Assessment Report
Department of Psychology
June, 1998
This report describes the assessment efforts of the Department of Psychology during the academic year 1997-98.
Below are the goals for our graduate and undergraduate programs. They were originally developed by the department chairs in consultation with the Planning, Policy, and Advisory Committee of the Department. They have not been modified since last year.
Undergraduate. The Department’s goals for its majors are that they develop a broad knowledge of the content of psychology including an understanding of the measurement and methodology procedures used in the field, that they be able to apply the knowledge and skills acquired to a variety of life situations, that they be successful in achieving jobs or entry into advanced training, and that they exhibit satisfaction in their program of study.
Graduate. The goals of the graduate clinical program are that the students have an in-depth content knowledge of the field, that they have developed a strong level of professional skills in assessment and intervention, that they have developed strong research skills, and that they have developed teaching skills. The goals of the graduate experimental program are that the students have an in-depth knowledge of the field, that they have strong research skills, and that they have developed teaching skills.
The assessment of student learning detailed in this report is based on three sources: (1) the Educational Testing Services Psychology II Major Field-Test, (2) the College Outcomes Measurement Program (COMP), and (3) the Follow-up Studies of Graduates and the Freshman-Senior Involvement Study conducted by the Office of Institutional Research, and (4) data from the Departing Senior Survey completed by this years’ graduating class.
ETS Major Field-Test. Data from the most recent psychology major field test indicated that our graduating seniors scored at the 93rd percentile compared to a set of 193 institutions that took the test. This is up from the 75th percentile from the prior year, although some of this fluctuation may be due to sample variation. The students scored above the 90th percentile in the areas of learning and cognition, perception, comparative and physiological, and clinical and abnormal psychology. They scored above the 85th percentile in the developmental and social area. These data indicate that our graduating seniors have developed a strong level of knowledge in the major content areas of the discipline, one of our major goals for the undergraduate program.
COMP Test. The Office of Institutional Research reports rolling three-year averages of the performance of psychology majors on the COMP test and each of its subscales. The most recent data based on tests given during 1995, 1996, and 1997 show that our students improved from the 31st percentile on the total COMP score during the freshman year to the 55th percentile during the senior year. This increase of 24 percentile points is significant but does represent a 9 point drop from the 33 percentile point increase shown in the prior three year rolling average. While these data arise from a small sample of only about five students per year, we plan on monitoring our student’s performance carefully over the next few years to determine whether this drop is a sampling issue or a performance trend. Our students continue to show good improvements from the freshman to senior years on the sub-test areas of Using Science and Technology (27 percentile point increase), Communicating (23 percentile point increase), Functioning within Social Institutions (19 percentile point increase), and Clarifying Values (19 percentile point increase). Somewhat worrisome is the 6-percentile point increase in the area of Solving Problems. This represents a decrease from the 24-percentile point increase shown in the prior three year rolling average. While we must be cautious interpreting this score because of the small sample size, we do need to consider other indices to determine if our students are appropriately developing general problem solving skills.
Institutional Research Data. Over the past year we have received two reports from the Office of Institutional Research which provide useful data regarding our undergraduate majors (Follow-up Studies of Graduates and Freshman-Senior Involvement Study).
Data from the follow-up Studies of Graduates showed that 43% of our 1996 graduates were extremely or very satisfied with their major courses, an increase from the 26% of our 1995 graduates and the 34% of 1994 graduates. The 1994 figure is above the college average (41%), but below the university average (58%). Recent changes in the way we deliver our undergraduate program may underlie the upward trend. Later in this report we detail other changes currently underway or being considered. The average salary of our 1996 graduates was $22,164 up from $20,177 in 1995 (9.8%) but still lags behind the average salaries of other college and university graduates. Forty-six percent of the 1996 psychology graduates were extremely or very satisfied with their present positions. This is less than the 56% figure of other college students and 66% of other university students. This difference has been long-standing over the years and may reflect the general character of the discipline. We continue to consider means to make our graduates who are not going on for advanced training more employable in high quality positions.
The Freshman-Senior Academic Involvement study is a longitudinal study comparing student involvement changes from the freshman to the senior year in the areas of academic involvement, social involvement and activities, and personal goals and adjustment to college. In this report we pay particular attention to the section on academic involvement, especially the frequency of academic conversations with faculty and advisors and ratings of importance and satisfaction.
One important issue for our department is the amount of contact between students and their advisors and faculty. The data from the Involvement Study indicate that about 80% of our freshman met with their advisor during the freshman year while 74% did during their senior year. However, the number of contacts went up from 1.7 to 2.9 per year. These data are similar in pattern but slightly lower than university average. In terms of non-advisor faculty contact only 53% of freshman have conversations with faculty while 80% of seniors do. These figures are very similar to the university pattern. About a third of our students have conversations about research and scholarship at both the freshman and senior year, slightly above university average. We are pleased to see, however, that the number of such conversation increases dramatically in psychology students from the freshman to the senior year (.7 to 3.2) whereas the figures are much smaller for the university in general (.6 to 1.0). This difference may reflect our push to involve our upper level students in scholarly and research activities, an important issue for our program.
The data from the importance and satisfaction section of the Involvement study provide some positive observations but raise some troublesome issues that we need to consider. Psychology students generally are very satisfied with the quality of instruction they have received at Ohio University (freshman 89%, seniors 91%), slightly higher than overall university figures. Between 65% to 70% of our students are satisfied with faculty availability outside of class although about 85% think such contact is important. Two worrisome patterns indicate that while 82% of our freshman are satisfied with their major courses, this figure drops to 73% by the senior year. University figures show a reverse pattern of 68% satisfied as freshman but 78% as seniors. Similarly in the advising area, 46% of our freshman are satisfied with their advising but only 29% of our seniors are. University figures are 45% and 48%. We detail later in these report efforts we have been considering which may address these patterns.
Departing Senior Survey Data. As mentioned above, we have for years been concerned that the data we get from institutional research comes from quite small samples. We decided late this year to try to survey the graduating class as they depart. All seniors in advanced level classes were given the questionnaire and a general mailing was sent to all majors with sufficient credit hours to graduate in June or August. To date 65 questionnaires have been returned. This is roughly 50% of our June-August graduating class. We anticipate using this questionnaire every year and with an earlier start next year an even higher return rate. While some mailings are still coming in, preliminary results have emerged. The questionnaire addresses most of the same concerns as does Institutional Research but the sample size is much larger and less likely to be skewed by a few extreme responses.
With regard to advising 25% of the respondents report that the availability of academic advisors was poor or very poor, 45% adequate, and 32% good to excellent. The quality of advice given had a slightly better profile with 29% poor to very poor, 36% adequate, but 38% good to excellent. These data are generally consistent with data from other sources which indicate our advising process could stand some improvement.
In terms of quality of instruction the picture was quite positive. With sections taught by graduate students rated only slightly lower than those taught by faculty, 4% of the respondents rated both TA and faculty teaching as poor to very poor. Adequate teaching was reported in 33% of TA sections and 15% of faculty sections. Good to very good ratings were reported for 63% of sections taught by TAs and 81% of the sections taught by faculty. These data are consistent with previous data which indicate that, while the quality of instruction is generally quite high, sections taught by faculty are slightly more positively evaluated.
The department’s computer access and quality were rated as adequate to excellent but several remarks were made regarding Mac vs. IBM, compatibility, etc. These are questions which will be considered in the next few years.
In terms of immediate plans 20% of the respondents state that they already have jobs which require knowledge gained in pursuing their degree. Seven percent report having jobs which do not require a degree and 30% are still looking but in the market for a job. It is not surprising that 30% have not found work yet since they are just now graduating. Forty three percent of those reporting have been accepted by graduate schools. The list of areas of admission while predominantly psychology also include counseling, law, social work, linguistics, and MBAs. Several of the students have been admitted to high quality schools such as North Carolina, Purdue, Ohio State University, University of California at Santa Barbara, Loyola of Chicago and several MAC schools. Six of the respondents are continuing here at Ohio University, two of them in our own program.
These data are much more reliable than the results provided by Institutional Research because the sample size is larger by a factor of 3 to 10. The general pattern of the data, however, is reasonably consistent with that from Institutional Research.
Summary. Our students appear to be doing quite well on standardized tests within the field, show significant improvement across their years at the university in acquiring general skills, and show strong increases in academic interactions. There are areas of concern regarding satisfaction with major courses and with academic advising.
III. Assessment of Student Accomplishments in the Graduate Program
The department offers the M.S. and Ph.D. in clinical psychology, experimental psychology, and industrial/organizational psychology. Of these three, only clinical has a periodic external review, which is conducted by the American Psychological Association. Following their most recent site visit (1997) the APA granted the clinical program accreditation for seven years, the maximum period for which they grant accreditation and implicitly their highest evaluation. Since that time the department has had its entire graduate program reviewed by the Ohio Board of Regents. Despite the large number of psychology doctoral program in Ohio, the Board made no move to curtail our training program, though it did recommend a sharper focus to our training program.
During 1997-98 the department awarded the Ph.D. degree to 7 students in clinical psychology and 8 students in experimental psychology. It also awarded the M.S. degree to 5 students in clinical psychology, 4 students in experimental psychology, and 4 students in industrial/organizational psychology.
Research activity has been high among our students. In the experimental psychology program 5 students had one or more research articles published and 6 students had submitted one or more manuscripts for review. In the industrial/organizational program 1 student had published two articles and submitted two more for review.
In the experimental psychology program 8 students had presented one or more papers at conferences (one student had accumulated 14 presentations) and 8 had submitted papers for review for upcoming conferences. In the industrial/organization program 5 students had presented one or more papers at conferences and 5 had also submitted presentations for review. In the clinical program 11 students presented papers at conferences.
Andrew Geers, one of our experimental psychology students, won an award for outstanding master’s thesis. The award was one of only three offered in a nationwide competition by the American Psychological Society.
Practicum training for clinical students has been extensive in the past year. About 12 students completed 12 months of traineeships in local and regional mental health agencies. These 15-hour/week traineeships are a significant part of the students’ preparation for clinical internships, which are full-time, 12-month training programs that are usually conducted in the last year of the student’s program. The internship sites are located across the U.S., and we require our students to apply only to those approved by the American Psychological Association. Of the 12 students who applied for clinical internships for 1998-99, all 12 were awarded an internship.
Recruitment of new students has been effective this year. For coming fall we have admitted 11 new students in clinical, 5 in experimental, and 3 in industrial/organizational, yielding an entering class of 19, slightly larger than average for us.
IV. Improvements in the Undergraduate Program
While the evaluation of undergraduate sections taught by TAs are good we feel that we can further improve instruction at the undergraduate level by putting as many professors in front of the class as possible. The faculty approved a recommendation that all faculty members should teach at least 1 undergraduate class per year. To the extent that faculty are less active in graduate education and research they will teach more undergraduate classes. Our first attempt to implement these policies have reduced the percentage of TA taught sections from 38% of all classes in 1997 to 18% of classes fall quarter of 98.
When graduate students must be used to teach undergraduate sections they will now be the most senior students in the program (3rd and 4th year Ph.D. students). We have decided to assign each TA to a faculty mentor who will advise and monitor the thoroughness and quality of TA instruction. We are encouraging faculty to increase the written component of class evaluation instead of objective tests. Faculty can bargain with the chair to reduce class enrollment to 25-30 with the stipulation that term papers or essays become a major component of student evaluations.
As class enrollments have grown several courses have dropped lab components. We will work with new and continuing faculty to develop lab sections such that, for example, physiological psychology 312 will be offered as 312 lecture with 312L as a lab section for at least some of the interested students. Credit toward a faculty members teaching load for lab instruction should encourage development of such lab courses.
We have decided to strengthen our departmental honors program. An insert will be added to the departmental brochure elaborating the availability and benefits of this program. The psychology department home page will feature this information. We will recruit students at the end of their freshman year to allow time for more concentration on research and the honors thesis.
Advising of undergraduates continues to be an issue of concern. We have centralized the handing out of DARS materials in the main office. All students are reminded to sign up to see their advisor. Sign-up sheets are posted by each faculty member outside their door and the assistant chair for undergraduate affairs monitors faculty to assure they have sign-up sheets and have enough office time set aside for advising. In the past the complaints of undergraduate students regarding advising were not always taken seriously by faculty when they show up in data collected by institutional research. The new data from the departmental survey will bolster the idea that it is not just one or two dissatisfied folks who skew limited returns. The current departmental administration is committed to increasing the role of faculty advisors and will discuss and push the issue in the upcoming year.
V. Improvements in the Graduate Program
For the first time in three years, the department did not undergo intensive review by outside agencies. The department’s Policy Planning, and Advisory Committee and the department’s two major sections (Clinical and Experimental) have approved a number of changes aimed at enhancing the quality of the graduate programs. Issues currently being considered or recently approved include the following: