Assessment Report

Department of Psychology

June, 1997

This report contains the assessment efforts of the Department of Psychology during the academic year 1996-1997.

The following are the goals for our graduate and undergraduate programs. They were originally developed by the prior chairman in consultation with the Planning, Policy, and Advisory Committee of the Department. They have not been modified since last year.

Undergraduate. The Department’s goals for its majors are that they develop a broad knowledge of the content of psychology including an understanding of the measurement and methodology procedures used in the field, that they be able to apply the knowledge and skills acquired to a variety of life situations, that they be successful in achieving jobs or entry into advanced training, and that they exhibit satisfaction in their program of study.

Graduate. The goals of the graduate clinical program are that the students have an in-depth content knowledge of the field, that they have developed a strong level of professional skills in assessment and intervention, that they have developed strong research skills, and that they have developed teaching skills. The goals of the graduate experimental program are that the students have an in-depth knowledge of the field, that they have strong research skills, and that they have developed teaching skills.

The assessment of learning is done through three means: (1) College Outcomes Measures Program (COMP), Educational Testing Service’s Psychology II Major Field Test, and the survey of graduates done by the Office of Institutional Research. All three of these have been done in previous years.

COMP Test. Three-year rolling averages of the COMP test have been collected for many years. Figure 1 contains the data for the amount of change, i.e., improvement, from the freshman to senior years on two scales--the Total score and the "Functioning Within Social Institutions" scale. The latter is the scale closer of the two to the content of the Department’s courses. Note that the Office of Institutional Research has no separate data for Psychology majors from 1996, so this figure contains data through 1995. (Since these are three-year rolling averages, the most recent data point actually contains data from 1993-1995.) The data indicate that psychology majors have always shown an improvement from the freshman to senior year on these scales. Since Figure 1 contains only change scores from the freshman to the senior year, it does not indicate the absolute score of our majors, which averaged at the 68th

Figure 1

percentile nationally for all the subscales at the most recent data point. Our majors did best on this test in the late 1980s, performed poorly in the early 1990s, and have rebounded in the last few years. Since our more recent majors improved about 30-35 percentage points nationally during their undergraduate careers here, this means that they started at about the 35th percentile as freshman, and our curriculum essentially doubled their score. (Attrition of weaker students certainly contributes to this rise in scores.) In general, we are satisfied with the trend in our majors’ recent performance.

2. ETS Major Field Test. We have no data from the 1996 field test. We did administer the test again in May of 1997, but the data have not yet been returned by ETS. The 1995 data indicate that our students scored at the 75th percentile compared to a set of 128 institutions administering the psychology test in the spring of 1995. This performance is slightly higher than our majors’ performance on the "Functioning Within Social Institutions" scale of the COMP test, but the ETS major field test is much closer to the content of our undergraduate courses than is the COMP test. Therefore we think that the 75th percentile is a good indicator of where our majors compare to senior psychology majors at other institutions.

3. Institutional Research Data. According to the most recent data (class of 1995) from the Office of Institutional Research, the proportion of Ohio University undergraduates who go on to graduate training is 28%; for Arts and Sciences the number is 46%; for Psychology majors the number is 57%. It is remarkable that over half of the Psychology majors who responded to the survey were in some kind of advanced training. This suggests that the Department is extremely successful in educating its majors in such a way that they can compete successfully for such training.

The fact that such a large fraction of our students are in graduate school makes the statistic on job salary somewhat misleading. The Department’s majors average about $2800 less in starting annual salary than Arts and Sciences majors as a whole. However compared to Arts and Sciences majors as a whole, many more of our graduates are in school rather than in the job market. Therefore the psychology majors’ salaries would be deflated due to this situation.

Summary of Accomplishments. Our majors do well on standardized tests compared to psychology majors in other institutions. Our majors also do very well in competing with other schools’ majors for slots in graduate schools.

The Office of Institutional Research data contain four categories in which the Department of Psychology’s 1990-1991 graduates were either extremely or very satisfied with some portion of their college experience at a level at least ten percentage points lower than the College of Arts and Sciences’ graduates. These are:

a. Interaction with Students;

b. Interactions with Faculty;

c. Academic Advising and Guidance;

d. Inspiration and Encouragement.

There was one category in which the Department’s 1995 graduates were either extremely or very satisfied with some portion of their college experience at a level at least ten percentage points lower than the College of Arts and Sciences’ graduates: Satisfaction with major courses.

We should also add that "Level of Rigor and Scholarship" exhibited a 9.5 point difference between the College’s and Department’s level of satisfaction in the 1990-1991 classes.

Our analysis of this data leads us to conclude that there are three problems being identified by our undergraduates. The first pertains to advising. We have approximately 725 majors and about 450 minors during the current academic year. The Office of Institutional Research’s "Longitudinal Involvement Study" shows that the proportion of psychology majors who have a conversation with their advisor drops off 13% from the freshman to the senior year, much more than the 1% drop-off exhibited by the College of Arts and Sciences as a whole. We have made efforts to rectify this problem. During the last few years outside each faculty member’s door we have put time sheets in which each advisee could write his or her name to make an appointment during pre-registration week. The number of students who actually make an appointment is quite small. We hand out the DARS reports in one room during advising week, but the 725 majors who avalanche into that room during that week generally do not then go to their advisor’s office after picking up their sheet. I have spoken to students, who are not of one mind with regard to whether they should be required to see an advisor in order to get their DARS report. Some curricula require this visitation, but I question whether we could accommodate the majors (and minors) if such a requirement were implemented. (At the University of Michigan, where I was a graduate student, some people were hired to work in the Arts and Science office to help with undergraduate advising for the Department of Psychology, which also had a large number of majors.) We have tried a few different strategies, but the large numbers defeat our efforts. This year I think we are going to have to formulate another solution to this problem, which I think is responsible for the low ratings on "Academic Advising and Guidance" and perhaps "Interactions with Faculty."

The second problem is another one we have struggled with over the years: staffing our courses with Type I faculty members. Recall that the 1995 graduates expressed low levels of satisfaction with their major courses. Here are two pieces of data to help explain that. The first is that in the 1992-1993 academic year, 10% of the 300-level courses were taught by graduate students. (These data exclude summer terms.) In 1993-1994 6% of the 300-level courses were taught be graduate students. In 1994-1995 17% of the 300-level courses were taught be graduate students. One hypothesis is that that the students in the class of 1995 had more courses (about 1/6) taught by non-faculty during their senior year. This may have diminished their satisfaction with their major courses.

The second group of data are depicted in Figure 2, which contains the teaching ratings of the faculty and graduate students during the past several quarters. Note that the grad students always are rated much lower than the faculty on our 5-point scale. Note also that this difference in the faculty and grad student ratings was greater between the winter of 1994 and the spring of 1995, just when the class of 1995 was taking the preponderance of their psychology courses. (The gap during these seniors’ last term, the spring of 1995, was the largest in our Department’s recent history.) With more grad students teaching them, and with the grad students teaching less well than usual, the class of 1995 may have been justified in denigrating their major courses.

The strain on our faculty to cover the undergraduate courses in relatively severe, and the reliance upon graduate students has been our generic solution. Figure 2 portrays the cost of that strategy.

Figure 2

The third problem is is revealed by the fact that the 1990-1991 graduates exhibited a 9.5 point difference between the College’s and Department’s level of satisfaction in "Level of Rigor and Scholarship." Recall that over half of the respondents to the survey said that they eventually went to graduate school. These psychology majors attended undergraduate classes in which they were the minority. Even in our 300-level courses, with the exception of Psychology 321, majors rarely outnumber non-majors. We simply have an enormous service responsibility in our Department, and that responsibility diminishes the rigor of the courses many of our faculty would like to teach. If we catered to the Psychology majors, many of whom are going on to grad school, we would have to raise the level of all of our courses from their current level of instruction. If we kept the courses at their current level, we would suffer the criticism of the Psychology majors with regard to the level of rigor and scholarship.

One solution was proposed to this problem: Perhaps we should have "majors only" sections of many of our courses. This proposed solution has drawbacks. First, some upper-level courses would require more sections than we currently offer. That would exacerbate our already serious current staffing problem. Second, many faculty expressed resistance to teaching the non-major sections, since the majors provide the main intellectual stimulation, according to some of our instructors. For example, the majors provide more than their share of the class discussion.

Graduate Program. Note that to this point the accomplishments and non-accomplishments have pertained just to the undergraduate program. This is due to the fact that the graduate program in clinical psychology was officially reviewed by the American Psychological Association (APA) last year, and the entire graduate program was reviewed by the Board of Regents as well. Both reports were positive.

The APA report concluded, "Overall, the program quality was judged to be very good to excellent. No major problems in any of the domains emerged as a function of either the program self study or the site visit." As a result of the favorable site visit, the APA issued continuing accreditation for the maximal time period, which is seven years.

The Board of Regents team was also quite positive: "Our team was very favorably impressed with the quality and the productivity of the Ohio University Graduate Program in Psychology. While the faculty shows a strong commitment to teaching, they are also quite rigorous in their commitments to research and scholarship . . . The graduate Program in Psychology appears to be a strong and healthy one. . . We note that graduate students from Ohio University are highly productive with respect to research and scholarship. On average, the students normally complete one published article or one convention presentation each year."

These assessments of our graduate program were sufficiently favorable that the faculty felt relatively satisfied with our current program.

V. Improvements and Enhancements Already Implemented. A major improvement the faculty has already enacted is the revitalization of our Departmental Honors program. This program has always been available to our best undergraduate students, but an average of only two students have entered it during most academic years. There have been a few reasons for this low level of activity. First and foremost, the faculty receive no reward to supervising the theses of Departmental honors students. Each thesis project usually involves two years of supervision. Also, since the undergraduate is relatively inexperienced in statistics and other procedures, the faculty mentor needs to take a very active role. This burden has been resisted by many faculty. The new Departmental Honors procedure gives faculty "points" toward release time. After the direction of approximately three such theses, the faculty member receives a course reduction.

The second reason for the low participation in the Departmental Honors program has been due to low publicity. We intend to raise the level of awareness of this program in order to attract many more good undergraduate majors.

Our goals in revitalizing the Departmental Honors program have been to reduce student dissatisfaction with the major’s courses, increase student-faculty interaction, and cater to our better students who are getting lost in the courses in which the Psychology majors are a small minority. We think the closer mentor-student relationship fostered by this program will also help provide better guidance and encouragement to these students.

VI. Changes Recommended in Curriculum and in the Assessment Process.

I am pleased with our Department’s assessment process. Therefore I anticipate no changes next year. However we will have a new chairman next year, and he can make whatever changes in the process he deems appropriate.

With regard to changes in the curriculum, I do anticipate some changes, but the faculty will have to make some hard decisions first. Some of these decisions will have to be made in consultation with the College of Arts and Sciences, since many of them have funding implications.

The first decision will pertain to staffing. Our graduate students teach a large number of our courses. Since they are less experienced than our faculty as classroom teachers, they quite understandably perform less well than the faculty. This has consequences for undergraduates’ satisfaction with the curriculum and graduate student degree progress. We could replace graduate students in the classroom with non-Type I faculty, but this is a more costly course of action than the status quo. Another complicating factor is that many of our graduate students want to teach a large number of courses, since they feel that this experience will make them more competitive for the type of jobs they will be seeking when they graduate. In general, our graduate students do not stay in graduate school for as long as the graduate students at the more prestigious universities. Thus our students do not have the number of publications as do many of their competitors from these other institutions. Hence our graduate students feel that they cannot compete as well in seeking job at research institutions but can compete well at small colleges. Such colleges seek good teachers, so our grad students want to prove they can teach a large number of courses and can teach them well. Of course, one must teach a lot to accomplish both of these goals, so our graduate students often wish to teach a great deal. Any strategy to reduce their number of teaching opportunities may be resisted, even if it would enhance undergraduates’ satisfaction with the courses. This is obviously a difficult problem.

The second decision pertains to a good strategy to accommodate our very large number of majors. They deserve good advising and good instruction. Assuming that our faculty advise no "undecided" students and no minors, both of which are false assumptions, each faculty would have to advise about 30 students during pre-registration if we required majors to see their advisor before they pre-registered. This would be resisted by many majors, particularly upperclass students. There is an irony here that many students don’t want to see an advisor, yet the data show that some students want better advising. One hopes that the same students don’t hold these two sentiments simultaneously. It may be that while they are in school students do not want to see their advisor, but when they fill our an alumni survey, they wish that they had received better advice. During the next year the faculty will have to decide what course of action they wish to take with regard to advising.

A related issue is the level of the courses taken by our majors. The extra mathematics and science requirements required in order to major in Psychology result in our majors being a very talented group of students. The fact that so many of them go on to graduate school attests to their ability and motivation. How we can be responsive to their needs while fulfilling our enormous service role to the University is a most difficult problem. We may have to go to "majors only" sections of a few courses as a compromise solution.

The Department’s annual report, which was turned in last week, contained a great deal of information about the enormous scholarly output of the faculty and graduate students. The reports by the Board of Regents and the APA both praised this aspect of our Department. This level of this output is characteristic of a department at a prestigious university. On the other hand, this assessment report contains information principally about our undergraduate instruction and advising. Such concerns are characteristic of a department at a fine four-year undergraduate institution. Here is the crux of the problem: Can we fulfill both roles at our current level of staffing? For example, about one-third of our faculty are supported by external grants. Obtaining external funding is a goal the University wishes to foster. The researchers who write such grants, execute this research, and supervise the graduate students who work on these large projects might not be able to advise adequately the 30 undergraduates assigned to them during registration week. This will, in turn, result in justifiably low undergraduate satisfaction with advising. These are the types of issues the faculty will have to confront when they meet next year to consider how to accomplish the goals currently not being met.