Sociology Program Assessment Report
The Sociology program assessment for 1995-1996 identified several elements in need of attention in our goals, assessment, and feedback efforts. In particular:
We concluded our report last year with five specific recommendations and problems based on our experience with the assessment process:
Action Taken: This has been accomplished with the introduction of Sociology 260-The American Justice System, removal of Sociology 361, Deviant Behavior, from the required sequence, and redesign of Sociology 363-Juvenile Delinquency.
Action Taken: A significantly revised sociology curriculum has been approved by the faculty and will be in the 1998-1999 catalog; needed new courses will be designed during the 1997-1998 academic year. The redesign includes adding a course in general social theory at the junior year level, introduction of senior level theory courses, and introduction of senior level research methods courses.
Action Taken: This is no longer a problem as faculty have been hired for these positions and courses are now available for students on a regular basis.
Action Taken: We have instituted a training program for new faculty to emphasize advising information and the role of the advisor in the academic program. We undertook an evaluation of advising in the department in the winter quarter. Data are discussed below.
Action Taken: While we have increased opportunities to over thirty slots this year, we are concerned about this as well and have not identified ways to provide more opportunities for students.
Goals and Objectives
At a department retreat before the 1996-1997 academic year, we reviewed and redefined the sociology program goals. The redefined goals are:
Students will learn how social behavior emerges, becomes organized and institutionalized, and changes. An important element of this learning is the application of sociological knowledge in personal and public situations. Students are expected to learn appropriate communication skills.
Two broad objectives are important elements of this goal:
1. The student will understand the "sociological imagination."
2. The student will understand and be able to apply concepts and principles of sociological analysis.
All sociology students are expected to learn problem solving and critical thinking skills, how to gather and analyze data, how to draw conclusions from data, and oral and written communication skills. Cognitive learning objectives differ for the two major programs in sociology. Students in the general sociology program are expected to learn (1) how social situations cause social behavior and how people construct social meanings; (2) how repeated patterns of social life in such institutions as the family and economy affect behavior; (3) the role of social inequality in allocation of rewards in society, how inequality is passed from one generation to another in society, and the role race, class, and gender have in social inequality.
Students in the criminology program are expected to learn the social causes of crime and deviance. This learning includes the important role of social inequality in its various forms as a cause of crime and deviant behavior. Students are also expected to learn methods and effects of various interventions employed by society to sanction criminal activity. Table 1 lists the assessment tools, goals, and data employed in this report.
Table 1: Sources of Data
|Assessment Tool||Assessment Function||Data|
|Major Field Assessment Test||Core knowledge and breadth of learning in sociology||Percentile in relation to the national sample|
|Advising Survey||Students perception about whether they are receiving good advice from faculty members||Rating of student satisfaction with advising and advisor/advice|
|Exit Questionnaire||Perceptions of learning experiences, experience with faculty, and the quality of the program||Qualitative essay responses to each item|
|Institutional Research Data:
First Year Retention Study
|Number of students returning and reasons for departure||Proportion of freshman students returning for university and major|
|Institutional Research Data:
Career and Further Education study
|Preparation for advanced work in the field or entry into the job market||Percent who are satisfied with major, with preparation for career goals, and with preparation for further study|
|Institutional Research Data:
Long Term Educational Outcomes Survey
|Report on five year assessment of student satisfaction and perceptions of education||Ratings of education, reports early career experiences|
Students in the Master of Arts program in Sociology are expected to develop greater depth of knowledge of sociological concepts, methods, and theoretical analyses than is expected of baccalaureate level students. Students are expected to be proficient in two content areas and the core areas of theory and methods. Assessment of competence in theory and methods of sociology is accomplished in required seminars offered in these areas of instruction. Assessment of competence in areas of the discipline is accomplished with comprehensive examinations in two substantive areas for non-thesis students and satisfactory completion of the thesis defense for students pursuing the thesis option.
Major Field Assessment Test Data
The data reported were collected in the Spring, 1996 and Spring, 1997 administrations of the Major Field Assessment Test (MFAT). Table 2 contains the mean scores for the sociology program and the appropriate national percentile rank for the overall test and the two major subscores of core sociology and critical thinking. The data stability across years is heartening. The 1996 group volunteered in response to inducements (N=9) while the 1997 group is from a required senior level class (N=30) in which the test was presented as a class exercise. Individual students scored from the thirty-first to the eighty-eighth percentile in 1996 and from the seventh to the ninety-first percentile in 1997.
Table 2. Overall mean scores for 1996 and 1997 administrations of MFAT in sociology.
|Overall Score||155/200 71||155/200 71|
|Core Sociology||54 60||55 69|
|Critical Thinking||68 82||68 82|
We correlated overall scores and subtest scores with the students overall grade point average to provide some measure of validity for each "measure." The grade point average is highly correlated with the overall test result (.691), with the critical thinking component of the test (.774), but not with the core sociology subscore (.181) in the 1996 data set. Grade point averages correlated moderately with students total score (.497), and critical thinking portion of the test (.443), but not with the core sociology component of the test (.222) for the 1997 data set. These correlations may shift in the future, given that only nine students in 1996 and thirty students in 1997 took the examination. Additional experience with the test is needed before conclusions regarding these relationships can be drawn.
One interpretation of this pattern of results is that we are very successful meeting part of our goals since students are to employ their knowledge (eighty-second percentile on critical thinking subtest). This is an element of both the first and second objective. It appears true regardless of academic success measured by grade point average. Our success with cognitive parts of our objectives are good but not outstanding (approximately seventieth percentile on total score and core knowledge).
In addition to the global assessment of student outcome, the MFAT provides scores for eight subareas in sociology. These indicators allow us to measure how well our students have learned material presented in major subdivisions in sociology and provide evidence relating to our learning goal about how society develops, persists, and changes. Table 3 contains data for the subfield indicators.
Table 3. Percentile scores for indicators of mastery in subfields of sociology.
While the overall scores for the two years are stable, there are some changes at the level of subfields within sociology. In 1996 our graduates were among the best in the United States in their knowledge of social problems, deviance, and crime issues; this did not change in 1997. They were proficient at a high level in social institutions in 1996, but less proficient in 1997. They were very proficient in multi cultural topics in both years and were proficient at a good level in theory and methods in sociology in 1996. In 1997 students were more proficient in methods than theory. The decline in the theory score may reflect the fact that students took the MFAT on the first day of their theory course. Our students have not done well in the study of population and rural and urban life. This is an area where we offer little instruction; the low score here is not surprising. The discouraging information is the mediocre performance in social psychology and the study of gender in 1996 and in social psychology in 1997. Improvement in learning about gender is evident as students have taken advantage of new courses on this topic. The relatively low performance in social psychology is being addressed in the redesign of the curriculum in sociology.
To assess advising, students responded to a short questionnaire about their experience with their academic advisor. The format was similar to the one employed in teaching evaluations in the College of Arts and Sciences with thirteen questions asked about elements of the student/advisor relationship. Eighty-seven of approximately 320 majors completed the form for sociology advising at the conclusion of their Winter, 1997 meeting with their advisor. Mean scores for three items indicate students see these relationships in very positive ways. Students are willing to recommend their advisor to a friend (mean of 4.42 out of 5.00), rate specific advice highly (4.56 out of 5.00), and rate their overall experience with advising very highly (4.57 out of 5.00).
Exit Questionnaire Data
The exit questionnaire was sent to forty randomly selected seniors in sociology and criminology. The questionnaire asked students to comment on the quality of the curriculum as manifested in courses and breadth of coverage, faculty in terms of quality and general treatment, and preparation for future career or educational plans. Sixteen students returned questionnaires. Data from questionnaires helped us identify areas of concern in last years assessment. Students who responded this year were more positive in their comments.
Eleven respondents gave positive responses regarding the overall quality of the undergraduate program. Many of these students specifically cited the variety of courses available, the strength of the curriculum and specific experiences in internships and upper level courses as positives for the program. Students expressed some concern for why we require statistics, a foreign language, and courses in the sociology curriculum designed to foster breadth of knowledge in the major field. This pattern of concerns reflects a tension for students who would prefer a vocationally more specific criminal justice program over the liberal arts criminology program we have in place. Students also expressed some concern that course content was too repetitive. We think this is the issue cited last year about the core criminology sequence which has been addressed in the redesigned curriculum. Students also mentioned a desire for more information about career choices, more variety in courses and instructors, and changes in times courses were offered.
Most students responded that the faculty as a group is excellent, concerned, and knowledgeable. The pattern of satisfaction with advising was repeated in the exit questionnaire. A small number of students reported some faculty did not care, were unmotivated, and were not a cohesive group. A few students reported their advisor was unhelpful, uninformed, and rushed them out the door.
Students generally saw the program as preparing them well for the future. Some students criticized course offerings in the criminology program. Specific concerns are addressed in the curriculum revision cited at the beginning of this report. There will probably be continued tension between our effort to mount a sound liberal arts criminology program and the interests of students whose goals are better matched in a vocational criminal justice program.
Institutional Research Data
Data from institutional research show sociology students persist from freshman to sophomore year at a rate that is above the university average. Eighty-six percent of first year sociology students returned to Ohio University in 1995-1996 compared to the university average of eighty-three percent. Over the last five years, average retention for sociology is at the university average of eighty-four percent. Data provided in 1996 show sociology students graduate in an average of four years and one quarter. This is encouraging because students often enter the sociology major after completing part of another program and start the major as late sophomores or early juniors.
Data from the Career and Further Education study for 1996 show 1995 graduates to be generally satisfied with their experiences in the sociology major -- one hundred percent indicate some level of satisfaction with major courses, preparation for career goals, and preparation for further academic work. Alumni who responded to the Long Term Educational Outcomes Study are also very positive about their experiences in the sociology major, even though many report they would change their major field of study .
Students in the criminology curriculum are less, but still mostly satisfied with their experience. Seventy percent reported some level of satisfaction with the program. Eighty nine percent felt the university prepared them well for their career goals. This result for recent graduates differs greatly from information obtained from criminology students who have five years in the work force. Alumni surveyed in the Long Term Educational Outcomes study in 1996 report generally high levels of satisfaction with their preparation. Ninety-five percent reported their major courses were helpful in acquisition of job skills, for instance..
Alumni report that Ohio University and their major program provided effective preparation in thinking analytically, applying the major, writing well, and coping with complex issues. Areas of concern reported in the Long Term Outcomes Study include lack of preparation in use of computers, oral communication, community involvement, and a perception that Career Planning and Services is not helpful. These patterns of satisfaction and concern suggest the sociology and criminology programs are generally effective but need to fine tune some aspects of the "hidden" curriculum in skill development areas. In particular, course requirements probably need to be upgraded in the area of modern computer skills and oral communication.
The pattern of dissatisfaction for new graduates in the criminology program, evidenced in the Career and Further Education Study and our exit questionnaire, compared with the general satisfaction seen after five years in the work force is an area of concern. We suspect expectations for immediate job placement are not met for this group of vocationally oriented students, and this may be a source of some frustration for them. Experience in the work force appears to mitigate this problem as students find positions and are able to identify ways in which their undergraduate experiences were beneficial.
Master of Arts in Sociology
Assessment of student outcomes in the M. A. Program is carried out in examination or thesis exercises for content areas of the discipline. Assessment of the core elements of theory and method is carried out in course work on these topics. Twenty-seven students matriculated in the program from Fall, 1993 through Winter, 1996. Of these, twenty will have completed the degree by the end of the current academic year. Two are expected to finish during Summer, 1997. Four other students are in various stages of completing their work. One is writing the final paper, another has completed two of three substantive examinations, and two others are in contact with professors regarding progress. We expect both of these students to complete the degree. One student is a full time employee of the university and takes courses intermittently. One student has dropped out of the program. Forty percent of students who finished between Fall, 1993 and Winter, 1996 wrote a thesis and sixty percent took qualifying examinations.
We have identified current career activity for ten of our recent MA graduates. Five are enrolled in Ph. D. programs and five are employed in related fields. Their employment includes the Ohio Youth Advocacy program, private research consultant, and SEPTA. We clearly need to follow our Master of Arts Alumni more carefully in order to assess our program and its outcomes more effectively.
Our assessment of the sociology program for 1996-1997 leads us to the following conclusions about our baccalaureate program:
We also need to develop goals for, and assess our general education service mission. Over seventy percent of our enrollment is service related, yet we have no direct evidence about student learning in this area.
Finally, we need to develop better strategies for gathering information from our Master of Arts students post degree. We are able to identify markers of success while students are in the program and at the point when they finish, but do not have good information for the early career period that is often critical to long-term educational benefit from the advanced degree.