1997 ASSESSMENT REPORT
Department of History
20 July 1997
Bruce E. Steiner, Chair
INTRODUCTION
The History Department offers a comprehensive program of studies, leading to the B.A., M.A., and Ph.D. degrees. Its courses cover, in addition to a range of time periods and topics in United States history, extensive topics and regions in European, Russian, Latin American, Middle Eastern, African, and Asian history, as well as more specialized thematic courses in, for example, sports history, the history of espionage, and the history of the U.S.-Vietnam war. In addition, we have the focused graduate offerings of the Contemporary History Institute. These courses attract an exceptionally diverse clientele. This circumstance, and the nature of our discipline — which is often, and for most students not entirely unjustly, viewed as an essential "liberal art" rather than a "professional" branch of study — means that History does not lend itself to easily quantified measurements and obvious outcomes judgments. Furthermore, it is worth pointing out here that the discipline of History does not possess any national accrediting body or universally accepted benchmark testing. Our departmental goals and methods of assessing progress toward them have therefore of necessity been cautiously evolved. Our first formal assessment document dates from 1994-95, the same year in which we also conducted an extensive assessment of our Ph.D. program for the Ohio Board of Regents. We are now only 18-24 months away from those efforts and resulting documents, and although we have taken some important steps (the results of which may in several cases not be clear for 4-5 years), there is still more that we would like to do.
GOALS AND OUTCOMES
Our goals for both undergraduates and graduates are based on our own sense of the best attributes of our discipline, but they have also taken conscious account of the Ohio University Mission Statement’s emphasis on broadened perspectives and disciplined thought; of the College of Arts and Sciences Statement of Purpose’s highlighting of the principles of the liberal educational tradition; and of the Ohio University General Education Requirement Statement’s focus on breadth of learning and ability to synthesize and communicate that synthesis.
Our B.A. program pursues the general goals stated above, and provides an entrance into a wide variety of careers, both discipline-based (for example, archival training, historical preservation, library science, legal studies, and teacher training) and general (for example, business and public service careers in which abilities in research, analysis, organization, synthesis, interpretation, and writing are both highly prized and applicable in practical, everyday ways).
Our program of graduate studies pursues the same general goals cited above but at greater depth and higher levels of sophistication. The graduate program provides advanced training appropriate for those who seek to enter careers directly related to or utilizing historians’ skills, as well as for those who will eventually benefit indirectly or personally from historical study. At the Ph.D. level, special emphasis is laid upon preparing individuals for careers in research and teaching. The Contemporary History Institute offers additional opportunities for application of historical skills to high-level careers outside the academy, such as domestic politics, international diplomacy, law, journalism, and economic and social analysis.
Undergraduate Outcomes
Students completing the B.A. program are expected to accomplish the following:
(1) Master a broad selection of factual knowledge, based on a diverse and individually-shaped selection of courses on the history of different cultural and geographical regions.
(2) Develop a sound acquaintance with basic research strategies and tools, including both printed and electronic sources, and demonstrate an ability to access and collect information effectively.
(3) Display the ability to work with historical information using skills of organization, analysis and synthesis, evaluation, and interpretation.
(4) Demonstrate the ability to express the resulting ideas effectively.
(5) Demonstrate an appreciation of the ways in which thinking about past, present, and future are connected.
(6) Demonstrate an awareness of the global context of the human past, and an appreciation of the different historical perspectives that cultural diversity offers.
Graduate Outcomes
Students completing the M.A. program in History are expected to have attained an appropriate level of achievement toward the undergraduate goals and outcomes, as formulated above, and to accomplish the following:
(1) Acquire a detailed understanding of the factual content, principal issues, and historiography of one field of history, carefully defined by geography and time. (The non-thesis option requires two fields of history.)
(2) Particularly for those students who take the thesis option, acquire and display research, organizational, analytical, and writing skills at an advanced level. These skills should be of a type and order that qualify students for continuation of study at the doctoral level or, alternatively, for effective teaching at the secondary level or for a variety of other careers in which historical skills may be directly or indirectly applied.
Students completing the Ph.D. program in History are expected to have attained an appropriate level of achievement at the M.A. level, as defined above, and to accomplish the following:
(1) Acquire a detailed understanding of the factual content, principal issues, and historiography of two major and two minor fields of history, carefully defined by geography and time.
(2) Develop and demonstrate, by completing a doctoral dissertation, the skills of historical research, organization, analysis, and written expression, as well as a capacity for original analysis and interpretation of a historical subject.
(3) Develop an understanding of historical methods and approaches at a level sufficient to teach history at the college or university level, and to permit active participation in general scholarly discourse in the field.
(4) Develop practical classroom skills—including lecturing, course and lecture preparation, and grading—appropriate to teaching, especially at the college and university level.
ASSESSMENT METHODS
Student assessment at a certain level is, of course, on-going and automatic; in our discipline, more than most, performance in course work is a crucial assessment tool. In particular, for example, undergraduate performance on the research paper assignment in the capstone-like History 301J, "Historical Research and Writing," is considered an important indicator. For graduate students, completion of requirements such as foreign language capability and theses or dissertations are obvious and critical sources of assessment information.
In addition, our student evaluations of individual courses, done every quarter in all courses at all levels, provide much valuable information. Our evaluation form offer both quantitative and qualitative data; their format has on a number of occasions served as a model in the College of Arts and Sciences.
In order to monitor long-range student learning and achievement, however, the Department of History relies on the following mechanisms:
Undergraduate
In addition to that represented by graded results in regular course work, the Department of History relies on the following mechanisms in order to monitor undergraduate student learning and achievement:
(1) Entrance and exit testing. All Freshman incoming majors are tested in the Fall quarter of each year. The test consists of a series of four essays, one covering each of United States, European, and Non-Western history, and one covering Historical Research and Interpretation. Those who continue as history majors are similarly tested in the quarter in which they graduate. The examinations are designed and graded in such a way as to highlight levels of mastery of factual information, organizational and analytical skills, and ability to express ideas effectively. Comparison of entrance and exit performances is the key indicator.
(2) Exit and post-graduation polling. In their final quarter, History graduates are polled by the Department regarding their satisfaction with their original program of study, and the ways in which their program has affected their abilities to think and perform, both as individuals in society and as members of the work force. This polling is based upon our own questionnaire, and supplements polling done by Institutional Research. Subsequent polling is done by Institutional Research, using their own materials and a special supplementary form designed by us.
Graduate
In addition to that represented by graded results in regular course-work, including research seminars, and by completion of other degree requirements, the Department of History relies on the following mechanisms in order to monitor graduate student learning and achievement:
(1) Exit questionnaire. Upon graduation, MA and Ph.D. students are asked to complete a questionnaire regarding their satisfaction with various parts of their program of study, and their plans for the future. This polling is supplementary to that done by Institutional Research and is designed to address our discipline and departmental concerns more directly.
(2) Post-graduate polling. Graduates of the MA and Ph.D. programs are asked periodically to comment on their graduate training in perspective, and on the relationship between this training and their current employment. This subsequent polling is done by Institutional Research, using their own materials and a special supplementary form designed by us.
ASSESSMENT DATA ANALYSIS PROCEDURES
The Department of History utilizes data generated through the quarterly course evaluation system — a particularly important source of information for us — by examining results quarterly in the Undergraduate and Graduate Committees, and annually in the Peer Evaluation Committee. Other data generated by our assessment procedures are reviewed in the following ways:
Undergraduate
(1) The Curriculum Committee reviews annually the compared entrance and exit test scores of majors, and prepares an interpretive report on this matter. This report is the foundation for department-wide consideration of improvement measures.
(2) The above data, in addition to the responses from exit and post-graduation polling, form the basis for a regular (every five years) review by the Curriculum Committee and the Undergraduate Committee of requirements for the undergraduate major, again with an eye to achieving improved results.
(3) Faculty participating in the Mentor Program for Teaching Assistants and Teachers of Record, and those teaching pedagogy courses (such as "Teaching Non-Western History"), are asked to provide summaries of the above data and interpretations to those who will be teaching undergraduates.
Graduate
(1) The Graduate Committee reviews annually the exit questionnaires of graduating M.A. and Ph.D. candidates, the undergraduate and faculty evaluations of teaching assistants and teachers of record, and other information, such as comparative data on graduation, enrollment, and the like. An interpretive report from the Committee is the basis for department-wide consideration of improvement measures.
(2) The above data and reports form the basis for a regular (every five years) review by the Graduate Committee of the requirements for the M.A. and Ph.D. programs, with an eye to achieving improved results.
(3) Faculty participating in the Mentor Program, and those teaching pedagogy courses (such as "Teaching Non-Western History"), are asked to pay special attention on an annual basis to the available assessment data, and to incorporate them into their approaches. This activity is encouraged by the Graduate Committee and evaluated annually by the Peer Evaluation Committee.
ASSESSMENT DATA ANALYSIS FINDINGS
Many of our assessment activities are too recently inaugurated for these procedures to have been fully implemented. For example, we began our entrance testing of majors only in the Fall of 1995-96; our first exit test, and therefore our earliest basis for comparison, will not be given until June 1999. It is also the case that our major effort to defend the Ph.D. program to the Ohio Board of Regents, which required enormous efforts in 1994-95 and part of 1995-96, delayed and interrupted a number of planned assessment activities. Although the defense was itself an important exercise in self-assessment, we found the disruption to our planned activities regrettable. Nevertheless, we have been looking at such assessment data as we do possess. The most important findings are as follows:
Undergraduate
(1) Institution Research data drawn from our B.A. graduates over the past five years shows a dramatic and steady decline in those "not at all satisfied" with their major classes, from 57% in 1991 to 8% in 1995. While performance in the "extremely satisfied" category seems to oscillate, it is significant that taken together the "very satisfied" and "somewhat satisfied" categories also show steady improvement, from 43% in 1991 to 85% in 1995. We are not certain what factors have produced these results, but hope that our attention to curricular offerings, improvement of Teacher of Record and Teaching Assistant performance, and insistence on high-quality new hires have paid off in this way. We are aware, however, that the comparatively small sample covered in these surveys makes conclusions difficult.
(2) Another indicator of satisfaction with our program is drawn from Institutional Research material on several combined years of graduates over the past decade. These figures show that between 1986 and 1991, those who would change their majors if they had it to do over again dropped from 70% to 45%. Similarly, those who found their major courses "helpful" in their employment rose from 60% in 1986/1987 to 78% in 1990/1991. And this is in spite of other indications that fewer of our B.A. and M.A. graduates seek and find employment in the traditional teaching or other history fields. We think the data may reflect our increasing efforts to emphasize to students the ways in which historians’ skills can be utilized outside the historical profession, in the everyday, educated world. (The same data set indicates that fewer of our students consider themselves to have actually applied their major to their work, 89% to 76%, but at the same time fewer think it necessary to do so, 70% to 53%.) Apparently we are succeeding better — though still not as well as we would like — at making history "relevant" to a broad clientele.
(3) Institutional Research data also tells us that in key areas connected with our goals and desired outcomes, we are meeting or exceeding graduates’ needs. For example, in writing, analysis and evaluation, creative thinking, conveying meaning, organizing, and placing problems in perspective, we score consistently well, and appear to be frequently exceeding the needs of our graduates. Somewhat unexpectedly, we also appear to be making an increasingly strong contribution in less tangible ways: we are more than meeting — and have been improving our effectiveness in meeting — our students’ growing needs in coping with complex moral and ethical issues, and in dealing sensitively with the feelings and perceptions of others.
(4) Our first exit survey of students completing their B.A. (using a form developed particularly to reflect our discipline’s nature and goals, and our own department’s circumstances) has been of great interest. With a 38% response, we received an overall rating of 3.25 on a 4-point scale. In areas such as broadening intellectual horizons, improving ability to access and digest information, and understanding the context of world affairs, we score 3.3 or higher. Our faculty expertise, dedication, and responsiveness to students rate 3.37, 3.43, and 3.13 respectively. Our contribution to improving writing skills is rated a high 3.43, and our sensitivity to gender issues in courses is rated 3.26; these two areas are of special significance to university-wide goals. Only two areas seem to call out for immediate attention: one is use of appropriate technology in classrooms, rated 2.0, and the other is physical facilities such as building and classrooms, rated a dismal 1.06. Comments from the qualitative portion of the survey emphasize heavily the strengths of a large and varied listing of courses, and a highly competent, enthusiastic faculty. The thrust of most comments is that the department is academically demanding, but worth the effort. (One student’s remark that our grading was widely thought by his contemporaries to be deliberately tough in order "to balance out the high grades given by other departments," and that they were therefore afraid to take history courses even when they had an interest in doing so, gave us some unexpected food for thought.) Advising, which was rated 2.93 overall, received several comments of high praise, one of which expressed the opinion that the History Department set itself apart from other departments in the College on the high quality of its advising. On the whole, we feel that this initial assessment survey of our just-graduated seniors shows that we are performing well in nearly all goal areas, and indeed all goal areas over which we have full control.
Graduate
(1) In our recent review of all 20 of our Ph.D. graduates down to 1993-4, every one had found employment in their chosen fields, 18 in academia and 2 in the private sector; 16 were "extremely satisfied" with their program, 2 were "very satisfied, and 2 were "satisfied." On our questionnaire covering 13 different aspects of the doctoral program, the average rating was precisely halfway between "extremely satisfied" and "very satisfied." Consistently the highest rated aspects of the program were teaching and seminar and specialized courses; lowest-rated by far was the departmental physical facilities. Of the 20 doctoral graduates, 19 found their graduate training "extremely useful" or "very useful" in carrying out their duties. As Anthony Newberry (1982) wrote: "Though some might think my current role [Vice Chancellor of Academic and Student Affairs, University of Kentucky Community College System] at odds with my Ph.D. in History, the Ohio University experience is in fact the foundation of whatever success I have had. The immense value of the doctoral training in history has become especially clear in recent years, when several opportunities have put me at the center of policy analysis and public advocacy on a wide range of issues. Week in and week out...I draw upon the powers of analysis and interpretation and the knowledge of history developed under the...outstanding members of the history faculty." Finally, the publication record of these 20 graduates is impressive: as of June 1995, 16 books, 54 book chapters, 157 scholarly journal articles, and hundreds of other scholarly or professional products. Taken together, this information gathered in the course of our assessment of the doctoral program indicates to us that we are performing well, and seeing strong long-term results.
(2) In our just-completed review of the 9 Ph.D. graduates between November 1994 and June 1997, two extremely encouraging facts emerge. First, all but one have full-time faculty positions in four-year colleges or universities (1 is tenured, 4 are in tenure-track positions, and 3 hold term appointments). This is a very strong record, as is the longer one it complements: we have verified that, down to June 1997, all but one of the Ph.D.-level students we have graduated since the beginning of our program has found appropriate employment, generally within a short time. (The one remaining individual is a 1996 graduate located in Korea, and we have not been able to contact him.) Second, of the 8 graduates between 1994-1996, half had books accepted by major university presses (University of North Carolina Press, New York University Press, Columbia University Press, University Press of Kansas) and in various stages of production as of June 1997. This is almost certainly a speed record in the transition from graduate student to publishing historian unequaled by any other History Ph.D. program in the United States. Judged by the two important criteria of job placement and publication, our doctoral graduates appear to reflect a strong program indeed.
ASSESSMENT APPLICATION
Formal assessment activities — that is, activities beyond our long-standing careful examination of student performance and evaluations — is quite new to our department, and most of the improvements or enhancements we have made during the past year (drawing on our 1996 Assessment Report) are still largely associated with establishing a workable assessment program. Nevertheless, we have since June 1996 taken the following steps:
(1) In order to begin to address feedback that our departmental use of technology in the classroom is poor, we have established a Technology in Learning Committee. The first efforts of this group were to survey technology needs of Bentley Hall classrooms used most frequently by History faculty, and to submit two proposals for House Bill 748 funding. (One was funded, the other — in our view, unaccountably — was not.) We expect this Committee to take an increasingly important role in 1997-98, its first full year of operation.
(2) In order to continue to improve the ways in which our undergraduate students see how historians’ skills can be applied to a wide range of occupations, many of them outside the purview of professional historians, we have designed a new course entitled "Careers in History, History in Careers." Its chief aim is to broaden our students’ view of future employment opportunities, and to facilitate the application of historical skills in a variety of workplaces. This course will be required of majors and taught at the 200 level. After a planning process involving a full 25% of our faculty in one way or another, the course will come before Curriculum Committees for approval during 1997-98.
(3) In our initial examination of the way in which our graduates were surveyed by Institutional Research, we found that a number of questions were not well suited to our discipline and to accurate assessment of our goals and desired outcomes. For example, on questions concerning whether the recent graduate’s major was applicable to their employment, or useful in obtaining it, we invariably did poorly, since comparatively few of those responding to the survey (itself not a large number) appear to have entered the teaching profession; those who eventually would enter were still in graduate school.) We therefore designed a supplementary questionnaire that more clearly assesses our achievement of departmental goals; this will be used by Institutional Research for the first time in next year’s survey. We also designed our own exit survey for B.A. graduates, used for the first time this month, with results reported above.
(4) In order to address the circumstance that apparently fewer B.A. graduates are now entering the history profession or history-related professions than was the case earlier, we have surveyed 40 historical institutions and organizations in Ohio, with an eye to creating summer and winter-break internships with them for our majors. Work will continue on this project during 1997-98.
(5) In order to begin to address the most serious problem referred to in our assessment data — poor physical facilities in Bentley Hall — we have attempted to pursue in every way open to us and as vigorously as possible the building of new facilities. We do not feel we have met with much success or even an accurate understanding of how critical this problem is, however.
(6) In order to address a problem which did not show up in any assessment data, but which we became aware of as we pursued other questions, we modified our Honors Tutorial College major to match our Arts and Sciences major by fashioning a formal non-Western track. This will be available to students beginning in 1997-98.
PROPOSED CHANGES
AND RECOMMENDATIONSOur assessment data and experience is still much sketchier than we would like, and undoubtedly improvements will occur to us in the coming academic year, based on continuing perusals of assessment data. At present, however, we have already made the following decisions based on readings of the material available to date:
(1) Our experience, over the past two or three years, with formal program assessment beyond what we traditionally accomplished has convinced us that such activities represent a far greater commitment of energy and resources that we were led to believe. They cannot be effectively carried out in our department’s present organizational structure. Therefore we propose to form, beginning in September 1997, an Assessment Committee to coordinate aspects of assessment now handled by three or four different departmental committees, and to handle the considerable work of compiling annual reports.
(2) Because our Ohio Board of Regents doctoral program review was so disruptive and time-consuming, many of the new procedures we intended to institute in 1995-96 regarding assessing M.A. and Ph.D. graduate program performance were overlooked. We intend to return to these procedures and institute them in the 1997-98 academic year; our exit survey for M.A. and Ph.D. graduates, for example, will begin then.
(3) Because so may of our assessment materials are new, we intend to re-examine as many of them as possible in 1997-98 and make any changes that seem desirable.
(4) We have come to believe that realistic departmental assessment cannot take place in isolation from assessment of the institution around it. There are a great many aspects of institutional support which are crucial to departmental success but which are beyond the control of the department itself. For example, departmental assessment data suggest strongly not only that our students have uniformly very negative evaluations of our Bentley Hall facilities, but that the condition of these facilities is a major factor in the slight (but potentially worrisome) downward trend in enrollments we have noted in the past three years. (The quality and diversity of our offerings seems to have been the major factor in our nearly doubling undergraduate enrollments between 1983 and 1994.) Yet we can do nothing to control the circumstances of our deteriorating facilities; our efforts in this area have been frustrating and, we think, ineffective. In order to begin to address this problem more powerfully, we intend to initiate an institutional effectiveness survey of our teachers starting in 1997-98. Full- and part-time faculty, doctoral candidates functioning as teachers of record, and teaching assistants annually will be asked to evaluate institutional support for all aspects of teaching and research, including physical facilities and technological capabilities. Our annual evaluation of institutional support will be shared with the dean of Arts and Sciences, and will also become part of our annual assessment report. This form of assessment will help ensure that assessment of our departmental performance is seen in proper context.