William W. Reeves, Professor ______________________________

Contact: 593-1451

James F. Fales, Chairman ______________________________

Contact: 593-1456

John Deno ______________________________

Associate Professor

Peter Klein ______________________________

Associate Professor

Timothy Sexton ______________________________

Associate Professor

List of Appended Material ii

Identification of Student Learning Goals 1

Methods of Assessment 2

Analysis of Data 2

Evidence of Goals Accomplishment Based on Assessment Data 5

Implemented Improvements or Enhancements Based on Assessment 7

Recommended Future Changes in Academic Program/Curriculum 9

Recommended Future Changes in Assessment 10

Appendices 11

A. Mission Statement and Goals for the

Department of Industrial Technology 11

B. Follow-up Survey of Recent Graduates of the

Department of Industrial Technology - 1996 13

C. Summary Results from the Office of Institutional

Research Career and Further Education Study - 1996 34

D. Student Responses to Class Assignments in the

Senior Seminar Course 41

E. Summary of Results of Student Performance on the

Society of Manufacturing Engineers Certified

Manufacturing Technologist Exam - 1996 52

F. Minutes from the Department of Industrial Technology

Industrial Advisory Board Meeting - May 1997 57

G. Competency and Performance Based Criteria From

Selected IT Course Syllabi 62

H. Selected Student and Supervisor Reports From

Co-op Experiences - AY 96-97 67

I. Employer Survey Results of Recently Graduated

IT Students - AY 95-96 129

ii

The student learning goals for the Department of Industrial Technology are encompassed within two of the four overall departmental goals (see Appendix A):

a. to provide effective quality education through personalized teaching,

advising, and applied laboratory activities.

b. to provide current and relevant technical and managerial content through

applied research and on-going interaction with industry.

Faculty were involved in establishing these goals in 1993. The goals are reviewed annually by the departmental faculty, our Industrial Advisory Board, and occasionally by other external bodies such as the National Association of Industrial Technology (our affiliated accrediting board). They are revised (if necessary) annually in reports to the Dean of the Russ College of Engineering and Technology, and other external reviewing bodies when appropriate. These goals were formulated to be consistent with the mission statement of the department (see Appendix A):

Student learning goals are assessed using clearly defined multiple methods that include annual faculty review and presentations to the dean of the Russ College of Engineering and Technology. Furthermore, these goals are systematically assessed through annual review by the Departmental Industrial Advisory Board, and on a planned six year cycle by the National Association of Industrial Technology (NAIT) Board of Accreditation. This review seems appropriate and ensures that the goals are pedagogically sound (through faculty review), are consistent with university goals (through the dean’s review), are reflective of expectations of potential student employers (Industrial Advisory Board review), and are consistent with the highest expectations of the profession (NAIT review).

These reviews are analyzed by the chair of the Department of Industrial Technology and the departmental faculty representative to the Russ College of Engineering and Technology’s Assessment Committee (Assessment Representative). Through this review, the two goals mentioned were not modified for the current year.

Multiple methods of student learning assessment are encompassed within the overall assessment plan the department uses to collect data. These methods include:

* Follow-up survey of IT graduates conducted annually by the department

* Office of Institutional Research follow-up surveys of graduates

* Capstone course criteria and performance

* Senior seminar feedback

* Student performance on standardized Society of Manufacturing Engineers (Certified Manufacturing Technologist) tests

* Annual interviews of students by the Industrial Advisory Board

* Competency and performance based activities outcomes in classes

* Graduate placement data from Departmental and University Career Planning and Placement

* Supervisor reports from Co-op placements

* Departmental program matriculation data

* Surveys of graduates’ employers, and

* Intra-faculty cooperative interaction.

Since our last analysis of student learning data, we became interested in the appropriateness of data received from the Office of Institutional Research Longitudinal Academic Involvement Study. Through meetings with the Director of Institutional Research, we learned that although the results of the Student Involvement Study were indicative of areas needing to be addressed in our program, the population was biased due to the fact that we have very few (typically less than 10) direct entry freshmen and over 100 seniors during any year. This study focuses on changes from the freshman to the senior year and reported data may give "false negatives" in our case. Although most of the results were positive it was apparent that students reported that they did not get to "socialize" with faculty as much as would appear to be appropriate.

3.1. Departmental Graduate Follow-up Survey. The Department administers its own follow-up graduate survey separate from the ones provided by Institutional Research (see Appendix B). Among other demographic data, graduates are asked to provide specific data on those courses that they deem to have been of special importance, those that may need improvement (or even potential elimination), areas of study requiring more emphasis, and potential new areas of study needed. These data are compiled by the Department Chair and are distributed to each faculty member and to the Industrial Advisory Board. Generally, these results are collectively discussed by the Faculty and Board during the annual May meeting to determine if areas of perceived weakness actually exist, and to plan strategies for overcoming identified deficiencies.

3.2. Institutional Research Graduate Follow-up Survey. Graduate follow-up surveys are administered by the Office of Institutional Research with the Ohio University Career and Further Education Study (see Appendix C). They provide an annual summary of the most recent graduates and a longitudinal study of graduates from five years prior. These data are analyzed by the Department Chair and the Assessment Representative. Close attention is paid to questions dealing with: "competencies needed and developed," "quality of instruction," "satisfaction with major courses," "overall satisfaction with education," and "changes graduates said they would make." Overwhelmingly, our graduates reported high levels of satisfaction in their education. In the total survey only two items reveal dissatisfaction at a level higher than 50% - they are Career Planning and Placement (60% not at all satisfied), and Participation in Community Service (43% said that OU was effective; however, only 23% said that this competency was needed). The most common change that graduates said they would make is to "have an internship (co-op)." 76% reported that they would participate in this type of activity if they had it to do all over again.

3.3. Capstone Course and Senior Seminar. Capstone course data are provided from assignments in IT 400 Senior Seminar and through performance reviews by instructors of IT 462 Product Manufacturing. The assignment data from IT 400 are analyzed by the instructor and the Department Chair (who happen to be the same person) and developed as anecdotal reports. The reports are distributed to each faculty member and to the Industrial Advisory Board to ascertain strengths and weaknesses in the program and curricular offerings as well as in student performance (see Appendix D). Student performance in IT 462 is continually monitored by the instructor of this course to provide feedback to other faculty teaching prerequisite courses. This feedback reveals strengths and weaknesses in specific areas of long-term retention of knowledge and skills, and results in strategies for overcoming perceived deficiencies while maintaining assets through faculty interaction.

3.4. Certification Examination. Twice annually, the department administers the Certified Manufacturing Technologist Examination, sponsored by the Society of Manufacturing Engineers. The Society provides a summary of performance within cognate areas on the test (see Appendix E). The faculty member who administers the test is given these data and compiles and distributes them to each faculty. The results are discussed at subsequent faculty meetings. Obviously, those cognate areas scoring highly are considered to be strengths and those scoring lowly are considered to be potential weaknesses.

3.5. Industrial Advisory Board Meetings. The Industrial Advisory Board meets with a group of no less than six and no more than twelve students during their annual meeting in May. Faculty are not present so as to allow for free interchange. During this two hour meeting, the Board solicits input from the students on perceived strengths and weaknesses in the program, and individual areas of concern from each student. This meeting takes place on a Friday afternoon. The Board compiles a summary of that meeting during the evening and their report is given to the faculty during a follow-up Saturday morning session. During the Saturday session courses-of-action are devised to remediate weaknesses and identify strengths. These are recorded in the official minutes of the Board (see Appendix F).

3.6. Competency and Performance Based Class Outcomes. Cognitive and psychomotor competencies have been devised for certain courses within the curriculum (see excerpts from selected course syllabi in Appendix G). In these courses, students are expected to perform operations and/or satisfactorily complete testing requirements before progressing to other higher order areas of study within the class (or pursue other horizontal areas of study). This method of study/performance assures that each and every student has gained an acceptable level of competence before completing the courses. Instructor analysis of the results of this competency approach, also reveals areas in which modification of instructional methods may be needed.

3.7. Co-op Placement Reports. Every co-op student is required to write a report on his/her experience. Every workplace supervisor is required to report on the performance of that co-op student and perceived deficiencies in the student’s preparation (see Appendix H). These reports are collected by the Assistant Dean for Career Development, compiled and forwarded to the department Co-op Coordinator and the Department Chair. Data from these reports are analyzed by all concerned parties, resulting in strategies for determination of continuation of the co-op placement site, and adequacy of the Departmental program to prepare students for the co-op experience.

3.8. Departmental Matriculation Data. The department keeps accurate data of the progress of students through the program, including date of entry (since most of our students transfer into the Department), number of hours attempted and completed before enrollment in our program, and number of hours attempted and completed when graduating. These baseline data provide an overview of the effectiveness and efficiency of our course offerings, help with scheduling of courses, and provide profile data of the "typical" IT graduate’s matriculation through the program.

3.9. Employer Survey. A new assessment instrument and procedure implemented during the past year is the employer survey (see Appendix I). In this survey, departmental graduates are asked to pass-along a brief survey to their employers. The results of this survey are analyzed by the Department Chair and the Assessment Representative to determine the perceived adequacy of the academic preparation of their employees (our graduates). To date, the returns from this survey are sparse; however, with refinement and modification, these types of data can be valuable assessment tools.

3.10. Intra-Faculty Cooperative Interaction. The informal interaction between faculty concerning student learning is one of the most difficult assessment methods to measure results. However, it is probably one of the most effective intervention mechanisms to improve student performance. In our small department, the faculty "get to know" our students and the faculty interact constantly with one another concerning student learning issues (both collectively and as individuals). Faculty, working in this cooperative network, plot strategies to improve student performance in class.

Evidence of accomplishment of the goal to provide effective quality education through personalized teaching, advising and applied laboratory activities is reflected in various data received and analyzed by the Department.

4.1. Strengths Reflected in Graduate Follow-up Studies. In this year’s Ohio University Career and Further Education Study, 84% of the departmental graduates responding reported that they were either "extremely satisfied" or "very satisfied" with the major courses that they took in their undergraduate program (0% reported "not at all satisfied") (see Appendix C). The student outcomes goals related to quality education are also reflected in the departmental Follow-up Survey of Recent IT Graduates. Although the questions related to quality of education are "open ended," graduates did (as expected) express the need for more course related to their specific current employment. These same students reported that the most relevant courses that they took during their undergraduate program were, of course, highly related to their current job responsibilities and reflect, collectively, the "worthiness" of nearly every course offered in the IT baccalaureate program (see Appendix B).

4.2. Strengths Reflected in Certification Examination. The results of the SME Certification Exam verify the quality of instruction provided in our undergraduate program (see Appendix E). This exam evaluates overall student performance in seven composite categories: Math/Applied Science, Materials, Design, Manufacturing Processes, Management/Economics, Quality Control, and Computer Applications. Students taking the latest offering of the test scored above the 50th percentile in all categories. It appears from these results that the strengths of our program lie in Materials (78th percentile), Manufacturing Processes (63rd percentile), and Management Economics (63rd percentile). Subcategorical areas within Materials that appear to be strengths include Non-metallic Materials (96th percentile), Heat Treatment (85th percentile), and Plastic/Rubber Processes/Assembly (83rd percentile). Subcategorically, the strengths in the Manufacturing Processes area include Production Planning and Control (92nd percentile), Jig and Fixture Design (78th percentile), and Assembly (76th percentile). Subcategorical strength areas within Management/Economics include Technical Writing (85th percentile) and Communications (83rd percentile). Subcategorically, it appears that strengths also exist in areas not having especially high overall performance including Metrication/SI (100th percentile), Fluid Power (85th percentile), and Production Planning and Control (92nd percentile).

4.3. Strengths Reflected by the Industrial Advisory Board. The Industrial Advisory Board in the minutes from their May 1997 meeting reported that the students they met with were highly competent and appeared to be progressing toward a goal of being prepared for employment following graduation (see Appendix F).

4.4. Strengths Reflected in Student Performance. Competency based exams (written and/or practicals) are currently given in many classes including IT 217 Production Metals Machining, IT 308 Industrial Plastics, IT 320 Hydraulic Controls, and IT 332 Electronics I. With pass rates well above 90% in these classes it is evident that this approach is providing the desired "positive" student outcomes. (These competencies also correlate highly with the higher scores on the SME certification exam.)

4.5. Strengths Reflected in Retaining Students. Departmental program matriculation data indicate that only one student major was dropped from the program between spring quarter 1996 and winter quarter 1997 (with an average of 150 majors this represents less than 1% retrenchment).

There are, however, some data that indicate that our goal of quality education is not being fully accomplished.

4.6. Academic Advising Concerns. Students in the IT 400 Senior Seminar course this year repeatedly reported the need for improved academic advising [although the Ohio University Career and Further Education Study indicated that graduates were either extremely/very satisfied or satisfied with the academic advising and guidance (83%) (see Appendix C)] and the need to have laboratories open and accessible during evening and weekend hours (see Appendix D).

4.7. Communications Skills Concerns. The Industrial Advisory Board reports that we continue to need to improve the communication skills of our students (see Appendix F). This appears to possibly be contradictory to the results of the SME Certification Exam (see Appendix E).

4.8. Technical Content/Achievement Test Concerns. SME Certification Exam results indicate weaknesses in the Math/Applied Sciences category (60th percentile) [especially in Physics (25th percentile) and Statics/Dynamics (25th percentile)], in all areas of the Design category (50th percentile) [Drafting/Engineering Drawing (45th percentile), CADD (57th percentile), Human Factors Engineering (57th percentile), and Product Design (47th percentile)], in the Quality Control category [especially in Quality Control/Quality Assurance (57th percentile), Statistical Process Control (35th percentile), and Inspection (58th percentile)] and in the Computer Applications category [especially in Computer Aided Manufacturing /Computer Integrated Manufacturing (25th percentile), Computer Numerical Controls/Numerical Controls Programming (58th percentile), and Robotics/Automatic Systems (47th percentile)]. For further detail see Appendix E.

4.9. Co-op and Career Planning and Placement Concerns. The results of the Ohio University Career and Further Education Study indicate that taking advantage of the internship/co-op experience would be greatly beneficial to students in our program (78%) although only 17% participated in the program while in school and that Career Planning and Placement Services were not satisfactory (60%) (see Appendix C).

4.10 Strengths and Weakness in Curricular Content. Evidence of accomplishment of the goal to provide current and relevant technical and managerial content through applied research and on-going interaction with industry is, first, manifest in the fact that industrially supported research funding in the department increased by 1376% during the last reporting period ($5,496 in AY 1994-95 to $81,118 in AY 1995-96). In addition, during the past year, four of the eight full time faculty members had textbooks published. The department is an active participant in the Russ College of Engineering and Technology Co-op program having had 25 student participants during the past year. Feedback from the industrial workplace supervisor seems to indicate that our students are well prepared for this experience (see Appendix H). Of course, the co-op experience itself results in student perspectives more reflective of current and relevant content in courses that they take when they return to campus. Faculty and students frequently visit industrial manufacturing facilities and students work as outside contractors for area industry on special projects. Faculty have also taken extensive industrial update training in their areas of expertise, as well as providing these types of services for industrialists and other educators. Unlike some other educational advisory boards our departmental advisory board is totally comprised of practicing industrialists, providing the unique perspectives of our students’ future employers.

There are probably many deficiencies in our current program associated with this goal. However, through analysis of input from the Industrial Advisory Board meeting we have learned that we need to expand our somewhat parochial approach to industrial outreach to reflect global perspectives (the manufacturing industrial workplace is now, more than ever a world-wide effort) (see Appendix F). Although we take pride in the broad based approach to industrial manufacturing evident in our program, feedback from former graduates and from the current senior level seminar course indicate that more depth of coursework, in some areas, would be appropriate. This is especially true in areas such as Engineering Drawing/Dimensional Analysis and Plastics. Many of our current graduates are finding employment in these two areas. Finally, it is apparent that we need to expand student participation in the co-op program. In discussions with students during advising periods, it was learned that many times parents are hesitant to have their daughters/sons take advantage of this program (due to lack of understanding of the program or reluctance to relocate to the workplace away from the Athens campus).

The assessment data reveal that we should continue doing what we do well: provide quality undergraduate educational experiences for students. However, some refinement and continual improvement has been planned based on the assessment data.

5.1. Technical Area Course Expansion. We have removed most prerequisites from the beginning Industrial Plastics course and moved it from the 300 level to the 200 level. This allowed us to redesign and expand this area of study to include not only the one required course, but also to improve the accessibility of two existing elective courses in this area, and to design one new elective course offering. Similarly, we have redesigned the Engineering Drawing area, changing software from Cadkey to AutoCad in response to graduate follow-up survey results, including coursework at all levels (100-400) incorporating renaming the 100 level required course "Engineering Graphics Fundamental," renumbering (to 200 level) and changing the name of the second level required course to "Engineering Graphics Applications," changing the elective "Geometric Dimensioning and Tolerancing" to a 300 level with a prerequisite of the 200 level "Engineering Drawing Applications" course, and proposing a new elective course at the 400 level "Dimensional Analysis."

5.2. Improving Academic Advising. Although graduates responding to the Ohio University Career and Further Education Study reported a high degree of satisfaction (83 %) with academic advising, students in the capstone seminar course deemed this an area needing attention. In response, we have somewhat altered our academic advising methods. Although we still do not require each student to meet with her/his academic advisor we have included a sheet in each DARS package that strongly encourages each student to schedule an advising appointment. We have further facilitated this suggestion by having a posted schedule of available times outside of each faculty/advisor’s office door during the university advising periods. Students may sign-up for an appointment on this posted schedule. Staff monitor the sign-up lists and procure appropriate documents and student record files for advisor/advisee use at those assigned times.

5.3. Expanded Laboratory Availability. In response to the capstone seminar course suggestions and recommendations from the Industrial Advisory Board, we have expanded the hours of open labs in certain areas of the department. The metals machining laboratory is operated under student employee and instructor supervision during three hour evening sessions on Tuesday, and Wednesday during the spring quarter of this year. The open hours of operation for the departmental computer laboratory are set by the College and those hours have not been expanded in the last two years. However, initiatives taken by the Engineering Student Council during this past year to define guidelines for a proposed College "technology fee" include a 24 hour supervised operation model for all computer laboratories and College Computer Systems Administrators are currently examining an expanded operating format.

5.4. Improved Communications Skills. Although graduate surveys and results from the SME certification test reveal both a high level of preparation satisfaction and actual performance in communications, the Departmental Industrial Advisory Board continues to demand increased student performance in this area. During the past year, the department chair has attended various sessions provided by the University Writing Program and intends to develop a model for "Writing Across the Industrial Technology Curriculum" program during next year.

5.5. Improved SME Certification Examination Results. It is anticipated that the redesign of the Engineering Graphics area within the Department will improve student performance on this examination. Currently, students take their Engineering Drawing courses during their Freshman year and little formal follow-up coursework is provided by the professional staff in that area. With the proposed redesign, students will have the opportunity to refresh and advance their knowledge and skills in this area with a 400 level course just prior to the typical examination period. Although we are not responsible for the physics instruction provided to our students, many of the follow-up courses in our power area (mechanical power, hydraulics, electronics, etc.) do require a prerequisite knowledge in this area. As a result of the weakness reflected in the physics area on this test, the primary instructor in the mechanical/electrical power sequence is currently designing an entry assessment instrument that he intends to administer to all students in the IT 221 Power Transmission course in an attempt to determine areas of strength and weaknesses brought from the Physics experience into our power sequence. Although, these will be the two main foci of intervention planned for next year, we will continue to monitor SME exam results and plan strategies for overcoming other weaknesses as revealed.

5.6. Increasing Co-op Participation. Two factors impeding participation in co-op activities are: 1) Some students just don’t want to leave Athens after they get here and Southeastern Ohio simply does not have a broad base of manufacturing industry co-op opportunities; 2) Due to the few numbers of credit hours awarded while on co-op, participation may result in extending a student’s graduation date by as much as a year. To encourange participation in the co-op program and help parents better understand its advantages and disadvantages we have planned special activities and presentations to parents during next years parents’ weekend (September 26-28). A committee of students, with faculty guidance, has planned activities and personally invited every parent of every IT student to attend. To date we have received responses totaling nearly 80 parents who will come to Athens during that weekend. Marty North, Assistant Dean for Student Careers in our College, will give a presentation to those parents on Saturday morning explaining the co-op program and benefits to the students that may be derived from participation.

5.7. Increased International Exposure For Students. In response to two different data input sources, the Industrial Advisory Board and the Department Follow-up Survey of Graduates, we have designed one and are pursuing another avenue for providing pertinent international experiences for our students. A new part-time faculty member (with extensive international manufacturing travel experience) has designed a for-credit (6 hours) course in which he and our department chairman will take up to 17 undergraduate students to Argentina to tour industrial manufacturing facilities for two weeks. Follow-up seminar sessions will be held during the winter 1998 quarter for these students as a portion of the requirements for credit. To date, seven of our students have registered for the course. We anticipate offering similar experiences during the December intersection on an annual basis (pending assessment of this original offering). Furthermore, one member of the Industrial Advisory Board has offered to take five students to Germany for five weeks at his company’s expense for a combination of work experience and various manufacturing facilities tours. We are currently pursuing this possibility.

Many major changes in Program/Curriculum were cited in previous sections of this document. It is apparent that with the increased technology in manufacturing our attempts to keep current and relevant will require new strategies. Foremost of these will be the combinating of some existing courses that appear to be overly emphasized. This will allow us to expand coursework in areas deemed to be more current and relevant. Currently, we do not have a graduate program and relatively few exist in Industrial Technology throughout the United States. One avenue of expanding time to cover the body-of-knowledge is to establish a masters degree program. We are, by nature, a hardware driven curriculum. This does not simply mean computers, but machine tools, tooling, fixturing, etc. The support necessary to provide such a curriculum is relatively expensive compared to the traditional sIT down lecture classroom approach. This experiential learning approach is, however, our major strength. With what appears to be hard-times ahead in support for higher education, we will need to develop other avenues to provide the necessary education to develop our future manufacturing engineers. Rigorously planned programs of external support in the forms of grants, gifts, and equipment loans need to be made. We need to expand our horizons beyond the Athens campus, and place students in the manufacturing workplace through co-ops, internships, and short-term project partnerships while providing the continuing education to these students through distance learning outreach.

We do a wonderful job of gathering assessment data considering of the size of our department. Examination of the two goals cited in this document relate to providing quality educational experience, and current and relevant content. If we are to reach these goals, we must constantly improve and change based upon the assessment data that we obtain from a variety of sources. Assessment is nothing new to us; we have been gathering these sorts of data for the last thirty years. Our national accreditation board demands it, our sources provide it, and we respond to it in keeping with our faculty’s collective judgment. Since we have such a long tradition of gathering this information, we have relied upon our own traditional methods and have not adequately adapted to the formats and procedures that the university has recently imposed upon us.

Primarily we have relied upon two individuals, the department chair and the department representative to the College Assessment Committee to gather and analyze assessment data. In reality, this represents 25% of our staff, and few other departments on this campus probably devote as high a percentage of their staff time to this effort. However, it has become apparent that we need a standing departmental assessment committee. Next year we will have one, and it will assess whether a committee approach is any more effective/efficient than the two individuals approach. Furthermore, it is apparent that our streamlined departmental goals statements do not adequately express things in term of student outcomes to satisfy the requirements of the University’s assessment reporting scheme. Therefore, a first directive to the new departmental assessment committee will be to review current goals and revise them, if necessary, or devise a new set of "student outcome goals" to be used for assessment purposes.