The report of the assessment committee of the School of Electrical Engineering and Computer Science contains a description of the activities of the committee during the 1996-97 school year and the rationale for those actions. Included in the report is a description of the formation and composition of the committee and also the purpose the committee feels it has in the school. After this, the report lists the main activities of the year in the areas of developing a mission statement and objectives for the school, analyzing data provided to the department from institutional research and implementing a pilot assessment loop to improve the consistency of material covered in courses. Finally, a brief outline of the activities that the committee will be focusing on next year is given.

The assessment committee of the School of Electrical Engineering and Computer Science was formed in February of 1997. Since the merger of the Department of Electrical and Computer Engineering and the Department of Computer Science in 1995, there have been two separate assessment committees. It was felt that it would be more efficient and would promote unity by having a single committee. The committee is composed of seven faculty members including the school chair and assistant chair. Members were selected to include a cross-section of faculty ranks, primary emphasis (teaching, research and service), and industrial experience. An effort was made to include faculty members that have expressed some skepticism about the assessment process. Table 1 lists members of the assessment committee.

The committee believes that there are two main purposes of the assessment process. The first is to document the performance of the school to interested external entities such as Accreditation Board of Engineering and Technology (ABET), Computer Science Accreditation Board (CSAB), the North Central Association of Colleges and Schools, and the Ohio Legislature. The second purpose is to attempt to improve the educational outcomes. While the committee will do what is necessary to satisfy the first purpose, it intends to focus its efforts on the later and more important purpose. To this end, the assessment committee has adopted the following statement of purpose:

The committee also believes that the assessment process should be an integrated part of the program rather than an activity added on to the program. However, the committee believes that the assessment process, while important, is only one part of our program and efforts spent on assessment must be balanced with the other activities of the faculty and staff of the school. This is important because the school’s programs in Electrical Engineering (EE) and Electrical Engineering with Computer Engineering Option (EE/CEO) will undergo accreditation review in Fall of 1998. Because the review process is greatly facilitated by a stable curriculum, no major changes will be made to the program until after that visit. After this review cycle, ABET is changing from a curriculum-based accreditation process to outcome-based process. The new criteria, called ABET 2000, requires that the programs have detailed, published objectives, that program graduates demonstrate at least eleven specific outcomes, and that programs have an assessment plan that shows that the actual outcomes match the stated objectives and use the results to improve the effectiveness of the program. The Computer Science (CS) program is currently not accredited but the curriculum was significantly revised for students entering in 1997 to meet the accreditation requirements of CSAB. The accreditation visit cannot occur until the first student graduates under an accreditable curriculum, which will be Spring of 2001. Again, until this time the curriculum will need to remain relatively stable. Instead of instituting systematic assessment procedures that would soon become obsolete, the committee has devoted its efforts this year to four tasks that they feel best serve the long term interests of the school: 1) Develop a mission statement and objectives for the school, 2) Analyze other existing assessment information for its importance to our objectives and address the most obvious problems, 3) Make recommendations based on Competencies Needed and Developed reported by Alumni in the Five-year Educational Outcomes Survey, and 4) Implement one assessment loop to address a significant problem in the school.

The committee’s first task was to develop a mission statement and objectives for the school. They will be used as the foundation on which decisions about such things as curriculum, course loads, allocation of resources, and the assessment procedure will be based. Instead of trying to merge the old EE and CS missions and objectives, the committee started over to develop a mission and objectives that were consistent with the needs of the constituents of the school and agreeable to the faculty who have to implement the objectives. We identified students, our faculty, employers, Russ College and Ohio University, ABET, CSAB, North Central, and the State of Ohio as constituents of our programs. After listing their needs, the committee identified the mission of the school as consisting of the four points listed in Figure 1.

Based on the mission statement, the committee derived the following eleven objectives of the school:

The objectives have now been issued to the faculty for their consideration, comments and discussion over the summer. It is planned that during Fall term the objectives will be modified as necessary and adopted by all the faculty. Since these objectives will guide many of the future activities of the school, the committee believes that it is very important that the entire faculty assent to the objectives.

Since the school has not adopted objectives, the committee could not take the approach of developing objectives, then determining outcomes that demonstrate the objectives, then specifying assessment activities and criteria that measure the outcomes, and finally deciding what actions to take if the criteria are not met. Instead, the committee tried to focus on data provided by Institutional Research to find information to improve our program. The committee first looked at the Competencies Needed and Developed reported by our alumni in the Five-Year Educational Outcomes: Survey of Alumni.

As always, good assessment practices should focus effort on those areas of a program which are important but exhibit a weakness. Those areas can be visualized graphically utilizing a plot of the need or the importance of a skill, knowledge or attitude versus the effectiveness or level of satisfaction in preparing students for it. The Competencies Needed and Developed section of the Five-Year Educational Outcomes: Survey of Alumni provides data that can easily be presented in this fashion. Figures 2 and 3 show graphs that plot the 86-91 EE and CS alumni’s average response, respectively, when asked about the need for each competency versus their perception of OU’s effectiveness in preparing them for it. Lines are drawn at the average values. As can be seen from the plot, in general most competencies appear in the upper right hand corner. That is, we are effectively preparing our graduates for the tasks they have found to be most important for them to perform their jobs. Items in the upper left quadrant are the competencies that are important but the alumni feel that they were the least prepared for. The only item in that quadrant for both programs is sensitivity to the feelings and perceptions of others. After this, the ability to organize and supervise the work of others and the ability to communicate orally are indicated as the next two items that should be worked on.

The committee first discussed ways to better prepare students in the area of sensitivity. It was felt that by nature, sensitivity comes to some people more easily than others, so our goal should not be to make our students sensitive but to assist them to better deal with others. Interacting with other people is probably the best way to do this, so the committee has recommend to the faculty a continued increase in the amount of teamwork done by students. The problem with sensitivity was further emphasized by the results of a survey of employers that found that the fourth most desired attribute of an entry-level engineer (after the ability to learn on their own, an in-depth technical knowledge and experience communicating technical information) is experience in working with people of different gender, race and cultural background. ET 280: Engineering and Technology: an Overview, which is required of all freshman, addresses many of the non-technical aspects of engineering. In 1992 an examination and discussion of the Meyers-Briggs work preferences was introduced in the course, but the effects of this would not yet have shown up on the five-year survey. This year a discussion of diversity in the workplace was added to this course to help students in this area.

The committee felt that increasing the amount of teamwork done by students will also help better prepare them to supervise others. This is generally not a problem graduates will have until some years after they have graduated, so an obvious solution like requiring a course in management might not be the most effective solution. Instead, giving the students the ability and motivation to engage in life-long learning, one of our new objectives, will enable them to seek out training in this and other areas when it will benefit them the most. The Russ College of Engineering and Technology has recently established the Robe Leadership Institute. We will investigate utilizing this resource when it becomes more developed.

In regards to oral communication, the survey was given to students who graduated prior to 1992. Graduates in EE after 1992 were required to take a course in public speaking, INCO 103. In addition, since this time a final requirement has been added in the EE capstone design course, EE 495 - Electrical Engineering Design. Even so, because of the importance of oral communication, it was decided that next year more instruction about presentations should be given to students in EE 495 and that the presentations should be publicized and open to all faculty. A recommendation will be given to the CS curriculum committee that ENG 305J and INCO 103 be made prerequisites for the CS capstone course CS 456: Software Design and Development.

In addition to the Competencies Needed and Developed, 61 different items that Institutional Research measures on the students and graduates of our program were identified. Again, the committee tried to focus on the most important items. This was done by having members of the assessment committee rank the twenty items they felt were most important in descending order. Based on this survey, the 61 measures are listed in Table 2, divided into the three groups: most important, not as important, and least important.

For the twenty measures deemed most important the committee discussed and decided a quantitative criteria which they felt that the school should achieve. In Table 3 each of the measures is listed with its rank, rating (lower is more important) and specified criteria.

After the committee decided criteria, it examined the data from Institutional research for the two programs. In Table 3 the results for each of the twenty most important measures are given for each program. The 5-year data is for 1990 and 1991 graduates, the one year data is for 1995 graduates, the longitudinal data is for 1995 seniors, and the COMP data is a rolling average of 1993 through 1995 seniors. As can be seen, 17 of the twenty criteria are met or exceeded in both EE and CS. From the table, the two most significant problems are dissatisfaction with the elective courses in the Computer Science curriculum, and the advising in Electrical Engineering and Computer Science.

The graduates in Computer Science in the 5-year survey all graduated from the old program located in the College of Arts & Sciences and had to meet the elective requirements of that college. Moving the program to the Russ College of Engineering and Technology has allowed for more flexibility determining the elective courses. The new Computer Science curriculum replaces some of the elective courses with specific courses that will be useful to the students in their profession. It also gives students more flexibility by allowing them to substitute technical courses for one year of foreign language requirement.

The most serious deficit was found in the satisfaction of the advising and guidance provided by the program. Additional measures were made in this area to determine the extent and cause of this problem. By counting the number of DARS forms returned to the office from advisors after advising week, it was found that approximately 75 to 80% of the students did not visit their advisor. Because of this, during Spring quarter, students who did not see their advisors were required to sign an advising waiver to obtain their DARS form. This was done to indicate that the school considers advising important and that the normal procedure should be for students to visit their advisor at least once each term. In addition to this, students were asked to check the reason they did not see their advisor. 46% said they did not need advising, 31% said that they were not able to schedule an appointment with their advisor and would talk to him or her later, 10% said they did not realize it was advising week and 13% said other. In a survey of Spring graduates a section was added requesting a complaint about advising and a suggestion for improvement. Although the analysis of these surveys is still in progress, initial indications are that there is dissatisfaction with both the technical aspects of advising (course offerings, perquisites, …) and the attitudes of the advisors. Corrective actions will be formulated over the summer by the school’s administration and implementation will start next Fall.

The assessment committee also wanted to implement one complete assessment loop which would take an objective, assess the outcome of that objective, and then take corrective action if necessary. The loop was desired to be relatively simple so that the committee could gain experience with the complete assessment process and the faculty could easily see that the assessment process can be used to help them educate their students. The committee started with the objective that our graduates of the undergraduate programs will have the theoretical, practical, and professional knowledge necessary for them to be immediately productive upon entering the work force or advanced study. The student outcome related to this objective is most easily assured by making students pass a set of required courses taken in a progressive fashion. Each subsequent course builds on the material of previous courses and the sum of the material in the courses constitutes the necessary body of knowledge. However, it was noted by the committee that there is no formal mechanism to determine if the specified material had actually been covered in a course. One means of determining this would be to administer an exam over this knowledge to seniors, however, this would require a substantial effort on the part of students and faculty. Instead, the committee has proposed that at the conclusion of each course, the instructor will complete a survey which indicates what topics have been covered in lecture and which have been included on exams. This information will be collected by the central office and can be used with samples of the exams to assess whether students have mastered the necessary knowledge.

Next year the committee will finish and build upon the work started this year. After the faculty completes and adopts a mission statement and objectives, the committee will start specifying the outcomes that will demonstrate the fulfillment of the objectives. The committee needs to find additional measures for the identified problems in sensitivity, supervision, oral communication and elective courses. Since much of the assessment activity for Electrical and Computer Engineering will probably occur in EE 495 and the assessment committee represents a good cross section of the faculty, this committee will be the core course committee in charge of EE 495. It will also need to monitor efforts made to improve advising and course content uniformity.

New activities will be centered on providing support to the redesign of the EE and EE/CEO curriculum planned to start next year. The committee will design and implement assessment activities such as alumni or employer surveys to provide information to faculty about what is needed in the curriculum. It will also make sure that the structure of the new curriculum facilitates assessment. In addition, the committee will filter the large amount of data that will be collected for the upcoming ABET visit and provide feedback to those areas that need attention. Finally, while this year the committee’s assessment efforts have been applied to the undergraduate programs, next year it will begin looking at other aspects of the program such as graduate education, research and service.