With three of the eight Group I faculty positions unfilled, the 1997-98 academic year was a challenging year for the IMSE department. The remaining five faculty balanced course work, research and service to meet the full IMSE program demands, with teaching help from retired faculty and group III instructors. Despite these challenges, we completed a successful job search and hired two outstanding new faculty members. Dr. Dale Masel, Ph.D. Penn. State, will join the faculty in the fall and will help with Plant Layout and Operations Research. Dr Masel will bring to the department important expertise in facilities design. Also, Dr. Tom Kaiser, Clemson University, currently, an assistant professor at Southern Colorado, joins the faculty in the fall. Dr. Kaiser was awarded the Outstanding Faculty Member award at the University of Southern Colorado and we are pleased that he will be joining the department.

Last year’s assessment identified several improvements and enhancements which are listed below. Under each, the steps taken to address each of these concerns is presented.

1. Identify strengths and weaknesses in the new curriculum. Using future assessments, the effectiveness of the new curriculum will be determined.

Assessment continues into the effect of curricular changes on the undergraduate program. As can be seen from this year’s analysis, no significant negative issues have yet to surface from the switch to senior specialty areas. In fact, several seniors praised the new curriculum in their exit interviews.

2. Refine our assessment measures. Using the ABET 2000 criteria, the IMSE department will develop evaluation procedures as specified by the engineering education community.

The College of Engineering and Technology has chosen to follow the old criteria for accreditation and the ABET 2000 assessment will begin after that accreditation review (fall 1998) has commenced.

3. Improve the oral and written communications for both undergraduate and graduate students.

In class assignments in undergraduate classes and in the winter graduate seminar (Dr. Koonce) we are stressing written and oral communications.

4. Improve the research skills of graduate students.

The objectives of the IMSE department are unchanged from the 1996-97 academic year. The following were the 1997 objectives of the IMSE department:

1. Students will be able to properly engineer the design of manufacturing systems and their associated facilities.
2. Students will be able to analyze the economic consequences of their engineered designs.
3. Students will be able to make oral and written presentations of their engineered designs.
4. Students will be able to perform their work in an ethical and professional manner.
5. In addition, recipients of the Masters of Science in Industrial and Systems Engineering degree will learn and demonstrate advanced knowledge of industrial and systems engineering.

The IMSE department is currently in the third year of a new undergraduate curriculum. The key aspects of the new curriculum are the senior specialty areas. We feel that continued assessment will identify strengths and weaknesses in the senior specialty areas.

The IMSE Department Assessment Committee feels that the three best indicators of the effectiveness of the Industrial and Systems Engineering program are:

1. External review - Using the engineers and managers who direct seniors projects in the capstone design course.
2. Internal review - Using exit interviews of graduating seniors by the chair of the department.
3. Student review - Using the two-year survey conducted by institutional research.

Responses from these three sources will be mapped into the objectives to identify successes and failures. Comments from the sources will be utilized to improve the program where appropriate.

At this point we should note that the previous year’s report included a survey of faculty. A weakness identified by the faculty, writing skills in undergraduate and graduates is being addressed by increased writing in undergraduate classes and the graduate seminar. However, we are replacing the faculty survey with senior exit interviews and an expanded questionnaire to the external reviewers.

The following three sections will detail the three data collection methods.

In their final two quarters, ISE seniors must complete the ISE 445A and B sequence. This capstone senior project course places the students in a real-life engineering design situation with engineers and managers, usually off-campus. For the projects, the students, in small teams, must apply their knowledge and skills to analyze a problem and design a workable solution. At the end of the project, we survey the external managers to determine how our students performed, including engineering analysis, design and professionalism. There are 24 questions in the survey, with respondents supplying a numeric response of 1 to 4. A score of 1 is inadequate, 2 adequate, 3 good and 4 is outstanding, room for comment is provided after each question. Table 1 gives the questions and responses.

Out of 158 question responses, 58 (36.7%) were outstanding and 97.5% were adequate or higher. The lowest team average was 2.4, the highest was 3.6, with a mean of 3.19 and variance of .17. The highest scores were tallied for ethical behavior (3.9) and ability to work with employees at different levels (3.8). The lowest scores were for detail design, visualization and writing communication (2.7). A more complete analysis will be done as the responses are mapped into the department goals.

This academic year, the department instituted exit interviews with graduating seniors. These were voluntary interviews and about one-half of the graduating seniors chose to participate. The students were asked several questions about their university and departmental experience. In general, the students were very positive about the university in general and the IMSE department in particular. There were a few specific concerns regarding some required courses that were taught outside the department. These "service" course concerns will be brought to the college service course committee next fall for discussion. There were also some program improvement suggestions and these will be discussed at the IMSE faculty meetings next fall. While these exit interviews were good and pointed out some current curriculum challenges, they will need to be reviewed again after the institutional survey results are compiled.

Each year, Institutional Research surveys students who graduated two years earlier. A request was made to IR on May 29, 1998 for the latest survey results for Bachelor of Science graduates in ISE. The report received was identical to the report received in May 1997. Therefore, no new information is available to assess this year’s performance. The remaining assessment will utilize external and internal review sources.

To assess the department, each goal will be evaluated using the two sources detailed previously. Strengths and weaknesses will be identified and methods for addressing the weaknesses will be presented.

Many questions in the senior project survey apply directly to this goal. Questions 1 through 12 and 14 through 16 provide an assessment of the student’s ability to properly engineer systems. The first phase of the design is to accurately assess the situation and identify the real problem, questions 1, 2, 3 and 4 address this phase. Table 2 gives the results for these questions and the average for all four of these questions was between good and outstanding. This appears to be an area where our students receive adequate instruction.

Individual comments include:

"Quality summary & documents most useful"

"…Would have liked to see a better cost/payback analysis…"

"…I feel the student’s comprehension of the operation and practices of the Signal Shop was great."

"They thought up more ideas and uses than we did."

The next major phase of design and analysis is to identify the requirements for a system to solve the problem. Questions 5 through 8 address the requirements determination and alternative evaluation necessary. Table 3 gives the results from the survey and shows that the averages were all 3 or above. Again, this area of engineering analysis appears to be adequately covered in the current curriculum.

Individual comments include:

"Needs outline fairly comprehensive"

"The methodology (#8) was too mechanical with insufficient common sense type reasoning"

"The design was highly creditable – I would liked to see some other alternatives"

"They identified three alternatives, didn’t go into much details on the two they discounted"

With the problem identified and the alternatives evaluated, the system design can begin. Questions 9 through 12 addressed the students’ ability to design effective solutions to the identified problem. Table 4 gives the results from the design questions. It can be seen that the averages for 10 and 12 were below average and reasons must be identified. In the detail design phase, preliminary designs must be expanded to develop workable systems. It is at this phase that many initial designs fail. Teams with low scores received comments that stressed that the level of detail was not sufficient. The other weakness was shown in visualization of the solution. Many charts and methodologies exist for visualization and we attempt to expose the students to many different methodologies for them during their program. However, as in most professional activities, the effective use of many of these techniques, e.g. IDEF0, only comes from long experience.

Individual comments include:

"I did not get the impression that they achieved the required level of detail"

"I am more interested in the conclusions and recommendations than the methodology"

"I never really saw in depth flow charts or a list of proposed functions."

"Program was developed on the fly, while in the implementation phase"

"The approach … is quite typical of what is found in the real world à Total Chaos"

Lastly, the solution provided must be implementable and effective. Questions 14 through 16 address the quality of the solution and its impact to the organization. The scores in this area averaged around 3. While we can teach the students the fundamentals of engineering, these skills often require experience.

Individual comments include:

"It will utilize all of our current computer equipment with minimal investment… It’s doable."

"It will speed up the process with less cost of information and equipment."

"We will use it right away."

"The ideas actually came from the client."

"Further in-house work necessary…"

"Not fully evaluated but very intriguing solution"

One of the primary criteria for a solution is its economic feasibility. Industrial engineers must be able to determine all costs associated with a solution and determine if the results justify the costs. Question 13 from the survey directly asks if the solution was analyzed from an economic perspective. While all but one response was good or outstanding (see Table 5), the average for this question was lower than expected at 2.8.

Individual comments include:

"Setup and equipment costs were in sufficient detail but labor costs were omitted in comparison of alternatives."

"They were able to justify everything that was recommended."

In their curriculum, ISE students practice analyzing and designing systems which are developed using the math, science and engineering skills learned. While in class, these solutions are often presented, for grading purposes, without much explanation. In practice, these solutions must be presented to management and other engineers who do not have either the technical background or detailed understanding of the problem to interpret the results. As such, poor oral and written communication skills limit an engineer’s effectiveness.

In their senior project, the ISE graduates must communicate effectively with all levels of employees to analyze the problems, develop the solutions, present solutions to management and implement the chosen solution. Questions 19, 20, 21 22 and 24 address these communication issues. Table 7 presents the results and shows that our graduates are strong in communication skills. It should be noted that faculty in the IMSE department have, through assignments, begun to stress reporting and presentation skills.

Individual comments include:

"I was impressed with this aspect of dealing with this group." (#20)

"Communication and conduct were excellent."

"The in class presentations were ok but, their meetings with me did not seem to accomplish much."

"Students answered every question asked."

A hallmark of engineering is the assumption that engineers are honest and ethical in their work. The IMSE department stresses ethical behavior in all professional activities. In the senior project survey, the questions 17, 18 and 23 evaluate the student’s behavior. Question 17 asks if the students acted in a professional manner and the average response was 3.6. Question 18 asked if the students acted in an ethical manner and the average was 3.9. The last question, 23, asked if the students kept the project manager informed about the status. While not a factor of ethics or direct professional behavior, maintaining contact and informing a client is crucial to the development of a successful system. The score for 23 was an average of 3.2. Table 8 gives the results of the individual teams on these questions.

Individual comments include:

"Most cooperative and professional. Well mannered"

"The group met with me once at their request… they had nothing prepared and nothing to show…"

The primary means for graduate students to learn and demonstrate "advanced knowledge of industrial and systems engineering" is through the successful defense of the thesis. The IMSE Department requires all MS students to complete a thesis. The thesis committee is comprised of at least 3 graduate faculty members including one from outside the department who represents the college.

The review process showed two possible areas for improving our undergraduate instruction: more detailed analytical techniques in courses and the use of economic analysis in other courses. We will review this preliminary finding and see what actually needs to be done.

Back to Assessments