J. Warren McClure School of Communication Systems Management

Assessment Report

Spring 1996

Submitted by the Faculty of the McClure School: Phyllis Bernt Phillip Campbell Hans KruseTony Mele Trevor Roycroft Varadharajan Sridhar

J. Warren McClure School of Communication Systems Management Assessment Report, Spring1996 Summary

The faculty did a total curricular re-write in 1993 after an exhaustive survey ofalumni and meetings with the Industry Advisory Board. Curriculum development is an ongoingprocess, with all faculty actively involved in analyzing the effectiveness of courses andhow well courses build on one another. This report addresses the assessment methodologiesidentified in the response to the North Central Accreditation Report, as well as otherassessment methods.

Alumni feedback, job placement: Faculty analyzed the Office of Institutional Research's"Career and Further Education Study" for alumni feedback and job placement. Inthe areas of satisfaction with present position, with major courses, and with careerpreparation, responses of McClure School alumni are significantly more positive than thoseof total Communication alumni and University alumni. In the area of job placement, 1994School alumni indicated that 74% obtained jobs prior to graduation, 21% obtained jobswithin 2 months; and the remaining 5% obtained jobs within 3 months of graduation. Anindicator of employer satisfaction with School alumni is the number of companies whichconsistently hire alumni. As an example, MCI had hired 50 alumni as of March 1996 and hasmade offers to six June graduates.

Course embedded assessment: A measure of course embedded assessment is the percentageof students who do not attain the required 2.0 in the first four courses in the majorsequence. Results suggest that these courses are serving as successful checkpoints toassure that students are demonstrating mastery of subject matter. As an example, 20% ofCOMT 214 students did not attain a 2.0 the first time.

Completion Rates: Faculty examined an Office of Institutional Research report on degreecompletion rates to assure that courses are being scheduled in a manner which allowsstudents to graduate in a timely manner. Completion rates for the School exceedUniversity-wide percentages.

Capstone Course: All students are required to complete COMT 444, a case-study-basedclass which serves as a capstone course. COMT 444 also serves as a barometer for howcomprehensive coverage of topics has been in earlier courses. Various enhancements haveresulted in lower-level COMT courses as a result of faculty analysis of COMT 444 outcomes.

Internship Program: Journals of the students completing internships are regularlyanalyzed to assure that students are adequately prepared and that tasks performed byinterns are covered in the curriculum. Faculty are considering requiringinternships/practica of all students.

Industry Survey: Results of an industry survey regarding important topics which shouldbe covered were compared to current course offerings. Several new course offerings wereidentified.

Changes Implemented: COMT 302 and 304 were expanded; network services and servicepricing components were added to COMT 220; lab exercises were added to COMT 222;client-server database course developed as experimental offering this summer.

Changes Proposed: Requiring internship/practica; adding courses in wireless technology,computer-telephony integration, and voice processing as regular offerings; analyzingoptimal placement of project management exercises; developing exit interview procedure forfurther assessment now that all students will have completed the new curriculum.

--Our students should have a basic knowledge of voice technologies; i.e., what is included in voice technologies, the basic technical underpinnings of these technologies, and the uses of these technologies.

--Our students should be able to analyze a real-life communication technology case study in order to identify the technical, regulatory, and management problems and challenges; formulate and present options; and make recommendations for optimal solutions. (p.98)

The methodologies to be used in assessing how well these goals/objectives have beenaccomplished were identified in the University's response to the North CentralAccreditation Report as including 1) alumni feedback; 2) course embedded assessment; 3)exit interviews; 4) a capstone course; and S) internship evaluation. The faculty of theSchool have applied all but one of these assessment methods to evaluate how welleducational goals/objectives are being met. Exit interviews have not yet been undertakenbecause of introduction of the new curriculum The last students subject to the oldcurriculum will be graduating this year. The faculty wished to wait until exit interviewscould be done with students who had all experienced a stabilized, fully implementedcurriculum.

Evidence of Accomplishment of Goals

Alumni Feedback:

The use of alumni feedback as an assessment tool is appropriate because the McClureSchool offers a professional program of study. Alumni, upon reflection and after someexperience, are able to provide feedback about how well the McClure School, and OhioUniversity, have prepared them for their current situations. The alumni surveys conductedby Institutional Research provide valuable information and feedback, especially in theareas of alumni satisfaction with their jobs; alumni satisfaction with their majorcourses; and alumni satisfaction with preparation for careers. These reports, because theyinclude data for alumni at the University, the College, and the School levels, alsoprovide a context for assessing successful and problem areas.

The latest "Ohio University Career and Further Education Study" revealed thefollowing about the 1994 graduating class (for comparison purposes, results from 1990-93surveys can be found in Appendix A of this report):

Alumni responses suggest that the School surpassed the College and the University inalumni satisfaction with their jobs, their major courses, and their preparation. Thefaculty were particularly interested in responses from the 1994 class because this was thefirst graduating class which had experienced some elements of the new curriculum.

Another dimension of alumni feedback is placement. It can be assumed that ease ofplacement is one reflection of alumni preparation The latest "Ohio University Careerand Further Education Study" report showed that, over a 5-year period, alumni of theSchool have experienced a consistently high job placement record:

While placement in the aggregate is an indicator of whether alumni have acquiredappropriate knowledge and skills, an additional indicator is the number of organizationsthat have made the School a recruiting resource. Organizations which, having hiredstudents from the School, return repeatedly to hire additional students, exhibit a highsatisfaction level with the type of preparation students are receiving in the School.School records show that, as of the March 1996 graduating class, the following companieshave consistently hired graduates of the program:

The numbers of graduates hired by these organizations vary by the organizations'internal hiring practices and growth needs. For example, Ameritech has hired very fewentry-level positions during the past five years, while LCI has hired most of the 23graduates listed above in the last two years. MCI has exhibited a consistent, annualhiring pattern for the past five years, and has already made commitments to hire sixstudents from the June graduating class. LCI has made commitments to hire at least threestudents from the June class. NCR is hiring two students; CompuServe has scheduledon-campus interviews for late May.

Because of consistent hiring by these employers, faculty members have developedcontacts and working relationships with recruiters coming to campus and have used thesecontacts as opportunities to discuss industry developments and preparation levels of thealumni. In response to input from recruiters, RFP exercises and cost/benefit analyses havebecome a larger part of the COMT 444 class.

Course Embedded Assessment:

In order to assure that students are learning the requisite skills and informationdetailed in the School's goals/objectives as they progress through the program, thefaculty, in designing the School's curriculum, have established a series of checkpointsThe first four courses in the curriculum (COMT 214, 220, 222, 302) build a necessaryfoundation of knowledge upon which students must build as they move through the balance ofthe curriculum Students must earn a grade of C (2.0) or better in those courses beforethey can progress further (Students are allowed to retake these courses subject toprovisions in the University Catalog.)

Data for the fall and winter quarters of the 1995-1996 academic year show that thesecourses, especially COMT 214, are serving as successful checkpoints

The relatively high percentage of students with grades below a 2.0 in COMT 214 may beindicative of the number of potential transfer students who enroll in that course as afirst step in determining whether they do indeed wish to become Communication SystemsManagement majors. While it is impossible to ascertain exactly why some students do notearn a 2.0 in a specific class, it does appear from these figures that students are notautomatically being passed through the curriculum without demonstrating some mastery ofthe subject matter.

Completion Rates:

The length of time it takes to graduate and the ability to get into required courses ina timely manner is a major concern of students, and their parents. The issue of timelygraduation and completion rates is of special concern in this program because of the largepercentage of transfer students. Of the students who are currently actively pursuing adegree in the School, 72% of them were transfers into the program; the other 28% werefreshmen admits. This 72% transfer student percentage is uncharacteristically low becauseof the unusually large number of freshmen admits this past fall. Information fromInstitutional Research and from School records indicate that the School is servingstudents well in this regard. According to an Office of Institutional Research Report ongraduation rates, the School fares well in comparison to the rest of the University.

When the program is examined from its inception, the percentage of students who havenot been able to complete the program is small. According to School records, as ofDecember 31, 1995, there were 537 alumni of the School; 44 inactive students (since thebeginning of the program in 1981); and 225 active students. Those who transferred out ofthe School are not included in these figures; the number of students transferring out ofthe program tends to be relatively small. Though a rather rough indicator, the inactivestudents, as a percentage of those who are in the School and who have graduated from it,is 5.5%. It appears that classes are being scheduled in a manner and in a quantity whichserve the students' needs. This is especially significant to the faculty because the newcurriculum increased COMT course requirements, first from 6 to 8 courses and then from 8to 11 courses.

Capstone Course:

All majors are required to complete COMT 444, "Management of CommunicationResources." Students enroll in COMT 444 after having completed virtually all otherrequired courses. This course serves as a capstone course in which students are requiredto analyze communication technology case studies in which they apply the knowledge andskills they have acquired as they have progressed through the program. Not only do facultyuse COMT 444 as a way of assessing each student's comprehension of desired skills andknowledge, they also use COMT 444 as a barometer for how comprehensive the coverage oftopics has been in the earlier Communication Systems Management courses. Faculty teachingCOMT 444 routinely share with the rest of the faculty at regular faculty meetings anytrends and problems they have identified.

Enhancements which have resulted from the COMT 444 analysis have included expansion ofthe regulatory sequence, COMT 302 and COMT 304, from two three-hour classes to twofour-hour classes because students in their COMT 444 case studies were not exhibiting anadequate knowledge of rate structures and tariff provisions. Enhancements which are beingimplemented this year have to do with additions to COMT 220 in the area of networkservices and service pricing and added lab exercises in the COMT 222 data applicationsclass. Between the laboratory hours required in COMT 312, which is primarily a laboratorycourse, and the laboratory exercises which are required in conjunction with the COMT 220and COMT 222 lecture courses, students receive 36 hours of hands-on laboratory experienceslaboratory exercises are a component in some of the elective COMT courses as well.

Internship Evaluation:

The School has since its inception had a competitive paid internship program. Studentswho have completed the first four courses in the program, and who have met specified GPArequirement, function as full-time, paid employees for a summer, or for a quarter duringthe school year. Students are expected to keep a journal of their experiences and theirsponsor is expected to evaluate their performance. Students enroll for credit in COMT 401as part of the internship. One of the faculty members serves as internship coordinator,identifying internship sponsors, coordinating interviews and placements, reading journalsand performance reviews, and assigning credit hours.

The internship coordinator, in reading the journals and the performance reviews, tracksthe type of projects and functions which interns are asked to complete and also assesseshow well students are prepared for the internship experience. The journals prepared by theinterns during the summer of 1995/6 showed that students worked on projects which arecovered in the curriculum: order tracking, RFP development, circuit testing, frame relayimplementation, PBX administration, LAN maintenance, voice mail administration.Evaluations by sponsors spoke to the professionalism of the interns and their ability tobe useful members of a work unit.

The issue of internship experiences has been a topic of discussion at faculty meetingsbecause of the growing concern on the part of the faculty that all students should havethe type of practical experience offered by an internship. This concern has grown asemployers are telling faculty members, and students, that they look for internship, orpracticum, experience in their hiring process. Faculty members have also noted a markeddifference in performance between students who have had practical experiences and thosewho have not. As a result, faculty members have begun creating, on an ad hoc basis,special projects, or practica, for students who do not have internships in order toprovide them with hands-on experiences. At this point, a little over half of the studentsare doing School-arranged internships. Other students, who do not meet the currentinternship requirements, are arranging for summer employment on their own, or are turningto the faculty for special project experience.

The issue of internships was a topic for the spring meeting of the School's IndustryAdvisory Board. The Advisory Board agreed with the faculty that the internship experienceshould be more widely available to students; indeed, the Board recommended thatinternships be required of all students. Those students unable to do an internship shouldbe required to complete a practicum experience. The faculty is considering making aninternship or practicum a requirement for graduation for all students in the program.

Industry Survey Results:

The faculty is currently in the process of writing a proposal for a master's degreeprogram. In aid of that undertaking, a survey of industry professionals has been conductedto accomplish two goals: to assess interest in the proposed program and to seek inputregarding the topics which should be covered in the course content. Because the graduateprogram will be modeled on the approach taken at the undergraduate level, the facultyfound the industry surveys enlightening regarding sufficiency of coverage by theundergraduate courses. The faculty read and discussed the results of the industry surveycarefully, identifying trends and also examining whether the topics mentioned are coveredin the current curriculum.

Comments on the surveys could be grouped into three areas: suggestions for technicalcourses; suggestions for policy courses; and suggestions for strategic management issuesThe faculty in analyzing the comments related to the technical areas found that currentcourses are covering LAN/WAN technologies (COMT 222 and 325); network management anddesign (COMT 329); ATM and frame relay technologies (COMT 220,222,325,391,312); systemrouters, hubs, and servers (COMT 222,325); traffic engineering (COMT 220, 312); systemsecurity issues (COMT 379); and voice communication (COMT 220,310,312). The topics whichare not covered by existing courses include wireless technologies, computer-telephonyintegration, voice processing applications, and client-server database applications. Someefforts have already been undertaken to develop these courses; an experimental course inwireless technologies has been offered and a client-server database course will be offeredthis summer with funding from the Provost's summer pool. Further work will be done indeveloping all of these course offerings. In the area of policy, the faculty found thatthe current courses cover the areas mentioned pretty fully; courses include the history ofthe industry (COMT 214), international issues (COMT 407); rate making fundamentals (COMT302,304); tariffing (COMT 302,304); competition and deregulation (COMT 302,304); and theimpact of current federal legislation (COMT 304,405). In the strategic management arena,current courses, especially COMT 444, cover RFP basics, case studies, tariff and contractmanagement, and cost-benefit analyses. The faculty found that the area of projectmanagement may need to be covered more thoroughly in the curriculum.

The issue of internships was a topic for the spring meeting of the School's IndustryAdvisory Board. The Advisory Board agreed with the faculty that the internship experienceshould be more widely available to students; indeed, the Board recommended thatinternships be required of all students. Those students unable to do an internship shouldbe required to complete a practicum experience. The faculty is considering making aninternship or practicum a requirement for graduation for all students in the program.

Improvements/Enhancements/Developments Implemented:

Improvements or enhancements already completed, or in the process of being implemented,are explained above; they include: --expansion of COMT 302 and COMT 304 to four-hourcourses --addition of network services and service pricing to COMT 220 --addition of labexercises to COMT 222 --development of a client-server database course to be offered thissummer

Recommended Changes for the Future:

The faculty is considering the following changes or enhancements: -requiringinternship/practicum of all students -adding courses in wireless technologies,computer-telephony integration, and voice processing as regular offerings in thecurriculum -examining where in the curriculum the inclusion of project managementexercises fit best in the curriculum. A topical seminar in project management has beenoffered, and the COMT 415 course, which could accommodate a project management component,is being evaluated. -developing an exit interview procedure for further assessment ofprogram outcomes

Appendix A Ohio University Career and Further Education Study

Source: Provost's Office

Back to Assessment Main Page