J. Warren McClure School of Communication Systems Management

Assessment Report, June 1998

Assessing the effectiveness of their curriculum and the level of preparation of their graduates is an ongoing, critical concern for the faculty of the J. Warren McClure School of Communication Systems Management (CSM). Because the voice/data/imaging telecommunications field is in a constant state of change in technology, regulation, and industry structure, graduates of a professional program like that offered in CSM must have both the grasp of current issues and technologies to gain entry into the field, and the broad-based skills and theoretical knowledge to adapt and prosper as the field changes. The CSM faculty seek feedback about curricular effectiveness and input regarding potential changes from a variety of sources, including alumni, the School's Advisory Board, recruiters, internship sponsors, and readings in trade publications. The relatively small size of the faculty makes it possible for every faculty member to play an integral role in curriculum development.

The mission statement of the School of Communication Systems Management, as stated in the School's Strategic Five-Year Plan, continues to be the following:

The School faculty made no changes to the mission statement this year. To accomplish this mission, the faculty articulated specific educational goals/objectives, incorporating changes suggested in last year’s assessment report.

1. Our students should have a clear understanding of who the regulators and policy makers are in the voice and data communications industry and of how they interact.
2. Our students should have a basic knowledge of Local Area Networks (LANs) and Wide Area Networks (WANs), of how they work, of what their functions are, and of how they are established.
3. Our students should have a basic knowledge of voice technologies; i.e., what is included in voice technologies, the basic technical underpinnings of these technologies, and the uses of these technologies.
4. Our students should be able to analyze a real-life communication technology case study in order to identify the technical, regulatory, and management problems and challenges; formulate and present options; and make recommendations for optimal solutions.
5. Our students should have an understanding of emerging communication and information technologies, their technological underpinnings, and their potential impact on the delivery of voice/data/imaging services.

The faculty continues to regard these as appropriate goals and objectives for the School.

The methodologies used to assess how well these goals/objectives are being accomplished include: 1) alumni feedback received from Institutional Research; 2) alumni surveys; 3) Advisory Board surveys; 4) use of a capstone course.

As indicated in the assessment report for the 1996/1997 school year, the faculty made a number of changes in the assessment process used in that year, compared to the 1995/1996 school year. It seems important to us at this time to maintain a consistent set of assessment tools to be able to spot trends which may develop quickly, either externally due to the rapid pace of change in the industry, or internally due to changes in the curriculum, teaching methods, or staffing of individual courses. The same tools as last year were therefore used as a basis for this year’s assessment. We were unable to conduct an in-depth review of internship journals this year, but intend to resume the use of that tool next year.

The faculty regard alumni feedback as an important assessment tool. The voice/data telecommunications field is one of constant change; the faculty hope to equip students with a grasp of the basic technical and regulatory theories which will help them understand and respond to a changing field. For this reason, feedback from recent graduates and older alumni are extremely useful indicators to the faculty in judging effectiveness of curriculum.

One source of alumni feedback regarding alumni satisfaction with their jobs; alumni satisfaction with their major courses; and alumni satisfaction with preparation for their career is the annual alumni survey conducted by Institutional Research. The latest Institutional Research report, titled "Ohio University Career and Further Education Study," revealed the following trends for the graduates of 1994 through 1996:

Alumni responses suggest that the School continues to surpass the University average in alumni satisfaction with their jobs, their major courses, and their preparation. The category "satisfaction with present position" were continues to be high. Satisfaction with courses in the major continues to rise. The alumni rate overall preparation for career goals slightly lower in the most recent set of responses. We will be watchful for any indications as to the source of that change, should it continue in future years.

Another dimension of alumni feedback measured by the "Ohio University Career and Future Education Study" is placement. For the five-year period 1992-1996, the report showed a consistently high placement rate for Communication Systems Management alumni:

From these figures, it is evident that placement for graduates of the CSM program continues to be high; well over half of all students are employed before graduation, and all respondents indicate that they have found employment within 4-8 months.

Starting with the 1996/1997 we have started an annual survey of CSM alumni. As outlined in the assessment report for that school year, surveys are sent to alumni 2 years and 5 years after graduation. Recent alumni are asked about their ability to enter the workforce having completed the CSM curriculum. Older alumni are asked to provide information on changes in the industry to gain a foundation for possible curriculum changes.

Surveys were sent to 47 members of the class of 1993, 10 responses were received. Key assessment-related information from these responses is:

• Of the 10 respondents, 7 work in the telecommunications industry.
• Of the 10 respondents, 9 felt better prepared than newly hired employees in similar jobs, 1 felt equally prepared. Aspects of the CSM program that contributed to their preparedness were identified as
• "Knowledge of a broad range of technologies", "general knowledge of many subjects" – this validates the concept of electives in the CSM curriculum
• "the business courses I took better prepared me for budgeting and overall business dealings", "I was prepared to communicate well with people" – these responses indicate that the non-CSM course required in the curriculum are effective.
• The respondents indicated a range of topics that were not part of the curriculum at the time of their graduation, as well as general replies on important aspects of the industry today:
• Wide area networking – this topic is now covered in more depth in COMT 222 and COMT 325.
• Stress Management – we are not equipped to provide that type of instruction.
• MS Office, Procomm, etc. – the curriculum now requires either CS 120 or MIS 100, and PC-based applications are used extensively in all courses.
• ATM, ISDN, Frame Relay – these topics are now covered in COMT 220, COMT 222, and COMT 325.
• Finance, Operations Management, TQM, advanced Accounting, International Business, Stock and Bond Markets – students have the ability to explore topics such as these using the area of concentration in the curriculum. Since students must obtain advisor approval for the area of concentration, the information provided by the alumni is available to current students, through their advisor.
• Data electronics – covered in COMT 310, COMT 312, and a seminar on digital systems.
• Routers, switches, data cabling – covered in COMT 222 and COMT 325.
• The closeness of voice & data communications – this is taken into account in COMT 220, which is no longer solely a voice communications class, but rather focuses on systems and applications in the public switched network.
• Call accounting (billing) – covered in COMT 220
• Wireless voice and data – this topic is currently covered in a seminar format, which was taught with expanded enrollment this last year. Adding a permanent class to the curriculum is still being investigated.
• The 1996 Telecommunications Reform Act – this is covered in detail in COMT 214, COMT 220, COMT 302 and COMT 304.
• Broadband technologies (Gigabit Ethernet, SONET) – covered in COMT 325; however, we are investigating the possibility of a course on emerging technologies.
• Voice over IP, Voice over Frame Relay – these topics are not currently covered, their inclusion in an appropriate course needs to be looked at.
• Use of the Internet – Internet protocols and operation are covered in detail in COMT 325, while some aspects of Internet-based applications are studied in the recently introduced COMT 335 class on client-server data bases. No conclusions have been reached yet regarding further coverage of Internet issues in the CSM curriculum.

Surveys were sent to 68 members of the class of 1996, 12 alumni responded. Among the 12 respondents, 10 work in the telecommunications industry. Nine respondents felt better prepared for their position than non-CSM graduates, 2 felt at least as well prepared (1 did not answer). Reasons cited for being selected for their first job included internship experience, knowledge of the industry (including jargon and acronyms), and labs. When asked for specific aspects of the program that proved valuable to their jobs, key points mentioned included:

1. Overall industry knowledge, balanced coverage of a wide range of topics, a "broad base from TCP/IP to key systems"
2. Network security
3. TCP/IP
4. Hands-on work in the labs

In addition to confirming the value of the broad-based curriculum developed by the CSM faculty, these responses also point to a number of recent changes in the curriculum as beneficial.

In response to the question about the least valuable aspects of the program, the following points emerged:

1. Programming was mentioned both as not valuable, and as not being covered enough (by different respondents). This divergence of job requirements is typical for the telecommunications industry. Students should be able to use the area of concentration to cover subject matter that they may wish to pursue as part of their career choices.
2. "Lack of hands-on experience" and the need for additional labs was cited repeatedly. We have already added a numerous lab exercises to many CSM classes. The recent introduction of the practicum course, and the internship requirement is of course too new to appear in this survey yet, so we look forward to tracking any effects from these changes.
3. Regulation was mentioned as less valuable by a couple of respondents. On the other hand, other respondents listed regulation as part of the broad education in telecommunications they felt was important. We do not intend to modify this balance in the curriculum.

As outlined in last years report, we are very mindful of the fact that many students enter the CSM curriculum as transfer students. A measure of successful service to our student population is the overall rate at which students complete the program, regardless of the time at which the entered.

Compared to the analysis presented last year, we have made some refinements to better capture the intent stated above. We define an inactive student as:

1. The student was not enrolled in OU classes in the Spring quarter, nor was the student studying abroad.
2. The student had at one point attempted to take the CSM introductory course (COMT 100 before 1993, or COMT 214 after 1993).

This definition leaves out exchange students as well as freshman or transfer students who are admitted to the program but leave the university before ever starting the CSM curriculum. This approach allows us to focus on the impact of our curriculum, and changes to the curriculum, on a student once they make an effort to get started in CSM.

For the 1997/1998 school year, we have found the following statistics:

Currently enrolled students: 216

Alumni: 667

Inactive Students:

(as defined above) 45

Completion Percentage: 95%

Going forward we will carefully track any changes in this number to spot problems in the curriculum.

The CSM advisory board consists of about 15 industry professionals from both communications users and communications service providers. They meet twice a year, and are routinely asked to participate in a strategic planning exercise for the school.

Given a slight but noticeable increase in non-traditional students in the program, we asked the question should the CSM curriculum contain offerings for non-residential students? From the perspective of the industry professionals, continuing education options are considered valuable. They noted that courses or workshops need to offer college credit to be attractive. A good portion of the discussion centered on delivery mechanisms. Distance education is clearly preferred by these professionals. They also expressed a preference for "asynchronous" courses, i.e. courses where at most a small portion of the course work involves a fixed schedule. On the other hand, "self-study" courses were not considered to be of much value; an instructor should be setting the "pace" of the course. We consider these points valuable input to an on-going discussion among the faculty about serving currently employed industry professionals.

Since our most recent curriculum revisions included mandating some form of practical experience before graduation, we feel that it is even more urgent to recruit students into the program as early as possible. We presented current OU recruiting mechanisms and scholarship award procedures to the group, and asked what suggestions do you have to increase freshman recruiting, including the recruitment of women and minorities? All board members agree that the CSM major should be attractive to high-school seniors due to the currently very strong job market in telecommunications. Several suggestions regarding the procedures for awarding scholarships were made; these involved not the actual selection criteria, but the timing and follow-up. We will study these before the next recruiting cycle. The board members also agreed to investigate any programs that their respective companies may have in place with local high-schools. During the next academic year we will attempt to participate in at least a few recruiting activities in conjunction with these existing programs.

COMT 444, "Management of Communication Resources," is required of all CSM majors; students complete COMT 444 after they have completed virtually all other program requirements. COMT 444 functions as a capstone course in which students are required to apply what they have learned as they have progressed through the curriculum to case studies involving communication technology, policy, and strategic issues. As a capstone course, COMT 444 serves two valuable purposes; the course measures how well individual students have mastered the subject matter of the CSM curriculum, and, just as importantly, the course measures the coverage of essential topics in the CSM courses leading up to COMT 444. One of the duties of the faculty teaching COMT 444 is to share with the rest of the faculty any trends or potential problems which he or she has detected in student knowledge and preparation.

The faculty member teaching COMT 444 has not reported any new concerns this year. We are still working to address the issues raised last year, namely:

• Familiarity with the circuit ordering process
• Ability to use PC spreadsheets in assigned projects.

As a result of the previous assessment activities, the faculty pursued several initiatives, not all of which could be completed:

As a result of alumni surveys, Advisory Board exercises, and the capstone course, the faculty will consider the following enhancements or additions to the curriculum:

2. Develop practicum projects based on the criteria defined this year.

Submitted by the CSM Faculty:

Back to Assessments