School of Nursing
College of Health and Human Services
1997 ASSESSMENT REPORT
June 1997
Introduction
The School of Nursing at Ohio University offers the upper-division work necessary for practicing registered nurses (RNs) to earn the baccalaureate degree with a major in nursing. The BSN was implemented in the mid-1970s and originally housed in the College of Education. Its mission has always been to provide an accessible, professionally relevant, and academically challenging degree program for the RNs in the 16 counties of southeastern Ohio.
The majority of the RNs who enter the OU program are graduates of local community colleges and hold the associate degree in nursing (AND). They are in their early to mid-30s with a family and have held a full time position in an acute care agency for 10.5 years. Approximately 18% of these RNs were LPNs (licensed practical nurses) prior to entering the associate degree program. While half of the students have attended one postsecondary institution prior to their enrollment at OU, the other half have attended several postsecondary institutions. They have spent their academic life in the maze of transfer credits, course equivalency evaluations, and the tightly organized structure of prelicensure nursing programs. Recently, this student body mix has been challenged with an increasing number of individuals who have completed the associate degree program, taken the licensing examination, and immediately continued the pursuit of the BSN. The classroom has become a lively environment of new grads with fresh book knowledge and very little clinical experience and experienced nurses with considerable clinical experience and
somewhat older book knowledge.
The RN to BSN Program is taught on all six of OU’s campuses by a faculty of eight who are academically housed in Athens. The use of the higher education microwave system (HEMS)
in combination with travel to regional campuses has enabled the faculty to deliver the program to such a widely dispersed population. The faculty will begin using compressed video in the fall of 1997, and it is anticipated that some courses will be offered through the Internet by the spring of 1998.
The following report is the second of annual assessment reports. It is based in information gleaned from discussions at faculty meetings with the director of the School preparing the final version. The report builds upon the information gained in preparing the 1996 Assessment Report and looks forward to the work yet to be done.
The report is organized by summarizing the School’s information regarding its end of program objectives in relation to each of the six questions posed by the guidelines for the 1997 Assessment Report.
1997 Assessment Report
School of Nursing
page 2
1. What are your unit’s goals for student learning? Indicate how your goals were developed and who was involved. Have they been modified based on last year’s report?
There were nine end-of-program objectives listed in the 1996 Assessment Report
and a description of the realization of how very few of them were measurable. These objectives had been reviewed by the faculty on a biennial basis but the focus had been "Is this what we still believe?" rather than "To what degree are we accomplishing these?" They were a mixture of beliefs and objectives.
Following the discussion of the 1996 Assessment Report the faculty reviewed these nine objectives in terms of the changing environment of health care delivery and the criteria established by nursing’s national accrediting organization. The current draft of the School of Nursing’s end-of-program objectives follows:
The School of Nursing provides the RN with the opportunity to develop an enhanced sense of professionalism through the acquisition of a liberal and professional education that will lead to the achievement of the following objectives and which will provide a foundation for both practice and advanced education in nursing:
1. Design and deliver nursing care that reflects use of
critical thinking.
2. Understand the theoretical basis of clinical practice.
3. Acquire competence in communication skills.
4. Develop management skills related to the delivery of
nursing care.
5. Establish collaborative relationships with other health
care professionals.
The end-of-program objectives in the School of Nursing have two separate but related sources: (a) the faculty and (b) national accreditation criteria. The current national accrediting body (National League for Nursing or NLN) requires "program" evaluation with some student learning outcomes embedded in the criteria. These criteria are:
Criterion 21.A.1: Reflect students’ skills in reasoning, analysis, research or decision making relevant to the discipline of nursing.
Criterion 21.A.2: Reflect students’ abilities in areas such as written, oral and nonverbal communication, group process, information technology and/or media production.
1997 Assessment Report
School of Nursing
page 3
Criterion 21.A.3: Reflect students’ abilities to perform theory-based nursing interventions including psychomotor and psychosocial therapeutics directed at individuals and/or aggregates.
Criterion 21.A.4: Reflects numbers entering, length of time in program and numbers graduated.
Criterion 21.A.5: Reflects a mission-relevant outcome selected by the program.
The OU, School of Nursing has chosen to address the needs of the RN to BSN
population.
The development of these objectives occurred with input from all faculty
members. Given the small size of the faculty (8) it is possible to make decisions as a committee-of- the-whole. This is the preferred style of decision making and uses the usual process of a subset of the faculty developing a draft of a specific document. The draft is distributed to the faculty and a discussion and decision occurs in one of the monthly faculty meetings.
The objectives as listed above are scheduled for additional discussion and (assuming approval) guidance in specifying the outcomes expected of the students. Questions that need to be addressed in relation to Objective #3 would be: Which communication skills? Oral and written have been assumed, but to what degree? How shall these be specifically assessed? Communication via a computer is included in this area. How shall we assess "computer literacy"?
2. How did you assess student learning? What testing instruments, methods and processes did you use to collect assessment data? Have these assessment activities been modified since last year?
Data were collected during the 1996-97 academic year using the School-
developed Program Evaluation instrument (see Attachment A) which was modified by the faculty from materials used by the University of Maryland. This instrument was completed by individuals in the senior capstone course in June of 1996. One section of the instrument asked students for their evaluation of the "Degree to which the program helped you to meet the outcomes." Each of the nine end-of-program objectives was listed and the mean response across the nine items was 4.31 on a 5-point scale (range was from 4.20 to 4.44). The data are consistent with those collected by the Office of Institutional Research in the OU Career and Further Education Study:
Satisfaction with major coursework: 83% extremely or very satisfied in 1995
OU prepare for career goals: 83% extremely or very well in 1995.
OU prepare for additional academic work: 89% extremely or very well in 1995 1997 Assessment Report
School of Nursing
page 4
The primary mechanism for assessing student learning in general has been
through the development and implementation of courses. The design of baccalaureate nursing courses is a task of the faculty. Each course is considered to be a part of a curriculum with the completion of the designed curriculum being the mechanism for accomplishing the School’s objectives. For example, any student who enrolls in NBSP 323 (counseling strategies) receives a syllabus that has been developed by the total faculty of the School. The assignments, the weight of these assignments, and some strategies are predetermined. Thus the underlying assumption is that if a learner completes the nursing curriculum successfully the objectives of the program have been met. There has been little attempt to evaluate the achievement of each of the end-of-program objectives as a separate entity.
When there is concern about a specific area of learning it is usually expressed by raising a concern about the performance of a specific group of learners in a specific course. These discussions have merit as an evaluative tool simply because the learning assignments and the content are consistent across the sections of a specific course with the structure and content of each course reviewed every two years. Such discussions are usually spontaneous and follow the format of a faculty member stating: "I am concerned about. . ." with other faculty requesting specific data about the concern. Typical questions asked are: How many are having trouble? Was there any indication of this concern the last time the course was taught (usually by a different instructor)? Thoughts about contributing factors?
Recently discussions of this type have highlighted a concern with written communication skills (Objective #1). There has been a persistent concern, i.e. someone mentions a concern with writing skills at every faculty meeting. Attempts to determine the source of the perceived drop in writing ability have addressed:
1. Any change in the number of individuals who have not completed English 151 or its equivalent prior to the beginning of nursing major courses?
Answer: No. Only occasionally does a student take the initial NBSP 300 and 310 without having completed freshman composition, and this person is invariably awaiting a decision regarding the equivalency of courses taken elsewhere.
2. Any change in the grades of the freshman composition courses?
Answer: No. A review of grades in freshman composition over the past five years has
shown no change in the proportion of As, Bs, and Cs.
3. Have fewer students completed the J course before entering major courses?
Answer: No. In fact, a greater number of students have completed the J course prior to enrolling in nursing courses. It is possible that the variable may be new grad/experienced
nurse. Nurses who have been practicing have also been writing daily, and the individuals
who return to complete the BSN are frequently those with some management responsibility which would increase the use of writing. This is being further explored.
1997 Assessment Data
School of Nursing
page 5
The discussions regarding written communication skills has raised the interest in the faculty to the extent that several are working on a draft for a junior level composition course. The majority of the BSN students are on the regional campuses where there is
a limited choice in J courses. A technical writing course that focuses on the type of writing required in the delivery of nursing care could be useful.
Data regarding critical thinking have been collected. Learners have been assessed using standardized critical thinking instruments at the beginning of the upper division major courses. Instruments used are: California Critical Thinking Dispositions Inventory, California Critical Thinking Skills Inventory, and the Watson Glaser Critical Thinking Appraisal. These instruments were used this spring to assess a graduating class of students. Data will be collated and prepared for analysis during the summer of 1997. Data will be ready for review by the faculty during the fall of 1997. (Note: Two critical
thinking instruments are being used for the time being until the value of the data from each can be assessed. The Watson Glaser instrument has been used consistently in nursing education. The California Critical Thinking instrument is newer and has been developed to be used following the California Critical Thinking Dispositions Inventory.)
No specific assessment of the learning related to end-of-program objectives #2, 4 and 5 have been completed.
As mentioned earlier each course is reviewed every two years with junior level courses being reviewed one year and senior level courses being reviewed the second year of a cycle. As part of this review comments written by students on the forms used in each course to evaluate teaching are reviewed by the School director. Comments that are directed more toward the curriculum are pulled from these comment sections and reviewed by the faculty as a whole when reviewing courses. An example: One of the items on the form used in the College for the evaluation of faculty is "textbooks were of instructive value." In the School of Nursing the textbook decision for each course is made by the total faculty with each faculty member using the same textbook for a given course; consequently, this input is more useful to the total faculty than to a specific faculty member. Information of this type is collected and made available to the faculty during the regularly scheduled review of a specific course.
1997 Assessment Report
School of Nursing
page 6
Clinical preceptors (individuals who are employed by the health care delivery agency in which the learner is completing a clinical experience and who serve as mentors to the student) also serve as a source of information. Each preceptor evaluates the learner on a set of specified activities and returns this evaluation form to the School and the faculty member. The evaluation of the learners in the senior capstone course
(NRSE 445) consistently rate the work and the presentations of the students between 4 and 5 on a 5-point scale. These ratings provide some indirect assessment of all five end- of-program objectives as the assignment is to design and present (verbally and written) a plan for some aspect of organizational change. The specific focus of the project is negotiated between the preceptor and the learner.
Assessment activities have not been modified since the 1996 Assessment Report.
Assessment of specific outcomes, especially objectives 2, 4 and 5 has been an ongoing discussion within the faculty but no modification of current activities has been approved.
3. Detail how you are analyzing the data and diagnosing strengths and weaknesses.
Data collection instruments are distributed by staff in the School of Nursing.
Data are collected either during class or through the mail. The information is collated in the School of Nursing with compiled raw data given to the faculty prior to a faculty meeting. Discussion and any necessary decisions are made by the faculty.
The determination of areas of strength and weakness have come through faculty consensus. While no decision rule has been established history indicates that a score less than 4 on a 5-point scale (with 5 indicating the most positive response) will indicate an area in need of attention. For example, data from the Program Evaluation tool mentioned above indicated that academic advising (M=3.86) and "the degree special learning needs were met" (M=3.51) are in need of attention. Additional information collected informally in classes has indicated the need for more detailed communication to occur between the School office in Athens and the regional campuses. Faculty are also developing a more specific evaluation of advising instrument.
Data from the standardized critical thinking instruments have received only preliminary review; consequently, they have not entered into the decision making process.
A large percentage of the BSN graduates have graduated with honors. Enough have done so to raise questions about the correlation between the rigor of the curriculum, the ability of the students, and grading practices within the School. The grade point averages of upper division major courses are statistically compared to the grade point averages of upper division courses taken outside of the major for every graduating class.There has been no statistically significant difference between these two sets of grades over
1997 Assessment Report
School of Nursing
page 7
the past five years. This finding is not difficult to understand given the self-selection
factor of any RN to BSN program.
4. What evidence (assessment data) have you gathered indicating that you are accomplishing your goals? What evidence indicates that they are not being fully accomplished?
Strengths
First, as mentioned above, initial data from the Program Evaluation instrument
indicate satisfaction with the program. This instrument will also be used to collect input
from graduates five years after graduation.
Second, data from the follow-up studies of graduates completed by the Office of Institutional Research (data from 1991 through 1995 graduates) indicate that the BSN graduates are satisfied with their experience at OU. The majority (83%) were extremely satisfied/satisfied with major courses; 83% thought OU prepared them extremely well/well given their career goals; and 89% thought OU prepared them extremely well/well for additional academic work.
Third, clinical preceptors in the students’ final clinical course consistently rate the
work and the presentations of the students between 4 and 5 on a 5-point scale. Comments mention the students’ willingness to be involved with the agency and to tackle the organizational change project with enthusiasm and professionalism. As mentioned earlier this serves as an indirect assessment of all five end-of-program objectives.
Fourth, unsolicited input from graduates comment on the understanding of the need and the willingness to provide program flexibility without lowering expectations.
There are consistent comments about how well the School’s graduates are prepared for graduate school. This information comes from the graduates and would be strengthened
if data were sought from the specific graduate program.
Fifth, one of the firm beliefs of the faculty is that one’s behavior tends to be consistent with one’s perception of self. The tight structure of prelicensure nursing programs and the current design of health care delivery agencies (even though changes are on the horizon) encourage a regimented, one-right-answer approach to the delivery of health care. Practitioners who learn in such an environment have a limited view of their strengths and potential. The most rewarding feedback that we hear (and we hear it frequently) is: This program has changed how I view myself, and I act differently at work. I speak up more. I initiate action more. I can better explain why a certain intervention should be taken.
School of Nursing
1997 Assessment Report
page 8
Last, but by no means least, is the fact that the faculty do discuss and review the strengths and limitations of each course frequently. Every member of the faculty has input into the design of each course which means that each individual has both the opportunity and the requirement to look at each course in terms of the entire program rather than as discrete entities.
Weaknesses
As mentioned previously in this document, advising needs to be more thoroughly assessed and improved.
The indication that students’ special learning needs were not being met needs further exploration. It is possible, given the nontraditional nature of RN students, that this comment refers to something that cannot be accommodated by the program. No determination can be made until the information is better understood.
The appropriate balance between the use of technology based classes and person- to-person contact with a faculty member has not been achieved. Faculty receive
lower teaching evaluations when the course is HEMS based versus single classroom based.
Currently the program is delivered in such a way that three campuses are taking the same set of courses on a given day, e.g. Courses #1, 2, and 3 are taught at Athens, Chillicothe, and Southern on every Monday. Of these three courses two may be taught with a separate section on each of the three campuses with a different faculty member responsible for a section on a given campus. The third course will be taught using HEMS (microwave based interactive television) with one faculty member being present on one of the regional campuses and teaching to all three campuses. This structure allows learners to take courses at the campus closest to their home and allows a campus with a small number of learners to continue to offer the program. Learners have a love-hate relationship with HEMS. They do not want to travel any greater distances to attend class, but they become very frustrated with the HEMS operation when the videos will not work, when doors to the HEMS classroom are locked, and when the system is not up and going when it is time for class to begin. There is room for improvement.
The amount and nature of classroom assignments needs attention. Comments
that are written on the forms used for the evaluation of teaching which are really
directed toward the program rather than the specific faculty member indicate a sense of
1997 Assessment Report
School of Nursing
page 9
more busy work than immediately relevant assignments. The numbers have not been enormous but they have tended to cluster around a couple of courses. "Busy work"
comments are not in and of themselves cause for immediate alarm, but they are more of a concern with nontraditional than with traditional students. Primarily because the
nontraditional student places more weight on use of time than the traditional college student.
The faculty have expressed concern about the writing skills of the seniors (objective #3). They are sensing a decline in the quality of the work being submitted. Several possible sources have been mentioned: (a) the students are younger which is a result of an increase the number of individuals who are continuing the BSN immediatelyfollowing completion of the associate degree program and licensure, (b) classes on some campuses are larger so there is less individual feedback on the writing, and (c) a perception of less conscientiousness in the students.
5. What improvements or enhancements have been implemented based on your assessment activity? How are you using information from your answer to question 3 to improve your program?
The course-by-course review that is currently being conducted is focusing on the
level of critical thinking that is being required in each course assignment. Consideration has been given as to ways to strengthen the writing component (objective #3) while emphasizing critical thinking (objective #1) and reducing the sense of busy work. This has led faculty to seek additional clinical scenarios for use in classroom assignments. Assistance in forging better links between the classroom and the work setting (objective #5) will be an agenda item at the fall meeting of the School’s advisory council.
The faculty is currently discussing the advantages and disadvantages of using student portfolios as a means of assessing the growth of students (especially in relation to objective #2 and #4) while they are completing the BSN. It is the belief that use of such a document would provide both faculty and the student with a better sense of where they started and what they have accomplished while in the School of Nursing.
6. What changes do you recommend for your unit in the future: (a) in academic program/curriculum, and (b) in your department’s assessment process (goals, objectives, methods)?
Academic program/curriculum changes
Faculty are working to improve learning in the area of communication. The
definition of "computer literacy" for undergraduates is being discussed with the intent
1997 Assessment Report
School of Nursing
page 10
of specifying certain baseline skills and then incorporating course assignments that would
require the use of these skills. An added bonus of specifying skills would be that of providing a mechanism whereby students could satisfy the program’s computer literacy requirement other than enrollment in the specified courses (MIS 100, CS 120, and
HS 309).
Once the baseline skills requirement is determined, the faculty will need to decide if these skills are sufficient for a nurse given the changes occurring in health care delivery, e.g. computers at the bedside, wireless computer which allows supervision of staff from a base office, access to professional journals through the Internet, or the design of courses that make use of virtual reality. There is a sense that a growing number of incoming students are comfortable and competent in basic word processing and data management skills but are in need of networking skills. We will be seeking the answer to these questions (beginning with a intensive workshop that is the product of collaboration between nursing service and education which is being attended by two faculty members this summer). Input from this session will be shared in the fall and appropriate adjustments to the curriculum will be made.
Faculty are also giving thoughtful consideration to ways to increase the writing and the feedback on that writing without creating an onerous burden for the faculty member. A couple of faculty are exploring the development of a junior level composition course that would provide greater consistency across the nursing majors and the six regional campuses.
The feedback concerning the use of the HEMS, the faculty’s concern with driving
time, and the increasing demands upon our students by their employers are stimulating exploration in the use of the Internet for certain courses or at least components of some courses.
Assessment Process
Change in the assessment process is more important than specific curriculum changes at this time. Objectives are clearer than they were at this time last year but specific outcomes are still not clear enough to make assessment smooth.
The annual calendar for review of data exists but needs to be more closely
followed. Several mechanisms for the evaluation of different aspects of the program have been designed but not implemented. Some data exist but have not been pulled together in a cohesive, easily retrievable format.
The primary task for the coming year will be to create an evaluation package
that will provide information about student learning outcomes to the same degree as it provides information about the structure and process of the School’s activities.
1997 Assessment Report
School of Nursing
Attachment A
Ohio University
School of Nursing
PROGRAM EVALUATION
DIRECTIONS:
1. Use a #2 pencil.
2. Write your Social Security number and today’s date in the appropriate areas on the scan sheet.
3. Please mark your level of agreement with each item using the following scale:
A=Completely satisfactory
B=Satisfactory
C=Unsatisfactory
D=Completely unsatisfactory
E=Not applicable
Program Utility and Efficacy
To what degree did your program:
1. help you acquire necessary nursing skills.
2. provide overall intellectual growth.
3. enhance your personal growth.
4. provide help toward achieving your career goals.
5. provide integration of courses and other learning experiences into a meaningful whole.
6. require you to use the skills you had prior to entry.
7. prepare you for a position appropriate to your level of education.
8. encourage collegial behavior among you and your peers.
9. enhance your commitment to professional nursing.
10. provide role models in nursing practice (including indirect & direct practice such as education or administration).
11. provide role models in scholarly conduct.
12. prepare you to assume responsibility for self assessment and continuing professional development.
13. provide you access to clinical agencies with suitable nursing role models.
Curriculum Options and Individualization
What was your level of satisfaction with:
14. alternatives in meeting course objectives.
15. alternatives in meeting program objectives.
16. academic advising.
17. course scheduling.
18. course sequencing.
19. the degree your former experiences were taken into account in curriculum planning.
20. the degree your special learning needs were met.
21. amount of input you had or could have had into your program.
Time, Efficiency and Student Demands
To what degree were you satisfied that there was:
22. no unnecessary duplication of clinical experiences.
23. no unnecessary duplication of theoretical content.
24. an overall emphasis on scientific basis for nursing practice.
25. a well articulated curriculum.
26. manageability of demands.
27. adequate time for learning required clinical content.
28. adequate presentation and application of the conceptual framework in school of nursing courses.
Faculty Student Relationships
How satisfied were you with faculty:
29. competence in theory presentations.
30. clinical competence.
31. concern for you as an individual.
32. concern for overall education.
33. availability (during office hours or by appointment).
34. sensitivity to your needs.
Learning Resources
To what degree were the following satisfactory:
35. depth of clinical experiences.
36. variety of clinical experiences.
37. variety of courses available to you.
38. depth of courses available to you.
39. financial assistance available to you.
40. classrooms.
41. conference rooms.
42. support services.
43. library holdings (books, journals, etc).
44. library hours.
45. library physical facilities.
46. library staff.
47. technical (hearing and seeing) aspects of HEMS (microwave/TV classes).
48. HEMS as an alternative to driving greater distances for class.
49. currency of materials in the media center.
50. materials produced by the media center.
51. working condition of media equipment.
52. adequacy of the School of Nursing physical facilities.
Overall Impressions of the Program
53. overall, how well satisfied are you with your program.
54. would you recommend the program to others (A=yes, B=no).
Degree to which the program helped you to meet the following outcomes:
55. acquire a liberal and professional education as a foundation for practice and advanced education.
56. develop an understanding of the use of theory in nursing practice.
57. use the nursing process in the design and delivery of nursing care.
58. develop management skills in traditional and emerging roles in response to the changing needs of society.
59. develop an attitude of intellectual inquiry and skills in critical thinking.
60. develop collegial relationships with nurses and others concerned with health care.
61. develop competence in a variety of communication modes.
62. become a critical reader of research related to clinical practice.
63. develop an enhanced sense of professionalism.