Pilot Program Improves Accurate Math Placement
by Emily Baxstrom
"We have an implicit promise when you become a Bobcat that we’re going to be here. We’re going to challenge you, but we’re going to support you and guide your success. If the very first thing we do when you walk through our doorstep is to put you in the wrong class, we’ve made it harder to deliver on that promise." —Bob Klein
Ohio University has long used a two-tiered system to place students in their first college-level math course. Students are initially assigned to a level based on their ACT/SAT scores or courses transferred from another institution; they can attempt to change their initial designation by taking an optional in-house math placement test through Blackboard. From this test, students are sorted into four placement levels:
- Developmental placement (DV) — Math D003, D004, or D005
- Placement level 1 (PL 1) — Tier 1 quantitative classes
- Placement level 2 (PL 2) — Pre-calculus, mathematical statistics, etc.
- Placement level 3 (PL 3) — Calculus I
While the levels continue to be important, the means of determining the levels at which students are most likely to be successful is being investigated. “Our existing system is good, but we think we could do better. And we’re not alone in taking a look at the validity and reliability of placement assessment,” said Bob Klein, associate professor and interim associate dean of Ohio University Lancaster. He said institutions across the country have been investigating the optimal way to place students in college-level math.
Klein started engaging colleagues from across different disciplines in conversation and participating in national meetings about placement options and consequences. This led him to pilot a system being used by many other institutions nationally: ALEKS.
What is ALEKS/QPASS?
Assessment and LEarning in Knowledge Spaces (ALEKS) is a web-based, artificially intelligent assessment and learning system that uses adaptive questioning to quickly and accurately determine exactly what a student knows and doesn’t know. At Ohio University, it is a “Quantitative Preparation for an Academically Strong Start (QPASS),” since it offers more than just an assessment of placement — it offers students a chance to spend time learning concepts before being reassessed. Advisors and students can then review progress and performance to help students succeed in their first quantitative course.
Students sign into the ALEKS/QPASS system and take a first assessment. The system uses adaptive reasoning to determine the questions it presents to each student. After the student’s first assessment, ALEKS/QPASS gives them an initial placement — for example, Placement Level 1 (PL 1). With this placement, ALEKS/QPASS breaks down the student’s strengths and weak areas so they can see what exactly that placement means.
After the first assessment, the student must spend at least three hours in a learning module that targets concepts the student needs to learn better to help them improve their next placement assessment. The student has the option to take the assessment a total of three times.. A 48-hour “cool-off” period between each assessment ensures that the assessment reflects knowledge mastery rather than more short-term “cram session” study techniques, enhancing the validity of the placement result. Throughout the student’s learning modules and placement assessments, ALEKS/QPASS tracks where the student struggles and adapts accordingly, giving detailed feedback to the student about where their knowledge is strong and where it needs to improve.
Klein admitted to being a suspicious person by nature, but what won him over to ALEKS/QPASS is the idea that students should be in charge of their own learning, and this program lets them do just that. “Students have agency, and students should be able to exercise that agency when it comes to their education,” said Klein. “Placement is often something that’s done to you; it’s a label that’s slapped on you and has little meaning. ALEKS/QPASS is a complete system that includes placement, but it also includes explanations and in-depth learning modules. This puts students in the driver seat while giving them a much better sense of what their placement means in terms of what they do and don’t know.”
Blackboard Math Placement vs. ALEKS/QPASS
ALEKS/QPASS differs from the traditional Blackboard math placement in five main ways:
|Traditional Blackboard Placement Test||ALEKS/QPASS Learning System*|
|Consistency||Four different tests distributed to first-year students based on different criteria (ACT/SAT scores, international/untraditional students)||One test is distributed to all first-year students|
|Question Options||Small pool of questions created in-house at OHIO||Enormous pool of questions, "practically an ocean of questions"|
|Adaptive Experience||Questions are not linked of adaptive||Questions are adaptive to students' strengths and weaknesses|
|Platform Support||Blackboard is hosted off-site; future surcharges to the University are possible given the size of the "organizations" that make up the placement tests in Blackboard||Platform is well supported and purpose-specific, 24/7 maintenance and call assistance and technical support included|
|Student Empowerment||Placements are given to students as labels without a broader sense of how those labels reflect actual concepts known or unmastered||Students are in charge of their own placement; placement label has meaning|
Academic advisors also can view the results from students’ ALEKS/QPASS assessment(s). Because the results are more in-depth and explain the strengths and weaknesses of each student, advisors can then help their advisees navigate courses even more effectively. Additionally, if there are major overall trends one year where a great deal of students show need for improvement in a specific area, math department leadership can adjust courses offerings or curriculum to reflect that cohort of students. For instance, if a cohort of students with PL 1 placement demonstrates overall weaknesses in the area of “ratios and proportional reasoning,” then the coordinator for College Algebra may elect to provide “just-in-time” support for that topic as it arises in the curriculum or perhaps even engage the Supplemental Instruction (SI) leaders in that kind of support.
Additionally, students have one year’s access to the ALEKS/QPASS learning modules, which may be used as a study supplement for their math classes throughout the year.
ALEKS/QPASS has the potential to improve the placement problem that exists at many higher education institutions. One way it can help students is that the level of challenges students are presented with determines their success. If something is too easy, students tune out; if it is too challenging, they become frustrated and give up. The ideal placement occurs when the student has a comfortable challenge and feels motivated and strong—they should have the tools and feel like they’re growing. “If we don’t place students accurately the first time,” said Klein, “we’re going to lose them in that class, lose them in mathematics, and lose them at the University. We don’t want that to happen.”
Accurate placement also affects the cost for the students’ education, and in turn, the University. If students are misplaced and have to retake a course, that’s two courses the student is paying for when it should only be one. In some cases, students are unable to finish their degree in four years. Klein looked at the number of students who retook mathematics courses, and using some basic assumptions, estimated almost $2 million in Athens student costs per year to retake math courses by scoring below a C or withdrawing. (Note: This calculation looked at a one-year period for algebra-to-calculus pipeline courses, assumed students retake courses, and assumed four-credit courses cost $1,400.)
“We have an implicit promise when you become a Bobcat that we’re going to be here. We’re going to challenge you, but we’re going to support you and guide your success,” said Klein. “If the very first thing we do when you walk through our doorstep is to put you in the wrong class, we’ve made it harder to deliver on that promise.”
Hand-in-hand with this promise is the need for students to take the assessment seriously. If students don’t honestly complete it to the best of their ability, they could be placed in the wrong course, potentially leading to the above-mentioned problems.
“The goal of a successful placement process is not to keep students out of courses, but to identify students’ strengths and weaknesses and work with them to develop a plan for them to succeed,” said J. Michael Pearson, executive director of the Mathematical Association of America (MAA).
ALEKS/QPASS pilot at OHIO
Klein has been working with many other departments, faculty, and staff across OHIO throughout the first pilot of ALEKS/QPASS. During the pilot, the ALEKS/QPASS assessment was sent to new-to-Athens students in fall 2017 that came in with a developmental placement. “We felt the developmental pool of students was the place we needed to focus on first,” said Klein. “We wanted to see if students can, given the right tools, lift themselves out of developmental math and into their first college-level mathematics course.” If a student placed in a PL 1, PL 2, or PL 3, they took the traditional Blackboard placement test and were not a part of the pilot.
The pilot has not been without its challenges, but Klein believes different results will reveal themselves when the assessment is distributed to a wider pool of students in the future. The struggle so far is getting students to complete more than one assessment. “We need more data that shows students they’re potentially saving $1,400 and an extra semester by taking this seriously,” said Klein.
Future of ALEKS/QPASS
The current stumbling block that is preventing ALEKS/QPASS from expanding at OHIO is funding. ALEKS/QPASS costs a flat rate of $15 per student. The pilot was funded for one academic year by the Office of Instructional Innovation. The in-house version created in Blackboard costs $0 per student on the face of it, but costs in supporting the well-worn Blackboard tests are mounting given changes in Blackboard-managed hosting and personnel changes at the University. Moreover, the retake costs to students merit taking placement seriously.
“Fifteen dollars is money well spent if students actually use the full year of help and assistance,” said Klein. “In the era of $400 textbooks and $1,400 course retakes, our institution pays for students who get D’s, F’s, and W’s; we pay for that in a loss of state subsidies, in loss of degree completion and four-year completions. So to me, that $15 seems like money well spent.”
In the early part of the 2018 calendar year, Klein wants to move the assessment forward on a bigger scale, doing a second round of the pilot that is focused on a wider range of students.
The Office of Instructional Innovation (OII) serves as a catalyst to spark bold experimentation and sustainable discovery of innovative instructional models that fulfill the University’s promise of a transformative educational experience. OII provides a variety of services to faculty, staff, and students in support of academic units and online programs, as well as to advance initiatives to further the institution’s mission. Visit our home page for more information.