After five years, hundreds of thousands of exams, CBTF still revolutionary

What started out as an off-hand comment during a faculty meeting like "Wouldn't it be great if…." has turned into a vital tool for students and faculty in engineering and other STEM fields. This fall, the Computer Based Testing Facility (CBTF) celebrated five years with a Lightning Symposium and visions of continued growth.

Under the auspices of the Academy for Excellence in Engineering Education (AE3), the CBTF uses web-based learning management systems to deliver exams in a convenient, proctored environment. Its services are especially useful to large lecture classes that don't have ideal classroom space to give exams to hundreds of students. It also supports students identified by the Disability Resources & Educational Services (DRES) that need extended time and/or reduced-distraction seating for testing. 

"The CBTF has tremendously improved our ability to meet our students' scheduling needs while maintaining high evaluation standards for our large, introductory Physics 101 and 102 sequence for non-engineering majors," said Elaine Schulte, senior lecturer and assistant director of Undergraduate Programs for the Department of Physics. "With the CBTF we are able to ensure all of our students' needs, including those with learning-related disabilities, are expertly met under the same evaluation conditions."

What started with 200 students in two classes has grown to over 8,900 unique students in 38 classes each semester. So far during the Fall 2019 semester alone, the CBTF facilities have given over 67,000 exams. 

"There is no other place in the United States that has a facility that can do what the CBTF can do at scale," said Carleen Sacris, the CBTF manager. "It fits in well with the mission of AE3. It is particularly beneficial for large classes. In fact, some classes like CS 125, which has over 800 students, test with us every week." 

Interested faculty reach out to the CBTF administrators and go through the process of understanding how the system works. There is an initial investment in time and effort in creating good question generators for the computer-based testing format but these can be used in perpetuity. After that first semester of adoption, instructors can spend time creating new types of questions for their subsequent semesters instead of re-writing the ones they used before.

The two CBTF locations, which are in the Grainger Engineering Library and the Digital Computer Laboratory, are open Monday-Fridays from 10 am-10 pm, Saturdays from 11 am-10 pm and Sundays from Noon-10 pm during the regular school year. The two facilities have a combined 123 stations. With the CBTF supporting Chemistry (3 courses with over 1,000 enrollees), Statistics, and other non-engineering STEM based courses, future plans will likely include opening a facility near the main quad, more convenient for students in liberal arts and sciences. 

Students can sign up online two weeks before their test and reserve a time within the three-day testing window. The system has enough flexibility to handle most conflicts and last-minute emergencies. It generally fills to 70 percent capacity to allow for occasional technical issues. Each station is built for security, making sure students only have access to the testing servers and whatever software is allowed to be used in the test. 

Students arrive 5-10 minutes before a scheduled exam, present identification to the proctors, and place their bags and other personal belongings on the racks at the front of the room. Proctors handle a variety of scheduling, technical, student, and academic integrity issues. Exams in the CBTF are automatically graded and, if instructors choose, provide immediate feedback to the students. 

"The CBTF's flexibility in allowing you to choose your slot for an exam is amazing because you can fit it into your schedule," said Rohan Khatu, who is majoring in computer science and math. "Getting an instant grade is so much better than anxiously waiting for the result." 

The need for a CBTF became clear after the creation of PrairieLearn, a tool for online assessment created by Matthew West, associate professor of Mechanical Science and Engineering, and Craig Zilles, associate professor of Computer Science and Education Innovation Fellow. With PrairieLearn, questions are randomized, so every student is asked something unique. Because of that, students who need extra practice can get it from the tool. 

Beyond the true-false and short answer questions, PrairieLearn accommodates coding questions, graphing and other drawing type problems that are commonplace in engineering, physics, chemistry, and other science-related courses. Faculty who are using PrairieLearn are supported by a community that helps troubleshoot issues and provides guidance on best practices. 

"Because exams are easier to give, instructors tend to give them more frequently, cutting down on the pressure of each individual exam," West said. "They can more easily give retakes and the whole process is more transparent to the students. Instead of grading, TAs get to spend more time interacting with students freeing them up to add projects and other in-depth, creative activities to the course."

"Before the CBTF, faculty and TAs spent a lot of time on the logistics of giving and grading the exams," Sacris added. "We are able to take care of that for them. From the student perspective, the scheduler is such a magnificent tool in part because the students themselves get to choose when they want to take their test. We are also able to take care of about 98 percent of DRES accommodations. I think it really helps them that we have long hours and are open during the weekends."