SP22 Survey Shines Light on Faculty Testing
Key themes: academic integrity, accommodations, testing ctr.
In an effort to gather information on how faculty design and administer tests, DoIT collaborated with the Faculty Development Center (FDC) and Student Disability Services (SDS) to create and distribute a short survey to all 941 instructional faculty teaching in Spring 22. There were 157 complete submissions, representing a 17% response rate. Of these survey participants, 78% were full-time faculty, and about half (54%) were Associate, Assistant, or Full Professor. Almost all were from CAHSS (48%), CNMS (24%), or COEIT (22%), and 86% primarily teach undergraduate students.
Academic integrity is approached in several ways, including the use of psychology (lecturing students and/or acknowledgment of an "honor" statement or pledge) and overall design of the assessments, such as question banks, randomization, synchronous testing, and, in direct conflict with student accomodation concerns, time constraints.1 These approaches are carried out in two different environments: in-person and online. In the case of the former, respondents indicated traditional means of face-to-face proctoring. With the latter, instructors may use Blackboard, along with the Respondus Lockdown Browser, and potentially the Monitor option. Other respondents described having their students turn their computer cameras on during the assessments. Plagiarism checkers such as SafeAssign and Turnitin are also leveraged.
There is an acknowledgement by some, however, that there may not be anything that can be done to fully redress issues concerning academic integrity "other than theatrics." As one respondent noted, "it is impossible to keep students from posting the questions (from memory) on the Internet as soon as they are done with their own test." Some stated, however, that they believe their tests are relatively impervious to cheating due to the tailoring and formatting of the questions. Others stated they didn't think they have a big problem with cheating, or did little to curtail it beyond making sure students aren't using their phones or looking at other students' papers, which can be of particular concern when rooms are at full seating capacity.
The student disability-based testing accommodations described universally involve faculty providing more time on assessments for individual students, or through the course of the pandemic, expanding the testing window for all students. Few faculty indicated a pervasiveness of need, with some emphasizing how few occurrences of student accommodations they've received in their teaching careers. Additionally, multiple respondents seemed to be under the false impression that SDS has a testing center to support this need, rather than just ad hoc space allocation that is only available for SDS registered students with disabilities. Meaning it is not for general make up test proctoring for all students, can only seat 15 students at a time, and has computer stations only intended for specialized assistive technology needs..
Interestingly, when asked about the possibility of a dedicated UMBC testing center, only 14% of respondents teaching 50 or more students don't believe a testing center would help them; none of the respondents teaching more than 100 students are of that opinion. In fact, several respondents voiced support for this kind of facility, with one saying "I would be THRILLED for a testing center at UMBC!"
Some faculty, however, never use high-stakes testing. They may choose not to due to a lack of familiarity or experience with it (e.g., having begun teaching during the pandemic), while other instructors stated they did not believe in using high-stakes assessments on pedagogical grounds, or employ qualitatively graded assessments, such as case studies or written assessments, rather than discrete answer assessments. These types of assessments, of course, can still be high stakes, although much more difficult to scale for larger classes than multiple choice bubble sheets. Other faculty described offering more "smaller" assignments or quizzes. One respondent specifically cited their PIVOT training for adoption of this "best practice."2 Of those who did make use of high-stakes assessments, some stated they were important to encourage students' motivation to learn. "You have to have some high-stakes testing to get students to study," responded one instructor.
The quantitative data, as illustrated below in Figure 1, indicate that faculty teaching larger courses and those teaching courses with high percentages of the overall grade contingent on a small number of tests (i.e., high-stakes testing) are considerably more concerned about academic integrity. Also, the survey's open responses suggest instructors believe assessments are important, but were not of a single voice in regards to balancing formative and summative approaches in face-to-face and virtual environments.
Figure 1: Percent of Final Grade from High-stakes Testing, by Course Enrollment
The majority of respondents (59%) taught classes of 50 or fewer students, and about one in five (19%) taught classes with more than 100 students. Additionally, of those instructors teaching courses with at least half of their students' final grades determined by high-stakes tests, 72% agreed or strongly agreed that "Academic integrity on high-stake assessments is a major concern in my course(s)." That same level of sentiment is shared with less than half (49%) of instructors with grading schemas that are not as dependent on high-stakes testing.
Academic integrity is approached in several ways, including the use of psychology (lecturing students and/or acknowledgment of an "honor" statement or pledge) and overall design of the assessments, such as question banks, randomization, synchronous testing, and, in direct conflict with student accomodation concerns, time constraints.1 These approaches are carried out in two different environments: in-person and online. In the case of the former, respondents indicated traditional means of face-to-face proctoring. With the latter, instructors may use Blackboard, along with the Respondus Lockdown Browser, and potentially the Monitor option. Other respondents described having their students turn their computer cameras on during the assessments. Plagiarism checkers such as SafeAssign and Turnitin are also leveraged.
There is an acknowledgement by some, however, that there may not be anything that can be done to fully redress issues concerning academic integrity "other than theatrics." As one respondent noted, "it is impossible to keep students from posting the questions (from memory) on the Internet as soon as they are done with their own test." Some stated, however, that they believe their tests are relatively impervious to cheating due to the tailoring and formatting of the questions. Others stated they didn't think they have a big problem with cheating, or did little to curtail it beyond making sure students aren't using their phones or looking at other students' papers, which can be of particular concern when rooms are at full seating capacity.
The student disability-based testing accommodations described universally involve faculty providing more time on assessments for individual students, or through the course of the pandemic, expanding the testing window for all students. Few faculty indicated a pervasiveness of need, with some emphasizing how few occurrences of student accommodations they've received in their teaching careers. Additionally, multiple respondents seemed to be under the false impression that SDS has a testing center to support this need, rather than just ad hoc space allocation that is only available for SDS registered students with disabilities. Meaning it is not for general make up test proctoring for all students, can only seat 15 students at a time, and has computer stations only intended for specialized assistive technology needs..
Interestingly, when asked about the possibility of a dedicated UMBC testing center, only 14% of respondents teaching 50 or more students don't believe a testing center would help them; none of the respondents teaching more than 100 students are of that opinion. In fact, several respondents voiced support for this kind of facility, with one saying "I would be THRILLED for a testing center at UMBC!"
Some faculty, however, never use high-stakes testing. They may choose not to due to a lack of familiarity or experience with it (e.g., having begun teaching during the pandemic), while other instructors stated they did not believe in using high-stakes assessments on pedagogical grounds, or employ qualitatively graded assessments, such as case studies or written assessments, rather than discrete answer assessments. These types of assessments, of course, can still be high stakes, although much more difficult to scale for larger classes than multiple choice bubble sheets. Other faculty described offering more "smaller" assignments or quizzes. One respondent specifically cited their PIVOT training for adoption of this "best practice."2 Of those who did make use of high-stakes assessments, some stated they were important to encourage students' motivation to learn. "You have to have some high-stakes testing to get students to study," responded one instructor.
In short, faculty vary in their approach to testing, but there are similarities based on course enrollment size and discipline/college. Also, adoption of a testing center could potentially benefit faculty conducting testing, particularly in high-stakes/enrollment environments to support academic integrity and accessibility needs.
[1] As one respondent points out, "Many institutions also offer disability accommodations within a broader testing center, which helps to reduce stigma for students with disabilities."
[2] About one in five respondents are now offering fewer high stakes tests since the pandemic shift to online learning.
~By Tom Penniston
Posted: May 2, 2022, 1:30 PM
