Effective Practices: Assessment

• Review your course map and learning objectives to ensure alignment between your assessments and course learning objectives.
• Review assessment tool options.
• Review 12 Effective Practices for Delivering Blackboard Tests
• Post a low-stakes quiz or assignment in Blackboard to provide you and your students an opportunity to practice using the online test tool.
• Deploy short quizzes to let students practice skills, demonstrate knowledge, or apply concepts.
• Alternatively, consider open-book exams with authentic scenarios for students to respond to in long-form. These do need additional grading, but require students to apply knowledge and skills.

Audience Response Systems

Audience or student response systems (aka “Clickers”) are typically used in a face-to-face environment. But since they are associated with a Blackboard course, you can upload the student response scores to the course gradebook. Poll Everywhere also can be used in face-to-face or online classroom environments as an audience or student response system. These tools could be used in the following ways:

• As an active F2F or online participation record that is maintained online (ask clicker / poll questions during class)
• As the individual recording device for the Team Based Learning RAP step #1 below (no need to manually grade during or after class)
• Used with Blackboard’s “Adaptive Release” you could limit review of online notes, screencasts or lectures to students who have participated in clicker quizzes in class. Thus, online materials are literally for “re-view” (not discovery) of material covered in class.

Peer Review

If you’re tired of being the first living soul who ever lays eyes on a student paper, make peer review of drafts a requirement before they can turn in a final draft–and assess their ability to critique each other. A few ideas to consider if you’re teaching a hybrid or online course:

• Commit to a rubric or check list of criteria that is not only used to grade the final assignment, but also to guide (and assess) the student’s review of a peer’s draft.
• Consider letting the students give a brief (2-3 minute) audio peer review using Blackboard Collaborate or even a podcast. Students may feel a written review is too daunting and will opt instead at a superficial focus on grammar or mechanics. An audio review is more high level, focused on what makes sense (or not) in the student writer’s draft. If students add their papers to VoiceThread, consider using the VoiceThread Assignment Student Gallery for students to provide feedback.
• Require the students to respond to the peer’s review when they submit the final (what they accepted or rejected and why).

Practice Quizzes

Instructors rightly worry about security of quizzes and exams in an online-only course, because it’s difficult to guarantee the person completing these assessments is the student registered for the course. But in a hybrid course, you can reserve F2F time for these “high stakes” assessments, and help students get ready for them through online practice quizzes. Consider the following:

• Developing an online quiz pool may be the most time-intensive, up-front hybrid course development effort, but once it’s done, students can learn at their own pace and based on deficiencies you and they discover.
• Consider adding a “syllabus” or “academic integrity” quiz after the first class that students must take (pass?) before they can turn in any other work for academic credit. This will tell you early on who’s engaged (technically or academically).

Team Based Learning

If you want to make sure students are prepared for class and group work, consider Team Based Learning (TBL) and its Readiness Assurance Process (RAP) that includes the following:

1. An individual quiz over assigned readings;
2. A team quiz over the same material using immediate feedback “scratch off” cards;
3. An open book “appeals” phase for incorrect “team answers” only.
4. Corrective instruction (mini lecture) by the instructor based on a real-time understanding of what students didn’t understand.

Participation Portfolios

If we assign online discussions, do we take a quantitative or qualitative approach to assess them? Using a best of both approach, students copy and paste their best 3-5 examples into a “participation portfolio” with a self-grade the instructor can raise, lower or accept, based on the quality of the evidence that students provide every 1-2 weeks. The key is defining (and sharing) a grading rubric of what constitutes a good post & reply. Now students (not the instructor) are responsible for hunting, gathering and initially evaluating their best work.

THE BASICS

1. Instructor defines grading rubric for good post & reply (this is THE hardest task for instructors).
2. Students propose grade they feel they deserve, based on 3-5 examples of each.
3. “Evidence” is taken from separate weeks to avoid end-of-term “dog pile.” Just assign and spread out a few portfolios.
4. Students copy and paste their “evidence” into a portfolio “template” and submit electronically.
5. Instructor can accept, raise or lower grade based on quality of evidence as defined by the rubric (#1).
6. Key point: Students (not instructor) hunt and gather their quality participation, which is a big time-save in an active course. They also carry on authentic conversations with each other, not one-way panderings to the prof.
7. TIP: Enable Discussion Board Notifications by Email (Original only)

MATERIALS

• Assignment “Handout” for Students & Sample Rubric (PDF)
• Participation Portfolio (MS Word Form)
• How to make MS Word Forms (a little dated, but use the “developer” ribbon in MS Word now).

PRESENTATIONS

• 04/27/09 NJ DLAAB Spring Meeting (Wimba Archive) | Wimba Setup Wizard Check
• 01/22/09 Educause Learning Initiative Presentation
• 07/23/2008 Wimba Distinguished Lecture Presentation ( Wimba Archive | MS PPT)
• 10/26/2007 Educause 2007 Presentation

REVIEWS

• March 2009 Duke University Blog Post
• January 2014 Team-Based Learning (TBL) Newsletter article

Online tests and assignments are an effective and convenient way to assess students in the virtual environment. Throughout the semester, faculty can leverage many different options and settings for digital assessments. Here are a dozen tips to improve your test administration and assignment collection as the semester reaches its conclusion. 

1. Avoid Force Complete on Original Tests

This legacy setting on Original tests prevents students from accessing the test environment if they prematurely leave. Unfortunately, if students lose internet access, have technical difficulties, double-click a link, they can be kicked out of the exam, requiring that you add a new attempt or reset the attempt altogether. Instead of using force complete, set your exam to Auto Submit, which will allow students back into the test and collect the student’s submission at the end of the allotted time whether or not the student is finished.

2. Practice Before Proctoring

Before you use Respondus Lockdown Browser (alone or with live proctoring or with Monitor enabled), create a no-points practice quiz with unlimited attempts so students can experience the testing environment and configure their computers before they take an exam. If you do not have a practice quiz, be sure to allow plenty of extra time for students to set up. This is especially important if students need to update their lockdown browser.

3. Describe Test Expectations

Let students know how many questions and what kind of questions they will have on the test. Post the test limit, whether the test will auto-submit when time is up, and if there is more than one attempt. Include start and end times. Link to relevant FAQs on test taking tips or using Respondus Lockdown Browser, if enabled.

4. Communicate Your Handling of Technical Difficulties

Tell students how you will address any potential technical issues during a test. Be fair and swift, especially if your test timer is limited. Consider whether you will respond to individual emails or post an announcement. Some technical issues may be deferred until business hours the next day so please consider how you will handle these types of questions. Know which vendors offer 24/7 support (e.g., Respondus) so you can refer students to those resources.

5. Stagger Access to the Test

Avoid having too many students access your test at the same time. You can set a wide window when students can start the test, but create a timer so once the test starts, students only have a fixed amount of time to complete it. You can also apply Adaptive Release (Original) or Conditional Release (Ultra) rules to restrict access to specific groups of students.

6. Set Question Display for the Right Situation

Showing all questions on one page can be great for short tests during a short amount of time. However, for longer tests (25+ questions or taking more than 1 hour), consider that students may benefit from displaying one question at a time. Additionally, Blackboard can time out if the test is too long: See #7 next on our list!

7. Break Up Large Tests

Blackboard will time out after three hours of general inactivity. Instead of delivering one long 2-hour test, provide four 30-minute tests. Chunking the test eases impact on Blackboard and student network connections while also providing opportunities for those screen and stretching breaks. Adaptive Release (Original) or Conditional Release (Ultra) rules can also be leveraged to open the next part of an assessment based on specific criteria such as a minimum grade or date/time. 

8. Allow Backtracking on Original Tests

Students like to check their answers or skip questions if they are uncertain about an answer. If you don’t allow students to backtrack, they will attempt to use the back arrow on the browser and inadvertently cause an issue with the test or lock themselves out. 

9. Dip Into the Pool

Leverage question pools to generate randomized and unique tests for students. Random blocks (Original) and Question pools (Ultra) allow you to choose a defined number of questions (5) out of a larger collection of questions (50) and randomly distribute those questions to students so each one sees a different set. Learn how CHEM 101 and CHEM 102 used nearly 1500 questions to create a robust assessment series. 

10. Set Access Codes on Ultra Tests to Prevent Late Submissions

Instead of hiding an Ultra test with a date restriction, use an access code after the test period ends. This allows students to see their grades and feedback and prevents any late submissions. 

11. Remind Students to Check Technology

Students should use the latest version of Chrome or Firefox when taking a Blackboard test, and they should check for browser updates regularly. To optimize performance, all other applications should be closed unless you allow access during the test. Students should verify that cookies are enabled for Blackboard, Collaborate, and related third-party tools. Respondus Lockdown Browser has its own requirements, including manual update of its application, and restrictions as well as settings for faculty if there are specific use cases for accessing websites, calculators, and spreadsheets.

12. Skip Timed Tests and Go Authentic

Last semester, students told us they appreciated flexible online test formats and alternative, authentic assessments. In Redesigning Assessments for Interaction and Engagement, a recorded presentation from Quality Matters, discover assessment types, examples, and strategies, including designing authentic assessments and using LMS tools, to support your assessment goals.

• Use questions pools (Ultra) and random blocks (Original) in Blackboard tests to create unique tests for each student.
• Consider using Respondus Lockdown Browser for high-stakes assessments. RLDB is a custom browser that secures down the testing environment within Blackboard. Students are unable to print, copy, go to another URL, or access other applications. Once the assessment is started, students are locked into it until they submit it for grading. Respondus Monitor records the student during the test.
• Use SafeAssign on assignments or test essay questions to determine the originality of student work.

• Use a rubric with Blackboard assignments to easily apply consistent grading.
• Embed feedback on student submissions with digital notes or record an audio comment for up to five minutes.