Leveraging Student Data to Enhance Online Assessments

One of the most powerful characteristics of an online classroom is its ability to maintain records. When enhancing your assessments, it’s important to look back at these records to see how students performed, including their responses to questions, their success rate, and even detailed components such as how long they spent on assignments. Although this type data won’t tell you everything, you can couple this information with what you know about your student body to take steps toward improvement. In this article, we’ll talk about how you can use common types of assessment data to inform decisions regarding misalignment, motivation, student preparedness, and feedback.

Assessment Data and Misalignment

We talk a lot about the course design triangle and how it underscores the importance of having your objectives, assessments, and instruction in alignment with one another. Equally important, however, is that your formative and summative assessments align.

One way to detect misalignment is to look at assessment scores. Are students performing well on the formative assessments but not so well on the summative assessments? If so, it’s possible that the formative assessments are testing skills that aren’t adequately measuring students’ progress toward learning objectives. Conversely, if students are performing well on summative assessments but not formative assessments, you might want to analyze the skills your formative assessments are measuring and make sure they’re relevant (and, on a loosely related note, covered appropriately in your instruction). Consider the following examples:

In an introductory psychology course, one macro-objective states that students will be able to analyze major issues in the field of social psychology, including ethnic, cognitive, cultural, and group variables. Students perform well on their formative assessments (which are automatically scoring quizzes that require them to define key terms), but they struggle on the summative assessment (which requires them to write a paper analyzing a major issue of their choice). In this instance, are the formative assessments really preparing students to succeed on the summative assessment? Knowing terms is important, but students probably need a slew of other skills to truly succeed. Adding formative assessments that help students analyze issues, write an analysis, research their topic, and so on, might truly help students succeed.

In a basic composition course, one macro-objective states that students will be able to write a multiparagraph composition with clear organization, including a strong thesis, three or more body paragraphs, and a strong conclusion. Students perform well on the summative assessment, but they do not perform well on a formative assessment covering how to cite research. Because the objective does not mention research, it isn’t included in the instruction, making it difficult for students to succeed. To address this misalignment, the instructor should make sure the instruction, formative assessment, and summative assessment align with one another and include any and all required skills.

Although misalignment between assessments looks different than misalignment between learning objectives and instruction, correcting these types of issues can help students more clearly see the relationships between separate assessments and the skills they are measuring. By looking at students’ scores as potential red flags or signals of possible issues, you can start to dig in to and eventually correct alignment problems that might exist in your course.

Assessment Data and Student Behavior

Assessment data can also be an indicator of student behavior. If your assessment is automatically scored, look at data points like how long students spent on the assessment, how many times they attempted it, and so on. While the specific data points might vary from one learning management system to the next, relevant data should be available nonetheless. For other types of nonautomatically scored assessments, consider your interactions with students and their behavior in the course. When did students submit the assessment? How interested are they in the way you’re assessing them? Is there any relationship between this interest and their performance? These questions can begin to show student attitudes toward the assessments you designed for them.

If your observations seem to suggest that students aren’t interested in your assessments, consider how you can positively impact student motivation by increasing value and expectancy. To do this, consider the following tips:

Create early opportunities for success.

Offer a low-stakes assessment in the first week of your course. This gives students the opportunity to build a positive outcome expectancy early on. For example, ask students to discuss why they’ve chosen to take the course or where they expect to encounter trouble. These activities don’t have to be complex; in many instances, you can even fold them into course icebreakers or an early formative assessment.

Be clear in your goals and expectations.

Use rubrics for all of your summative assessments (and consider using them for your formative assessments as well) that not only offer clear grading criteria, but also articulate the learning objective related to the assessment. Students should be aware of exactly what they need to know to succeed on your assessments.

Offer opportunities for reflection.

After summative assessments, create a low- or no-stakes discussion forum as a place where students can discuss the challenges they faced in the assessment. (Open this forum when appropriate so students don’t have an open forum to discuss an exam that others may not have taken.) Alternatively, create a journaling assignment where students periodically reflect on their learning. (“By now, you should have an understanding of x, y, and z. Do you feel like you have a good handle on these topics at this point? Why or why not?”) Based on the specifics of your class, this may be better done individually (an upload assignment) or collectively (a discussion forum).

Assessment Data and Student Preparedness

One reason students might struggle in your course is because your instruction might not have adequately prepared them for assessments. Many learning management systems give you the ability to see if students accessed a resource and, in some instances, how long they spent on it. If you notice that students have engaged in the instructional material but struggled on the corresponding assessments, it’s important to return to the course design triangle and ensure that your assessments align with your instruction.

To do this, evaluate how well the verbs in your objectives align with your assessments and the instruction. For example, if your objective states, “By the end of this course, students will be able to articulate a classroom management plan,” the instruction should prepare students to articulate a classroom management plan, and the assessment should require them to articulate a classroom management plan (writing one out, putting together a presentation, etc.). If the instruction is a lecture on classroom management plans, is it preparing students to actually articulate one? Probably not. By paying attention to your verbs, you’ll be better prepared to see just how well the assessments prove that students have mastered the course content you’ve designed for them.

Conversely, it’s possible that students have ignored the instructional material altogether. While alignment might be the culprit for this, too—after all, if the instruction isn’t relevant to the assessments, students have no motivation to engage in it—one thing to consider is whether the context you’ve provided is clear. Consider your course’s connective elements—the module introductions, assessment instructions, and item descriptions—and whether they’re adequately explaining what students should do when and why they should do it. Looking specifically at your instructional material, make sure students know what they can expect out of it and how it will prepare them for the corresponding assessment. This can help establish the motivation students need.

Another matter to consider related to preparedness is whether students have the prerequisite knowledge they need to succeed in your course. If your course has prerequisites, it’s often valuable to offer a prior knowledge assessment at the beginning of your course. Even if your course doesn’t have prerequisites, it might also benefit from a prior knowledge assessment. In both cases, it can help you to better identify areas of student need at the beginning of the course.

Conclusion

Coupling your learning management system’s data with your anecdotal notes can prove tremendously helpful when trying to enhance your course’s assessments. When doing this, it’s important to note that correlation doesn’t necessarily imply causation; that is, just because two variables appear related doesn’t mean that one causes the other. Even so, the data you use to identify concerns like misalignment, student behavior, and students’ preparedness can be a fantastic starting-off point for addressing issues in your online course. With that in mind, here are a few takeaways from this article:

• Just as important as alignment between objectives, assessments, and instruction is alignment between formative and summative assessments. Your students’ scores on these assessments can help you determine if you might need to address this in your course.
• If your data seems to indicate that students aren’t interested in or don’t see the relevance of your assessments, consider creating early opportunities for success, clarifying your goals and expectations, and offering structured opportunities for reflection.
• Use assessment data to see if students are adequately prepared for their assessments. By looking at factors such as your course’s prerequisites and student activity in instructional material, you can begin to see whether students are properly engaged in your instruction.