Pacing and progression in online learning

In virtual learning, students are often told they can learn and complete coursework “any time, any place, any pace.” However, previous research suggests that the timing of students’ assignment submissions (their pace), in fact, does matter (Kwon, 2018; Zweig, 2023). For example, students who submitted an assignment within the first week of a course had higher final course scores than students who missed this window (Zweig, 2023). 

In addition to the timing of assignment submissions, it is also important to consider the order in which students submit their assignments. Because the content in many courses is scaffolded, moving through a course sequentially should help students build foundational skills, receive timely feedback on their comprehension, and understand instructor expectations before moving on to increasingly complex topics. 

The impact of deviations from course pacing guides

Across two reports (linked below), researchers from the Michigan Virtual Learning Research Institute examined how the order of students’ assignment submissions was related to course performance by benchmarking student progress against Michigan Virtual’s course pacing guides, which are provided to help students stay on track in their courses. Both reports identified that as students become increasingly out of alignment with course pacing guides, final course scores tend to decline. 

Diving deeper into this relationship, researchers divided students into four equal groups based on how much they deviated from course pacing guides—students who deviated the least were in the first group, and students who deviated the most were in the fourth group. In the first report, which focused on Michigan Virtual’s STEM courses, researchers found a 9.5-point difference in final course score (out of 100) between students in group 1 (the least out of order) and group 4 (the most out of order). In the second report, which focused on Michigan Virtual’s World Language courses, this difference was 9.6 points. Effectively, this translates to about a letter grade difference. Across both reports, final course scores steadily decreased as students’ deviation from course pacing guides increased. 

It is important to be able to identify student characteristics that may be related to virtual course outcomes, as this could help teachers more quickly identify students who may need additional support. While pacing is associated with students’ final course scores, Michigan Virtual’s 2023-2024 Effectiveness Report highlights differences in virtual course pass rates by poverty level and race/ethnicity. For example, Freidhoff and colleagues (2025) note that the virtual pass rate for students in poverty was 58% while students not in poverty had a pass rate of 77%. Given that deviating from course pacing guides is associated with lower final course scores, understanding the extent to which groups with varying demographic characteristics complete assignments out of sequence could help inform proactive supports and interventions. As such, using data from the second report, this blog (part of a blog series exploring the impact of student assignment submission patterns) examines pacing guide deviation based on the demographic makeup of students’ home school buildings.

Methodology snapshot

Student-level assignment and performance data and building-level demographic data were analyzed for students enrolled in Michigan Virtual World Languages courses in Spring 2024. Variables were created to measure the amount and the extent to which students submitted assignments out of alignment with course pacing guides. The “percentage of assignments completed out of order” variable reflects the number of assignments students submitted out of the intended pacing guide order out of all assignments submitted. The “average magnitude” variable refers to the average difference between the intended submission order of consecutively submitted assignments for all of the assignments submitted by a student. For a complete description of the study methodology, please review the full report.

To get a better sense of how the poverty level of schools might be associated with pacing behaviors, schools were categorized based on the percentage of all learners at the school (not just virtual learners) who qualified for free or reduced-price lunch:

• Low Poverty (≤25%)
• Mid-Low Poverty (>25% to ≤50%)
• Mid-High Poverty (>50%)¹

School-level poverty data were available for 1,674 students. Approximately 45% (n = 748) were students from “Mid-Low Poverty (>25% to ≤50%)” buildings. In contrast, 23% (n = 385) came from “Mid-High Poverty (>50%)” buildings. 

To better understand how school demographics may relate to pacing, schools were also categorized by the percentage of Non-White students.

• Non-White School Population ≤25%
• Non-White School Population >25% and ≤50%
• Non-White School Population >50%²

Data on the Non-White School Population was available for 1,676 students. Approximately 68% (n = 1140) were from buildings where the Non-White School Population was ≤25%. Just under 10% (n = 164) of students came from buildings where the Non-White School Population was >50%.

Connecting pacing patterns to school demographics

Cuccolo and Green’s report (2025) revealed that most students (97%) deviate from course pacing guides at least once. When examining pacing guide deviations by the poverty level of students’ home schools, the percentage of students who submitted at least one assignment out of the intended order remained remarkably consistent (approximately 97%), varying by only about one percentage point. Further highlighting the commonality of moving out of sequence, almost 98% of students from Mid-High Poverty (>50%) buildings went out of sequence at least once. Review Table 1 for a detailed breakdown of sequencing behaviors by school poverty level.

 Table 1. Pacing Groups by School’s Poverty Level 

A similar pattern was observed when analyzing pacing guide deviations by the Racial/Ethnic makeup of students’ home school buildings. About 97% of students attending schools where the Non-White population was ≤25% submitted at least one assignment out of order. While students from these schools submitted assignments out of sequence most frequently, this value is within two percentage points of those observed in the other categories. Further, the percentage of students who submitted at least one assignment out of order was within .02% across schools where the Non-White student population was between >25% and ≤50%, and >50%. Review Table 2 for more details.

Table 2. Pacing Groups by Percent of Schools’ Non-White Population 

Connecting pacing patterns to school poverty level

Inspecting the average frequency of course pacing guide deviation by school poverty level revealed that the percentage of assignments submitted out of order was highest among students from Mid-High Poverty (>50%) buildings, on average (M = 47.15, SD = 24.39). This was approximately two to four percentage points higher than the other categories. Overall, the average percentage of assignments submitted out of order was fairly comparable across economic categories (approximately 43-47%).

The average magnitude variable provided a look at how “off” pace students were when they submitted assignments out of order. While there was consistency in average magnitude values across economic cateogories, students from Low-Poverty (≤25%) buildings had the largest values on average (M = 3.74, SD = 3.12) while students from Mid-Low Poverty (>25% to ≤50%) buildings had the smallest values on average (M = 3.39, SD = 2.95). It is worth noting the similarity of these means, as they are within 0.35 percentage points of each other. Taken together, across economic categories, the extent to which students are “off” pace is typically between three and four assignments. Review Table 3 for the average percentage of assignments submitted out of order and the average magnitude for each group of students.

Table 3. Out of Order Assignments and Average Magnitude by School’s Poverty Level

Connecting pacing patterns to the percentage of schools’ Non-White population

Breaking down the percentage of assignments submitted out of order by the school’s Non-White population suggested that students from school buildings where >50% of the population was Non-White submitted the greatest percentage of assignments out of order, on average (M = 48.99, SD = 26.91). On the other hand, students from buildings where the Non-White School Population was >25% and ≤50% had the lowest percentage of assignments submitted out of order, on average (M = 43.81, SD = 25.97). Overall, this was fairly similar to the trends observed across poverty levels, as the percentage of assignments submitted out of order varied by approximately one to five percentage points across ethnic/racial categories. 

There was remarkable consistency in magnitude values when looking across schools’ Non-White populations. The highest average magnitude values were noted among students whose school had a Non-White population of >50% (M = 3.76, SD = 3.01), which was only 0.3 percentage points greater than the values observed in the two remaining categories. Across buildings with various Non-White populations, students were approximately three and a half to four assignments “off” pace on average. Review Table 4 for the average percentage of assignments submitted out of order and the average magnitude for each group.

Table 4. Out of Order Assignments and Average Magnitude by School’s Non-White Population

Key findings

On average, students from schools with varying economic and racial/ethnic makeups deviated from pacing guides by approximately 3-4 assignments and submitted just under half of the course content out of order. While this was a near-universal behavior, several patterns stood out:

• High prevalence of out-of-sequence submissions: Over 95% of students from schools in every demographic group submitted at least one assignment out of order.
• Pacing trends by poverty level: There was consistency in the percentage of assignments submitted out of order across poverty levels, with a difference of approximately four percentage points between the group with the lowest and highest values. On average, students submitted just under half of their assignments out of order, regardless of building type.
• Pacing trends by percentage of schools’ Non-White population: There was consistency in the percentage of assignments submitted out of order across buildings with varying Non-White student population percentages—a difference of approximately five percentage points between the group with the lowest and the highest values. Across buildings with varying makeups, students submitted just under half of their assignments out of their intended order. 
• Extent of deviation: Across building types, students were typically between three and four assignments “off” the intended assignment sequence, on average.
• Performance thresholds: Cuccolo & Green (2025) found that a drop in final course scores may occur when students submit over 25% of assignments out of order or are more than one assignment “off” from pacing recommendations—on average, all demographic groups exceeded these thresholds.

Implications for educators

These trends suggest that pacing guide deviations are common, but not trivial, among students whose schools have a variety of demographic makeups. Since students who stray from their course pacing guide tend to earn lower grades, early identification is key. Mentors and instructors can support students by:

• Actively monitoring gradebooks for early signs of pacing issues
• Reinforcing pacing expectations clearly and consistently
• Offering feedback and support targeted at helping students stay, or get back, on track

It is important to note that a variety of student, course, and school factors likely interact to contribute to students’ pacing behavior. Although school demographics do not cause pacing behaviors, understanding these patterns may help educators intervene sooner and do so more effectively.

You can check out the full reports below: 

• Out of Order, Out of Reach: Navigating Assignment Sequences for STEM Success (2024)
• Out of Order, Still Out of Reach: Navigating Assignment Sequences for Michigan Virtual World Language Courses (2025)

References

Cuccolo, K. & DeBruler, K. (2024). Out of Order, Out of Reach: Navigating Assignment Sequences for STEM Success. Michigan Virtual. https://michiganvirtual.org/research/publications/out-of-order-out-of-reach-navigating-assignment-sequences-for-stem-success/ 

Cuccolo, K. & Green, C. (2025). Out of Order, Still Out of Reach: Navigating Assignment Sequences for MV World Language Courses. Michigan Virtual. https://michiganvirtual.org/research/publications/navigating-assignment-sequences-for-mv-world-language-courses/ 

Freidhoff, J. R., DeBruler, K., Cuccolo, K., & Green, C. (2025). Michigan’s k-12 virtual learning effectiveness report 2023-24. Michigan Virtual. https://michiganvirtual.org/research/publications/michigans-k-12-virtual-learning-effectiveness-report-2023-24/

Kwon, J. B. (2018). Learning trajectories in online mathematics courses. Lansing, MI: Michigan Virtual University. Retrieved from https://michiganvirtual.org/research/publications/learning-trajectories-in-online-mathematics-courses/

Zweig. J. (2023). The first week in an online course: Differences across schools. Michigan Virtual. https://michiganvirtual.org/research/publications/first-weeks-in-an-online-course/

1. Due to low ns, the ‘High Poverty >75%’ category was combined with the ‘Mid-High Poverty (>50% to ≤75%)’ category to form the existing ‘Mid-High Poverty (>50%)’ category.
   ↩︎
2. Due to low ns, the ‘Non-White Population >75%’ category was combined with the ‘Mid-High Poverty (>50% to ≤75%)’ category to form the existing ‘Non-White School Population >50%’ category. ↩︎

Self-paced asynchronous online courses offer students significant flexibility in when and where learning occurs. Recent research by the Michigan Virtual Research Institute examined how student pacing, particularly the order in which they submit assignments, is related to online STEM and World Language course performance. Understanding students’ pacing behavior and its relationship to course performance can help inform the strategies educators and mentors use when working with students in self-paced online courses.  

In the following interview from our “Interview with a Researcher” blog series, the lead researchers behind Michigan Virtual Learning Research Institute’s (MVLRI) STEM and World Languages reports synthesize some take-home messages about students’ assignment submission patterns. 

Why is it important to consider online learners’ assignment submission patterns? 

Assignment submission patterns are a part of a set of student behaviors called pacing—how students progress through a course. Pacing has traditionally been thought of as the timing of students’ assignment submissions. When conceptualizing pacing in this way, we often ask questions like, Are students waiting until the last minute to submit assignments? Are they submitting assignments early or late? Are they submitting a lot of assignments close together? It’s well-established that the timing of student assignment submissions is related to course outcomes. However, our team wanted to know more about the possible impact of out-of-order assignment submissions because, anecdotally, this was a pattern Michigan Virtual (MV) instructors were noticing within these asynchronous courses. Assignment sequencing is the term we gave to describe the order in which students submit their assignments. When we looked at this behavior across two domains (STEM and World Languages), we found evidence that it is related to course outcomes. Specifically, as students submit more assignments out of their intended course order, final course scores tend to decline.  

Why did you feel like it was important to look at assignment sequencing in World Language courses?

Great question! The original hypothesis that prompted this research was that submitting assignments out of their intended order would be detrimental to student performance because it would undermine the scaffolding built into the courses. So, based on this hypothesis, our first study on assignment sequencing used a sample of students enrolled in Michigan Virtual STEM courses since they are highly scaffolded. Of course, scaffolding is likely to vary by subject area and course, so we felt it was important to expand our research. After preliminary analyses of assignment sequencing in several other subject areas, World Language had a high percentage of students who moved out of alignment with course pacing guides and is a distinct subject area from STEM, making it an ideal choice for expanding our previous research. Looking across these two subject areas also allows us to understand the generalizability of our findings, compare and contrast key differences, and provide data-backed recommendations to instructors and mentors of students in these subject areas. 

What did students’ assignment submission patterns look like in World Language courses? Could you explain the relationship between students’ assignment sequencing and their final course scores?

We found that it was really common for students to deviate from course pacing guides! 97% of students submitted at least one assignment out of alignment with their course pacing guide. Among these students, approximately 45% of completed course assignments were submitted out of order. While the volume of assignments submitted out of order was fairly high, students were about three assignments “off” from the intended pacing guide order.

Looking across the spectrum of student performance, we observed that students’ final course scores steadily declined as their assignment submissions became increasingly out of order, both in terms of the number of assignments submitted out of order and how far “off” students were from the pacing guide expectation. To put this in perspective, students who submitted the fewest assignments out of order had average final course scores as much as a full letter grade higher than students who had the greatest number of assignments submitted out of order. 

You mentioned that the first study in this series looked at assignment sequencing in online STEM courses. Were there any notable differences between that study and this one? Did you see any patterns across these two studies?

The general pattern of results was similar across the two studies in that students’ assignment submission patterns had a relationship with final course scores. The biggest difference, however, was in the percentage of students who went out of order in each subject area. While both studies showed high rates of out-of-sequence behavior, 93% of students went out of order in the STEM study compared to 97% in World Languages. Across both studies, course scores steadily declined as students submitted a greater percentage of assignments out of order and strayed further from the intended assignment sequence. In STEM courses, there was a 9.5 point difference in average final course scores between students with the fewest and greatest number of assignments submitted out of order, whereas in World Language courses, there was a 9.6 point difference. The relationship between assignment sequencing and final course scores was really similar across the two studies, which suggests that monitoring and encouraging proper pacing is important for student performance in both subject areas.

Based on your findings across these two studies, what recommendations do you have for online instructors and mentors?

Our findings indicate that it is common for students to deviate from course pacing guides at least once during their time enrolled in MV online asynchronous courses. Some deviation is to be expected and is unlikely to negatively impact student performance, especially if that deviation is infrequent or small (e.g., within a unit). However, if students are consistently moving between units or submitting a high volume (more than 25%) of their assignments out of order, online instructors may want to flag these behaviors (and students) and monitor for performance declines. 

In general, adhering to best practices for online teaching and mentoring is recommended to help online learners be as successful as possible. Communicating course expectations early on (informing students of the structure, workload, pacing, and demands of self-paced online learning) may help students adjust their expectations and approach to their course(s). Regularly checking the gradebook and benchmarking student progress against course pacing guides can help teachers and mentors identify students who may be struggling with course pacing. Mentors and instructors should also communicate regularly about students’ progress and work collaboratively to address pacing issues.

It is also possible that submitting assignments out of order may have a greater impact on some students’ performance than others. For example, students with less content knowledge may miss key benefits of built-in scaffolding when submitting assignments out of order, which may negatively impact course performance. Further, because the design of these studies limits our ability to make cause-and-effect statements, it is likely that other factors interact with pacing to affect student performance. In particular, encouraging the development of metacognitive, time management, and self-regulated learning skills may help students reflect and make adjustments to their own learning behaviors. In this regard, providing students with personalized feedback may be useful.

Final Thoughts

Across two reports, the relationship between pacing and final course scores has consistently shown that final course scores decline as students become increasingly out of alignment with their course pacing guides. Instructors and mentors can help students succeed by paying particular attention to students’ pacing within their online courses. 

You can check out the full reports below: 

Out of Order, Out of Reach: Navigating Assignment Sequences for STEM Success

Out of Order, Still Out of Reach: Navigating Assignment Sequences for Michigan Virtual World Language Courses

In addition, this blog is part of a blog series exploring the impact of student assignment submission patterns.

Introduction

Effective communication between teachers and students is one of the foundational elements of online education. Without face-to-face interactions, teachers must rely on digital tools and platforms to build relationships, support students, and foster engagement. In this installment of our “Interview with a Researcher” blog series, the lead researchers behind the Michigan Virtual Learning Research Institute’s (MVLRI) report Starting Strong: Understanding Teacher-Student Communication in Online Courses share key findings from the study, specifically addressing the importance of communication for relationship-building.

Research Findings and Implications

Can you provide an overview of your research on teacher-student communication in online courses? 

Online courses create unique challenges when it comes to communicating and building relationships with students. With the growing prevalence of online learning, understanding how teachers communicate and connect with students in these environments is more critical than ever. Our research focused on identifying the methods, frequency, and reasons teachers and students communicate and what relationship-building practices teachers use, particularly within the first four weeks of a course. Our research also focused on determining if the frequency of teacher-student communication correlates with students’ final course grades. 

What were the key findings from your research specific to online teacher-student communication?

We found that during the first four weeks of an online course, teachers primarily relied on BrightSpace (Michigan Virtual’s LMS or learning management system), the Student Learning Portal (SLP), and email to communicate with students. The SLP was a particularly effective communication tool, because students have to log in here before accessing their courses, so this ensures that messages are seen. While most students communicated through the SLP, email, or text, teachers highlighted the importance of adapting to student preferences. 

Most teachers surveyed reported that they communicated daily with individual students, and sent out course-wide communications once a week. Teachers sent an average of two messages per student during this four-week time period. When teachers communicated with students, it was primarily to provide reminders, answer student-initiated communications, or provide feedback. Teachers emphasized that they took care to ensure their communications were timely and had a welcoming and compassionate tone to help build positive relationships, with some incorporating personal details—students’ preferred names or references to aforementioned hobbies—and using tools like Grammarly to ensure their messages have the desired tone. 

Though the study showed no statistically significant link between the number of messages students receive and their final course grade, the consistency in teacher communication practices suggests teachers were already following best practices and using effective communication strategies. 

What are considered best practices for online teacher-student communication? How does this align with Michigan Virtual teacher training, behaviors, and recommendations?

Best practices for online teacher-student communication at Michigan Virtual (MV) are grounded in established educational frameworks, including the National Standards for Quality Online Teaching (NSQOT) and Danielson’s Framework for Teaching (FFT). These frameworks emphasize the importance of timely, personalized communication to foster academic engagement and success. For instance, Michigan Virtual teachers are expected to reply to student-initiated communications within 24 hours, a practice that aligns with both frameworks’ focus on creating supportive learning environments. Teachers also use a variety of communication methods—such as email, the Student Learning Portal (SLP), and text messaging—to ensure accessibility and foster meaningful relationships with their students.

One key best practice is providing specific, personalized, and timely feedback. Michigan Virtual teachers are encouraged to score and offer feedback on assignments within 72 hours of submission (96 hours for ELA and AP courses). Research and Michigan Virtual’s training materials both highlight feedback as pivotal for student success, with over 81% of MV teachers identifying it as a highly effective relationship-building strategy. Providing students with timely and personalized feedback can provide them with important insights into their learning, motivate them to engage deeply with assignments/content, and help them stay on track.

Additionally, relationship-building is integral to effective communication. Michigan Virtual encourages teachers to reach out proactively to students, mentors, and guardians, particularly if a student is disengaged. Personalized outreach, whether through emails, phone calls, or other formats, helps create a sense of connection in a virtual environment. Ultimately, the combination of timely responses, individualized feedback, and proactive communication strategies forms the foundation of effective teacher-student interactions in online courses.

Were there any findings that surprised you or challenged your assumptions about online communication?

We were surprised to learn that although a positive relationship was observed between the number of messages sent and student grades, it was not statistically significant. The uniformity in the number of messages sent by teachers may have obscured the relationship between grades and communication. Teachers indicated they used certain highly effective communication and relationship-building strategies with their students from the beginning of a course, meaning that all students likely had an opportunity to benefit from those strategies. Alternatively, this study was focused on the first four weeks of a course, so teachers may not have had enough data to identify struggling students. Despite the lack of statistical significance, the importance of communication remains clear. Teachers should continue using best practices—personalized, timely feedback and communication—as these strategies are supported by research and teacher experience. 

Based on your findings, what practical steps can online educators take to improve communication with their students?

To improve communication in online courses, educators can take several practical steps. First, pairing communication tools like the Student Learning Portal (SLP) with email ensures that messages reach students effectively. Flexibility in communication methods is important, as students’ preferences vary, and teachers should be responsive to what works best for their students. Crafting communications with a welcoming tone and personalizing feedback helps build rapport.

Prompt responses are also key—replying to student inquiries within 24 hours shows attentiveness, while timely, constructive feedback (specific, consistent, and balanced in terms of corrective comments and motivation) supports academic success. 

Finally, while no significant link was found between communication and grades, consistent, personalized communication remains crucial for building strong relationships that support student engagement and success in online courses. Michigan Virtual offers extensive training and support to our own teachers around communication and relationship-building and also shares best practices statewide through courses offered in our Professional Learning Portal, including a series of courses (written by MV teachers) specific to online teaching and learning and focused on these best practice strategies. 

Final Thoughts

This research underscores the importance of effective teacher-student communication in online learning environments. Teachers can use a combination of timely, consistent, and personalized communication methods to enhance student engagement, support learning, and build better relationships to create a more connected virtual classroom.

As computer science (CS) continues to grow in importance in K-12 education, understanding what motivates students to pursue this field is becoming increasingly vital. In a study, Dr. Aman Yadav from Michigan State University and Dr. Kristen DeBruler from Michigan Virtual studied how students’ motivation – beliefs about their abilities (self-efficacy), the perceived challenges of learning CS (cost), and the perceived value of the subject (value) –  shape their intentions to continue studying CS. 

Connecting Motivation to Computer Science

When applying these concepts to computer science, it becomes clear why motivation is crucial. CS is often seen as challenging, requiring complex problem-solving skills and a significant time investment. This perception can either motivate students who see the effort as worthwhile or discourage those who find the challenge too daunting. Moreover, understanding how CS is applied in real-world careers, like data analysis, can enhance students’ appreciation for its utility and relevance.

The Study: High School Students’ Experiences in Online CS Courses

The researchers focused on 44 high school students enrolled in online AP Computer Science courses, examining how self-efficacy, cost, and utility influenced their intention to continue studying CS. Here’s what they found:

Self-efficacy initially appeared to be a significant factor in predicting students’ intent to pursue CS. This means those who felt more capable in their CS courses were more inclined to continue. However, when other factors (cost and utility) were included in the analysis, self-efficacy’s impact diminished.

Perceived cost had a surprising effect. Students who believed that studying CS would require significant effort were actually more likely to want to continue! This finding challenges the assumption that high perceived cost always discourages engagement. It suggests that students might associate CS with a meaningful challenge worth their time and effort.

Utility value showed an unexpected negative relationship with intent to pursue CS. Students who saw a higher utility in studying CS were less likely to want to continue. One possible explanation is that students may feel the subject’s relevance but find the commitment to learning it too demanding, especially in an online setting where support and guidance might be limited.

What Does This Mean for Teaching Computer Science?

The findings highlight the complex ways in which students’ perceptions influence their motivation to study computer science. The idea that high perceived cost can increase motivation suggests that students who view CS as a challenge are willing to tackle it if they see the effort as rewarding. However, the negative relationship between utility value and intent to pursue suggests that even if students understand the importance of CS, they might need more support to overcome perceived difficulties.

For educators, these insights are essential. As more high schools introduce CS courses, especially online options, it’s crucial to:

• Provide support and resources to help students overcome the challenges of studying CS, ensuring they feel capable and confident.
• Highlight the real-world applications of CS, clarifying the subject’s utility and emphasizing how students can succeed despite the challenges.

To learn more and explore related research, you can read the following papers:

Lishinki, A. & Yadav, A. (2021). Self-evaluation interventions: Impact on self-efficacy and performance in introductory programming. ACM Transactions on Computing Education. DOI: 10.1145/3447378  

Lishinski, A., Yadav, A., Good, J., & Enbody, R. (2016). Learning to program: Gender differences and interactive effects of students’ motivation, goals, and self-efficacy on performance. In Proceedings of International Computing Educational Research (pp. 211-220). Melbourne, Australia: Association for Computing Machinery. DOI: 10.1145/2960310.2960328.

Introduction

In this installment of our “Interview with a Researcher” blog series, we explore the key findings from two recent reports by the Michigan Virtual Learning Research Institute (MVLRI): Evaluating Professional Learning Course Offerings and Educator Engagement and Maximizing Professional Learning through Educators’ Perceptions of Utility and Self-Efficacy in Pedagogy-Focused Courses. We sat down with Dr. Kelly Cuccolo and Christa Green, the team behind these studies, to discuss the findings and implications that can help both educators and administrators optimize professional learning.

Research Findings and Implications

What motivated the research behind evaluating professional learning course offerings and educator engagement?

Professional learning is a critical component in the continuous development of educators. However, not all professional learning experiences are created equal. Our motivation was to dig deeper into the effectiveness of these offerings—are they meeting the needs of educators? Are they engaging enough to facilitate real change in teaching practices? Through this research, we wanted to provide insights that can help improve the quality and impact of professional learning programs.

Your reports highlight specific metrics used to evaluate course offerings. Can you elaborate on these metrics?

Certainly! We employed a variety of metrics to evaluate the course offerings, including course completion rates, educator satisfaction, and the perceived relevance of course content to educators’ day-to-day classroom needs. Additionally, we analyzed engagement levels during the courses through metrics such as assignment completion rates and intention to apply course content in practical settings. These metrics provide a comprehensive picture of how effective and engaging professional learning courses are for educators.

Your research shows that many courses taken by educators didn’t count toward their license renewal or recertification. What were the main motivations for educators enrolling in these courses?

Interestingly, our research found that the majority of professional learning courses taken were non-SCECH bearing, meaning they didn’t directly contribute to educators’ license renewal or recertification. Despite this, the most frequently reported motivation for taking these courses was that it was required—42.8% of educators enrolled because their school administrators mandated it. This suggests that while state standards might not drive course enrollment, local administrative requirements play a significant role in educators’ participation.

How did the requirement of courses impact educators’ satisfaction levels?

The data revealed a notable connection between course requirements and educator satisfaction. Among those who reported being unsatisfied with their course, 59.1% had enrolled because either the PD in general or the specific course was required. This finding highlights the potential importance of offering educators more agency over their professional learning choices. When educators feel that they have a say in their learning journey, satisfaction—and likely engagement—tends to be higher.

Your study also found differences in completion rates between SCECH and non-SCECH courses. Could you elaborate on that?

Yes, the completion rate for courses overall was just above 50%, but we observed that SCECH courses had a slightly lower completion rate (47.4%) than non-SCECH courses (52.5%). Notably, SCECH courses had a higher drop rate (34.9%) than non-SCECH courses (11.9%). There could be many reasons for this finding. We observed a pattern where the more courses educators enroll in, the more likely they are to drop. It’s possible educators are enrolling in several SCECH courses and later pruning their selection to meet their needs.

Based on your findings, what elements of professional learning courses do educators find most valuable and engaging?

Educators indicate they value engaging course elements that provide practical, real-world examples. The top three most useful and valuable elements were video/audio, readings, and pedagogical scenarios. These elements help educators envision how to apply new skills in their classrooms. This underscores the importance of including relevant examples and opportunities to apply new skills within professional learning, as research suggests this positively benefits educators—and is something they want.

What do your findings suggest about the importance of self-efficacy and reflection in professional learning?

Self-efficacy—the belief in one’s ability to succeed—is crucial. Most educators reported feeling confident in their ability to apply course content, with just under half planning to use it directly in their classrooms. After completing their courses, many also believed strongly in their role as reflective practitioners, which is a positive sign since self-efficacy is associated with better teaching outcomes and job satisfaction. Our findings underscore the importance of reinforcing educators’ confidence and providing opportunities for reflection within professional learning courses.

How can education leaders use these insights to improve professional learning offerings in their districts?

Education leaders can use these insights to better tailor professional learning course offerings to meet the needs and preferences of their educators. These findings suggest that providing more agency in course selection, ensuring that courses are engaging and relevant, ensuring courses are supported by relevant and accessible resources, and focusing on building educators’ self-efficacy could all contribute to more effective professional learning experiences. By prioritizing these elements, leaders can create professional development experiences that engage educators and drive real improvements in teaching practices and student outcomes. 

Final Thoughts

The findings from these reports highlight the complex motivations behind educators’ participation in professional learning and the critical role of engagement and self-efficacy. The findings also underscore the importance of continuously evaluating and refining professional learning programs. By focusing on these elements, education leaders can enhance the effectiveness of professional development and better support educators in their continuous growth.

In an era where virtual learning is becoming increasingly prevalent, understanding the best practices for engaging students online is crucial. Researchers at the Michigan Virtual Learning Research Institute (MVLRI) have conducted a comprehensive study to uncover effective strategies used by virtual educators, particularly those that help disengaged and struggling students succeed. 

The following interview, part of our “Interview with a Researcher” blog series, shares some highlights from this research.

Why was it important to examine effective practices in virtual learning environments, especially for struggling students?

The shift to emergency remote instruction during the COVID-19 pandemic highlighted a significant disparity in the success of virtual learning implementations. Schools with pre-existing, well-established virtual teaching practices fared much better. We wanted to identify what made these programs successful and how new virtual teachers and administrators could adopt these practices to better engage all students, particularly those who are disengaged or struggling.

What were some of the key strategies identified for engaging disengaged or struggling students in virtual environments?

One of the most frequently used and effective strategies was providing frequent and specific feedback, which was reported by nearly 79% of educators. This type of feedback not only supports academic progress but also helps in building strong teacher-student relationships. Additionally, involving other adults, such as onsite mentors and parents, was idenfied as being crucial. Around 69% of educators communicated with the student’s onsite mentor, and 61% encouraged parental involvement. These strategies help bridge the gap created by the lack of physical presence in virtual learning.

Communication seems to be a recurring theme. Can you elaborate on the importance of communication in virtual learning environments?

Absolutely. Communication is the backbone of virtual education. Effective communication strategies include maintaining regular contact through various channels like LMS messaging, phone calls, and web conferencing tools. Many educators also emphasized the importance of being available for students through scheduled office hours or drop-in times. Establishing clear communication channels helps ensure that students, parents, and educators are on the same page, which is vital for student engagement and success.

The study also looked at professional development for virtual educators. What sources of professional development were found to be most effective?

Our findings showed that optional opportunities provided by the virtual school or program were considered the most effective, with over 50% of educators endorsing them. Conferences and informal peer mentoring were also highly valued. These professional development sources are preferred because they are immediately applicable and foster a sense of community among educators, which is essential for sharing best practices and support.

What challenges did educators face in virtual teaching, particularly in connecting with disengaged students?

One of the biggest challenges is the lack of face-to-face interaction, which makes it difficult to read body language and establish personal connections. This physical separation also complicates identifying the specific reasons behind a student’s disengagement. Additionally, educators mentioned difficulties in effectively communicating with parents and guardians, who are crucial allies in supporting student engagement and progress.

Based on your research, what recommendations would you give to new virtual teachers working with disengaged or struggling students?

Focus on building strong relationships with your students from the beginning. Use frequent, specific feedback to show students that you care about their progress. Keep open channels of communication and be flexible with your teaching methods to accommodate diverse learning needs. Also, involve parents and onsite mentors whenever possible to create a supportive network around the student. Flexibility, patience, and a personalized approach are key.

The insights from this study underscore the importance of tailored strategies, consistent communication, and community support in virtual learning environments. By focusing on relationship-building, providing specific feedback, and involving parents and mentors, educators can significantly improve engagement and success for all students, especially those who struggle. As virtual learning continues to evolve, these findings offer a valuable roadmap for educators seeking to enhance their practices and better support their students in a digital age.

Introduction

Helping students succeed in their online courses is a team effort involving leveraging course design principles and technology to facilitate learning. Instructional designers use technology to design courses that optimize students’ and teachers’ experiences. Technology operations staff help implement and adapt systems to meet users’ needs. Pacing, or how students move through a course, is important to student success. Cramming and submitting assignments out of their intended order are associated with poor course performance (DeBruler, 2021; Cuccolo & DeBruler, 2024; Michigan Virtual Learning Research Institute, 2019). Because online courses are dynamic environments facilitating learning “anytime, anyplace,” it is crucial to leverage technology and course design principles to support students’ pacing to optimize their experience. In this blog, we’ll explore the pivotal roles of instructional designers and technology operations in students’ experiences with pacing in online courses.

Expert Interviews

Michigan Virtual Learning Research Institute researchers talked with Kim Garvison and Megan Riggers from Michigan Virtual’s Instructional Product Development (iPD) team, and Kristen Crain Senior Director of Technology Operations about the interplay between technology and course design principles in addressing pacing. This blog highlights central themes that cut across our conversations. The transcript has been edited for clarity and brevity.

Functionality: Supporting Students, Teachers, and Guardians through LMS Features

Centering Students and Teachers

How is course pacing important within the context of course design?

We try to ensure the workload is evenly distributed. We consider when students might be doing these assignments and how we’re balancing intensity in terms of cognitive load for the student and the teacher. Another way we consider pacing is through how we distribute auto and teacher-graded assignments. Even though we have a short turnaround time for teachers to grade assignments, balancing the type of assignment does help move students through their course without having roadblocks.

What design choices have you found helpful for supporting students struggling with pacing in their online courses?

As designers, we have to think about our end users and how much is realistic for them to handle. What grade are they in? What’s their age? What’s their reading level? What can they handle? What experience might they have in an online setting? We try to create courses with predictable structures: approximately the same number of lessons and each unit follows the same general structure. That way, both teachers and students know what to expect. Predictability also helps mentors and parents support students more effectively during their courses.

We also ensure our content is accessible for students with different learning needs. For example, our LMS helps ensure we have appropriate alt text (descriptive text concisely conveying the meaning of an image). Having hurdles for students with different learning needs is a big deal for their pacing, and therefore is a big deal to us when designing a course.

Organizational Tools Aid in User Experience

Organizational tools provided by the LMS are critical for helping students navigate the course efficiently. For instructors, organization allows them to serve students more effectively by creating a centralized location where they can assess students’ progress.

How can technology or the LMS be leveraged to address pacing?

We are very intentional with setting up course navigation. We don’t want students thinking “What do I do next? How do I find the quiz? Where do I click?” That’s the background instructional design part that helps students work through their content. Having a good LMS helps us have students get friction from content and learning, and not from navigating their course. I think that the technology is what allows us to not think about the technology. The more none of us have to think about the technology, that’s a good sign that it’s working well for us.

Our LMS is heavily invested in simplifying the instructor and teacher experience. Brightspace centralizes everything so an instructor can see who is accessing content, who submitted content, and student grades in one view. LMSs also do a great job creating buckets of content, nesting units together, and keeping students moving through all the content linearly.

Reminders

Reminder tools within the LMS were viewed positively as a way to provide guide rails for students as they move through content at their own pace.

Is there anything that helps students adhere more closely to the pacing guides?

Instead of thinking, “Okay, on this date, we have to blast out this reminder,” for our courses, location is more important than time because we don’t know where the students will be within a course at a specific date.

In addition to strategically placed reminders within the LMS, data from the LMS can be used to create reminders that are pushed outward to students, and guardians.

LMSs can compile extensive amounts of data in a consumable way for a school, an instructor, and even parents. They can leverage the data into tools, widgets, calendars, emails, and other things that make the students’ experience more streamlined. For example, certain LMSs let parents opt into features such as automated daily emails telling them what’s due for their child.

Leveraging LMS to Promote Sequential Course Progression

How do you encourage students to follow the logical progression of the course content? Do you use any specific tools?

We use LMS features that should help students get back on track. For example, we have a checklist of assignments at the beginning of each unit. There are also LMS features at the end of lessons and units pointing out incomplete assignments and reminding students to go back. Our LMS also provides the option to put a password on an assignment. We do this for all final exams as another way to say, “Hey, this is important.”

We make sure everything is scaffolded so it’s clear to students that they have to go through certain assignments before they can do the next project. If students skip content, the teacher can point out that they don’t meet the rubric requirements because we develop rubrics that emphasize the lesson’s content.

We also have an internally designed pacing guide application that’s accessible by the student and the mentor so they can see a week-by-week breakdown of what students should accomplish including graded and non-graded activities.

Addressing Assignment Cherry-Picking

We’ve conducted research showing students tend to favor auto-graded and higher-point assignments. What are some potential workarounds for addressing this cherry-picking?

We try to anticipate it and consider the point spread and the ratio of auto-graded to teacher-graded assignments. We don’t want a student to be able to only take the auto-graded quizzes and pass the course. Usually, we go for approximately 40% auto-graded to 60% teacher-graded.

One way to address this is to incorporate conditional releases which can function based on completion and unlock specific content, units, or modules. We don’t use this tool often, but it can create different learning methods for students. One student may come into a course and say, “I want to learn today via video.” Then they go into the content, choose the method they want, and it unlocks content that’s all video-based instruction.

We leave everything open though, and that’s a transparency piece so the student can see the whole scope of the class and every graded object upfront. Sometimes waiting for a teacher to grade an assignment before being able to move on frustrates students. In addition, only allowing students to see one unit at a time (rather than the whole course) can add another layer of frustration. From an instruction design standpoint, it seems easy – just lock it until they pass. However, from the user side, that’s not always how it goes.

Thinking Ahead

What would an LMS that perfectly addresses students’ pacing needs look like?

One that automatically sets a student’s pace as they start the course based on its duration, and is flexible enough to empower teachers or administrators to override, re-pace, or modify students’ pace in bulk since they’re dealing with large course loads of students.

It would be neat if courses had their own AI bot that could provide students with reminders like, “Hey, you should be working on this assignment,” or “Hey, it looks like you are behind here. Your new due date is.”

Course Pacing Blog Series

In our Course Pacing blog series, we discuss pacing and how it impacts student success with input from several different subject matter experts. Our hope with this series is to bring to light how different organizations and experts approach course pacing, share their insights and struggles, provide relevant research and resources, and determine areas for future research. Stay up to date on future blogs in this series by signing up for email notifications!

One of the benefits of online learning is that students can work at their own pace. However, not all students have developed the time management skills to work through a course consistently. Perhaps unsurprisingly, research has shown that consistent course pacing results in higher student achievement (DeBruler, 2021). 

To explore course pacing through the lens of an online program administrator, researchers from the Michigan Virtual Learning Research Institute (MVLRI) interviewed Andrea McKay, Director of Instruction for Michigan Virtual in April 2024. The transcript from our interview has been edited for clarity and brevity. During our conversation, we explored how through intentional decisions made by online program administrators and support from both online teachers and on-site mentors, students are guided and supported to stay on pace in their online courses. 

Understanding Course Pacing and Developing Pacing Guides

How is course pacing currently addressed in Michigan Virtual courses? For example, how do you ensure students keep up without feeling rushed?

We provide pacing guides within our courses that show students a week-by-week breakdown of assignments to complete. Our teachers can access and adjust the pacing guide if students get behind and need a new plan to help them catch up. Once students have enrolled, their start date remains the same, and the length of the term remains the same; however, we can adjust the pacing guide so students see what assignments they need to complete within the shorter working time they have as a result of falling behind. 

How do you develop pacing guides for your courses? 

After a course is designed, the pacing guide is developed by splitting up the content and assignments over the number of weeks in the length of the term. Subject matter experts who understand the content and assignment expectations evaluate the pacing guide to ensure the pace is realistic. In addition, our courses are regularly updated which includes re-evaluating the pacing guide and making necessary adjustments. 

Tools for Pacing, Progress Tracking, and Data Utilization

Are there any tools or platforms that Michigan Virtual uses to help students keep pace and track their progress?

Teachers send monthly progress reports to the student, mentor, and parent so everyone understands the student’s course progress. Progress reports are personalized to each student by pulling data from our LMS (Brightspace) and student information system (SIS) such as the number of complete and incomplete assignments, current grade, and total assignments in the course. Some teachers add information such as personalized comments or a reminder of the course end date. For example, “Are you on track to complete the course by [course end date]? With a few adjustments on the back end, this tool within our LMS allows teachers to send these personalized progress reports to their entire course roster. In addition, teachers use our SIS to sort students in a course by start date and, based on that, send timely communication to help keep students on pace. Being able to sort students within a course by start date is very important in terms of accurately tracking student progress as we offer many different course start dates within each semester to better meet school districts’ needs for course start dates to align with their school calendar.

Online Program Course Pacing: Challenges and Solutions

What major challenges do you run into regarding course pacing in an online setting?

One major challenge is grade reporting. A common request from schools is to know a student’s current grade to determine whether that student is eligible to play sports. Unfortunately, the way our systems report students’ scores does not give them that information. In our courses, students start with zero points and build up their total course points with every assessment submitted. We provide an accurate display of their overall score at all times, but it’s not the same as a weekly grade that schools may be more used to. However, adjusting our grade reporting process would result in far fewer start and end date options and reduced flexibility in submission deadlines.

What do you think are the biggest pacing hurdles for students learning online? 

While the flexibility to work at your own pace is a common reason why many students take courses with us, some students need help to learn how to manage their time effectively. If a student gets behind, assignments add up quickly, resulting in a poor learning experience and a mountain of assignments to submit. 

What challenges do teachers face with pacing in their online courses? 

If instructors are overloaded with a flurry of assignment submissions at the end of a course, they cannot provide the same feedback quality while also meeting grading turnaround expectations. The resulting assignment feedback isn’t as effective because those students procrastinating and turning in numerous assignments during the last few weeks are probably not as concerned about whether they didn’t quite meet a learning target, what content they might need to revisit, or how to use and grow from instructor feedback—they’re just trying to get through it. An additional problem when students leave so many assignments for the end of the course is that they might be tempted to take some shortcuts and plagiarize, which turns into another huge headache at the end of a term. We have found the number of plagiarism incidents increases drastically in the last few weeks of a course. As a result, teachers communicate progress regularly and try to keep students on pace. 

Given our recent finding that the extent to which students submit assignments out of order is associated with lower grades, what’s your take on getting students to stick to the order and pace you set out?

Teachers understand that students may choose to complete assignments based on which ones will earn them a higher score more quickly, otherwise known as cherry-picking. Unfortunately, this means that rather than working sequentially through courses designed to scaffold and build skills as students progress, students are sometimes more focused on points than learning. Our teachers recognize this and will reach out to students to get them back on track. 

Pacing: Mentor and Parent Roles

Do mentors and parents play a part in monitoring students’ pacing? What does that collaboration look like?

There is a triangle of student support between the teacher, mentor, and (ideally) the parent as they are a critical component in overall student success (Borup & Stimson, 2017; Borup et al., 2017). Both mentors and parents are copied on progress reports to remain aware of student pace. In addition, specific mentor reports and monitoring tools are available through our LMS, Brightspace, to help mentors support students most effectively. Teachers understand that parents and mentors may know students in ways that they do not, so we all value the roles and support they provide and work hard to establish and maintain open lines of communication. 

Looking Down the Road

Are there any big changes you would like to see or pacing challenges you are preparing for in K-12 online learning?

In addition to continuing to meet districts’ needs for different course start/end dates and providing the flexibility of an adjustable pacing guide, we would love to offer a different view of students’ scores based on current progress and performance as school districts so frequently request it. This is a current limitation of our flexible scheduling options and how they interact with the LMS; however, we are working to find solutions. Despite students having access to a pacing guide and inevitably tallying up points, my other hope is to get to a point where students think more about their learning rather than the points. 

Course Pacing Blog Series

In this Course Pacing Blog Series, we discuss pacing and how it impacts student success with input from several different subject matter experts. Our hope with this series is to bring to light how different organizations and experts approach course pacing, share their insights and struggles, provide relevant research and resources, and determine areas for future research. Stay up to date on future blogs in this series by signing up for email notifications!

References

Borup, J. & Stimson, R. (2017). Helping students be successful: Mentor responsibilities. Michigan Virtual University. https://michiganvirtual.org/research/publications/helping-online-students-be-successful-mentor-responsibilities/  

Borup, J., Chambers, C. B., Stimson, R. (2017). Helping online students be successful: Parental engagement. Michigan Virtual University. https://michiganvirtual.org/research/publications/helping-online-students-be-successful-parental-engagement/ 

DeBruler, K. (2021). Research On K-12 Online Best Practices. Michigan Virtual. https://michiganvirtual.org/blog/research-on-k-12-online-best-practices

Acknowledgments

The author would like to thank Andrea McKay and Dr. Shannon Smith from Michigan Virtual’s Learning Services team as well as Dr. Kelly Cuccolo and Dr. Kristen DeBruler from the Michigan Virtual Learning Research Institute for their contributions and advice in developing this blog post.

In the ever-evolving landscape of education, online learning has become increasingly prevalent, offering students flexibility and accessibility to course materials. Recent research has delved into the dynamics of online coursework completion, particularly focusing on the sequencing of assignments and its impact on student success. Understanding how students navigate through their coursework, whether adhering to prescribed pacing guides or deviating from them, provides valuable insights for educators aiming to enhance student outcomes. 

The following interview from our “Interview with a Researcher” blog series shares some highlights from this research.

Is it common for students to go out of sequence when completing online course assignments?

Absolutely. Our study revealed that a vast majority of students, around 93%, tended to go out of sequence at least once when completing assignments in online STEM courses. This suggests that flexibility in pacing is a common practice among online learners.

How does the average number of assignments submitted out of sequence affect students’ performance?

We found that, on average, students submitted approximately 17.5 assignments out of order, accounting for about 38.15% of all course assignments. While some deviation from the prescribed order is expected, this trend indicates that a significant portion of students are not strictly following the intended sequencing of assignments. We observed that as students became increasingly out of sequence (submitted a higher proportion of assignments out of order) their grades dropped such that students with the highest number of assignments out of order had the lowest final course grades.

Could you explain the relationship between students’ course progression and their final grades?

Certainly. Our research uncovered a significant negative relationship between the proportion of assignments completed out of order, as well as the magnitude of these deviations, and students’ final grades. Essentially, as students submitted more assignments out of sequence, their final grades tended to decrease accordingly.

Are there noticeable differences in final grades between students who go out of sequence and those who stay in sequence?

Yes, indeed. We observed a clear distinction in final grades between students who adhered to the intended sequencing of assignments and those who did not. On average, students who stayed in sequence achieved a final grade of 89.2, whereas those who went out of sequence averaged 79.7. When we grouped students by how often they submitted assignments out of sequence the discrepancies in final grades were even more pronounced. Students in the bottom 25% for being out of sequence (the least proportion of “out of order” assignment submissions) consistently had the highest grades on average, and those in the top 25% (the highest proportion of “out of order” assignments) had the lowest, 86.8 compared to 74.1–a difference of 12.7 points. This sizable difference underscores the impact of following the prescribed order of assignments on student success.

What recommendations do you have for educators to support students in navigating online course assignments?

Educators play a pivotal role in guiding students toward effective online learning strategies. It is essential to emphasize the significance of following pacing guides and completing assignments in the intended order. Providing clear expectations, scaffolding course content, and explaining the purpose of assignments can help students grasp the value of pacing and organization. Moreover, supporting students in developing self-regulatory skills and effective time management practices empowers them to succeed in online learning environments.

The insights gleaned from this study illuminate the intricate relationship between assignment sequencing and student success in online STEM courses. As online learning continues to evolve, understanding how students navigate their coursework, the impact of pacing deviations, and the role of educators in guiding them toward success becomes increasingly vital. By embracing these findings, educators can empower students to effectively manage their coursework, enhance engagement, and ultimately achieve academic success in the digital realm of education.

Be sure to check out the full research report for more information! In addition, this blog is part of a blog series exploring the impact of student assignment submission patterns.