Out of Order, Still Out of Reach: Variations in Pacing among World Language Students

Cuccolo & Green’s (2025) report highlighted the relationship between students’ assignment submission patterns and final course scores. Given that pacing has important implications for student performance, knowing what assignment submission patterns look like across schools with varying demographics could help prompt early identification and intervention. As such, this blog explores students’ assignment submission patterns based on school-level demographic information.
Facebook
LinkedIn
X/Twitter
Psychological concept of choices and Finding or Choosing the right life path with group of diverse people following intersecting paths to doors with central signpost on arrow, flat vector illustration

Pacing and progression in online learning

In virtual learning, students are often told they can learn and complete coursework “any time, any place, any pace.” However, previous research suggests that the timing of students’ assignment submissions (their pace), in fact, does matter (Kwon, 2018; Zweig, 2023). For example, students who submitted an assignment within the first week of a course had higher final course scores than students who missed this window (Zweig, 2023). 

In addition to the timing of assignment submissions, it is also important to consider the order in which students submit their assignments. Because the content in many courses is scaffolded, moving through a course sequentially should help students build foundational skills, receive timely feedback on their comprehension, and understand instructor expectations before moving on to increasingly complex topics. 

The impact of deviations from course pacing guides

Across two reports (linked below), researchers from the Michigan Virtual Learning Research Institute examined how the order of students’ assignment submissions was related to course performance by benchmarking student progress against Michigan Virtual’s course pacing guides, which are provided to help students stay on track in their courses. Both reports identified that as students become increasingly out of alignment with course pacing guides, final course scores tend to decline. 

Diving deeper into this relationship, researchers divided students into four equal groups based on how much they deviated from course pacing guides—students who deviated the least were in the first group, and students who deviated the most were in the fourth group. In the first report, which focused on Michigan Virtual’s STEM courses, researchers found a 9.5-point difference in final course score (out of 100) between students in group 1 (the least out of order) and group 4 (the most out of order). In the second report, which focused on Michigan Virtual’s World Language courses, this difference was 9.6 points. Effectively, this translates to about a letter grade difference. Across both reports, final course scores steadily decreased as students’ deviation from course pacing guides increased. 

It is important to be able to identify student characteristics that may be related to virtual course outcomes, as this could help teachers more quickly identify students who may need additional support. While pacing is associated with students’ final course scores, Michigan Virtual’s 2023-2024 Effectiveness Report highlights differences in virtual course pass rates by poverty level and race/ethnicity. For example, Freidhoff and colleagues (2025) note that the virtual pass rate for students in poverty was 58% while students not in poverty had a pass rate of 77%. Given that deviating from course pacing guides is associated with lower final course scores, understanding the extent to which groups with varying demographic characteristics complete assignments out of sequence could help inform proactive supports and interventions. As such, using data from the second report, this blog (part of a blog series exploring the impact of student assignment submission patterns) examines pacing guide deviation based on the demographic makeup of students’ home school buildings.

Methodology snapshot

Student-level assignment and performance data and building-level demographic data were analyzed for students enrolled in Michigan Virtual World Languages courses in Spring 2024. Variables were created to measure the amount and the extent to which students submitted assignments out of alignment with course pacing guides. The “percentage of assignments completed out of order” variable reflects the number of assignments students submitted out of the intended pacing guide order out of all assignments submitted. The “average magnitude” variable refers to the average difference between the intended submission order of consecutively submitted assignments for all of the assignments submitted by a student. For a complete description of the study methodology, please review the full report.

To get a better sense of how the poverty level of schools might be associated with pacing behaviors, schools were categorized based on the percentage of all learners at the school (not just virtual learners) who qualified for free or reduced-price lunch:

  • Low Poverty (≤25%)
  • Mid-Low Poverty (>25% to ≤50%)
  • Mid-High Poverty (>50%)1

School-level poverty data were available for 1,674 students. Approximately 45% (n = 748) were students from “Mid-Low Poverty (>25% to ≤50%)” buildings. In contrast, 23% (n = 385) came from “Mid-High Poverty (>50%)” buildings. 

To better understand how school demographics may relate to pacing, schools were also categorized by the percentage of Non-White students.

  • Non-White School Population ≤25%
  • Non-White School Population >25% and ≤50%
  • Non-White School Population >50%2

Data on the Non-White School Population was available for 1,676 students. Approximately 68% (n = 1140) were from buildings where the Non-White School Population was ≤25%. Just under 10% (n = 164) of students came from buildings where the Non-White School Population was >50%.

Connecting pacing patterns to school demographics

Cuccolo and Green’s report (2025) revealed that most students (97%) deviate from course pacing guides at least once. When examining pacing guide deviations by the poverty level of students’ home schools, the percentage of students who submitted at least one assignment out of the intended order remained remarkably consistent (approximately 97%), varying by only about one percentage point. Further highlighting the commonality of moving out of sequence, almost 98% of students from Mid-High Poverty (>50%) buildings went out of sequence at least once. Review Table 1 for a detailed breakdown of sequencing behaviors by school poverty level.

 Table 1. Pacing Groups by School’s Poverty Level 

Poverty LevelnIn-SequenceOut-of-Sequence
Low Poverty (≤25%)5413.88%96.12%
Mid-Low Poverty (>25% to ≤50%)7482.41%97.59%
Mid-High Poverty (>50%)3852.34%97.66%

A similar pattern was observed when analyzing pacing guide deviations by the Racial/Ethnic makeup of students’ home school buildings. About 97% of students attending schools where the Non-White population was ≤25% submitted at least one assignment out of order. While students from these schools submitted assignments out of sequence most frequently, this value is within two percentage points of those observed in the other categories. Further, the percentage of students who submitted at least one assignment out of order was within .02% across schools where the Non-White student population was between >25% and ≤50%, and >50%. Review Table 2 for more details.

Table 2. Pacing Groups by Percent of Schools’ Non-White Population 

%Non-White CategorynIn-SequenceOut-of-Sequence
Non-White School Population ≤25%11402.54%97.46%
Non-White School Population >25% and ≤50%3724.03%95.97%
Non-White School Population >50%1643.05%96.95%

Connecting pacing patterns to school poverty level

Inspecting the average frequency of course pacing guide deviation by school poverty level revealed that the percentage of assignments submitted out of order was highest among students from Mid-High Poverty (>50%) buildings, on average (M = 47.15, SD = 24.39). This was approximately two to four percentage points higher than the other categories. Overall, the average percentage of assignments submitted out of order was fairly comparable across economic categories (approximately 43-47%).

The average magnitude variable provided a look at how “off” pace students were when they submitted assignments out of order. While there was consistency in average magnitude values across economic cateogories, students from Low-Poverty (≤25%) buildings had the largest values on average (M = 3.74, SD = 3.12) while students from Mid-Low Poverty (>25% to ≤50%) buildings had the smallest values on average (M = 3.39, SD = 2.95). It is worth noting the similarity of these means, as they are within 0.35 percentage points of each other. Taken together, across economic categories, the extent to which students are “off” pace is typically between three and four assignments. Review Table 3 for the average percentage of assignments submitted out of order and the average magnitude for each group of students.

Table 3. Out of Order Assignments and Average Magnitude by School’s Poverty Level

Economic CategoryMean (SD)MinMedianMax
Percentage Out of Order
Low Poverty (≤25%)45.02 (26.17)0.0048.1597.22
Mid-Low Poverty (>25% to ≤50%)43.55 (25.05)0.0046.1197.70
Mid-High Poverty (>50%)47.15 (24.39)0.0050.0095.38
Average Magnitude
Low Poverty (≤25%)3.74 (3.12)0.002.9313.89
Mid-Low Poverty (>25% to ≤50%)3.39 (2.95)0.002.5014.16
Mid-High Poverty (>50%)3.42 (2.81)0.002.6214.56

Connecting pacing patterns to the percentage of schools’ Non-White population

Breaking down the percentage of assignments submitted out of order by the school’s Non-White population suggested that students from school buildings where >50% of the population was Non-White submitted the greatest percentage of assignments out of order, on average (M = 48.99, SD = 26.91). On the other hand, students from buildings where the Non-White School Population was >25% and ≤50% had the lowest percentage of assignments submitted out of order, on average (M = 43.81, SD = 25.97). Overall, this was fairly similar to the trends observed across poverty levels, as the percentage of assignments submitted out of order varied by approximately one to five percentage points across ethnic/racial categories. 

There was remarkable consistency in magnitude values when looking across schools’ Non-White populations. The highest average magnitude values were noted among students whose school had a Non-White population of >50% (M = 3.76, SD = 3.01), which was only 0.3 percentage points greater than the values observed in the two remaining categories. Across buildings with various Non-White populations, students were approximately three and a half to four assignments “off” pace on average. Review Table 4 for the average percentage of assignments submitted out of order and the average magnitude for each group.

Table 4. Out of Order Assignments and Average Magnitude by School’s Non-White Population

%Non-White CategoryMean (SD)MinMedianMax
Percentage Out of Order
Non-White School Population ≤25%44.54 (24.81)0.0047.1197.70
Non-White School Population >25% and ≤50%43.81 (25.97)0.0045.9497.22
Non-White School Population >50%48.99 (26.91)0.0054.2395.00
Average Magnitude
Non-White School Population ≤25%3.47 (2.98)0.002.5814.56
Non-White School Population >25% and ≤50%3.48 (2.94)0.002.7012.22
Non-White School Population >50% 3.76 (3.01)0.003.0013.95

Key findings

On average, students from schools with varying economic and racial/ethnic makeups deviated from pacing guides by approximately 3-4 assignments and submitted just under half of the course content out of order. While this was a near-universal behavior, several patterns stood out:

  • High prevalence of out-of-sequence submissions: Over 95% of students from schools in every demographic group submitted at least one assignment out of order.
  • Pacing trends by poverty level: There was consistency in the percentage of assignments submitted out of order across poverty levels, with a difference of approximately four percentage points between the group with the lowest and highest values. On average, students submitted just under half of their assignments out of order, regardless of building type.
  • Pacing trends by percentage of schools’ Non-White population: There was consistency in the percentage of assignments submitted out of order across buildings with varying Non-White student population percentages—a difference of approximately five percentage points between the group with the lowest and the highest values. Across buildings with varying makeups, students submitted just under half of their assignments out of their intended order. 
  • Extent of deviation: Across building types, students were typically between three and four assignments “off” the intended assignment sequence, on average.
  • Performance thresholds: Cuccolo & Green (2025) found that a drop in final course scores may occur when students submit over 25% of assignments out of order or are more than one assignment “off” from pacing recommendations—on average, all demographic groups exceeded these thresholds.

Implications for educators

These trends suggest that pacing guide deviations are common, but not trivial, among students whose schools have a variety of demographic makeups. Since students who stray from their course pacing guide tend to earn lower grades, early identification is key. Mentors and instructors can support students by:

  • Actively monitoring gradebooks for early signs of pacing issues
  • Reinforcing pacing expectations clearly and consistently
  • Offering feedback and support targeted at helping students stay, or get back, on track

It is important to note that a variety of student, course, and school factors likely interact to contribute to students’ pacing behavior. Although school demographics do not cause pacing behaviors, understanding these patterns may help educators intervene sooner and do so more effectively.

You can check out the full reports below: 

References

Cuccolo, K. & DeBruler, K. (2024). Out of Order, Out of Reach: Navigating Assignment Sequences for STEM Success. Michigan Virtual. https://michiganvirtual.org/research/publications/out-of-order-out-of-reach-navigating-assignment-sequences-for-stem-success/ 

Cuccolo, K. & Green, C. (2025). Out of Order, Still Out of Reach: Navigating Assignment Sequences for MV World Language Courses. Michigan Virtual. https://michiganvirtual.org/research/publications/navigating-assignment-sequences-for-mv-world-language-courses/ 

Freidhoff, J. R., DeBruler, K., Cuccolo, K., & Green, C. (2025). Michigan’s k-12 virtual learning effectiveness report 2023-24. Michigan Virtual. https://michiganvirtual.org/research/publications/michigans-k-12-virtual-learning-effectiveness-report-2023-24/

Kwon, J. B. (2018). Learning trajectories in online mathematics courses. Lansing, MI: Michigan Virtual University. Retrieved from https://michiganvirtual.org/research/publications/learning-trajectories-in-online-mathematics-courses/

Zweig. J. (2023). The first week in an online course: Differences across schools. Michigan Virtual. https://michiganvirtual.org/research/publications/first-weeks-in-an-online-course/

  1. Due to low ns, the ‘High Poverty >75%’ category was combined with the ‘Mid-High Poverty (>50% to ≤75%)’ category to form the existing ‘Mid-High Poverty (>50%)’ category.
    ↩︎
  2. Due to low ns, the ‘Non-White Population >75%’ category was combined with the ‘Mid-High Poverty (>50% to ≤75%)’ category to form the existing ‘Non-White School Population >50%’ category. ↩︎
Facebook
LinkedIn
X/Twitter
Picture of Michigan Virtual Learning Research Institute

Michigan Virtual Learning Research Institute

The Michigan Virtual Learning Research Institute (MVLRI) is a non-biased organization that exists to expand Michigan’s ability to support new learning models, engage in active research to inform new policies in online and blended learning, and strengthen the state’s infrastructures for sharing best practices. MVLRI works with all online learning environments to develop the best practices for the industry as a whole.

You might also like...

Top-down View of Children in Classroom Use Tablets and Laptops, Connectivity by Lines of AI Data, Information. Diverse Kids Study with Interactive E-Learning, Modern Education, Digital Literacy

Have You Considered AI in Your Classroom? A Khanmigo Pilot Story

In a two-phase pilot across Michigan schools, educators used Khanmigo, an AI-powered tutor and teaching assistant, to explore how AI might support teaching and learning. Their reflections surfaced both opportunities and challenges. The big takeaway?...
Illustration of three people using virtual reality (VR) headsets. Each person is interacting with a colorful, cosmic-themed environment featuring abstract planets, stars, and orbits. One person points at a floating circle, another raises their hand toward an orange sphere, and the third holds VR controllers, immersed in the experience. The background features deep purples, blues, and greens with celestial shapes like the moon and stars.

From Curiosity to Career: Exploring Possibilities with VR

Explore how immersive VR simulations helped students step into real-world roles: from EMTs to chefs, all without leaving the classroom....
A cheerful classroom scene with elementary-aged students working on laptops. A smiling boy in the foreground looks directly at the camera, while other students and a teacher engage in learning activities in the background. The classroom is softly lit, with large windows and warm yellow curtains.

The AI Horizon: Case Studies in Michigan Education’s Transformation (Part 2)

The work to integrate artificial intelligence in Michigan schools is gaining momentum. In Part 1 of this series, we looked at how four districts are building their own foundations through task forces, learning councils, and...

A classroom scene where a diverse group of elementary-aged students work on laptops at their desks. A smiling teacher leans over to assist one student, creating an engaging and supportive learning environment. Sunlight streams through large windows with yellow curtains in the background.

The AI Horizon: Case Studies in Michigan Education’s Transformation (Part 1)

The integration of Artificial Intelligence (AI) is rapidly shifting the landscape of education, presenting both unprecedented opportunities and novel challenges for K-12 institutions. Across Michigan, a wave of innovation is taking shape as K-12 schools,...

Two coworkers discussing work over lunch outside their office building while on a break

Mastering Change: Coaching as the Lever for Transformation in Education

Originally published in the Spring 2025 issue of the MASB Leaderboard Magazine. Introduction A few years ago, I came across Master of Change by Brad Stulberg, and it couldn’t have landed at a more fitting...

Woman looking at a computer screen displaying a bar graph

Out of Order, Still Out of Reach: An Interview with a Researcher

In this blog, MVLRI researchers synthesize the key findings from two research studies about student assignment submission patterns in Michigan Virtual online courses....

The Digital Backpack

Get our latest articles sent straight to your inbox every week!