OUAnalyse has been developed to predict whether students would submit their next assignment and presents the outcomes to teachers in a colour-coded dashboard. The predictions were built on data from student demographics, engagement with the virtual learning environment, assessment data and information from previous presentations of a course. Delivered weekly, the indicators help teaching staff take proactive action to support and save students from failing.
Using 220 machine learning models with human readable justification, weekly automated predictions were generated, with quality checks to discover shifts or any other issues in the data. Following several pilots, it was discovered that OUAnalyse usage by teachers was one of two significant predictors of students completing and passing a course.
Teachers that accessed OUAnalyse for at least 10% of the duration of a course showed better student pass and completion rates. It was also found that teachers had better student outcomes the year they were accessing OUAnalyse than the previous years when they had no access.
Proving the value of the system led to an OU-wide implementation. In the last year, the performance of 161,261 students was predicted in 530 modules. However, the focus was always on using data analytics to make teaching more effective, not to replace teachers. To ensure its successful adoption, OUAnalyse was introduced together with strictly-followed ethical guidelines.
A similar model was tweaked for classroom-based education at the Faculty of Mechanical engineering at Czech Technical University (CTU) to tackle high first-year student dropout rates. From 2015 to 2019, the retention rate of first-year students has increased by more than 30%, while saving almost £1 million for CTU.
While proving direct support to teachers, a new version of the dashboard is now being tested directly with students. This delivers personalised study recommendations, with the vision of changing the student-teacher interaction in distance higher education. OUAnalyse is one of the very few analytics systems available that has been tested, applied and rolled out, and has shown, through systematic evaluation, to improve student learning at a large scale.
