Can we use learning analytics to improve student unit completion?

Do you want to improve student unit completion? Are you interested in participating in a trial using a new learning analytics plugin in iLearn? If so, read on.

The recent field of learning analytics is starting to make an appearance in higher education. One of the first issues that learning analytics seeks to address is to identify students who are at risk of becoming disengaged and not completing a unit. Macquarie currently has a project via a teaching and delivery grant to use a learning analytics tool to examine this issue.

This project is based on the assumption that once a student who is at risk of becoming disengaged is identified, the unit convenor will send them a targeted message. The other assumption is that the student wants to receive this message and will act on it to address the issue.

Student attitudes to being contacted

To explore student attitudes to being contacted by unit convenors we conducted a survey in semester 1. It involved 754 students in 12 units. Of the respondents 6% were international students and 66% were first year students. The link to the survey was included in the unit and convenors informed students via announcements in iLearn. We found that 90% of students wanted to be contacted if their performance in the unit was unsatisfactory. Surprisingly 87% wanted to be contacted via their university email as opposed to 33% for face-to-face. Students wanted to be contacted as soon as the drop off in their performance occurs 79%, the first time it occurs 31% and following assessment results 30%. The specific behaviours that students wanted to be contacted about included low scores in assessments/assignments 86%, how they were doing compared to the rest of the class 67%, if there was missing work 56% and if there was a lack of participation or effort on their behalf 43%.

“Students using laptops” by Tony Dwyer ©2012 Macquarie University
“Students using laptops” by Tony Dwyer ©2012 Macquarie University

The trial of a learning analytics plugin

To explore the issues around the use of learning analytics by unit convenors and student responses to receiving messages we are running a trial. The trial involves unit convenors use a plugin on iLearn that will help them do two things:

  1. track which students are becoming disengaged (based on indicators such as late assessments, logins, marks in the gradebook and forum posts), and
  2. send targeted messages to encourage students to engage in the unit and access the various support services and resources available.

We are inviting unit convenors at Macquarie to participate in a trial for semester 2. You can participate by contacting the project manager chris.froissard@mq.edu.au. This is a continuation of the trial from semester 1 where we piloted a learning analytics plugin with a number of units from Macquarie. These included AHIS170, GEN110, COMP125, COMP249, ECON111 and PSYC104.

A bit about the plugin

The Moodle Engagement Analytics Plugin (MEAP) uses a formula that provides a total risk rating for each student . The risk rating attempts to measure the level of disengagement of a student.

The unit convenor selects from a number of indicators to construct a formula that is specific to their unit. They then identify and select at risk students to send targeted and personalised emails.When composing an email, the unit convenor chooses from a bank of messages. They can also write custom messages.

“MEAP mailer” by Chris Froissard © Macquarie University

“MEAP mailer” by Chris Froissard © Macquarie University

“MEAP message composition” by Chris Froissard © Macquarie University
“MEAP message composition” by Chris Froissard © Macquarie University

What are the indicators?

The indicators unit convenors can use in MEAP include whether students are:

  1. late in submitting assessments or quizzes (Assessment Activity indicator),
  2. infrequently login onto iLearn (Login Activity indicator),
  3. not obtaining sufficient marks in their tasks (Gradebook indicator),
  4. not participating in forums (Forum Activity indicator).

Is my unit appropriate for MEAP?

Units that exhibit the following characteristics may be suitable for MEAP:

  1. Include a number of online activities on iLearn e.g. forums, quizzes, assignments and resources which are closely aligned to final grades.
  2. Have a non-completion rate of at least 10% of the student cohort, and
  3. There are at least 70 students in the unit.

What’s involved in the trail?

If you would like to participate in the trial we would ask you to use MEAP to identify and contact students that are at risk of becoming disengaged. What is determined to be a reason for disengagement and what messages to send are totally up to you. Also after the trial we would like to conduct a 30-45 minute interview with you about your experiences. Finally we would like to ask your students to complete an online survey at the end of semester about their attitudes and experiences (if they are contacted by you) of receiving messages.

I’m interested but I’m uncertain about the time commitment and it looks too complex

If you are interested but worried that you may not have sufficient time we would anticipate that once MEAP was set up that you may use it on as little as 2-3 occasions. These would be crucial milestones in your unit and for example could be just before census date, after mid semester and after an indicative assessment task.

If you prefer, student support officers from some faculties are involved in the project and can support you in using MEAP. This could involve setting it up and sending messages on your behalf.

In addition project members would work with you to help you setup and use it.

If you are interested in finding out more

To find out more about this project read the Teche Blog article Learning analytics and student retention

We will be running a workshop during Learning and Teaching week where you will be able to explore MEAP in your own unit.

Also you can send an email to the project manager chris.froissard@mq.edu.au if you have any questions.

I want to participate what do I do?

Simply send an email to the project manager chris.froissard@mq.edu.au. We’ll get in contact with you shortly after.

Has anyone already tried MEAP?

If you have already used MEAP or have any data about its effectiveness we would like to hear from you. Please email the project manager chris.froissard@mq.edu.au.