“typical classical” by Krembo1 2008 CC BY-NC 2.0

What’s happening under the hood (of my iLearn site)?

At the end of the session, unit convenors ask themselves, how did my students do in the unit, and how can I improve it for next time? Of course, they have access to grades, anecdotal feedback from tutors and students and various survey instruments such as LEUs. However, using Reports they can also see what is going on under the “hood” of their iLearn site. The Reports section in the Toolbar of iLearn can give convenors access to reports and data that can tell them how students are using resources and navigating throughout the site.

I have found after speaking to many unit convenors that do engage in unit review, that most of them do not use, or are not familiar with the information in the Reports section. Here we will look at a few reports and processes that convenors have available to them.

“Location of reports in iLearn Toolbar” by Chris Froissard ©2016 Macquarie University
“Location of reports in iLearn Toolbar” by Chris Froissard ©2016 Macquarie University

The Activity report

The Activity report shows the number of views of activities (quizzes, Turnitin etc)  and resources (URLs, documents etc) within each topic. This can give you a quick overview of which activities or resources were not accessed by your students. If you identify resources that were not accessed by many students, you may want to think about why this may have been the case. For example, was it easily discoverable, was it obvious that you expected them to access it? Also, you may need to reconsider whether it is a resource you will use next time.

“Activity report” by Chris Froissard ©2016 Macquarie University
“Activity report” by Chris Froissard ©2016 Macquarie University

In ‘Activity report’ above, there were approximately 24 students in this unit. The report shows that there were 100 views of the unit guide, suggesting that students were frequently accessing it during the session. Also, the dialogue tool with 80 views suggests that students found this useful.

“Example of an Activity completion report for a unit” by Chris Froissard ©2016 Macquarie University
“Example of an Activity completion report for a unit” by Chris Froissard ©2016 Macquarie University

In ‘Example of an Activity completion report’ above, the unit with approximately 24 students, shows that in week 1 most if not all students were accessing readings and lectures notes. However, during some later weeks, there are as little as 8 views for the readings and lecture notes. Perhaps this may warrant further investigation. Can the drop-off in engagement in some of those later weeks be due to assessment submissions or something else? As this example illustrates, looking at reports might not provide answers, but it may help you to start investigating what may be happening . Asking questions and looking for possible answers may provide insights that could be used to improve future iterations of the unit.

Unit participation report

This report lets you drill down to the student level in relation to a specific resources or activities. For example, if you want to see which students have and have not accessed your lecture notes, you would use the unit participation report. This report also lets you specifically look at the activity of your students. Also, you can look at a particular time period (ie look back 1 week or 3 weeks).

“unit participation report” by Chris Froissard ©2016 Macquarie University
“unit participation report” by Chris Froissard ©2016 Macquarie University

Statistics

This report gives you a graphical representation of the weekly views of your iLearn site and posts to your forums. It shows the peaks and troughs of student activity during the session. This will either confirm or challenge your expectations of student engagement. For example, typically students demonstrate a lot of online activity in the first week or so of a unit, which then tapers off gradually to ever dwindling numbers by week 13, occasionally spiking around major assessment tasks. Do you see this pattern in your report? If not, why do you think this might be the case? Also, if you have designed forum activities in your unit, has this been successful as per the report, or are the numbers of posts lower than expected?

Here is an example of a statistics report. The first data point is 11 August (end of week 2) and the last data point is 8 September (end of week 6). We can clearly see that the peaks in student activity are on Tuesdays and the troughs are on Thursdays and Fridays. Is this what you would expect to happen? If not, why not? This might be worth investigating.

“Statistics report” by Chris Froissard ©2016 Macquarie University
“Statistics report” by Chris Froissard ©2016 Macquarie University

You could also use this report to see what the impact was on student engagement on your site following the introduction of a new task. For example, if you included formative quizzes, has this lead to increased participation? Of course, you would need to compare this report with a previous offering that did not include the new task. However, this may give you an indication that the changes you’ve made have had an expected or an unexpected change in student engagement.

Logs

If you want to get very detailed information about what each individual student is doing (live logs) or has done (logs) you can access data in tables from iLearn. These reports are rich in information but do require a number of steps to become meaningful. In relation to logs, you will need to download the data into Excel and may need to clean the data by removing extraneous data. Also, you may need to do some processing of the data to make sense of it. Finally, you will need to understand the data to interpret it. This type of analysis, whilst being potentially time-consuming, can allow you to ask questions of your unit, and use the data to answer them. You may want to access support from the Learning and Teaching team in your faculty to get the most out of this process.

Activity completion report

The Activity completion report is only relevant if you have switched on activity completion throughout your unit, and have selected completion tracking for the activities or resources you want to track. Essentially (when activity completion is activated) iLearn tracks when a student has completed an activity or accessed a resource that you have selected. This information is then presented in a matrix with student names in the first column, and the activities or resources you are tracking across the top of the first row. As a student completes the various activities a tick appears in the relevant cell. You can set completion tracking to either be manual (students need to tick when they have completed the activity) or automatic (upon students doing something like clicking on a link or getting a particular mark in a quiz). The former allows the students to tick off the activities and resources like a checklist. An example of what you might want to track could be the unit guide. Then you can see which students have not accessed the unit guide and may need encouragement to become familiar with it. To start using it, first go to Tools>Edit settings>Completion tracking and select Yes for Enable completion tracking. Next Turn editing on and go the activity or resource you want to track, and select Edit (the cog wheel icon). Then in the section called Activity completion make your selections.

“Activity completion report” by Chris Froissard ©2016 Macquarie University
“Activity completion report” by Chris Froissard ©2016 Macquarie University

Final comments

As you can see there is a whole host of reports and tools that you can use to help you understand what is happening in your unit. All these reports will help you make the invisible, visible, and hopefully, help you to make improvements to your unit that rely not only on feedback but also on data. Don’t forget that your friendly faculty learning designers are always available to answer questions about these reports and to work with you to interpret what they may mean.

Powered By DT Author Box

Written by Chris Froissard

Chris Froissard

Chris is a keen and passionate educational technologist. He has worked in many different educational sectors over the years. He recently completed a Masters of Education at Macquarie University. He is one of the Senior Learning Designers at the Faculty of Arts for Macquarie University. His aim is to bring the worlds of educational practitioners together with educational researchers to create effective and transformative teaching and learning experiences.

2 thoughts on “What’s happening under the hood (of my iLearn site)?”

  1. Thanks for the post Chris.

    It prompted me to take a break from marking and look under the hood of the first year law unit I convene.

    Here’s a quick highlight of the views:
    Announcements – 19,696
    Private Communication with staff tool – 4,987
    Assessment info – 20,042
    Discussion forum – 30,370

    That last one stands out for me – I deliver my unit in a flipped model to 460 students and its important to me that I answer students questions when they post them. There are a few super keen students who post questions often, but if I hadn’t “looked under the hood” I wouldn’t have realized how many other students must be following the conversations. Good to know.

    And some stats of the weekly views of unit material
    1) 1761
    3) 1309
    5) 828
    7) 661
    9) 662
    11) 508
    13) 752

  2. Hi Lise, yes that stat for discussion forum does stand out. It would be interesting to look at posts to these forums to get a handle on how many students are actively contributing. Some forums would result in more posts, and if so, why might that be the case? Is it because they are in the first few weeks, or does the topic or question particularly inspire students to comment? There are always many interesting questions.

Leave a Reply

Your email address will not be published. Required fields are marked *