Assessing the impact on the student experience of embedding information resources in the Guided Learner Journey at the University of Hertfordshire
The Guided Learner Journey
At the University of Hertfordshire, a series of consultative workshops with academic staff and students established the aims and principles for the student Guided Learner Journey (GLJ) back in 2014. The GLJ aims included providing an excellent student learning experience, meeting sound pedagogic principles and student expectations, taking advantage of technological developments and a full range of online services for a holistic student experience.
The GLJ also established key principles including a consistent delivery of learning activities and resources, students’ ability to comment on and supplement resources and learning activities and customisation and simpler mechanisms to provide links to journals, ebooks and other external resources.
The university procured a VLE to replace its own inhouse system, StudyNet. Canvas was chosen as the closest to helping us provide the framework for the Guided Learner Journey, and it was decided to implement Talis Aspire reading lists at the same time. A Talis Reading List template was designed to reflect the same structure as the Canvas module. The full reading list is embedded in the Canvas module and can also be linked at unit level using an LTI (Learning Tools Interoperability).
In 2016 we undertook a pilot of Talis for approximately 30 modules to help define workflows for book acquisitions and other processes. Following this pilot all first year undergraduate modules moved to the new learning environment in 2017-18 (approximately 500 modules), with the rollout to all other modules ready for the academic year 2018-19. Prior to this, reading was either in the Online Library section of StudyNet, in module guides or powerpoint slides.
Background to research project
Other institutions have demonstrated improvement in students’ use of resources with the Canvas platform and the Talis Resource List Management System, for example Harper and James (2017) at Birmingham.
At Loughborough University the importance of developing a reading strategy was highlighted (Brewerton, 2014), including giving clear guidance to students and the importance of keeping lists up to date. Stokes and Martin (2008) established the need for discussion around reading lists, and greater understanding of expectations from both staff and students.
At Hertfordshire, reading principles were drawn up and were discussed with academic Schools. These provide general principles for staff in using reading lists and set draft ratios of the number of copies of books depending on the importance level allocated.
This paper reports on a project which received funding from the university’s Learning and Teaching Innovation Centre. Working alongside the project to deliver the Guided Learner Journey, we aimed to use a scholarly approach to validate both the GLJ principles and the reading principles, and to assess their impact on the staff and student experience. We measured the impact of using Canvas on students’ learning experience and staff perceptions of student engagement by focusing on a sample of first year undergraduate Semester A modules
The methodology used comprised both quantitative research to review levels and patterns of engagement and qualitative research to understand the student experience and the learning and teaching impact of engaging with embedded reading.
Method
We selected 7 first year undergraduate modules across a variety of disciplines where embedded reading was used for a variety of purposes:
Sports Science: students were asked to watch videos before practical sessions
Engineering: students were asked to read book chapters and manuals before workshops
Business: students were asked to read articles and book chapters each week, to be assessed by exam
Child Nursing: students were asked to read prior to a lecture (flipped classroom approach)
Law, Creative Arts and Humanities had more traditional lists broken down into weekly topics. For each module we set up a module profile to note the number of students, and identify key milestones related to significant reading activity such as assessment dates.
Several of the lists had been set up during the reading list pilot. A new level 5 Law module in Canvas was included so students could compare this with their previous experience of the StudyNet module. The length of the lists varied from 14 items (Business) to much longer lists for Humanities (162 items) and Creative Arts (304 items).
The reading lists were all structured by weekly topics and annotated with guidance from academic staff such as specific chapter or pages from the core textbook and the purpose of reading specific items. We recorded a 5 minute video to show students the functionality of the lists
Quantitative data
Early in the project we found that the Talis analytics via the Talis Dashboard did not work when the lists were embedded in the Canvas modules. As an alternative, colleagues in our department ran API scripts every week to show student behaviour by recording Canvas clicks to the Reading units across time. Other colleagues obtained usage data to show use of print and ebooks and digitisations for each reading list. We were not able to record use of journal articles or websites. The harvested data was represented by our data analyst using Tableau, a data visualisation tool.
Although we are still analysing the data, a clear picture of student behaviour is emerging. We can see the number and timing of clicks on Reading items in Canvas, for example in Fig 1 below where Engineering students were asked to read in preparation for laboratory work. This is useful in terms of staff seeing the benefits of designing activities that encourage their students to read.
We can also see use of e-books and print books for each module, along with digitisation requests. The very long reading lists had some books that were never borrowed. Further work is needed to look at the impact on use of books where tutors have added reading labels such as Essential or Recommended, and guidance for students.
Fig 1: Data showing clicks on weekly Reading links in an Engineering module
In an overview, Chad and Anderson (2017) predict the role that analytics, including data from reading lists, will play when combined with data on student performance and retention and this is something that we would like to pursue in future.
Qualitative data
We sent surveys using Microsoft Forms to the 7 module leaders and all the students. Of the approximately 700 students we obtained feedback from 83, over 11%. We then ran focus groups for five of these students in a newly designed student study room within one of our Learning Resources Centres, using user experience (UX) techniques, as suggested by Priestner and Borg (2016). The techniques used were observation, annotation of lists, post-it notes answering questions on our glass write wall, and love and break up letters. These provided us with rich feedback in a varied and fun environment and allowed us to engage with the students in an active way.
Fig 2: Focus groups using UX techniques
Main findings
The main themes that emerged were consistent with the findings from previous research and our own checklist of good practice criteria which listed the importance of a structured list, whether the list had been sent for review, use of importance labels and tutor guidance.
Back in 2003, Thompson, Mahon and Thomas (2003, p.60) established that students valued tutors’ annotations of lists and found evidence of use of readings lists in citations in students’ work.
(Siddall, 2016) recommends using institutionally agreed labels and the use of reading lists broken down into sections to aid navigation. An evaluation at Huddersfield (McGuinn, Stone, Sharman, & Davison, 2017) provided feedback showing that students appreciated lists to aid understanding, as a guide to further reading, for ease of use, and that they appreciated advice from tutors, weekly reading and labels, and the need for lists to be an appropriate length and up to date.
The findings from our project can be grouped into student behaviour and preferences, the learning and teaching impact and staff views.
Student behaviour and preferences
In our research, we found evidence that students liked having advance notice of the module reading. This reinforces the results of the university’s induction survey results from 2017. When asked about book purchase intentions, one student said “Definitely! I wanted to buy all books prior to starting the year while I had the money”, whilst another reported that “The prices for books are usually high and the majority of the books can be found in the library.”
Weekly reading was unanimously popular, with 90% of replies finding this helpful. Comments include: “Having it weekly helps me schedule my time, can’t imagine trying to do it without” and “weeks were better. Long list would have been a disaster”. When asked whether students preferred to consult the whole list or the weekly readings, one student said “I used the reading list to buy the books at the start of the year, but then accessed week by week to see the chapters needed for each week.”
The focus group students appreciated guidance from their tutors: “Yes, helps me identify the hard concepts” and liked weekly reminders about the reading.
We were interested in the work of Piscioneri and Hlavac (2013) who looked at shorter, quality annotated lists for Arts and Humanities students. In our research, some lists were seen to be too short “perhaps a few more books so if you don’t like one you have the option to choose another” whilst others complained about the very long lists, for example in this ‘break-up letter’: “You’re too long! I enjoy the reading and want to get ahead but so much extra reading is intimidating.” “There was far too much to read every week on top of everything else you have to read in other modules.” Given the length of the Humanities and Creative Arts lists, this is probably not surprising. In the survey 67% of respondents said the length of the list was about right.
88% of students said the level of the reading was about right, while 11% found the reading too advanced: “Some of the readings were easy to understand while others were more complicated and required a lot of time.”
Fig 3: Student survey results example
Learning and teaching impact
A member of staff commented that “Having the resources embedded allowed us to build the session around the pre-course reading.” Students found it helpful to have the list to prepare for workshops and assignments in this flipped classroom model: “enables easy access to specific and required content for lab sessions and assignments.”
98% of students in the survey agreed that the reading aided their understanding: ”The reading helped me to consolidate my ideas, and to understand more.” 84% of respondents found the reading helped with their revision and assignments: ”The reading was very important for this module because the entire exam was based on the readings.”
63% of respondents said that the reading also acted as a springboard for further reading with comments such as: “Found very interesting and did further research” and “Some of the reading I found interesting and lead to further reading” and “I found that by reading it gave me a base line of understanding helping me in practical sessions and also other modules.” Further reading was found in the university’s Online Library, on Google Scholar, YouTube and the Internet.
Staff views on embedding reading and student engagement
Staff were positive about the benefits of having a structured template corresponding to Canvas units, averaging a 5 out of 5 star rating. Ease of use and embedding the Talis links into Canvas were rated with 4 stars, with one member of staff saying that he did not find this intuitive.
Regarding set up time, one tutor said “It works fine once the time has been taken to set it up,” another said it only took 2 hours to set up whilst those with longer lists complained about the length of time involved.
The staff were also enthusiastic about embedding weekly reading: “The guided learner journey helps to point the students in the right direction to get through the course.” Another member of staff commented: “Overall Talis is good because it allows students to have a structured approach to their reading, and it is easier for them to find the relevant material and access it remotely week-by-week.”
All the staff felt that the embedded reading had a positive impact on student engagement. Comments included: “Student practical grades have increased this year”, “The more integrated function is more uniform and looks more professional” and “I think students were more likely to do the reading.”
When asked what ideas they have for encouraging reading, there were some valuable suggestions, for example: “I ran an exercise to get them to think about what they found interesting in the article /chapter - rather than what they thought was important. We snowballed this and voted on what the most interesting things were” Another member of staff commented: “Student are very assessment driven - so potentially telling them it will benefit their coursework or final exams.”
Fig 4: Staff survey results example
Conclusions
In addition to assessing the impact of the student experience of embedded reading, the project aimed to disseminate good practice. Some good ideas about promoting use of the lists to staff are advocated by Atkinson et al., (2011) whilst the importance of using the Review function was highlighted by Cameron, C. and Siddall (2015).
It is clear that academic staff value a simple system that is easy to use so clear training is needed. Library staff need to ensure that academic staff see the value of reviewing their lists. The findings from this project can be used to demonstrate the benefits to the student experience of embedded reading. As part of the project, we made a short video of staff discussing the benefits.
We have drawn up the following Guidelines for good practice based on the evidence from the research:
Early access to reading lists may better prepare students before they start their university course.
Students value structured lists that reflect the structure of their modules and can aid students’ organisation and time management skills.
Embedded reading can aid students’ preparation, understanding, and revision.
Students appreciate their tutors’ guidance, in the form of reading importance and annotations.
Students are more likely to use the lists if an activity is centred round the reading and not overlong.
Students often don’t buy the recommended core textbook due to costs but rely on ebook versions if available.
In addition, we will publicise the top tips to students from their fellow students:
Reading is important and a valuable part of studying, “1 hour reading will help in the long run”
Use the reading list to prepare for lecture or seminar and afterwards to reinforce learning and extend understanding
Practise reading academic texts before starting your course
REFERENCES
Atkinson, S., Conway, N., Taylorson, B., & Smith, C. (2011). Reading lists – pitching it right! | SCONUL. Retrieved from http://www.sconul.ac.uk/publication/reading-lists-–-pitching-it-right
Brewerton, G. (2014). Implications of Student and Lecturer Qualitative Views on Reading Lists: A Case Study at Loughborough University, UK. New Review of Academic Librarianship. http://doi.org/10.1080/13614533.2013.864688
Cameron, C. and Siddall, G. (2015). Academic staff perceptions and use of reading lists for book ordering | SCONUL. Retrieved April 7, 2018, from https://www.sconul.ac.uk/publication/academic-staff-perceptions-and-use-of-reading-lists-for-book-ordering
Chad, K., & Anderson, H. (2017). The new role of the library in teaching and learning outcomes Briefing paper. http://doi.org/10.13140/RG.2.2.14688.89606
Harper, P. and James, A. (2017). issue 68 | SCONUL. Retrieved April 7, 2018, from https://www.sconul.ac.uk/tags/issue-68?page=1
McGuinn, K., Stone, G., Sharman, A., & Davison, E. (2017). Student reading lists: evaluating the student experience at the University of Huddersfield. The Electronic Library. http://doi.org/10.1108/EL-12-2015-0252
Piscioneri, M., & Hlavac, J. (2013). The minimalist reading model: Rethinking reading lists in arts and education subjects. Arts and Humanities in Higher Education, 12(4), 424–445. http://doi.org/10.1177/1474022212459035
Priestner, A., & Borg, M. (2016). User Experience in Libraries: Applying Ethnography and Human-Centred Design. http://doi.org/http://dx.doi.org/10.3998/weave.12535642.0001.505
Siddall, G. (2016). University academics’ perceptions of reading list labels. New Library World. http://doi.org/10.1108/NLW-02-2016-0012
Stokes, P., & Martin, L. (2008). Reading lists: A study of tutor and student perceptions, expectations and realities. Studies in Higher Education, 33(2), 113–125. http://doi.org/10.1080/03075070801915874
Thompson, L., Mahon, C., & Thomas, L. (2003). Reading lists -How do you eat yours? Background and rationale. Retrieved from www.wlv.ac.uk/celt