Background
Analytics involves collecting data about individual users and their practices to make inferences about users' behavior. As an inferential technology, analytics can help examine practices that benefit or work against an institution's or individual’s service or academic goal. Many companies use data collection methods to personalize online advertisements to users’ interests. Universities also use data collection to learn about an individual’s background or academic engagement. Library analytics is a relatively new practice that has been explored within the past decade. The 2010 publication, “The Value of Academic Libraries: A Comprehensive Research Review and Report,” from The Association of College and Research Libraries, kickstarted many libraries into learning about the possible values of using library analytics as a method to assess and to demonstrate the impact of libraries with evidence.
The University of Michigan Library wanted to experiment with a library analytics approach by using student data to understand how U-M students use the library’s resources, and ways the library could improve certain services, etc. For example, the library could examine the connection between library use patterns among students and correlation to student GPAs. Another example of library analytics could be using book checkout patterns to generate related recommendations for future checkouts. Before implementing these practices, it is always wise to seek out the opinions of the target audience/user base. Enter: the Library Engagement Fellows Program.
Our Process & Results
The Library Engagement Fellows Program pairs undergraduate or graduate students with a library staff member on a year-long project. In 2020-2021 the Library Analytics team consisted of Craig Smith, Assessment Specialist, and two students, Zoe Garden and Delaney Jorgensen.
Because student opinions on learning analytics at university libraries are a relatively new area of study, our research team decided to begin the data collection process by conducting a series of interviews with individual undergraduate students. The decision to start with interviews instead of a more extensive survey was beneficial for discovering a more broad range of student opinions. By collecting data first from a small sample and using an interview format that allowed for open-ended questioning, our team understood what questions might be impactful to ask of a larger sample later. In addition, the interviews themselves were an opportunity to refine questions and evaluate which topics within the broader learning analytics framework were most meaningful to undergraduate students. Eleven undergraduate students participated in these interviews.
After concluding this round of interviews, we presented our findings to the U-M Library’s Analytics Task Force and other Engagement Fellows and their supervisors. Despite conducting only eleven interviews, we observed a striking variety of viewpoints on library analytics and library privacy among students. Therefore, for initial reporting, we grouped participants into four “personas,” or profiles of a particular viewpoint that were useful for typifying student opinion while preserving participants’ anonymity. Here we summarize our findings through the signature “phrase” of each persona created to indicate the range of opinions that students held. These four invented phrases, based on real opinions expressed in interviews, display all levels of concern regarding the privacy challenges of using an analytics approach to data analysis.
-
“I care deeply about my data privacy, including the data I create as a student at the university.”
-
“I wouldn’t personally be affected, but I see why other students might care about data being collected on sensitive topics.”
-
“Everyone in our generation expects data collection.”
-
“Analytics could definitely help my academic success, and it could make researching easier and faster.”
Personas are a useful type of analysis because they encourage us, as researchers, to remember that there are people behind these responses and that library analytics affect students each time they interact with library systems. However, personas can also equalize opinions when some groups may, in actuality, represent only a small minority of responses.
For further analysis, we then coded participants’ responses to each question into categories to help us visualize student opinion through charts and graphs.
Early in the interview, we showed participants a website summarizing the types of data that the U-M Library collects on patrons. Students’ initial reactions to learning what types of data the library collects on them are shown in the chart below.
“What are your reactions to hearing about the types of data the library collects on you?”
-
Not surprised / expected: 55%
-
Surprised at amount of data collected: 27%
-
Some of it feels unnecessary: 27%
-
Not concerned / fine with it: 9%
-
Some and/or all of it feels invasive: 9%
Overall, a majority of the students interviewed were not surprised by the types of data that the library collects. However, significant percentages of participants found some and/or all of the data collected unnecessary or were surprised by the library’s amount of data on student users.
We asked another question about students’ level of trust in the library to responsibly handle the data they collect. As indicated in the chart below, a higher percentage of interview participants trusted the library to be responsible with student data, and relatively low numbers of participants expressly distrusted the U-M Library with their data.
“We’re interested in your level of trust in the library, with regard to responsible data management. For example, some types of data mismanagement include data leaks, data selling, and the sharing of identifiable information. Do you have concerns about any of these, with regard to Library data?”
-
No concerns at all: 64%
-
Trusts library to be responsible with data: 55%
-
Resigned to data collection: 18%
-
Concerned after learning new info: 18%
-
Mildly concerned: 9%
Despite some initial surprise at the amount of data that the library collects, we see the percentage of students that trust the library to handle that data responsibly remains high. Still, 18% of students were concerned about the library’s data collection and management practices after learning the extent of the information collected on them.
Next, we described library analytics and technology concepts to interviewees’ U-M Library data types. We then asked the question: “The library is considering using an analytics approach to data. How do you feel about this?”
“The library is considering using an analytics approach to data. How do you feel about this?”
-
In favor of analytics: 64%
-
No pressing concerns: 64%
-
Concerned about library analytics: 27%
-
Not personally affected by analytics: 18%
-
Uncertain/needs more information: 18%
-
Not sure if student data is applicable to analytics: 9%
-
Not personally affected, but sees how other students may be: 9%
The majority of participants were in favor of analytics and felt no pressing concerns about implementing an analytics approach to data collection. However, that leaves a fair percentage of students either unsure of their opinion on library analytics or with hesitations about analytics being used on their library data. The number of students concerned about analytics also increased by 9% from the previous question about trust in the library’s responsible data management, which indicates a heightened concern with library analytics than the basic act of library data collection.
What We Learned
The data collection and analysis from this round of interviews were beneficial to understand the landscape of student opinion on library analytics technology. While it seems that a majority of students may support the use of library analytics, there are still particular concerns that the U-M Library should address to ensure that all students feel comfortable sharing their library data with researchers. First, the library should assess which data types are helpful to collect and store about patrons, and which data may not be necessary. Next, the library should be more transparent about what library analytics is and how they potentially may affect the services we provide. Many of the concerns expressed in the interviews were from students who felt they did not know enough about library analytics to feel comfortable sharing their data or sharing their opinions on analytics topics. And finally, more efforts will be needed to understand the campus climate surrounding learning analytics, which we intend to address in the next stage of this project.
Next Steps
In the next phase of this project, we will expand our knowledge by talking to more students and instructors in a focus group setting. With results from a focus group method, we will be able to prepare a survey that will expand the reach of our research to graduate students and faculty, which will have the positive effect of incorporating even more campus populations’ opinions on library analytics into our results. Gathering varied perspectives will aid us in accumulating a consensus on if the University of Michigan Library should implement library analytic practices. These efforts will inform the work of the Library Analytics Task Force going forward, as it articulates strategies for campus learning analytic engagement.
About the authors:
Zoe Garden graduated from the College of Literature, Science & The Arts with a Bachelor of Arts in Communication and Media in Winter 2021. As a postgraduate, Zoe works full-time as a Marketing Manager for a law firm in Virginia.
Delaney Jorgensen is a senior studying French and History and will graduate with a Bachelor of Arts in April 2022. Delaney has worked within the U-M Library’s Student Engagement Program for three years as a Student Engagement Ambassador and an Engagement Fellow. After graduating, she intends to either attend graduate school for a Master’s degree in library science or teach English abroad in a francophone country.