The Consultation and Discovery Services team (formerly known as User Information & Discovery Services) in the Learning & Teaching division provides reference and research assistance to our community not only in-person but also virtually, via our Ask a Librarian email and instant messaging (IM) services.
Members from this team wanted to better understand the landscape of our virtual service domain. Anecdotally, we believed that email as a communication tool was dead as far as undergraduate students were concerned. We wanted to learn why someone would choose to email their questions to a library service, when we had other, perhaps more convenient methods available for requesting assistance (IM, texting, phone, in-person). We also wanted to confirm our understanding about why IM service users were motivated to instant message us. We knew from past experience that most users present a course- or research-related question in the IM environment, but we were hoping for more quantitative and qualitative data to support that assumption. We were also considering if we needed to change technical and staffing elements of the IM service, and wanted to solicit ideas from our user base. These goals determined our methodology and targeted population: from Fall semester 2016 through Winter semester 2017, virtual service providers employed a survey methodology to seek feedback from the email and IM services’ current users.
All email and IM service providers included an invitation to most users to complete a short Qualtrics survey, giving every user a chance to participate in the feedback process. Researchers were invited to participate in the surveys at the end of the Ask a Librarian transactions, either by appending an invitation with a survey link to the end of an email thread, or sending a link to the survey at the conclusion of an IM interaction. (Note that sometimes users ended the IM session before the link could be sent.)
The email service users were asked to provide feedback about their motivation for seeking help via email, as well as their satisfaction with the interaction. The IM service users were asked to provide feedback on their service experience, their motivation for using IM, and ideas for future service enhancements. We conducted an email user survey during Fall 2016 first, to pilot our questions and to test out our survey engagement processes. During Winter 2017, we administered both an email and IM service survey to our users. (See links to our full reports at the end of this blog post for complete details about each survey.)
“I’ve used the Ask a Librarian email and online chat services before (I do a lot of research-based projects), and they always respond so quickly with exactly what I needed, and that’s really helpful to me.”
Email Service Results
The number of respondents to the email feedback request overall was 149 (66 in the Fall semester survey and 83 in the Winter semester survey). By way of comparison during these periods, email service providers answered 2,400 email queries (1,181 in Fall semester and 1,219 in Winter semester). While the response rate was not as high as we would have liked, we felt that the feedback collected was still valid and potentially helpful.
Of those individuals who did provide feedback, the completion rate for the feedback form was 87% in the Fall semester and 95% in the Winter semester. The majority of respondents in each of the two surveys were UM students (undergraduate and graduate; they made up 57.4% of users in the Fall and 41.5% in Winter). In each semester, the next largest group of respondents (almost one-third) were guests (non-UM affiliates). The remaining respondents were UM faculty, UM staff, and affiliates of UM-Flint or UM-Dearborn campuses.
Overall, respondents were extremely satisfied with the service they received via email; 92.3% in the Fall semester and 90.4% in the Winter semester said they “completely” received the information or help needed.
The leading reasons (about one-third of total responses) given for seeking help via our email service during both terms were (not surprisingly):
- help for academic, project, or grant research, and
- help for a course-related assignment or project, thesis, dissertation
The top need expressed within the realm of academic or course-related help was across both semesters was "trouble accessing online library resource(s)," which accounted for roughly half of queries in each term (52.5% in Fall and 48.8% in Winter). The second most common need was "help finding a specific item," which was roughly one-third of queries each term.
And, finally, the top reasons for using the email service over other communication methods were related to "reporting a linking problem" and a preference for, or the convenience of, email.
“I use this service frequently as a doctoral student, and consider it vital. It is efficient and allows me to allocate my own time more efficiently. I LOVE THIS SERVICE.”
Instant Messaging Service Results
The results of the IM service survey were a little different, in that 456 users responded to the feedback invitation during Winter semester. IM service providers answered 1,662 queries during the eight-week period, which translates to just over one-quarter (27.4%) of IM users during the collection period contributed as feedback respondents. Of those submitted feedback forms, 420 were finished, for a completion rate of 92%. Students were overwhelmingly the largest respondent group (62%); 420 respondents were affiliated with the University of Michigan.
Like the email service feedback results, a majority of respondents (348 or 80.2%) indicated that they completely received the help or information that they needed. Only 47 respondents (or 10.6%) indicated that they received some help, and 26 respondents (6.0%) indicated that they did not get the information they needed but that a librarian offered to follow up personally or referred them to a different service or individual. In general, most respondents cited convenience or preference for instant messaging and texting when seeking library assistance. Almost 20% cited the need for immediate help as the reason for using instant messaging for their communication, and almost 15% chose IM because they had successfully used our IM service before.
The one question we asked to solicit ideas for service enhancements concerned us a bit (“What technological features or additional services might improve your next experience with the Ask a Librarian instant messaging and texting service?”). As an open-ended question, we were not certain if respondents would take the time to think about this question let alone provide illuminating suggestions or comments.
In response to this particular question, 165 users gave a wide variety of suggestions, and 101 stated that there were no additional features they would recommend. Additionally, many respondents included comments about how much they appreciate the IM service. The overall themes found in the responses for this question include a larger dialog box or widget, larger font within the dialog box/widget, a method for the dialog box/widget to pop out of the browser window (which already existed), and the ability to easily share computer screens between user and service provider. Generally, the comments submitted by respondents centered on praise for the service, the immediacy and convenience of responses, and the individualized assistance.
What Did We Learn?
Granted that our response rates for each of these three surveys were nowhere near statistically significant levels, we nevertheless generally confirmed some of our assumptions or learned some new things that prompted us to reflect and react. For example, we confirmed:
- Undergraduate students use IM mostly for their research needs, as compared to email
- More guests pose their questions via email, as compared to IM
- Our virtual reference services were providing highly satisfactory service to users
- Most students use the email service for reporting online access issues (which is means that we have made the reporting process easily discoverable and responsive to their needs?)
We learned that:
- An increase in the size of the IM dialog box/widget was necessary to support great communication
- Embedded IM functionality and/or features were not obvious to our users and that improvements as well as user education were needed
We also learned that we can do short-term, non-rigorous surveys to solicit feedback and data that can improve or impact our service delivery in positive ways. (It should be noted that these three surveys provided a foundation to a larger investigation into the virtual reference services conducted by the L&T staff during 2016/2017.)
Our team discovered that easy access to online, customized assistance and instructional guidance wherever an individual may be in library applications, resources, web sites, tools, as well as in course sites and MOOCs, is still a valid service proposition, and will continue to be a valid service proposition. Because current and future students, as digital natives, expect technology they encounter in their daily lives to be present when pursuing their academic research and individual learning or scholarly research-related tasks, the Library and our team will continue to decide which technologies will complement, enhance or improve our services while connecting with our campus community and remote scholars. Based on these results we anticipate that no one communication method will suit all learners or researchers, but future user surveys will continue to provide us with insights and data that will shape our services.
Access to the full reports is available via the Library’s institutional repository, Deep Blue.
- Ask a Librarian Instant Messaging/Text Reference Service User Feedback Report (Winter 2017)
- Final Report: Ask a Librarian Email Reference Service User Feedback (Winter 2017)
- Interim Report: Ask a Librarian Email Reference Service User Feedback (Fall 2016)
If you have questions about our surveys or reports, please contact the authors Dave Carter [superman@umich.edu], Karen Reiman-Sendi [karsendi@umich.edu], or Harold Tuckett [htuckett@umich.edu].
(Submitted by Dave Carter, Karen Reiman-Sendi, and Harold Tuckett.)