Background
March 2020 seems like a long time ago. Since that day when our campuses shut down, our library communities have learned quite a bit about society, higher education, and individual learning needs.
During the summer of 2020, I was part of an Instruction Assessment Task Force, which was charged to design and administer the two assessments during the 2020/2021 academic year. Bottomline, colleagues wanted to respond knowledgeably to our users’ “new normal” away from campus during a global pandemic, and we needed data to do that. One of the instruction-related assessments was to discover more about student learning during research consultations related to 300- and 400- level courses (upper undergraduate) on the Ann Arbor campus.
The assessment team’s research questions included:
- What did students hope to learn during winter 2021 research consultations with librarians?
- Did students feel they learned what they needed?
- Were they able to apply that learning to their coursework?
- Could they apply it to other academic situations?
- What did consultation service providers/librarians hope to teach in this context?
Why? And Why Then?
A general analysis of curriculum-related instruction data showed that historically, fewer library information literacy workshops or in-person library research instruction sessions were taught for undergraduate students at the 300- and 400- level. Subject liaisons reported that most library instruction for students at this level happened in one-on-one research consultations. When the campus moved to remote learning in a crisis situation, we assumed that requests for research consultations would increase. We wanted to quickly respond to these new conditions, by gathering new and potentially rich data that could inform our immediate and future service directions. At the time, we didn’t know enough to frame our research questions or our methodology to consider mental health needs, living situations, and unique academic goals from a remote location.
Methodologies
Given the remote learning and work environment, and unpredictable availability of individuals during the COVID-19 pandemic, the assessment team landed on survey methodology as the best and efficient path to data collection. We mapped out various steps in our process to engage survey participants (library employees and student researchers).
After meeting with four colleagues to learn more about the nature of their research consultations with a key college on campus, the assessment team formulated survey questions aimed at all library research consultation service providers. In December 2020, we invited all library employees to answer a three-question survey, and asked for volunteers to participate in another step of our process during the next semester if they typically provided research consultations to undergraduate students in 300- or 400- level courses. Through this effort, the assessment team learned that most students arranged research consultations via personal email and via our Springshare LibCal appointment scheduling tool. Other methods to arrange consultations included in-person, virtually (likely Zoom or Google Meet), phone, and Google calendar. This information helped us formulate our process for the next step to understand our research consultation service environment.
During Winter semester 2021, we invited students to complete a short survey after every research consultation they had with our volunteer library experts. Most of these post-consultation survey invitations were automatically sent via email, using features in our scheduling tool. The volunteer library experts also completed short surveys about their experience after each research consultation with students in 300- or 400-level courses.
Summary of Findings
“[Consultation service provider] was amazing! [They were] incredibly approachable and helpful in helping me get a better sense of how I wanted to narrow down my research question, how I can have a more strategic and targeted approach to finding sources, and how I can use publicly available datasets to help me answer my research question.”
During the Winter 2021 semester 114 consultations were booked through LibCal, and 37 were identified as supporting students in 300- or 400- level courses. Our volunteer consultation service providers submitted 17 post-consultation surveys, and 13 students in 300- or 400-level courses submitted responses. (Thirty-five other students submitted surveys.)
These students reported learning about specific skills and concepts that mirrored those reported by library research consultation providers, such as learning how to use specific library tools or software, understanding or navigating the library, collections, spaces, etc., developing a topic and/or identifying keywords and search strategies, and finding relevant sources.
All respondents strongly agreed or somewhat agreed that they understood why particular skills or concepts were taught or recommended during their research consultation. Most respondents had received formal library instruction prior to their consultation meeting, either through course-related information literacy presentation (in-person or Zoom), or through a self-paced Library Canvas module or library-provided videos. And half of the respondents had met with someone in the Library about their projects or research in previous semesters. It seems that a prior relationship with a library expert might be one basis for research consultations requested by undergraduates.
For details about this assessment effort, see the team’s Instruction Assessment Task Force Final Report.
Impacts on Assessment Practice
Besides discovering more about our students’ learning needs as represented in research consultations and more details about how colleagues met those learning needs, I also learned how valuable personalized help from library experts were (and continue to be) to our community. As I reflect on that period of pandemic living and working, specific constraints impacted my personal assessment practice, resulting in some lessons learned:
- COVID-19 Pandemic - It was an unprecedented year that brought anxiety and stress to our whole community. That anxiety was apparent in every interaction between colleagues, and assessment participants. By Fall semester 2020, the assessment team was very much aware of mental health/well being concerns, and that context was important when implementing processes, conducting surveys and analyzing data. A major lesson learned here: Clarifying the “why” of an assessment project within your given context and environment helps you understand the value (or lack thereof) of your efforts. Basically, I learned to “do no harm” and be humane to all campus populations. Another lesson that I remain conscious of is that student living situations can be very fluid, not very private, and fraught with complications that can impact student academic success. Being empathetic and patient with participants – and not making assumptions about individual student campus experiences – were really important.
- Funding freeze - A lack of incentives for participants probably impacted the level of participation. While students generally didn’t mind connecting with our services and assessment processes, we really needed incentives to not only encourage participation, but also to recognize individuals who committed to sharing something about their situation when their world was so chaotic. Gestures of appreciation are important in any circumstance.
- Employee capacity - This study relied on volunteer library employees, which resulted in a smaller pool of participants than we had hoped for. That said, those volunteers were keen to help. While the assessment team could have designed a different assessment plan without engaging our consultation service providers, we would have missed an important perspective in this service. Generous volunteers made the project more complete as well as “do-able,” given the constraints in our environment and organization. Expressing gratitude for those individuals’ efforts was key to maintaining trusting and collaborative relationships.
Gratitude, kindness, creating a sense of belonging and community, and valuing the all voices — these are the things I carry with me as a result of assessment efforts during a global pandemic.