This post was also authored by Nabeela Jaffer with contributions by other members of the MHydra prototype team.
The open source software Hydra is, by its name and nature, modular and complex. It’s a growing framework for building repository and access solutions, with an active, engaged, library-focused community. It’s thoroughly open source, with all the joys and pains that come with a large, distributed project. Using this technology gives the University of Michigan the opportunity to participate in the development of an increasingly-adopted suite of tools with the flexibility to accommodate a host of needs and engage in the spirit and philosophy of open source software development. With open source, we must concern ourselves not just with our own institution’s needs and priorities, but those of a broader community. It’s the ultimate in collaborative efforts, which makes it both exciting and challenging.
The University of Michigan Library has recently become a Duraspace Hydra partner but has been assessing Hydra for some time - considering how to implement this technology more broadly, how to engage with the broader Hydra community, and what level of customization is needed to make Hydra meet our particular needs (and in the process re-examine UM needs in light of Hydra’s capabilities). This process has required much thought and discussion about where Hydra fits into our planning and infrastructure - what it can replace, what it can enhance, where it can integrate. This is certainly an ongoing discussion here at Michigan; keep an eye out for updates on this blog and other UM blogs!
The MHydra idea came into existence in early September, when Digital Library Production Service (DLPS) team members were working with the Hydra community to develop supporting Hydra gems (a "gem" being a Ruby package containing an application or library) for Sufia 7, a Hydra gem which will be the base for our research data repository (another Hydra effort at the University of Michigan). Hydra development seemed to be challenging and complex. Although the Hydra community is very welcoming and helpful, the Hydra code is quite complicated to follow for a newbie. Working on different Hydra gems, and knowing the basic architecture of the Hydra, we thought we could gain a deeper understanding of Hydra technology by developing our own Hydra app. For this reason, we came up with the idea of working locally on a Hydra prototype - a Hydra head that could utilize the promising Hydra technology.
Our prototype work started with the vision of “creating a Hydra head with a Fedora-based digital repository”. We planned on using existing Hydra gems to create a repository and using a sampling of data from Deep Blue, our institutional repository (which uses DSpace as their repository). We used Agile methodology, and in particular Scrum, for sprint planning, iteration reviews and retrospectives. We planned on creating and contributing to an ongoing technical “lessons learned” document about problems, frustrations, solutions, and decisions the team comes across as we learn about the technology.
Our initial team members from DLPS were:
- Jose Blanco (Applications Programmer-Analyst Senior) has been working on DSpace for a long time. He helped us understand the functionality of DSpace that we wanted to duplicate in MHydra. Jose and Kayla worked closely together as dual Product Owners to write user stories & clarify requirements.
- Roger Espinosa (Applications Programmer-Analyst Senior) played an important role as a developer and a guide. He has been exploring mapping image and text data models to the Hydra platform. We were very lucky to have Roger’s expertise in our group.
- Nabeela Jaffer (Applications Programmer) acted as Scrum Master, helping to facilitate Scrum meetings. She also brought extensive programming experience to the group.
- Gordon Leacock (Applications Programmer) is new to Hydra development, but has extensive programming experience. He helped us use tools like RubyMine to debug the code.
- Kayla Ondracek (Project Manager) also acted as Product Owner, helping to create user stories and communicate with stakeholders.
Staff members from Michigan Publishing (another department at UM Library) expressed interest in our work early on, particularly in areas where our efforts could overlap with theirs. They joined our team to help discuss and develop our features:
- Jonathan McGlone (Front End Developer and UI Designer) helped work on item metadata display and contributed significantly to the discussion.
- Jeremy Morse (Directory of Publishing Technology) joined several of our meetings to stay apprised of our activities and discuss overlap in features between Deep Blue and Michigan Publishing’s needs.
It was a big plus for us to have a cross-functional team with expertise from Publishing and Deep Blue.
During our initial sprints, we focused on developing major features such as permissions and roles, customizing metadata for collections, making content searchable, and working on faceted search. Each sprint was two weeks long. We followed the Scrum standards of sprint planning, daily standups, review and retrospectives. We also held occasional product backlog refinement meetings as needed. During sprint planning meetings, we played ‘planning poker’ to assign story points to our user stories, which helped us measure the ‘velocity’ of our team per sprint. The daily standups were held by Google Hangouts - we met every day for fifteen minutes to talk about what was accomplished and what needed to be done. At the end of the two week sprints, we held sprint review meetings to go over the accomplishments. We also held a sprint retrospective to see how we could improve our productivity in the next sprint. We enjoyed applying the concept of ‘continuous learning and improvement’. The whole process helped us emerge as an empowered team.
The tools we used to support the project work and record information were mainly Waffle board, Github and Github’s Wiki. Waffle is a good, easy-to-setup workflow management system, and is used by Hydra community. Our MHydra source code is on Github; the Wiki was updated by all team members to document all the necessary information. We built the application in our internal development server environment. Our Core Services department was instrumental in resolving issues with the server and making the server environment stable for our Hydra development.
After two sprints, we held a one week ‘Knowledge-Sharing-Summit’ to share our knowledge with each other. Team members, as well as others in the organization involved in Hydra development, met daily to crawl through the Hydra gems code. It was a good learning experience for the whole team. The experience was so useful that we, as a team, decided to approach the next sprint more collaboratively by continuing daily code crawls and discussions. We met every day for an hour to go over a feature and coded it as a group. We then worked individually following up on our collaborative learnings. While this does not exactly follow Scrum methodology, we learned that pure Scrum may not work best for us.
The MHydra prototype work helped the whole team become more confident and comfortable in diving into Hydra world. Our take-aways from MHydra include:
- Hydra development might be complex, but when we work as a group, we can do it! - Nabeela
- It is fun and rewarding to read the code as a group, and we will continue our Knowledge-Sharing activity once a week - Nabeela
- “Leveling up” training and activities have been important for us - Gordon
- Group work has improved our team’s focus and productivity - Gordon
- Developing user stories has improved our understanding of user and stakeholder’s needs - Gordon
- Working towards a common goal, and sharing ideas in a welcoming environment has allowed all of us to make progress - Jose
- Refinement sessions with the team are important for discussing and raising questions for user stories - the programmer perspective is vital - Kayla
- Documentation is helpful for tracking decisions made and for helping team members participate in the project - particularly those members with less technical expertise! - Kayla
- Good start in teasing out how the Hydra stack will let us move beyond our current separate repositories - Roger
Our prototyping work is coming to a close along with the end of the calendar year. We feel that it is not the end of MHydra, but the start of the work towards more substantial Hydra head development and production solutions. What the team learned through this process will have a huge impact on approaching, planning, and implementing future Hydra projects.