Why usability testing should be a part of regular library activity

Guest post from Library Lab fellow and UX ninja, Anneli Friberg, Linköping University Library

The interest for user experience (UX) and usability in libraries has grown rapidly over the past years and now has become an essential tool for developing and assessing a library’s digital services and physical spaces. It is necessary, though, to recognize that UX incorporates much more than just usability. Norman and Nielsen (n.d.) summarize user experience as something that ‘encompasses all aspects of the end-user’s interaction with the company, its services, and its products’ and continues:

The first requirement for an exemplary user experience is to meet the exact needs of the customer, without fuss or bother. Next comes simplicity and elegance that produce products that are a joy to own, a joy to use. True user experience goes far beyond giving customers what they say they want, or providing checklist features. In order to achieve high-quality user experience in a company’s offerings there must be a seamless merging of the services of multiple disciplines, including engineering, marketing, graphical and industrial design, and interface design.

Furthermore, they state that it is important to separate the overall user experience from usability, since the latter ‘is a quality attribute of the UI [user interface], covering whether the system is easy to learn, efficient to use, pleasant, and so forth.’ (Norman and Nielsen, n.d.).

At Linköping University Library (LiUB) we are slowly moving towards a ‘culture of usability’ where users are being observed interacting with both physical and virtual spaces, the way Godfrey (2015) advocates, but this paper will only focus on the library’s online presence. The main objective with this paper is to argue for continuous usability testing, as a part of regular library activity.

Usability testing per se is nothing new within the library sector, but it is usually done in the process of launching a new or redesigned website/UI or implementing a new library system. Most often it has a distinct focus on web development, and is not so much used to develop other services or physical spaces. This is confirmed in numerous articles and UX-blog posts (e.g. Gasparini 2015; Godfrey 2015; Broadwater 2016; Dominguez, Hamill & Brillat 2015). Sometimes the tests are not conducted by library staff, but by external consultants. Our approach, however, is to use an in-house, continuous process which is applied not only to the library’s website structure, but also to other digital services such as the search box on the library start page and link resolver user interface and the link resolver icon in the discovery tool.

Rettig (2014) asks whether such a thing as ‘grassroots UX’ exists in libraries. She wonders if ‘the UX hopeful, [who] do not have the mandate or team or job title’, can find ‘ways to apply UX methods to smaller-scale, day-to-day work in the library?’ I am inclined to say that it is possible. A UX perspective can and should be integrated in any development project, big or small. The UX philosophy does not have to be initiated as a top-down initiative, and in a sense LiUB’s systematic way of doing usability testing started out as a grassroots initiative.

Context

Linköping University (LiU) is one of 16 Universities in Sweden. LiU has four campuses in three cities (Linköping, Norrköping and Stockholm) and has four faculties: Science & Engineering, Medicine & Health Science, Arts & Science and Educational Sciences. LiUB consists of four physical libraries, one on each campus, with approximately 90 staff members in total.

In order to make sure that LiUB contributes in a useful and valuable way to student learning and research, we have tried to find different ways to understand our users’ needs and behaviour. We use our insights to improve the digital library in order to provide a user-friendly and intuitive way for students and researchers at LiU to access the information they need for their studies and research.

The groundwork for the library’s systematic user involvement was done within a web strategy project in the spring of 2014. Throughout the project we had the opportunity to test different methods for collecting user data. During this time we also formed a usability team at the library. The team consists of five people (of which three are librarians), including myself, with different skills and roles such as system manager, computer programmer, webmaster, UX expert and cognitive scientist. Over the last 24 months, the usability team has gathered once a month to do testing. The advantage of having a permanent usability team is that the library does not have to mobilize a team whenever the need occurs. This approach is also advocated by Nichols, Bobal & McEvoy (2009):

A permanent usability team allows an organization to build expertise and tackle more usability projects than ad hoc teams. Having a usability team already in place makes it more likely that usability studies will be done on projects that may otherwise have been overlooked because of the “burden” of asking staff to be part of another project on top of their already busy schedule.

The LiU Library Experience

The web strategy project in 2014 established usability and user benefits as central to the continuous web development process. In order to accomplish a user-centered library website we decided to find a doable model for user-involvement. The book Rocket Surgery Made Easy: the Do-It-Yourself Guide to Finding and Fixing Usability Problems by Krug (2010) became our inspiration. The workflow for usability testing at LiUB is illustrated in Fig. 1.

fig-1

Fig 1.: The workflow for usability testing at LiUB

When we first started, we asked ourselves how many test participants were needed. According to Nielsen (2012) five users are enough when doing usability testing, because then ‘you almost get close to user testing’s maximum benefit-cost ratio.’ Krug (2010, p. 43) on the other hand claims that three users are good enough for ‘the do-it-yourselfer’, considering ‘you’re not interested in what it takes to uncover most of the problems; you only care about what it takes to uncover as many problems as you can fix.’

As we evidently belong in the category of ‘do-it-yourselfers’ we started with three test participants per session during the first year. The previous semester we decided to increase the number to four users per session, since we thought we had the capacity to expand. Although, after our last evaluation we decided to go back to only three users again, since it was difficult for me as facilitator, but also for the observers, to stay focused and perceptive with four users and to get enough time for summarizing and debriefing. Krug (2010, p. 43) made a list of arguments why three test participants are enough, and after trying with four, I am willing to agree. Some of Krug’s reasons are:

  • The first three users are very likely to encounter many of the most significant problems related to the tasks you’re testing.
  • Finding three participants is less work than finding more.
  • Testing with three users makes it possible to test and debrief in the same day.
  • When you test with more than three at a time, you often end up with more notes than anyone has time to process – many of them about things that are really “nits”. This can make it harder to see the most serious problems – the “can’t see the forest for the trees” effect.

For the tests we use randomly chosen employees and/or students as test participants. In my experience, engaging face to face is the most successful way to recruit users. For example, I usually recruit students I meet in the library. Regarding employees we always recruit research or teaching staff such as PhD students, lecturers, university teachers and professors. My experience is that most students and employees I ask are willing to help us as long as they can find the time for it. They all want to be part of a process that aims to improve the user experience.

When it comes to deciding what to test, we make a preliminary plan at the beginning of each semester. This plan sometimes changes during the semester. What we actually test depends on different projects in progress at the library. We never test systems or interfaces that we can’t alter or modify ourselves to some extent.

We conduct usability testing monthly during each semester, which gives us approximately eight test sessions per year. This enables an agile and iterative approach to assessing the users’ experiences of the digital library as well as helping in the development of our digital services.

On the test day, the usability team divides into two groups in two different locations: a test room (see Fig. 2) and an observation room (see Fig. 3). The facilitator and one observer goes to the test room, while the rest of the team goes to the observation room. Often the latter are accompanied by other observers and stakeholders; sometimes colleagues from other departments within the University such as the division for IT Services, sometimes external such as librarians from other universities.

fig-2-1
Fig 2.: Test room

fig-3
Fig 3.: Observation room

We combine different methods like observation, think-aloud protocol and capturing screen activity. By using different practices that complement each other, we avoid the uncertainty of using just one method. One of the benefits of triangulation of data is that we get a more complete picture of the usability issues that need to be addressed.

Each test person is given a specific assignment based on a common user scenario for the service to be tested. The test person attempts to complete the assignment while thinking aloud. If needed, the facilitator encourages the test participant to think aloud and describe what he/she is trying to do. At the same time, the team in the observation room records what the test person says and does. We use Camtasia to record screen activity, and we set up an Adobe Connect meeting to share screens between the test room and the observation room. Obviously we do not record anything without permission from the users. Before we begin the test session, the test participant signs a written consent.

After the test, the facilitator and observer from the test room join the rest of the usability team in the observation room and a debriefing session starts. We then collect and discuss the usability problems we have noticed and put them together in an aggregated list of feasible improvements. We also prioritize the things on the list.

After each test session the usability team starts to improve the things listed. Depending on what the problems are and what has to be done, we involve different colleagues outside the usability team. The recordings have proven valuable for the analyses and development in between the test sessions. They are an essential complement to the observers’ notes.

Another valuable complement is so called guerrilla testing, which we do sometimes in between the monthly test sessions. This type of testing is both agile and flexible. It is a ‘low cost method of user testing. The term “guerrilla” refers to its “out in the wild” style, in the fact that it can be conducted anywhere…’ (GOV.UK n.d.) When we perform guerrilla testing we approach people in the library and ask them to give quick feedback. This fits well with our thinking that some testing is better than no testing.

Outcomes

The improvements we have made as a result of what we have seen during our usability testing ranges from very small terminological changes to more structural changes on our website. One of the first things we tested was the information architecture for a new library website. For that, we used a tool called Treejack. We did one test session with students and one with employees. This enabled us to get valuable feedback on the site structure.

For several years we had a tabbed search box on the library start page (see Fig. 4). Last year we decided to renew the design, inspired by MIT Libraries. Before we launched the new search box (see Fig. 5) we made a prototype which we used to perform both regular usability testing and guerrilla testing. The feedback we got gave us useful input to the design process.

fig-4
Fig 4.: Old desgin of the library start page with a tabbed search box

fig-5
Fig 5.: The new search box

We have also tested different features and new services for the discovery tool, such as a new search service for e-publications. We tested this service twice – once with undergraduate students and once with PhD students. In addition to getting feedback on what adjustments to do, we also learned that undergraduate students have quite a different attitude to journals than PhD students have. We have seen this in other situations, for instance when doing interviews as part of the web strategy project in 2014, but seeing this again during usability testing confirmed our previous insights.

Things we have also tested and improved are terminology, holdings information and link resolver user interface. Sometimes we make changes and then we do a new round of testing, but more often we get indirect feedback on changes we have done while testing new things.

Conclusion

A vast understanding about our users is the foundation of any user-centered development. By combining qualitative and quantitative methods and applying a UX-perspective we are better equipped to meet our users’ changing needs and behaviour. It allows a more agile workflow. The trick is to keep it simple. We do not consider ourselves researchers. What we do are continuous modifications based on input we get from real users. Our motivation is to enhance users’ experiences of the library’s digital services.

Based on our experiences from the last 24 months we have found that systematic usability testing can and should be a part of the regular library activity and that it can encompass so much more than just the website structure. The key to success is the model itself, particularly when it is carried out monthly during the academic year. By involving real users continuously, we avoid getting stuck in our own internal assumptions of how users interact with the library’s digital services.

Additionally, usability testing is an excellent way to make our services more visible to users.

References

Broadwater, T 2016, Why am I doing this to our users? A case study about the wrong turns taken during a redesign project and the impact of design-by-committee on team morale, viewed 11 July 2016, < http://libux.co/why-am-i-doing-this-to-our-users/>

Dominguez, G, Hammill, SJ, Brillat, AI 2015, ‘Toward a usable academic library web site: a case study of tried and tested usability practices’, Journal of Web Librarianship, vol. 9, issue 2-3, pp. 99-120.

Gasparini, AA 2015, ‘A holistic approach to user experience in the context of an academic library interactive system’. Lecture Notes in Computer Science, vol. 9188, pp. 173-184.

Godfrey, K 2015, ‘Creating a culture of usability’, Weave: Journal of Library User Experience, vol. 1, issue 3, viewed 8 October 2015, <http://dx.doi.org/10.3998/weave.12535642.0001.301>

GOV.UK n.d., Guerrilla testing: getting input into products and services, viewed 27 September 2016, <https://www.gov.uk/service-manual/user-centred-design/user-research/guerrilla-testing.html>

Krug, S 2010, Rocket surgery made easy: the do-it-yourself guide to finding and fixing usability problems, New Riders Publishing, Berkeley, CA.

Nichols, J, Bobal, AM, McEvoy, S 2009, ‘Using a permanent usability team to advance user-centered design in libraries’, Electronic Journal of Academic and Special Librarianship, vol. 10, no. 2, viewed 13 July 2016, <http://southernlibrarianship.icaap.org/content/v10n02/nichols_j01.html>

Nielsen, J 2012, How many test users in a usability study?, viewed 8 July 2016, <https://www.nngroup.com/articles/how-many-test-users/>

Norman, D and Nielsen, J n.d., The definition of user experience, viewed 8 July 2016, <https://www.nngroup.com/articles/definition-user-experience/>

Rettig, M 2014, Grassroots UXD in the library: a review essay, Weave: Journal of Library User Experience, vol. 1, issue 1, viewed 1 April 2016, <http://dx.doi.org/10.3998/weave.12535642.0001.103>

Upcoming talk at #REBIUN2016: Towards Rubicon: Libraries that turn and face the strange 

Towards Rubicon: A Story About Libraries That Turn And Face The Strange. That’s the title of my keynote at the annual REBIUN conference at Palma de Mallorca, Spain, November 9th to 11th.

The title reaches for both history and Bowie – two things that I find great for explaining and framing stuff.

I was asked if I would do a talk on struggles and challenges for academic libraries but I find that a bit to gloomy a subject for my taste (I’m a LEGO playing library director after all) but also a theme in lack of some important elements and aspect in the discussion of the future of libraries. Where does challenges come from? Where does opportunities come from? They are mainly driven by changes in our surroundings and changes, if we like them or not, are here to stay. Not much in this world are static (except maybe the magic of David Bowie) and in order to develop and grow I strongly believe that libraries needs to embrace the change and face the strange. Not ignore it or fight it. So my talk will be about that.

Changes 

Changes often comes with uncertainty and I think many human beings and institutions even if they will admit it or not, really are fans of status quo (myself included on many days). You know that you got but you don’t know what you gonna get, right? But it’s an unhealthy position to take in order to navigate libraries wisely through changes and it often brings out The Let’s Save Libraries Ghost. The Let’s Save Libraries Ghost is not saying boo because of governmental cuts on library budgets (that’s a different kind of change) but it’s scares us to a place where we look at changes in technology, infrastructure, culture and behavior and thinking how do I save the library I know in all this. But that is wrong – leading a library of status quo through tides of changes is a bad trip and will in the end result in a library out of sync with it’s community and the needs it serve.

slide3
The Burning Platform methafor and attitude is an evention of The Let’s Save Libraries Ghost

Ask the right questions

In my talk I will address the following questions which I think is crucial for libraries to navigate and continuous create value in research and higher education.

  • What’s the purpose of higher education?
  • What changes effects higher education?
  • Which challenges and opportunities does those changes bring to higher education?
  • To which of these challenges and opportunities in higher education are the library the answer?

My talk looks at those questions and will pinpoint some ways to face challenges, opportunities and changes but most of all, I hope it will offer reflections and a direction on how the culture of libraries can deal with changes.

I’m looking forward to walk towards Rubicon with the REBIUN participants, hopefully cross it, and go back to our institutions ready to embrace the change and face the strange.

slide2
Hopefully the REBIUN 2016 participants will be all be dancing like Bowie after my talk

Some thoughts on The University Library anno 2035

You can not escape the responsibility of tomorrow by evading it today
– Abraham Lincoln

A few days ago a got a fun task from a co-worker at Copenhagen University Library; he is the editor of The Faculty Library of Humanities’ newsletter and they where doing a piece on the future of university libraries. He asked if I wanted to contribute with my thoughts and answer the question:

How does the University Library looks like in the year 2035?

Sure! What an excellent and totally mind twisting question. It was really fun and healthy little mind experiment to imagine the University Library in 2035. I think my picture of The University Library 19 years from now might be colored by an inherent optimism on behalf of libraries and the fact, that I wrote down my 2035 scenario in the co-drivers seat en route from Copenhagen to Skaelskoer with a cold Sierra Nevada Torpedo Extra IPA in my hand.

You find my University Library anno 2035 below: The space was limited, the context is Denmark but anyway I think the result is worth a share and I recommend the mind experiment for everyone who cares about the future of libraries (I do also recommend a good beer for this kind of work).

29814930814_30d42f1a96_o
Thinking big 

One University Library, four primary tasks

In 2035 the wave of mergers and centralization has left Denmark with three regional universities – one in Jutland, one on Fyn and one on Zealand. Together the three universities cover all relevant research disciplines and educations relevant to society. They are served by only one Danish University Library, simply called The University Library.

The University Library has four primary tasks:

  1. information supply
  2. support for information and data literacy
  3. consulting services in connection with the university’s research
  4. physical learning environments


Information supply
In 2035 the information resources used in higher education is 100% digital. The traditional commercial publishing system succumbed several years ago and has been replaced by 100% open access publishing. In addition to the supply of scientific literature The University Library curates datasets with open data relevant for research and education as well as other materials such as 3D scans of molecules and historical artifacts which can be searched, downloaded and elaborated. All materials is accessibly via a single search interface.

Support of information and data literacy
The amount of information is complex and actors in research and education use many different types of materials, sources and formats in their work. The University Library teaches and supervise academic staff and students at the university in information seeking, source criticism, reference management, data management, data handling and data visualization. The instructions is embedded in the curriculum and is carried out both physically and online.

Partner in research
The librarians of The University Library acts as consultants and partners in university research projects. They assist on systematic reviews, bibliometric analyzes, instructions for data handling and data management and several other aspects of the research process. Often the librarian embed in larger research projects over a period of time.

Where people meet
In 2035 the physical space is more important than ever. The digitization of society activities has not – as some predicted – changed the basic human need to meet physically for dialogue and cooperation in order to develop and learn about the world. The University Library has a number of physical devices attached to the academic environments of the three regional universities. The physical devices, simply called Libraries, functions primarily as hubs for projects, events and workshops related to research, education and learning. The decor of the Libraries are flexible and inspiring with furniture and technology to support creative processes and cooperation; Should one kick start a research project or pitch an idea for fellow students it happens at the Library.

That was my University Library anno 2035. Might look different next year. How does your University Library look in 2035?

Cheers

Christian

 

Share, inspire, connect: Library related Twitter hashtags

Twitter is an outstanding tool for gaining inspiration  with whatever turns you on and engaging with people of similar interests. For me Twitter has been a major boost for my professional life in terms of insights, inspiration, debates and network with the global library community.

I slowly started using Twitter in 2012 when I was on the TICER Summer School in Tillburg and it took me a while to crack the code and come to a place where I got the full potential of this great tool. A part of it was related to the size of the network; where do you even start to build your professional learning network on Twitter? Well, a good place to start is via hashtags that binds various topics and discussions together. So if you are library newbie to Twitter or just want to go exploring on library and library related hashtags here is a list (got additions to the list please write them in the commentary section):

Mobile Hashtag Horizontal Concept

 

Everyday library life

There is a number of hashtags that a great for sharing and get inspiration on what’s going on in everyday library land.

#librarylife

A quiet broad hashtag which is often used in tweets related to the daily experiences librarians do in there job around the world.

#libraryproblems or #librarianproblems

You often find broken staplers or jammed printers under this hashtag

#saturdaylibrarian

A hashtag to unite all the happiness and sorrow of librarians working the weekends

#librarianwardrobe

On of my fave hashtags that contains all the awesome socks, cardigans, glasses and cat sweaters out there – librarians got style yo!

Chats

Twitter chats are excellent for discussions on various subjects with a quit broad and diverse group of people, get inspiration and connect with others. Some chats is organized with questions on topic published in advance, moderators and suggested readings. Others are more casual organized.

#edtechchat

Chats on technology within education and instruction. The hashtag is both used in actual chats but also appears in Tweets disconnected to a chat.

#radlibchat

Monthly chat hosted by Radical Librarian Collective about an article or research paper relevant to the principles upon which Radical Librarians Collective operates: challenging, provoking, improving and developing the communications between like-minded radicals, to galvanise our collective solidarity against the marketisation of libraries.

#libchat

To my knowledge a hashtag started by Hack Library School but I’m not sure it’s currently function as an actually hashtag for live chats (but I might be mistaken) but it got some fair use on relevant content.

#UKlibchat

A monthly discussion group on library related issues and topics. The organization and participation of #UKlibchat is always great – read more here: https://uklibchat.wordpress.com/ 

#critlib

Critlib is short for “critical librarianship,” a movement of library workers dedicated to bringing social justice principles into our work in libraries. The Critlib chats are always really well prepared with suggested readings on the current chat topic, well moderated and with a lot of great thoughts and discussions on librarianship. Read more: http://critlib.org/ 

#auslibchat

Chat hosted by Australian Library and Information Association’s New Graduates Advisory Council (ALIA NGAC) on different topics related to libraries and librarianship. Even though it’s hosted from Australia I find the topics of chats relevant for the global library community. Read more: https://alianewgrads.wordpress.com/new-generation-advisory-committee/auslibchat/

#lisprochat

Chat on broad trends and topics in the library and information industries including education and job search. Read more: http://lisprochat.blogspot.dk/

#mashcat

Mashcat is a mashed library event / community focussing on cataloguing data and is for cataloguers, developers and anyone else with an interest in how library catalogue data can be created, manipulated, used and re-used by computers and software. They got a very cool archive with all previous chats + link to Storify: http://www.mashcat.info/twitter-chat/

This hashtag also function as a subject specific hashtag for tweets on cataloguing data. Read more: http://www.mashcat.info/

#tlchat

Chat for librarians working with teaching and instructions. Hashtag is also used regular on tweets relevant to teacher librarians. Check http://tlchat.wikispaces.com/  for info an chat dates.

Subject specific hashtags 

#datalibs

Things involving data services and working with data in libraries

#UXlibs and #libux

All things User Experience and Libraries – very active hashtag with lots of great content.

#libweb

All things library web services

#interlibnet

The hashtag for tweets involving activities connected to International Librarians Network (ILN). ILN is a peer-mentoring facilitated program aimed at helping librarians develop international networks. Read more about ILN: http://interlibnet.org/  

#openaccess

An extremely broad hashtag with everything related to Open Access. Much is the stuff is not from a library point of view but I only find that valuable and of inspiration.

#scholcomm

Great hashtag on all things scholarly communication

#ebliprg

A evidence-based library and information practice reading group / chat

#libtech

Stuff on technology in libraries but also holds some noise because it’s used by a snowboard company…

#infolit

Great and very active hashtag about all things related til information literacy and libraries.

#medlibs

Very active hashtags for health librarianship and  medical librarians. There are also regional hashtags in extension to the main one: #canmedlibs and #ukmedlibs are pretty active.

#lawlib

Tweets on law librarianship. The hashtag #lawlibs is sometimes also used for tweets on this subject.

#libleadgender

Great and active hashtag on gender issues in library leadership. Important input and discussions taking place here.

Keep ’em coming

I’m positive that the above list of library related hashtags is far from comprehensive but hopefully it can get some library folks new to Twitter going and be of inspiration for some Twitter old timers. If there is any Hashtags that should be added to the list please write them in the commentary field

Happy sharing, inspiring and connecting

Christian