Hollis Empty Search

 
hollis_banner.gif

PROJECT MOTIVATIONS:

Empty states—e.g., moments in a user’s experience with a product where there is nothing to display—are some of the most overlooked aspects of UX. Useful empty state design, however, can let the user know what’s happening, why it’s happening, and what to do about it in a manner that not only enhances the user experience but also bolsters a product and brand. Empty states are easy to overlook in UX design because they occur infrequently and aren’t always well understood. There currently are no standard guidelines regarding how to design for empty search queries.

RESEARCH AIMS:

·  To determine what is the ideal empty state design for search on the HOLLIS website based off users’ mental models surrounding search (e.g., Do users expect feedback? A chance at exploration? Or do they want nothing?)

·  To understand what casual and expert researchers expect when they execute empty (no content) and asterisk (*) searches in HOLLIS and how those compares to their expectations on two other comparison/control sites, GOOGLE SEARCH and the YALE ORBIS library catalog

·  To learn what kinds of information users would find relevant in response to submitting empty searches in HOLLIS (and how it compares to other search engines)

This project was inspired by an analysis of the top 1,000 search themes submitted on the HOLLIS site since 2016 (asterisks were in the top 400 query themes) and by a landscape review of popular academic and non-academic search engines’ empty state designs.

METHOD:

  • Participants were Harvard affiliates (e.g., staff; undergraduate students; graduate students) split into novice and experienced HOLLIS users according to a pre-test survey question regarding their familiarity with HOLLIS and frequency of use

  • User tests of interactive prototypes of websites (on Sketch) involving think aloud method and Quicktime audio recording

  • All participants first described their expectations regarding the output of an empty search query on HOLLIS. Participants then described their expectations regarding the output of submitting an asterisk (*) and a blank space in the HOLLIS search box. Participants provided their expectations for the HOLLIS site first followed by their expectations of the same searches (e.g., empty search; asterisk; blank space) for GOOGLE and YALE ORBIS library catalog. Presentation orders for the second and third websites were counterbalanced across participants.

ANALYSIS:

  • A clustering approach was used to identify the prevalence (e.g., proportion) of particular themes from participants’ survey and qualitative interview responses

 

NO CONTENT QUERY RESULTS (click on image to expand):

  • Expectations surrounding empty queries varied as a function of the different websites, with the proportion of participants who expected something to happen falling at 50% for HOLLIS, > 50% for GOOGLE, and < 50% for YALE ORBIS

  • Specific empty query expectations varied as a function of the different websites, with the majority of participants expecting an empty query to produce either an error message or a “No Results Found” message for HOLLIS and YALE ORBIS. Participants expected an empty query in GOOGLE to lead to exploration (e.g., a game or interesting piece of information) or search guidance (e.g., a list of suggested things to search for)

 

ASTERISK QUERY RESULTS (click on image to expand):

  • For the HOLLIS and YALE ORBIS websites, participants expected asterisk queries to search for everything within the respective search engines due to prior knowledge and uses of the asterisk as a wildcard operator in many programming languages

  • For the GOOGLE website, participants did not expect an asterisk query to function as a wildcard operator, however, and instead expected the SERP to feature all content topically related to asterisks

 

USER RECOMMENDATIONS FOR EMPTY QUERY RESULTS (click on image to expand):

  • Overall, the majority of users recommended that empty queries facilitate PERSONALIZATION (most popular) and SEARCH GUIDANCE

  • PERSONALIZATION included (1) providing user histories of prior searches, (2) redirecting to most visited sites, and (3) enabling a shortcut to relevant or most-accessed databases

  • SEARCH GUIDANCE included (1) a pop-up with advanced search options, (2) a contextual menu with more search options, and (3) information on how to search in HOLLIS

  • Preference for PERSONALIZATION was evenly-split between novice and experienced HOLLIS users whereas preference for SEARCH GUIDANCE was greater among novice users compared to experienced users. This latter finding suggests that added information about how to search in HOLLIS could be useful for novice users and also suggests ways new user onboarding in HOLLIS can be facilitated through its search empty state design.

 

INSIGHTS & DESIGN RECOMMENDATIONS

CLICK ON IMAGE TO EXPAND

Insights from the user interviews suggest that empty search output design on HOLLIS should feature an error message with search guidance in the form of suggested search content types, particularly for casual users not yet logged into the site. Once logged in, search might enable more personalization.

Asterisk searches on HOLLIS, on the other hand, should directly produce all content, in line with users’ expectations.

Users’ recommendations were not evaluated in a mutually-exclusive manner; future work might consider having participants rank-order the four major themes that emerged from the qualitative interviews to determine which are most relevant.