Hollis Empty Search



There currently are no standard guidelines regarding how to design for empty queries. What are users’ expectations surrounding the kinds of output that should result in response to submitting no content in search bars (either intentionally or accidentally)? Do users want feedback? A chance at exploration? Or do they want nothing? The following user testing explored this question by assessing users’ expectations of varying kinds of queries across different search engines with the goal of improving the HOLLIS search experience.


·  To determine what is the ideal search bar & SERP design for the HOLLIS website

·  To determine what novice vs experienced HOLLIS users expect when submitting no content, an asterisk (*), and a blank space in the search bar and how it compares to their expectations on two other comparison/control sites, GOOGLE SEARCH and the YALE ORBIS LIBRARY CATALOG

·  To learn what kinds of information users would like to see in response to submitting no content in the HOLLIS search bar (and how it compares to other search engines)

This project was inspired by an analysis of the top 1,000 search terms and themes submitted on the HOLLIS site since 2016 (asterisks were within the top 400 queries) and by a landscape review of popular academic and non-academic search engines.


  • Participants were Harvard affiliates (e.g., staff; undergraduate students; graduate students) split into novice and experienced HOLLIS users according to a pre-test survey question regarding their familiarity with HOLLIS and frequency of use

  • User tests of interactive prototypes of websites (on Sketch) involving think aloud method and Quicktime audio recording

  • All participants first described their expectations regarding the output of an empty search query on HOLLIS. Participants then described their expectations regarding the SERP of submitting an asterisk (*) and a blank space in the HOLLIS search box. Participants provided their expectations for the HOLLIS site first followed by their expectations of the same searches (e.g., no content; asterisk; blank space) for GOOGLE and YALE ORBIS LIBRARY CATALOG. Presentation orders for the second and third websites were counterbalanced across participants.


  • A clustering approach was used to identify the prevalence (e.g., proportion) of particular themes from participants’ survey and interview responses


NO CONTENT QUERY RESULTS (click on image to expand):

  • Expectations surrounding empty queries varied as a function of the different websites, with the proportion of participants who expected something to happen falling at 50% for HOLLIS, > 50% for GOOGLE, and < 50% for YALE ORBIS

  • Specific empty query expectations varied as a function of the different websites, with the majority of participants expecting an empty query to produce either an error message or a “No Results Found” message for HOLLIS and YALE ORBIS. Participants expected an empty query in GOOGLE to lead to exploration (e.g., a game or interesting piece of information) or search guidance (e.g., a list of suggested things to search for)


ASTERISK QUERY RESULTS (click on image to expand):

  • For the HOLLIS and YALE ORBIS websites, participants expected asterisk queries to search for everything within the respective search engines due to prior knowledge and uses of the asterisk as a wildcard operator in many programming languages

  • For the GOOGLE website, participants did not expect an asterisk query to function as a wildcard operator, however, and instead expected the SERP to feature all content topically related to asterisks



  • Overall, the majority of users preferred for empty queries to facilitate PERSONALIZATION and/or SEARCH GUIDANCE

  • PERSONALIZATION included (1) providing user histories of prior searches, (2) redirecting to most visited sites, and (3) enabling a shortcut to relevant or most-accessed databases

  • SEARCH GUIDANCE included (1) a pop-up with advanced search options, (2) a contextual menu with more search options, and (3) auto-suggestion