04 Sep 2013 by
A More Natural Approach to Usability Testing

Often in usability tests, we ask participants to go to the homepage of the website under evaluation and navigate around. The problem is that people do not use the web like this… at least not anymore. People are more inclined to search for what they are looking for and enter a website already on the page they need. This got us thinking: How can we let users act naturally on a usability test? The answer is to ‘fake’ Google.

Fake Google Usability Test

Several months ago, we tried out a BYOD (bring your own device) usability testing session at the office, which Lisa wrote about in an earlier blog post. It was a way to capture quickly a wide range of feedback, as well as observe how participants interacted with the site on their preferred devices. We thought it would be the perfect opportunity to try out ‘Fake Google’ with actual participants.

Redirecting Users to ‘Fake Google’

The devices that participants brought covered a wide spectrum, from Mac and Windows-based laptops, to Android handhelds, iPhones and iPads, and an off-brand, feature-reduced tablet. All of the devices could connect to the internet using Wi-Fi.

Tech Alert: The next paragraph gets a bit technical. Basically, we configured a wireless router to send Google searches to our server instead of Google’s servers. The participant did not have to enter a special website address or change any settings on their device. This allowed the participant to proceed with the task how they would outside of the usability testing session, rather than from a prescribed starting point.

For the technophiles, we had a spare NETGEAR WNDR3700 wireless router that was flashed with the DD-WRT firmware and created a guest network for the participants to connect to. The DD-WRT firmware has a built-in DNS server, enabling us to intercept and modify traffic before it leaves the local network. The DNS server was configured so that a participant who went to google.com or submitted a search query from their browser’s search bar was actually sent to our local server while still maintaining Google’s URL structure.

Manipulating the Search Results

The site that participants would be redirected to looked exactly like the Google experience they were accustomed to on their handheld or laptop device. The site would capture the participant’s search query and get actual results from Google in the background. We manipulated the results before displaying them, redirecting results for the site we were evaluating to the redesigned version on the test server. Additional relevant results were also inserted as the top three results, allowing titles and descriptions to be tested with participants. The page titles and breadcrumb of the injected results matched the format of the redesigned website, and the descriptions contained phrases from the usability test task.

Benefits and Observations

By manipulating search results, participants could interact with the site how they would if they were at home or work. Not only were participants more at ease, the observations were more representative of the real world.

One of the more surprising findings is that not all of the additional search results we inserted at the top of the list resonated as the most appropriate result. Some participants indicated the titles or summary did not appear authentic, despite containing keywords from their query. In the future, injected results should look more realistic and less like a word-for-word match to the usability test task, which may have contributed to the links being perceived as search engine spam.

With the technical aspects figured out, ‘Fake Google’ can easily be adjusted to work with any other site or search engine, and is a great tool when conducting BYOD testing with more of a real world process.

09 Aug 2013 by

An alternative to post-it notes and sharpies: Using butcher paper and pre-printed sticky labels for workshop facilitation and analysis

There have historically been three sure things in IA – post-it notes, sharpies and the Polar Bear Book. We’ve recently expanded our IA toolkit to include a roll of butcher paper and a stock of Avery mailing labels. Having used … Read More

01 Jan 2014 by

Edmonton Public Schools – Taming Complexity

Edmonton Public Schools’ (EPS) public- and staff-facing web properties evolved over many years and represented a wide variety of purposes, audiences and designs. In 2012 EPS redesigned the district’s website and implemented a new content management system within an environment … Read More