Improving search functionality
Citizens Advice aims to provide the advice people need for the problems they face and to improve the policies and practices that affect people’s lives. They provide free, independent, confidential and impartial advice to everyone on their rights and responsibilities.
The problem: “Search is broken”
Advisers have been reporting problems with search. They used to be able to find what they were looking for, but it is becoming increasingly difficult to surface the information they need.
Advisers need to find specific, detailed information at a moment’s notice to give the right advice to a client. Giving correct advice could mean people won’t lose their benefits, their job or be declared bankrupt.
Adviser search vs public search
Citizens Advice has 2 core user groups: The public and advisers. They use search in very different ways:
Advisers need to find information quickly to help their clients. They predominantly rely on search to recover information that they know exists, whereas the public use search to discover information about an issue affecting them. The public predominantly reach our advice pages via Google search, and thus rely less on internal search.
8% of public sessions use internal site search, while 48% of adviser sessions do.25% of the public refine their search in comparison to 45% of advisers.
Refinement was clearly an issue for advisers. The high number of refined searches led us to identify that advisers could not find what they were looking for in their first search. They had to perform multiple searches to find what they were looking for.
Setting an objective
We quickly agreed that our goal should be to reduce search refinement. This would mean that advisers were finding the right content faster, saving time and allowing them to focus on the client. Saving time in advice sessions could even enable advisers to see more clients as a result. As a team, we defined our objective:
“To provide quick, clear, relevant, device agnostic search results”
Our core KPI was to reduce the number of secondary searches by Advisers by 30% in 6 months. It was a bit of a ‘finger in the air’ number, but I convinced the team that having an unambiguous target was important for drive. We would know in 6 months whether the number had been wildly inaccurate, and thus be able to set a more accurate goal for the following 6 months.
We had a clear focus, and this would guide how we prioritised the workload throughout the project.
Gathering ideas
I organised a workshop for the team. Armed with the web stats that we had already gathered, some anecdotal information from advisers, and our combined industry knowledge we created a list of hypotheses of what we could do to improve search. We all had some fairly clear ideas, many of which overlapped, confirming our suspicions that they were solid, educated suggestions. And from them prioritised the following:
Spellcheck: replace ‘did you mean’ with ‘showing results for’ Improve the designImprove the metadataGive better guidance on the ‘no results’ page
We built a search dashboard to track our KPI and some other core health metrics. We didn’t want to improve secondary searches, only to find out that we had negatively impacted our bounce rate. We added a survey to the bottom of the search results page where Advisers could leave us more detailed feedback. This, we hoped, would add more depth and detail to the anecdotal information we already had.
Spellcheck: replaced ‘did you mean’ with ‘showing results for’
A lot of secondary searches were performed due to spelling mistakes. If a user typed Baliff we would show a page with the message “Did you mean Bailiff”. The user would then click on the link to Bailiff to reach their search results. The new journey takes the user straight to the results page for ‘Bailiff’ and offers the message “Showing results for Bailiff, search instead for Baliff. It’s a mental model users of Google Search are familiar with.
Improve the design
The search results page used an out of date stylesheet that was not in keeping with the rest of the website. It also had a block of text before the results that unnecessarily explained what was in the results. The results were split into 3 tabbed sections, and our stats told us that only 2% of users were clicking on these. Social sharing was available on the results page, and these too were rarely interacted with. We decided that a simpler approach would make readability and scannability of the results page much easier for the user. It wasn’t visually clear whether a result was an index page or an advice page, so we wanted to highlight these with a sitelinks design pattern.
Improve the metadata
A lot of the metadata had been poorly written and was keyword stuffed. We didn’t have the resource to embark on manually rewriting all the metadata on the site, so we selected our top 100 adviser search terms and rewrote the metadata for those pages. In doing so, we created new guidelines and training for how to create good metadata going forward. We bandied around the idea of auto removing metadata from all pages of a certain age, but decided that it might make too large a change, and we were seeking to incrementally improve rankings. A big change would be jarring for the advisers who had become accustomed to expecting certain results from certain search terms.
The content designers were encouraged to think of the meta description as a normal line of content, designed to tell the public and advisers what the page is about. They should use the same language as the rest of the content and show users that the page will answer their need. They shouldn’t try to list everything the page covers – just explain the main topic in a way that doesn’t repeat the title.
Give better guidance on the ‘no results’ page
Our no results page was unhelpful and busy:
The new design clearly explains that there are no results, and offers a better onward journey.
So, what happened?
Our KPI was to reduce the number of secondary searches by Advisers by 30% in 6 months. We reduced them by approximately 8% in 6 months. We were relatively happy with the result.
However, we continued to receive complaints about poor search results.
The eye opening moment was when we undertook further observational research. We discovered that secondary searches were not necessarily because advisers couldn’t find what they were looking for, but rather they were a way of navigating to a section of the site before searching in more detail. It was a bit of a blow to learn that hitting our KPI was not indicative of success.
What did we learn?
Did we agree on our KPI too early? Perhaps.
Should we have done more upfront research? Definitely.
The quantitative stats were telling us that secondary searches were an issue. We reached our own conclusion about why. More research would have revealed that the issue was with the information architecture, not the search results. We should have investigated the anecdotal information from advisers in more detail.
We undertook some really great work, and have definitely made the experience of using search better. However, the problem remains that the results are simply not good enough.
Cue phase 2
So, what’s on our radar for phase 2 of the search improvements project? Setting a new KPI for one! We’ll also be looking more closely at the search engine itself. Here are a few of the things we are considering:
Edit the weightings of our content
Add filtering functionality
Improve our synonyms dictionary
Investigate our usage of stop words
Offer an alternate way of exploring the advice
Display related topics and related results
Offer search as you type functionality
Auto remove metadata from pages over a certain age
New search engine
Watch this space for an update!
Image credit: Google