Improving how our customers ask their questions proved one of the most impactful projects of my career, resulting in delighted customers, tens of millions of cost-savings for Intuit, numerous patents and an ACM peer-reviewed paper presented at the Intelligent User Interfaces 2018 conference in Tokyo.
Our Self Help portal taught us that the right question-posting design can substantially deflect duplicate questions and reduce contacts, while getting customers the answer they need.
Seeking to improve their customer-success experience, and address a forecasted self-help surge in 2017, TurboTax approached us for help. Improving the experience and expected surge led us to numerous opportunities, and we settled on Reducing Duplicates, and Improving Helpfulness.
Reducing Duplicate Questions
Key to duplicate-deflection is providing results based on the customers question prior to posting. Research showed that customers tended to search with keyword strategies and often had a hard time articulating their question due to a lack of domain expertise. By searching against a fully formed question, we could often closely match with answered duplicates. The design challenge was in how to provide these results in a way that customers found valuable, and not as an unexpected detour.
Rapid-prototyping led us to a design that set expectations with a 3-step accordion pattern. The potential duplicates were presented prior to investing time in details, and with a quick-view feature that made it a safe-click. This design also compartmentalized the tasks, so customers could focus on finding their answer rather than rushing to post.
By analyzing the questions that were being posted, I discovered ways that we could improve over the prior design. This included people thinking they were chatting privately to agents, posting queries rather than questions, duplicating their question in the details area, and providing too much, or too little information. The final design set the context for a public Q&A system, and helped customers form effective questions.
The results were dramatic, averaging 50% deflection. This addressed the surge in self-help, and improved both conversion rates, helpfulness rates, and contributor effectiveness. It also reduced escalation to expensive and time-consuming assisted-care.
Increasing Helpfulness with Question Optimizer™
With duplicate-deflection performing well, we moved on to improving the quality of questions. This began with an ad-hoc conversation with my data scientist, who revealed correlations between question and its likelihood of receiving a helpful response.
This led to numerous patents, and a peer-reviewed article that was presented at the 2018 ACM Intelligent User Interfaces conference in Tokyo.
Key insights included that closed-ended questions performed best, while why questions performed worst. I saw this as a design opportunity, and we partnered to develop a question-posting experience that provided interactive guidance.
At the time, only Quora.com provided question-asking tips, but they were static, lacked analysis and the insights we knew resulted in better questions. We settled on three approaches: upfront guidance, upfront constraints (usually favored in usability), and iterative reflective guidance.
The iterative approach proved most successful, and facilitated gamification and dynamic, personalized tips with each re-write.
We had a breakthrough when rapid-experiments proved that people write higher-scoring questions when re-writing their questions from scratch. I used a mid-fidelity prototype to prove out how dynamic tips and a fresh input-box can facilitate this.
TurboTax reduced contact-rate by 20%, and deflected over 50% of potentially duplicate questions. The design helped 72% find their answer before posting. All this despite 18% more questions overall!
TurboTax and QuickBooks are now eager to leverage the Question Optimizer™ widget. Design refinements continue, particularly in the form of a Conversational-UI, which offers a familiar dialog that's well suited to the nature of the problem.