Sponsored Projects NPS Research

Led UX research and synthesized data to fuel immediate change and increase NPS by 39 points.

Role: Senior UX Designer and Researcher

Team: Caitlyn Holland (Senior UX Designer), Chelsea Phelan (Product Manager)

My Contribution: UX Research

Employer: Cayuse

Skills: Interviewing, Data Parsing and Synthesizing

Tools: Pendo, Calendly, Zoom, Airtable

Timeline: 3 months

User Problem

One of our flagship products, Sponsored Projects (SP), had a NPS (Net Promoter Score) that consistently lingered in the negative double digits. Since NPS is a measurement of customer loyalty and overall satisfaction, we knew we were in trouble and at risk of customer churn, and needed to turn it around — fast.

Goal

Gain a deep, actionable understanding of the factors contributing to low NPS scores among our customer base. These insights would inform and prioritize product improvements, enhance customer support, and contribute to targeted retention strategies.

Process

Recruit users and conduct interviews
Parse and synthesize raw data
Create summary document to inform roadmap planning

Recruit users and conduct interviews
Using Pendo, our insight and analytics tool, we recruited users who gave a detractor NPS (0 – 6) and left an insightful comment. I wrote the initial and follow-up emails that (along with a $25 gift card) resulted in five SP users signing up to share their insights. Over the course of three weeks, our two-person team used the questions that I prepared to conduct 30-45 min interviews with each user. We knew we were dealing with unhappy people, so we used that small window of time to build rapport, learn about their usage in the system, and discover their biggest pain points.

What’s the first thing you do after you sign in?

How do you know if you are successful?

What’s one thing you would change about the product?

2. Parse and synthesize raw data

With Zoom transcripts in hand, we parsed the interviews and entered the data into Airtable, a relational database that allowed us to add multiple themes and tags. Each data point was designated as a positive comment, neutral comment, pain point, or a suggestion. As themes began to emerge, we grouped and sorted the data to better understand users' roles and responsibilities, workflows, good and bad days, and the things they wanted to change most about our system.

I entered datapoints from five interviews into Airtable for organizing and sorting.

“If I could fix one thing it would be navigability. There’s no ‘Back to search results’ button. If I search by Sponsor and they have 30 different projects and I select the wrong one, I have to seach all over.”

— Sponsored Projects user

“I love the potential of Cayuse, where you can create an ecosystem around a project. Any complex project requires tens of emails, many documents, different versions of documents. If I can have a web-base product that can keep track of this for me, with a history, separated from other projects- that is an added value. Otherwise, I have to do it with email.”

— Sponsored Projects user

“Because the system doesn't auto-calculate things, I have to keep an Excel workbook open to keep track of everything and there is STILL human error risk.”

— Sponsored Projects user

3. Create summary document to inform Roadmap Planning

To simplify the large amount of data compiled in Airtable, we created a thorough summary document that could be shared with other teams across the company. We boiled it down to high-level takeaways of the themes and included direct quotes (because sometimes, it’s more impactful to hear it in the user’s own words). We also wrote a strategic action plan to address the major pain points, and focused first on those that could be alleviated with the smallest amount of developer effort. Tackling the highest-value, lowest-effort issues could deliver high impact within a two-week sprint, and more robust functionality could take a quarter or more.

I compiled the results of the research into a 19-page document to share across the company.

Results

The NPS research enabled us to have a much deeper understanding of SP users and their pain points. The insights revealed opportunities for feature enhancements within SP, as well as a bigger picture of where we could improve touch points across the entire customer and user experience. User’s frustrations helped to fuel immediate change across the company, as we shared the findings with Sales, Implementation, and Customer Success. The strategic action plan clearly laid out the next steps for SP, and as the prioritized changes were incorporated into sprints, we saw SP’s NPS score increase 39 points.

Implementation-related improvements:

Involve end users in implementation meetings
Confirm that legacy data transferred properly into Cayuse

Usability improvements

Improve in-system automation to generate sponsor info, validate form fields, etc.

Create configurable views for Reporting
Routing needs to be more flexible to avoid work-arounds

Customer education improvements:

Continue to build out Pendo Help Center
Continue to build robust documentation
Create short, targeted training videos for reference

Pendo’s monthly NPS displays a 39 point increase.

“Throughout our collaboration, Caitlyn actively participated in brainstorming sessions, conducted usability testing, and hosted user interviews, offering valuable insights and creative solutions to design challenges. Her collaborative approach fostered a productive and transparent working environment, driving our team towards shared goals and objectives.”

— Chlesea Phelan, Project Manager for Sponsored Projects

About Resume More