Case Study Series
Case Study Series
TL;DR
Most of our customer case studies were outdated, overly promotional and not very useful as a source of customer insight. When the company decided to refresh them, I volunteered to lead the project, especially since I saw this as an opportunity to gather more insights for the ongoing personas project.
I designed the recruitment process, defined participant criteria, prepared the interview questions and coordinated the study with our data analyst. Across two rounds, we completed nine case studies. Besides producing fresh marketing content, the project gave us valuable qualitative evidence that helped validate and refine several of our personas.
My role: full ownership
The problem
We needed new customer stories, but there was also an opportunity to learn much more than that.
Outdated case studies
Most of our existing case studies were several years old. The newer ones were few and far between, which left us with a limited pool of recent customer stories and quotes for social proof.
Too promotional to be informative
Many of the older case studies were short and written in strongly promotional language. They worked reasonably well as testimonials, but revealed very little about how customers actually used the product, what they were trying to achieve or how Elastic Email fitted into their workflow.
Personas still needed evidence
At the same time, our proto-personas still contained assumptions that needed stronger validation. A case study series was not the same as a in-depth interview study, but it offered a rare opportunity to speak directly with customers and compare their real behaviours with the profiles we had created.
Challenges
Balancing marketing and research goals
The final articles were meant to be success stories, so I could not approach the conversations as open-ended research interviews. Questions about frustrations or reasons for leaving would not fit the into the project. I therefore had to look for useful behavioural insights without losing sight of the marketing objective.
Depending on customer participation
Previous attempts to recruit customers for research had produced disappointing response rates. There was a real possibility that very few people would volunteer. To improve our chances, I secured a budget for a $50 gift card for every completed case study.
Recruiting the right clients
Not every client was suitable. Participants needed to be paying users, relatively recent, and in good standing with very low spam rates. To identify them, I worked with our data analyst, who prepared a query matching the recruitment criteria.
The process
The process which led me to the final result was long and took some unexpected turns, however it involved three distinctive elements.
Designing the recruitment process
Once the budget was approved, I prepared the invitation email and defined the eligibility criteria for participation.
I wanted the process to be as convenient as possible, so participants could choose between two formats: answering the questions by email or joining me for a live conversation.
For those who preferred a call, I prepared a Calendly schedule with meeting slots available several weeks in advance.
Preparing questions that could serve two goals
I created a structured set of questions covering the customer's business, reasons for choosing Elastic Email, most-used parts of the product, typical workflows and how they discovered us.
The primary goal was still to create a publishable customer story, but I also wanted to learn whether participants resembled our personas, whether they used the product in expected ways and whether any new behaviours emerged.
Screening and interviewing participants
Once the invitation campaign was sent, responses started coming in quickly.
Some participants had to be excluded because their businesses conflicted with company policies, while others stopped responding during the process. Even so, we completed six case studies in the first round and another three in the second.
Most participants preferred answering by email, but two chose live interviews. These gave me the opportunity to explore their workflows in greater depth and ask follow-up questions in real time.
When any responses were unclear or particularly interesting, I followed up with additional questions, which allowed for even deeper understanding of the respondents.
Outcome
For the first time, I had detailed descriptions of how individual customers actually worked with Elastic Email.
The participants were not just account IDs or analytics segments anymore. They were people with specific businesses, goals, technical abilities and workflows, often using the product in ways that were difficult to infer from quantitative data alone.
Stronger evidence for the agency developer persona
Three participants shared a very similar profile: web developers working in digital or marketing agencies and managing email services for clients.
This matched a pattern that had also appeared in our review analysis and gave us much stronger evidence that our original SaaS microfounder persona was missing the mark.
We changed the developer persona to reflect this more common agency-based customer.
Confirmation of the low-effort small business user
Several responses supported the idea behind our small-business persona.
For these customers, email marketing was only a small part of their job. They wanted the shortest possible path to sending a newsletter and had little reason to explore more advanced functionality.
This strengthened our assumption that simplicity and low effort were important benefits for this group.
More confidence in the creator persona
We also found examples that broadly supported the creator persona and the way we expected this group to use the product.
One blogger, for example, used an RSS merge field to automatically send new blog posts to subscribers. Once the email template had been created, the process required almost no further interaction with the editor.
This was a good example of how a creator could use automation to keep email running in the background rather than treating it as a central part of their work.
Evidence for unexpected product behaviour
The case studies also surfaced behaviours that did not fit neatly into our original expectations.
One recurring example was a marketing-focused customer who was technical enough to use our API product instead of the marketing application, mainly because it was cheaper and gave them enough flexibility to build their own workflow.
This gave us evidence for a user type that had appeared in earlier discussions but had never been clearly confirmed.
Impact
Combined with the review analysis, the case study series helped move our personas further away from assumptions and toward observed customer behaviour.
It contributed directly to changes in the developer persona and strengthened our confidence in several other profiles and workflows.
At the same time, the interviews produced nine fresh customer stories that could be published on the company blog and reused as social proof across the website and other marketing materials.