So you've done an experiment - now what?
How I got better at analysing results
Hi friends,
You did an experiment. Whether it is at work or in your personal life, I’m glad you decided to do so.
The experiment is just the beginning. The real value comes from how it did in comparison to our goals.
Side note: If you want to learn how to run good experiments, go read this and come back!
Experimentation is nothing without effective and efficient data or results analysis.
If I had to simplify the process into 3 steps, it would be to:
Look at the results (in comparison to your goals)
Call out out the most important takeaways
Outline the next steps
It sounds easy but the data analysis process can actually get quite time consuming and challenging — depending on the nuances of the experiment and data volume.
Working in a fast paced industry, especially in a startup environment, you simply cannot afford to move slowly.
I've been practicing to analyse data faster and better. Here's what I've learned:
Get to know your metrics really well, then create a framework
The metrics you set out to improve are the most important starting point.
Know them so well that you can precisely identify what you are trying to compare.
When you don’t have a clear idea on what you are measuring, you will find it 10x more difficult when it comes to analysing the results.
Unclear metrics
Increase in users
Increase in customer satisfaction
Consistency
Clear metrics
% of users (as of total traffic) clickthrough to next step
Average satisfaction score of 4 out of 5
No. of times you showed up in a month
This is a step to be done before you start your experiment.
Make sure you plan and set up analytics correctly and thoughtfully.
Do you have your events eg: Google analytics set up?
Do you have a tracking app or device?
Do you have a feedback form or survey you plan to send out?
Once you are measuring the right things, data analysis becomes much easier.
Say you already have all that covered, you can set up frameworks you can use regularly to measure progress.
Whether you're evaluating a feature, funnel, or qualitative data, a good framework should include:
Key questions, issues or objectives
Solution or modifications
Timeframes: before and after the change
Results (Primary and secondary metrics)
Takeaways and next steps
Example:
Issue: I’ve been unmotivated to go work out. When I do go, I rush my sessions.
Solution: Get a friend to hold me accountable
Timeframe: 2 months before and after
Results:
No. of times a week going to the gym increased from 2 times to 4 times
Average gym session duration increased from 30 mins to 45 mins
Takeaways and next steps:
Having a workout buddy is effective but relying on 1 workout buddy may not be sustainable.
Look into group workouts, getting a personal trainer or more friends to work out with.
Set a benchmark for comparison
Once you have calculated the figures or percentages eg: clickthrough rate or conversion rate, you need to conclude whether the change is positive, negative, or neutral.
This is straightforward when you are building upon past results. However, if you are collecting data for the first time, you might not have a baseline for comparison.
In such cases, consider these alternative benchmarks:
Industry standards
Comparable features
Competitors
Example: If you're in email marketing, you could compare your email open rate to average rates in your industry.
Set aside time to revisit results
Stay on top of your game by setting reminders to check your results. It's easy to forget when you made changes, especially when there is a lot going on.
Jot down when you implement new changes and set reminders to review the results regularly. I'll be honest, at first I found data reporting a real struggle. It took forever and I got super frustrated. It was not something that I wanted to dedicate time to do.
But here's the thing - if you don't make time for it, you could end up steering in the completely wrong direction.
Set aside some time to do it right.
Presentation is everything
Once you've done all the hard work of analysing your data, the final step is presenting your findings effectively. This is where you showcase what you’ve found in a digestible format to help you and others make decisions.
How you present your findings really matters and I’m still working on getting better at it.
There is a reason why first-year consultants get nitpicked on their formatting and all the details on their slides.
Here are a few tips to consider when putting together your presentation:
Highlight the key points
Identify the most important findings from your analysis. What are the top 2-3 takeaways that your audience needs to know? Focus on these to avoid overwhelming your audience with too much information.
Distill the learnings
Take complex data and simplify it into clear, concise messages. What does the data mean for you, your project or business? What actions should be taken based on these insights?
Create simple visuals
Visual representations of data can be incredibly powerful. Choose the right type of chart or graph for your data. Make sure they're clear, easy to understand, and visually appealing. Pay attention to details like color choices, labels, and scales.
One final tip → Tell a story.
Don't just present numbers - weave your data into a narrative using the framework above.
I promise you will get better with practice.
I would love to hear more from you. Do you have to do any data reporting or analysis?
How do you make the process easier?
Viv’s collection
🖥️Tech: iOS 18 and Mac OS Sequoia
I recently upgraded to iOS 18 on iPhone and Mac OS to Sequoia. There are so many features but the upgrades I actually find useful or interesting are:
Control center customisation:
I use it a lot more now and I love the easy access to the motion sickness feature.
Screensavers and widget on Desktop:
The forest screensaver is beautiful and so calming. The widgets on the main desktop screen are very sleek.
Notes app calculations:
You can do calculations for splitting your bills and even currency conversions!
📺Creators/content worth your time
i am a collection of dismantled almosts - Postcards by Elle
I loved this piece about dwelling too much on ‘almosts’.
More on “low-stakes friendships” - Maalvika
A lot of great points to ponder on how to approach adult friendships.
In case you missed it: In my last newsletter, I share my overwhelming thoughts lately.
A question for you!
What small victory did you achieve today, and how did it make you feel?
Always open to learning more and hearing from you 😊
Thanks for reading!
Stay inspired,
Viv