What we cover in Episode 25
How can you build a world-class product team?
We talk to Claire Vo who is the co-founder of Experiment Engine (sold to Optimizely) and now SVP of Product at Optimizley about this very subject.
– Why world-class product teams are entrepreneurial, autonomous and focus on customer value.
– How to build a diverse product team, and why that’s a great thing
– Why experimentation should be part of your product teams culture
The PM’s at Optimizely are responsible for the health of the business, not just the health of the products. It means they need to think like both a PM and GM.
To do this, you need to have an entrepreneurial mindset, thrive on risk-taking and ownership, plus be greedy for continual success.
Although each product team has a lot of autonomy, they all share the same objectives and metrics through the OKR methodology. They have three key objectives:
– Increase the number of experiments Optimizely customers are running
– Increase customer maturity by helping customers to get better at experimentation
– Build enterprise-grade software that meets the needs of the largest brands in the world.
The objectives are tied back to metrics shared by the entire product team, for example – the sum of ARR (annual recurring revenue) of accounts that are running four or more experiments a month, and how much ARR of their accounts are running server-side experiments.
Wait, what’re server-side experiments?
Server-side experiments are where Claire sees a lot of experimentation moving. Instead of focusing on testing different elements of the front-end, you can run more complex and robust tests by experimenting with the backend data layer:
Optimizely’s OKR set is hyper-focused on improving the value customers are getting from their products.
But does all of that autonomy comes at a price? Don’t PM’s get overwhelmed by how much choice they have to build new products?
It’s something Claire actively screens for on her product team:
If you have a rigorous hiring framework, it can result in you building a team full of the same people. It means you’ll often have a very singular approach to ideas and problem solving from your team.
If you want to build a more diverse group of people, you have to be clear on the things you won’t budge on, and those you will.
Claire looks for entrepreneurial spirit, intellectual honesty and culture ad. These are things she won’t budge on:
What she does budge on is:
What is intellectual honesty?
Intellectual honesty helps you think about the solution to a problem from multiple viewpoints, not just your own.
Given the PM’s in Claire’s teams are wearing many hats, e.g., what’s right for the customer, the business, having intellectual honesty means they’re honest about the solution they’ve come up with, the potential pro’s and cons and potentially if there are unknown problems they don’t have great answers for.
Like Pinterest and SurveyMonkey, no individual or team own experimentation, instead, it’s part of every PM’s job.
Every Wednesday morning Optimizely have an experiment review meeting, where anyone across the organization can recommend experiments they want to see run.
Each experiment idea must be in a specific format with a lot of focus on the hypothesis:
The experiments are focused on user behavior that drive metrics listed in the OKR model. Each experiment should list what the potential ROI is if successful.
Along with the hypothesis and potential ROI, Claire puts a lot of focus on consistency of experimentation, she wants her team to get into a regular cadence of running experiments.
The podcast provides a more in-depth look at these topics, so if you enjoyed reading the above, please do give it a listen.
And until next time,
>> Google Play