Keynote: Make experimentation ridiculously easy for 2000 bol.com employees

Keynote: Make experimentation ridiculously easy for 2000 bol.com employees

Denise Visser – Product manager Experimentation @ Bol.com

1 minute video preview

Frank – Data Scientist, feedback through our #CH2021 attendee survey:

Cool story, interesting insight in how a large organization like Bol.com is doing this.

Slides

Notes

This is the link to the live notes of Denise her talk

Questions asked by attendees through our #CH2021 app:

• What kind of teams do experiments? And which teams you find it hardest to start experimenting with? And where do you think it depends on whether they do or don’t?
• So…. Good to hear about the process. Can you share a test outcome you did not expect? Always cool to hear.
• What content did you share with the board to get buy-in and how did you keep them involved after the approval for more funding?
• You talk about building your own platform. Why did you decide the “make” rather than “buy”? Which criteria did decide this?
• Option a b or c ? A. Police office, b. coach or c. Lead by example? Bonus with c. You can keep testing yourself !?
• Could you tell us how the holacracy way of working has helped you to spread the experimentation virus? (as it did for us at the Consumentenbond)
• What test was actually considered the first real experiment at Bol.com
• Why don’t you want to show segmented results or different KPIs and their sign level?
• Why do you use visits instead of users in the experiments result calculation?
• How did you motivate teams not knowledgable about testing to contribute?
• I assume many more people where included making hypotheses. How accurate are the hypotheses of your colleagues? And do you see an incline in accuracy when involving more colleagues from different team
• Where does the research and design happen? Within the productteams? And do you control the quality of the opportunity?
• Did I understand well that you have one KPI per test? When you have CR as a goal, do you also check AOV as a secondary goal?
• How do you take pricing and price competitiveness into account in your experiments?
• Have you convinced legal dept to join?
• How do we teach our company that all these tips should be implementeer?