#001 - Geoffrey Bell, Microsoft
In this conversation, Alexandre Suon interviews Geoffrey Bell, a program manager for experimentation at Microsoft. They discuss Geoffrey's role in experimentation at Xbox and his background in sales and marketing.
Summary
In this conversation, Alexandre Suon interviews Geoffrey Bell, a program manager for experimentation at Microsoft. They discuss Geoffrey's role in experimentation at Xbox and his background in sales and marketing. They also talk about the tools and processes used for experimentation at Microsoft, the importance of testing everything, and how experiments are prioritized. They touch on the challenges of running experiments at different types of companies and the role of data in decision-making. In this conversation, Geoffrey Bell discusses the challenges of sharing experiment results across teams and the importance of having a standardized template to present the findings. He also talks about the challenges of providing equal visibility to all publishers and third-party companies on the Xbox platform. Personalization is another key aspect of their work, using algorithms to recommend games based on previous purchases and user demographics. Geoffrey emphasizes the importance of documentation and maintaining an up-to-date library of resources. He defines experimentation as data-backed decision-making and highlights the need to challenge the status quo and avoid relying on past practices. Finally, he mentions the competition in the gaming industry and the shift towards free-to-play games.
Keywords
experimentation, program manager, Xbox, sales, marketing, Microsoft, tools, processes, testing, prioritization, challenges, data-driven, experiment sharing, standardized template, visibility, personalization, documentation, decision-making, gaming industry, free-to-play games
Takeaways
Geoffrey Bell is a program manager for experimentation at Microsoft, specifically focusing on the Xbox ecosystem.
Experimentation at Xbox involves testing various elements such as game design, user experience, and marketing strategies to improve conversion and engagement.
Microsoft has an internal experimentation tool called EXP, which is highly sophisticated and covers multiple business areas.
The prioritization of experiments is based on factors such as potential impact, development cost, and alignment with company goals.
Experimentation teams at different companies face unique challenges, such as limited traffic or resources, but the goal remains the same: to make data-driven decisions and improve the customer experience. Having a standardized template for sharing experiment results can help teams communicate findings effectively.
Providing equal visibility to all publishers and third-party companies is a challenge in the gaming industry.
Personalization algorithms can recommend games based on previous purchases and user demographics.
Documentation is crucial for maintaining a library of resources and sharing knowledge within the organization.
Experimentation enables data-backed decision-making and challenges the status quo.
The gaming industry is competitive, with a shift towards free-to-play games.
Titles
Exploring Experimentation at Xbox
The Role of Data in Decision-Making Experimentation: Data-Backed Decision-Making
Documentation: Building a Library of Resources
Sound Bites
"The truth is in the numbers."
"I consider myself a spoke on a wheel on an eighteen-wheeler."
"How do we select those two to fifteen crucial tests?"
"How do you share your experiments across a wider team? You guys have a standardized template?"
"How do we democratically offer everyone a little bit of visibility?"
"Personalization is kind of like an icing on a cake."
Chapters
00:00 Introduction and Location
00:45 Geoffrey's Role at Microsoft
05:53 Experimentation at Xbox
08:04 Tools and Processes for Experimentation at Microsoft
11:43 Testing Everything and Production Releases
12:24 Comparison of Experimentation at Different Companies
16:15 Setting Priorities and OKRs
21:07 Optimizing Experience, Design, and Content
23:19 The Power of Filters in Experimentation
24:23 Data-Driven Decisions and Gut Feelings
25:30 Sharing Experiment Results and Standardized Templates
29:02 Equal Visibility for Publishers and Third-Party Companies
35:07 Personalization in Gaming
46:20 Documentation and Building a Library of Resources
48:17 Experimentation and Data-Backed Decision-Making
51:26 The Gaming Industry and the Rise of Free-to-Play Games
