The Experimentation Approach to GenAI: Ensuring High-ROI to Your Projects
« The real value of AI isn’t in what it can do, but in what it can do for you. »
In today's digital landscape, Generative AI (GenAI) is more than a trend—it's a strategic move for companies aiming to stay relevant and competitive. According to a recent report by McKinsey & Company, the potential of GenAI extends far beyond basic automation, offering transformative capabilities in areas such as personalized marketing, customer service, and product recommendations. Chief Product Officers (CPOs) and product directors are increasingly focused on integrating GenAI into their strategies to drive innovation and maintain a competitive edge.
However, the journey from conception to successful implementation is fraught with challenges, including high costs, wasted resources, and failed products. A study by Boston Consulting Group (BCG) highlights that many companies struggle with the practical application of GenAI, often due to a lack of strategic alignment and insufficient user relevance. To navigate these obstacles, companies can leverage lessons from experimentation to ensure their GenAI projects are not only innovative but also successful.
The Current Landscape of GenAI Adoption
Leading companies have already made significant strides with GenAI, showcasing its transformative potential. For instance, Accenture has documented how enterprises like Netflix and Amazon are leveraging GenAI to enhance their recommendation engines, leading to substantial increases in user engagement and revenue. Similarly, Deloitte reports that financial institutions are using GenAI to streamline customer service through advanced chatbots and automated advisors, resulting in improved customer satisfaction and reduced operational costs.
However, many other companies have struggled with costly, unsuccessful launches. A Gartner survey reveals that 30% of GenAI projects fail to meet their objectives, often due to misaligned goals and inadequate user insights. This reality underscores the need for a structured approach to GenAI project development, grounded in experimentation and iterative improvement.
Lesson 1: Align Projects with Current Business Objectives
One important lesson from experimentation is the need to align new projects with existing business objectives. While it's tempting to pursue every new opportunity, companies must ensure that their GenAI initiatives are relevant to their current goals. This means taking the time to complete ongoing projects and achieve current objectives before embarking on new GenAI ventures. For most businesses, this focus can prevent the dilution of resources and ensure strategic consistency.
Experimentation Advice: Conduct a thorough assessment of your current business objectives and identify how GenAI can specifically enhance or support these goals. Run small-scale experiments to validate the potential impact of GenAI on your existing objectives before committing significant resources.
Lesson 2: Ensure Product Relevance to Users
When a new technology like GenAI emerges, there is often a rush to adopt it without fully considering its relevance to the user. It's essential to remember that not all innovations will benefit every user or solve their problems. Companies should revisit their initial user problems and customer journeys to determine how GenAI can enhance the user experience.
Examples of Previous Technologies Poorly Used:
NFTs (Non-Fungible Tokens): Initially heralded as a revolutionary way to own digital assets, many NFT projects failed because they did not offer genuine value to users. Instead, they focused on the novelty of the technology itself. For example, several NFT marketplaces launched without considering the user experience or the actual utility of the NFTs being sold, leading to poor adoption and user dissatisfaction.
Augmented Reality (AR): While AR has great potential, many early AR applications focused more on showcasing the technology rather than solving real user problems. For instance, several AR shopping apps allowed users to visualize products in their homes but often lacked accurate rendering or practical functionality, resulting in a gimmicky rather than useful experience.
Blockchain for Supply Chain: Blockchain has immense potential for supply chain management, but some early implementations focused more on using the blockchain technology than on solving real supply chain issues. These projects often lacked scalability and user-friendly interfaces, making them impractical for everyday use.
Experimentation Advice: Start with user research to understand current pain points and opportunities for improvement. Use A/B testing to compare user engagement and satisfaction with and without the GenAI enhancements. This data-driven approach ensures that the technology truly meets user needs.
Lesson 3: Prioritize Based on Impact, Cost, and Effort
Effective prioritization is crucial for the success of any project. Companies should evaluate potential GenAI initiatives using a framework that considers impact, cost, and effort. Prioritizing projects that target the largest user segments with minimal effort and cost can yield the best return on investment. By focusing on high-ROI projects, businesses can maximize the benefits of their GenAI implementations without overextending their resources.
Examples of GenAI Projects in E-Commerce with Potential High ROI:
Personalized Marketing On-Site or Email Messages: Implementing GenAI to create personalized marketing messages for on-site displays or email campaigns based on user behavior and preferences. This can significantly increase engagement rates, conversion rates, and overall sales.
Improved Product Descriptions Based on Personas: Using GenAI to generate tailored product descriptions that resonate with different customer personas. This can enhance the shopping experience, improve product discoverability, and boost sales.
Description Ranking of Content: Leveraging GenAI to rank product descriptions and content based on relevance and user engagement metrics. This ensures that the most impactful and engaging content is prioritized, leading to better user experiences and higher conversion rates.
Customer Service Chatbots: Developing a GenAI-powered chatbot to handle common customer inquiries. This can reduce customer service costs and improve response times, enhancing the overall customer experience.
Experimentation Advice: Develop a prioritization matrix that evaluates each GenAI initiative based on its potential impact, cost, and effort required. Conduct pilot tests on high-priority projects to gather initial results and refine your approach before full-scale implementation.
Lesson 4: Embrace the MVP Approach
Launching a minimum viable product (MVP) and conducting early user tests are essential strategies in experimentation. These approaches allow companies to gather valuable feedback and make iterative improvements before a full-scale launch. Many companies mistakenly aim to release a final, polished product immediately, which can lead to missed opportunities for refinement and optimization. Embracing MVPs and user testing can help identify the most promising GenAI applications and ensure they meet user needs effectively.
Experimentation Advice: Begin with an MVP that addresses the core functionality of your GenAI project. Use techniques like fake door testing or smoke tests to validate user interest and gather feedback. Iterate based on user feedback and performance data to enhance the product incrementally.
Conclusion
Experimentation offers valuable lessons for companies looking to implement GenAI projects successfully. By aligning projects with business objectives, ensuring product relevance to users, prioritizing effectively, and embracing the MVP approach, businesses can navigate the complexities of GenAI adoption.
Consider the example of Amazon, which has used experimentation extensively to integrate GenAI into its recommendation systems. By running numerous A/B tests and starting with MVPs, Amazon has been able to refine its algorithms, resulting in a personalized shopping experience that significantly boosts user engagement and sales.
Similarly, financial institutions like JPMorgan Chase have successfully implemented GenAI in their customer service operations by starting small, testing rigorously, and scaling based on proven results. These examples highlight how a thoughtful, user-centered experimentation approach can lead to successful GenAI projects that drive meaningful innovation and business growth.
At Henkan & Partners, we specialize in helping companies uncover and seize opportunities in GenAI through these proven methods. Our expertise in product analytics, experimentation, and strategic alignment enables us to guide businesses in making data-driven decisions that lead to impactful and sustainable innovations. By partnering with us, companies can ensure their GenAI initiatives are not only innovative but also aligned with their business objectives and user needs, maximizing their chances of success in this rapidly evolving field.
