A conversation with Kevin Weil (OpenAI CPO), Mike Krieger (Anthropic CPO), Sarah Guo (Conviction)

Kevin Weil's Role at OpenAI and Initial Reactions

• Kevin Weil took the role of Chief Product Officer at OpenAI, which he found to be one of the most interesting and impactful roles, with many challenges to figure out, including building a product with a constantly evolving technology base, as computers can do something new every two months that they have never been able to do before in the history of the world 1m14s.
• Kevin's friends and team generally reacted with excitement to his new role, and he has been having a blast working on the inside as AI gets developed 1m18s.

Mike Krieger's Move to Anthropic and His Experience

• Mike Krieger, the founder of Instagram, joined Anthropic as Chief Product Officer, and people's reactions to the news varied, with some thinking it made sense, others wondering why he would take the job, and some being impressed that Anthropic could hire the founder of Instagram 2m21s.
• Mike couldn't resist the opportunity to work on something new and was drawn to the company's research-driven approach, and he has been enjoying learning about enterprise and serving customers different from those he worked with at Instagram 2m37s.
• Mike has been surprised by the childish delight he feels in learning about new things, such as enterprise and research-driven organizations, and he appreciates the opportunity to have a different experience every year, as he had vowed to do when he was 18 3m31s.

The Nature of Enterprise Sales and Customer Feedback

• Enterprise sales have a different pace compared to other sales, with a longer timeline that can take six months from the initial conversation to deployment, requiring adjustment to different timelines 4m5s.
• The feedback and engagement from enterprise customers can be more rewarding, as they have a financial incentive to provide honest feedback on the product's performance 4m31s.
• In enterprise sales, the focus is not just on the product, but also on the buyer's goals, and building a great product does not necessarily guarantee success 5m2s.
• Enterprise customers may have specific requirements, such as advance notice of product launches, which can be challenging to accommodate 5m29s.

Product Development at OpenAI and the Role of Model Capabilities

• At OpenAI, they have multiple products, including consumer, enterprise, and developer products, which can be managed simultaneously 5m44s.
• Instincts can be helpful in product development, but only in about half of the job, especially when the product is in the final stages of development 5m53s.
• The beginning of product development can be uncertain, with unknown capabilities and emergent properties of models, requiring a wait-and-see approach 6m31s.
• The product development process can be influenced by the capabilities of the model, and the product that is built will depend on the model's performance, which can vary from 60% to 99% 7m1s.
• The research team's progress is regularly checked to determine the model's capabilities and potential applications 7m18s.
• Model training is a research process where the outcome is not always certain, making it exciting and stochastic, similar to the experience of working at Instagram during Apple's WWDC announcements, where a new feature could either be awesome or cause chaos, but in this case, the disruption comes from within the company 7m22s.
• The cycle of discovering new capabilities and planning for the next set of features is challenging when the outcome is uncertain, but it's possible to plan by squinting at the advancements in intelligence and building products around expected capabilities 8m2s.

Approaches to Product Development with Evolving AI Models

• There are three ways to approach this: watching the advancements in intelligence, deciding on product capabilities and fine-tuning with research teams, and co-designing and co-researching with the actual research teams 8m29s.
• Embedding designers early in the process is crucial, but it's essential to understand that the output of experimentation should be learning, not perfect products, and partnering with research should lead to demos or informative things that spark product ideas 9m4s.
• Research is both product-oriented and academically focused, and sometimes, new capabilities are discovered by chance, leading to unexpected opportunities 9m32s.
• When investing in new capabilities, it's essential to consider whether a model can be useful even if it's only 60% successful at a task, especially if the task is valuable and important 10m13s.
• Evaluating progression on a task and deciding what to prioritize involves considering the importance and value of the task, even if the model is not 99% successful 10m30s.
• The burden of product design is to make AI models work gracefully, even when they're not perfect, and to expect human involvement in the loop, especially when models are only 60% right, which can still be valuable for users 10m40s.

The Importance of Imperfect AI Models and Real-World Testing

• GitHub Co-Pilot, an AI product that assists with coding, was launched with a model that wasn't perfect but still provided significant value by getting the code partially correct, allowing users to edit and complete it 10m59s.
• Similar experiences will occur with the shift towards agents and longer-form tasks, where models may not be perfect but can still save users time and be valuable, especially if they can understand their limitations and ask for help 11m45s.
• The 60% benchmark is not a fixed number, but rather a rough estimate, and AI models often perform well on some tasks and poorly on others, making it essential to design for these variations 12m9s.
• Pilot programs with customers have shown that AI models can receive vastly different feedback, with some companies finding them highly effective and others finding them less useful, highlighting the importance of real-world testing 12m27s.
• The effectiveness of AI models can be influenced by various factors, including custom data sets, internal use cases, and prompting styles, which can lead to unexpected results when deployed in the real world 12m58s.

Evaluating and Improving AI Models

• Current AI models are not limited by their intelligence but rather by their evaluation methods, and they can be taught to perform better on a wider range of tasks with proper training and evaluation 13m13s.
• The lack of evaluation methods has hindered the development of AI models, and it's essential to establish clear success metrics to improve their performance 13m47s.
• The problem often solved involves determining what success looks like for a task and iteratively improving it, with tools like Claud able to automate evaluations and grading, but requiring input on what success entails 13m51s.
• At Anthropic, the interview process involves making candidates improve a prompt from a "crappy eval" to a good one, showcasing their thought process, as the company believes writing evaluations is a crucial skill for product managers (PMs) 14m23s.
• There is a lack of talent with this skill, and the company is trying to teach people how to write evaluations, considering it a core skill for PMs 14m31s.

The Changing Role of Product Managers in the Age of AI

• The job of a PM in 2024-2025, building AI-powered features, is looking more like the role of research PMs, who work on model capabilities and development, rather than product surface PMs or API PMs 14m55s.
• The quality of a feature is now gated on how well PMs have done evaluations and prompts, making the PM definition more meritorious 15m20s.
• Anthropic set up a boot camp to teach PMs how to write evaluations and the difference between good and bad evaluations, but still needs to iterate and improve 15m29s.
• To develop intuition for getting good at evaluations and iteration, one can use the models themselves, asking for sample evaluations and learning from the results 16m3s.
• Looking at data and examining cases where models fail is also crucial, as it can reveal issues with the grader rather than the model itself 16m27s.

Challenges and Evolution of AI Model Evaluation

• Every model release has a model card, and some model evaluations have shown that even the golden evaluations can be improved 16m47s.
• Evaluating the performance of AI models is challenging, and even grading them is difficult, so it's essential to look at the actual answers and evolve the evaluation methods as the models improve 16m53s.
• As AI models move towards longer-form and more agentic tasks, evaluation will become more nuanced and personalized, requiring a softer grading approach 17m18s.
• The concept of capabilities in AI models may evolve to resemble a career ladder, with evaluation resembling performance reviews that assess whether the model meets or exceeds expectations 18m8s.
• The increasing ability of AI models to beat humans at certain tasks raises questions about the role of humans in writing evaluations and the potential need for new evaluation methods 18m36s.

Skills and Adaptation for Working with AI

• To effectively work with AI models, product people should learn to write evaluations that assess the models' skills and abilities 18m52s.
• Prototyping with AI models is an underused skill that can be useful for quickly testing and evaluating different ideas and approaches 19m1s.
• The use of AI models will push product managers to go deeper into the tech stack and develop a deeper understanding of the technology 19m46s.
• The skills required to work with AI models will continue to evolve over time, and product people should be prepared to adapt and learn new skills 19m51s.
• Product managers (PMs) do not need to be researchers, but having an appreciation for and understanding of how AI works can be beneficial in building products that utilize AI 20m4s.

Building Products with Stochastic AI Systems and User Feedback

• AI systems are stochastic and non-deterministic, making it challenging to design products where the outcome is not entirely predictable 20m22s.
• To address this challenge, PMs need to establish feedback mechanisms to understand when the model is not working as intended and collect feedback rapidly 20m35s.
• This requires a different set of skills, as the traditional bug report is no longer applicable, and PMs need to understand the output of the AI across multiple outputs and users 20m51s.
• Adapting to non-deterministic user interfaces is a new challenge, and even tech-savvy individuals are still adjusting to this new paradigm 21m8s.

User Research and Adapting to AI-Powered Products

• Building products with AI requires considering the user's perspective and understanding how they will interact with the product, which can have both positive and negative consequences 21m44s.
• Conducting user research is essential in understanding how users interact with AI-powered products, and it can be surprising to see how users react to new features and the model's output 22m11s.
• PMs need to be prepared to let go of control and be flexible when working with AI, as the outcome is not always predictable 22m40s.
• The development of AI products is happening rapidly, and PMs and technical people need to develop intuition for how to use them effectively 22m55s.

Rapid Technological Advancements and User Education

• The rapid advancement of technology, such as ChatPT, has led to a situation where people quickly adapt to new innovations, and what was once considered "magic" becomes outdated in a short period, with the current state of technology expected to be seen as inferior in 12 months 24m24s.
• The speed of adaptation is also influenced by people's excitement and understanding that the world is moving in the direction of technological advancements, making it essential to make the best possible progress 24m43s.
• To address the challenge of educating end-users at scale, efforts are being made to make products more educational, such as providing information about the product itself and its features, which was not done initially 24m55s.
• User research has shown that people want to know how to use the product, and providing clear instructions and documentation can help solve UI problems and user confusion 25m14s.

Educating Users in Enterprise Settings and Empowering Power Users

• The approach to educating users is different in an Enterprise setting, where there is a status quo for how things are done, and organizational processes need to be considered when introducing productivity improvements or new technologies 25m47s.
• In the Enterprise context, power users are often early adopters who are familiar with technology, but there is also a long tail of users who may need more guidance and education on how to use new products and features effectively 26m8s.
• Non-technical users are being exposed to chat-powered LLMs for the first time, and it's essential to learn from these experiences to teach the next 100 million people how to use these UIs effectively 26m16s.
• Power users within organizations are creating custom GPTs to make AI more accessible and valuable for those who might not know how to use it otherwise, and these power users can act as evangelists 26m50s.
• The organizations mentioned are comprised of power users who are living in a "pocket of the future" and are finding innovative ways to utilize AI 27m16s.

Internal Use Cases of AI within Organizations

• Internally, the organizations are using AI to automate tasks, such as ordering pizzas, and are exploring various use cases, including UI testing and data manipulation 27m28s.
• AI is being used for UI testing, which is typically challenging and brittle, but early signs indicate that it works well for testing whether a UI functions as intended 28m12s.
• The organizations are also exploring the use of AI for agentic tasks that involve data manipulation, such as automating repetitive tasks and filling out forms 28m38s.
• The goal is to automate "drudgery" tasks, allowing humans to focus on more creative and high-value tasks 28m55s.

Workflows and Orchestration between Models

• Many sophisticated customers and internal teams are experimenting with workflows and orchestration between models, utilizing each model for its strengths, such as reasoning, but also acknowledging limitations like time to think and multimodality 29m9s.
• Reasoning in this context refers to the ability to form hypotheses, refute or affirm them, and continue reasoning, similar to how humans solve complex problems or make scientific breakthroughs 29m52s.

Scaling Intelligence and the Evolution of Reasoning in AI Models

• The concept of scaling pre-training is well-known, where models like GPT2, 3, 4, and 5 are trained on increasingly larger datasets, resulting in smarter models, but with limitations, such as system one thinking, which provides immediate answers without much thought 29m59s.
• In contrast, the new approach to scaling intelligence, as seen in models like 01, involves doing it at query time, allowing the model to pause, think, and reason before providing an answer, similar to human problem-solving 30m52s.
• This new approach has the potential to revolutionize problem-solving, as models can think for extended periods, refining their answers, and can be used in various applications, including cybersecurity, where models can be fine-tuned to work together to achieve precise results 31m43s.
• The use of models in concert with each other enables them to check each other's outputs, ensuring more accurate results, and can be applied to various tasks, such as finding and fine-tuning models to be good at specific tasks 32m32s.
• The current state of this new approach to scaling intelligence is still in its early stages, similar to the GPT1 phase, but it has the potential to significantly impact various fields and applications 31m53s.
• Models will be able to realize when something doesn't make sense and ask to try again, providing more value in specific use cases and orchestrations of models working together to accomplish complex tasks 32m40s.

The Future of AI: Proactive, Asynchronous, and Interactive Models

• The future of AI may involve models becoming more proactive, such as monitoring emails and spotting interesting trends to provide users with proactive recaps and research 33m47s.
• Another aspect of the future of AI is being more asynchronous, allowing users to expand their time horizon and not expect immediate answers, enabling them to work on other tasks while the model is processing 34m20s.
• This asynchronicity will allow users to ask more complex questions and tasks, such as fleshing out a mini project plan, fixing bugs, or adapting a product requirement document (PRD) for new market conditions 35m4s.
• The models are expected to get smarter at an accelerating rate, which will contribute to the development of these capabilities 35m28s.
• A key aspect of the future of AI is seeing models interact in various ways, enabling more complex and powerful applications 35m35s.

Advancements in Voice Mode and Natural Interactions with AI

• Humans interact with AI systems mostly through typing, but advancements in voice mode are changing this, allowing for more natural interactions like speaking and seeing, with the potential to become commonplace fast 35m42s.
• The launch of advanced voice mode has enabled users to have conversations with people who speak different languages, acting as a universal translator, and has the potential to increase people's willingness to travel to new places 35m54s.
• The combination of voice mode and other AI capabilities is creating new experiences, such as young people using voice mode to pour their hearts out and interact with AI in ways that are becoming increasingly natural 37m0s.
• The digitally native generation is growing up with the expectation that AI will be able to understand and interact with them in various ways, including voice conversations 37m17s.
• Children as young as 5 and 7 are already interacting with AI systems like ChatGPT, asking it bizarre questions and having weird conversations, and are perfectly happy talking to an AI 37m35s.
• Children are also using AI to create their own entertainment, such as telling stories and asking AI to create images in real-time, showcasing a new way of creating and interacting with content 38m14s.

Developing Empathy and Understanding Nuances in AI Interactions

• The most surprising behavior seen in AI products recently is the development of a nuanced understanding of the AI model and its capabilities, with users forming a kind of two-way empathy and befriending the AI 38m38s.
• Users are also noticing differences in the behavior of new AI models, such as feeling smarter but more distant, and are developing a relationship with the AI based on its nuances 39m0s.
• Developing AI products requires empathy, as they involve shipping intelligence and empathy, which are key components of interpersonal relationships, making it essential to consider how users will adapt to and interact with these products 39m9s.

The Personality of AI Models and User Preferences

• The personality of AI models is crucial, and there are interesting questions around how much they should customize versus having a single personality, such as OpenAI and Claude having distinct personalities 39m40s.
• Users may choose to use one AI model over another based on their personality, which is a human-like preference, as people tend to be friends with those they like and have an affinity for 40m1s.
• A recent experiment where an AI model described users based on their past interactions went viral on Twitter, showcasing how people are starting to interact with AI models in a more personal and human-like way 40m13s.
• This experiment demonstrated how AI models can be seen as entities that users can interact with, and their reactions to these interactions can be fascinating and provide valuable insights 40m39s.

Conclusion: Perspectives on the Future of AI

• Kevin Weil and Mike Krieger shared their perspectives on the future of AI and its development, providing a glimpse into the potential of these technologies 40m46s.