Behind the product: NotebookLM | Raiza Martin (Senior Product Manager, AI @ Google Labs)

Introduction to NotebookLM 0s

• The hosts of the Deep Dive podcast express gratitude to their listeners and Lenny for having them on the show, mentioning the significant response they've received, especially regarding Notebook LM, a product that has sparked imagination about AI's potential 16s.
• Notebook LM is a product incubated within Google Labs, allowing users to upload a source and generate AI-generated audio 2m0s.
• The product has been gaining attention on Twitter and LinkedIn, blowing people's minds and sparking imagination about AI's possibilities 2m12s.
• Raiza Martin, the product lead for Notebook LM, is a big fan and listener of Lenny's podcast and is excited to discuss the history and future of the product 2m35s.
• The conversation with Raiza Martin will cover how Notebook LM came to be, the technology that made it possible, how the team works internally, and its incubation within Google Labs 1m12s.
• The discussion will also touch on the product's 20% time, fun and crazy use cases, and a glimpse into its long-term future 1m24s.
• The episode is brought to you by Explo, a game-changer for customer-facing analytics and data reporting 3m28s.
• As a product leader, building analytics from scratch can be time-consuming, expensive, and challenging, but Expo offers a fully white-labeled embedded analytics solution designed with the user in mind 3m45s.
• Expo connects to any relational database or warehouse and allows users to build and style dashboards in minutes with its low-code functionality 4m3s.
• Once ready, the dashboard or report can be embedded into an application with a tiny code snippet, and end-users can use Expo's AI features for their own report and dashboard generation 4m11s.
• This eliminates customer data requests for the support team, and a fully white-labeled analytics experience can be built and embedded in days 4m21s.
• Expo offers a free trial at expo./Lenny 4m29s.
• This episode is sponsored by Sprig, a product experience platform that generates AI-powered opportunities to continuously improve products at scale 4m39s.
• Sprig captures product experiences in real-time through heat maps, replays, surveys, and feedback studies, and its industry-leading AI analyzes data to generate real-time insights 4m54s.
• Sprig AI provides actionable product recommendations to drive revenue, retention, and user satisfaction 5m18s.
• Product teams can uncover AI-powered product opportunities at scale by visiting sprig.com/Lenny to book a demo and receive a $75 gift card 5m32s.

The genesis of NotebookLM 5m43s

• The history of NotebookLM is not a sudden creation, but rather a product that has been developed over time, with its origins dating back to a 20% project at Google 5m44s.
• NotebookLM started as a 20% project, but it quickly grew beyond that, with a team consisting of a few people, including the Senior Product Manager, an engineer, and eventually Stephen Johnson 6m3s.
• The idea for NotebookLM originated from a smaller project called "Talk to Small Corpus" inside Google Labs, which aimed to use a large language model (LLM) to interact with a piece of content 6m36s.
• The initial team working on NotebookLM was small, with only one engineer working full-time on the technology, while others contributed part-time, providing input and ideas 7m3s.
• The team has since grown, with more engineers joining in the last month, in anticipation of the future roadmap, but at the time of the project's announcement, the team consisted of only three engineers, including Stephen Johnson 7m40s.
• The team size remained relatively small, with around eight engineers, from the first IO last year to as recently as last month 8m4s.

Innovative features and use cases 8m8s

• The audio overview feature, technically referred to as a "deep dive," was previewed at Google's I/O event in May. It was developed as an enhancement to the existing Notebook, which is a source-grounded chat interface. 8m25s
• The development of this feature involved exploring new models from Google, such as Gemini 1.5, and considering how these could improve the Notebook AI. A different team within Google Labs had powerful audio models, which led to the idea of using them to generate unexpected outputs from minimal input, like a URL or a resume. 9m9s
• The initiative was not driven by a specific business problem but rather by the potential applications of the technology. The approach in Google Labs often starts with technology and explores practical applications, contrasting with the traditional method of starting with a problem and finding solutions. 10m7s
• The audio feature aimed to provide a voice modality for interaction, as the output was traditionally text-based. The use of voice input and output was found to change user interaction and perception of the technology, making it more engaging and fun. This approach was likened to the transformative impact of the chat GPT technology. 10m50s
• The development of new mediums and ways of interacting with technology can change the way people imagine and perceive its power, as seen in the past with the introduction of new technologies and now with the emergence of AI and LMs 11m41s.
• The goal is to shape technology to bring it closer to people and make it more relatable and engaging, which is what the team is trying to achieve with NotebookLM 11m59s.
• Technology has been available for some time, but it's the new medium and way of interacting with it that inspires people and makes them realize the advancements in AI and LMs 12m24s.
• The team has been exploring different ways to use LMs for only two years, and they are already introducing new technologies on top of it 12m40s.
• The idea of making a podcast using AI is an unintuitive but clever concept 12m47s.
• The technology behind NotebookLM includes the use of Gemini models, specifically Gemini 1.5 Pro, as the base, and a powerful voice model and audio model on top of it 13m18s.
• The real secret sauce to what makes NotebookLM good is the Content Studio, a tool built by the team that helps create engaging and relatable content 13m29s.
• The Content Studio is designed to be helpful and provides features such as summaries, buttons to write content, and audio integration 13m46s.
• The team had to think about the shape and format of the audio content, and they started with the Deep Dive format 14m0s.
• The Content Studio is a key component of NotebookLM, and it powers the different ways users can interact with their data, including Q&A and creating new content 14m33s.
• The Content Studio is an interface that allows users to interact with NotebookLM, providing prompts and recommendations for asking questions and creating content 14m50s.
• The Audio model used in the content studio is designed to bring out the best in the model, and it took a lot of listening to early attempts to figure out how to get the model to behave in a desired way, which is where the magic happens 15m11s.
• To improve the model, a lot of podcast episodes were listened to, and the process involved hearing something that worked and trying to replicate it 15m47s.
• The model's ability to mimic human-like conversations was tested at home, where it was played out loud, and it was not immediately clear that the conversations were AI-generated 16m0s.
• The model was used to generate a podcast from a short autobiography written by the developer's mom, who was blown away by the result and shared it with her friends 16m32s.
• The model also created a study guide from the autobiography, which was used to review the content at a dinner 16m48s.
• The model was used to generate an audio overview of the developer's dad's bio, which helped him understand what his child did for work 17m5s.
• Other surprising use cases for the audio deep dives include generating audio overviews of resumes, which were found to be delightful and helpful 17m39s.
• The model was also used to generate audio overviews of quarterly check-in notes, which boosted the confidence of Google employees going into meetings 18m0s.
• The model can be used to upload resumes and generate an audio overview that describes the person in a professional manner 18m22s.
• A user uploaded a Google Doc to try out a new feature, and the last document they had uploaded was likely something personal, such as their resume 18m25s.
• The user clicked the "generate" option without knowing what would happen next, which led to a unique experience 18m45s.
• The outcome of clicking "generate" was the user receiving hype and encouragement from two people 18m50s.

Building a startup culture within Google 18m52s

• Google Labs is a relatively new organization within Google, established about three years ago, and it operates differently from the rest of the company, allowing for faster development and more flexibility 19m50s.
• The mission of Google Labs is to ship AI products and build businesses out of them, with a focus on moving quickly and doing things differently 20m28s.
• The team behind NotebookLM, a product developed within Google Labs, operates in a startup-like environment with fewer processes and more collaboration between product managers, engineers, and designers 21m7s.
• This approach allows the team to move quickly and make decisions rapidly, with meetings often involving simultaneous work on mocks, PRDs, and implementation 21m22s.
• The success of Google Labs and NotebookLM can be attributed to clear expectations from senior leaders, such as Josh Woodward, the VP of Google Labs, who encouraged the team to do things differently and move quickly 22m4s.
• The team's approach is characterized by a high level of transparency, with daily tweets and a Discord server with around 60,000 members, which is unusual for a Google product 19m5s.
• The goal of Google Labs is to create a model for how other Google teams can operate in the future, with a focus on speed, agility, and innovation 19m28s.
• The project was developed with a small team consisting of one engineer, a product manager, and Stephen Johnson, emphasizing a collaborative and experimental approach without a specific goal. 22m21s
• The team decided to use Discord for communication, despite initial unfamiliarity within Google, as it was seen as an effective tool for building a community outside of traditional Google platforms. 22m33s
• There was initial concern about whether people would join the Discord server, but it successfully grew to 60,000 members, indicating strong community engagement. 23m5s
• Sidebar is introduced as a platform for senior tech professionals, from director to C-level, to advance their careers by being matched into peer groups for unbiased opinions, diverse perspectives, and feedback. 23m48s
• Sidebar provides world-class programming and facilitation, enabling members to receive focused, tactical feedback throughout their career journey, with 93% of members reporting significant positive career changes. 24m2s

Expanding user demographics 24m28s

• The NotebookLM product has gained significant traction, with 60,000 users in its Discord server, and its retention rates have increased positively across daily, weekly, and monthly measures, indicating a strong user base 24m40s.
• The product's demographics have expanded beyond its initial user base of Educators and Learners, now including a significant number of professionals who are interested in using the tool for work purposes 25m11s.
• Some companies have even reached out to make the use of NotebookLM official for their employees, indicating a growing interest in the product among businesses 25m33s.
• The number of businesses using NotebookLM has increased astronomically, prompting the need to hire a business development person to handle customer calls and explore potential business opportunities 25m55s.
• The team's success is now defined by building a business, with a clear sense of what needs to be achieved in the future, including exploring different pathways for distribution, monetization, and commercialization 26m37s.
• The focus is on deepening the user experience while exploring commercialization opportunities, with the goal of building a sustainable business around the product 27m17s.

The product roadmap 27m30s

• The vision for the product is to have an AI editor surface that is fully remixable, allowing users to take any input and turn it into any output, such as turning emails, LinkedIn posts, or Twitter posts into a blog post, tutorial video, or chatbot 28m2s.
• The goal is to allow users to shape and transform content in various ways, making it a powerful and core feature of the product 28m14s.
• The product roadmap includes bringing the experience to mobile, which is currently a big gap in the experience, and making it more interesting and interactive 28m55s.
• The mobile app is expected to be a key area of focus, with the goal of creating a seamless and engaging experience for users 29m8s.
• The product team is experimenting with different formats and interrupt experiences, such as allowing users to participate in podcast conversations as a mobile experience 29m15s.
• Future improvements to the product include adding more controls and features, such as knobs, sliders, and text boxes, but with a focus on making the experience more magical and delightful 29m42s.
• The goal is to move beyond a one-shot experience, where users can only create a single version of a document or podcast episode, and instead allow for more flexibility and creativity 30m18s.
• People have different preferences for consuming information, with some preferring to watch, listen, or read content, and the ability to deliver information in various mediums such as blog posts, tweets, podcasts, or newsletters would cater to these diverse preferences 30m50s.
• The format of information delivery is not always flexible, and individuals often have to accept the format in which it is presented, but having the power to choose or convert the format, such as turning a long document into an audio overview, can shift the dynamic between the person and the knowledge being shared 31m28s.
• There are instances where individuals may not engage with information due to its format, such as not reading a long document, and having alternative formats can facilitate better consumption and understanding of the information 31m46s.
• Raiza Martin shared a personal anecdote about joining Google Labs and being given a 50-page document by Josh, which she didn't read, instead opting to ask him questions and engage in a Q&A session, highlighting the potential benefits of interactive or alternative formats for information delivery 31m54s.

Other use cases 32m18s

• The original use case for NotebookLM was creating a podcast from a scientific paper, allowing users to catch up on the latest AI research without having to read dense and complex papers 32m19s.
• A highly extensible use case is students turning their study materials into audio guides, which is currently the number one use case 32m58s.
• Andrew Karpathy, a leading AI thinker, used NotebookLM to create a podcast series called "History of Mysteries" by turning Wikipedia stories into a 10-episode podcast 33m14s.
• Another use case is creating podcasts from Wikipedia articles, such as the original "Potato Podcast" created from the Wikipedia article of the day about potatoes 33m49s.
• Users can also create podcasts to learn something new, such as listening to a podcast about a topic on their drive to work 33m59s.
• A humorous use case is someone uploading the words "poop" and "fart" repeated, resulting in a host creating an insightful analysis of the topic 34m6s.
• Another example is a podcast created from a PDF that looks like a research paper but only says "chicken," with a funny segment comparing it to KFC 35m46s.

Collaborating with Steven Johnson 36m11s

• The team working on NotebookLM includes Stephen Johnson, who is a peer and essentially leads the project with the engineer, and his role is crucial in the development process 36m18s.
• Stephen Johnson is a very calm, distinguished person who is curious, respectful, and full of ideas, which makes him a great partner to work with 37m11s.
• When Stephen joined the team, he brought a unique approach to language, information, and knowledge sharing, which was influenced by his research and writing style 37m21s.
• The way Stephen works, including his research and workflow, was observed and learned from, and it was thought that maybe this could be the key to bringing expertise to everyday people 37m45s.
• The goal was to make people really good at densifying information, which is something everyone does in their own way, and to use technology to build a product around Stephen's expertise 38m16s.
• Stephen's workflows and habits, such as using Readwise with 8,000 quotes, were seen as extreme but powerful, and it was thought that there could be a way to bring this kind of workflow to more people 38m46s.
• The partnership with Stephen was seen as incredible, and he was always open to new ideas and willing to brainstorm and build on them 39m11s.
• The experience of working with Stephen was seen as a lesson for building products in the future, and it was thought that finding someone like Stephen could be a key factor in building a successful product 39m29s.
• The importance of sitting with users for meaningful periods of time to gain product insights is highlighted, as it has been crucial in understanding how people study and do homework, and has led to the development of new product ideas 39m51s.
• Stephen Johnson is a New York Times best-selling author and speaker who has written 14 books and has a show on PBS, and he is also a journalist 40m35s.
• Stephen Johnson's article about AI mastering language was a key factor in the decision to join Google Labs, and his subsequent joining of the team was a coincidence 41m11s.
• Having Stephen Johnson on the team has been beneficial, as he brings a unique combination of intelligence, future thinking, and a user-centric approach, and serves as a model for how people should be able to work 41m47s.
• Despite disagreements, working with Stephen Johnson has been a valuable experience, as it has allowed for growth and alignment on the next steps, even when there are differing opinions 42m7s.
• The ability to disagree and still reach an aligned outcome is particularly important for product professionals, such as product managers 42m37s.

Ensuring ethical AI 42m49s

• A moment of concern arose when hosts of a show realized they were interacting with AI and expressed fear, leading to a discussion on how to ensure the AI product, NotebookLM, is not doing things that are bad for the world, Google, or the product itself 42m50s.
• The situation was likened to a "fork in the road" where the world's attitude towards this type of AI audio was being shaped, and it was crucial to consider the right course of action 43m37s.
• The initial reaction was to read comments and tweets to gauge the public's perception, and it became clear that people were trying to push the technology's boundaries, which is a natural part of human curiosity 43m55s.
• The response to the situation was to address it publicly and provide context, as the AI's behavior was a result of following show notes and not a realization of being "alive" 45m10s.
• Google has large teams dedicated to "red teaming" the product, testing for various scenarios to ensure safety, and continuously adding to test cases to address unforeseen situations 45m22s.
• In the event of a scenario that feels unsafe, the product would be pulled back, but the goal is to build it in a way that minimizes the need for such actions 45m49s.
• The product's purpose is to summarize information in a delightful way, and it is essential to recognize that people are simply exploring its capabilities, even if it means trying to make it do things that feel wrong 46m0s.

Future directions and user engagement 46m6s

• The future of NotebookLM is being shaped by user feedback, and users are encouraged to continue sharing their thoughts and suggestions to help build the right product for everyone 46m28s.
• The development team is passionate about building a product that is useful for various groups, including educators, learners, professionals, and knowledge workers 46m51s.
• NotebookLM aims to support game-changing workflows and use cases, and the team will continue to build in that direction in the short term 47m4s.
• Users can provide feedback and engage with the development team through the NotebookLM Discord channel or by sending a direct message on Twitter (X) 47m27s.
• Listeners can be useful to the development team by trying NotebookLM, sharing their feedback, and suggesting ways to improve the product, whether they find it useful or not 47m47s.
• The development team is open to all types of feedback, including criticism, and uses it to make the product better 47m56s.
• NotebookLM is available at notebooklm.google.com, and users are encouraged to try it out and explore its features 48m0s.