Open Models vs Frontier Models: Who Actually Wins? | The $100K Token Budget Every Engineer Will Need

Sierra's Business Model and Market Position

• The demand for frontier levels of intelligence is unbounded, and part of the driver of this difference is the willingness of Chinese companies to do scaled distillation of frontier models, with companies like Sierra, which has raised over $1.5 billion and is valued at almost $16 billion, working with 40% of the Fortune 50 10s.
• Sierra's approach is to distill frontier models and offer them up, as not every company can build these models themselves, and the company has changed its engineering interview process to focus on ready-to-go candidates, with some of the most effective employees being 22 or 23 years old and having been completely AIP 2m6s.
• Clay Pavore, co-founder of Sierra, has a background of 18 years at Google, where he met his co-founder Brett in the associate product management program, and they stayed in touch through a monthly poker group, with Brett trying to hire or start a company with Clay several times before Sierra 4m30s.
• Clay's decision to leave Google and start Sierra was due to the alignment of the planets, with language models becoming a key technology and the right timing for a new company, and he took with him the experience and knowledge gained from his time at Google, including working with extraordinary people and having managers who took bets on him 6m40s.
• The conversation also touches on Clay's personal background, including starting a modest company at the age of 13 and always wanting to start another company, and the importance of finding the right co-founder, with Brett being excellent in competence and character, and the right timing for starting a new venture 8m20s.
• The discussion highlights the significant growth and valuation of Sierra, as well as its work with major companies, and the company's approach to building and distilling frontier models, with Clay's experience and insights providing a unique perspective on the industry and the future of AI 10m30s.

Company Scale and Technology Stack Decisions

• The scale of a company, whether it is a small enterprise software company or a large corporation like Google with roughly 150,000 people, can significantly impact its approach to technology and product development 10s.
• A key takeaway from experience is the willingness to invest as far down the technology stack as needed to build the desired service and product, which is evident in Google's early days of building its own data centers, cluster architectures, and using commodity hardware 42s.
• The company realized the potential of language models and agents as early as April 2023 and decided to invent frameworks for building these things from scratch, hiring a founding head of research who literally wrote the paper on language model-based agents 1m30s.
• The decision was made not to train their own models, due to the high capital expense and ongoing costs, and instead to use open weights models and fine-tune them for specific company needs, allowing the company to slipstream behind the investments made by labs and hyperscalers 2m6s.
• The company has a set of proprietary fine-tune models built on top of open weights models, but does not go all the way down to mega cluster training runs, and believes in being in control of its own destiny without feeling the need to go further than necessary 3m20s.

Demand for Frontier Models and Open Models

• The future may be open models fine-tuned to specific company needs, but the demand for frontier levels of intelligence is unbounded, and while not every domain requires it, having access to such intelligence can be highly beneficial 5m10s.
• The cost of training frontier models is a significant barrier, with the capital expense and ongoing costs being too high for all but a small number of companies, making it important for companies to appreciate the value of open models and fine-tuning 4m40s.
• The demand for greater levels of intelligence will be unbounded in certain domains, such as coding, science, material science, and law, where the stakes are high and complexity is great, and frontier models will be needed to meet this demand 10s.
• As open models become more advanced, they will be able to perform a larger set of tasks, and companies will use a combination of open and frontier models, depending on the task at hand, with open models being used for tasks where they are sufficient and less expensive 42s.
• The majority of enterprise tasks cannot be automated today, and this is due to a combination of factors, including a model gap, a technology diffusion gap, and an application layer gap 2m6s.
• The set of tasks that can be performed by open models will continue to grow as they become more capable, but the demand for frontier intelligence will remain high, driven by the need for invention, discovery, and the creation of new products and services 2m6s.

Token Economics and Compute Constraints

• The evolution of token costs will be driven by several factors, including the development of new hardware that can produce more tokens at equivalent cost, the migration of certain workloads to open weights models, and the increasing use of reasoning models that drive up token use 2m6s.
• The use of reasoning models, such as the 01 model from OpenAI, which can think out loud to themselves and demonstrate logarithmic improvements in performance with increased compute time, will also impact token economics 2m6s.
• The availability of compute is a key driver that is hard to predict how it will play out across both open weights models and frontier models, with the rate limiter being the number of available GPUs, which affects the cost of tokens due to energy and compute costs 10s.
• The demand for frontier-level intelligence or GPUs to run open weights models is unbounded, and even if supply is increased, it can still be sold out quickly, as evidenced by the founder of Nebius stating that if they 10x their supply, they could still sell out in a day 42s.
• Open weights models may be cheaper because they avoid some of the margin stack in hosted frontier models, but the fundamental input of GPU capacity and power is still constrained 1m6s.
• Running models locally, such as on a cluster of Mac minis or on-device on phones, could slightly alleviate the constraint, but it is unlikely to alleviate the server-side challenge due to thermal limits and the need for large amounts of compute 2m6s.

Global AI Ecosystem and Competitive Landscape

• Companies like Open AI, Anthropic, and Google are dominant leaders in the US, but Chinese models are seen as advanced and impressive, and the US has a challenging open ecosystem, with Chinese companies willing to do scaled distillation of frontier models from US labs 4m10s.
• The difference in the open ecosystem between the US and China may be driven by the willingness of Chinese companies to distill frontier models, and US-based labs and hyperscalers may not release open weights models that compete with their own frontier models 6m0s.

Team Structure and Productivity in the AI Era

• The concept of enterprise being a team sport and the importance of having advanced software engineers internally is highlighted, with examples such as Lovable announcing a significant milestone with a small team 8m0s.
• The future of teams is expected to involve smaller, higher-leverage teams, with software engineers becoming more productive due to the use of AI tools, estimating a productivity gain of between three and 20 times, with this trend expected to touch all parts of every company 10s.
• Companies like Sierra, which serve large enterprises, including 40% of the Fortune 50, will still require larger teams to sell and implement their products and solutions successfully, as this process involves deeply understanding customers' business outcomes and objectives, as well as building relationships and earning trust 42s.

Internal AI Tools and Productivity Enhancements

• Sierra has developed an internal agent called Pine Cone, which is a purpose-built harness that incorporates the MCP gateway, allowing employees to access and interrogate the entirety of the company's information, making decisions, and getting things done more efficiently, with engineers using Pine Cone being phenomenally productive 2m6s.
• The MCP gateway is a single server that aggregates all of the main systems and services used to run the company, providing full access to these systems with the necessary permissions, and Pine Cone has a whole harness around the engineering of the platform, agent architecture, and agent studio, speeding up software development 2m6s.
• The use of Pine Cone and other AI tools is expected to change the way companies operate, with employees having access to a shared library of skills that can be built and deployed, including private skills, and the company having a whole bunch of skills, including one that is a clay scanner of interview packets 2m6s.
• The company uses a tool called Pine Cone to review and approve hires, which has become an indispensable tool for running the company, and is also working on other projects such as Sierra Brain, a strategy thought partner that knows the company inside and out 10s.

Strategic Planning and Company Direction

• Sierra Brain is a tool that starts with a 20 or 30 page document that grounds any agent in what the company is, what it does, and its strengths and weaknesses, and is used to reason about what the company should be doing 2m6s.
• The company is considering how to approach token budgeting for its developers, with top engineers spending over $100,000 on tokens per year, and is thinking about allocating a token budget on a per employee basis 42s.
• The direction the company is headed is towards token budgeting, where capital allocation will look like allocating opex, including headcount, salaries, and tokens associated with headcount, with the goal of learning quickly and seeing what works 42s.
• The company is not yet at the point of allocating a specific token budget, but is preferring to learn quickly and see what works, and is interested in understanding what percent of developer salary will be spent on tokens in the future 42s.

Token Budgeting and Developer Economics

• The percentage of developer salary spent on tokens is expected to be around 3.8% based on larger companies' spending, but if it goes up to 20%, some companies may be undervalued, highlighting the need to understand the dynamics of token spending and its impact on company valuation 2m6s.
• The idea that companies will spend 3.8% of their budget on tokens is considered to be wildly off, with a more likely steady state being around 20%, which would make a $100,000 token budget per year seem normal for a great developer in the valley who earns around $500,000 per year 10s.
• The gains from using tokens in software engineering seem unequivocally there, with the potential to double the size of an engineering team, and the debate is more about whether the gain is 2x, 10x, or 20x 1m42s.

Customer Experience and Market Positioning

• Retaining a real product focus and closeness to customers is not necessarily difficult for large enterprise companies, as evidenced by the fact that the company's founders are constantly building agents themselves and are in the products, allowing them to stay close to the end experience 4m6s.
• The company is serving hundreds of millions of interactions and will soon be serving billions, which will require them to stay close to the end experience and prioritize voice fluency, latency, and the quality of the experience 6m30s.
• The market for AI-powered customer service is highly competitive, with many startups and incumbents like Salesforce, Atlassian, and Zenesk, but customers are voting with their feet and choosing the company's products, which is growing faster than its competitors 8m40s.
• The market is expected to evolve significantly over the next 5 to 10 years, but it's hard to predict whether it will be like an Uber-Lyft market or an AWS-Google Cloud-Azure market 11m30s.

Enterprise Sales and Deployment Strategies

• The concept of economies of scale in terms of depth and breadth of platform, experience in specific industry verticals, and other factors is expected to have a significant impact, potentially leading to a market similar to Uber, with the company poised to be a major player 10s.
• Selling to large enterprises requires a specific approach, and having a forward-deployed team can be beneficial in helping customers achieve impact quickly, although it is not necessarily a requirement, as evidenced by the company's experience working with 40 of the top 50 enterprises 2m6s.
• The company's approach to selling and supporting its product involves working closely with customers, including embedding engineers within customer companies, which has been successful in helping customers like Next and Sigma achieve rapid deployment and time-to-market 2m6s.
• The use of a forward-deployed team has enabled the company to take companies live with their product in a short amount of time, such as six weeks for Next and 58 days for Sigma, and has been an important catalyst for achieving impact quickly 2m6s.

Market Demand and Expansion

• The current market demand for AI products is unique, with effectively unbounded demand in areas such as coding agents and the company's category-leading space, driven by buyer pull and the fact that every CEO is being asked by their board about their AI strategy 42s.
• The company has experienced rapid growth, including expanding to 100 people in Europe and acquiring a company in Japan, Opera Technologies, in order to meet the high demand for its products and services 42s.
• The concept of omotanashi, or extreme hospitality, is expected in Japanese service, and this is what is intended to be built, with a team that can be attuned to the cultural nuances of Japan 10s.

Company Vision and Vertical Expansion

• To scale into the desired company, it is necessary to move out of customer support into complete life cycle management, and the direction of the company can be indicated by the example of Rocket, which begins with search and discovery of a home and includes helping customers with loan servicing 2m6s.
• The company's approach is not just limited to service and support, but also includes sales, such as personalized product recommendations, and building solutions that fit the needs of customers, similar to a Fortune 50 or Fortune 500 company, but more consumerized 2m6s.
• The company intends to be at a larger scale than Palanteer, with a focus on building real domain expertise and specific industry verticals, and applying lessons learned from customer deployments in a scaled way 4m30s.
• The company's approach to building products is to focus on commonality and strengthening the platform, but if a customer needs a unique solution, they will build it, especially if it's a large company, and this has become more feasible with the use of coding agents and the pace at which solutions can be built 6m40s.
• The company's goal is to build a platform that can be applied across customer bases, but also be able to build custom solutions for specific customers, and this is made possible by the ability to extend the platform in ways that may apply to a single customer 8m20s.

Board Governance and Company Culture

• The company has a unique approach to board meetings, with a six-week cadence that includes a three-hour meeting and a one and a half hour meeting, allowing them to adapt quickly to changes in the AI landscape 2m6s.
• Instead of using board decks, the company uses board memos, typically six to 10 pages long, written by the founders, Brett and the other founder, to clearly outline their thoughts and provide board members with time to review and prepare 4m42s.
• The board memos include an honest assessment of the company's performance, highlighting areas where they are doing well, but also areas where they can improve, and the board meetings are used to discuss and challenge these ideas 6m15s.
• The company prioritizes transparency and self-awareness, with the founders writing about their own weaknesses and areas for improvement, such as a memorable instance where they failed to hire quickly enough to meet demand 8m45s.
• When it comes to fundraising and discussing price with investors, the founders focus on determining the amount of capital needed to reach the next milestone in terms of revenue and company scale, rather than focusing on valuation 12m10s.

Fundraising and Capital Allocation

• The company has a strong team and can choose from a range of investors, but they aim to find a balance between raising enough capital to drive growth and avoiding overly high valuations 13m40s.
• The approach to funding is considered milestone to milestone, with sensitivity to dilution, and in every round, a lower price is taken than what could have been achieved 10s.

Company Values and Culture

• The company values craftsmanship, intensity, family, trust, and customer obsession, with craftsmanship and intensity being unique and important aspects of the company culture 42s.
• Craftsmanship is about doing things with excellence, and it is essential for building a great company, which is an aggregation of thousands of great things, including people, processes, products, and culture 1m6s.
• The company's focus on craftsmanship also extends to how they interact with customers, showing up with professionalism, care, and a high level of detail, which helps build trust with customers 2m6s.
• Intensity is about pace, winning, and being competitive in a giant market, where companies need to interact with customers via sophisticated agents, and it is essential to maintain intensity through hiring the right people and creating a culture that values excellence and pace 4m30s.
• The company looks for people who are smart, nice, and tense, which is a rare combination, and when found, it can create a fantastic work environment, with intensity being related to craftsmanship and doing things with excellence 6m20s.
• Intensity with scale is maintained by the founders, Brett and the other founder, who set the pace and are the examples of intensity, and this shows up in how they manage the company, constantly questioning if things can be done better and faster 10s.

Leadership and Decision-Making

• The founders determine what they should be involved in and what they shouldn't by using judgment and editing, focusing on things that won't happen or won't happen as quickly without their direct applied force, such as the next generation agent architecture 2m6s.
• Ambitious goals have a way of becoming self-fulfilling, and the founders try to be selective about where they engage at a high level, using techniques like suspending disbelief and imagining what would have to be true to achieve a certain goal, such as having a giant business in Japan 4m42s.
• The founders' values, including "third family", come directly from them, and they use a technique called "think apart think together" to initialize on a prompt and get the best of their independent thinking, which led to a shocking amount of overlap in their values 8m10s.

Work-Life Balance and Personal Values

• The value of "family" comes from the founders' personal experiences, with both of them having young kids and prioritizing their families, and they believe that it's possible to be part of a growing and intense work environment while still prioritizing family and personal relationships 10m40s.
• The importance of working smart and finding balance in life is emphasized, as it allows individuals to focus on things that matter beyond just work, and this balance is crucial for people with families, such as having four kids, which can be a lot to handle 10s.
• Despite having a large family, it is possible to work intensely and efficiently, making the most out of every hour, and having a complex networking setup can help maintain uninterrupted connectivity, allowing for maximum productivity 2m6s.

Team Building and In-Person Culture

• The value of in-person interactions, especially for young companies, is highlighted, as it facilitates building a culture, shared norms, camaraderie, and apprenticeship, which are essential for team sports like enterprise software development 4m42s.
• The concept of paying it forward and learning from experienced people is also stressed, and renowned computer scientist Richard Hamming's talk "You and Your Research" is referenced, which emphasizes the importance of finding great people to work with and learn from 6m15s.
• Hamming's talk also discusses how knowledge and hard work can have a compound interest effect, and it is essential for young people to focus on learning as much as they can as early as possible to change their life's trajectory 8m10s.

Learning and Mentorship

• Personal anecdotes are shared, including doing five shows a week and not having a thousand listeners per show for three years, highlighting the importance of perseverance and using opportunities as a method to learn from others 10m40s.
• The conversation concludes by acknowledging the challenges faced by young people today, who are uncertain about their place in the world and what to do, and the importance of learning from others and finding great people to work with 12m20s.

AI's Impact on the Workforce and Hiring

• The upcoming AI tsunami is expected to have significant implications on jobs, particularly entry-level positions, but young people with experience in AI tools are highly valued by companies, and many are being hired with no prior work experience, with some of the most effective employees being 22 or 23 years old 10s.
• The hiring process for engineers has changed to include an AI-native interview process, where candidates are given a prompt and $150 to spend on coding agents, and are allowed to use their own laptops and tools, with the goal of assessing their ability to build applications and think through problems 2m6s.

Cybersecurity and AI Ethics

• Cyber security is considered a crucial area of importance due to the proliferation of code generated by AI that may not be secure, and it is expected that AI tools will play a significant role in both offensive and defensive capabilities 4m42s.
• Disagreements between leaders, such as the one between Clay and Brett, are resolved through a truth-seeking approach, where they converge on a solution by interrogating each other's ideas and considering multiple perspectives, with the goal of finding the best approach 6m15s.

Leadership Structure and Roles

• The company's leadership structure is organized around "majors and minors" for each part of the company, with Brett's majors being sales and engineering, and this structure allows for a more flexible and collaborative approach to decision-making 8m30s.
• The individuals involved have a strong background in software engineering and sales, with one being exceptionally skilled at selling software and the other having a solid understanding of system design and architecture, with Brett having learned from his time at Salesforce under Mark's guidance 10s.
• The split of responsibilities between the two individuals is roughly based on their strengths, with one handling operations, finance, and legal matters, and the other focusing on product and sales, with a mutual trust in each other's judgment on their respective areas of expertise 42s.

Leadership Insights and Personal Growth

• A valuable lesson learned from working with Sundar is the ability to look at problems from different zoom levels, having a dynamic range of thinking that can zoom in and out of details, from high-level strategy to minute details, which is a skill worth emulating 2m6s.
• Google's success can be attributed to the alignment of an ambitious mission, incredibly smart people, and a culture that values truth and building in service of that mission, which allows the company to solve complex problems and drive invention and discovery 4m10s.

Entrepreneurship and Innovation

• A book recommendation for entrepreneurs and inventors is "The Wright Brothers" by David McCullough, which tells the story of the invention of the first heavier-than-air aircraft and serves as a portrait of entrepreneurship and invention, highlighting the importance of perseverance and networking 6m40s.
• Parenting is considered a great gift and a life-changing experience, and the advice given is to appreciate the privilege of having children and to be prepared for the changes and challenges that come with it, with the individual having personal experience as a parent of four kids 10m20s.

Family and Personal Life

• The fastest rate of change that anyone experiences in their life, after being born, is having their first child, and it is essential to carve out time for family and create rituals and discipline around making time and space, which can be achieved by having clear goals and good habits 10s.
• Making kids' interests one's own is crucial, as seen in the example of being terrible at basketball but learning about the sport to support and enjoy the child's interest, and this approach can be applied to each child to understand what gets them excited and engaged 2m6s.
• A great marriage is a partnership where both individuals work together in pursuit of shared goals, and having shared interests in what they are pursuing as a partnership is deeply important, but it is also possible to have different interests and still support each other in becoming the best version of themselves 4m42s.
• The kindest thing that anyone has ever done is the support and encouragement provided by parents, who saw the interest in computers and supported it, even when they did not fully understand it, and this support had a direct impact on future career choices and successes 10m10s.

Gratitude and Personal Reflections

• The importance of parents' support is highlighted by the example of going to Macworld and spending the day learning about new technologies, which was made possible by the parents' efforts to enable and push their child's interests, and this has led to a successful career, including 18 years at Google 12m20s.
• The guest is being complimented on being a good person, with their goodness being described as tangible 10s.
• The interviewer mentions that they interview many intellectually brilliant people, but the guest stands out for being a genuinely good person 42s.
• The guest is being thanked for participating in the interview, with the interviewer expressing appreciation for their presence and describing them as an incredible guest 2m6s.