AI’s Next Race: Cost, Control, and Compute

The Postfrontier Era of AI and Open Models

• AI is entering its postfrontier era, where the model is no longer the whole product, and the value will be in routing, cost control, and compute, with open-weight models potentially squeezing out frontier labs 10s.
• Over 90% of tokens created are expected to come from open-weight models over the next 18 to 24 months, possibly even by the end of the year, making AI more affordable and widely distributed 42s.
• The average Fortune 500 business may have thousands of internal applications that are being modernized with AI, and using open models is in the best interest of the business for productivity 1m6s.
• The idea of developers being empowered is what fuels the industry, and there is a resistance to early oligopolies, with the open model squeeze bringing cheaper models, more control, and a challenge to the idea that a few frontier labs will own the AI stack 1m30s.

Shift in AI Development and Deployment Focus

• The AI race had a clean scorecard in the past, with bigger models, better benchmarks, and more GPUs, but the conversation is now moving from model rankings to deployment reality, where cost control and data matters more 2m6s.
• Companies like Door Dash are building their own tests for AI code review, instead of relying on public benchmarks or model launch claims, and are finding that no single model wins everything, but rather a system of multiple models orchestrated around the actual job 3m0s.
• Perplexity is previewing a new system that starts with a cheaper open model and calls in a stronger, more expensive model only when needed, pointing to the idea that the product is no longer just the model, but the system around it 4m0s.

Model Orchestration and System Integration

• The postfrontier era of AI may look different from the last few years, with a focus on token value per watt as a defining metric, and companies like Perplexity building on open-source Chinese AI 5m0s.
• The company Olama is making it easier for developers and enterprises to download, run, and manage open models, which is a significant claim that explains why Benchmark was an early investor in Olama 10s.
• The importance of model routing has increased exponentially since the advent of agent harnesses, and the value of orchestration and model routing automatically increases as models start specializing in different capabilities and cost performance trade-offs 2m6s.
• According to Anthropic CEO Dario Amode, models are actually a lot more differentiated than cloud, and different models are good at different capabilities at different cost performance trade-offs, which is why the value lies in every company having their own eval harness model and loop to iterate on this 2m6s.
• The model alone is no longer the product, but rather the harness and orchestration system that puts the model inside a capable harness and pairs it with tools, allowing companies to serve unique value to their customers and optimize for performance and cost trade-offs 4m30s.

Open Models and Frontier Models: Competition and Collaboration

• The conversation has shifted from just focusing on better models for better tasks or cheaper models for cheaper tasks, to considering the entire AI stack, including cost, control, and compute, and how these factors impact the overall value proposition 5m20s.
• Dario Amode's prediction that one model would not be used for everything was made in a different environment, and while the explosion of open-source Chinese models was on the horizon, it is unclear if he expected the current level of competitiveness and progress in the field 6m40s.
• Perplexity is the only company that can claim to benefit from competition at all layers of the AI stack and contribute to all layers of the AI stack, which is a unique position in the industry 8m10s.
• The progress of open models and frontier models competing with each other to offer maximum intelligence for the lowest cost benefits companies that can orchestrate these models to provide maximum performance with minimal cost, which is referred to as maximum token value per watt per user 10s.

Post-Training and Hybrid Compute Strategies

• The goal is to provide the best return on investment for token spend with minimal power consumption, and this is achieved through post-training, specifically by embedding skills inside open source models to escalate to frontier models, serving as advisor models, and using a specific agent harness to serve production workloads for knowledge work 1m5s.
• This approach allows for a paradigm shift where the model is no longer important on its own, but rather as part of a vertically integrated system, with the frontier model serving as a sub-agent or tool inside the harness, enabling various forms of inference compute and paradigms, including running on local devices 2m6s.
• The use of hybrid compute, such as Nvidia's DGX Spark architecture, which combines a GPU chip with unified memory on a local device, enables efficient local models to orchestrate frontier models on the server, and also allows for more efficient models like GLM to escalate to advisor models like Opus 5 or GPD56 when necessary 4m10s.

The Gap Between Open and Frontier Models

• The gap between frontier models and Chinese open source models is estimated to be around six months, with open source models potentially catching up to frontier models like MTOS, Fable, or GPD 5 in the near future, but it's unclear if this gap will persist or narrow over time 8m20s.
• The limits of scaling have not been reached yet, suggesting that the gap between frontier models and open source models may continue to evolve, with the possibility of open source models becoming more competitive with frontier models in the future 10m40s.
• The current paradigm of training models, known as GB300, has just begun and is expected to bring advances from models post-trained on larger pre-trains, with both Frontier models and open-source models making progress over the next year or two 10s.
• The key to unlocking the potential of these models lies not in their capabilities, but in grounding them in a context that can be harnessed by small businesses and enterprises, allowing them to own and control the evaluation loop, with a focus on deployment and orchestration 42s.
• The gap between open-source and Frontier models may shrink from 6 months to a month or two, or potentially disappear, but the cost of serving models on GPUs and servers remains a significant factor, which is why hybrid compute and local models on devices are being explored 2m6s.

New Metrics and Industry Implications

• A new metric, "intelligence per watt," is being considered, which expands the conversation beyond tech companies to include other industries, and raises questions about how non-tech companies should think about hosting their own models and who Nvidia Spark is for, including consumers, developers, and enterprises 2m6s.
• The goal is for AI value to diffuse widely throughout society, but this requires giving people control over where models run, what knowledge they have access to, and the ability to run them locally, without relying on data centers, which can be vulnerable to attacks or shutdowns 4m30s.
• Local compute, such as with Nvidia's DGX Spark, solves this problem by providing freedom, cost control, flexibility, and sovereignty, and can run on low power, making it a more efficient and scalable solution for widespread adoption 6m20s.

Local Compute and Power Efficiency

• Models like Quen 35B are becoming more efficient at running local workloads, and with post-training strategies, can be used to escalate to Frontier models on servers, creating a win-win situation where local devices can use remote tools when needed 8m40s.
• The landscape of how people spend tokens is going to change completely, and this change is related to the way computing is done, with a shift towards local or deskside computing, which would reduce the need for a large infrastructure buildout 10s.
• One of the main reasons why AI revenue is not higher is the lack of compute, which is often due to power bottlenecks, and a solution to this problem is to use more power-efficient solutions, such as unified memory across the GPU and CPU, which is achieved by DGXR 2m6s.

AI as the New Computer and Enterprise Adoption

• For non-tech companies that are hesitant to host their own compute due to risk concerns, the solution is to have admin control over each device, similar to how it is done with Microsoft servers, and this can be achieved with DGX parks on every desk 4m30s.
• The concept of AI as the new computer is introduced, where PCs are seen as outdated and legacy, and the compute runtime becomes the computer, with AI becoming the operating system, and this is what is being realized through Perplexity Computer 6m20s.
• The enterprise landscape is expected to change entirely, with people doing work through AIs rather than manually, and local compute becoming the computer that every enterprise buys for every desk 8m10s.

Open Source Models and Their Capabilities

• The use of open-source models, such as GLM52, is also an important part of this thesis, as it allows for the ability to download and use these models on local systems, and GLM52 is a efficient model with 700 billion parameters and approximately 40 billion active parameters 10m40s.
• A company has trained a model called Z AI and post-trained it on tasks that their customers care about, resulting in a significant reduction in cost, approximately one-third of what they were paying Opus, and every enterprise is expected to create their own evaluations and harnesses tailored to their business 10s.
• The future of AI deployment involves creating purpose-built harnesses for specific evaluations, training models to excel within those harnesses, and continuously improving them through iterative deployment and data collection, allowing companies to own and control the cost of intelligence 42s.

Customization and Benchmarking in AI

• Companies like Door Dash and Perplexity are likely to have their own benchmarks, as public benchmarks may not accurately relate to a company's specific needs, and it's only a matter of time before companies develop their own customized benchmarks 2m6s.
• There is a perception that Chinese models are secure because they can be hosted on a company's own servers, but some companies may still be skeptical, and there is a risk of backlash towards Chinese models as they gain adoption and potentially take market share away from American labs 2m6s.
• Nvidia is working on developing models like Neatron 3 Ultra, which is expected to match the quality of Chinese models, and with post-training, it can be made to perform as well as other high-end models like Opus or Sonnet, but at a lower cost 4m30s.

Strategic Infrastructure and Policy Considerations

• The US government should consider supporting the open-source AI ecosystem as a strategic infrastructure, similar to chips and data centers, to promote competition and innovation, rather than relying solely on export controls 8m20s.
• It's not advisable to put all hopes on a single company like Nvidia, given the number of AI labs in China working on open-source models, and a more open approach to competition and collaboration could be beneficial for the US 10m0s.
• The idea of open models as strategic infrastructure is gaining traction, with some experts arguing that it's better for the US to be more open and compete on capabilities rather than relying on export controls, as this can lead to more innovation and growth 12m0s.

Security, Trust, and Open Source AI

• The capabilities of AI that may seem scary can be managed by building guardrails, and open-source software can help achieve this, as it allows people to patch vulnerabilities quickly, similar to how Linux and Android operate, and this can be applied to AI models to improve cybersecurity defenses 10s.
• To widely distribute the benefits of AI to small businesses in America and its allied countries, it is necessary to make AI more affordable, which can be achieved through open-source models that can be served at the price of compute, rather than the markups charged by labs 2m6s.

Economic and Strategic Implications of Open Source AI

• Supporting open-source AI models can help the economy move forward, and it is in America's interest to invest in open-source initiatives, which could involve investing in the university system or supporting companies like Nvidia that are already working on open-source AI 2m6s.
• Competing with Chinese open-source models in an open-source arena can drive innovation and affordability, leading to more AI-enabled businesses and increased entrepreneurship in America, and this competition can help solve the affordability problem and allow small businesses to flourish 2m6s.
• Protecting proprietary data is crucial for enterprises, as the value of a business lies in its unique tokens and tacet knowledge, and it is not in a company's interest to rely solely on external labs, but rather to have its own weights, deploy them, and create a loop to continuously improve 6m42s.

Local AI and Data Sovereignty

• Local AI enables users to have control and privacy over their data by setting up their own models and giving them access to personal data, such as audio and photos, without storing it in a data center, and this can be particularly useful for future applications like physical and robotics at home that require high frame rates of inference and processing of private data 10s.
• The cost argument for local AI is also favorable, even for people who do not care much about privacy, and this could lead to a future where everyone has a form of data center at their home, allowing for more sovereignty and flexibility over their data and models 2m6s.
• The prediction that 90% plus of tokens could come from open-weight models over the next 18 to 24 months is seen as potentially amazing, but the focus should be on creating valuable tokens rather than just a high number of tokens, and this can be achieved by making agent loops run with open-weight models 4m42s.

Value Creation and Token Economics

• The value created by open models should accrue to the user, and companies that help achieve this will be the ones to win, rather than the value accruing to the frontier, and this is because businesses can only flourish if users benefit in the long term 6m15s.
• The value created by AI can be measured by cost-performance curves, and the goal is to be on the optimality curve, where the token value per watt per user is maximized, and this is what will determine which companies will get the value 8m10s.
• Perplexity, a consumer AI company, is seeing growth in its business, particularly in the enterprise sector, where companies are asking about open source, model routing, and orchestration, but there is still a need for education and simplification of these concepts to create more value in the economy 10m0s.

AI Adoption in Small Businesses and Entrepreneurs

• Many small businesses and entrepreneurs are using AI products, and they often start with the consumer version before converting to the enterprise product, with some companies making significant revenue and spending a substantial amount on these products, such as one single-person company with a yearly revenue of around $3 million spending $800,000 a year on an enterprise computer product 10s.
• The Software as a Service (SAS) model is considered outdated, and what matters in AI is delivering output value for the tokens, with a focus on helping small businesses flourish rather than trying to sell to legacy companies that do not understand AI 2m6s.
• There are a lot of small businesses, with around $3 trillion worth of payroll being spent by them, and it is essential to diffuse the output value of AI to these businesses rather than having the value locked in a few large companies 4m30s.
• A bottoms-up approach, where adoption starts at the grassroots level with small businesses, can be an effective way for new trends, especially in tech, to take off, as seen with companies like Apple and Nvidia, which initially sold to small companies and gamers before becoming much larger 6m40s.

Open Models in Enterprise Adoption

• Open models are moving from developer experimentation to real enterprise adoption, with companies like Alama, which allows users to download and run open AI models on their own infrastructure, positioning themselves to take advantage of this trend 10m50s.
• The growth of the open AI model ecosystem has exceeded expectations, and companies like Alama are well-positioned to capitalize on this momentum, with investors like Benchmark recognizing the potential of open models and their potential impact on the AI industry 12m10s.
• The idea of open models is becoming increasingly popular, with thousands of businesses considering building with them, and developers taking control and ownership of their AI journey, allowing them to build applications from chatbots to long-horizon agents 10s.
• Enterprise companies are starting to realize that they can use open models to manage their costs better, and these models are good enough or almost as good as the frontier models, which could lead to a significant shift in the industry 1m42s.

Performance and Cost Advantages of Open Models

• It is believed that 90 plus percent of the tokens created will come out of open-weight models over the next 18 to 24 months, driven by two main forces: cost and the fine-tuning of models for lower latency and higher performance 2m6s.
• The use of open-weight models can result in orders of magnitude reduction in time to first token, making them more appealing to companies like Sierra and other applied AI companies for performance reasons 2m6s.
• The goal is to empower developers to discover which open model is best suited for their task, giving them choice and control, and having a platform like Llama as a destination for developers to achieve this 3m30s.

Expanding Open Model Adoption to Fortune 500

• To get beyond the current users of open models, such as Sierras and AI companies, and into the Fortune 500, it will be necessary to address the hesitation to embrace these models and demonstrate their value in various use cases 5m10s.
• Both frontier models and open models will thrive, with frontier models pursuing complex use cases and open models being used for more straightforward applications where they are good enough and can be run without the markup of the foundation models 6m20s.
• The shift towards open models is expected to happen as companies move from discovery to optimization and ultimately driving excellence in total cost, with open models being used for internal, confidential information where companies have total control over the data 7m30s.

Adoption Trends and Industry Examples

• Over 85% of the Fortune 500 companies have adopted Lama, with the majority coming from highly regulated industries such as aviation, insurance, and healthcare, due to the trust relationship built by running models in their environment securely and safely 10s.
• These companies start with small models and then adopt larger ones as they progress in their journey with open models, with the ability to provide cloud computing environments in the US and Europe being a key factor 42s.
• The use of open source models has evolved from setting up internal chat GPT type experiences to adopting long-running coding agents, with adoption seen across various industries, including healthcare and power plants 1m26s.
• There is a healthy mix of models from different geographies, including American labs' open-source models and Chinese models, with each having their own focus and driving different use cases 2m6s.

Chinese Models and Global Competition

• The Chinese models, such as GLM and ZI, have gained a dominant share in terms of tokens, and are delivering high-performance results, especially for agentic long-running and more complicated reasoning tasks 3m15s.
• The compute constraint is a significant variable in the adoption of open weight models, with companies forming long-term relationships with providers like Fireworks and Neocloud to support the demand for open weight models at a lower cost and higher performance 4m30s.
• The market is seeing a shift towards open weight models, with many companies anticipating the release of new models, such as the deepseek model, and the proximity of Chinese models to the performance of Opus models 5m10s.

Economic and Infrastructure Constraints

• The development of open-weight models is complex, and despite 90% of tokens being generated by these models, 90% of the revenue may still come from frontier models, with a pricing difference of three to five times more for frontier models, 10s.
• The cost difference between open-weight and frontier models puts economic pressure on the system, and it is uncertain when capacity constraints will be resolved, which could be as late as 2030 or 2035, 1m42s.
• The scarcity of infrastructure, particularly compute power, may become a mediating factor in the adoption rate of AI, and companies like Meta and SpaceX have already started hosting compute for others, 2m6s.

Performance and Pricing Dynamics

• Some open-source models, such as GLM, are almost as good as or even better than the latest models from companies like Anthropic, which could lead to pricing pressure on these companies, 3m30s.
• It is unlikely that companies like Open AI and Anthropic will start hosting open-source models, as it would require a significant evolution of their business models, and instead, they may move into vertical applications where margins are more defensible, 5m10s.
• Applied AI companies are not switching to open-weight models solely for cost reasons, but also for performance, as they need direct control over the models to fine-tune them, and once a task is understood, they can move to a high-performance, low-cost inference model, 7m20s.
• The ability to fine-tune models is highly valuable, and cost is often a secondary consideration, with performance being the primary driver, which will ultimately impact cost, 9m40s.

Security and Geopolitical Considerations

• Companies are worried about security issues with open source models from China, but hosting them in the United States can alleviate these concerns, and corporations are now interested in open source models and are looking to shift some of their workloads, with questions focusing on where the models run and how they are managed 10s.
• The key considerations for businesses are the geographic location where the model operates, such as the US or Europe, and the need for tooling to run the models safely, including performance, monitoring, and customization, which can be provided by services like Alama's cloud 2m6s.
• There are risks involved in building on Chinese models, as it could lead to a Chinese ecosystem, and American AI companies should pay more attention to developing their own open source models to maintain control and access 4m42s.

Open Models as Strategic Infrastructure

• The open weight model ecosystem benefits from heterogeneity and multiple geographies, and companies should recognize the importance of having a strategy that enables access and control for developers, rather than creating closed systems 6m15s.
• Policy makers should avoid blocking access to open weight models, as this could disadvantage the US technology market and instead, they should proactively support and embrace open models to stay competitive with other regions, including China 10m45s.

Policy and Regulatory Considerations

• The risk of regulatory capture is a concern, but it is believed that frontier models will ultimately want to have a super majority developers strategy, and a losing strategy would be counterproductive even for them 10s.
• There is a misconception that open source AI cannot be monetized, but Chinese labs are proving that it can be done through API access and other ways, which is an important point to consider when discussing the adoption of open source AI 42s.

Monetization and Adoption Strategies

• To effectively adopt AI, it is essential to understand the needs of customers, especially US-based ones, and help them adopt AI in a way that is safe, secure, and productive, which requires getting to know them and their AI journey 2m6s.
• The importance of having an American developer tool, such as Alama, is that it provides trust and security for companies that are worried about using Chinese models, and having partners around the world can help launch models like the Canadian coding model 2m6s.
• The debate surrounding data is a critical issue, with some arguing that companies should be more protective of their data from big labs, and others believing that open models can provide economic value within enterprises by capturing internal workflow data and providing ownership and portability 2m6s.

Data Ownership and Enterprise AI

• The idea of seeding context to a closed model company seems incoherent, as it would require giving up ownership of internal workflows and proprietary know-how, which is essential for a company's competitive advantage, and open models can provide a solution to this problem by allowing companies to maintain control over their data 2m6s.
• The concept of context and orchestration is crucial in applying AI to productive use cases, and companies like Merkore are working on providing task mapping of internals of a company, their workflows, and proprietary data onto models, which requires capturing internal workflow data in a way that provides ownership and portability 2m6s.
• Enterprises should develop ownership and invest in AI models to apply them to their business, rather than giving their data to other models and potentially losing it to competitors, allowing them to benefit from these models 10s.

Enterprise AI Stack and Model Diversity

• When considering working with forward-deployed engineers from companies like OpenAI or Anthropic, it's essential to ask questions about data ownership and portability to other models, as these companies may resist protections for the enterprise, highlighting the need for a robust independent ecosystem 42s.
• The enterprise AI stack is expected to be a mix of different models, with a majority of tokens coming from open models, which are more efficient, lower latency, and higher performance, and will be used to modernize internal applications and create AI-augmented agents 2m6s.
• Open models are expected to play a significant role in the future of AI, with thousands of internal applications being modernized with AI, and open models being used to process hundreds of millions of tokens, due to their efficiency and performance 2m6s.

Developer Empowerment and Innovation

• The idea of developers being empowered is crucial, and open-weight models and open source are enabling a constellation of use cases to emerge from the creativity of developers, leading to a more diverse and innovative AI landscape 4m30s.
• There is excitement about the opportunity for long-horizon agents that can work over many days, weeks, and months, which requires optimizations that open-weight models provide, and this is an area of significant interest and potential growth 6m0s.
• The AI landscape is moving quickly, and it's essential to stay informed and adapt to the changing environment, with a focus on open models, developer empowerment, and the potential for new use cases and innovations 8m0s.