GitHub Universe 2024 opening keynote: delivering phase two of AI code generation

GitHub Universe timeline of key product launches 0s

GitHub CEO Thomas Dohmke intro on phase two of AI and software development 54s

• GitHub Universe is being held for the 10th time, and it's the first time at Fort Mason, marking a decade of open-source creation and consumption, with open source having "won" as it can no longer be denied that it has become a major force in the industry 1m9s.
• Since 2015, developers have solved big problems by closing over 67 million GitHub issues, merging over 180 million pull requests, and sparking over 320 million public projects on GitHub, making open source the greatest team sport on Earth 2m5s.
• The age of AI has not just begun, but has crossed the chasm, with a 98% year-over-year growth in AI creation in 2024, and Python becoming the number one programming language on GitHub, surpassing JavaScript 3m11s.
• The primary use case for GitHub is expanding, with open source creation becoming a big trend for data science, academia, and research, and the programming language of AI is on the rise 3m49s.
• GitHub has evolved into the world's largest creator network for the age of AI, with developers using AI to build AI, and laying the blueprint for what it means to partner with intelligent machines 4m9s.
• Copilot, launched three years ago, is the most adopted AI developer tool on the planet, and is now launching into its next phase, with the first phase of AI code generation being built on three key pillars: AI-infused, conversational coding, and multimodel functionality 4m43s.
• Phase two of AI code generation has arrived, with the partnership between developers and AI going to the next level, and AI becoming native to the entire developer experience, with AI-native developer workflows and agentic coding, where AI assistance executes tasks at the developer's direction in natural language 6m11s.

Multi-model choice for developers 7m0s

• GitHub is moving from multimodal functionality to multimodal choice in phase two, allowing developers to choose the models that work best for them 7m1s.
• There has been a boom of high-quality models in 2024, and GitHub is responding by providing developers with the agency to build with the models they prefer 7m8s.
• GitHub is an open developer platform, and developers expect to have the freedom to choose the models that suit their needs 7m15s.
• GitHub is delivering multimodal choice by providing public access to 01 Preview and 01 Mini for all Co-Pilot users 7m23s.
• 01 Preview has been found to have strong reasoning capabilities, allowing for a deeper understanding of code constraints and edge cases, and producing efficient and quality results 7m51s.
• Developers can now use 01 in Co-Pilot Chat by simply clicking on it and getting started 8m2s.

Anthropic's Claude 3.5 Sonnet + GitHub Copilot 8m9s

• A model has gained significant attention from developers this year, specifically Claude 3.5 Sonnet, developed by Anthropic. 8m9s
• A new partnership between GitHub and Anthropic is being announced, which brings the power of Claude to GitHub Copilot. 8m21s
• As a result of this partnership, users can select Claude in the model palette, starting today. 8m35s
• The integration of Claude with GitHub Copilot is now available on users' computers. 8m39s

GitHub CEO Thomas Dohmke + Anthropic Co-Founder Jared Kaplan 8m45s

• Jared Kaplan, co-founder and chief scientist of Anthropic, was welcomed on stage, and his background as a theoretical physicist was discussed 8m46s.
• Kaplan explained that he was always fascinated by science and AI, but initially pursued physics due to the lack of tools for software development and AI research at the time 9m10s.
• He eventually transitioned to AI after being inspired by friends working in the field, including Dario Amodei, who was leading research at Open AI 9m41s.
• Kaplan discussed the inception of Anthropic and the development of Claude 3.5 Sonnet, which was designed to make AI models more accessible to developers 10m4s.
• The goal of Anthropic is to steer AI in a beneficial direction and make frontier model capabilities available to as many people as possible, as quickly as possible 10m12s.
• Claude 3.5 Sonnet was optimized for software engineering and is intended to be a great model for developers to work with 10m57s.
• Kaplan believes that the partnership between GitHub and Anthropic will be amazing for developers, providing them with access to the latest tools and models 11m5s.
• He emphasized the importance of getting hands-on experience with AI tools, rather than just reading academic papers, and encouraged physicists to learn tools like GitHub 11m26s.
• Kaplan discussed the roadmap for Claude, which includes improving the complexity of tasks that AI models can perform and expanding the environments in which they can work 11m49s.
• Future releases are expected to enable models to perform more complex workflows, work alongside humans, and collaborate in various environments, including software testing and app development 12m10s.
• Kaplan expressed excitement about Claude being able to work alongside humans in many different contexts 12m46s.

Google's Gemini 1.5 Pro + GitHub Copilot 13m11s

• A partnership has been formed with Google to bring Gemini 1.5 Pro to GitHub Copilot, allowing developers to build with this new model in the future 13m21s.
• Gemini 1.5 Pro is natively multimodal, enabling it to process code, images, video, and text simultaneously, making it suitable for code suggestions, writing documentation, and explaining code 13m36s.
• The multimodal capabilities of Gemini 1.5 Pro will be integrated into many of GitHub Copilot's surface areas in the coming weeks, providing developers with more options and flexibility 13m50s.
• Gemini 1.5 Pro will be available in GitHub Copilot's new model Pickers, allowing developers to choose the model that best suits their needs 13m31s.

GitHub Copilot in Visual Studio Code 14m0s

• GitHub Copilot has been enhanced in Visual Studio Code to provide a better experience, with the goal of reducing context switching and increasing productivity 14m0s.
• Cassidy, the presenter, showcases the new features of GitHub Copilot, highlighting its evolution into an intelligent platform with multiple tools and services 14m24s.
• GitHub Copilot is demonstrated on a website called CoPilot Airways, where the presenter uses it to autocomplete code and explain its functionality in different languages 15m5s.
• The presenter uses GitHub Copilot to ask for suggestions on how to improve a table on the profile page, and the AI tool suggests using a line graph that already exists in the codebase 16m49s.
• GitHub Copilot's intent detection feature is showcased, where it figures out what the user is asking and uses the workspace agent and repo indexing to provide relevant suggestions 17m0s.
• The presenter decides to create a new component, an area graph, and notes that this would require multiple changes to the code, including creating a new component, changing the profile page, adding tests, and updating the README 17m21s.
• The presenter sets the stage for demonstrating how GitHub Copilot can be used to edit multiple files at once, streamlining the development process 17m35s.

Multi-file editing 17m36s

• GitHub's Co-Pilot now supports multi-file editing, allowing users to make changes across multiple files at once 17m43s.
• A new edit mode has been introduced, enabling users to select a model, such as Claude, and provide instructions for the AI to generate code 17m51s.
• Users can provide specific instructions, such as creating a new reusable area graph from a points activity service, and the AI will generate the necessary code and tests 18m1s.
• The AI will create step-by-step instructions for the code generation process, including creating a chart component, adding it to a profile page, and creating tests for the new component 18m28s.
• The generated code is based on the existing codebase and includes green diffs to highlight new changes 18m41s.
• The AI also generates unit tests for every single component, which can be reviewed and saved 19m0s.
• The generated code can be accepted and saved, and the changes will be reflected in the relevant pages, such as the profile page 19m15s.
• The feature includes a little feature flag under the hood, which can be accessed by going back to the monthly points area chart 19m31s.

Copilot Extensions 19m40s

• GitHub Copilot Extensions was released in May, allowing users to extend GitHub Copilot's capabilities to perform various tasks, such as integrating with feature flag services like Launch Darkly 19m41s.
• The Launch Darkly GitHub Copilot extension enables users to create flags based on specific conditions, such as a monthly points chart, and toggle them directly within VS Code 20m2s.
• GitHub Copilot Custom Instructions is a feature that allows users to add custom instructions to their GitHub repository using a markdown file, enabling them to specify specific actions for Copilot to perform, such as adding certain docstrings or following specific linting rules 20m29s.
• The custom instructions file can be used to automate various tasks, such as adding comments or ensuring that reusable React components include specific services 20m46s.
• GitHub Copilot Extensions are available now, with a range of extensions to choose from, including those from Superbase, Atlassian, and Century, and users can also build their own extensions 22m5s.
• The extension marketplace will continue to grow, offering users a wide range of choices, and GitHub Copilot's capabilities will extend beyond VS Code, with support for Apple developers 22m28s.

GitHub Copilot + Xcode 22m34s

• GitHub Copilot is available in Xcode today in public preview, allowing developers who build for the Apple ecosystem to leverage its power as they develop, test, and deploy 22m35s.
• The integration of GitHub Copilot with Xcode aims to simplify the lives of developers, providing them with the agency to make their own choices, regardless of where they work 22m58s.
• Developers can access GitHub Copilot in Xcode whether they work on mobile, desktop, remotely on GitHub.com, or locally on their machine 22m52s.

GitHub Models 23m7s

• GitHub co-pilot is being expanded to include multimodal choice, allowing developers to build on higher layers of abstraction with models of various sizes to stay up-to-speed 23m21s.
• GitHub launched a new tool called GitHub models, an interactive playground for AI engineers to build applications right where their code is, allowing users to go from model to playground to deployment quickly 23m47s.
• GitHub models was launched with the latest models in the industry, including French junqua with Mistral large and Mistral Nemo, as well as smaller language models like 53 24m7s.
• The platform also includes open-source models, such as Meta's Llama Suite, and has added more models from Co:here, Command, Bria AI, and Open AI 24m45s.
• GitHub models allows users to explore and compare different language models (LLMs) to find the best one for their specific task, and Co-pilot can guide users in selecting the most suitable model 26m11s.
• The app models extension in Co-pilot chat enables users to ask for advice on which model to use for a particular task, and Co-pilot will provide recommendations based on data from the model Marketplace 26m34s.
• A demonstration of GitHub models and Co-pilot was shown, where a user wanted to extract JSON data from travel reviews and used Co-pilot to find the most suitable Mistral model for the task 26m48s.
• GitHub models allow users to try out different models, such as Mr. LLaMA and Mr. Nemo, and compare their performance on specific tasks 27m0s.
• The compare feature enables users to run two models in parallel for the same task and easily compare their results 27m30s.
• Users can add system prompts to explain to the model what task to perform, and the prompt is automatically applied to both models being compared 27m58s.
• After running the models in parallel, users can compare the results and choose the model that best suits their needs 28m27s.
• The chosen model can then be used to generate code, such as extracting JSON data from a review, which can be easily integrated into an application 28m46s.
• GitHub models provide a code tab where users can pick their language of choice and get a copy-and-paste code sample to interact with the model 29m26s.
• Users can use their GitHub token for authentication, eliminating the need to sign up for new services or jump through hoops 29m39s.
• When ready to go to production, users can get an API key for Azure AI, which has higher rate limits and can be scaled 29m50s.
• GitHub models also include multimodal models, which can handle multiple types of input and output 30m6s.
• Multimodal models are capable of accepting images as well as text, allowing for more diverse inputs and interactions 30m12s.
• A demonstration of a multimodal model, Open AI's GPT 40 mini, is given, showcasing its ability to describe an image of the Golden Gate Bridge in one word: "Bridge" 30m30s.
• The GitHub Multiverse is being extended, and GitHub models, including the app models extension, are now available to all GitHub users 31m1s.
• With GitHub models, users will have access to the playground, side-by-side comparison, and more features to aid in their development 31m16s.
• The availability of GitHub models aims to enable over 100 million developers worldwide to become AI engineers 31m22s.

Copilot Workspace 31m34s

• Nille, a high school senior, is the lead programmer for the robotics team Orange Dynamite, which has been working on a project for 7 months 31m34s.
• Nille is interested in aerospace engineering and has found GitHub Copilot Workspace to be a valuable tool, surpassing other AI offerings in its ability to understand programming tasks and real-world solutions 31m50s.
• GitHub Copilot Workspace represents the future of GitHub Copilot, allowing users to assign tasks and guide the AI as it implements solutions, making it possible to understand complex concepts in a matter of minutes 32m6s.
• With GitHub Copilot Workspace, tasks that conventionally took a long time can now be completed quickly, and the tool is deeply integrated into Nille's workflow, making it hard to imagine working without it 32m21s.
• Nille considers GitHub Copilot Workspace a game-changer that continues to surprise and improve their work 32m33s.

Copilot Workspace demo + GitHub Next 32m44s

• GitHub Next is the in-house research team building for the future of software development, and they launched a technical preview of Co-Pilot Workspace six months ago on April 29th 33m3s.
• Co-Pilot Workspace is an AI-native development environment where users build with agents and the power of AI at every step from idea to execution, starting with an issue and having three agents helping: a spec agent, a plan agent, and an implementation agent 33m25s.
• Since the launch, developers on the technical preview have created over 15,000 pull requests using Workspace, and over 10,000 of those pull requests have merged 34m6s.
• Thousands of developers have engaged with Workspace, providing feedback that has been generous and helpful, whether positive or constructive 34m24s.
• Much of the feedback has been positive, with some users sharing how Co-Pilot Workspace helped them bootstrap an app easily and shrink timelines from days to hours or minutes 34m46s.
• Other feedback has been constructive, pointing out issues such as the difficulty of validating code inside Workspace, which the team is addressing 34m57s.
• The team has made over 100 changes to Co-Pilot Workspace since the start of the technical preview, including redesigns, new features, paper cuts, and quality of life improvements 35m20s.
• Two new agents have been added to the workflow: Brainstorm for ideation and another for building and repairing code, bringing the total number of agents to five 35m51s.
• A VS Code extension has been released, and Workspace is being expanded to pull requests, with AI acting as a second brain and a truly helpful partner alongside developers 36m9s.
• The next iteration of Co-Pilot Workspace is available, and it continues to build on the existing features and functionality 36m23s.
• Co-Pilot Airways is being used as an example to demonstrate the capabilities of Co-Pilot Workspace, with a feature being added to allow images to be added to each post 36m47s.
• Co-Pilot Workspace allows users to work on issues and implement features without having to create a new branch and start thinking about implementation from scratch 37m25s.
• GitHub's Copilot Workspace is a tool that helps developers implement issues and features in their code by providing a specification of how to implement the issue, and it can be used iteratively based on the context provided by the brainstorm feature 37m29s.
• The brainstorm feature shares suggested questions and ideas to get started with implementing an issue, and users can also ask their own questions to generate more ideas 37m52s.
• Copilot Workspace can derive context from multiple sources, including GitHub issues and brainstorming sessions, to generate a plan for implementing a feature 38m34s.
• The plan generated by Copilot Workspace includes a list of changes to make in which files, and users can edit the plan if needed before asking Copilot to implement it 38m48s.
• Copilot Workspace can make changes to multiple files across a project, and users can ask for revisions to the plan and implementation by talking to Copilot in natural language 39m31s.
• Copilot Workspace supports multiple languages, including Hindi, and can understand and respond to user requests in those languages 39m43s.
• The tool also allows users to run tests and debug their code, and can even help fix errors by suggesting solutions with the new build and repair agent 40m31s.
• Users can configure commands for building, testing, and running their code within Copilot Workspace, and can also run tests and debug their code within the tool 40m33s.
• If a test fails, Copilot Workspace can help users fix the error by suggesting a solution, and users can apply the suggested fix and re-run the test to see if it resolves the issue 40m56s.
• Once a feature is implemented and tested, users can preview what they've built by running a command within Copilot Workspace, which can suggest a command if one hasn't been configured already 41m44s.
• GitHub is showcasing the capabilities of its co-pilot feature, specifically co-pilot workspace, which allows developers to brainstorm, build, and repair code in a natural language environment 42m36s.
• The co-pilot workspace feature enables developers to work efficiently, moving as fast as their creativity, and is intended to be the developer environment of the future 42m55s.
• Co-pilot workspace is expanding to include pull requests, allowing developers to address feedback from their team and AI agents, making it a part of the development team 43m5s.
• The feature is designed with the developer at the center and has the potential to be the single most revolutionary thing built for professional developers since the inception of GitHub 44m16s.
• Co-pilot workspace is being further developed to make it accessible to people who want to build software but cannot write code, making coding more inclusive 44m36s.
• The demos shown were live deployed to production on GitHub.com, and the company looks forward to getting co-pilot workspace into the hands of more developers 43m56s.
• The co-pilot workspace feature is part of the next iteration of co-pilot and is expected to be a game-changer for professional developers 43m32s.
• The feature is being developed with the goal of making coding more accessible and efficient, and GitHub is excited about its potential impact on the development community 44m10s.

GitHub Spark 44m50s

• GitHub Spark is an AI-native tool that allows users to build applications entirely in natural language, blurring the line between developers and everyday people 44m50s.
• With GitHub Spark, anyone can create and share an application in a matter of minutes, regardless of their experience 45m12s.
• GitHub Spark is not a classic IDE, and users do not need to install programming languages or figure out how to use the terminal 45m32s.
• The tool features an input field for creating applications, a model picker with different models, and a list of recent Sparks 45m39s.
• Users can create various applications, such as games, using natural language prompts 45m55s.
• A live demo showcased the creation of a Tic Tac Toe game with ducks and hippos using GitHub Spark 46m8s.
• The model behind the scenes does all the work, and the application is streamed in digitally, with a console available for refinement 46m13s.
• Users can refine their applications, add themes, change the scale, and modify the accent color 47m1s.
• GitHub Spark also allows users to add custom instructions, store data in a key-value store backed by Azure Cosmos DB, and scan conference badges 47m21s.
• A demo app was created using GitHub Spark, which scans conference badges and displays the user's information 47m35s.
• The app was created entirely in natural language, and every step of the process is visible, including all the prompts used 47m41s.
• The app is functional and can be used on a mobile device 48m22s.

Closing remarks 48m44s

• A new wave of personal applications is on the horizon, and the floodgates of innovation have swung wide open, as demonstrated in the presentation 48m49s.
• For a long time, there has been a significant barrier separating a vast majority of the world's citizens from creating software, due to factors such as language, educational access, or the perceived difficulty of coding 49m6s.
• This barrier has limited opportunities in the technology economy, making the industry a "Walled Garden" 49m35s.
• GitHub Spark aims to put one billion cracks in this concrete barrier, and with the help of the community, it is possible to break this barrier and create a more inclusive environment 49m40s.
• The goal is to realize a world where not just 100 million but one billion people on GitHub can build software together 50m5s.
• The new creator Network is designed for the age of AI and is intended to be a creator Network for all, which is GitHub's guiding aspiration 50m16s.
• GitHub is committed to making this vision a reality and invites the community to enjoy the 10th Annual GitHub Universe, with more content and announcements to come 50m31s.