Jueves de Quack - GitHub Copilot en la era UBB: contexto, modos y presupuesto

Introduction and Context

• The speaker is broadcasting from a coworking space due to an internet outage in their community, which forced them to find an alternative location for the session. They mention leaving someone named Wilson at home and express gratitude to the audience for joining the session. 10s
• The session's topic is about budgeting, which is not the speaker's favorite subject. They aim to educate the audience on how to work with GitHub Copilot more effectively without overspending. The speaker introduces themselves as Andrea Griffis from Colombia. 2m6s
• There is a change in how GitHub Copilot usage is billed, moving from a credit-based system to a usage-based billing system called "usage billing" or "UBB." This change has caused stress and concern among users. The speaker emphasizes the importance of using Copilot intelligently to make each interaction count. 2m6s
• The speaker plans to share two resources to help optimize the use of Copilot under the new billing system. These resources include a guide that provides step-by-step instructions on optimizing workflows to manage costs effectively. The guide is unofficial and aims to address questions left unanswered by GitHub's official guidance. 2m6s
• The repository currently contains a version in Spanish and there is a desire to clean it up and potentially make it open source, with a community task of translating it. 10s

GitHub Copilot Billing Changes

• The focus is on maximizing the use of GitHub Copilot by understanding the starting point and the changes in billing from June 1st, where Copilot Business and Enterprise will charge based on credits rather than the number of requests. 42s
• The cost of using Copilot will now depend on several factors: the input prompts, the output generated by the model, the overhead of context such as active instructions and open tabs, and the overhead of tools like agent mode. 2m6s
• There is a need to shift from relying on subsidies to working more intelligently, focusing on changing habits rather than creating budgets, as the era of subsidies is ending. 2m6s
• Personal reflection on the wasteful use of Copilot due to the privilege of not having to worry about costs, and the need to focus on efficient usage moving forward. 2m6s
• The focus is on changing habits to reduce waste rather than strictly limiting budgets, although organizing budgets is also important. A guide is provided to help organize defaults and optimize for quick gains, with a checklist for focusing efforts effectively. 10s

Optimizing GitHub Copilot Usage

• When using GitHub Copilot, it is possible to specify that only code is needed without explanations, especially if the project is well-defined. This approach helps in reducing unnecessary costs associated with verbose interactions. 42s
• There are three modes for working with agents: ask, edit, and agent. The ask mode is used for questions and is more economical. The edit mode is for when the objective is clear, providing more control with less overhead. The agent mode is the most powerful and expensive, as it involves Copilot making decisions and executing tasks. 2m6s
• A common mistake is overusing the agent mode for simple tasks, which can be costly. It is more efficient to use the appropriate mode for each task to optimize resource usage. 2m6s
• One common mistake is assigning tasks to agents that can be efficiently handled by the Edit mode, which incurs costs in planning, reasoning, and tool calls. It is more efficient to use Edit mode for straightforward tasks, supported by GitHub Copilot, without overburdening the agent unless a premium model is necessary for complex tasks like large-scale refactoring 10s.
• When planning a project, using a high-quality model is beneficial for comprehensive planning. However, for execution and optimization, a less expensive model may suffice if the plan is well-detailed and clear, reducing the need for a high-powered model 42s.
• The Auto mode in GitHub Copilot, initially perceived as a cost-saving measure for the company, has evolved to intelligently select the most suitable model for the task at hand, balancing budget efficiency with task-specific model assignment. This mode now considers both cost and task requirements, rather than just minimizing expenses 2m6s.
• Users often have preferences for specific models based on their experiences, such as using certain models for design or engineering tasks. However, these preferences are often based on empirical opinions rather than formal benchmarking, highlighting the importance of Auto mode's intelligent model selection 2m6s.

Creating Effective Prompts

• The importance of understanding how models work and creating effective prompts is crucial, especially when working with models like Opus 4.7, which can provide more in-depth investigations, but currently lacks benchmarking and evaluation to validate its effectiveness 10s.
• To optimize the use of models and avoid high interaction costs, it is essential to focus on creating well-structured prompts that consider the problem, the desired change, and the required output, allowing for more efficient and cost-effective interactions 1m20s.
• When creating prompts, it is vital to define the objectives, such as identifying the problem, determining the required change, and specifying the desired output, to ensure that the model provides relevant and useful results 2m6s.
• Providing constraints, such as security limits, API usage, and testing requirements, can help the model understand the context and provide more accurate and relevant outputs, making it essential to include these details in the prompt 3m30s.
• Defining the desired output format, such as JSON or CESB, is also crucial to ensure that the model provides the results in the required format, making it easier to work with the output and reducing the need for additional processing 4m40s.
• By focusing on creating well-structured prompts and considering the problem, change, and output, users can optimize their interactions with models, reduce costs, and achieve more effective results 5m50s.

Managing Context and Instructions

• When using GitHub Copilot, providing specific prompts can be more efficient and cost-effective than vague ones, as they reduce the search space and eliminate the need for guessing the file or problem, leading to shorter and more accurate responses 10s.
• The context in which Copilot operates is crucial, as it involves loading repository instructions and agent instructions with each interaction, which can become burdensome if not managed properly 2m6s.
• An audit of personal projects revealed unnecessary bloat in Copilot instruction files, indicating that excessive or irrelevant context and tokens are being sent with each request, which can be counterproductive 2m6s.
• It is important to optimize the Copilot instruction files to avoid sending unnecessary tokens, as this can lead to inefficiencies and increased costs in interactions with the model 2m6s.
• GitHub Copilot instructions that are always active should be treated as essential information to avoid costly errors, and instructions with scope can be loaded when referencing specific files to work with budgets 10s.
• Instructions can be categorized, such as those related to APIs, which are only loaded when working with APIs, allowing for more efficient execution of intelligence when providing the necessary context for Copilot to produce the desired output 42s.
• Prompts and scripts are loaded when invoked, and the agent determines when they are needed, eliminating unnecessary costs when not in use 1m15s.
• Automatically generated context files can increase token usage by 20-33% in medium-sized sessions, making it essential to review and update instructions regularly to avoid unnecessary costs 2m6s.

Using Hooks and Skills

• Evaluating Copilot instructions and agent instructions is crucial to determine if they prevent real errors or are simply documentation that may not be read, and a checklist can be used to assess their effectiveness 3m20s.
• Creating a skill can help optimize token usage, and utilizing tools like Ghoocco Pilot Hooks can provide a more efficient way to manage projects and reduce costs 4m40s.
• Managing resources such as MSP, skills, and hooks is essential to control consumption, and only maintaining necessary resources can help minimize costs 6m10s.
• Utilizing GitHub Copilot Hooks and other tools can help automate tasks, save time, and improve overall efficiency, with resources like Gocileo Hooks providing more in-depth documentation and guidance 8m0s.
• Hooks can be used to personalize behavior in GitHub Copilot, allowing actions to be executed only when needed, which optimizes performance without loading the entire context. 10s
• Using skills and hooks together can save requests in a chain of language models (LLM) by running analysis with skills and executing hooks based on that analysis. Hooks are configured using a simple JSON file placed in the root of a GitHub repository. 42s
• Proper configuration of hooks is crucial to avoid them becoming spam vectors. Hooks should be deterministic, with clear triggers to ensure they execute as intended. 2m6s
• It is important to be methodical about what is executed, when, and why, rather than providing all context and authorizing all tools. This involves changing habits, setting early alerts, and testing model policies purposefully. 2m6s

Evaluating and Controlling Costs

• Efficiency and budget considerations are becoming more important, as excessive context and model output can be counterproductive. Creating benchmarks and evaluating metrics can help reduce costs and improve efficiency. 2m6s
• To view the usage and billing information, users can go to their billing section, where a CSB will be generated and sent to them via email, and there is a tool that can be used to help with the build process, providing detailed information about usage and costs 10s.
• The tool provides details about the users, such as the number of requests, interactions, and costs associated with GitHub Copilot, allowing administrators to evaluate and make changes to their usage and budget 42s.
• The information provided by the tool includes the type of requests, interactions, and equipment used, as well as the products and models utilized, giving administrators a clear understanding of their costs and usage 1m6s.
• Administrators can use this information to educate their teams and make changes to their usage, focusing on areas such as user management, models, products, and organizations, to optimize their costs and budget 1m42s.
• After June 1st, users will be able to view a complete report of their usage and costs, which will provide a more expansive view of their usage and costs, and there is also a checklist available that outlines the steps users can take to manage their costs 2m6s.
• Additionally, there is a guide available that provides more in-depth information about GitHub Copilot, including how it works, how tokens are counted, and how to manage costs, which can be useful for both personal and enterprise-level users 2m42s.

Understanding Billing and Usage

• When running a VB report, it is important to recognize that some user issues may stem from a lack of education rather than abuse, presenting an opportunity for learning about token management and request handling, especially in companies broadly adopting AI technologies across various departments like legal and HR 10s.
• Understanding the cost of outputs compared to inputs is crucial, as outputs can be more expensive. For example, a cloud output might cost $5, while another model costs $4.5. This cost structure is determined by external laboratories, and having knowledge about these costs can aid in making informed decisions 42s.
• Using local models with less context and lower costs can yield good results for specific tasks, highlighting the importance of balancing model selection with the need for context to avoid overwhelming the system with unnecessary information 2m6s.
• Enterprise administrators should be aware of changes effective from June 1st, which will alter the way billing is handled. It is advised to set budgets in advance and not wait until the deadline to organize them. Administrators have the ability to assign budgets to users, which can restrict access to certain features if set to zero 2m6s.
• Model selection is a controllable aspect for administrators, and it is important to review models before activation to ensure that users can effectively utilize GitHub Copilot while maintaining responsible budget management 2m6s.
• The cost of certain models, such as Antropic, can be high, with prices like $25, and it is essential to consider the budget and explore other options that may be more cost-effective 10s.
• Education about auto models is crucial, and having instructions with specific scopes is vital, as well as considering different organizations and their needs 42s.

Model Selection and Budget Management

• Evaluating the budget and understanding where the money is being spent is necessary, and it is essential to assess what is important for each team and whether certain modules are necessary or if access needs to be restricted 1m6s.
• Creating a checklist and using the default enterprise setting, which includes utilizing auto and having preset budgets, can help with cost control and ensure that everyone is aware of the available budget 2m6s.
• Monitoring internal usage of GitHub Copilot and identifying the actors who are using the most budget can help optimize costs, and it is essential to be mindful of costs as an administrator 2m42s.
• Implementing controls, such as optimizing models for specific users and providing access to premium models only when necessary, can help reduce costs, and it is essential to consider the execution and scaling of interactions to avoid high costs 3m6s.
• Utilizing ASMO and agent modes only when necessary, as well as evaluating and optimizing instructions and outputs, can also help reduce costs and improve efficiency 4m6s.
• The importance of considering the team's needs and evaluating the usage of GitHub Copilot, as well as implementing measures to control costs, such as creating a checklist and monitoring internal usage, is crucial for effective budget management 5m6s.
• Not all tasks require the agent mode, and it is recommended to set the selector to auto, which provides insight into the rubrics used for model selection, and this information can be found in a previously recorded segment 10s.

Optimizing Model Usage and Costs

• MSP is a significant expense, but there are tasks that require it, and using it efficiently is possible with the right knowledge, such as not overloading unused models and being precise in model assignment 1m42s.
• Chronicle is a function of the Copilot CLI that helps track interactions with Copilot, and it can identify repeated tasks and suggest turning them into skills to save requests, context, and budget 2m6s.
• The Chronicle function can also help identify patterns and provide insights into how Copilot is being used, such as identifying areas where skills can be created to improve efficiency 2m6s.
• An experiment was attempted to create a skill for auditing learned tasks using a local model, but it was unsuccessful due to internet connectivity issues, and an alternative approach will be explored in a future demonstration 4m30s.
• The goal is to utilize a local model for automated auditing, which will provide continued access to this functionality even after changes take effect on June 1st 5m40s.
• A skill has been developed to perform a mini-audit, which can help optimize the way files are written and manage budgets. This skill is not a complete solution but can be used for personal projects to assist in auditing and optimization. 10s
• The process involves sharing a terminal to demonstrate the execution of the skill in real-time, using a project called Issue Crush. The audit checks various aspects, such as the number of characters in a file, build and test commands, and the presence of duplicate files. 42s

Utilizing Skills for Auditing and Optimization

• The audit identifies that there are 2381 characters in the file and many build and test commands, which should be used as sub-instructions. It also checks for duplicate files and suggests using a YAML file to help with budgeting. 2m6s
• The audit reveals that there is only one MCP configured, which is the GitHub MSP, necessary for certain application calls. It also notes the presence of hardcoded premium model references, which could be reevaluated using the Copilot SDK. 2m6s
• The audit provides results with high and medium priority issues and highlights six areas that are being handled well. It notes that the Copilot instructions currently use 595 tokens, suggesting there is room for optimization by tightening the context and reevaluating the necessity of certain commands. 2m6s
• The discussion involves optimizing processes by specifying desired outputs and using more cost-effective models, particularly in applications that analyze issue content to provide succinct descriptions. This approach does not require highly intelligent models, which can save resources. 10s
• YAML files are suggested for inclusion as they can contribute to cost savings. Implementing these strategies can streamline workflows and reduce expenses. 42s
• Skills are highlighted as fully editable and configurable, allowing for personalized use. They serve as a starting point for conducting audits with checklists, which can be beneficial at both individual and enterprise levels. This methodical approach helps in evaluations without worrying about costs. 2m6s
• There is an emphasis on changing mental models rather than panicking about budget constraints. The focus is on achieving efficiency and improving work processes, which can be more engaging than merely discussing budgets. 2m6s

Conclusion and Future Plans

• The speaker plans to edit and upload a video with improved sound quality, as the original recording was made in a noisy public place. The video will cover the same topics discussed but in greater depth. 2m6s
• The discussion involves making edits to various files and running a skill to evaluate the audit process, emphasizing the importance of education and information sharing within teams. 10s
• There is a focus on the inevitability of using models and agents in the future, highlighting the need to learn how to work effectively with them. 42s
• Gratitude is expressed to participants, with a reminder that the session is available on YouTube for those who wish to view it from the beginning. 2m6s
• Concerns about costs after June 1st are mentioned, but it is advised not to panic. Enterprises are reminded of the advantage of receiving credits for the next three months to aid in the transition. 2m6s
• Enterprises are encouraged to contact their sales representatives to understand the value and duration of these credits, and to review their accounts to assess usage and efficiency. 2m6s
• The speaker offers to communicate difficult messages to companies on behalf of individuals who do not want to be seen as the "bad guy." 2m6s
• The session concludes with thanks and a farewell, inviting participants to join the next session. 2m6s