Context is the Key to the Agentic Architecture Revolution: A Conversation with Baruch Sadogursky

Introduction to AI Adoption and Architectural Challenges

• The decisions made about AI adoption, architecture trade-offs, and team collaboration will have a long-lasting impact on systems, and making the right choices is challenging due to the rapidly changing landscape 10s.
• UCON San Francisco has been connecting senior engineers with practitioners who are ahead in solving similar problems for 20 years, and this year's event will feature 60-plus speakers sharing their experiences on what works and what doesn't in production 42s.
• Baruch Sadogursky, the guest, has a background in working with Java before generics, DevOps before Docker, and DevRel before it had a name, and he is currently focused on context engineering management and sharing, particularly with AI agents 2m6s.
• Baruch Sadogursky became an architect through on-the-job training, working with his former boss and mentor, Schlom Benhim, and other experienced consultants, where he learned to understand systems, complexities, and when architectural expertise is needed 2m6s.

Evolution of AI in Code Generation and Context Engineering

• The concept of using AI agents to write code is not new, as there have been previous attempts to create requirements definition languages for page tools to generate code automatically, and the challenge now is to express requirements in an unambiguous way, which is related to context engineering 2m6s.
• The use of AI agents in coding is different from previous attempts, as it involves a reasoning machine that can assist in the process, which was not available before, and this changes the approach to context engineering and prompt engineering 2m6s.
• The downsides of human languages, such as English, as programming languages, have always been a barrier to overcome, but with the emergence of reasoning entities that can interpret ambiguities, this barrier is becoming less significant, allowing for the possibility of making programming languages more accessible 10s.
• Expressing ambiguity clearly in languages is a challenge, and architects often have to deal with clients who present solution statements rather than problem statements, requiring the architect to ask clarifying questions to understand the client's needs 1m30s.
• To address this issue, architects can use techniques such as asking questions like "fast compared to what" and providing scenarios and monetary tradeoffs to help clients clarify their requirements, and AI agents can be designed to do the same 2m45s.
• The intent integrity kit is a framework that includes clarifying loops to ensure that the reasoning system understands the user's intentions, and it can be used to write specs and ask questions to clarify requirements at every step of the process 5m10s.
• The intent integrity kit was built on top of the spec kit by GitHub, and its development was driven by the understanding that ambiguities between what people want, what they say they want, and what the machine understands are a major barrier to creating effective programming languages 7m20s.

Clarifying Ambiguity and the Role of AI Agents

• To force AI models to ask clarifying questions, prompts can be designed to include instructions such as "ask at least three clarifying questions" to encourage the model to think critically and find multiple questions to ask 10m30s.
• The process of triggering reflection can be done today by setting the right context for the agent, which is essential for avoiding ambiguities and ensuring the agent receives the correct information 10s.
• Prompt engineering has been a topic of emphasis, but it is not a true engineering discipline, as it involves tweaking prompts in the hopes that the agent will miraculously receive the missing context, rather than providing a clear and intentional context 42s.
• Context engineering is a more appropriate approach, which involves thinking about requirements, such as providing clear intent from the human to the machine, and measuring the effectiveness of the context provided, with the goal of creating a feedback loop to improve the context 2m6s.
• This approach to context engineering amplifies the importance of human expertise, as humans are responsible for defining the requirements and providing the correct context for the agent to do the right thing, and no amount of prompt hacking can replace this 10s.

Context Engineering as a Discipline

• A three-way conversation between the agent, the architect, and the developer team, and the client is necessary to ensure that the agent has the correct context, and this conversation requires client management skills, which architects may not always possess 42s.
• The approach to context engineering involves thinking about how to measure the effectiveness of the context, creating a feedback loop, and scaling the solution, making it a proper engineering discipline that differs from the voodoo conditions of prompt hacking 2m6s.
• Baruch Sadogursky emphasizes that context engineering is about providing missing instructions and clear intent from the human to the machine, and that this requires a thoughtful and intentional approach, rather than relying on magical incantations or divine revelation 42s.
• The architect can act as an intermediate to translate the client's language into context for the agent, or a system can be built where both the architect and client access the agent directly, allowing the agent to ask clarifying questions to ensure understanding of the client's requirements 10s.
• To preserve context and history between interactions, a technical solution is needed, which can be achieved by saving clarifying questions and answers in a document or UI, making it easy to refer back to the conversation history 2m6s.
• Clients may attempt to remove the architect from the process, but they often lack technological sophistication, and the agent can be engineered to request additional context or clarification from the architect if necessary 2m6s.

Liability and Responsibility in AI-Driven Systems

• The use of independent agents raises questions about liability, as it may be unclear who is responsible if an agent makes a mistake, and the fault could be at the level of the model, the harness, or the external context provided 4m6s.
• The agent can be designed to request additional information or context from the client or architect if it is unclear or incomplete, which can help to ensure that the client understands the importance of providing accurate and sufficient information 4m6s.
• The relationship between the client, architect, and agent is complex, and the agent can be used to facilitate communication and ensure that all parties are on the same page, which can help to prevent mistakes and misunderstandings 4m6s.
• The model's weaknesses can be a genuine limitation, and even if the model's producer is aware of these limitations, they may not be considered liable, but if there is a problem with the context provided, it is the responsibility of the person who provided the context 10s.
• The boundaries between the model's limitations, the context provided, and the client's instructions can be hard to define, and it can be challenging to determine what went wrong, but the definition is simple, and documentation is crucial for assigning blame and continuous improvement 42s.
• Agent logs are essential for evaluating system performance, assigning blame, and continuous improvement, and they should be preserved for these purposes, allowing for an engineering discipline with feedback loops and continuous improvement 2m6s.

Predicting and Improving Code Quality with AI

• The legal sphere will likely struggle with the concept of liability in software development, especially with the use of agents and artificial intelligence, and it may be unavoidable to figure out who to sue in case of errors or violations 4m10s.
• Successfully created designs with implementations can be tested and put into production, and feedback from customers and the system can be used to improve the architecture, and this process can be shifted left into a negative territory, where code can be assessed before it is even written 6m30s.
• The context provided to agents determines the quality of the code produced, and assessing the context can give a good idea of the code's quality, even before it is written, and this can lead to producing software of higher quality than ever before 8m40s.
• A practical example of assessing code quality before it is written is a demo that showed how an agent can write code based on a prompt, and a tool like Sonar can catch errors and bad code on commit, preventing it from going into production 10m50s.
• The process of improving code quality involves using a reasoning agent as a sidekick to enhance the prompt, which eventually becomes a context artifact that produces higher-quality code, and this can be achieved with a high degree of confidence 10s.
• It is now possible to predict the quality of the code that will be generated and improve it before even starting to generate a single line of code, which was previously considered impossible 10s.

Managing Change and Spec-Driven Development

• When dealing with changes or feedback from customers, there are old and new ways to proceed, including going back to the spec, changing it, and running the whole process again, or using the new approach where the spec is the source of truth and any changes are made to it first 2m6s.
• The new approach involves using an intent integrity chain that guarantees what's in the spec will end up in the code, and any changes made to the spec will be reflected in the code, making the spec the single source of truth 2m6s.
• To make delta changes, it's necessary to use small pieces of code or small modules, such as microservices, which are now more important than ever because they allow for fast, cheap, and efficient regeneration of code 2m6s.
• The use of microservices is crucial when driving code generation from the spec, as it allows for smaller scopes and better performance by the agent and model, making small features, small modules, and microservices more important than ever 2m6s.
• The importance of microservices is highlighted when considering the limitations of context and context window, where smaller scopes perform better, and the use of monoliths may not be suitable for agent-generated coherent pieces of code 2m6s.

Architectural Responsibilities and System Illities

• The illities, such as scalability and security, are emerging properties that fall under the responsibility of the architect, as they cannot be written into a use case and require someone to oversee them 10s.
• The tradeoff of microservices is that the complexity of a unit is traded for the complexity of orchestration, and since agents can only deal with small pieces, orchestrating those pieces becomes more complicated and requires an architect 42s.
• The current model consists of three tiers: essential services designed by AI agents, a user interface that may be designed by agents or humans, and a middle tier where the application is built and requires an architect to orchestrate the pieces 2m6s.
• The agentic revolution may lead to a future where agents can grasp and implement complicated architectures, but for now, architects are still necessary to orchestrate microservices 4m30s.

The Future of Architectural Learning and AI Agents

• The development of agents is replicating how humans learn to be architects, starting with small tasks and increasing complexity as understanding grows 6m15s.
• Context engineering is important and requires consistency, a feedback loop, and distributability, but the implementation of context engineering is also crucial, which involves the creation of context artifacts 8m40s.
• Context artifacts consist of three pieces: skills, which are rules for agents to follow, rules of what to do and not to do, and scripts, which are deterministic logic that agents can use to operate 10m20s.

Implementing and Distributing Context Artifacts

• Creating context artifacts is essential for nondeterministic systems like AI, and these artifacts need to be engineered, run through rigorous feedback, tested, versioned, and distributed, similar to npm modules, jar files, or Docker images 10s.
• To get started with building agents, one can begin by asking their AI agent to write a context artifact for a particular part of knowledge, and then work with modern agent code, such as Gemini or Codex, without requiring any secret knowledge or special tools 2m6s.

Resources and Further Learning

• For those looking to learn more, recommended resources include the Tesla blog at tesl.io/blog, which features articles on this topic written by various colleagues, and a keynote from the Arc of AI conference, which discusses how AI can become a team member that interacts with architects and users 42s.

The Emotional and Philosophical Side of Being an Architect

• The favorite part of being an architect is the magic of creation, where ideas are turned into complex systems through code, and solving hard questions like software architecture and tradeoffs, which can be a pleasant and rewarding experience 42s.
• The least favorite part of being an architect is dealing with tradeoffs that cannot be solved with a win-win, which can be frustrating and annoying, as complexity cannot be eliminated and will always be present in some form 10s.
• The complexity of the problem domain cannot be eliminated, and it is the developer's job to minimize and manage this complexity, which represents real-world complexity that must be paid for either in code or orchestration 10s.
• Solving complex real-life problems translates to complex software, and the trade-off between microservices and monoliths is a prime example of this, where complexity must be paid for somewhere 42s.
• Problem-solving is engaging on all levels, including technically, spiritually, and emotionally, and being a software architect can be creatively and emotionally engaging, similar to being a physical architect, but with the ability to create something by typing on a keyboard 2m6s.
• One of the frustrating aspects of being an architect is dealing with human imperfections, such as customers who do not understand what they want or how to express it, but AI agents may be able to shield architects from this frustration in the future 4m30s.

The Role of Technology in Enhancing Architectural Work

• The use of certain technologies, such as generative AI, can be extremely beneficial and make creation easier, allowing architects to generate and test ideas quickly, and making them happier and more productive in their work 6m15s.
• Generative AI is forgiving of mistakes, such as typos and syntax errors, and can ask for clarification if it does not understand something, making it a powerful tool for architects and a key part of their workflow 8m40s.
• The ability to manage complexity in an engineering way is what is loved about architecture, as it allows for knowledgeable decisions to be made about complexity, which is a fundamental aspect of the field 10s.
• Complexity is a reality of life that cannot be eliminated, but it can be minimized, massaged, and made less disruptible, which is a key goal of architecture 42s.

Reflections on the Architect Profession and Alternatives

• Being a developer advocate is a fulfilling profession that combines software architecture with speaking to people at conferences and communicating with other software architects and developers 2m6s.
• If a different profession were to be chosen, being a real-world architect would be a satisfying career, as it is an admired profession that requires skills that are not currently possessed 2m6s.
• Architectural engineering tools can create 3D models and visualizations, allowing people to do more than what human limitations previously allowed, and exploring these tools could be a viable option 4m30s.
• Being an architect is not just a day job, but a way of thinking that involves seeing the world in systems and complexity tradeoffs, and it is something that cannot be undone once it is developed 6m15s.

Closing Thoughts and Recommendations

• The most important feedback that can be received from clients or teams is that the intent has been captured, and that the project was completed on time, on budget, and with valuable lessons that can be shared and applied to future projects 8m0s.
• The conversation has provided a clear glimpse into the future of architecture, and the importance of context in the agentic architecture revolution, which is a valuable and insightful perspective 10m30s.
• Attendees are encouraged to attend conferences, such as the Cucon conference, which are highly recommended for their value and quality 12m0s.