Formal Methods for Every Engineer in an AI-Powered Future

Introduction to AI and Formal Methods in Software Engineering

• The decisions made about AI adoption, architecture trade-offs, and team collaboration will have a long-lasting impact on systems, and making the right choices is challenging due to the rapidly changing landscape 10s.
• QCon San Francisco has been connecting senior engineers with practitioners who are ahead in solving similar problems for 20 years, and this year's event will feature 60-plus speakers sharing their experiences and knowledge from November 16th to 20th 42s.
• Gabriella Moreira is a formal methods person who has been working to make formal methods more accessible for about 8 years, and her interest in the field began when she was introduced to TLA+, a formal specification language, while deciding on her bachelor's thesis 2m6s.
• Gabriella's background includes research work on type systems, specifically for Haskell, and developing tooling for TLA+, which led to her joining Formal Systems, a company that shared her interests, and building a new specification language called Quint 2m6s.

Overview of Quint and Formal Methods Accessibility

• Quint is a more accessible version of TLA+, designed to address the difficulties new users faced when learning TLA+, and Gabriella has been leading the development of Quint for 4 years, recently transitioning into the CEO role as Formal Systems spins out Quint 2m6s.

Why Software Engineers Should Care About Formal Methods

• Software engineers should care about formal methods because they already face the problem of not being able to think about all the edge cases and corner cases in complex systems, and formal methods can provide a solution to this problem, rather than being something they feel responsible for learning 4m10s.
• Formal methods help define sequences, behaviors, and properties of systems, and can be applied at various levels, such as the scheduler of an operational system or user interactions with an interface, to find corner cases and understand systems better 10s.

AI and Formal Methods in Practice

• When facing complex environments where sequences of events have an impact, software engineers can use AI to write formal specifications, such as TLA+ specifications, which can be easier to start with than learning a new language or tools 2m6s.
• Running the specification produced by AI, such as model checking, can provide the most value, as it allows engineers to see the behaviors of the state machine and understand their system 2m6s.

Formal Methods vs. Traditional Testing

• Formal methods differ from good test design in that they can reason about complex and abstract levels, such as distributed or concurrent systems, where it is hard to write tests, and can help come up with scenarios for testing 4m42s.

Integrating Formal Methods into Software Workflows

• Formal methods should be integrated into workflows, such as testing and production monitoring, and can be used for model-based testing, where the model or specification is used to write tests 8m10s.
• The integration of formal methods into workflows is easier in 2026, as AI can generate specifications and help with testing, allowing engineers to focus on defining properties and behaviors, and then using the model for testing 10m0s.

Model-Based Testing and AI Assistance

• Model-based testing requires writing code, known as glue code or harness code, to replay behaviors from a model against the actual implementation, which can be annoying to write and maintain, but with AI, this process can be delegated and made easier to generate and maintain 10s.
• The development of open-source libraries, such as Quint Connect for Rust, has made it possible to define glue code with a nice interface that works as guardrails for AI, allowing for easier generation and maintenance of testing code 2m6s.

Runtime Monitoring and Generative AI

• Leveraging models for runtime monitoring and trace validation is also being explored, as the input from production and runtime is important, and AI can make these techniques easier to implement and more feasible 4m42s.
• Defining behavior with precision is crucial, especially in the realm of generative AI, and requires a lot of understanding, as there is no easy way out or checkmark that guarantees correctness, but rather a process of refining understanding and defining properties and behaviors 6m15s.

Getting Started with Formal Methods

• To get started with formal methods, an engineer should begin by defining their system or solution in a formal language and then use model checking to understand the specification, starting with random executions of the model to gain insight 10m30s.
• The process of formal methods involves defining properties and behaviors, refining the state machine, and leveraging these definitions for testing and determining correct behavior, which can be applied to both testing and production environments 8m45s.
• When a model exhibits random behavior, it can be refined by writing or refining the model or its properties to prevent unwanted behavior from occurring, and then running the model again to verify the updated properties 10s.

Defining System Behavior and Failure Modes

• Defining failure modes is crucial when dealing with distributed systems, as it helps understand how the system behaves and responds to failures, and this information can be used to determine what happens when the system has more failures than it can tolerate 1m42s.
• Witnesses can be defined as properties that are expected to break in order to witness certain behaviors, and they can be used to challenge the tools and see how useful they are for finding rare and difficult edge cases 2m6s.
• Using witnesses in combination with properties can help ensure that properties are not being checked in a vacuum, and defining specific behaviors that matter the most can provide valuable documentation that can be executed and communicated to colleagues 3m10s.

Formal Languages and Abstraction

• Formal languages can be used to make abstractions concrete and executable, allowing for reproducible behaviors that can be documented, executed, and checked, which is a valuable aspect of formal systems 4m30s.
• Personal experience with formal systems has shown that it can be challenging to understand complex protocols, but working with experts in the field can help overcome these challenges and provide a deeper understanding of distributed systems and fault tolerance 5m40s.
• Writing a formal specification for an algorithm can help with understanding and confidence, as it allows for the identification of corner cases and the creation of reproducible examples, which was a personal experience that felt amazing and led to a deeper understanding of the algorithm 10s.

Model-Code Synchronization and Challenges

• The relationship between a model and a code base is that the model should be more abstract than the code, otherwise the power of abstraction is lost, and the model is basically just reasoning about the code itself, although in some cases it still makes sense to have a more concrete model 2m6s.
• There are two techniques to keep the model and code base in sync: model-based testing, which involves replaying traces from the model against the code and checking that the resulting state in the code matches the resulting state from the model, and trace validation, which involves logging production executions and checking if the behavior is compliant with the model 2m6s.
• Historically, people have written models and then left them unused, which can lead to the code deviating from the spec over time, resulting in bugs in production that were not present in the model, as seen in a case from AWS where a DNS outage occurred due to a bug that was not present in the original design 4m42s.

Preventing Divergence Between Model and Code

• The use of formal methods, such as model-based testing and trace validation, can help prevent such problems by keeping the model and code base in sync, and this is not a new problem in software engineering, but rather a new solution to an old problem 6m15s.
• The case from AWS, which was presented at a re:Invent conference, demonstrates the effectiveness of using formal methods and AI tools to compare the model with the code and identify divergences, allowing for the correction of the code and prevention of similar problems in the future 8m10s.

Formal Methods and AI Collaboration

• Formal methods have been around for at least 10 years, and they provide a better solution when combined with AI, as AI can generate glue code, instrumentation, or parse logs to extract behavior and check it against a specification 10s.
• The integration of formal methods with AI can help engineers define what behaviors are correct or not, and this part of the profession cannot be taken away by AI, as it requires human thinking about edge cases 2m6s.
• While AI can generate code, it is not capable of replacing the human aspect of defining complex behaviors, especially in cases where concurrency issues and difficult event orderings are involved, and this is where formal methods can help 2m6s.
• By using formal methods, engineers can understand the system better, even if AI is writing the code, and assert that the AI-generated code is correct, which can lead to confidence in the product and value in the profession 2m6s.

Conclusion and Further Engagement

• For those interested in learning more, Gabriela can be found on LinkedIn, where she is active and open to connections and continuing the conversation 10m0s.