Developing Regulated Software at the Speed of Innovation: Insights from Erez Kaminski

Erez Kaminski and Krix

• Erez Kaminski is the founder and CEO of a company called Krix, which helps people develop regulated software at the speed of regular software 46s.
• He has a background in physics and started his career working on control methods and automation for fusion reactors and fusion reactions 56s.
• Kaminski worked at Wolfram Research, developing Mathematica, Wolfram Alpha, and the Wolfram Cloud, and was part of the team that developed the largest expert rule-based engine ever built 1m37s.
• He later became the head of AI for the medical device group at Amgen, a large multinational pharmaceutical company, where he worked on the design and development of safety-critical systems 1m58s.
• Kaminski left Amgen to pursue higher education at MIT, where he started Krix, aiming to combine his passion for developer tooling and the developer ecosystem with the need to develop regulated software at the same speed as regular software 2m11s.
• He believes that the math used in physics is far ahead of what is used in day-to-day products, and he wants to help apply advanced applied mathematics through software in the realm of safety-critical systems 2m52s.
• Kaminski thinks that developing safety-critical systems is a challenging journey that requires considering the potential risks and consequences of the software, which is often overlooked in non-safety-critical software development 3m32s.
• Safety-critical software development requires control, reliability, and complexity to ensure the software executes as intended, especially with featureful software that makes decisions and uses powerful algorithms 4m3s.
• The goal of safety-critical product development is to prove that the software consistently and reliably performs its intended function in a maintainable fashion 4m24s.

Safety-Critical Software and Examples

• Examples of safety-critical software include medical devices, such as the Klar implant, which replaces the human sense of sound and uses complicated algorithms to function 4m43s.
• Ensuring the safety, reliability, and serviceability of such devices over an extended period, even after the original engineers have left the company, is a significant challenge 5m2s.
• As software complexity increases, so do the potential issues and mistakes, which can be mitigated through software validation and risk management 5m33s.

Software Validation and Risk Management

• Software validation involves providing objective evidence that a system conforms to its intended use, while risk management involves understanding the hazards that emerge from the software's functions and development 5m40s.
• Risk management requires critical thinking and planning to ensure that the cost and safety profile of each component are appropriate for the intended use 7m6s.

Challenges in Safety-Critical Software Development

• The increasing complexity of safety-critical software has made it difficult to manage their life cycle and documented evidence, leading to slow development times 7m29s.
• There is a need to develop a faster way to manage the life cycle of safety-critical software, as its use is becoming more widespread in society 7m39s.
• The frequency of critical systems failing is increasing over time, resulting in more catastrophic and newsworthy events, which can be mitigated by changing the way products are built for such scenarios 7m48s.
• Developing safety-critical systems, such as pacemaker software, takes more time due to the brakes put on the process, but the current amount of time and brakes may not make sense anymore 8m21s.
• The methodologies used for developing safety-critical systems are often old school, and the tools developed to reduce software complexity, such as task management and devops tools, were not built with control for safety-critical systems in mind 8m36s.

Regulation and Societal Expectations

• Society expects quality, reliability, and safety checks in products, especially in regulated industries like medical devices, pharmaceuticals, and automotive, which is why regulators are in place to ensure safety 9m32s.
• Regulators give society a voice in ensuring that products are safe, and people generally expect safety checks, especially when it comes to products that can injure or kill, such as pacemakers 9m40s.
• While some developers may prioritize rapid release cycles, most people agree that safety-critical systems require more checks and validation to ensure they work as claimed 10m12s.
• Society has voted to regulate certain industries through their voting acts, and while not all regulation is good, many regulations are appropriate and necessary to ensure safety 10m46s.
• Being regulated means building a product that matters to society, and society has decided that there need to be rules about developing such products for specific use cases 11m11s.
• Every country has an agency that monitors the development of medical products, highlighting the importance of regulation in this field 11m28s.

Standard Writing and Committees

• Standard writing involves creating guidelines, technical reports, or standards to help others develop safe products, often requiring patience, organization, and committee work 11m41s.
• There are various committees and societies, such as ISO, AAMI, and ISP, that publish standards for different regions and industries, which companies use to show conformance 12m44s.
• Standards aim to ensure that developers do not forget essential steps, even if they lack experience, by providing a checklist of obvious things to do 13m11s.
• The standard writing process is distinct from software development, requiring more organization, committees, and patience 13m21s.
• Mathematicians and developers may find it challenging to adapt to the slow pace of standard writing, as they are accustomed to working quickly 13m30s.
• One of the standards worked on is for risk management in machine learning and medical devices, and another is for medical clouds, focusing on compliance with regulations and safety requirements 13m49s.

Modern Challenges and Validated DevOps

• The increasing automation of devices and systems, including in the medical field, raises questions about what validation looks like in these spaces 14m17s.
• Currently, validation often involves massive teams and thousands of pages of documented evidence, but there is a desire to make this process more efficient and developer-friendly 14m41s.
• Regulated software development involves producing extensive documentation, often hundreds of pages long, to provide evidence of process steps and ensure the software meets its intended use, with examples including staff training, rigorous process analysis, design verification, and testing associated with features and requirements specifications 15m21s.
• This documentation serves as objective evidence that the system can reliably and safely meet its intended use, and while it may seem burdensome, it is essential for ensuring safety and maintainability, particularly in critical applications such as pacemakers 16m32s.
• The introduction of validated DevOps, which combines development, operations, and computer automation, is expected to make the production of this evidence less burdensome and more efficient, allowing for faster development while maintaining safety and reliability 16m57s.
• Validated DevOps involves connecting different IT systems to ensure that activities done in one system are prerequisite for activities done in another, and connecting them in a robust manner to prevent errors and ensure traceability 18m26s.
• This approach is expected to become more widespread in the future, not just for safety-critical applications but also for B2B mission-critical applications, as it provides a way to prove that something works and can help prevent costly challenges 17m39s.
• Validated DevOps is integrated into CI/CD pipelines to prevent errors and ensure that modifications are assessed to prevent unintended changes, allowing for faster development while maintaining safety and reliability 19m1s.
• Implementing automated and verifiable quality assurance systems can provide teams with more freedom to work faster while ensuring compliance with regulations, as these systems can generate evidence of compliance and force developers to follow necessary controls 19m20s.

Cloud-Connected Medical Devices and Security

• Cloud-connected medical devices, such as pacemakers and infusion pumps, can provide benefits like real-time notifications and remote monitoring, but also introduce complexity and security risks 20m46s.
• Infusion pumps, in particular, have many different parts and require access to various data sources, creating complexity and potential attack vectors 21m14s.
• Connecting medical devices to different systems and using various open-source libraries and versions can lead to configuration management challenges and reliability issues 21m53s.
• Breaking down the architecture of medical devices to ensure safety-critical parts are isolated and secure is a significant challenge 22m20s.
• Ensuring the reliability and security of medical devices, especially those connected to the cloud, is crucial to prevent critical errors and ensure patient safety 22m16s.
• Regulated software development requires a significant amount of evidence to demonstrate understanding of system architecture, reliability, and security to ensure that things cannot go wrong, as required by the FDA and EU regulators 22m47s.
• Even with vulnerabilities in libraries used by manufacturers, proper system architecture and barriers can prevent bad actors from controlling devices remotely, as seen in the case of a large medical device company's infusion pump 23m14s.

Interoperability and Continuous Development

• Interoperability of modern medical devices and consumer expectations pose a challenge for manufacturers who have not considered this aspect before, requiring a different way of thinking about product development 23m55s.
• Developing safety-critical products continuously is essential, as it is hard for those who do not do so to think about safety and security, and it requires a different paradigm of thinking about software development 24m14s.
• A developer who worked on language design for languages like Rescript noted that he was taught to only increase functionality, but not to think about reducing features to ensure safety, highlighting the need for a different approach to software development 24m25s.

Future of Safe Software Development

• The need for safe and secure software development will grow in the next 10-15 years, driven by emerging technologies like deep tech, fusion reactors, autonomous vehicles, and AI for pharmaceutical development and medical devices 24m56s.
• The developer community will face a big fracture as more emphasis is placed on creating the right features in a safe way, rather than just adding more features, and some developers may struggle to adapt to this new approach 25m29s.
• The goal is to make it easier for developers to create safe and secure software, so they do not leave regulated industries like healthcare after a short time due to frustration with documentation and the desire to focus on development 25m38s.
• AI-enabled systems will continue to play a significant role in regulated software development, with both opportunities and risks that need to be considered 26m3s.
• The development of safety-critical AI systems is crucial, especially when they are deployed to millions of patients with severe illnesses, and it is essential to think about the limits of AI and how it will impact people's lives 26m11s.

AI and Automation in Regulated Industries

• In the future, most commerce, B2B interactions, and personal life will be dominated by automation, including routine automation, traditional machine learning, and generative machine learning, which will require validation to ensure they interact and work correctly 26m49s.
• Validating generative AI systems is challenging due to the numerous use cases, but companies are starting to perform proper risk management, and a huge amount of software will be developed in validated DevOps, with agents checking each other to ensure they are doing the right things 27m12s.
• The FDA has approved over a thousand medical devices with machine learning, and companies are trying to figure out how to develop these devices in the right way, considering the risk they are taking and the risk to patients 27m48s.
• Revolutionary devices, such as those developed by HeartFlow, are changing the way people work and saving lives, with their AI system being used by a quarter of a million patients every year to detect potential heart problems and provide results equivalent to an interventional lead insertion 28m21s.
• HeartFlow's AI system allows patients to get a CT scan and receive results the same day or the next day, reducing the delay and risk associated with traditional methods, and it is an example of how advanced AI systems can save lives every day 29m4s.

Need for Augmented Abilities and Automation in Healthcare

• Most software in the consumer and cloud web domain is not suitable for industries that need it the most, such as healthcare, and it's impossible to train physicians and subject matter experts fast enough for the growing population 29m38s.
• There is a need to augment the abilities of these professionals, reduce their cognitive load, and make their work easier to train and more productive 29m58s.
• Medical device development and surgeries can be physically challenging, and there is a need to make these procedures easier to learn and perform, potentially through automation and AI 30m22s.
• Automation can make certain medical procedures more accessible, such as cataract removal, which can now be done in more clinics, even in rural areas 31m10s.

The Future of Automation and Safety

• Society is heading towards automation, and it's essential to figure out how to make this automation safe and reliable, especially for the safety of children 31m45s.
• The development of autonomous vehicles has not yet earned a good reputation, but it's expected to improve over time 31m30s.
• The goal is to make automation dominant in life while ensuring it's safe and reliable 31m42s.

Continuing the Conversation

• People can continue the conversation on Twitter and LinkedIn 32m2s.