Shreya Rajpal on Guardrails for Large Language Models

Guardrails AI

• Guardrails AI is an open-source framework that enhances the reliability and safety of large language model (LLM) applications.
• It acts as a "firewall" around LLMs, checking inputs and outputs for correctness and filtering out unreliable or harmful content.
• Guardrails AI employs various techniques, including few-shot prompting, validators, and machine learning models, to ensure correctness.
• It operates as a "side car" that runs alongside the LLM, checking prompts before they are sent and outputs before they are returned to the application.
• Correctness criteria can be customized to meet specific user needs.

Applications of Guardrails AI

• Common applications of Guardrails AI include chatbots, structured data extraction, resume screening, healthcare support chatbots, and contract analysis.

Implementation and Customization

• Implementing Guardrails AI involves additional compute and latency, but it can be configured to balance risk mitigation with performance.
• Guardrails AI supports customizability, allowing users to create their own guardrails and integrate with different language models.

Recent Updates and Techniques

• Recent updates to Guardrails AI include improved logging, visibility, and stability.
• Guardrails AI employs a "reasing" paradigm, where incorrect outputs are sent back to the language model for self-correction.
• Reasing optimizes the prompt to focus on correcting only the incorrect parts of the output, reducing computational costs.

Prompts and Verification

• The speaker emphasizes the importance of understanding the specific correctness criteria for the application and configuring Guardrails AI accordingly, rather than relying solely on structured prompts.
• Prompts do not guarantee correct outputs due to the non-deterministic nature of LMs and the possibility of incorrect responses despite specified instructions.
• Safer prompts can be used to prime the LM for more accurate responses, but verification is necessary to ensure that the desired conditions are respected.
• Verification systems are crucial to ensure that the conditions the user cares about are adhered to.