Measuring the Effectiveness and Performance of AI Guardrails in Generative AI Applications

Safeguarding AI agents and other conversational AI applications to ensure safe, on-brand and reliable behavior is essential for enterprises. NVIDIA NeMo Guardrails offers robust protection with AI guardrails for content safety, topic control, jailbreak detection, and more to evaluate and optimize guardrail performance. 

In this post, we explore techniques for measuring and optimizing your AI guardrails’ effectiveness, ensuring that your AI applications operate within carefully defined boundaries while meeting user experience and other design requirements.

NeMo Guardrails provides an evaluation tool for monitoring policy compliance rates given specific AI guardrail policies. In addition to policy compliance rates for LLM-generated responses, the tool offers insights into key performance metrics such as latency, and LLM token usage efficiency. This post guides you through the evaluation tool, highlighting how each metric plays a role in optimizing your AI applications. 

As enterprises continue to push the boundaries of what AI can do, balancing performance and cost efficiency has become a top priority. The NeMo Guardrails evaluation tool helps navigate key trade-offs, whether maximizing throughput or optimizing token-usage, and provides comprehensive insights to guide decision-making. Here’s how leveraging these insights can further optimize your generative AI applications for improved performance and reliability. 

Evaluating AI guardrails configuration with NeMo Guardrails

NVIDIA NeMo Guardrails introduces a robust evaluation methodology that uses policy-based guardrails to enforce the desired behavior of your AI agent or chatbot assistant. 

At the core of this approach is the idea that each guardrail configuration should be designed to align with a set of well-defined policies, whether it’s preventing toxic content, ensuring on-topic responses, or delivering factually correct information. 

This methodology is based on using a curated dataset of interactions to measure policy compliance. Each interaction in the dataset is matched against the expected behavior as defined by your policies, enabling you to compute a policy compliance rate, which is the percentage of interactions that fully comply with your guardrail rules. This single metric, akin to accuracy in traditional machine learning evaluations, provides an immediate insight into the overall effectiveness of your configuration. For a comprehensive understanding, it’s important to also consider metrics such as LLM response latency, token usage, and overall throughput. 

In the sections that follow, we discuss the details of setting up your evaluation configuration. For more information about defining precise policies, creating and annotating interaction datasets, and employing both automated LLM-based judges and manual reviews to assess policy compliance, see the comprehensive evaluation methodology guide.

Defining evaluation policies

For this example, consider the example ABC bot which is a simple RAG chatbot with a knowledge base composed of company information. The behavior of the chatbot is defined in the policies.yml file with the following policies:

• Content moderation: Llama 3.1 NemoGuard 8B ContentSafety NIM microservice enhances the safety and moderation capabilities of LLMs by not letting the bot engage in toxic, unsafe content.
• Topic control: Llama 3.1 NemoGuard 8B TopicControl NIM microservice ensures that the bot keeps conversations focused on approved topics, avoiding inappropriate content.
• Jailbreak detection: NemoGuard JailbreakDetect NIM microservice for protection from adversarial attempts.

To create an effective guardrail evaluation configuration, follow the essential steps of creating a comprehensive interactions dataset, using an LLM-as-judge for an initial evaluation and manually annotating any complex interactions to ensure accuracy and reliability. 

Creating the interactions dataset

We’ve curated an in-house dataset of 215 interactions, with approximately 10% being multi-turn interactions. These multi-turn interactions offer insights into the dialogue dynamics by capturing extended exchanges between the user and the chatbot. 

Each interaction includes an expected_output attribute that specifies the desired response, for example, refusal when evaluating an input-moderation policy. 

To build these interaction sets, both synthetic data generation and real, expert-annotated data are effective approaches. Synthetic generation offers a straightforward method when annotated datasets are unavailable, though it requires iterative refinement and filtering, while real data ensures the highest level of relevance and accuracy.

Using the LLM as a judge

A powerful LLM can serve as an effective judge for computing the policy compliance rate by determining if the actual responses adhere to the expected outputs. To ensure high accuracy in this automatic evaluation, we recommend establishing strong, clear rules for the LLM-as-a-judge, run the judge multiple times on the same dataset to check for inconsistencies, and validate results with a subset of manual annotations for each policy. 

In particular, manual annotations should always be applied—especially for interactions where the LLM’s evaluations are inconsistent—to ensure a robust and reliable overall assessment.

For more information about these steps, see the evaluation configuration offered by NeMo Guardrails. 

Evaluation workflow

Figure 1 shows how user requests flow through various components of the evaluation tool. Starting from the user, the query is routed through to NeMo Guardrails for initial input processing. The latest AI safeguard NVIDIA NIM microservices are integrated into NeMo Guardrails to analyze the user request for content safety, topic control, and jailbreak detection. 

If the conversation requires retrieval of external information, the system can query a vector database that holds relevant documents, for example, HR policies or generic company information. The LLM then processes both the user query and retrieved chunks, generating an appropriate response. 

Throughout this process, the evaluation tooling tracks the interactions, computing policy compliance rates, latencies, and more. This holistic flow ensures that both user-facing and internal guardrail logic are rigorously monitored and measured.

Running evaluations and analyzing results

The evaluation tooling manages data collection and reporting. You can use the CLI to perform the following tasks:

• nemoguardrails eval run: Run a set of interactions against your guardrail configuration.
• nemoguardrails eval check-policy compliance: Check policy compliance using an LLM judge.
• nemoguardrails eval ui: Launch the Eval UI for an interactive view of the results.

Through the Eval interface, you can visually inspect policy compliance rates, resource usage (number of tokens and calls), and latency statistics. This interface helps you pinpoint any gaps in your guardrails or areas where performance might be improved.