Glossary

• Agent: conversational agent configured through an API endpoint. They can be evaluated and tested within the Hub.

• Knowledge Base: domain-specific collection of information. You can have several knowledge bases for different areas of your business.

• Dataset: a collection of conversations used to evaluate your agents.

• Checks: a set of evaluations designed to ensure that an Agent exhibits appropriate behavior according to specified criteria. Checks can include correctness, groundedness, conformity, and others.

• Conversations: a collection of messages along with evaluation parameters (checks), such as the expected answer or rules the agent must follow when responding.

• Correctness: Verifies if the agent’s response matches the expected output (reference answer).

• Conformity: Ensures the agent’s response adheres to the rules, such as “The agent must be polite.”

• Groundedness: Ensures the agent’s response is grounded in the conversation.

• String Matching: Checks if the agent’s response contains a specific string, keyword, or sentence.

• Reference: A reference answer that the agent must match when generating a response.

• Rules: A list of requirements the agent must meet when generating an answer. For example, “The agent must be polite”.

• Context: The context in which the agent must ground its response.

• Keyword: A specific string, keyword, or sentence that the agent’s response must contain. For example, “I don’t know”.