Skip to content

datarobot_genai.dragent.eval.guideline_adherence

guideline_adherence

nat eval plugin: in-process guideline adherence (DataRobot moderations OOTB scorer).

DataRobotGuidelineAdherenceEvaluatorConfig

Bases: EvaluatorLLMConfig

Guideline adherence evaluator using the DataRobot moderations LlamaIndex judge.

Source code in datarobot_genai/dragent/eval/guideline_adherence.py 
36
37
38
39
40
41
42
43
class DataRobotGuidelineAdherenceEvaluatorConfig(
    EvaluatorLLMConfig, name="agent_guideline_adherence"
):  # type: ignore[call-arg]
    """Guideline adherence evaluator using the DataRobot moderations LlamaIndex judge."""

    agent_guideline: str = Field(
        description="Natural-language guideline the agent response must follow.",
    )