Skip to content

datarobot_genai.dragent.eval.agent_goal_accuracy

agent_goal_accuracy

nat eval plugin: in-process agent goal accuracy (DataRobot moderations scorer).

DataRobotAgentGoalAccuracyEvaluatorConfig

Bases: EvaluatorLLMConfig

Agent goal accuracy evaluator using the DataRobot moderations LLM judge.

Source code in datarobot_genai/dragent/eval/agent_goal_accuracy.py 
38
39
class DataRobotAgentGoalAccuracyEvaluatorConfig(EvaluatorLLMConfig, name="agent_goal_accuracy"):  # type: ignore[call-arg]
    """Agent goal accuracy evaluator using the DataRobot moderations LLM judge."""