Skip to content

LLMs in workflow.yaml

This matches what appears under llms: in the examples, e.g. e2e-tests/dragent/nat/workflow.yaml.

What you usually declare

llms:
  datarobot_llm:
    _type: datarobot-llm-component

datarobot-llm-component is the flexible option: it follows the same gateway / deployment / NIM / external routing as your environment variables (see LLM configuration (shared)). You can name the key anything (datarobot_llm is just the label used elsewhere as llm_name).

Other _type values you may see

_type When it appears
datarobot-llm-gateway Gateway only; no per-deployment id in the block.
datarobot-llm-deployment Fixed LLM deployment; often includes deployment id and optional headers in YAML.
datarobot-nim NIM deployment on DataRobot.
datarobot-litellm External LiteLLM providers; provider keys still come from the environment.
datarobot-llm-router Primary + fallback LLMs with automatic failover via LiteLLM Router; includes primary, fallbacks, and optional tuning field num_retries.

The exact fields inside each block mirror what you would set in env for routing (model name, gateway on/off, deployment ids). Prefer the same env vars as the e2e tests unless you need to pin something in YAML for a deployment.

Passing extra kwargs with extra_body

Add extra_body to any LLM block to forward arbitrary key-value pairs in the request body. Works with every _type. This is a dedicated top-level field; LangGraph clients move it into model_kwargs.extra_body automatically.

llms:
  datarobot_llm:
    _type: datarobot-llm-component
    extra_body:
      mock_response: "this is a mock response"

Enable LLM extended reasoning (parameters format for Anthropic model before version 4.6).

llms:
  datarobot_llm:
    _type: datarobot-llm-component
    extra_body:
      thinking:
        type: enabled
        budget_tokens: 1024

See litellm documentation for a specific provider to setup correct extra_body for your model.

Linking workflows to an LLM

Any workflow or functions.* entry that needs a model sets llm_name: to the key under llms: (e.g. llm_name: datarobot_llm).