datarobot_genai.drmcpbase.tool_metrics

tool_metrics

OTel metrics for MCP tool calls (shared by User MCP and Global MCP servers).

Until now tool calls were observable through spans only — per-tool call counts, latency, and failure taxonomy had to be re-derived by scanning traces. This module adds SLI instruments plus the OTel-recommended server operation duration histogram:

• mcp.server.operation.duration histogram (OTel GenAI MCP semconv)
• datarobot.mcp.tool.calls_total counter, by tool + outcome
• datarobot.mcp.tool.call_failure_total counter, by tool + error.kind + error.type

Latency follows OpenTelemetry GenAI MCP semantic conventions <https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/mcp.md>_ (mcp.server.operation.duration with mcp.method.name=tools/call, gen_ai.tool.name, gen_ai.operation.name=execute_tool, and error.type on failure). MCP attribute registry: https://opentelemetry.io/docs/specs/semconv/registry/attributes/mcp/

error.kind on the failure counter carries the :class:~datarobot_genai.drmcputils.exceptions.ToolErrorKind value. The middleware sees failures AFTER log_execution converted them to FastMCP ToolError strings, so the kind is recovered from the exception's kind attribute when present, else parsed from the "[kind] …" message prefix, else recorded as "unknown".

Metrics ship through the OTLP MeterProvider bootstrap_metrics_provider installs during initialize_telemetry; without OTLP env it resolves to a no-op provider. That makes recording cheap, not infallible, so :func:record_tool_call swallows its own errors — it runs on the success path before the tool's result is returned.

ToolCallInstruments dataclass

OTel instruments backing per-tool MCP SLIs and semconv latency.

Source code in datarobot_genai/drmcpbase/tool_metrics.py

@dataclass
class ToolCallInstruments:
    """OTel instruments backing per-tool MCP SLIs and semconv latency."""

    calls_total: Any  # Counter: denominator + success-rate numerator
    server_operation_duration: Any  # Histogram (seconds): mcp.server.operation.duration
    call_failures: Any  # Counter: failure-taxonomy SLI (by kind + type)

build_instruments

build_instruments(meter: Any) -> ToolCallInstruments

Create the tool-call instruments on meter (an OTel Meter).

Source code in datarobot_genai/drmcpbase/tool_metrics.py

def build_instruments(meter: Any) -> ToolCallInstruments:
    """Create the tool-call instruments on ``meter`` (an OTel ``Meter``)."""
    return ToolCallInstruments(
        calls_total=meter.create_counter(
            TOOL_CALLS_TOTAL,
            unit="1",
            description=(
                "MCP tool calls, labeled by gen_ai.tool.name and outcome (success|failure)."
            ),
        ),
        server_operation_duration=meter.create_histogram(
            MCP_SERVER_OPERATION_DURATION,
            unit="s",
            description=(
                "MCP server operation duration for tools/call "
                "(mcp.method.name, gen_ai.tool.name, gen_ai.operation.name, error.type)."
            ),
            explicit_bucket_boundaries_advisory=_MCP_SERVER_OPERATION_DURATION_BUCKETS,
        ),
        call_failures=meter.create_counter(
            TOOL_CALL_FAILURES,
            unit="1",
            description=(
                "MCP tool call failures, labeled by gen_ai.tool.name, error.kind "
                "(ToolErrorKind value or 'unknown'), and error.type (exception class)."
            ),
        ),
    )

get_instruments

get_instruments() -> ToolCallInstruments

Build (and cache) instruments from the global meter.

Source code in datarobot_genai/drmcpbase/tool_metrics.py

def get_instruments() -> ToolCallInstruments:
    """Build (and cache) instruments from the global meter."""
    provider = metrics.get_meter_provider()
    if _STATE["instruments"] is None or _STATE["provider"] is not provider:
        _STATE["instruments"] = build_instruments(metrics.get_meter(METER_NAME))
        _STATE["provider"] = provider
    return _STATE["instruments"]

resolve_error_kind

resolve_error_kind(error: BaseException) -> str

Resolve the error.kind label value for a tool failure.

Prefers the exception's own kind (drmcputils ToolError); falls back to the "[kind] …" message prefix log_execution puts on the FastMCP-facing error; else "unknown".

Source code in datarobot_genai/drmcpbase/tool_metrics.py

def resolve_error_kind(error: BaseException) -> str:
    """Resolve the ``error.kind`` label value for a tool failure.

    Prefers the exception's own ``kind`` (drmcputils ``ToolError``); falls
    back to the ``"[kind] …"`` message prefix ``log_execution`` puts on the
    FastMCP-facing error; else ``"unknown"``.
    """
    kind = getattr(error, "kind", None)
    if isinstance(kind, ToolErrorKind):
        return kind.value
    parsed = tool_error_kind_from_message(str(error))
    if parsed is not None:
        return parsed.value
    return ERROR_KIND_UNKNOWN

record_tool_call

record_tool_call(tool_name: str, duration_s: float, error: BaseException | None, *, instruments: ToolCallInstruments | None = None) -> None

Emit the SLI and semconv metrics for one MCP tools/call.

Never raises. The success path calls this before returning the tool's result, so an exception escaping here would turn a tool call that worked into a client-visible error — telemetry is not worth that. "No-op without OTLP env" is not the same guarantee: it only holds in the configuration where nothing is recorded at all.

Source code in datarobot_genai/drmcpbase/tool_metrics.py

def record_tool_call(
    tool_name: str,
    duration_s: float,
    error: BaseException | None,
    *,
    instruments: ToolCallInstruments | None = None,
) -> None:
    """Emit the SLI and semconv metrics for one MCP tools/call.

    Never raises. The success path calls this *before* returning the tool's result, so an
    exception escaping here would turn a tool call that worked into a client-visible error
    — telemetry is not worth that. "No-op without OTLP env" is not the same guarantee: it
    only holds in the configuration where nothing is recorded at all.
    """
    try:
        _record_tool_call(tool_name, duration_s, error, instruments=instruments)
    except Exception:
        logger.debug("Failed to record MCP tool-call metrics for %r", tool_name, exc_info=True)