Skip to content

datarobot_genai.eval.dataset

dataset

to_byob_jsonl 

to_byob_jsonl(dataset: list[dict[str, Any]], output_path: str) -> None

Write the dataset to the JSONL the BYOB runner consumes.

Every field of each case is passed through verbatim — NeMo BYOB places the whole row into sample.metadata, so a benchmark can read any field it needs (context, canary, constraints, match_mode, entity_types, …) with no change required here. We only guarantee the two keys the runner itself relies on: input (the prompt template key, defaulted so a malformed row surfaces in the scorer rather than at render time) and id (required — a missing one is a genuine error).

Source code in datarobot_genai/eval/dataset.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
def to_byob_jsonl(dataset: list[dict[str, Any]], output_path: str) -> None:
    """Write the dataset to the JSONL the BYOB runner consumes.

    Every field of each case is passed through verbatim — NeMo BYOB places the
    whole row into ``sample.metadata``, so a benchmark can read any field it
    needs (``context``, ``canary``, ``constraints``, ``match_mode``,
    ``entity_types``, …) with no change required here. We only guarantee the two
    keys the runner itself relies on: ``input`` (the prompt template key,
    defaulted so a malformed row surfaces in the scorer rather than at render
    time) and ``id`` (required — a missing one is a genuine error).
    """
    lines = []
    for case in dataset:
        row = {"input": "", **case, "id": case["id"]}
        lines.append(json.dumps(row))
    Path(output_path).write_text("\n".join(lines))