Notebooks as Tests¶

A notebook run is a test run, and it fails the way a test does: a cell raises AssertionError. Nothing switches it on — write the rules, then assert.

Checks report instead of raising, so every rule in a cell runs and the whole board is visible before anything fails. That is the shape: evaluate everything, then gate once.

Write the rules¶

A check takes the series you already queried, so its window is the window you queried for:

frame = agent.query("bus0/BMS_message/cells.*", start=params.window)
agent.check.that(frame["bus0/BMS_message/cells.cell_0"], ">", params.floor)

Or it takes a signal handle and its own window, when you want a rule that does not depend on a query above it:

agent.check.that(
    agent.signal("bus0/BMS_message/status.pack_current"),
    "<",
    40.0,
    last="3m",
)

last="3m" means "every sample in the last three minutes" — a window implies always. With no window at all, a check reads the latest sample.

The full operator list and the tolerance operators are in Checks.

Read the evidence¶

A check result renders itself — as a board in the app and in an HTML export, as a row in the terminal. Every row carries the sample that proves it:

2026-09-04 17:06:58.855  cells.cell_0 (3.628 V) > 3 V  always  PASSED
2026-09-04 17:06:46.840  cells.cell_3 (2.994 V) > 3 V  always  FAILED

A failure names the first violating sample; a pass names the closest the data came to breaking the rule. Both are timestamped, so a red run points at the second to go look at rather than at a boolean. In code, that is result.evidence — a CheckEvidence with time_ns, signal and value.

Gate on them¶

One rule is one assert — a result is true when it passed:

assert agent.check.that(frame["bus0/BMS_message/cells.cell_0"], ">", params.floor)

For a scoreboard, collect the results and end the notebook with raise_if_failed(). It raises AssertionError naming every check that failed or errored:

from zelos_sdk import CheckResults

results = CheckResults(
    agent.check.that(series, ">", params.floor, name=path)
    for path, series in frame.short_names().items()
)
results.raise_if_failed()

agent.check.suite("checks/live.json") returns the same object for rules kept in a JSON file.

Read the outcome¶

The run's last line names the outcome, and the process exit code matches it. The rule is pytest's:

Outcome	When	Exit
`succeeded`	Every cell ran	`0`
`failed`	A cell raised `AssertionError`	`1`
`errored`	A cell raised anything else	`1`
`interrupted`	Ctrl-C	`130`

Exit 2 is an infrastructure failure — the agent connection, the session, or the stream — not a statement about the notebook.

notebook: 4 cell(s) in 2.0s — failed

Run it in CI¶

A run happens inside an agent, and that agent reads the notebook file from its own disk. The CLI sends the agent a path, not the file's contents. So the notebook must be on the machine where the agent runs. A hosted runner that checks out the repo and points ZELOS_AGENT_URL at a bench agent fails with exit 2, because the bench has no file at the runner's path.

Three setups work.

A self-hosted runner on the bench¶

Register a CI runner on the bench machine, next to the agent that records the bench. The checkout lands on the agent's disk, and the CLI reaches the agent on localhost:2300 with no configuration:

name: bench gate
on: [push]

jobs:
  gate:
    runs-on: [self-hosted, bench]
    steps:
      - uses: actions/checkout@v4
      - name: Run the gate
        run: zelos notebook run ./release-gate.md -o release-gate.html
      - name: Keep the report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: release-gate
          path: release-gate.html

Install the Zelos CLI on the bench once. The bench agent must keep samples to answer queries: run it with --store-type memory or --store-type disk. The default metadata store keeps only the catalog. See Install the Agent.

if: always() matters: the run you most want to keep is the one that failed. -o release-gate.html writes a single page with every table, chart, and check board. Vega-Lite charts, including plot() output, render with no network access. Plotly charts load the Plotly library from the web when the page opens.

An agent on the runner¶

Install zelos-agent on the runner itself, as in Install the Agent. The checkout and the agent then share a disk, so zelos notebook run ./release-gate.md works against localhost:2300. The cells still need data from somewhere:

A trace file in the repo. Open a committed .trz with agent.trace(). Pass an absolute path, such as str(Path("data/run.trz").resolve()), because the agent opens the file, not the cell.
The bench agent. Set ZELOS_AGENT_URL in the runner agent's environment, not the CLI's. Every cell's connect() then reaches the bench. A cell can also name the bench directly with connect("bench-01.internal:2300"). Either way, the runner must reach the bench over the network.
A test harness on the runner. Code that streams to the runner's agent needs that agent to keep samples, so start it with --store-type memory.

The first run on a fresh runner downloads Python and the notebook's packages, so it needs network access.

A notebook kept on the bench¶

A bare name that does not exist in the current directory goes to the agent unchanged. The agent looks it up in its own notebooks folder. So a notebook saved in the bench's notebooks folder runs from anywhere that can reach the bench:

zelos notebook --host bench-01.internal:2300 run release-gate.md -o release-gate.html

Run this from a directory that has no release-gate.md. A local file of that name wins, and its path means nothing to the bench. -o still writes the report on the machine that ran the command. To put a published notebook on the bench, pull it once through the bench agent:

zelos notebook --host bench-01.internal:2300 pull <notebook-id>

The pull lands in the bench's notebooks folder and prints the file name. It never overwrites, so pull once rather than on every run. Copying the file into that folder works too.

Measuring instead of judging¶

count() counts how many samples matched. A count never fails a check. It is the right tool for a threshold you are still calibrating, and for reporting alongside a rule:

below = agent.check.count(frame["bus0/BMS_message/cells.cell_3"], "<", 3.5)
below
# 2026-09-04 17:27:23.400  cells.cell_3 < 3.5 V  count · 93  PASSED

count() returns a CheckResult, like that(), so it renders as a row. int(below) is the number. int() raises TypeError when the check never got as far as counting, for example when the signal is missing, rather than reading 0.

Waive a threshold on the command line¶

zelos notebook run release-gate.md --param cell_floor=2.5

The override is visible in the run command, in the run's log, and in the rendered page — never by editing the rulebook.

Read a red run¶

In the Zelos App. Each check renders under the cell that produced it. A cell that raised shows its traceback, and the status bar counts the errors, for example 1 error. Select that count to scroll to the first cell that failed.

In Zelos Cloud. Publish the run — zelos notebook run … --publish, or Share in the app — and the version keeps the source snapshot and the settled outputs, checks included. Failed runs publish too, because a failed test is exactly the run someone needs to see. See Publishing notebooks.

Where to next¶

A release gate in CI

The complete, runnable example this page describes.
Checks

Every operator, the temporal words, tolerances, and the pytest fixture.
The notebook format

Front matter, params, {.norun} fences, and environments.