Docs / Developer mode

Verifying a change

ivralo verify runs your project's own checks over a change, cheapest first, and stops at the first that fails; the agents run the same pipeline before they call a change done. This page covers the command, the roles in IVRALO.md, how tests are selected, and how failures are read from JUnit and SARIF reports.

The command

ivralo verify                       # what git says changed since HEAD
ivralo verify src/app/totals.py     # these files
ivralo verify --scope full          # the whole test command, before a pull request

Flag	Effect
`[paths]`	The changed files; without them, what git says changed since `HEAD`
`--scope targeted`	The default: the tests the change selects, then one ring wider
`--scope full`	The whole declared test command after the other checks
`--yes`, `-y`	Approve the commands that would ask
`--trust`	Trust this folder, and remember it, before running

stdout is the report. At a terminal, the commands the shell policy would ask about are listed and confirmed once; in a pipe they are refused unless --yes approves them.

Exit code	Meaning
`0`	Nothing stopped the pipeline
`1`	A stage failed and the pipeline stopped
`2`	Nothing to verify with: not a repository, or no command declared

Inside a turn, the agents call the same pipeline as the verify_changes tool. Its dialog lists the commands it would run in a runs row.

Roles: the `# Commands` section

The pipeline runs four roles, in this order, from the # Commands section of IVRALO.md:

# Commands

- format: `uv run ruff format --check .`
- lint: `uv run ruff check .`
- types: `uv run mypy`
- test: `uv run pytest -q`

Role	Note
`format`	Declare the check form (`ruff format --check .`, `prettier --check .`). The command runs as written, so a rewriting form would change your files; if one does, the report names them
`lint`	Any linter
`types`	Any type checker
`test`	First the tests the change touches, then one ring wider

It stops at the first stage that fails and names it (stopped at lint). A role you did not declare is reported as not declared and skipped: ivralo never guesses a command. Other names in the section (build, e2e) are not stages.

Declaring a command permits nothing. Each one is still judged by the shell policy as a run_shell command would be: a read-only or allowed command runs at once, one that asks shows a dialog, a denied one never runs. In a folder you have not trusted, the declared commands are refused.

Slots for paths

The selected test paths are appended to your test command. A runner that takes them somewhere else declares a test-paths line with a {paths} slot:

- test: `make test`
- test-paths: `make test FILES="{paths}"`

The symbol edits' formatter uses a format-paths line the same way (Code intelligence).

Which tests run

The selection reads only your repository's files. A test file is one whose name has the word test in it (test_totals.py, totals_test.go, money.test.ts, FooTest.java).

Ring 1: the tests named after a changed file, the tests that mention it, and the tests you changed.
Ring 2: their directories.
A change nothing matches, such as a config file, widens to the nearest test directory and then to the whole test command. It never skips.

Every line of the report says why it was that wide and which test areas it left out. The report never overstates what ran: targeted tests passed (12 selected ...), then wider ring passed, then the suite was not run. Only --scope full says the suite passed.

A project whose test files carry no test word (*.spec.ts, a spec/ tree) has nothing selected, so every change runs the whole test command.

Flaky tests

A test stage that fails and then passes on one rerun is labelled flaky. Its first run's output is kept and the pipeline goes on. A second failure is a failure. ivralo reports a flaky test; it does not try to fix it.

Reading a failure: `# Reports`

A failing test run can be thousands of lines, and the end of it often loses the failing test's name. So a failed stage is read first, and the report leads with what it came to:

what failed in test (targeted):
  tests: 2 failed, 1 error, 38 passed, 3 skipped (report:junit)
  failures (report:junit):
    1. tests.test_totals::test_refund - AssertionError: assert 7 == 5
       at tests/test_totals.py:22
  in the project: src/ledger/totals.py:18, tests/test_totals.py:22 (paths)
  outcome: exit 1 · 2 failed, 1 error, 38 passed in 0.31s (outcome)

It is read from three places, and ivralo knows none of your tools by name:

The reports your commands write. If a command writes a JUnit XML or a SARIF file, name it under # Reports, one per role, as a path or glob relative to the repository. ivralo never adds an option to your command, so the command has to write the file itself.
Your project's own paths in the output. Every path:line that names a file git lists is a frame of your project, shown in the order printed, with the one frame outside it where your code called into a library.
The last line: the exit code and the output's last line.

# Commands

- lint: `uv run ruff check --output-format sarif -o reports/lint.sarif .`
- test: `uv run pytest -q --junitxml=reports/junit.xml`

# Reports

- lint: reports/lint.sarif
- test: reports/junit.xml

The format is read from the file, never its name. JUnit gives the counts and each failing test with what it raised; SARIF gives each finding's rule, level, message and place. TAP is not read.

A report this run did not write (an old one left behind), or one that is missing, malformed, empty or too large, becomes a note: line and never a count: ivralo never says "0 failed" about a report it could not read. A report that counts no failure while the command failed, such as a collection error, is called out too.