Check a deployed build from CI
BetaStart Scout from your pipeline on the task a branch implements, get the verdict on the task and fail the build when the task is broken.
Call Scout from your CI pipeline once a branch is deployed to a preview or staging address. Scout does the branch's task on that build like a visitor would, posts its verdict on the task with a link to the run and its recording, and your pipeline reads the same verdict.
Before you begin
- You need the Editor, Admin or Owner role.
- Connect Jira, GitHub Issues or Trello to the project in Settings > Integrations. See Connect an issue tracker.
- Create an account token in Settings > API & MCP and store it as a secret in your CI, for example
FLAMIO_TOKEN. See Connect Claude, Cursor or Codex over MCP. A per-project key cannot start Scout. - Copy the project id from the address of the project's page under Projects:
/dashboard/projects/<id>.
Each check spends tokens
A check is billed like a run you start from the Scout form. Set budgetTokens to cap it.
Start a check
Send the build and the task
POST https://api.flamio.org/api/scout-checks with your account token:
curl -sS https://api.flamio.org/api/scout-checks \
-H "Authorization: Bearer $FLAMIO_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"projectId": "YOUR_PROJECT_ID",
"url": "https://preview-123.example.com",
"boardTasks": {
"provider": "jira",
"query": { "kind": "links", "urls": ["https://example.atlassian.net/browse/KAN-33"] }
},
"siteCredentials": { "login": "qa@example.com", "password": "TEST_ACCOUNT_PASSWORD" }
}'| Field | Required | What it does |
|---|---|---|
projectId | Yes | The Flamio project the run belongs to. |
url | Yes | The deployed build to check. |
boardTasks | One of | provider (jira, github or trello) and query: { "kind": "links", "urls": [...] }, or { "kind": "version", "name": "2.4.0" }, or { "kind": "status", "name": "In review" }. |
instruction | One of | One scenario in words, when there is no task to read it from. |
siteCredentials | No | A test account Scout signs in with first. Used for this run only, never stored. Add "signInUrl" beside the login and password when the product is signed in to on another domain than url. |
width | No | desktop (default), mobile or both. |
ownershipAttested | No | true lets Scout complete flows that change data, like allow destructive actions on the Scout form. |
budgetTokens | No | Where the run stops. |
Scout reads each task from your board, so it checks what the task says, not what the request says. The answer carries the runId. Sending the same check again while it runs answers with that run and "alreadyRunning": true — nothing new starts.
Wait for the verdict
Poll GET https://api.flamio.org/api/scout-checks/RUN_ID with the same token until verdict is not running:
verdict | Meaning |
|---|---|
running | Scout is still working, or the verdicts are not on the tasks yet. |
passed | Every task works. |
failed | At least one task is broken. |
inconclusive | The run failed, or a task could not be checked. error says why. |
reported | No tasks were given: the report is ready, with no pass or fail. |
tasks lists each task with its outcome and reason, and runUrl opens the run in Flamio.
Example: GitHub Actions
A job that runs after the preview deploy. It reads the task key from a branch named like feature/KAN-33-signin:
scout:
needs: deploy-preview
runs-on: ubuntu-latest
steps:
- name: Check the branch's task with Flamio Scout
env:
FLAMIO_TOKEN: ${{ secrets.FLAMIO_TOKEN }}
PREVIEW_URL: ${{ needs.deploy-preview.outputs.url }}
QA_PASSWORD: ${{ secrets.QA_PASSWORD }}
run: |
KEY=$(echo "$GITHUB_HEAD_REF" | grep -oE '[A-Z]+-[0-9]+' | head -1)
RUN=$(curl -sS https://api.flamio.org/api/scout-checks \
-H "Authorization: Bearer $FLAMIO_TOKEN" -H "Content-Type: application/json" \
-d "{\"projectId\":\"YOUR_PROJECT_ID\",\"url\":\"$PREVIEW_URL\",
\"boardTasks\":{\"provider\":\"jira\",\"query\":{\"kind\":\"links\",
\"urls\":[\"https://example.atlassian.net/browse/$KEY\"]}},
\"siteCredentials\":{\"login\":\"qa@example.com\",\"password\":\"$QA_PASSWORD\"}}" \
| jq -r .response.runId)
while true; do
CHECK=$(curl -sS -H "Authorization: Bearer $FLAMIO_TOKEN" \
https://api.flamio.org/api/scout-checks/$RUN)
VERDICT=$(echo "$CHECK" | jq -r .response.verdict)
[ "$VERDICT" != "running" ] && break
sleep 30
done
echo "$CHECK" | jq .response
[ "$VERDICT" = "passed" ]Every answer is wrapped in response, as in the rest of the Flamio API.
Result
Each task gets a comment from Scout with the verdict, the reason and Open the Scout run — the run page holds the recording of what Scout did. The run also appears on the Flamio Scout page, with the same Board tasks card as a run started from the form.