Flamio Scout

How Flamio runs your delivery

Beta

The rules Flamio follows at each step of a change — what it checks, when a pull request may not merge, what it skips — and where you set them.

Flamio follows a set of rules at each step of a change on its way to production: which changes Scout checks, when a pull request may not merge, which changes it skips, and what it checks once a release is live. You set them on Delivery > Rules, or keep them in a .flamio.yml file in your repository — see the .flamio.yml reference.

Before you begin

  • Changing the rules needs the Editor, Admin or Owner role. Everyone in the organization can read them.
  • Scout checks pull requests from your pipeline, with a CI key — see Check every deployed preview from CI.

Set the rules

Open the rules

Click Delivery in the sidebar, then Rules.

Pick a preset

A preset sets every rule at once:

PresetWhat it does
StrictBlocks a merge on a medium finding or worse.
BalancedChecks every pull request; blocking critical findings runs as a dry run.
Report onlyChecks and reports; never blocks a merge.

A rule you change keeps its value when you switch. It has a bar on its left edge, and its menu has Reset to the preset.

Set each rule

Click a rule's mode and select On, Dry run or Off, then fill in its fields. Each step's rules are listed in The steps.

Save

Click Save on the bar at the bottom, or Discard to drop your changes. The Monthly Scout budget above the rules caps what every check spends in a month.

Modes

ModeWhat the rule does
OnActs: starts a check, blocks a merge, skips a change.
Dry runActs on nothing, and records what it would have done — under the rule, Would fire.
OffDoes nothing.

Under each rule, the number of times it fired in the last 7 days and Recent list what it did — Fired, Would fire, No action or Failed — with the pull request or commit it looked at. Try a rule in Dry run for a while before you turn it On.

The steps

Pull requests

RuleWhat it doesStrictBalancedReport only
Check pull requests
pull_requests.check
Flamio checks a pull request’s preview when it is deployed.
Not acting yet: your pipeline starts each check, with its own settings.
OnOnOn
Block merges
pull_requests.block_merges
What the check finds decides whether the pull request may merge.OnDry runOff
Skip docs-only changes
pull_requests.skip_paths
A change that only touches these paths is not checked.OnOnOn

Block merges reads what the check of the pull request found:

  • Findings that block — the mildest severity that blocks.
  • On pages — only findings on these pages block. Leave it empty for every page.
  • When the check could not decide — Block the merge or Let it through.

When the rule blocks, the pipeline job fails and its report says Merge rule: blocks the merge with the reason. Make that job a required check in your repository's branch rules, so a blocked pull request cannot merge. With Bitbucket connected, the commit's build status fails too — see Block a merge when a check fails. The job follows the rule only when your workflow leaves fail-on unset — see The verdict.

Skip docs-only changes leaves out a change that touches nothing but its Paths: the job passes without a check and says why.

Staging

RuleWhat it doesStrictBalancedReport only
Check staging
staging.check
A deploy to staging starts a check of its address.
Acting in part: each deploy gets a whole-site check on desktop; audit, tasks, scenarios, width and budget are not used yet.
OnOnOn

Production

RuleWhat it doesStrictBalancedReport only
Check production
production.check
A new site version in production starts a release check.OnOnOn

Check production is the release check — see Check every release automatically.

Where your settings live

  • The rules live on Delivery > Rules, or in .flamio.yml. The file holds the whole set: once it manages the rules, the page says Managed in .flamio.yml, and Copy file replaces Save — change the file instead.
  • YAML on the Rules page shows the rules as the file: File holds only what differs from the preset, Full config every key. Download saves it. With GitHub repositories on Delivery, Move to repository opens GitHub's new-file page with the file filled in. A file too long for the link, or a repository whose branch Flamio has not seen in a deploy or a pull request yet, is copied instead, for you to paste on GitHub.
  • The monthly Scout budget and the test account stay on their pages: a file never raises what Flamio may spend, and it names the test account without holding its password.

Keep the rules in your repository

With the repository connected — through the Flamio app on GitHub, see Connect GitHub, or on Bitbucket, see Connect Bitbucket — Flamio reads .flamio.yml from its default branch by itself. Flamio only reads the file: you commit it.

Put the file in the repository

On Delivery > Rules, click YAML. On GitHub, click Move to repository: GitHub opens a new file with the rules filled in, or copied for you to paste. On Bitbucket, click Download and add the file at the root of the repository. Commit it to the default branch.

Push changes to the default branch

Flamio applies the file on every push to the default branch. The Rules page says Managed in .flamio.yml on that branch, with the commit it was read at, and Edit in GitHub or Edit in Bitbucket opens the file there.

A Bitbucket repository that updates every 10 minutes — you are not an admin of it — is read overnight instead of on each push.

Review a change before it merges

A pull request that changes the file gets a "Flamio config" check — a report on the pull request in Bitbucket: what each changed setting is now and what it will be after the merge. A Bitbucket repository read every 10 minutes gets none. To draft a change on the Rules page, change the rules there and click Copy file, then paste the file in the repository.

A file with errors changes nothing: the rules that ran keep running. The Rules page lists the errors above the rules, the commit's "Flamio config" check fails with each error on its line, and the project's admins get a notification.

To manage the rules on the page again, delete the file from the default branch: the rules keep the file's values. If Flamio can no longer read the repository — the app removed, Bitbucket access removed, the repository gone — the page says the file is no longer read, and the rules are edited there meanwhile.

Apply the file from your pipeline

On GitLab or another host, apply the file from the pipeline of your default branch:

Create a key that may change the rules

Open Settings > API & MCP, click Create CI key, and check Can change delivery settings. Keep the key in a protected variable that only the default branch reads: a GitHub environment with protection rules, a protected GitLab variable, or a Bitbucket deployment variable.

Apply on every push to the default branch

FLAMIO_CI_KEY=$FLAMIO_CONFIG_KEY npx @flamioai/cli config apply

FLAMIO_CONFIG_KEY — the key from the step above. The .flamio.yml reference lists validate, plan and pull too.

Limitations

  • Rules block a merge, not a release. Check production reports what a live release broke; nothing stops the deploy. To keep a change out of production, block its merge.
  • Check pull requests is saved but not acting yet: your pipeline starts each pull request check with its own settings.
  • Check staging acts in part: a deploy to a staging environment it names gets a whole-site check on desktop. Its audit, tasks, scenarios, width and budget are not used yet.