Check every release automatically

Have Scout re-check your site once for each new version real visitors reach, and see which problems the release fixed.

When real visitors reach a new version of your site, Scout re-checks the site once, compares it with your last audit and marks the problems it no longer finds. It can also check the release's tasks on your board.

Before you begin

You need the Editor, Admin or Owner role.
Flamio On-Live must record your site with the web SDK. See Install the web SDK.
Run one complete Scout audit of the same site first. Each release check compares with it.
To check the release's board tasks, connect Jira, GitHub or Trello to the project. See Connect an issue tracker.

On by default

Release checks start switched on. Once the project has a completed audit and your visitors report a version, every new version starts a check that spends tokens from your balance, up to the monthly budget — nobody has to click Launch Scout.

Set up release checks

Pass your release version to the SDK

Add the version option to Flamio.init. A new value seen on real visits counts as a new release.

flamio.ts

import { Flamio } from '@flamioai/web-sdk';

Flamio.init({
  projectKey: 'YOUR_KEY_HERE',
  endpoint: 'https://api.flamio.org/api',
  version: 'YOUR_RELEASE_VERSION',
});

YOUR_KEY_HERE — the key from Flamio On-Live > Setup.
YOUR_RELEASE_VERSION — your release, for example 1.4.2. Set it at build or deploy time.

Open the Release checks card

Open Flamio Scout. The Release checks card sits beside the launch form. Check every new release automatically is on by default.

Set the monthly budget

Enter the Monthly budget (tokens) that release checks may spend, and press Enter. The line under it reads how much they spent this month. When the budget is reached, further releases are skipped until next month.

Optional: check the release's board tasks

Turn on Also check the release's tasks on the board. If more than one board is connected, pick the board. Then pick which tasks:

Tasks of the release's version — taken from the Jira fix version or GitHub milestone named like the version your SDK reports.
Tasks in a status or column — pick the status or column.

Each task gets the verdict as a comment on the board. Without a connected board, the card reads "Connect Jira, GitHub or Trello to also check each release's tasks on the board." with a Connect a board link.

Follow each release

Each new version appears in the list with the date it was first seen and what Scout did:

Label	Meaning
Starting point	A version that was live before release checks started. Not checked.
Checked	Scout checked this release.
Skipped — checks are off	Release checks were turned off.
Skipped — monthly budget reached	The month's budget was spent.
Skipped — another release was checked under 6 hours ago	The next release's check covers this one.
Skipped — run one Scout audit of this site first	No earlier audit to compare with.
Could not start — check the token balance	The organization's balance could not cover the check.

Click Open check to open that release's report.

Result

The release's report opens with "Release check of version" and the version, linked to the previous audit. Findings the previous audit did not report carry Not in the previous check. Two checks of an unchanged site can differ, so this label is a pointer, not proof that the release introduced the problem.

On the previous audit's report, problems the check no longer found read "Not found again by the check of" and the version. With On-Live recording, each shows real visitors on that page before and after.

Until the SDK reports a version, the list reads "No versions seen yet."

Check every release automatically

On this page