How Vortex IQ measures execution

Version 1, September 2026. Any change to a definition is dated in the change log, so figures from different periods stay comparable.

At a glance

What this page is for

These three numbers only help if they mean the same thing whenever quoted. This page is the definition of record.

The execution gap explains why we chose them. Here we state exactly how each is calculated, so workspace figures trace to a written rule, and anyone using the same names can be asked whether they mean the same thing.

The unit of measurement

The unit is a defined corrective job. It begins when an issue is first recorded, is owned by one crew, and ends when a correction is verified in the live store against a completion criterion set when the job opened.

Not everything the AI workforce for ecommerce does is a fix. Research briefs, launches and full migrations are projects with their own measures (accepted delivery, launch completeness, verified recovery) and are not counted here. A remediation inside such a project, for example repairing a broken redirect found during a migration, is a fix.

Six crews

Measures apply to corrective work in every crew, reported per crew and in total.

The record every fix leaves behind

Every job produces one record. The three numbers are calculated from it and from nothing else:

• Detection: time first recorded, and the source (monitor, shopper report, manual entry)
• Priority: business weight at detection
• First proposal: ready time, evidence, proposed change, expected effect
• Review decision: who, when, and the outcome (approved unchanged, revised or rejected) with reason category
• Execution: what ran, when, against what, and the rollback point where supported
• Reuse: whether a validated solution was used, which one, and the fit check
• Verification: check performed, pass or fail time, evidence retained
• Reopen: recurrence within the validation window, with the original detection time preserved

A number that cannot be traced to one of these fields is not reported.

Time to Fix (TTF)

Definition. Elapsed time from first recorded detection to verification that the intended correction is live and working.

TTF = verification time − first detection time

Start. The clock starts at first detection. A ticket opened two days later still includes those two days, because the merchant carried the problem.

Finish. The clock stops at evidence, not activity. A deploy, a field update or an API success are intermediate steps. Completion needs a check suited to the issue (checkout journey test, live catalogue state, template render check), set when the job opens and recorded when it runs.

Three components (approval included)

Approval waits sit inside the headline figure, because merchants experience them. They are also shown separately:

Make review easier. Never remove the reviewer.

Example. Checkout fault detected Monday 09:00. Proposal ready 11:00. Approved 14:00. Journey test passes Tuesday 10:00. TTF is about 25 hours: preparation 2h, approval 3h, execution and verification 20h.

Failure keeps the clock. Failed verification keeps TTF accruing. If an issue reopens within the validation window, the record is flagged, the original detection time is retained, and resolution is measured from that time. A fix that did not hold is not a fast fix.

Reporting. Median and P90 (never the mean alone), plus the count and age of open issues. Figures are segmented by type, severity, platform and whether human approval was required. A missing image and an intermittent checkout fault are different work.

Does not claim: how much the problem cost. Read TTF alongside business priority, not instead of it.

Fix Acceptance Rate (FAR)

Definition. Percentage of reviewed issues whose first submitted fix proposal was approved without human edits or a request for revision.

FAR = (first proposals approved unchanged ÷ first proposals with a review decision) × 100

• One first proposal per issue. Revised-then-approved is not first-pass. The original decision stands in FAR.
• Pending excluded. Reported as pending, and entered into FAR only once decided.
• Unchanged means unchanged. No alteration and no revision request. Non-altering comments are not edits. Changes to scope, wording, target, timing or implementation are.
• Standing approvals separate. Classes of change a merchant has authorised under standing policy are counted on their own line and never blended into FAR, which covers human decisions only.

Reason categories (non-acceptance)

Revisions and rejections are categorised at decision time:

Reasons often matter more than the rate. Reasons clustering on missing context point to a data problem. Reasons clustering on scope point to an instruction problem.

Example. 40 first proposals reviewed: 28 approved unchanged, 8 revised, 4 rejected, and 5 still pending. FAR = 28 ÷ 40 × 100 = 70%. Pending proposals stay out of the denominator.

Reporting. Reviewed, pending, approved unchanged, revised, rejected, the reason breakdown, and the verification outcomes of approved proposals. TTF approval time is shown beside FAR, because a high rate means little without real review time and evidence.

Does not claim: correctness. An approved proposal must still execute and pass verification. FAR measures handover quality, not outcome quality.

Fix Reuse Rate (FRR)

Definition. Percentage of verified completed fixes that materially used a previously validated solution.

FRR = (verified fixes using a validated solution ÷ all verified completed fixes) × 100

Validated means verified on an earlier job, with recorded conditions, pre-checks, the change, and post-evidence. A lookalike suggestion is not validated.

Material use means a procedure, implementation pattern or substantive change component that helped resolve this issue, applied under recorded conditions and checked in the current store. Generic prompts, the same connector, or resemblance alone do not count.

Within-store and across-store reuse are reported separately. Within-store reuse shows retained context. Across-store reuse shows methods that travel. Claims that the workforce learns across the merchant base rest on the across-store figure only.

Methods travel. Merchant data does not. Transferable methods are separated from the store data they were learned on, and applied elsewhere only where reuse is permitted and conditions match. Each store keeps its own context, permissions and checks.

Rising FRR shows reuse, not benefit. Pair it with lower delivery cost or less human effort on comparable work, with quality maintained. Novelty is not failure: declining a mismatched pattern is competent execution.

Example. 50 verified fixes, of which 18 reused a validated redirect-repair pattern (12 within-store, 6 across-store). FRR = 36%. The within-store and across-store split is reported alongside.

Reporting. Verified fixes, reuse count, the within-store and across-store split, plus the solution reference and fit check for each reused fix.

Does not claim: cost or effort savings by itself. Without that evidence, FRR is a count, not a result.

Reading the three together

The useful question is how they move relative to each other on comparable work:

Counts and rates are still read against business impact and work left undone.

The reporting standard

Any report of these figures states:

• Observation period
• Number of stores and of issues
• Mix of work by type and severity
• Approval policy in force, including standing approvals
• Verification window per issue type
• Version of these definitions used

Unfinished, failed and reopened work gets the same prominence as successes. Credibility comes from explaining delays and rejections as clearly as wins.

Where the numbers appear

Merchants see their own figures in the Vortex IQ workspace, per crew and in total, updated as jobs complete and summarised monthly. Asking how a number was reached surfaces the jobs, not a footnote.

Change log

Version 1, September 2026. First published.

Frequently asked questions

Essentials only. The full rules live in the sections above.

What counts as a fix?

A defined corrective job: a recorded issue, owned by one crew, ending in a verified live correction against the opening criterion. Research, launches and full migrations are not fixes. See the unit of measurement section above.

Does approval wait count in TTF?

Yes. It sits inside the headline figure and is also shown as its own component. See the Time to Fix section above.

What if verification fails or the issue returns?

The clock keeps running, and reopened issues retain their original detection time. See the Time to Fix section above.

Are standing or policy approvals in FAR?

No. They are reported separately. FAR measures a human decision on a first proposal. See the Fix Acceptance Rate section above.

What counts as reuse for FRR?

A previously validated solution, applied under recorded conditions, checked in-store, and material to the fix. See the Fix Reuse Rate section above.