How it works

What happens between the ticket and production.

BDD scenarios, questions to people, the QA agent and honest limits. The details the pipeline stands on.

The flow

From request to production. Agents in the loop, decisions with people.

One flow, several layers. Pick a layer to see who does what: the development pipeline, the QA pipeline, the deployment pipeline and the points where your people decide.

From request to production. Tickets come from your people, production users and the QA agent, the PM agent reviews and assigns them, and agents work in loops up to the merge. Then CD promotes the change through the environments.

CI fails review fixes unclear ticket answer new tickets user bugreports ← CD: promotion between environmentsreport: what to fix and whyno agent access Your people Ticket PM agent Scenario agent Developer agent CI on themerge request Reviewer agent Lead review Decision: merge Development Staging Preprod Production Clarification QA agent Monitoring agent Release approval CI fails reviewfixes newtickets user bugreports CD: promotion ↓no agent access Your people Ticket PM agent Scenarioagent Developeragent CI on themerge request Revieweragent Lead review Decision:merge Development Staging Preprod Production Clarification QA agent Monitoringagent Releaseapproval
agent human system no agent access

Agents work in an isolated zone with limited permissions and a full log of their actions. They have no access to production. Every ticket gets a note from the agent: what was done, how and why.

Division of work

The full delivery chain

What agents do in each area, and what we oversee. From backend to release.

AreaWhat agents doWhat we oversee
Backend APIs, logic, integrations, migrations, unit and integration tests Architecture, contracts, security
Frontend Screens from the design system, states, forms, component tests Fidelity to design, accessibility
BDD tests Scenario agent: Given / When / Then scenarios from the spec, characterisation tests, full suite on every change. QA agent: edge and corner cases on staging and preprod Scenario quality, product owner sign-off, path coverage
Other tests Unit, integration, API contract, browser e2e, regression Test strategy, quality gates, test data
Review Checks against the ticket, fixes minor defects Final decision, review rules
Releases Release checks against tickets, acceptance tests on staging and preprod, pipeline monitoring with a report on what to fix and why, merge request hygiene Promotion between environments, release approval, fixes on higher environments, rollback
Security Work in an isolated zone with limited permissions, every action logged Production access for people and the pipeline only
Process Triage of user reports (ticket or answer), backlog prioritisation, clarifying questions, new tickets from tests, ticket updates (what was done, how and why), morning report Who decides and when

Tests first

We start with tests of how your product behaves.

An agent is only as good as the thing it checks its work against. A Given / When / Then scenario is at once the acceptance criterion, the agent’s task and the test that verifies it.

features/invoices/export.feature
Feature: Export invoices to CSV   Scenario: Accountant exports invoices for a selected month    Given 3 invoices were issued in October    When the accountant exports October invoices    Then they receive a CSV file with 3 rows    And each row contains the VAT ID and gross amount
$ behave 1 scenario (1 passed) · 4 steps (4 passed)
01

The scenario is the contract

The agent implements and runs the scenarios. When they pass, the change goes to review. When they fail, it goes back to the agent.

02

The scenario is the documentation

An always up-to-date description of what the product does. New people and new agents learn from the same files.

03

The scenario prevents regressions

Every change, human or agent, runs through all existing scenarios.

Does your product have tests like this? If not, that is where we start.

Your current testsWhere we start
BDD scenarios or good functional tests Pipeline on the first ticket class straight away
Unit tests only About 1 week of BDD foundation for the first ticket class
No functional tests 1-3 weeks of BDD foundation for the key paths, partly in parallel with the pipeline build

Underneath sits the full pyramid: unit, integration, API contract and browser e2e tests. But the scenarios decide whether a ticket is done.

When an agent needs a decision

An agent does not guess. It asks in the ticket, with a ready answer.

When a requirement is unclear or there are several reasonable solutions, the agent moves the ticket to “Needs clarification” and writes one comment. You answer in a line, without a meeting.

#142 Validate VAT ID in form Needs clarification

developer agent03:40

Problem

The ticket says “validate the VAT ID”. A number can be checked by format only, or online in the EU VIES registry, which is sometimes unavailable.

Question

Should the order be blocked when VIES does not answer?

Options

  1. A

    Format check only

    Fast and never blocks, but accepts made-up numbers.

  2. B

    Format now, VIES in the background recommended

    The order goes through, an invalid number flags the invoice for the accountant.

  3. C

    Block until VIES confirms

    Safest, but orders fail whenever VIES is down.

Recommended path

B. The checkout scenario “order completes in under 3 s” would fail with C, and B follows how the payment webhook already retries. Meanwhile I am working on the other scenarios of this ticket.

product owner08:15

B. And email the accountant when an invoice is flagged.

developer agent08:16

Added the scenario “Invalid VAT ID flags the invoice and notifies the accountant”. Resuming.

01

The agent stops instead of guessing

Unclear requirements, several reasonable solutions or a change in product behaviour mean a question, not an assumption hidden in the code.

02

One comment, everything needed to decide

The problem in context, one concrete question, two or three options with their consequences and the path the agent recommends, with the reason.

03

The answer goes straight into the work

The agent turns the decision into a BDD scenario, updates the ticket and resumes. The decision stays in the ticket history, open questions appear in the morning report.

QA agent

The QA agent finds scenarios on its own. Including the ones nobody thought of.

This is not a standard tester. The scenario agent writes the main-path scenarios before any code. The QA agent works after development: it reads the specification, acceptance criteria and the scenarios in the repo, tests changes on staging and preprod, and deliberately hunts for edge and corner cases. Your team decides what the expected behaviour is.

QA agent pipeline

  1. 1

    Reads the specification

    Ticket, acceptance criteria, mock-ups, API contracts and existing scenarios.

  2. 2

    Checks the main path

    Runs the scenario agent’s scenarios on staging and preprod, with real integrations and test data.

  3. 3

    Hunts for edge and corner cases

    Boundary values, empty and invalid data, permissions, concurrency, time zones, integration failures.

  4. 4

    Asks when the spec is silent

    If the specification does not say how the product should behave, the case goes to clarification instead of guesswork.

  5. 5

    Opens tickets on its own

    When a test exposes a bug in existing code or a coverage gap, the agent opens a ticket with a scenario that reproduces it. The PM agent reviews it, sets a priority and assigns it to other agents.

  6. 6

    Learns from your decisions

    Approved and rejected scenarios go into the project’s pattern library. Next proposals get closer to how your team thinks.

the next ticket starts with more knowledge

QA agent · export invoices to CSV

From the invoice export scenario, the QA agent derives cases the ticket does not mention:

  • edge Month with no invoices: file with just the header approved
  • edge Credit note with a negative gross amount approved
  • corner Invoice from 31 Oct, 23:30 UTC, accountant in Europe/Warsaw needs clarification
  • edge EU customer: VAT number instead of a Polish NIP approved
  • corner 20,000 invoices in a month: background export and an email link needs clarification
  • edge User without invoice permissions approved
  • edge Semicolon in the customer name breaks CSV columns bug → new ticket SHS-152
  • edge Invoice with no line items rejected: already covered

8 proposals · 4 approved · 2 need clarification · 1 new ticket · 1 rejected

The QA agent does not replace the tester on your team. It gives them broad coverage to start with, and a human decides which cases matter to the business.

Honestly

Where agents work and where they don’t

We don’t promise “10x”. We promise measurement: a monthly report with metrics that are easy to check.

Works well

  • Tickets with BDD scenarios or clear criteria
  • CRUD, API integrations, migrations, tests, docs
  • Bugfix with a reproduction
  • Screens from an existing design system

Works poorly

  • Vague requirements like “make it better”
  • Architecture decisions and cross-cutting changes across services
  • Performance, concurrency, security-critical code
  • Code without functional tests, until we build the BDD foundation

In the monthly report

  • behaviour coverage by BDD scenarios and its growth
  • share of tickets merged without rewrites
  • share of tickets with a question and response time
  • AI cost per merged ticket
  • lead time from ticket to production and release count
  • change failure rate and review time

What we need from you

  • a tracker with tickets
  • a repo and CI, or readiness to build CI in step 3
  • an environment where tests can run
  • a product owner who approves scenarios
  • one decision maker who answers agent questions daily

Contact

First call. 30 minutes.

You tell us how you deliver today; we tell you where to start and what it costs.

kontakt@shipsteer.eu

Number of developers
Topic

We use form data only to contact you about this request. This site uses no tracking cookies.