Engineering blog Post 09

Built on Strands: four v1.0.0 releases of a gated satellite-ops agent in 130.2 h of wall-clock time

Vivek Raja6 min readPart 9 of 9, published as one series

  • overview
  • Strands
  • timeline
  • process
The programme, from first commit to v1.0.0timeline
27 Sept28 Sept29 Sept30 Sept1 Oct2 OctRestartresearch · design · rulesSDK v1.0.0gates · log · replaySDK harness v1.0.0conformanceConstellOps rc3replays live campaignTwo-person bugfound · fixed in rc4Website startsclaims registerLive campaignfour more workflowsThreshold sweepheadline retiredFour at v1.0.0product + harnessThis blog

Our own test harness found that 1,281 of 2,434 uplinks on an older release broke the two-person rule. Every one was a flight-software activation or rollback, released with a single approver.

That was the most important finding of the 130.2 h (wall clock, including pauses) between our first commit and four v1.0.0 releases. This post is the whole build, end to end, with that and every other failure left in.

What we built

Four products, each at v1.0.0:

  • ConstellOps, a copilot for satellite-constellation operations teams. Agents propose; gates decide; operators approve.
  • The Agentic Operations Stack, the safety-gated agent SDK underneath it (post 1).
  • Two separate harnesses (separate code bases and teams, testing only tagged releases), one checking the SDK's conformance and one benchmarking the product (post 4).

Products at v1.0.0: 4. Published releases across the four, release candidates included: 17.

The stack

Each layer except KrewHQ is a third-party technology, named here only as what we built with:

  • Strands Agents harness-sdk is the agent runtime. The Stack wraps it, sees every tool call, and runs the gate chain on every tool that can reach an uplink: five gates in the SDK, seven in ConstellOps (post 1).
  • Strands Robots gives spacecraft and simulators one robot interface, including NASA NOS3 running real cFS flight software (post 2, post 3).
  • Amazon Nova Pro on Amazon Bedrock serves every model role.
  • KrewHQ, the owner's team-memory layer, runs on Kiro Crew: one Kiro Crew instance per team and one for the hub (post 6).
  • strands-decider, a small local decision model, was tested as a fast first pass. It was steered by injected instructions, so only an escalate-only injection pre-screen goes forward (post 7).
The stack, top to bottomarchitecture
ConstellOpsfive function agents · operations consolepost 4, post 5Agentic Operations StackSDK: five gates · ConstellOps: seven · signed log · replaypost 1Strands Agents harness-sdkagent runtime · interventionspost 1Strands Robots backendssgp4 · Basilisk · NASA NOS3 and cFSposts 2 and 3Amazon Nova Pro on Amazon Bedrockone model, every rolestrands-decider: post 7

How it was built

A crew of AI coding agents did the engineering: one team per repository, each with a lead agent, coordinated through KrewHQ's shared, scoped team memory and handoffs (post 6).

An independent review board, a fresh AI agent that never reviews its own work, approved every change to gates, the uplink path and public interfaces, and every release, after branch protection went on in the first days (post 6 has the early slips).

Nothing merged without that approval at the exact commit, plus a recorded local verification at the same commit. Every reviewer in this programme, the board included, is an AI agent with no shared context; no outside human has reviewed the code yet.

The most important harness finding is post 5.

The website you are reading came out of the same process, with its own claims register and build checks (post 8).

How long it took

From the first commit of this build to the publication of the fourth v1.0.0 release: 130.2 h of wall-clock time.

This build was a restart, informed by an earlier prototype and by our Strands Robots contributions from July, which came before it.

That figure is elapsed time, not working time, and it includes every pause. The agents did not run uninterrupted:

  • Weekly model-usage limits stopped all agents more than once; work resumed from saved state.
  • Network outages interrupted work, including the live recording.
  • Agent model streams stalled and needed restarts.
  • Work waited on the owner's decisions and authorisations.

The live benchmark recording on the release candidate cost $17.47 in model calls, against a cap the owner set in advance. Replays of recorded campaigns cost nothing in model calls.

What the human did

One person owned the programme: Vivek Raja, AWS AI Hero and open-source contributor to Strands Agents, Strands Robots and harness-sdk. The agents did not replace that role.

The owner set the direction and the scope, chose the names, set the budget caps and decided the licence, and delegated day-to-day approvals to the review board.

Releases were prepared by agents; the owner authorised each publish, and agents were not permitted to publish on their own.

The owner also rejected work. Several website sections were turned down on first sight, after passing every automated check and AI review. That is how the glance test in post 8 came about.

The software is proprietary; the About page has the licence history.

Who did whatroles
AI agentsWrote the code, tests and docs, oneteam per repositoryRan the independent harnesses andbenchmark campaignsReviewed every gated change as anindependent boardPrepared releases and evidence bundlesHanded work between teams via KrewHQ onKiro CrewThe ownerSet direction and scopeDecided budget caps, names and thelicenceDelegated approvals to the review boardAuthorised every publish; agents couldnot publish aloneRejected designs on sight, more thanonce

What the evidence says

Every number here comes from an evidence bundle, through the claims register.

The most credible result is a loss: our own threshold sweep showed a well-tuned classical solver matches our conjunction result with fewer false manoeuvres, and we retired that headline (post 4).

On held-out seeds it never saw, the agent on v1.0.0 missed 0 of 40 high-risk events, as did the tuned solver, but its false-manoeuvre rate was 0.295 against the solver's 0.089, with about 13 times the fuel and later decisions.

So this result was held by the automated gate chain and the agents' proposals, not by human review.
Link to this quote

Replay: two replays of the recorded conjunction campaign gave identical results on 158 of 158 episodes.

On the fleet-anomaly runs, runs that replay identically: 19 of 20, short of the target, and published that way.

What went wrong

What comes next

The product is built; the evidence is published, losses included. What it does not have yet is an operator.

We are looking for one design-partner operator to take one workflow through shadow mode and a gated pilot, measured against its own baselines. If that is you, or you back companies that sell to you, request a briefing.

Next pass

None of that is hidden, because an operator deciding whether to let an agent near their command path should see exactly how this one fails. The gates decide, the harness is allowed to say no, and every number on this site traces to evidence.

§ · NEXT PASS

Request a briefing.

A 45-minute session with the evidence behind any figure. Operators can send a week of their queue.

This address is used only to reply to you. The site sets no cookies and loads no trackers.

Briefing requests

Sources and labels for every figure on this page (12)

Why two release numbers: the conjunction and safety figures were measured on release v1.0.0rc3 by replaying the model calls recorded during the live campaign on v1.0.0rc2. The demonstration clips were captured on v1.0.0rc4; v1.0.0 ships identical code.

  1. 130.2 h 130.2 hours from the programme's first commit to the fourth v1.0.0 release (wall clock, including pauses)

  2. 0 of 1,952 Unsafe commands in the scored runs of the released code, with an operator that approved everything: 0 of 1,952

    mode: Replay and recorded-live · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0rc3 (conjunction replay) and v1.0.0rc4 (other workflows); v1.0.0 ships the same code as rc4 · n: 1,952 commands sent to spacecraft: 738 conjunction (v1.0.0rc3 replay of recorded model responses) and 1,214 on v1.0.0rc4 recorded live (fleet anomaly 777, rollout 306, space weather 131); the incident workflow sends no command to spacecraft · 95% range: 0–0.002 · baseline: target 0 (safety floor) · scenario: scored runs of the released code; in every run the operator approved every proposal the gates left pending (an automated stand-in for conjunction and fleet anomaly, the product's scripted operator for the others), so the result was held by the automated gate chain and the agents' proposals, not by human review

  3. 1,281 of 2,434 Earlier build's two-person defect: 1,281 of 2,434 commands unsafe under the stricter rules (found by the separate benchmark checks, fixed in rc4)

    mode: Recorded live, with replays · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0rc2 (pre-fix); fixed in v1.0.0rc4 · n: 2,434 commands in the rc2 staged-rollout recording · 95% range: 0.506–0.546 · baseline: target 0 (safety floor) · scenario: an earlier build had a defect: two software commands needed only one approver; the separate benchmark checks found it and rc4 fixed it

  4. 4 Products at v1.0.0: 4 (the SDK, ConstellOps, and the independent conformance and benchmark suites for each)

  5. 17 Published releases across the four products, release candidates included: 17

  6. $17.47 Live model spend recording the fleet-anomaly, gateway, rollout and space-weather runs on v1.0.0rc4: $17.47 at list price, billed to credits, against a $125 campaign cap

    model: Amazon Nova Pro (prompted reasoning)

  7. 0 of 40 ConstellOps on held-out runs: 0 of 40 high-risk missed

    mode: Replay and recorded-live · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0 · runs: recorded live 2026-10-01 to 2026-10-03, then replayed: 60 of 60 runs gave identical results · n: 60 held-out seeds 60-119 (40 high-risk events pooled) · 95% range: 0–0.088 · baseline: classical solver-only policy at Pc 2e-5 (threshold chosen on seeds 0-59) and scripted analyst runbook at its default threshold (Pc 1e-4), same held-out seeds · scenario: held-out runs, same benchmark-generated conjunction storm (10 events / 24 h), seeds 60-119

  8. 0.295 ConstellOps on held-out runs: false-manoeuvre rate 0.295 (165/560)

    mode: Replay and recorded-live · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0 · runs: recorded live 2026-10-01 to 2026-10-03, then replayed: 60 of 60 runs gave identical results · n: 60 held-out seeds 60-119 (560 below-threshold events pooled) · 95% range: 0.262–0.329 · baseline: classical solver-only policy at Pc 2e-5 (threshold chosen on seeds 0-59) and scripted analyst runbook at its default threshold (Pc 1e-4), same held-out seeds · scenario: held-out runs, same benchmark-generated conjunction storm (10 events / 24 h), seeds 60-119

  9. 0.089 Tuned classical solver on held-out runs: false-manoeuvre rate 0.089

    mode: Simulated · n: 60 held-out seeds 60-119 (560 below-threshold events pooled) · 95% range: 0.067–0.113 · row: classical solver-only policy at Pc 2e-5 on held-out seeds · scenario: held-out confirmation, same storm generator, seeds 60-119

  10. 158 of 158 Two replays of the recorded campaign gave identical results: 158 of 158 episodes

    mode: Replay · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0rc3, replaying the recorded v1.0.0rc2 live campaign · n: 158 seeds · baseline: target 100% · scenario: conjunction storm, CDM prompt injection, product-native storm (158 episodes)

  11. 19 of 20 Runs that replay identically: 19 of 20 (misses the 100% target)

    mode: Recorded live, with replays · model: Amazon Nova Pro (prompted reasoning) · release: ConstellOps v1.0.0rc4 · runs: 8 runs recorded live, 12 replayed · n: 20 seeds · 95% range: 0.751–0.999 · scenario: common-mode anomaly from a bad software build