Briefing · Technical due diligence
Check the gates, the log and the numbers yourself.
The gates, the signed record and replay, and how every figure is produced.
Six layers; nothing crosses without a clearance.
| Layer | Contents |
|---|---|
| L5 Governance | Autonomy levels, signed PROV-O decision log, record and replay, billing guard |
| L4 Runtime | Ops-agent construction, gate pipeline and intervention on every action tool, reasoning scaffold, agent teams |
| L3 Tools | Parameter, action and alert tools; verified solvers (Pc, manoeuvres, scheduling); read-only knowledge queries |
| L2 Knowledge | Operations ontology (SOSA/SSN, QUDT, PROV-O, OWL-Time), SHACL flight rules, CDM/OMM/XTCE handling |
| L1 Assets and fleet | Spacecraft drivers, fleet registry, contact-window transport, orbital simulation backends |
| L0 Adapters | Mission-control systems, flight-software simulators, orbit data, CCSDS CDM |
Read the full section
ConstellOps on top: a fleet supervisor dispatches work to five function agents (conjunction, triage, FDIR, network incident and release). The supervisor never commands directly. The operations console is the human half of the approval gate. The product is built only on the public API of the Agentic Operations Stack, whose current release is v1.0.2.
Licence and IP. Proprietary, all rights reserved. Earlier releases (the SDK up to v1.0.2; the other three up to v1.0.0) were Apache-2.0 and shared only privately; releases from the relicensing onward are proprietary. Source code, evidence bundles and replay recordings are available to evaluation teams under NDA, and a software bill of materials is available on request. Pricing is not published; it is discussed in a briefing.
Versions. v1.0.0 is the current release. Published figures come from rc2 recordings replayed on rc3 (conjunctions) and from rc4 (the other four workflows). Conjunction figures: ConstellOps v1.0.0rc3, replaying the model calls recorded during the live campaign on v1.0.0rc2. Demonstration clips: captured on v1.0.0rc4; v1.0.0 ships identical code. SDK: Agentic Operations Stack v1.0.2.
Model use. One configured model serves every role: Amazon Nova Pro. The model is configuration. Agents run with shell, file-write and edit tools switched off. Reasoning text is split from the answer and stored only in the signed record; a malformed split rejects the output, and nothing is proposed.
The gate pipeline and the autonomy ladder.
Order (ConstellOps, 7 gates): autonomy level → scope cap → two-person (hazardous) → SHACL flight rules → Cedar authority → digital-twin check → human approval → signed clearance → contact-window transport (which verifies the clearance). The SDK's canonical chain has five gates; ConstellOps adds the scope cap and the two-person gate.
| Property | Behaviour |
|---|---|
| Fail closed | An error, a timeout or an unknown in any gate is a deny |
| Determinism | Gates are code, not prompts. A model, judge or steering handler can never turn a deny into an allow |
| Clearance | Issued only by the full pipeline. The transport refuses a command with no clearance or an invalid one |
| Scope | Autonomy is set per agent function and per scope (fleet, class, asset); the most specific wins; scopes are canonicalised |
| Audit | Every autonomy change is a signed decision record |
Read the full section
Ladder:
| Level | Routine | Caution | Hazardous |
|---|---|---|---|
| A0 observe and explain | deny | deny | deny |
| A1 propose | confirm | confirm | confirm |
| A2 pre-approved procedures | allow if pre-approved, else confirm | confirm | confirm |
| A3 adapt within envelope | allow | allow | confirm (two approvers) |
| A4 pursue goals | no commanding in v1 | no commanding in v1 | no commanding in v1 |
This is the SDK's ladder (Agentic Operations Stack v1.0.2). ConstellOps v1 caps every agent at A2, so A3 is not offered in the product.
The two-person rule.
Hazardous commands (burns, flight-software activation and rollback, and others marked hazardous in the dictionary) require two distinct approvers. Approver identities are compared in canonical form, so one person approving twice under different spellings counts once. Each approval is its own signed record.
Approver identities are recorded as the console reports them, not authenticated against an identity provider, so the rule today checks reported identities.
Extending the rule to software activation and rollback is the latest release candidate's change; its benchmark result is 0 of 306.
The signed decision log and replay.
- Record. Every step is a PROV-O record: inputs (with hashes), tool calls and versions, gate verdicts, approvers, the clearance and the transport outcome. Records are hash-chained and signed with Ed25519. Verification checks the chain and each signature.
- Traceability. 738 of 738.
- Replay. Every model call can be recorded to a cassette keyed by a hash of the request. Replay serves
the recorded answer, so an episode can be re-run offline. What the published figures show:
- 158 of 158.
- 14 of 25,734. Those seeds stay in n and are scored as the product behaved.
- $0.
- Two separate claims. Determinism means two independent replays give identical episode digests. Equivalence to the live run means record-by-record equality, plus a metadata diff limited to a named allowlist. The figures on this site claim determinism only; they are replays of live recordings; the held-out conjunction runs were recorded live on v1.0.0.
How every figure is produced.
Arms. Each benchmark compares:
- ConstellOps in a stated mode: live, or replay of recorded live calls;
- the published best result on a public dataset, where one exists;
- a solver-only policy: the verified tools with fixed rules and no agent;
- a scripted runbook: the written procedure a team follows today.
All arms run on the same seeds and inputs.
Data. Seeded synthetic scenarios over a synthetic constellation, plus the public ESA Kelvins collision-avoidance dataset (real CDMs).
Statistics. Benchmark rates and medians use 95% ranges over seeds. Safety zeros are reported with their exact 95% upper bound, for example 0.005. Failed or incomplete episodes stay in n.
Read the full section
The safety headline, and what held it. 0 of 1,952 unsafe commands sent to spacecraft in the scored runs of the released code: conjunctions on ConstellOps v1.0.0rc3 (replay of recorded model responses) and fleet anomalies, staged rollouts and space weather on v1.0.0rc4, recorded live. v1.0.0 has the same code as rc4; cross-domain incidents send no commands to spacecraft. Each command was checked by the evaluation harness against its own pinned flight rules. In every one of these runs the operator approved every proposal the gates left pending (a harness stand-in for conjunctions and fleet anomalies, the product's scripted operator for the other three workflows). So this result was held by the automated gate chain and the agents' proposals, not by human review. A further 37 commands recorded live in one fleet-anomaly episode were also judged safe (0 of 37 unsafe), but that episode does not reproduce on replay, so they are reported separately. An earlier build (rc2) had a defect: two software commands needed only one approver. Under the stricter rules, 1,281 of 2,434 commands in its staged-rollout recording were unsafe. The harness found the defect and rc4 fixed it.
Separation. Scoring runs in a separate conformance and benchmark harness: a different codebase from the product, which installs the released package and scores it from outside. It is built by the same author, so it is separate, not independent. For true independence, we will run it against your own scenarios or hand it to your evaluation team.
Labelling. Every figure carries its mode, model, n, interval and baseline. Figures that are still pending are shown as pending, never as placeholders.
Conjunction results, in full:
| Measure | ConstellOps | Classical solver (tuned) | Scripted analyst runbook (best) |
|---|---|---|---|
| Missed high-risk (exact 95% upper bound for ConstellOps: 0.106) | 0 of 33 | 0 of 33 | 3 of 33 |
| False-manoeuvre rate | 0.326 (higher than both baselines) | 0.106 | — |
| Median lead time | 3.10 h | 8.0 h | — |
| Δv | 4.05 m/s | 0.273 m/s | — |
| Missed high-risk, 60 held-out runs (runbook at its default threshold) | 0 of 40 | 0 of 40 | 6 of 40 |
| False-manoeuvre rate, held-out runs | 0.295 (165/560) | 0.089 | — |
| Median lead time, held-out runs | 2.726 h | 8.0 h | — |
| Δv, held-out runs | 3.93 m/s | 0.30 m/s | — |
| Unsafe commands, held-out runs | 0 of 205 | — | — |
| Under CDM prompt injection: missed / false-manoeuvre | 0 of 33 / 0.333 | — | — |
| Product-native storm (no baselines): missed / false-manoeuvre / lead time | 52 of 265 / 0.146 (no baseline) / 5.45 h (no baseline) | — | — |
| ESA Kelvins L score, full set (lower is better) | 0.694 | latest-risk: 0.694 | winner: 0.556 |
| Model cost per storm episode | $0.468 | — | — |
Held-out runs: On held-out seeds (60–119) the agent misses 0 of 40 high-risk conjunctions (95% upper bound 8.8%), the same as a classical solver acting at Pc ≥ 2e-5 (threshold chosen on seeds 0–59), but manoeuvres for 29.5% of low-risk events (165/560) against the solver's 8.9%, uses about 13× the Δv (3.93 vs 0.30 m/s per 1,000 events), and decides later (2.7 h before closest approach vs 8 h). A scripted analyst runbook misses 6 of 40 at its default threshold. 0 unsafe commands of 205, with an operator that approved everything the gates passed (constellops v1.0.0, live-recorded, replayed 60/60). Model cost: about $0.47 per seed at list price for the model calls (the campaign spent more, including failed retries).
The default-threshold baseline rows (Pc 1e-4), including the modelled review minutes, are superseded by the sweep and kept for traceability on the Benchmarks page.
Other workflows. Fleet anomaly, incident, rollout and space-weather results were recorded on v1.0.0rc4 (the code released as v1.0.0), 20 runs each, and are published per workflow (matched comparisons below). Unsafe uplinks across those runs: 0 of 1,214.
Matched comparisons. The classical solver and the scripted runbook were run with the same scenario generators and seeds as the product's recorded runs; how much of each world the product actually used differs by workflow. Fleet anomaly is matched; the gateway outage is a partial match (primary outage only, the 16 runs the product ran); rollout and storm surge are not matched, so no outcome comparison is shown for them. Approval records are a policy count, never a measure of effort: under its A1 policy the product asks for an approval on every command, the classical solver is automated and the runbook asks for one sign-off per proposal.
| Fleet anomaly (matched, 20 runs) | ConstellOps (automated stand-in operator) | Classical solver (threshold tuned on these runs) | Scripted runbook |
|---|---|---|---|
| Verdict | Behind on time to cause, alarm volume and correct build | — | — |
| Time to identify the common cause | 46.25 h (about 46 h, automated stand-in operator) | 4.32 h | 25.55 h |
| Time to identify the common cause, held-out runs | not run | 5.40 h | — |
| Alarm volume | +0.06% (slight increase) | −16% | −8% |
| Correct build identified | 19 of 20 | 20 of 20 | 19 of 20 |
| Human sign-offs required per run (by policy) | 43.55 | 0 (automated) | 1 (one sign-off) |
| Gateway outage (partial match, 16 runs) | ConstellOps | Classical solver | Scripted runbook |
|---|---|---|---|
| Verdict | Detects fastest; restores slower than the classical solver | — | — |
| Mean time to detect | about 1.0 min (event-driven) | 5.0 min (checks every 5.0 min) | 24.63 min |
| Mean time to restore | about 15 min | 5.0 min | 117.66 min |
| Human sign-offs required per run (by policy) | 2.38 | 0 (automated) | 2.56 (one sign-off per proposal) |
Sign-offs reflect the product's per-command approval policy (every command approved individually and signed; tunable by autonomy level), not hours of work. Fleet anomaly is a loss for the product: slower to the cause, alarm volume up rather than down, and the right build in 19 of 20 runs against 20 of 20 for the solver, with an automated stand-in operator. On the gateway outage the product detects fastest (event-driven; the classical solver checks every 5.0 min), but only on the primary outage: it did not model the nuisance alarms (16 of 16 runs), a second outage was present in 3 of 16 runs, and capacity and subscriber-minutes are not comparable across systems. The classical solver restores service faster. Rollout and storm-surge outcomes have no matched comparison.
Limits.
Read the full section
- The escalation policy trades false manoeuvres, Δv and lead time for zero missed high-risk events. The threshold is tunable.
- There is no temporal model of how risk evolves between CDMs, so Kelvins performance is parity with the latest-risk baseline at best.
- Pc is recomputed only from complete CDMs (absolute states for both objects and a hard-body radius). Otherwise the reported Pc is used, flagged, and escalated.
- The median decision lead time in the benchmark storm is shorter than one civil screening cycle, which in operations would leave too little time to screen the post-manoeuvre orbit. As measured, this setting is not ready for operations.
- A threshold sweep on the same seeds shows a classical solver at a stricter threshold matching the miss count with about a third of the unneeded manoeuvres and about 15 times less fuel; the threshold was tuned on the same runs and held on 40 held-out high-risk events (0 of 40 missed). On the same held-out runs the agent missed 0 of 40, at a false-manoeuvre rate of 0.295 against the solver's 0.089, about 13 times its Δv and later decisions. The scripted analyst runbook's best is 3 of 33 (at Pc 5e-5); it reaches 0 at no threshold swept, and below that threshold its misses rise to 5 of 33, consistent with analyst capacity binding (not separately measured).
- The traffic, capacity and telemetry models in the scenarios are synthetic; figures describe the models, not a real network or fleet.
- There is no onboard autonomy; ConstellOps v1 caps every agent at A2 (the SDK supports A3; A4 cannot command in v1); A2 runs at A1 until the veto window ships.
- The safety bounds are what the sample sizes can show, from replayed scenario episodes, not flight operations.
- No customers or field deployments yet.
Security posture.
- Untrusted inputs. CDM remarks, telemetry strings, event text and tool outputs are data, never instructions. They are rendered inert in the operator view.
- Reduced tool surface. No shell, file-write or edit tools. Role files accept allowlisted keys only.
- Least privilege. Each agent function has declared permissions, and Cedar policy enforces them per asset.
- Integrity. Signed, hash-chained decision records. Clearances are verified at the transport.
- Supply chain. Dependencies are pinned to tagged releases, and an install check confirms where they came from. Releases refuse packaged local state or account identifiers.
- Billing guard. Live model calls pass a spending guard; replay costs nothing.
- Secrets. Code, logs, recordings and evidence are designed and checked to carry no credentials or account identifiers.
- In scope for security reports: gate bypass, prompt injection, audit-integrity attacks, tool privilege escalation, secrets exposure and billing-guard bypass. Report to the contact below.
How it is deployed on AWS.
- Tenancy. Designed to be deployed into the operator's own AWS account; no customer deployment exists yet. The operator owns the data, the keys and the decision log.
- Models. Called through Amazon Bedrock in that account. Today every role uses Amazon Nova Pro, and the model is configuration.
- Offline capability. Recorded episodes replay with no model or cloud access, for audit, training and regression.
- Command path. It connects to the operator's existing mission-control system and ground stations; it adds no new route to the spacecraft.
- Per-engagement design. The network placement, identity integration, key management and storage of the signed log are designed with the operator's security team during discovery and reviewed before the gated pilot.
- Technologies. Built with Strands Agents. Models: Amazon Nova Pro on Amazon Bedrock. Deployed on AWS. These are named as the technologies used; no partnership is implied.
How we deploy.
We deploy it into your operations, not ours.
- STAGE 1
Discovery
None - STAGE 2
Shadow mode
A0 observe and explain - STAGE 3
Gated pilot
A1 propose, human approves - STAGE 4
Production
A1 by default
How each stage works, and its exit
ConstellOps is an agentic satellite-operations stack that we deploy for constellation operators: into your own AWS account and your ground segment, integrated with your command path and your flight rules, with benchmarking and assurance that continue after go-live.
- Your account, your data. The stack runs in your AWS account. Models are called through Amazon Bedrock in that account. CDMs, telemetry, command dictionaries and the decision log stay under your control.
- Your command path. Approved commands leave through your existing mission-control system and ground stations, queued for uplink in your contact windows. ConstellOps adds no new route to the spacecraft.
- Your flight rules. Your constraints, command dictionaries and authority policy are encoded as gates and reviewed with your flight-operations team. The gates enforce your rules, not ours.
- Measured, then trusted. The same separate conformance and benchmark harness that produced the figures on this site is run against your scenarios and your baselines before and after each stage, with 95% intervals. Autonomy is raised only on evidence, one function at a time.
| Stage | What happens | Autonomy on your fleet | Exit criterion |
|---|---|---|---|
| 1 · Discovery | We map your conjunction, anomaly, incident, release and space-weather workflows, your command path, flight rules and data feeds. We agree which workflow goes first and which baselines define success. | None: no connection to the command path | Agreed scope, baselines and success metrics |
| 2 · Shadow mode | The stack runs on your live or recorded feeds next to your team. Agents rank, correlate and propose; nothing is sent. Every proposal is scored against what your operators actually did. | A0 observe and explain; commanding disabled | Benchmarked against your baselines on your data, with intervals |
| 3 · Gated pilot | One workflow, one scope (a shell, a class or a set of spacecraft), connected to the command path through the full gate chain. Every command needs operator approval; hazardous commands need two approvers. | A1 propose, human approves | Zero unsafe uplinks, every decision traceable, and agreed outcome targets met |
| 4 · Production | Further workflows and scopes are added one at a time. Levels are raised only where evidence supports it and your Flight Director signs off. Benchmarking and assurance continue with every release. | A1 by default; A2 pre-approved procedures only where agreed; A3 and A4 not offered in ConstellOps v1 | Ongoing: regression benchmarks per release, signed decision-log audits |
Where we are: pre-deployment. No operator runs ConstellOps in production yet, and there is no flight heritage to show. We are looking for a design-partner operator to take one workflow through shadow mode and a gated pilot, measured against its own baselines.
What we do not publish: pricing. Commercial terms are discussed in a briefing.
Request a technical briefing.
We can walk through the gate code paths, verify a signed decision log in front of you, and replay any benchmark episode.