detached solve cycle decision (epic SRQEK5, tasks WV62TX + 4QKZE3 + SF3QLP)

Context

Epic BXZBWY shipped streamed solve orchestration (dedicated tokio solve runtime + per-path delivery) but kept ONE engine-Mutex hold spanning the whole solve cycle — for a heavy 1700-path cycle that hold is multi-second, and register_path/ deregister_path (GIL-released, concurrent with the pump) serialize behind it. The offline detach-strategy A/B probe (commit 881c098f5, heavy-CL corpus, 156 paths x 5 passes) measured the merge seam directly:

disposition	engine-Mutex hold	cycle return	makespan
in-cycle (baseline)	2370-3812 us (one hold)	makespan-equal	equal
detach-always	9-18 us (per-result)	60-170 us (enqueue-end)	equal
hybrid 2x-median budget	141-155 us	budget-expiry	degenerates to detach

Makespan-equal at every thread count + an order-of-magnitude hold win = detach-always (no fast/slow tier; the budget arm was recorded degenerate).

Design (user-approved 2026-09-02)

The detached cycle is an enqueue-and-return contract: the enqueue half keeps the resolve/gate/deferred-path bookkeeping, the SOLVES run on a plain std::thread per LPT bin (a scoped rayon install JOINS its tasks and can starve against a held parking_lot guard — the deadlock diagnosis in the task record), and each result flows through an unbounded mpsc to the merge sidecar, a plain std::thread that applies each straggler under the engine Mutex.

The five theses (all match the shipped code)

1. Stale policy (Q1a): apply-if-unchanged, drop-on-touched. Each item rides its enqueue-time per-hop pool_update_block stamp; merge_detached_item re-reads the LIVE pool clocks at merge: ANY advance (a swap, or a price-neutral liquidity event — V3 Mint/Burn, V4 ModifyLiquidity both bump update_block AND re-classify the pools dirty) drops the straggler pre-insert. A de-registered path drops too. Code: solver_dispatch.rs::merge_detached_item; tests: detached_straggler_with_stale_update_stamp_is_dropped, detached_straggler_after_deregister_is_dropped.

2. In-flight: self-consistent gauge. Bin threads bump the gauge at SEND time, the sidecar decrements per terminal disposition — a bin that dies before sending never leaks a count. Code: detached_outstanding (Arc<AtomicU64>). WFF6MM: DETACHED_INFLIGHT_CAP survives only as the clamp bound for the admission target depth; it is no longer a dispatch gate.

3. Backpressure: shed+carry, never block, never re-topologize. The admission draw (budget = max(0, target_depth − in-flight), taken under the engine lock at on_resolve) is the ONE consumption decision: a zero-budget draw SHEDS the whole cycle before any enqueue, retaining the keys in the ledger for a later cycle, and a positive-budget draw ALWAYS submits down the one (detached) arm. WFF6MM retired the cap-based degrade and the in-cycle fallback; with the engine mutex released at enqueue-end there is nothing to re-topologize and no in-cycle hold to block on.

4. Observability: hotpath labels arb_solve.detached_enqueue / arb_solve.detached_merge; metrics degenbot.detached.in_flight (gauge), .stale_dropped, .applied (counters; stubbed no-ops in the no-otel build). In-cycle counters/labels unchanged for the fallback mode. The ADR-021 publish tripwire is UNCHANGED and stays the correctness backstop: an APPLIED straggler re-extends last_solved_path_ids at merge so the next publish’s verifier diff covers late merges (pinned by adr021_detached_stragglers_stay_scoped_to_the_publish_verifier); a stale-DROPPED straggler reaches the publish path never.

5. No fallback (WFF6MM hard cutover). DEGENBOT_DETACHED_SOLVES and its solve.detached_solves TOML key retired; a surviving CLI/config override fails the load as an unknown key. The detached arm is unconditional, and the old in-cycle dispatch (and its Saturated machine state, tick_in_cycle ledger seq, FanInTally, and lock-context duality) is deleted.

Policy matrix

axis	values	shipped default	effect
DEGENBOT_DETACHED_SOLVES / solve.detached_solves	— (retired)	always detached	RETIRED at the WFF6MM cutover: the in-cycle fallback arm is deleted, so there is no stance to set; a surviving CLI/env/TOML key fails the config load (unknown key)
DEGENBOT_STREAMING_DELIVERY	unset / 0	streaming (per-path micro-batches) / debounce sweep	T3 default flip; A/B opt-out keeps the debounce sweep; the sweep still owns expired/removed + end-of-cycle metadata either way
DEGENBOT_SOLVE_EXECUTOR	— (retired)	fleet	RETIRED at the P6YXA6 hard cutover; the tokio-stance fallback itself retired at LW-T9 (ergo CQLMM2): the fleet is the ONLY solve executor (fleet is the only stance since LW-T9), and the env var fails the config load loudly for one release
DEGENBOT_FLEET / fleet.stance	— (retired)	fleet	RETIRED at the LW-T9 hard cutover: the stance flag is gone with the legacy tokio-stance code path (solve_executor.rs deleted); a surviving env var or TOML key fails the config load loudly for one release
DEGENBOT_SOLVE_SIM_INFLIGHT / solve.solve_sim_inflight	— (retired)	fleet	RETIRED at the LW-T9 hard cutover with the SimSlots semaphore (legacy arb-sim detached threads): SimDriver capacity is fleet.sim_slot_cap, inline-sim sizing solve.inline_sim_workers; a surviving env var fails the config load loudly for one release
admission draw	max(0, target_depth − in-flight) keys	zero-budget draw sheds the cycle; keys carried	shed+carry backpressure (WFF6MM); the gauge is the draw’s input
Q1a oracle	stamp vs live clocks	mismatch or deregistered => drop	stale policy thesis

What a detached cycle guarantees

• rebuild_and_solve_affected RETURNS at enqueue-end (results_block stamped at the solve block); drift between enqueue and merge is bounded by the sidecar drain, which re-acquires the engine Mutex per item (the register/deregister serialization invariant holds — per-item acquisitions take the same Mutex).

• A straggler result lands in results iff its intake is UNCHANGED live ( oracle above) — otherwise it is dropped and the world stays consistent (the touch that invalidated it re-marks the pools dirty, so the next cycle re-solves fresh).

• The Python consumer contract drifts NOTHING: the result channel still only ever carries clamp-passed above-threshold batches (streaming micro-batches per solve completion + the end-of-cycle debounce sweep).

What we measured (10.2-min mainnet soak, dry-run, shipped stack, 2026-09-02)

• Blocks 25892125-25892175 (51 blocks); 98 detached cycles; 3686 of 3712 dispatch phases = 99.3% single-candidate batches (per-path micro-batches with streaming DEFAULT-UNSET — the flag flip itself validated live); the multi-candidate remnants are the end-of-cycle debounce sweeps + startup accumulation (6..109 candidates).

• 86 Q1a stale drops (pools genuinely moved between resolve and sidecar merge); 0 error/panic/abort lines; ADR-021 publish tripwire silent for the whole run.

• Evidence: logs/t3_soak_batch_shapes.csv, logs/t3_soak_summary.json, logs/t3_soak_hotpath.json (hotpath timed-exit report), logs/bot_run.log.

Allocator-lane combined soak (2026-09-02 21:15Z, the SF3QLP closer)

The COMBINED config — DEGENBOT_STREAMING_DELIVERY=1 EXPLICIT + DEGENBOT_DETACHED_SOLVES=1 — re-soaked from the allocator terminal on a fresh log segment (zero contamination): 10.4 min live mainnet, blocks 25892238-25892288 (51 blocks), 3966 of 4000 dispatch phases = 99.15% single-candidate micro-batches, 96 detached cycles (enqueue-end + sidecar merge every cycle), 0 Q1a drops this window, 0 errors, publish tripwire silent. Evidence: logs/t3_lane_batch_shapes.csv, logs/t3_lane_summary.json, logs/t3_lane_hotpath.json. The decision table shape is unchanged on live data from both lanes; the deferred cross-cycle straggler rejection analysis has no trigger.

Terminology note (epic MROOY7): the seam names this decision record uses — SolveCoordinator, DispatchOwner, DrainSink, EngineHandle, DrainWork — were retired in SZJUKL (ADR-041). The detached cycle itself remains, and since the WFF6MM cutover it is the ONLY solve arm: DEGENBOT_DETACHED_SOLVES retired with the in-cycle fallback.

WFF6MM hard cutover (2026-09-12)

Phase A soaked the flag-ON posture (solve.admission_shed, DEGENBOT_SOLVE_ADMISSION=1) live: 95/95 healthy samples held in_flight at 0 with shed_total flat; a 7 s SIGSTOP stall shed exactly one zero-budget cycle (in_flight=269 >> target_depth=8) and carried the keys; recovery resumed at ~17 merges/s with no further shed or degrade (see docs/tasks/WFF6MM-cutover.md for the full evidence table). Phase B then cut over:

• deleted the in-cycle dispatch arm and every construct that existed only to switch to it — the machine’s Saturated state, begin_cycle’s cap/stance verdict (it now takes no argument), tick_in_cycle, FanInTally, LaneArmPolicy::lock/claim_all_lanes and LaneDrainLockContext;

• retired the solve.detached_solves schema key + DEGENBOT_DETACHED_SOLVES env name (unknown key => load failure), keeping degenbot.detached.degraded_cycles exported at 0;

• the ONE drain (drain_lane_outcomes) is unchanged except that its only caller is now the detached sidecar; Suppressed/Failed stay keyless (no claim).

Watchdog + publish-verifier audit

• B3 frozen-drainer backstop: detached drain items COMPLETE at enqueue-end (return is enqueue-end, not apply-end), so no stall accrues across cycles — pinned by detached_stragglers_do_not_trip_the_frozen_drainer_backstop through SolveCoordinator-in-DispatchOwner, the production cadence; the synthetic-freeze abort pair (no_progress_frozen_drainer_aborts_proc et al.) stays green in the same suite.

• EngineHandle::solve_dirty’s hold-time invariant now documents the detached exception: the hold collapses to enqueue-end (~us) while the stance is ON; register/deregister serialization is preserved by the per-item sidecar acquisitions (engine_handle.rs).