Reconfiguration architecture

Reconfiguration changes a running monitor's specification and effective input/output interface at a locally ordered control boundary. Both reconfigurable runtimes retain their opened input and output sessions. The dataflow runtime updates one persistent monitor owner in place, while the semisynchronous runtime replaces the monitor/evaluation generation and carries compatible history into the next one.

flowchart TB
    accTitle: Reconfiguration models and boundaries
    accDescr: The shared input driver emits data already pending when it observes a typed control item, then delivers a local reconfiguration barrier. Independent data and control producers have no cross-stream total order. Both reconfigurable runtimes retain persistent input and output owners. The dataflow runtime updates its monitor in place, while the semisynchronous runtime replaces the monitor/evaluation generation and carries compatible history into it. Nested dynamic expressions use a separate tick-local mechanism.

    data["Data items observed by input driver"] --> driver["Shared window and barrier driver"]
    request["Observed ReconfigurationRequest"] --> driver
    independent["Independent data and control producers"] -. "no cross-stream total order" .-> driver
    driver -->|"emit locally pending data first"| barrier["Delivered control barrier"]
    barrier --> dataflow["In-place dataflow cutover"]
    barrier --> semisync["Monitor generation replacement"]
    dataflow --> persistent["Persistent input, output, and monitor owners"]
    semisync --> generation["Replace monitor generation; retain I/O sessions"]
    nested["dynamic / defer source value"] --> tick["Nested tick-local activation"]
    tick --> persistent

Reading rule. Solid arrows show the ordering established after the shared driver observes items: pending window data is emitted before the delivered control barrier. The dashed edge denies a stronger guarantee—independent data and control producers have no cross-stream total order. Root requests replace the monitor definition and effective I/O interface; nested dynamic and defer activations do not run that protocol.

Principal entities

Two supported runtime models

These are separate runtime implementations, not execution tiers of one evaluator.

Constructing the control request

ReconfigurationRequest is the public transport-independent envelope delivered at the control barrier. Its constructor starts with unchanged default input and output configurations, so a request can replace only the monitor specification:

fn build_reconfiguration_request() -> anyhow::Result<()> {
    use trustworthiness_checker::io::ReconfigurationRequest;

    let replacement = "in x: Int\n\
        out alert: Bool\n\
        out total: Int\n\
        out scaled: Int\n\
        alert = total > 40\n\
        total = default(total[1], 0) + scaled\n\
        scaled = x * 2";

    let request = ReconfigurationRequest::new(replacement);
    request.validate()?;
    assert_eq!(request.specification, replacement);
    Ok(())
}

validate checks the request's transport-independent structure: the specification is nonempty and the nested input/output descriptions are structurally valid. It does not parse or compile the replacement, resolve it against the runtime's durable I/O registries, acquire resources, or apply a cutover. Those operations begin when the owner loop plans the delivered request.

Requests that change interfaces populate the public input and output fields with InputConfiguration and OutputConfiguration; those values describe the requested effective bindings, while InputPipeline and OutputPipeline remain the owners of durable local source and destination configuration.

One root replacement

A representative request changes one input source binding, one output route, and part of the specification while retaining compatible stream state.

Phase 1 states only the local order produced after the shared input driver observes control, not a total order over independent producers. Phase 2 is resource-free; mutation begins in phase 3. Phase 8 confirms local application, not remote destination consumption.

The interaction view exposes the old-batch callback, the owners crossed during application, and the acknowledgement boundary:

sequenceDiagram
    accTitle: Root replacement crosses persistent owners before acknowledgement
    accDescr: InputPipelineSession yields a delivered reconfiguration request to DataflowRuntime. The runtime first builds a resource-free plan. It then consumes old-side batches through the input rebind callback, flushes old output, applies output changes, applies the monitor plan, commits both session revisions, rebuilds layouts, and finally sends an acknowledgement. On failure, the runtime consumes the input session's deadline-bounded drain before flushing pending output and closing output owners. Cleanup does not reverse earlier effects.

    participant input as InputPipelineSession
    participant runtime as DataflowRuntime
    participant machine as DirectDataflowEngine and DataflowMonitor
    participant output as OutputPipelineSession
    participant ack as ReconfigurationAckSink

    input-->>runtime: Reconfigure(request) after local barrier
    runtime->>runtime: plan_runtime_reconfiguration(request)
    alt planning fails
        runtime->>runtime: finish_reconfigurable_active(error)
        runtime->>input: into_drain_with_deadline(deadline)
        input-->>runtime: cleanup items, errors, then EOF
        runtime->>output: flush pending rows and close owners
        output-->>runtime: cleanup result, then terminate owner loop
    else resource-free plan ready
        Note over runtime,machine: A failed call skips later calls and enters cleanup
        runtime->>input: rebind(input_plan, process_old)
        loop Already-admitted old input
            input-->>runtime: Data(batch)
            runtime->>machine: evaluate batch under old definition
            machine-->>runtime: old-side output row retained
        end
        input-->>runtime: rebound InputPipelineSession
        runtime->>machine: flush_reconfiguration_barrier()
        machine->>output: feed and flush old-side rows
        output-->>machine: flush complete
        machine-->>runtime: old side drained
        runtime->>output: apply_reconfiguration(output_plan)
        output-->>runtime: interfaces and routing applied
        runtime->>machine: apply_reconfiguration_plan(monitor plan)
        machine-->>runtime: ReconfigurationReport
        runtime->>input: commit_revision(input_revision)
        runtime->>output: commit_revision(output_revision)
        runtime->>machine: rebuild_monitor_layout()
        machine-->>runtime: transient layouts rebuilt
        runtime->>ack: send(ReconfigurationAck)
        alt acknowledgement accepted
            ack-->>runtime: send complete
            runtime->>input: next().await with new ActiveRuntime
        else acknowledgement fails
            ack-->>runtime: error
            runtime->>runtime: finish_reconfigurable_active(error)
            Note over input,ack: Cleanup does not restore detached or updated owners
            runtime->>input: into_drain_with_deadline(deadline)
            input-->>runtime: cleanup items, errors, then EOF
            runtime->>output: flush pending rows and close owners
            output-->>runtime: cleanup result, then terminate owner loop
        end
    end

Reading rule. Solid arrows are calls, ownership-changing requests, or submitted old-side output; dashed arrows are yielded items, returned artifacts, completions, or terminal outcomes. The consuming input rebind callback keeps admitted old work on the old monitor side until the local boundary, and acknowledgement is attempted only after input, output, monitor, session-revision, and transient-layout changes. Failure cleanup stops and drains input before output flush and close under the same absolute deadline; completed effects remain active. Lifeline spacing is causal order rather than logical tick spacing.

Candidate planning

DataflowRuntime plans the complete target before opening or stopping resources:

1. validate the request structure;
2. parse and compile the replacement specification;
3. resolve and plan the complete target input;
4. resolve and plan the complete target output;
5. plan monitor retention, cold installation, or state transfer.

A failure in this phase leaves active owners structurally unchanged. The DataflowRuntime owner loop nevertheless treats the request failure as terminal rather than retrying it.

Application order and partial failure

Mutation begins at phase 3 of the sequence above, and each mutating phase completes before the next begins. What that costs is stated per phase: the effect below survives the failure of any later phase.

Phase 8 is last, so nothing later can fail; a failed acknowledgement still leaves phases 3 to 7 applied.

There is no rollback across these owners. A source can already be detached, or one destination interface already updated, when a later operation fails. The architecture guarantees serial ownership and an explicit ordering boundary, not all-or-nothing replacement across transports, monitor state, and remote systems.

Identity and persistence

Replacement preserves state by semantic identity, not schedule position or storage coincidence. Compatible streams match by variable identity and StreamStateKey; unmapped or changed state starts cold. MonitorRevision tracks accepted semantic activations, while InterfaceRevision changes only when the effective input/output interface changes.

The exact mapping and transfer rules are documented in replacement identity and context transfer.

Nested expression activation

A dynamic or defer source is evaluated by a nested Evaluator inside one DataflowMonitor logical tick. Source prerequisites run first, active nested programs are resolved, dependency order is repaired if necessary, and the remaining streams run once before the common temporal commit.

A changed dynamic body receives a fresh evaluator unless compatible state transfers from the immediately previous body. Returning later to old source text does not revive its former evaluator. defer seals its first active body and releases source prerequisites only after a successful execution boundary. See dynamic properties.

Failure boundary

Planning, input drain, source opening, output flushing, interface update, monitor transfer, and acknowledgement delivery can each terminate the dataflow owner loop. Cleanup is still attempted, but already-applied cutover work is not reversed. A failed monitor tick publishes no row and commits no monitor history for that tick.

The complete containment ladder is in failure and termination.

Implementation mapping

• Root request planning, application, revisions, and acknowledgement: src/runtime/dataflow.rs.
• Persistent input ownership and cutover: src/io/reconfigurable_input.rs.
• Persistent output owners and interface handoff: src/io/output/pipeline.rs.
• Monitor replacement planning and state application: src/dataflow/monitor/reconfiguration.rs and src/dataflow/execution/monitor_execution/reconfiguration.rs.
• Semisynchronous monitor-generation replacement with retained I/O sessions: src/runtime/reconfigurable_semi_sync.rs.

Reading route

This page compares the two runtime models. The mechanism lives in the dataflow area, whose Replacement and containment group covers root cutover, replacement identity, context transfer, and failure containment in order — start from dataflow architecture, which owns the reading route for all of them.