Dataflow architecture

The dataflow layer gives DSRV stream equations their synchronous execution model. Its organising model follows the synchronous dataflow tradition exemplified by Lustre: equations define streams, current data dependencies form a graph, and one logical tick evaluates that graph in a dependency-valid order before state becomes visible to the next tick (Halbwachs et al., 1991). DSRV adds sparse/special values, runtime-defined properties, and live replacement to that foundation.

Layer responsibility

A dataflow graph is a directed graph of computed streams. Each node is one stream equation; a current edge a → b means that b needs a's value from the same logical tick. Current edges determine execution order and must be acyclic. A positive historical read such as a[1] observes committed state from an earlier tick, so it crosses time rather than adding a current graph edge.

The layer compiles that graph once, schedules its currently active edges, evaluates each computed stream through an independent persistent state owner, commits temporal effects once, and projects one complete output row. It does not own input transports, output destinations, or asynchronous driving; DataflowRuntime adapts those external concerns to the synchronous row interface.

Canonical execution

The canonical view removes asynchronous adapters, runtime-defined dependency changes, execution-tier selection, and root replacement. It shows the synchronous machine that those mechanisms must preserve.

flowchart TB
    accTitle: Canonical execution of one logical dataflow tick
    accDescr: One logical input row is loaded into DataflowMonitor's current environment. DataflowProgram supplies the immutable stream graph, stable slots, and per-stream StreamProgram values. Scheduler supplies a dependency-valid order over computed streams. MonitorExecution invokes each persistent Evaluator once through canonical graph evaluation, producing a complete current row. Staged temporal writes commit once after ordered evaluation, outputs are projected, and committed state is visible only to a later tick.

    input["One logical input row"] --> monitor["DataflowMonitor<br/>load the current environment"]
    program["DataflowProgram<br/>stream graph · stable slots · StreamProgram values"] --> scheduler["Scheduler<br/>dependency-valid computed-stream order"]
    program --> owners["Independent per-stream Evaluator owners<br/>persistent EvaluatorState"]

    monitor --> execute["MonitorExecution<br/>canonical ordered evaluation"]
    scheduler --> execute
    owners --> execute
    previous["Committed earlier-tick state"] -. "historical reads" .-> execute

    execute --> row["Complete current row<br/>each computed stream published once"]
    row --> commit["One post-row temporal commit"]
    commit --> outputs["Output projection"]
    commit -. "visible only later" .-> later["Committed state for a later tick"]

Reading rule. Solid arrows are the canonical path through one logical tick. Inputs are loaded before scheduled work and are not computed-stream schedule entries. The evaluator-owner node is separate from Scheduler because persistent language state does not belong to an order position. Dashed arrows cross tick boundaries: only committed state can satisfy a later historical read.

Full dataflow architecture

The full view restores the surrounding runtime, active dynamic dependencies, replaceable quickened and native routes, accelerator fallback, and root reconfiguration control. Each added path remains constrained by the canonical model above.

flowchart TB
    accTitle: Dataflow graph, scheduling, evaluator ownership, and acceleration tiers
    accDescr: Input architecture supplies logical ticks to DataflowRuntime, which presents rows to the synchronous layer. DataflowProgram defines the immutable stream graph and stable layout. Scheduler combines fixed and active dynamic dependencies into a ScheduledExecutionPlan. Canonical, quickened, and feature-gated JIT routes execute against stable independent evaluator state, converge on one row publication and temporal commit, and pass complete rows to output architecture. A quickened region declines a row before executing it; a native artifact can miss a guard mid-tick. Both hand the work to canonical execution.

    sources["External input sources"] --> input["Input architecture: logical ticks"]
    input --> runtime["DataflowRuntime: batches to rows"]

    subgraph layer["Synchronous dataflow layer"]
        direction TB
        program["DataflowProgram: immutable stream graph and stable layout"]
        active["Active dynamic dependency edges"]
        scheduler["Scheduler: dependency-valid stream order"]
        plan["ScheduledExecutionPlan"]
        route{"Physical execution route"}
        canonical["Canonical graph evaluation"]
        quick["Quickened mixed execution"]
        jit["JIT native artifact when enabled"]
        owners["Independent per-stream Evaluator state owners"]
        execute["Execute ordered stream work"]
        commit["Publish complete row and commit temporal state"]

        program --> scheduler
        program --> owners
        active --> scheduler
        scheduler --> plan --> route
        route --> canonical
        route --> quick
        route --> jit
        canonical --> execute
        quick --> execute
        jit --> execute
        owners --> execute
        execute --> commit
        execute -. "new active dynamic edges" .-> active
        quick -. "declines row" .-> canonical
        jit -. "guard miss" .-> canonical
    end

    runtime --> execute
    commit --> output["Output architecture: delivery policies and destination owners"]
    control["Root reconfiguration control"] -. "ordered cutover" .-> runtime

Reading rule. Solid arrows show graph definition, scheduling, alternative physical routes, evaluation, and complete-row flow. The evaluator-owner node is separate from the schedule because language state does not move when order or execution tier changes. Dashed edges show dynamic schedule feedback, the two different ways work returns to canonical semantics, and root control rather than ordinary row data. The quickened route has no mid-execution escape: a region checks its inputs and declines the whole row before running any of it, so canonical evaluation is the only thing that ever partially executes.

Organising concepts

Principal entities

A compiled monitor does not own transports or drive itself. Input and output batching may change physical granularity, but it must not add, merge, or reorder logical ticks unless an explicit input reduction says so.

Cross-cutting invariants

Every execution route preserves these facts:

1. one successful monitor call is one logical tick;
2. current dependencies are evaluated before their consumers;
3. historical reads observe only previously committed ticks;
4. each computed stream publishes once to a stable row location;
5. temporal writes become historical only at the common post-row commit;
6. persistent language state belongs to stable evaluator identities, not schedule positions;
7. a failed tick publishes no output row and commits no monitor history;
8. optimization may replace routing, never the semantic authority of the canonical program and state.

The running example

The guide reuses one specification throughout its conceptual pages:

in x: Int
out alert: Bool
out total: Int
out scaled: Int
alert  = total > 20
total  = default(total[1], 0) + scaled
scaled = x * 2

For x = 4, the monitor evaluates scaled = 8, then total = 8, then alert = false. After the row is complete, total = 8 becomes available to total[1] on the next tick. The execution model traces this scenario in detail.

Progressive reading path

Each group below refines the one before it and contradicts none of them. Stopping at the end of any group leaves a correct model, not a partial one — the later groups add mechanism, never revise meaning.

Canonical semantics

1. Execution model defines rows, absence, dependencies, and the running trace.
2. Compilation explains how equations become immutable programs and plans.
3. Runtime ownership separates immutable meaning, persistent state, mutable order, and replaceable routing.
4. Scheduling explains how fixed and active dependencies become source and main execution orders.
5. Tick execution establishes phase order and the common commit boundary.

Stop here and you can predict what any tick produces.

Stateful language mechanisms

6. Temporal state explains staging, delay rings, and next-tick visibility — and why the commit boundary must exist at all.
7. Language state covers lifting, conditionals, functions, and recursive frames.
8. Dynamic properties covers nested expression activation and exact active dependency discovery.

Stop here and you understand everything the language guarantees.

Physical execution

9. Fusion and regions motivates the second representation, shows what the schedule contributes, and defines the two region scopes.
10. The scalar IR defines the representation every region is expressed in.
11. Execution tiers places quickening and native execution over the canonical machine, and states who owns a region's state.
12. Typed monitors covers the caller-side path that removes Value from the boundary.

Everything in this group is routing. None of it changes what a tick means.

External ownership

13. Runtime adapter connects asynchronous input batches to synchronous rows.
14. Input and output sessions defines live transport ownership.

Replacement and containment

15. Root cutover, replacement identity, and context transfer explain live replacement.
16. Failure and termination states containment and resulting runtime state.
17. Implementation mapping maps these concepts and entities to source and tests.

Nested activation and root replacement

The word reconfiguration covers two distinct mechanisms.

Nested expression activation is dynamic and defer. It happens inside one tick, at the source barrier, to one node of one stream. It compiles a body from a string, may change which same-tick edges are active, and is governed by the enclosing monitor's own evaluation. It is explained in dynamic properties.

Root monitor replacement swaps the whole compiled definition. It happens between ticks, is driven by a control request through the runtime, is serial rather than atomic, and can leave named partial states if it fails part-way. It is explained in root cutover and context transfer.

Both install something compiled at runtime. They differ in trigger, granularity, failure model, and the lifetime of the state involved.

The separate input architecture, output architecture, and reconfiguration overview place the synchronous machine in the wider runtime.

Reference

N. Halbwachs, P. Caspi, P. Raymond, and D. Pilaud, “The Synchronous Data Flow Programming Language LUSTRE,” Proceedings of the IEEE, 79(9), 1305–1320, 1991. doi:10.1109/5.97300.

D. Biernacki, J.-L. Colaço, G. Hamon, and M. Pouzet, “Clock-Directed Modular Code Generation for Synchronous Data-Flow Languages,” LCTES, 2008.