Skip to Content
ArchitectureProjection and read modelOverview
Architecture · Read modelRead modelThe database tables used by the map, Ask and Find. NovusLens rebuilds these tables from its history of recorded changes.

The read model, rebuilt from the log.

NovusLens keeps what it knows in two places, on purpose. The canonical logCanonical logThe permanent history of changes to an organisation’s model. New entries are added at the end. Earlier entries stay when the database is rebuilt. lives in the MyceliumMyceliumThe application infrastructure underneath NovusLens. Its connected services exchange updates and keep the ordered structural log. meshMeshA set of connected Mycelium services that exchange updates and keep the NovusLens log.: an append-only, time-ordered record of every commit about the organisation. The read model lives in PostgresPostgresThe database software that stores the tables read by the NovusLens screens, along with document bodies, access records and protected personnel records.: tables the web application and the reasoning service can query quickly. Most of the read model can be rebuilt from the log. Some records exist only in Postgres and are described below. This split is called command query responsibility segregationCQRSCommand query responsibility segregation: keeping the record of changes separate from the tables used to answer questions. NovusLens writes to its log and reads from database tables built from it. (CQRS). Writes go to the log; reads use the database built from it.

Commits go to the canonical log in the mesh. The projection worker applies each entry to Postgres, and the web application and reasoning service read from there. The dashed band holds the three stores that a reset leaves alone.

The projection worker scans the commit log, applies each entry to Postgres, then refreshes the measurement engine.

How a change reaches the screen

  1. A build agent, connectorConnectorSoftware that reads records from a source system, such as an incident tool. A customer-written file connects those records to teams or services in NovusLens. or findingFindingA difference in the records that needs a decision. It includes its sources and can describe a slow handoff, disputed fact or missing owner. analyser writes an event to the mesh log. The event records an interview answer, an observation or a finding.
  2. The projectionProjectionThe regular pass that brings the read model up to date with the canonical log. It runs every 30 seconds by default. worker, ./lens exec -- python -m conway_reasoning.project, runs at a set interval. The Compose service projector starts it, every 30 seconds by default (NOVUSLENS_PROJECTION_INTERVAL).
  3. For each organisation it asks the materialiserMaterialiserThe component that reads recorded changes in order and updates the database. It records its place so it can continue after a restart. for the log entries newer than its cursorCursorA saved position in a sequence of records. A program uses it to continue from where it stopped. and applies each one to Postgres in its own transactionTransactionA group of database changes that is saved together. If the group fails, its changes are not saved..
  4. If anything was applied, it refreshes the measurement engine: critical path lengthCPLCritical path length. The total coordination weight of the longest chain of dependencies in the model. NovusLens reports it as index points, not elapsed time. (CPL), the told, shown and measured split, the history trail, and the derived findings.
  5. It records the pass in projection_runs. Every screen reads the latest row as “records as of”.

Rebuilding

Because the log is canonical, the read model can be wiped and replayed. ./lens graph reset empties the read-model tables, and materialise then re-reads the log from the start. Group and federationFederationSharing selected signed facts and summaries between separate regional twins, while their detailed records stay in their own regions. tables are also emptied, and they are re-created through the group tooling, not from the log. Three stores are Postgres-primary and survive a reset because nothing could rebuild them: the evidence bodies, the sealed person-to-unit join and the query audit trail. The projection page lists exactly what is and is not rebuilt.

The architectural units

Last updated