Skip to Content
ArchitectureConnectors and meaningSimulated sources and estate
Architecture · Acquisition

Simulated sources and estate, labelled at every step.

Meridian GridMeridian GridThe fictional demonstration organisation used by the demonstration walk and its sample data. Nothing about Meridian Grid describes a real company. is a fictional demonstration organisation. Every person, company, ticket, metric, incident and agent named on this page belongs to it. None of them is a real customer’s data. The simulated sources and the mock agent estateAgent estateThe collection of AI agents an organisation operates, together with the systems and providers they use. let the acquisition machinery run, be tested and be shown before a customer hands over credentials.

The mark set by a fixture server or the mock estate follows the data to the screen. A real system lacks the marker, so the same code labels it real.

What it is for

NovusLens reads customer systems: ticket queues, metrics, continuous integrationContinuous integrationAutomated checks run when code changes, usually including building the code and running tests. (CI) runs, incident tools and, for agent governance, the boundary feedBoundary feedRecords supplied by the customer’s own systems about activity at the edge of an agent or service. NovusLens reads those records; it does not watch the agent. that a customer’s own agent estate publishes. A demonstration needs those inputs without a customer. The acceptance gates need them too, so that the same acquirersAcquirerA program that reads a customer's operational records, calculates quantities from them and records the results in NovusLens. a real integration uses are exercised. The simulated pieces stand in for those systems. They do not change what NovusLens does with the data. Every simulated record is marked, and the mark travels to the screen.

How it works

There are three simulated parts, and each has a different job.

Synthetic feeds (synthetic.py). SyntheticConnector writes two kinds of entry to the twinTwinNovusLens’s model of one organisation, built from records about its teams, services and how they rely on one another. It is only as current as those records.’s meshMeshA set of connected Mycelium services that exchange updates and keep the NovusLens log.: one edge entry per capabilityCapabilityWhat a team can provide or an agent can do. NovusLens records the claim and the evidence that supports it. (keys.telemetry_edge) and one cellCellA box on the map that stands for a team, service, supplier or AI agent. It is a unit you can inspect, not necessarily a department or a person. Complexity, Agility and RecoverabilityRecoverabilityThe R in C/A/R: how well a team or service restores acceptable service after a failure. Recovery times and tested alternatives can supply evidence. (C/A/RC/A/RThree separate scores for a team or service: Complexity, Agility and Recoverability. Each uses current evidence and the customer’s stated scoring rules.) entry per cell (keys.telemetry_cell). The lead timesLead timeThe elapsed time between starting a request or handoff and completing it. come from the canonical demonstration measurements, demo_org.OBSERVED_LEAD_DAYS and OBSERVED_CAR, unless a --leads-file supplies another dataset. The demonstration values follow fixed formulas. Ticket age is 0.8 times the lead time. Mean recovery time is 4.0 plus 1.5 times the lead, in hours. Change failure rate is 5 plus 2.5 times the lead, in per cent, capped at 45. Each entry carries synthetic: true, the source name synthetic:ServiceNow and its refresh_interval_ms.

The default refresh interval is 2000 ms (DEFAULT_REFRESH_MS). stream refreshes on that interval for a number of passes (five by default). once makes a single pass, and the walk runbook calls once --refresh-ms 30000. The module documents that readers discard an entry more than three times older than its refresh interval. A connectorConnectorSoftware that reads records from a source system, such as an incident tool. A customer-written file connects those records to teams or services in NovusLens. that stops refreshing therefore goes stale instead of lingering as current evidenceCurrent evidenceA view that uses only measurements recent enough to count. The “What we were told” view uses recorded statements instead..

FixtureFixtureA prepared set of records used for a test or demonstration. A fixture is sample data, not a live customer's records. servers (sim_sources.py). Each server reads one JSONJSONJavaScript Object Notation: a text format for data, using named fields, lists and values. Configuration files and service messages use it. fixture and responds in a real system’s format. This lets the test exercise the same acquirer as a real integration. The module states that a real integration changes only the URL and the token. The servers listen on 127.0.0.1 only.

ShapeEndpointsEndpointA particular address on a service that accepts a defined kind of request. servedFixture
ServiceNow Table APIAPIApplication programming interface: a defined way for one program to request data or actions from another.GET /api/now/table/incidentticket fixture
Jira Cloud searchGET /rest/api/2/searchticket fixture
Prometheus HTTPHTTPHypertext Transfer Protocol: the rules a browser or program uses to send a request to a web service and receive its response. APIGET /api/v1/query, GET /api/v1/label/job/valuesmetrics fixture (series)
GitHub ActionsGET /repos/{owner}/{repo}/actions/runs, GET /orgs/{owner}/reposCI fixture (repos)
PagerDuty REST APIGET /services, GET /incidentsincidents fixture (services)

The Prometheus shape accepts only selectors for values at one point in time (instant vectorsInstant vectorA set of measurement values selected for one point in time, rather than a series of values over a period.). Any other query returns HTTP 400, as a real server rejects malformed Prometheus Query LanguagePromQLPrometheus Query Language: the language used to select and calculate values from Prometheus measurements. (PromQL). An unknown GitHub repository returns 404. Unknown paths return 404. The walk runbook starts four servers, one per source system, with fixtures under apps/connectors/fixtures/: ServiceNow on 8600, Prometheus on 8601, GitHub CI on 8602 and PagerDuty on 8603. The module’s own default port is 8600.

Mock agent estate (sim_estate.py). Four characters run as live MyceliumMyceliumThe application infrastructure underneath NovusLens. Its connected services exchange updates and keep the ordered structural log. agents on the estate’s own mesh. That mesh has gatewaysGatewayA service that accepts requests and passes them into another system. The Mycelium gateway gives NovusLens access to the mesh. on 8410 and 8411 and gossipGossipA way for connected services to pass updates to one another until the updates spread through the group. ports 57020 and 57021. The twin’s mesh uses gateways 8400 to 8402 and gossip ports 57010 to 57012, and the two are never peered. The estate is the customer’s side of the layer boundaryLayer A and Layer B boundaryThe division between advice and action. NovusLens records and explains evidence in Layer A. The customer’s systems run and control agents in Layer B. (Layer B). NovusLens holds a read-only client against the estate’s gateway and reads only the boundary feed.

CharacterWhat it declaresWhat it doesIts numbers
auto-hedginghedge-execution with delegated authorityAuthorityWhat a team or agent is allowed to decide or do. For an agent, this comes from a named person or system that grants permission. and no accountable ownerAccountable ownerThe person recorded as answerable for a team, service or agent. NovusLens does not assume that this person is also the one who runs it day to day.Also takes positions, which its declaration does not coverDeclared service-level agreementSLAService-level agreement: a recorded service commitment, such as a target percentage of availability. A declared target is kept separate from measured attainment. (SLA) target 99.9 per cent; attainment fixed at 96.0 per cent
grid-balancerload-balancing, owned by Head of Grid OpsStays within its declarationDeclared 99.5 per cent; all SLA windows met
settlement-reconcilersettlement-reconciliation, owned by Head of Settlement OpsStays within its declarationDeclared 99.0 per cent; all SLA windows met
rogueSigned AgentFactsAgentFactsA signed statement from an agent about its allowed work, its permissions and the person answerable for it. for an agent named roguePublishes facts altered after signing, with no behaviourRefused by the standard ingestIngestRead supplied records or documents and add their information to NovusLens, after the required checks.

Each refresh pass, every character writes conway/estate/boundary/<agent_id>, a signed envelopeEnvelopeThe surrounding fields that identify a message, its sender, its type and its time, alongside the message's data. covering both its facts and its behaviour, and conway/estate/presence/<agent_id>. The estate writes every 2000 ms by default, but it declares a refresh interval of 120000 ms. The module records why the two were separated: when the write cadence and the declared freshness were the same number, evidence expired before anyone could open the lens.

./lens ops --version dev estate starts the estate alone. It writes an enrollment file for the estate, bound to one tenant and endpoint, and prints the command that acquires it onto the twin. The enrollment is in demo mode and uses public, name-derived keys for the estate.

Labels and how the flag travels

The simulated marker is a body key in each object response (x_novuslens_simulated: true) and a response header on every response (X-NovusLens-Simulated: true). The header is needed because list responses cannot carry a body key. The estate uses its own marker, x_novuslens_simulated_estate, inside the signed envelope.

  1. Fixture to reader. paging.py reads either form on each page and reports simulated to the caller.
  2. Reader to entry. acquire.py writes the observed entry with synthetic set from the source’s own declaration. It appends (simulated source) to the source name. The other acquirers add the same suffix. composite_acquire.py marks a composed calibrationCalibrationThe customer's own rule that turns raw figures, such as a gauge, a workflow history or an incident log, into a C/A/R score. A score moves when the calibration moves, and the change is recorded. as simulated when any contributing source is simulated, so mixed data never wears a real label.
  3. Real systems. A real system’s responses lack the marker, so the same acquirer labels them as real.
  4. Estate to findingFindingA difference in the records that needs a decision. It includes its sources and can describe a slow handoff, disputed fact or missing owner.. boundary_acquire.py copies the estate’s marker into the behaviour entry as estate: "simulated" and reports simulated_estate on the result. governance.py then words the finding as measured “from the estate’s live boundary feed (simulated estateSimulated estateA set of example agents supplied for testing and demonstrations. Their findings carry the label “(simulated estate)”. Real customer reports do not carry that label.)”.
  5. Production refusal. boundary_contract.py refuses a simulated estate marker under a production enrollment.
LabelAppears onSet by
(simulated source)the source name of an acquired readingacquirers, from the fixture’s marker
(simulated estate)the wording of a governance findinggovernance.py, from estate: simulated
synthetic: truesynthetic telemetryTelemetryMeasurements and activity records sent by a running system, such as ticket timings, network traffic or workflow results. entriessynthetic.py

Synthetic telemetry carries synthetic: true and the name synthetic:ServiceNow. It does not carry the (simulated source) text, because that text is added by the acquirers, not by synthetic.py.

Interfaces

Entry pointWhat it does
./lens connectors stream --org <org> [--cycles N] [--refresh-ms MS] [--leads-file FILE]Refreshes the synthetic feeds on their interval.
./lens connectors once --org <org> [--refresh-ms MS] [--leads-file FILE]One synthetic refresh pass.
./lens exec -- python -m conway_connectors.sim_sources --fixture FILE [--port 8600]Serves one fixture in one source system’s API shape.
./lens exec -- python -m conway_connectors.sim_estate [--port 8410] [--cycles N] [--cycle-ms MS]Runs the mock estate. --cycles 0, the default, runs forever.
./lens connectors boundary --org <org> --gateway HOST:PORT --enrollment FILEReads an estate’s boundary feed, read-only, onto the twin.
./lens ops --version dev walk, ./lens ops --version dev estate, ./lens ops --version dev agentsStart the fixture servers, the estate alone, or the estate for the agents check.

./lens exec uploads the host file passed with --fixture before starting the simulated source.

The global --host and --port options default to NOVUSLENS_MESH_HTTP_ADDR and NOVUSLENS_MESH_HTTP_BASE, then 127.0.0.1 and 8400. Those options address the twin’s mesh, not the estate’s.

Invariants and failure behaviour

  • The synthetic feeds write only while a process runs. When the process stops, its entries age out and the readers stop counting them.
  • A fixture server answers only the shapes listed above. It returns 404 or 400 for anything else, and it never returns a success for a shape it does not speak.
  • An acquisition that does not retrieve a complete listing writes nothing. The report says how many items were retrieved and how many were advertised.
  • Unmapped ticket groups are counted and reported. They are never guessed onto a cell.
  • The estate’s envelope is signed, and the rogue character’s facts are altered after signing. The standard ingest refuses them. Invalid facts can raise refusal findings, but they never contribute behaviour.
  • Without an activated enrollment and policy, boundary evidence cannot support a finding. The acquisition still succeeds, and the findings are withheld.
  • The acceptance agents check starts the estate, and the walk runbook starts the fixture servers. The walk is labelled as an operator runbook, not a gate.

The simulated estate lets us test and demonstrate governance findings from source to screen, without involving a customer’s agents. A real estate publishes the same feed without the marker, and the same acquirer labels its findings accordingly.

The twin's mesh and the estate's mesh, the fixture servers, the synthetic feeds and the boundary read path. The two meshes are never peered.

Last updated