Adding a connector, without adding a way to guess.

An acquirerAcquirerA program that reads a customer's operational records, calculates quantities from them and records the results in NovusLens. reads a customer’s operational records and calculates a measured quantity. It records the result in the twinTwinNovusLens’s model of one organisation, built from records about its teams, services and how they rely on one another. It is only as current as those records. as observed evidence. This page shows how to add one for a new system. The worked example is the continuous integrationContinuous integrationAutomated checks run when code changes, usually including building the code and running tests. (CI) acquirer, ci_acquire.py, which turns workflow runs into Complexity, Agility and RecoverabilityRecoverabilityThe R in C/A/R: how well a team or service restores acceptable service after a failure. Recovery times and tested alternatives can supply evidence. (C/A/RC/A/RThree separate scores for a team or service: Complexity, Agility and Recoverability. Each uses current evidence and the customer’s stated scoring rules.) scores for a cellCellA box on the map that stands for a team, service, supplier or AI agent. It is a unit you can inspect, not necessarily a department or a person..

NovusLens measures and advises. An acquirer only reads. It never writes to the customer’s system.

What it is for

Every figure on the twin says which system measured it, over which population and when. A new connectorConnectorSoftware that reads records from a source system, such as an incident tool. A customer-written file connects those records to teams or services in NovusLens. is how a new system joins that evidence. Without a connector, that source cannot supply measured evidence. Other documents can still support the shown rungRungOne of three levels of support for a recorded value: told, shown or measured. The level appears beside the value.. With a connector, its observations can reach measured. See the evidence ladder.

How it works

Every connection has three parts, and you author only the acquirer (the cookbook in docs/mvp/instrumentation_connections.md says the same).

1. The source. The customer’s system, speaking its own APIAPIApplication programming interface: a defined way for one program to request data or actions from another..
2. The mapping. A small JSONJSONJavaScript Object Notation: a text format for data, using named fields, lists and values. Configuration files and service messages use it. file the customer writes. It joins the source’s names to the twin’s: for the CI kind, {"cells": {"payments": {"repo": "acme/payments"}}} plus an axes calibrationCalibrationThe customer's own rule that turns raw figures, such as a gauge, a workflow history or an incident log, into a C/A/R score. A score moves when the calibration moves, and the change is recorded..
3. The acquirer. A module in apps/connectors/src/conway_connectors/ with one entry function, run_acquire_<kind>, which the acquire command in __main__.py calls.

The acquirer shape

Follow ci_acquire.py. It has four steps, each a plain function you can test alone.

Reuse the shared parts rather than copying them:

• paging.walk(page_url, items_of, token, ...) follows a listing. You supply two small functions: how to build a page URL and how to read items and the advertised total from the API’s own shape. It carries the page budget (MAX_PAGES, 50), de-duplication and the repeat-page guard. It returns a record of fetched, advertised, pages and complete.
• metrics_acquire.load_mapping validates the mapping and its calibration. score_axes applies it.
• refused_partial(collected, target, noun) turns an incomplete listing into a refusal.

The measuring rules

These are the rules the existing acquirers follow. A new one should keep them.

• Use measurements, never fill gaps with estimates. An in-progress run has no outcome and is skipped and counted. A failure with no later success in its own workflow and branch cannot yield a recovery time, so it is counted as unrecovered.
• Refuse a partial listing. If the source advertised more than it supplied, or the page budget ran out, the target is refused with the counts fetched and advertised. It is never scored over a partial page.
• Report the unmapped, never guess. A repository present under the owner but absent from the mapping is listed in the result (unmapped_repos), not placed on a cell.
• Calibration is the customer’s. The score is the calibration’s base plus each per_unit times its value, clamped to 0–100. NovusLens applies it as declared and never tunes it.

Where the entries go

Entries are plain last-write-winsLast-write-winsA rule for competing updates in which the update with the latest ordering stamp supplies the stored value. keys in the meshMeshA set of connected Mycelium services that exchange updates and keep the NovusLens log., under the conway/ prefix only. Build every key through conway_mesh/keys.py, which passes it through assert_ours. That guard refuses the MyceliumMyceliumThe application infrastructure underneath NovusLens. Its connected services exchange updates and keep the ordered structural log.’s reserved prefixes (grp/, sys/, consensus/, cap/, log/, wiki/ and the rest listed in the repository’s CLAUDE.md). Never write a raw key with agent.set(f"…"); a test greps for it.

The one exception, mcp_tool_registration, writes the Mycelium’s own tools/ convention and is not for connectors.

Keep mesh entries lean, because writes are size-gated. Evidence bodies go to PostgresPostgresThe database software that stores the tables read by the NovusLens screens, along with document bodies, access records and protected personnel records.. write_cells stores a score-observation artifactArtifactA stored source file or evidence body that a record can cite. Keeping the exact content lets a reader check what supported the record. through evidence.store_artifact, and the mesh entry carries only the artifact id, the calibration reference and the triple.

Calibration, provenance and simulated labelling

Every entry must carry the facts a reader needs to judge it:

• source: the system and host, such as github:<host>. A simulated source gets the suffix (simulated source).
• acquired_from: the URL polled.
• synthetic: true when the source declared itself simulated. The source does this with an x_novuslens_simulated key in an object body, or an X-NovusLens-Simulated header on a list response. paging.get_json reads both. The flag then propagates into every entry, so simulated data can never wear a real label. A real system lacks the marker, so the same code labels it real. There is no flag to forget.
• refresh_interval_ms and written_at_ms. Readers treat an entry as stale after three times the refresh interval, so the connector must be run on a cadence.
• The calibration, kept as a stored score-calibration artifact and cited by reference. The calibration is what lets a reader see how a raw quantity became a score.

How the corrector commits

Acquirers write observations. They do not change the twin. The correctorCorrectorThe NovusLens component that compares measurements with recorded statements and adds observations and differences to the log., run_correction in corrector.py, reads edge and cell telemetryTelemetryMeasurements and activity records sent by a running system, such as ticket timings, network traffic or workflow results. by prefix. A new feed under that prefix needs no entry in a name list.

1. It takes a distributed lockDistributed lockA lock that coordinates programs running in separate processes or on separate machines, so only one holder is recognised at a time. only long enough to get a fencing tokenFencing tokenA number that identifies a turn to update shared data. NovusLens rejects an earlier correction’s writes after a newer correction is applied..
2. Cell telemetry goes through car_axes.correct, which commits one car_axis_observation per axis with the token, observed_at_ms and valid_until_ms.
3. Edge telemetry is compared with the asserted account. A difference becomes an asserted-observed-delta findingFindingA difference in the records that needs a decision. It includes its sources and can describe a slow handoff, disputed fact or missing owner.. A first measurement of a couplingCouplingA link where one team or service depends on another. Its Loose, Buffered or Rigid class describes the time needed for a handoff. nobody had estimated becomes a first-measurement finding instead.
4. Every observed write is added to the structural log with a hybrid logical clockHybrid logical clockA number attached to a log entry to order it against other entries. It combines clock time with a counter, so entries can be ordered even when clocks do not agree exactly. (HLC) stamp. It does not require a separate consensusConsensusA process in which connected services agree on one result before accepting it as a shared decision. decision.

Contested structure is never settled by the tool. It is surfaced as a finding for a person to resolve.

Applying a new event in the materialiser

If your connector fits an existing event, such as the cell axis observation, you add no new event and no migrationMigrationAn ordered change to a database's tables or rules. Applied migration files are recorded so the same file is not run again.. Stop there.

If it needs a new kind of observation, as the network-flow acquirer did, you add three things together:

1. A commit builder in conway_mesh/commits.py, such as network_flow_observation. It fixes the required fields, rejects any other field set and validates enumerations.
2. A materialiserMaterialiserThe component that reads recorded changes in order and updates the database. It records its place so it can continue after a restart. branch in conway_graph/materialiser.py, in _apply. It resolves the cell and the cited artifacts, and it calls _correction_epoch_admits. That check protects the database from older correction passes: an observed write with a token below the correction epochCorrection epochThe highest correction token already applied for one organisation. An observation with a lower token belongs to an older correction pass and is refused. already applied is refused and quarantinedQuarantineA holding area for a log entry that could not be applied. The entry keeps its error and waits for a retry or an operator..
3. A migration in apps/graph/migrations/, numbered after the latest, such as 0051_network_flow_observations.sql. Give the table the fencing_token, observed_at_ms, valid_until_ms and last_hlc columns, with a check that the window is valid. The installation’s init service applies it on the next ./lens start.

Read a new table through the read modelRead modelThe database tables used by the map, Ask and Find. NovusLens rebuilds these tables from its history of recorded changes. only. See the read model.

The command

Add the kind to the --kind choices in __main__.py. Add a branch calling run_acquire_<kind>. Print its result, including unmeasured and unmapped targets. The CI branch is the template.

Interfaces

The --token bearerBearer tokenA secret value sent with a request to prove that the caller holds it. Anyone who obtains it can use it, so it must be protected. is sent as an unredirected header, so a redirect to another host never receives it.

Invariants and failure behaviour

• A cell is scored whole or not at all, unless the calibration sets partial_axes.
• A partial listing writes nothing for that target.
• A stale holder of the correction lock is refused at the sink and quarantined, not applied.
• A source that goes quiet is not read as healthy. Entries expire, and the twin says so.

Tests and gates

Build your tests on apps/connectors/tests/test_ci_acquire.py.

1. Add a fixtureFixtureA prepared set of records used for a test or demonstration. A fixture is sample data, not a live customer's records. of raw records and a mapping under apps/connectors/fixtures/, as meridian_ci_runs.json and meridian_ci_mapping.json are. Meridian GridMeridian GridThe fictional demonstration organisation used by the demonstration walk and its sample data. Nothing about Meridian Grid describes a real company. is a fictional demonstration organisation.
2. Serve the fixture in the real API shape from sim_sources.py (sim_sources.serve(fixture, port)). Add a handler there if your API shape is new.
3. Assert that the derived quantities reproduce the exact expected score, then test each refusal: partial paging, unmapped targets, a missing quantity and the simulated label.
4. If you added an event, add tests under apps/graph/tests/ that exercise the materialiser, as test_integrity.py does for correction tokens.

Then run the release checks that apply. The test check runs the unit tests and the supply-chain auditSupply-chain auditA check of the software packages the product depends on against reported security problems. An incomplete report cannot pass.. kit tests a clean organisation through product paths and refuses organisation-specific tokens there. Keep customer and demonstration names out of connector code. See the release checks and the command reference.