Skip to Content
ArchitectureProjection and read modelAgent and governance tables
Architecture · Read modelRead modelThe database tables used by the map, Ask and Find. NovusLens rebuilds these tables from its history of recorded changes.

Agent evidence is recorded, never acted on.

What it is for

These tables hold signed evidence about agentsAgentA program that takes actions for an organisation. NovusLens records what it is allowed to do and what its activity reports show., each subject’s recorded remitRemitA written list of an agent’s allowed actions, resources and destinations for information. The list states whether it is complete., and the findingsFindingA difference in the records that needs a decision. It includes its sources and can describe a slow handoff, disputed fact or missing owner. derived from both. A remit records the scopeScopeThe work, data, activity or access that a rule, statement or calculation covers. For access, it names the records an account can see. For team measures, “Own operations” covers the selected unit. “Including descendants” adds the units beneath it. the agent is permitted to act within. Agent views, intervention authorityAuthorityWhat a team or agent is allowed to decide or do. For an agent, this comes from a named person or system that grants permission. assessments, fleet incidentsFleet incidentA finding about several agents that completed the same operation above their recorded limits in one period. The agents must share something recorded in the model., remit driftRemit driftReported agent activity that breaks a recorded exclusion or falls outside a list declared complete. An incomplete list alone does not prove that activity is forbidden. and control-gap findings all read from this layer. The in-product GuideGuideThe help pages inside NovusLens. They explain the pages, input formats and tasks in the version you use. They are product instructions, not evidence about the organisation. is loaded by the same package, but it is product documentation and is not organisational evidence.

The findings separate three questions that are often confused. CapabilityCapabilityWhat a team can provide or an agent can do. NovusLens records the claim and the evidence that supports it. is what an agent is declared to be able to do. Authority is what it has been granted permission to do. ResponsibilityResponsibilityThe work or result that a team or agent answers for. NovusLens records the person answerable for it. is who is answerable for it. The governance findings below are built on that separation.

A signed batch becomes admitted records, the authority evaluators read those records, and the analysers turn their results into findings.

How it works

Admission of evidence

A source is enrolled in operator policy. Each activation of that policy is a row in agent_evidence_policies, one per epochEpochFor timestamps, the starting point is 1 January 1970 at midnight in the shared world time reference. For evidence policy, it is the recorded version number.. NovusLens records the activation time itself, and activate_on in conway_reasoning/agent_policy.py refuses a clock that precedes the last activation. The policy also carries two operator settings: fleet_window_ms, which bounds fleetFleetA group of AI agents considered together. Related activity can be grouped only when the records establish what the agents share. grouping, and evidence_horizon_ms, which bounds the observations an org-wide read considers.

A signed batch reaches stage() in conway_reasoning/agent_evidence.py. The source must be enrolled and enabled, and its signing key must match the enrolled key. Each record must name a subject that the enrolmentEnrolmentThe operator's recorded permission for a source to send specified record types about named agents, using an identified signing key. links to a cellCellA box on the map that stands for a team, service, supplier or AI agent. It is a unit you can inspect, not necessarily a department or a person., and a type the source is allowed to send. Intake also checks enrolled grant issuers, authorisers and acceptors. Remit issuer eligibility is checked when records are read for an assessment. The signed bytes are stored as their own artifactArtifactA stored source file or evidence body that a record can cite. Keeping the exact content lets a reader check what supported the record..

enqueue() in conway_graph/agent_evidence_store.py then stages the batch in the caller’s transactionTransactionA group of database changes that is saved together. If the group fails, its changes are not saved.. It stamps each record with a server-side received_at, so caller-supplied receive times are discarded. It accepts between 1 and 1,000 records and up to 2 MiBKiB and MiBBinary size units: one KiB is 1,024 bytes; one MiB is 1,048,576 bytes. Evidence limits use these units., and it writes the artifact and an agent_evidence_outbox row together. The source is acknowledged only after that commit. A retry with the same batch ID and the same content returns the same artifact. The same batch ID with different content is refused.

publish_pending() publishes committed outboxOutboxA database list of saved messages waiting to be published. A message stays on the list until its progress is recorded. rows as agent_evidence_batch commits through the meshMeshA set of connected Mycelium services that exchange updates and keep the NovusLens log. writer, at most 100 at a time. A row whose publication has not been recorded is published again after 60 seconds. A failed publication is never marked delivered. agent_delivery.deliver() runs this per organisation under an advisory lockAdvisory lockA database lock that cooperating programs take to prevent conflicting work at the same time. They must explicitly use the lock., then materialises. Its state sits in agent_evidence_delivery, with a status of pending, projected or retrying. A failed delivery sets retrying. Attempts are capped at 10, and only an error class name is stored.

apply_batch() runs when the materialiserMaterialiserThe component that reads recorded changes in order and updates the database. It records its place so it can continue after a restart. projects a batch. The artifact body must match its identifier. The bundle needs version 1 and 1–1,000 records, each naming the bundle’s source. A record ID already stored with the same hashHashA fixed-length fingerprint calculated from content. Comparing fingerprints helps detect changes and match records. is skipped. A record ID stored with a different hash is refused. The batch is then recorded in agent_evidence_batches, so replaying it changes nothing.

read_history() returns the history for one subject as known at a given time. It resolves each record body, refuses evidence that has been purgedPurgeRemoving a stored document’s contents while keeping a record that it was removed and why. Conclusions that used it must be checked again., and passes the records to history() in packages/common_py/src/common_py/agent_evidence.py. That function keeps every received record and applies correctionsCorrectionA change to a recorded value or record. The earlier version stays in the history, and any conclusion that depended on it is checked again. and withdrawalsWithdrawalA source’s removal of a value or record from current use. Its history stays, and NovusLens checks conclusions that depended on it again. to their targets. An unresolved amendment is listed as pending with a reason. The target may be unavailable, belong to another source or subject, or itself be an amendment.

agent_cursor.advance() tracks source progress in agent_evidence_cursors. It stores positions supplied by each source and stream without interpreting them. Each advance must start from the current head of its stream, may not reuse a position, and runs under the same intake lock.

Authority evaluation

conway_graph/agent_authority.py holds three pure functions. Each takes records and eligible record identifiers. It reads no database and returns a status with the identifiers used. The eligible sets are not taken from the payloadPayloadThe data carried inside a message, apart from the surrounding fields that identify and date it.. The reasoning layer’s _gather() checks the current policy. Eligibility needs an enabled source, matching admitted key, unchanged subject binding, allowed type and an available signed source. For grants, authorisations, acceptances and remit declarations, the issuer must also be enrolled.

FunctionWhat it answersStatuses it can return
evaluate_operationOne operation against the grant that applies to its subject, operation and resource.insufficient-evidence, conflicting-evidence, within-checked-boundary, outside-checked-boundary. An outside result on a completed operation also sets completed_breach.
evaluate_caseOne intervention request: whether it was authorised, whether it was executed, and whether the observed outcome supports the stated criterion.Authorisation: approved, refused, not-evidenced, conflicting-evidence. Outcome: insufficient-evidence, pending-window, conflicting-evidence, supported, contradicted.
evaluate_activityOne observed activity against the recorded remit for its subject.within-recorded-remit, outside-recorded-remit, authority-not-established, insufficient-evidence, conflicting-evidence.

evaluate_operation compares only the per-operation upper bound. It does not aggregate across operations. evaluate_case needs an explicit criterion from the caller (a name, unit, window and expected value). It counts an authorisation only when both that record and its cited grant are eligible. The grant must apply to the authoriser and cover the request’s action, resource, unit and value. The authorisation’s event time must be at or after the request’s, and within the grant’s validity interval. Several executions of one request are reported as conflicting, not added together.

evaluate_activity checks three axes: operation, resource and destination. On each axis the result is one of three things. The axis is checked when a value was observed and compared. It is not applicable when the remit does not constrain that axis. It is unobserved when the remit constrains the axis but the source recorded no value. The operation axis is unobserved if the connectorConnectorSoftware that reads records from a source system, such as an incident tool. A customer-written file connects those records to teams or services in NovusLens. could not map the original call to a business operation. A replaced or removed catalogue mapping has the same result. The effect of an activity is taken from the runtimeRuntimeThe software that runs a model or agent. A model runtime answers model requests; an agent runtime carries out the customer's agent actions.’s own record. It is none only when the resource reported the call as blocked. A refusal on its own does not establish that nothing ran, so it is recorded as unknown until the resource says otherwise.

Missing evidence gives insufficient-evidence or authority-not-established. It is never read as permission. An incomplete remit cannot prove prohibition. Activity outside its listed permissions is therefore reported as authority-not-established.

Acceptance records

conway_graph/agent_acceptance.py lists the acceptance records that name a request and its resource. Each one carries a reference_status: references-current, references-changed, references-unavailable, decision-superseded-or-withdrawn, publisher-not-eligible or updating. An acceptance is shown as a record of an external decision. It is not treated as assurance, and it does not change the authority statuses above.

Fleet incidents

conway_graph/fleet.py groups completed over-limit operations, and activities outside a recorded remit, into incidents across several agents. An incident is derived from the evaluations at read time. It has no evidence record of its own. A group needs at least two different agent cells, and all of its members must fall inside one window. Over-limit operations must share the same operation and resource. Remit drift members must share the same operation and fail the remit in the same way. A member joins at most one incident, using the first basis that holds:

BasisHolds when
parentThe members’ agent cells serve the same unit, through parent_cell_id.
providerThe members depend on a common provider through an asserted edge.
grantThe members act under the same grant issuer and version. Remit drift uses the remit issuer and version.
Members are tried against the three bases in order and join the first that holds.

The window defaults to 15 minutes (DEFAULT_WINDOW_MS). The operator may set fleet_window_ms on the active policy, between one minute and one day. A source cannot set it. Incidents are stored as fleet-incident findings. Each states what its members have in common and that it is evidence of a shared pattern, not a verdict on intent. An incident keeps its identity when its membership changes. A member whose support lapses stays named, and each membership change is recorded as a transition. The analyser never resolves or retracts a finding.

Remit drift findings

conway_graph/remit.py raises one remit-drift finding per subject, agent cell and remit, grouped by window. Only results with the status outside-recorded-remit count. Attempts that the resource reported as blocked are excluded. Each finding lists the failed sets, such as an explicitly excluded destination. It keeps the detector’s classification unchanged and states how the activity was attributed. Attribution is group-level where a credential is shared. A request that was refused at the resource yet recorded as completed is named as an enforcement gap. The finding says what was observed. It does not offer a diagnosis.

A result of authority-not-established never becomes a finding.

Governance control-gap findings

conway_graph/governance.py reads the agent cells (their granted authority and accountable ownerAccountable ownerThe person recorded as answerable for a team, service or agent. NovusLens does not assume that this person is also the one who runs it day to day.), the external dependenciesDependencyA link where one unit needs something from another. A dependency is not a reporting line: it shows where work relies on something else., and the capabilities declared on asserted edges. It then raises one of four kinds of control-gap finding:

Finding kindRaised when
control-gap-overdelegationAn agent holds declared authority but names no accountable owner.
control-gap-declaration-missingAn agent has observed activity but no mapped capability declaration.
control-gap-beyond-authorityAn agent is observed acting on capabilities that its declaration does not include.
control-gap-declared-control-unmetThe measured SLASLAService-level agreement: a recorded service commitment, such as a target percentage of availability. A declared target is kept separate from measured attainment. attainment is below the declared target, for an agent or an external dependency.

The behaviour telemetryTelemetryMeasurements and activity records sent by a running system, such as ticket timings, network traffic or workflow results. carries its own label. Seeded data is marked synthetic and described as illustrative. A live feed from a simulated agent estateAgent estateThe collection of AI agents an organisation operates, together with the systems and providers they use. is marked simulated-estate. Other sources are described as enrolled-source or unverified-publisher reports, and their observation method and coverageCoverageWhat the available records cover, and what is missing. A count such as “3 of 5 dependencies measured” names the measured portion and the gap. are described as not independently established. The three boundary-measured kinds carry generated_support, which is built by the helpers in conway_graph/boundary_support.py.

The Guide loader

conway_graph/guide.py loads the product Guide. By default it reads config/guide.json at the repository root. The --path option overrides that. The file must be version 1 and contain a pages list. Each page needs non-empty id, section, title, audience, summary and body_md fields, or the loader raises a ValueError. Each page is upsertedUpsertAdd a record if its identifier is new, or update the record with that identifier if it already exists. into guide_pages by its ID. Any row whose ID is not in the file is then deleted, so the Guide always describes the build that is running. The table has no organisation column. The Guide is never a finding, never coverage and never part of a brief. The init service (deploy/docker/bootstrap.py) runs the loader, and ./lens graph guide status reports what is loaded.

Remit authoring

conway_reasoning/remit_authoring.py keeps drafts and approvals in remit_declarations and remit_templates. Approval publishes a signed remit_declaration from a product-owned source. This record passes the same intake checks as other sources. The approval rules are set out in that module.

Interfaces

Tables

TableMigrationMigrationAn ordered change to a database's tables or rules. Applied migration files are recorded so the same file is not run again.Written byHolds
agent_evidence_outbox0028enqueue()Committed batches waiting to be published.
agent_evidence_batches0028apply_batch()Batches that have been projected, with the last HLCHybrid logical clockA number attached to a log entry to order it against other entries. It combines clock time with a counter, so entries can be ordered even when clocks do not agree exactly. applied.
agent_evidence_records0028apply_batch()One row per admitted record: its subject, type, hash and position in its artifact.
agent_evidence_delivery0029agent_deliveryPer-organisation delivery state. This is operational state, not an evidence verdict.
agent_evidence_policies0030activate_on()Operator activation history, one row per epoch.
agent_evidence_cursors0031agent_cursor.advance()Producer cursorCursorA saved position in a sequence of records. A program uses it to continue from where it stopped. positions per source and stream.
boundary_evidence_heads0032boundary_support.record_head()The latest legacy boundary acquisition per source, and whether it conflicted.
product_sources0041Remit authoringThe product-owned source’s key pairSigning key pairTwo related keys: the private key makes a signature and the public key checks it. The sender must keep the private key secret.. The private keyPrivate keyA secret value used to create digital signatures. The source keeps it so others cannot sign evidence in its name. is held on the customer host.
remit_templates0041, 0042Remit authoringVersioned remit templates, with their membership and a draft or approved status.
remit_declarations0041, 0042, 0043, 0044Remit authoringRemit declarations per subject. At most one is approved per subject. Each keeps the cell it was scoped to.
activity_catalogues0042conway_reasoning/catalogue.pyNamed, versioned catalogues of business activities, with their owner and coverage gaps.
remit_exports0042Remit authoringA record of each exported policy artifact, with its format, digest and the clauses the target cannot carry.
guide_pages0039guide.load()The Guide pages for the running release.

The table definitions in full are on the schema page.

Calls in and out

Calls in come from conway_reasoning/agent_evidence.py (ingest, refresh, refresh_derived, assess and activities), from the materialiser, which calls apply_batch(), and from the delivery worker in conway_reasoning/agent_delivery.py. Calls out go to the mesh writer (TwinWriter, with commits.agent_evidence_batch and commits.finding_open) and to the evidence artifact store (store_artifact_in_transaction and resolve_citation).

Invariants and failure behaviour

  • Intake is idempotentIdempotentRepeating the same operation has the same effect as doing it once: it creates no extra copies or repeated changes. by batch ID, and record IDs are immutable. Reusing an ID with different content is refused. Nothing is overwritten.
  • A projected batch is checked against its artifact hash before any record is written. A mismatch is refused.
  • A failed publication is retried, and it is never marked delivered. Delivery failures store only an error class name.
  • Purged evidence is refused on read. Its records also stop being eligible, so they no longer support any status.
  • Historical policy is not inferred. A cutoff earlier than any recorded policy is refused.
  • The fleet and remit analysers open or refresh findings in place. They never resolve or retract one. Membership changes are recorded, and support is checked again at read time.
  • Findings from every analyser are evidence to be reviewed. None of them is a verdict on an agent or a person.

The projection page describes how these tables are rebuilt from the log, and which of them survive a reset.

Last updated