Cloud and datacentre inventories, the infrastructure under each cell.
An estate export lists the infrastructure an organisation runs: cloud accounts, projects and subscriptions, the resources inside them, and the datacentre configuration items. NovusLens turns each export into infrastructure cellsCellA box on the map that stands for a team, service, supplier or AI agent. It is a unit you can inspect, not necessarily a department or a person. and places them under the team cells they serve. Where the export declares that one resource depends on another, the dependencyDependencyA link where one unit needs something from another. A dependency is not a reporting line: it shows where work relies on something else. becomes an edge in the mapMapThe view of teams, services and their dependencies for the selected part of the organisation. Details appear beside the map..
Everything the export establishes lands at the shownShownEvidence that a document supports a value. A recovery plan that states a target shows the target, but it does not show that the target was achieved. rungRungOne of three levels of support for a recorded value: told, shown or measured. The level appears beside the value. on the evidence ladder. It is what the export says, not what the platform observed. TelemetryTelemetryMeasurements and activity records sent by a running system, such as ticket timings, network traffic or workflow results. is what raises a claim to measuredMeasuredThe top evidence level: a connected source recorded the observation within a stated period. A measurement shows what was observed on that occasion, not that it always holds..
What the inventories carry
Each sourceSourceA document, interview statement, measurement or export that supports a value. Each source has a date and a stated scope. contributes three things: its resources as infrastructure units, the containersCloud resource containerA cloud account, project, folder, resource group or business service that holds other resources. Its position in the hierarchy determines which tags the importer reads. those resources sit in, and any dependencies the export declares. One command reads all five shapes. The --source option chooses the shape.
A CMDBCMDBConfiguration management database: an inventory of systems and the relationships recorded between them. export declares no dependencies. Those come later, from telemetry.
How a unit reaches a cell
Every shape carries a domain tag, label or field. Here, domain names a team cell created by the org chart; it does not name a business area in the domain taxonomyDomain taxonomyThe list of business areas, their owners and the teams or services that do the work in each area.. Its value is matched against the cells already asserted in the map. The match uses the words in cell ids, rather than an exact character match. A name sharing those words can resolve. A tag that matches no cell opens an unknown-domain-<org>-<domain> finding. The tag is read from the resource itself, or from its immediate containerCloud resource containerA cloud account, project, folder, resource group or business service that holds other resources. Its position in the hierarchy determines which tags the importer reads.: the AWSAWSAmazon Web Services: a cloud platform whose account and resource exports NovusLens can read. account, the GCPGCPGoogle Cloud Platform: a cloud platform whose project and resource exports NovusLens can read. project, the Azure resource group, or the CMDB business service.
A tag higher up, on an organisational unit, a folder or a subscription, names that containerCloud resource containerA cloud account, project, folder, resource group or business service that holds other resources. Its position in the hierarchy determines which tags the importer reads.’s cell. It does not reach the resources under it. A resource with no tag on itself or its immediate containerCloud resource containerA cloud account, project, folder, resource group or business service that holds other resources. Its position in the hierarchy determines which tags the importer reads. is not placed under a domain.
Docker commands
For the Compose installation, run commands from the checkout that holds .novuslens.env and compose.yml.
Prepare the input described below, then import it with ./lens, which uploads the export into the running reasoning containerContainerA running program with its own packaged software and isolated environment, managed by Docker on the host.:
./lens connectors ingest --org harbour --source aws --fixture estate.jsonUse your organisation id and the chosen account wherever a command asks for them.
For a cross-provider link, upload every export with ./lens exec -- sh -c 'cat > /tmp/gcp.json' < gcp.json and give each --estate its /tmp/ path.
Follow the complete Operations manual for accounts, configuration and command access.
Before you start
- Import the org chart first, so the team cells exist. The
domaintags in an estate export name the team cells, which must already be in the map. - Export each source in its own file, in one of the five shapes below.
- Run each command from the installation directory.
./lensruns it in the installation’s reasoning service, with the installation’s configuration. It first copies any local file named by--fixture,--file,--mappingor--enrolmentinto the service. - Choose the organisation id once and use it in every command.
The five shapes
AWS
The file has three top-level lists:
| List | Fields per entry |
|---|---|
organizational_units | id, name, optional tags.domain |
accounts | id, name, optional ou and tags. ou names a unit. Tags can include domain, region and iac. |
resources | name, optional account, type, region, tags.domain and depends_on. depends_on lists resource names. |
{
"organizational_units": [{"id": "ou-1", "name": "Operations", "tags": {"domain": "operations"}}],
"accounts": [{"id": "111122223333", "name": "bookings-prod", "ou": "Operations", "tags": {"domain": "bookings", "region": "eu-west-2"}}],
"resources": [
{"name": "bookings-api", "account": "bookings-prod", "type": "ecs-service", "depends_on": ["bookings-db"]},
{"name": "bookings-db", "account": "bookings-prod", "type": "rds"}
]
}GCP
The file has three top-level lists:
| List | Fields per entry |
|---|---|
folders | name, displayName, optional labels.domain |
projects | projectId, optional displayName, projectNumber, folder and labels. folder names a folder’s display name. Labels can include domain and region. |
resources | name, optional project, type, region, labels.domain and depends_on |
{
"folders": [{"name": "folders/100", "displayName": "Customs", "labels": {"domain": "customs"}}],
"projects": [{"projectId": "harbour-customs-prod", "folder": "Customs", "labels": {"domain": "customs", "region": "europe-west2"}}],
"resources": [{"name": "customs-filing-run", "project": "harbour-customs-prod", "type": "cloud-run", "depends_on": ["customs-declarations-db"]},
{"name": "customs-declarations-db", "project": "harbour-customs-prod", "type": "cloud-sql"}]
}Azure
The file has four top-level lists:
| List | Fields per entry |
|---|---|
management_groups | id, displayName, optional parent and tags.domain |
subscriptions | subscriptionId, displayName, optional managementGroup and tags.domain |
resource_groups | name, optional subscription, location and tags. Tags can include domain and iac. |
resources | name, optional resource_group, type, location, tags.domain and depends_on |
parent matches another group’s displayName converted to lower case with hyphens. It does not match that group’s id.
{
"management_groups": [{"id": "mg-harbour", "displayName": "Harbour", "parent": null}],
"subscriptions": [{"subscriptionId": "0000-wh", "displayName": "Warehousing Prod", "managementGroup": "Harbour", "tags": {"domain": "warehousing"}}],
"resource_groups": [{"name": "rg-wms", "subscription": "Warehousing Prod", "location": "uksouth", "tags": {"domain": "warehousing"}}],
"resources": [{"name": "wms-app", "resource_group": "rg-wms", "type": "Microsoft.Web/sites", "depends_on": ["wms-sql"]},
{"name": "wms-sql", "resource_group": "rg-wms", "type": "Microsoft.Sql/servers"}]
}Terraform
The state file itself. Each resource is read for its type, name, optional module, and provider. Its attributes are read for tags or labels (for domain), and for region or location. Its dependencies become the declared dependencies.
Every instance of a resource is read, not only the first. An instance that carries an index key has that key appended to its name. A unit’s id is qualified by its module, so a resource called api in the module bookings becomes bookings-api.
{
"version": 4,
"resources": [
{"module": "bookings", "type": "aws_ecs_service", "name": "api",
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
"instances": [{"attributes": {"region": "eu-west-2", "tags": {"domain": "bookings"}},
"dependencies": ["module.bookings.aws_db_instance.main"]}]},
{"module": "bookings", "type": "aws_db_instance", "name": "main",
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
"instances": [{"attributes": {"region": "eu-west-2", "tags": {"domain": "bookings"}}}]}
]
}CMDB
Top level: configuration_items. Each needs ci_id and ci_name. Optional fields are ci_class, business_service (the cell), datacentre, and a dcim object containing rack and power details. A CMDB export declares no dependencies; those come from telemetry later.
{
"configuration_items": [
{"ci_id": "CI0001", "ci_name": "wms-db-01", "ci_class": "server", "business_service": "warehousing", "datacentre": "felixstowe-dc1"},
{"ci_id": "CI0002", "ci_name": "wms-app-01", "ci_class": "server", "business_service": "warehousing", "datacentre": "felixstowe-dc1"}
]
}How to send it
From the installation directory:
./lens connectors ingest --org harbour --source aws --fixture estate.json--orgis a top-level option, so it comes before the subcommand nameingest.--sourceis one ofaws,gcp,azure,terraformorcmdb. Any other value is refused by the command line.--fixtureis the path to the export file.--solicited-byis optional. It links the import to an open request for this artefactArtifactA stored source file or evidence body that a record can cite. Keeping the exact content lets a reader check what supported the record., if one exists.--hostand--portdefault to the meshMeshA set of connected Mycelium services that exchange updates and keep the NovusLens log. address in the environment, as described on the org chart format.
The estate exports have no browser import. Send them from the command line only.
What a re-run does
The table below is the guide’s description of a re-run. Where the code differs, the difference is stated below the table.
| Situation | What happens |
|---|---|
| Stable identity | The resource’s name converted to lower case with hyphens, rather than its cloud id. Group cells are named by provider and group. |
| The same export twice | The same artefact; cells and edges written again with the same values. |
| A resource whose domain tag changes | Re-parented under the new domain. |
| A renamed resource | A new cell; the old one stays. |
| Two resources with the same name in different accounts or projects | Both are left out of the map and a duplicate-unit-id finding names them. Qualify the names or split the export. |
| A resource or dependency omitted from the export | Untouched. Nothing is ever removed from the implementation layer; an export cannot say a resource is gone. |
| A resource the provider reports and the infrastructure codeInfrastructure codeFiles that describe the computing resources an organisation intends to create and how they should be configured. does not describe | An estate-drift finding names both accounts. Neither is corrected. |
| A resource the code declares and the provider does not report | The same: a drift finding, both accounts kept. |
| A tag naming a domain the map lacks | The resource is not created; an undeclared-domain finding opens. Re-run once the cell exists. The finding does not close by itself. |
| Cross-provider edges | Only after the link command, which re-writes them on each run. |
| When visible | After the projectionProjectionThe regular pass that brings the read model up to date with the canonical log. It runs every 30 seconds by default. service’s next pass. |
Two differences from the table, as the code behaves:
- The Terraform unit id is qualified by its module, so a resource is named
<module>-<name>rather than by its name alone. A renamed module therefore creates new cells. - Terraform reads every instance of a resource, not only the first.
Linking cross-provider dependencies
Each ingest run reads one source. Dependencies that cross a provider boundary, such as an AWS service that calls a GCP one, are resolved by the link command. Run it after every source has been ingestedIngestRead supplied records or documents and add their information to NovusLens, after the required checks.:
./lens exec -- sh -c 'cat > /tmp/estate.json' < estate.json
./lens exec -- sh -c 'cat > /tmp/gcp.json' < gcp.json
./lens connectors link --org harbour --estate aws=/tmp/estate.json --estate gcp=/tmp/gcp.json--estateis repeated, once for each source, in the formSOURCE=FIXTURE.- Each run of the link command re-writes the cross-provider edges. It also runs the estate drift analysis, so drift findings are raised by the link command as well.
What you will see
On success the command exits without error and prints one summary line for each ingest run:
ingest: 4 units → 3 infra cells; structure documented for bookings, operations (artifact 3f9a1c0b2d4e…)If any tag named a domain the map lacks, the line ends with UNKNOWN domains flagged: <ids>. The figures above belong to the illustrative example, not to a real run.
The link command prints a summary of the form link: N service dependency edges resolved across the estate; R cross-provider/cross-region resilience findings surfaced; D declared-vs-actual estate drift finding(s).
The map shows the result after the projection service’s next pass, about 30 seconds by default. Each infrastructure cell sits under its domain cell, and each declared dependency appears as an edge.
When something goes wrong
The estate parsers have no refusal messages of their own. A file that is not valid JSONJSONJavaScript Object Notation: a text format for data, using named fields, lists and values. Configuration files and service messages use it. stops the command with a Python traceback. Valid JSON can still lack a field the parser needs. The command then stops with a traceback. The last line of the traceback names the missing field. The next line names a file with the full traceback. That file can hold secrets: see Configuration. The command exits with a non-zero status.
Some problems do not stop the command. They raise findings and leave the affected item out of the map:
- Two resources that share a name in the same export are held out of the map. A duplicate-unit-id finding names them.
- A tag that names a domain the map lacks creates an undeclared-domain finding. The resource is not created until the domain cell exists.
Fix the export, or create the missing cell, and run the import again. Check the findings after the next pass, because a finding does not close by itself.