Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ On a PR you can comment `@coderabbitai review` (incremental) or `@coderabbitai f

## Development guidelines

- Keep collectors **read-only** toward cloud accounts by default. `om scan workload` is the opt-in exception: it may create and delete snapshots in the account, and it stores package identifiers only
- Keep collectors **read-only** toward cloud accounts by default. `om scan workload` (and its `azure` and `gcp` subcommands) is the opt-in exception: it may create and delete snapshots in the account, and it stores package identifiers only
- Document graph schema changes in `docs/`
- Prefer small, reviewable PRs
- Add tests for graph queries and normalization logic
Expand Down
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ OpenSourceOM Core is the platform behind [opensourceom.org](https://opensourceom

Traditional scanners flood you with CVEs and misconfigurations. OpenSourceOM connects the dots — showing which findings sit on paths from the internet to your sensitive data and privileged identities.

> **Status:** Phase 3 has shipped. Phase 4 is open: `om scan workload` inventories AWS Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors ([#10](https://github.com/OpenSourceOM/core/issues/10)). See the [roadmap](./docs/ROADMAP.md).
> **Status:** Phase 3 has shipped. Phase 4 is open: `om scan workload` inventories AWS, Azure, and GCE Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors ([#10](https://github.com/OpenSourceOM/core/issues/10)). See the [roadmap](./docs/ROADMAP.md).

## Why this exists

Expand Down Expand Up @@ -95,6 +95,9 @@ export AWS_REGION=us-east-1
./om scan aws
# On an EC2 instance in that account, as root, after the API scan:
# sudo -E ./om scan workload
# On an Azure VM or a GCE instance in that region, as root:
# sudo -E ./om scan workload azure
# sudo -E ./om scan workload gcp
./om scan k8s # requires kubeconfig

# Or ingest a custom collector (stdout is a graph batch)
Expand Down Expand Up @@ -135,9 +138,11 @@ An API token is created only when an admin asks for one, from the console or wit
```bash
./om scan demo # sample environment, no credentials
./om scan aws
./om scan workload # root on an EC2 worker; Linux root-volume packages
./om scan azure # requires AZURE_SUBSCRIPTION_ID + az login
./om scan gcp # requires GCP_PROJECT_ID + ADC
./om scan workload # root on an EC2 worker; Linux root-volume packages
./om scan azure # requires AZURE_SUBSCRIPTION_ID + az login
./om scan workload azure # root on an Azure VM; Linux OS-disk packages
./om scan gcp # requires GCP_PROJECT_ID + ADC
./om scan workload gcp # root on a GCE instance; Linux boot-disk packages
./om scan k8s # requires kubeconfig or in-cluster credentials
./om scan plugin -- ./my-collector
```
Expand Down Expand Up @@ -165,7 +170,7 @@ Full documentation: [opensourceom.org](https://opensourceom.org) (docs at [opens
| **1** | Attack path queries, CVE enrichment, web UI, Azure/GCP collectors |
| **2** | CSPM rules, blast radius, K8s connector, exports |
| **3** | Plugin SDK, Helm, embedded rule pack, demo graph, inventory-backed CVEs, crown-jewel datastores, attack-path findings, and cloud audit context (CloudTrail, Activity Log, Admin Activity). Shipped. |
| **4** | Package inventory from persistent workload storage. AWS Linux root volumes ship with `om scan workload`. The other clouds, boot images, container layers, function bundles, and shared filesystems are open. |
| **4** | Package inventory from persistent workload storage. AWS, Azure, and GCE Linux root volumes ship with `om scan workload`. Boot images, container layers, function bundles, and shared filesystems are open. |
| **5** | Secret and malware findings on storage Phase 4 already opens. Kind and path only. Open. |

Details: [docs/ROADMAP.md](./docs/ROADMAP.md)
Expand Down
2 changes: 1 addition & 1 deletion collectors/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The **demo** collector loads a fixed environment (internet-exposed web tier, pro

The **AWS** collector emits properties the CIS pack matches on: `open_ingress`, `imdsv2`, `public_ip`, S3 `encryption` / `versioning` / `public_access_block`, and IAM user `mfa` / `unused_access_keys`. It also records RDS and Aurora instances as datastores. S3 bucket tags and the RDS `TagList` copy `sensitivity` or `data-class` onto the datastore. `sensitivity` wins when both are set. The same scan reads CloudTrail management events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. `om scan aws` does not read instance disks. `om scan workload` is the separate command that inventories packages on Linux root volumes.

The **Azure** collector records logical SQL servers. The same scan reads administrative Activity Log events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Entra ID sign-in logs are not part of this scan. The **GCP** collector records Cloud SQL instances. The same scan reads Admin Activity audit logs from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Data Access logs are not part of this scan. A workload gets `CAN_ACCESS` only when a security group, firewall, or VPC path allows it. Azure copies the crown-jewel mark from storage-account and SQL-server tags. GCP copies it from a bucket label or a Cloud SQL user label. The keys are `sensitivity` and `data-class`. A plugin may set `sensitivity` on a datastore directly.
The **Azure** collector records logical SQL servers. `om scan azure` and `om scan gcp` do not read OS disks. `om scan workload azure` and `om scan workload gcp` are the separate commands that inventory packages on those disks. The same scan reads administrative Activity Log events from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Entra ID sign-in logs are not part of this scan. The **GCP** collector records Cloud SQL instances. The same scan reads Admin Activity audit logs from the last 24 hours and stores the ones that name an identity and a resource on an exposed path. A failed lookup omits those events. Data Access logs are not part of this scan. A workload gets `CAN_ACCESS` only when a security group, firewall, or VPC path allows it. Azure copies the crown-jewel mark from storage-account and SQL-server tags. GCP copies it from a bucket label or a Cloud SQL user label. The keys are `sensitivity` and `data-class`. A plugin may set `sensitivity` on a datastore directly.

The **Kubernetes** collector records Ingress objects and whether a NetworkPolicy selects each pod. A pod is internet-reachable from a LoadBalancer or an Ingress backend unless that policy governs ingress and does not allow the world. NodePort alone is not exposure.

Expand Down
3 changes: 2 additions & 1 deletion deploy/helm/opensourceom/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ resources:
# Secret, or serviceAccountAuth). Kubernetes is off unless enabled; without
# a kubeconfig Secret the chart creates a read-only ServiceAccount for this
# cluster. `om scan demo` and `om scan workload` are not scheduled.
# The workload scan mounts disks and runs on an EC2 instance in the account.
# The workload scan mounts disks and runs on a VM in the account: EC2 for
# `om scan workload`, an Azure VM for `azure`, and a GCE instance for `gcp`.
scan:
schedule: "0 */6 * * *"
successfulJobsHistoryLimit: 3
Expand Down
8 changes: 4 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ SPDX-License-Identifier: Apache-2.0

# Architecture

> **Status:** Phase 3 has shipped. The collector plugin SDK, Helm, the embedded rule pack, and cloud audit context from CloudTrail, Activity Log, and Admin Activity logs are in the tree, alongside Phase 2 CSPM rules, blast radius, Kubernetes ingest, and exports. Phase 4 is open: `om scan workload` inventories AWS Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors.
> **Status:** Phase 3 has shipped. The collector plugin SDK, Helm, the embedded rule pack, and cloud audit context from CloudTrail, Activity Log, and Admin Activity logs are in the tree, alongside Phase 2 CSPM rules, blast radius, Kubernetes ingest, and exports. Phase 4 is open: `om scan workload` inventories AWS, Azure, and GCE Linux root volumes. Phase 5 (secret and malware findings on that storage) is open. Further rule packs stay open for contributors.

## Overview

Expand All @@ -32,7 +32,7 @@ SPDX-License-Identifier: Apache-2.0
| Component | Location | Notes |
|-----------|----------|-------|
| **Collectors** | `internal/collectors/` | AWS, Azure, GCP, Kubernetes, demo; RDS, Azure SQL, and Cloud SQL are datastores; AWS emits CIS pack properties and recent CloudTrail management events; Azure emits recent Activity Log events; GCP emits recent Admin Activity audit logs |
| **Workload packages** | `internal/workload/` | `om scan workload` reads AWS Linux root volumes and sets `packages` |
| **Workload packages** | `internal/workload/` | `om scan workload` reads AWS, Azure, and GCE Linux root volumes and sets `packages` |
| **Plugin SDK** | `sdk/collector`, `internal/plugins/` | External executables; `om scan plugin` |
| **Graph store** | `internal/graph/`, `migrations/` | PostgreSQL `nodes` + `edges` |
| **Path queries** | `internal/graph/query.go` | Named queries including `internet-to-datastore` and `internet-to-sensitive-datastore` |
Expand All @@ -52,7 +52,7 @@ SPDX-License-Identifier: Apache-2.0

Findings link to affected resources via `VIOLATES` edges. CSPM and CVE findings share the same `Finding` node type but use `finding_type` and `rule_id` / `cve_id` properties for provenance.

See [ADR 001](./adr/001-graph-schema-v0.md), [ADR 002](./adr/002-phase1-findings-ui.md), [ADR 003](./adr/003-phase2-cspm-rbac.md), [ADR 004](./adr/004-collector-plugin-sdk.md), [ADR 005](./adr/005-console-admins.md), and [ADR 006](./adr/006-aws-root-volume-packages.md).
See [ADR 001](./adr/001-graph-schema-v0.md), [ADR 002](./adr/002-phase1-findings-ui.md), [ADR 003](./adr/003-phase2-cspm-rbac.md), [ADR 004](./adr/004-collector-plugin-sdk.md), [ADR 005](./adr/005-console-admins.md), [ADR 006](./adr/006-aws-root-volume-packages.md), and [ADR 007](./adr/007-azure-gce-root-volume-packages.md).

## Technology choices

Expand Down Expand Up @@ -87,7 +87,7 @@ Further community rule packs (PCI and additional CIS mappings) stay open for con

Phase 4 writes `packages` (CPE 2.3 names and package URLs) from persistent workload storage onto the workload node that enrichment already matches. The sources, in order, are AWS Linux root volumes, Azure VM and GCE root volumes, boot images, container image layers and function bundles, and shared filesystems (EFS, Azure Files, Filestore). The scan is opt-in. `om scan aws`, `om scan azure`, and `om scan gcp` stay read-only API collectors.

`om scan workload` is the AWS root-volume scan. It runs as root on an EC2 instance in the account, snapshots the root volume, mounts the copy read-only, and stores package identifiers only. A later `om scan aws` keeps `packages`. Windows, LVM, and non-root disks are skipped. The other Phase 4 sources are still open. See [ADR 006](./adr/006-aws-root-volume-packages.md).
`om scan workload` is the AWS root-volume scan. `om scan workload azure` and `om scan workload gcp` are the same walk for Azure OS disks and GCE boot disks. Each runs as root on a worker in that account and region, snapshots the root disk, mounts the copy read-only, and stores package identifiers only. A later API scan keeps `packages`. Windows, LVM, and non-root disks are skipped. Boot images, container layers, function bundles, and shared filesystems are still open. See [ADR 006](./adr/006-aws-root-volume-packages.md) and [ADR 007](./adr/007-azure-gce-root-volume-packages.md).

Phase 5 adds a finding for a secret pattern or a malware signature on storage that Phase 4 reader already opened. The finding stores the kind and the path. Object bodies and database rows are in neither phase.

Expand Down
4 changes: 2 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,12 @@ The scan is opt-in and separate from `om scan aws`, `om scan azure`, and `om sca
The path, in order:

- [x] **AWS Linux root volumes** — [#102](https://github.com/OpenSourceOM/core/issues/102)
- [ ] **Azure VM and GCE root volumes** — the same walker, those snapshot APIs — [#103](https://github.com/OpenSourceOM/core/issues/103)
- [x] **Azure VM and GCE root volumes** — the same walker, those snapshot APIs — [#103](https://github.com/OpenSourceOM/core/issues/103)
- [ ] **Boot images** — an AMI and the Azure and GCE equivalents, scanned once and applied to instances launched from that image — [#104](https://github.com/OpenSourceOM/core/issues/104)
- [ ] **Container image layers and function bundles** — layers for images already stored on Kubernetes workloads, plus Lambda deployment packages and the Azure and GCP equivalents. No volume attach — [#105](https://github.com/OpenSourceOM/core/issues/105)
- [ ] **Shared filesystems** — EFS, Azure Files, and Filestore, when the application is installed there and a root-volume scan would miss it — [#106](https://github.com/OpenSourceOM/core/issues/106)

`om scan workload` reads Linux root volumes in the current AWS account. It runs as root on an EC2 instance in that account, snapshots the root volume, mounts the copy read-only, and writes CPE 2.3 names and package URLs onto `packages`. A later `om scan aws` leaves `packages` in place. Windows, LVM, and disks other than the root volume are skipped.
`om scan workload` reads Linux root volumes in the current AWS account. It runs as root on an EC2 instance in that account, snapshots the root volume, mounts the copy read-only, and writes CPE 2.3 names and package URLs onto `packages`. `om scan workload azure` does the same for managed OS disks from an Azure VM in that region, and `om scan workload gcp` does it for GCE boot disks from an instance in that region. A later `om scan aws`, `om scan azure`, or `om scan gcp` leaves `packages` in place. Windows, LVM, and disks other than the root volume are skipped.

Object bodies and database rows stay out. They are customer data, not the installed software. A public bucket or database is already an exposure finding from the API scan.

Expand Down
44 changes: 44 additions & 0 deletions docs/adr/007-azure-gce-root-volume-packages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
<!--
Copyright 2026 OpenSourceOM
SPDX-License-Identifier: Apache-2.0
-->

# ADR 007: Azure and GCE root-volume package inventory

**Status:** Accepted
**Date:** 2026-10-15
**Phase:** 4

## Context

[ADR 006](./006-aws-root-volume-packages.md) defined `packages` and `om scan workload` for AWS Linux root volumes. `om scan azure` and `om scan gcp` still record the virtual machine from the cloud API and do not read the OS disk, so a package installed on an Azure VM or a GCE instance never reaches `packages`.

The filesystem walk is the same. The snapshot and attach APIs are not.

## Decision

`om scan workload azure` and `om scan workload gcp` are subcommands of the same opt-in scan. `om scan azure` and `om scan gcp` stay read-only.

`om scan workload azure` runs as root on an Azure VM in the subscription. For each Linux VM in that region with a managed OS disk, it creates a snapshot and a disk in the worker's resource group and zone, attaches the disk to the worker, and mounts it read-only. `om scan workload gcp` runs as root on a GCE instance in the project. For each Linux instance in that region with a persistent boot disk, it creates a snapshot, creates a disk in the worker's zone, and attaches that disk read-only.

Both commands then run the package walk from ADR 006 and write CPE 2.3 names and versioned package URLs onto the workload node `om scan azure` or `om scan gcp` already stored. They delete the snapshot and the disk, including when a later step fails. The graph, logs, and exports store package identifiers. They do not store file contents.

Windows, ephemeral Azure OS disks, unmanaged VHDs, GCE local SSD boot disks, customer-supplied encryption keys, LVM physical volumes, and data disks are skipped. A customer-managed key is not copied into the graph. The identity needs the unwrap permission listed below, and the snapshot keeps the source disk's key.

Azure copies are tagged `opensourceom:scan=workload` and are created with public network access disabled. GCE label keys cannot contain a colon, so those copies are labeled `opensourceom-scan=workload`.

The Azure actions are `Microsoft.Compute/snapshots/write`, `Microsoft.Compute/snapshots/delete`, `Microsoft.Compute/disks/write`, `Microsoft.Compute/disks/delete`, `Microsoft.Compute/virtualMachines/attachDetachDataDisks/action`, and `Microsoft.KeyVault/vaults/keys/unwrap/action` (with `keys/wrap/action` when the snapshot keeps the key). The GCP permissions are `compute.disks.createSnapshot`, `compute.snapshots.delete`, `compute.disks.create`, `compute.disks.delete`, `compute.instances.attachDisk`, `compute.instances.detachDisk`, and `cloudkms.cryptoKeyVersions.useToDecrypt` (with `useToEncrypt`). The API collectors do not gain these permissions.

Boot images, container layers, function bundles, and shared filesystems stay on later Phase 4 issues. Secret and malware findings stay on Phase 5.

## Consequences

Each command has to run on a VM in the subscription or project, in the region being scanned. A process killed with SIGKILL can leave a snapshot or disk; the tag or label above is how to find it. The Helm chart does not schedule these commands.

`om enrich cve` is unchanged. It attaches a CVE only when one of these identifiers matches. The Azure graph id is still `azure:<subscription>:<AZURE_LOCATION>:workload:<vm name>`, which is the name alone. Two virtual machines with the same name in different resource groups already share that id.

## References

- [docs/ROADMAP.md](../ROADMAP.md)
- [ADR 006](./006-aws-root-volume-packages.md)
- [Issue #103](https://github.com/OpenSourceOM/core/issues/103)
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/OpenSourceOM/core
go 1.27.2

require (
cloud.google.com/go/compute/metadata v0.9.1
cloud.google.com/go/iam v1.12.0
cloud.google.com/go/storage v1.69.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.3
Expand Down Expand Up @@ -42,7 +43,6 @@ require (
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.23.3 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.1 // indirect
cloud.google.com/go/monitoring v1.30.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.13.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.8.0 // indirect
Expand Down
2 changes: 1 addition & 1 deletion internal/cmd/scan_workload.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Run it as root on an EC2 instance in that account. The command snapshots each ro

om scan aws stays a read-only API scan. A later om scan aws keeps packages already stored on the workload. Snapshots and volumes are tagged opensourceom:scan=workload, which is how to find one left behind by a killed process.

This command reads Linux root volumes. Windows instances are skipped. LVM and disks other than the root volume are skipped. Azure, GCE, boot images, container layers, function bundles, and shared filesystems are later scans.
This command reads Linux root volumes. Windows instances are skipped. LVM and disks other than the root volume are skipped. Azure OS disks are "om scan workload azure". GCE boot disks are "om scan workload gcp". Boot images, container layers, function bundles, and shared filesystems are later scans.

IAM actions:
` + strings.Join(awsv.Permissions, "\n ")
Expand Down
Loading
Loading