RootTrace

RootTrace Security Model

RootTrace is designed as a read-only diagnostics and incident intelligence platform by default.

Tenant Isolation

  • Tenant-owned MongoDB collections include organization_id.
  • API endpoints enforce organization membership before returning or mutating tenant data.
  • Similar incident search is scoped to the same organization_id.
  • Elasticsearch is used only for embedding/vector mirror data. Each workspace uses its own vector index named roottrace_<organization_id>, and MongoDB stores the reverse mapping to the authoritative incident, issue, comment, and postmortem documents.
  • Production MongoDB sharding is strict by default. Startup shards every managed collection and fails if any collection cannot be sharded safely.
  • Shard keys preserve unique indexes: global auth tokens shard by token hash, users shard by email, and tenant-owned operational data shards by organization_id where possible.

API Authentication

  • User API calls use asymmetric JWTs with a kid header. The default algorithm is post-quantum ML-DSA-65. Current validated-provider FIPS mode requires RS256 or ES256.
  • JWT signing keys are initialized in MongoDB and loaded from the jwt_signing_keys collection.
  • JWT private keys are encrypted at rest with key material derived from the database-backed application secret in app_secrets.
  • Production does not depend on local JWT key files. Do not use shared-secret JWT signing in production.
  • Each token carries a per-user uniqueId and sequence; the sequence is checked against users.token_sequences on every protected API call.
  • Expired access tokens are rotated using the same sequence model and returned with X-New-Token.
  • API middleware enforces bearer authentication across /api/* except public auth endpoints and collector-token ingestion endpoints.
  • Request bodies are size-limited at the ASGI boundary before JSON, protobuf, Pydantic, or webhook-signature parsing. Chunked requests cannot bypass the limit.
  • Passwords are hashed with PBKDF2-SHA512 and per-password salt.

Network Boundaries

  • A non-empty organization IP allowlist applies after tenant resolution, not only in middleware. It covers users, service accounts, SCIM provisioning, and collector/OTLP ingest.
  • IP allowlist lookup or enforcement errors fail closed with HTTP 503. An empty or absent allowlist remains allow-all for backward compatibility.
  • User-configured webhook and repository SSH destinations cannot reach private, loopback, link-local, reserved, or metadata addresses unless an operator explicitly lists the CIDR in ROOTTRACE_OUTBOUND_ALLOWED_PRIVATE_CIDRS.
  • HTTP redirects are checked through the same outbound resolver. Repository SSH connections are pinned to the checked address to prevent DNS rebinding.
  • OIDC discovery, token, and JWKS requests use the protected resolver and bounded JSON responses.
  • /metrics requires ROOTTRACE_METRICS_BEARER_TOKEN; outside development it is unavailable when no token is configured.
  • Flutter web builds bundle CanvasKit and its WebAssembly files locally. The dashboard does not depend on the Google Flutter CDN at runtime.
  • CSP and clickjacking controls are response headers in both the self-hosted API/nginx path and CloudFront. Cloud conversion tracking is narrowly allowlisted; on-prem image preparation removes the tracking scripts.

Collector Model

  • Collector tokens are scoped to one organization and one environment.
  • Collector tokens are returned once, hashed at rest, and revocable.
  • The default collector mode is read_only.
  • Collectors are outbound-only and do not accept inbound command execution.
  • Linux installs run as the non-login roottrace-collector user with a hardened systemd unit.
  • Docker and Kubernetes installs run as non-root, drop Linux capabilities, and support read-only filesystems.
  • Kubernetes host mounts are read-only and are used only for node diagnostics.
  • Assisted remediation requires explicit organization settings and still does not execute commands from AI output.

AI Safety

  • Diagnostic payloads are recursively redacted before storage and before AI summarization.
  • AI summaries include evidence, confidence, and a fallback when evidence is insufficient.
  • AI responses include source events, suggested next action, and a not_enough_data indicator.
  • AI endpoints do not execute commands.
  • Recommendations are manual instructions by default.

No Remote Execution Path

  • The server ships no mechanism for running commands on a monitored host: no SSH client for remediation, no configuration-management runner, and no command channel back down to a collector. This is absence, not a disabled feature: there is no flag that turns one on.
  • The only outbound SSH is read-only git fetch against a linked repository, authenticated with a per-repository deploy key. See change tracking.
  • Assisted remediation renders a runbook for a human to read. The commands in it are text.

Secrets

  • On-prem configuration comes from the container environment. Compose supplies it from a mode-0600 deploy/.env and mounts the application secret separately; the application never opens either file itself.
  • An on-prem image contacts AWS Parameter Store only when ROOTTRACE_SSM_PARAMETER_PREFIX is explicitly set. The supported stack, and every air-gapped deployment, leave it unset.
  • No configuration file is copied into an image layer, and the image build has no access to application parameters.
  • Do not commit private keys, cloud credentials, webhooks, API keys, collector tokens, or JWT secrets. Treat anything previously committed as exposed and rotate it.

See the server configuration reference for precedence details.

Production JWT Key Setup

The supported Compose deployment mounts an external application secret through ROOTTRACE_APP_SECRET_FILE. Keep that same secret with every database backup: it encrypts JWT private keys and integration secrets. If neither an inline nor file-backed secret is supplied, RootTrace generates one and stores it in app_secrets, but production operators should not depend on that fallback. When upgrading an older Compose installation that used it, run scripts/migrate_compose_app_secret.sh before the first target startup. The server refuses an external value that differs from the stored value so an accidental replacement cannot make existing encrypted data unreadable.

RootTrace creates the first ML-DSA-65 JWT signing key in MongoDB during API startup: no key generation step for the operator, and no key file on disk. Set ROOTTRACE_ENVIRONMENT=prod so the production baseline applies (production and live are accepted spellings).

The startup bootstrap stores active keys in jwt_signing_keys. Each record has an ObjectId _id, a kid, the public PEM, and an encrypted private PEM.

Signing keys rotate before a platform-admin algorithm update returns. Every API replica resolves the configured algorithm and active keys from MongoDB, so no process restart or cache invalidation is required. Older active public keys remain available for verification during the overlap. With ROOTTRACE_FIPS_MODE=true, startup accepts only RS256 or ES256 and also verifies that the OpenSSL FIPS provider is active.

Security Profiles and Workspace Administration

ROOTTRACE_SECURITY_PROFILE selects the process-wide default, regulated, or air_gapped startup baseline. Production defaults to regulated. The stricter profiles fail startup when required controls conflict with the selected posture. A platform admin can store the profile and JWT signing algorithm through the Flutter administration screen without restarting the API. The profile is rejected unless the running deployment already satisfies it. The API accepts all three standardized ML-DSA parameter sets, EdDSA, RS256, or ES256 and rotates the database signing key immediately after an algorithm change. ML-DSA-65 is the balanced post-quantum default standardized by NIST FIPS 204 and RFC 9964. Its signatures are larger than classical JWT signatures.

Workspace admins use Flutter workspace security settings for tenant-scoped controls: the workspace policy profile, privileged MFA, idle timeout, failed login lockout, concurrent-session cap, inactive-account disablement, and the logon banner. The screen also displays the effective process-wide FIPS, JWT/WebAuthn, token, password, secret, metrics, rate-limit, audit, request-size, egress, email, search TLS, proxy/CDN, CORS, collector-token, and TLS posture. Those values are read-only at workspace scope so one tenant cannot change another tenant's runtime boundary.

Deployment settings

Use the Compose install guide for the complete required configuration. Its template constructs ROOTTRACE_MONGO_URI and requires the public app/API identity, exact CORS and WebAuthn origins, JWT issuer, file-backed application secret, first-owner bootstrap token, and authenticated metrics before the stack starts. The published image already contains the license-verification key.

Optional:

  • ROOTTRACE_ENABLE_MONGO_SHARDING=true for sharded MongoDB clusters; production enables this by default
  • ROOTTRACE_MONGO_SHARDING_STRICT=true to fail startup if sharding cannot be applied to every collection; production enables this by default
  • ROOTTRACE_MONGO_SHARD_INITIAL_CHUNKS to control initial chunks for hashed shard keys
  • ROOTTRACE_ELASTICSEARCH_HOST when vector search is enabled; leave it empty otherwise
  • ROOTTRACE_ALLOWED_ORIGIN_REGEX only when exact ROOTTRACE_ALLOWED_ORIGINS entries cannot express a controlled origin policy
  • ROOTTRACE_COLLECTOR_INGEST_RATE_LIMIT_ENABLED=false to disable database-backed collector ingest rate limiting. Collector ingest limits are scoped to collector key plus host, not source IP, so large clusters behind a NAT gateway do not share one bucket.
  • ROOTTRACE_COLLECTOR_INGEST_MIN_INTERVAL_SECONDS=45 to control the minimum interval between accepted non-clock diagnostics from the same collector key and host
  • ROOTTRACE_API_MAX_REQUEST_BODY_BYTES and ROOTTRACE_API_MAX_STREAM_BODY_BYTES to tune the pre-parser request ceilings
  • ROOTTRACE_METRICS_BEARER_TOKEN to enable authenticated Prometheus scraping outside development
  • ROOTTRACE_OUTBOUND_ALLOWED_PRIVATE_CIDRS to permit narrowly scoped internal webhook, status-watch, or repository destinations
  • Local development allows localhost, 127.0.0.1, and [::1] on any port for Flutter web preflight requests. Production does not use this development regex by default; set exact ROOTTRACE_ALLOWED_ORIGINS values for deployed UI origins.
  • ROOTTRACE_COLLECTOR_IMAGE to control generated Docker/Kubernetes image references