From 4c8db1ec9433292916879634dade635535c61f14 Mon Sep 17 00:00:00 2001 From: max Date: Fri, 25 Sep 2026 22:12:37 +0200 Subject: [PATCH] Split state machines into modular, focused subsections for PKI enrollment, sync, forwarder discovery, and mTLS streaming --- README.md | 213 +++++++++++++++++++++++------------------------------- 1 file changed, 90 insertions(+), 123 deletions(-) diff --git a/README.md b/README.md index e9dde46..65faf08 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,13 @@ 2. [End-to-End System Architecture](#end-to-end-system-architecture) 3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles) - 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine) - - 3.2 [Master License Authority & WebSocket Enrollment State Machine](#32-master-license-authority--websocket-enrollment-state-machine) - - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine](#33-edge-forwarder-auto-enrollment--mtls-lifecycle-state-machine) - - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning-state-machine) + - 3.2 [Master License Authority & PKI Lifecycles](#32-master-license-authority--pki-lifecycles) + - 3.2.1 [Bootstrap & Sub-CA Auto-Enrollment State Machine](#321-bootstrap--sub-ca-auto-enrollment-state-machine) + - 3.2.2 [Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload](#322-live-websocket-heartbeat-seat-expansion--in-flight-reload) + - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycles](#33-edge-forwarder-auto-enrollment--mtls-lifecycles) + - 3.3.1 [Machine Identity Discovery & Zero-Touch Enrollment](#331-machine-identity-discovery--zero-touch-enrollment) + - 3.3.2 [mTLS 1.3 Streaming, Proactive Renewal & Self-Healing](#332-mtls-13-streaming-proactive-renewal--self-healing) + - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning) 4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing) 5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment) 6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning) @@ -119,152 +123,115 @@ stateDiagram-v2 --- -### 3.2 Master License Authority & WebSocket Enrollment State Machine -Details the bidirectional cryptographic enrollment, seat leasing, and hot-reload lifecycle between `logar.eibl.tech` and the LOGAR Server Hub: +### 3.2 Master License Authority & PKI Lifecycles + +#### 3.2.1 Bootstrap & Sub-CA Auto-Enrollment State Machine +Details how a newly deployed LOGAR Server Hub imports initial credentials, connects to `logar.eibl.tech` over WebSocket, and secures its signed Sub-CA certificate: ```mermaid stateDiagram-v2 - [*] --> Idle_Unlicensed: Server Hub initial launch (no active license) - - state "Master Authority (logar.eibl.tech)" as MasterAuth { - BootstrapGenerated: Generates Bootstrap .lic with Comm Cert & CSR Template - WS_Listening: WebSocket Hub listening on /ws/license - ValidateCSR: Validates Server CSR & Hardware Fingerprint - SignSubCA: Signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1) - } + [*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority + GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub + ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR + LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license + WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR + ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1) + SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota + QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active +``` - state "LOGAR Server Hub PKI Engine" as ServerPKI { - ImportLic: Admin imports .lic file via CLI or WebUI - GenerateLocalCSR: Server generates local Sub-CA Keypair & CSR - WS_Connecting: Opens WebSocket to wss://logar.eibl.tech/ws/license - InstallSubCA: Installs signed Sub-CA Cert & Master Root Trust - ActivateSeatQuota: Parses Seat OID & initializes SQLite seat quota - mTLS_Listening: Starts mTLS 1.3 Listener on Port 9443 - - HeartbeatSync: Sends periodic check_updates heartbeat over WebSocket - InFlightReload: Hot-reloads ssl.SSLContext without restarting service - - RevokedState: License revoked by Master Authority - DropConnections: Drops all incoming mTLS connections immediately - } +#### 3.2.2 Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload +Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior: - BootstrapGenerated --> ImportLic: Operator deploys .lic - ImportLic --> GenerateLocalCSR - GenerateLocalCSR --> WS_Connecting - WS_Connecting --> WS_Listening: action="enroll_or_sync" - WS_Listening --> ValidateCSR - ValidateCSR --> SignSubCA - SignSubCA --> InstallSubCA: status="success", ca_cert, master_root_ca - InstallSubCA --> ActivateSeatQuota - ActivateSeatQuota --> mTLS_Listening +```mermaid +stateDiagram-v2 + [*] --> ActiveListener: Server Hub actively servicing edge forwarders + ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat - mTLS_Listening --> HeartbeatSync: Scheduled or manual sync - HeartbeatSync --> WS_Listening: action="check_updates" - WS_Listening --> InFlightReload: Updated seat quota / renewed cert - InFlightReload --> mTLS_Listening + state WebSocketHeartbeat <> + WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota + WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack + WebSocketHeartbeat --> Revoked: Authority revoked license_id - WS_Listening --> RevokedState: Authority revokes license_id - RevokedState --> DropConnections + QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight + InFlightReload --> ActiveListener: Hot reload complete (Zero downtime) + SteadyState --> ActiveListener + Revoked --> DropConnections: Drops all incoming mTLS connections immediately ``` --- -### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine -Details how edge nodes discover their identity, bootstrap mTLS trust, and stream logs without local state: +### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycles + +#### 3.3.1 Machine Identity Discovery & Zero-Touch Enrollment +Details how forwarders automatically resolve host attribution and lease seats: ```mermaid stateDiagram-v2 - [*] --> HostDiscovery: Edge Client (Win_Client / Linux_Client) launches + [*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches + DiscoverIdentity --> QueryFQDN: socket.getfqdn() + QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf + ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution - state "Machine & Domain Identification" as Disc { - HostDiscovery --> QueryFQDN: socket.getfqdn() - QueryFQDN --> OS_DomainDiscovery: Windows AD env / Linux resolv.conf - OS_DomainDiscovery --> BuildClientID: Computes FQDN client_id & Site Attribution - } - - state "Zero-Touch Enrollment" as Enroll { - CheckLocalCert: Inspects certs/client.crt & certs/ca.crt - EnrollRequest: POST /api/client/enroll with enrollment_secret - VerifyQuota: Server verifies secret & active_seats < max_seats - StoreCerts: Saves client.crt, client.key, ca.crt - } - - state "Secure Streaming & Self-Healing" as Stream { - mTLS_Handshake: Mutual TLS 1.3 Handshake (Port 9443) - ValidateClientCN: Server checks client_id status in clients table - ActiveStreaming: Streams candidate log frames (INFO..CRITICAL) - CertExpiryCheck: Checks client cert validity before each run - ProactiveRenew: Renews cert when expiring in < 30 days - ReactiveSelfHeal: Re-enrolls automatically on CA rotation / handshake error - } - - BuildClientID --> CheckLocalCert - CheckLocalCert --> mTLS_Handshake: Valid local certificates exist - CheckLocalCert --> EnrollRequest: Certificates missing or expired - EnrollRequest --> VerifyQuota - VerifyQuota --> StoreCerts: 200 OK (Certificate Issued) - VerifyQuota --> [*]: 403 Forbidden (Seat Quota Full / Bad Secret) - StoreCerts --> mTLS_Handshake + BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt + state CheckCertificates <> + CheckCertificates --> ReadyToStream: Certificates valid & unexpired + CheckCertificates --> RequestEnrollment: Certificates missing or expired - mTLS_Handshake --> ValidateClientCN - ValidateClientCN --> ActiveStreaming: Client active in DB - ValidateClientCN --> [*]: Client seat revoked + RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret + state ValidateQuota <> + ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid) + ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret) - ActiveStreaming --> CertExpiryCheck - CertExpiryCheck --> ProactiveRenew: Expiry < 30 days - ProactiveRenew --> EnrollRequest - ActiveStreaming --> ReactiveSelfHeal: Handshake failure / CA changed - ReactiveSelfHeal --> EnrollRequest + StoreCerts --> ReadyToStream +``` + +#### 3.3.2 mTLS 1.3 Streaming, Proactive Renewal & Self-Healing +Details the secure streaming loop, proactive certificate lifecycle, and resilience against CA rotation: + +```mermaid +stateDiagram-v2 + [*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443) + mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table + + state CheckClientCN <> + CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota) + CheckClientCN --> DropClient: Client seat revoked in DB + + ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run + PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll) + ProactiveRenew --> mTLSHandshake + + ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls) + ReactiveHeal --> mTLSHandshake + DropClient --> [*] ``` --- -### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine -Details how log events are extracted, deduplicated, temporally tracked, and provisioned directly to Hermes AI: +### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning +Details how log events are extracted, deduplicated, temporally tracked across the 12-hour window, and provisioned to Hermes AI: ```mermaid stateDiagram-v2 - [*] --> EventCaptured: Windows Event Log / Linux journalctl event + [*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl + EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old + NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature}) + FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443) - state "Edge Filtering (Zero State)" as EdgeFilter { - EventCaptured --> NoiseStripping: Discards DEBUG, trace logs & events > 24h old - NoiseStripping --> FrameEncoding: Normalizes to JSON frame (severity, site, server, signature) - } - - state "Server Ingestion & 12h Temporal Window" as ServerIngest { - FrameIngested: mTLS Socket reads length-prefixed frame - ComputeFingerprint: Fingerprint = {site}:{server}:{signature} - EvaluateTemporalWindow: Compares (now - last_seen) against 12h window - - state EvaluateTemporalWindow <> - WithinWindow: (now - last_seen) <= 12 hours - WindowExpired: (now - last_seen) > 12 hours (First run in new window) - - UpdatePersistence: Increment consecutive_runs, update last_seen - ResetWindow: Set consecutive_runs = 1, update first_seen & last_seen - - UpsertSQLite: Upserts active_issues with status="VERIFIED" - } - - state "Centralized Provisioning & Hermes AI" as AIProvision { - API_Report: GET /api/hermes/report (Port 8443) - JSON_Payload: Delivers structured verified anomalies with temporal metadata - HermesThinking: Hermes AI executes multi-factor reasoning & autonomous decisions - } - - FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 - FrameIngested --> ComputeFingerprint - ComputeFingerprint --> EvaluateTemporalWindow - EvaluateTemporalWindow --> WithinWindow: Existing issue seen recently - EvaluateTemporalWindow --> WindowExpired: New issue or inactive > 12h - WithinWindow --> UpdatePersistence - WindowExpired --> ResetWindow - UpdatePersistence --> UpsertSQLite - ResetWindow --> UpsertSQLite + FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window + state EvaluateWindow <> + EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours + EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours - UpsertSQLite --> API_Report - API_Report --> JSON_Payload - JSON_Payload --> HermesThinking + WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen + WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps + + UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED" + ResetRunCount --> UpsertDB + + UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443) + ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions ``` ---