Split state machines into modular, focused subsections for PKI enrollment, sync, forwarder discovery, and mTLS streaming

2026-09-25 22:12:37 +02:00
parent 5fe74325ec
commit 4c8db1ec94
1 changed files with 90 additions and 123 deletions
@@ -10,9 +10,13 @@
 2. [End-to-End System Architecture](#end-to-end-system-architecture)
 3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles)
   - 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine)
-   - 3.2 [Master License Authority & WebSocket Enrollment State Machine](#32-master-license-authority--websocket-enrollment-state-machine)
-   - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine](#33-edge-forwarder-auto-enrollment--mtls-lifecycle-state-machine)
-   - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning-state-machine)
+   - 3.2 [Master License Authority & PKI Lifecycles](#32-master-license-authority--pki-lifecycles)
+     - 3.2.1 [Bootstrap & Sub-CA Auto-Enrollment State Machine](#321-bootstrap--sub-ca-auto-enrollment-state-machine)
+     - 3.2.2 [Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload](#322-live-websocket-heartbeat-seat-expansion--in-flight-reload)
+   - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycles](#33-edge-forwarder-auto-enrollment--mtls-lifecycles)
+     - 3.3.1 [Machine Identity Discovery & Zero-Touch Enrollment](#331-machine-identity-discovery--zero-touch-enrollment)
+     - 3.3.2 [mTLS 1.3 Streaming, Proactive Renewal & Self-Healing](#332-mtls-13-streaming-proactive-renewal--self-healing)
+   - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning)
 4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing)
 5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment)
 6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning)
@@ -119,152 +123,115 @@ stateDiagram-v2

 ---

-### 3.2 Master License Authority & WebSocket Enrollment State Machine
-Details the bidirectional cryptographic enrollment, seat leasing, and hot-reload lifecycle between `logar.eibl.tech` and the LOGAR Server Hub:
+### 3.2 Master License Authority & PKI Lifecycles
+
+#### 3.2.1 Bootstrap & Sub-CA Auto-Enrollment State Machine
+Details how a newly deployed LOGAR Server Hub imports initial credentials, connects to `logar.eibl.tech` over WebSocket, and secures its signed Sub-CA certificate:

 ```mermaid
 stateDiagram-v2
-    [*] --> Idle_Unlicensed: Server Hub initial launch (no active license)
-    
-    state "Master Authority (logar.eibl.tech)" as MasterAuth {
-        BootstrapGenerated: Generates Bootstrap .lic with Comm Cert & CSR Template
-        WS_Listening: WebSocket Hub listening on /ws/license
-        ValidateCSR: Validates Server CSR & Hardware Fingerprint
-        SignSubCA: Signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
-    }
+    [*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority
+    GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub
+    ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR
+    LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
+    WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR
+    ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
+    SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota
+    QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active
+```

-    state "LOGAR Server Hub PKI Engine" as ServerPKI {
-        ImportLic: Admin imports .lic file via CLI or WebUI
-        GenerateLocalCSR: Server generates local Sub-CA Keypair & CSR
-        WS_Connecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
-        InstallSubCA: Installs signed Sub-CA Cert & Master Root Trust
-        ActivateSeatQuota: Parses Seat OID & initializes SQLite seat quota
-        mTLS_Listening: Starts mTLS 1.3 Listener on Port 9443
-        
-        HeartbeatSync: Sends periodic check_updates heartbeat over WebSocket
-        InFlightReload: Hot-reloads ssl.SSLContext without restarting service
-        
-        RevokedState: License revoked by Master Authority
-        DropConnections: Drops all incoming mTLS connections immediately
-    }
+#### 3.2.2 Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload
+Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior:

-    BootstrapGenerated --> ImportLic: Operator deploys .lic
-    ImportLic --> GenerateLocalCSR
-    GenerateLocalCSR --> WS_Connecting
-    WS_Connecting --> WS_Listening: action="enroll_or_sync"
-    WS_Listening --> ValidateCSR
-    ValidateCSR --> SignSubCA
-    SignSubCA --> InstallSubCA: status="success", ca_cert, master_root_ca
-    InstallSubCA --> ActivateSeatQuota
-    ActivateSeatQuota --> mTLS_Listening
+```mermaid
+stateDiagram-v2
+    [*] --> ActiveListener: Server Hub actively servicing edge forwarders
+    ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat
    
-    mTLS_Listening --> HeartbeatSync: Scheduled or manual sync
-    HeartbeatSync --> WS_Listening: action="check_updates"
-    WS_Listening --> InFlightReload: Updated seat quota / renewed cert
-    InFlightReload --> mTLS_Listening
+    state WebSocketHeartbeat <<choice>>
+    WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota
+    WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack
+    WebSocketHeartbeat --> Revoked: Authority revoked license_id
    
-    WS_Listening --> RevokedState: Authority revokes license_id
-    RevokedState --> DropConnections
+    QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight
+    InFlightReload --> ActiveListener: Hot reload complete (Zero downtime)
+    SteadyState --> ActiveListener
+    Revoked --> DropConnections: Drops all incoming mTLS connections immediately
 ```

 ---

-### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine
-Details how edge nodes discover their identity, bootstrap mTLS trust, and stream logs without local state:
+### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycles
+
+#### 3.3.1 Machine Identity Discovery & Zero-Touch Enrollment
+Details how forwarders automatically resolve host attribution and lease seats:

 ```mermaid
 stateDiagram-v2
-    [*] --> HostDiscovery: Edge Client (Win_Client / Linux_Client) launches
+    [*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches
+    DiscoverIdentity --> QueryFQDN: socket.getfqdn()
+    QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf
+    ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution
    
-    state "Machine & Domain Identification" as Disc {
-        HostDiscovery --> QueryFQDN: socket.getfqdn()
-        QueryFQDN --> OS_DomainDiscovery: Windows AD env / Linux resolv.conf
-        OS_DomainDiscovery --> BuildClientID: Computes FQDN client_id & Site Attribution
-    }
-
-    state "Zero-Touch Enrollment" as Enroll {
-        CheckLocalCert: Inspects certs/client.crt & certs/ca.crt
-        EnrollRequest: POST /api/client/enroll with enrollment_secret
-        VerifyQuota: Server verifies secret & active_seats < max_seats
-        StoreCerts: Saves client.crt, client.key, ca.crt
-    }
-
-    state "Secure Streaming & Self-Healing" as Stream {
-        mTLS_Handshake: Mutual TLS 1.3 Handshake (Port 9443)
-        ValidateClientCN: Server checks client_id status in clients table
-        ActiveStreaming: Streams candidate log frames (INFO..CRITICAL)
-        CertExpiryCheck: Checks client cert validity before each run
-        ProactiveRenew: Renews cert when expiring in < 30 days
-        ReactiveSelfHeal: Re-enrolls automatically on CA rotation / handshake error
-    }
-
-    BuildClientID --> CheckLocalCert
-    CheckLocalCert --> mTLS_Handshake: Valid local certificates exist
-    CheckLocalCert --> EnrollRequest: Certificates missing or expired
-    EnrollRequest --> VerifyQuota
-    VerifyQuota --> StoreCerts: 200 OK (Certificate Issued)
-    VerifyQuota --> [*]: 403 Forbidden (Seat Quota Full / Bad Secret)
-    StoreCerts --> mTLS_Handshake
+    BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt
+    state CheckCertificates <<choice>>
+    CheckCertificates --> ReadyToStream: Certificates valid & unexpired
+    CheckCertificates --> RequestEnrollment: Certificates missing or expired
    
-    mTLS_Handshake --> ValidateClientCN
-    ValidateClientCN --> ActiveStreaming: Client active in DB
-    ValidateClientCN --> [*]: Client seat revoked
+    RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret
+    state ValidateQuota <<choice>>
+    ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid)
+    ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret)
    
-    ActiveStreaming --> CertExpiryCheck
-    CertExpiryCheck --> ProactiveRenew: Expiry < 30 days
-    ProactiveRenew --> EnrollRequest
-    ActiveStreaming --> ReactiveSelfHeal: Handshake failure / CA changed
-    ReactiveSelfHeal --> EnrollRequest
+    StoreCerts --> ReadyToStream
+```
+
+#### 3.3.2 mTLS 1.3 Streaming, Proactive Renewal & Self-Healing
+Details the secure streaming loop, proactive certificate lifecycle, and resilience against CA rotation:
+
+```mermaid
+stateDiagram-v2
+    [*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443)
+    mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table
+    
+    state CheckClientCN <<choice>>
+    CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota)
+    CheckClientCN --> DropClient: Client seat revoked in DB
+    
+    ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run
+    PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll)
+    ProactiveRenew --> mTLSHandshake
+    
+    ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls)
+    ReactiveHeal --> mTLSHandshake
+    DropClient --> [*]
 ```

 ---

-### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine
-Details how log events are extracted, deduplicated, temporally tracked, and provisioned directly to Hermes AI:
+### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning
+Details how log events are extracted, deduplicated, temporally tracked across the 12-hour window, and provisioned to Hermes AI:

 ```mermaid
 stateDiagram-v2
-    [*] --> EventCaptured: Windows Event Log / Linux journalctl event
+    [*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl
+    EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old
+    NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature})
+    FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443)
    
-    state "Edge Filtering (Zero State)" as EdgeFilter {
-        EventCaptured --> NoiseStripping: Discards DEBUG, trace logs & events > 24h old
-        NoiseStripping --> FrameEncoding: Normalizes to JSON frame (severity, site, server, signature)
-    }
-
-    state "Server Ingestion & 12h Temporal Window" as ServerIngest {
-        FrameIngested: mTLS Socket reads length-prefixed frame
-        ComputeFingerprint: Fingerprint = {site}:{server}:{signature}
-        EvaluateTemporalWindow: Compares (now - last_seen) against 12h window
-        
-        state EvaluateTemporalWindow <<choice>>
-        WithinWindow: (now - last_seen) <= 12 hours
-        WindowExpired: (now - last_seen) > 12 hours (First run in new window)
-        
-        UpdatePersistence: Increment consecutive_runs, update last_seen
-        ResetWindow: Set consecutive_runs = 1, update first_seen & last_seen
-        
-        UpsertSQLite: Upserts active_issues with status="VERIFIED"
-    }
-
-    state "Centralized Provisioning & Hermes AI" as AIProvision {
-        API_Report: GET /api/hermes/report (Port 8443)
-        JSON_Payload: Delivers structured verified anomalies with temporal metadata
-        HermesThinking: Hermes AI executes multi-factor reasoning & autonomous decisions
-    }
-
-    FrameEncoding --> FrameIngested: Streamed over mTLS 1.3
-    FrameIngested --> ComputeFingerprint
-    ComputeFingerprint --> EvaluateTemporalWindow
-    EvaluateTemporalWindow --> WithinWindow: Existing issue seen recently
-    EvaluateTemporalWindow --> WindowExpired: New issue or inactive > 12h
-    WithinWindow --> UpdatePersistence
-    WindowExpired --> ResetWindow
-    UpdatePersistence --> UpsertSQLite
-    ResetWindow --> UpsertSQLite
+    FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window
+    state EvaluateWindow <<choice>>
+    EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours
+    EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours
    
-    UpsertSQLite --> API_Report
-    API_Report --> JSON_Payload
-    JSON_Payload --> HermesThinking
+    WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen
+    WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps
+    
+    UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED"
+    ResetRunCount --> UpsertDB
+    
+    UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443)
+    ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions
 ```

 ---