Split state machines into modular, focused subsections for PKI enrollment, sync, forwarder discovery, and mTLS streaming
CI Test Suite / Run Component Tests & Pipeline Verification (push) Failing after 19s
CI Test Suite / Run Component Tests & Pipeline Verification (push) Failing after 19s
This commit is contained in:
@@ -10,9 +10,13 @@
|
|||||||
2. [End-to-End System Architecture](#end-to-end-system-architecture)
|
2. [End-to-End System Architecture](#end-to-end-system-architecture)
|
||||||
3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles)
|
3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles)
|
||||||
- 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine)
|
- 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine)
|
||||||
- 3.2 [Master License Authority & WebSocket Enrollment State Machine](#32-master-license-authority--websocket-enrollment-state-machine)
|
- 3.2 [Master License Authority & PKI Lifecycles](#32-master-license-authority--pki-lifecycles)
|
||||||
- 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine](#33-edge-forwarder-auto-enrollment--mtls-lifecycle-state-machine)
|
- 3.2.1 [Bootstrap & Sub-CA Auto-Enrollment State Machine](#321-bootstrap--sub-ca-auto-enrollment-state-machine)
|
||||||
- 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning-state-machine)
|
- 3.2.2 [Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload](#322-live-websocket-heartbeat-seat-expansion--in-flight-reload)
|
||||||
|
- 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycles](#33-edge-forwarder-auto-enrollment--mtls-lifecycles)
|
||||||
|
- 3.3.1 [Machine Identity Discovery & Zero-Touch Enrollment](#331-machine-identity-discovery--zero-touch-enrollment)
|
||||||
|
- 3.3.2 [mTLS 1.3 Streaming, Proactive Renewal & Self-Healing](#332-mtls-13-streaming-proactive-renewal--self-healing)
|
||||||
|
- 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning)
|
||||||
4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing)
|
4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing)
|
||||||
5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment)
|
5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment)
|
||||||
6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning)
|
6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning)
|
||||||
@@ -119,152 +123,115 @@ stateDiagram-v2
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3.2 Master License Authority & WebSocket Enrollment State Machine
|
### 3.2 Master License Authority & PKI Lifecycles
|
||||||
Details the bidirectional cryptographic enrollment, seat leasing, and hot-reload lifecycle between `logar.eibl.tech` and the LOGAR Server Hub:
|
|
||||||
|
#### 3.2.1 Bootstrap & Sub-CA Auto-Enrollment State Machine
|
||||||
|
Details how a newly deployed LOGAR Server Hub imports initial credentials, connects to `logar.eibl.tech` over WebSocket, and secures its signed Sub-CA certificate:
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
stateDiagram-v2
|
stateDiagram-v2
|
||||||
[*] --> Idle_Unlicensed: Server Hub initial launch (no active license)
|
[*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority
|
||||||
|
GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub
|
||||||
state "Master Authority (logar.eibl.tech)" as MasterAuth {
|
ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR
|
||||||
BootstrapGenerated: Generates Bootstrap .lic with Comm Cert & CSR Template
|
LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
|
||||||
WS_Listening: WebSocket Hub listening on /ws/license
|
WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR
|
||||||
ValidateCSR: Validates Server CSR & Hardware Fingerprint
|
ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
|
||||||
SignSubCA: Signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
|
SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota
|
||||||
}
|
QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active
|
||||||
|
```
|
||||||
|
|
||||||
state "LOGAR Server Hub PKI Engine" as ServerPKI {
|
#### 3.2.2 Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload
|
||||||
ImportLic: Admin imports .lic file via CLI or WebUI
|
Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior:
|
||||||
GenerateLocalCSR: Server generates local Sub-CA Keypair & CSR
|
|
||||||
WS_Connecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
|
|
||||||
InstallSubCA: Installs signed Sub-CA Cert & Master Root Trust
|
|
||||||
ActivateSeatQuota: Parses Seat OID & initializes SQLite seat quota
|
|
||||||
mTLS_Listening: Starts mTLS 1.3 Listener on Port 9443
|
|
||||||
|
|
||||||
HeartbeatSync: Sends periodic check_updates heartbeat over WebSocket
|
|
||||||
InFlightReload: Hot-reloads ssl.SSLContext without restarting service
|
|
||||||
|
|
||||||
RevokedState: License revoked by Master Authority
|
|
||||||
DropConnections: Drops all incoming mTLS connections immediately
|
|
||||||
}
|
|
||||||
|
|
||||||
BootstrapGenerated --> ImportLic: Operator deploys .lic
|
```mermaid
|
||||||
ImportLic --> GenerateLocalCSR
|
stateDiagram-v2
|
||||||
GenerateLocalCSR --> WS_Connecting
|
[*] --> ActiveListener: Server Hub actively servicing edge forwarders
|
||||||
WS_Connecting --> WS_Listening: action="enroll_or_sync"
|
ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat
|
||||||
WS_Listening --> ValidateCSR
|
|
||||||
ValidateCSR --> SignSubCA
|
|
||||||
SignSubCA --> InstallSubCA: status="success", ca_cert, master_root_ca
|
|
||||||
InstallSubCA --> ActivateSeatQuota
|
|
||||||
ActivateSeatQuota --> mTLS_Listening
|
|
||||||
|
|
||||||
mTLS_Listening --> HeartbeatSync: Scheduled or manual sync
|
state WebSocketHeartbeat <<choice>>
|
||||||
HeartbeatSync --> WS_Listening: action="check_updates"
|
WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota
|
||||||
WS_Listening --> InFlightReload: Updated seat quota / renewed cert
|
WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack
|
||||||
InFlightReload --> mTLS_Listening
|
WebSocketHeartbeat --> Revoked: Authority revoked license_id
|
||||||
|
|
||||||
WS_Listening --> RevokedState: Authority revokes license_id
|
QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight
|
||||||
RevokedState --> DropConnections
|
InFlightReload --> ActiveListener: Hot reload complete (Zero downtime)
|
||||||
|
SteadyState --> ActiveListener
|
||||||
|
Revoked --> DropConnections: Drops all incoming mTLS connections immediately
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycle State Machine
|
### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycles
|
||||||
Details how edge nodes discover their identity, bootstrap mTLS trust, and stream logs without local state:
|
|
||||||
|
#### 3.3.1 Machine Identity Discovery & Zero-Touch Enrollment
|
||||||
|
Details how forwarders automatically resolve host attribution and lease seats:
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
stateDiagram-v2
|
stateDiagram-v2
|
||||||
[*] --> HostDiscovery: Edge Client (Win_Client / Linux_Client) launches
|
[*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches
|
||||||
|
DiscoverIdentity --> QueryFQDN: socket.getfqdn()
|
||||||
|
QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf
|
||||||
|
ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution
|
||||||
|
|
||||||
state "Machine & Domain Identification" as Disc {
|
BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt
|
||||||
HostDiscovery --> QueryFQDN: socket.getfqdn()
|
state CheckCertificates <<choice>>
|
||||||
QueryFQDN --> OS_DomainDiscovery: Windows AD env / Linux resolv.conf
|
CheckCertificates --> ReadyToStream: Certificates valid & unexpired
|
||||||
OS_DomainDiscovery --> BuildClientID: Computes FQDN client_id & Site Attribution
|
CheckCertificates --> RequestEnrollment: Certificates missing or expired
|
||||||
}
|
|
||||||
|
|
||||||
state "Zero-Touch Enrollment" as Enroll {
|
|
||||||
CheckLocalCert: Inspects certs/client.crt & certs/ca.crt
|
|
||||||
EnrollRequest: POST /api/client/enroll with enrollment_secret
|
|
||||||
VerifyQuota: Server verifies secret & active_seats < max_seats
|
|
||||||
StoreCerts: Saves client.crt, client.key, ca.crt
|
|
||||||
}
|
|
||||||
|
|
||||||
state "Secure Streaming & Self-Healing" as Stream {
|
|
||||||
mTLS_Handshake: Mutual TLS 1.3 Handshake (Port 9443)
|
|
||||||
ValidateClientCN: Server checks client_id status in clients table
|
|
||||||
ActiveStreaming: Streams candidate log frames (INFO..CRITICAL)
|
|
||||||
CertExpiryCheck: Checks client cert validity before each run
|
|
||||||
ProactiveRenew: Renews cert when expiring in < 30 days
|
|
||||||
ReactiveSelfHeal: Re-enrolls automatically on CA rotation / handshake error
|
|
||||||
}
|
|
||||||
|
|
||||||
BuildClientID --> CheckLocalCert
|
|
||||||
CheckLocalCert --> mTLS_Handshake: Valid local certificates exist
|
|
||||||
CheckLocalCert --> EnrollRequest: Certificates missing or expired
|
|
||||||
EnrollRequest --> VerifyQuota
|
|
||||||
VerifyQuota --> StoreCerts: 200 OK (Certificate Issued)
|
|
||||||
VerifyQuota --> [*]: 403 Forbidden (Seat Quota Full / Bad Secret)
|
|
||||||
StoreCerts --> mTLS_Handshake
|
|
||||||
|
|
||||||
mTLS_Handshake --> ValidateClientCN
|
RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret
|
||||||
ValidateClientCN --> ActiveStreaming: Client active in DB
|
state ValidateQuota <<choice>>
|
||||||
ValidateClientCN --> [*]: Client seat revoked
|
ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid)
|
||||||
|
ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret)
|
||||||
|
|
||||||
ActiveStreaming --> CertExpiryCheck
|
StoreCerts --> ReadyToStream
|
||||||
CertExpiryCheck --> ProactiveRenew: Expiry < 30 days
|
```
|
||||||
ProactiveRenew --> EnrollRequest
|
|
||||||
ActiveStreaming --> ReactiveSelfHeal: Handshake failure / CA changed
|
#### 3.3.2 mTLS 1.3 Streaming, Proactive Renewal & Self-Healing
|
||||||
ReactiveSelfHeal --> EnrollRequest
|
Details the secure streaming loop, proactive certificate lifecycle, and resilience against CA rotation:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443)
|
||||||
|
mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table
|
||||||
|
|
||||||
|
state CheckClientCN <<choice>>
|
||||||
|
CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota)
|
||||||
|
CheckClientCN --> DropClient: Client seat revoked in DB
|
||||||
|
|
||||||
|
ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run
|
||||||
|
PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll)
|
||||||
|
ProactiveRenew --> mTLSHandshake
|
||||||
|
|
||||||
|
ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls)
|
||||||
|
ReactiveHeal --> mTLSHandshake
|
||||||
|
DropClient --> [*]
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning State Machine
|
### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning
|
||||||
Details how log events are extracted, deduplicated, temporally tracked, and provisioned directly to Hermes AI:
|
Details how log events are extracted, deduplicated, temporally tracked across the 12-hour window, and provisioned to Hermes AI:
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
stateDiagram-v2
|
stateDiagram-v2
|
||||||
[*] --> EventCaptured: Windows Event Log / Linux journalctl event
|
[*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl
|
||||||
|
EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old
|
||||||
|
NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature})
|
||||||
|
FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443)
|
||||||
|
|
||||||
state "Edge Filtering (Zero State)" as EdgeFilter {
|
FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window
|
||||||
EventCaptured --> NoiseStripping: Discards DEBUG, trace logs & events > 24h old
|
state EvaluateWindow <<choice>>
|
||||||
NoiseStripping --> FrameEncoding: Normalizes to JSON frame (severity, site, server, signature)
|
EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours
|
||||||
}
|
EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours
|
||||||
|
|
||||||
state "Server Ingestion & 12h Temporal Window" as ServerIngest {
|
|
||||||
FrameIngested: mTLS Socket reads length-prefixed frame
|
|
||||||
ComputeFingerprint: Fingerprint = {site}:{server}:{signature}
|
|
||||||
EvaluateTemporalWindow: Compares (now - last_seen) against 12h window
|
|
||||||
|
|
||||||
state EvaluateTemporalWindow <<choice>>
|
|
||||||
WithinWindow: (now - last_seen) <= 12 hours
|
|
||||||
WindowExpired: (now - last_seen) > 12 hours (First run in new window)
|
|
||||||
|
|
||||||
UpdatePersistence: Increment consecutive_runs, update last_seen
|
|
||||||
ResetWindow: Set consecutive_runs = 1, update first_seen & last_seen
|
|
||||||
|
|
||||||
UpsertSQLite: Upserts active_issues with status="VERIFIED"
|
|
||||||
}
|
|
||||||
|
|
||||||
state "Centralized Provisioning & Hermes AI" as AIProvision {
|
|
||||||
API_Report: GET /api/hermes/report (Port 8443)
|
|
||||||
JSON_Payload: Delivers structured verified anomalies with temporal metadata
|
|
||||||
HermesThinking: Hermes AI executes multi-factor reasoning & autonomous decisions
|
|
||||||
}
|
|
||||||
|
|
||||||
FrameEncoding --> FrameIngested: Streamed over mTLS 1.3
|
|
||||||
FrameIngested --> ComputeFingerprint
|
|
||||||
ComputeFingerprint --> EvaluateTemporalWindow
|
|
||||||
EvaluateTemporalWindow --> WithinWindow: Existing issue seen recently
|
|
||||||
EvaluateTemporalWindow --> WindowExpired: New issue or inactive > 12h
|
|
||||||
WithinWindow --> UpdatePersistence
|
|
||||||
WindowExpired --> ResetWindow
|
|
||||||
UpdatePersistence --> UpsertSQLite
|
|
||||||
ResetWindow --> UpsertSQLite
|
|
||||||
|
|
||||||
UpsertSQLite --> API_Report
|
WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen
|
||||||
API_Report --> JSON_Payload
|
WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps
|
||||||
JSON_Payload --> HermesThinking
|
|
||||||
|
UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED"
|
||||||
|
ResetRunCount --> UpsertDB
|
||||||
|
|
||||||
|
UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443)
|
||||||
|
ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
Reference in New Issue
Block a user