From 930a4cb34c3563d42f78e3d3d99abcedfb26b4f5 Mon Sep 17 00:00:00 2001 From: max Date: Sat, 26 Sep 2026 13:06:58 +0200 Subject: [PATCH] docs(wiki): add detailed technical documentation pages --- API-Reference.md | 167 +++++++++++++++++++++++++++++++ Architecture-and-Design.md | 74 ++++++++++++++ Deployment-and-Installation.md | 92 +++++++++++++++++ Home.md | 104 ++++++++++++++++++- PKI-and-Licensing.md | 76 ++++++++++++++ State-Machines-and-Lifecycles.md | 138 +++++++++++++++++++++++++ Testing-and-Releases.md | 83 +++++++++++++++ 7 files changed, 732 insertions(+), 2 deletions(-) create mode 100644 API-Reference.md create mode 100644 Architecture-and-Design.md create mode 100644 Deployment-and-Installation.md create mode 100644 PKI-and-Licensing.md create mode 100644 State-Machines-and-Lifecycles.md create mode 100644 Testing-and-Releases.md diff --git a/API-Reference.md b/API-Reference.md new file mode 100644 index 0000000..38d87ec --- /dev/null +++ b/API-Reference.md @@ -0,0 +1,167 @@ +# API Reference + +The LOGAR Server Hub exposes a comprehensive REST API on default port `8443` for edge enrollment, client seat management, Hermes AI log ingestion, and WebUI administration. + +--- + +## 1. Authentication & Security + +- **Administrator Endpoints**: Require either an authenticated session cookie (established via `/login` or `/api/auth/login`) or an `Authorization: Bearer ` header. +- **Hermes Endpoints**: Configurable for open access or Bearer token authentication via `Authorization: Bearer `. +- **Brute-Force Rate Limiting**: All authentication and enrollment endpoints enforce in-memory IP rate limiting. +- **Sanitization**: Dynamic WebUI rendering applies full `html.escape` sanitization against Stored XSS. + +--- + +## 2. Forwarder Enrollment API + +### `POST /api/client/enroll` +Enrolls a new edge client or proactively renews an existing client's certificate. + +**Request Body**: +```json +{ + "client_id": "db-prod-01.corp.internal", + "hostname": "db-prod-01", + "os": "linux", + "enrollment_secret": "SECURE_SHARED_SECRET" +} +``` + +**Success Response (`200 OK`)**: +```json +{ + "ca_cert": "-----BEGIN CERTIFICATE-----\nMIID...", + "client_cert": "-----BEGIN CERTIFICATE-----\nMIID...", + "client_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE..." +} +``` + +**Error Responses**: +- `403 Forbidden`: Invalid enrollment secret or seat quota exceeded. +- `429 Too Many Requests`: Rate limit exceeded. + +--- + +## 3. Client & Seat Management API + +### `GET /api/clients` +Returns seat quota status and registered client telemetry *(Requires Admin Auth)*. + +**Response (`200 OK`)**: +```json +{ + "active_seats": 2, + "max_seats": 25, + "clients": [ + { + "client_id": "db-prod-01.corp.internal", + "hostname": "db-prod-01", + "os_type": "linux", + "cert_fingerprint": "7D5B660B08...", + "status": "active", + "first_seen": "2026-09-26 10:00:00", + "last_seen": "2026-09-26 12:45:00", + "last_seen_relative": "15m ago" + } + ] +} +``` + +### `POST /api/client/revoke` +Revokes an edge client's seat license *(Requires Admin Auth)*. + +**Request Body**: +```json +{ + "client_id": "db-prod-01.corp.internal" +} +``` + +### `POST /api/client/approve` +Reactivates a revoked edge client *(Requires Admin Auth)*. + +**Request Body**: +```json +{ + "client_id": "db-prod-01.corp.internal" +} +``` + +### `POST /api/client/delete` +Permanently removes a forwarder record from SQLite *(Requires Admin Auth)*. + +**Request Body**: +```json +{ + "client_id": "db-prod-01.corp.internal" +} +``` + +### `GET /api/client/export` +Generates a downloadable `client_config.json` profile pre-configured for edge deployment *(Requires Admin Auth)*. + +--- + +## 4. Hermes AI & Log Telemetry API + +### `GET /api/hermes/report` +Returns all **verified anomalies** within the active 12-hour evaluation window with full temporal metadata. + +**Response (`200 OK`)**: +```json +[ + { + "fingerprint": "corp.internal:db-prod-01.corp.internal:PostgresConnectionLimitExceeded", + "site": "corp.internal", + "server": "db-prod-01.corp.internal", + "signature": "PostgresConnectionLimitExceeded", + "severity": "WARNING", + "message": "Maximum client connections reached (100/100)", + "os_type": "linux", + "first_seen": "2026-09-26T08:00:00+00:00", + "last_seen": "2026-09-26T12:00:00+00:00", + "consecutive_runs": 5, + "evaluation_window": "12h", + "verified": true, + "status": "VERIFIED" + } +] +``` + +### `GET /api/hermes/all` +Diagnostic endpoint listing all raw tracked issues in SQLite *(Requires Admin Auth)*. + +--- + +## 5. Licensing & Sub-CA Management API + +### `GET /api/ca/csr` +Returns the current Subordinate CA Certificate Signing Request in PEM format. + +### `POST /api/license/update` +Imports a signed Subordinate CA certificate (`ca.crt`) or activated `.lic` bundle *(Requires Admin Auth)*. + +### `POST /api/license/sync` +Triggers an immediate background synchronization over WebSocket with the Master Authority (`wss://logar.eibl.tech/ws/license`) *(Requires Admin Auth)*. + +--- + +## 6. Authentication & Health API + +### `POST /api/auth/login` +Authenticates administrator credentials and establishes session cookie. + +**Request Body**: +```json +{ + "username": "admin", + "password": "YOUR_ADMIN_PASSWORD" +} +``` + +### `POST /api/auth/logout` +Destroys the active administrator session cookie. + +### `GET /health` +Returns hub health, encryption fingerprint, active listener ports, and mTLS status. diff --git a/Architecture-and-Design.md b/Architecture-and-Design.md new file mode 100644 index 0000000..ac06f46 --- /dev/null +++ b/Architecture-and-Design.md @@ -0,0 +1,74 @@ +# Architecture and Design + +**LOGAR** is engineered as a zero-state, high-throughput log aggregation and provisioning system designed for enterprise server fleets comprising both Windows and Linux hosts. + +--- + +## 1. Core Design Philosophy + +### Centralized Log Aggregation & Zero-State Edge +Edge agents (`Win_Client` and `Linux_Client`) follow a strict **zero-state architecture**: +- **Zero Local Databases**: Edge forwarders do not maintain local SQLite databases or persistent files on disk. +- **Zero Disk Buffering**: Log events are filtered, formatted in memory, and immediately streamed over mTLS 1.3 sockets. +- **Fail-Safe Self-Healing**: In the event of network disconnection, connection reset, or CA rotation, clients re-establish connections or re-enroll automatically without state corruption or data loss. + +### Direct Hermes Intelligence +LOGAR decouples data aggregation and delivery from semantic log analysis: +- **No Arbitrary Edge Filtering**: All anomalous log events (`WARNING`, `ERROR`, `CRITICAL`) within the active 12-hour evaluation window are ingested and normalized. +- **Hermes AI Reasoning**: **Hermes AI** queries the centralized server hub (`GET /api/hermes/report`) and applies multi-factor reasoning, cross-system correlation, severity ranking, and autonomous remediation actions directly on verified telemetry. + +--- + +## 2. Component Subsystems + +### Edge Forwarders +1. **Windows Client (`Win_Client.exe` / `Win_Client.py`)**: + - Queries Windows Event Logs (`System`, `Application`, `Security`) via `win32evtlog`. + - Normalizes Windows event categories into standard severity levels (`INFO`, `WARNING`, `ERROR`, `CRITICAL`). + - Dynamic host/domain identification via Active Directory environment variables. +2. **Linux Client (`Linux_Client.bin` / `Linux_Client.py`)**: + - Ingests system logs via `systemd journalctl -p warning..emerg --output=json`. + - Strips noisy trace entries and parses unit attribution. + - Dynamic host/domain identification via `/etc/resolv.conf` and canonical DNS resolution. + +### Central Server Hub (`Server.py`) +1. **mTLS 1.3 Ingestion Engine (Port 9443)**: + - Enforces bidirectional TLS 1.3 with mutual certificate verification (`ssl.CERT_REQUIRED`). + - Validates client certificates against the Subordinate CA in real time. + - Checks active seat status on every handshake and drops revoked edge nodes. +2. **Temporal Persistence Engine**: + - Tracks active anomalies in SQLite (`active_issues`). + - Formats unique fingerprints: `{site_name}:{server}:{signature}`. + - Maintains consecutive run counts within a rolling 12-hour evaluation window. +3. **Log Provisioning REST API (Port 8443)**: + - Exposes clean, verified log telemetry to Hermes AI via `/api/hermes/report`. + - Provides administrator WebUI and seat management endpoints. + +--- + +## 3. Data Flow + +```mermaid +sequenceDiagram + autonumber + participant Edge as Edge Forwarder (Win/Linux) + participant Hub as LOGAR Server Hub + participant DB as logar_state.db + participant Hermes as Hermes AI Engine + + Edge->>Hub: TLS 1.3 Handshake (Client Cert :9443) + Hub->>DB: Check client_id status (active?) + DB-->>Hub: Approved (Seat valid) + Hub-->>Edge: TLS Handshake OK + + loop Streaming Logs + Edge->>Hub: Stream JSON Log Frame ({site, server, signature, msg}) + Hub->>DB: Upsert active_issues (Update run_count & last_seen) + end + + Hermes->>Hub: GET /api/hermes/report (:8443) + Hub->>DB: SELECT verified issues within 12h window + DB-->>Hub: Return active issue records + Hub-->>Hermes: 200 OK [JSON Telemetry Stream] + Hermes->>Hermes: Deep Reasoning & Automated Remediation +``` diff --git a/Deployment-and-Installation.md b/Deployment-and-Installation.md new file mode 100644 index 0000000..a777d9f --- /dev/null +++ b/Deployment-and-Installation.md @@ -0,0 +1,92 @@ +# Deployment and Installation + +LOGAR is designed for rapid, automated deployment across both Linux and Windows environments with native service integration. + +--- + +## 1. Central Server Hub Deployment + +### Prerequisites +- Python 3.10+ (or standalone Linux binary / Docker container) +- Inbound access on ports `9443` (mTLS Ingestion) and `8443` (WebUI & API) + +### Automated Linux Server Installation +Run the automated systemd installer script on your server: + +```bash +sudo ./compilation/install_linux_server.sh +``` + +This script: +1. Deploys the application files to `/opt/logar-server`. +2. Sets up Python dependencies and virtual environment. +3. Generates the initial PKI keys and configuration. +4. Installs and starts `logar-server.service` with `LimitNOFILE=65536`. + +### Windows Server Installation +1. Compile or download `LOGAR-Server-Setup.exe`. +2. Execute the installer. It will install the server binary to `{autopf}\LOGAR` and register the `LOGAR_Server` Windows service via NSSM. + +--- + +## 2. Linux Client Deployment + +### Automated Non-Interactive Installer +Deploy edge forwarders effortlessly across your Linux fleet: + +```bash +sudo ./compilation/install_linux_client.sh "http://:8443" "" +``` + +The installer performs the following: +1. Queries `/etc/machine-id` and canonical FQDN. +2. Contacts `/api/client/enroll` on the server hub using the shared enrollment secret. +3. Receives and stores `ca.crt`, `client.crt`, and `client.key` in `/etc/logar/certs` with strict permissions (`chmod 700`). +4. Generates `/etc/logar/client_config.json`. +5. Installs and activates `logar-client.service` systemd unit. + +### Systemd Service Management +```bash +# Check status +systemctl status logar-client.service + +# View forwarder logs +journalctl -u logar-client.service -f +``` + +--- + +## 3. Windows Client Deployment + +### Inno Setup Automated Installer +1. Download `LOGAR-Client-Setup.exe` from the latest release. +2. Run the installer or deploy silently via Group Policy / SCCM: + ```cmd + LOGAR-Client-Setup.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART + ``` +3. Place `client_config.json` (exported from Server WebUI) into `C:\Program Files\LOGAR\client_config.json`. +4. The service automatically enrolls on first start and streams Windows Event Logs continuously. + +### Windows Service Management +- **Service Name**: `LOGAR_Client` +- **Log Output**: `C:\ProgramData\LOGAR\client.log` +- Managed via standard Windows Services MMC (`services.msc`) or `sc.exe`: + ```cmd + sc query LOGAR_Client + sc stop LOGAR_Client + sc start LOGAR_Client + ``` + +--- + +## 4. Dynamic Machine & Domain Identification + +Client configurations intentionally contain **no hardcoded machine name or site name**. Both forwarders dynamically identify their host and domain at runtime via `get_machine_identifier()`: +1. **Fully Qualified Domain Name (FQDN)**: Checked via `socket.getfqdn()`. +2. **OS-Specific Domain Discovery**: + - **Windows**: Checks Active Directory environment variables (`USERDNSDOMAIN`, `USERDOMAIN`). + - **Linux**: Parses `/etc/resolv.conf` `domain` and `search` directives. +3. **Reverse DNS Lookup**: Resolves canonical hostname via `socket.gethostbyaddr`. +4. **Fallback**: Local hostname `socket.gethostname()`. + +The server automatically infers site attribution from domain qualifiers (e.g. `node01.corp.internal` $\rightarrow$ site `corp.internal`). diff --git a/Home.md b/Home.md index fcc721f..a27b3f4 100644 --- a/Home.md +++ b/Home.md @@ -1,3 +1,103 @@ -# LOGAR Wiki +# LOGAR Documentation Wiki -Welcome to the LOGAR documentation wiki. +Welcome to the official technical documentation wiki for **LOGAR: Fleet Log Aggregation & Centralized Provisioning System**. + +--- + +## Navigation & Wiki Index + +1. **[Architecture and Design](Architecture-and-Design)** + - Core philosophy & data provisioning model + - End-to-end system architecture + - Zero-state edge forwarders vs centralized hub + - Hermes AI reasoning integration + +2. **[State Machines and Lifecycles](State-Machines-and-Lifecycles)** + - End-to-end high-level system state machine + - Sub-CA bootstrap & auto-enrollment lifecycle + - Live WebSocket synchronization & in-flight hot reload + - Forwarder identity discovery & zero-touch enrollment + - mTLS 1.3 streaming, proactive renewal & self-healing + - 12-hour temporal evaluation window + +3. **[PKI and Fleet Licensing](PKI-and-Licensing)** + - Hierarchical PKI architecture & custom X.509 OIDs + - Dynamic seat quotas and licensing enforcement + - Bootstrap `.lic` package handling + - In-flight certificate watchdog & hot SSLContext reloading + - WebUI Sub-CA & Licensing dashboard + +4. **[API Reference](API-Reference)** + - REST API specifications & JSON request/response schemas + - Forwarder enrollment (`/api/client/enroll`) + - Seat management & client telemetry (`/api/clients`) + - Hermes AI log streaming (`/api/hermes/report`) + - Administrator session authentication & health endpoints + +5. **[Deployment and Installation](Deployment-and-Installation)** + - Central server hub setup & configuration + - Linux client deployment (`install_linux_client.sh`) + - Windows client & server setup (Inno Setup & NSSM) + - Dynamic machine and domain identification + +6. **[Testing and Releases](Testing-and-Releases)** + - Automated unit test suite (35 tests) & end-to-end pipeline test + - Continuous Integration (`ci.yml`) + - Automated cross-compilation release workflow (`release.yml`) + - Local binary packaging & Gitea asset publisher + +--- + +## High-Level Architecture Overview + +```mermaid +graph TB + subgraph Authority [Master License Authority - logar.eibl.tech] + M_CA[Master Root CA
RSA-4096 / SHA-256] + AUTH_WS[WebSocket Licensing Hub
/ws/license] + AUTH_DB[(Authority DB
Tenants, Users, 2FA, Seats)] + AUTH_UI[Authority WebUI
User Mgmt & Fleet Quotas] + AUTH_WS --- AUTH_DB + AUTH_UI --- AUTH_DB + end + + subgraph ServerHub [LOGAR Server Hub - Central Aggregation & Provisioning] + SUB_CA[Subordinate CA
Signed with Seat OID 1.3.6.1.4.1.59999.1.1] + SEAT_MGR[Dynamic Seat Manager
Enforces Seat Quota in SQLite] + MTLS_SRV[mTLS 1.3 Ingestion Server
Port 9443 - ssl.CERT_REQUIRED] + TEMP_ENG[12h Temporal Persistence Engine
Correlates First/Last Seen & Run Counts] + HUB_DB[(logar_state.db
active_issues, clients, config)] + PROV_API[Log Provisioning & Hermes API
Port 8443 - GET /api/hermes/report] + + SUB_CA --> SEAT_MGR + SEAT_MGR --> MTLS_SRV + MTLS_SRV --> TEMP_ENG + TEMP_ENG --> HUB_DB + HUB_DB --> PROV_API + end + + subgraph EdgeNodes [Heterogeneous Edge Forwarders] + WIN[Win_Client.exe / Win_Client.py
Windows Event Log Ingestion] + LIN[Linux_Client.bin / Linux_Client.py
systemd journalctl -p warning] + end + + subgraph Downstream [Hermes AI Reasoning Layer] + HERMES[Hermes AI Engine
Autonomous Analysis & Remediation] + end + + %% PKI & Licensing Relationships + M_CA -->|Signs Sub-CA Cert via /ws/license| SUB_CA + AUTH_WS <==>|Bidirectional Sync & Live Seat Expansion| SUB_CA + + %% Forwarder Enrollment & mTLS Transport + WIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA + LIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA + SUB_CA -->|2. Issues Client Cert & Root Trust| WIN + SUB_CA -->|2. Issues Client Cert & Root Trust| LIN + + WIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV + LIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV + + %% Log Provisioning & Hermes Consumption + PROV_API ==>|Centralized Log Delivery| HERMES +``` diff --git a/PKI-and-Licensing.md b/PKI-and-Licensing.md new file mode 100644 index 0000000..68b610d --- /dev/null +++ b/PKI-and-Licensing.md @@ -0,0 +1,76 @@ +# PKI and Fleet Licensing + +LOGAR implements a hierarchical, cryptographically enforced dynamic PKI architecture that seamlessly manages machine authentication, mutual TLS transport security, and enterprise seat licensing. + +--- + +## 1. Cryptographic Hierarchy + +```mermaid +graph TD + ROOT[Master Root CA
logar.eibl.tech
RSA-4096 / SHA-256] + SUB[Subordinate CA
LOGAR Server Hub
RSA-4096 / SHA-256] + SRV[Server TLS Certificate
Port 9443 mTLS Listener] + CLI_WIN[Client Cert: Win_Client
CN: win-dc-01.corp.internal] + CLI_LIN[Client Cert: Linux_Client
CN: db-node-01.corp.internal] + + ROOT -->|Signs with custom OID_LOGAR_SEATS| SUB + SUB -->|Issues on first run| SRV + SUB -->|Issues on /api/client/enroll| CLI_WIN + SUB -->|Issues on /api/client/enroll| CLI_LIN +``` + +--- + +## 2. Custom Enterprise X.509 OIDs + +LOGAR embeds license metadata directly into the Subordinate CA certificate X.509 extensions under enterprise OID prefix `1.3.6.1.4.1.59999.1`: + +| OID | Constant Name | Description | Example Value | +| :--- | :--- | :--- | :--- | +| `1.3.6.1.4.1.59999.1.1` | `OID_LOGAR_SEATS` | Maximum approved forwarder seat quota | `100` | +| `1.3.6.1.4.1.59999.1.2` | `OID_LOGAR_LICENSE_ID` | Unique enterprise license identifier | `LIC-A1B2C3D4E5F6` | +| `1.3.6.1.4.1.59999.1.3` | `OID_LOGAR_ISSUER_URL` | WebSocket authority endpoint | `wss://logar.eibl.tech/ws/license` | + +When the LOGAR server loads its certificate chain, it parses these custom extensions using `cryptography.x509` and dynamically enforces the extracted seat limit in its SQLite database (`config.max_seats`). + +--- + +## 3. Bootstrap `.lic` Packages + +To initialize a new server hub without manual key transfers, the Master Authority issues a **Bootstrap `.lic` Package** (OpenPGP-encrypted JSON envelope): +- **Master Root CA Certificate**: Installs authority trust anchor. +- **Dedicated Communication Keypair**: Mutual TLS credentials used strictly for communicating with the Master Authority. +- **Sub-CA CSR Template**: Pre-configured Subject Distinguished Names and assigned enterprise tenant. +- **Endpoint Coordinates**: Pre-configured WebSocket URI (`wss://logar.eibl.tech/ws/license`). + +### Importing a `.lic` Package +- **CLI**: `python src/Server.py --import-lic ` +- **WebUI**: Navigate to **📜 Sub-CA & Licensing** tab $\rightarrow$ Drag-and-drop or paste `.lic` file. + +Upon import, the server hub connects to the Master Authority over WebSocket, submits its local Sub-CA CSR, verifies hardware fingerprints, receives the signed certificate, and immediately activates its mTLS listeners. + +--- + +## 4. In-Flight Watchdog & Hot SSLContext Reloading + +The server hub runs a continuous background watchdog coroutine (`cert_validity_watchdog`, running every 12 hours): +1. Checks the expiration date of `ca.crt` and `server.crt`. +2. If remaining validity is less than 30 days: + - Initiates automated certificate renewal with the Master License Authority. + - Replaces certificate files atomically on disk. + - Dynamically reloads the active `ssl.SSLContext` in-memory. +3. Active mTLS client connections and log streams remain completely uninterrupted (Zero Downtime). + +--- + +## 5. WebUI Sub-CA & Licensing Dashboard + +The Server Hub WebUI includes a dedicated **📜 Sub-CA & Licensing** management tab: +- **Authority Connection Badge**: Real-time status indicator of Master Authority WebSocket connection. +- **Hardware Fingerprint**: Machine binding fingerprint displayed for license validation. +- **Seat Progress Gauge**: Visual representation of active seats leased vs. maximum quota. +- **Interactive Actions**: + - **Export .lic Request**: Downloads a signed request bundle for air-gapped environments. + - **View & Copy CA CSR**: Displays the current Subordinate CA CSR for external signing. + - **Sync & Renew via WebSocket**: Triggers immediate upstream synchronization and seat expansion. diff --git a/State-Machines-and-Lifecycles.md b/State-Machines-and-Lifecycles.md new file mode 100644 index 0000000..895150d --- /dev/null +++ b/State-Machines-and-Lifecycles.md @@ -0,0 +1,138 @@ +# State Machines and Lifecycles + +This document provides formal state machine diagrams detailing the operational lifecycles of all subsystems across the LOGAR architecture. + +--- + +## 1. High-Level System State Machine + +Macroscopic overview of the complete end-to-end system lifecycle from initial PKI bootstrapping to AI provisioning: + +```mermaid +stateDiagram-v2 + [*] --> 1_PKI_Licensing: Master Authority generates Bootstrap .lic Package + 1_PKI_Licensing --> 2_Hub_Ready: Server Hub imports .lic & signs Sub-CA over WebSocket + 2_Hub_Ready --> 3_Edge_Enrollment: Edge Forwarders auto-enroll & lease licensed seats + 3_Edge_Enrollment --> 4_mTLS_Streaming: Forwarders stream framed logs over mTLS 1.3 (:9443) + 4_mTLS_Streaming --> 5_Temporal_Aggregation: Server Hub normalizes & correlates logs in 12h window + 5_Temporal_Aggregation --> 6_AI_Provisioning: Logs provisioned to Hermes AI via GET /api/hermes/report + 6_AI_Provisioning --> 4_mTLS_Streaming: Continuous operational loop +``` + +--- + +## 2. Master License Authority & Sub-CA Auto-Enrollment + +Details how a freshly initialized LOGAR Server Hub imports initial credentials, connects to the Master License Authority (`logar.eibl.tech`) via WebSocket, and acquires its signed Sub-CA certificate: + +```mermaid +stateDiagram-v2 + [*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority + GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub + ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR + LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license + WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR + ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1) + SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota + QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active +``` + +--- + +## 3. Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload + +Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior without dropping client streams: + +```mermaid +stateDiagram-v2 + [*] --> ActiveListener: Server Hub actively servicing edge forwarders + ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat + + state WebSocketHeartbeat <> + WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota + WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack + WebSocketHeartbeat --> Revoked: Authority revoked license_id + + QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight + InFlightReload --> ActiveListener: Hot reload complete (Zero downtime) + SteadyState --> ActiveListener + Revoked --> DropConnections: Drops all incoming mTLS connections immediately +``` + +--- + +## 4. Forwarder Identity Discovery & Zero-Touch Enrollment + +Details dynamic machine identity resolution and initial certificate enrollment on Windows and Linux edge forwarders: + +```mermaid +stateDiagram-v2 + [*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches + DiscoverIdentity --> QueryFQDN: socket.getfqdn() + QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf + ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution + + BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt + state CheckCertificates <> + CheckCertificates --> ReadyToStream: Certificates valid & unexpired + CheckCertificates --> RequestEnrollment: Certificates missing or expired + + RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret + state ValidateQuota <> + ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid) + ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret) + + StoreCerts --> ReadyToStream +``` + +--- + +## 5. mTLS 1.3 Streaming, Proactive Renewal & Reactive Self-Healing + +Details the secure streaming loop, proactive certificate lifecycle checks, and automated recovery against CA rotation: + +```mermaid +stateDiagram-v2 + [*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443) + mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table + + state CheckClientCN <> + CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota) + CheckClientCN --> DropClient: Client seat revoked in DB + + ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run + PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll) + ProactiveRenew --> mTLSHandshake + + ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls) + ReactiveHeal --> mTLSHandshake + DropClient --> [*] +``` + +--- + +## 6. 12-Hour Temporal Window & AI Provisioning + +Details log frame extraction, deduplication, temporal correlation across the 12-hour evaluation window, and delivery to Hermes AI: + +```mermaid +stateDiagram-v2 + [*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl + EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old + NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature}) + FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443) + + FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window + state EvaluateWindow <> + EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours + EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours + + WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen + WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps + + UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED" + ResetRunCount --> UpsertDB + + UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443) + ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions +``` diff --git a/Testing-and-Releases.md b/Testing-and-Releases.md new file mode 100644 index 0000000..e090a64 --- /dev/null +++ b/Testing-and-Releases.md @@ -0,0 +1,83 @@ +# Testing and Releases + +This document describes the automated test infrastructure, CI/CD pipeline, and cross-compilation release distribution workflow for LOGAR. + +--- + +## 1. Automated Test Suite + +LOGAR includes comprehensive unit and integration test suites: + +### Running Unit Tests +```bash +python -m unittest discover -s tests -v +``` + +The test suite executes **35 unit tests** verifying: +- Dynamic Root CA generation and server TLS certificate issuance. +- Dynamic client certificate issuance with CN and authority key extensions. +- Enrollment secret authentication, seat limits, and certificate revocation. +- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload. +- Subordinate CA CSR generation, WebSocket enrollment, and dynamic seat quota updates. +- Administrator session authentication, PBKDF2 hashing, and Hermes Bearer token lifecycle. +- Windows & Linux event log collection, deduplication, and mTLS certificate verification. +- 12-hour evaluation window and direct Hermes anomaly verification. + +### End-to-End Pipeline Integration Test +```bash +# 1. Initialize test configuration +python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json + +# 2. Launch server in background +python src/Server.py & + +# 3. Run integration test +python tests/test_pipeline.py +``` +Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output. + +--- + +## 2. Continuous Integration (`ci.yml`) + +Every push or pull request to `main` triggers `.gitea/workflows/ci.yml`: +- Checks Python code syntax. +- Runs all 35 unit tests in an isolated environment. +- Executes end-to-end mTLS pipeline integration tests. + +--- + +## 3. Releases & Binary Cross-Compilation (`release.yml`) + +Releases are published automatically upon pushing a git tag: + +```bash +git tag v3.0.1 +git push origin v3.0.1 +``` + +### Linux-Based Cross-Compilation Pipeline +The workflow runs on a single `ubuntu-latest` runner executing `python compilation/package_dist.py --target all`: +1. **Linux Binaries**: Standalone native ELF executables (`Linux_Client.bin`, `Server.bin`). +2. **Windows Executables & Bundles**: Cross-compiled standalone binaries (`Win_Client.exe`, `Server.exe`) and `.zip` distribution packages. +3. **SHA256 Checksums**: Generated automatically as `sha256sums.txt`. +4. **Gitea Release Publication**: `upload_release.py` uploads all compiled assets and changelogs directly to the Gitea release page. + +### Manual Workflow Dispatch +Releases can also be triggered on demand via the Gitea UI: +1. Navigate to **Actions** $\rightarrow$ **Release Binaries & Installers** (`release.yml`). +2. Click **Run workflow**, specify the release tag (e.g. `v3.0.1`), and execute. + +--- + +## 4. Local Packaging & Asset Publishing + +To build binaries and upload releases directly from a developer workstation: + +```powershell +# 1. Build all target binaries locally +python compilation/package_dist.py --target all + +# 2. Upload assets and release notes to Gitea +python compilation/upload_release.py --tag v3.0.1 --token --skip-build +```