diff --git a/README.md b/README.md index 769ae35..916458a 100644 --- a/README.md +++ b/README.md @@ -1,566 +1,161 @@ # LOGAR: Fleet Log Aggregation & Centralized Provisioning System ### *Secure Log Aggregation, PKI Fleet Licensing & Centralized Delivery for Autonomous AI Analysis* +[![Version](https://img.shields.io/badge/version-3.0.1-blue.svg)](RELEASE_NOTES.md) +[![Tests](https://img.shields.io/badge/tests-35%20passed-success.svg)](tests/) +[![Security](https://img.shields.io/badge/mTLS-TLS%201.3-green.svg)](https://gitea.eibl.tech/me0nline/LOGAR/wiki/PKI-and-Licensing) + **LOGAR** is an enterprise-grade log aggregation, normalization, and centralized provisioning system built for heterogeneous server fleets (Windows & Linux). It pairs ultra-lightweight, zero-state edge forwarders with a secure centralized server hub that enforces mutual TLS 1.3 (**mTLS**) authentication, dynamic PKI fleet licensing with cryptographic seat leasing, 12-hour temporal persistence tracking, and direct anomaly pass-throughβ€”delivering clean, structured, verified log telemetry directly to **Hermes AI** for autonomous deep reasoning, contextual root-cause analysis, and operational decision-making. -> πŸ“š **Detailed Technical Documentation & Specifications**: Complete architecture deep-dives, state machine lifecycles, PKI specifications, and full API references have been published to the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)**. +> πŸ“š **Comprehensive Technical Documentation**: Full architectural specifications, state machines, PKI X.509 OID definitions, and complete REST API references are maintained on the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)**. --- -## Table of Contents -1. [Core Purpose & Philosophy (Log Aggregation & Provisioning)](#core-purpose--philosophy) -2. [End-to-End System Architecture](#end-to-end-system-architecture) -3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles) - - 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine) - - 3.2 [Master License Authority & PKI Lifecycles](#32-master-license-authority--pki-lifecycles) - - 3.2.1 [Bootstrap & Sub-CA Auto-Enrollment State Machine](#321-bootstrap--sub-ca-auto-enrollment-state-machine) - - 3.2.2 [Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload](#322-live-websocket-heartbeat-seat-expansion--in-flight-reload) - - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycles](#33-edge-forwarder-auto-enrollment--mtls-lifecycles) - - 3.3.1 [Machine Identity Discovery & Zero-Touch Enrollment](#331-machine-identity-discovery--zero-touch-enrollment) - - 3.3.2 [mTLS 1.3 Streaming, Proactive Renewal & Self-Healing](#332-mtls-13-streaming-proactive-renewal--self-healing) - - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning) -4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing) -5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment) -6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning) -7. [Agentic Hermes & Client Management API](#agentic-hermes--client-management-api) -8. [Dynamic Machine & Domain Identification](#dynamic-machine--domain-identification) -9. [Automated Service Installers (Linux & Windows)](#automated-service-installers-linux--windows) -10. [Repository & Shippables Structure](#repository--shippables-structure) -11. [Getting Started & Deployment](#getting-started--deployment) -12. [Running Tests](#running-tests) -13. [Automated Releases via Gitea Actions](#automated-releases-via-gitea-actions) +## Key Capabilities + +- **Zero-State Edge Forwarders**: Windows (`Win_Client`) and Linux (`Linux_Client`) agents maintain zero local databases, zero cache files, and minimal memory footprint. +- **Mutual TLS 1.3 (`:9443`)**: Enforces bidirectional cryptographic identity verification (`ssl.CERT_REQUIRED`) for all streaming log frames. +- **Hierarchical PKI & Seat Licensing**: Subordinate CAs automatically enroll and sync seat quotas with the Master Authority (`logar.eibl.tech`) over WebSockets. +- **12-Hour Temporal Persistence**: Evaluates anomalies over a rolling 12-hour temporal window, tracking occurrence velocity and first/last seen metrics. +- **Direct Hermes AI Provisioning (`:8443`)**: Exposes verified anomalous telemetry directly to Hermes AI via `/api/hermes/report` without heuristic bottlenecks. +- **WebUI Dashboard**: Authenticated, 2-tab management console for live anomaly telemetry, client seat leasing, and Sub-CA lifecycle management. --- -## Core Purpose & Philosophy - -### 1. Centralized Log Aggregation & Provisioning -The foundational objective of LOGAR is reliable, secure collection and structured delivery of system events: -- **Heterogeneous Fleet Aggregation**: Ingests Windows Event Logs (System, Application, Security) and Linux `systemd journalctl` events into a unified, normalized JSON data format. -- **Centralized Provisioning**: Serves as the authoritative clearinghouse and real-time streaming endpoint (`/api/hermes/report`), providing structured logs with temporal metadata to external consumers and automated AI agents. -- **Zero Local Edge Footprint**: Forwarders maintain zero local databases, zero cache files, and zero disk overhead. If a network disruption occurs, agents automatically re-establish mTLS streaming without state corruption. - -### 2. Direct Hermes Intelligence (No Heuristic Bottlenecks) -LOGAR strictly separates **data aggregation/provisioning** from **semantic reasoning**: -- **Removal of Arbitrary Heuristics**: The legacy 4-run throttling rule was eliminated. All ingested candidate logs (`INFO`, `WARNING`, `ERROR`, `CRITICAL`) within the active 12-hour temporal evaluation window are immediately verified and provisioned. -- **Hermes Autonomous Thinking**: **Hermes AI** performs all multi-factor reasoning, cross-server correlation, urgency prioritization, and automated remediation decisions directly on the provisioned log stream. - -### 3. Hierarchical Dynamic PKI & Fleet Licensing -- **Authoritative Master PKI**: Subordinate server hubs do not require manual certificate generation or static keys. The **Master License Authority** (`logar.eibl.tech`) issues cryptographically signed Sub-CA certificates with custom X.509 seat quotas (`1.3.6.1.4.1.59999.1.1`). -- **Dynamic Seat Leasing**: Server hubs enforce local seat allocations in real-time, instantly revoking or admitting edge nodes while synchronizing with the central authority over bidirectional WebSockets. - ---- - -## End-to-End System Architecture +## System Architecture ```mermaid graph TB subgraph Authority [Master License Authority - logar.eibl.tech] - M_CA[Master Root CA
RSA-4096 / SHA-256] - AUTH_WS[WebSocket Licensing Hub
/ws/license] - AUTH_DB[(Authority DB
Tenants, Users, 2FA, Seats)] - AUTH_UI[Authority WebUI
User Mgmt & Fleet Quotas] - AUTH_WS --- AUTH_DB - AUTH_UI --- AUTH_DB + M_CA[Master Root CA] + AUTH_WS[WebSocket Licensing Hub] end - subgraph ServerHub [LOGAR Server Hub - Central Aggregation & Provisioning] - SUB_CA[Subordinate CA
Signed with Seat OID 1.3.6.1.4.1.59999.1.1] - SEAT_MGR[Dynamic Seat Manager
Enforces Seat Quota in SQLite] - MTLS_SRV[mTLS 1.3 Ingestion Server
Port 9443 - ssl.CERT_REQUIRED] - TEMP_ENG[12h Temporal Persistence Engine
Correlates First/Last Seen & Run Counts] - HUB_DB[(logar_state.db
active_issues, clients, config)] - PROV_API[Log Provisioning & Hermes API
Port 8443 - GET /api/hermes/report] + subgraph ServerHub [LOGAR Server Hub] + SUB_CA[Subordinate CA] + SEAT_MGR[Dynamic Seat Manager] + MTLS_SRV[mTLS 1.3 Ingestion Server
Port 9443] + TEMP_ENG[12h Temporal Persistence Engine] + PROV_API[Hermes Provisioning & WebUI API
Port 8443] SUB_CA --> SEAT_MGR SEAT_MGR --> MTLS_SRV MTLS_SRV --> TEMP_ENG - TEMP_ENG --> HUB_DB - HUB_DB --> PROV_API + TEMP_ENG --> PROV_API end subgraph EdgeNodes [Heterogeneous Edge Forwarders] - WIN[Win_Client.exe / Win_Client.py
Windows Event Log Ingestion] - LIN[Linux_Client.bin / Linux_Client.py
systemd journalctl -p warning] + WIN[Win_Client.exe
Windows Event Logs] + LIN[Linux_Client.bin
systemd journalctl] end - subgraph Downstream [Hermes AI Reasoning Layer] - HERMES[Hermes AI Engine
Autonomous Analysis & Remediation] + subgraph Downstream [Hermes AI Layer] + HERMES[Hermes AI Engine] end - %% PKI & Licensing Relationships - M_CA -->|Signs Sub-CA Cert via /ws/license| SUB_CA - AUTH_WS <==>|Bidirectional Sync & Live Seat Expansion| SUB_CA + M_CA -->|Signs Sub-CA Cert| SUB_CA + AUTH_WS <==>|Live WebSocket Sync| SUB_CA - %% Forwarder Enrollment & mTLS Transport - WIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA - LIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA - SUB_CA -->|2. Issues Client Cert & Root Trust| WIN - SUB_CA -->|2. Issues Client Cert & Root Trust| LIN - - WIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV - LIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV - - %% Log Provisioning & Hermes Consumption - PROV_API ==>|Centralized Log Delivery| HERMES + WIN -->|1. Auto-Enrollment| SUB_CA + LIN -->|1. Auto-Enrollment| SUB_CA + WIN ==>|2. mTLS 1.3 Stream :9443| MTLS_SRV + LIN ==>|2. mTLS 1.3 Stream :9443| MTLS_SRV + PROV_API ==>|3. GET /api/hermes/report| HERMES ``` --- -## System State Machines & Subsystem Lifecycles - -### 3.1 High-Level System State Machine -A simplified macroscopic view of the end-to-end LOGAR lifecycle: - -```mermaid -stateDiagram-v2 - [*] --> 1_PKI_Licensing: Master Authority generates Bootstrap .lic Package - 1_PKI_Licensing --> 2_Hub_Ready: Server Hub imports .lic & signs Sub-CA over WebSocket - 2_Hub_Ready --> 3_Edge_Enrollment: Edge Forwarders auto-enroll & lease licensed seats - 3_Edge_Enrollment --> 4_mTLS_Streaming: Forwarders stream framed logs over mTLS 1.3 (:9443) - 4_mTLS_Streaming --> 5_Temporal_Aggregation: Server Hub normalizes & correlates logs in 12h window - 5_Temporal_Aggregation --> 6_AI_Provisioning: Logs provisioned to Hermes AI via GET /api/hermes/report - 6_AI_Provisioning --> 4_mTLS_Streaming: Continuous operational loop -``` - ---- - -### 3.2 Master License Authority & PKI Lifecycles - -#### 3.2.1 Bootstrap & Sub-CA Auto-Enrollment State Machine -Details how a newly deployed LOGAR Server Hub imports initial credentials, connects to `logar.eibl.tech` over WebSocket, and secures its signed Sub-CA certificate: - -```mermaid -stateDiagram-v2 - [*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority - GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub - ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR - LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license - WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR - ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1) - SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota - QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active -``` - -#### 3.2.2 Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload -Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior: - -```mermaid -stateDiagram-v2 - [*] --> ActiveListener: Server Hub actively servicing edge forwarders - ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat - - state WebSocketHeartbeat <> - WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota - WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack - WebSocketHeartbeat --> Revoked: Authority revoked license_id - - QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight - InFlightReload --> ActiveListener: Hot reload complete (Zero downtime) - SteadyState --> ActiveListener - Revoked --> DropConnections: Drops all incoming mTLS connections immediately -``` - ---- - -### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycles - -#### 3.3.1 Machine Identity Discovery & Zero-Touch Enrollment -Details how forwarders automatically resolve host attribution and lease seats: - -```mermaid -stateDiagram-v2 - [*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches - DiscoverIdentity --> QueryFQDN: socket.getfqdn() - QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf - ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution - - BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt - state CheckCertificates <> - CheckCertificates --> ReadyToStream: Certificates valid & unexpired - CheckCertificates --> RequestEnrollment: Certificates missing or expired - - RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret - state ValidateQuota <> - ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid) - ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret) - - StoreCerts --> ReadyToStream -``` - -#### 3.3.2 mTLS 1.3 Streaming, Proactive Renewal & Self-Healing -Details the secure streaming loop, proactive certificate lifecycle, and resilience against CA rotation: - -```mermaid -stateDiagram-v2 - [*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443) - mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table - - state CheckClientCN <> - CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota) - CheckClientCN --> DropClient: Client seat revoked in DB - - ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run - PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll) - ProactiveRenew --> mTLSHandshake - - ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls) - ReactiveHeal --> mTLSHandshake - DropClient --> [*] -``` - ---- - -### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning -Details how log events are extracted, deduplicated, temporally tracked across the 12-hour window, and provisioned to Hermes AI: - -```mermaid -stateDiagram-v2 - [*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl - EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old - NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature}) - FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443) - - FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window - state EvaluateWindow <> - EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours - EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours - - WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen - WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps - - UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED" - ResetRunCount --> UpsertDB - - UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443) - ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions -``` - ---- - -## Master License Authority & PKI Fleet Licensing - -### 1. TLS 1.3 Mutual Authentication (mTLS) -- **Port 9443**: Ingestion occurs exclusively over TLS 1.3 sockets with `ssl.CERT_REQUIRED`. -- Both the hub and edge clients verify each other's certificates: - - Client verifies server certificate against `ca.crt`. - - Server verifies client certificate against the Subordinate/Root CA. -- **Client CN Identification & Revocation**: In the TLS handshake, the server extracts the `commonName` attribute (`client_id`), validates that the client is marked `active` in the `clients` table, updates the `last_seen` timestamp, and drops unregistered or revoked certificates immediately. - -### 2. Master License Authority (`logar.eibl.tech`), Multi-Tenancy & Subordinate CA -- **Master Root Authority**: Standalone PKI repository & submodule ([`LOGAR-LICENSING/`](file:///u:/max/Desktop/LOGAR/LOGAR-LICENSING/README.md), upstream at [`https://gitea.eibl.tech/me0nline/LOGAR-LICENSING.git`](https://gitea.eibl.tech/me0nline/LOGAR-LICENSING.git)) deployed at `logar.eibl.tech`. -- **Multi-Tenant Fleet Isolation & User Management**: - - Supports multiple isolated **Tenants** (e.g. `DEFAULT`, `CORP_A`, `PRODUCTION_FLEET`) with live tenant filtering. - - Built-in user management with PBKDF2 password hashing, RBAC (Administrators & Operators), and self-service password updates. -- **Custom Enterprise X.509 OIDs**: - - `OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)`: Embeds the exact licensed seat quota directly inside the signed Subordinate CA certificate. - - `OID_LOGAR_LICENSE_ID (1.3.6.1.4.1.59999.1.2)`: Uniquely tracks enterprise license entitlement. - - `OID_LOGAR_ISSUER_URL (1.3.6.1.4.1.59999.1.3)`: Embedded endpoint coordinate (`https://logar.eibl.tech`). -- **Bootstrap `.lic` Packages & Instant WebSocket Auto-Enrollment**: - - The License Authority (`logar.eibl.tech`) generates initial **Bootstrap `.lic` Packages** containing Master Root CA, dedicated communication credentials (`communication_cert` + `communication_key`), CSR templates, assigned tenant, and WebSocket coordinates bound directly to the authority server host. - - **1-Click Import & Live Signing**: Importing the `.lic` file on the client server (`python src/Server.py --import-lic ` or via WebUI) connects directly via **WebSocket**, submits the server Sub-CA CSR, and immediately installs the officially signed Sub-CA license certificate. -- **Self-Updating Client Servers & Live WebSocket Sync**: - - Client servers update themselves against the master authority (`wss://logar.eibl.tech/ws/license`) over WebSocket (`python src/Server.py --sync-license` or via WebUI / automated watchdog). - - Seat quotas can be increased with each renewal or updated on-demand in the License Authority WebUI. - - The client server automatically receives the updated seat quota in its signed certificate, updates its SQLite database and runtime configuration, and reloads its in-flight `ssl.SSLContext` without dropping client connections or restarting services. -- **Automated Container Pipeline & Ephemeral NPM Pass-Through**: - - The license authority repository includes a dedicated CI/CD workflow ([`.gitea/workflows/build-container.yml`](file:///u:/max/Desktop/LOGAR/LOGAR-LICENSING/.gitea/workflows/build-container.yml)) publishing to `gitea.eibl.tech/me0nline/logar-licensing:latest` (authenticated via `giteatoken` secret). - - Works seamlessly with ephemeral Nginx Proxy Manager (NPM) by enabling the standard **Websockets Support** toggle for `logar.eibl.tech`. - - Admin WebUI is protected by authenticated login with user and tenant administration. - -### 3. Integrated Fleet Licensing & WebUI Dashboard -- **WebUI URL**: Accessible at `http://:8443/webui` (or `/`). Protected by authenticated session login (`/login`). -- **Tab 1: πŸ“‹ Active Anomalies & Telemetry**: - - Real-time tabular stream of verified anomalies within the active 12-hour evaluation window. - - Interactive search filter and Hermes AI status metrics. - - **Visual Seat Quota Meter**: Real-time progress bar displaying active forwarders vs. allocated seat limit. - - **Seat Management & Revocation Table**: Lists all connected Windows and Linux edge forwarders with 1-click **Revoke Seat**, **Reactivate Seat**, and **Export Client Configuration** actions. -- **Tab 2: πŸ“œ Sub-CA & Licensing**: - - Dedicated PKI and Subordinate CA lifecycle management interface. - - Displays Master Authority WebSocket status (`logar.eibl.tech`), hardware fingerprint, and embedded seat quota. - - **Licensing Actions**: - - 1-click **Export .lic Request** file download. - - Interactive **View & Copy CA CSR** modal. - - Drag-and-drop or paste **Import License (.lic / ca.crt)** modal. - - **Sync & Renew via WebSocket** button for immediate upstream synchronization and seat expansion. - -### 4. Cloud-Side In-Flight Certificate Watchdog -- **Continuous Hub PKI Watchdog**: - - The server hub runs a continuous background watchdog coroutine (`cert_validity_watchdog`, running every 12 hours). - - Automatically renews Subordinate CA certificates over mTLS with `logar.eibl.tech` and Server TLS certificates in-flight, dynamically reloading the active `ssl.SSLContext`. -- **Client Proactive Check & Reactive Self-Healing**: - - **Proactive Renewal**: Edge clients inspect `client.crt` before every run cycle. If expiring within 30 days, they contact `/api/client/enroll` to renew. - - **Reactive Self-Healing**: If the server CA rotates or a handshake error occurs, edge clients catch the exception, re-bootstrap enrollment against the hub, and re-establish streaming cleanly. - ---- - -## Cloud-Side Temporal Persistence & Direct Hermes Reasoning - -Incoming candidate logs are tracked in SQLite table `active_issues`: -- **Issue Fingerprint**: Formatted as `{site_name}:{server}:{signature}`. -- **12-Hour Evaluation Window**: - - The hub compares `(now - last_seen)`. - - If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to `run_count = 1`. -- **Direct Verification & Temporal Telemetry**: - - Ingested anomalies are immediately promoted to `VERIFIED` and made available to Hermes with complete temporal metadata (first seen, last seen, consecutive run counts). - - Hermes performs all contextual analysis, pattern recognition, and alerting logic. - ---- - -## Agentic Hermes & Client Management API - -The server hub exposes a management and reporting REST API (default port `8443`): - -### `POST /api/client/enroll` -Client enrollment endpoint: -- **Request**: - ```json - { - "client_id": "web-worker-01.corp.internal", - "hostname": "web-worker-01", - "os": "linux", - "enrollment_secret": "" - } - ``` -- **Response**: - ```json - { - "ca_cert": "-----BEGIN CERTIFICATE-----\n...", - "client_cert": "-----BEGIN CERTIFICATE-----\n...", - "client_key": "-----BEGIN RSA PRIVATE KEY-----\n..." - } - ``` - -### `GET /api/clients` -Returns seat quota status and registered client telemetry *(Requires Administrator session cookie or Bearer auth)*: -```json -{ - "active_seats": 2, - "max_seats": 10, - "clients": [ - { - "client_id": "web-worker-01.corp.internal", - "hostname": "web-worker-01", - "os_type": "linux", - "cert_fingerprint": "7D5B660B...", - "status": "active", - "first_seen": "2026-09-04 18:00:00", - "last_seen": "2026-09-04 19:15:00" - } - ] -} -``` - -### `GET /api/hermes/report` -Returns all **verified anomalies** within the active 12-hour evaluation window with full temporal metadata for direct Hermes reasoning: -```json -[ - { - "fingerprint": "corp.internal:web-worker-01.corp.internal:PostgresPoolExhausted", - "site": "corp.internal", - "server": "web-worker-01.corp.internal", - "signature": "PostgresPoolExhausted", - "severity": "WARNING", - "message": "Connection pool saturated (>95%)", - "os_type": "linux", - "first_seen": "2026-09-04T07:00:00+00:00", - "last_seen": "2026-09-04T19:00:00+00:00", - "consecutive_runs": 4, - "evaluation_window": "12h", - "verified": true, - "status": "VERIFIED" - } -] -``` - -### `GET /api/hermes/all` -Diagnostic endpoint listing all tracked anomalies in SQLite *(Requires Administrator auth)*. - -### `POST /api/auth/login` & `POST /api/auth/logout` -Session authentication endpoints for Administrator WebUI access. - -### `GET /health` -Returns hub health, encryption fingerprint, listener ports, and mTLS status. - ---- - -## Dynamic Machine & Domain Identification - -Client configurations intentionally contain **no hardcoded machine name or site name**. Both forwarders dynamically identify their host and domain at runtime via `get_machine_identifier()`: -1. **Fully Qualified Domain Name (FQDN)**: Checked via `socket.getfqdn()`. -2. **OS-Specific Domain Discovery**: - - **Windows**: Checks Active Directory environment variables (`USERDNSDOMAIN`, `USERDOMAIN`). - - **Linux**: Parses `/etc/resolv.conf` `domain` and `search` directives. -3. **Reverse DNS Lookup**: Resolves canonical hostname via `socket.gethostbyaddr`. -4. **Fallback**: Local hostname `socket.gethostname()`. - -The server automatically infers site attribution from domain qualifiers (e.g. `node01.corp.internal` $\rightarrow$ site `corp.internal`). - ---- - -## Automated Service Installers (Linux & Windows) - -LOGAR provides production-grade installation scripts and installer builders for automated service deployment: - -### 1. Linux Service Installers -- **Client Installer (`compilation/install_linux_client.sh`)**: - - Non-interactive script deploying to `/opt/logar-client`. - - Automatically queries `/etc/machine-id` and enrolls with the hub via `curl`. - - Installs and enables `logar-client.service` systemd unit. - ```bash - sudo ./compilation/install_linux_client.sh "http://hub.example.com:8443" "" - ``` -- **Server Installer (`compilation/install_linux_server.sh`)**: - - Deploys server to `/opt/logar-server`. - - Configures logging and installs `logar-server.service` with `LimitNOFILE=65536`. - ```bash - sudo ./compilation/install_linux_server.sh - ``` - -### 2. Windows Inno Setup Installers -- Built using **Inno Setup 6** and bundled with **NSSM** (`compilation/nssm.exe`): - - **Client Setup (`compilation/installer_client.iss`)**: Compiles `LOGAR-Client-Setup.exe`. Installs `Win_Client.exe` into `{autopf}\LOGAR`, sets up `LOGAR_Client` service via NSSM with stdout/stderr redirection to `{commonappdata}\LOGAR\client.log`, and starts the service. Clean uninstallation stops and removes the service. - - **Server Setup (`compilation/installer_server.iss`)**: Compiles `LOGAR-Server-Setup.exe`. Installs `Server.exe` and sets up `LOGAR_Server` Windows service via NSSM. - ---- - -## Repository & Shippables Structure - -``` -LOGAR/ -β”œβ”€β”€ .gitea/ -β”‚ └── workflows/ -β”‚ β”œβ”€β”€ ci.yml # CI pipeline: syntax, 35 unit tests & mTLS pipeline test -β”‚ └── release.yml # Consolidated release workflow (Cross-compilation on Linux runner) -β”œβ”€β”€ compilation/ # Packaging, installers, and release automation -β”‚ β”œβ”€β”€ install_linux_client.sh # Automated Linux client systemd installation script -β”‚ β”œβ”€β”€ install_linux_server.sh # Automated Linux server systemd installation script -β”‚ β”œβ”€β”€ installer_client.iss # Inno Setup Windows Client installer script -β”‚ β”œβ”€β”€ installer_server.iss # Inno Setup Windows Server installer script -β”‚ β”œβ”€β”€ nssm.exe # Official 64-bit NSSM service manager binary -β”‚ β”œβ”€β”€ package_dist.py # Standalone binary cross-compiler & packager -β”‚ β”œβ”€β”€ requirements.txt # Unified project dependencies -β”‚ └── upload_release.py # Gitea REST API release asset publisher -β”œβ”€β”€ src/ # Core application source modules -β”‚ β”œβ”€β”€ __init__.py -β”‚ β”œβ”€β”€ Server.py # Central mTLS server, temporal engine, and Hermes REST API -β”‚ β”œβ”€β”€ server_enrollment.py # Dynamic PKI, Root CA, and client certificate generator -β”‚ β”œβ”€β”€ Win_Client.py # Windows edge forwarder with auto-enrollment -β”‚ └── Linux_Client.py # Linux edge forwarder with auto-enrollment -β”œβ”€β”€ tests/ # Automated test suites -β”‚ β”œβ”€β”€ test_license_authority.py # Subordinate CA, WebSocket sync, and WebUI integration tests -β”‚ β”œβ”€β”€ test_linux_client.py # Linux client unit tests & mTLS certificate validation -β”‚ β”œβ”€β”€ test_pipeline.py # End-to-end mTLS integration & direct anomaly verification test -β”‚ β”œβ”€β”€ test_server.py # Server unit tests, PKI generation, user auth & seat quota tests -β”‚ └── test_win_client.py # Windows client unit tests & mTLS certificate validation -β”œβ”€β”€ .gitignore # Ignores venv, caches, DBs, and private keys -β”œβ”€β”€ README.md # Architecture and usage documentation -β”œβ”€β”€ RELEASE_NOTES.md # Release history and changelog -β”œβ”€β”€ server_config.sample.json # Reference server configuration -└── out/ # Component guides and sample configs - β”œβ”€β”€ linux_server/ - β”‚ β”œβ”€β”€ README.md - β”‚ └── server_config.sample.json - β”œβ”€β”€ win_server/ - β”‚ β”œβ”€β”€ README.md - β”‚ └── server_config.sample.json - β”œβ”€β”€ linux_client/ - β”‚ β”œβ”€β”€ README.md - β”‚ └── client_config.sample.json - └── win_client/ - β”œβ”€β”€ README.md - └── client_config.sample.json -``` - ---- - -## Getting Started & Deployment +## Quick Start ### 1. Central Server Hub -1. **Install dependencies**: - ```bash - pip install -r compilation/requirements.txt - ``` -2. **Start the server** (generates `server_config.json`, Root CA, and server certs on first run): - ```bash - python src/Server.py - ``` -3. **Export a client configuration**: - ```bash - python src/Server.py --create-client-config --server-host --server-port 9443 --client-out client_config.json - ``` +```bash +# Install dependencies +pip install -r compilation/requirements.txt -### 2. Windows Client Deployment -1. Download `LOGAR-Client-Setup.exe` from releases and run it, or place `Win_Client.exe` and `client_config.json` in `C:\Program Files\LOGAR`. -2. On first run with `client_config.json`, `Win_Client.exe` automatically enrolls with the hub, receives its mTLS certificates, and establishes secure streaming. +# Start Server (Generates initial keys and server_config.json on first run) +python src/Server.py -### 3. Linux Client Deployment -1. Run the automated installer: - ```bash - sudo ./compilation/install_linux_client.sh "http://:8443" "" - ``` -2. The installer enrolls the client, configures `/etc/logar/certs` (with `chmod 700`), and activates `logar-client.service`. +# WebUI available at: http://:8443/webui (Default: admin / admin_logar_2026!) +``` + +### 2. Linux Client Deployment + +```bash +sudo ./compilation/install_linux_client.sh "http://:8443" "" +``` +*Enrolls client certificates to `/etc/logar/certs`, writes configuration, and enables `logar-client.service`.* + +### 3. Windows Client Deployment + +1. Download and run `LOGAR-Client-Setup.exe` from [Releases](https://gitea.eibl.tech/me0nline/LOGAR/releases). +2. Place `client_config.json` (exported from Server WebUI) into `C:\Program Files\LOGAR\client_config.json`. +3. The `LOGAR_Client` Windows service starts and auto-enrolls immediately. --- -## Running Tests +## Repository Structure + +``` +LOGAR/ +β”œβ”€β”€ .gitea/workflows/ # CI/CD pipelines (ci.yml, release.yml) +β”œβ”€β”€ compilation/ # Build scripts, Linux installers & Inno Setup scripts +β”‚ β”œβ”€β”€ install_linux_client.sh # Linux client systemd auto-installer +β”‚ β”œβ”€β”€ install_linux_server.sh # Linux server systemd auto-installer +β”‚ β”œβ”€β”€ package_dist.py # Standalone binary cross-compiler +β”‚ └── upload_release.py # Gitea release asset publisher +β”œβ”€β”€ src/ # Application source code +β”‚ β”œβ”€β”€ Server.py # Central mTLS hub, temporal engine & REST API +β”‚ β”œβ”€β”€ server_enrollment.py # Sub-CA PKI and certificate generator +β”‚ β”œβ”€β”€ Win_Client.py # Windows edge forwarder +β”‚ └── Linux_Client.py # Linux edge forwarder +β”œβ”€β”€ tests/ # Automated test suites (35 unit tests + pipeline test) +β”œβ”€β”€ LOGAR-LICENSING/ # Standalone Master License Authority (Git submodule) +β”œβ”€β”€ RELEASE_NOTES.md # Version history and security changelog +└── README.md # Repository overview +``` + +--- + +## Ports & REST API Summary + +| Port | Protocol | Service | Description | +| :--- | :--- | :--- | :--- | +| **`9443`** | `TCP / TLS 1.3` | **mTLS Log Ingestion** | Mutual TLS socket listener for edge forwarders | +| **`8443`** | `HTTP / REST` | **Management & Hermes API** | WebUI, forwarder enrollment, seat management, and AI reports | + +### Core Endpoints + +- **`POST /api/client/enroll`**: Forwarder auto-enrollment and proactive certificate renewal. +- **`GET /api/clients`**: Returns active seat quota and registered edge forwarder telemetry *(Admin Auth)*. +- **`GET /api/hermes/report`**: Structured JSON anomaly stream for Hermes AI reasoning. +- **`POST /api/license/sync`**: Triggers immediate WebSocket sync with `logar.eibl.tech` *(Admin Auth)*. +- **`GET /health`**: Server health, encryption fingerprint, and mTLS status. + +--- + +## Testing & Cross-Compilation -### 1. Component Unit Tests ```bash +# Run full unit test suite (35 tests) python -m unittest discover -s tests -v -``` -Runs all **35 unit tests**, covering: -- Dynamic Root CA generation and server TLS certificate issuance. -- Dynamic client certificate issuance with CN and authority key extensions. -- Enrollment secret authentication, seat limits, and certificate revocation. -- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload. -- Subordinate CA CSR generation, WebSocket enrollment, and dynamic seat quota updates. -- Administrator session authentication, PBKDF2 hashing, and Hermes Bearer token lifecycle. -- Windows & Linux event log collection, deduplication, and mTLS certificate verification. -- 12-hour evaluation window and direct Hermes anomaly verification. -### 2. End-to-End Pipeline Integration Test -```bash -# 1. Initialize test configuration -python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json - -# 2. Launch server in background -python src/Server.py & - -# 3. Run integration test +# Run end-to-end mTLS pipeline integration test python tests/test_pipeline.py + +# Cross-compile native Linux and Windows binaries on Linux runner +python compilation/package_dist.py --target all ``` -Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output. --- -## Releases & Binary Distribution +## Documentation & Deep Dives -Releases are cross-compiled and published through automated or manual triggers: +For in-depth operational guides, lifecycle state machines, and PKI specifications, consult the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)**: -### 1. Tag Push Automation (Gitea Actions) -Pushing a release tag automatically triggers the build workflow: -```bash -git tag v3.0.1 -git push origin v3.0.1 -``` -The consolidated workflow **`release.yml`** runs on a single Ubuntu runner (`ubuntu-latest`) using cross-compilation (`python compilation/package_dist.py --target all`): -- **Linux Binaries**: Native ELF standalone executables (`Linux_Client.bin`, `Server.bin`). -- **Windows Executables & Bundles**: Cross-compiled standalone binaries (`Win_Client.exe`, `Server.exe`) and `.zip` distribution packages. -- **Automated Checksums & Publishing**: Generates `sha256sums.txt` and publishes all release assets and release notes directly to Gitea. - -### 2. Manual Workflow Dispatch (Gitea UI) -Workflows can be manually triggered on demand from the Gitea web interface: -1. Navigate to **Actions** $\rightarrow$ **Release Binaries & Installers** (`release.yml`). -2. Click **Run workflow**, set the release tag (defaults to `v3.0.1`), and run. - -### 3. Local Native Build & Direct Release Upload -To build and upload binaries locally from an administrative workstation: -```powershell -# 1. Build binaries locally (windows, linux, or all) -python compilation/package_dist.py --target all - -# 2. Upload assets and release notes directly to the Gitea release -python compilation/upload_release.py --tag v3.0.1 --token --skip-build -``` +* [Architecture & Design Details](https://gitea.eibl.tech/me0nline/LOGAR/wiki/Architecture-and-Design) +* [System State Machines & Subsystem Lifecycles](https://gitea.eibl.tech/me0nline/LOGAR/wiki/State-Machines-and-Lifecycles) +* [PKI Architecture & Custom X.509 Seat OIDs](https://gitea.eibl.tech/me0nline/LOGAR/wiki/PKI-and-Licensing) +* [Complete REST API Specification](https://gitea.eibl.tech/me0nline/LOGAR/wiki/API-Reference) +* [Production Deployment & Service Management](https://gitea.eibl.tech/me0nline/LOGAR/wiki/Deployment-and-Installation) +* [CI/CD Pipeline & Cross-Compilation Workflow](https://gitea.eibl.tech/me0nline/LOGAR/wiki/Testing-and-Releases)