LOGAR/README.md

# LOGAR: Fleet Log Aggregation & Centralized Provisioning System
### *Secure Log Aggregation, PKI Fleet Licensing & Centralized Delivery for Autonomous AI Analysis*

**LOGAR** is an enterprise-grade log aggregation, normalization, and centralized provisioning system built for heterogeneous server fleets (Windows & Linux). It pairs ultra-lightweight, zero-state edge forwarders with a secure centralized server hub that enforces mutual TLS 1.3 (**mTLS**) authentication, dynamic PKI fleet licensing with cryptographic seat leasing, 12-hour temporal persistence tracking, and direct anomaly pass-through—delivering clean, structured, verified log telemetry directly to **Hermes AI** for autonomous deep reasoning, contextual root-cause analysis, and operational decision-making.

---

## Table of Contents
1. [Core Purpose & Philosophy (Log Aggregation & Provisioning)](#core-purpose--philosophy)
2. [End-to-End System Architecture](#end-to-end-system-architecture)
3. [System State Machines & Subsystem Lifecycles](#system-state-machines--subsystem-lifecycles)
   - 3.1 [High-Level Overview State Machine](#31-high-level-system-state-machine)
   - 3.2 [Master License Authority & PKI Lifecycles](#32-master-license-authority--pki-lifecycles)
     - 3.2.1 [Bootstrap & Sub-CA Auto-Enrollment State Machine](#321-bootstrap--sub-ca-auto-enrollment-state-machine)
     - 3.2.2 [Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload](#322-live-websocket-heartbeat-seat-expansion--in-flight-reload)
   - 3.3 [Edge Forwarder Auto-Enrollment & mTLS Lifecycles](#33-edge-forwarder-auto-enrollment--mtls-lifecycles)
     - 3.3.1 [Machine Identity Discovery & Zero-Touch Enrollment](#331-machine-identity-discovery--zero-touch-enrollment)
     - 3.3.2 [mTLS 1.3 Streaming, Proactive Renewal & Self-Healing](#332-mtls-13-streaming-proactive-renewal--self-healing)
   - 3.4 [Fleet Log Aggregation, 12h Temporal Window & AI Provisioning](#34-fleet-log-aggregation-12h-temporal-window--ai-provisioning)
4. [Master License Authority & PKI Fleet Licensing](#master-license-authority--pki-fleet-licensing)
5. [mTLS 1.3 Transport Security & Forwarder Auto-Enrollment](#mtls-13-transport-security--forwarder-auto-enrollment)
6. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning)
7. [Agentic Hermes & Client Management API](#agentic-hermes--client-management-api)
8. [Dynamic Machine & Domain Identification](#dynamic-machine--domain-identification)
9. [Automated Service Installers (Linux & Windows)](#automated-service-installers-linux--windows)
10. [Repository & Shippables Structure](#repository--shippables-structure)
11. [Getting Started & Deployment](#getting-started--deployment)
12. [Running Tests](#running-tests)
13. [Automated Releases via Gitea Actions](#automated-releases-via-gitea-actions)

---

## Core Purpose & Philosophy

### 1. Centralized Log Aggregation & Provisioning
The foundational objective of LOGAR is reliable, secure collection and structured delivery of system events:
- **Heterogeneous Fleet Aggregation**: Ingests Windows Event Logs (System, Application, Security) and Linux `systemd journalctl` events into a unified, normalized JSON data format.
- **Centralized Provisioning**: Serves as the authoritative clearinghouse and real-time streaming endpoint (`/api/hermes/report`), providing structured logs with temporal metadata to external consumers and automated AI agents.
- **Zero Local Edge Footprint**: Forwarders maintain zero local databases, zero cache files, and zero disk overhead. If a network disruption occurs, agents automatically re-establish mTLS streaming without state corruption.

### 2. Direct Hermes Intelligence (No Heuristic Bottlenecks)
LOGAR strictly separates **data aggregation/provisioning** from **semantic reasoning**:
- **Removal of Arbitrary Heuristics**: The legacy 4-run throttling rule was eliminated. All ingested candidate logs (`INFO`, `WARNING`, `ERROR`, `CRITICAL`) within the active 12-hour temporal evaluation window are immediately verified and provisioned.
- **Hermes Autonomous Thinking**: **Hermes AI** performs all multi-factor reasoning, cross-server correlation, urgency prioritization, and automated remediation decisions directly on the provisioned log stream.

### 3. Hierarchical Dynamic PKI & Fleet Licensing
- **Authoritative Master PKI**: Subordinate server hubs do not require manual certificate generation or static keys. The **Master License Authority** (`logar.eibl.tech`) issues cryptographically signed Sub-CA certificates with custom X.509 seat quotas (`1.3.6.1.4.1.59999.1.1`).
- **Dynamic Seat Leasing**: Server hubs enforce local seat allocations in real-time, instantly revoking or admitting edge nodes while synchronizing with the central authority over bidirectional WebSockets.

---

## End-to-End System Architecture

```mermaid
graph TB
    subgraph Authority [Master License Authority - logar.eibl.tech]
        M_CA[Master Root CA<br/>RSA-4096 / SHA-256]
        AUTH_WS[WebSocket Licensing Hub<br/>/ws/license]
        AUTH_DB[(Authority DB<br/>Tenants, Users, 2FA, Seats)]
        AUTH_UI[Authority WebUI<br/>User Mgmt & Fleet Quotas]
        AUTH_WS --- AUTH_DB
        AUTH_UI --- AUTH_DB
    end

    subgraph ServerHub [LOGAR Server Hub - Central Aggregation & Provisioning]
        SUB_CA[Subordinate CA<br/>Signed with Seat OID 1.3.6.1.4.1.59999.1.1]
        SEAT_MGR[Dynamic Seat Manager<br/>Enforces Seat Quota in SQLite]
        MTLS_SRV[mTLS 1.3 Ingestion Server<br/>Port 9443 - ssl.CERT_REQUIRED]
        TEMP_ENG[12h Temporal Persistence Engine<br/>Correlates First/Last Seen & Run Counts]
        HUB_DB[(logar_state.db<br/>active_issues, clients, config)]
        PROV_API[Log Provisioning & Hermes API<br/>Port 8443 - GET /api/hermes/report]
        
        SUB_CA --> SEAT_MGR
        SEAT_MGR --> MTLS_SRV
        MTLS_SRV --> TEMP_ENG
        TEMP_ENG --> HUB_DB
        HUB_DB --> PROV_API
    end

    subgraph EdgeNodes [Heterogeneous Edge Forwarders]
        WIN[Win_Client.exe / Win_Client.py<br/>Windows Event Log Ingestion]
        LIN[Linux_Client.bin / Linux_Client.py<br/>systemd journalctl -p warning]
    end

    subgraph Downstream [Hermes AI Reasoning Layer]
        HERMES[Hermes AI Engine<br/>Autonomous Analysis & Remediation]
    end

    %% PKI & Licensing Relationships
    M_CA -->|Signs Sub-CA Cert via /ws/license| SUB_CA
    AUTH_WS <==>|Bidirectional Sync & Live Seat Expansion| SUB_CA
    
    %% Forwarder Enrollment & mTLS Transport
    WIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA
    LIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA
    SUB_CA -->|2. Issues Client Cert & Root Trust| WIN
    SUB_CA -->|2. Issues Client Cert & Root Trust| LIN
    
    WIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV
    LIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV

    %% Log Provisioning & Hermes Consumption
    PROV_API ==>|Centralized Log Delivery| HERMES
```

---

## System State Machines & Subsystem Lifecycles

### 3.1 High-Level System State Machine
A simplified macroscopic view of the end-to-end LOGAR lifecycle:

```mermaid
stateDiagram-v2
    [*] --> 1_PKI_Licensing: Master Authority generates Bootstrap .lic Package
    1_PKI_Licensing --> 2_Hub_Ready: Server Hub imports .lic & signs Sub-CA over WebSocket
    2_Hub_Ready --> 3_Edge_Enrollment: Edge Forwarders auto-enroll & lease licensed seats
    3_Edge_Enrollment --> 4_mTLS_Streaming: Forwarders stream framed logs over mTLS 1.3 (:9443)
    4_mTLS_Streaming --> 5_Temporal_Aggregation: Server Hub normalizes & correlates logs in 12h window
    5_Temporal_Aggregation --> 6_AI_Provisioning: Logs provisioned to Hermes AI via GET /api/hermes/report
    6_AI_Provisioning --> 4_mTLS_Streaming: Continuous operational loop
```

---

### 3.2 Master License Authority & PKI Lifecycles

#### 3.2.1 Bootstrap & Sub-CA Auto-Enrollment State Machine
Details how a newly deployed LOGAR Server Hub imports initial credentials, connects to `logar.eibl.tech` over WebSocket, and secures its signed Sub-CA certificate:

```mermaid
stateDiagram-v2
    [*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority
    GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub
    ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR
    LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
    WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR
    ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
    SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota
    QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active
```

#### 3.2.2 Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload
Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior:

```mermaid
stateDiagram-v2
    [*] --> ActiveListener: Server Hub actively servicing edge forwarders
    ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat
    
    state WebSocketHeartbeat <<choice>>
    WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota
    WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack
    WebSocketHeartbeat --> Revoked: Authority revoked license_id
    
    QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight
    InFlightReload --> ActiveListener: Hot reload complete (Zero downtime)
    SteadyState --> ActiveListener
    Revoked --> DropConnections: Drops all incoming mTLS connections immediately
```

---

### 3.3 Edge Forwarder Auto-Enrollment & mTLS Lifecycles

#### 3.3.1 Machine Identity Discovery & Zero-Touch Enrollment
Details how forwarders automatically resolve host attribution and lease seats:

```mermaid
stateDiagram-v2
    [*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches
    DiscoverIdentity --> QueryFQDN: socket.getfqdn()
    QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf
    ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution
    
    BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt
    state CheckCertificates <<choice>>
    CheckCertificates --> ReadyToStream: Certificates valid & unexpired
    CheckCertificates --> RequestEnrollment: Certificates missing or expired
    
    RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret
    state ValidateQuota <<choice>>
    ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid)
    ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret)
    
    StoreCerts --> ReadyToStream
```

#### 3.3.2 mTLS 1.3 Streaming, Proactive Renewal & Self-Healing
Details the secure streaming loop, proactive certificate lifecycle, and resilience against CA rotation:

```mermaid
stateDiagram-v2
    [*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443)
    mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table
    
    state CheckClientCN <<choice>>
    CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota)
    CheckClientCN --> DropClient: Client seat revoked in DB
    
    ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run
    PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll)
    ProactiveRenew --> mTLSHandshake
    
    ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls)
    ReactiveHeal --> mTLSHandshake
    DropClient --> [*]
```

---

### 3.4 Fleet Log Aggregation, 12h Temporal Window & AI Provisioning
Details how log events are extracted, deduplicated, temporally tracked across the 12-hour window, and provisioned to Hermes AI:

```mermaid
stateDiagram-v2
    [*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl
    EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old
    NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature})
    FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443)
    
    FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window
    state EvaluateWindow <<choice>>
    EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours
    EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours
    
    WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen
    WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps
    
    UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED"
    ResetRunCount --> UpsertDB
    
    UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443)
    ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions
```

---

## Master License Authority & PKI Fleet Licensing

### 1. TLS 1.3 Mutual Authentication (mTLS)
- **Port 9443**: Ingestion occurs exclusively over TLS 1.3 sockets with `ssl.CERT_REQUIRED`.
- Both the hub and edge clients verify each other's certificates:
  - Client verifies server certificate against `ca.crt`.
  - Server verifies client certificate against the Subordinate/Root CA.
- **Client CN Identification & Revocation**: In the TLS handshake, the server extracts the `commonName` attribute (`client_id`), validates that the client is marked `active` in the `clients` table, updates the `last_seen` timestamp, and drops unregistered or revoked certificates immediately.

### 2. Master License Authority (`logar.eibl.tech`), Multi-Tenancy & Subordinate CA
- **Master Root Authority**: Standalone PKI repository & submodule ([`LOGAR-LICENSING/`](file:///u:/max/Desktop/LOGAR/LOGAR-LICENSING/README.md), upstream at [`https://gitea.eibl.tech/me0nline/LOGAR-LICENSING.git`](https://gitea.eibl.tech/me0nline/LOGAR-LICENSING.git)) deployed at `logar.eibl.tech`.
- **Multi-Tenant Fleet Isolation & User Management**:
  - Supports multiple isolated **Tenants** (e.g. `DEFAULT`, `CORP_A`, `PRODUCTION_FLEET`) with live tenant filtering.
  - Built-in user management with PBKDF2 password hashing, RBAC (Administrators & Operators), and self-service password updates.
- **Custom Enterprise X.509 OIDs**:
  - `OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)`: Embeds the exact licensed seat quota directly inside the signed Subordinate CA certificate.
  - `OID_LOGAR_LICENSE_ID (1.3.6.1.4.1.59999.1.2)`: Uniquely tracks enterprise license entitlement.
  - `OID_LOGAR_ISSUER_URL (1.3.6.1.4.1.59999.1.3)`: Embedded endpoint coordinate (`https://logar.eibl.tech`).
- **Bootstrap `.lic` Packages & Instant WebSocket Auto-Enrollment**:
  - The License Authority (`logar.eibl.tech`) generates initial **Bootstrap `.lic` Packages** containing Master Root CA, dedicated communication credentials (`communication_cert` + `communication_key`), CSR templates, assigned tenant, and WebSocket coordinates bound directly to the authority server host.
  - **1-Click Import & Live Signing**: Importing the `.lic` file on the client server (`python src/Server.py --import-lic <file.lic>` or via WebUI) connects directly via **WebSocket**, submits the server Sub-CA CSR, and immediately installs the officially signed Sub-CA license certificate.
- **Self-Updating Client Servers & Live WebSocket Sync**:
  - Client servers update themselves against the master authority (`wss://logar.eibl.tech/ws/license`) over WebSocket (`python src/Server.py --sync-license` or via WebUI / automated watchdog).
  - Seat quotas can be increased with each renewal or updated on-demand in the License Authority WebUI.
  - The client server automatically receives the updated seat quota in its signed certificate, updates its SQLite database and runtime configuration, and reloads its in-flight `ssl.SSLContext` without dropping client connections or restarting services.
- **Automated Container Pipeline & Ephemeral NPM Pass-Through**:
  - The license authority repository includes a dedicated CI/CD workflow ([`.gitea/workflows/build-container.yml`](file:///u:/max/Desktop/LOGAR/LOGAR-LICENSING/.gitea/workflows/build-container.yml)) publishing to `gitea.eibl.tech/me0nline/logar-licensing:latest` (authenticated via `giteatoken` secret).
  - Works seamlessly with ephemeral Nginx Proxy Manager (NPM) by enabling the standard **Websockets Support** toggle for `logar.eibl.tech`.
  - Admin WebUI is protected by authenticated login with user and tenant administration.

### 3. Integrated Fleet Licensing & WebUI Dashboard
- **WebUI URL**: Accessible at `http://<server_host>:8443/webui` (or `/`). Protected by authenticated session login (`/login`).
- **Tab 1: 📋 Active Anomalies & Telemetry**:
  - Real-time tabular stream of verified anomalies within the active 12-hour evaluation window.
  - Interactive search filter and Hermes AI status metrics.
  - **Visual Seat Quota Meter**: Real-time progress bar displaying active forwarders vs. allocated seat limit.
  - **Seat Management & Revocation Table**: Lists all connected Windows and Linux edge forwarders with 1-click **Revoke Seat**, **Reactivate Seat**, and **Export Client Configuration** actions.
- **Tab 2: 📜 Sub-CA & Licensing**:
  - Dedicated PKI and Subordinate CA lifecycle management interface.
  - Displays Master Authority WebSocket status (`logar.eibl.tech`), hardware fingerprint, and embedded seat quota.
  - **Licensing Actions**:
    - 1-click **Export .lic Request** file download.
    - Interactive **View & Copy CA CSR** modal.
    - Drag-and-drop or paste **Import License (.lic / ca.crt)** modal.
    - **Sync & Renew via WebSocket** button for immediate upstream synchronization and seat expansion.

### 4. Cloud-Side In-Flight Certificate Watchdog
- **Continuous Hub PKI Watchdog**:
  - The server hub runs a continuous background watchdog coroutine (`cert_validity_watchdog`, running every 12 hours).
  - Automatically renews Subordinate CA certificates over mTLS with `logar.eibl.tech` and Server TLS certificates in-flight, dynamically reloading the active `ssl.SSLContext`.
- **Client Proactive Check & Reactive Self-Healing**:
  - **Proactive Renewal**: Edge clients inspect `client.crt` before every run cycle. If expiring within 30 days, they contact `/api/client/enroll` to renew.
  - **Reactive Self-Healing**: If the server CA rotates or a handshake error occurs, edge clients catch the exception, re-bootstrap enrollment against the hub, and re-establish streaming cleanly.

---

## Cloud-Side Temporal Persistence & Direct Hermes Reasoning

Incoming candidate logs are tracked in SQLite table `active_issues`:
- **Issue Fingerprint**: Formatted as `{site_name}:{server}:{signature}`.
- **12-Hour Evaluation Window**:
  - The hub compares `(now - last_seen)`.
  - If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to `run_count = 1`.
- **Direct Verification & Temporal Telemetry**:
  - Ingested anomalies are immediately promoted to `VERIFIED` and made available to Hermes with complete temporal metadata (first seen, last seen, consecutive run counts).
  - Hermes performs all contextual analysis, pattern recognition, and alerting logic.

---

## Agentic Hermes & Client Management API

The server hub exposes a management and reporting REST API (default port `8443`):

### `POST /api/client/enroll`
Client enrollment endpoint:
- **Request**:
  ```json
  {
    "client_id": "web-worker-01.corp.internal",
    "hostname": "web-worker-01",
    "os": "linux",
    "enrollment_secret": "<SECRET>"
  }
  ```
- **Response**:
  ```json
  {
    "ca_cert": "-----BEGIN CERTIFICATE-----\n...",
    "client_cert": "-----BEGIN CERTIFICATE-----\n...",
    "client_key": "-----BEGIN RSA PRIVATE KEY-----\n..."
  }
  ```

### `GET /api/clients`
Returns seat quota status and registered client telemetry *(Requires Administrator session cookie or Bearer auth)*:
```json
{
  "active_seats": 2,
  "max_seats": 10,
  "clients": [
    {
      "client_id": "web-worker-01.corp.internal",
      "hostname": "web-worker-01",
      "os_type": "linux",
      "cert_fingerprint": "7D5B660B...",
      "status": "active",
      "first_seen": "2026-09-04 18:00:00",
      "last_seen": "2026-09-04 19:15:00"
    }
  ]
}
```

### `GET /api/hermes/report`
Returns all **verified anomalies** within the active 12-hour evaluation window with full temporal metadata for direct Hermes reasoning:
```json
[
  {
    "fingerprint": "corp.internal:web-worker-01.corp.internal:PostgresPoolExhausted",
    "site": "corp.internal",
    "server": "web-worker-01.corp.internal",
    "signature": "PostgresPoolExhausted",
    "severity": "WARNING",
    "message": "Connection pool saturated (>95%)",
    "os_type": "linux",
    "first_seen": "2026-09-04T07:00:00+00:00",
    "last_seen": "2026-09-04T19:00:00+00:00",
    "consecutive_runs": 4,
    "evaluation_window": "12h",
    "verified": true,
    "status": "VERIFIED"
  }
]
```

### `GET /api/hermes/all`
Diagnostic endpoint listing all tracked anomalies in SQLite *(Requires Administrator auth)*.

### `POST /api/auth/login` & `POST /api/auth/logout`
Session authentication endpoints for Administrator WebUI access.

### `GET /health`
Returns hub health, encryption fingerprint, listener ports, and mTLS status.

---

## Dynamic Machine & Domain Identification

Client configurations intentionally contain **no hardcoded machine name or site name**. Both forwarders dynamically identify their host and domain at runtime via `get_machine_identifier()`:
1. **Fully Qualified Domain Name (FQDN)**: Checked via `socket.getfqdn()`.
2. **OS-Specific Domain Discovery**:
   - **Windows**: Checks Active Directory environment variables (`USERDNSDOMAIN`, `USERDOMAIN`).
   - **Linux**: Parses `/etc/resolv.conf` `domain` and `search` directives.
3. **Reverse DNS Lookup**: Resolves canonical hostname via `socket.gethostbyaddr`.
4. **Fallback**: Local hostname `socket.gethostname()`.

The server automatically infers site attribution from domain qualifiers (e.g. `node01.corp.internal` $\rightarrow$ site `corp.internal`).

---

## Automated Service Installers (Linux & Windows)

LOGAR provides production-grade installation scripts and installer builders for automated service deployment:

### 1. Linux Service Installers
- **Client Installer (`compilation/install_linux_client.sh`)**:
  - Non-interactive script deploying to `/opt/logar-client`.
  - Automatically queries `/etc/machine-id` and enrolls with the hub via `curl`.
  - Installs and enables `logar-client.service` systemd unit.
  ```bash
  sudo ./compilation/install_linux_client.sh "http://hub.example.com:8443" "<ENROLLMENT_SECRET>"
  ```
- **Server Installer (`compilation/install_linux_server.sh`)**:
  - Deploys server to `/opt/logar-server`.
  - Configures logging and installs `logar-server.service` with `LimitNOFILE=65536`.
  ```bash
  sudo ./compilation/install_linux_server.sh
  ```

### 2. Windows Inno Setup Installers
- Built using **Inno Setup 6** and bundled with **NSSM** (`compilation/nssm.exe`):
  - **Client Setup (`compilation/installer_client.iss`)**: Compiles `LOGAR-Client-Setup.exe`. Installs `Win_Client.exe` into `{autopf}\LOGAR`, sets up `LOGAR_Client` service via NSSM with stdout/stderr redirection to `{commonappdata}\LOGAR\client.log`, and starts the service. Clean uninstallation stops and removes the service.
  - **Server Setup (`compilation/installer_server.iss`)**: Compiles `LOGAR-Server-Setup.exe`. Installs `Server.exe` and sets up `LOGAR_Server` Windows service via NSSM.

---

## Repository & Shippables Structure

```
LOGAR/
├── .gitea/
│   └── workflows/
│       ├── ci.yml                     # CI pipeline: syntax, 35 unit tests & mTLS pipeline test
│       └── release.yml                # Consolidated release workflow (Cross-compilation on Linux runner)
├── compilation/                       # Packaging, installers, and release automation
│   ├── install_linux_client.sh        # Automated Linux client systemd installation script
│   ├── install_linux_server.sh        # Automated Linux server systemd installation script
│   ├── installer_client.iss           # Inno Setup Windows Client installer script
│   ├── installer_server.iss           # Inno Setup Windows Server installer script
│   ├── nssm.exe                       # Official 64-bit NSSM service manager binary
│   ├── package_dist.py                # Standalone binary cross-compiler & packager
│   ├── requirements.txt               # Unified project dependencies
│   └── upload_release.py              # Gitea REST API release asset publisher
├── src/                               # Core application source modules
│   ├── __init__.py
│   ├── Server.py                      # Central mTLS server, temporal engine, and Hermes REST API
│   ├── server_enrollment.py           # Dynamic PKI, Root CA, and client certificate generator
│   ├── Win_Client.py                  # Windows edge forwarder with auto-enrollment
│   └── Linux_Client.py                # Linux edge forwarder with auto-enrollment
├── tests/                             # Automated test suites
│   ├── test_license_authority.py      # Subordinate CA, WebSocket sync, and WebUI integration tests
│   ├── test_linux_client.py           # Linux client unit tests & mTLS certificate validation
│   ├── test_pipeline.py               # End-to-end mTLS integration & direct anomaly verification test
│   ├── test_server.py                 # Server unit tests, PKI generation, user auth & seat quota tests
│   └── test_win_client.py             # Windows client unit tests & mTLS certificate validation
├── .gitignore                         # Ignores venv, caches, DBs, and private keys
├── README.md                          # Architecture and usage documentation
├── RELEASE_NOTES.md                   # Release history and changelog
├── server_config.sample.json          # Reference server configuration
└── out/                               # Component guides and sample configs
    ├── linux_server/
    │   ├── README.md
    │   └── server_config.sample.json
    ├── win_server/
    │   ├── README.md
    │   └── server_config.sample.json
    ├── linux_client/
    │   ├── README.md
    │   └── client_config.sample.json
    └── win_client/
        ├── README.md
        └── client_config.sample.json
```

---

## Getting Started & Deployment

### 1. Central Server Hub

1. **Install dependencies**:
   ```bash
   pip install -r compilation/requirements.txt
   ```
2. **Start the server** (generates `server_config.json`, Root CA, and server certs on first run):
   ```bash
   python src/Server.py
   ```
3. **Export a client configuration**:
   ```bash
   python src/Server.py --create-client-config --server-host <SERVER_IP> --server-port 9443 --client-out client_config.json
   ```

### 2. Windows Client Deployment
1. Download `LOGAR-Client-Setup.exe` from releases and run it, or place `Win_Client.exe` and `client_config.json` in `C:\Program Files\LOGAR`.
2. On first run with `client_config.json`, `Win_Client.exe` automatically enrolls with the hub, receives its mTLS certificates, and establishes secure streaming.

### 3. Linux Client Deployment
1. Run the automated installer:
   ```bash
   sudo ./compilation/install_linux_client.sh "http://<HUB_HOST>:8443" "<ENROLLMENT_SECRET>"
   ```
2. The installer enrolls the client, configures `/etc/logar/certs` (with `chmod 700`), and activates `logar-client.service`.

---

## Running Tests

### 1. Component Unit Tests
```bash
python -m unittest discover -s tests -v
```
Runs all **35 unit tests**, covering:
- Dynamic Root CA generation and server TLS certificate issuance.
- Dynamic client certificate issuance with CN and authority key extensions.
- Enrollment secret authentication, seat limits, and certificate revocation.
- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload.
- Subordinate CA CSR generation, WebSocket enrollment, and dynamic seat quota updates.
- Administrator session authentication, PBKDF2 hashing, and Hermes Bearer token lifecycle.
- Windows & Linux event log collection, deduplication, and mTLS certificate verification.
- 12-hour evaluation window and direct Hermes anomaly verification.

### 2. End-to-End Pipeline Integration Test
```bash
# 1. Initialize test configuration
python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json

# 2. Launch server in background
python src/Server.py &

# 3. Run integration test
python tests/test_pipeline.py
```
Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output.

---

## Releases & Binary Distribution

Releases are cross-compiled and published through automated or manual triggers:

### 1. Tag Push Automation (Gitea Actions)
Pushing a release tag automatically triggers the build workflow:
```bash
git tag v3.0.1
git push origin v3.0.1
```
The consolidated workflow **`release.yml`** runs on a single Ubuntu runner (`ubuntu-latest`) using cross-compilation (`python compilation/package_dist.py --target all`):
- **Linux Binaries**: Native ELF standalone executables (`Linux_Client.bin`, `Server.bin`).
- **Windows Executables & Bundles**: Cross-compiled standalone binaries (`Win_Client.exe`, `Server.exe`) and `.zip` distribution packages.
- **Automated Checksums & Publishing**: Generates `sha256sums.txt` and publishes all release assets and release notes directly to Gitea.

### 2. Manual Workflow Dispatch (Gitea UI)
Workflows can be manually triggered on demand from the Gitea web interface:
1. Navigate to **Actions** $\rightarrow$ **Release Binaries & Installers** (`release.yml`).
2. Click **Run workflow**, set the release tag (defaults to `v3.0.1`), and run.

### 3. Local Native Build & Direct Release Upload
To build and upload binaries locally from an administrative workstation:
```powershell
# 1. Build binaries locally (windows, linux, or all)
python compilation/package_dist.py --target all

# 2. Upload assets and release notes directly to the Gitea release
python compilation/upload_release.py --tag v3.0.1 --token <GITEA_TOKEN> --skip-build
```