docs(wiki): add detailed technical documentation pages
+167
@@ -0,0 +1,167 @@
|
|||||||
|
# API Reference
|
||||||
|
|
||||||
|
The LOGAR Server Hub exposes a comprehensive REST API on default port `8443` for edge enrollment, client seat management, Hermes AI log ingestion, and WebUI administration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Authentication & Security
|
||||||
|
|
||||||
|
- **Administrator Endpoints**: Require either an authenticated session cookie (established via `/login` or `/api/auth/login`) or an `Authorization: Bearer <ADMIN_TOKEN>` header.
|
||||||
|
- **Hermes Endpoints**: Configurable for open access or Bearer token authentication via `Authorization: Bearer <HERMES_TOKEN>`.
|
||||||
|
- **Brute-Force Rate Limiting**: All authentication and enrollment endpoints enforce in-memory IP rate limiting.
|
||||||
|
- **Sanitization**: Dynamic WebUI rendering applies full `html.escape` sanitization against Stored XSS.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Forwarder Enrollment API
|
||||||
|
|
||||||
|
### `POST /api/client/enroll`
|
||||||
|
Enrolls a new edge client or proactively renews an existing client's certificate.
|
||||||
|
|
||||||
|
**Request Body**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"client_id": "db-prod-01.corp.internal",
|
||||||
|
"hostname": "db-prod-01",
|
||||||
|
"os": "linux",
|
||||||
|
"enrollment_secret": "SECURE_SHARED_SECRET"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Success Response (`200 OK`)**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ca_cert": "-----BEGIN CERTIFICATE-----\nMIID...",
|
||||||
|
"client_cert": "-----BEGIN CERTIFICATE-----\nMIID...",
|
||||||
|
"client_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE..."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Error Responses**:
|
||||||
|
- `403 Forbidden`: Invalid enrollment secret or seat quota exceeded.
|
||||||
|
- `429 Too Many Requests`: Rate limit exceeded.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Client & Seat Management API
|
||||||
|
|
||||||
|
### `GET /api/clients`
|
||||||
|
Returns seat quota status and registered client telemetry *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
**Response (`200 OK`)**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"active_seats": 2,
|
||||||
|
"max_seats": 25,
|
||||||
|
"clients": [
|
||||||
|
{
|
||||||
|
"client_id": "db-prod-01.corp.internal",
|
||||||
|
"hostname": "db-prod-01",
|
||||||
|
"os_type": "linux",
|
||||||
|
"cert_fingerprint": "7D5B660B08...",
|
||||||
|
"status": "active",
|
||||||
|
"first_seen": "2026-09-26 10:00:00",
|
||||||
|
"last_seen": "2026-09-26 12:45:00",
|
||||||
|
"last_seen_relative": "15m ago"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `POST /api/client/revoke`
|
||||||
|
Revokes an edge client's seat license *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
**Request Body**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"client_id": "db-prod-01.corp.internal"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `POST /api/client/approve`
|
||||||
|
Reactivates a revoked edge client *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
**Request Body**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"client_id": "db-prod-01.corp.internal"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `POST /api/client/delete`
|
||||||
|
Permanently removes a forwarder record from SQLite *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
**Request Body**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"client_id": "db-prod-01.corp.internal"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `GET /api/client/export`
|
||||||
|
Generates a downloadable `client_config.json` profile pre-configured for edge deployment *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Hermes AI & Log Telemetry API
|
||||||
|
|
||||||
|
### `GET /api/hermes/report`
|
||||||
|
Returns all **verified anomalies** within the active 12-hour evaluation window with full temporal metadata.
|
||||||
|
|
||||||
|
**Response (`200 OK`)**:
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"fingerprint": "corp.internal:db-prod-01.corp.internal:PostgresConnectionLimitExceeded",
|
||||||
|
"site": "corp.internal",
|
||||||
|
"server": "db-prod-01.corp.internal",
|
||||||
|
"signature": "PostgresConnectionLimitExceeded",
|
||||||
|
"severity": "WARNING",
|
||||||
|
"message": "Maximum client connections reached (100/100)",
|
||||||
|
"os_type": "linux",
|
||||||
|
"first_seen": "2026-09-26T08:00:00+00:00",
|
||||||
|
"last_seen": "2026-09-26T12:00:00+00:00",
|
||||||
|
"consecutive_runs": 5,
|
||||||
|
"evaluation_window": "12h",
|
||||||
|
"verified": true,
|
||||||
|
"status": "VERIFIED"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
### `GET /api/hermes/all`
|
||||||
|
Diagnostic endpoint listing all raw tracked issues in SQLite *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Licensing & Sub-CA Management API
|
||||||
|
|
||||||
|
### `GET /api/ca/csr`
|
||||||
|
Returns the current Subordinate CA Certificate Signing Request in PEM format.
|
||||||
|
|
||||||
|
### `POST /api/license/update`
|
||||||
|
Imports a signed Subordinate CA certificate (`ca.crt`) or activated `.lic` bundle *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
### `POST /api/license/sync`
|
||||||
|
Triggers an immediate background synchronization over WebSocket with the Master Authority (`wss://logar.eibl.tech/ws/license`) *(Requires Admin Auth)*.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Authentication & Health API
|
||||||
|
|
||||||
|
### `POST /api/auth/login`
|
||||||
|
Authenticates administrator credentials and establishes session cookie.
|
||||||
|
|
||||||
|
**Request Body**:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"username": "admin",
|
||||||
|
"password": "YOUR_ADMIN_PASSWORD"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### `POST /api/auth/logout`
|
||||||
|
Destroys the active administrator session cookie.
|
||||||
|
|
||||||
|
### `GET /health`
|
||||||
|
Returns hub health, encryption fingerprint, active listener ports, and mTLS status.
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Architecture and Design
|
||||||
|
|
||||||
|
**LOGAR** is engineered as a zero-state, high-throughput log aggregation and provisioning system designed for enterprise server fleets comprising both Windows and Linux hosts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Core Design Philosophy
|
||||||
|
|
||||||
|
### Centralized Log Aggregation & Zero-State Edge
|
||||||
|
Edge agents (`Win_Client` and `Linux_Client`) follow a strict **zero-state architecture**:
|
||||||
|
- **Zero Local Databases**: Edge forwarders do not maintain local SQLite databases or persistent files on disk.
|
||||||
|
- **Zero Disk Buffering**: Log events are filtered, formatted in memory, and immediately streamed over mTLS 1.3 sockets.
|
||||||
|
- **Fail-Safe Self-Healing**: In the event of network disconnection, connection reset, or CA rotation, clients re-establish connections or re-enroll automatically without state corruption or data loss.
|
||||||
|
|
||||||
|
### Direct Hermes Intelligence
|
||||||
|
LOGAR decouples data aggregation and delivery from semantic log analysis:
|
||||||
|
- **No Arbitrary Edge Filtering**: All anomalous log events (`WARNING`, `ERROR`, `CRITICAL`) within the active 12-hour evaluation window are ingested and normalized.
|
||||||
|
- **Hermes AI Reasoning**: **Hermes AI** queries the centralized server hub (`GET /api/hermes/report`) and applies multi-factor reasoning, cross-system correlation, severity ranking, and autonomous remediation actions directly on verified telemetry.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Component Subsystems
|
||||||
|
|
||||||
|
### Edge Forwarders
|
||||||
|
1. **Windows Client (`Win_Client.exe` / `Win_Client.py`)**:
|
||||||
|
- Queries Windows Event Logs (`System`, `Application`, `Security`) via `win32evtlog`.
|
||||||
|
- Normalizes Windows event categories into standard severity levels (`INFO`, `WARNING`, `ERROR`, `CRITICAL`).
|
||||||
|
- Dynamic host/domain identification via Active Directory environment variables.
|
||||||
|
2. **Linux Client (`Linux_Client.bin` / `Linux_Client.py`)**:
|
||||||
|
- Ingests system logs via `systemd journalctl -p warning..emerg --output=json`.
|
||||||
|
- Strips noisy trace entries and parses unit attribution.
|
||||||
|
- Dynamic host/domain identification via `/etc/resolv.conf` and canonical DNS resolution.
|
||||||
|
|
||||||
|
### Central Server Hub (`Server.py`)
|
||||||
|
1. **mTLS 1.3 Ingestion Engine (Port 9443)**:
|
||||||
|
- Enforces bidirectional TLS 1.3 with mutual certificate verification (`ssl.CERT_REQUIRED`).
|
||||||
|
- Validates client certificates against the Subordinate CA in real time.
|
||||||
|
- Checks active seat status on every handshake and drops revoked edge nodes.
|
||||||
|
2. **Temporal Persistence Engine**:
|
||||||
|
- Tracks active anomalies in SQLite (`active_issues`).
|
||||||
|
- Formats unique fingerprints: `{site_name}:{server}:{signature}`.
|
||||||
|
- Maintains consecutive run counts within a rolling 12-hour evaluation window.
|
||||||
|
3. **Log Provisioning REST API (Port 8443)**:
|
||||||
|
- Exposes clean, verified log telemetry to Hermes AI via `/api/hermes/report`.
|
||||||
|
- Provides administrator WebUI and seat management endpoints.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Data Flow
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
autonumber
|
||||||
|
participant Edge as Edge Forwarder (Win/Linux)
|
||||||
|
participant Hub as LOGAR Server Hub
|
||||||
|
participant DB as logar_state.db
|
||||||
|
participant Hermes as Hermes AI Engine
|
||||||
|
|
||||||
|
Edge->>Hub: TLS 1.3 Handshake (Client Cert :9443)
|
||||||
|
Hub->>DB: Check client_id status (active?)
|
||||||
|
DB-->>Hub: Approved (Seat valid)
|
||||||
|
Hub-->>Edge: TLS Handshake OK
|
||||||
|
|
||||||
|
loop Streaming Logs
|
||||||
|
Edge->>Hub: Stream JSON Log Frame ({site, server, signature, msg})
|
||||||
|
Hub->>DB: Upsert active_issues (Update run_count & last_seen)
|
||||||
|
end
|
||||||
|
|
||||||
|
Hermes->>Hub: GET /api/hermes/report (:8443)
|
||||||
|
Hub->>DB: SELECT verified issues within 12h window
|
||||||
|
DB-->>Hub: Return active issue records
|
||||||
|
Hub-->>Hermes: 200 OK [JSON Telemetry Stream]
|
||||||
|
Hermes->>Hermes: Deep Reasoning & Automated Remediation
|
||||||
|
```
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
# Deployment and Installation
|
||||||
|
|
||||||
|
LOGAR is designed for rapid, automated deployment across both Linux and Windows environments with native service integration.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Central Server Hub Deployment
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
- Python 3.10+ (or standalone Linux binary / Docker container)
|
||||||
|
- Inbound access on ports `9443` (mTLS Ingestion) and `8443` (WebUI & API)
|
||||||
|
|
||||||
|
### Automated Linux Server Installation
|
||||||
|
Run the automated systemd installer script on your server:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo ./compilation/install_linux_server.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This script:
|
||||||
|
1. Deploys the application files to `/opt/logar-server`.
|
||||||
|
2. Sets up Python dependencies and virtual environment.
|
||||||
|
3. Generates the initial PKI keys and configuration.
|
||||||
|
4. Installs and starts `logar-server.service` with `LimitNOFILE=65536`.
|
||||||
|
|
||||||
|
### Windows Server Installation
|
||||||
|
1. Compile or download `LOGAR-Server-Setup.exe`.
|
||||||
|
2. Execute the installer. It will install the server binary to `{autopf}\LOGAR` and register the `LOGAR_Server` Windows service via NSSM.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Linux Client Deployment
|
||||||
|
|
||||||
|
### Automated Non-Interactive Installer
|
||||||
|
Deploy edge forwarders effortlessly across your Linux fleet:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo ./compilation/install_linux_client.sh "http://<HUB_HOST>:8443" "<ENROLLMENT_SECRET>"
|
||||||
|
```
|
||||||
|
|
||||||
|
The installer performs the following:
|
||||||
|
1. Queries `/etc/machine-id` and canonical FQDN.
|
||||||
|
2. Contacts `/api/client/enroll` on the server hub using the shared enrollment secret.
|
||||||
|
3. Receives and stores `ca.crt`, `client.crt`, and `client.key` in `/etc/logar/certs` with strict permissions (`chmod 700`).
|
||||||
|
4. Generates `/etc/logar/client_config.json`.
|
||||||
|
5. Installs and activates `logar-client.service` systemd unit.
|
||||||
|
|
||||||
|
### Systemd Service Management
|
||||||
|
```bash
|
||||||
|
# Check status
|
||||||
|
systemctl status logar-client.service
|
||||||
|
|
||||||
|
# View forwarder logs
|
||||||
|
journalctl -u logar-client.service -f
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Windows Client Deployment
|
||||||
|
|
||||||
|
### Inno Setup Automated Installer
|
||||||
|
1. Download `LOGAR-Client-Setup.exe` from the latest release.
|
||||||
|
2. Run the installer or deploy silently via Group Policy / SCCM:
|
||||||
|
```cmd
|
||||||
|
LOGAR-Client-Setup.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART
|
||||||
|
```
|
||||||
|
3. Place `client_config.json` (exported from Server WebUI) into `C:\Program Files\LOGAR\client_config.json`.
|
||||||
|
4. The service automatically enrolls on first start and streams Windows Event Logs continuously.
|
||||||
|
|
||||||
|
### Windows Service Management
|
||||||
|
- **Service Name**: `LOGAR_Client`
|
||||||
|
- **Log Output**: `C:\ProgramData\LOGAR\client.log`
|
||||||
|
- Managed via standard Windows Services MMC (`services.msc`) or `sc.exe`:
|
||||||
|
```cmd
|
||||||
|
sc query LOGAR_Client
|
||||||
|
sc stop LOGAR_Client
|
||||||
|
sc start LOGAR_Client
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Dynamic Machine & Domain Identification
|
||||||
|
|
||||||
|
Client configurations intentionally contain **no hardcoded machine name or site name**. Both forwarders dynamically identify their host and domain at runtime via `get_machine_identifier()`:
|
||||||
|
1. **Fully Qualified Domain Name (FQDN)**: Checked via `socket.getfqdn()`.
|
||||||
|
2. **OS-Specific Domain Discovery**:
|
||||||
|
- **Windows**: Checks Active Directory environment variables (`USERDNSDOMAIN`, `USERDOMAIN`).
|
||||||
|
- **Linux**: Parses `/etc/resolv.conf` `domain` and `search` directives.
|
||||||
|
3. **Reverse DNS Lookup**: Resolves canonical hostname via `socket.gethostbyaddr`.
|
||||||
|
4. **Fallback**: Local hostname `socket.gethostname()`.
|
||||||
|
|
||||||
|
The server automatically infers site attribution from domain qualifiers (e.g. `node01.corp.internal` $\rightarrow$ site `corp.internal`).
|
||||||
+102
-2
@@ -1,3 +1,103 @@
|
|||||||
# LOGAR Wiki
|
# LOGAR Documentation Wiki
|
||||||
|
|
||||||
Welcome to the LOGAR documentation wiki.
|
Welcome to the official technical documentation wiki for **LOGAR: Fleet Log Aggregation & Centralized Provisioning System**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Navigation & Wiki Index
|
||||||
|
|
||||||
|
1. **[Architecture and Design](Architecture-and-Design)**
|
||||||
|
- Core philosophy & data provisioning model
|
||||||
|
- End-to-end system architecture
|
||||||
|
- Zero-state edge forwarders vs centralized hub
|
||||||
|
- Hermes AI reasoning integration
|
||||||
|
|
||||||
|
2. **[State Machines and Lifecycles](State-Machines-and-Lifecycles)**
|
||||||
|
- End-to-end high-level system state machine
|
||||||
|
- Sub-CA bootstrap & auto-enrollment lifecycle
|
||||||
|
- Live WebSocket synchronization & in-flight hot reload
|
||||||
|
- Forwarder identity discovery & zero-touch enrollment
|
||||||
|
- mTLS 1.3 streaming, proactive renewal & self-healing
|
||||||
|
- 12-hour temporal evaluation window
|
||||||
|
|
||||||
|
3. **[PKI and Fleet Licensing](PKI-and-Licensing)**
|
||||||
|
- Hierarchical PKI architecture & custom X.509 OIDs
|
||||||
|
- Dynamic seat quotas and licensing enforcement
|
||||||
|
- Bootstrap `.lic` package handling
|
||||||
|
- In-flight certificate watchdog & hot SSLContext reloading
|
||||||
|
- WebUI Sub-CA & Licensing dashboard
|
||||||
|
|
||||||
|
4. **[API Reference](API-Reference)**
|
||||||
|
- REST API specifications & JSON request/response schemas
|
||||||
|
- Forwarder enrollment (`/api/client/enroll`)
|
||||||
|
- Seat management & client telemetry (`/api/clients`)
|
||||||
|
- Hermes AI log streaming (`/api/hermes/report`)
|
||||||
|
- Administrator session authentication & health endpoints
|
||||||
|
|
||||||
|
5. **[Deployment and Installation](Deployment-and-Installation)**
|
||||||
|
- Central server hub setup & configuration
|
||||||
|
- Linux client deployment (`install_linux_client.sh`)
|
||||||
|
- Windows client & server setup (Inno Setup & NSSM)
|
||||||
|
- Dynamic machine and domain identification
|
||||||
|
|
||||||
|
6. **[Testing and Releases](Testing-and-Releases)**
|
||||||
|
- Automated unit test suite (35 tests) & end-to-end pipeline test
|
||||||
|
- Continuous Integration (`ci.yml`)
|
||||||
|
- Automated cross-compilation release workflow (`release.yml`)
|
||||||
|
- Local binary packaging & Gitea asset publisher
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## High-Level Architecture Overview
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TB
|
||||||
|
subgraph Authority [Master License Authority - logar.eibl.tech]
|
||||||
|
M_CA[Master Root CA<br/>RSA-4096 / SHA-256]
|
||||||
|
AUTH_WS[WebSocket Licensing Hub<br/>/ws/license]
|
||||||
|
AUTH_DB[(Authority DB<br/>Tenants, Users, 2FA, Seats)]
|
||||||
|
AUTH_UI[Authority WebUI<br/>User Mgmt & Fleet Quotas]
|
||||||
|
AUTH_WS --- AUTH_DB
|
||||||
|
AUTH_UI --- AUTH_DB
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph ServerHub [LOGAR Server Hub - Central Aggregation & Provisioning]
|
||||||
|
SUB_CA[Subordinate CA<br/>Signed with Seat OID 1.3.6.1.4.1.59999.1.1]
|
||||||
|
SEAT_MGR[Dynamic Seat Manager<br/>Enforces Seat Quota in SQLite]
|
||||||
|
MTLS_SRV[mTLS 1.3 Ingestion Server<br/>Port 9443 - ssl.CERT_REQUIRED]
|
||||||
|
TEMP_ENG[12h Temporal Persistence Engine<br/>Correlates First/Last Seen & Run Counts]
|
||||||
|
HUB_DB[(logar_state.db<br/>active_issues, clients, config)]
|
||||||
|
PROV_API[Log Provisioning & Hermes API<br/>Port 8443 - GET /api/hermes/report]
|
||||||
|
|
||||||
|
SUB_CA --> SEAT_MGR
|
||||||
|
SEAT_MGR --> MTLS_SRV
|
||||||
|
MTLS_SRV --> TEMP_ENG
|
||||||
|
TEMP_ENG --> HUB_DB
|
||||||
|
HUB_DB --> PROV_API
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph EdgeNodes [Heterogeneous Edge Forwarders]
|
||||||
|
WIN[Win_Client.exe / Win_Client.py<br/>Windows Event Log Ingestion]
|
||||||
|
LIN[Linux_Client.bin / Linux_Client.py<br/>systemd journalctl -p warning]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Downstream [Hermes AI Reasoning Layer]
|
||||||
|
HERMES[Hermes AI Engine<br/>Autonomous Analysis & Remediation]
|
||||||
|
end
|
||||||
|
|
||||||
|
%% PKI & Licensing Relationships
|
||||||
|
M_CA -->|Signs Sub-CA Cert via /ws/license| SUB_CA
|
||||||
|
AUTH_WS <==>|Bidirectional Sync & Live Seat Expansion| SUB_CA
|
||||||
|
|
||||||
|
%% Forwarder Enrollment & mTLS Transport
|
||||||
|
WIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA
|
||||||
|
LIN -->|1. Auto-Enrollment POST /api/client/enroll| SUB_CA
|
||||||
|
SUB_CA -->|2. Issues Client Cert & Root Trust| WIN
|
||||||
|
SUB_CA -->|2. Issues Client Cert & Root Trust| LIN
|
||||||
|
|
||||||
|
WIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV
|
||||||
|
LIN ==>|3. Continuous mTLS 1.3 Stream :9443| MTLS_SRV
|
||||||
|
|
||||||
|
%% Log Provisioning & Hermes Consumption
|
||||||
|
PROV_API ==>|Centralized Log Delivery| HERMES
|
||||||
|
```
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
# PKI and Fleet Licensing
|
||||||
|
|
||||||
|
LOGAR implements a hierarchical, cryptographically enforced dynamic PKI architecture that seamlessly manages machine authentication, mutual TLS transport security, and enterprise seat licensing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Cryptographic Hierarchy
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
ROOT[Master Root CA<br/>logar.eibl.tech<br/>RSA-4096 / SHA-256]
|
||||||
|
SUB[Subordinate CA<br/>LOGAR Server Hub<br/>RSA-4096 / SHA-256]
|
||||||
|
SRV[Server TLS Certificate<br/>Port 9443 mTLS Listener]
|
||||||
|
CLI_WIN[Client Cert: Win_Client<br/>CN: win-dc-01.corp.internal]
|
||||||
|
CLI_LIN[Client Cert: Linux_Client<br/>CN: db-node-01.corp.internal]
|
||||||
|
|
||||||
|
ROOT -->|Signs with custom OID_LOGAR_SEATS| SUB
|
||||||
|
SUB -->|Issues on first run| SRV
|
||||||
|
SUB -->|Issues on /api/client/enroll| CLI_WIN
|
||||||
|
SUB -->|Issues on /api/client/enroll| CLI_LIN
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Custom Enterprise X.509 OIDs
|
||||||
|
|
||||||
|
LOGAR embeds license metadata directly into the Subordinate CA certificate X.509 extensions under enterprise OID prefix `1.3.6.1.4.1.59999.1`:
|
||||||
|
|
||||||
|
| OID | Constant Name | Description | Example Value |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| `1.3.6.1.4.1.59999.1.1` | `OID_LOGAR_SEATS` | Maximum approved forwarder seat quota | `100` |
|
||||||
|
| `1.3.6.1.4.1.59999.1.2` | `OID_LOGAR_LICENSE_ID` | Unique enterprise license identifier | `LIC-A1B2C3D4E5F6` |
|
||||||
|
| `1.3.6.1.4.1.59999.1.3` | `OID_LOGAR_ISSUER_URL` | WebSocket authority endpoint | `wss://logar.eibl.tech/ws/license` |
|
||||||
|
|
||||||
|
When the LOGAR server loads its certificate chain, it parses these custom extensions using `cryptography.x509` and dynamically enforces the extracted seat limit in its SQLite database (`config.max_seats`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Bootstrap `.lic` Packages
|
||||||
|
|
||||||
|
To initialize a new server hub without manual key transfers, the Master Authority issues a **Bootstrap `.lic` Package** (OpenPGP-encrypted JSON envelope):
|
||||||
|
- **Master Root CA Certificate**: Installs authority trust anchor.
|
||||||
|
- **Dedicated Communication Keypair**: Mutual TLS credentials used strictly for communicating with the Master Authority.
|
||||||
|
- **Sub-CA CSR Template**: Pre-configured Subject Distinguished Names and assigned enterprise tenant.
|
||||||
|
- **Endpoint Coordinates**: Pre-configured WebSocket URI (`wss://logar.eibl.tech/ws/license`).
|
||||||
|
|
||||||
|
### Importing a `.lic` Package
|
||||||
|
- **CLI**: `python src/Server.py --import-lic <package.lic>`
|
||||||
|
- **WebUI**: Navigate to **📜 Sub-CA & Licensing** tab $\rightarrow$ Drag-and-drop or paste `.lic` file.
|
||||||
|
|
||||||
|
Upon import, the server hub connects to the Master Authority over WebSocket, submits its local Sub-CA CSR, verifies hardware fingerprints, receives the signed certificate, and immediately activates its mTLS listeners.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. In-Flight Watchdog & Hot SSLContext Reloading
|
||||||
|
|
||||||
|
The server hub runs a continuous background watchdog coroutine (`cert_validity_watchdog`, running every 12 hours):
|
||||||
|
1. Checks the expiration date of `ca.crt` and `server.crt`.
|
||||||
|
2. If remaining validity is less than 30 days:
|
||||||
|
- Initiates automated certificate renewal with the Master License Authority.
|
||||||
|
- Replaces certificate files atomically on disk.
|
||||||
|
- Dynamically reloads the active `ssl.SSLContext` in-memory.
|
||||||
|
3. Active mTLS client connections and log streams remain completely uninterrupted (Zero Downtime).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. WebUI Sub-CA & Licensing Dashboard
|
||||||
|
|
||||||
|
The Server Hub WebUI includes a dedicated **📜 Sub-CA & Licensing** management tab:
|
||||||
|
- **Authority Connection Badge**: Real-time status indicator of Master Authority WebSocket connection.
|
||||||
|
- **Hardware Fingerprint**: Machine binding fingerprint displayed for license validation.
|
||||||
|
- **Seat Progress Gauge**: Visual representation of active seats leased vs. maximum quota.
|
||||||
|
- **Interactive Actions**:
|
||||||
|
- **Export .lic Request**: Downloads a signed request bundle for air-gapped environments.
|
||||||
|
- **View & Copy CA CSR**: Displays the current Subordinate CA CSR for external signing.
|
||||||
|
- **Sync & Renew via WebSocket**: Triggers immediate upstream synchronization and seat expansion.
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# State Machines and Lifecycles
|
||||||
|
|
||||||
|
This document provides formal state machine diagrams detailing the operational lifecycles of all subsystems across the LOGAR architecture.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. High-Level System State Machine
|
||||||
|
|
||||||
|
Macroscopic overview of the complete end-to-end system lifecycle from initial PKI bootstrapping to AI provisioning:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> 1_PKI_Licensing: Master Authority generates Bootstrap .lic Package
|
||||||
|
1_PKI_Licensing --> 2_Hub_Ready: Server Hub imports .lic & signs Sub-CA over WebSocket
|
||||||
|
2_Hub_Ready --> 3_Edge_Enrollment: Edge Forwarders auto-enroll & lease licensed seats
|
||||||
|
3_Edge_Enrollment --> 4_mTLS_Streaming: Forwarders stream framed logs over mTLS 1.3 (:9443)
|
||||||
|
4_mTLS_Streaming --> 5_Temporal_Aggregation: Server Hub normalizes & correlates logs in 12h window
|
||||||
|
5_Temporal_Aggregation --> 6_AI_Provisioning: Logs provisioned to Hermes AI via GET /api/hermes/report
|
||||||
|
6_AI_Provisioning --> 4_mTLS_Streaming: Continuous operational loop
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Master License Authority & Sub-CA Auto-Enrollment
|
||||||
|
|
||||||
|
Details how a freshly initialized LOGAR Server Hub imports initial credentials, connects to the Master License Authority (`logar.eibl.tech`) via WebSocket, and acquires its signed Sub-CA certificate:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> GeneratedBootstrap: Operator generates Bootstrap .lic Package on Authority
|
||||||
|
GeneratedBootstrap --> ServerImport: Admin imports .lic package on Server Hub
|
||||||
|
ServerImport --> LocalCSR: Server generates local Sub-CA Keypair & CSR
|
||||||
|
LocalCSR --> WSConnecting: Opens WebSocket to wss://logar.eibl.tech/ws/license
|
||||||
|
WSConnecting --> ValidatedCSR: Authority validates hardware fingerprint & CSR
|
||||||
|
ValidatedCSR --> SubCASigned: Authority signs Sub-CA with OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1)
|
||||||
|
SubCASigned --> QuotaActive: Server parses Seat OID & initializes SQLite seat quota
|
||||||
|
QuotaActive --> ServerHubReady: mTLS Listener (Port 9443) & Hermes API (Port 8443) active
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Live WebSocket Heartbeat, Seat Expansion & In-Flight Reload
|
||||||
|
|
||||||
|
Details the continuous synchronization loop, on-demand quota scaling, and hot-reload behavior without dropping client streams:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> ActiveListener: Server Hub actively servicing edge forwarders
|
||||||
|
ActiveListener --> WebSocketHeartbeat: Sends periodic check_updates heartbeat
|
||||||
|
|
||||||
|
state WebSocketHeartbeat <<choice>>
|
||||||
|
WebSocketHeartbeat --> QuotaExpanded: Authority increased seat quota
|
||||||
|
WebSocketHeartbeat --> SteadyState: Quota unchanged / heartbeat ack
|
||||||
|
WebSocketHeartbeat --> Revoked: Authority revoked license_id
|
||||||
|
|
||||||
|
QuotaExpanded --> InFlightReload: Updates SQLite & reloads SSLContext in-flight
|
||||||
|
InFlightReload --> ActiveListener: Hot reload complete (Zero downtime)
|
||||||
|
SteadyState --> ActiveListener
|
||||||
|
Revoked --> DropConnections: Drops all incoming mTLS connections immediately
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Forwarder Identity Discovery & Zero-Touch Enrollment
|
||||||
|
|
||||||
|
Details dynamic machine identity resolution and initial certificate enrollment on Windows and Linux edge forwarders:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> DiscoverIdentity: Edge Client (Win_Client / Linux_Client) launches
|
||||||
|
DiscoverIdentity --> QueryFQDN: socket.getfqdn()
|
||||||
|
QueryFQDN --> ParseDomain: Resolves Windows AD env / Linux resolv.conf
|
||||||
|
ParseDomain --> BuildClientID: Computes FQDN client_id & Site Attribution
|
||||||
|
|
||||||
|
BuildClientID --> CheckCertificates: Inspects local certs/client.crt & certs/ca.crt
|
||||||
|
state CheckCertificates <<choice>>
|
||||||
|
CheckCertificates --> ReadyToStream: Certificates valid & unexpired
|
||||||
|
CheckCertificates --> RequestEnrollment: Certificates missing or expired
|
||||||
|
|
||||||
|
RequestEnrollment --> ValidateQuota: POST /api/client/enroll with secret
|
||||||
|
state ValidateQuota <<choice>>
|
||||||
|
ValidateQuota --> StoreCerts: 200 OK (Seats available & secret valid)
|
||||||
|
ValidateQuota --> [*]: 403 Forbidden (Seat quota full / bad secret)
|
||||||
|
|
||||||
|
StoreCerts --> ReadyToStream
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. mTLS 1.3 Streaming, Proactive Renewal & Reactive Self-Healing
|
||||||
|
|
||||||
|
Details the secure streaming loop, proactive certificate lifecycle checks, and automated recovery against CA rotation:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> mTLSHandshake: Initiates TLS 1.3 Handshake (Port 9443)
|
||||||
|
mTLSHandshake --> CheckClientCN: Server validates client_id in SQLite clients table
|
||||||
|
|
||||||
|
state CheckClientCN <<choice>>
|
||||||
|
CheckClientCN --> ActiveStreaming: Client active in DB (Within seat quota)
|
||||||
|
CheckClientCN --> DropClient: Client seat revoked in DB
|
||||||
|
|
||||||
|
ActiveStreaming --> PeriodicCheck: Checks client.crt validity before each run
|
||||||
|
PeriodicCheck --> ProactiveRenew: Expiry < 30 days (Auto-renews via /api/client/enroll)
|
||||||
|
ProactiveRenew --> mTLSHandshake
|
||||||
|
|
||||||
|
ActiveStreaming --> ReactiveHeal: Handshake error / CA changed (Re-enrolls)
|
||||||
|
ReactiveHeal --> mTLSHandshake
|
||||||
|
DropClient --> [*]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 12-Hour Temporal Window & AI Provisioning
|
||||||
|
|
||||||
|
Details log frame extraction, deduplication, temporal correlation across the 12-hour evaluation window, and delivery to Hermes AI:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
stateDiagram-v2
|
||||||
|
[*] --> EventCaptured: Event captured from Windows Event Log / Linux journalctl
|
||||||
|
EventCaptured --> NoiseStripping: Strips DEBUG, trace entries & logs > 24h old
|
||||||
|
NoiseStripping --> FrameEncoding: Encodes JSON frame ({site}:{server}:{signature})
|
||||||
|
FrameEncoding --> FrameIngested: Streamed over mTLS 1.3 (Port 9443)
|
||||||
|
|
||||||
|
FrameIngested --> EvaluateWindow: Compares (now - last_seen) against 12h window
|
||||||
|
state EvaluateWindow <<choice>>
|
||||||
|
EvaluateWindow --> WithinWindow: (now - last_seen) <= 12 hours
|
||||||
|
EvaluateWindow --> WindowExpired: (now - last_seen) > 12 hours
|
||||||
|
|
||||||
|
WithinWindow --> UpdateRunCount: Increment consecutive_runs, update last_seen
|
||||||
|
WindowExpired --> ResetRunCount: Reset consecutive_runs = 1, update timestamps
|
||||||
|
|
||||||
|
UpdateRunCount --> UpsertDB: Upserts active_issues with status="VERIFIED"
|
||||||
|
ResetRunCount --> UpsertDB
|
||||||
|
|
||||||
|
UpsertDB --> ProvisionReport: Provisioned via GET /api/hermes/report (Port 8443)
|
||||||
|
ProvisionReport --> HermesThinking: Hermes AI executes multi-factor reasoning & actions
|
||||||
|
```
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# Testing and Releases
|
||||||
|
|
||||||
|
This document describes the automated test infrastructure, CI/CD pipeline, and cross-compilation release distribution workflow for LOGAR.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Automated Test Suite
|
||||||
|
|
||||||
|
LOGAR includes comprehensive unit and integration test suites:
|
||||||
|
|
||||||
|
### Running Unit Tests
|
||||||
|
```bash
|
||||||
|
python -m unittest discover -s tests -v
|
||||||
|
```
|
||||||
|
|
||||||
|
The test suite executes **35 unit tests** verifying:
|
||||||
|
- Dynamic Root CA generation and server TLS certificate issuance.
|
||||||
|
- Dynamic client certificate issuance with CN and authority key extensions.
|
||||||
|
- Enrollment secret authentication, seat limits, and certificate revocation.
|
||||||
|
- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload.
|
||||||
|
- Subordinate CA CSR generation, WebSocket enrollment, and dynamic seat quota updates.
|
||||||
|
- Administrator session authentication, PBKDF2 hashing, and Hermes Bearer token lifecycle.
|
||||||
|
- Windows & Linux event log collection, deduplication, and mTLS certificate verification.
|
||||||
|
- 12-hour evaluation window and direct Hermes anomaly verification.
|
||||||
|
|
||||||
|
### End-to-End Pipeline Integration Test
|
||||||
|
```bash
|
||||||
|
# 1. Initialize test configuration
|
||||||
|
python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json
|
||||||
|
|
||||||
|
# 2. Launch server in background
|
||||||
|
python src/Server.py &
|
||||||
|
|
||||||
|
# 3. Run integration test
|
||||||
|
python tests/test_pipeline.py
|
||||||
|
```
|
||||||
|
Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Continuous Integration (`ci.yml`)
|
||||||
|
|
||||||
|
Every push or pull request to `main` triggers `.gitea/workflows/ci.yml`:
|
||||||
|
- Checks Python code syntax.
|
||||||
|
- Runs all 35 unit tests in an isolated environment.
|
||||||
|
- Executes end-to-end mTLS pipeline integration tests.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Releases & Binary Cross-Compilation (`release.yml`)
|
||||||
|
|
||||||
|
Releases are published automatically upon pushing a git tag:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git tag v3.0.1
|
||||||
|
git push origin v3.0.1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Linux-Based Cross-Compilation Pipeline
|
||||||
|
The workflow runs on a single `ubuntu-latest` runner executing `python compilation/package_dist.py --target all`:
|
||||||
|
1. **Linux Binaries**: Standalone native ELF executables (`Linux_Client.bin`, `Server.bin`).
|
||||||
|
2. **Windows Executables & Bundles**: Cross-compiled standalone binaries (`Win_Client.exe`, `Server.exe`) and `.zip` distribution packages.
|
||||||
|
3. **SHA256 Checksums**: Generated automatically as `sha256sums.txt`.
|
||||||
|
4. **Gitea Release Publication**: `upload_release.py` uploads all compiled assets and changelogs directly to the Gitea release page.
|
||||||
|
|
||||||
|
### Manual Workflow Dispatch
|
||||||
|
Releases can also be triggered on demand via the Gitea UI:
|
||||||
|
1. Navigate to **Actions** $\rightarrow$ **Release Binaries & Installers** (`release.yml`).
|
||||||
|
2. Click **Run workflow**, specify the release tag (e.g. `v3.0.1`), and execute.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Local Packaging & Asset Publishing
|
||||||
|
|
||||||
|
To build binaries and upload releases directly from a developer workstation:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 1. Build all target binaries locally
|
||||||
|
python compilation/package_dist.py --target all
|
||||||
|
|
||||||
|
# 2. Upload assets and release notes to Gitea
|
||||||
|
python compilation/upload_release.py --tag v3.0.1 --token <GITEA_TOKEN> --skip-build
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user