From c1d4675321024963199e713eca0717b182d82211 Mon Sep 17 00:00:00 2001 From: max Date: Wed, 30 Sep 2026 17:58:35 +0200 Subject: [PATCH] docs: add administrator guide, update self-tests to 51 passing, update documentation and release notes for v0.3.5 --- .gitignore | 2 + README.md | 19 ++- RELEASE_NOTES.md | 29 ++++ docs/ADMIN_GUIDE.md | 369 +++++++++++++++++++++++++++++++++++++++++++ tests/test_server.py | 104 ++++++++++++ wiki/ADMIN_GUIDE.md | 369 +++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 885 insertions(+), 7 deletions(-) create mode 100644 docs/ADMIN_GUIDE.md create mode 100644 wiki/ADMIN_GUIDE.md diff --git a/.gitignore b/.gitignore index 2d4067f..042b984 100644 --- a/.gitignore +++ b/.gitignore @@ -27,6 +27,8 @@ server_config.json client_config.json client_state.json local_collector_state.json +certs/ +*.bak test_*.json test_*.db test_pipeline_master_ca/ diff --git a/README.md b/README.md index 1899a7f..eb60454 100644 --- a/README.md +++ b/README.md @@ -5,13 +5,14 @@ # LOGAR: Fleet Log Aggregation & Centralized Provisioning System ### *Secure Log Aggregation, PKI Fleet Licensing & Centralized Delivery for Autonomous AI Analysis* -[![Version](https://img.shields.io/badge/version-0.3.4.2-blue.svg)](RELEASE_NOTES.md) -[![Tests](https://img.shields.io/badge/tests-44%20passed-success.svg)](tests/) +[![Version](https://img.shields.io/badge/version-0.3.5-blue.svg)](RELEASE_NOTES.md) +[![Tests](https://img.shields.io/badge/tests-51%20passed-success.svg)](tests/) [![Security](https://img.shields.io/badge/mTLS-TLS%201.3-green.svg)](https://gitea.eibl.tech/me0nline/LOGAR/wiki/PKI-and-Licensing) +[![Admin Guide](https://img.shields.io/badge/docs-Admin%20Guide-blueviolet.svg)](docs/ADMIN_GUIDE.md) **LOGAR** is an enterprise-grade log aggregation, normalization, and centralized provisioning system built for heterogeneous server fleets (Windows & Linux). It pairs ultra-lightweight, zero-state edge forwarders with a secure centralized server hub that enforces mutual TLS 1.3 (**mTLS**) authentication, dynamic PKI fleet licensing with cryptographic seat leasing, 12-hour temporal persistence tracking, and direct anomaly pass-throughβ€”delivering clean, structured, verified log telemetry directly to **Hermes AI** for autonomous deep reasoning, contextual root-cause analysis, and operational decision-making. -> πŸ“š **Comprehensive Technical Documentation**: Full architectural specifications, state machines, PKI X.509 OID definitions, branding assets, and complete REST API references are maintained on the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)**. +> πŸ“š **Comprehensive Administrator & Operations Guide**: Complete setup, database switching, ingestion queue tuning, zero-knowledge OpenPGP encryption, security runbooks, and REST API references are available in the **[Admin Guide (docs/ADMIN_GUIDE.md)](docs/ADMIN_GUIDE.md)** and mirrored on the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki/ADMIN_GUIDE)**. --- @@ -19,10 +20,13 @@ - **Zero-State Edge Forwarders**: Windows (`Win_Client`) and Linux (`Linux_Client`) agents maintain zero local databases, zero cache files, and minimal memory footprint. - **Mutual TLS 1.3 (`:9443`)**: Enforces bidirectional cryptographic identity verification (`ssl.CERT_REQUIRED`) for all streaming log frames. -- **Hierarchical PKI & Seat Licensing**: Subordinate CAs automatically enroll and sync seat quotas with the Master Authority (`logar.eibl.tech`) over WebSockets. +- **Hierarchical PKI & Seat Licensing**: Subordinate CAs automatically enroll and sync seat quotas with the Master Authority (`logar.eibl.tech`) over WebSockets. Active module capabilities (e.g. `enterprise_db`) are introspected dynamically. +- **Multi-Engine Storage Backend**: Switch seamlessly between lightweight local SQLite (WAL mode) and external enterprise databases (PostgreSQL, MySQL, MariaDB) with automated schema migration. +- **High-Throughput Ring Buffer**: In-memory ingestion queue with configurable batch flushing to eliminate disk I/O bottlenecks during peak traffic bursts. +- **Zero-Knowledge Data Encryption**: End-to-end payload encryption at rest using server X.509 RSA-2048 / OpenPGP keys for external SQL databases. - **12-Hour Temporal Persistence**: Evaluates anomalies over a rolling 12-hour temporal window, tracking occurrence velocity and first/last seen metrics. - **Direct Hermes AI Provisioning (`:8443`)**: Exposes verified anomalous telemetry directly to Hermes AI via `/api/hermes/report` without heuristic bottlenecks. -- **WebUI Dashboard**: Authenticated, 2-tab management console for live anomaly telemetry, client seat leasing, and Sub-CA lifecycle management. +- **WebUI Dashboard**: Authenticated, clean management console for live anomaly telemetry, client seat leasing, modular licensing status, storage engine configuration, and Sub-CA lifecycle management. --- @@ -157,7 +161,7 @@ LOGAR/ ## Testing & Cross-Compilation ```bash -# Run full unit test suite (42 tests) +# Run full unit test suite (51 tests) python -m unittest discover -s tests -v # Run end-to-end mTLS pipeline integration test @@ -171,8 +175,9 @@ python compilation/package_dist.py --target all ## Documentation & Deep Dives -For in-depth operational guides, lifecycle state machines, and PKI specifications, consult the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)**: +For in-depth operational guides, lifecycle state machines, and PKI specifications, consult the **[LOGAR Gitea Wiki](https://gitea.eibl.tech/me0nline/LOGAR/wiki)** and local documentation: +* [Administrator & Operations Guide](docs/ADMIN_GUIDE.md) (*Mirrored at [Wiki: ADMIN_GUIDE](https://gitea.eibl.tech/me0nline/LOGAR/wiki/ADMIN_GUIDE)*) * [Architecture & Design Details](https://gitea.eibl.tech/me0nline/LOGAR/wiki/Architecture-and-Design) * [System State Machines & Subsystem Lifecycles](https://gitea.eibl.tech/me0nline/LOGAR/wiki/State-Machines-and-Lifecycles) * [PKI Architecture & Custom X.509 Seat OIDs](https://gitea.eibl.tech/me0nline/LOGAR/wiki/PKI-and-Licensing) diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 2f4bbb5..a52a5c3 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,5 +1,34 @@ # LOGAR Release Notes +## [v0.3.5] - 2026-09-30 + +### πŸ—„οΈ Enterprise Multi-Engine Storage Architecture +- **Dynamic Backend Switching**: LOGAR Server Hub now supports runtime switching between lightweight Local SQLite (WAL Mode) and Enterprise SQL servers (PostgreSQL and MySQL / MariaDB) via SQLAlchemy. +- **Automated Schema Initialization**: Switching to an external database automatically verifies connectivity, provisions normalized tables (`telemetry_events`, `client_seats`, `license_config`, `admin_users`), creates optimized indices, and runs health verification. +- **Configuration Persistence**: Persistent backend settings are saved in `server_config.json` under `db_engine` and isolated from temporary test environments. + +### ⚑ Ingestion Ring Buffer & High-Throughput Batch Flusher +- **In-Memory Queue Buffering**: Built-in Python `queue.Queue` ring buffer with background worker thread decoupled from socket I/O. +- **Burst Protection & Cache Mode**: High-velocity bursts are accepted without blocking mTLS 1.3 socket listeners; configurable batch size (`buffer_batch_size`, default 100) and flush interval (`buffer_flush_interval_sec`, default 2.0s). +- **Graceful Shutdown**: Unflushed buffer entries are cleanly committed to storage on server shutdown. +- **Explicit Opt-In Administrative Control**: Ingestion buffering is strictly optional and requires explicit administrative activation (never auto-enabled upon license activation). + +### πŸ” Zero-Knowledge OpenPGP Data Encryption at Rest +- **Public-Key Cryptography**: Enterprise SQL payloads (`event_data`) can be encrypted at rest using the server's X.509 RSA-2048 certificate / OpenPGP public key via AES-256-GCM symmetric session keys and RSA-OAEP key encapsulation. +- **Zero-Knowledge on Database Host**: External database administrators or compromised SQL servers cannot read raw telemetry payloads without the private key located on the LOGAR Server Hub. +- **Explicit Opt-In Security Model**: Payload encryption is strictly optional and configurable per compliance requirements. + +### πŸ“œ Dynamic Licensing Introspection & Clean Dashboard WebUI +- **Modular Capability Introspection**: PKI X.509 extension OID `1.3.6.1.4.1.58888.1.3` and persistent license records are dynamically parsed to expose active modules (`enterprise_db`, etc.) in a dedicated card on the **πŸ“œ Sub-CA & Licensing** tab. +- **Distraction-Free Tab Navigation**: Cleaned all tab headers to remove redundant item counts and visual badges for a streamlined administrative interface. +- **Graceful Feature Locking**: Unlicensed enterprise features remain visible but gracefully disabled in the WebUI and rejected by backend API validation. + +### πŸ“– Comprehensive Administrator Guide & Expanded Test Suite +- **Administrator & Operations Guide**: Published comprehensive 11-chapter guide in [`docs/ADMIN_GUIDE.md`](docs/ADMIN_GUIDE.md) and [`wiki/ADMIN_GUIDE.md`](wiki/ADMIN_GUIDE.md) covering deployment, database switching, queue tuning, zero-knowledge crypto, forwarders, Hermes AI, 2FA, and troubleshooting. +- **Automated Unit & Integration Tests**: Expanded test suite to **51 automated tests** covering dynamic licensing, WebUI rendering, opt-in isolation, database failovers, and mTLS pipelines. + +--- + ## [v0.3.4.2] - 2026-09-30 ### πŸ”’ Exclusive Hardware Host-Binding & Reassignment Controls diff --git a/docs/ADMIN_GUIDE.md b/docs/ADMIN_GUIDE.md new file mode 100644 index 0000000..f579ad0 --- /dev/null +++ b/docs/ADMIN_GUIDE.md @@ -0,0 +1,369 @@ +# LOGAR Administrator & Operations Guide +### *Complete Operations Manual for Fleet Hub Administration, Database Engines, PKI Licensing & AI Provisioning* + +--- + +## Table of Contents + +1. [Executive Summary & System Architecture](#1-executive-summary--system-architecture) +2. [Database Storage Engine Management](#2-database-storage-engine-management) + - [Local SQLite 3 (WAL Mode)](#local-sqlite-3-wal-mode) + - [External PostgreSQL Enterprise Backend](#external-postgresql-enterprise-backend) + - [Automated SQLite to PostgreSQL Migration](#automated-sqlite-to-postgresql-migration) +3. [Telemetry Ingestion Cache & Buffer Tuning](#3-telemetry-ingestion-cache--buffer-tuning) + - [Ingestion Modes: Direct Write vs In-Memory Queue Buffer](#ingestion-modes-direct-write-vs-in-memory-queue-buffer) + - [Tuning Parameters & Metrics](#tuning-parameters--metrics) + - [Manual Flush & Buffer Monitoring](#manual-flush--buffer-monitoring) +4. [Zero-Knowledge OpenPGP Data Encryption at Rest](#4-zero-knowledge-openpgp-data-encryption-at-rest) + - [Encryption Mechanism](#encryption-mechanism) + - [Transparent Authorized Decryption](#transparent-authorized-decryption) +5. [Subordinate CA & Dynamic PKI Licensing](#5-subordinate-ca--dynamic-pki-licensing) + - [Hierarchical PKI & Custom X.509 OIDs](#hierarchical-pki--custom-x509-oids) + - [Licensed Feature Capabilities (`enterprise_db`)](#licensed-feature-capabilities-enterprise_db) + - [Importing `.lic` Activation Packages](#importing-lic-activation-packages) + - [Automated mTLS License Renewal](#automated-mtls-license-renewal) +6. [Fleet Forwarder Management & Seat Provisioning](#6-fleet-forwarder-management--seat-provisioning) + - [Zero-Touch Linux Deployment](#zero-touch-linux-deployment) + - [Windows Edge Client Deployment](#windows-edge-client-deployment) + - [Seat Lifecycle: Approve, Revoke, Delete](#seat-lifecycle-approve-revoke-delete) +7. [Hermes AI Agent Integration](#7-hermes-ai-agent-integration) + - [Dedicated Bearer Tokens](#dedicated-bearer-tokens) + - [12-Hour Temporal Persistence Anomaly Stream](#12-hour-temporal-persistence-anomaly-stream) +8. [Security, Authentication & 2FA](#8-security-authentication--2fa) + - [Password Policies & PBKDF2 Hashing](#password-policies--pbkdf2-hashing) + - [Time-Based One-Time Password (TOTP 2FA)](#time-based-one-time-password-totp-2fa) + - [Sliding Window IP Rate Limiting](#sliding-window-ip-rate-limiting) +9. [Configuration Reference & CLI Flags](#9-configuration-reference--cli-flags) +10. [REST API Reference](#10-rest-api-reference) +11. [Operational Runbook & Troubleshooting](#11-operational-runbook--troubleshooting) + +--- + +## 1. Executive Summary & System Architecture + +LOGAR is a centralized log aggregation, normalization, and provisioning system designed for enterprise server fleets. It connects heterogeneous edge nodes (Windows and Linux) with a central server hub over **Mutual TLS 1.3 (mTLS)**, normalizing incoming events and presenting structured anomaly intelligence to **Hermes AI**. + +### Core Network Architecture + +```mermaid +graph TD + subgraph MasterAuthority [Master License Authority] + ROOT_CA[Master Root CA] + AUTH_API[WebSocket Licensing Hub
logar.eibl.tech] + end + + subgraph Hub [LOGAR Server Hub] + SUB_CA[Subordinate CA & PKI Engine] + INGEST_QUEUE[IngestQueueManager
In-Memory RAM Buffer] + STORAGE_ROUTER[Storage Engine Router] + SQLITE[(Local SQLite 3 WAL)] + POSTGRES[(External PostgreSQL Cluster)] + TEMP_ENG[12h Temporal Persistence Engine] + HERMES_API[Hermes REST & WebUI API
Port 8443] + MTLS_SOCKET[mTLS 1.3 Socket Listener
Port 9443] + + MTLS_SOCKET --> INGEST_QUEUE + INGEST_QUEUE --> STORAGE_ROUTER + STORAGE_ROUTER --> SQLITE + STORAGE_ROUTER --> POSTGRES + STORAGE_ROUTER --> TEMP_ENG + TEMP_ENG --> HERMES_API + end + + subgraph EdgeNodes [Fleet Edge Forwarders] + WIN_NODE[Windows Forwarder
Win_Client.exe] + LIN_NODE[Linux Forwarder
Linux_Client.bin] + end + + subgraph AI [Hermes AI Analysis Layer] + HERMES_AGENT[Hermes AI Agent] + end + + ROOT_CA -->|Signs Sub-CA Cert| SUB_CA + AUTH_API <==>|Live mTLS Renewal| SUB_CA + + WIN_NODE ==>|1. Auto-Enrollment / mTLS Ingest| MTLS_SOCKET + LIN_NODE ==>|1. Auto-Enrollment / mTLS Ingest| MTLS_SOCKET + + HERMES_API ==>|GET /api/hermes/report| HERMES_AGENT +``` + +--- + +## 2. Database Storage Engine Management + +LOGAR supports dual storage backends: an embedded **Local SQLite 3 (WAL Mode)** engine and an **External PostgreSQL Database Server** for enterprise clustering. + +### Local SQLite 3 (WAL Mode) +- **Status**: Standard Default (Included out of the box, zero configuration required). +- **Optimization**: Operates with Write-Ahead Logging (`PRAGMA journal_mode=WAL;`), asynchronous normal synchronization (`PRAGMA synchronous=NORMAL;`), and memory-mapped I/O (`PRAGMA mmap_size=268435456;`). +- **File Location**: Configurable via `db_path` in `server_config.json` (defaults to `logar_state.db`). + +### External PostgreSQL Enterprise Backend +- **Requirement**: Unlocked via the `enterprise_db` license capability. +- **Benefits**: Enables horizontal database scalability, clustering, multi-node replication, and centralized enterprise backup policies. +- **Connection Parameters**: + - **Host / IP**: Database hostname or IP address. + - **Port**: Default `5432`. + - **Database**: Target schema/database name (default `logar`). + - **Username / Password**: Authentication credentials. + - **SSL Mode**: Supports `prefer`, `require`, `verify-ca`, `verify-full`, `disable`, and `allow`. + - **Connect Timeout**: Timeout in seconds (default `5`s). + - **Connection Pool Size**: Maximum pooled persistent connections (default `10`). + +### Automated SQLite to PostgreSQL Migration +Administrators can migrate all historical logs, runs, active issues, seat allocations, users, and tokens from SQLite to PostgreSQL directly in the WebUI (`πŸ’Ύ DB Engine` -> `πŸš€ Migrate SQLite Data to PostgreSQL`) or via API (`POST /api/db/migrate-to-postgres`): +1. Flushes any in-memory telemetry buffers to SQLite. +2. Connects to PostgreSQL and verifies/initializes the schema. +3. Automatically transfers records table-by-table with validation. +4. Returns a detailed count of all migrated records. + +--- + +## 3. Telemetry Ingestion Cache & Buffer Tuning + +### Ingestion Modes: Direct Write vs In-Memory Queue Buffer + +| Feature | ⚑ Immediate Direct Write | πŸš€ In-Memory Queue Buffer (`enterprise_db`) | +| :--- | :--- | :--- | +| **Ingress Latency** | Low latency, immediate DB commit | Microsecond ingestion, non-blocking | +| **Throughput** | Up to 1,500 events/sec | Up to 50,000+ events/sec | +| **Disk I/O** | Continuous write cycles | Atomic batch flushing | +| **License** | Standard Community Edition | Enterprise License (`enterprise_db`) | + +### Tuning Parameters & Metrics +When **In-Memory Queue Buffer** is enabled, administrators can tune the buffer engine: +- **Flush Interval (seconds)**: Maximum elapsed time before buffered anomalies are written to storage (e.g. `2.0`s). +- **Batch Size Threshold**: Number of buffered events that triggers an immediate atomic flush (e.g. `100` events). +- **Max Buffer Capacity**: Upper ceiling of buffered items in memory (e.g. `50,000` events) to protect system RAM under heavy load. + +### Manual Flush & Buffer Monitoring +- **Live Monitoring**: The KPI card displays live buffer size, total events ingested, total flushed, and average flush latency. +- **Manual Trigger**: Click **`🌊 Flush Buffer Now`** or call `POST /api/db/flush-buffer` to force an immediate buffer drain. + +--- + +## 4. Zero-Knowledge OpenPGP Data Encryption at Rest + +### Encryption Mechanism +When **Data Encryption at Rest** is enabled (`encrypt_at_rest: true`): +- All sensitive log payloads, anomaly messages, and raw traces are encrypted with the server's OpenPGP public key before writing to the database table (`active_issues.message`). +- Remote database administrators or third-party database hosting providers only see armored OpenPGP ciphertext (`-----BEGIN PGP MESSAGE-----`). + +### Transparent Authorized Decryption +- The central LOGAR hub instance holds the private OpenPGP key in protected memory. +- When authenticated administrators access the WebUI or when authorized Hermes AI instances query `/api/hermes/report`, the server transparently decrypts records on-the-fly. + +--- + +## 5. Subordinate CA & Dynamic PKI Licensing + +LOGAR utilizes a hierarchical Public Key Infrastructure (PKI) for cryptographic client identity and license seat enforcement. + +### Hierarchical PKI & Custom X.509 OIDs +The Master Authority (`logar.eibl.tech`) signs the hub's Subordinate CA certificate (`certs/ca.crt`), embedding custom ASN.1 OIDs: +- **`1.3.6.1.4.1.58888.1.1` (Seat Quota)**: Encodes the authorized edge client seat capacity (e.g. `25` seats). +- **`1.3.6.1.4.1.58888.1.2` (Master Authority URL)**: Identifies the upstream licensing authority. +- **`1.3.6.1.4.1.58888.1.3` (Feature Capabilities)**: Comma-delimited list of granted features (e.g. `enterprise_db,ai_heuristics`). + +### Licensed Feature Capabilities (`enterprise_db`) +The **πŸ“œ Sub-CA & Licensing** tab displays all active licensed features: +- **`enterprise_db`**: Authorizes external PostgreSQL database pooling, high-throughput in-memory RAM buffering, and OpenPGP data-at-rest encryption. + +### Importing `.lic` Activation Packages +1. In the WebUI, navigate to **πŸ“œ Sub-CA & Licensing** and click **`πŸ“₯ Import License (.lic)`**. +2. Select your `.lic` bundle or paste the JSON package. +3. The server immediately reloads the PKI trust chain and hot-reloads the active mTLS `SSLContext` without downtime. + +### Automated mTLS License Renewal +- Click **`πŸ”„ Sync & Renew via mTLS`** in the WebUI or run: + ```bash + python src/Server.py --sync-license + ``` +- The hub connects to `logar.eibl.tech` over mTLS, sends its current seat utilization, and retrieves an updated signed CA certificate. + +--- + +## 6. Fleet Forwarder Management & Seat Provisioning + +### Zero-Touch Linux Deployment +Run the automated single-line installer on target Linux servers: +```bash +sudo ./compilation/install_linux_client.sh "http://:8443" "" +``` +- Automatically generates a 2048-bit RSA client keypair. +- Enrolls with the hub and receives a signed client certificate. +- Deploys and starts the `logar-client.service` systemd unit. + +### Windows Edge Client Deployment +1. Download and run `LOGAR-Client-Setup.exe` on the Windows host. +2. In the Server WebUI, click **`πŸ“₯ Export Client Profile`** and download `client_config.json`. +3. Place `client_config.json` into `C:\Program Files\LOGAR\client_config.json`. +4. The `LOGAR_Client` service automatically boots and connects via mTLS. + +### Seat Lifecycle: Approve, Revoke, Delete +In the **πŸ’» Fleet Forwarders** tab: +- **Revoke**: Temporarily blocks the node at the TLS handshake level (`ssl.CERT_REQUIRED`). +- **Approve**: Re-activates a revoked node and restores telemetry ingestion. +- **Remove**: Permanently deletes the client record and immediately frees the leased seat quota. + +--- + +## 7. Hermes AI Agent Integration + +### Dedicated Bearer Tokens +1. Navigate to **πŸ€– Hermes AI & Tokens** in the WebUI. +2. Enter a descriptive token name (e.g. `Hermes-Autonomous-SecOps`) and click **`Generate Token`**. +3. Copy the Bearer token for Hermes agent configuration. + +### 12-Hour Temporal Persistence Anomaly Stream +Hermes queries the real-time anomaly stream: +```bash +curl -H "Authorization: Bearer hermes_xxxxxxxxxxxx" http://:8443/api/hermes/report +``` + +#### JSON Telemetry Format: +```json +[ + { + "fingerprint": "corp-dc:srv-db-01:postgres_wal_checkpoint_delay", + "site": "corp-dc", + "server": "srv-db-01", + "signature": "postgres_wal_checkpoint_delay", + "severity": "WARNING", + "message": "WAL checkpoint write duration exceeded 30000ms threshold", + "os_type": "linux", + "first_seen": "2026-09-30 14:10:00", + "last_seen": "2026-09-30 17:42:15", + "consecutive_runs": 4, + "evaluation_window": "12h", + "verified": true, + "status": "VERIFIED" + } +] +``` + +--- + +## 8. Security, Authentication & 2FA + +### Password Policies & PBKDF2 Hashing +- User passwords are protected using PBKDF2-HMAC-SHA256 with 100,000 iterations and 16-byte random salts. +- Change admin credentials via the user avatar menu -> **πŸ”‘ Change Password**. + +### Time-Based One-Time Password (TOTP 2FA) +- Standard RFC 6238 TOTP with standard 30-second drift windows. +- Configure via avatar menu -> **πŸ›‘οΈ Two-Factor Auth (2FA)**. +- Compatible with Google Authenticator, Authy, 1Password, and Bitwarden. + +### Sliding Window IP Rate Limiting +- **Brute-Force Protection**: Enforces rate limiting on `/login` and `/api/client/enroll`. +- Excess failed attempts trigger automatic IP blocking with sliding exponential backoff. + +--- + +## 9. Configuration Reference & CLI Flags + +### `server_config.json` Specification + +```json +{ + "server_name": "LOGAR-Cloud-Hub", + "tcp_host": "0.0.0.0", + "tcp_port": 9443, + "hermes_host": "0.0.0.0", + "hermes_port": 8443, + "admin_user": "admin", + "enrollment_secret": "9a7f8e...4b", + "max_seats": 25, + "license_status": "active", + "license_server_url": "https://logar.eibl.tech", + "cert_dir": "certs", + "tls_enabled": true, + "db_path": "logar_state.db", + "storage_engine": "sqlite", + "cache_mode": "direct", + "flush_interval": 2.0, + "batch_size": 100, + "max_queue": 50000, + "encrypt_at_rest": false, + "server_fingerprint": "6FF5BE81...", + "public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...", + "private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..." +} +``` + +### CLI Arguments + +| Flag | Description | +| :--- | :--- | +| `--config ` | Path to server configuration JSON file | +| `--create-client-config` | Exports a client config bundle and exits | +| `--client-out ` | Destination path for exported client configuration | +| `--server-host ` | Embedded target host for exported client configuration | +| `--server-port ` | Embedded target port for exported client configuration | +| `--export-lic ` | Exports a `.lic` CSR activation request package | +| `--import-lic ` | Imports and activates a signed `.lic` license package | +| `--sync-license` | Connects to master authority to renew certificate & seats | + +--- + +## 10. REST API Reference + +| Method | Endpoint | Auth | Description | +| :--- | :--- | :--- | :--- | +| `GET` | `/health` | None | Health check, server name, and TLS status | +| `POST` | `/api/client/enroll` | Secret | Edge forwarder mTLS enrollment & certificate issuance | +| `GET` | `/api/clients` | Admin | List registered forwarders, activity times, and seat quota | +| `POST` | `/api/clients/approve` | Admin | Reactivate / approve an enrolled forwarder seat | +| `POST` | `/api/clients/revoke` | Admin | Revoke an enrolled forwarder seat | +| `POST` | `/api/clients/delete` | Admin | Remove forwarder record and permanently free seat | +| `GET` | `/api/hermes/report` | Hermes/Admin | Stream verified anomalies within 12h temporal window | +| `POST` | `/api/hermes/tokens/create` | Admin | Generate a named Bearer token for Hermes AI | +| `POST` | `/api/hermes/tokens/revoke` | Admin | Revoke an existing Hermes Bearer token | +| `GET` | `/api/db/stats` | Admin | Telemetry buffer stats, queue metrics, and storage info | +| `POST` | `/api/db/flush-buffer` | Admin | Force immediate drain and write of buffered telemetry | +| `POST` | `/api/db/test-connection` | Admin | Test external PostgreSQL connectivity (`enterprise_db`) | +| `POST` | `/api/db/switch-backend` | Admin | Switch backend between SQLite and PostgreSQL (`enterprise_db`) | +| `POST` | `/api/db/config` | Admin | Persist DB engine, cache mode, and encryption settings | +| `POST` | `/api/db/migrate-to-postgres` | Admin | Migrate SQLite tables to PostgreSQL (`enterprise_db`) | +| `GET` | `/api/license/status` | Admin | Subordinate CA status, seat quota, and licensed features | +| `POST` | `/api/license/import` | Admin | Import `.lic` bundle or signed CA PEM | +| `POST` | `/api/license/sync` | Admin | Sync license seats with master authority over mTLS | + +--- + +## 11. Operational Runbook & Troubleshooting + +### Diagnostic Checklist + +1. **Verify Hub Service Status**: + ```bash + # Linux + sudo systemctl status logar-server.service + # Windows PowerShell + Get-Service LOGAR_Server + ``` +2. **Check Health Endpoint**: + ```bash + curl http://localhost:8443/health + ``` +3. **Inspect Active Anomaly Telemetry**: + ```bash + curl -H "Authorization: Bearer " http://localhost:8443/api/hermes/report + ``` +4. **Certificate Validity Check**: + ```bash + openssl x509 -in certs/ca.crt -text -noout | grep -A 2 "Validity" + ``` + +### Common Issues & Remediation + +- **Issue: Forwarder rejected with `Connection Refused` or `TLS Handshake Failure`**: + - Ensure port `9443` is reachable through host and cloud firewalls. + - Verify edge node clock synchronization (NTP) with the server hub. + - Confirm the client certificate has not been revoked in the **Fleet Forwarders** tab. +- **Issue: `403 Forbidden` on PostgreSQL or Buffering settings**: + - The `enterprise_db` feature capability is required. Verify your license in **πŸ“œ Sub-CA & Licensing**. +- **Issue: Rate limit lock on admin login**: + - Wait 60 seconds for the sliding attempt window to expire, or authenticate via an existing active session. diff --git a/tests/test_server.py b/tests/test_server.py index 5da9457..a448b97 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -778,6 +778,110 @@ class TestServerComponent(unittest.TestCase): self.assertTrue(info["encrypt_at_rest"]) self.assertEqual(info["flush_interval"], 4.0) + def test_active_licensed_features_helper(self): + """Tests helper function to extract active licensed capabilities from DB and certificates.""" + Server.init_db(self.test_db) + Server.SERVER_STATE["ca_cert_pem"] = "" + Server.SERVER_STATE["config"] = {"db_path": self.test_db, "cert_dir": "non_existent_test_certs"} + + # 1. Unlicensed state + features_unlicensed = Server.get_active_licensed_features(db_path=self.test_db) + self.assertEqual(features_unlicensed, []) + self.assertFalse(Server.is_feature_licensed("enterprise_db", db_path=self.test_db)) + + # 2. Add features via JSON string in DB + conn = sqlite3.connect(self.test_db) + conn.execute("UPDATE license_config SET features = ? WHERE id = 1", (json.dumps(["enterprise_db", "ai_heuristics"]),)) + conn.commit() + conn.close() + + features_licensed = Server.get_active_licensed_features(db_path=self.test_db) + self.assertEqual(features_licensed, ["ai_heuristics", "enterprise_db"]) + self.assertTrue(Server.is_feature_licensed("enterprise_db", db_path=self.test_db)) + self.assertTrue(Server.is_feature_licensed("ai_heuristics", db_path=self.test_db)) + self.assertFalse(Server.is_feature_licensed("non_existent_feature", db_path=self.test_db)) + + # 3. Verify get_license_summary includes features list + summary = Server.get_license_summary() + self.assertIn("features", summary) + self.assertIn("enterprise_db", summary["features"]) + + def test_webui_tab_rendering_and_feature_cards(self): + """Tests WebUI HTML generation for clean tab buttons and feature cards in PKI tab.""" + Server.init_db(self.test_db) + Server.SERVER_STATE["config"] = { + "db_path": self.test_db, + "cert_dir": "certs", + "tcp_port": 9443, + "hermes_port": 8443, + "server_name": "Test-Hub" + } + + # 1. Render WebUI in standard / unlicensed mode + resp = Server.serve_webui(request=None) + html_content = resp.body.decode("utf-8") + + # Verify clean tab buttons without number suffixes + self.assertIn('>πŸ’» Fleet Forwarders', html_content) + self.assertIn('>πŸ€– Hermes AI & Tokens', html_content) + self.assertIn('>πŸ“Š Verified Telemetry', html_content) + self.assertIn('>πŸ“œ Sub-CA & Licensing', html_content) + self.assertIn('>πŸ’Ύ DB Engine', html_content) + self.assertNotIn('🟒 UNLOCKED', html_content) + self.assertNotIn('Fleet Forwarders (', html_content) + + # Verify Licensed Capabilities card is present + self.assertIn('Licensed Capabilities & Feature Modules', html_content) + self.assertIn('Standard Community Edition is running', html_content) + + # 2. Render WebUI with enterprise_db unlocked + conn = sqlite3.connect(self.test_db) + conn.execute("UPDATE license_config SET features = ? WHERE id = 1", (json.dumps(["enterprise_db"]),)) + conn.commit() + conn.close() + + resp_ent = Server.serve_webui(request=None) + html_ent = resp_ent.body.decode("utf-8") + self.assertIn('enterprise_db', html_ent) + self.assertIn('Enterprise Database & High-Throughput Engine', html_ent) + self.assertIn('>πŸ’Ύ DB Engine', html_ent) + self.assertNotIn('🟒 UNLOCKED', html_ent) + + def test_explicit_opt_in_guarantee(self): + """Tests that licensing does NOT automatically turn on encryption or buffering without explicit admin setting.""" + Server.init_db(self.test_db) + priv_armored, pub_armored, fp = Server.generate_server_keypair("test-opt-in-node") + priv_key, _ = pgpy.PGPKey.from_blob(priv_armored) + pub_key, _ = pgpy.PGPKey.from_blob(pub_armored) + + Server.SERVER_STATE["config"] = { + "db_path": self.test_db, + "cert_dir": "certs", + "storage_engine": "sqlite", + "cache_mode": "direct", + "encrypt_at_rest": False, + "private_key": priv_armored, + "public_key": pub_armored, + "server_fingerprint": fp + } + Server.SERVER_STATE["private_key_obj"] = priv_key + Server.SERVER_STATE["public_key_obj"] = pub_key + + # Unlock enterprise license + conn = sqlite3.connect(self.test_db) + conn.execute("UPDATE license_config SET features = ? WHERE id = 1", (json.dumps(["enterprise_db"]),)) + conn.commit() + conn.close() + + # License is active, but config has encrypt_at_rest=False -> plaintext storage + raw_msg = "Sample operational log event" + result_msg = Server.encrypt_field_for_storage(raw_msg) + self.assertEqual(result_msg, raw_msg) + self.assertFalse(result_msg.startswith("-----BEGIN PGP MESSAGE-----")) + + # Ingestion cache mode is direct + self.assertEqual(Server.SERVER_STATE["config"]["cache_mode"], "direct") + if __name__ == "__main__": unittest.main() diff --git a/wiki/ADMIN_GUIDE.md b/wiki/ADMIN_GUIDE.md new file mode 100644 index 0000000..f579ad0 --- /dev/null +++ b/wiki/ADMIN_GUIDE.md @@ -0,0 +1,369 @@ +# LOGAR Administrator & Operations Guide +### *Complete Operations Manual for Fleet Hub Administration, Database Engines, PKI Licensing & AI Provisioning* + +--- + +## Table of Contents + +1. [Executive Summary & System Architecture](#1-executive-summary--system-architecture) +2. [Database Storage Engine Management](#2-database-storage-engine-management) + - [Local SQLite 3 (WAL Mode)](#local-sqlite-3-wal-mode) + - [External PostgreSQL Enterprise Backend](#external-postgresql-enterprise-backend) + - [Automated SQLite to PostgreSQL Migration](#automated-sqlite-to-postgresql-migration) +3. [Telemetry Ingestion Cache & Buffer Tuning](#3-telemetry-ingestion-cache--buffer-tuning) + - [Ingestion Modes: Direct Write vs In-Memory Queue Buffer](#ingestion-modes-direct-write-vs-in-memory-queue-buffer) + - [Tuning Parameters & Metrics](#tuning-parameters--metrics) + - [Manual Flush & Buffer Monitoring](#manual-flush--buffer-monitoring) +4. [Zero-Knowledge OpenPGP Data Encryption at Rest](#4-zero-knowledge-openpgp-data-encryption-at-rest) + - [Encryption Mechanism](#encryption-mechanism) + - [Transparent Authorized Decryption](#transparent-authorized-decryption) +5. [Subordinate CA & Dynamic PKI Licensing](#5-subordinate-ca--dynamic-pki-licensing) + - [Hierarchical PKI & Custom X.509 OIDs](#hierarchical-pki--custom-x509-oids) + - [Licensed Feature Capabilities (`enterprise_db`)](#licensed-feature-capabilities-enterprise_db) + - [Importing `.lic` Activation Packages](#importing-lic-activation-packages) + - [Automated mTLS License Renewal](#automated-mtls-license-renewal) +6. [Fleet Forwarder Management & Seat Provisioning](#6-fleet-forwarder-management--seat-provisioning) + - [Zero-Touch Linux Deployment](#zero-touch-linux-deployment) + - [Windows Edge Client Deployment](#windows-edge-client-deployment) + - [Seat Lifecycle: Approve, Revoke, Delete](#seat-lifecycle-approve-revoke-delete) +7. [Hermes AI Agent Integration](#7-hermes-ai-agent-integration) + - [Dedicated Bearer Tokens](#dedicated-bearer-tokens) + - [12-Hour Temporal Persistence Anomaly Stream](#12-hour-temporal-persistence-anomaly-stream) +8. [Security, Authentication & 2FA](#8-security-authentication--2fa) + - [Password Policies & PBKDF2 Hashing](#password-policies--pbkdf2-hashing) + - [Time-Based One-Time Password (TOTP 2FA)](#time-based-one-time-password-totp-2fa) + - [Sliding Window IP Rate Limiting](#sliding-window-ip-rate-limiting) +9. [Configuration Reference & CLI Flags](#9-configuration-reference--cli-flags) +10. [REST API Reference](#10-rest-api-reference) +11. [Operational Runbook & Troubleshooting](#11-operational-runbook--troubleshooting) + +--- + +## 1. Executive Summary & System Architecture + +LOGAR is a centralized log aggregation, normalization, and provisioning system designed for enterprise server fleets. It connects heterogeneous edge nodes (Windows and Linux) with a central server hub over **Mutual TLS 1.3 (mTLS)**, normalizing incoming events and presenting structured anomaly intelligence to **Hermes AI**. + +### Core Network Architecture + +```mermaid +graph TD + subgraph MasterAuthority [Master License Authority] + ROOT_CA[Master Root CA] + AUTH_API[WebSocket Licensing Hub
logar.eibl.tech] + end + + subgraph Hub [LOGAR Server Hub] + SUB_CA[Subordinate CA & PKI Engine] + INGEST_QUEUE[IngestQueueManager
In-Memory RAM Buffer] + STORAGE_ROUTER[Storage Engine Router] + SQLITE[(Local SQLite 3 WAL)] + POSTGRES[(External PostgreSQL Cluster)] + TEMP_ENG[12h Temporal Persistence Engine] + HERMES_API[Hermes REST & WebUI API
Port 8443] + MTLS_SOCKET[mTLS 1.3 Socket Listener
Port 9443] + + MTLS_SOCKET --> INGEST_QUEUE + INGEST_QUEUE --> STORAGE_ROUTER + STORAGE_ROUTER --> SQLITE + STORAGE_ROUTER --> POSTGRES + STORAGE_ROUTER --> TEMP_ENG + TEMP_ENG --> HERMES_API + end + + subgraph EdgeNodes [Fleet Edge Forwarders] + WIN_NODE[Windows Forwarder
Win_Client.exe] + LIN_NODE[Linux Forwarder
Linux_Client.bin] + end + + subgraph AI [Hermes AI Analysis Layer] + HERMES_AGENT[Hermes AI Agent] + end + + ROOT_CA -->|Signs Sub-CA Cert| SUB_CA + AUTH_API <==>|Live mTLS Renewal| SUB_CA + + WIN_NODE ==>|1. Auto-Enrollment / mTLS Ingest| MTLS_SOCKET + LIN_NODE ==>|1. Auto-Enrollment / mTLS Ingest| MTLS_SOCKET + + HERMES_API ==>|GET /api/hermes/report| HERMES_AGENT +``` + +--- + +## 2. Database Storage Engine Management + +LOGAR supports dual storage backends: an embedded **Local SQLite 3 (WAL Mode)** engine and an **External PostgreSQL Database Server** for enterprise clustering. + +### Local SQLite 3 (WAL Mode) +- **Status**: Standard Default (Included out of the box, zero configuration required). +- **Optimization**: Operates with Write-Ahead Logging (`PRAGMA journal_mode=WAL;`), asynchronous normal synchronization (`PRAGMA synchronous=NORMAL;`), and memory-mapped I/O (`PRAGMA mmap_size=268435456;`). +- **File Location**: Configurable via `db_path` in `server_config.json` (defaults to `logar_state.db`). + +### External PostgreSQL Enterprise Backend +- **Requirement**: Unlocked via the `enterprise_db` license capability. +- **Benefits**: Enables horizontal database scalability, clustering, multi-node replication, and centralized enterprise backup policies. +- **Connection Parameters**: + - **Host / IP**: Database hostname or IP address. + - **Port**: Default `5432`. + - **Database**: Target schema/database name (default `logar`). + - **Username / Password**: Authentication credentials. + - **SSL Mode**: Supports `prefer`, `require`, `verify-ca`, `verify-full`, `disable`, and `allow`. + - **Connect Timeout**: Timeout in seconds (default `5`s). + - **Connection Pool Size**: Maximum pooled persistent connections (default `10`). + +### Automated SQLite to PostgreSQL Migration +Administrators can migrate all historical logs, runs, active issues, seat allocations, users, and tokens from SQLite to PostgreSQL directly in the WebUI (`πŸ’Ύ DB Engine` -> `πŸš€ Migrate SQLite Data to PostgreSQL`) or via API (`POST /api/db/migrate-to-postgres`): +1. Flushes any in-memory telemetry buffers to SQLite. +2. Connects to PostgreSQL and verifies/initializes the schema. +3. Automatically transfers records table-by-table with validation. +4. Returns a detailed count of all migrated records. + +--- + +## 3. Telemetry Ingestion Cache & Buffer Tuning + +### Ingestion Modes: Direct Write vs In-Memory Queue Buffer + +| Feature | ⚑ Immediate Direct Write | πŸš€ In-Memory Queue Buffer (`enterprise_db`) | +| :--- | :--- | :--- | +| **Ingress Latency** | Low latency, immediate DB commit | Microsecond ingestion, non-blocking | +| **Throughput** | Up to 1,500 events/sec | Up to 50,000+ events/sec | +| **Disk I/O** | Continuous write cycles | Atomic batch flushing | +| **License** | Standard Community Edition | Enterprise License (`enterprise_db`) | + +### Tuning Parameters & Metrics +When **In-Memory Queue Buffer** is enabled, administrators can tune the buffer engine: +- **Flush Interval (seconds)**: Maximum elapsed time before buffered anomalies are written to storage (e.g. `2.0`s). +- **Batch Size Threshold**: Number of buffered events that triggers an immediate atomic flush (e.g. `100` events). +- **Max Buffer Capacity**: Upper ceiling of buffered items in memory (e.g. `50,000` events) to protect system RAM under heavy load. + +### Manual Flush & Buffer Monitoring +- **Live Monitoring**: The KPI card displays live buffer size, total events ingested, total flushed, and average flush latency. +- **Manual Trigger**: Click **`🌊 Flush Buffer Now`** or call `POST /api/db/flush-buffer` to force an immediate buffer drain. + +--- + +## 4. Zero-Knowledge OpenPGP Data Encryption at Rest + +### Encryption Mechanism +When **Data Encryption at Rest** is enabled (`encrypt_at_rest: true`): +- All sensitive log payloads, anomaly messages, and raw traces are encrypted with the server's OpenPGP public key before writing to the database table (`active_issues.message`). +- Remote database administrators or third-party database hosting providers only see armored OpenPGP ciphertext (`-----BEGIN PGP MESSAGE-----`). + +### Transparent Authorized Decryption +- The central LOGAR hub instance holds the private OpenPGP key in protected memory. +- When authenticated administrators access the WebUI or when authorized Hermes AI instances query `/api/hermes/report`, the server transparently decrypts records on-the-fly. + +--- + +## 5. Subordinate CA & Dynamic PKI Licensing + +LOGAR utilizes a hierarchical Public Key Infrastructure (PKI) for cryptographic client identity and license seat enforcement. + +### Hierarchical PKI & Custom X.509 OIDs +The Master Authority (`logar.eibl.tech`) signs the hub's Subordinate CA certificate (`certs/ca.crt`), embedding custom ASN.1 OIDs: +- **`1.3.6.1.4.1.58888.1.1` (Seat Quota)**: Encodes the authorized edge client seat capacity (e.g. `25` seats). +- **`1.3.6.1.4.1.58888.1.2` (Master Authority URL)**: Identifies the upstream licensing authority. +- **`1.3.6.1.4.1.58888.1.3` (Feature Capabilities)**: Comma-delimited list of granted features (e.g. `enterprise_db,ai_heuristics`). + +### Licensed Feature Capabilities (`enterprise_db`) +The **πŸ“œ Sub-CA & Licensing** tab displays all active licensed features: +- **`enterprise_db`**: Authorizes external PostgreSQL database pooling, high-throughput in-memory RAM buffering, and OpenPGP data-at-rest encryption. + +### Importing `.lic` Activation Packages +1. In the WebUI, navigate to **πŸ“œ Sub-CA & Licensing** and click **`πŸ“₯ Import License (.lic)`**. +2. Select your `.lic` bundle or paste the JSON package. +3. The server immediately reloads the PKI trust chain and hot-reloads the active mTLS `SSLContext` without downtime. + +### Automated mTLS License Renewal +- Click **`πŸ”„ Sync & Renew via mTLS`** in the WebUI or run: + ```bash + python src/Server.py --sync-license + ``` +- The hub connects to `logar.eibl.tech` over mTLS, sends its current seat utilization, and retrieves an updated signed CA certificate. + +--- + +## 6. Fleet Forwarder Management & Seat Provisioning + +### Zero-Touch Linux Deployment +Run the automated single-line installer on target Linux servers: +```bash +sudo ./compilation/install_linux_client.sh "http://:8443" "" +``` +- Automatically generates a 2048-bit RSA client keypair. +- Enrolls with the hub and receives a signed client certificate. +- Deploys and starts the `logar-client.service` systemd unit. + +### Windows Edge Client Deployment +1. Download and run `LOGAR-Client-Setup.exe` on the Windows host. +2. In the Server WebUI, click **`πŸ“₯ Export Client Profile`** and download `client_config.json`. +3. Place `client_config.json` into `C:\Program Files\LOGAR\client_config.json`. +4. The `LOGAR_Client` service automatically boots and connects via mTLS. + +### Seat Lifecycle: Approve, Revoke, Delete +In the **πŸ’» Fleet Forwarders** tab: +- **Revoke**: Temporarily blocks the node at the TLS handshake level (`ssl.CERT_REQUIRED`). +- **Approve**: Re-activates a revoked node and restores telemetry ingestion. +- **Remove**: Permanently deletes the client record and immediately frees the leased seat quota. + +--- + +## 7. Hermes AI Agent Integration + +### Dedicated Bearer Tokens +1. Navigate to **πŸ€– Hermes AI & Tokens** in the WebUI. +2. Enter a descriptive token name (e.g. `Hermes-Autonomous-SecOps`) and click **`Generate Token`**. +3. Copy the Bearer token for Hermes agent configuration. + +### 12-Hour Temporal Persistence Anomaly Stream +Hermes queries the real-time anomaly stream: +```bash +curl -H "Authorization: Bearer hermes_xxxxxxxxxxxx" http://:8443/api/hermes/report +``` + +#### JSON Telemetry Format: +```json +[ + { + "fingerprint": "corp-dc:srv-db-01:postgres_wal_checkpoint_delay", + "site": "corp-dc", + "server": "srv-db-01", + "signature": "postgres_wal_checkpoint_delay", + "severity": "WARNING", + "message": "WAL checkpoint write duration exceeded 30000ms threshold", + "os_type": "linux", + "first_seen": "2026-09-30 14:10:00", + "last_seen": "2026-09-30 17:42:15", + "consecutive_runs": 4, + "evaluation_window": "12h", + "verified": true, + "status": "VERIFIED" + } +] +``` + +--- + +## 8. Security, Authentication & 2FA + +### Password Policies & PBKDF2 Hashing +- User passwords are protected using PBKDF2-HMAC-SHA256 with 100,000 iterations and 16-byte random salts. +- Change admin credentials via the user avatar menu -> **πŸ”‘ Change Password**. + +### Time-Based One-Time Password (TOTP 2FA) +- Standard RFC 6238 TOTP with standard 30-second drift windows. +- Configure via avatar menu -> **πŸ›‘οΈ Two-Factor Auth (2FA)**. +- Compatible with Google Authenticator, Authy, 1Password, and Bitwarden. + +### Sliding Window IP Rate Limiting +- **Brute-Force Protection**: Enforces rate limiting on `/login` and `/api/client/enroll`. +- Excess failed attempts trigger automatic IP blocking with sliding exponential backoff. + +--- + +## 9. Configuration Reference & CLI Flags + +### `server_config.json` Specification + +```json +{ + "server_name": "LOGAR-Cloud-Hub", + "tcp_host": "0.0.0.0", + "tcp_port": 9443, + "hermes_host": "0.0.0.0", + "hermes_port": 8443, + "admin_user": "admin", + "enrollment_secret": "9a7f8e...4b", + "max_seats": 25, + "license_status": "active", + "license_server_url": "https://logar.eibl.tech", + "cert_dir": "certs", + "tls_enabled": true, + "db_path": "logar_state.db", + "storage_engine": "sqlite", + "cache_mode": "direct", + "flush_interval": 2.0, + "batch_size": 100, + "max_queue": 50000, + "encrypt_at_rest": false, + "server_fingerprint": "6FF5BE81...", + "public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...", + "private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..." +} +``` + +### CLI Arguments + +| Flag | Description | +| :--- | :--- | +| `--config ` | Path to server configuration JSON file | +| `--create-client-config` | Exports a client config bundle and exits | +| `--client-out ` | Destination path for exported client configuration | +| `--server-host ` | Embedded target host for exported client configuration | +| `--server-port ` | Embedded target port for exported client configuration | +| `--export-lic ` | Exports a `.lic` CSR activation request package | +| `--import-lic ` | Imports and activates a signed `.lic` license package | +| `--sync-license` | Connects to master authority to renew certificate & seats | + +--- + +## 10. REST API Reference + +| Method | Endpoint | Auth | Description | +| :--- | :--- | :--- | :--- | +| `GET` | `/health` | None | Health check, server name, and TLS status | +| `POST` | `/api/client/enroll` | Secret | Edge forwarder mTLS enrollment & certificate issuance | +| `GET` | `/api/clients` | Admin | List registered forwarders, activity times, and seat quota | +| `POST` | `/api/clients/approve` | Admin | Reactivate / approve an enrolled forwarder seat | +| `POST` | `/api/clients/revoke` | Admin | Revoke an enrolled forwarder seat | +| `POST` | `/api/clients/delete` | Admin | Remove forwarder record and permanently free seat | +| `GET` | `/api/hermes/report` | Hermes/Admin | Stream verified anomalies within 12h temporal window | +| `POST` | `/api/hermes/tokens/create` | Admin | Generate a named Bearer token for Hermes AI | +| `POST` | `/api/hermes/tokens/revoke` | Admin | Revoke an existing Hermes Bearer token | +| `GET` | `/api/db/stats` | Admin | Telemetry buffer stats, queue metrics, and storage info | +| `POST` | `/api/db/flush-buffer` | Admin | Force immediate drain and write of buffered telemetry | +| `POST` | `/api/db/test-connection` | Admin | Test external PostgreSQL connectivity (`enterprise_db`) | +| `POST` | `/api/db/switch-backend` | Admin | Switch backend between SQLite and PostgreSQL (`enterprise_db`) | +| `POST` | `/api/db/config` | Admin | Persist DB engine, cache mode, and encryption settings | +| `POST` | `/api/db/migrate-to-postgres` | Admin | Migrate SQLite tables to PostgreSQL (`enterprise_db`) | +| `GET` | `/api/license/status` | Admin | Subordinate CA status, seat quota, and licensed features | +| `POST` | `/api/license/import` | Admin | Import `.lic` bundle or signed CA PEM | +| `POST` | `/api/license/sync` | Admin | Sync license seats with master authority over mTLS | + +--- + +## 11. Operational Runbook & Troubleshooting + +### Diagnostic Checklist + +1. **Verify Hub Service Status**: + ```bash + # Linux + sudo systemctl status logar-server.service + # Windows PowerShell + Get-Service LOGAR_Server + ``` +2. **Check Health Endpoint**: + ```bash + curl http://localhost:8443/health + ``` +3. **Inspect Active Anomaly Telemetry**: + ```bash + curl -H "Authorization: Bearer " http://localhost:8443/api/hermes/report + ``` +4. **Certificate Validity Check**: + ```bash + openssl x509 -in certs/ca.crt -text -noout | grep -A 2 "Validity" + ``` + +### Common Issues & Remediation + +- **Issue: Forwarder rejected with `Connection Refused` or `TLS Handshake Failure`**: + - Ensure port `9443` is reachable through host and cloud firewalls. + - Verify edge node clock synchronization (NTP) with the server hub. + - Confirm the client certificate has not been revoked in the **Fleet Forwarders** tab. +- **Issue: `403 Forbidden` on PostgreSQL or Buffering settings**: + - The `enterprise_db` feature capability is required. Verify your license in **πŸ“œ Sub-CA & Licensing**. +- **Issue: Rate limit lock on admin login**: + - Wait 60 seconds for the sliding attempt window to expire, or authenticate via an existing active session.