LOGAR: Edge-Thin Log Analysis & Temporal Verification System
LOGAR is an enterprise log aggregation, verification, and anomaly detection architecture designed for heterogeneous server fleets (Windows & Linux). It combines lightweight zero-state edge forwarders with a centralized cloud hub that applies mutual TLS 1.3 (mTLS) authentication, dynamic PKI licensing and quota management, temporal persistence tracking across 12-hour evaluation windows, and direct anomaly pass-through enabling Hermes AI to perform all deep reasoning, anomaly evaluation, and autonomous decision-making.
Table of Contents
- Core Philosophy
- Architecture & Data Flow
- mTLS Security, Dynamic PKI & Licensing
- Cloud-Side Temporal Persistence & Direct Hermes Reasoning
- Agentic Hermes & Client Management API
- Dynamic Machine & Domain Identification
- Automated Service Installers (Linux & Windows)
- Repository & Shippables Structure
- Getting Started & Deployment
- Running Tests
- Automated Releases via Gitea Actions
Core Philosophy
1. Edge Thinness & Zero State
Site agents running on Windows and Linux act strictly as lightweight forwarders:
- No Local Database: Clients maintain zero local SQLite or heavy cache files.
- Source-Level Filtering: Agents stream candidate entries from informational events up to errors (
INFO,WARNING,ERROR,CRITICAL), stripping verbose debugging noise (DEBUG, trace entries) and skipping events older than 24 hours. - Transport Security (mTLS 1.3): Logs are streamed directly over mutual TLS 1.3 sockets with hardware-bound / machine-unique client certificates.
- Zero Configuration Overhead: Clients auto-bootstrap certificate enrollment on first run if configured with an enrollment secret.
2. Cloud-Side Temporal Persistence & Direct Hermes Reasoning
The central Python hub handles state and temporal tracking:
- State tracking is managed centrally in SQLite (
logar_state.db). - Candidate issues are tracked over a 12-hour temporal evaluation window with consecutive run counts and first/last-seen timestamps.
- Direct Hermes Intelligence: All ingested actionable anomalies (
WARNING,ERROR,INFO) are immediately promoted toVERIFIEDand surfaced directly to Hermes. Hermes performs all multi-factor reasoning, pattern evaluation, and contextual thinking rather than relying on arbitrary hardcoded threshold rules.
3. Agentic Integration with Hermes
Instead of engineers manually sifting through logs, Hermes ingests verified anomalies directly from the cloud hub (GET /api/hermes/report), treating errors and warnings with full temporal metadata (run counts, timestamps) as actionable system artifacts to trigger precise notifications and remediations.
Architecture & Data Flow
graph TB
subgraph Edge Nodes [Lightweight Edge Forwarders]
W[Win_Client.exe / Win_Client.py<br/>Windows Event Log Ingestion]
L[Linux_Client.bin / Linux_Client.py<br/>systemd journalctl -p warning]
end
subgraph Enrollment [Dynamic PKI & Licensing]
ENR[POST /api/client/enroll<br/>License Quota & Secret Validation]
CA[Internal Root CA<br/>Signs RSA-2048 Client Cert]
end
subgraph Transport [mTLS 1.3 Security Layer]
MTLS[Mutual TLS 1.3 Handshake<br/>Port 9443 - Client Cert Required]
AUTH[Extract Client CN & Fingerprint<br/>Validate Active License in SQLite]
end
subgraph Hub [LOGAR Server Hub]
INGEST[Length-Prefixed Frame Ingestion]
DB[(SQLite Persistence<br/>active_issues, clients, license_config)]
RULE{12h Window &<br/>Temporal Ingestion}
end
subgraph Downstream [Hermes Agent & Monitoring]
API[FastAPI Reporting & Management<br/>Port 8443]
HERMES[Hermes Agent<br/>GET /api/hermes/report]
end
W -->|Auto-Enrollment| ENR
L -->|Auto-Enrollment| ENR
ENR --> CA
CA -->|ca.crt, client.crt, client.key| W
CA -->|ca.crt, client.crt, client.key| L
W -->|mTLS Stream| MTLS
L -->|mTLS Stream| MTLS
MTLS --> AUTH
AUTH --> INGEST
INGEST --> RULE
RULE --> DB
DB --> API
API --> HERMES
mTLS Security, Dynamic PKI & Fleet Licensing
1. TLS 1.3 Mutual Authentication (mTLS)
- Port 9443: Ingestion occurs exclusively over TLS 1.3 sockets with
ssl.CERT_REQUIRED. - Both the hub and edge clients verify each other's certificates:
- Client verifies server certificate against
ca.crt. - Server verifies client certificate against the Subordinate/Root CA.
- Client verifies server certificate against
- Client CN Identification & Revocation: In the TLS handshake, the server extracts the
commonNameattribute (client_id), validates that the client is markedactivein theclientstable, updates thelast_seentimestamp, and drops unregistered or revoked certificates immediately.
2. Master License Authority (logar.eibl.tech) & Seat-Bearing Subordinate CA
-
Master Root Authority: Standalone PKI repository & submodule (
LOGAR-LICENSING/, upstream athttps://gitea.eibl.tech/me0nline/LOGAR-LICENSING.git) deployed atlogar.eibl.tech. -
Custom Enterprise X.509 OIDs:
OID_LOGAR_SEATS (1.3.6.1.4.1.59999.1.1): Embeds the exact licensed seat quota directly inside the signed Subordinate CA certificate.OID_LOGAR_LICENSE_ID (1.3.6.1.4.1.59999.1.2): Uniquely tracks enterprise license entitlement.OID_LOGAR_ISSUER_URL (1.3.6.1.4.1.59999.1.3): Embedded endpoint coordinate (https://logar.eibl.tech).
-
Bootstrap
.licPackages & Instant WebSocket Auto-Enrollment:- The License Authority (
logar.eibl.tech) generates initial Bootstrap.licPackages containing Master Root CA, dedicated communication credentials (communication_cert+communication_key), CSR templates, and WebSocket coordinates (wss://logar.eibl.tech/ws/license). - 1-Click Import & Live Signing: Importing the
.licfile on the client server (python src/Server.py --import-lic <file.lic>or via WebUI) connects directly via WebSocket, submits the server Sub-CA CSR, and immediately installs the officially signed Sub-CA license certificate.
- The License Authority (
-
Self-Updating Client Servers & Live WebSocket Sync:
- Client servers update themselves against the master authority (
wss://logar.eibl.tech/ws/license) over WebSocket (python src/Server.py --sync-licenseor via WebUI / automated watchdog). - Seat quotas can be increased with each renewal or updated on-demand in the License Authority WebUI.
- The client server automatically receives the updated seat quota in its signed certificate, updates its SQLite database and runtime configuration, and reloads its in-flight
ssl.SSLContextwithout dropping client connections or restarting services.
- Client servers update themselves against the master authority (
-
Automated Container Pipeline & Ephemeral NPM Pass-Through:
- The license authority repository includes a dedicated CI/CD workflow (
.gitea/workflows/build-container.yml) publishing togitea.eibl.tech/me0nline/logar-licensing:latest(authenticated viagiteatokensecret). - Works seamlessly with ephemeral Nginx Proxy Manager (NPM) by enabling the standard Websockets Support toggle for
logar.eibl.tech. - Admin WebUI is protected by session login (
/login, default:admin/admin_license_2026!).
- The license authority repository includes a dedicated CI/CD workflow (
3. Integrated Fleet Licensing & WebUI Dashboard
- WebUI URL: Accessible at
http://<server_host>:8443/webui(or/). - Visual Seat Quota Meter: Real-time progress bar displaying active forwarders vs. allocated seat limit.
- Licensing Actions:
- 1-click Export .lic Request file download.
- Interactive View & Copy CA CSR modal.
- Drag-and-drop or paste Import License (.lic / ca.crt) modal.
- Sync & Renew via mTLS button for immediate upstream synchronization and seat expansion.
- Seat Management & Revocation Table:
- Lists all connected Windows and Linux edge forwarders.
- Revoke Seat: Instantly drops and blocks the client at the TLS socket handshake layer.
- Reactivate Seat: Restores the client to active status if within seat quota limits.
4. Cloud-Side In-Flight Certificate Watchdog
- Continuous Hub PKI Watchdog:
- The server hub runs a continuous background watchdog coroutine (
cert_validity_watchdog, running every 12 hours). - Automatically renews Subordinate CA certificates over mTLS with
logar.eibl.techand Server TLS certificates in-flight, dynamically reloading the activessl.SSLContext.
- The server hub runs a continuous background watchdog coroutine (
- Client Proactive Check & Reactive Self-Healing:
- Proactive Renewal: Edge clients inspect
client.crtbefore every run cycle. If expiring within 30 days, they contact/api/client/enrollto renew. - Reactive Self-Healing: If the server CA rotates or a handshake error occurs, edge clients catch the exception, re-bootstrap enrollment against the hub, and re-establish streaming cleanly.
- Proactive Renewal: Edge clients inspect
Cloud-Side Temporal Persistence & Direct Hermes Reasoning
Incoming candidate logs are tracked in SQLite table active_issues:
- Issue Fingerprint: Formatted as
{site_name}:{server}:{signature}. - 12-Hour Evaluation Window:
- The hub compares
(now - last_seen). - If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to
run_count = 1.
- The hub compares
- Direct Verification & Temporal Telemetry:
- Ingested anomalies are immediately promoted to
VERIFIEDand made available to Hermes with complete temporal metadata (first seen, last seen, consecutive run counts). - Hermes performs all contextual analysis, pattern recognition, and alerting logic.
- Ingested anomalies are immediately promoted to
Agentic Hermes & Client Management API
The server hub exposes a management and reporting REST API (default port 8443):
POST /api/client/enroll
Client enrollment endpoint:
- Request:
{ "client_id": "web-worker-01.corp.internal", "hostname": "web-worker-01", "os": "linux", "enrollment_secret": "<SECRET>" } - Response:
{ "ca_cert": "-----BEGIN CERTIFICATE-----\n...", "client_cert": "-----BEGIN CERTIFICATE-----\n...", "client_key": "-----BEGIN RSA PRIVATE KEY-----\n..." }
GET /api/clients
Returns seat quota status and registered client telemetry:
{
"active_seats": 2,
"max_seats": 10,
"clients": [
{
"client_id": "web-worker-01.corp.internal",
"hostname": "web-worker-01",
"os_type": "linux",
"cert_fingerprint": "7D5B660B...",
"status": "active",
"first_seen": "2026-09-04 18:00:00",
"last_seen": "2026-09-04 19:15:00"
}
]
}
GET /api/hermes/report
Returns all verified anomalies within the active 12-hour evaluation window with full temporal metadata for direct Hermes reasoning:
[
{
"fingerprint": "corp.internal:web-worker-01.corp.internal:PostgresPoolExhausted",
"site": "corp.internal",
"server": "web-worker-01.corp.internal",
"signature": "PostgresPoolExhausted",
"severity": "WARNING",
"message": "Connection pool saturated (>95%)",
"os_type": "linux",
"first_seen": "2026-09-04T07:00:00+00:00",
"last_seen": "2026-09-04T19:00:00+00:00",
"consecutive_runs": 4,
"evaluation_window": "12h",
"verified": true,
"status": "VERIFIED"
}
]
GET /api/hermes/all
Diagnostic endpoint listing all tracked anomalies in SQLite.
GET /health
Returns hub health, encryption fingerprint, listener ports, and mTLS status.
Dynamic Machine & Domain Identification
Client configurations intentionally contain no hardcoded machine name or site name. Both forwarders dynamically identify their host and domain at runtime via get_machine_identifier():
- Fully Qualified Domain Name (FQDN): Checked via
socket.getfqdn(). - OS-Specific Domain Discovery:
- Windows: Checks Active Directory environment variables (
USERDNSDOMAIN,USERDOMAIN). - Linux: Parses
/etc/resolv.confdomainandsearchdirectives.
- Windows: Checks Active Directory environment variables (
- Reverse DNS Lookup: Resolves canonical hostname via
socket.gethostbyaddr. - Fallback: Local hostname
socket.gethostname().
The server automatically infers site attribution from domain qualifiers (e.g. node01.corp.internal \rightarrow site corp.internal).
Automated Service Installers (Linux & Windows)
LOGAR provides production-grade installation scripts and installer builders for automated service deployment:
1. Linux Service Installers
- Client Installer (
compilation/install_linux_client.sh):- Non-interactive script deploying to
/opt/logar-client. - Automatically queries
/etc/machine-idand enrolls with the hub viacurl. - Installs and enables
logar-client.servicesystemd unit.
sudo ./compilation/install_linux_client.sh "http://hub.example.com:8443" "<ENROLLMENT_SECRET>" - Non-interactive script deploying to
- Server Installer (
compilation/install_linux_server.sh):- Deploys server to
/opt/logar-server. - Configures logging and installs
logar-server.servicewithLimitNOFILE=65536.
sudo ./compilation/install_linux_server.sh - Deploys server to
2. Windows Inno Setup Installers
- Built using Inno Setup 6 and bundled with NSSM (
compilation/nssm.exe):- Client Setup (
compilation/installer_client.iss): CompilesLOGAR-Client-Setup.exe. InstallsWin_Client.exeinto{autopf}\LOGAR, sets upLOGAR_Clientservice via NSSM with stdout/stderr redirection to{commonappdata}\LOGAR\client.log, and starts the service. Clean uninstallation stops and removes the service. - Server Setup (
compilation/installer_server.iss): CompilesLOGAR-Server-Setup.exe. InstallsServer.exeand sets upLOGAR_ServerWindows service via NSSM.
- Client Setup (
Repository & Shippables Structure
LOGAR/
├── .gitea/
│ └── workflows/
│ ├── ci.yml # CI pipeline: syntax, 25 unit tests & mTLS pipeline test
│ └── release.yml # Consolidated release workflow (Linux binaries & Windows installers)
├── compilation/ # Packaging, installers, and release automation
│ ├── install_linux_client.sh # Automated Linux client systemd installation script
│ ├── install_linux_server.sh # Automated Linux server systemd installation script
│ ├── installer_client.iss # Inno Setup Windows Client installer script
│ ├── installer_server.iss # Inno Setup Windows Server installer script
│ ├── nssm.exe # Official 64-bit NSSM service manager binary
│ ├── package_dist.py # Standalone binary compiler & packager
│ ├── requirements.txt # Unified project dependencies
│ └── upload_release.py # Gitea REST API release asset publisher
├── src/ # Core application source modules
│ ├── __init__.py
│ ├── Server.py # Central mTLS server, temporal engine, and Hermes REST API
│ ├── server_enrollment.py # Dynamic PKI, Root CA, and client certificate generator
│ ├── Win_Client.py # Windows edge forwarder with auto-enrollment
│ └── Linux_Client.py # Linux edge forwarder with auto-enrollment
├── tests/ # Automated test suites
│ ├── test_linux_client.py # Linux client unit tests & mTLS certificate validation
│ ├── test_pipeline.py # End-to-end mTLS integration & direct anomaly verification test
│ ├── test_server.py # Server unit tests, PKI generation, and seat quota tests
│ └── test_win_client.py # Windows client unit tests & mTLS certificate validation
├── .gitignore # Ignores venv, caches, DBs, and private keys
├── README.md # Architecture and usage documentation
├── RELEASE_NOTES.md # Release history and changelog
├── server_config.sample.json # Reference server configuration
└── out/ # Component guides and sample configs
├── linux_server/
│ ├── README.md
│ └── server_config.sample.json
├── win_server/
│ ├── README.md
│ └── server_config.sample.json
├── linux_client/
│ ├── README.md
│ └── client_config.sample.json
└── win_client/
├── README.md
└── client_config.sample.json
Getting Started & Deployment
1. Central Server Hub
- Install dependencies:
pip install -r compilation/requirements.txt - Start the server (generates
server_config.json, Root CA, and server certs on first run):python src/Server.py - Export a client configuration:
python src/Server.py --create-client-config --server-host <SERVER_IP> --server-port 9443 --client-out client_config.json
2. Windows Client Deployment
- Download
LOGAR-Client-Setup.exefrom releases and run it, or placeWin_Client.exeandclient_config.jsoninC:\Program Files\LOGAR. - On first run with
client_config.json,Win_Client.exeautomatically enrolls with the hub, receives its mTLS certificates, and establishes secure streaming.
3. Linux Client Deployment
- Run the automated installer:
sudo ./compilation/install_linux_client.sh "http://<HUB_HOST>:8443" "<ENROLLMENT_SECRET>" - The installer enrolls the client, configures
/etc/logar/certs, and activateslogar-client.service.
Running Tests
1. Component Unit Tests
python -m unittest discover -s tests -v
Runs all 25 unit tests, covering:
- Dynamic Root CA generation and server TLS certificate issuance.
- Dynamic client certificate issuance with CN and authority key extensions.
- Enrollment secret authentication, seat limits, and certificate revocation.
- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload.
- Windows & Linux event log collection, deduplication, and mTLS certificate verification.
- 12-hour evaluation window and direct Hermes anomaly verification.
2. End-to-End Pipeline Integration Test
# 1. Initialize test configuration
python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json
# 2. Launch server in background
python src/Server.py &
# 3. Run integration test
python tests/test_pipeline.py
Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output.
Releases & Binary Distribution
Releases can be built and published through three complementary channels:
1. Tag Push Automation (Gitea Actions)
Pushing a release tag automatically triggers the build workflows:
git tag v2.0.1
git push origin v2.0.1
The consolidated workflow release.yml defines two parallel jobs:
release-linux(ubuntu-latest): Compiles standalone native ELF binaries (Linux_Client.bin,Server.bin) and checksums.release-windows(windows-latest): Compiles Windows executables (Win_Client.exe,Server.exe), builds Inno Setup installers, and publishes checksums (requires self-hosted Windows Act Runner).
2. Manual Workflow Dispatch (Gitea UI)
Workflows can be manually triggered on demand from the Gitea web interface:
- Navigate to Actions
\rightarrowRelease Binaries & Installers (release.yml). - Click Run workflow, set the release tag (defaults to
v2.0.1), and run.
3. Native Local Windows Build & Direct Release Upload
For environments without a registered Windows CI runner, Windows executables can be built and published directly to Gitea releases:
# 1. Build Windows binaries locally
python compilation/package_dist.py --target windows
# 2. Upload assets and release notes directly to the Gitea release
python compilation/upload_release.py --tag v2.0.1 --token <GITEA_TOKEN> --skip-build