LOGAR is an enterprise log aggregation, verification, and anomaly detection architecture designed for heterogeneous server fleets (Windows & Linux). It combines lightweight zero-state edge forwarders with a centralized cloud hub that applies mutual TLS 1.3 (mTLS) authentication, dynamic PKI licensing and quota management, temporal persistence tracking across 12-hour evaluation windows, an automated 4-run rule to filter transient warnings, and immediate pass-through for critical errors before reporting verified anomalies to Hermes.

Core Philosophy
Architecture & Data Flow
mTLS Security, Dynamic PKI & Licensing
Cloud-Side Temporal Persistence & 4-Run Rule
Agentic Hermes & Client Management API
Dynamic Machine & Domain Identification
Automated Service Installers (Linux & Windows)
Repository & Shippables Structure
Getting Started & Deployment
Running Tests
Automated Releases via Gitea Actions

Core Philosophy

1. Edge Thinness & Zero State

Site agents running on Windows and Linux act strictly as lightweight forwarders:

No Local Database: Clients maintain zero local SQLite or heavy cache files.
Source-Level Filtering: Agents stream candidate entries from informational events up to errors (INFO, WARNING, ERROR, CRITICAL), stripping verbose debugging noise (DEBUG, trace entries) and skipping events older than 24 hours.
Transport Security (mTLS 1.3): Logs are streamed directly over mutual TLS 1.3 sockets with hardware-bound / machine-unique client certificates.
Zero Configuration Overhead: Clients auto-bootstrap certificate enrollment on first run if configured with an enrollment secret.

2. Cloud-Side Temporal Persistence & Severity Routing

The central Python hub handles state and verification:

State tracking is managed centrally in SQLite (logar_state.db).
Candidate issues are evaluated over a 12-hour temporal evaluation window.
Warning Persistence (4-Run Rule): WARNING issues must persist across at least 4 consecutive runs / cycles within the 12-hour window to be confirmed as genuine anomalies, automatically filtering transient infrastructure blips.
Immediate Error Pass: Critical errors (ERROR, CRITICAL, FATAL) bypass the 4-run threshold and are promoted immediately to VERIFIED on their first occurrence.

3. Agentic Integration with Hermes

Instead of engineers manually sifting through logs, Hermes ingests pre-filtered anomalies directly from the cloud hub (GET /api/hermes/report), treating verified errors and 4-run validated warnings as actionable system artifacts to trigger precise notifications and remediations.

Architecture & Data Flow

graph TB
    subgraph Edge Nodes [Lightweight Edge Forwarders]
        W[Win_Client.exe / Win_Client.py<br/>Windows Event Log Ingestion]
        L[Linux_Client.bin / Linux_Client.py<br/>systemd journalctl -p warning]
    end

    subgraph Enrollment [Dynamic PKI & Licensing]
        ENR[POST /api/client/enroll<br/>License Quota & Secret Validation]
        CA[Internal Root CA<br/>Signs RSA-2048 Client Cert]
    end

    subgraph Transport [mTLS 1.3 Security Layer]
        MTLS[Mutual TLS 1.3 Handshake<br/>Port 9443 - Client Cert Required]
        AUTH[Extract Client CN & Fingerprint<br/>Validate Active License in SQLite]
    end

    subgraph Hub [LOGAR Server Hub]
        INGEST[Length-Prefixed Frame Ingestion]
        DB[(SQLite Persistence<br/>active_issues, clients, license_config)]
        RULE{12h Window &<br/>4-Run Rule}
    end

    subgraph Downstream [Hermes Agent & Monitoring]
        API[FastAPI Reporting & Management<br/>Port 8443]
        HERMES[Hermes Agent<br/>GET /api/hermes/report]
    end

    W -->|Auto-Enrollment| ENR
    L -->|Auto-Enrollment| ENR
    ENR --> CA
    CA -->|ca.crt, client.crt, client.key| W
    CA -->|ca.crt, client.crt, client.key| L

    W -->|mTLS Stream| MTLS
    L -->|mTLS Stream| MTLS
    MTLS --> AUTH
    AUTH --> INGEST
    INGEST --> RULE
    RULE --> DB
    DB --> API
    API --> HERMES

mTLS Security, Dynamic PKI & Licensing

1. TLS 1.3 Mutual Authentication (mTLS)

Port 9443: Ingestion occurs exclusively over TLS 1.3 sockets with ssl.CERT_REQUIRED.
Both the hub and edge clients verify each other's certificates:
- Client verifies server certificate against ca.crt.
- Server verifies client certificate against the Root CA.
Client CN Identification: In the TLS handshake, the server extracts the commonName attribute (client_id), validates that the client is marked active in the clients table, updates the last_seen timestamp, and drops unregistered or revoked certificates immediately.

2. Dynamic PKI Hub Engine (`src/server_enrollment.py`)

Root CA: On first run, Server.py creates a self-signed Root CA (ca.crt / ca.key) valid for 10 years.
Server TLS Certificate: Generated automatically with Subject Alternative Names (SANs) for localhost, 127.0.0.1, server IP, and hostnames.
Authority Key Identifiers: Full compliance with OpenSSL 3.x and Python 3.12–3.14 via SubjectKeyIdentifier and AuthorityKeyIdentifier extensions.
Dynamic Client Certificates: RSA-2048 keys and X.509 client certificates are issued on the fly via the enrollment API.

3. Seat Accounting & Licensing

Stored in SQLite table license_config:
- max_seats: Maximum concurrent active client licenses (default: 10).
- enrollment_secret: Cryptographic secret required for initial client enrollment.
Stored in SQLite table clients:
- client_id: Unique client identifier (machine GUID or hardware hash).
- hostname, os_type, cert_fingerprint, status (active / revoked), first_seen, last_seen.
When a new client enrolls:
- If active_seats >= max_seats, the hub rejects registration with HTTP 403 (License seat limit reached).
- Existing registered clients can re-enroll / renew seamlessly without consuming additional seats.

4. In-Flight Certificate Watchdog & Automated Self-Healing Renewal

Continuous Hub PKI Watchdog:
- The server hub runs a continuous background watchdog coroutine (cert_validity_watchdog, running every 12 hours) alongside startup checks.
- The hub automatically inspects expiration dates of both the Root CA (ca.crt) and the Server TLS certificate (server.crt).
- If either certificate is within 30 days of expiration, the server regenerates certificates (backing up previous keys as ca.crt.<timestamp>.bak) and dynamically reloads its active ssl.SSLContext in memory without dropping socket connections or restarting the service.
Client Proactive Check & Reactive Self-Healing:
- Proactive Renewal: Edge clients inspect client.crt before every run cycle. If the certificate expires in less than 30 days, it automatically contacts /api/client/enroll to renew its certificate.
- Reactive Self-Healing: If the server hub Root CA rotates or a handshake fails with ssl.SSLError / SSLCertVerificationError, edge clients catch the verification exception, re-bootstrap certificate enrollment against the hub, and re-establish the connection cleanly without human intervention.

Cloud-Side Temporal Persistence & 4-Run Rule

Incoming candidate logs are tracked in SQLite table active_issues:

Issue Fingerprint: Formatted as {site_name}:{server}:{signature}.
12-Hour Evaluation Window:
- The hub compares (now - last_seen).
- If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to run_count = 1.
4-Run Rule for Warnings:
- The 4-run persistence threshold applies to WARNING (and INFO) events to eliminate transient operational noise.
- Each distinct run batch increments run_count.
- Warnings with run_count < 4 are marked as TRANSIENT and excluded from Hermes reports.
- When run_count >= 4 within the active 12-hour window, the warning transitions to VERIFIED.
Immediate Verification for Errors:
- High-severity events (ERROR, CRITICAL, FATAL) always pass immediately.
- On their very first ingestion (run_count = 1), errors are promoted directly to VERIFIED and surfaced to Hermes without waiting for 4 runs.

Agentic Hermes & Client Management API

The server hub exposes a management and reporting REST API (default port 8443):

`POST /api/client/enroll`

Client enrollment endpoint:

Request:

{
  "client_id": "web-worker-01.corp.internal",
  "hostname": "web-worker-01",
  "os": "linux",
  "enrollment_secret": "<SECRET>"
}

Response:

{
  "ca_cert": "-----BEGIN CERTIFICATE-----\n...",
  "client_cert": "-----BEGIN CERTIFICATE-----\n...",
  "client_key": "-----BEGIN RSA PRIVATE KEY-----\n..."
}

`GET /api/clients`

Returns seat quota status and registered client telemetry:

{
  "active_seats": 2,
  "max_seats": 10,
  "clients": [
    {
      "client_id": "web-worker-01.corp.internal",
      "hostname": "web-worker-01",
      "os_type": "linux",
      "cert_fingerprint": "7D5B660B...",
      "status": "active",
      "first_seen": "2026-09-04 18:00:00",
      "last_seen": "2026-09-04 19:15:00"
    }
  ]
}

`GET /api/hermes/report`

Returns all verified anomalies (immediate critical errors and warnings verified after 4 consecutive runs within the 12-hour window):

[
  {
    "fingerprint": "corp.internal:web-worker-01.corp.internal:PostgresPoolExhausted",
    "site": "corp.internal",
    "server": "web-worker-01.corp.internal",
    "signature": "PostgresPoolExhausted",
    "severity": "WARNING",
    "message": "Connection pool saturated (>95%) across 4 runs",
    "os_type": "linux",
    "first_seen": "2026-09-04T07:00:00+00:00",
    "last_seen": "2026-09-04T19:00:00+00:00",
    "consecutive_runs": 4,
    "evaluation_window": "12h",
    "verified": true,
    "status": "VERIFIED"
  }
]

`GET /api/hermes/all`

Diagnostic endpoint listing all candidate issues (TRANSIENT and VERIFIED).

`GET /health`

Returns hub health, encryption fingerprint, listener ports, and mTLS status.

Dynamic Machine & Domain Identification

Client configurations intentionally contain no hardcoded machine name or site name. Both forwarders dynamically identify their host and domain at runtime via get_machine_identifier():

Fully Qualified Domain Name (FQDN): Checked via socket.getfqdn().
OS-Specific Domain Discovery:
- Windows: Checks Active Directory environment variables (USERDNSDOMAIN, USERDOMAIN).
- Linux: Parses /etc/resolv.conf domain and search directives.
Reverse DNS Lookup: Resolves canonical hostname via socket.gethostbyaddr.
Fallback: Local hostname socket.gethostname().

The server automatically infers site attribution from domain qualifiers (e.g. node01.corp.internal \rightarrow site corp.internal).

Automated Service Installers (Linux & Windows)

LOGAR provides production-grade installation scripts and installer builders for automated service deployment:

1. Linux Service Installers

Client Installer (compilation/install_linux_client.sh):
- Non-interactive script deploying to /opt/logar-client.
- Automatically queries /etc/machine-id and enrolls with the hub via curl.
- Installs and enables logar-client.service systemd unit.
```
sudo ./compilation/install_linux_client.sh "http://hub.example.com:8443" "<ENROLLMENT_SECRET>"
```
Server Installer (compilation/install_linux_server.sh):
- Deploys server to /opt/logar-server.
- Configures logging and installs logar-server.service with LimitNOFILE=65536.
```
sudo ./compilation/install_linux_server.sh
```

2. Windows Inno Setup Installers

Built using Inno Setup 6 and bundled with NSSM (compilation/nssm.exe):
- Client Setup (compilation/installer_client.iss): Compiles LOGAR-Client-Setup.exe. Installs Win_Client.exe into {autopf}\LOGAR, sets up LOGAR_Client service via NSSM with stdout/stderr redirection to {commonappdata}\LOGAR\client.log, and starts the service. Clean uninstallation stops and removes the service.
- Server Setup (compilation/installer_server.iss): Compiles LOGAR-Server-Setup.exe. Installs Server.exe and sets up LOGAR_Server Windows service via NSSM.

Repository & Shippables Structure

LOGAR/
├── .gitea/
│   └── workflows/
│       ├── ci.yml                     # CI pipeline: syntax, 25 unit tests & mTLS pipeline test
│       └── release.yml                # Consolidated release workflow (Linux binaries & Windows installers)
├── compilation/                       # Packaging, installers, and release automation
│   ├── install_linux_client.sh        # Automated Linux client systemd installation script
│   ├── install_linux_server.sh        # Automated Linux server systemd installation script
│   ├── installer_client.iss           # Inno Setup Windows Client installer script
│   ├── installer_server.iss           # Inno Setup Windows Server installer script
│   ├── nssm.exe                       # Official 64-bit NSSM service manager binary
│   ├── package_dist.py                # Standalone binary compiler & packager
│   ├── requirements.txt               # Unified project dependencies
│   └── upload_release.py              # Gitea REST API release asset publisher
├── src/                               # Core application source modules
│   ├── __init__.py
│   ├── Server.py                      # Central mTLS server, temporal engine, and Hermes REST API
│   ├── server_enrollment.py           # Dynamic PKI, Root CA, and client certificate generator
│   ├── Win_Client.py                  # Windows edge forwarder with auto-enrollment
│   └── Linux_Client.py                # Linux edge forwarder with auto-enrollment
├── tests/                             # Automated test suites
│   ├── test_linux_client.py           # Linux client unit tests & mTLS certificate validation
│   ├── test_pipeline.py               # End-to-end mTLS integration & 4-run verification test
│   ├── test_server.py                 # Server unit tests, PKI generation, and seat quota tests
│   └── test_win_client.py             # Windows client unit tests & mTLS certificate validation
├── .gitignore                         # Ignores venv, caches, DBs, and private keys
├── README.md                          # Architecture and usage documentation
├── RELEASE_NOTES.md                   # Release history and changelog
├── server_config.sample.json          # Reference server configuration
└── out/                               # Component guides and sample configs
    ├── linux_server/
    │   ├── README.md
    │   └── server_config.sample.json
    ├── win_server/
    │   ├── README.md
    │   └── server_config.sample.json
    ├── linux_client/
    │   ├── README.md
    │   └── client_config.sample.json
    └── win_client/
        ├── README.md
        └── client_config.sample.json

Getting Started & Deployment

1. Central Server Hub

Install dependencies:

pip install -r compilation/requirements.txt

Start the server (generates server_config.json, Root CA, and server certs on first run):
```
python src/Server.py
```

Export a client configuration:

python src/Server.py --create-client-config --server-host <SERVER_IP> --server-port 9443 --client-out client_config.json

2. Windows Client Deployment

Download LOGAR-Client-Setup.exe from releases and run it, or place Win_Client.exe and client_config.json in C:\Program Files\LOGAR.
On first run with client_config.json, Win_Client.exe automatically enrolls with the hub, receives its mTLS certificates, and establishes secure streaming.

3. Linux Client Deployment

Run the automated installer:

sudo ./compilation/install_linux_client.sh "http://<HUB_HOST>:8443" "<ENROLLMENT_SECRET>"

The installer enrolls the client, configures /etc/logar/certs, and activates logar-client.service.

Running Tests

1. Component Unit Tests

python -m unittest discover -s tests -v

Runs all 25 unit tests, covering:

Dynamic Root CA generation and server TLS certificate issuance.
Dynamic client certificate issuance with CN and authority key extensions.
Enrollment secret authentication, seat limits, and certificate revocation.
Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload.
Windows & Linux event log collection, deduplication, and mTLS certificate verification.
12-hour evaluation window and 4-run rule progression.

2. End-to-End Pipeline Integration Test

# 1. Initialize test configuration
python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json

# 2. Launch server in background
python src/Server.py &

# 3. Run integration test
python tests/test_pipeline.py

Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, warning 4-run rule promotion, immediate error promotion, and Hermes report output.

Releases & Binary Distribution

Releases can be built and published through three complementary channels:

1. Tag Push Automation (Gitea Actions)

Pushing a release tag automatically triggers the build workflows:

git tag v2.0.1
git push origin v2.0.1

The consolidated workflow release.yml defines two parallel jobs:

release-linux (ubuntu-latest): Compiles standalone native ELF binaries (Linux_Client.bin, Server.bin) and checksums.
release-windows (windows-latest): Compiles Windows executables (Win_Client.exe, Server.exe), builds Inno Setup installers, and publishes checksums (requires self-hosted Windows Act Runner).

2. Manual Workflow Dispatch (Gitea UI)

Workflows can be manually triggered on demand from the Gitea web interface:

Navigate to Actions \rightarrow Release Binaries & Installers (release.yml).
Click Run workflow, set the release tag (defaults to v2.0.1), and run.

3. Native Local Windows Build & Direct Release Upload

For environments without a registered Windows CI runner, Windows executables can be built and published directly to Gitea releases:

# 1. Build Windows binaries locally
python compilation/package_dist.py --target windows

# 2. Upload assets and release notes directly to the Gitea release
python compilation/upload_release.py --tag v2.0.1 --token <GITEA_TOKEN> --skip-build

README.md Unescape Escape

LOGAR: Edge-Thin Log Analysis & Temporal Verification System

Table of Contents