LOGAR: Edge-Thin Log Analysis & Temporal Verification System

LOGAR is an enterprise log aggregation, verification, and anomaly detection architecture designed for heterogeneous server fleets (Windows & Linux). It combines lightweight zero-state edge forwarders with a centralized cloud hub that applies OpenPGP encryption, authenticated TCP streaming, temporal persistence tracking across 12-hour evaluation windows, an automated 4-run rule to filter transient warnings, and immediate pass-through for critical errors before reporting verified anomalies to Hermes.

---

Table of Contents

1. Core Philosophy
2. Architecture & Data Flow
3. Security & Cryptographic Model
4. Cloud-Side Temporal Persistence & 4-Run Rule
5. Agentic Hermes Integration
6. Dynamic Machine & Domain Identification
7. Repository & Shippables Structure
8. Getting Started & Installation
9. Running Tests
10. Automated Releases via Gitea Actions

---

Core Philosophy

1. Edge Thinness & Zero State

Site agents running on Windows and Linux act strictly as lightweight forwarders:

• No Local Database: Clients maintain zero state and no local SQLite or cache files.
• Source-Level Filtering: Agents upload candidate entries spanning from informational events up to errors (INFO, WARNING, ERROR, CRITICAL), while stripping verbose debugging noise (DEBUG, audit entries) and skipping any entries older than 24 hours.
• End-to-End Encryption: Logs are encrypted using the server's OpenPGP public key before leaving the edge node.
• Secure TCP Sockets: Ingestion occurs over low-overhead authenticated TCP sockets rather than bulky HTTP/HTTPS endpoints.

2. Cloud-Side Temporal Persistence & Severity Routing

The central Python/TCP hub handles the heavy lifting:

• State tracking is managed centrally in SQLite (logar_state.db).
• Candidate issues are evaluated over a 12-hour temporal evaluation window.
• Warning Persistence (4-Run Rule): WARNING level issues must persist across at least 4 consecutive runs / cycles within the 12-hour window to be confirmed as genuine anomalies, automatically filtering out transient blips.
• Immediate Error Pass: Critical errors (ERROR, CRITICAL, FATAL) bypass the 4-run threshold and are promoted immediately to VERIFIED on their first occurrence.

3. Agentic Integration with Hermes

Instead of human engineers manually diving through noisy logs, Hermes ingests pre-filtered anomalies directly from the cloud hub (GET /api/hermes/report), treating verified errors and 4-run validated warnings as actionable system artifacts to trigger precise team notifications.

---

Architecture & Data Flow

graph TB
    subgraph Edge Nodes [State-Tracking Edge Forwarders]
        W[Win_Client.py / Win_Client.exe / Win_Client.pyz<br/>Windows Event Log Application]
        L[Linux_Client.py / Linux_Client.bin<br/>systemd journalctl -p info]
    end

    subgraph Security Layer [Security & Framing]
        E[OpenPGP Payload Encryption<br/>Server Public Key & Fingerprint]
        S[Length-Prefixed Framing<br/>4-byte Big-Endian + Auth Envelope]
    end

    subgraph Cloud Hub [LOGAR Central Server Hub]
        TCP[Authenticated TCP Listener<br/>Port 9443]
        DEC[OpenPGP Decryption<br/>Server Private Key]
        DB[(SQLite Persistence<br/>active_issues & ingest_runs)]
        RULE{12h Window &<br/>4-Run Rule}
    end

    subgraph Agentic Reporting [Downstream Integration]
        API[FastAPI / Uvicorn Reporting<br/>Port 8443]
        HERMES[Hermes Agent<br/>GET /api/hermes/report]
    end

    W --> E
    L --> E
    E --> S
    S -->|TCP Stream| TCP
    TCP --> DEC
    DEC --> RULE
    RULE --> DB
    DB --> API
    API --> HERMES

---

Security & Cryptographic Model

Pure-Python OpenPGP (RFC 4880)

• Zero OS Binary Dependency: Utilizes pgpy and cryptography in pure Python. No native GnuPG or gpg binary installation is required on the server, Windows nodes, or Linux nodes.
• First-Run Automatic Key Generation: On the first launch, if server_config.json is missing, Server.py automatically generates:
  • An OpenPGP RSA 2048 keypair with encryption-only usage flags.
  • An armored private key (private_key) and public key (public_key).
  • A SHA-256 public encryption fingerprint (server_fingerprint).
  • A cryptographically random authentication secret token (auth_token).
• Client Configuration Exporter:

  python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json
  

  Produces an anonymous client config containing only the server socket coordinates, authentication token, and the encryption-only public key & fingerprint.
• Socket Protocol Framing:
  • [4 bytes big-endian unsigned int] : Total envelope length.
  • [JSON Envelope] :

    {
      "auth_token": "<SECRET_TOKEN>",
      "timestamp": "2026-09-03T...",
      "encrypted_payload": "-----BEGIN PGP MESSAGE-----\n..."
    }
    

  • Unauthorized clients or invalid authentication tokens are rejected immediately.

---

Cloud-Side Temporal Persistence & 4-Run Rule

Incoming candidate logs are tracked in SQLite table active_issues:

• Issue Fingerprint: Formatted as {site_name}:{server}:{signature}.
• 12-Hour Evaluation Window:
  • When an issue is observed, the hub compares (now - last_seen).
  • If more than 12 hours have passed since the issue was last recorded, the previous window is expired and the cycle resets to run_count = 1.
• 4-Run Rule for Warnings:
  • The 4-run persistence threshold specifically applies to WARNING (and INFO) events to eliminate transient operational noise.
  • For each distinct run batch, run_count increments.
  • Warnings with run_count < 4 are marked as TRANSIENT and excluded from Hermes reports.
  • When run_count >= 4 within the active 12-hour window, the warning transitions to VERIFIED.
• Immediate Verification for Errors:
  • High-severity events (ERROR, CRITICAL, FATAL) always pass immediately.
  • On their very first ingestion (run_count = 1), errors are promoted directly to VERIFIED and surfaced to Hermes without waiting for 4 runs.

---

Agentic Hermes Integration

The server hub serves a REST reporting API (default port 8443):

GET /api/hermes/report

Returns all verified anomalies (immediate critical errors and warnings verified after 4 consecutive runs within the 12-hour evaluation window):

[
  {
    "fingerprint": "corp.internal:web-app-01.corp.internal:NginxWorkerCrash",
    "site": "corp.internal",
    "server": "web-app-01.corp.internal",
    "signature": "NginxWorkerCrash",
    "severity": "ERROR",
    "message": "Worker process 4120 terminated with signal 11",
    "os_type": "linux",
    "first_seen": "2026-09-03T09:00:00+00:00",
    "last_seen": "2026-09-03T21:00:00+00:00",
    "consecutive_runs": 4,
    "evaluation_window": "12h",
    "verified": true,
    "status": "VERIFIED"
  }
]

GET /api/hermes/all

Diagnostic endpoint listing all active issues (both TRANSIENT candidate blips and VERIFIED anomalies).

GET /health

Returns hub health, encryption fingerprint, and listener ports.

---

Dynamic Machine & Domain Identification

Client configurations intentionally contain no machine name or site name. Both forwarders dynamically identify their host and domain at runtime via get_machine_identifier():

1. Fully Qualified Domain Name (FQDN): Checked via socket.getfqdn().
2. OS-Specific Domain Discovery:
   • Windows: Checks Active Directory environment variable USERDNSDOMAIN / USERDOMAIN.
   • Linux: Parses /etc/resolv.conf domain and search directives.
3. Reverse DNS Lookup: Resolves canonical hostname via socket.gethostbyaddr.
4. Fallback: Local hostname socket.gethostname().

The server automatically infers site attribution from domain qualifiers (e.g. node01.corp.internal \rightarrow site corp.internal).

---

Repository & Shippables Structure

LOGAR/
├── .gitea/
│   └── workflows/
│       ├── ci.yml                  # Continuous Integration automated test suite (runs on every push)
│       ├── release-linux.yml       # Linux release workflow (compiles Server.bin, Linux_Client.bin, checksums)
│       └── release-windows.yml     # Windows release workflow (compiles Server.exe, Win_Client.exe, checksums)
├── compilation/                    # Build, packaging, and release automation tools
│   ├── package_dist.py             # Multi-platform standalone binary packaging script
│   ├── requirements.txt            # Unified project dependencies
│   └── upload_release.py           # Direct Gitea REST API release asset publisher
├── src/                            # Core application source modules
│   ├── __init__.py
│   ├── Server.py                   # Central TCP server and Hermes API
│   ├── Win_Client.py               # Windows edge forwarder
│   └── Linux_Client.py             # Linux edge forwarder
├── tests/                          # Automated test suites
│   ├── test_linux_client.py        # Linux client unit tests
│   ├── test_pipeline.py            # End-to-end integration test
│   ├── test_server.py              # Server unit tests
│   └── test_win_client.py          # Windows client unit tests
├── .gitignore                      # Ignore venv, caches, DBs, and private keys
├── README.md                       # Comprehensive documentation
├── RELEASE_NOTES.md                # Release history and changelog
├── server_config.sample.json       # Central server sample configuration
└── out/                            # Standalone deployment documentation & sample configs
    ├── linux_server/
    │   ├── README.md               # Linux systemd service installation & hub guide
    │   └── server_config.sample.json # Reference server configuration
    ├── win_server/
    │   ├── README.md               # Windows service (NSSM/Task Scheduler) installation guide
    │   └── server_config.sample.json # Reference server configuration
    ├── linux_client/
    │   ├── README.md               # Linux service & timer installation guide
    │   └── client_config.sample.json # Reference client configuration
    └── win_client/
        ├── README.md               # Windows service installation & configuration guide
        └── client_config.sample.json # Reference client configuration

---

Getting Started & Installation

1. Central Server Hub

1. Install dependencies:

   pip install -r compilation/requirements.txt
   

2. Start the server (generates server_config.json and keypair on first run):

   python src/Server.py
   

3. Export a client configuration:

   python src/Server.py --create-client-config --server-host <SERVER_IP> --server-port 9443 --client-out client_config.json
   

2. Windows Client Deployment

Option A: Precompiled Standalone Executable (Recommended)

1. Download Win_Client.exe (or Win_Client.pyz) from the repository releases.
2. Place client_config.json (exported from the server) in the same directory.
3. Run manually or schedule via Task Scheduler (every 3 hours):

   .\Win_Client.exe --hours 24
   

Option B: Python Source Execution

1. Copy Win_Client.py, requirements.txt, and client_config.json to the target machine.
2. Install client dependencies:

   python -m pip install -r requirements.txt
   

3. Run manually or schedule via Task Scheduler:

   python Win_Client.py --hours 24
   

3. Linux Client Deployment

Option A: Precompiled Standalone Binary (Recommended)

1. Download Linux_Client.bin from the repository releases.
2. Place Linux_Client.bin and client_config.json into /opt/logar/ and make it executable:

   chmod +x /opt/logar/Linux_Client.bin
   

3. Run via cron or systemd timer:

   0 */3 * * * /opt/logar/Linux_Client.bin --hours 24
   

Option B: Python Source Execution

1. Copy Linux_Client.py, requirements.txt, and client_config.json to /opt/logar/.
2. Install client dependencies:

   python3 -m pip install -r requirements.txt
   

3. (Optional) Run out/linux_client/build_bin.sh to compile a standalone ELF binary locally if desired.
4. Run via cron or systemd timer:

   0 */3 * * * python3 /opt/logar/Linux_Client.py --hours 24
   

---

Running Tests

1. Component-Specific Unit Tests

The test suite is located in tests/ and exercises all components:

# Run all unit tests
python -m unittest discover -s tests

# Or run component tests individually:
python -m unittest tests/test_server.py
python -m unittest tests/test_win_client.py
python -m unittest tests/test_linux_client.py

2. End-to-End Pipeline Integration Test

The pipeline test exercises invalid token rejection, encrypted socket streaming, database persistence, status promotion upon the 4th run, and the Hermes API report output.

1. Start the server in Shell 1 (creates server_config.json on first run):

   python src/Server.py
   

2. Export client configuration in Shell 2 (required for testing):

   python src/Server.py --create-client-config --server-host 127.0.0.1 --server-port 9443 --client-out client_config.json
   

3. Execute the integration test in Shell 2:

   python tests/test_pipeline.py
   

Continuous Integration via Gitea Actions

Continuous integration is automated via .gitea/workflows/ci.yml and triggers automatically on every push and pull request:

1. Syntax Compilation: Validates all Python scripts (src/Server.py, src/Win_Client.py, src/Linux_Client.py, compilation/package_dist.py, compilation/upload_release.py, tests/test_pipeline.py, and test suites).
2. Component Unit Tests: Discovers and runs all unit tests in tests/ (test_server.py, test_win_client.py, test_linux_client.py).
3. End-to-End Pipeline Verification: Automatically spins up the LOGAR server hub, generates test configs, runs tests/test_pipeline.py (testing socket authentication, 4-run rule persistence, Hermes API report, and client integrations), and shuts down the test instance.

---

Automated Releases via Gitea Actions

Release builds are automated via two dedicated Gitea Actions workflows running concurrently on native platform runners:

• .gitea/workflows/release-linux.yml (ubuntu-latest)
• .gitea/workflows/release-windows.yml (windows-latest)

Publishing a Release

Whenever you want to release a new version with compiled standalone binaries:

git tag v1.0.1
git push origin v1.0.1

(You can also trigger builds manually via the Gitea UI using the Run workflow button (workflow_dispatch) on either workflow).

Automated Multi-Platform Compilation:

1. Linux Runner (release-linux.yml):

   • Compiles native Linux ELF executables: Linux_Client.bin and Server.bin.
   • Generates dedicated SHA-256 checksum files:
     • linux_client_sha256sum (verification for Linux_Client.bin)
     • linux_agent_sha256sum (alias for client/agent integrations)
     • linux_server_sha256sum (verification for Server.bin)
     • SHA256SUMS_linux.txt (summary manifest)
   • Attaches all Linux assets to the Gitea release.

2. Windows Runner (release-windows.yml):

   • Compiles native Windows PE executables: Win_Client.exe and Server.exe.
   • Generates dedicated SHA-256 checksum files:
     • win_client_sha256sum (verification for Win_Client.exe)
     • win_agent_sha256sum (alias for client/agent integrations)
     • win_server_sha256sum (verification for Server.exe)
     • SHA256SUMS_windows.txt (summary manifest)
   • Attaches all Windows assets to the Gitea release.

3. Concurrent Publishing & Conflict Handling: upload_release.py includes automatic retry and conflict resolution so concurrent Windows and Linux runners attach their respective assets to the release without collision.

Verifying Checksums

• On Linux:

  sha256sum -c linux_client_sha256sum
  # or
  sha256sum -c linux_server_sha256sum
  

• On Windows (PowerShell):

  Get-FileHash .\Win_Client.exe -Algorithm SHA256
  Get-Content .\win_client_sha256sum
  

Local Packaging & Manual Upload

You can also compile and package binaries locally anytime:

# Windows
py -3.12 compilation/package_dist.py --target windows

# Linux
python3 compilation/package_dist.py --target linux

To upload local builds directly to Gitea:

python compilation/upload_release.py --tag v1.0.1 --token <YOUR_GITEA_TOKEN>

(Environment variables GITEA_TOKEN, GITEA_SERVER_URL, GITEA_REPOSITORY, and GITEA_REF_NAME are also supported automatically).