6 Commits
6 changed files with 58 additions and 71 deletions
+18 -24
View File
@@ -1,6 +1,6 @@
# LOGAR: Edge-Thin Log Analysis & Temporal Verification System # LOGAR: Edge-Thin Log Analysis & Temporal Verification System
**LOGAR** is an enterprise log aggregation, verification, and anomaly detection architecture designed for heterogeneous server fleets (Windows & Linux). It combines lightweight zero-state edge forwarders with a centralized cloud hub that applies mutual TLS 1.3 (**mTLS**) authentication, dynamic PKI licensing and quota management, temporal persistence tracking across 12-hour evaluation windows, an automated 4-run rule to filter transient warnings, and immediate pass-through for critical errors before reporting verified anomalies to **Hermes**. **LOGAR** is an enterprise log aggregation, verification, and anomaly detection architecture designed for heterogeneous server fleets (Windows & Linux). It combines lightweight zero-state edge forwarders with a centralized cloud hub that applies mutual TLS 1.3 (**mTLS**) authentication, dynamic PKI licensing and quota management, temporal persistence tracking across 12-hour evaluation windows, and direct anomaly pass-through enabling **Hermes AI** to perform all deep reasoning, anomaly evaluation, and autonomous decision-making.
--- ---
@@ -8,7 +8,7 @@
1. [Core Philosophy](#core-philosophy) 1. [Core Philosophy](#core-philosophy)
2. [Architecture & Data Flow](#architecture--data-flow) 2. [Architecture & Data Flow](#architecture--data-flow)
3. [mTLS Security, Dynamic PKI & Licensing](#mtls-security-dynamic-pki--licensing) 3. [mTLS Security, Dynamic PKI & Licensing](#mtls-security-dynamic-pki--licensing)
4. [Cloud-Side Temporal Persistence & 4-Run Rule](#cloud-side-temporal-persistence--4-run-rule) 4. [Cloud-Side Temporal Persistence & Direct Hermes Reasoning](#cloud-side-temporal-persistence--direct-hermes-reasoning)
5. [Agentic Hermes & Client Management API](#agentic-hermes--client-management-api) 5. [Agentic Hermes & Client Management API](#agentic-hermes--client-management-api)
6. [Dynamic Machine & Domain Identification](#dynamic-machine--domain-identification) 6. [Dynamic Machine & Domain Identification](#dynamic-machine--domain-identification)
7. [Automated Service Installers (Linux & Windows)](#automated-service-installers-linux--windows) 7. [Automated Service Installers (Linux & Windows)](#automated-service-installers-linux--windows)
@@ -28,15 +28,14 @@ Site agents running on Windows and Linux act strictly as lightweight forwarders:
- **Transport Security (mTLS 1.3)**: Logs are streamed directly over mutual TLS 1.3 sockets with hardware-bound / machine-unique client certificates. - **Transport Security (mTLS 1.3)**: Logs are streamed directly over mutual TLS 1.3 sockets with hardware-bound / machine-unique client certificates.
- **Zero Configuration Overhead**: Clients auto-bootstrap certificate enrollment on first run if configured with an enrollment secret. - **Zero Configuration Overhead**: Clients auto-bootstrap certificate enrollment on first run if configured with an enrollment secret.
### 2. Cloud-Side Temporal Persistence & Severity Routing ### 2. Cloud-Side Temporal Persistence & Direct Hermes Reasoning
The central Python hub handles state and verification: The central Python hub handles state and temporal tracking:
- State tracking is managed centrally in SQLite (`logar_state.db`). - State tracking is managed centrally in SQLite (`logar_state.db`).
- Candidate issues are evaluated over a **12-hour temporal evaluation window**. - Candidate issues are tracked over a **12-hour temporal evaluation window** with consecutive run counts and first/last-seen timestamps.
- **Warning Persistence (4-Run Rule)**: `WARNING` issues must persist across **at least 4 consecutive runs / cycles** within the 12-hour window to be confirmed as genuine anomalies, automatically filtering transient infrastructure blips. - **Direct Hermes Intelligence**: All ingested actionable anomalies (`WARNING`, `ERROR`, `INFO`) are immediately promoted to `VERIFIED` and surfaced directly to Hermes. Hermes performs all multi-factor reasoning, pattern evaluation, and contextual thinking rather than relying on arbitrary hardcoded threshold rules.
- **Immediate Error Pass**: Critical errors (`ERROR`, `CRITICAL`, `FATAL`) bypass the 4-run threshold and are promoted immediately to `VERIFIED` on their first occurrence.
### 3. Agentic Integration with Hermes ### 3. Agentic Integration with Hermes
Instead of engineers manually sifting through logs, **Hermes** ingests pre-filtered anomalies directly from the cloud hub (`GET /api/hermes/report`), treating verified errors and 4-run validated warnings as actionable system artifacts to trigger precise notifications and remediations. Instead of engineers manually sifting through logs, **Hermes** ingests verified anomalies directly from the cloud hub (`GET /api/hermes/report`), treating errors and warnings with full temporal metadata (run counts, timestamps) as actionable system artifacts to trigger precise notifications and remediations.
--- ---
@@ -62,7 +61,7 @@ graph TB
subgraph Hub [LOGAR Server Hub] subgraph Hub [LOGAR Server Hub]
INGEST[Length-Prefixed Frame Ingestion] INGEST[Length-Prefixed Frame Ingestion]
DB[(SQLite Persistence<br/>active_issues, clients, license_config)] DB[(SQLite Persistence<br/>active_issues, clients, license_config)]
RULE{12h Window &<br/>4-Run Rule} RULE{12h Window &<br/>Temporal Ingestion}
end end
subgraph Downstream [Hermes Agent & Monitoring] subgraph Downstream [Hermes Agent & Monitoring]
@@ -139,21 +138,16 @@ graph TB
--- ---
## Cloud-Side Temporal Persistence & 4-Run Rule ## Cloud-Side Temporal Persistence & Direct Hermes Reasoning
Incoming candidate logs are tracked in SQLite table `active_issues`: Incoming candidate logs are tracked in SQLite table `active_issues`:
- **Issue Fingerprint**: Formatted as `{site_name}:{server}:{signature}`. - **Issue Fingerprint**: Formatted as `{site_name}:{server}:{signature}`.
- **12-Hour Evaluation Window**: - **12-Hour Evaluation Window**:
- The hub compares `(now - last_seen)`. - The hub compares `(now - last_seen)`.
- If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to `run_count = 1`. - If more than 12 hours have elapsed since the issue was last recorded, the previous window expires and the cycle resets to `run_count = 1`.
- **4-Run Rule for Warnings**: - **Direct Verification & Temporal Telemetry**:
- The 4-run persistence threshold applies to `WARNING` (and `INFO`) events to eliminate transient operational noise. - Ingested anomalies are immediately promoted to `VERIFIED` and made available to Hermes with complete temporal metadata (first seen, last seen, consecutive run counts).
- Each distinct run batch increments `run_count`. - Hermes performs all contextual analysis, pattern recognition, and alerting logic.
- Warnings with `run_count < 4` are marked as `TRANSIENT` and excluded from Hermes reports.
- When `run_count >= 4` within the active 12-hour window, the warning transitions to `VERIFIED`.
- **Immediate Verification for Errors**:
- High-severity events (`ERROR`, `CRITICAL`, `FATAL`) **always pass immediately**.
- On their very first ingestion (`run_count = 1`), errors are promoted directly to `VERIFIED` and surfaced to Hermes without waiting for 4 runs.
--- ---
@@ -202,7 +196,7 @@ Returns seat quota status and registered client telemetry:
``` ```
### `GET /api/hermes/report` ### `GET /api/hermes/report`
Returns all **verified anomalies** (immediate critical errors and warnings verified after 4 consecutive runs within the 12-hour window): Returns all **verified anomalies** within the active 12-hour evaluation window with full temporal metadata for direct Hermes reasoning:
```json ```json
[ [
{ {
@@ -211,7 +205,7 @@ Returns all **verified anomalies** (immediate critical errors and warnings verif
"server": "web-worker-01.corp.internal", "server": "web-worker-01.corp.internal",
"signature": "PostgresPoolExhausted", "signature": "PostgresPoolExhausted",
"severity": "WARNING", "severity": "WARNING",
"message": "Connection pool saturated (>95%) across 4 runs", "message": "Connection pool saturated (>95%)",
"os_type": "linux", "os_type": "linux",
"first_seen": "2026-09-04T07:00:00+00:00", "first_seen": "2026-09-04T07:00:00+00:00",
"last_seen": "2026-09-04T19:00:00+00:00", "last_seen": "2026-09-04T19:00:00+00:00",
@@ -224,7 +218,7 @@ Returns all **verified anomalies** (immediate critical errors and warnings verif
``` ```
### `GET /api/hermes/all` ### `GET /api/hermes/all`
Diagnostic endpoint listing all candidate issues (`TRANSIENT` and `VERIFIED`). Diagnostic endpoint listing all tracked anomalies in SQLite.
### `GET /health` ### `GET /health`
Returns hub health, encryption fingerprint, listener ports, and mTLS status. Returns hub health, encryption fingerprint, listener ports, and mTLS status.
@@ -296,7 +290,7 @@ LOGAR/
│ └── Linux_Client.py # Linux edge forwarder with auto-enrollment │ └── Linux_Client.py # Linux edge forwarder with auto-enrollment
├── tests/ # Automated test suites ├── tests/ # Automated test suites
│ ├── test_linux_client.py # Linux client unit tests & mTLS certificate validation │ ├── test_linux_client.py # Linux client unit tests & mTLS certificate validation
│ ├── test_pipeline.py # End-to-end mTLS integration & 4-run verification test │ ├── test_pipeline.py # End-to-end mTLS integration & direct anomaly verification test
│ ├── test_server.py # Server unit tests, PKI generation, and seat quota tests │ ├── test_server.py # Server unit tests, PKI generation, and seat quota tests
│ └── test_win_client.py # Windows client unit tests & mTLS certificate validation │ └── test_win_client.py # Windows client unit tests & mTLS certificate validation
├── .gitignore # Ignores venv, caches, DBs, and private keys ├── .gitignore # Ignores venv, caches, DBs, and private keys
@@ -362,7 +356,7 @@ Runs all **25 unit tests**, covering:
- Enrollment secret authentication, seat limits, and certificate revocation. - Enrollment secret authentication, seat limits, and certificate revocation.
- Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload. - Proactive certificate validity checks, Root CA auto-renewal, and in-flight server SSLContext reload.
- Windows & Linux event log collection, deduplication, and mTLS certificate verification. - Windows & Linux event log collection, deduplication, and mTLS certificate verification.
- 12-hour evaluation window and 4-run rule progression. - 12-hour evaluation window and direct Hermes anomaly verification.
### 2. End-to-End Pipeline Integration Test ### 2. End-to-End Pipeline Integration Test
```bash ```bash
@@ -375,7 +369,7 @@ python src/Server.py &
# 3. Run integration test # 3. Run integration test
python tests/test_pipeline.py python tests/test_pipeline.py
``` ```
Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, warning 4-run rule promotion, immediate error promotion, and Hermes report output. Tests client enrollment, secret rejection, mTLS TLS 1.3 socket handshake, direct warning promotion, immediate error promotion, and Hermes report output.
--- ---
+3 -3
View File
@@ -14,7 +14,7 @@ Standalone compiled binary and automated systemd service distribution for Linux
- **Master License Authority (`logar.eibl.tech`) & Seat-Bearing CA**: Integrates with the upstream authority. Subordinate CA certificates embed exact seat quotas via custom enterprise X.509 OID `1.3.6.1.4.1.59999.1.1`. - **Master License Authority (`logar.eibl.tech`) & Seat-Bearing CA**: Integrates with the upstream authority. Subordinate CA certificates embed exact seat quotas via custom enterprise X.509 OID `1.3.6.1.4.1.59999.1.1`.
- **Zero-Touch mTLS Renewals**: Once initial signed CA cert is imported, the server hub automatically executes in-flight mTLS renewals with `https://logar.eibl.tech/api/license/renew` when nearing expiration (< 30 days). - **Zero-Touch mTLS Renewals**: Once initial signed CA cert is imported, the server hub automatically executes in-flight mTLS renewals with `https://logar.eibl.tech/api/license/renew` when nearing expiration (< 30 days).
- **Fleet Management & WebUI Dashboard (Port 8443)**: Modern browser WebUI accessible at `http://localhost:8443/webui` to monitor seat quotas, export `.lic` request files, copy CA CSRs, activate signed CA certs, and revoke/reactivate forwarder seats with immediate socket enforcement. - **Fleet Management & WebUI Dashboard (Port 8443)**: Modern browser WebUI accessible at `http://localhost:8443/webui` to monitor seat quotas, export `.lic` request files, copy CA CSRs, activate signed CA certs, and revoke/reactivate forwarder seats with immediate socket enforcement.
- **Warning Persistence & Immediate Error Routing**: High-severity `ERROR`, `CRITICAL`, and `FATAL` events are promoted to `VERIFIED` immediately on their first occurrence. Operational `WARNING` and `INFO` events require persistence across at least 4 distinct client transmission cycles within a rolling 12-hour evaluation window. - **Direct Hermes Intelligence & Temporal Telemetry**: All ingested actionable anomalies (`WARNING`, `ERROR`, `INFO`) are immediately promoted to `VERIFIED` and surfaced directly to Hermes with rich temporal metadata (run counts, timestamps) across a rolling 12-hour evaluation window, allowing Hermes AI to perform all contextual reasoning and decisions.
- **Embedded Hermes Reporting API (Port 8443)**: Integrated REST API exposing `/api/hermes/report`, `/api/clients`, and `/api/client/enroll`. - **Embedded Hermes Reporting API (Port 8443)**: Integrated REST API exposing `/api/hermes/report`, `/api/clients`, and `/api/client/enroll`.
- **State Database**: Stores issue lifecycle records, client telemetry, and licensing quotas in a local SQLite database (`logar_state.db`). - **State Database**: Stores issue lifecycle records, client telemetry, and licensing quotas in a local SQLite database (`logar_state.db`).
@@ -58,7 +58,7 @@ Output:
Transport Security: mTLS (TLS 1.3) Transport Security: mTLS (TLS 1.3)
License Quota: 10 Active Seats License Quota: 10 Active Seats
Server Encryption Fingerprint: 375388960531264EA0648EC0D2C4E4ABC6F22AC2 Server Encryption Fingerprint: 375388960531264EA0648EC0D2C4E4ABC6F22AC2
Evaluation Window: 12 hours | 4-Run Rule: Warnings | Immediate Pass: Errors Evaluation Window: 12 hours | Direct Hermes Reasoning Enabled
============================================================ ============================================================
[*] LOGAR mTLS TLSv1.3 Socket Server listening on 0.0.0.0:9443 [*] LOGAR mTLS TLSv1.3 Socket Server listening on 0.0.0.0:9443
[*] Hermes Reporting API available at http://0.0.0.0:8443/api/hermes/report [*] Hermes Reporting API available at http://0.0.0.0:8443/api/hermes/report
@@ -81,7 +81,7 @@ The generated `server_config.json` contains:
"tls_enabled": true, "tls_enabled": true,
"db_path": "logar_state.db", "db_path": "logar_state.db",
"evaluation_window_hours": 12, "evaluation_window_hours": 12,
"min_persistence_runs": 4, "min_persistence_runs": 1,
"server_fingerprint": "375388960531264EA0648EC0D2C4E4ABC6F22AC2", "server_fingerprint": "375388960531264EA0648EC0D2C4E4ABC6F22AC2",
"public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...", "public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...",
"private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..." "private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..."
+3 -3
View File
@@ -14,7 +14,7 @@ Standalone compiled executable and installer distribution for Windows Server env
- **Master License Authority (`logar.eibl.tech`) & Seat-Bearing CA**: Integrates with the upstream authority. Subordinate CA certificates embed exact seat quotas via custom enterprise X.509 OID `1.3.6.1.4.1.59999.1.1`. - **Master License Authority (`logar.eibl.tech`) & Seat-Bearing CA**: Integrates with the upstream authority. Subordinate CA certificates embed exact seat quotas via custom enterprise X.509 OID `1.3.6.1.4.1.59999.1.1`.
- **Zero-Touch mTLS Renewals**: Once initial signed CA cert is imported, the server hub automatically executes in-flight mTLS renewals with `https://logar.eibl.tech/api/license/renew` when nearing expiration (< 30 days). - **Zero-Touch mTLS Renewals**: Once initial signed CA cert is imported, the server hub automatically executes in-flight mTLS renewals with `https://logar.eibl.tech/api/license/renew` when nearing expiration (< 30 days).
- **Fleet Management & WebUI Dashboard (Port 8443)**: Modern browser WebUI accessible at `http://localhost:8443/webui` to monitor seat quotas, export `.lic` request files, copy CA CSRs, activate signed CA certs, and revoke/reactivate forwarder seats with immediate socket enforcement. - **Fleet Management & WebUI Dashboard (Port 8443)**: Modern browser WebUI accessible at `http://localhost:8443/webui` to monitor seat quotas, export `.lic` request files, copy CA CSRs, activate signed CA certs, and revoke/reactivate forwarder seats with immediate socket enforcement.
- **Warning Persistence & Immediate Error Routing**: High-severity `ERROR`, `CRITICAL`, and `FATAL` events are promoted to `VERIFIED` immediately on their first occurrence. Operational `WARNING` and `INFO` events require persistence across at least 4 distinct transmission cycles within a rolling 12-hour evaluation window. - **Direct Hermes Intelligence & Temporal Telemetry**: All ingested actionable anomalies (`WARNING`, `ERROR`, `INFO`) are immediately promoted to `VERIFIED` and surfaced directly to Hermes with rich temporal metadata (run counts, timestamps) across a rolling 12-hour evaluation window, allowing Hermes AI to perform all contextual reasoning and decisions.
- **Embedded Hermes Reporting API (Port 8443)**: Integrated REST API exposing `/api/hermes/report`, `/api/clients`, and `/api/client/enroll`. - **Embedded Hermes Reporting API (Port 8443)**: Integrated REST API exposing `/api/hermes/report`, `/api/clients`, and `/api/client/enroll`.
- **State Database**: Stores issue lifecycle records, client telemetry, and licensing quotas in a local SQLite database (`logar_state.db`). - **State Database**: Stores issue lifecycle records, client telemetry, and licensing quotas in a local SQLite database (`logar_state.db`).
@@ -58,7 +58,7 @@ Output:
Transport Security: mTLS (TLS 1.3) Transport Security: mTLS (TLS 1.3)
License Quota: 10 Active Seats License Quota: 10 Active Seats
Server Encryption Fingerprint: 375388960531264EA0648EC0D2C4E4ABC6F22AC2 Server Encryption Fingerprint: 375388960531264EA0648EC0D2C4E4ABC6F22AC2
Evaluation Window: 12 hours | 4-Run Rule: Warnings | Immediate Pass: Errors Evaluation Window: 12 hours | Direct Hermes Reasoning Enabled
============================================================ ============================================================
[*] LOGAR mTLS TLSv1.3 Socket Server listening on 0.0.0.0:9443 [*] LOGAR mTLS TLSv1.3 Socket Server listening on 0.0.0.0:9443
[*] Hermes Reporting API available at http://0.0.0.0:8443/api/hermes/report [*] Hermes Reporting API available at http://0.0.0.0:8443/api/hermes/report
@@ -81,7 +81,7 @@ The generated `server_config.json` contains:
"tls_enabled": true, "tls_enabled": true,
"db_path": "logar_state.db", "db_path": "logar_state.db",
"evaluation_window_hours": 12, "evaluation_window_hours": 12,
"min_persistence_runs": 4, "min_persistence_runs": 1,
"server_fingerprint": "375388960531264EA0648EC0D2C4E4ABC6F22AC2", "server_fingerprint": "375388960531264EA0648EC0D2C4E4ABC6F22AC2",
"public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...", "public_key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n...",
"private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..." "private_key": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n..."
+11 -19
View File
@@ -37,7 +37,7 @@ except ImportError:
CONFIG_FILE_NAME = "server_config.json" CONFIG_FILE_NAME = "server_config.json"
DEFAULT_DB_FILE = "logar_state.db" DEFAULT_DB_FILE = "logar_state.db"
EVALUATION_WINDOW_HOURS = 12 EVALUATION_WINDOW_HOURS = 12
RUN_THRESHOLD = 4 RUN_THRESHOLD = 1
app = FastAPI(title="LOGAR Cloud Ingestion & Hermes Hub", version="2.0.1") app = FastAPI(title="LOGAR Cloud Ingestion & Hermes Hub", version="2.0.1")
@@ -260,8 +260,8 @@ def export_license_request_lic(config: Dict[str, Any]) -> Tuple[Dict[str, Any],
def process_ingested_logs(payload: Dict[str, Any], db_path: str, window_hours: int, min_runs: int) -> Dict[str, Any]: def process_ingested_logs(payload: Dict[str, Any], db_path: str, window_hours: int, min_runs: int) -> Dict[str, Any]:
""" """
Evaluates candidate issues against the 12-hour evaluation window and 4-run rule. Evaluates ingested log anomalies and updates temporal run tracking.
Zero-state clients send raw candidate entries; this engine handles temporal state. All anomalies are promoted directly to VERIFIED for Hermes AI reasoning and decision-making.
""" """
client_server = payload.get("server", "unknown-host") client_server = payload.get("server", "unknown-host")
site_name = payload.get("site_name") or (client_server.split(".", 1)[1] if "." in client_server else "default") site_name = payload.get("site_name") or (client_server.split(".", 1)[1] if "." in client_server else "default")
@@ -288,9 +288,6 @@ def process_ingested_logs(payload: Dict[str, Any], db_path: str, window_hours: i
if severity in ["DEBUG", "TRACE"]: if severity in ["DEBUG", "TRACE"]:
continue continue
# Errors are always passed immediately; the 4-run rule only concerns warnings
is_error = severity in ["ERROR", "CRITICAL", "FATAL"]
signature = log.get("signature", "unknown") signature = log.get("signature", "unknown")
server = log.get("server", client_server) server = log.get("server", client_server)
message = log.get("message", "") message = log.get("message", "")
@@ -312,21 +309,17 @@ def process_ingested_logs(payload: Dict[str, Any], db_path: str, window_hours: i
# 12-hour evaluation window expiry check # 12-hour evaluation window expiry check
if (now - last_seen_dt) > timedelta(hours=window_hours): if (now - last_seen_dt) > timedelta(hours=window_hours):
# Window elapsed: reset to new cycle
new_runs = 1 new_runs = 1
new_first_seen = now_iso new_first_seen = now_iso
new_status = "VERIFIED" if is_error else "TRANSIENT"
else: else:
# Same run guard: only increment count once per distinct run batch
if last_run_id != run_id: if last_run_id != run_id:
new_runs = run_count + 1 new_runs = run_count + 1
else: else:
new_runs = run_count new_runs = run_count
new_first_seen = first_seen_str new_first_seen = first_seen_str
# 4-run rule applies to warnings; errors are always passed immediately as VERIFIED
new_status = "VERIFIED" if (is_error or new_runs >= min_runs) else "TRANSIENT"
if new_status == "VERIFIED" and current_status != "VERIFIED": new_status = "VERIFIED"
if current_status != "VERIFIED":
promoted_to_verified += 1 promoted_to_verified += 1
cursor.execute(""" cursor.execute("""
@@ -335,8 +328,7 @@ def process_ingested_logs(payload: Dict[str, Any], db_path: str, window_hours: i
WHERE fingerprint = ? WHERE fingerprint = ?
""", (new_runs, now_iso, new_first_seen, new_status, run_id, message, severity, fp)) """, (new_runs, now_iso, new_first_seen, new_status, run_id, message, severity, fp))
else: else:
initial_status = "VERIFIED" if (is_error or 1 >= min_runs) else "TRANSIENT" initial_status = "VERIFIED"
if initial_status == "VERIFIED":
promoted_to_verified += 1 promoted_to_verified += 1
cursor.execute(""" cursor.execute("""
INSERT INTO active_issues INSERT INTO active_issues
@@ -548,7 +540,7 @@ async def handle_socket_client(reader: asyncio.StreamReader, writer: asyncio.Str
if client_id and "server" not in log_payload: if client_id and "server" not in log_payload:
log_payload["server"] = client_id log_payload["server"] = client_id
# Ingest and apply 12h window / 4-run rule # Ingest and evaluate anomalies with temporal run tracking
res = process_ingested_logs( res = process_ingested_logs(
log_payload, log_payload,
db_path=SERVER_STATE["config"]["db_path"], db_path=SERVER_STATE["config"]["db_path"],
@@ -694,8 +686,8 @@ def list_clients():
def get_verified_anomalies_for_hermes(): def get_verified_anomalies_for_hermes():
""" """
Ingestion endpoint for Hermes agentic workflows. Ingestion endpoint for Hermes agentic workflows.
Returns only verified anomalies that have satisfied the 4-run persistence rule Returns verified anomalies within the active 12-hour evaluation window
within the active 12-hour evaluation window. Transient blips (< 4 runs) are excluded. for Hermes AI direct reasoning and autonomous decision-making.
""" """
db_path = SERVER_STATE["config"]["db_path"] db_path = SERVER_STATE["config"]["db_path"]
conn = sqlite3.connect(db_path) conn = sqlite3.connect(db_path)
@@ -1284,7 +1276,7 @@ def serve_webui():
<span class="logo-icon">🛰️</span> <span class="logo-icon">🛰️</span>
<div> <div>
<h1>LOGAR Fleet Hub & License Management</h1> <h1>LOGAR Fleet Hub & License Management</h1>
<div class="subtitle">Cloud Hub: <strong>__SERVER_NAME__</strong> | Evaluator: 12h Rolling Window & 4-Run Rule</div> <div class="subtitle">Cloud Hub: <strong>__SERVER_NAME__</strong> | Evaluator: 12h Rolling Window & Direct Hermes Intelligence</div>
</div> </div>
</div> </div>
<div class="system-tags"> <div class="system-tags">
@@ -1380,7 +1372,7 @@ def serve_webui():
<div class="card"> <div class="card">
<div class="card-header"> <div class="card-header">
<span class="card-title" style="font-size: 1rem;">Hermes Verified Anomalies (4-Run Rule & Errors)</span> <span class="card-title" style="font-size: 1rem;">Hermes Verified Anomalies</span>
<span class="badge badge-active">Active Window: 12h</span> <span class="badge badge-active">Active Window: 12h</span>
</div> </div>
<table> <table>
+5 -5
View File
@@ -146,7 +146,7 @@ def run_tests():
resp_bytes.extend(chunk) resp_bytes.extend(chunk)
return json.loads(resp_bytes.decode("utf-8")) return json.loads(resp_bytes.decode("utf-8"))
print("\n=== [3] Testing Temporal Persistence & 4-Run Rule for Warnings over mTLS ===") print("\n=== [3] Testing Temporal Persistence & Direct Anomaly Verification over mTLS ===")
test_signature = "TestServiceDegraded" test_signature = "TestServiceDegraded"
candidate_log = [{ candidate_log = [{
"server": client_id, "server": client_id,
@@ -161,10 +161,10 @@ def run_tests():
assert resp.get("status") == "success", f"Run {run_num} failed: {resp}" assert resp.get("status") == "success", f"Run {run_num} failed: {resp}"
promoted = resp.get("promoted_verified", 0) promoted = resp.get("promoted_verified", 0)
print(f"[Run {run_num}/4] Ingested successfully via mTLS. Promoted to verified: {promoted}") print(f"[Run {run_num}/4] Ingested successfully via mTLS. Promoted to verified: {promoted}")
if run_num < 4: if run_num == 1:
assert promoted == 0, f"Expected 0 promoted on run {run_num} for warning, got {promoted}" assert promoted == 1, f"Expected 1 promoted on run 1 for warning, got {promoted}"
else: else:
assert promoted == 1, f"Expected 1 promoted on run 4 for warning, got {promoted}" assert promoted == 0, f"Expected 0 newly promoted on run {run_num} (already verified), got {promoted}"
# Inspect SQLite database directly # Inspect SQLite database directly
conn = sqlite3.connect(server_conf.get("db_path", "logar_state.db")) conn = sqlite3.connect(server_conf.get("db_path", "logar_state.db"))
@@ -178,7 +178,7 @@ def run_tests():
print(f"[DB Verification] Issue '{test_signature}' -> run_count: {run_count}, status: {status}") print(f"[DB Verification] Issue '{test_signature}' -> run_count: {run_count}, status: {status}")
assert run_count >= 4, f"Expected run_count >= 4, got {run_count}" assert run_count >= 4, f"Expected run_count >= 4, got {run_count}"
assert status == "VERIFIED", f"Expected status 'VERIFIED', got {status}" assert status == "VERIFIED", f"Expected status 'VERIFIED', got {status}"
print("[OK] 4-Run Rule verified: Warning promoted to VERIFIED anomaly on 4th run over mTLS!") print("[OK] Direct verification confirmed: Warning immediately verified for Hermes with run count tracking over mTLS!")
print("\n=== [4] Testing Immediate Pass for Errors over mTLS ===") print("\n=== [4] Testing Immediate Pass for Errors over mTLS ===")
error_signature = "TestServiceCrashImmediate" error_signature = "TestServiceCrashImmediate"
+17 -16
View File
@@ -75,7 +75,7 @@ class TestServerComponent(unittest.TestCase):
if os.path.exists(client_out): if os.path.exists(client_out):
os.remove(client_out) os.remove(client_out)
def test_4_run_rule_and_12h_window(self): def test_direct_hermes_anomaly_and_12h_window(self):
Server.init_db(self.test_db) Server.init_db(self.test_db)
log_entry = { log_entry = {
"server": "app-worker-01.corp.local", "server": "app-worker-01.corp.local",
@@ -89,23 +89,24 @@ class TestServerComponent(unittest.TestCase):
"logs": [log_entry] "logs": [log_entry]
} }
# Runs 1 to 3: WARNING should remain TRANSIENT # Run 1: WARNING is immediately VERIFIED for Hermes AI reasoning
for run_idx in range(1, 4): res1 = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=1)
res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=4) self.assertEqual(res1["status"], "success")
self.assertEqual(res["status"], "success") self.assertEqual(res1["promoted_verified"], 1)
self.assertEqual(res["promoted_verified"], 0)
conn = sqlite3.connect(self.test_db) conn = sqlite3.connect(self.test_db)
c = conn.cursor() c = conn.cursor()
c.execute("SELECT run_count, status FROM active_issues WHERE signature = ?", ("PostgresConnWarning",)) c.execute("SELECT run_count, status FROM active_issues WHERE signature = ?", ("PostgresConnWarning",))
row = c.fetchone() row = c.fetchone()
conn.close() conn.close()
self.assertEqual(row[0], 3) self.assertEqual(row[0], 1)
self.assertEqual(row[1], "TRANSIENT") self.assertEqual(row[1], "VERIFIED")
# Run 4: promotes WARNING to VERIFIED! # Runs 2 to 4: Increments consecutive run count while remaining VERIFIED
res4 = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=4) for run_idx in range(2, 5):
self.assertEqual(res4["promoted_verified"], 1) res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=1)
self.assertEqual(res["status"], "success")
self.assertEqual(res["promoted_verified"], 0)
conn = sqlite3.connect(self.test_db) conn = sqlite3.connect(self.test_db)
c = conn.cursor() c = conn.cursor()
@@ -130,7 +131,7 @@ class TestServerComponent(unittest.TestCase):
} }
# Run 1: ERROR must immediately promote to VERIFIED # Run 1: ERROR must immediately promote to VERIFIED
res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=4) res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=1)
self.assertEqual(res["status"], "success") self.assertEqual(res["status"], "success")
self.assertEqual(res["promoted_verified"], 1) self.assertEqual(res["promoted_verified"], 1)
@@ -156,7 +157,7 @@ class TestServerComponent(unittest.TestCase):
{"server": "app-worker-01", "signature": "SigTrace", "severity": "TRACE", "message": "Trace msg", "os_type": "linux"} {"server": "app-worker-01", "signature": "SigTrace", "severity": "TRACE", "message": "Trace msg", "os_type": "linux"}
] ]
} }
res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=4) res = Server.process_ingested_logs(payload, self.test_db, window_hours=12, min_runs=1)
self.assertEqual(res["status"], "success") self.assertEqual(res["status"], "success")
conn = sqlite3.connect(self.test_db) conn = sqlite3.connect(self.test_db)
@@ -171,10 +172,10 @@ class TestServerComponent(unittest.TestCase):
self.assertNotIn("SigDebug", rows) self.assertNotIn("SigDebug", rows)
self.assertNotIn("SigTrace", rows) self.assertNotIn("SigTrace", rows)
# SigErr is immediately VERIFIED; SigWarn and SigInfo are TRANSIENT on run 1 # All actionable issues (ERROR, WARNING, INFO) are promoted to VERIFIED immediately for Hermes reasoning
self.assertEqual(rows["SigErr"], "VERIFIED") self.assertEqual(rows["SigErr"], "VERIFIED")
self.assertEqual(rows["SigWarn"], "TRANSIENT") self.assertEqual(rows["SigWarn"], "VERIFIED")
self.assertEqual(rows["SigInfo"], "TRANSIENT") self.assertEqual(rows["SigInfo"], "VERIFIED")
def test_license_schema_and_pki_generation(self): def test_license_schema_and_pki_generation(self):
secret = "test-secret-12345" secret = "test-secret-12345"