Compare commits

..
2 Commits
Author SHA1 Message Date
Julian Boldt b020e38d0a Release v1.31.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Regel-Bereinigung: Entfernen, Bestätigen, Löschen, Wiederherstellen:** Im Tab **Bereinigung** (Regelwerk › Regel-Analyse) lassen sich Kandidaten jetzt auswählen
  (Checkbox, höchstens so viele Regeln je Lauf wie in den Einstellungen; „vermutlich“ nur mit einer ausdrücklichen Einzelbestätigung je Regel, „unbekannt“ nie).
  **Vorschau** zeigt je Firewall (im Cluster zuerst der Backup) die Regeln mit Befund, Sicherheit und Schutzprüfung; **Ausrollen** deaktiviert die Regeln und wendet an.
  Danach läuft ein Countdown: **Bestätigen** heißt „ich erreiche die Firewall noch“, ohne Bestätigung setzt OPNexus **automatisch zurück**; bis dahin geht auch
  **Jetzt zurücksetzen**. Bestätigte Regeln lassen sich nach der Wartezeit (einstellbar, Standard 7 Tage) **löschen**; vorher sichert OPNexus den Regelinhalt, und
  unter **Verlauf der Bereinigungen** kommt die Regel mit **Wiederherstellen** deaktiviert und möglichst an alter Position zurück (sonst mit Hinweis am Ende der Gruppe);
  **Wieder aktivieren** hat wieder ein Bestätigungsfenster. Wichtig: **Ausrollen, Wiederholen und Wiederherstellen darf nur ein Administrator**, eine Vorschau gehört
  dem, der sie erstellt hat (ein Admin öffnet selbst eine neue); **Bestätigen geht nur in der Browser-Sitzung**, nie per API-Token. **Die Bereinigung bleibt gesperrt,
  bis unter Einstellungen die Verwaltungsquellen** (IP-Adressen oder Netze, von denen aus verwaltet wird) **eingetragen sind**: Regeln, die diesen Zugang treffen
  könnten, deaktiviert OPNexus nie. Dort stehen auch die Obergrenze je Lauf, die Wartezeit bis zum Löschen und **weitere Verwaltungsports** (zusätzlich zu
  API-Port, 443, 80, 22). Tragen Sie die Adresse ein, mit der OPNexus selbst bei den Firewalls ankommt (im Cluster für jeden Knoten). Quellen, Ports und
  Firewall-Adressen sehen nur Administratoren. Der Hinweis „Entfernen folgt“ entfällt.
- **Update: neuere CLI aus dem Zielrelease:** `opnexus update` (auch per Klick im Webinterface) holt sich nach dem Pull die CLI aus dem Updater-Image des
  Zielreleases und startet sich damit neu, damit Korrekturen am Update-Ablauf sofort greifen statt erst beim übernächsten Update. Das geschieht nur, wenn
  `cosign` und `cosign.pub` vorhanden sind und die Signatur des Image-**Digests** in diesem Lauf geprüft wurde (ausgeführt wird genau dieser Digest);
  sonst und bei jedem Fehler läuft das Update mit der vorhandenen CLI weiter. Abschalten: `OPNEXUS_NO_CLI_REFRESH=1`. Wirksam wird es ab dem Update
  *von* dieser Version.
2026-10-22 21:04:32 +02:00
Julian Boldt abd15318a3 Release v1.30.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
  Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
  trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
  dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
  höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
  verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
  Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
  stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
  Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
  verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
  Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
  automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
  erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
  Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
  15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
  Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
  mehr verdeckte Regeln als bisher, nie weniger belegte.
2026-10-22 10:27:02 +02:00
91 changed files with 11744 additions and 117 deletions

No files matched your search

+43
View File
@@ -31,6 +31,49 @@ den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin …
## [1.31.0] "Vulture" — 2026-10-22
- **Regel-Bereinigung: Entfernen, Bestätigen, Löschen, Wiederherstellen:** Im Tab **Bereinigung** (Regelwerk › Regel-Analyse) lassen sich Kandidaten jetzt auswählen
(Checkbox, höchstens so viele Regeln je Lauf wie in den Einstellungen; „vermutlich“ nur mit einer ausdrücklichen Einzelbestätigung je Regel, „unbekannt“ nie).
**Vorschau** zeigt je Firewall (im Cluster zuerst der Backup) die Regeln mit Befund, Sicherheit und Schutzprüfung; **Ausrollen** deaktiviert die Regeln und wendet an.
Danach läuft ein Countdown: **Bestätigen** heißt „ich erreiche die Firewall noch“, ohne Bestätigung setzt OPNexus **automatisch zurück**; bis dahin geht auch
**Jetzt zurücksetzen**. Bestätigte Regeln lassen sich nach der Wartezeit (einstellbar, Standard 7 Tage) **löschen**; vorher sichert OPNexus den Regelinhalt, und
unter **Verlauf der Bereinigungen** kommt die Regel mit **Wiederherstellen** deaktiviert und möglichst an alter Position zurück (sonst mit Hinweis am Ende der Gruppe);
**Wieder aktivieren** hat wieder ein Bestätigungsfenster. Wichtig: **Ausrollen, Wiederholen und Wiederherstellen darf nur ein Administrator**, eine Vorschau gehört
dem, der sie erstellt hat (ein Admin öffnet selbst eine neue); **Bestätigen geht nur in der Browser-Sitzung**, nie per API-Token. **Die Bereinigung bleibt gesperrt,
bis unter Einstellungen die Verwaltungsquellen** (IP-Adressen oder Netze, von denen aus verwaltet wird) **eingetragen sind**: Regeln, die diesen Zugang treffen
könnten, deaktiviert OPNexus nie. Dort stehen auch die Obergrenze je Lauf, die Wartezeit bis zum Löschen und **weitere Verwaltungsports** (zusätzlich zu
API-Port, 443, 80, 22). Tragen Sie die Adresse ein, mit der OPNexus selbst bei den Firewalls ankommt (im Cluster für jeden Knoten). Quellen, Ports und
Firewall-Adressen sehen nur Administratoren. Der Hinweis „Entfernen folgt“ entfällt.
- **Update: neuere CLI aus dem Zielrelease:** `opnexus update` (auch per Klick im Webinterface) holt sich nach dem Pull die CLI aus dem Updater-Image des
Zielreleases und startet sich damit neu, damit Korrekturen am Update-Ablauf sofort greifen statt erst beim übernächsten Update. Das geschieht nur, wenn
`cosign` und `cosign.pub` vorhanden sind und die Signatur des Image-**Digests** in diesem Lauf geprüft wurde (ausgeführt wird genau dieser Digest);
sonst und bei jedem Fehler läuft das Update mit der vorhandenen CLI weiter. Abschalten: `OPNEXUS_NO_CLI_REFRESH=1`. Wirksam wird es ab dem Update
*von* dieser Version.
## [1.30.0] "Owl" — 2026-10-22
- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
mehr verdeckte Regeln als bisher, nie weniger belegte.
## [1.29.0] "Caracara" — 2026-10-22
- **Fehlende API-Rechte werden sichtbar:** Darf der API-Schlüssel einer Firewall den Gateway- oder Dienststatus nicht lesen (OPNsense antwortet
+3 -3
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
> **Stand:** `1.29.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
> **Stand:** `1.31.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen
@@ -26,7 +26,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
- **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
- **Automatisierung** — persönliche API-Tokens (Lesen / Ausrollen / Admin), Prometheus-Metriken, ein nur lesender MCP-Endpunkt für KI-Assistenten und ein OpenAPI-Schema.
- **Sicherheitslage** — ein Wert von 0 bis 100 je Firewall aus Härtungs- und Regel-Hygiene-Prüfungen (offene WAN-Regeln, schwaches SSH/SNMP, verdeckte Regeln, alte Backups …), nur lesend.
- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend.
- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend. Eine Bereinigungsliste nennt ungenutzte, verdeckte, doppelte und lange deaktivierte Regeln (jeweils mit Sicherheit); ein Administrator kann ausgewählte Regeln abschalten (Vorschau, dann Ausrollen mit automatischer Rücknahme ohne Bestätigung), später löschen und aus einer Sicherung des Regelinhalts wiederherstellen. Jede Regel hat einen Änderungsverlauf – auch für Änderungen außerhalb von OPNexus.
- **Backups & Wiederherstellung** — geplante Konfigurationssicherungen mit Aufbewahrung, verschlüsselte Offsite-Kopie auf S3-kompatiblen Speicher, Änderungsansicht zwischen Sicherungen und eine vollständige verschlüsselte Sicherung von OPNexus selbst (`opnexus backup --full`).
- **Single Sign-on** — Anmeldung über deinen Identitätsanbieter (OIDC); die Rolle kommt bei jeder Anmeldung aus dessen Gruppen.
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Operator / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
@@ -65,7 +65,7 @@ Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelie
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.29.0
OPNEXUS_VERSION=1.31.0
```
```bash
+3 -3
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](docs/screenshots/readme-rules-en.webp)
> **Status:** `1.29.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
> **Status:** `1.31.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does
@@ -26,7 +26,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
- **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
- **Automation** — personal API tokens (read / deploy / admin), Prometheus metrics, a read-only MCP endpoint for AI assistants and an OpenAPI schema.
- **Security posture** — a 0–100 score per firewall from hardening and rule-hygiene checks (open WAN rules, weak SSH/SNMP, shadowed rules, stale backups …), read-only.
- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only.
- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only. A cleanup list names unused, shadowed, duplicate and long-disabled rules (each with its confidence); an administrator can switch selected rules off (preview, then roll-out with an automatic revert unless confirmed), delete them later and restore them from a backup of their content. Every rule has a change history, including changes made outside OPNexus.
- **Backups & recovery** — scheduled config backups with retention, an encrypted offsite copy to S3-compatible storage, a redacted change view between backups, and a full encrypted backup/restore of OPNexus itself (`opnexus backup --full`).
- **Single sign-on** — sign in with your identity provider (OIDC); roles come from its groups at every login.
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / operator / read-only), keyboard operation, dark mode, German and English UI.
@@ -67,7 +67,7 @@ For access from other machines and for production use with TLS (bundled Caddy pr
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.29.0
OPNEXUS_VERSION=1.31.0
```
```bash
+7
View File
@@ -160,6 +160,13 @@ class Settings(BaseSettings):
# wireguard_key_store.py.
wireguard_key_secret_dir: str = "/data/wireguard-key-secrets"
# ── Rule observation (usage counters + change history of the live rules) ─
# Two read-only GETs per enabled firewall per cycle (searchRule, ruleStats). The interval is an operator
# setting (env RULE_OBSERVE_INTERVAL_SECONDS); it can not go below the floor, which keeps a typo from
# polling every firewall in a tight loop.
rule_observe_interval_seconds: int = 900
rule_observe_min_interval_seconds: int = 60
# ── Resource monitoring ──────────────────────────────────────────────
# Fixed interval, no admin-editable settings row/UI for this — v1
# deliberately kept minimal (see resource_monitor_service.py).
+25
View File
@@ -171,6 +171,7 @@ class ErrorCode(str, Enum):
OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED"
OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY"
OPEN_CLEANUP_BLOCKS_DEPLOY = "OPEN_CLEANUP_BLOCKS_DEPLOY"
UPSTREAM_REJECTED = "UPSTREAM_REJECTED"
INVALID_CHARACTERS = "INVALID_CHARACTERS"
SERVICE_NOT_FOUND = "SERVICE_NOT_FOUND"
@@ -222,6 +223,30 @@ class ErrorCode(str, Enum):
ANALYSE_INTERFACE_UNBEKANNT = "ANALYSE_INTERFACE_UNBEKANNT"
ANALYSE_PARAMETER_UNGUELTIG = "ANALYSE_PARAMETER_UNGUELTIG"
ANALYSE_FIREWALL_NICHT_LESBAR = "ANALYSE_FIREWALL_NICHT_LESBAR"
# Rule cleanup (1.31, services/rule_cleanup_service.py)
BEREINIGUNG_NICHT_GEFUNDEN = "BEREINIGUNG_NICHT_GEFUNDEN"
BEREINIGUNG_AUSWAHL_UNGUELTIG = "BEREINIGUNG_AUSWAHL_UNGUELTIG"
BEREINIGUNG_VORSCHAU_VERALTET = "BEREINIGUNG_VORSCHAU_VERALTET"
BEREINIGUNG_REGEL_NICHT_AENDERBAR = "BEREINIGUNG_REGEL_NICHT_AENDERBAR"
BEREINIGUNG_OFFENE_OPERATION = "BEREINIGUNG_OFFENE_OPERATION"
BEREINIGUNG_ZU_VIELE_REGELN = "BEREINIGUNG_ZU_VIELE_REGELN"
BEREINIGUNG_BESTAETIGUNG_FEHLT = "BEREINIGUNG_BESTAETIGUNG_FEHLT"
BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN = "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG = "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG"
BEREINIGUNG_FRIST_ABGELAUFEN = "BEREINIGUNG_FRIST_ABGELAUFEN"
BEREINIGUNG_NICHT_BESTAETIGBAR = "BEREINIGUNG_NICHT_BESTAETIGBAR"
BEREINIGUNG_LIVESTAND_ABWEICHEND = "BEREINIGUNG_LIVESTAND_ABWEICHEND"
BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR = "BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR"
BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT = "BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT"
BEREINIGUNG_SNAPSHOT_FEHLT = "BEREINIGUNG_SNAPSHOT_FEHLT"
BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR = "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
BEREINIGUNG_AKTION_NICHT_MOEGLICH = "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
BEREINIGUNG_NICHT_AUSGEFUEHRT = "BEREINIGUNG_NICHT_AUSGEFUEHRT"
BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN = "BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN"
BEREINIGUNG_AUSGANG_UNKLAR = "BEREINIGUNG_AUSGANG_UNKLAR"
BEREINIGUNG_ABGEBROCHEN_VERWAIST = "BEREINIGUNG_ABGEBROCHEN_VERWAIST"
BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET = "BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET"
BEREINIGUNG_EINSTELLUNG_UNGUELTIG = "BEREINIGUNG_EINSTELLUNG_UNGUELTIG"
def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:
+36 -1
View File
@@ -53,6 +53,7 @@ from app.routers import scheduled_maintenance as scheduled_maintenance_router
from app.routers import maintenance_windows as maintenance_windows_router
from app.routers import rollouts as rollouts_router
from app.routers import rule_analysis as rule_analysis_router
from app.routers import rule_cleanup as rule_cleanup_router
from app.routers import gui_access as gui_access_router
from app.services import scheduled_maintenance_service
from app.services import rollout_engine
@@ -61,6 +62,8 @@ from app.services import audit_export_service
from app.services import overlay_service, variables_service
from app.services import backup_schedule_service
from app.services import carp_event_service, carp_maintenance_window_service
from app.services import rule_history_service
from app.services import rule_cleanup_service
# Without this, Python's root logger defaults to WARNING with no handler,
# so every logger.info() call in the 5 background loops below (CARP role
@@ -141,6 +144,13 @@ async def _deploy_revert_sweep_loop():
logger.info("Auto-reverted %d unconfirmed policy-hierarchy deployment(s)", len(reverted))
except Exception:
logger.exception("Policy-hierarchy deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await rule_cleanup_service.sweep_expired_cleanups(db)
if reverted:
logger.info("Regel-Bereinigung: %d Operation(en) automatisch zurückgenommen/abgeschlossen", len(reverted))
except Exception:
logger.exception("Rule-cleanup revert sweep failed")
await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
@@ -234,6 +244,9 @@ async def _resource_poll_loop():
deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
if deleted:
logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
pruned = await rule_history_service.prune(db)
if pruned:
logger.info("Regelverlauf-Retention: %d alte Einträge gelöscht", pruned)
except Exception:
logger.exception("Resource-Poll fehlgeschlagen")
await asyncio.sleep(settings.resource_poll_interval_seconds)
@@ -322,6 +335,25 @@ async def _backup_poll_loop():
await asyncio.sleep(_BACKUP_POLL_SECONDS)
_RULE_OBSERVE_FIRST_RUN_SECONDS = 150
async def _rule_observe_loop():
"""Ninth background loop: reads the rules and their counters of every enabled firewall (two GETs each),
keeps the usage sampling current and records what changed since the last look (services/rule_history_service.py).
Writes only to the database. The first look at a firewall only stores the starting point."""
await asyncio.sleep(_RULE_OBSERVE_FIRST_RUN_SECONDS)
while True:
try:
async with SessionLocal() as db:
result = await rule_history_service.observe_all(db)
if result["events"] or result["failed"]:
logger.info("Regelverlauf: %s", result)
except Exception:
logger.exception("Regelverlauf fehlgeschlagen")
await asyncio.sleep(max(settings.rule_observe_interval_seconds, settings.rule_observe_min_interval_seconds))
@asynccontextmanager
async def lifespan(app: FastAPI):
conflicts = settings.gui_origin_conflicts()
@@ -351,6 +383,7 @@ async def lifespan(app: FastAPI):
compliance_task = asyncio.create_task(_compliance_poll_loop())
audit_export_task = asyncio.create_task(_audit_export_poll_loop())
backup_task = asyncio.create_task(_backup_poll_loop())
rule_observe_task = asyncio.create_task(_rule_observe_loop())
try:
yield
finally:
@@ -362,7 +395,8 @@ async def lifespan(app: FastAPI):
compliance_task.cancel()
audit_export_task.cancel()
backup_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task):
rule_observe_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task, rule_observe_task):
try:
await task
except asyncio.CancelledError:
@@ -536,5 +570,6 @@ app.include_router(scheduled_maintenance_router.router, prefix="/api")
app.include_router(maintenance_windows_router.router, prefix="/api")
app.include_router(rollouts_router.router, prefix="/api")
app.include_router(rule_analysis_router.router, prefix="/api")
app.include_router(rule_cleanup_router.router, prefix="/api")
app.include_router(gui_access_router.router, prefix="/api")
app.include_router(gui_access_router.internal_router)
+6
View File
@@ -102,3 +102,9 @@ __all__ += ["RolloutPlan", "RolloutPlanWave", "RolloutRun", "RolloutRunHost", "R
from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
__all__ += ["GuiAccessSettings", "GuiSession", "GuiTicket"]
from app.models.rule_history import LiveRuleState, RuleChangeEvent
__all__ += ["LiveRuleState", "RuleChangeEvent"]
from app.models.rule_cleanup import RuleCleanupOperation, RuleCleanupItem
__all__ += ["RuleCleanupOperation", "RuleCleanupItem"]
+12 -1
View File
@@ -1,5 +1,5 @@
"""Separation of duties (singleton row): may operators roll changes out themselves?"""
from sqlalchemy import Boolean, Integer
from sqlalchemy import JSON, Boolean, Integer
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
@@ -11,3 +11,14 @@ class GovernanceSettings(Base):
id: Mapped[int] = mapped_column(Integer, primary_key=True)
# False: an operator prepares changes (edit, preview) but only an admin may deploy them.
operators_may_deploy: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Rule cleanup (1.31). At most this many rules per cleanup operation (default 10, ceiling 25).
cleanup_max_rules: Mapped[int] = mapped_column(Integer, nullable=False, default=10)
# A rule must have been switched off (and confirmed) this many days before it may be deleted (minimum 1).
cleanup_delete_min_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7)
# IPs/CIDRs the admin manages the firewalls from; no rule that could match them may be cleaned up. Empty list =
# the cleanup is locked (fail closed). Only an admin may change it.
cleanup_management_sources: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# Extra TCP ports the firewalls are managed on (besides the API port of base_url, 443, 80 and 22): a rule that
# could match a management source on one of them is protected too. Only an admin may change it.
cleanup_management_ports: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
+106
View File
@@ -0,0 +1,106 @@
"""Rule cleanup (1.31, B4 part 2): switching rules off or deleting them through preview -> deploy -> confirm.
``RuleCleanupOperation`` is one cleanup run on a host (or a cluster pair, anchored at one host); its id doubles as
the preview token. ``RuleCleanupItem`` is one rule on one node of that run. The columns only carry what the design
(DESIGN-1.31) needs to resume, revert and restore after a crash; the logic lives in services/rule_cleanup_service.py.
"""
import uuid
from datetime import datetime, timezone
from typing import Optional
from sqlalchemy import JSON, DateTime, ForeignKey, Index, Integer, String, Text, text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
# kind: disable | enable (re-enable after a restore) | delete | restore
OPERATION_KINDS = ("disable", "enable", "delete", "restore")
# preview -> running -> awaiting_confirm -> confirmed; the rest are the side exits.
OPERATION_STATUSES = (
"preview", "running", "awaiting_confirm", "confirmed", "partial", "apply_failed", "reverting", "reverted",
"revert_failed", "done", "failed", "cancelled", "expired",
)
ITEM_STATES = (
"queued", "disabling", "disabled", "confirmed", "reverting", "reverted", "deleting", "deleted", "restoring",
"restored", "failed", "unknown", "revert_failed",
# never attempted (the run stopped at an earlier error); not an open state, so the rule is free again
"skipped",
)
# A rule in one of these states is being worked on or its outcome is unresolved: no second item may claim it.
OPEN_ITEM_STATES = ("queued", "disabling", "disabled", "reverting", "deleting", "restoring", "unknown", "revert_failed")
def _now() -> datetime:
return datetime.now(timezone.utc)
class RuleCleanupOperation(Base):
__tablename__ = "rule_cleanup_operations"
__table_args__ = (
Index("ix_rule_cleanup_operations_host", "host_id", "created_at"),
Index("ix_rule_cleanup_operations_status", "status"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
cluster_name: Mapped[Optional[str]] = mapped_column(String(120), nullable=True)
kind: Mapped[str] = mapped_column(String(16), nullable=False)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="preview")
actor: Mapped[str] = mapped_column(String(150), nullable=False)
# The stable account id next to the name (see migration 0026); no foreign key, a deleted account stays gone.
actor_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=_now)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
# Written at every checkpoint, so another process can tell an abandoned ``running`` operation from a live one.
heartbeat_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
# Auto-revert deadline (disable/enable only); NULL once confirmed or for kinds without a window.
revert_deadline: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
confirmed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
# sha256 over everything the preview showed; the deploy recomputes it and refuses on any difference.
preview_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
# unused_days, acknowledged rules (with the text version), protected management sources.
options: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
class RuleCleanupItem(Base):
__tablename__ = "rule_cleanup_items"
__table_args__ = (
Index("ix_rule_cleanup_items_operation", "operation_id"),
# Independent of any lock: one rule on one node can be in only one unfinished item at a time.
Index(
"uq_cleanup_rule_open", "host_id", "rule_uuid", unique=True,
postgresql_where=text("state IN (" + ", ".join(f"'{s}'" for s in OPEN_ITEM_STATES) + ")"),
),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
operation_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rule_cleanup_operations.id", ondelete="CASCADE"), nullable=False)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
seq: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
carp_role: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
state: Mapped[str] = mapped_column(String(16), nullable=False, default="queued")
# What the preview showed.
description: Mapped[str] = mapped_column(Text, nullable=False, default="")
categories: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
confidence: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
row_before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
fp_before: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
# shadowed/duplicate: the covering rule must stay unchanged until the deploy.
cover_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
cover_fp: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
# Flattened getRule, raw answer and firmware; written BEFORE a rule is deleted.
snapshot: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
# Where the rule sat (neighbours in the same prio_group by sort_order), for the restore.
anchor_next_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
anchor_prev_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
prio_group: Mapped[Optional[str]] = mapped_column(String(32), nullable=True)
# delete -> the confirmed disable item it builds on; restore -> the delete item it undoes. Plain id, no foreign key.
source_item_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
restored_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
changed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
+59
View File
@@ -0,0 +1,59 @@
"""Change history of the rules that are live on a firewall (all rules, also hand-made ones).
``live_rule_states`` is the last observed state per rule (a fingerprint plus the row itself), maintained by the
``rule_observe`` loop; ``rule_change_events`` is what changed between two observations. See
services/rule_history_service.py. Neither table holds anything the firewall itself would not show on its rule page.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import JSON, Boolean, DateTime, ForeignKey, Index, Integer, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class LiveRuleState(Base):
__tablename__ = "live_rule_states"
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True)
# The rule's UUID/label on the firewall; BASELINE_KEY marks "this host has been observed once".
rule_uuid: Mapped[str] = mapped_column(String(64), primary_key=True)
fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
row: Mapped[dict] = mapped_column(JSON, nullable=False)
# False = the rule is gone (kept as a marker so a returning UUID continues the numbering).
present: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Counts the observed changes of this rule; the event of a change gets the new number, which makes the
# observation idempotent when two processes see the same change.
version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
changed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class RuleChangeEvent(Base):
__tablename__ = "rule_change_events"
__table_args__ = (
UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
Index("ix_rule_change_events_rule", "host_id", "rule_uuid", "observed_at"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
seq: Mapped[int] = mapped_column(Integer, nullable=False)
observed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
kind: Mapped[str] = mapped_column(String(16), nullable=False) # added | changed | removed | enabled | disabled
before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
after: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
origin: Mapped[str] = mapped_column(String(16), nullable=False) # opnexus | external
actor: Mapped[Optional[str]] = mapped_column(String(150), nullable=True)
# The audit entry that explains an OPNexus-made change; no foreign key, the audit trail outlives everything.
audit_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
# Start of the time since the previous observation of the host: only audit entries from then on can explain
# this change (a later reconcile must not reach back to an older deployment).
window_start: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
+2
View File
@@ -205,6 +205,8 @@ async def remove_previous_target(alias_id: uuid.UUID, host_id: uuid.UUID, approv
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in alias.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_alias_item(deployment.host, deployment.opnsense_alias_uuid)
+17 -4
View File
@@ -1,14 +1,15 @@
from datetime import datetime
from fastapi import APIRouter, Depends, Query
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import select, func, or_
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.audit import AuditEvent
from app.services.audit_service import ADMIN_ONLY_DETAILS
router = APIRouter(prefix="/audit", tags=["audit"])
@router.get("")
async def history(kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
async def history(request: Request, kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
status: str | None = None, action: str | None = None, actor: str | None = None,
before: datetime | None = None, since: datetime | None = None,
q: str = Query(default="", max_length=200), offset: int = Query(default=0, ge=0),
@@ -24,5 +25,17 @@ async def history(kind: str | None = None, host_id: str | None = None, entity_id
clauses.append(or_(*(c.ilike('%'+q+'%') for c in [AuditEvent.entity_name, AuditEvent.host_name, AuditEvent.actor, AuditEvent.message])))
total = await db.scalar(select(func.count()).select_from(AuditEvent).where(*clauses))
rows = (await db.scalars(select(AuditEvent).where(*clauses).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).offset(offset).limit(limit))).all()
return {"total": total, "offset": offset, "limit": limit,
"items": [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]}
items = [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]
if not full_view(request):
for item in items:
if isinstance(item.get("details"), dict) and ADMIN_ONLY_DETAILS in item["details"]:
item["details"] = {**{k: v for k, v in item["details"].items() if k != ADMIN_ONLY_DETAILS},
ADMIN_ONLY_DETAILS: "redacted"}
return {"total": total, "offset": offset, "limit": limit, "items": items}
def full_view(request: Request | None) -> bool:
"""An admin account, and with a token only one of scope admin."""
user = getattr(getattr(request, "state", None), "user", None)
token = getattr(getattr(request, "state", None), "api_token", None)
return getattr(user, "role", None) == "admin" and (token is None or token.scope == "admin")
+105 -6
View File
@@ -1,6 +1,15 @@
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it)."""
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it).
The rule-cleanup settings (1.31) live here too: how many rules one cleanup may touch, how long a rule must stay
switched off before it may be deleted, the management sources (addresses the firewalls are managed from; no rule
that could match them is ever cleaned up) and extra management ports (besides the API port, 443, 80 and 22). Only an admin writes any of it (middleware: ``/api/settings/...`` is
admin-only for operators and tokens below scope admin; checked again here) and every change is audited with the
old and the new values."""
import ipaddress
from typing import Any
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
@@ -8,12 +17,83 @@ from app.errors import ErrorCode, api_error
from app.models.governance import GovernanceSettings
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.services import audit_service
router = APIRouter(prefix="/settings/governance", tags=["governance"])
MAX_CLEANUP_RULES = 25 # same ceiling as services/rule_cleanup_service.MAX_RULES_CEILING
MAX_DELETE_MIN_DAYS = 365
MAX_MANAGEMENT_SOURCES = 20
MAX_MANAGEMENT_PORTS = 20
class GovernanceRead(BaseModel):
operators_may_deploy: bool
cleanup_max_rules: int = 10
cleanup_delete_min_days: int = 7
# The list itself is shown to admins only; everybody else sees just whether it is set.
cleanup_management_sources: list[str] = Field(default_factory=list)
cleanup_management_sources_configured: bool = False
# Extra management ports (besides API port, 443, 80, 22): also admins only, others see the flag.
cleanup_management_ports: list[int] = Field(default_factory=list)
cleanup_management_ports_configured: bool = False
class GovernanceUpdate(BaseModel):
operators_may_deploy: bool | None = None
cleanup_max_rules: int | None = None
cleanup_delete_min_days: int | None = None
cleanup_management_sources: list[str] | None = None
cleanup_management_ports: list[Any] | None = None # validated in clean_management_ports (no bool coercion)
def _bad(field: str, reason: str):
return api_error(422, ErrorCode.BEREINIGUNG_EINSTELLUNG_UNGUELTIG, field=field, reason=reason)
def clean_management_sources(values: list[str]) -> list[str]:
"""IP addresses and CIDR networks only (no host names: they could resolve differently later), canonical form,
without duplicates; not the whole address space, which would protect every rule."""
if len(values) > MAX_MANAGEMENT_SOURCES:
raise _bad("cleanup_management_sources", "too_many")
cleaned: list[str] = []
for raw in values:
text = str(raw).strip()
try:
net = ipaddress.ip_network(text, strict=False)
except ValueError:
raise _bad("cleanup_management_sources", "not_an_address") from None
if net.prefixlen == 0:
raise _bad("cleanup_management_sources", "everything")
entry = str(net.network_address) if net.prefixlen == net.max_prefixlen else str(net)
if entry not in cleaned:
cleaned.append(entry)
return cleaned
def clean_management_ports(values) -> list[int]:
"""TCP ports 1..65535, at most MAX_MANAGEMENT_PORTS, without duplicates, sorted."""
if len(values) > MAX_MANAGEMENT_PORTS:
raise _bad("cleanup_management_ports", "too_many")
ports: set[int] = set()
for raw in values:
text = str(raw).strip()
if isinstance(raw, bool) or not text.isdigit() or not 1 <= int(text) <= 65535:
raise _bad("cleanup_management_ports", "not_a_port")
ports.add(int(text))
return sorted(ports)
def _read(row: GovernanceSettings, user: AdminUser) -> GovernanceRead:
sources = list(row.cleanup_management_sources or [])
ports = list(row.cleanup_management_ports or [])
return GovernanceRead(
operators_may_deploy=row.operators_may_deploy, cleanup_max_rules=row.cleanup_max_rules or 10,
cleanup_delete_min_days=row.cleanup_delete_min_days or 7,
cleanup_management_sources=sources if user.role == "admin" else [],
cleanup_management_sources_configured=bool(sources),
cleanup_management_ports=ports if user.role == "admin" else [],
cleanup_management_ports_configured=bool(ports))
async def get_settings(db: AsyncSession) -> GovernanceSettings:
@@ -28,14 +108,33 @@ async def get_settings(db: AsyncSession) -> GovernanceSettings:
@router.get("", response_model=GovernanceRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Readable by everybody signed in (the UI explains why a deploy was refused)."""
return await get_settings(db)
return _read(await get_settings(db), user)
@router.put("", response_model=GovernanceRead)
async def update(payload: GovernanceRead, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
async def update(payload: GovernanceUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
row = await get_settings(db)
row.operators_may_deploy = payload.operators_may_deploy
changes: dict[str, dict] = {}
if payload.cleanup_max_rules is not None:
if not 1 <= payload.cleanup_max_rules <= MAX_CLEANUP_RULES:
raise _bad("cleanup_max_rules", f"1..{MAX_CLEANUP_RULES}")
if payload.cleanup_delete_min_days is not None:
if not 1 <= payload.cleanup_delete_min_days <= MAX_DELETE_MIN_DAYS:
raise _bad("cleanup_delete_min_days", f"1..{MAX_DELETE_MIN_DAYS}")
sources = clean_management_sources(payload.cleanup_management_sources) if payload.cleanup_management_sources is not None else None
ports = clean_management_ports(payload.cleanup_management_ports) if payload.cleanup_management_ports is not None else None
for field, value in (("cleanup_max_rules", payload.cleanup_max_rules), ("cleanup_delete_min_days", payload.cleanup_delete_min_days),
("cleanup_management_sources", sources), ("cleanup_management_ports", ports)):
if value is not None and value != getattr(row, field):
changes[field] = {"old": getattr(row, field), "new": value}
setattr(row, field, value)
if payload.operators_may_deploy is not None:
row.operators_may_deploy = payload.operators_may_deploy
if changes:
db.add(audit_service.make_event(
kind="settings", action="cleanup_settings", status="accepted", entity_id="governance",
message="Einstellungen der Regel-Bereinigung geändert", details={"changes": changes}))
await db.commit()
return row
return _read(row, user)
+8
View File
@@ -147,6 +147,11 @@ async def get_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _get_host_or_404(db, host_id)
if (payload.base_url is not None or payload.enabled is not None or payload.clear_cluster_name
or payload.cluster_name is not None):
# An open rule cleanup (1.31) reverts through this address and this cluster pairing: not while it is open.
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
if payload.name is not None:
existing = await db.scalar(
@@ -188,6 +193,9 @@ async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
# An open rule cleanup (1.31) holds a revert deadline and snapshots for this host (and its cluster peers).
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
# Both deployment tables FK to opnsense_hosts with ondelete=CASCADE, so
# deleting the host would silently drop OPNexus's only record of what
# it pushed there — the rule/alias itself stays live on the actual
+2
View File
@@ -193,6 +193,8 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_child_uuid:
+17
View File
@@ -113,6 +113,23 @@ TOOLS = [
# The per-rule trace is for the UI; the verdict, deciding rule and reasons are what matters here.
"drop": ("trace",),
},
{
"name": "rule_cleanup_candidates",
"description": "Cleanup candidates of one firewall: rules that are unused (no hit for N days), shadowed or "
"duplicate (an earlier quick rule already decides everything they match), or disabled for a "
"long time. Each with its confidence (sure/probable/unknown), the last observed hit and "
"whether it could be removed (not for automatic, legacy or OPNexus-managed rules). Reads only.",
"inputSchema": {
"type": "object", "required": ["host_id"],
"properties": {
"host_id": {"type": "string", "description": "Firewall id from list_hosts"},
"unused_days": {"type": "integer", "minimum": 1, "maximum": 3650, "default": 30,
"description": "observation window for unused / disabled rules"},
},
},
"path": "/api/hosts/{host_id}/rule-analysis/cleanup",
"query": ("unused_days",),
},
{
"name": "api_get",
"description": "Read any other OPNexus API endpoint (GET only), e.g. /api/rules, /api/aliases, /api/logs. "
+2
View File
@@ -231,6 +231,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_one_to_one_rule(deployment.host, deployment.opnsense_one_to_one_uuid)
+2
View File
@@ -195,6 +195,8 @@ async def remove_previous_target(instance_id: uuid.UUID, host_id: uuid.UUID, app
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in instance.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_instance_uuid:
+2
View File
@@ -238,6 +238,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_outbound_nat_rule(deployment.host, deployment.opnsense_outbound_nat_uuid)
+2
View File
@@ -330,6 +330,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_policy_hierarchy_uuid)
await opnsense_client.apply(deployment.host)
+2
View File
@@ -233,6 +233,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_port_forward_rule(deployment.host, deployment.opnsense_port_forward_uuid)
+37
View File
@@ -17,6 +17,8 @@ from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup
from app.models.rule_history import LiveRuleState
from app.services import rule_history_service
from app.services.rule_analysis import service
logger = logging.getLogger(__name__)
@@ -89,3 +91,38 @@ async def fleet_match(
return await _run(host, q, refresh)
return list(await asyncio.gather(*(one(h) for h in hosts)))
@router.get("/hosts/{host_id}/rule-analysis/cleanup")
async def host_cleanup(
host_id: uuid.UUID, unused_days: int = Query(30, ge=1, le=3650, description="observation window for 'unused' / 'disabled for a long time'"),
refresh: bool = False, db: AsyncSession = Depends(get_db),
):
"""Cleanup candidates (unused, shadowed, duplicate, disabled for a long time) with confidence and whether
each rule could be offered for removal. Read-only; removal is not part of this endpoint."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return await service.cleanup_candidates(db, host, unused_days=unused_days, refresh=refresh)
except HTTPException:
raise
except Exception as exc: # noqa: BLE001 - an unreachable firewall is a 502, not a crash
logger.info("cleanup candidates failed for %s: %s", host.name, exc)
raise api_error(502, ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR) from exc
@router.get("/hosts/{host_id}/live-rules/{rule_uuid}/history")
async def live_rule_history(host_id: uuid.UUID, rule_uuid: str, limit: int = Query(100, ge=1, le=500),
db: AsyncSession = Depends(get_db)):
"""What happened to one rule on one firewall: observed changes (also hand-made ones), saved revisions of
rules OPNexus manages and the audit entries, newest first. Reads the database only."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
baseline = await db.get(LiveRuleState, (host_id, rule_history_service.BASELINE_KEY))
return {
"host_id": str(host.id), "rule_uuid": rule_uuid.lower()[:64],
"watching_since": baseline.first_seen_at if baseline else None,
"events": await rule_history_service.rule_timeline(db, host, rule_uuid, limit),
}
+194
View File
@@ -0,0 +1,194 @@
"""Rule cleanup (1.31, B4 part 2): switch hand-made rules off, delete them later, restore them (DESIGN-1.31 §4).
POST /hosts/{id}/rule-cleanup/preview operator, admin prepare (writes nothing to a firewall)
POST /hosts/{id}/rule-cleanup/{op}/cancel operator, admin the preview's own actor
POST /hosts/{id}/rule-cleanup/{op}/deploy ADMIN, token scope admin
POST /hosts/{id}/rule-cleanup/{op}/confirm operator, admin BROWSER SESSION ONLY, never a token
POST /hosts/{id}/rule-cleanup/{op}/revert admin; operator only with operators_may_deploy
POST /hosts/{id}/rule-cleanup/{op}/retry ADMIN, token scope admin
POST /hosts/{id}/rule-cleanup/items/{item}/restore ADMIN, token scope admin
GET /hosts/{id}/rule-cleanup, /{op} everybody signed in (read tokens too)
Who may do what is enforced twice: AuthMiddleware (services/api_token_service.py: _ADMIN_ONLY_CLEANUP,
_SESSION_ONLY, _ROLLOUT_POST) and here, so a gap in one layer does not open the route. The audit trail is the
service's (one ``rule_cleanup`` entry per step and rule) plus the middleware's request entry (kind ``host``,
action = last path segment; ``/preview`` is not logged by it, the service writes its own entry).
The services raise ``api_error``; nothing is translated here. MCP stays read-only (no tool, ``api_get`` is GET).
"""
import uuid
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.user import AdminUser
from app.routers import governance
from app.routers.auth import resolve_user
from app.schemas.rule_cleanup import CleanupOperationRead, CleanupPreviewRead, ConfirmName, PreviewRequest
from app.services import rule_cleanup_service as service
from app.services.rule_cleanup_service import ACK_TEXT_VERSION
router = APIRouter(prefix="/hosts/{host_id}/rule-cleanup", tags=["rule-cleanup"])
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
def _token(request: Request):
return getattr(request.state, "api_token", None)
def _writer(user: AdminUser, request: Request) -> None:
"""Any write: not for a read-only account, not with a read-scope token."""
if user.role == "read_only":
raise api_error(403, ErrorCode.READ_ONLY_ACCESS)
token = _token(request)
if token is not None and token.scope == "read":
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
def _admin_only(user: AdminUser, request: Request) -> None:
"""Deploy, retry, restore: an admin account, and with a token only one of scope admin."""
_writer(user, request)
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
token = _token(request)
if token is not None and token.scope != "admin":
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
async def _deployer(user: AdminUser, request: Request, db: AsyncSession) -> None:
"""Revert: an admin, or an operator while the governance switch operators_may_deploy is on."""
_writer(user, request)
if user.role == "admin":
return
if user.role == "operator" and (await governance.get_settings(db)).operators_may_deploy:
return
raise api_error(403, ErrorCode.DEPLOY_NEEDS_ADMIN)
def _session_only(user: AdminUser, request: Request) -> None:
"""Confirm: "I still have access" is a human decision in a browser session, never a token's."""
_writer(user, request)
token = _token(request)
if token is not None:
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
def _full_view(request: Request) -> bool:
"""Admins (session or admin-scope token) see the management sources/ports and firewall addresses in a plan."""
from app.routers.audit import full_view
return full_view(request)
async def _names(db: AsyncSession, operations: list[dict], request: Request) -> list[dict]:
"""The firewall names the UI shows next to the ids; for non-admins without the security-relevant plan parts."""
if not _full_view(request):
operations = [service.redact(op) for op in operations]
ids = {uuid.UUID(op["host_id"]) for op in operations} | {uuid.UUID(i["host_id"]) for op in operations for i in op["items"]}
names = {h.id: h.name for h in (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(ids)))).all()} if ids else {}
for op in operations:
op["host_name"] = names.get(uuid.UUID(op["host_id"]))
for item in op["items"]:
item["host_name"] = names.get(uuid.UUID(item["host_id"]))
return operations
async def _one(db: AsyncSession, operation: dict, request: Request) -> dict:
return (await _names(db, [operation], request))[0]
@router.get("", response_model=list[CleanupOperationRead])
async def list_operations(host_id: uuid.UUID, request: Request, limit: int = Query(50, ge=1, le=100),
db: AsyncSession = Depends(get_db)):
"""The newest operations of this host (and its cluster peers) with their items; unexpired previews included."""
host = await _host(db, host_id)
return await _names(db, await service.list_operations(db, host, limit), request)
@router.post("/preview", response_model=CleanupPreviewRead)
async def preview(host_id: uuid.UUID, payload: PreviewRequest, request: Request, db: AsyncSession = Depends(get_db)):
"""Recompute the plan from the live firewall(s) and store a 5-minute preview bound to the caller."""
user = await resolve_user(request, db)
_writer(user, request)
host = await _host(db, host_id)
kind, uuids, acknowledge = payload.resolved()
if kind is None:
raise api_error(422, ErrorCode.BEREINIGUNG_AUSWAHL_UNGUELTIG, field="kind")
result = await service.preview(
db, host, actor=user.username, actor_id=user.id, kind=kind, uuids=uuids, acknowledge=acknowledge,
**({"unused_days": payload.unused_days} if payload.unused_days else {}),
ack_text_version=payload.ack_text_version or ACK_TEXT_VERSION)
result["operation"] = await _one(db, await service.get_operation(db, host, uuid.UUID(result["preview_id"])), request)
return result if _full_view(request) else service.redact(result)
@router.get("/{op_id}", response_model=CleanupOperationRead)
async def get_operation(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
return await _one(db, await service.get_operation(db, host, op_id), request)
@router.post("/{op_id}/cancel", response_model=CleanupOperationRead)
async def cancel(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Drop an unused preview (only its own actor can)."""
user = await resolve_user(request, db)
_writer(user, request)
host = await _host(db, host_id)
return await _one(db, await service.cancel(db, host, op_id, actor=user.username), request)
@router.post("/{op_id}/deploy", response_model=CleanupOperationRead)
async def deploy(host_id: uuid.UUID, op_id: uuid.UUID, payload: ConfirmName, request: Request,
db: AsyncSession = Depends(get_db)):
"""Execute a preview. Admin only; the live state must still give the fingerprint of the preview."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.deploy(db, host, op_id, actor=user.username, confirm_name=payload.confirm_name), request)
@router.post("/{op_id}/confirm", response_model=CleanupOperationRead)
async def confirm(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
""""I still have access": re-reads every node first. Browser session only, never with an API token."""
user = await resolve_user(request, db)
_session_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.confirm(db, host, op_id), request)
@router.post("/{op_id}/revert", response_model=CleanupOperationRead)
async def revert(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Take the change back now (the same code the sweeper runs when the window passes)."""
user = await resolve_user(request, db)
await _deployer(user, request, db)
host = await _host(db, host_id)
return await _one(db, await service.revert_now(db, host, op_id), request)
@router.post("/{op_id}/retry", response_model=CleanupOperationRead)
async def retry(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Revert again after revert_failed, or read the firewall again for a delete/restore with an unclear item."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.retry(db, host, op_id), request)
@router.post("/items/{item_id}/restore", response_model=CleanupOperationRead)
async def restore(host_id: uuid.UUID, item_id: uuid.UUID, payload: ConfirmName, request: Request,
db: AsyncSession = Depends(get_db)):
"""Bring a deleted rule back, switched OFF, at its old position (a new UUID). The result is the restore run;
``warnings`` can hold BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.restore_item(
db, host, item_id, actor=user.username, actor_id=user.id, confirm_name=payload.confirm_name), request)
+6
View File
@@ -108,6 +108,10 @@ async def adopt_rule(payload: AdoptRequest, db: AsyncSession = Depends(get_db)):
host = await db.get(OpnsenseHost, payload.host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
# Taking over a hand-made rule that a rule cleanup (1.31) is switching off/deleting would make it "managed"
# under the cleanup's feet: not while one is open on this host or its cluster.
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
try:
rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid, link_to=payload.link_to)
except rule_adopt_service.RuleNotFoundOnHost as exc:
@@ -276,6 +280,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_rule_uuid)
await opnsense_client.apply(deployment.host)
+2
View File
@@ -207,6 +207,8 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_peer_uuid:
+117
View File
@@ -0,0 +1,117 @@
"""Request and response shapes of the rule-cleanup API (routers/rule_cleanup.py, DESIGN-1.31 §4).
Responses mirror services/rule_cleanup_service.read_operation. Texts are not part of the API: ``message`` holds an
ErrorCode name (``errors.<CODE>`` in the UI, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR) or the firewall's
own wording; ``warnings`` is a list of ErrorCode names.
"""
from __future__ import annotations
from typing import Any, Literal
from pydantic import BaseModel, Field, model_validator
CleanupKind = Literal["disable", "enable", "delete"]
# Longest list the API accepts at all; the effective limit is the setting cleanup_max_rules (ceiling 25).
MAX_SELECTION = 100
class PreviewItem(BaseModel):
"""One rule of the selection. ``kind`` (alias ``mode``) must be the same for all items of one preview."""
uuid: str = Field(min_length=1, max_length=64)
kind: CleanupKind | None = None
mode: CleanupKind | None = None
# The per-rule acknowledgement for a "probable" rule (the UI's checkbox).
ack: bool = False
class PreviewRequest(BaseModel):
"""``{kind, uuids, acknowledge}`` or ``{items:[{uuid, kind|mode, ack}]}`` (or both: they are merged)."""
kind: CleanupKind | None = None
uuids: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
items: list[PreviewItem] = Field(default_factory=list, max_length=MAX_SELECTION)
acknowledge: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
unused_days: int | None = Field(default=None, ge=1, le=3650)
ack_text_version: str | None = Field(default=None, max_length=16)
@model_validator(mode="after")
def _single_kind(self):
kinds = {k for k in [self.kind, *[i.kind or i.mode for i in self.items]] if k}
if len(kinds) > 1:
raise ValueError("kind must be the same for every item")
return self
def resolved(self) -> tuple[str | None, list[str], list[str]]:
kind = self.kind or next((i.kind or i.mode for i in self.items if i.kind or i.mode), None)
uuids = [*self.uuids, *[i.uuid for i in self.items]]
acknowledge = [*self.acknowledge, *[i.uuid for i in self.items if i.ack]]
return kind, list(dict.fromkeys(u.strip() for u in uuids)), list(dict.fromkeys(a.strip() for a in acknowledge))
class ConfirmName(BaseModel):
"""Deploy and restore: the exact host name, typed by a human."""
confirm_name: str = Field(max_length=255)
class CleanupItemRead(BaseModel):
id: str
host_id: str
host_name: str | None = None
rule_uuid: str
seq: int
carp_role: str | None = None
# queued, disabling, disabled, confirmed, reverting, reverted, deleting, deleted, restoring, restored, failed,
# unknown, revert_failed, skipped (never attempted: the run stopped earlier)
state: str
description: str | None = None
categories: Any = None
confidence: str | None = None
cover_uuid: str | None = None
prio_group: str | None = None
anchor_next_uuid: str | None = None
anchor_prev_uuid: str | None = None
source_item_id: str | None = None
restored_uuid: str | None = None
changed_at: str | None = None
message: str | None = None
has_snapshot: bool = False
# A deleted rule with a stored snapshot that has not been restored yet (POST items/{id}/restore, admin).
restorable: bool = False
class CleanupOperationRead(BaseModel):
id: str
host_id: str
host_name: str | None = None
cluster_name: str | None = None
kind: str # disable | enable | delete | restore
# preview, running, awaiting_confirm, confirmed, partial, apply_failed, reverting, reverted, revert_failed, done,
# failed, cancelled, expired
status: str
actor: str
created_at: str | None = None
expires_at: str | None = None
started_at: str | None = None
heartbeat_at: str | None = None
revert_deadline: str | None = None
confirmed_at: str | None = None
finished_at: str | None = None
message: str | None = None
options: dict[str, Any] = Field(default_factory=dict)
# Only while status == "preview".
plan: dict[str, Any] | None = None
items: list[CleanupItemRead] = Field(default_factory=list)
can_confirm: bool = False
can_revert: bool = False
can_retry: bool = False
# ErrorCode names, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR (restore only).
warnings: list[str] = Field(default_factory=list)
class CleanupPreviewRead(BaseModel):
preview_id: str
expires_at: str | None = None
kind: str
plan: dict[str, Any]
max_rules: int
window_seconds: int
operation: CleanupOperationRead
@@ -130,6 +130,9 @@ async def sweep_expired_alias_deployments(db: AsyncSession) -> list[AliasDeploym
async def undeploy_alias(db: AsyncSession, alias: FirewallAlias) -> None:
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, AliasDeployment, AliasDeployment.alias_id == alias.id)
failures = []
result = await db.execute(select(AliasDeployment).where(AliasDeployment.alias_id == alias.id))
for deployment in result.scalars().all():
+16 -3
View File
@@ -39,7 +39,9 @@ _LAST_USED_GRANULARITY = timedelta(minutes=1)
_SAFE_METHODS = ("GET", "HEAD", "OPTIONS")
# GUI tickets open a firewall's web interface in the browser: an interactive admin session only.
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$")
# Rule cleanup (1.31): "I still have access" is a human decision, so confirming a cleanup is never possible with a
# token, whatever its scope (a script must not be able to confirm blindly).
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$|^/api/hosts/[^/]+/rule-cleanup/[^/]+/confirm/?$")
# Reads that expose secrets (config.xml holds password hashes and VPN keys, the
# notification settings the webhook URL/SMTP data): admin scope only.
_ADMIN_SCOPE_READS = re.compile(r"^/api/hosts/[^/]+/backups/[^/]+$|^/api/notifications(/|$)|^/api/settings/|^/api/overlay/peers/[^/]+/instructions$")
@@ -55,6 +57,13 @@ _ADMIN_ONLY_WRITES = re.compile(
)
# Rule cleanup: executing a cleanup, retrying a failed revert/reconcile and restoring a deleted rule are admin-only.
# operators_may_deploy defaults to True, so _ROLLOUT_POST alone cannot express "admin only"; this pattern is checked
# for operator accounts (operator_denies) and for tokens below scope admin (denies). Not part of _ADMIN_ONLY_WRITES:
# that one means "who may access OPNexus / where data goes". The router checks the same rule once more.
_ADMIN_ONLY_CLEANUP = re.compile(r"^/api/hosts/[^/]+/rule-cleanup/([^/]+/(deploy|retry)|items/[^/]+/restore)/?$")
def _now() -> datetime:
return datetime.now(timezone.utc)
@@ -73,7 +82,7 @@ def denies(scope: str, method: str, path: str) -> bool:
return scope != "admin" and bool(_ADMIN_SCOPE_READS.match(path))
if scope == "read":
return True
if scope == "deploy" and _ADMIN_ONLY_WRITES.match(path):
if scope == "deploy" and (_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path)):
return True
return False
@@ -82,7 +91,8 @@ def operator_denies(method: str, path: str) -> bool:
"""True if an `operator` account must not make this request: the writes that
change who may access OPNexus or where it sends data. (Their own theme at
PUT /api/settings stays open; account endpoints are not matched here.)"""
return method not in _SAFE_METHODS and path != "/api/settings" and bool(_ADMIN_ONLY_WRITES.match(path))
return method not in _SAFE_METHODS and path != "/api/settings" and bool(
_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path))
# Writes that put a change onto the fleet (as opposed to preparing one): deploy, remove/rollback a deployment,
@@ -92,6 +102,9 @@ def operator_denies(method: str, path: str) -> bool:
_ROLLOUT_POST = re.compile(
r"/(deploy|push|sign|poll|register|reassert-ordering"
r"|deployments/[^/]+/(remove|retry-rollback)"
# Rule cleanup (1.31): switching rules off (/deploy matches above) and taking it back. Confirming and
# previewing stay open, like confirm_rule.
r"|rule-cleanup/[^/]+/(revert|retry)"
r"|maintenance/(execute|carp-execute|schedule)"
r"|network/changes"
r"|services/[^/]+/(start|stop|restart)"
+3
View File
@@ -13,6 +13,9 @@ from app.database import SessionLocal
from app.models.audit import AuditEvent
context = ContextVar("audit_context", default=None)
# A key in ``details`` whose content only admins (session or admin-scope token) get to read (routers/audit.py):
# firewall addresses and management sources of the rule cleanup.
ADMIN_ONLY_DETAILS = "admin_only"
# Column widths of audit_events. entity_id/host_id/action are taken from the
+30 -31
View File
@@ -26,6 +26,7 @@ from app.models.host import OpnsenseHost
from app.services import backup_store, rule_hit_service
from app.services import opnsense_client as client
from app.services.opnsense_client import OpnsenseAPIError
from app.services.rule_analysis import coverage
MAX_CONFIG_BYTES = 32 * 1024 * 1024
BACKUP_MAX_AGE_DAYS = 7
@@ -83,10 +84,15 @@ def _mvc_rule(rule: Element) -> dict:
"sport": _text(rule, "source_port"),
"dst": _text(rule, "destination_net") or "any", "dst_not": _text(rule, "destination_not") == "1",
"dport": _text(rule, "destination_port"),
"floating": False,
"if_not": _text(rule, "interfacenot") == "1",
"floating": False, # decided in extract_rules (needs the list of interfaces)
"quick": _text(rule, "quick") != "0",
"descr": _text(rule, "description") or "",
"order": int(_text(rule, "sequence") or 0),
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
"received_on": _text(rule, "received-on") or "",
"icmptype": ",".join(filter(None, [_text(rule, "icmptype"), _text(rule, "icmp6type")])),
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
}
@@ -101,17 +107,29 @@ def _legacy_rule(rule: Element, index: int) -> dict:
"ipprotocol": _text(rule, "ipprotocol") or "inet",
"protocol": (_text(rule, "protocol") or "any").lower(),
"src": src, "src_not": src_not, "sport": sport, "dst": dst, "dst_not": dst_not, "dport": dport,
"if_not": rule.find("interfacenot") is not None,
"floating": _text(rule, "floating") == "yes",
"quick": _text(rule, "quick") != "0",
# Legacy rules: an interface rule is always evaluated "quick"; only a floating rule has the option,
# and it is off unless <quick> says yes.
"quick": _text(rule, "floating") != "yes" or (_text(rule, "quick") or "").lower() in ("yes", "1"),
"descr": _text(rule, "descr") or "",
"order": index,
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
"received_on": _text(rule, "received-on") or "", "icmptype": _text(rule, "icmptype") or "",
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
}
def extract_rules(root: Element) -> list[list[dict]]:
"""Rule lists in evaluation order, one per source (MVC, legacy)."""
mvc = sorted((_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")), key=lambda r: r["order"])
mvc = [_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")]
known = {node.tag for node in root.findall("interfaces/*")}
for rule in mvc: # same rule as OPNsense: not exactly one interface (or "not") = floating
rule["floating"] = len(rule["interfaces"]) != 1 or rule["if_not"]
rule["group"] = not rule["floating"] and rule["interfaces"][0] not in known
mvc.sort(key=coverage.config_order)
legacy = [_legacy_rule(r, i) for i, r in enumerate(root.findall("filter/rule"))]
legacy.sort(key=coverage.config_order)
return [group for group in (mvc, legacy) if group]
@@ -133,41 +151,22 @@ def _is_any_any(rule: dict) -> bool:
def _covers(a: dict, b: dict) -> bool:
"""Every packet matched by rule b is already matched by the earlier rule a.
Only exact-or-any comparisons, so it never claims coverage it cannot prove."""
if not (a["enabled"] and b["enabled"]) or a["floating"] or b["floating"] or not a["quick"]:
return False
if not a["interfaces"] or not set(b["interfaces"]) <= set(a["interfaces"]):
return False
if a["direction"] != b["direction"] or a["ipprotocol"] != b["ipprotocol"]:
return False
if a["protocol"] != "any" and a["protocol"] != b["protocol"]:
return False
for side in ("src", "dst"):
if a[side + "_not"] or b[side + "_not"]:
if (a[side], a[side + "_not"]) != (b[side], b[side + "_not"]):
return False
elif a[side] != "any" and a[side] != b[side]:
return False
for port in ("sport", "dport"):
if a[port] and a[port] != b[port]:
return False
return True
def _label(rule: dict) -> str:
return rule["descr"] or f"{rule['action']} {rule['src']} → {rule['dst']}"
def analyse_rules(groups: list[list[dict]]) -> dict:
"""Shadowed and duplicate rules. The coverage logic lives in rule_analysis/coverage.py (shared with the
cleanup candidates); the rules of one list are compared in evaluation order."""
shadowed, duplicates = [], []
for rules in groups:
for j, later in enumerate(rules):
for earlier in rules[:j]:
if _covers(earlier, later):
(duplicates if earlier["action"] == later["action"] and _covers(later, earlier) else shadowed).append(_label(later))
break
ordered = sorted(rules, key=coverage.config_order)
norm = [coverage.from_config_rule(rule) for rule in ordered]
by_identity = {id(n): r for n, r in zip(norm, ordered)}
for overlap in coverage.analyse(norm):
if overlap.by.order == overlap.rule.order:
continue # equal position in config.xml: the real order is unknown, claim nothing
(duplicates if overlap.kind == "duplicate" else shadowed).append(_label(by_identity[id(overlap.rule)]))
return {"shadowed": shadowed, "duplicates": duplicates}
@@ -34,7 +34,27 @@ async def peers(db, host):
return [host]
async def ensure_no_maintenance(db, hosts):
async def ensure_no_open_cleanup(db, hosts):
"""A rule cleanup (1.31) that is running, waiting for its confirmation or unresolved owns the host and its
cluster peers: nothing else may write to the firewall until it is confirmed, reverted or settled."""
from app.services.rule_cleanup_service import has_open_cleanup
ids = set()
for host in hosts:
ids.update(h.id for h in await peers(db, host))
if ids and await has_open_cleanup(db, ids):
raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)
async def ensure_no_open_cleanup_on(db, model, condition):
"""ensure_no_open_cleanup for every host that has a deployment row of ``model`` matching ``condition`` (the
undeploy paths: deleting a central object removes it from all its firewalls)."""
ids = set((await db.scalars(select(model.host_id).where(condition))).all())
hosts = [h for h in [await db.get(OpnsenseHost, i) for i in ids] if h is not None]
await ensure_no_open_cleanup(db, hosts)
async def ensure_no_maintenance(db, hosts, *, cleanup=True):
"""cleanup=False: only the cleanup service itself, which reports an open cleanup with its own error code."""
ids = set()
for host in hosts:
ids.update(h.id for h in await peers(db, host))
@@ -42,6 +62,8 @@ async def ensure_no_maintenance(db, hosts):
raise api_error(409, ErrorCode.OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY)
if ids and await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
raise api_error(409, ErrorCode.OPEN_RESTORE_BLOCKS_DEPLOY)
if cleanup and ids:
await ensure_no_open_cleanup(db, hosts)
async def _shared_change_blockers(db, fleet):
@@ -56,6 +78,9 @@ async def _shared_change_blockers(db, fleet):
blocked.append("Eine frühere Wartungsaktion im Host/Cluster muss zuerst geprüft und abgeschlossen werden.")
if await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(fleet_ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
blocked.append("Eine Wiederherstellung im Host/Cluster ist offen und muss zuerst bestätigt oder abgebrochen werden.")
from app.services.rule_cleanup_service import has_open_cleanup
if await has_open_cleanup(db, fleet_ids):
blocked.append("Eine Regel-Bereinigung im Host/Cluster ist offen und muss zuerst bestätigt oder zurückgenommen werden.")
return blocked
@@ -303,6 +303,9 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel) -> None:
for this entity to gate on) — deletes Child/Local/Remote before their
parent Connection (the ModelRelationField dependency direction), then
the PreSharedKey (independent of the other 4), then reconfigures."""
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, IpsecTunnelDeployment, IpsecTunnelDeployment.tunnel_id == tunnel.id)
failures = []
result = await db.execute(select(IpsecTunnelDeployment).where(IpsecTunnelDeployment.tunnel_id == tunnel.id))
for deployment in result.scalars().all():
@@ -201,6 +201,62 @@ async def check_revert_failed(db: AsyncSession) -> dict[str, dict]:
"host_name": host.name if host else None,
"message": deployment.message,
}
problems.update(await _cleanup_problems(db))
return problems
# Rule cleanup (1.31) states that need a human: the automatic revert failed, or a run ended half-way.
_CLEANUP_STILL_CHANGED = ("revert_failed", "unknown", "disabling", "disabled", "failed")
async def _cleanup_problems(db: AsyncSession) -> dict[str, dict]:
"""Rule cleanups whose automatic revert failed (per rule that could not be switched back) and runs stuck in
``partial``/``apply_failed`` (a decision is needed before the window ends), plus delete/restore runs with an
unresolved item. Same ``revert_failed`` alert as the deployments. Keys: ``rule_cleanup:<item id>`` for one rule,
``rule_cleanup_op:<operation id>`` for a whole run, so one open problem is one message."""
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
problems: dict[str, dict] = {}
operations = (await db.scalars(select(RuleCleanupOperation).where(
RuleCleanupOperation.status.in_(("revert_failed", "partial", "apply_failed", "failed"))))).all()
for op in operations:
items = list((await db.scalars(select(RuleCleanupItem).where(RuleCleanupItem.operation_id == op.id)
.order_by(RuleCleanupItem.seq))).all())
host = await db.get(OpnsenseHost, op.host_id)
base = {
"deployment_type": "rule_cleanup", "deployment_id": str(op.id), "host_id": str(op.host_id),
"host_name": host.name if host else None, "operation": op.kind, "operation_status": op.status,
}
if op.status == "revert_failed":
stuck = [i for i in items if i.state in _CLEANUP_STILL_CHANGED]
for item in stuck:
member = await db.get(OpnsenseHost, item.host_id)
problems[f"rule_cleanup:{item.id}"] = {
**base, "entity_id": item.rule_uuid, "host_id": str(item.host_id),
"host_name": member.name if member else None, "rule": item.description, "item_state": item.state,
"message": item.message or op.message,
"hinweis": "Bereinigung: Die Regel konnte nicht automatisch zurückgesetzt werden und ist möglicherweise "
"noch geändert. Auf der Firewall prüfen und in OPNexus „Erneut zurücknehmen“ wählen.",
}
if not stuck:
problems[f"rule_cleanup_op:{op.id}"] = {**base, "entity_id": str(op.id), "message": op.message,
"hinweis": "Bereinigung: Das automatische Zurücknehmen ist fehlgeschlagen. Zustand der Firewall prüfen."}
elif op.status in ("partial", "apply_failed"):
problems[f"rule_cleanup_op:{op.id}"] = {
**base, "entity_id": str(op.id), "message": op.message,
"regeln": ", ".join(i.rule_uuid for i in items if i.state in ("failed", "unknown")) or None,
"hinweis": "Bereinigung nur teilweise ausgeführt (oder das Anwenden ist fehlgeschlagen). Bestätigen ist gesperrt; "
"jetzt zurücknehmen oder die automatische Rücknahme am Fristende abwarten.",
}
elif op.kind in ("delete", "restore"):
for item in (i for i in items if i.state == "unknown"):
member = await db.get(OpnsenseHost, item.host_id)
problems[f"rule_cleanup:{item.id}"] = {
**base, "entity_id": item.rule_uuid, "host_id": str(item.host_id),
"host_name": member.name if member else None, "rule": item.description, "item_state": item.state,
"message": item.message or op.message,
"hinweis": "Bereinigung: Der Ausgang beim Löschen/Wiederherstellen ist unklar. Mit „Erneut prüfen“ nachlesen.",
}
return problems
@@ -135,6 +135,9 @@ async def sweep_expired_one_to_one_deployments(db: AsyncSession) -> list[OneToOn
async def undeploy_one_to_one(db: AsyncSession, rule: OneToOneRule) -> None:
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, OneToOneDeployment, OneToOneDeployment.one_to_one_id == rule.id)
failures = []
result = await db.execute(select(OneToOneDeployment).where(OneToOneDeployment.one_to_one_id == rule.id))
for deployment in result.scalars().all():
@@ -371,6 +371,9 @@ async def undeploy_instance(db: AsyncSession, instance: OpenVpnInstance) -> None
for this entity to gate on) — deletes the Instance before its
referenced StaticKey (the ModelRelationField dependency direction),
then reconfigures."""
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, OpenVpnInstanceDeployment, OpenVpnInstanceDeployment.instance_id == instance.id)
failures = []
result = await db.execute(select(OpenVpnInstanceDeployment).where(OpenVpnInstanceDeployment.instance_id == instance.id))
for deployment in result.scalars().all():
+135 -4
View File
@@ -10,6 +10,7 @@ Auth is HTTP Basic: username = API key, password = API secret.
"""
import logging
import json
import re
from typing import Any
import httpx
@@ -33,6 +34,11 @@ class OpnsenseAmbiguousError(OpnsenseAPIError):
outcome of a write is unknown and must be reconciled, not retried blindly."""
class OpnsenseNotFoundError(OpnsenseAPIError):
"""The firewall answered, but there is no such object (an empty `getRule` answer: unknown UUID, or a rule that is
not part of the rules model - legacy or automatic). A refusal, never an unknown outcome."""
class OpnsenseForbiddenError(OpnsenseAPIError):
"""The firewall answered 403: the API key lacks the privilege for this endpoint. A refusal, not an unknown
outcome; callers that only read can tell "no rights" from "unreachable". 401 (wrong key or secret) stays a
@@ -241,6 +247,51 @@ def _mutation_result(data: Any, operation: str, results: set[str]) -> dict:
return data
# -- Filter rules (FilterController -> ApiMutableModelControllerBase, rules.rule) --------------------------------
# Evidence for add/set/get/del/toggle/moveRuleBefore, checked 2026-10-22 against BOTH
# source opnsense/core tags 26.7.7, 26.1.9 and 25.7.11, src/opnsense/mvc/app/controllers/OPNsense/:
# Firewall/Api/FilterController.php (the *RuleAction methods, moveRuleBeforeAction, ittrRules),
# Firewall/Api/FilterBaseController.php (applyAction),
# Base/ApiMutableModelControllerBase.php (addBase, setBase, getBase, delBase, toggleBase)
# live OPNsense 26.7.7 on two lab test firewalls (scratch rules whose description began with
# `opnexus-b4-probe`, deleted again and applied afterwards; nothing was left behind).
# Same in all three source lines unless said otherwise; the live checks cover 26.7.7 only.
#
# addRule POST, body {"rule": {...}}. Answer {"result": "saved", "uuid": "<new>"}; a validation failure is HTTP 200
# {"result": "failed", "validations": {"rule.<field>": "<text>"}} (so _mutation_result raises).
# `sequence` is a FilterSequenceField (models/OPNsense/Firewall/Filter.xml: 1..999999, default 1;
# `sort_order` and `prio_group` are volatile, derived). Live: without `sequence` the rule gets the
# highest sequence of ALL rules + 100 (566, 666, 766, ...), so a copy always lands at the END of its
# prio_group; an explicit `sequence` IS honoured (sequence "3" -> sort_order 400000.0000003) and is not
# checked for uniqueness. A restore can therefore either send the old number or call moveRuleBefore.
# setRule POST /{uuid}, body {"rule": {...}}; answer {"result": "saved"}. It is an upsert (a well-formed unknown
# UUID creates the rule) - never call it for a rule that may be gone.
# delRule POST /{uuid}. {"result": "deleted"} or {"result": "not found"} (unknown, legacy and automatic UUIDs all
# answer "not found", HTTP 200, nothing changes). 26.1 and 26.7 accept a comma-separated list; 25.7 a
# single UUID. Consequence: "not found" proves nothing about a rule being gone - re-read it.
# getRule GET /{uuid} -> {"rule": {...}} (every field of the rules model, select fields as
# {"<option>": {"value": "<label>", "selected": 0|1}}, `categories` a select dict - or [] when the rule has
# none -; no `uuid` key; read-only: `sequence`, `sort_order`, `prio_group` and the nested `audit`).
# Live: an unknown, legacy (hex label) or automatic UUID answers HTTP 200 with a bare `[]`. 25.7 and 26.1
# also know `?fetchmode=copy` (pre-fills `sequence`); 26.7.7 no longer has it - not used here.
# Without a UUID it returns the template of a new rule (see get_rule_template above).
# toggleRule POST /{uuid}/{enabled} - `enabled` is "0" or "1" and EXPLICIT; without it the rule flips (never do that).
# Answer {"result": "Enabled"|"Disabled", "changed": true|false}. The explicit form is idempotent: repeating
# it answers the same result with "changed": false (live: 0,0 -> true,false; 1,1 -> true,false). Any other
# value answers {"result": "failed", "changed": false}; an unknown or legacy UUID, a GET, or a model that
# has no such node answers {"result": "failed"} without "changed" - all HTTP 200. It only edits the
# configuration; nothing reaches pf before apply() (the rule keeps its sort_order, searchRule shows
# enabled "0"/"1" at once). 26.1/26.7 take a list of UUIDs (explicit value only); 25.7 one UUID.
# moveRuleBefore POST /{selected}/{target}. {"status": "ok"} (no "result" key). Both rules must be in the same
# `prio_group` - the sort_order prefix - which is NOT the interface: every single-interface rule of every
# interface shares 400000 (lan and wan rules can be moved before each other), floating rules (several or
# no interface) are 200000, interface-group rules 3000xx. Live: unknown UUID, same UUID twice and
# different groups all answer HTTP 500 {"errorMessage": "Unexpected error, check log for details",
# "errorTitle": "Filter"}, a GET answers 200 {"status": "error"}. A 500 is Ambiguous in _do_request, so a
# caller that wants to tell "refused" from "unknown" must read sort_order back. The move only touches
# `sequence` of the moved rule (and, when the gap is too small, of the rules behind the target); legacy
# rules are not in the model and cannot be an anchor.
# apply POST. {"status": "OK\n\n"} after `filter reload skip_alias`. FilterBaseController has no savepoint here.
async def add_rule(host: OpnsenseHost, payload: dict) -> dict:
return _mutation_result(await _request(host, "POST", "/firewall/filter/addRule", json={"rule": payload}), "Regel anlegen", {"saved"})
@@ -253,18 +304,93 @@ async def del_rule(host: OpnsenseHost, rule_uuid: str) -> dict:
return _mutation_result(await _request(host, "POST", f"/firewall/filter/delRule/{rule_uuid}"), "Regel entfernen", {"deleted", "not found"})
_RULE_REF = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
def _rule_ref(rule_uuid: str) -> str:
"""A rule reference ends up in the URL path: refuse anything that could leave it."""
if not isinstance(rule_uuid, str) or not _RULE_REF.match(rule_uuid):
raise OpnsenseAPIError("Ungültige Regel-UUID")
return rule_uuid
async def get_rule(host: OpnsenseHost, rule_uuid: str) -> dict:
"""The full model record of one rule (`getRule/{uuid}`, the inner `rule` dict, select fields still in
OPNsense's option form - reduce them with rule_write_payload()). A rule that is not in the rules model (unknown,
legacy hex label, automatic) is answered with `[]`: raises OpnsenseNotFoundError."""
data = await _request(host, "GET", f"/firewall/filter/getRule/{_rule_ref(rule_uuid)}")
rule = data.get("rule") if isinstance(data, dict) else None
if isinstance(rule, dict) and rule:
return rule
if data == [] or data == {}:
raise OpnsenseNotFoundError(f"Regel {rule_uuid} gibt es auf {host.name} nicht (oder sie ist keine bearbeitbare Regel)")
raise OpnsenseAPIError(f"Regel lesen: Keine auswertbare Antwort von {host.name}")
async def toggle_rule(host: OpnsenseHost, rule_uuid: str, enabled: bool) -> dict:
"""Sets the rule's `enabled` flag to exactly `enabled` (`toggleRule/{uuid}/1|0`; never the flipping form) and
returns OPNsense's answer, e.g. {"result": "Disabled", "changed": False}. Repeating it is harmless. Only the
configuration changes; call apply() afterwards. A refusal (unknown or legacy UUID) is a plain OpnsenseAPIError,
a lost answer an OpnsenseAmbiguousError - then read the rule back before doing anything else."""
if not isinstance(enabled, bool):
raise TypeError("toggle_rule: 'enabled' must be a bool (an implicit flip would not be idempotent)")
data = await _request(host, "POST", f"/firewall/filter/toggleRule/{_rule_ref(rule_uuid)}/{1 if enabled else 0}")
return _mutation_result(data, "Regel ein-/ausschalten", {"enabled" if enabled else "disabled"})
# Fields of a getRule record that a write must not send: the UUID is the rule's identity, `sequence` is assigned
# on creation (sort_order and prio_group are derived from it), `audit` is metadata OPNsense maintains itself.
RULE_READONLY_FIELDS = frozenset({"uuid", "sequence", "sort_order", "prio_group", "audit"})
def rule_write_payload(rule: dict) -> dict[str, str]:
"""Turns a getRule record into the flat form addRule/setRule accept: select fields become their selected
option(s), comma-separated (`lan,wan`; none selected -> ""), the empty-list form of an empty multi-select
becomes "", read-only fields are left out. Live round trip on 26.7.7 (getRule -> this -> addRule -> getRule)
gave an identical record except uuid/sequence/sort_order/audit for single-interface, multi-interface (floating),
no-interface (floating), interface-group, ICMP and any-protocol rules with categories, aliases, port ranges,
TCP flags, tags and state limits."""
out: dict[str, str] = {}
for key, value in rule.items():
if key in RULE_READONLY_FIELDS:
continue
if isinstance(value, dict):
if value and all(isinstance(o, dict) and "selected" in o for o in value.values()):
out[key] = ",".join(k for k, o in value.items() if o.get("selected") and k != "")
# any other nested dict is metadata and is not sent
elif isinstance(value, list):
out[key] = ",".join(str(v) for v in value)
else:
out[key] = "" if value is None else str(value)
return out
async def apply(host: OpnsenseHost) -> dict:
return _mutation_result(await _request(host, "POST", "/firewall/filter/apply"), "Änderungen anwenden", set())
async def get_hasync(host: OpnsenseHost) -> dict:
"""System > High Availability settings, read-only: `core/hasync/get` (Core/Api/HasyncController extends
ApiMutableModelControllerBase, model Core/Hasync.xml; identical in opnsense/core 25.7.11, 26.1.9 and 26.7.7).
Answer {"hasync": {"pfsyncinterface": {"": {"value": "Disabled", ...}, "lan": {"value": "LAN", "selected": 1},
...}, "pfsyncpeerip": "<address or empty>", "synchronizetoip": ..., "syncitems": {...}}} - `pfsyncinterface` is an
InterfaceField (the selected option, "" = disabled), `pfsyncpeerip` a NetworkField without mask. Live on 26.7.7
(both nodes of the lab cluster): pfsyncinterface `lan`, a peer address, synchronizetoip empty. Used by the rule
cleanup to keep pfsync rules out of reach (rule_analysis/protection.py); it reads nothing secret (the sync
password field comes back, but is never stored or logged by the caller)."""
return await _request(host, "GET", "/core/hasync/get")
async def move_rule_before(host: OpnsenseHost, selected_uuid: str, target_uuid: str) -> dict:
"""`firewall/filter/moveRuleBefore/{selected_uuid}/{target_uuid}`
(FilterController::moveRuleBeforeAction) — repositions `selected_uuid`
to sit directly before `target_uuid` via gap-numbered `sequence`
floats, scoped to rules sharing the same `prio_group` (interface/type).
Verified against OPNsense core source: no "move to top/bottom"
sentinel exists, and both UUIDs must resolve to existing rules on the
SAME interface/type or the controller throws. On success it returns
floats, scoped to rules sharing the same `prio_group` (the sort_order
prefix, see the evidence block above - not the interface). Verified against OPNsense core source and live
(26.7.7): no "move to top/bottom"
sentinel exists, and both UUIDs must resolve to existing rules in the
SAME prio_group or the controller throws (HTTP 500, which this client
reports as OpnsenseAmbiguousError). On success it returns
`{"status": "ok"}` (confirmed against FilterController.php directly —
unlike addRule/setRule/delRule this has no "result" key at all), so
this is dispatched through _mutation_result with an empty `results`
@@ -442,6 +568,11 @@ async def get_pf_rule_statistics(host: OpnsenseHost) -> dict:
# ifname, members (comma separated identifiers), sequence; the built-in groups (enc0, openvpn,
# wireguard) appear with their name as uuid and empty members.
async def get_rule_stats(host: OpnsenseHost) -> dict:
"""Per-label counters, already summed over every pf line of a rule. Used for the rule usage sampling
(rule_hit_service). Live check 2026-10-22 on all three test firewalls (26.7, one with unmigrated legacy
rules): HTTP 200, ``stats`` has exactly one key per label of ``pfStatistics/rules`` (30/30/25 labels,
including the 32-hex label of the legacy user rule on the third), and each ``packets`` equals the sum over
that label's pf lines (differences only by traffic between the two reads)."""
return await _request(host, "GET", "/firewall/filter_util/ruleStats")
@@ -147,6 +147,9 @@ async def sweep_expired_outbound_nat_deployments(db: AsyncSession) -> list[Outbo
async def undeploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule) -> None:
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, OutboundNatDeployment, OutboundNatDeployment.outbound_nat_id == rule.id)
failures = []
result = await db.execute(select(OutboundNatDeployment).where(OutboundNatDeployment.outbound_nat_id == rule.id))
for deployment in result.scalars().all():
@@ -185,6 +185,9 @@ async def reassert_group_ordering(db: AsyncSession, group: PolicyHierarchyGroup)
for h in g.hosts:
hosts[h.id] = h
errors: list[str] = []
# moveRuleBefore changes rule positions: not while a rule cleanup (1.31) owns one of the firewalls.
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [h for h in hosts.values() if h.enabled])
for host in sorted(hosts.values(), key=lambda h: (h.name, str(h.id))):
if not host.enabled:
continue
@@ -300,9 +303,13 @@ async def sweep_expired_hierarchy_deployments(db: AsyncSession) -> list[PolicyHi
async def undeploy_hierarchy_rule(db: AsyncSession, rule: PolicyHierarchyRule) -> None:
from app.services.host_maintenance_service import ensure_no_open_cleanup
failures = []
result = await db.execute(select(PolicyHierarchyRuleDeployment).where(PolicyHierarchyRuleDeployment.policy_hierarchy_id == rule.id))
for deployment in result.scalars().all():
deployments = list(result.scalars().all())
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them.
await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])
for deployment in deployments:
host = await db.get(OpnsenseHost, deployment.host_id)
if host and deployment.opnsense_policy_hierarchy_uuid:
try:
@@ -147,6 +147,9 @@ async def sweep_expired_port_forward_deployments(db: AsyncSession) -> list[PortF
async def undeploy_port_forward(db: AsyncSession, rule: PortForwardRule) -> None:
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, PortForwardDeployment, PortForwardDeployment.port_forward_id == rule.id)
failures = []
result = await db.execute(select(PortForwardDeployment).where(PortForwardDeployment.port_forward_id == rule.id))
for deployment in result.scalars().all():
@@ -0,0 +1,120 @@
"""Cleanup candidates of one firewall: unused, shadowed, duplicate and long-disabled rules (read-only).
Nothing here changes a firewall. The list says *which rules are worth a look and how sure that is*; removing
them is a separate step (not part of this module).
Categories
* ``unused`` OPNexus has watched the rule's pf counters for at least N days and saw no hit
(services/rule_hit_service.py; N is a parameter, default 30).
* ``shadowed`` an earlier quick rule already decides everything the rule matches (rule_analysis/coverage.py).
* ``duplicate`` like shadowed, with the same action and the rule also covering the earlier one.
* ``disabled_long`` disabled, and OPNexus has seen it unchanged for N days or more (rule history).
Confidence (never higher than what is proven; what is not proven is never "sure")
* ``sure`` the evidence is complete: unused = the rule is loaded in pf, has no schedule, the counters were
sampled recently and over the whole window; shadowed/duplicate = literal comparison, order from
``sort_order``, the covering rule is loaded; disabled_long = observed unchanged for N days.
* ``probable`` relies on a guess or on static alias content, or pf's loaded ruleset is not available, or the rule
has a schedule (a hit might happen at another time), or (cluster) some node has no usage data for
the rule. In a cluster "unused" is judged over all nodes (rule_hit_service.merge_cluster_hits):
the CARP backup sees almost no traffic, so its zero alone never makes a rule unused.
* ``unknown`` there is a hint but no support: the rule is not loaded in pf (no counters can exist), or the
counters are stale.
``removable`` says whether the rule could be offered for removal at all; the reason is a code:
``automatic`` (generated by OPNsense), ``legacy`` (not yet migrated, migrate it in OPNsense first),
``managed`` (OPNexus manages the rule: remove it through the normal flow of the rules page).
"""
from __future__ import annotations
from datetime import datetime, timedelta
from typing import Iterable
from app.services import rule_hit_service
from app.services.rule_analysis import coverage
from app.services.rule_analysis.context import Context
from app.services.rule_analysis.model import SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO, NormRule
CATEGORIES = ("unused", "shadowed", "duplicate", "disabled_long")
DEFAULT_UNUSED_DAYS = 30
MIN_DAYS, MAX_DAYS = 1, 3650
def removal_block(rule: NormRule, managed: dict | None) -> str | None:
"""Why the rule could not be offered for removal (None = it could)."""
if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_NAT_AUTO):
return "automatic"
if rule.legacy:
return "legacy"
if managed is not None:
return "managed"
return None
def unused_confidence(rule: NormRule, hit, now: datetime) -> str:
"""Confidence of an 'unused' verdict (the caller already knows the window passed without a hit)."""
if rule.loaded is False:
return "unknown" # pf did not load it (schedule, missing interface, gateway down): no counter can exist
if hit.last_sampled_at is None or now - hit.last_sampled_at > rule_hit_service.SAMPLE_FRESH:
return "unknown" # stale counters say nothing about the present
if rule.loaded is None or rule.sched:
return "probable" # pf's ruleset not available / the rule only works at certain times
if getattr(hit, "peer_missing", False):
return "probable" # cluster: some node has no usage data for this rule (rule_hit_service.ClusterHit)
return "sure"
def _name(rule: NormRule) -> str:
return rule.description or rule.label or rule.uuid or (f"@{rule.pf_numbers[0]}" if rule.pf_numbers else "")
def build(rules: list[NormRule], ctx: Context, *, hits: dict, states: dict, managed: dict[str, dict],
now: datetime, unused_days: int = DEFAULT_UNUSED_DAYS, order_guessed: bool = False) -> list[dict]:
"""Candidate rows, in evaluation order. ``hits``: RuleHit per rule label, ``states``: LiveRuleState per label,
``managed``: {label: {"kind", "id", "name"}} of the rules OPNexus deployed to this firewall."""
unused_days = max(MIN_DAYS, min(MAX_DAYS, int(unused_days)))
overlaps = {id(o.rule): o for o in coverage.analyse(rules, ctx, order_guessed=order_guessed)}
rows: list[dict] = []
for rule in rules:
if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
continue
categories: list[dict] = []
key = rule.uuid.lower()
hit = hits.get(key)
if hit is not None and rule_hit_service.classify(hit, now, unused_days) == "unused":
category = {"kind": "unused", "confidence": unused_confidence(rule, hit, now), "days": unused_days}
if getattr(hit, "peer_missing", False):
category["notes"] = ["CLUSTER_PEER_USAGE_UNKNOWN"]
categories.append(category)
overlap = overlaps.get(id(rule))
if overlap is not None:
categories.append({
"kind": overlap.kind, "confidence": overlap.confidence,
"by": {"uuid": overlap.by.uuid, "description": _name(overlap.by), "section": overlap.by.section,
"automatic": overlap.by.automatic},
})
state = states.get(key)
if (not rule.enabled and state is not None and state.present
and now - state.changed_at >= timedelta(days=unused_days)):
categories.append({"kind": "disabled_long", "confidence": "sure", "days": unused_days,
"since": state.changed_at})
if not categories:
continue
block = removal_block(rule, managed.get(key))
rows.append({
**rule.summary(), "enabled": rule.enabled, "categories": categories,
"hit_state": rule_hit_service.classify(hit, now, unused_days) if hit is not None else None,
"last_hit_at": hit.last_hit_at if hit is not None else None,
"watched_since": hit.first_seen_at if hit is not None else None,
"removable": block is None, "not_removable_reason": block,
"managed_by": managed.get(key),
})
return rows
def summarize(rows: Iterable[dict]) -> dict:
counts = {kind: 0 for kind in CATEGORIES}
for row in rows:
for category in row["categories"]:
counts[category["kind"]] += 1
return counts
@@ -0,0 +1,290 @@
"""Does an earlier rule make a later one pointless? (shadowed / duplicate)
Pure functions on ``NormRule``; no I/O. Used by the security check (``compliance_service``, rules taken
from config.xml) and by the cleanup candidates (rules taken from ``searchRule``).
pf evaluates all rules in order, the last matching rule wins unless a matching rule is ``quick``. So a later
rule b can only be made dead by an earlier rule a when
* a is ``quick`` (a non-quick match is overridden by later rules and decides nothing by itself),
* a is in force (enabled, not known to be unloaded) and has no condition we cannot compare
(schedule, tags, ``received-on``, TCP flags, rate limits), and
* every packet b matches is also matched by a: set coverage per field (interfaces, direction, address
family, protocols, source/destination, ports, ICMP types).
The order is ``sort_order`` (floating 200000 before groups 300000+ before interfaces 400000, then the
sequence); nothing is re-implemented here, ``NormRule.order`` already holds it.
Every comparison answers ``"sure"`` (literal values or identical names), ``"probable"`` (the result relies
on the static content of aliases or on a guessed order) or ``None`` (not provable, so no claim is made).
Whatever is not proven is never "sure": the weakest link decides.
"""
from __future__ import annotations
import ipaddress
import re
from dataclasses import dataclass
from app.services.rule_analysis.context import Context
from app.services.rule_analysis.model import (
SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID, SECTION_NAT_AUTO,
Endpoint, NormRule,
)
from app.services.rule_analysis.normalize import _af, _protos
from app.services.rule_analysis.resolve import ADDRESS_TYPES, MAX_ALIAS_DEPTH, port_ranges
SURE, PROBABLE = "sure", "probable"
_RANK = {None: 0, PROBABLE: 1, SURE: 2}
def weakest(*values: str | None) -> str | None:
"""The lowest confidence of the arguments (None beats everything: one unproven part spoils the claim)."""
return min(values, key=lambda v: _RANK[v]) if values else SURE
def _empty_context() -> Context:
return Context(rows=[], pf=None, interfaces={}, groups=None, aliases={}, port_forwards=None)
# ── Addresses ─────────────────────────────────────────────────────────────
Range = tuple[int, int, int] # (ip version, low, high)
def _literal(token: str) -> Range | None:
"""CIDR, single address or ``low-high`` range as an integer interval; None for names."""
token = token.strip()
try:
if "-" in token and not token.startswith("-"):
low, _, high = token.partition("-")
lo, hi = ipaddress.ip_address(low.strip()), ipaddress.ip_address(high.strip())
return (lo.version, int(lo), int(hi)) if lo.version == hi.version and lo <= hi else None
net = ipaddress.ip_network(token, strict=False)
except ValueError:
return None
return (net.version, int(net.network_address), int(net.broadcast_address))
def _alias_ranges(name: str, ctx: Context, depth: int = 0, seen: tuple = ()) -> list[Range] | None:
"""Static content of a host/network alias as intervals; None when anything about it is not static
(tables, URLs, FQDNs, exclusions, unknown names, cycles)."""
alias = ctx.aliases.get(name)
if alias is None or name in seen or depth > MAX_ALIAS_DEPTH:
return None
if str(alias.get("type") or "").lower() not in ADDRESS_TYPES:
return None
if str(alias.get("enabled", "1")).strip().lower() in ("0", "false", "no", "off"):
return None # a disabled alias is an empty table in pf: its configured content proves nothing
result: list[Range] = []
for entry in (e.strip() for e in re.split(r"[\n,]", str(alias.get("content") or "")) if e.strip()):
if entry.startswith("!"):
return None
literal = _literal(entry)
if literal is not None:
result.append(literal)
continue
nested = _alias_ranges(entry, ctx, depth + 1, (*seen, name)) if entry in ctx.aliases else None
if nested is None:
return None
result += nested
return result or None
def _ranges_of(token: str, ctx: Context) -> tuple[list[Range] | None, bool]:
"""(intervals, came_from_alias)"""
literal = _literal(token)
if literal is not None:
return [literal], False
return _alias_ranges(token, ctx), True
def _covered(inner: list[Range], outer: list[Range]) -> bool:
return all(any(o[0] == i[0] and o[1] <= i[1] and i[2] <= o[2] for o in outer) for i in inner)
def _item_covers(x: str, y: str, ctx: Context) -> str | None:
"""Does the address item x contain everything the item y contains?"""
x, y = x.strip(), y.strip()
if x == y:
return SURE
outer, outer_alias = _ranges_of(x, ctx)
inner, inner_alias = _ranges_of(y, ctx)
if outer is None or inner is None or not _covered(inner, outer):
return None
return PROBABLE if (outer_alias or inner_alias) else SURE
def endpoint_covers(a: Endpoint, b: Endpoint, ctx: Context) -> str | None:
if a.is_any and not a.negated:
return SURE
if b.is_any and not b.negated:
return None # a is narrower than "any"
if a.negated != b.negated:
return None
if not a.negated: # every item of b lies inside some item of a
parts = []
for y in b.items:
best = None
for x in a.items:
got = _item_covers(x, y, ctx)
if _RANK[got] > _RANK[best]:
best = got
parts.append(best)
return weakest(*parts)
# "!A" covers "!B" when A lies inside B (outside of B is a subset of outside of A)
parts = []
for x in a.items:
best = None
for y in b.items:
got = _item_covers(y, x, ctx)
if _RANK[got] > _RANK[best]:
best = got
parts.append(best)
return weakest(*parts)
# ── Ports, interfaces, the rest ───────────────────────────────────────────
def _norm_ports(spec: str) -> str:
return re.sub(r"\s+", "", spec or "").lower()
def ports_cover(a: str, b: str, ctx: Context) -> str | None:
if _norm_ports(a) in ("", "any"):
return SURE
if _norm_ports(b) in ("", "any"):
return None
if _norm_ports(a) == _norm_ports(b):
return SURE
outer, _ = port_ranges(a, ctx)
inner, _ = port_ranges(b, ctx)
if outer is None or inner is None:
return None
if not all(any(lo <= il and ih <= hi for lo, hi in outer) for il, ih in inner):
return None
alias_involved = any(part.strip() in ctx.aliases for spec in (a, b) for part in re.split(r"[\n,]", spec))
return PROBABLE if alias_involved else SURE
def _expand(names: set[str], ctx: Context) -> set[str]:
result: set[str] = set()
for name in names:
result |= (ctx.groups[name] | {name}) if ctx.groups and name in ctx.groups else {name}
return result
def interfaces_cover(a: NormRule, b: NormRule, ctx: Context) -> str | None:
if not a.interfaces and not a.if_not:
return SURE # every interface
if a.if_not or b.if_not:
return SURE if (a.interfaces, a.if_not) == (b.interfaces, b.if_not) else None
if not b.interfaces:
return None # b applies everywhere, a only to some interfaces
return SURE if _expand(b.interfaces, ctx) <= _expand(a.interfaces, ctx) else None
def _conditions_ok(a: NormRule) -> bool:
"""a must not depend on something we cannot compare: schedule, tags, received-on, TCP flags, limits."""
return not (a.sched or a.tagged or a.received_on or a.extra_notes)
def covers_fields(a: NormRule, b: NormRule, ctx: Context | None = None) -> str | None:
"""Every packet rule b matches is already matched by rule a (order and enabled state not considered)."""
ctx = ctx or _empty_context()
if a.quick is not True or not _conditions_ok(a):
return None
if a.direction != "any" and a.direction != b.direction:
return None
if not b.af <= a.af:
return None
if a.protos is not None and (b.protos is None or not b.protos <= a.protos):
return None
if a.icmptypes is not None and (b.icmptypes is None or not b.icmptypes <= a.icmptypes):
return None
parts = [
interfaces_cover(a, b, ctx),
endpoint_covers(a.src, b.src, ctx), endpoint_covers(a.dst, b.dst, ctx),
ports_cover(a.sport, b.sport, ctx), ports_cover(a.dport, b.dport, ctx),
]
return weakest(*parts)
# ── Order-aware analysis ──────────────────────────────────────────────────
def _in_force(rule: NormRule) -> bool:
return rule.enabled and rule.loaded is not False and rule.section not in (SECTION_INVALID, SECTION_NAT_AUTO)
# normalize() notes after which the evaluation order is not trustworthy: findings stay at "probable" at most.
ORDER_DOUBT = frozenset({"ORDER_GUESSED", "ORDER_MISMATCH"})
def order_doubtful(notes) -> bool:
return bool(ORDER_DOUBT & set(notes))
@dataclass
class Overlap:
rule: NormRule # the later rule that is made pointless
by: NormRule # the earlier rule that already decides everything the later one matches
kind: str # "shadowed" | "duplicate"
confidence: str # "sure" | "probable"
def analyse(rules: list[NormRule], ctx: Context | None = None, *, order_guessed: bool = False) -> list[Overlap]:
"""``rules`` in evaluation order. Each later rule is reported at most once, with the first earlier rule that
covers it. Duplicate = same action and the later rule also covers the earlier one."""
ctx = ctx or _empty_context()
result: list[Overlap] = []
for index, later in enumerate(rules):
if not later.enabled or later.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
continue
if later.action is None:
continue
for earlier in rules[:index]:
if not _in_force(earlier) or earlier.action is None:
continue
confidence = covers_fields(earlier, later, ctx)
if confidence is None:
continue
# (Equal sort_order without pf line numbers always comes with loaded=None, i.e. "probable" here.)
if order_guessed or not (earlier.order_known and later.order_known) or earlier.loaded is None:
confidence = weakest(confidence, PROBABLE)
duplicate = earlier.action == later.action and covers_fields(later, earlier, ctx) is not None
if duplicate:
confidence = weakest(confidence, covers_fields(later, earlier, ctx))
result.append(Overlap(later, earlier, "duplicate" if duplicate else "shadowed", confidence or PROBABLE))
break
return result
# ── Rules read from a config.xml (security check) ─────────────────────────
def config_order(rule: dict) -> tuple[int, int]:
"""sort_order of a rule from config.xml in OPNsense's terms: floating 200000, interface group 300000,
single interface 400000, each by sequence."""
return (200000 if rule["floating"] else 300000 if rule.get("group") else 400000, rule["order"])
def _config_endpoint(raw: str, negated: bool) -> Endpoint:
raw = (raw or "").strip()
items = [] if raw.lower() in ("", "any") else [i.strip() for i in raw.split(",") if i.strip()]
return Endpoint(raw=raw, negated=negated, items=items)
def from_config_rule(rule: dict) -> NormRule:
"""Adapter for the rule dicts of compliance_service (MVC and legacy shapes already merged). Nothing is known
about the loaded pf ruleset there, so ``loaded`` stays None (coverage then caps at "probable")."""
order = config_order(rule)
section = SECTION_FLOATING if rule["floating"] else SECTION_GROUP if rule.get("group") else SECTION_INTERFACE
return NormRule(
uuid="", label="", description=rule["descr"], order=order, order_known=True, section=section,
enabled=rule["enabled"], action=rule["action"] if rule["action"] in ("pass", "block", "reject") else None,
quick=bool(rule["quick"]), interfaces=set(rule["interfaces"]), if_not=bool(rule.get("if_not")),
direction=rule["direction"] or "in", af=_af(rule["ipprotocol"]), protos=_protos(rule["protocol"]),
src=_config_endpoint(rule["src"], rule["src_not"]), dst=_config_endpoint(rule["dst"], rule["dst_not"]),
sport=rule["sport"] or "", dport=rule["dport"] or "",
icmptypes={t.strip() for t in (rule.get("icmptype") or "").split(",") if t.strip()} or None,
tagged=rule.get("tagged") or "", received_on=rule.get("received_on") or "", sched=rule.get("sched") or "",
extra_notes=["RATE_LIMIT"] if rule.get("limited") else [],
)
@@ -117,6 +117,31 @@ def rule_fields(r: NormRule, q: Query, direction: str, iface: str, ctx: Context,
return fields
def match_rule(r: NormRule, q: Query, ctx: Context, *, iface: str | None = None,
source: Outcome | None = None, overrides: dict[str, Outcome] | None = None) -> Outcome:
"""Could the rule's own fields match the packet ``q`` arriving on ``iface``? Order, ``quick``, other rules and
whether the rule is enabled or loaded play no part: this is the question "is this rule about such a packet at
all", which the cleanup asks about the way the admin reaches the firewall (DESIGN-1.31 §3.2).
``iface`` defaults to ``q.iface`` and then to the interface guessed from the address; an interface that cannot
be resolved is UNKNOWN. ``source`` replaces the source field (the caller judged it for a whole network, see
rule_analysis/protection.py); ``overrides`` replaces further fields by name (e.g. ``interface`` for a packet
arriving on a device that is not an assigned interface). Tags set by earlier rules are not known here, so ``tagged`` is UNKNOWN. Only
``False`` is a proof that the rule does not match; ``None`` (unknown) must be treated like ``True``."""
iface = iface or q.iface or guess_iface(q.src if q.direction == "in" else q.dst, ctx)
if iface is None:
return Outcome(None, ["IFACE_UNRESOLVED"])
fields = rule_fields(r, q, q.direction, iface, ctx, set(), None)
if source is not None:
fields["source"] = source
for name, outcome in (overrides or {}).items():
if name in fields or name == "interface":
fields[name] = outcome
value = t_and(*(f.value for f in fields.values()))
reasons = sorted({reason for f in fields.values() if f.value is None for reason in f.reasons}) if value is None else []
return Outcome(value, reasons)
def _step(r: NormRule, fields: dict[str, Outcome], result, effect: str) -> dict:
return {
"rule": r.summary(), "result": tri_str(result), "effect": effect,
@@ -13,6 +13,8 @@ NAT, 600000 = interface missing, never loaded).
"""
from __future__ import annotations
import re
from app.services.rule_analysis.context import Context
from app.services.rule_analysis.model import (
SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID,
@@ -169,6 +171,11 @@ def normalize_row(row: dict, ctx: Context) -> NormRule:
return rule
# Words in an unlabeled pf line that restrict it in ways antispoof_rule() does not model (protocol, ports, flags,
# tags, user/group, received-on, ICMP types, OS): such a line must never be used as a covering rule.
_UNMODELLED = re.compile(r"\b(proto|port|flags|tagged|received-on|user|group|icmp6?-type|os|tag|label)\b")
def antispoof_rule(line: PfLine, order: tuple[int, int]) -> NormRule:
"""An unlabeled pf line OPNsense generated itself (antispoof / own-address block)."""
return NormRule(
@@ -179,6 +186,7 @@ def antispoof_rule(line: PfLine, order: tuple[int, int]) -> NormRule:
src=Endpoint(raw="" if line.src == "any" else line.src, items=[] if line.src == "any" else [line.src]),
dst=Endpoint(raw="" if line.dst == "any" else line.dst, items=[] if line.dst == "any" else [line.dst]),
sport="", dport="", automatic=True, loaded=True, pf_numbers=[line.number], pf_line=line,
extra_notes=["UNMODELLED_CONDITION"] if _UNMODELLED.search(line.text) else [],
)
@@ -188,7 +196,9 @@ def normalize(ctx: Context) -> tuple[list[NormRule], list[str]]:
rules = [normalize_row(row, ctx) for row in ctx.rows]
if any(not r.order_known for r in rules):
notes.append("ORDER_GUESSED")
rules.sort(key=lambda r: (r.order, r.uuid))
# Equal sort_order (e.g. two rules with the same sequence): what pf actually loaded decides, the uuid only as
# a last resort (then coverage.analyse() lowers its confidence, it cannot know the real order).
rules.sort(key=lambda r: (r.order, min(r.pf_numbers) if r.pf_numbers else 10**9, r.uuid))
if ctx.pf is not None:
loaded = [r for r in rules if r.loaded and r.pf_numbers]
firsts = [min(r.pf_numbers) for r in loaded]
@@ -0,0 +1,256 @@
"""May a rule be switched off without touching the way the admin reaches the firewall? (DESIGN-1.31 §3.2/§3.3)
Pure functions on ``NormRule`` and the analysis ``Context``; no I/O.
Management access (§3.2). For every configured management source (IP or network), every address of the
firewall (the address in ``host.base_url`` plus every interface address and VIP) and every management port
(the API port of ``base_url``, 443, 80, 22) and every interface the packet could arrive on, the rule's own
fields are compared with that packet (``engine.match_rule``). A rule is safe only when EVERY such comparison is
``False``. ``True`` or unknown means the rule is about management traffic: it is protected.
Why that is enough without simulating "the ruleset afterwards": a rule that does not match a packet takes no part
in its evaluation at all - it neither decides it nor tags it (pf only tags on a match) - so removing it cannot
change what happens to that packet. This holds whatever the verdict for the packet is, even when the analysis
of the whole ruleset is only "probable". The packet's ingress interface is a routing question OPNexus cannot prove
(asymmetric routes, VPNs, failover), so every interface is tried: that only ever protects more.
A management source that is a network is judged for all of its addresses at once: up to 256 addresses are
enumerated (exact, with every resolver rule of the analysis), larger networks are compared as address ranges
with the rule's literal/alias/interface addresses; anything that cannot be resolved is unknown = protected.
Infrastructure (§3.3, reason ``infrastructure``): CARP (IP protocol 112) and pfsync (240) rules, every rule that
could match CARP advertisements (to 224.0.0.18 / ff02::12 from the interface's own networks) on an interface with
a CARP VIP, and every rule that could match pfsync traffic on the pfsync interface (System > High Availability,
``core/hasync/get``; when that cannot be read on a cluster node, on every interface). The second leash of the
design ("(self) with port 22/80/443/API" when §3.2 is not False) is part of §3.2 already: such a rule is
protected there.
"""
from __future__ import annotations
import ipaddress
from urllib.parse import urlsplit
from app.services.rule_analysis import coverage, resolve
from app.services.rule_analysis.context import BUILTIN_GROUP_DEVICES, Context
from app.services.rule_analysis.engine import Query, match_rule
from app.services.rule_analysis.model import Endpoint, IPAddr, IPNet, NormRule, Outcome, o_or, t_not
MANAGEMENT_PORTS = (443, 80, 22)
# A management network up to this size is checked address by address (exact); larger ones as address ranges.
ENUMERATE_LIMIT = 256
CARP_PROTO, PFSYNC_PROTO = "112", "240"
CARP_GROUPS = {4: ipaddress.ip_address("224.0.0.18"), 6: ipaddress.ip_address("ff02::12")}
PFSYNC_GROUP = ipaddress.ip_address("224.0.0.240")
_LOOPBACK = {ipaddress.ip_address("127.0.0.1"), ipaddress.ip_address("::1")}
def parse_sources(values) -> list[IPNet]:
"""The configured management sources as networks (an address becomes a /32 or /128). Invalid entries are
dropped; the caller treats an empty result as "not configured" (the cleanup is then locked)."""
nets: list[IPNet] = []
for value in values or []:
try:
net = ipaddress.ip_network(str(value).strip(), strict=False)
except ValueError:
continue
if net not in nets:
nets.append(net)
return nets
def management_targets(base_url: str, ctx: Context, extra_ports=()) -> tuple[list[IPAddr], list[int]]:
"""Firewall addresses and ports the management traffic goes to; ``extra_ports`` = the admin's
``cleanup_management_ports`` (an SSH or GUI port moved away from the defaults)."""
parts = urlsplit(base_url or "")
addrs: list[IPAddr] = []
try:
addrs.append(ipaddress.ip_address((parts.hostname or "").strip("[]")))
except ValueError:
pass # a DNS name: the interface addresses below cover the address it resolves to on the firewall
for addr in ctx.self_addrs:
if addr not in _LOOPBACK and addr not in addrs:
addrs.append(addr)
try:
api_port = parts.port or 443
except ValueError:
api_port = 443
ports = [api_port]
for port in (*MANAGEMENT_PORTS, *extra_ports):
if isinstance(port, int) and 1 <= port <= 65535 and port not in ports:
ports.append(port)
return addrs, ports
def _interfaces(ctx: Context) -> list[str]:
return sorted(i for i in ctx.interfaces if i != "lo0")
# A packet can also arrive on a device that is not an assigned interface: a WireGuard instance (wg0), an OpenVPN
# server (ovpns1), IPsec (enc0). They have no identifier, so they are not in ``ctx.interfaces``, and the built-in
# groups that stand for them (``wireguard``, ``openvpn``, ``enc0``) list no members in ``firewall/group/searchItem``
# (live fixture tests/fixtures/rule_analysis_fw.json: members ""). One extra, virtual ingress is checked per query:
UNASSIGNED = "(unassigned)"
def unassigned_interface(rule: NormRule, ctx: Context) -> Outcome:
"""Could the rule's interface field match a packet arriving on an unassigned device? An assigned interface or a
user-defined group (its members are assigned interfaces) cannot; a built-in group or an unknown name might
(members unknown = protected); no interface (floating "any") always does."""
if not rule.interfaces:
return Outcome(True)
results = []
for name in rule.interfaces:
if name in ctx.interfaces:
results.append(Outcome(False))
elif name in BUILTIN_GROUP_DEVICES or ctx.groups is None or name not in ctx.groups:
results.append(Outcome(None, ["IFACE_MEMBERS_UNKNOWN"]))
else:
results.append(Outcome(False))
result = o_or(results)
return Outcome(t_not(result.value), result.reasons) if rule.if_not else result
def _ingress(rule: NormRule, ctx: Context) -> list[tuple[str, dict | None]]:
"""(interface, field overrides) for every ingress a management packet could take."""
virtual = {"interface": unassigned_interface(rule, ctx)}
if rule.received_on:
virtual["received_on"] = Outcome(None, ["RECEIVED_ON"])
return [(i, None) for i in _interfaces(ctx)] + [(UNASSIGNED, virtual)]
def _item_ranges(token: str, ctx: Context) -> list[coverage.Range] | None:
"""Address ranges an endpoint item stands for; None when that is not static."""
ranges, _alias = coverage._ranges_of(token, ctx)
if ranges is not None:
return ranges
name = token.strip().lower()
if name in ("(self)", "self", "this firewall"):
return [(a.version, int(a), int(a)) for a in ctx.self_addrs]
if name.endswith("ip") and name[:-2] in ctx.interfaces:
info = ctx.interfaces[name[:-2]]
return [(a.version, int(a), int(a)) for a in (*info.addrs, *info.vips)]
if name.startswith("__") and name.endswith("_network"):
name = name[2:-len("_network")]
if name in ctx.interfaces:
return [(n.version, int(n.network_address), int(n.broadcast_address)) for n in ctx.interfaces[name].nets]
return None
def _covers_all(ranges: list[coverage.Range], version: int, low: int, high: int) -> bool:
cursor = low
for _v, a, b in sorted(r for r in ranges if r[0] == version):
if a > cursor:
return False
cursor = max(cursor, b + 1)
if cursor > high:
return True
return cursor > high
def source_outcome(endpoint: Endpoint, net: IPNet, ctx: Context) -> Outcome:
"""Does the endpoint match at least one address of ``net``? (False is a proof for every address.)"""
if endpoint.is_any:
return Outcome(True)
if net.num_addresses <= ENUMERATE_LIMIT:
return o_or([resolve.endpoint_match(endpoint, ip, ctx) for ip in net])
ranges: list[coverage.Range] = []
for item in endpoint.items:
got = _item_ranges(item, ctx)
if got is None:
return Outcome(None, ["SOURCE_NOT_RESOLVABLE"])
ranges += got
low, high = int(net.network_address), int(net.broadcast_address)
if endpoint.negated:
return Outcome(not _covers_all(ranges, net.version, low, high))
return Outcome(any(v == net.version and a <= high and b >= low for v, a, b in ranges))
def _query(src: IPAddr, dst: IPAddr, proto: str, label: str, dport: int | None) -> Query:
return Query(src=src, dst=dst, proto=proto, proto_label=label, dport=dport, direction="in", egress="none")
def management_check(rule: NormRule, ctx: Context, sources: list[IPNet], targets: list[IPAddr],
ports: list[int]) -> dict:
"""{"result": "false" | "true" | "unknown", "queries": n, "reasons": [...], "match": {...} | None}.
Only "false" makes the rule removable."""
reasons: set[str] = set()
queries = 0
unknown_at = None
interfaces = _ingress(rule, ctx)
for net in sources:
src = source_outcome(rule.src, net, ctx)
if src.value is False:
dsts = [d for d in targets if d.version == net.version]
queries += len(dsts) * len(ports) * len(interfaces)
continue
for dst in (d for d in targets if d.version == net.version):
for port in ports:
q = _query(net.network_address, dst, "6", "tcp", port)
for iface, overrides in interfaces:
queries += 1
out = match_rule(rule, q, ctx, iface=iface, source=src, overrides=overrides)
where = {"source": str(net), "destination": str(dst), "port": port, "interface": iface}
if out.value is True:
return {"result": "true", "queries": queries, "reasons": [], "match": where}
if out.value is None:
reasons.update(out.reasons)
unknown_at = unknown_at or where
if not queries:
# No address of the firewall in the family of any source: nothing could be proven.
return {"result": "unknown", "queries": 0, "reasons": ["NO_MANAGEMENT_QUERY"], "match": None}
if unknown_at is not None:
return {"result": "unknown", "queries": queries, "reasons": sorted(reasons), "match": unknown_at}
return {"result": "false", "queries": queries, "reasons": [], "match": None}
def _hasync_interface(hasync: dict | None) -> str | None:
"""The pfsync interface from ``core/hasync/get`` ("" = HA sync disabled); None when not readable."""
node = (hasync or {}).get("hasync") if isinstance(hasync, dict) else None
if not isinstance(node, dict) or "pfsyncinterface" not in node:
return None
value = node.get("pfsyncinterface")
if isinstance(value, dict):
return next((k for k, o in value.items() if isinstance(o, dict) and str(o.get("selected")) == "1"), "")
return str(value or "")
def _hasync_peer(hasync: dict | None) -> IPAddr | None:
node = (hasync or {}).get("hasync") if isinstance(hasync, dict) else None
try:
return ipaddress.ip_address(str((node or {}).get("pfsyncpeerip") or "").strip())
except ValueError:
return None
def infrastructure_reason(rule: NormRule, ctx: Context, *, hasync: dict | None, clustered: bool) -> str | None:
"""'infrastructure' when the rule could carry CARP or pfsync traffic, else None."""
if rule.protos is not None and rule.protos & {CARP_PROTO, PFSYNC_PROTO}:
return "infrastructure"
for info in ctx.interfaces.values():
if info.identifier == "lo0" or not info.vips:
continue
for net in info.nets:
group = CARP_GROUPS.get(net.version)
out = match_rule(rule, _query(net.network_address, group, CARP_PROTO, "carp", None), ctx,
iface=info.identifier, source=source_outcome(rule.src, net, ctx))
if out.value is not False:
return "infrastructure"
pfsync_if = _hasync_interface(hasync)
if pfsync_if == "" or (pfsync_if is None and not clustered):
return None # HA sync off (or a single firewall whose HA settings are not readable)
peer = _hasync_peer(hasync)
candidates = [ctx.interfaces[pfsync_if]] if pfsync_if in ctx.interfaces else \
[i for i in ctx.interfaces.values() if i.identifier != "lo0"] # unknown: every interface
for info in candidates:
for net in info.nets:
if peer is not None and peer.version == net.version:
src = Outcome(resolve.endpoint_match(rule.src, peer, ctx).value)
src_ip = peer
else:
src, src_ip = source_outcome(rule.src, net, ctx), net.network_address
for dst in [*(a for a in info.addrs if a.version == net.version)] + ([PFSYNC_GROUP] if net.version == 4 else []):
out = match_rule(rule, _query(src_ip, dst, PFSYNC_PROTO, "pfsync", None), ctx,
iface=info.identifier, source=src)
if out.value is not False:
return "infrastructure"
return None
@@ -3,11 +3,25 @@ from __future__ import annotations
import ipaddress
import logging
from datetime import datetime, timezone
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.models.rule_hit import RuleHit
from app.models.rule_history import LiveRuleState
from app.services import rule_hit_service
from app.services.rule_history_service import BASELINE_KEY
from app.services.rule_analysis import cleanup, coverage
from app.services.rule_analysis import context as ctx_mod
from app.services.rule_analysis.engine import Query, evaluate
from app.services.rule_analysis.normalize import normalize
from app.services.rule_analysis.normalize import PROTO_NUMBERS, norm_proto
logger = logging.getLogger(__name__)
@@ -74,3 +88,75 @@ async def analyze_host(host: OpnsenseHost, q: Query, *, refresh: bool = False) -
if any(ctx.refs.get(ip) is None for ip in (str(q.src), str(q.dst))):
result["notes"] = sorted({*result["notes"], "ALIAS_REFERENCES_UNAVAILABLE"})
return result
async def _managed_rules(db: AsyncSession, host: OpnsenseHost) -> dict[str, dict]:
"""{label on the firewall: what manages it} for every rule OPNexus deployed to this host."""
managed: dict[str, dict] = {}
for dep_uuid, rule_id, description in (await db.execute(
select(RuleDeployment.opnsense_rule_uuid, FirewallRule.id, FirewallRule.description)
.join(FirewallRule, FirewallRule.id == RuleDeployment.rule_id)
.where(RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid.is_not(None)))).all():
managed[dep_uuid.lower()] = {"kind": "rule", "id": str(rule_id), "name": description}
for dep_uuid, rule_id, description in (await db.execute(
select(PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid, PolicyHierarchyRule.id, PolicyHierarchyRule.description)
.join(PolicyHierarchyRule, PolicyHierarchyRule.id == PolicyHierarchyRuleDeployment.policy_hierarchy_id)
.where(PolicyHierarchyRuleDeployment.host_id == host.id,
PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid.is_not(None)))).all():
managed[dep_uuid.lower()] = {"kind": "policy_hierarchy", "id": str(rule_id), "name": description}
return managed
async def _hits(db: AsyncSession, host_id) -> dict:
return {r.rule_uuid.lower(): r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host_id))).all()}
async def cluster_hits(db: AsyncSession, host: OpnsenseHost) -> tuple[dict, dict]:
"""(usage to judge "unused" with, the host's own hit rows). For a cluster member the usage is merged over all
nodes (DESIGN-1.31 §3.4): "unused" only when it holds on every node."""
own = await _hits(db, host.id)
if not host.cluster_name:
return own, own
others = (await db.scalars(select(OpnsenseHost.id).where(
OpnsenseHost.cluster_name == host.cluster_name, OpnsenseHost.id != host.id))).all()
if not others:
return own, own
return rule_hit_service.merge_cluster_hits(own, [await _hits(db, other) for other in others]), own
async def cleanup_view(db: AsyncSession, host: OpnsenseHost, *, unused_days: int = cleanup.DEFAULT_UNUSED_DAYS,
refresh: bool = False, now: datetime | None = None):
"""(context, normalized rules, candidate payload, managed rules) - the cleanup service needs all of them."""
now = now or datetime.now(timezone.utc)
ctx = await ctx_mod.load_context(host, use_cache=not refresh)
rules, notes = normalize(ctx)
usage, hits = await cluster_hits(db, host)
states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
managed = await _managed_rules(db, host)
rows = cleanup.build(rules, ctx, hits=usage, states=states, managed=managed, now=now,
unused_days=unused_days, order_guessed=coverage.order_doubtful(notes))
return ctx, rules, _payload(host, rules, rows, hits, states, notes, ctx, unused_days, now), managed
async def cleanup_candidates(db: AsyncSession, host: OpnsenseHost, *, unused_days: int = cleanup.DEFAULT_UNUSED_DAYS,
refresh: bool = False, now: datetime | None = None) -> dict:
"""Read-only. Raises when the firewall cannot be read (callers map that)."""
return (await cleanup_view(db, host, unused_days=unused_days, refresh=refresh, now=now))[2]
def _payload(host, rules, rows, hits, states, notes, ctx, unused_days, now) -> dict:
all_notes = sorted({*notes, *ctx.notes})
if host.cluster_name and any("CLUSTER_PEER_USAGE_UNKNOWN" in c.get("notes", ()) for r in rows for c in r["categories"]):
all_notes = sorted({*all_notes, "CLUSTER_PEER_USAGE_UNKNOWN"})
return {
"host_id": str(host.id), "host_name": host.name, "cluster_name": host.cluster_name,
"unused_days": max(cleanup.MIN_DAYS, min(cleanup.MAX_DAYS, int(unused_days))), "generated_at": now,
"rules_total": sum(1 for r in rules if not r.automatic and r.section not in ("antispoof", "nat_auto", "invalid")),
"candidates": rows, "counts": cleanup.summarize(rows), "notes": all_notes,
"usage": {
"tracked": len(hits),
"collecting": sum(1 for h in hits.values() if rule_hit_service.classify(h, now, unused_days) == "collecting"),
"watching_since": min((h.first_seen_at for h in hits.values()), default=None),
},
"history": {"watching_since": baseline.first_seen_at if (baseline := states.get(BASELINE_KEY)) else None},
}
File diff suppressed because it is too large. Load diff
+5 -1
View File
@@ -155,9 +155,13 @@ async def sweep_expired_deployments(db: AsyncSession) -> list[RuleDeployment]:
async def undeploy_rule(db: AsyncSession, rule: FirewallRule) -> None:
from app.services.host_maintenance_service import ensure_no_open_cleanup
failures = []
result = await db.execute(select(RuleDeployment).where(RuleDeployment.rule_id == rule.id))
for deployment in result.scalars().all():
deployments = list(result.scalars().all())
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them.
await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])
for deployment in deployments:
host = await db.get(OpnsenseHost, deployment.host_id)
if host and deployment.opnsense_rule_uuid:
try:
@@ -0,0 +1,330 @@
"""Change history of the rules that are live on a firewall, also those OPNexus did not create.
Every ``rule_observe`` cycle reads ``searchRule`` (and the counters, for the usage sampling) of each enabled
firewall and compares the user rules with the last observed state (``live_rule_states``). A difference becomes
one ``rule_change_events`` row: added, changed, removed, enabled or disabled, with the rule as it was and as it
is. The first observation of a firewall only records the starting point (no flood of "added" events).
Origin: a change is ``opnexus`` when the audit trail has a deployment event for that rule UUID on that host in
the time since the previous observation (the actor is taken from it), else ``external`` (changed in the
firewall's own GUI, by a script, ...). The audit entry is written a moment after the change reaches the firewall,
so an event classified ``external`` is re-checked in the next cycles (``reconcile_origins``).
Two processes may run the loop at once (overlapping web updates, a replica): every change is claimed with an
optimistic ``version`` on the state row and the event carries that version in a unique key, so a change is
recorded exactly once.
Only the rows' own fields are kept; derived display fields (``%…``, ``alias_meta_*``, colours, priorities) are
dropped so that a changed alias or colour does not look like a rule change.
"""
import hashlib
import json
import logging
import uuid as uuid_lib
from datetime import datetime, timedelta, timezone
from sqlalchemy import delete, select, update
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.audit import AuditEvent
from app.models.host import OpnsenseHost
from app.models.policy import RuleRevision
from app.models.rule_deployment import RuleDeployment
from app.models.rule_history import LiveRuleState, RuleChangeEvent
from app.services import opnsense_client as client
from app.services import rule_hit_service
from app.services.opnsense_client import OpnsenseAPIError
logger = logging.getLogger(__name__)
BASELINE_KEY = "__baseline__"
RETENTION_DAYS = 365
# Audit kinds whose events carry the UUID of a rule on a firewall in details["remote_uuid"]. "rule_cleanup": the
# cleanup assistant (rule_cleanup_service) switching a hand-made rule off/on, deleting or restoring it.
AUDIT_KINDS = ("rule", "policy_hierarchy", "rule_cleanup")
# Audit statuses that mean the firewall's rule was NOT changed by that event (so it cannot explain a change).
# "confirmed": the cleanup's confirmation only re-reads the firewall.
AUDIT_STATUSES_NO_CHANGE = ("requested", "interrupted", "error", "confirmed")
# The audit entry is written right after the firewall changed; allow for the observer running just before it.
AUDIT_SLACK = timedelta(minutes=2)
RECHECK_WINDOW = timedelta(minutes=45)
_DROP_PREFIXES = ("%", "alias_meta", "#")
_DROP_KEYS = {"category_colors", "categories_colors", "icons", "ref"}
def clean_row(row: dict) -> dict:
"""The part of a ``searchRule`` row that describes the rule itself."""
return {k: v for k, v in row.items() if not k.startswith(_DROP_PREFIXES) and k not in _DROP_KEYS}
def fingerprint(row: dict) -> str:
return hashlib.sha256(json.dumps(row, sort_keys=True, separators=(",", ":"), default=str).encode()).hexdigest()
def _enabled(row: dict) -> bool:
return str(row.get("enabled", "1")).strip().lower() in ("1", "true", "yes", "on")
# Fields that only say where the rule sits in the order. Another rule being inserted or a group being re-sorted
# changes them for many rules at once; that is a "moved", not an edit of the rule.
ORDER_FIELDS = frozenset({"sort_order", "prio_group", "sequence"})
# Kinds after which what OPNexus learned about the rule's traffic no longer describes it.
RESTARTS_USAGE = frozenset({"added", "changed", "enabled", "disabled"})
def change_kind(before: dict | None, after: dict | None) -> str:
if before is None:
return "added"
if after is None:
return "removed"
differing = {k for k in {*before, *after} if before.get(k) != after.get(k)}
if differing == {"enabled"}:
return "enabled" if _enabled(after) else "disabled"
if differing and differing <= ORDER_FIELDS:
return "moved"
return "changed"
def diff_fields(before: dict | None, after: dict | None, limit: int = 12) -> list[dict]:
"""Short field-by-field difference: [{field, before, after}] (a missing side is None)."""
before, after = before or {}, after or {}
changes = [{"field": key, "before": before.get(key), "after": after.get(key)}
for key in sorted({*before, *after}) if before.get(key) != after.get(key)]
return changes[:limit]
def _tracked(search: dict) -> dict[str, dict] | None:
"""{rule label: cleaned row} of the user rules; None when the answer is not a rule list at all (an error
page must never look like 'every rule was removed')."""
rows = search.get("rows") if isinstance(search, dict) else None
if not isinstance(rows, list):
return None
result: dict[str, dict] = {}
for row in rows:
if not isinstance(row, dict) or not row.get("uuid") or row.get("is_automatic") in (True, 1, "1"):
continue
result[str(row["uuid"]).lower()[:64]] = clean_row(row)
return result
async def _find_audit(db: AsyncSession, host_id, rule_uuid: str, start: datetime, end: datetime) -> AuditEvent | None:
"""The newest deployment audit entry of this rule after ``start`` that no other event has claimed yet. The
audit entry is written after the firewall changed, so it can only lie after the previous observation."""
claimed = select(RuleChangeEvent.audit_id).where(RuleChangeEvent.audit_id.is_not(None))
return await db.scalar(
select(AuditEvent).where(
AuditEvent.host_id == str(host_id), AuditEvent.kind.in_(AUDIT_KINDS),
AuditEvent.created_at >= start, AuditEvent.created_at <= end + AUDIT_SLACK,
AuditEvent.id.notin_(claimed),
AuditEvent.status.notin_(AUDIT_STATUSES_NO_CHANGE),
AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
).order_by(AuditEvent.created_at.desc()).limit(1))
async def observe_rows(db: AsyncSession, host: OpnsenseHost, search: dict, now: datetime | None = None) -> dict:
"""Compare a ``searchRule`` answer with the stored state of the host. Returns {"baseline": bool, "events": n}."""
now = now or datetime.now(timezone.utc)
current = _tracked(search)
if current is None or not rule_hit_service.looks_complete(search):
# Not a full rule list (no automatic rules in it): never read an empty or partial answer as "all removed".
return {"baseline": False, "events": 0, "skipped": True, "touched": []}
states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
baseline = states.get(BASELINE_KEY)
since = baseline.last_seen_at if baseline else now
events = 0
touched: list[str] = []
if baseline is not None:
known = [u for u, st in states.items() if u != BASELINE_KEY and st.present]
missing = sorted(u for u in known if u not in current)
if known and len(missing) * 2 > len(known):
# More than half of the known rules vanished at once: possible, but far more often a broken answer.
# Only the same picture on two observations in a row counts.
marker = hashlib.sha256("\n".join(missing).encode()).hexdigest()
if (baseline.row or {}).get("pending") != marker:
await db.execute(update(LiveRuleState).where(
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY)
.values(row={"pending": marker}).execution_options(synchronize_session=False))
await db.commit()
return {"baseline": False, "events": 0, "skipped": True, "touched": [], "unconfirmed": len(missing)}
async def claim(rule_uuid: str, state: LiveRuleState, new_row: dict | None) -> int | None:
"""Move the state to its new version; None when another process already did."""
values = {"version": LiveRuleState.version + 1, "changed_at": now, "last_seen_at": now,
"present": new_row is not None}
if new_row is not None:
values.update(fingerprint=fingerprint(new_row), row=new_row)
else:
values.update(fingerprint="")
return await db.scalar(update(LiveRuleState).where(
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == rule_uuid, LiveRuleState.version == state.version,
).values(**values).returning(LiveRuleState.version).execution_options(synchronize_session=False))
async def record(rule_uuid: str, seq: int, before: dict | None, after: dict | None) -> None:
nonlocal events
kind = change_kind(before, after)
if kind in RESTARTS_USAGE:
touched.append(rule_uuid)
audit = await _find_audit(db, host.id, rule_uuid, since, now)
stmt = pg_insert(RuleChangeEvent).values(
id=uuid_lib.uuid4(), host_id=host.id, rule_uuid=rule_uuid, seq=seq, observed_at=now,
kind=kind, before=before, after=after,
origin="opnexus" if audit else "external", actor=audit.actor if audit else None,
audit_id=audit.id if audit else None, window_start=since,
).on_conflict_do_nothing(constraint="uq_rule_change_event")
events += 1 if (await db.execute(stmt)).rowcount else 0
for rule_uuid, row in current.items():
state = states.get(rule_uuid)
if state is None:
stmt = pg_insert(LiveRuleState).values(
host_id=host.id, rule_uuid=rule_uuid, fingerprint=fingerprint(row), row=row, present=True, version=1,
first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing()
inserted = (await db.execute(stmt)).rowcount
if inserted and baseline is not None:
await record(rule_uuid, 1, None, row)
elif not state.present:
seq = await claim(rule_uuid, state, row)
if seq is not None:
await record(rule_uuid, seq, None, row)
elif state.fingerprint != fingerprint(row):
seq = await claim(rule_uuid, state, row)
if seq is not None:
await record(rule_uuid, seq, state.row, row)
if baseline is not None:
for rule_uuid, state in states.items():
if rule_uuid == BASELINE_KEY or rule_uuid in current or not state.present:
continue
seq = await claim(rule_uuid, state, None)
if seq is not None:
await record(rule_uuid, seq, state.row, None)
unchanged = [u for u in current if u in states and states[u].present]
if unchanged:
await db.execute(update(LiveRuleState).where(
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid.in_(unchanged)).values(last_seen_at=now)
.execution_options(synchronize_session=False))
if baseline is None:
await db.execute(pg_insert(LiveRuleState).values(
host_id=host.id, rule_uuid=BASELINE_KEY, fingerprint="", row={}, present=False, version=1,
first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing())
else:
await db.execute(update(LiveRuleState).where(
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY).values(last_seen_at=now, row={})
.execution_options(synchronize_session=False))
await db.commit()
return {"baseline": baseline is None, "events": events, "touched": touched}
async def reconcile_origins(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
"""Recent 'external' events may only look so because the audit entry was written after the observation."""
now = now or datetime.now(timezone.utc)
recent = (await db.scalars(select(RuleChangeEvent).where(
RuleChangeEvent.host_id == host.id, RuleChangeEvent.origin == "external",
RuleChangeEvent.observed_at >= now - RECHECK_WINDOW).order_by(RuleChangeEvent.observed_at, RuleChangeEvent.seq))).all()
fixed = 0
for event in recent: # oldest first: an audit entry belongs to the earliest change it can explain
start = event.window_start or (event.observed_at - RECHECK_WINDOW)
audit = await _find_audit(db, host.id, event.rule_uuid, start, event.observed_at)
if audit is not None:
event.origin, event.actor, event.audit_id = "opnexus", audit.actor, audit.id
# (autoflush makes the claim visible to the next event's query)
fixed += 1
if fixed:
await db.commit()
return fixed
async def observe_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> dict:
"""Two GETs: the rules and their counters. One read serves the usage sampling and the history."""
# First settle the origin of earlier events (their audit entries may have arrived late), so that an entry is
# not taken by a newer change of the same rule that happens to fall in its window.
await reconcile_origins(db, host, now)
search = await client.search_rules(host, all_rows=True)
try:
counters = await rule_hit_service.read_counters(host)
except OpnsenseAPIError as exc:
counters = None
logger.info("Regelzähler von %s nicht lesbar: %s", host.name, exc)
result = await observe_rows(db, host, search, now)
if not result.get("skipped"): # a rejected or unconfirmed answer must not touch the usage data either
if counters is not None:
await rule_hit_service.apply_sample(db, host, counters, search, now)
await rule_hit_service.restart_window(db, host.id, result["touched"], now)
return result
async def observe_all(db: AsyncSession, now: datetime | None = None) -> dict:
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all()
summary = {"hosts": 0, "failed": 0, "events": 0}
for host in hosts:
try:
result = await observe_host(db, host, now)
summary["hosts"] += 1
summary["events"] += result["events"]
except OpnsenseAPIError as exc:
summary["failed"] += 1
logger.warning("Regelverlauf von %s nicht lesbar: %s", host.name, exc)
except Exception: # noqa: BLE001 - one host must never stop the cycle for the others
summary["failed"] += 1
await db.rollback()
logger.exception("Regelverlauf von %s fehlgeschlagen", host.name)
return summary
async def prune(db: AsyncSession, retention_days: int = RETENTION_DAYS) -> int:
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
events = await db.execute(delete(RuleChangeEvent).where(RuleChangeEvent.observed_at < cutoff))
gone = await db.execute(delete(LiveRuleState).where(
LiveRuleState.present.is_(False), LiveRuleState.rule_uuid != BASELINE_KEY, LiveRuleState.changed_at < cutoff))
# Finished cleanup operations (with the snapshots of deleted rules) are kept as long as the history.
from app.services.rule_cleanup_service import prune_operations
cleanups = await prune_operations(db, cutoff)
await db.commit()
return (events.rowcount or 0) + (gone.rowcount or 0) + cleanups
# ── Reading the history of one rule ───────────────────────────────────────
def _iso(value: datetime | None) -> str | None:
return value.isoformat() if value else None
async def rule_timeline(db: AsyncSession, host: OpnsenseHost, rule_uuid: str, limit: int = 200) -> list[dict]:
"""Observed changes, saved revisions (OPNexus-managed rules) and audit entries of one rule, newest first."""
rule_uuid = rule_uuid.lower()[:64]
items: list[dict] = []
for event in (await db.scalars(select(RuleChangeEvent).where(
RuleChangeEvent.host_id == host.id, RuleChangeEvent.rule_uuid == rule_uuid)
.order_by(RuleChangeEvent.observed_at.desc()).limit(limit))).all():
items.append({
"source": "observed", "at": event.observed_at, "kind": event.kind, "origin": event.origin, "actor": event.actor,
"changes": diff_fields(event.before, event.after), "audit_id": event.audit_id,
})
audits = (await db.scalars(select(AuditEvent).where(
AuditEvent.host_id == str(host.id), AuditEvent.kind.in_(AUDIT_KINDS),
AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
AuditEvent.status.notin_(("requested",))).order_by(AuditEvent.created_at.desc()).limit(limit))).all()
explained = {item["audit_id"] for item in items if item["audit_id"]} # already shown as the observed change
for audit in audits:
if audit.id in explained:
continue
items.append({
"source": "audit", "at": audit.created_at, "kind": audit.action, "status": audit.status, "actor": audit.actor,
"origin": "opnexus", "message": audit.message, "audit_id": audit.id, "changes": [],
})
deployment = await db.scalar(select(RuleDeployment).where(
RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid == rule_uuid))
if deployment is not None:
revisions = (await db.scalars(select(RuleRevision).where(RuleRevision.rule_id == deployment.rule_id)
.order_by(RuleRevision.created_at.desc()).limit(limit))).all()
ordered = sorted(revisions, key=lambda r: r.created_at)
previous = None
for revision in ordered:
items.append({
"source": "revision", "at": revision.created_at, "kind": "revision", "origin": "opnexus", "actor": None,
"note": revision.note, "changes": diff_fields(previous, revision.snapshot) if previous is not None else [],
"audit_id": None, "managed_rule_id": deployment.rule_id,
})
previous = revision.snapshot
items.sort(key=lambda item: item["at"], reverse=True)
return items[:limit]
+125 -14
View File
@@ -1,23 +1,31 @@
"""Rule usage from pf's counters: which rules never match anything?
``sample_host`` reads the per-rule packet counters (``diagnostics/firewall/pfStatistics/rules``) and the
configured rules (``searchRule``) and updates one row per rule: the time of the last observed hit.
``sample_host`` reads the per-rule packet counters and the configured rules (``searchRule``) and updates one
row per rule: the time of the last observed hit. The counters come from ``firewall/filter_util/ruleStats``
(already summed per label by the firewall; verified live on OPNsense 26.7 on all three test firewalls, see
the evidence block in opnsense_client.py, where its numbers equal the sum over ``pfStatistics/rules``). Older
releases without that endpoint fall back to parsing ``diagnostics/firewall/pfStatistics/rules``.
* A counter that grew since the last sample is a hit "now" (at most one sampling interval late).
* A counter that is *lower* than before means pf reloaded its ruleset (an apply or a reboot), which
zeroes everything. Whatever it shows above zero was hit since the reload, so that also counts as a hit now;
zero tells nothing, the old ``last_hit_at`` stays.
zero tells nothing, the old ``last_hit_at`` stays. (Live, 2026-10-22, OPNsense 26.7.7: ``firewall/filter/apply``
WITHOUT any pending change already zeroes every ``ruleStats`` counter - 16228 -> 0 packets on the busiest rule -
so every apply is such a reload; writers sample right before their own apply, see rule_cleanup_service.)
* A rule is called unused only after OPNexus has watched it for ``UNUSED_AFTER_DAYS`` without a hit.
A new rule, or a brand-new installation, is "still collecting data" instead of a false alarm.
Only rules with a UUID label are tied to a configured rule (rules created in the rule editor or API; OPNsense's
built-in automatic rules carry hash labels and are not user rules).
A counter belongs to a configured rule by its label: the UUID of a rule created in the rule editor or API, or the
32-hex label of a not yet migrated legacy rule (the ``searchRule`` row carries the same string as ``uuid``).
OPNsense's built-in automatic rules also have hash labels but are flagged ``is_automatic`` there, so they are
never tracked.
"""
import logging
import re
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from sqlalchemy import delete, select
from sqlalchemy import delete, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.host import OpnsenseHost
@@ -28,7 +36,12 @@ from app.services.opnsense_client import OpnsenseAPIError
logger = logging.getLogger(__name__)
UNUSED_AFTER_DAYS = 30
_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})"')
_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}|[0-9a-fA-F]{32})"')
# A sample counts as fresh for the "unused" confidence when it is at most this old.
SAMPLE_FRESH = timedelta(hours=24)
# A pf reload zeroes the counters; whatever was hit between the last sample and the reload is lost. After a gap
# this long the loss is too big to call the rule unused on that evidence: the observation window starts over.
SAMPLE_GAP = timedelta(hours=2)
def parse_counters(stats: dict) -> dict[str, dict]:
@@ -45,18 +58,65 @@ def parse_counters(stats: dict) -> dict[str, dict]:
return result
def parse_rule_stats(data: dict) -> dict[str, dict] | None:
"""{label: {packets, bytes, evaluations}} from ``filter_util/ruleStats``; None when the answer is not
that shape (older release, error page), so the caller can fall back to pfStatistics."""
stats = (data or {}).get("stats") if isinstance(data, dict) else None
if not isinstance(stats, dict) or (data or {}).get("status") != "ok":
return None
result: dict[str, dict] = {}
for label, values in stats.items():
if isinstance(values, dict):
result[str(label).lower()] = {key: int(values.get(key) or 0) for key in ("packets", "bytes", "evaluations")}
return result
async def read_counters(host: OpnsenseHost) -> dict[str, dict]:
"""Counters per rule label: ruleStats first, the pf ruleset text as fallback."""
try:
parsed = parse_rule_stats(await client.get_rule_stats(host))
except OpnsenseAPIError:
parsed = None
if parsed is not None:
return parsed
return parse_counters(await client.get_pf_rule_statistics(host))
def looks_complete(search) -> bool:
"""A usable ``searchRule`` answer: a list that contains OPNsense's own automatic rules. A real firewall always
has some, so an empty or partial answer (a firewall just restarting a service, a proxy error page) is rejected
instead of being read as "every rule is gone"."""
rows = search.get("rows") if isinstance(search, dict) else None
if not isinstance(rows, list) or not rows:
return False
dicts = [r for r in rows if isinstance(r, dict)]
if not any("is_automatic" in r for r in dicts):
return bool(dicts) # firmware that does not flag automatic rules at all: a non-empty list is all we can ask for
return any(r.get("is_automatic") in (True, 1, "1") for r in dicts)
def configured_rules(search: dict) -> dict[str, dict]:
rows = (search or {}).get("rows") or []
return {
str(r["uuid"]).lower(): {"description": r.get("description") or "", "enabled": r.get("enabled") in ("1", 1, True)}
for r in rows if r.get("uuid") and not r.get("is_automatic")
for r in rows if r.get("uuid") and r.get("is_automatic") not in (True, 1, "1")
}
async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
"""Read the counters and the rules from the firewall and update the hit rows."""
counters = await read_counters(host)
return await apply_sample(db, host, counters, await client.search_rules(host, all_rows=True), now)
async def apply_sample(db: AsyncSession, host: OpnsenseHost, counters: dict[str, dict], search: dict,
now: datetime | None = None) -> int:
"""Update the hit rows from counters (per label) and a ``searchRule`` answer that are already read
(the observe loop shares one read between usage and change history)."""
now = now or datetime.now(timezone.utc)
counters = parse_counters(await client.get_pf_rule_statistics(host))
rules = configured_rules(await client.search_rules(host, all_rows=True))
if not looks_complete(search):
return 0 # not a full rule list: an error or partial answer must not look like "every rule was deleted"
rules = configured_rules(search)
rows = {r.rule_uuid: r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host.id))).all()}
for uuid_, meta in rules.items():
counter = counters.get(uuid_, {"packets": 0, "bytes": 0})
@@ -68,6 +128,10 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
row.last_hit_at = now # already matching when first seen
elif counter["packets"] > row.last_packets or (counter["packets"] < row.last_packets and counter["packets"] > 0):
row.last_hit_at = now
elif counter["packets"] < row.last_packets and now - row.last_sampled_at > SAMPLE_GAP:
row.first_seen_at = now # reloaded after a long gap: the hits in between are unknown
if meta["enabled"] and not row.enabled:
row.first_seen_at = now # a rule that was off has not been "waiting for traffic" while it was off
row.description = meta["description"][:255]
row.enabled = meta["enabled"]
row.last_packets, row.last_bytes, row.last_sampled_at = counter["packets"], counter["bytes"], now
@@ -78,6 +142,18 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
return len(rules)
async def restart_window(db: AsyncSession, host_id, rule_uuids, now: datetime | None = None) -> int:
"""The rules were edited, enabled or created: what OPNexus saw before does not describe them any more, so the
observation window for "unused" starts over (an earlier last hit is kept)."""
keys = [u.lower() for u in rule_uuids]
if not keys:
return 0
result = await db.execute(update(RuleHit).where(RuleHit.host_id == host_id, RuleHit.rule_uuid.in_(keys))
.values(first_seen_at=now or datetime.now(timezone.utc)).execution_options(synchronize_session=False))
await db.commit()
return result.rowcount or 0
async def sample_all(db: AsyncSession) -> int:
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)))).all()
sampled = 0
@@ -92,14 +168,49 @@ async def sample_all(db: AsyncSession) -> int:
return sampled
def classify(row: RuleHit, now: datetime) -> str:
"""'used' | 'unused' | 'collecting' | 'disabled'"""
@dataclass
class ClusterHit:
"""The usage of one rule over all nodes of a cluster (DESIGN-1.31 §3.4). Has the attributes ``classify`` and
``cleanup.unused_confidence`` read from a ``RuleHit``. Traffic goes through the CARP master, a backup sees
almost nothing: a rule is only unused when it is unused everywhere, so the newest hit of any node counts, the
window starts at the latest start of any node and the counters are only as fresh as the oldest node's."""
rule_uuid: str
enabled: bool
first_seen_at: datetime
last_hit_at: datetime | None
last_sampled_at: datetime
# The rule is not tracked on some node (missing there, or the node was never sampled): nothing is known
# about its traffic there, so "unused" is never better than "probable".
peer_missing: bool = False
def merge_cluster_hits(own: dict[str, RuleHit], peers: list[dict[str, RuleHit]]) -> dict[str, ClusterHit]:
"""Per rule over the union of all nodes' rows (a rule this node has no row for is still judged, as
``peer_missing``: what this node saw of it is unknown)."""
merged: dict[str, ClusterHit] = {}
for key in {*own, *(k for p in peers for k in p)}:
rows = [own.get(key), *(p.get(key) for p in peers)]
present = [r for r in rows if r is not None]
hits = [r.last_hit_at for r in present if r.last_hit_at is not None]
mine = own.get(key) or present[0]
merged[key] = ClusterHit(
rule_uuid=mine.rule_uuid, enabled=mine.enabled,
first_seen_at=max(r.first_seen_at for r in present),
last_hit_at=max(hits) if hits else None,
last_sampled_at=min(r.last_sampled_at for r in present),
peer_missing=len(present) < len(rows),
)
return merged
def classify(row: RuleHit, now: datetime, days: int = UNUSED_AFTER_DAYS) -> str:
"""'used' | 'unused' | 'collecting' | 'disabled'; ``days`` is the observation window."""
if not row.enabled:
return "disabled"
if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=UNUSED_AFTER_DAYS):
if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=days):
return "used"
watched = now - row.first_seen_at
if watched < timedelta(days=UNUSED_AFTER_DAYS):
if watched < timedelta(days=days):
return "used" if row.last_hit_at is not None else "collecting"
return "unused"
@@ -374,6 +374,9 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: WireguardTunnel) -> None:
dependency direction; deleting the Peer first also auto-unlinks it
from the Server's `peers` list server-side, per delClientAction's own
behavior — see module docstring), then reconfigures."""
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, WireguardTunnelDeployment, WireguardTunnelDeployment.tunnel_id == tunnel.id)
failures = []
result = await db.execute(select(WireguardTunnelDeployment).where(WireguardTunnelDeployment.tunnel_id == tunnel.id))
for deployment in result.scalars().all():
+2 -2
View File
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
Versioning scheme: see CHANGELOG.md ("Versionierung").
"""
APP_VERSION = "1.29.0"
APP_CODENAME = "Caracara"
APP_VERSION = "1.31.0"
APP_CODENAME = "Vulture"
@@ -0,0 +1,62 @@
"""Änderungsverlauf der Regeln auf den Firewalls (1.30): letzter beobachteter Stand und Änderungsereignisse.
Revision ID: 0027
Revises: 0026
Create Date: 2026-10-22
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0027"
down_revision = "0026"
branch_labels = None
depends_on = None
UUID = postgresql.UUID(as_uuid=True)
TS = sa.DateTime(timezone=True)
def upgrade() -> None:
# A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("live_rule_states"):
op.create_table(
"live_rule_states",
sa.Column("host_id", UUID, nullable=False),
sa.Column("rule_uuid", sa.String(64), nullable=False),
sa.Column("fingerprint", sa.String(64), nullable=False),
sa.Column("row", sa.JSON(), nullable=False),
sa.Column("present", sa.Boolean(), nullable=False),
sa.Column("version", sa.Integer(), nullable=False),
sa.Column("first_seen_at", TS, nullable=False),
sa.Column("changed_at", TS, nullable=False),
sa.Column("last_seen_at", TS, nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("host_id", "rule_uuid"),
)
if not inspector.has_table("rule_change_events"):
op.create_table(
"rule_change_events",
sa.Column("id", UUID, nullable=False),
sa.Column("host_id", UUID, nullable=False),
sa.Column("rule_uuid", sa.String(64), nullable=False),
sa.Column("seq", sa.Integer(), nullable=False),
sa.Column("observed_at", TS, nullable=False),
sa.Column("kind", sa.String(16), nullable=False),
sa.Column("before", sa.JSON(), nullable=True),
sa.Column("after", sa.JSON(), nullable=True),
sa.Column("origin", sa.String(16), nullable=False),
sa.Column("actor", sa.String(150), nullable=True),
sa.Column("audit_id", UUID, nullable=True),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
)
op.create_index("ix_rule_change_events_observed_at", "rule_change_events", ["observed_at"])
op.create_index("ix_rule_change_events_rule", "rule_change_events", ["host_id", "rule_uuid", "observed_at"])
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS rule_change_events")
op.execute("DROP TABLE IF EXISTS live_rule_states")
@@ -0,0 +1,21 @@
"""Änderungsverlauf: Fensteranfang je Ereignis für die Herkunftsbestimmung (1.30).
Revision ID: 0028
Revises: 0027
Create Date: 2026-10-22
"""
from alembic import op
revision = "0028"
down_revision = "0027"
branch_labels = None
depends_on = None
def upgrade() -> None:
# IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have the column.
op.execute("ALTER TABLE rule_change_events ADD COLUMN IF NOT EXISTS window_start timestamptz")
def downgrade() -> None:
op.execute("ALTER TABLE rule_change_events DROP COLUMN IF EXISTS window_start")
@@ -0,0 +1,103 @@
"""Regel-Bereinigung (1.31, B4 Teil 2): Operationen, Einträge, Einstellungen.
Revision ID: 0029
Revises: 0028
Create Date: 2026-10-22
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0029"
down_revision = "0028"
branch_labels = None
depends_on = None
UUID = postgresql.UUID(as_uuid=True)
TS = sa.DateTime(timezone=True)
# Must equal OPEN_ITEM_STATES in app/models/rule_cleanup.py (the schema comparison of the tests notices a drift).
_OPEN_STATES = ("queued", "disabling", "disabled", "reverting", "deleting", "restoring", "unknown", "revert_failed")
_SETTINGS = [
("cleanup_max_rules", "integer NOT NULL DEFAULT 10"),
("cleanup_delete_min_days", "integer NOT NULL DEFAULT 7"),
("cleanup_management_sources", "json NOT NULL DEFAULT '[]'::json"),
("cleanup_management_ports", "json NOT NULL DEFAULT '[]'::json"),
]
def upgrade() -> None:
# A pre-Alembic install is bridged with create_all() first (app/migrate.py), which may already have made all of it.
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("rule_cleanup_operations"):
op.create_table(
"rule_cleanup_operations",
sa.Column("id", UUID, nullable=False),
sa.Column("host_id", UUID, nullable=False),
sa.Column("cluster_name", sa.String(120), nullable=True),
sa.Column("kind", sa.String(16), nullable=False),
sa.Column("status", sa.String(20), nullable=False),
sa.Column("actor", sa.String(150), nullable=False),
sa.Column("actor_id", UUID, nullable=True),
sa.Column("created_at", TS, nullable=False),
sa.Column("expires_at", TS, nullable=False),
sa.Column("started_at", TS, nullable=True),
sa.Column("heartbeat_at", TS, nullable=True),
sa.Column("revert_deadline", TS, nullable=True),
sa.Column("confirmed_at", TS, nullable=True),
sa.Column("finished_at", TS, nullable=True),
sa.Column("message", sa.String(1000), nullable=True),
sa.Column("preview_fingerprint", sa.String(64), nullable=False),
sa.Column("options", sa.JSON(), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_rule_cleanup_operations_host", "rule_cleanup_operations", ["host_id", "created_at"])
op.create_index("ix_rule_cleanup_operations_status", "rule_cleanup_operations", ["status"])
if not inspector.has_table("rule_cleanup_items"):
op.create_table(
"rule_cleanup_items",
sa.Column("id", UUID, nullable=False),
sa.Column("operation_id", UUID, nullable=False),
sa.Column("host_id", UUID, nullable=False),
sa.Column("rule_uuid", sa.String(64), nullable=False),
sa.Column("seq", sa.Integer(), nullable=False),
sa.Column("carp_role", sa.String(16), nullable=True),
sa.Column("state", sa.String(16), nullable=False),
sa.Column("description", sa.Text(), nullable=False),
sa.Column("categories", sa.JSON(), nullable=False),
sa.Column("confidence", sa.String(16), nullable=True),
sa.Column("row_before", sa.JSON(), nullable=True),
sa.Column("fp_before", sa.String(64), nullable=True),
sa.Column("cover_uuid", sa.String(64), nullable=True),
sa.Column("cover_fp", sa.String(64), nullable=True),
sa.Column("snapshot", sa.JSON(), nullable=True),
sa.Column("anchor_next_uuid", sa.String(64), nullable=True),
sa.Column("anchor_prev_uuid", sa.String(64), nullable=True),
sa.Column("prio_group", sa.String(32), nullable=True),
sa.Column("source_item_id", UUID, nullable=True),
sa.Column("restored_uuid", sa.String(64), nullable=True),
sa.Column("changed_at", TS, nullable=True),
sa.Column("message", sa.String(1000), nullable=True),
sa.ForeignKeyConstraint(["operation_id"], ["rule_cleanup_operations.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_rule_cleanup_items_operation", "rule_cleanup_items", ["operation_id"])
op.create_index(
"uq_cleanup_rule_open", "rule_cleanup_items", ["host_id", "rule_uuid"], unique=True,
postgresql_where=sa.text("state IN (" + ", ".join(f"'{s}'" for s in _OPEN_STATES) + ")"),
)
for name, ddl in _SETTINGS:
op.execute(f"ALTER TABLE governance_settings ADD COLUMN IF NOT EXISTS {name} {ddl}")
# The default only fills the existing row; the model supplies it for new ones (and the schema comparison
# cannot compare a JSON server default).
op.execute(f"ALTER TABLE governance_settings ALTER COLUMN {name} DROP DEFAULT")
def downgrade() -> None:
for name, _ddl in reversed(_SETTINGS):
op.execute(f"ALTER TABLE governance_settings DROP COLUMN IF EXISTS {name}")
op.execute("DROP TABLE IF EXISTS rule_cleanup_items")
op.execute("DROP TABLE IF EXISTS rule_cleanup_operations")
+116
View File
@@ -0,0 +1,116 @@
{
"_comment": "Anonymized from live OPNsense 26.7 test firewalls (2026-10-22): firewall/filter_util/ruleStats (3 automatic labels, 3 MVC rules, 1 unmigrated legacy rule with a 32-hex label; counters as read) and the matching searchRule row of the legacy rule. No addresses involved.",
"ruleStats": {
"status": "ok",
"stats": {
"02f4bab031b57d1e30553ce08e0ec131": {
"pf_rules": 2,
"evaluations": 17734,
"packets": 250,
"bytes": 28670,
"states": 0
},
"7355b5da99f40893221b6c4140de1006": {
"pf_rules": 6,
"evaluations": 860,
"packets": 0,
"bytes": 0,
"states": 0
},
"80f90f0d4c5aba0c28d1c539e1e35766": {
"pf_rules": 12,
"evaluations": 1219,
"packets": 0,
"bytes": 0,
"states": 0
},
"fccf1b9d-2461-43d6-938f-2c3c39b3ddee": {
"pf_rules": 1,
"evaluations": 171,
"packets": 458,
"bytes": 35302,
"states": 0
},
"920a850f-68c3-42f6-a5a4-5d06984e417b": {
"pf_rules": 2,
"evaluations": 8,
"packets": 4,
"bytes": 756,
"states": 0
},
"2ec1c6db-61e1-4476-b3f6-0599a28990a2": {
"pf_rules": 1,
"evaluations": 9625,
"packets": 234816,
"bytes": 99582131,
"states": 40
},
"db610dbc7c6467261cf08e8a72ade096": {
"pf_rules": 1,
"evaluations": 4712,
"packets": 110719,
"bytes": 55995167,
"states": 24
}
}
},
"legacy_row": {
"@attributes": {
"uuid": "a00d53d2-8107-4d87-a71d-53ac85c1e9da"
},
"interface": "WAN",
"ipprotocol": "inet",
"statetype": "keep state",
"direction": "in",
"quick": "1",
"updated": {
"username": "[email protected]",
"time": "1791536720.85",
"description": "/firewall_rules_edit.php made changes"
},
"created": {
"username": "[email protected]",
"time": "1791536720.85",
"description": "/firewall_rules_edit.php made changes"
},
"seq": 0,
"#priority": 400000,
"enabled": "1",
"action": "pass",
"replyto": "",
"description": "ALLOW ANY",
"source_net": "any",
"source_port": "",
"destination_net": "any",
"destination_port": "",
"ref": "ui/firewall/migration",
"uuid": "db610dbc7c6467261cf08e8a72ade096",
"source_not": "0",
"destination_not": "0",
"%direction": "In",
"%action": "Pass",
"%ipprotocol": "IPv4",
"protocol": "",
"%protocol": "",
"sort_order": "400000.1000023",
"legacy": true,
"is_automatic": false,
"category_colors": [],
"alias_meta_source_net": [
{
"value": "any",
"%value": "any",
"isAlias": false,
"description": ""
}
],
"alias_meta_destination_net": [
{
"value": "any",
"%value": "any",
"isAlias": false,
"description": ""
}
]
}
}
+2 -1
View File
@@ -6,6 +6,7 @@ Each mutation: {"file": ..., "old": ..., "new": ..., "tests": "pytest selection"
The file is patched, the selected tests must FAIL, then the file is restored.
"""
import json
import shlex
import subprocess
import sys
from pathlib import Path
@@ -23,7 +24,7 @@ def main(spec_path: str) -> int:
continue
try:
path.write_text(original.replace(m["old"], m["new"]))
result = subprocess.run([sys.executable, "-m", "pytest", "-q", "-x", "-p", "no:cacheprovider", *m["tests"].split()],
result = subprocess.run([sys.executable, "-m", "pytest", "-q", "-x", "-p", "no:cacheprovider", *shlex.split(m["tests"])],
capture_output=True, text=True)
finally:
path.write_text(original)
+380
View File
@@ -0,0 +1,380 @@
[
{
"name": "fingerprint not compared at deploy",
"file": "app/services/rule_cleanup_service.py",
"old": " if plan_fingerprint(plan) != op.preview_fingerprint:",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_deploy_refuses_a_stale_preview_without_writing"
},
{
"name": "management UNKNOWN treated as FALSE (service)",
"file": "app/services/rule_cleanup_service.py",
"old": " if check[\"result\"] != \"false\":",
"new": " if check[\"result\"] == \"true\":",
"tests": "tests/test_rule_cleanup.py::test_rules_that_must_not_be_switched_off_are_refused"
},
{
"name": "management UNKNOWN treated as FALSE (protection)",
"file": "app/services/rule_analysis/protection.py",
"old": " return {\"result\": \"unknown\", \"queries\": queries, \"reasons\": sorted(reasons), \"match\": unknown_at}",
"new": " return {\"result\": \"false\", \"queries\": queries, \"reasons\": [], \"match\": None}",
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_rules_that_must_not_be_switched_off_are_refused"
},
{
"name": "network source: ranges ignored",
"file": "app/services/rule_analysis/protection.py",
"old": " return Outcome(any(v == net.version and a <= high and b >= low for v, a, b in ranges))",
"new": " return Outcome(False)",
"tests": "tests/test_rule_cleanup_protection.py"
},
{
"name": "CARP/pfsync protocol not excluded",
"file": "app/services/rule_analysis/protection.py",
"old": " if rule.protos is not None and rule.protos & {CARP_PROTO, PFSYNC_PROTO}:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_protection.py"
},
{
"name": "pfsync interface not checked",
"file": "app/services/rule_analysis/protection.py",
"old": " if pfsync_if == \"\" or (pfsync_if is None and not clustered):",
"new": " if True:",
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_cluster_pfsync_interface_rules_are_infrastructure"
},
{
"name": "no management sources not locked",
"file": "app/services/rule_cleanup_service.py",
"old": " if not sources:\n raise api_error(409, ErrorCode.BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN)\n clustered",
"new": " if False:\n raise api_error(409, ErrorCode.BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN)\n clustered",
"tests": "tests/test_rule_cleanup.py::test_without_management_sources_everything_is_locked"
},
{
"name": "unused sure below 30 days",
"file": "app/services/rule_cleanup_service.py",
"old": " if unused_days < MIN_SURE_UNUSED_DAYS:",
"new": " if unused_days < 7:",
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_short_window_and_external_change_cap_at_probable"
},
{
"name": "unused block rule not capped",
"file": "app/services/rule_cleanup_service.py",
"old": " if rule.action in (\"block\", \"reject\"):",
"new": " if rule.action in (\"reject\",):",
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_probable_needs_an_explicit_acknowledgement_per_rule"
},
{
"name": "external change not capped",
"file": "app/services/rule_cleanup_service.py",
"old": " if external and best is not None:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_short_window_and_external_change_cap_at_probable"
},
{
"name": "probable without acknowledgement",
"file": "app/services/rule_cleanup_service.py",
"old": " if level == \"probable\" and given not in acknowledge:",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_probable_needs_an_explicit_acknowledgement_per_rule"
},
{
"name": "unknown confidence selectable",
"file": "app/services/rule_cleanup_service.py",
"old": " if level == \"unknown\":\n refuse(\"confidence\")",
"new": " if False:\n refuse(\"confidence\")",
"tests": "tests/test_rule_cleanup.py::test_managed_legacy_and_unknown_confidence_are_refused"
},
{
"name": "cluster usage merge removed",
"file": "app/services/rule_analysis/service.py",
"old": " return rule_hit_service.merge_cluster_hits(own, [await _hits(db, other) for other in others]), own",
"new": " return own, own",
"tests": "tests/test_rule_cleanup.py::test_cluster_usage_is_judged_over_all_nodes"
},
{
"name": "write order master first",
"file": "app/services/rule_cleanup_service.py",
"old": " return sorted(nodes, key=lambda n: (n.role == \"MASTER\", n.host.name))",
"new": " return sorted(nodes, key=lambda n: (n.role != \"MASTER\", n.host.name))",
"tests": "tests/test_rule_cleanup.py::test_cluster_writes_backup_first_and_reverts_master_first"
},
{
"name": "revert order backup first",
"file": "app/services/rule_cleanup_service.py",
"old": " for item in sorted(items, key=lambda i: (i.carp_role != \"MASTER\", str(i.host_id))):",
"new": " for item in sorted(items, key=lambda i: (i.carp_role == \"MASTER\", str(i.host_id))):",
"tests": "tests/test_rule_cleanup.py::test_cluster_writes_backup_first_and_reverts_master_first"
},
{
"name": "no apply after a partial failure",
"file": "app/services/rule_cleanup_service.py",
"old": " if any(i.state in (\"disabled\", \"unknown\") for i in node_items):\n # Also after an error",
"new": " if not stop and any(i.state in (\"disabled\", \"unknown\") for i in node_items):\n # Also after an error",
"tests": "tests/test_rule_cleanup.py::test_refused_toggle_stops_writing_but_applies_what_is_switched"
},
{
"name": "write continues after a refusal",
"file": "app/services/rule_cleanup_service.py",
"old": " item.state, item.message = \"failed\", str(exc)[:1000]\n _audit(db, op, item, node.host, action=op.kind, status=\"error\", message=item.message)\n stop = True",
"new": " item.state, item.message = \"failed\", str(exc)[:1000]\n _audit(db, op, item, node.host, action=op.kind, status=\"error\", message=item.message)",
"tests": "tests/test_rule_cleanup.py::test_refused_toggle_stops_writing_but_applies_what_is_switched"
},
{
"name": "unclear toggle counted as done",
"file": "app/services/rule_cleanup_service.py",
"old": " item.state = \"disabled\" if seen is target else (\"failed\" if seen is not None else \"unknown\")",
"new": " item.state = \"disabled\"",
"tests": "tests/test_rule_cleanup.py::test_unclear_toggle_is_settled_by_reading"
},
{
"name": "confirm with unreachable firewall",
"file": "app/services/rule_cleanup_service.py",
"old": " except OpnsenseAPIError as exc:\n raise api_error(502, ErrorCode.BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR, host=member.name) from exc\n if not rule_hit_service.looks_complete(search):",
"new": " except OpnsenseAPIError as exc:\n continue\n if not rule_hit_service.looks_complete(search):",
"tests": "tests/test_rule_cleanup.py::test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different"
},
{
"name": "confirm ignores re-enabled rule",
"file": "app/services/rule_cleanup_service.py",
"old": " if row is None or _truthy(row.get(\"enabled\", \"1\")) is not target \\\n or _content_fp",
"new": " if row is None \\\n or _content_fp",
"tests": "tests/test_rule_cleanup.py::test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different"
},
{
"name": "confirm after the deadline",
"file": "app/services/rule_cleanup_service.py",
"old": " RuleCleanupOperation.revert_deadline > now).values(\n status=\"confirmed\"",
"new": " RuleCleanupOperation.revert_deadline > now - timedelta(days=1)).values(\n status=\"confirmed\"",
"tests": "tests/test_rule_cleanup.py::test_deadline_passing_while_confirm_reads_the_firewall_refuses tests/test_rule_cleanup.py::test_confirm_and_sweeper_at_the_deadline_decide_once"
},
{
"name": "sweeper does not revert expired",
"file": "app/services/rule_cleanup_service.py",
"old": " or_(and_(RuleCleanupOperation.status.in_(SWEEP_STATUSES), RuleCleanupOperation.revert_deadline <= now),",
"new": " or_(and_(RuleCleanupOperation.status.in_(SWEEP_STATUSES), RuleCleanupOperation.revert_deadline <= now - timedelta(days=1)),",
"tests": "tests/test_rule_cleanup.py::test_confirm_after_the_deadline_is_refused_and_the_sweeper_reverts tests/test_rule_cleanup.py::test_failed_apply_keeps_the_deadline_and_the_sweeper_reverts"
},
{
"name": "sweeper retries revert_failed",
"file": "app/services/rule_cleanup_service.py",
"old": "SWEEP_STATUSES = (\"awaiting_confirm\", \"partial\", \"apply_failed\")",
"new": "SWEEP_STATUSES = (\"awaiting_confirm\", \"partial\", \"apply_failed\", \"revert_failed\")",
"tests": "tests/test_rule_cleanup.py::test_revert_now_and_failed_revert_is_not_retried_automatically"
},
{
"name": "heartbeat not checked",
"file": "app/services/rule_cleanup_service.py",
"old": " if not _stale(op, len(items), nodes, now):\n # alive",
"new": " if False:\n # alive",
"tests": "tests/test_rule_cleanup.py::test_sweeper_skips_a_live_run_and_takes_over_an_abandoned_one"
},
{
"name": "delete before the waiting time",
"file": "app/services/rule_cleanup_service.py",
"old": " if now < due:",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change"
},
{
"name": "delete of a changed rule",
"file": "app/services/rule_cleanup_service.py",
"old": " if _content_fp(row, False) != _content_fp(item.row_before or {}, False):",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change"
},
{
"name": "snapshot not committed before delRule",
"file": "app/services/rule_cleanup_service.py",
"old": " item.state, item.changed_at = \"deleting\", _now()\n await _commit(db) # heartbeat + snapshot + intent, durable before delRule\n",
"new": " item.state, item.changed_at = \"deleting\", _now()\n",
"tests": "tests/test_rule_cleanup.py::test_delete_commits_the_snapshot_before_delrule"
},
{
"name": "unclear delete counted as deleted",
"file": "app/services/rule_cleanup_service.py",
"old": " return \"failed\" # still there",
"new": " return \"deleted\"",
"tests": "tests/test_rule_cleanup.py::test_unclear_delete_is_settled_with_getrule"
},
{
"name": "restore anchor ignored",
"file": "app/services/rule_cleanup_service.py",
"old": " await client.move_rule_before(member, new_uuid, anchor)",
"new": " pass",
"tests": "tests/test_rule_cleanup.py::test_restore_adds_disabled_at_the_anchor_with_a_new_uuid"
},
{
"name": "restore position not verified",
"file": "app/services/rule_cleanup_service.py",
"old": " return i > 0 and position[i - 1] == new_uuid.lower()",
"new": " return True",
"tests": "tests/test_rule_cleanup.py::test_restore_with_a_refused_move_is_verified_by_reading_and_only_warns"
},
{
"name": "R1 unassigned ingress not checked",
"file": "app/services/rule_analysis/protection.py",
"old": " return [(i, None) for i in _interfaces(ctx)] + [(UNASSIGNED, virtual)]",
"new": " return [(i, None) for i in _interfaces(ctx)]",
"tests": "tests/test_rule_cleanup_protection.py"
},
{
"name": "R1 builtin group members treated as known",
"file": "app/services/rule_analysis/protection.py",
"old": " elif name in BUILTIN_GROUP_DEVICES or ctx.groups is None or name not in ctx.groups:",
"new": " elif ctx.groups is None or name not in ctx.groups:",
"tests": "tests/test_rule_cleanup_protection.py"
},
{
"name": "R2 sweeper takes over a live run past its deadline",
"file": "app/services/rule_cleanup_service.py",
"old": " if not _stale(op, len(items), nodes, now):",
"new": " if not (op.revert_deadline is not None and op.revert_deadline <= now) and not _stale(op, len(items), nodes, now):",
"tests": "tests/test_rule_cleanup.py::test_sweeper_leaves_a_live_run_alone_even_past_its_deadline"
},
{
"name": "R2 heartbeat before a write not conditional",
"file": "app/services/rule_cleanup_service.py",
"old": " RuleCleanupOperation.id == op.id, RuleCleanupOperation.status == expect).values(heartbeat_at=now)",
"new": " RuleCleanupOperation.id == op.id).values(heartbeat_at=now)",
"tests": "tests/test_rule_cleanup.py::test_a_run_taken_over_by_another_backend_writes_nothing_more"
},
{
"name": "R3 extra management ports ignored",
"file": "app/services/rule_analysis/protection.py",
"old": " for port in (*MANAGEMENT_PORTS, *extra_ports):",
"new": " for port in MANAGEMENT_PORTS:",
"tests": "tests/test_rule_cleanup.py::test_extra_management_ports_protect_rules_on_them"
},
{
"name": "R3 management ports shown to non-admins",
"file": "app/routers/governance.py",
"old": " cleanup_management_ports=ports if user.role == \"admin\" else [],",
"new": " cleanup_management_ports=ports,",
"tests": "tests/test_rule_cleanup_api.py::test_management_ports_are_validated_normalised_audited_and_hidden"
},
{
"name": "R3 port range not validated",
"file": "app/routers/governance.py",
"old": " if isinstance(raw, bool) or not text.isdigit() or not 1 <= int(text) <= 65535:",
"new": " if not text.isdigit():",
"tests": "tests/test_rule_cleanup_api.py::test_management_ports_are_validated_normalised_audited_and_hidden"
},
{
"name": "R4 any host may act on any operation",
"file": "app/services/rule_cleanup_service.py",
"old": " if op.host_id != host.id and not (cluster and host.cluster_name and op.host_id in {h.id for h in await _fleet(db, host)}):",
"new": " if op.host_id != host.id and not cluster:",
"tests": "tests/test_rule_cleanup.py::test_cluster_peer_may_confirm_and_read_but_strangers_may_not"
},
{
"name": "R4 peers refused",
"file": "app/services/rule_cleanup_service.py",
"old": " if op.host_id != host.id and not (cluster and host.cluster_name and op.host_id in {h.id for h in await _fleet(db, host)}):",
"new": " if op.host_id != host.id:",
"tests": "tests/test_rule_cleanup.py::test_cluster_peer_may_confirm_and_read_but_strangers_may_not"
},
{
"name": "R5 half-deleted node not skipped",
"file": "app/services/rule_cleanup_service.py",
"old": " if clustered and kind == \"delete\" and await _already_deleted(db, node.host.id, u):",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
},
{
"name": "R5 failed delete counts as basis",
"file": "app/services/rule_cleanup_service.py",
"old": " if not (o.kind == \"delete\" and i.state in (\"failed\", \"skipped\"))]",
"new": " ]",
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
},
{
"name": "R5 nothing left still planned",
"file": "app/services/rule_cleanup_service.py",
"old": " if not any(i[\"given\"] == given for n in plan_nodes for i in n[\"items\"]):",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
},
{
"name": "R5 restore needs all siblings deleted",
"file": "app/services/rule_cleanup_service.py",
"old": " if s.state == \"deleted\" and not s.restored_uuid]",
"new": " ]\n if any(s.state != \"deleted\" or s.restored_uuid for s in siblings):\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=\"partial\")",
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_restored_where_it_was_deleted"
},
{
"name": "R7 enabling a block rule without acknowledgement",
"file": "app/services/rule_cleanup_service.py",
"old": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] != \"false\":",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless"
},
{
"name": "R7 unknown protection counted as harmless",
"file": "app/services/rule_cleanup_service.py",
"old": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] != \"false\":",
"new": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] == \"true\":",
"tests": "tests/test_rule_cleanup.py::test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless"
},
{
"name": "R8 router does not redact for non-admins",
"file": "app/routers/rule_cleanup.py",
"old": " if not _full_view(request):\n operations = [service.redact(op) for op in operations]",
"new": " if False:\n operations = [service.redact(op) for op in operations]",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R8 preview answer not redacted",
"file": "app/routers/rule_cleanup.py",
"old": " return result if _full_view(request) else service.redact(result)",
"new": " return result",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R8 audit admin_only not stripped",
"file": "app/routers/audit.py",
"old": " if not full_view(request):",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R8 read token of an admin sees everything",
"file": "app/routers/audit.py",
"old": " return getattr(user, \"role\", None) == \"admin\" and (token is None or token.scope == \"admin\")",
"new": " return getattr(user, \"role\", None) == \"admin\"",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R8 base_url kept in plan",
"file": "app/services/rule_cleanup_service.py",
"old": " out[\"nodes\"] = [{**{k: v for k, v in n.items() if k != \"base_url\"},",
"new": " out[\"nodes\"] = [{**{k: v for k, v in n.items()},",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R8 protection match kept",
"file": "app/services/rule_cleanup_service.py",
"old": " return {k: v for k, v in check.items() if k != \"match\"}",
"new": " return check",
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
},
{
"name": "R10 stored restored_uuid ignored",
"file": "app/services/rule_cleanup_service.py",
"old": " if item.restored_uuid:\n try:\n await client.get_rule(member, item.restored_uuid)",
"new": " if False:\n try:\n await client.get_rule(member, item.restored_uuid)",
"tests": "tests/test_rule_cleanup.py::test_unclear_restore_uses_the_stored_uuid_before_searching_by_content"
},
{
"name": "R10 twins resolved to the first",
"file": "app/services/rule_cleanup_service.py",
"old": " if len(found) == 1:\n return \"found\", found[0]",
"new": " if found:\n return \"found\", found[0]",
"tests": "tests/test_rule_cleanup.py::test_unclear_restore_with_twins_and_no_stored_uuid_stays_unknown"
},
{
"name": "R11 any acknowledgement text version accepted",
"file": "app/services/rule_cleanup_service.py",
"old": " if str(ack_text_version) != ACK_TEXT_VERSION:",
"new": " if False:",
"tests": "tests/test_rule_cleanup.py::test_an_outdated_acknowledgement_text_is_refused"
}
]
@@ -0,0 +1,373 @@
[
{
"name": "middleware: confirm not session-only",
"file": "app/services/api_token_service.py",
"old": "|^/api/hosts/[^/]+/rule-cleanup/[^/]+/confirm/?$\")",
"new": "\")",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: retry not admin-only",
"file": "app/services/api_token_service.py",
"old": "(deploy|retry)|items",
"new": "(deploy)|items",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: restore not admin-only",
"file": "app/services/api_token_service.py",
"old": "|items/[^/]+/restore)/?$\")",
"new": ")/?$\")",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: deploy-scope token passes admin-only cleanup routes",
"file": "app/services/api_token_service.py",
"old": "if scope == \"deploy\" and (_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path)):",
"new": "if scope == \"deploy\" and (_ADMIN_ONLY_WRITES.match(path)):",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: operator passes admin-only cleanup routes",
"file": "app/services/api_token_service.py",
"old": " _ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path))",
"new": " _ADMIN_ONLY_WRITES.match(path))",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: revert not a rollout (operators_may_deploy ignored)",
"file": "app/services/api_token_service.py",
"old": "r\"|rule-cleanup/[^/]+/(revert|retry)\"",
"new": "r\"|rule-cleanup/[^/]+/(retry)\"",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "middleware: retry not a rollout",
"file": "app/services/api_token_service.py",
"old": "r\"|rule-cleanup/[^/]+/(revert|retry)\"",
"new": "r\"|rule-cleanup/[^/]+/(revert)\"",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: deploy only needs a writer",
"file": "app/routers/rule_cleanup.py",
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.deploy(",
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.deploy(",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: retry only needs a writer",
"file": "app/routers/rule_cleanup.py",
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.retry(",
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.retry(",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: restore only needs a writer",
"file": "app/routers/rule_cleanup.py",
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.restore_item(",
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.restore_item(",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: confirm only needs a writer (tokens may confirm)",
"file": "app/routers/rule_cleanup.py",
"old": " _session_only(user, request)\n",
"new": " _writer(user, request)\n",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: revert needs no deployer",
"file": "app/routers/rule_cleanup.py",
"old": " await _deployer(user, request, db)\n",
"new": " _writer(user, request)\n",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: admin role not checked",
"file": "app/routers/rule_cleanup.py",
"old": " if user.role != \"admin\":\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n token = _token(request)",
"new": " if False:\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n token = _token(request)",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: admin token scope not checked",
"file": "app/routers/rule_cleanup.py",
"old": " if token is not None and token.scope != \"admin\":",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: session-only token check removed",
"file": "app/routers/rule_cleanup.py",
"old": " if token is not None:\n raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)",
"new": " if False:\n raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: operator revert ignores operators_may_deploy",
"file": "app/routers/rule_cleanup.py",
"old": "if user.role == \"operator\" and (await governance.get_settings(db)).operators_may_deploy:",
"new": "if user.role == \"operator\":",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: read-only account may write",
"file": "app/routers/rule_cleanup.py",
"old": " if user.role == \"read_only\":\n raise api_error(403, ErrorCode.READ_ONLY_ACCESS)",
"new": " if False:\n raise api_error(403, ErrorCode.READ_ONLY_ACCESS)",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: read-scope token may write",
"file": "app/routers/rule_cleanup.py",
"old": " if token is not None and token.scope == \"read\":",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: preview needs no writer",
"file": "app/routers/rule_cleanup.py",
"old": " user = await resolve_user(request, db)\n _writer(user, request)\n host = await _host(db, host_id)\n kind, uuids",
"new": " user = await resolve_user(request, db)\n host = await _host(db, host_id)\n kind, uuids",
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
},
{
"name": "router: actor is not the account name",
"file": "app/routers/rule_cleanup.py",
"old": "actor=user.username, actor_id=user.id, kind=kind",
"new": "actor='x', actor_id=user.id, kind=kind",
"tests": "tests/test_rule_cleanup_api.py -k 'preview_belongs_to_its_actor or shapes_and_audit'"
},
{
"name": "settings: any role may write",
"file": "app/routers/governance.py",
"old": " if user.role != \"admin\":\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n row = await get_settings(db)\n changes",
"new": " if False:\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n row = await get_settings(db)\n changes",
"tests": "tests/test_rule_cleanup_api.py -k only_an_admin_writes"
},
{
"name": "settings: whole address space accepted",
"file": "app/routers/governance.py",
"old": " if net.prefixlen == 0:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: host names accepted",
"file": "app/routers/governance.py",
"old": " raise _bad(\"cleanup_management_sources\", \"not_an_address\") from None",
"new": " continue",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: unlimited list",
"file": "app/routers/governance.py",
"old": " if len(values) > MAX_MANAGEMENT_SOURCES:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: sources not canonical",
"file": "app/routers/governance.py",
"old": " entry = str(net.network_address) if net.prefixlen == net.max_prefixlen else str(net)",
"new": " entry = text",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: sources shown to everybody",
"file": "app/routers/governance.py",
"old": "cleanup_management_sources=sources if user.role == \"admin\" else [],",
"new": "cleanup_management_sources=sources,",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: change not audited",
"file": "app/routers/governance.py",
"old": " if changes:\n db.add(audit_service.make_event(",
"new": " if False:\n db.add(audit_service.make_event(",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: max rules unbounded",
"file": "app/routers/governance.py",
"old": " if not 1 <= payload.cleanup_max_rules <= MAX_CLEANUP_RULES:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "settings: min days unbounded",
"file": "app/routers/governance.py",
"old": " if not 1 <= payload.cleanup_delete_min_days <= MAX_DELETE_MIN_DAYS:",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
},
{
"name": "lock: ensure_no_maintenance ignores a cleanup",
"file": "app/services/host_maintenance_service.py",
"old": " if cleanup and ids:\n await ensure_no_open_cleanup(db, hosts)",
"new": " if False:\n await ensure_no_open_cleanup(db, hosts)",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: ensure_no_open_cleanup never raises",
"file": "app/services/host_maintenance_service.py",
"old": " if ids and await has_open_cleanup(db, ids):\n raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)",
"new": " if False:\n raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: maintenance inspection ignores a cleanup",
"file": "app/services/host_maintenance_service.py",
"old": " if await has_open_cleanup(db, fleet_ids):\n blocked.append(",
"new": " if False:\n blocked.append(",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: host delete ignores a cleanup",
"file": "app/routers/hosts.py",
"old": " await ensure_no_open_cleanup(db, [host])\n # Both deployment tables FK",
"new": " # Both deployment tables FK",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: host re-addressing ignores a cleanup",
"file": "app/routers/hosts.py",
"old": " await ensure_no_open_cleanup(db, [host])\n\n if payload.name is not None:",
"new": " pass\n\n if payload.name is not None:",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: rule deletion ignores a cleanup",
"file": "app/services/rule_deploy_service.py",
"old": " await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: hierarchy rule deletion ignores a cleanup",
"file": "app/services/policy_hierarchy_deploy_service.py",
"old": " await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: hierarchy reordering ignores a cleanup",
"file": "app/services/policy_hierarchy_deploy_service.py",
"old": " await ensure_no_open_cleanup(db, [h for h in hosts.values() if h.enabled])",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: removing a rule target ignores a cleanup",
"file": "app/routers/rules.py",
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: removing a hierarchy target ignores a cleanup",
"file": "app/routers/policy_hierarchy.py",
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: adoption ignores a cleanup",
"file": "app/routers/rules.py",
"old": " await ensure_no_open_cleanup(db, [host])\n try:\n rule = await rule_adopt_service.adopt(",
"new": " try:\n rule = await rule_adopt_service.adopt(",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
},
{
"name": "lock: cleanup service trips over itself",
"file": "app/services/rule_cleanup_service.py",
"old": "await ensure_no_maintenance(db, [host], cleanup=False)",
"new": "await ensure_no_maintenance(db, [host])",
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or preview_input_forms or shapes_and_audit or other_open_work'"
},
{
"name": "service: lost revert claim hits an expired row",
"file": "app/services/rule_cleanup_service.py",
"old": "\n await db.refresh(op) # _revert rolled back when it lost the claim: the status is expired\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=op.status)\n await db.commit()",
"new": "\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=op.status)\n await db.commit()",
"tests": "tests/test_rule_cleanup_api.py -k revert_is_for_admins"
},
{
"name": "service: lost retry claim hits an expired row",
"file": "app/services/rule_cleanup_service.py",
"old": "\n await db.refresh(op) # _revert rolled back when it lost the claim: the status is expired",
"new": "",
"tests": "tests/test_rule_cleanup_api.py -k retry_after_a_failed"
},
{
"name": "notify: failed revert not reported",
"file": "app/services/notification_checks.py",
"old": " if op.status == \"revert_failed\":",
"new": " if False:",
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
},
{
"name": "notify: partial run not reported",
"file": "app/services/notification_checks.py",
"old": " elif op.status in (\"partial\", \"apply_failed\"):",
"new": " elif False:",
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
},
{
"name": "notify: unclear delete not reported",
"file": "app/services/notification_checks.py",
"old": " elif op.kind in (\"delete\", \"restore\"):",
"new": " elif False:",
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
},
{
"name": "notify: every item of a failed revert is reported",
"file": "app/services/notification_checks.py",
"old": " stuck = [i for i in items if i.state in _CLEANUP_STILL_CHANGED]",
"new": " stuck = list(items)",
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
},
{
"name": "notify: cleanup problems not part of the revert alert",
"file": "app/services/notification_checks.py",
"old": " problems.update(await _cleanup_problems(db))",
"new": " pass",
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
},
{
"name": "preview writes no own audit entry",
"file": "app/services/rule_cleanup_service.py",
"old": " db.add(audit_service.make_event(\n kind=\"rule_cleanup\", action=\"preview\"",
"new": " (lambda *a, **k: None)(audit_service.make_event(\n kind=\"rule_cleanup\", action=\"preview\"",
"tests": "tests/test_rule_cleanup_api.py -k shapes_and_audit"
},
{
"name": "R9 alias undeploy without cleanup lock",
"file": "app/services/alias_deploy_service.py",
"old": " await ensure_no_open_cleanup_on(db, AliasDeployment, AliasDeployment.alias_id == alias.id)\n",
"new": "",
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
},
{
"name": "R9 alias target removal without cleanup lock",
"file": "app/routers/aliases.py",
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes\n",
"new": "",
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
},
{
"name": "R9 wireguard undeploy without cleanup lock",
"file": "app/services/wireguard_deploy_service.py",
"old": " await ensure_no_open_cleanup_on(db, WireguardTunnelDeployment, WireguardTunnelDeployment.tunnel_id == tunnel.id)\n",
"new": "",
"tests": "tests/test_rule_cleanup_api.py::test_every_undeploy_and_target_removal_checks_for_an_open_cleanup_first"
},
{
"name": "R9 helper checks nothing",
"file": "app/services/host_maintenance_service.py",
"old": " ids = set((await db.scalars(select(model.host_id).where(condition))).all())",
"new": " ids = set()",
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
}
]
+241
View File
@@ -0,0 +1,241 @@
"""A small stateful OPNsense for the rule-cleanup tests (respx side effects).
It keeps its rules in memory and answers searchRule, getRule, toggleRule, delRule, addRule, moveRuleBefore, apply,
ruleStats, pfStatistics (generated from the enabled rules, like pf after an apply), interfacesInfo, groups, aliases,
port forwards, getVipStatus and core/hasync/get in the shapes seen live (OPNsense 26.7, see opnsense_client.py).
``fail`` injects errors per call: {("toggle", uuid): httpx.ReadTimeout | httpx.Response | callable}.
Addresses: documentation/private ranges only.
"""
from __future__ import annotations
import copy
import re
import uuid as uuid_lib
import httpx
R_BASE = "aaaaaaaa-0000-4000-8000-000000000100"
R_DUP = "aaaaaaaa-0000-4000-8000-000000000200"
R_UNUSED = "aaaaaaaa-0000-4000-8000-000000000300"
R_BLOCK = "aaaaaaaa-0000-4000-8000-000000000400"
R_MGMT = "aaaaaaaa-0000-4000-8000-000000000500"
R_CARP = "aaaaaaaa-0000-4000-8000-000000000600"
R_ALIAS = "aaaaaaaa-0000-4000-8000-000000000700"
R_TAIL = "aaaaaaaa-0000-4000-8000-000000000800"
R_LEGACY = "0123456789abcdef0123456789abcdef"
AUTO = "02f4bab031b57d1e30553ce08e0ec131"
MGMT_SOURCE = "203.0.113.10"
DEVICES = {"wan": "vtnet0", "lan": "vtnet1"}
def rule(uuid, seq, *, action="pass", interface="lan", proto="TCP", src="any", dst="any", dport="", enabled="1",
description=None, **extra) -> dict:
row = {
"uuid": uuid, "sort_order": f"400000.{seq:07d}", "prio_group": "400000", "sequence": str(seq),
"enabled": enabled, "action": action, "quick": "1", "interface": interface, "interfacenot": "0",
"direction": "in", "ipprotocol": "inet", "protocol": proto, "source_net": src, "source_not": "0",
"source_port": "", "destination_net": dst, "destination_not": "0", "destination_port": dport,
"gateway": "", "sched": "", "statetype": "keep", "tagged": "", "log": "0", "categories": "",
"description": description or f"cleanup-{seq}",
}
row.update(extra)
return row
def default_rules() -> list[dict]:
return [
{"uuid": AUTO, "sort_order": "000001.1000001", "legacy": True, "is_automatic": True, "enabled": "1",
"action": "block", "quick": False, "direction": "in", "ipprotocol": "inet46", "protocol": "",
"source_net": "", "source_port": "", "destination_net": "", "destination_port": "",
"description": "Default deny / state violation rule"},
rule(R_BASE, 100, src="10.20.30.0/24", dport="9000", description="base 9000"),
rule(R_DUP, 200, src="10.20.30.0/24", dport="9000", description="dup 9000"),
rule(R_UNUSED, 300, src="10.20.30.0/24", dport="8081", description="unused 8081"),
rule(R_BLOCK, 400, action="block", proto="UDP", dport="9999", description="block 9999"),
rule(R_MGMT, 500, interface="wan", src="203.0.113.0/24", dport="443", description="mgmt 443"),
rule(R_CARP, 600, proto="CARP", description="carp"),
rule(R_ALIAS, 700, interface="wan", src="tbl", dport="443", description="alias 443"),
rule(R_TAIL, 800, src="10.20.30.7", dport="7000", description="tail 7000"),
]
class FakeFirewall:
def __init__(self, base: str, *, role: str | None = None, rules: list[dict] | None = None,
pfsync: str = "", peer: str = "", lan_addr: str = "10.20.30.2", wan_addr: str = "203.0.113.57",
log: list | None = None):
self.base = base
self.log = log # shared across firewalls: the order of calls over a cluster
self.role = role
self.rules: list[dict] = copy.deepcopy(rules if rules is not None else default_rules())
self.pfsync, self.peer = pfsync, peer
self.lan_addr, self.wan_addr = lan_addr, wan_addr
self.fail: dict = {}
self.calls: list[tuple[str, str]] = []
self.applied = 0
self.down = False
self.on_call = None # callable(kind, uuid) run before a write is answered (e.g. look at the database)
# ── state helpers ──
def get(self, uuid):
return next((r for r in self.rules if r["uuid"].lower() == uuid.lower()), None)
def _inject(self, key):
effect = self.fail.get(key)
if effect is None:
return None
if isinstance(effect, list):
effect = effect.pop(0) if effect else None
if effect is None:
return None
if isinstance(effect, type) and issubclass(effect, Exception):
raise effect("injected")
if isinstance(effect, Exception):
raise effect
if callable(effect) and not isinstance(effect, httpx.Response):
return effect()
return effect
def pf_lines(self) -> dict:
lines = {"@0 scrub in all fragment reassemble": {}}
n = 1
for r in sorted(self.rules, key=lambda r: r["sort_order"]):
if str(r.get("enabled")) != "1":
continue
if r.get("is_automatic"):
lines[f'@{n} block drop in log inet all label "{r["uuid"]}"'] = {"evaluations": 0}
else:
words = ["block" if r["action"] == "block" else "pass", "in", "quick", f"on {DEVICES.get(r['interface'], r['interface'])}", "inet"]
if r["protocol"] not in ("", "any"):
words.append(f"proto {r['protocol'].lower()}")
words.append(f'all label "{r["uuid"]}"')
lines[f"@{n} " + " ".join(words)] = {"evaluations": 0, "packets": 0}
n += 1
return {"rules": {"filter rules": lines}}
def model(self, row: dict) -> dict:
"""getRule shape: select fields as option dicts, no uuid key."""
out = {k: v for k, v in row.items() if k not in ("uuid", "is_automatic", "legacy")}
out["action"] = {a: {"value": a, "selected": 1 if row["action"] == a else 0} for a in ("pass", "block", "reject")}
out["interface"] = {i: {"value": i.upper(), "selected": 1 if i in str(row["interface"]).split(",") else 0} for i in ("lan", "wan")}
out["categories"] = []
out["audit"] = {"created": "x", "updated": "y"}
return out
# ── respx wiring ──
def mount(self, respx_mock):
respx_mock.route(url__startswith=self.base).mock(side_effect=self.handle)
return self
def handle(self, request: httpx.Request) -> httpx.Response:
path = request.url.path.removeprefix("/api")
method = request.method
self.calls.append((method, path))
if self.log is not None:
self.log.append((self.base, method, path))
if self.down:
raise httpx.ConnectError("down")
if method == "GET" and path == "/diagnostics/interface/getVipStatus":
rows = [] if self.role is None else [{"interface": "LAN", "vhid": "10", "status": self.role}]
return httpx.Response(200, json={"rows": rows, "carp": {"allow": "1", "maintenancemode": False}})
if method == "GET" and path == "/firewall/filter/searchRule":
got = self._inject(("search", ""))
if got is not None:
return got
return httpx.Response(200, json={"rows": copy.deepcopy(sorted(self.rules, key=lambda r: r["sort_order"]))})
if method == "GET" and path == "/diagnostics/firewall/pfStatistics/rules":
return httpx.Response(200, json=self.pf_lines())
if method == "GET" and path == "/firewall/filter_util/ruleStats":
return httpx.Response(200, json={"status": "ok", "stats": {}})
if method == "GET" and path == "/interfaces/overview/interfacesInfo":
return httpx.Response(200, json={"rows": [
{"identifier": "wan", "device": "vtnet0", "description": "WAN", "ipv4": [{"ipaddr": f"{self.wan_addr}/24"}],
"ipv6": [], "gateways": ["203.0.113.1"], "routes": ["default"]},
{"identifier": "lan", "device": "vtnet1", "description": "LAN",
"ipv4": [{"ipaddr": f"{self.lan_addr}/24"}, {"ipaddr": "10.20.30.1/24", "vhid": "10"}],
"ipv6": [], "gateways": [], "routes": ["10.20.30.0/24"]},
{"identifier": "lo0", "device": "lo0", "description": "Loopback", "ipv4": [{"ipaddr": "127.0.0.1/8"}], "ipv6": []},
]})
if method == "GET" and path == "/firewall/group/searchItem":
return httpx.Response(200, json={"rows": []})
if method == "GET" and path == "/firewall/alias/searchItem":
return httpx.Response(200, json={"rows": [{"name": "tbl", "type": "urltable", "content": "https://203.0.113.9/list"}]})
if method == "GET" and path == "/firewall/d_nat/searchRule":
return httpx.Response(200, json={"rows": []})
if method == "GET" and path == "/core/hasync/get":
sel = {"": {"value": "Disabled", "selected": 1 if not self.pfsync else 0},
"lan": {"value": "LAN", "selected": 1 if self.pfsync == "lan" else 0}}
return httpx.Response(200, json={"hasync": {"pfsyncinterface": sel, "pfsyncpeerip": self.peer, "password": "x"}})
m = re.fullmatch(r"/firewall/filter/getRule/([^/]+)", path)
if method == "GET" and m:
got = self._inject(("get", m.group(1)))
if got is not None:
return got
row = self.get(m.group(1))
if row is None or row.get("is_automatic") or row.get("legacy"):
return httpx.Response(200, json=[])
return httpx.Response(200, json={"rule": self.model(row)})
m = re.fullmatch(r"/firewall/filter/toggleRule/([^/]+)/([01])", path)
if method == "POST" and m:
if self.on_call:
self.on_call("toggle", m.group(1))
row = self.get(m.group(1))
if row is not None and not row.get("legacy"):
pending = ("toggle", m.group(1))
if pending in self.fail and self.fail.get(("toggle_applies", m.group(1))):
row["enabled"] = m.group(2) # applied on the firewall, but the answer is lost
got = self._inject(pending)
if got is not None:
return got
changed = row["enabled"] != m.group(2)
row["enabled"] = m.group(2)
return httpx.Response(200, json={"result": "Enabled" if m.group(2) == "1" else "Disabled", "changed": changed})
return httpx.Response(200, json={"result": "failed"})
m = re.fullmatch(r"/firewall/filter/delRule/([^/]+)", path)
if method == "POST" and m:
if self.on_call:
self.on_call("delete", m.group(1))
row = self.get(m.group(1))
applies = self.fail.get(("delete_applies", m.group(1)))
if applies and row is not None:
self.rules.remove(row)
got = self._inject(("delete", m.group(1)))
if got is not None:
return got
if row is None or row.get("legacy"):
return httpx.Response(200, json={"result": "not found"})
self.rules.remove(row)
return httpx.Response(200, json={"result": "deleted"})
if method == "POST" and path == "/firewall/filter/addRule":
got = self._inject(("add", ""))
if got is not None:
return got
import json as _json
body = _json.loads(request.content)["rule"]
new = str(uuid_lib.uuid4())
top = max(int(float(r.get("sequence") or 0)) for r in self.rules if not r.get("is_automatic"))
row = {**body, "uuid": new, "sequence": str(top + 100), "sort_order": f"400000.{top + 100:07d}", "prio_group": "400000"}
self.rules.append(row)
return httpx.Response(200, json={"result": "saved", "uuid": new})
m = re.fullmatch(r"/firewall/filter/moveRuleBefore/([^/]+)/([^/]+)", path)
if method == "POST" and m:
got = self._inject(("move", m.group(1))) or self._inject(("move", "*"))
if got is not None:
return got
moving, target = self.get(m.group(1)), self.get(m.group(2))
if moving is None or target is None or moving is target:
return httpx.Response(500, json={"errorMessage": "Unexpected error, check log for details"})
seq = int(target["sequence"]) - 1
moving["sequence"], moving["sort_order"] = str(seq), f"400000.{seq:07d}"
return httpx.Response(200, json={"status": "ok"})
if method == "POST" and path == "/firewall/filter/apply":
if self.on_call:
self.on_call("apply", "")
got = self._inject(("apply", ""))
if got is not None:
return got
self.applied += 1
return httpx.Response(200, json={"status": "OK\n\n"})
return httpx.Response(404, json={"message": f"unexpected {method} {path}"})
def writes(self) -> list[tuple[str, str]]:
return [(m, p) for m, p in self.calls if m == "POST"]
+57 -4
View File
@@ -12,12 +12,12 @@ from tests.test_rules import _create_host
def _mvc_rule(*, interface="lan", action="pass", src="any", dst="any", protocol="any", descr="r", seq=1,
enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0"):
enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0", extra=""):
return (f"<rule><enabled>{enabled}</enabled><action>{action}</action><quick>{quick}</quick><interface>{interface}</interface>"
f"<direction>{direction}</direction><ipprotocol>{ipprotocol}</ipprotocol><protocol>{protocol}</protocol>"
f"<source_net>{src}</source_net><source_not>{src_not}</source_not><destination_net>{dst}</destination_net>"
f"<destination_not>0</destination_not><destination_port>{dport}</destination_port>"
f"<description>{descr}</description><sequence>{seq}</sequence></rule>")
f"<description>{descr}</description><sequence>{seq}</sequence>{extra}</rule>")
def _config(*, rules="", legacy="", protocol="https", ssh="", snmpd="", timeservers="0.pool.ntp.org", syslog="",
@@ -105,6 +105,52 @@ def test_rules_are_ordered_by_sequence_not_file_order():
assert _run(_config(rules=rules))["rules_shadowed"]["params"]["examples"] == ["narrow"]
def test_coverage_uses_networks_and_the_floating_section_like_the_firewall_does():
"""1.30: a CIDR inside a CIDR is shadowed, a floating quick rule is evaluated before every interface rule,
and an interface rule never shadows a floating one that comes first."""
shadow = _run(_config(rules=(
_mvc_rule(src="10.0.0.0/8", action="block", descr="wide", seq=1)
+ _mvc_rule(src="10.1.0.0/16", descr="inside", seq=2))))
assert shadow["rules_shadowed"]["params"]["examples"] == ["inside"]
floating = _run(_config(rules=(
_mvc_rule(interface="lan", src="10.1.0.0/16", descr="interface rule", seq=1)
+ _mvc_rule(interface="lan,wan", src="10.0.0.0/8", action="block", descr="floating block", seq=9))))
assert floating["rules_shadowed"]["params"]["examples"] == ["interface rule"]
reverse = _run(_config(rules=(
_mvc_rule(interface="lan", src="10.0.0.0/8", action="block", descr="interface rule", seq=1)
+ _mvc_rule(interface="lan,wan", src="10.1.0.0/16", descr="floating narrow", seq=9))))
assert reverse["rules_shadowed"]["passed"] is True
def test_a_legacy_floating_rule_is_only_quick_when_it_says_so():
"""mutation: defaulting legacy rules to quick makes a plain floating rule 'cover' everything after it."""
def legacy_rule(descr, *, floating, quick=None, src="any"):
return (f"<rule><type>block</type><interface>lan,wan</interface><ipprotocol>inet</ipprotocol>"
f"{'<floating>yes</floating>' if floating else ''}{f'<quick>{quick}</quick>' if quick else ''}"
f"<descr>{descr}</descr><source><any>1</any></source><destination><any>1</any></destination></rule>"
if src == "any" else
f"<rule><type>pass</type><interface>lan</interface><ipprotocol>inet</ipprotocol><descr>{descr}</descr>"
f"<source><address>{src}</address></source><destination><any/></destination></rule>")
later = legacy_rule("later", floating=False, src="10.0.0.0/8")
assert _run(_config(legacy=legacy_rule("not quick", floating=True) + later))["rules_shadowed"]["passed"] is True
quick = _run(_config(legacy=legacy_rule("quick", floating=True, quick="yes") + later))
assert quick["rules_shadowed"]["params"]["examples"] == ["later"]
# a non-floating legacy rule is always quick, whatever <quick> says
plain = _run(_config(legacy=legacy_rule("interface rule", floating=False) .replace("lan,wan", "lan") + later))
assert plain["rules_shadowed"]["params"]["examples"] == ["later"]
def test_rules_at_the_same_position_in_the_config_are_not_compared():
rules = _mvc_rule(src="any", action="block", descr="first", seq=5) + _mvc_rule(src="10.0.0.0/8", descr="second", seq=5)
assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True
def test_a_scheduled_or_tagged_rule_does_not_shadow_anything():
for extra in ("<sched>office</sched>", "<tagged>mark</tagged>", "<received-on>wan</received-on>"):
rules = _mvc_rule(src="any", action="block", descr="timed", seq=1, extra=extra) + _mvc_rule(src="10.0.0.0/8", descr="later", seq=2)
assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True, extra
def test_ssh_levels():
on = "<ssh><enabled>enabled</enabled><permitrootlogin>{}</permitrootlogin><passwordauth>{}</passwordauth></ssh>"
worst = _run(_config(ssh=on.format(1, 1)))["ssh_hardened"]
@@ -159,6 +205,8 @@ def _mock_host(respx_mock, xml, firmware_status="none"):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
respx_mock.get(url__regex=r".*/core/backup/download/this").mock(return_value=httpx.Response(200, content=xml))
respx_mock.get(url__regex=r".*/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": firmware_status}))
# An older release without ruleStats: the usage sampling falls back to the pf ruleset text.
respx_mock.get(url__regex=r".*/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={"message": "not found"}))
respx_mock.get(url__regex=r".*/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
respx_mock.get(url__regex=r".*/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
@@ -246,7 +294,10 @@ def test_pf_counters_are_tied_to_rule_uuids_and_summed_over_expanded_lines():
stats["rules"]["filter rules"][f'@7 pass in quick on vtnet0 inet6 all label "{U1}"'] = {"evaluations": 2, "packets": 3, "bytes": 30}
got = rh.parse_counters(stats)
assert got[U1] == {"packets": 8, "bytes": 530, "evaluations": 12}
assert "2f2c37025cd02b5cd29fd330f8e48954" not in got # md5 labels are automatic rules, not user rules
# An md5 label is either an automatic rule or an unmigrated legacy rule: only searchRule (is_automatic) tells them apart.
assert got["2f2c37025cd02b5cd29fd330f8e48954"]["packets"] == 9
assert "2f2c37025cd02b5cd29fd330f8e48954" not in rh.configured_rules(
{"rows": [{"uuid": "2f2c37025cd02b5cd29fd330f8e48954", "is_automatic": True}]})
assert rh.parse_counters({}) == {} and rh.parse_counters({"rules": None}) == {}
@@ -259,7 +310,8 @@ async def _sample(host_id, stats, search, at):
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, __import__("uuid").UUID(host_id))
from unittest.mock import AsyncMock, patch
with patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
with patch.object(rh.client, "get_rule_stats", AsyncMock(side_effect=rh.OpnsenseAPIError("404"))), \
patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
patch.object(rh.client, "search_rules", AsyncMock(return_value=search)):
return await rh.sample_host(db, host, now=at)
@@ -373,6 +425,7 @@ async def test_one_truncated_config_does_not_stop_the_others(client, respx_mock)
for base, body in (("bad", b"<opnsense><system><webgui>"), ("good", _config())):
respx_mock.get(f"https://{base}.test/api/core/backup/download/this").mock(return_value=httpx.Response(200, content=body))
respx_mock.get(f"https://{base}.test/api/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": "none"}))
respx_mock.get(f"https://{base}.test/api/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={}))
respx_mock.get(f"https://{base}.test/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
respx_mock.get(f"https://{base}.test/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
resp = await client.post("/api/compliance/check")
+119 -1
View File
@@ -331,5 +331,123 @@ async def test_0026_adds_account_ids_matches_existing_names_and_is_idempotent(sc
await _alembic(scratch, "upgrade", "head")
await _alembic(scratch, "downgrade", "0025")
await _alembic(scratch, "upgrade", "head")
assert await _version(scratch) == _head() == "0026"
assert await _version(scratch) == _head()
assert await _schema_diff(scratch) == []
async def test_0027_creates_the_rule_history_tables_and_is_idempotent(scratch):
"""1.29 -> 1.30: two new tables, the downgrade drops them; running the upgrade over tables that a pre-Alembic
bridge (create_all) already made changes nothing."""
await _alembic(scratch, "upgrade", "0026")
async with scratch.connect() as conn:
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
assert "live_rule_states" not in tables and "rule_change_events" not in tables
await _alembic(scratch, "upgrade", "head")
assert await _version(scratch) == _head()
assert await _schema_diff(scratch) == []
host = "11111111-1111-1111-1111-111111111111"
async with scratch.begin() as conn:
await conn.execute(text(
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
"VALUES (:h, 'fw', 'https://fw.test', 'k', true, true, now())"), {"h": host})
await conn.execute(text(
"INSERT INTO rule_change_events (id, host_id, rule_uuid, seq, observed_at, kind, origin) "
"VALUES (gen_random_uuid(), :h, 'r', 1, now(), 'added', 'external')"), {"h": host})
with pytest.raises(DBAPIError): # the same change can only be recorded once
async with scratch.begin() as conn:
await conn.execute(text(
"INSERT INTO rule_change_events (id, host_id, rule_uuid, seq, observed_at, kind, origin) "
"VALUES (gen_random_uuid(), :h, 'r', 1, now(), 'added', 'external')"), {"h": host})
async with scratch.begin() as conn: # deleting the host takes its history with it
await conn.execute(text("DELETE FROM opnsense_hosts"))
assert (await conn.execute(text("SELECT count(*) FROM rule_change_events"))).scalar() == 0
await _alembic(scratch, "downgrade", "0026")
async with scratch.connect() as conn:
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
assert "live_rule_states" not in tables and "rule_change_events" not in tables
await _alembic(scratch, "upgrade", "head")
await _alembic(scratch, "upgrade", "head") # second run: nothing to do
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
async def test_0028_adds_the_event_window_and_is_idempotent(scratch):
await _alembic(scratch, "upgrade", "0027")
async with scratch.connect() as conn:
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'rule_change_events'"))).scalars().all())
assert "window_start" not in cols
await _alembic(scratch, "upgrade", "head")
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
await _alembic(scratch, "downgrade", "0027")
async with scratch.connect() as conn:
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'rule_change_events'"))).scalars().all())
assert "window_start" not in cols
await _alembic(scratch, "upgrade", "head")
await _alembic(scratch, "upgrade", "head")
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
async def test_0029_creates_the_rule_cleanup_tables_and_settings_and_is_idempotent(scratch):
"""1.30 -> 1.31: two tables with a partial unique index (one unfinished item per rule and node) and three
columns on governance_settings; the downgrade removes all of it again."""
await _alembic(scratch, "upgrade", "0028")
async with scratch.begin() as conn:
await conn.execute(text("INSERT INTO governance_settings (id, operators_may_deploy) VALUES (1, true)"))
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'governance_settings'"))).scalars().all())
assert "rule_cleanup_operations" not in tables and "rule_cleanup_items" not in tables
assert "cleanup_max_rules" not in cols
await _alembic(scratch, "upgrade", "head")
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
async with scratch.connect() as conn: # the existing settings row got the defaults of the design
row = (await conn.execute(text(
"SELECT cleanup_max_rules, cleanup_delete_min_days, cleanup_management_sources, cleanup_management_ports "
"FROM governance_settings"))).one()
assert tuple(row) == (10, 7, [], [])
host, op_id = "11111111-1111-1111-1111-111111111111", "22222222-2222-2222-2222-222222222222"
async with scratch.begin() as conn:
await conn.execute(text(
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
"VALUES (:h, 'fw', 'https://fw.test', 'k', true, true, now())"), {"h": host})
await conn.execute(text(
"INSERT INTO rule_cleanup_operations (id, host_id, kind, status, actor, created_at, expires_at, preview_fingerprint, options) "
"VALUES (:o, :h, 'disable', 'preview', 'admin', now(), now(), 'f', '{}')"), {"o": op_id, "h": host})
async def item(conn, rule: str, state: str):
await conn.execute(text(
"INSERT INTO rule_cleanup_items (id, operation_id, host_id, rule_uuid, seq, state, description, categories) "
f"VALUES (gen_random_uuid(), :o, :h, '{rule}', 0, '{state}', 'r', '[]')"), {"o": op_id, "h": host})
async with scratch.begin() as conn:
await item(conn, "r1", "disabled")
await item(conn, "r2", "disabled") # another rule is fine
await item(conn, "r1", "confirmed") # finished items do not count
await item(conn, "r1", "reverted")
for blocked in ("queued", "unknown", "revert_failed", "deleting"): # r1 is still open as "disabled"
with pytest.raises(DBAPIError):
async with scratch.begin() as conn:
await item(conn, "r1", blocked)
async with scratch.begin() as conn: # on another node the same UUID is a different rule
await conn.execute(text(
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
"VALUES ('33333333-3333-3333-3333-333333333333', 'fw2', 'https://fw2.test', 'k', true, true, now())"))
await conn.execute(text(
"INSERT INTO rule_cleanup_items (id, operation_id, host_id, rule_uuid, seq, state, description, categories) "
"VALUES (gen_random_uuid(), :o, '33333333-3333-3333-3333-333333333333', 'r1', 0, 'disabled', 'r', '[]')"), {"o": op_id})
async with scratch.begin() as conn: # the host takes its operations and their items with it
await conn.execute(text("DELETE FROM opnsense_hosts WHERE id = :h"), {"h": host})
assert (await conn.execute(text("SELECT count(*) FROM rule_cleanup_operations"))).scalar() == 0
assert (await conn.execute(text("SELECT count(*) FROM rule_cleanup_items"))).scalar() == 0
await _alembic(scratch, "downgrade", "0028")
async with scratch.connect() as conn:
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'governance_settings'"))).scalars().all())
assert "rule_cleanup_operations" not in tables and "rule_cleanup_items" not in tables
assert not {"cleanup_max_rules", "cleanup_delete_min_days", "cleanup_management_sources", "cleanup_management_ports"} & cols
await _alembic(scratch, "upgrade", "head")
await _alembic(scratch, "upgrade", "head") # second run: nothing to do
assert await _version(scratch) == _head() == "0029" and await _schema_diff(scratch) == []
+2 -2
View File
@@ -146,9 +146,9 @@ def test_what_counts_as_a_rollout(method, path, rollout):
async def test_with_separation_of_duties_operators_prepare_but_only_admins_deploy(client):
op = await _operator(client, "dutyop")
try:
assert (await op.get("/api/settings/governance")).json() == {"operators_may_deploy": True}
assert (await op.get("/api/settings/governance")).json()["operators_may_deploy"] is True
assert (await op.put("/api/settings/governance", json={"operators_may_deploy": False})).status_code == 403
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": False})).json() == {"operators_may_deploy": False}
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": False})).json()["operators_may_deploy"] is False
# Preparing still works ...
assert (await op.post("/api/groups", json={"name": "prepared"})).status_code == 201
# ... rolling out does not, whatever the target (the 403 comes before any lookup).
+246
View File
@@ -0,0 +1,246 @@
"""Client calls for filter rules that the cleanup assistant (1.31) builds on: toggleRule, getRule, the
getRule -> addRule payload, and the request shape / failure classification of add/del/move/apply.
The JSON fixtures are shaped like live answers of OPNsense 26.7.7 (checked 2026-10-22 on a test firewall) with
documentation addresses and made-up UUIDs. Evidence for each endpoint: comment block in services/opnsense_client.py.
"""
import json
import uuid
import httpx
import pytest
from app.models.host import OpnsenseHost
from app.services import opnsense_client as c
BASE = "https://fw.test/api/firewall/filter"
RULE = "5c6f3bfa-1718-4216-81c8-04a75527960f"
OTHER = "74190f58-f35f-4402-8794-5410e99ea364"
@pytest.fixture
def host(monkeypatch):
monkeypatch.setattr(c.opnsense_secret_store, "read_secret", lambda host_id: "secret")
return OpnsenseHost(id=uuid.uuid4(), name="fw-a", base_url="https://fw.test", api_key="key", verify_tls=False)
def _opts(selected: list[str], options: dict[str, str]) -> dict:
return {k: {"value": v, "selected": 1 if k in selected else 0} for k, v in options.items()}
def live_rule() -> dict:
"""A `getRule/<uuid>` record: select fields as option dicts, `categories` an option dict, read-only extras."""
return {
"enabled": "1", "sequence": "566", "sort_order": "200000.0000566", "prio_group": "200000", "quick": "1",
"action": _opts(["pass"], {"pass": "Pass", "block": "Block", "reject": "Reject"}),
"interface": _opts(["lan", "wan"], {"lan": "LAN", "wan": "WAN", "wireguard": "WireGuard (Group)"}),
"direction": _opts(["in"], {"in": "In", "out": "Out", "any": "Both"}),
"protocol": _opts(["TCP"], {"any": "any", "TCP": "TCP", "UDP": "UDP"}),
"gateway": _opts([""], {"": "None", "WAN_GW": "WAN_GW - 203.0.113.1"}),
"categories": _opts(["c1", "c2"], {"c1": "one", "c2": "two", "c3": "three"}),
"tcpflags1": _opts([], {"syn": "SYN", "ack": "ACK"}),
"source_net": "bogons", "destination_net": "203.0.113.9", "destination_port": "8080-8443", "source_port": "",
"description": "opnexus-b4-probe", "statetimeout": None, "log": 1,
"audit": {"created": {"username": "", "time": "", "description": ""}, "userdata": {"note": ""}},
}
def _body(route) -> dict:
return json.loads(route.calls.last.request.content)
# ── toggle_rule ────────────────────────────────────────────────────────────
@pytest.mark.asyncio
@pytest.mark.parametrize("enabled, tail, answer", [(False, "0", "Disabled"), (True, "1", "Enabled")])
async def test_toggle_rule_sends_the_explicit_value_and_returns_the_answer(host, respx_mock, enabled, tail, answer):
route = respx_mock.post(f"{BASE}/toggleRule/{RULE}/{tail}").mock(
return_value=httpx.Response(200, json={"result": answer, "changed": True}))
assert await c.toggle_rule(host, RULE, enabled) == {"result": answer, "changed": True}
assert route.call_count == 1 and not route.calls.last.request.content # never the flipping form, no body
@pytest.mark.asyncio
async def test_toggle_rule_is_idempotent_an_unchanged_answer_is_success(host, respx_mock):
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(
return_value=httpx.Response(200, json={"result": "Disabled", "changed": False}))
assert (await c.toggle_rule(host, RULE, False))["changed"] is False
@pytest.mark.asyncio
@pytest.mark.parametrize("bad", [0, 1, "0", "1", None, "false"])
async def test_toggle_rule_refuses_anything_but_a_bool_without_a_request(host, respx_mock, bad):
route = respx_mock.post(url__regex=r".*/toggleRule/.*").mock(return_value=httpx.Response(200, json={}))
with pytest.raises(TypeError):
await c.toggle_rule(host, RULE, bad)
assert route.call_count == 0
@pytest.mark.asyncio
async def test_toggle_rule_answer_for_the_other_state_is_not_accepted(host, respx_mock):
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(return_value=httpx.Response(200, json={"result": "Enabled", "changed": True}))
with pytest.raises(c.OpnsenseAPIError) as info:
await c.toggle_rule(host, RULE, False)
assert not isinstance(info.value, c.OpnsenseAmbiguousError)
@pytest.mark.asyncio
@pytest.mark.parametrize("answer", [{"result": "failed"}, {"result": "failed", "changed": False}])
async def test_toggle_rule_unknown_uuid_is_a_plain_refusal(host, respx_mock, answer):
"""Live: an unknown or legacy UUID answers HTTP 200 {"result": "failed"} (no "changed" key)."""
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(return_value=httpx.Response(200, json=answer))
with pytest.raises(c.OpnsenseAPIError) as info:
await c.toggle_rule(host, RULE, False)
assert type(info.value) is c.OpnsenseAPIError
@pytest.mark.asyncio
async def test_toggle_rule_failure_classes(host, respx_mock):
route = respx_mock.post(f"{BASE}/toggleRule/{RULE}/0")
route.mock(return_value=httpx.Response(403, json={"status": 403, "message": "Forbidden"}))
with pytest.raises(c.OpnsenseForbiddenError):
await c.toggle_rule(host, RULE, False)
route.mock(return_value=httpx.Response(500, text="boom"))
with pytest.raises(c.OpnsenseAmbiguousError): # the change may have been saved
await c.toggle_rule(host, RULE, False)
route.mock(side_effect=httpx.ReadTimeout("slow"))
with pytest.raises(c.OpnsenseAmbiguousError):
await c.toggle_rule(host, RULE, False)
route.mock(side_effect=httpx.ConnectError("down")) # never sent: a plain error, not an unknown outcome
with pytest.raises(c.OpnsenseAPIError) as info:
await c.toggle_rule(host, RULE, False)
assert not isinstance(info.value, c.OpnsenseAmbiguousError)
@pytest.mark.asyncio
@pytest.mark.parametrize("bad", ["../x", "a/b", "a?b", "", "x" * 65, "a b"])
async def test_rule_references_cannot_leave_the_url_path(host, respx_mock, bad):
route = respx_mock.route().mock(return_value=httpx.Response(200, json={}))
with pytest.raises(c.OpnsenseAPIError):
await c.toggle_rule(host, bad, False)
with pytest.raises(c.OpnsenseAPIError):
await c.get_rule(host, bad)
assert route.call_count == 0
# ── get_rule ───────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_get_rule_returns_the_inner_record(host, respx_mock):
route = respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json={"rule": live_rule()}))
rule = await c.get_rule(host, RULE)
assert route.call_count == 1 and rule["description"] == "opnexus-b4-probe" and rule["interface"]["lan"]["selected"] == 1
@pytest.mark.asyncio
@pytest.mark.parametrize("answer", [[], {}])
async def test_get_rule_empty_answer_means_not_found(host, respx_mock, answer):
"""Live: unknown, legacy (hex label) and automatic UUIDs answer HTTP 200 with a bare `[]`."""
respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json=answer))
with pytest.raises(c.OpnsenseNotFoundError):
await c.get_rule(host, RULE)
@pytest.mark.asyncio
async def test_get_rule_failure_classes(host, respx_mock):
route = respx_mock.get(f"{BASE}/getRule/{RULE}")
route.mock(return_value=httpx.Response(200, json={"rule": "nonsense"}))
with pytest.raises(c.OpnsenseAPIError) as info:
await c.get_rule(host, RULE)
assert not isinstance(info.value, c.OpnsenseNotFoundError) # an unreadable answer is not proof that the rule is gone
route.mock(return_value=httpx.Response(403, text="no"))
with pytest.raises(c.OpnsenseForbiddenError):
await c.get_rule(host, RULE)
route.mock(return_value=httpx.Response(502, text="bad gateway"))
with pytest.raises(c.OpnsenseAmbiguousError):
await c.get_rule(host, RULE)
# ── rule_write_payload (getRule -> addRule) ───────────────────────────────
def test_rule_write_payload_reduces_selects_and_drops_read_only_fields():
payload = c.rule_write_payload(live_rule())
assert payload["interface"] == "lan,wan" # multi-interface: comma separated
assert payload["categories"] == "c1,c2"
assert payload["action"] == "pass" and payload["protocol"] == "TCP" and payload["direction"] == "in"
assert payload["gateway"] == "" # the "None" option has the key ""
assert payload["tcpflags1"] == "" # nothing selected
assert payload["statetimeout"] == "" and payload["log"] == "1" and payload["source_port"] == ""
assert payload["enabled"] == "1" and payload["destination_port"] == "8080-8443"
assert not set(payload) & {"uuid", "sequence", "sort_order", "prio_group", "audit"}
assert all(isinstance(v, str) for v in payload.values())
def test_rule_write_payload_empty_category_list_becomes_an_empty_string():
"""Live: a rule without categories answers `"categories": []` (PHP's empty array), not an option dict."""
rule = live_rule() | {"categories": []}
assert c.rule_write_payload(rule)["categories"] == ""
def test_rule_write_payload_does_not_mutate_its_input():
rule = live_rule()
before = json.dumps(rule, sort_keys=True)
c.rule_write_payload(rule)
assert json.dumps(rule, sort_keys=True) == before
@pytest.mark.asyncio
async def test_a_copy_is_added_with_exactly_the_payload_and_stays_disabled(host, respx_mock):
"""The restore path: getRule -> payload -> addRule(enabled=0). The body must carry the payload unchanged."""
respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json={"rule": live_rule()}))
add = respx_mock.post(f"{BASE}/addRule").mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": OTHER}))
payload = c.rule_write_payload(await c.get_rule(host, RULE)) | {"enabled": "0"}
assert (await c.add_rule(host, payload))["uuid"] == OTHER
sent = _body(add)["rule"]
assert sent == payload and sent["enabled"] == "0" and "sequence" not in sent and "uuid" not in sent
# ── add / set / del / move / apply: request shape and classification ─────
@pytest.mark.asyncio
async def test_add_rule_validation_failure_is_a_plain_refusal(host, respx_mock):
respx_mock.post(f"{BASE}/addRule").mock(return_value=httpx.Response(
200, json={"result": "failed", "validations": {"rule.destination_port": "Please specify a valid portnumber"}}))
with pytest.raises(c.OpnsenseAPIError) as info:
await c.add_rule(host, {"destination_port": "1,2"})
assert type(info.value) is c.OpnsenseAPIError and "destination_port" in str(info.value)
@pytest.mark.asyncio
@pytest.mark.parametrize("answer", ["deleted", "not found"])
async def test_del_rule_accepts_deleted_and_not_found(host, respx_mock, answer):
route = respx_mock.post(f"{BASE}/delRule/{RULE}").mock(return_value=httpx.Response(200, json={"result": answer}))
assert (await c.del_rule(host, RULE))["result"] == answer and route.call_count == 1
@pytest.mark.asyncio
async def test_del_rule_failed_answer_and_5xx(host, respx_mock):
route = respx_mock.post(f"{BASE}/delRule/{RULE}")
route.mock(return_value=httpx.Response(200, json={"result": "failed"}))
with pytest.raises(c.OpnsenseAPIError):
await c.del_rule(host, RULE)
route.mock(return_value=httpx.Response(500, text="x"))
with pytest.raises(c.OpnsenseAmbiguousError):
await c.del_rule(host, RULE)
@pytest.mark.asyncio
async def test_move_rule_before_url_and_answer(host, respx_mock):
route = respx_mock.post(f"{BASE}/moveRuleBefore/{RULE}/{OTHER}").mock(return_value=httpx.Response(200, json={"status": "ok"}))
assert await c.move_rule_before(host, RULE, OTHER) == {"status": "ok"} and route.call_count == 1
@pytest.mark.asyncio
async def test_move_rule_before_refusal_is_http_500_and_therefore_ambiguous(host, respx_mock):
"""Live: unknown UUID, the same UUID twice and rules of different prio_groups all answer HTTP 500
{"errorMessage": "Unexpected error, check log for details"}. Callers must read the position back."""
respx_mock.post(f"{BASE}/moveRuleBefore/{RULE}/{OTHER}").mock(return_value=httpx.Response(
500, json={"errorMessage": "Unexpected error, check log for details", "errorTitle": "Filter", "errorLevel": "warning"}))
with pytest.raises(c.OpnsenseAmbiguousError):
await c.move_rule_before(host, RULE, OTHER)
@pytest.mark.asyncio
async def test_apply_url_and_answer(host, respx_mock):
route = respx_mock.post(f"{BASE}/apply").mock(return_value=httpx.Response(200, json={"status": "OK\n\n"}))
assert (await c.apply(host))["status"].strip() == "OK" and route.call_count == 1
+994
View File
@@ -0,0 +1,994 @@
"""Rule cleanup (1.31, B4 part 2): preview, deploy, confirm, revert/sweeper, delete and restore against a stateful
fake firewall (tests/rule_cleanup_fake.py) and the test database. Numbers refer to DESIGN-1.31 §6. Roles and API
tokens are checked in the router (package 6), not here."""
import asyncio
import uuid as uuid_lib
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
import httpx
import pytest
import pytest_asyncio
from fastapi import HTTPException
from sqlalchemy import select, update
from app.config import settings
from app.database import SessionLocal
from app.models.audit import AuditEvent
from app.models.governance import GovernanceSettings
from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
from app.models.rule_hit import RuleHit
from app.models.rule_history import LiveRuleState, RuleChangeEvent
from app.services import rule_cleanup_service as svc
from app.services import rule_history_service as rhs
from app.services.rule_analysis import context as ctx_mod
from tests.rule_cleanup_fake import (
AUTO, MGMT_SOURCE, R_ALIAS, R_BASE, R_BLOCK, R_CARP, R_DUP, R_LEGACY, R_MGMT, R_TAIL, R_UNUSED, FakeFirewall,
default_rules, rule,
)
from tests.test_rules import _create_host
FW1, FW2 = "https://203.0.113.57", "https://203.0.113.58"
UNUSED_SEEDED = (R_UNUSED, R_BLOCK, R_MGMT, R_CARP, R_ALIAS, R_TAIL)
@pytest.fixture(autouse=True)
def _fresh_cache():
ctx_mod.clear_cache()
yield
ctx_mod.clear_cache()
def _now():
return datetime.now(timezone.utc)
async def _settings(sources=(MGMT_SOURCE,), max_rules=10, min_days=7, ports=()):
async with SessionLocal() as db:
row = await db.get(GovernanceSettings, 1)
if row is None:
row = GovernanceSettings(id=1, operators_may_deploy=True)
db.add(row)
row.cleanup_management_sources = list(sources)
row.cleanup_management_ports = list(ports)
row.cleanup_max_rules, row.cleanup_delete_min_days = max_rules, min_days
await db.commit()
async def _seed_hits(host_id, uuids, *, days=100, hit_days=None, sampled_minutes=10):
async with SessionLocal() as db:
now = _now()
for u in uuids:
db.add(RuleHit(host_id=host_id, rule_uuid=u.lower(), description=u, enabled=True,
first_seen_at=now - timedelta(days=days), last_packets=0, last_bytes=0,
last_sampled_at=now - timedelta(minutes=sampled_minutes),
last_hit_at=None if hit_days is None else now - timedelta(days=hit_days)))
await db.commit()
async def call(host_id, fn, *args, **kw):
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, host_id)
return await fn(db, host, *args, **kw)
async def refused(host_id, fn, *args, **kw) -> dict:
with pytest.raises(HTTPException) as info:
await call(host_id, fn, *args, **kw)
return {"status": info.value.status_code, **info.value.detail}
async def preview(host_id, uuids, actor="admin", **kw):
kw.setdefault("kind", "disable")
return await call(host_id, svc.preview, actor=actor, uuids=uuids, **kw)
async def deploy(host_id, preview_id, actor="admin", name=None):
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, host_id)
return await svc.deploy(db, host, uuid_lib.UUID(preview_id), actor=actor, confirm_name=name or host.name)
async def _items(op_id):
async with SessionLocal() as db:
return {i.rule_uuid: i for i in (await db.scalars(select(RuleCleanupItem).where(
RuleCleanupItem.operation_id == uuid_lib.UUID(str(op_id))))).all()}
async def _op(op_id):
async with SessionLocal() as db:
return await db.get(RuleCleanupOperation, uuid_lib.UUID(str(op_id)))
async def _backdate(op_id, **fields):
async with SessionLocal() as db:
await db.execute(update(RuleCleanupOperation).where(RuleCleanupOperation.id == uuid_lib.UUID(str(op_id))).values(**fields))
await db.commit()
async def _sweep():
async with SessionLocal() as db:
return await svc.sweep_expired_cleanups(db)
@pytest_asyncio.fixture
async def env(client, respx_mock):
log: list = []
fw = FakeFirewall(FW1, log=log).mount(respx_mock)
host_id = uuid_lib.UUID((await _create_host(client, "fw-a", FW1))["id"])
await _settings()
await _seed_hits(host_id, UNUSED_SEEDED)
fw.calls.clear()
log.clear()
return SimpleNamespace(fw=fw, host_id=host_id, log=log)
@pytest_asyncio.fixture
async def cluster(client, respx_mock):
log: list = []
backup = FakeFirewall(FW2, role="BACKUP", pfsync="lan", peer="10.20.30.2", lan_addr="10.20.30.3",
wan_addr="203.0.113.58", log=log).mount(respx_mock)
master = FakeFirewall(FW1, role="MASTER", pfsync="lan", peer="10.20.30.3", log=log).mount(respx_mock)
a = uuid_lib.UUID((await _create_host(client, "fw-a", FW1))["id"])
b = uuid_lib.UUID((await _create_host(client, "fw-b", FW2))["id"])
async with SessionLocal() as db:
await db.execute(update(OpnsenseHost).where(OpnsenseHost.id.in_([a, b])).values(cluster_name="ha-test"))
await db.commit()
await _settings()
await _seed_hits(a, UNUSED_SEEDED)
await _seed_hits(b, UNUSED_SEEDED)
for f in (master, backup):
f.calls.clear()
log.clear()
return SimpleNamespace(master=master, backup=backup, a=a, b=b, log=log)
def _writes(log):
return [(base, path) for base, method, path in log if method == "POST"]
# ── Preview ───────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_preview_recomputes_from_the_live_firewall_and_stores_a_bound_token(env):
result = await preview(env.host_id, [R_UNUSED, R_DUP])
plan = result["plan"]
items = {i["uuid"]: i for i in plan["nodes"][0]["items"]}
assert set(items) == {R_UNUSED, R_DUP}
assert items[R_UNUSED]["level"] == "sure" and items[R_UNUSED]["protection"]["result"] == "false"
assert items[R_DUP]["cover_uuid"] == R_BASE and items[R_DUP]["cover_fp"]
assert env.fw.writes() == [] # nothing is written by a preview
assert ("GET", "/firewall/filter/searchRule") in env.fw.calls # read live, not from a cache or the request
op = await _op(result["preview_id"])
assert op.status == "preview" and op.actor == "admin" and len(op.preview_fingerprint) == 64
assert timedelta(minutes=4) < op.expires_at - _now() <= timedelta(minutes=5)
async with SessionLocal() as db:
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "rule_cleanup"))).all()
assert [(a.action, a.status) for a in audit] == [("preview", "requested")]
assert "remote_uuid" not in audit[0].details # a preview never appears as a change in the rule history
# a second preview of the same actor replaces the first
again = await preview(env.host_id, [R_UNUSED])
assert (await _op(result["preview_id"])).status == "cancelled" and (await _op(again["preview_id"])).status == "preview"
@pytest.mark.asyncio
async def test_without_management_sources_everything_is_locked(env):
await _settings(sources=())
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
await _settings(sources=("not-an-address",))
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
@pytest.mark.asyncio
@pytest.mark.parametrize("uuid,reason", [
(R_MGMT, "management"), # allows the management source to the firewall's 443
(R_ALIAS, "management"), # source is a URL table: match unknown = protected (case 18)
(R_CARP, "infrastructure"), # CARP protocol
(R_BASE, "not_candidate"), # nothing says it is removable
])
async def test_rules_that_must_not_be_switched_off_are_refused(env, uuid, reason):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[uuid], acknowledge=[uuid])
assert got["status"] == 409 and got["code"] == "BEREINIGUNG_REGEL_NICHT_AENDERBAR"
assert got["params"]["reason"] == reason and got["params"]["uuid"] == uuid
assert env.fw.writes() == []
@pytest.mark.asyncio
async def test_managed_legacy_and_unknown_confidence_are_refused(env):
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
env.fw.rules.append(rule(R_LEGACY, 900, src="10.20.30.0/24", dport="6000", legacy=True))
await _seed_hits(env.host_id, [R_LEGACY])
async with SessionLocal() as db:
managed = FirewallRule(description="Managed", interface="lan", action="pass", direction="in", protocol="tcp",
source_net="any", destination_net="any", destination_port="8081")
db.add(managed)
await db.flush()
db.add(RuleDeployment(rule_id=managed.id, host_id=env.host_id, status="success", opnsense_rule_uuid=R_UNUSED))
# stale counters (sampled two days ago): "unused" is unknown, never selectable
await db.execute(update(RuleHit).where(RuleHit.rule_uuid == R_TAIL).values(last_sampled_at=_now() - timedelta(days=2)))
await db.commit()
for uuid, reason in ((R_UNUSED, "managed"), (R_LEGACY, "legacy"), (R_TAIL, "confidence")):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[uuid], acknowledge=[uuid])
assert got["params"]["reason"] == reason, (uuid, got)
@pytest.mark.asyncio
async def test_probable_needs_an_explicit_acknowledgement_per_rule(env):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_BLOCK, R_UNUSED])
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT" and got["params"]["uuids"] == R_BLOCK
ok = await preview(env.host_id, [R_BLOCK, R_UNUSED], acknowledge=[R_BLOCK], ack_text_version="1")
item = next(i for i in ok["plan"]["nodes"][0]["items"] if i["uuid"] == R_BLOCK)
assert item["level"] == "probable" and item["needs_ack"] and "UNUSED_BLOCK_RULE" in item["notes"]
op = await _op(ok["preview_id"])
assert op.options["acknowledged"] == [{"uuid": R_BLOCK, "text_version": "1"}]
@pytest.mark.asyncio
async def test_short_window_and_external_change_cap_at_probable(env):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED], unused_days=14)
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
async with SessionLocal() as db:
db.add(RuleChangeEvent(host_id=env.host_id, rule_uuid=R_UNUSED, seq=1, observed_at=_now() - timedelta(days=2),
kind="changed", before={}, after={}, origin="external"))
await db.commit()
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
ok = await preview(env.host_id, [R_UNUSED], acknowledge=[R_UNUSED])
assert "EXTERNAL_CHANGE" in ok["plan"]["nodes"][0]["items"][0]["notes"]
@pytest.mark.asyncio
async def test_rule_limit_and_selection_validation(env):
await _settings(max_rules=2)
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED, R_DUP, R_TAIL])
assert got["status"] == 422 and got["code"] == "BEREINIGUNG_ZU_VIELE_REGELN" and got["params"]["max"] == 2
for uuids in ([], ["../apply"]):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=uuids)
assert got["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG"
got = await refused(env.host_id, svc.preview, actor="admin", kind="wipe", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG"
@pytest.mark.asyncio
async def test_maintenance_open_deployments_and_open_cleanups_block(env):
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
async with SessionLocal() as db:
op = HostOperation(host_id=env.host_id, host_name="fw-a", actor="admin", action="reboot", status="submitted",
expires_at=_now(), fingerprint="x", snapshot={})
db.add(op)
await db.commit()
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
async with SessionLocal() as db:
await db.execute(update(HostOperation).values(status="done"))
managed = FirewallRule(description="x", interface="lan", action="pass", direction="in", protocol="tcp",
source_net="any", destination_net="any")
db.add(managed)
await db.flush()
dep = RuleDeployment(rule_id=managed.id, host_id=env.host_id, status="pending", opnsense_rule_uuid=R_TAIL)
db.add(dep)
await db.commit()
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" and got["params"]["reason"] == "deployment"
async with SessionLocal() as db:
await db.execute(update(RuleDeployment).values(status="success"))
await db.commit()
first = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, first["preview_id"])
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_DUP])
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" and got["params"]["reason"] == "cleanup"
async with SessionLocal() as db:
assert await svc.has_open_cleanup(db, [env.host_id])
# ── Deploy ────────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_deploy_switches_off_samples_before_apply_and_waits_for_confirmation(env):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
env.fw.calls.clear()
result = await deploy(env.host_id, p["preview_id"])
assert result["status"] == "awaiting_confirm" and result["can_confirm"]
writes = env.fw.writes()
assert sorted(writes[:2]) == sorted([("POST", f"/firewall/filter/toggleRule/{R_DUP}/0"),
("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/0")])
assert writes[2:] == [("POST", "/firewall/filter/apply")] # one apply per node, after the toggles
apply_at = env.fw.calls.index(("POST", "/firewall/filter/apply"))
last_toggle = max(i for i, c in enumerate(env.fw.calls) if "toggleRule" in c[1])
assert ("GET", "/firewall/filter_util/ruleStats") in env.fw.calls[last_toggle:apply_at] # sampled before the apply
assert env.fw.get(R_UNUSED)["enabled"] == "0" and env.fw.get(R_DUP)["enabled"] == "0"
op = await _op(p["preview_id"])
window = timedelta(seconds=settings.deploy_confirm_window_seconds)
assert window - timedelta(seconds=10) < op.revert_deadline - _now() <= window
items = await _items(p["preview_id"])
assert {i.state for i in items.values()} == {"disabled"}
assert items[R_UNUSED].fp_before and items[R_UNUSED].row_before["enabled"] == "1"
async with SessionLocal() as db:
audits = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "rule_cleanup", AuditEvent.action == "disable"))).all()
assert {(a.details["remote_uuid"], a.status) for a in audits} == {(R_UNUSED, "accepted"), (R_DUP, "accepted")}
assert all(a.host_id == str(env.host_id) and a.details["preview_fingerprint"] == op.preview_fingerprint for a in audits)
@pytest.mark.asyncio
async def test_rule_history_shows_the_cleanup_as_opnexus(env):
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, env.host_id)
await rhs.observe_rows(db, host, {"rows": env.fw.rules}) # baseline
p = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, p["preview_id"])
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, env.host_id)
await rhs.observe_rows(db, host, {"rows": env.fw.rules})
event = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.rule_uuid == R_UNUSED))).one()
assert event.kind == "disabled" and event.origin == "opnexus" and event.audit_id is not None
@pytest.mark.asyncio
@pytest.mark.parametrize("change", ["edited", "moved", "cover_edited", "hit"])
async def test_deploy_refuses_a_stale_preview_without_writing(env, change):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
if change == "edited":
env.fw.get(R_UNUSED)["destination_port"] = "8082"
elif change == "moved":
env.fw.get(R_UNUSED)["sort_order"] = "400000.0000301"
elif change == "cover_edited":
env.fw.get(R_BASE)["description"] = "base changed"
else: # a hit since the preview: the fresh sample at deploy sees the counter grow
async with SessionLocal() as db:
await db.execute(update(RuleHit).where(RuleHit.rule_uuid == R_UNUSED).values(last_hit_at=_now()))
await db.commit()
env.fw.calls.clear()
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(p["preview_id"]), actor="admin", confirm_name="fw-a")
expected = R_DUP if change == "cover_edited" else R_UNUSED
if change == "hit":
assert got["code"] == "BEREINIGUNG_REGEL_NICHT_AENDERBAR" and got["params"]["uuid"] == R_UNUSED
else:
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET" and got["params"]["uuids"] == expected
assert env.fw.writes() == []
assert (await _op(p["preview_id"])).status == "expired"
@pytest.mark.asyncio
async def test_preview_token_is_single_use_bound_to_actor_and_name(env):
p = await preview(env.host_id, [R_UNUSED])
pid = uuid_lib.UUID(p["preview_id"])
got = await refused(env.host_id, svc.deploy, pid, actor="someone-else", confirm_name="fw-a")
assert got["status"] == 404
got = await refused(env.host_id, svc.deploy, pid, actor="admin", confirm_name="FW-A")
assert got["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
await deploy(env.host_id, p["preview_id"])
got = await refused(env.host_id, svc.deploy, pid, actor="admin", confirm_name="fw-a")
assert got["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
# an expired preview
await call(env.host_id, svc.revert_now, pid)
q = await preview(env.host_id, [R_DUP])
await _backdate(q["preview_id"], expires_at=_now() - timedelta(seconds=1))
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(q["preview_id"]), actor="admin", confirm_name="fw-a")
assert got["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
@pytest.mark.asyncio
async def test_refused_toggle_stops_writing_but_applies_what_is_switched(env):
uuids = [R_DUP, R_UNUSED, R_BLOCK, R_TAIL]
p = await preview(env.host_id, uuids, acknowledge=[R_BLOCK])
env.fw.fail[("toggle", R_BLOCK)] = httpx.Response(200, json={"result": "failed"})
env.fw.calls.clear()
result = await deploy(env.host_id, p["preview_id"])
states = {i.rule_uuid: i.state for i in (await _items(p["preview_id"])).values()}
assert states == {R_DUP: "disabled", R_UNUSED: "disabled", R_BLOCK: "failed", R_TAIL: "queued"}
assert ("POST", f"/firewall/filter/toggleRule/{R_TAIL}/0") not in env.fw.calls # nothing after the error
assert env.fw.writes()[-1] == ("POST", "/firewall/filter/apply") # but the switched ones are applied
assert result["status"] == "partial" and not result["can_confirm"] and result["can_revert"]
assert (await _op(p["preview_id"])).revert_deadline > _now()
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
assert got["code"] == "BEREINIGUNG_NICHT_BESTAETIGBAR"
@pytest.mark.asyncio
@pytest.mark.parametrize("applied,readable,state", [(True, True, "disabled"), (False, True, "failed"), (False, False, "unknown")])
async def test_unclear_toggle_is_settled_by_reading(env, applied, readable, state):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
items_order = [i["uuid"] for i in p["plan"]["nodes"][0]["items"]]
first = items_order[0]
env.fw.fail[("toggle", first)] = httpx.ReadTimeout
env.fw.fail[("toggle_applies", first)] = applied
if not readable:
env.fw.fail[("get", first)] = httpx.ConnectError
result = await deploy(env.host_id, p["preview_id"])
items = await _items(p["preview_id"])
assert items[first].state == state
# no further write after an unclear answer (never attempted: "skipped" once nothing changed at all)
assert items[items_order[1]].state == ("skipped" if state == "failed" else "queued")
expect_apply = state in ("disabled", "unknown")
assert (("POST", "/firewall/filter/apply") in env.fw.calls) is expect_apply
assert result["status"] == {"disabled": "partial", "failed": "failed", "unknown": "partial"}[state]
assert not result["can_confirm"]
@pytest.mark.asyncio
async def test_failed_apply_keeps_the_deadline_and_the_sweeper_reverts(env):
p = await preview(env.host_id, [R_UNUSED])
env.fw.fail[("apply", "")] = [httpx.Response(500, json={}), None]
result = await deploy(env.host_id, p["preview_id"])
assert result["status"] == "apply_failed" and result["revert_deadline"]
await _sweep()
assert (await _op(p["preview_id"])).status == "apply_failed" # deadline not reached yet
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
env.fw.calls.clear()
await _sweep()
assert (await _op(p["preview_id"])).status == "reverted"
assert env.fw.writes() == [("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/1"), ("POST", "/firewall/filter/apply")]
assert env.fw.get(R_UNUSED)["enabled"] == "1"
# ── Confirm ───────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different(env):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
await deploy(env.host_id, p["preview_id"])
pid = uuid_lib.UUID(p["preview_id"])
env.fw.down = True
got = await refused(env.host_id, svc.confirm, pid)
assert got["status"] == 502 and got["code"] == "BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR"
assert (await _op(pid)).status == "awaiting_confirm" # stays open: the sweeper reverts it
env.fw.down = False
env.fw.get(R_DUP)["enabled"] = "1" # someone switched it back on in the GUI
got = await refused(env.host_id, svc.confirm, pid)
assert got["code"] == "BEREINIGUNG_LIVESTAND_ABWEICHEND" and got["params"]["uuids"] == R_DUP
env.fw.get(R_DUP)["enabled"] = "0"
env.fw.calls.clear()
result = await call(env.host_id, svc.confirm, pid)
assert result["status"] == "confirmed" and {i["state"] for i in result["items"]} == {"confirmed"}
assert env.fw.writes() == [] # confirming only reads
op = await _op(pid)
assert op.revert_deadline is None and op.confirmed_at is not None
assert await _sweep() == []
@pytest.mark.asyncio
async def test_confirm_after_the_deadline_is_refused_and_the_sweeper_reverts(env):
p = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, p["preview_id"])
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
assert got["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
assert len(await _sweep()) == 1
assert env.fw.get(R_UNUSED)["enabled"] == "1"
assert {i.state for i in (await _items(p["preview_id"])).values()} == {"reverted"}
async with SessionLocal() as db:
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "revert"))).one()
assert audit.actor == "system:rollback" and audit.details["remote_uuid"] == R_UNUSED
@pytest.mark.asyncio
async def test_revert_now_and_failed_revert_is_not_retried_automatically(env):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
await deploy(env.host_id, p["preview_id"])
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
result = await call(env.host_id, svc.revert_now, uuid_lib.UUID(p["preview_id"]))
assert result["status"] == "revert_failed" and result["can_retry"]
states = {i["rule_uuid"]: i["state"] for i in result["items"]}
assert states == {R_UNUSED: "reverted", R_DUP: "revert_failed"} # the others are still reverted
assert env.fw.get(R_UNUSED)["enabled"] == "1" and env.fw.writes()[-1] == ("POST", "/firewall/filter/apply")
env.fw.calls.clear()
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(minutes=5))
await _sweep()
assert env.fw.writes() == [] # never retried by the sweeper
del env.fw.fail[("toggle", R_DUP)]
result = await call(env.host_id, svc.retry, uuid_lib.UUID(p["preview_id"]))
assert result["status"] == "reverted" and env.fw.get(R_DUP)["enabled"] == "1"
# ── Sweeper and two backends ──────────────────────────────────────────────
@pytest.mark.asyncio
async def test_sweeper_skips_a_live_run_and_takes_over_an_abandoned_one(env):
p = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, p["preview_id"])
# simulate a backend that died mid-run: running, item half-way
async with SessionLocal() as db:
await db.execute(update(RuleCleanupOperation).values(status="running", heartbeat_at=_now(),
revert_deadline=_now() + timedelta(minutes=10)))
await db.execute(update(RuleCleanupItem).values(state="disabling"))
await db.commit()
env.fw.calls.clear()
assert await _sweep() == [] # (b) fresh heartbeat: someone is still working on it
assert env.fw.writes() == []
await _backdate(p["preview_id"], heartbeat_at=_now() - timedelta(hours=1))
assert await _sweep() == [p["preview_id"]] # (c) stale heartbeat: taken over and reverted
assert env.fw.writes() == [("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/1"), ("POST", "/firewall/filter/apply")]
assert (await _op(p["preview_id"])).status == "reverted"
@pytest.mark.asyncio
async def test_two_deploys_of_one_preview_write_once(env):
p = await preview(env.host_id, [R_UNUSED, R_DUP])
outcomes = await asyncio.gather(deploy(env.host_id, p["preview_id"]), deploy(env.host_id, p["preview_id"]),
return_exceptions=True)
errors = [o for o in outcomes if isinstance(o, HTTPException)]
assert len(errors) == 1 and errors[0].detail["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
assert sum(1 for c in env.fw.calls if "toggleRule" in c[1]) == 2 # (a) each rule exactly once
@pytest.mark.asyncio
async def test_second_operation_on_the_same_rule_is_refused(env):
a = await preview(env.host_id, [R_UNUSED], actor="admin")
b = await preview(env.host_id, [R_UNUSED], actor="admin2")
await deploy(env.host_id, a["preview_id"])
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(b["preview_id"]), actor="admin2", confirm_name="fw-a")
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" # (d)
# and the partial unique index holds even without the service's checks
async with SessionLocal() as db:
op = await db.get(RuleCleanupOperation, uuid_lib.UUID(b["preview_id"]))
db.add(RuleCleanupItem(operation_id=op.id, host_id=env.host_id, rule_uuid=R_UNUSED, state="queued", description=""))
with pytest.raises(Exception):
await db.commit()
@pytest.mark.asyncio
async def test_confirm_and_sweeper_at_the_deadline_decide_once(env):
p = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, p["preview_id"])
await _backdate(p["preview_id"], revert_deadline=_now() + timedelta(milliseconds=300))
async def late_confirm():
await asyncio.sleep(0.4)
return await call(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
async def late_sweep():
await asyncio.sleep(0.4)
return await _sweep()
outcomes = await asyncio.gather(late_confirm(), late_sweep(), return_exceptions=True)
op = await _op(p["preview_id"])
assert op.status == "reverted" # (e) the deadline passed: confirm refused, exactly one revert
assert isinstance(outcomes[0], HTTPException) and outcomes[0].detail["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
assert sum(1 for c in env.fw.calls if c[1].endswith(f"{R_UNUSED}/1")) == 1
# ── Cluster ───────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_cluster_writes_backup_first_and_reverts_master_first(cluster):
p = await preview(cluster.a, [R_UNUSED])
assert [n["role"] for n in p["plan"]["nodes"]] == ["BACKUP", "MASTER"]
await deploy(cluster.a, p["preview_id"])
writes = _writes(cluster.log)
assert writes == [(FW2, f"/firewall/filter/toggleRule/{R_UNUSED}/0"), (FW2, "/firewall/filter/apply"),
(FW1, f"/firewall/filter/toggleRule/{R_UNUSED}/0"), (FW1, "/firewall/filter/apply")]
items = await _items(p["preview_id"])
assert len(items) == 1 # (same rule_uuid on both nodes: keyed by uuid here)
async with SessionLocal() as db:
roles = {i.host_id: i.carp_role for i in (await db.scalars(select(RuleCleanupItem))).all()}
assert roles == {cluster.a: "MASTER", cluster.b: "BACKUP"}
cluster.log.clear()
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
await _sweep()
assert _writes(cluster.log) == [(FW1, f"/firewall/filter/toggleRule/{R_UNUSED}/1"), (FW1, "/firewall/filter/apply"),
(FW2, f"/firewall/filter/toggleRule/{R_UNUSED}/1"), (FW2, "/firewall/filter/apply")]
@pytest.mark.asyncio
async def test_cluster_master_failure_leaves_backup_switched_and_unconfirmable(cluster):
p = await preview(cluster.a, [R_UNUSED])
cluster.master.fail[("toggle", R_UNUSED)] = httpx.Response(200, json={"result": "failed"})
result = await deploy(cluster.a, p["preview_id"])
assert result["status"] == "partial" and not result["can_confirm"]
assert cluster.backup.get(R_UNUSED)["enabled"] == "0" and cluster.master.get(R_UNUSED)["enabled"] == "1"
@pytest.mark.asyncio
async def test_cluster_refuses_missing_uuid_offline_peer_and_two_masters(cluster):
cluster.backup.rules = [r for r in cluster.backup.rules if r["uuid"] != R_UNUSED]
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG" and got["params"]["reason"] == "uuid_missing"
cluster.backup.rules = default_rules()
cluster.backup.role = "MASTER"
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG" and got["params"]["reason"] == "roles"
cluster.backup.role = "BACKUP"
cluster.backup.down = True
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["status"] == 502 and got["params"]["host"] == "fw-b"
@pytest.mark.asyncio
async def test_cluster_usage_is_judged_over_all_nodes(cluster):
# case 16: the backup saw no hit, the master did: not unused
async with SessionLocal() as db:
await db.execute(update(RuleHit).where(RuleHit.host_id == cluster.a, RuleHit.rule_uuid == R_UNUSED)
.values(last_hit_at=_now() - timedelta(days=1)))
await db.commit()
got = await refused(cluster.b, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
assert got["params"]["reason"] == "not_candidate"
# no usage data on one node: at best probable
async with SessionLocal() as db:
await db.execute(RuleHit.__table__.delete().where(RuleHit.host_id == cluster.a, RuleHit.rule_uuid == R_TAIL))
await db.commit()
got = await refused(cluster.b, svc.preview, actor="admin", kind="disable", uuids=[R_TAIL])
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
@pytest.mark.asyncio
async def test_cluster_pfsync_interface_rules_are_infrastructure(cluster):
# "any" from the LAN net on the pfsync interface (LAN) would also carry pfsync
for fw in (cluster.master, cluster.backup):
fw.rules.append(rule("aaaaaaaa-0000-4000-8000-000000000900", 900, proto="any", src="10.20.30.0/24", description="lan any"))
await _seed_hits(cluster.a, ["aaaaaaaa-0000-4000-8000-000000000900"])
await _seed_hits(cluster.b, ["aaaaaaaa-0000-4000-8000-000000000900"])
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=["aaaaaaaa-0000-4000-8000-000000000900"])
assert got["params"]["reason"] == "infrastructure"
# ── Delete and restore ────────────────────────────────────────────────────
async def _confirmed_disable(host_id, uuids, *, days_ago=8):
p = await preview(host_id, uuids)
await deploy(host_id, p["preview_id"])
await call(host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=days_ago))
return p["preview_id"]
@pytest.mark.asyncio
async def test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change(env):
await _confirmed_disable(env.host_id, [R_UNUSED], days_ago=2)
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT" and got["params"]["days"] == 5
async with SessionLocal() as db:
await db.execute(update(RuleCleanupOperation).values(confirmed_at=_now() - timedelta(days=8)))
await db.commit()
ok = await preview(env.host_id, [R_UNUSED], kind="delete")
assert ok["plan"]["nodes"][0]["items"][0]["basis"]["type"] == "confirmed"
env.fw.get(R_UNUSED)["destination_port"] = "8099"
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET"
env.fw.get(R_UNUSED)["destination_port"] = "8081"
async with SessionLocal() as db:
db.add(RuleChangeEvent(host_id=env.host_id, rule_uuid=R_UNUSED, seq=5, observed_at=_now() - timedelta(days=1),
kind="changed", before={}, after={}, origin="external"))
await db.commit()
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET"
env.fw.get(R_DUP)["enabled"] = "1"
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_DUP])
assert got["params"]["reason"] == "not_candidate" # an enabled rule is never deleted
@pytest.mark.asyncio
async def test_delete_commits_the_snapshot_before_delrule(env, monkeypatch):
from app.services import opnsense_client as client_mod
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
seen: list = []
real_del = client_mod.del_rule
async def spy(host, uuid): # what another session sees in the database at the moment delRule goes out
async with SessionLocal() as db:
item = (await db.scalars(select(RuleCleanupItem).where(
RuleCleanupItem.operation_id == uuid_lib.UUID(p["preview_id"])))).one()
seen.append((item.state, bool(item.snapshot), item.anchor_next_uuid))
return await real_del(host, uuid)
monkeypatch.setattr(client_mod, "del_rule", spy)
result = await deploy(env.host_id, p["preview_id"])
assert seen == [("deleting", True, R_BLOCK)] # durable before the remote delete
assert result["status"] == "done"
item = next(iter((await _items(p["preview_id"])).values()))
assert item.state == "deleted" and item.snapshot["rule"]["destination_port"] == "8081"
assert item.snapshot["rule"]["action"] == "pass" and "uuid" not in item.snapshot["rule"]
assert item.prio_group == "400000" and item.anchor_prev_uuid == R_DUP
assert env.fw.get(R_UNUSED) is None and env.fw.writes()[-1] == ("POST", "/firewall/filter/apply")
@pytest.mark.asyncio
@pytest.mark.parametrize("applied,state", [(True, "deleted"), (False, "failed")])
async def test_unclear_delete_is_settled_with_getrule(env, applied, state):
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
env.fw.fail[("delete", R_UNUSED)] = httpx.ReadTimeout
env.fw.fail[("delete_applies", R_UNUSED)] = applied
result = await deploy(env.host_id, p["preview_id"])
assert next(iter((await _items(p["preview_id"])).values())).state == state
assert result["status"] == ("done" if state == "deleted" else "failed")
@pytest.mark.asyncio
async def test_restore_adds_disabled_at_the_anchor_with_a_new_uuid(env):
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
await deploy(env.host_id, p["preview_id"])
deleted = next(iter((await _items(p["preview_id"])).values()))
env.fw.calls.clear()
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["status"] == "done" and result["warnings"] == []
item = result["items"][0]
new = item["restored_uuid"]
assert new and new != R_UNUSED and item["state"] == "restored"
restored = env.fw.get(new)
assert restored["enabled"] == "0" and restored["destination_port"] == "8081"
assert ("POST", f"/firewall/filter/moveRuleBefore/{new}/{R_BLOCK}") in env.fw.calls
order = [r["uuid"] for r in sorted(env.fw.rules, key=lambda r: r["sort_order"]) if not r.get("is_automatic")]
assert order.index(new) == order.index(R_BLOCK) - 1 and order.index(new) == order.index(R_DUP) + 1
async with SessionLocal() as db:
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "restore"))).one()
source = await db.get(RuleCleanupItem, deleted.id)
assert audit.details["remote_uuid"] == new and audit.details["restored_from"] == R_UNUSED
assert source.restored_uuid == new
got = await refused(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert got["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH" # restored once only
# re-enabling the restored rule is an operation with a window
e = await preview(env.host_id, [new], kind="enable")
env.fw.calls.clear()
r = await deploy(env.host_id, e["preview_id"])
assert r["status"] == "awaiting_confirm" and ("POST", f"/firewall/filter/toggleRule/{new}/1") in env.fw.calls
@pytest.mark.asyncio
async def test_restore_without_anchor_warns_and_keeps_the_rule(env):
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
await deploy(env.host_id, p["preview_id"])
deleted = next(iter((await _items(p["preview_id"])).values()))
env.fw.rules = [r for r in env.fw.rules if r["uuid"] != R_BLOCK] # the anchor is gone meanwhile
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["status"] == "done" and result["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
assert result["items"][0]["message"] == "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
assert env.fw.get(result["items"][0]["restored_uuid"])["enabled"] == "0"
@pytest.mark.asyncio
async def test_restore_refused_by_the_firewall_is_failed_with_its_message(env):
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
await deploy(env.host_id, p["preview_id"])
deleted = next(iter((await _items(p["preview_id"])).values()))
env.fw.fail[("add", "")] = httpx.Response(200, json={"result": "failed", "validations": {"rule.source_net": "alias missing"}})
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["status"] == "failed" and "alias missing" in result["items"][0]["message"]
async with SessionLocal() as db:
assert (await db.get(RuleCleanupItem, deleted.id)).restored_uuid is None # can be tried again
@pytest.mark.asyncio
async def test_cluster_delete_restore_and_reenable_follow_the_restore_link(cluster):
p = await preview(cluster.a, [R_UNUSED])
await deploy(cluster.a, p["preview_id"])
await call(cluster.a, svc.confirm, uuid_lib.UUID(p["preview_id"]))
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=8))
d = await preview(cluster.a, [R_UNUSED], kind="delete")
cluster.log.clear()
assert (await deploy(cluster.a, d["preview_id"]))["status"] == "done"
assert [w for w in _writes(cluster.log) if "delRule" in w[1]] == [(FW2, f"/firewall/filter/delRule/{R_UNUSED}"),
(FW1, f"/firewall/filter/delRule/{R_UNUSED}")]
async with SessionLocal() as db:
deleted = (await db.scalars(select(RuleCleanupItem).where(
RuleCleanupItem.operation_id == uuid_lib.UUID(d["preview_id"]), RuleCleanupItem.host_id == cluster.a))).one()
restored = await call(cluster.a, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert restored["status"] == "done" and len(restored["items"]) == 2 # both nodes, backup first
new = {i["host_id"]: i["restored_uuid"] for i in restored["items"]}
assert new[str(cluster.a)] != new[str(cluster.b)] # addRule: a new UUID on every node
assert cluster.backup.get(new[str(cluster.b)])["enabled"] == "0" and cluster.master.get(new[str(cluster.a)])["enabled"] == "0"
# re-enable through the master's UUID: the backup's copy is found through the restore operation
e = await preview(cluster.a, [new[str(cluster.a)]], kind="enable")
assert {n["host_name"]: n["items"][0]["uuid"] for n in e["plan"]["nodes"]} == {"fw-a": new[str(cluster.a)], "fw-b": new[str(cluster.b)]}
cluster.log.clear()
r = await deploy(cluster.a, e["preview_id"])
assert r["status"] == "awaiting_confirm"
assert [w for w in _writes(cluster.log) if "toggleRule" in w[1]] == [
(FW2, f"/firewall/filter/toggleRule/{new[str(cluster.b)]}/1"), (FW1, f"/firewall/filter/toggleRule/{new[str(cluster.a)]}/1")]
@pytest.mark.asyncio
async def test_restore_with_a_refused_move_is_verified_by_reading_and_only_warns(env):
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
await deploy(env.host_id, p["preview_id"])
deleted = next(iter((await _items(p["preview_id"])).values()))
# moveRuleBefore answers HTTP 500 (an Ambiguous error for the client, P3): the position is read back
env.fw.fail[("move", "*")] = httpx.Response(500, json={"errorMessage": "Unexpected error, check log for details"})
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["status"] == "done" and result["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
@pytest.mark.asyncio
async def test_deadline_passing_while_confirm_reads_the_firewall_refuses(env):
import time
p = await preview(env.host_id, [R_UNUSED])
await deploy(env.host_id, p["preview_id"])
await _backdate(p["preview_id"], revert_deadline=_now() + timedelta(milliseconds=300))
env.fw.fail[("search", "")] = lambda: time.sleep(0.6) # a slow firewall: the window closes during the read
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
assert got["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
assert (await _op(p["preview_id"])).status == "awaiting_confirm" # left to the sweeper
@pytest.mark.asyncio
async def test_sweeper_leaves_a_live_run_alone_even_past_its_deadline(env, monkeypatch): # review finding 2
from app.services import opnsense_client as client_mod
p = await preview(env.host_id, [R_UNUSED, R_DUP])
real = client_mod.toggle_rule
swept: list = []
async def toggle(host, uuid, enabled): # a second backend's sweeper starts while the first toggle is on the wire
if not swept:
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
swept.append(asyncio.create_task(_sweep())) # it gets the lock in the deploy's next checkpoint gap
await asyncio.sleep(0.05)
return await real(host, uuid, enabled)
monkeypatch.setattr(client_mod, "toggle_rule", toggle)
result = await deploy(env.host_id, p["preview_id"])
assert await swept[0] == [] # fresh heartbeat: not taken over although the deadline passed
assert result["status"] == "awaiting_confirm"
assert {i.state for i in (await _items(p["preview_id"])).values()} == {"disabled"}
@pytest.mark.asyncio
async def test_a_run_taken_over_by_another_backend_writes_nothing_more(env, monkeypatch): # review finding 2
from app.services import opnsense_client as client_mod
p = await preview(env.host_id, [R_UNUSED, R_DUP])
real = client_mod.toggle_rule
calls: list = []
async def toggle(host, uuid, enabled):
calls.append(uuid)
if len(calls) == 1: # another backend took the operation over while this toggle was on the wire
await _backdate(p["preview_id"], status="reverting")
return await real(host, uuid, enabled)
monkeypatch.setattr(client_mod, "toggle_rule", toggle)
await deploy(env.host_id, p["preview_id"])
assert len(calls) == 1 # the conditional heartbeat before the second toggle found the operation gone
assert (await _op(p["preview_id"])).status == "reverting" # left to the new owner, not overwritten
states = sorted(i.state for i in (await _items(p["preview_id"])).values())
assert states == ["disabled", "queued"]
@pytest.mark.asyncio
async def test_extra_management_ports_protect_rules_on_them(env): # review finding 3
custom = "aaaaaaaa-0000-4000-8000-000000000910"
env.fw.rules.append(rule(custom, 910, interface="wan", src="203.0.113.0/24", dport="8443", description="gui 8443"))
await _seed_hits(env.host_id, [custom])
ok = await preview(env.host_id, [custom]) # 8443 is not one of the fixed ports: not about management
assert ok["plan"]["ports"] == []
await _settings(ports=(8443,))
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[custom])
assert got["params"]["reason"] == "management"
@pytest.mark.asyncio
async def test_cluster_peer_may_confirm_and_read_but_strangers_may_not(cluster, client, respx_mock): # review finding 4
p = await preview(cluster.a, [R_UNUSED])
await deploy(cluster.a, p["preview_id"])
pid = uuid_lib.UUID(p["preview_id"])
assert (await call(cluster.b, svc.get_operation, pid))["id"] == p["preview_id"]
FakeFirewall("https://203.0.113.59").mount(respx_mock)
other = uuid_lib.UUID((await _create_host(client, "fw-c", "https://203.0.113.59"))["id"])
for fn in (svc.get_operation, svc.confirm, svc.revert_now, svc.retry):
assert (await refused(other, fn, pid))["status"] == 404
got = await refused(cluster.b, svc.cancel, pid, actor="admin") # previews/deploys stay bound to the anchor
assert got["status"] == 404
result = await call(cluster.b, svc.confirm, pid)
assert result["status"] == "confirmed"
async def _half_deleted_cluster(cluster):
p = await preview(cluster.a, [R_UNUSED])
await deploy(cluster.a, p["preview_id"])
await call(cluster.a, svc.confirm, uuid_lib.UUID(p["preview_id"]))
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=8))
d = await preview(cluster.a, [R_UNUSED], kind="delete")
cluster.master.fail[("delete", R_UNUSED)] = httpx.Response(200, json={"result": "failed"})
result = await deploy(cluster.a, d["preview_id"])
assert result["status"] == "failed"
assert cluster.backup.get(R_UNUSED) is None and cluster.master.get(R_UNUSED) is not None
del cluster.master.fail[("delete", R_UNUSED)]
return d["preview_id"]
@pytest.mark.asyncio
async def test_half_done_cluster_delete_can_be_finished(cluster): # review finding 5
await _half_deleted_cluster(cluster)
again = await preview(cluster.a, [R_UNUSED], kind="delete")
assert [(n["host_name"], len(n["items"])) for n in again["plan"]["nodes"]] == [("fw-b", 0), ("fw-a", 1)]
cluster.log.clear()
assert (await deploy(cluster.a, again["preview_id"]))["status"] == "done"
assert [w for w in _writes(cluster.log) if "delRule" in w[1]] == [(FW1, f"/firewall/filter/delRule/{R_UNUSED}")]
assert cluster.master.get(R_UNUSED) is None
got = await refused(cluster.a, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED]) # nothing left
assert got["params"]["reason"] == "not_candidate"
@pytest.mark.asyncio
async def test_half_done_cluster_delete_can_be_restored_where_it_was_deleted(cluster): # review finding 5
delete_op = await _half_deleted_cluster(cluster)
async with SessionLocal() as db:
deleted = (await db.scalars(select(RuleCleanupItem).where(
RuleCleanupItem.operation_id == uuid_lib.UUID(delete_op), RuleCleanupItem.state == "deleted"))).one()
result = await call(cluster.a, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["status"] == "done" and [i["host_id"] for i in result["items"]] == [str(cluster.b)]
assert cluster.backup.get(result["items"][0]["restored_uuid"])["enabled"] == "0"
assert cluster.master.get(R_UNUSED)["enabled"] == "0" # the master's copy was never deleted: untouched
@pytest.mark.asyncio
async def test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless(env): # review finding 7
lock_out = "aaaaaaaa-0000-4000-8000-000000000920"
env.fw.rules.append(rule(lock_out, 920, action="block", interface="wan", src="203.0.113.0/24", dport="443",
enabled="0", description="block mgmt"))
unknown = "aaaaaaaa-0000-4000-8000-000000000940" # source is a URL table: match unknown
env.fw.rules.append(rule(unknown, 940, action="block", interface="wan", src="tbl", dport="443", enabled="0",
description="block table"))
harmless = "aaaaaaaa-0000-4000-8000-000000000930"
env.fw.rules.append(rule(harmless, 930, action="block", proto="UDP", dport="5353", enabled="0", description="block mdns"))
async with SessionLocal() as db: # both were switched off by a confirmed cleanup earlier
op = RuleCleanupOperation(host_id=env.host_id, kind="disable", status="confirmed", actor="admin",
expires_at=_now(), preview_fingerprint="x", options={}, confirmed_at=_now())
db.add(op)
await db.flush()
for u in (lock_out, harmless, unknown):
db.add(RuleCleanupItem(operation_id=op.id, host_id=env.host_id, rule_uuid=u, state="confirmed", description=u))
await db.commit()
got = await refused(env.host_id, svc.preview, actor="admin", kind="enable", uuids=[lock_out, harmless, unknown])
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT" and got["params"]["uuids"] == ", ".join(sorted([lock_out, unknown]))
ok = await preview(env.host_id, [lock_out, harmless, unknown], kind="enable", acknowledge=[lock_out, unknown])
items = {i["uuid"]: i for i in ok["plan"]["nodes"][0]["items"]}
assert items[lock_out]["needs_ack"] and items[lock_out]["notes"] == ["ENABLE_MAY_BLOCK_MANAGEMENT"]
assert not items[harmless]["needs_ack"] and items[harmless]["protection"]["result"] == "false"
async def _unknown_restore(env):
"""A restore whose addRule answer was lost: the item stays unknown."""
await _confirmed_disable(env.host_id, [R_UNUSED])
p = await preview(env.host_id, [R_UNUSED], kind="delete")
await deploy(env.host_id, p["preview_id"])
deleted = next(iter((await _items(p["preview_id"])).values()))
snapshot_row = dict(deleted.snapshot["row"])
env.fw.fail[("add", "")] = httpx.ReadTimeout
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
assert result["items"][0]["state"] == "unknown"
del env.fw.fail[("add", "")]
return result, snapshot_row
@pytest.mark.asyncio
async def test_unclear_restore_uses_the_stored_uuid_before_searching_by_content(env): # review finding 10
result, row = await _unknown_restore(env)
# on the firewall: the real copy and a content-identical twin someone made by hand
real, twin = "cccccccc-0000-4000-8000-000000000001", "cccccccc-0000-4000-8000-000000000002"
for u, seq in ((real, 950), (twin, 960)):
env.fw.rules.append({**row, "uuid": u, "sequence": str(seq), "sort_order": f"400000.{seq:07d}"})
async with SessionLocal() as db:
await db.execute(update(RuleCleanupItem).where(RuleCleanupItem.id == uuid_lib.UUID(result["items"][0]["id"]))
.values(restored_uuid=real))
await db.commit()
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"]))
assert done["items"][0]["state"] == "restored" and done["items"][0]["restored_uuid"] == real
@pytest.mark.asyncio
async def test_unclear_restore_with_twins_and_no_stored_uuid_stays_unknown(env): # review finding 10
result, row = await _unknown_restore(env)
for u, seq in (("cccccccc-0000-4000-8000-000000000003", 950), ("cccccccc-0000-4000-8000-000000000004", 960)):
env.fw.rules.append({**row, "uuid": u, "sequence": str(seq), "sort_order": f"400000.{seq:07d}"})
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"]))
assert done["items"][0]["state"] == "unknown" and done["items"][0]["message"] == "BEREINIGUNG_AUSGANG_UNKLAR"
env.fw.rules = [r for r in env.fw.rules if not r["uuid"].startswith("cccccccc")]
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"])) # nothing there: it was never created
assert done["items"][0]["state"] == "failed"
@pytest.mark.asyncio
async def test_an_outdated_acknowledgement_text_is_refused(env): # review finding 11
for version in ("0", "2", ""):
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_BLOCK],
acknowledge=[R_BLOCK], ack_text_version=version)
assert got["status"] == 409 and got["code"] == "BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET", version
assert env.fw.calls == [] # refused before anything is read
ok = await preview(env.host_id, [R_BLOCK], acknowledge=[R_BLOCK], ack_text_version=svc.ACK_TEXT_VERSION)
assert ok["plan"]["acknowledged"] == [R_BLOCK]
+910
View File
@@ -0,0 +1,910 @@
"""Rule cleanup API (1.31, package 6): router, rights, locks in both directions, notification, settings.
Who may do what is enforced by TWO layers (AuthMiddleware via services/api_token_service.py, and the router itself).
The role matrix therefore runs three times: with both layers, with the middleware's rights checks switched off (the
router alone must refuse) and with the router's checks switched off (the middleware alone must refuse). A mutation
of either layer is then caught by one of the variants (tests/mutations_cleanup_1_31_api.json).
Addresses: documentation/private ranges only."""
import uuid as uuid_lib
from datetime import timedelta
import httpx
import pytest
import pytest_asyncio
from sqlalchemy import select, update
from app import auth_middleware
from app.database import SessionLocal
from app.models.audit import AuditEvent
from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
from app.models.rule_deployment import RuleDeployment
from app.routers import rule_cleanup as router_mod
from app.schemas.rule_cleanup import CleanupItemRead, CleanupOperationRead
from app.services import api_token_service, notification_checks
from app.services import rule_cleanup_service as svc
from tests.rule_cleanup_fake import MGMT_SOURCE, R_ALIAS, R_BASE, R_BLOCK, R_DUP, R_MGMT, R_TAIL, R_UNUSED
from tests.test_api_tokens import _bearer, _login, _make
from tests.test_rule_cleanup import ( # noqa: F401 (fixtures are used by name)
FW1, FW2, _backdate, _confirmed_disable, _fresh_cache, _items, _now, _op, _settings, cluster, env, refused,
)
PW = "cleanup-api-test-pw-1"
NOHOST = "00000000-0000-0000-0000-0000000000aa"
NOID = "00000000-0000-0000-0000-0000000000bb"
# ── Principals ────────────────────────────────────────────────────────────
async def _user(client, name: str, role: str):
resp = await client.post("/api/users", json={"username": name, "password": PW, "role": role})
assert resp.status_code in (201, 409), resp.text # the account survives between tests
return await _login(name, PW)
async def _token_of(session, name: str, scope: str):
resp = await session.post("/api/tokens", json={"name": name, "scope": scope, "current_password": PW})
assert resp.status_code == 201, resp.text
return _bearer(resp.json()["token"])
@pytest_asyncio.fixture
async def who(client):
"""Every kind of caller: sessions of an admin, an operator and a read-only account, and tokens of each scope
(an admin's read/deploy/admin, an operator's deploy, a read-only account's read)."""
operator = await _user(client, "cl-operator", "operator")
read_only = await _user(client, "cl-readonly", "read_only")
callers = {
"admin": client, "operator": operator, "read_only": read_only,
"tok_read": _bearer((await _make(client, "read", name="cl-read"))["token"]),
"tok_deploy": _bearer((await _make(client, "deploy", name="cl-deploy"))["token"]),
"tok_admin": _bearer((await _make(client, "admin", name="cl-admin"))["token"]),
"tok_op_deploy": await _token_of(operator, "cl-op-deploy", "deploy"),
"tok_ro_read": await _token_of(read_only, "cl-ro-read", "read"),
}
yield callers
for name, session in callers.items():
if name != "admin":
await session.aclose()
async def _governance(client, may_deploy: bool):
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": may_deploy})).status_code == 200
# ── The role matrix ───────────────────────────────────────────────────────
ROUTES = {
"list": ("GET", "", None),
"get": ("GET", "/{op}", None),
"preview": ("POST", "/preview", {"kind": "disable", "uuids": ["a"]}),
"cancel": ("POST", "/{op}/cancel", None),
"deploy": ("POST", "/{op}/deploy", {"confirm_name": "x"}),
"confirm": ("POST", "/{op}/confirm", None),
"revert": ("POST", "/{op}/revert", None),
"retry": ("POST", "/{op}/retry", None),
"restore": ("POST", "/items/{op}/restore", {"confirm_name": "x"}),
}
READ_ONLY_CODES = {"READ_ONLY_ACCESS", "API_TOKEN_SCOPE_FORBIDDEN"}
def verdict(route: str, caller: str, may_deploy: bool):
"""None: the caller passes the gate (the unknown host then answers 404). Otherwise the acceptable 403 codes."""
if route in ("list", "get"):
return None
if caller in ("read_only", "tok_ro_read"):
return READ_ONLY_CODES
if caller == "tok_read":
return {"API_TOKEN_SCOPE_FORBIDDEN"}
if route in ("preview", "cancel"):
return None
if route == "confirm": # a human in a browser session; no token, whatever its scope
return None if caller in ("admin", "operator") else {"API_TOKEN_SCOPE_FORBIDDEN"}
if route in ("deploy", "retry", "restore"): # admin only, even though operators_may_deploy is on
if caller in ("admin", "tok_admin"):
return None
return {"admin": None, "operator": {"ADMIN_REQUIRED"}, "tok_deploy": {"API_TOKEN_SCOPE_FORBIDDEN"},
"tok_op_deploy": {"API_TOKEN_SCOPE_FORBIDDEN", "ADMIN_REQUIRED"}}[caller]
assert route == "revert" # an admin, or an operator while operators_may_deploy is on
if caller in ("admin", "tok_deploy", "tok_admin"):
return None
return None if may_deploy else {"DEPLOY_NEEDS_ADMIN"}
async def _gate(session, route: str):
method, path, body = ROUTES[route]
url = f"/api/hosts/{NOHOST}/rule-cleanup" + path.format(op=NOID)
resp = await session.request(method, url, **({"json": body} if body is not None else {}))
code = (resp.json().get("detail") or {}).get("code") if resp.headers.get("content-type", "").startswith("application/json") else None
return resp.status_code, code
@pytest.mark.asyncio
@pytest.mark.parametrize("may_deploy", [True, False])
@pytest.mark.parametrize("variant", ["both", "router_only", "middleware_only"])
async def test_every_route_for_every_caller(client, who, monkeypatch, variant, may_deploy):
await _governance(client, may_deploy)
try:
if variant == "router_only": # switch the middleware's rights checks off: the router must still refuse
monkeypatch.setattr(api_token_service, "denies", lambda *a, **k: False)
monkeypatch.setattr(api_token_service, "operator_denies", lambda *a, **k: False)
monkeypatch.setattr(api_token_service, "is_rollout", lambda *a, **k: False)
class _Everything: # the read-only gate of the middleware lets this request through
def __contains__(self, _):
return True
monkeypatch.setattr(auth_middleware, "_READ_ONLY_ALLOWED_WRITES", _Everything())
if variant == "middleware_only": # switch the router's checks off: the middleware must still refuse
async def _nothing_async(*a, **k):
return None
for name in ("_writer", "_admin_only", "_session_only"):
monkeypatch.setattr(router_mod, name, lambda *a, **k: None)
monkeypatch.setattr(router_mod, "_deployer", _nothing_async)
problems = []
for route in ROUTES:
for caller, session in who.items():
want = verdict(route, caller, may_deploy)
got = await _gate(session, route)
ok = got == (404, "HOST_NICHT_GEFUNDEN") if want is None else (got[0] == 403 and got[1] in want)
if not ok:
problems.append((route, caller, got, want))
assert not problems, problems
finally:
await _governance(client, True)
@pytest.mark.parametrize("method,path,session_only,admin_only,rollout", [
("POST", "/api/hosts/h/rule-cleanup/preview", False, False, False),
("POST", "/api/hosts/h/rule-cleanup/o/cancel", False, False, False),
("POST", "/api/hosts/h/rule-cleanup/o/confirm", True, False, False),
("POST", "/api/hosts/h/rule-cleanup/o/confirm/", True, False, False),
("POST", "/api/hosts/h/rule-cleanup/o/deploy", False, True, True),
("POST", "/api/hosts/h/rule-cleanup/o/deploy/", False, True, False),
("POST", "/api/hosts/h/rule-cleanup/o/revert", False, False, True),
("POST", "/api/hosts/h/rule-cleanup/o/retry", False, True, True),
("POST", "/api/hosts/h/rule-cleanup/items/i/restore", False, True, False),
("GET", "/api/hosts/h/rule-cleanup", False, False, False),
("GET", "/api/hosts/h/rule-cleanup/o", False, False, False),
])
def test_pattern_table(method, path, session_only, admin_only, rollout):
denied_for_read = method == "POST"
assert api_token_service.denies("read", method, path) is denied_for_read
# a token below scope admin: admin-only routes and confirm are closed
assert api_token_service.denies("deploy", method, path) is (session_only or admin_only)
# an admin-scope token reaches everything except what only a session may do
assert api_token_service.denies("admin", method, path) is session_only
assert api_token_service.operator_denies(method, path) is admin_only
assert api_token_service.is_rollout(method, path) is rollout
def test_governance_settings_are_admin_writes_for_operators_and_deploy_tokens():
assert api_token_service.operator_denies("PUT", "/api/settings/governance") is True
assert api_token_service.denies("deploy", "PUT", "/api/settings/governance") is True
assert api_token_service.denies("admin", "PUT", "/api/settings/governance") is False
# ── A full run over the API: shapes, audit, input forms ──────────────────
def _base(env_or_id) -> str:
return f"/api/hosts/{getattr(env_or_id, 'host_id', env_or_id)}/rule-cleanup"
async def _audit(**filters):
async with SessionLocal() as db:
rows = (await db.scalars(select(AuditEvent).where(*[getattr(AuditEvent, k) == v for k, v in filters.items()])
.order_by(AuditEvent.created_at))).all()
return rows
async def _open(session, env, uuids=(R_UNUSED, R_DUP), name="fw-a"):
"""Preview + deploy over the API; the run waits for its confirmation."""
pv = await session.post(_base(env) + "/preview", json={"kind": "disable", "uuids": list(uuids)})
assert pv.status_code == 200, pv.text
op = await session.post(f"{_base(env)}/{pv.json()['preview_id']}/deploy", json={"confirm_name": name})
assert op.status_code == 200, op.text
return op.json()
@pytest.mark.asyncio
async def test_preview_deploy_confirm_over_the_api_with_shapes_and_audit(client, env):
pv = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED, R_DUP]})
assert pv.status_code == 200, pv.text
body = pv.json()
assert {"preview_id", "expires_at", "kind", "plan", "max_rules", "window_seconds", "operation"} <= set(body)
assert body["max_rules"] == 10 and body["kind"] == "disable" and body["window_seconds"] > 0
assert {i["uuid"] for i in body["plan"]["nodes"][0]["items"]} == {R_UNUSED, R_DUP}
operation = body["operation"]
assert set(operation) == set(CleanupOperationRead.model_fields) # the UI gets exactly the documented fields
assert operation["status"] == "preview" and operation["plan"] and operation["items"] == []
assert operation["actor"] == "admin" and operation["host_name"] == "fw-a"
# The middleware does not log /preview; the service writes the one entry (and it is no history change).
assert [(a.action, a.status) for a in await _audit(kind="rule_cleanup")] == [("preview", "requested")]
assert await _audit(kind="host", action="preview") == []
assert env.fw.writes() == []
pid = body["preview_id"]
wrong = await client.post(f"{_base(env)}/{pid}/deploy", json={"confirm_name": "fw-x"})
assert wrong.status_code == 409 and wrong.json()["detail"]["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
deployed = await client.post(f"{_base(env)}/{pid}/deploy", json={"confirm_name": "fw-a"})
assert deployed.status_code == 200, deployed.text
op = deployed.json()
assert set(op) == set(CleanupOperationRead.model_fields) and all(set(i) == set(CleanupItemRead.model_fields) for i in op["items"])
assert op["status"] == "awaiting_confirm" and op["revert_deadline"] and op["plan"] is None
assert (op["can_confirm"], op["can_revert"], op["can_retry"]) == (True, True, False)
assert {(i["rule_uuid"], i["state"], i["host_name"]) for i in op["items"]} == {(R_UNUSED, "disabled", "fw-a"), (R_DUP, "disabled", "fw-a")}
assert env.fw.get(R_UNUSED)["enabled"] == "0"
# Audit: one request pair from the middleware (kind host), one entry per rule from the service - no duplicates.
# the wrong-name attempt and the real one: a request entry and an outcome each, from the middleware only
assert sorted(a.status for a in await _audit(kind="host", action="deploy")) == ["accepted", "rejected", "requested", "requested"]
assert [a.status for a in await _audit(kind="rule_cleanup", action="disable")] == ["accepted", "accepted"]
assert await _audit(kind="rule_cleanup", action="deploy") == [] and await _audit(kind="host", action="disable") == []
listing = (await client.get(_base(env))).json()
assert [o["id"] for o in listing] == [pid] and listing[0]["status"] == "awaiting_confirm"
one = (await client.get(f"{_base(env)}/{pid}")).json()
assert one == listing[0]
confirmed = await client.post(f"{_base(env)}/{pid}/confirm")
assert confirmed.status_code == 200, confirmed.text
c = confirmed.json()
assert c["status"] == "confirmed" and c["revert_deadline"] is None and {i["state"] for i in c["items"]} == {"confirmed"}
assert (c["can_confirm"], c["can_revert"], c["can_retry"]) == (False, False, False)
assert [a.status for a in await _audit(kind="rule_cleanup", action="confirm")] == ["confirmed", "confirmed"]
assert sorted(a.status for a in await _audit(kind="host", action="confirm")) == ["accepted", "requested"]
again = await client.post(f"{_base(env)}/{pid}/confirm")
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
missing = await client.get(f"{_base(env)}/{NOID}")
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
assert (await client.get(f"/api/hosts/{NOHOST}/rule-cleanup")).status_code == 404
@pytest.mark.asyncio
async def test_preview_input_forms_and_validation(client, env):
url = _base(env) + "/preview"
# items[] with per-rule kind/mode and acknowledgement; a "probable" rule needs the acknowledgement
plain = {"items": [{"uuid": R_UNUSED, "mode": "disable"}, {"uuid": R_BLOCK, "kind": "disable"}]}
got = await client.post(url, json=plain)
assert got.status_code == 409 and got.json()["detail"]["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
assert R_BLOCK in got.json()["detail"]["params"]["uuids"]
ack = {"items": [{"uuid": R_UNUSED, "mode": "disable"}, {"uuid": R_BLOCK, "kind": "disable", "ack": True}]}
done = await client.post(url, json=ack)
assert done.status_code == 200, done.text
assert {i["uuid"] for i in done.json()["plan"]["nodes"][0]["items"]} == {R_UNUSED, R_BLOCK}
assert done.json()["plan"]["acknowledged"] == [R_BLOCK]
# the old form: kind + uuids + acknowledge
old = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED, R_BLOCK], "acknowledge": [R_BLOCK]})
assert old.status_code == 200 and (await _op(done.json()["preview_id"])).status == "cancelled" # one preview per actor
# bad input
for bad in ({"uuids": [R_UNUSED]}, {"items": []}):
r = await client.post(url, json=bad)
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG", bad
for bad in ({"kind": "disable", "uuids": []}, {"kind": "disable", "uuids": ["../x"]}):
r = await client.post(url, json=bad)
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG", bad
mixed = {"items": [{"uuid": R_UNUSED, "kind": "disable"}, {"uuid": R_DUP, "kind": "delete"}]}
assert (await client.post(url, json=mixed)).status_code == 422
assert (await client.post(url, json={"kind": "restore", "uuids": [R_UNUSED]})).status_code == 422
assert (await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED], "unused_days": 0})).status_code == 422
assert (await client.post(f"/api/hosts/{NOHOST}/rule-cleanup/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).status_code == 404
# the settings reach the service: the limit and the fail-closed lock without management sources
assert (await client.put("/api/settings/governance", json={"cleanup_max_rules": 1})).status_code == 200
many = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED, R_DUP]})
assert many.status_code == 422 and many.json()["detail"]["code"] == "BEREINIGUNG_ZU_VIELE_REGELN" and many.json()["detail"]["params"]["max"] == 1
assert (await client.put("/api/settings/governance", json={"cleanup_management_sources": []})).status_code == 200
locked = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED]})
assert locked.status_code == 409 and locked.json()["detail"]["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
@pytest.mark.asyncio
async def test_a_preview_belongs_to_its_actor_and_can_be_cancelled(client, env, who):
operator = who["operator"]
mine = (await operator.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).json()
assert mine["operation"]["actor"] == "cl-operator"
# the admin neither sees it as his own nor deploys it; the operator cannot deploy at all
stranger = await client.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
assert stranger.status_code == 404 and stranger.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
refused_op = await operator.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
assert refused_op.status_code == 403 and refused_op.json()["detail"]["code"] == "ADMIN_REQUIRED"
assert (await client.post(f"{_base(env)}/{mine['preview_id']}/cancel")).status_code == 404 # not the admin's
cancelled = await operator.post(f"{_base(env)}/{mine['preview_id']}/cancel")
assert cancelled.status_code == 200 and cancelled.json()["status"] == "cancelled"
# the admin's refused attempt (not his preview) and the operator's real cancel: a request entry and an outcome each
assert sorted(a.status for a in await _audit(kind="host", action="cancel")) == ["accepted", "rejected", "requested", "requested"]
again = await operator.post(f"{_base(env)}/{mine['preview_id']}/cancel")
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
gone = await client.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
assert gone.status_code == 404 # (not the admin's preview)
assert env.fw.writes() == []
# an operator prepares, the admin prepares his own and executes; the operator may confirm afterwards
own = (await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).json()
deployed = await client.post(f"{_base(env)}/{own['preview_id']}/deploy", json={"confirm_name": "fw-a"})
assert deployed.status_code == 200
confirmed = await operator.post(f"{_base(env)}/{own['preview_id']}/confirm")
assert confirmed.status_code == 200 and confirmed.json()["status"] == "confirmed"
@pytest.mark.asyncio
async def test_confirm_never_works_with_a_token_and_changes_nothing(client, env, who):
op = await _open(client, env)
for name in ("tok_admin", "tok_deploy", "tok_op_deploy"):
r = await who[name].post(f"{_base(env)}/{op['id']}/confirm")
assert r.status_code == 403 and r.json()["detail"]["code"] == "API_TOKEN_SCOPE_FORBIDDEN", name
state = await _op(op["id"])
assert state.status == "awaiting_confirm" and state.revert_deadline is not None
assert {i.state for i in (await _items(op["id"])).values()} == {"disabled"}
# the same token may read the run, and the browser session confirms
assert (await who["tok_read"].get(f"{_base(env)}/{op['id']}")).json()["can_confirm"] is True
assert (await client.post(f"{_base(env)}/{op['id']}/confirm")).json()["status"] == "confirmed"
denied = await _audit(kind="host", action="confirm", status="rejected")
assert not denied # the middleware refused before the router: no request entry for a refused token confirm
@pytest.mark.asyncio
async def test_revert_is_for_admins_and_for_operators_only_while_they_may_deploy(client, env, who):
op = await _open(client, env)
url = f"{_base(env)}/{op['id']}/revert"
await _governance(client, False)
try:
assert (await who["operator"].post(url)).status_code == 403
assert (await who["tok_op_deploy"].post(url)).status_code == 403
assert (await who["read_only"].post(url)).status_code == 403
assert (await _op(op["id"])).status == "awaiting_confirm" and env.fw.get(R_UNUSED)["enabled"] == "0"
finally:
await _governance(client, True)
reverted = await who["operator"].post(url)
assert reverted.status_code == 200, reverted.text
r = reverted.json()
assert r["status"] == "reverted" and {i["state"] for i in r["items"]} == {"reverted"} and not r["can_revert"]
assert env.fw.get(R_UNUSED)["enabled"] == "1" and env.fw.get(R_DUP)["enabled"] == "1"
again = await client.post(url)
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
@pytest.mark.asyncio
async def test_retry_after_a_failed_revert_is_admin_only(client, env, who):
op = await _open(client, env)
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
failed = await client.post(f"{_base(env)}/{op['id']}/revert")
assert failed.status_code == 200 and failed.json()["status"] == "revert_failed" and failed.json()["can_retry"] is True
del env.fw.fail[("toggle", R_DUP)]
url = f"{_base(env)}/{op['id']}/retry"
for name, code in (("operator", "ADMIN_REQUIRED"), ("tok_deploy", "API_TOKEN_SCOPE_FORBIDDEN"), ("read_only", "READ_ONLY_ACCESS")):
r = await who[name].post(url)
assert r.status_code == 403 and r.json()["detail"]["code"] == code, name
assert (await _op(op["id"])).status == "revert_failed"
done = await who["tok_admin"].post(url)
assert done.status_code == 200 and done.json()["status"] == "reverted" and env.fw.get(R_DUP)["enabled"] == "1"
nothing = await client.post(url)
assert nothing.status_code == 409 and nothing.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
@pytest.mark.asyncio
async def test_delete_and_restore_over_the_api(client, env, who):
await _confirmed_disable(env.host_id, [R_UNUSED])
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
assert pv["operation"]["kind"] == "delete"
done = await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})
assert done.status_code == 200, done.text
op = done.json()
assert op["status"] == "done" and op["kind"] == "delete" and (op["can_confirm"], op["can_revert"], op["can_retry"]) == (False,) * 3
item = op["items"][0]
assert item["state"] == "deleted" and item["restorable"] is True and item["has_snapshot"] is True
assert env.fw.get(R_UNUSED) is None
assert [a.status for a in await _audit(kind="rule_cleanup", action="delete")] == ["accepted"]
url = f"{_base(env)}/items/{item['id']}/restore"
for name, code in (("operator", "ADMIN_REQUIRED"), ("tok_deploy", "API_TOKEN_SCOPE_FORBIDDEN"), ("read_only", "READ_ONLY_ACCESS")):
r = await who[name].post(url, json={"confirm_name": "fw-a"})
assert r.status_code == 403 and r.json()["detail"]["code"] == code, name
wrong = await client.post(url, json={"confirm_name": "nope"})
assert wrong.status_code == 409 and wrong.json()["detail"]["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
assert env.fw.get(R_UNUSED) is None
restored = await who["tok_admin"].post(url, json={"confirm_name": "fw-a"})
assert restored.status_code == 200, restored.text
r = restored.json()
assert r["kind"] == "restore" and r["status"] == "done" and r["warnings"] == []
new = r["items"][0]["restored_uuid"]
assert new and r["items"][0]["state"] == "restored" and env.fw.get(new)["enabled"] == "0"
listed = {o["id"]: o for o in (await client.get(_base(env))).json()}
source = next(i for i in listed[op["id"]]["items"] if i["id"] == item["id"])
assert source["restorable"] is False and source["restored_uuid"] == new
assert [a.status for a in await _audit(kind="rule_cleanup", action="restore")] == ["accepted"]
again = await client.post(url, json={"confirm_name": "fw-a"})
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
nothing = await client.post(f"{_base(env)}/{op['id']}/retry")
assert nothing.status_code == 409 # nothing unclear in a finished delete
@pytest.mark.asyncio
async def test_restore_reports_a_lost_position_as_a_warning(client, env):
await _confirmed_disable(env.host_id, [R_UNUSED])
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
op = (await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})).json()
env.fw.rules = [r for r in env.fw.rules if r["uuid"] != R_BLOCK] # the anchor is gone
r = await client.post(f"{_base(env)}/items/{op['items'][0]['id']}/restore", json={"confirm_name": "fw-a"})
assert r.status_code == 200, r.text
assert r.json()["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
assert r.json()["items"][0]["message"] == "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
missing = await client.post(f"{_base(env)}/items/{NOID}/restore", json={"confirm_name": "fw-a"})
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
# ── Settings ──────────────────────────────────────────────────────────────
GOV = "/api/settings/governance"
@pytest.mark.asyncio
async def test_management_sources_are_validated_normalised_and_audited(client):
put = await client.put(GOV, json={"cleanup_management_sources": [" 203.0.113.10 ", "10.20.30.5/24", "203.0.113.10", "2001:db8::1"],
"cleanup_max_rules": 25, "cleanup_delete_min_days": 30})
assert put.status_code == 200, put.text
got = put.json()
assert got["cleanup_management_sources"] == ["203.0.113.10", "10.20.30.0/24", "2001:db8::1"]
assert got["cleanup_management_sources_configured"] is True and got["cleanup_max_rules"] == 25 and got["cleanup_delete_min_days"] == 30
assert (await client.get(GOV)).json() == got
audit = await _audit(kind="settings", action="cleanup_settings")
assert len(audit) == 1 and audit[0].actor == "admin"
changes = audit[0].details["changes"]
assert changes["cleanup_management_sources"]["new"] == got["cleanup_management_sources"]
assert changes["cleanup_max_rules"] == {"old": 10, "new": 25}
# an unchanged value writes no further entry; a partial update leaves the rest alone
assert (await client.put(GOV, json={"cleanup_max_rules": 25, "operators_may_deploy": True})).json()["cleanup_delete_min_days"] == 30
assert len(await _audit(kind="settings", action="cleanup_settings")) == 1
bad_sources = [["fw.example.test"], ["203.0.113.999"], ["0.0.0.0/0"], ["::/0"], ["203.0.113.1", ""], [f"10.20.30.{i}" for i in range(21)], ["10.20.30.0/33"]]
for sources in bad_sources:
r = await client.put(GOV, json={"cleanup_management_sources": sources})
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_EINSTELLUNG_UNGUELTIG", sources
assert r.json()["detail"]["params"]["field"] == "cleanup_management_sources"
for field, value in (("cleanup_max_rules", 0), ("cleanup_max_rules", 26), ("cleanup_delete_min_days", 0), ("cleanup_delete_min_days", 366)):
r = await client.put(GOV, json={field: value})
assert r.status_code == 422 and r.json()["detail"]["params"]["field"] == field, (field, value)
assert (await client.get(GOV)).json() == got # nothing of the refused requests was stored
assert len(await _audit(kind="settings", action="cleanup_settings")) == 1
assert (await client.put(GOV, json={"cleanup_management_sources": [f"10.20.30.{i}" for i in range(20)]})).status_code == 200
@pytest.mark.asyncio
async def test_management_ports_are_validated_normalised_audited_and_hidden(client, who): # review finding 3
put = await client.put(GOV, json={"cleanup_management_ports": [8443, "2222", 8443]})
assert put.status_code == 200, put.text
assert put.json()["cleanup_management_ports"] == [2222, 8443] and put.json()["cleanup_management_ports_configured"] is True
audit = await _audit(kind="settings", action="cleanup_settings")
assert audit[-1].details["changes"]["cleanup_management_ports"] == {"old": [], "new": [2222, 8443]}
for ports in ([0], [65536], ["ssh"], [True], [-1], list(range(1, 22))):
r = await client.put(GOV, json={"cleanup_management_ports": ports})
assert r.status_code == 422 and r.json()["detail"]["params"]["field"] == "cleanup_management_ports", ports
assert (await client.get(GOV)).json()["cleanup_management_ports"] == [2222, 8443]
for name in ("operator", "read_only"):
seen = (await who[name].get(GOV)).json()
assert seen["cleanup_management_ports"] == [] and seen["cleanup_management_ports_configured"] is True, name
r = await who[name].put(GOV, json={"cleanup_management_ports": [1]})
assert r.status_code == 403, name
assert (await client.put(GOV, json={"cleanup_management_ports": []})).json()["cleanup_management_ports_configured"] is False
@pytest.mark.asyncio
@pytest.mark.parametrize("variant", ["both", "router_only"])
async def test_only_an_admin_writes_the_cleanup_settings(client, who, monkeypatch, variant):
await client.put(GOV, json={"cleanup_management_sources": ["203.0.113.10"]})
if variant == "router_only":
monkeypatch.setattr(api_token_service, "denies", lambda *a, **k: False)
monkeypatch.setattr(api_token_service, "operator_denies", lambda *a, **k: False)
# router_only: an admin-owned deploy/read token IS an admin account: only the token layer (switched off) stops it
attempts = {"operator": {"ADMIN_REQUIRED"}, "tok_op_deploy": {"API_TOKEN_SCOPE_FORBIDDEN", "ADMIN_REQUIRED"},
"read_only": {"READ_ONLY_ACCESS", "ADMIN_REQUIRED"}}
if variant == "both":
attempts.update({"tok_deploy": {"API_TOKEN_SCOPE_FORBIDDEN"}, "tok_read": {"API_TOKEN_SCOPE_FORBIDDEN"}})
for name, codes in attempts.items():
r = await who[name].put(GOV, json={"cleanup_management_sources": ["198.51.100.7"], "cleanup_max_rules": 3})
assert r.status_code == 403 and r.json()["detail"]["code"] in codes, (name, r.text)
assert (await client.get(GOV)).json()["cleanup_management_sources"] == ["203.0.113.10"]
assert (await client.get(GOV)).json()["cleanup_max_rules"] == 10
# an admin-scope token acts as the admin and the audit names the token
ok = await who["tok_admin"].put(GOV, json={"cleanup_management_sources": ["198.51.100.7"]})
assert ok.status_code == 200, ok.text
assert [a.actor for a in await _audit(kind="settings", action="cleanup_settings")][-1] == "admin (API: cl-admin)"
@pytest.mark.asyncio
async def test_others_only_see_whether_management_sources_are_set(client, who):
await client.put(GOV, json={"cleanup_management_sources": ["203.0.113.10"]})
assert (await client.get(GOV)).json()["cleanup_management_sources"] == ["203.0.113.10"]
for name in ("operator", "read_only"):
seen = (await who[name].get(GOV)).json()
assert seen["cleanup_management_sources"] == [] and seen["cleanup_management_sources_configured"] is True, name
assert seen["cleanup_max_rules"] == 10 and seen["operators_may_deploy"] is True
# ── Locks in both directions ──────────────────────────────────────────────
async def _central_rule(client, env, **extra):
resp = await client.post("/api/rules", json={
"description": "managed ssh", "interface": "lan", "direction": "in", "protocol": "tcp", "action": "pass",
"source_net": "any", "destination_net": "lan", "destination_port": "22",
"target_host_ids": [str(env.host_id)], **extra})
assert resp.status_code == 201, resp.text
return resp.json()
async def _deployed_row(rule_id, host_id, firewall_uuid, status="success"):
async with SessionLocal() as db:
dep = RuleDeployment(rule_id=uuid_lib.UUID(rule_id), host_id=host_id, status=status, opnsense_rule_uuid=firewall_uuid)
db.add(dep)
await db.commit()
@pytest.mark.asyncio
async def test_an_open_cleanup_blocks_other_writes_to_the_firewall(client, env):
rule = await _central_rule(client, env)
alias = await client.post("/api/aliases", json={"name": "cl_alias", "type": "host", "content": "203.0.113.5",
"target_host_ids": [str(env.host_id)]})
assert alias.status_code == 201, alias.text
hid = env.host_id
await _open(client, env)
writes = list(env.fw.writes())
blocked = [
("POST", f"/api/rules/{rule['id']}/deploy", None),
("POST", f"/api/aliases/{alias.json()['id']}/deploy", None),
("POST", f"/api/hosts/{hid}/snapshots", {"name": "x"}),
("POST", f"/api/hosts/{hid}/services/dhcpd/restart", None),
("POST", f"/api/hosts/{hid}/maintenance/check-updates", None),
("POST", f"/api/hosts/{hid}/network/changes", {"kind": "vlan", "action": "add", "fields": {}, "preview_token": "0" * 64}),
("DELETE", f"/api/hosts/{hid}/snapshots/abc", None),
]
for method, path, body in blocked:
r = await client.request(method, path, **({"json": body} if body is not None else {}))
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", (path, r.text)
assert env.fw.writes() == writes # nothing reached the firewall
# the cleanup's own follow-up steps are not blocked by itself, and a second cleanup is refused with its own code
second = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_BLOCK], "acknowledge": [R_BLOCK]})
assert second.status_code == 409 and second.json()["detail"]["code"] == "BEREINIGUNG_OFFENE_OPERATION"
assert second.json()["detail"]["params"]["reason"] == "cleanup"
op = (await client.get(_base(env))).json()[0]
assert (await client.post(f"{_base(env)}/{op['id']}/revert")).status_code == 200
again = await client.post(f"/api/hosts/{hid}/maintenance/check-updates")
assert again.status_code != 409 or again.json()["detail"]["code"] != "OPEN_CLEANUP_BLOCKS_DEPLOY"
@pytest.mark.asyncio
async def test_the_block_covers_the_cluster_and_the_maintenance_inspection(cluster, client):
from app.services.host_maintenance_service import _shared_change_blockers, ensure_no_maintenance, ensure_no_open_cleanup, peers
pv = await client.post(_base(cluster.a) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})
assert pv.status_code == 200, pv.text
assert (await client.post(f"{_base(cluster.a)}/{pv.json()['preview_id']}/deploy", json={"confirm_name": "fw-a"})).status_code == 200
async with SessionLocal() as db:
a, b = await db.get(OpnsenseHost, cluster.a), await db.get(OpnsenseHost, cluster.b)
for host in (a, b): # the partner node is just as busy
for call in (ensure_no_maintenance, ensure_no_open_cleanup):
with pytest.raises(Exception) as info:
await call(db, [host])
assert info.value.detail["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
blockers = await _shared_change_blockers(db, await peers(db, b))
assert any("Bereinigung" in text for text in blockers)
await ensure_no_maintenance(db, [a], cleanup=False) # the cleanup service's own check does not trip over itself
# deleting or re-addressing either node is refused while it is open
for host_id in (cluster.a, cluster.b):
r = await client.delete(f"/api/hosts/{host_id}")
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", host_id
op = (await client.get(_base(cluster.a))).json()[0]
assert (await client.post(f"{_base(cluster.a)}/{op['id']}/revert")).status_code == 200
async with SessionLocal() as db:
assert await _shared_change_blockers(db, await peers(db, await db.get(OpnsenseHost, cluster.b))) == []
await ensure_no_maintenance(db, [await db.get(OpnsenseHost, cluster.b)])
@pytest.mark.asyncio
async def test_host_delete_and_readdressing_are_refused_while_a_cleanup_is_open(client, env):
hid = env.host_id
await _open(client, env)
gone = await client.delete(f"/api/hosts/{hid}")
assert gone.status_code == 409 and gone.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
for body in ({"base_url": "https://203.0.113.99"}, {"enabled": False}, {"cluster_name": "ha-x"}, {"clear_cluster_name": True}):
r = await client.patch(f"/api/hosts/{hid}", json=body)
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", body
assert (await client.patch(f"/api/hosts/{hid}", json={"name": "fw-a2"})).status_code == 200 # a rename is harmless
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, hid)
assert host.base_url == FW1 and host.enabled and host.name == "fw-a2"
op = (await client.get(_base(env))).json()[0]
assert (await client.post(f"{_base(env)}/{op['id']}/revert")).status_code == 200
assert (await client.delete(f"/api/hosts/{hid}")).status_code == 204 # settled: the host may go (its runs go with it)
async with SessionLocal() as db:
assert (await db.scalars(select(RuleCleanupOperation))).all() == []
@pytest.mark.asyncio
async def test_a_confirmed_cleanup_blocks_nothing(client, env):
op = await _open(client, env)
assert (await client.post(f"{_base(env)}/{op['id']}/confirm")).status_code == 200
async with SessionLocal() as db:
assert not await svc.has_open_cleanup(db, [env.host_id])
r = await client.post(f"/api/hosts/{env.host_id}/maintenance/check-updates")
assert r.status_code != 409 or r.json()["detail"]["code"] != "OPEN_CLEANUP_BLOCKS_DEPLOY"
@pytest.mark.asyncio
async def test_rule_deletion_and_removal_of_targets_wait_for_the_cleanup(client, env):
rule = await _central_rule(client, env, target_host_ids=[])
await _deployed_row(rule["id"], env.host_id, R_TAIL)
token = (await client.get(f"/api/rules/{rule['id']}/preview")).json()["token"]
await _open(client, env)
env.fw.calls.clear()
deleted = await client.delete(f"/api/rules/{rule['id']}")
assert deleted.status_code == 409 and deleted.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
removed = await client.post(f"/api/rules/{rule['id']}/deployments/{env.host_id}/remove", json={"preview_token": token})
assert removed.status_code == 409 and removed.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", removed.text
assert env.fw.writes() == [] and env.fw.get(R_TAIL) is not None
op = (await client.get(_base(env))).json()[0]
await client.post(f"{_base(env)}/{op['id']}/revert")
assert (await client.delete(f"/api/rules/{rule['id']}")).status_code == 204 # the same call works afterwards
assert env.fw.get(R_TAIL) is None
@pytest.mark.asyncio
async def test_hierarchy_rule_deletion_and_reordering_wait_for_the_cleanup(client, env):
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from tests.test_policy_hierarchy import _create_group, _hrule_payload
group = await _create_group(client, "Cleanup-Root")
assert (await client.patch(f"/api/hosts/{env.host_id}", json={"policy_hierarchy_group_id": group["id"]})).status_code == 200
rule = (await client.post("/api/policy-hierarchy/rules", json=_hrule_payload(group["id"], description="guard"))).json()
async with SessionLocal() as db:
db.add(PolicyHierarchyRuleDeployment(policy_hierarchy_id=uuid_lib.UUID(rule["id"]), host_id=env.host_id, status="success",
opnsense_policy_hierarchy_uuid=R_TAIL))
await db.commit()
await _open(client, env)
env.fw.calls.clear()
deleted = await client.delete(f"/api/policy-hierarchy/rules/{rule['id']}")
assert deleted.status_code == 409 and deleted.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", deleted.text
reorder = await client.post(f"/api/policy-hierarchy/groups/{group['id']}/reassert-ordering")
assert reorder.status_code == 409 and reorder.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", reorder.text
assert env.fw.writes() == []
@pytest.mark.asyncio
async def test_removing_a_former_hierarchy_target_waits_for_the_cleanup(client, env):
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from tests.test_policy_hierarchy import _create_group, _hrule_payload
group = await _create_group(client, "Cleanup-Former")
rule = (await client.post("/api/policy-hierarchy/rules", json=_hrule_payload(group["id"], description="former"))).json()
async with SessionLocal() as db: # deployed on the host, which is not in the group (any more): "retained"
db.add(PolicyHierarchyRuleDeployment(policy_hierarchy_id=uuid_lib.UUID(rule["id"]), host_id=env.host_id, status="success",
opnsense_policy_hierarchy_uuid=R_TAIL))
await db.commit()
plan = (await client.get(f"/api/policy-hierarchy/rules/{rule['id']}/preview")).json()
assert [h["can_remove"] for h in plan["hosts"]] == [True]
await _open(client, env)
env.fw.calls.clear()
r = await client.post(f"/api/policy-hierarchy/rules/{rule['id']}/deployments/{env.host_id}/remove", json={"preview_token": plan["token"]})
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
assert env.fw.writes() == [] and env.fw.get(R_TAIL) is not None
@pytest.mark.asyncio
async def test_adopting_a_rule_waits_for_the_cleanup(client, env):
await _open(client, env)
r = await client.post("/api/rules/adopt", json={"host_id": str(env.host_id), "rule_uuid": R_BASE})
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
@pytest.mark.asyncio
async def test_other_open_work_blocks_a_new_cleanup_over_the_api(client, env):
url = _base(env) + "/preview"
body = {"kind": "disable", "uuids": [R_UNUSED]}
async with SessionLocal() as db:
db.add(HostOperation(host_id=env.host_id, host_name="fw-a", actor="admin", action="reboot", status="submitted",
expires_at=_now(), fingerprint="x", snapshot={}))
await db.commit()
r = await client.post(url, json=body)
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
async with SessionLocal() as db:
await db.execute(update(HostOperation).values(status="done"))
await db.commit()
rule = await _central_rule(client, env)
await _deployed_row(rule["id"], env.host_id, R_TAIL, status="pending")
r = await client.post(url, json=body)
assert r.status_code == 409 and r.json()["detail"]["code"] == "BEREINIGUNG_OFFENE_OPERATION" and r.json()["detail"]["params"]["reason"] == "deployment"
async with SessionLocal() as db:
await db.execute(update(RuleDeployment).values(status="success"))
await db.commit()
assert (await client.post(url, json=body)).status_code == 200
assert env.fw.writes() == []
# ── Notification ──────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_failed_reverts_and_unclear_runs_are_reported_with_stable_keys(client, env):
op = await _open(client, env)
async with SessionLocal() as db:
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
# a run that ended half-way (decision needed): one problem for the run
async with SessionLocal() as db:
await db.execute(update(RuleCleanupOperation).where(RuleCleanupOperation.id == uuid_lib.UUID(op["id"])).values(status="partial"))
await db.commit()
problems = await notification_checks.check_revert_failed(db)
key = f"rule_cleanup_op:{op['id']}"
assert set(problems) == {key} and problems[key]["deployment_type"] == "rule_cleanup"
assert problems[key]["host_name"] == "fw-a" and "teilweise" in problems[key]["hinweis"]
# the automatic revert failed: one problem per rule that is still changed (the key is the item, not the run)
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
await client.post(f"{_base(env)}/{op['id']}/revert")
items = await _items(op["id"])
assert (await _op(op["id"])).status == "revert_failed"
async with SessionLocal() as db:
first = await notification_checks.check_revert_failed(db)
second = await notification_checks.check_revert_failed(db)
stuck = f"rule_cleanup:{items[R_DUP].id}"
assert set(first) == {stuck} and set(second) == set(first) # stable: the same problem is not reported twice
assert first[stuck]["entity_id"] == R_DUP and first[stuck]["host_name"] == "fw-a" and "Erneut zurücknehmen" in first[stuck]["hinweis"]
assert first[stuck]["operation_status"] == "revert_failed" and first[stuck]["item_state"] == "revert_failed"
# solved: gone
del env.fw.fail[("toggle", R_DUP)]
assert (await client.post(f"{_base(env)}/{op['id']}/retry")).json()["status"] == "reverted"
async with SessionLocal() as db:
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
@pytest.mark.asyncio
async def test_an_unclear_delete_is_reported_until_it_is_read_again(client, env):
await _confirmed_disable(env.host_id, [R_UNUSED])
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
env.fw.fail[("delete", R_UNUSED)] = httpx.ReadTimeout
def _then_unreadable(kind, rule_uuid): # the firewall stops answering once delRule went out
if kind == "delete":
env.fw.fail[("get", R_UNUSED)] = httpx.ReadTimeout
env.fw.on_call = _then_unreadable
op = (await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})).json()
assert op["status"] == "failed" and op["items"][0]["state"] == "unknown" and op["can_retry"] is True
async with SessionLocal() as db:
problems = await notification_checks.check_revert_failed(db)
key = f"rule_cleanup:{op['items'][0]['id']}"
assert set(problems) == {key} and problems[key]["operation"] == "delete" and "unklar" in problems[key]["hinweis"]
env.fw.on_call = None
del env.fw.fail[("get", R_UNUSED)]
del env.fw.fail[("delete", R_UNUSED)]
assert (await client.post(f"{_base(env)}/{op['id']}/retry")).status_code == 200
async with SessionLocal() as db:
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
# ── MCP and the schema ────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_mcp_stays_read_only_for_the_cleanup(client, env, who):
from app.routers.mcp import TOOLS
names = {t["name"] for t in TOOLS}
assert not [n for n in names if "cleanup" in n and n != "rule_cleanup_candidates"]
assert all(t.get("path", "").startswith("/api/") for t in TOOLS if "path" in t)
op = await _open(client, env)
def call(path):
return {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "api_get", "arguments": {"path": path}}}
async with _bearer((await _make(client, "admin", name="mcp-admin"))["token"]) as api:
for tail in ("/preview", f"/{op['id']}/confirm", f"/{op['id']}/revert", f"/{op['id']}/deploy", f"/{op['id']}/retry"):
result = (await api.post("/api/mcp", json=call(f"/api/hosts/{env.host_id}/rule-cleanup{tail}"))).json()["result"]
assert result["isError"] is True, tail # a GET on a POST route: refused, nothing runs
listing = (await api.post("/api/mcp", json=call(f"/api/hosts/{env.host_id}/rule-cleanup"))).json()["result"]
assert listing["isError"] is False and op["id"] in listing["content"][0]["text"]
assert (await _op(op["id"])).status == "awaiting_confirm"
# a plain POST of a tool name that would write does not exist
async with _bearer((await _make(client, "admin", name="mcp-admin2"))["token"]) as api:
r = (await api.post("/api/mcp", json={"jsonrpc": "2.0", "id": 2, "method": "tools/call",
"params": {"name": "rule_cleanup_deploy", "arguments": {}}})).json()
assert "error" in r or r["result"]["isError"] is True
@pytest.mark.asyncio
async def test_openapi_knows_the_routes_and_their_models(client):
schema = (await client.get("/api/openapi.json")).json()
base = "/api/hosts/{host_id}/rule-cleanup"
paths = schema["paths"]
assert set(paths[base]) == {"get"} and set(paths[base + "/preview"]) == {"post"} and set(paths[base + "/{op_id}"]) == {"get"}
for tail in ("cancel", "deploy", "confirm", "revert", "retry"):
assert set(paths[f"{base}/{{op_id}}/{tail}"]) == {"post"}, tail
assert set(paths[base + "/items/{item_id}/restore"]) == {"post"}
assert {"CleanupOperationRead", "CleanupItemRead", "CleanupPreviewRead", "PreviewRequest", "ConfirmName"} <= set(schema["components"]["schemas"])
ops = schema["components"]["schemas"]["CleanupOperationRead"]["properties"]
assert {"can_confirm", "can_revert", "can_retry", "plan", "warnings", "items", "revert_deadline", "message"} <= set(ops)
assert {"restorable", "state", "host_name", "has_snapshot"} <= set(schema["components"]["schemas"]["CleanupItemRead"]["properties"])
def test_every_new_error_code_has_a_text_in_both_languages():
import json
import pathlib
from app.errors import ErrorCode
root = pathlib.Path(__file__).resolve().parents[2] / "frontend" / "lib" / "i18n"
if not root.is_dir():
pytest.skip("the frontend is not part of this image (scripts/check_i18n.py checks the same from the repo root)")
for lang in ("de", "en"):
errors = json.loads((root / f"{lang}.json").read_text(encoding="utf-8"))["errors"]
for code in ErrorCode:
if code.name.startswith("BEREINIGUNG_") or code.name == "OPEN_CLEANUP_BLOCKS_DEPLOY":
assert errors.get(code.value), (lang, code.value)
@pytest.mark.asyncio
async def test_a_refused_operator_attempt_is_audited_and_writes_nothing(client, env, who):
op = await _open(client, env)
r = await who["operator"].post(f"{_base(env)}/{op['id']}/retry")
assert r.status_code == 403
denied = await _audit(kind="host", action="retry", status="denied")
assert [a.actor for a in denied] == ["cl-operator"] and denied[0].host_id == str(env.host_id)
assert (await _op(op["id"])).status == "awaiting_confirm"
def _leaks(value) -> bool:
"""Anything that names a management source or a firewall's base URL."""
import json as _json
text = _json.dumps(value)
return MGMT_SOURCE in text or "203.0.113.57" in text
@pytest.mark.asyncio
async def test_sources_and_firewall_addresses_are_for_admins_only(client, env, who): # review finding 8
mine = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})
assert mine.json()["plan"]["sources"] == [f"{MGMT_SOURCE}/32"] and mine.json()["plan"]["nodes"][0]["base_url"]
theirs = await who["operator"].post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_DUP]})
assert theirs.status_code == 200, theirs.text
body = theirs.json()
assert "sources" not in body["plan"] and body["plan"]["sources_configured"] is True and not _leaks(body)
assert body["plan"]["nodes"][0]["items"][0]["protection"]["result"] == "false"
for name in ("operator", "read_only", "tok_read", "tok_deploy", "tok_ro_read"):
listed = await who[name].get(_base(env))
assert listed.status_code == 200 and listed.json() and not _leaks(listed.json()), name
one = await who[name].get(f"{_base(env)}/{mine.json()['preview_id']}")
assert one.status_code == 200 and not _leaks(one.json()), name
audit = await who[name].get("/api/audit", params={"kind": "rule_cleanup"})
assert audit.status_code == 200 and audit.json()["items"] and not _leaks(audit.json()), name
assert {i["details"]["admin_only"] for i in audit.json()["items"]} == {"redacted"}, name
for name in ("admin", "tok_admin"):
assert _leaks((await who[name].get(_base(env))).json()), name
audit = (await who[name].get("/api/audit", params={"kind": "rule_cleanup"})).json()
assert all(isinstance(i["details"]["admin_only"], dict) for i in audit["items"]), name
# Review finding 9: undeploying a central object and removing a former target write to the firewall too.
_UNDEPLOY_SITES = [
("app.services.alias_deploy_service", "undeploy_alias"), ("app.services.openvpn_deploy_service", "undeploy_instance"),
("app.services.wireguard_deploy_service", "undeploy_tunnel"), ("app.services.port_forward_deploy_service", "undeploy_port_forward"),
("app.services.outbound_nat_deploy_service", "undeploy_outbound_nat"), ("app.services.one_to_one_deploy_service", "undeploy_one_to_one"),
("app.services.ipsec_deploy_service", "undeploy_tunnel"),
] + [(f"app.routers.{r}", "remove_previous_target") for r in
("aliases", "port_forward", "one_to_one", "outbound_nat", "wireguard", "openvpn", "ipsec")]
@pytest.mark.parametrize("module,name", _UNDEPLOY_SITES)
def test_every_undeploy_and_target_removal_checks_for_an_open_cleanup_first(module, name):
import importlib
import inspect
source = inspect.getsource(getattr(importlib.import_module(module), name))
lock = min(i for i in (source.find("ensure_no_open_cleanup_on("), source.find("ensure_no_open_cleanup(")) if i >= 0)
first_write = source.find("opnsense_client.")
first_write = min(i for i in (first_write, source.find("client.del_"), source.find("_client.del")) if i >= 0)
assert 0 <= lock < first_write, (module, name)
@pytest.mark.asyncio
async def test_alias_delete_and_target_removal_wait_for_the_cleanup(client, env):
from app.models.alias_deployment import AliasDeployment
deployed = await client.post("/api/aliases", json={"name": "cl_gone", "type": "host", "content": "203.0.113.5",
"target_host_ids": [str(env.host_id)]})
former = await client.post("/api/aliases", json={"name": "cl_former", "type": "host", "content": "203.0.113.6",
"target_host_ids": []})
assert deployed.status_code == 201 and former.status_code == 201, (deployed.text, former.text)
async with SessionLocal() as db:
for alias in (deployed.json(), former.json()):
db.add(AliasDeployment(alias_id=uuid_lib.UUID(alias["id"]), host_id=env.host_id, status="success",
opnsense_alias_uuid="aaaaaaaa-1111-4111-8111-111111111111", last_payload={"name": alias["name"]}))
await db.commit()
preview = (await client.get(f"/api/aliases/{former.json()['id']}/preview")).json()
assert next(h for h in preview["hosts"] if h["host_id"] == str(env.host_id))["can_remove"], preview
await _open(client, env)
writes = list(env.fw.writes())
r = await client.delete(f"/api/aliases/{deployed.json()['id']}")
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
preview = (await client.get(f"/api/aliases/{former.json()['id']}/preview")).json()
r = await client.post(f"/api/aliases/{former.json()['id']}/deployments/{env.host_id}/remove", json={"preview_token": preview["token"]})
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
assert env.fw.writes() == writes
@@ -0,0 +1,193 @@
"""Pure parts of the rule cleanup (1.31): engine.match_rule, the management-access protection, the infrastructure
exclusion, the confidence caps and the cluster usage merge. No database, no firewall."""
import ipaddress
from datetime import datetime, timedelta, timezone
from app.services import rule_hit_service
from app.services.rule_analysis import cleanup, protection
from app.services.rule_analysis.context import Context, IfInfo
from app.services.rule_analysis.engine import Query, match_rule
from app.services.rule_analysis.model import Endpoint, NormRule
from app.services.rule_cleanup_service import selectable_level
NOW = datetime(2026, 10, 22, 12, 0, tzinfo=timezone.utc)
ip = ipaddress.ip_address
net = ipaddress.ip_network
def ctx(**kw) -> Context:
interfaces = {
"wan": IfInfo("wan", "vtnet0", "WAN", nets=[net("203.0.113.0/24")], addrs=[ip("203.0.113.57")], has_gateway=True),
"lan": IfInfo("lan", "vtnet1", "LAN", nets=[net("10.20.30.0/24")], addrs=[ip("10.20.30.2")], vips=[ip("10.20.30.1")]),
}
return Context(rows=[], pf=None, interfaces=interfaces, groups={}, aliases=kw.get("aliases", {}), port_forwards=[])
def nrule(*, src="", dst="", dport="", protos=frozenset({"6"}), interfaces=frozenset({"wan"}), action="pass", negated=False,
tagged="", uuid="u1") -> NormRule:
items = [i for i in src.split(",") if i]
return NormRule(uuid=uuid, label=uuid, description=uuid, order=(400000, 1), order_known=True, section="interface",
enabled=True, action=action, quick=True, interfaces=set(interfaces), if_not=False, direction="in",
af={4}, protos=None if protos is None else set(protos), src=Endpoint(raw=src, negated=negated, items=items),
dst=Endpoint(raw=dst, items=[i for i in dst.split(",") if i]), sport="", dport=dport, tagged=tagged)
SOURCES = [net("203.0.113.10/32")]
def check(rule, c=None, sources=SOURCES):
c = c or ctx()
targets, ports = protection.management_targets("https://203.0.113.57:4443", c)
return protection.management_check(rule, c, sources, targets, ports)
def test_targets_are_the_api_address_and_port_plus_every_firewall_address():
targets, ports = protection.management_targets("https://203.0.113.57:4443", ctx())
assert targets[0] == ip("203.0.113.57") and ip("10.20.30.1") in targets and ip("127.0.0.1") not in targets
assert ports == [4443, 443, 80, 22]
def test_a_rule_for_the_management_path_is_protected():
assert check(nrule(src="203.0.113.0/24", dport="443"))["result"] == "true"
assert check(nrule(src="", dport="4443"))["result"] == "true" # any source, API port
assert check(nrule(src="203.0.113.10", dport="", protos=None))["result"] == "true"
def test_only_a_proven_non_match_is_removable():
assert check(nrule(src="10.20.30.0/24", dport="443"))["result"] == "false" # other source
assert check(nrule(src="203.0.113.0/24", dport="8081"))["result"] == "false" # other port
assert check(nrule(src="203.0.113.0/24", dport="443", protos={"17"}))["result"] == "false" # udp
def test_unknown_is_protected(): # case 18
unknown = check(nrule(src="tbl", dport="443"), ctx(aliases={"tbl": {"name": "tbl", "type": "urltable"}}))
assert unknown["result"] == "unknown" and unknown["match"]["port"] in (4443, 443)
tagged = check(nrule(src="", dport="443", tagged="mark"))
assert tagged["result"] == "unknown" and "TAGGED" in tagged["reasons"]
q = Query(src=ip("203.0.113.10"), dst=ip("203.0.113.57"), proto="6", proto_label="tcp", dport=443)
assert match_rule(nrule(interfaces={"nosuchgroup"}), q, ctx(), iface="wan").value is None
def test_network_sources_are_judged_for_every_address():
small = [net("203.0.113.0/28")]
assert check(nrule(src="203.0.113.14", dport="443"), sources=small)["result"] == "true" # one address inside
assert check(nrule(src="203.0.113.20", dport="443"), sources=small)["result"] == "false"
big = [net("10.0.0.0/8")]
assert check(nrule(src="10.20.30.0/24", dport="443"), sources=big)["result"] == "true" # ranges overlap
assert check(nrule(src="192.0.2.0/24", dport="443"), sources=big)["result"] == "false"
assert check(nrule(src="lan", dport="443"), sources=big)["result"] == "true" # interface network
assert check(nrule(src="10.0.0.0/8", dport="443", negated=True), sources=big)["result"] == "false" # !whole net
assert check(nrule(src="tbl", dport="443"), ctx(aliases={"tbl": {"type": "urltable"}}), sources=big)["result"] == "unknown"
def test_no_query_possible_is_protected():
got = check(nrule(src="", dport="443"), sources=[net("2001:db8::/64")]) # no IPv6 address on the firewall
assert got["result"] == "unknown" and got["reasons"] == ["NO_MANAGEMENT_QUERY"]
def test_carp_and_pfsync_rules_are_infrastructure():
c = ctx()
assert protection.infrastructure_reason(nrule(protos={"112"}), c, hasync=None, clustered=False) == "infrastructure"
assert protection.infrastructure_reason(nrule(protos={"240"}), c, hasync=None, clustered=False) == "infrastructure"
lan_any = nrule(src="10.20.30.0/24", protos=None, interfaces={"lan"})
assert protection.infrastructure_reason(lan_any, c, hasync=None, clustered=False) == "infrastructure" # CARP adverts
lan_tcp = nrule(src="10.20.30.0/24", dport="8081", interfaces={"lan"})
assert protection.infrastructure_reason(lan_tcp, c, hasync=None, clustered=True) is None
wan_any = nrule(src="", protos=None, interfaces={"wan"})
off = {"hasync": {"pfsyncinterface": {"": {"selected": 1}, "wan": {"selected": 0}}, "pfsyncpeerip": ""}}
on = {"hasync": {"pfsyncinterface": {"": {"selected": 0}, "wan": {"selected": 1}}, "pfsyncpeerip": "203.0.113.58"}}
assert protection.infrastructure_reason(wan_any, c, hasync=off, clustered=True) is None
assert protection.infrastructure_reason(wan_any, c, hasync=on, clustered=True) == "infrastructure"
# HA settings not readable on a cluster node: every interface counts as a possible pfsync interface
assert protection.infrastructure_reason(wan_any, c, hasync=None, clustered=True) == "infrastructure"
def test_confidence_caps():
rule = nrule()
unused = {"categories": [{"kind": "unused", "confidence": "sure"}]}
assert selectable_level(unused, rule, unused_days=30, external=False)[0] == "sure"
assert selectable_level(unused, rule, unused_days=29, external=False)[0] == "probable"
assert selectable_level(unused, nrule(action="block"), unused_days=90, external=False)[0] == "probable"
assert selectable_level(unused, nrule(action="reject"), unused_days=90, external=False)[0] == "probable"
assert selectable_level(unused, rule, unused_days=90, external=True)[0] == "probable"
both = {"categories": [{"kind": "unused", "confidence": "unknown"}, {"kind": "duplicate", "confidence": "sure"}]}
assert selectable_level(both, nrule(action="block"), unused_days=7, external=False)[0] == "sure" # best category
assert selectable_level({"categories": [{"kind": "shadowed", "confidence": "unknown"}]}, rule, unused_days=30, external=False)[0] == "unknown"
assert selectable_level({"categories": [{"kind": "disabled_long", "confidence": "sure"}]}, rule, unused_days=30, external=False)[0] is None
class Hit:
def __init__(self, uuid, first_days, hit_days=None, sampled_minutes=10, enabled=True):
self.rule_uuid, self.enabled = uuid, enabled
self.first_seen_at = NOW - timedelta(days=first_days)
self.last_hit_at = None if hit_days is None else NOW - timedelta(days=hit_days)
self.last_sampled_at = NOW - timedelta(minutes=sampled_minutes)
def test_cluster_usage_merge(): # case 16
master = {"r": Hit("r", 100, hit_days=1)}
backup = {"r": Hit("r", 100)}
merged = rule_hit_service.merge_cluster_hits(backup, [master])
assert rule_hit_service.classify(merged["r"], NOW, 30) == "used"
both_idle = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{"r": Hit("r", 20)}])
assert rule_hit_service.classify(both_idle["r"], NOW, 30) == "collecting" # window = latest start
stale = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{"r": Hit("r", 100, sampled_minutes=60 * 48)}])
assert stale["r"].last_sampled_at == NOW - timedelta(hours=48) # as fresh as the oldest node
missing = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{}])
assert missing["r"].peer_missing and rule_hit_service.classify(missing["r"], NOW, 30) == "unused"
rule = nrule()
rule.loaded = True
assert cleanup.unused_confidence(rule, missing["r"], NOW) == "probable"
assert cleanup.unused_confidence(rule, both_idle["r"], NOW) == "sure"
def _live_rule(**kw):
"""A rule normalized inside the live fixture (tests/fixtures/rule_analysis_fw.json: built-in groups wireguard,
openvpn, enc0 with members "" and unassigned devices without identifier)."""
from app.services.rule_analysis.normalize import normalize
from tests.test_rule_analysis import build, mvc
row = mvc("bbbbbbbb-0000-4000-8000-000000000001", seq=900, proto="tcp", **kw)
c = build([row])
rules, _ = normalize(c)
return next(r for r in rules if r.uuid == row["uuid"]), c
def test_rules_on_builtin_groups_without_known_members_are_protected(): # review finding 1
for group in ("wireguard", "openvpn", "enc0"):
rule, c = _live_rule(interface=group, src="203.0.113.10", dst="(self)", dport="443")
got = check(rule, c)
assert got["result"] == "unknown" and "IFACE_MEMBERS_UNKNOWN" in got["reasons"], (group, got)
assert got["match"]["interface"] == protection.UNASSIGNED
# the virtual ingress does not make every rule protected
rule, c = _live_rule(interface="wireguard", src="203.0.113.10", dst="(self)", dport="8081")
assert check(rule, c)["result"] == "false"
rule, c = _live_rule(interface="lan", src="10.20.30.0/24", dst="(self)", dport="443")
assert check(rule, c)["result"] == "false"
# floating "any interface" and "not lan" also match a packet on wg0
rule, c = _live_rule(interface="", src="203.0.113.10", dst="(self)", dport="22")
assert check(rule, c)["result"] == "true"
rule, c = _live_rule(interface="lan", interfacenot="1", src="203.0.113.10", dst="(self)", dport="22")
assert check(rule, c)["result"] == "true"
def test_unassigned_interface_outcome():
c = ctx()
c.groups = {"mygroup": {"lan"}, "wireguard": set()}
assert protection.unassigned_interface(nrule(interfaces={"lan"}), c).value is False
assert protection.unassigned_interface(nrule(interfaces={"mygroup"}), c).value is False
assert protection.unassigned_interface(nrule(interfaces={"wireguard"}), c).value is None
assert protection.unassigned_interface(nrule(interfaces={"nosuch"}), c).value is None
assert protection.unassigned_interface(nrule(interfaces=set()), c).value is True
def test_redacted_plan_names_no_source_port_or_firewall_address(): # review finding 8
from app.services.rule_cleanup_service import redact
plan = {"kind": "enable", "sources": ["203.0.113.10/32"], "ports": [2222], "nodes": [
{"host_id": "h", "base_url": "https://203.0.113.57", "items": [
{"uuid": "u", "protection": {"result": "true", "queries": 3, "reasons": [],
"match": {"source": "203.0.113.10/32", "destination": "203.0.113.57"}}}]}]}
out = redact({"plan": plan, "operation": {"plan": plan}})
for p in (out["plan"], out["operation"]["plan"]):
assert "203.0.113" not in str(p) and "2222" not in str(p)
assert p["sources_configured"] is True and p["nodes"][0]["items"][0]["protection"]["result"] == "true"
assert plan["sources"] and plan["nodes"][0]["base_url"] # the stored plan itself is untouched
+391
View File
@@ -0,0 +1,391 @@
"""Coverage (shadowed/duplicate) and cleanup candidates: pure functions on NormRule (1.30, backlog B4 part 1).
Rules are built the way the live ``searchRule`` delivers them (tests/test_rule_analysis.py: fixture of a real
26.7 firewall plus synthetic MVC rows with matching pf lines). Cases that must stay red when the logic is broken
are marked "mutation" in their docstring.
"""
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
import pytest
from app.services.rule_analysis import cleanup, coverage
from app.services.rule_analysis.normalize import normalize
from tests.test_rule_analysis import build, mvc
NOW = datetime(2026, 10, 22, 12, 0, tzinfo=timezone.utc)
def overlaps(rows, **kw):
ctx = build(rows, **kw)
rules, notes = normalize(ctx)
found = coverage.analyse(rules, ctx, order_guessed=coverage.order_doubtful(notes))
return {o.rule.uuid: o for o in found}
# ── coverage ──────────────────────────────────────────────────────────────
def test_a_narrower_rule_behind_a_broad_quick_rule_is_shadowed_and_sure():
found = overlaps([
mvc("broad", seq=1000, action="block", src="10.20.30.0/24"),
mvc("narrow", seq=1001, src="10.20.30.0/25", dport="443", proto="tcp"),
])
assert found["narrow"].kind == "shadowed" and found["narrow"].by.uuid == "broad" and found["narrow"].confidence == "sure"
assert "broad" not in found
def test_an_identical_rule_with_the_same_action_is_a_duplicate_a_different_action_is_only_shadowed():
found = overlaps([
mvc("first", seq=1000, src="10.20.30.0/24", dport="443", proto="tcp"),
mvc("second", seq=1001, src="10.20.30.0/24", dport="443", proto="tcp"),
mvc("other", seq=1002, src="10.20.30.0/24", dport="443", proto="tcp", action="block"),
])
assert (found["second"].kind, found["second"].by.uuid) == ("duplicate", "first")
assert found["other"].kind == "shadowed"
def test_the_wider_later_rule_is_not_covered_by_the_narrower_earlier_one():
found = overlaps([
mvc("narrow", seq=1000, src="10.20.30.64/26"),
mvc("wide", seq=1001, src="10.20.30.0/24"),
])
assert found == {}
def test_only_a_quick_rule_covers():
"""mutation: ignoring `quick` makes the last-match-wins rule look like a cover."""
found = overlaps([
mvc("floaty", seq=1000, quick="0", action="block"),
mvc("later", seq=1001, src="10.20.30.0/24"),
])
assert found == {}
def test_sets_ranges_and_protocol_families():
rows = [
mvc("all-ports", seq=1000, proto="tcp/udp", ipprotocol="inet46", dport="1000:2000", src="10.20.30.0/24"),
mvc("inside", seq=1001, proto="tcp", ipprotocol="inet", dport="1500", src="10.20.30.5"),
mvc("udp-edge", seq=1002, proto="udp", ipprotocol="inet", dport="2000", src="10.20.30.0/24"),
mvc("outside", seq=1003, proto="tcp", ipprotocol="inet", dport="2001", src="10.20.30.5"),
mvc("v6-inside-v46", seq=1004, proto="tcp", ipprotocol="inet6", dport="1500", src="2001:db8::5"),
mvc("no-port", seq=1005, proto="tcp", ipprotocol="inet", src="10.20.30.5"),
mvc("icmp", seq=1006, proto="icmp", ipprotocol="inet", src="10.20.30.5"),
]
found = overlaps(rows)
assert set(found) == {"inside", "udp-edge"} # "v6-inside-v46": an IPv6 address is outside the 10.20.30.0/24 source
assert all(o.by.uuid == "all-ports" for o in found.values())
def test_inet_does_not_cover_inet46_but_inet46_covers_inet6():
found = overlaps([mvc("v4", seq=1000, ipprotocol="inet"), mvc("both", seq=1001, ipprotocol="inet46")])
assert found == {}
found = overlaps([mvc("both", seq=1000, ipprotocol="inet46"), mvc("v6", seq=1001, ipprotocol="inet6")])
assert found["v6"].by.uuid == "both"
def test_a_narrow_earlier_rule_never_covers_an_any_later_one():
found = overlaps([mvc("web", seq=1000, proto="tcp", dport="443"), mvc("any", seq=1001)])
assert found == {}
def test_interface_sets_and_floating_order():
rows = [
mvc("float-quick", prio=200000, seq=5, interface="lan,wan", action="block", src="10.20.30.0/24", loaded=True),
mvc("on-lan", seq=1000, interface="lan", src="10.20.30.7"),
mvc("on-wan", seq=1001, interface="wan", src="10.20.30.7"),
mvc("on-all", seq=1002, interface="", src="10.20.30.7"),
]
found = overlaps(rows)
assert found["on-lan"].by.uuid == "float-quick" and found["on-wan"].by.uuid == "float-quick"
assert "on-all" not in found # applies to interfaces the floating rule does not name
def test_a_later_floating_rule_cannot_be_covered_by_an_interface_rule():
"""mutation: sorting by `sequence` instead of `sort_order` would put the interface rule first."""
found = overlaps([
mvc("interface-rule", prio=400000, seq=1, interface="lan", src="any"),
mvc("floating-rule", prio=200000, seq=9, interface="lan,wan", src="10.20.30.7"),
])
assert "floating-rule" not in found
# and the other way round: the floating rule is evaluated first and wins
found = overlaps([
mvc("interface-rule", prio=400000, seq=9, interface="lan", src="10.20.30.7"),
mvc("floating-rule", prio=200000, seq=1, interface="lan,wan", src="10.20.30.0/24"),
])
assert found["interface-rule"].by.uuid == "floating-rule"
def test_a_group_rule_comes_before_the_interface_rule():
found = overlaps([
mvc("plain", prio=400000, seq=1, interface="lan", src="10.20.30.7"),
mvc("grouped", prio=300005, seq=900, interface="lan", src="10.20.30.0/24"),
])
assert found["plain"].by.uuid == "grouped"
def test_conditions_the_engine_cannot_compare_prevent_a_claim():
for extra in ({"sched": "work-hours"}, {"tagged": "mark"}, {"received-on": "wan"}):
found = overlaps([mvc("cond", seq=1000, **extra), mvc("later", seq=1001, src="10.20.30.5")])
assert found == {}, extra
def test_disabled_and_unloaded_rules_do_not_cover():
found = overlaps([mvc("off", seq=1000, enabled="0"), mvc("later", seq=1001, src="10.20.30.5")])
assert found == {}
found = overlaps([mvc("not-loaded", seq=1000, loaded=False), mvc("later", seq=1001, src="10.20.30.5")])
assert found == {}
def test_negation_is_compared_exactly():
rows = [
mvc("not-a", seq=1000, source_not="1", src="10.20.30.0/24"),
mvc("same", seq=1001, source_not="1", src="10.20.30.0/24"),
mvc("plain", seq=1002, src="10.20.30.5"),
]
found = overlaps(rows)
assert set(found) == {"same"} and found["same"].kind == "duplicate"
def test_a_negated_source_covers_a_negated_source_that_excludes_more():
"""mutation: using plain containment for '!' lists claims the wrong direction."""
narrow_exclusion = mvc("not-narrow", seq=1000, source_not="1", src="10.1.0.0/16")
wide_exclusion = mvc("not-wide", seq=1001, source_not="1", src="10.0.0.0/8")
assert overlaps([narrow_exclusion, wide_exclusion])["not-wide"].by.uuid == "not-narrow"
assert overlaps([mvc("not-wide", seq=1000, source_not="1", src="10.0.0.0/8"),
mvc("not-narrow", seq=1001, source_not="1", src="10.1.0.0/16")]) == {}
def test_a_negated_rule_does_not_cover_a_plain_one_and_the_reverse():
"""mutation: dropping the negation guard lets '!host' appear to cover a network that contains the host."""
assert overlaps([mvc("not-host", seq=1000, source_not="1", src="10.20.30.5"), mvc("net", seq=1001, src="10.20.30.0/24")]) == {}
assert overlaps([mvc("net", seq=1000, src="10.20.30.0/24"), mvc("not-host", seq=1001, source_not="1", src="10.20.30.5")]) == {}
def test_order_comes_from_sort_order_with_and_without_the_pf_ruleset():
"""mutation: sorting by sequence alone puts the interface rule (sequence 1) before the group rule."""
for pf in (True, False):
found = overlaps([
mvc("plain", prio=400000, seq=1, interface="lan", src="10.20.30.7"),
mvc("grouped", prio=300005, seq=900, interface="lan", src="10.20.30.0/24"),
], pf=pf)
assert found["plain"].by.uuid == "grouped", pf
def test_equal_sort_order_is_decided_by_the_pf_line_numbers_not_the_uuid():
"""mutation: sorting ties by uuid turns the effective block rule into a 'sure' shadow of the pass rule."""
rows = [mvc("zzz-narrow", seq=1000, src="10.20.30.0/24", dport="443", proto="tcp"),
mvc("aaa-broad", seq=1000, action="block")]
assert overlaps(rows) == {} # pf evaluates zzz-narrow first; the broad rule is not covered by the narrow one
rows.reverse() # pf now evaluates the broad quick block first
found = overlaps(rows)
assert found["zzz-narrow"].by.uuid == "aaa-broad" and found["zzz-narrow"].confidence == "sure"
def test_equal_sort_order_without_pf_data_is_at_most_probable():
rows = [mvc("aaa-broad", seq=1000, action="block"), mvc("zzz-narrow", seq=1000, src="10.20.30.0/24")]
assert overlaps(rows, pf=False)["zzz-narrow"].confidence == "probable"
def test_a_pf_order_mismatch_or_guessed_order_caps_confidence():
"""mutation: removing ORDER_MISMATCH from the doubt set."""
assert coverage.order_doubtful(["ORDER_MISMATCH"]) and coverage.order_doubtful(["ORDER_GUESSED", "X"])
assert not coverage.order_doubtful(["PF_RULESET_UNAVAILABLE"]) and not coverage.order_doubtful([])
ctx = build([mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")])
rules, _ = normalize(ctx)
assert coverage.analyse(rules, ctx, order_guessed=True)[0].confidence == "probable"
def test_an_unlabeled_pf_line_with_conditions_is_never_a_cover():
"""mutation: ignoring proto/port/... in the unlabeled line lets a 'tcp only' block shadow every protocol."""
from app.services.rule_analysis import pf_ruleset
from app.services.rule_analysis.normalize import antispoof_rule
plain = antispoof_rule(pf_ruleset.parse_line(5, "block drop in quick inet from 10.20.30.0/24 to any"), (400000, 0))
assert plain.extra_notes == []
for text in ("block drop in quick inet proto tcp from 10.20.30.0/24 to any",
"block drop in quick inet from 10.20.30.0/24 port 53 to any",
"block drop in quick inet from 10.20.30.0/24 to any flags S/SA",
"block drop in quick inet from 10.20.30.0/24 to any user 0",
"block drop in quick inet from 10.20.30.0/24 to any tagged x",
"block drop in quick inet from 10.20.30.0/24 to any received-on vtnet0"):
assert antispoof_rule(pf_ruleset.parse_line(6, text), (400000, 0)).extra_notes == ["UNMODELLED_CONDITION"], text
victim = antispoof_rule(pf_ruleset.parse_line(7, "block drop in quick inet from 10.20.30.5 to any"), (400000, 1))
victim.extra_notes, victim.interfaces = [], set()
assert coverage.covers_fields(plain, victim) == "sure"
conditional = antispoof_rule(pf_ruleset.parse_line(8, "block drop in quick inet proto tcp from 10.20.30.0/24 to any"), (400000, 0))
assert coverage.covers_fields(conditional, victim) is None
def test_a_disabled_alias_is_an_empty_table_and_never_evaluated():
"""mutation: using the content of a disabled alias claims coverage by something pf does not have."""
aliases = {"OFF": {"type": "host", "content": "10.20.30.0/24", "enabled": "0"},
"ON": {"type": "host", "content": "10.20.30.0/24", "enabled": "1"}}
rows = [mvc("a", seq=1000, src="OFF", dport="80", proto="tcp"), mvc("b", seq=1001, src="10.20.30.5", dport="80", proto="tcp")]
assert overlaps(rows, aliases=aliases) == {}
rows[0] = mvc("a", seq=1000, src="ON", dport="80", proto="tcp")
assert overlaps(rows, aliases=aliases)["b"].confidence == "probable"
def test_aliases_same_name_sure_other_name_probable_unresolvable_never():
aliases = {
"WEB": {"type": "host", "content": "10.20.30.5\n10.20.30.6", "enabled": "1"},
"WEB_ONE": {"type": "host", "content": "10.20.30.5", "enabled": "1"},
"NESTED": {"type": "network", "content": "WEB_ONE", "enabled": "1"},
"FEED": {"type": "urltable", "content": "https://example.invalid/list", "enabled": "1"},
}
rows = [
mvc("by-name", seq=1000, src="WEB", dport="80", proto="tcp"),
mvc("same", seq=1001, src="WEB", dport="80", proto="tcp"),
mvc("static", seq=1002, src="WEB_ONE", dport="80", proto="tcp"),
mvc("nested", seq=1003, src="NESTED", dport="80", proto="tcp"),
mvc("table-a", seq=1004, src="FEED", dport="81", proto="tcp"),
mvc("table-b", seq=1005, src="WEB_ONE", dport="81", proto="tcp"),
]
found = overlaps(rows, aliases=aliases)
assert (found["same"].kind, found["same"].confidence) == ("duplicate", "sure")
assert (found["static"].kind, found["static"].confidence) == ("shadowed", "probable")
assert found["nested"].confidence == "probable"
assert "table-b" not in found # FEED has no static content: nothing is claimed
def test_port_alias_comparison():
aliases = {"P1": {"type": "port", "content": "80\n443", "enabled": "1"}, "P2": {"type": "port", "content": "443", "enabled": "1"}}
rows = [mvc("a", seq=1000, proto="tcp", dport="P1"), mvc("b", seq=1001, proto="tcp", dport="P2"),
mvc("c", seq=1002, proto="tcp", dport="P1")]
found = overlaps(rows, aliases=aliases)
assert (found["b"].kind, found["b"].confidence) == ("shadowed", "probable")
assert (found["c"].kind, found["c"].confidence) == ("duplicate", "sure")
def test_without_the_loaded_pf_ruleset_nothing_is_sure():
"""What is not proven is never sure."""
rows = [mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")]
assert overlaps(rows, pf=False)["b"].confidence == "probable"
assert overlaps(rows, pf=True)["b"].confidence == "sure"
def test_a_guessed_order_lowers_the_confidence():
rows = [mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")]
for row in rows:
row.pop("sort_order")
found = overlaps(rows)
assert found["b"].confidence == "probable"
def test_the_unmodified_live_fixture_has_no_overlap_among_user_rules():
ctx = build([])
rules, _ = normalize(ctx)
assert [o.rule.description for o in coverage.analyse(rules, ctx) if not o.rule.automatic] == []
# ── candidates ────────────────────────────────────────────────────────────
def hit(*, enabled=True, first_days=100, last_hit_days=None, sampled_hours=1):
return SimpleNamespace(
enabled=enabled, first_seen_at=NOW - timedelta(days=first_days),
last_hit_at=None if last_hit_days is None else NOW - timedelta(days=last_hit_days),
last_sampled_at=NOW - timedelta(hours=sampled_hours), last_packets=0, last_bytes=0)
def candidates(rows, *, hits=None, states=None, managed=None, days=30, **kw):
ctx = build(rows, **kw)
rules, notes = normalize(ctx)
out = cleanup.build(rules, ctx, hits=hits or {}, states=states or {}, managed=managed or {}, now=NOW,
unused_days=days, order_guessed=coverage.order_doubtful(notes))
return {c["uuid"]: c for c in out}
def kinds(candidate):
return {c["kind"]: c["confidence"] for c in candidate["categories"]}
def test_unused_needs_the_whole_window_without_a_hit():
rows = [mvc("quiet", seq=1000, proto="tcp", dport="1"), mvc("busy", seq=1001, proto="tcp", dport="2"),
mvc("young", seq=1002, proto="tcp", dport="3"), mvc("never", seq=1003, proto="tcp", dport="4")]
hits = {"quiet": hit(last_hit_days=60), "busy": hit(last_hit_days=3), "young": hit(first_days=5), "never": hit()}
got = candidates(rows, hits=hits)
assert set(got) == {"quiet", "never"}
assert kinds(got["quiet"]) == {"unused": "sure"}
def test_the_threshold_is_a_parameter():
rows = [mvc("r", seq=1000)]
hits = {"r": hit(first_days=100, last_hit_days=45)}
assert "r" not in candidates(rows, hits=hits, days=90)
assert "r" in candidates(rows, hits=hits, days=30)
assert "r" not in candidates(rows, hits={"r": hit(first_days=20)}, days=30) # watched less than the window
def test_unused_is_never_sure_without_evidence_the_rule_can_be_hit():
"""mutation: dropping the loaded/schedule/freshness checks lets every unused rule through as sure."""
rows = [mvc("plain", seq=1000, proto="tcp", dport="1"), mvc("timed", seq=1001, sched="work-hours", proto="tcp", dport="2"),
mvc("unloaded", seq=1002, loaded=False, proto="tcp", dport="3"), mvc("stale", seq=1003, proto="tcp", dport="4")]
hits = {u: hit() for u in ("plain", "timed", "unloaded")} | {"stale": hit(sampled_hours=72)}
got = candidates(rows, hits=hits)
assert kinds(got["plain"])["unused"] == "sure"
assert kinds(got["timed"])["unused"] == "probable"
assert kinds(got["unloaded"])["unused"] == "unknown"
assert kinds(got["stale"])["unused"] == "unknown"
without_pf = candidates(rows, hits=hits, pf=False)
assert kinds(without_pf["plain"])["unused"] == "probable"
def test_a_rule_can_be_in_several_categories():
rows = [mvc("broad", seq=1000, src="10.20.30.0/24"), mvc("narrow", seq=1001, src="10.20.30.5")]
got = candidates(rows, hits={"narrow": hit()})
assert kinds(got["narrow"]) == {"unused": "sure", "shadowed": "sure"}
assert got["narrow"]["categories"][1]["by"]["uuid"] == "broad"
def test_disabled_for_a_long_time_comes_from_the_observed_history():
rows = [mvc("old-off", seq=1000, enabled="0"), mvc("new-off", seq=1001, enabled="0"), mvc("never-seen", seq=1002, enabled="0")]
states = {
"old-off": SimpleNamespace(present=True, changed_at=NOW - timedelta(days=40)),
"new-off": SimpleNamespace(present=True, changed_at=NOW - timedelta(days=3)),
}
got = candidates(rows, states=states)
assert set(got) == {"old-off"} and kinds(got["old-off"]) == {"disabled_long": "sure"}
def test_disabled_rules_are_neither_unused_nor_shadowing_candidates_of_the_unused_kind():
got = candidates([mvc("off", seq=1000, enabled="0")], hits={"off": hit(enabled=False)})
assert got == {}
def test_not_removable_reasons():
rows = [mvc("plain", seq=1000, proto="tcp", dport="1"), mvc("managed-one", seq=1001, proto="tcp", dport="2"),
mvc("legacy-one", seq=1002, legacy=True, proto="tcp", dport="3")]
hits = {u: hit() for u in ("plain", "managed-one", "legacy-one")}
got = candidates(rows, hits=hits, managed={"managed-one": {"kind": "rule", "id": "x", "name": "Managed"}})
assert (got["plain"]["removable"], got["plain"]["not_removable_reason"]) == (True, None)
assert (got["managed-one"]["removable"], got["managed-one"]["not_removable_reason"]) == (False, "managed")
assert got["managed-one"]["managed_by"]["name"] == "Managed"
assert (got["legacy-one"]["removable"], got["legacy-one"]["not_removable_reason"]) == (False, "legacy")
def test_removal_block_for_automatic_rules():
ctx = build([])
rules, _ = normalize(ctx)
automatic = next(r for r in rules if r.automatic and r.section != "antispoof")
assert cleanup.removal_block(automatic, None) == "automatic"
assert cleanup.removal_block(automatic, {"kind": "rule"}) == "automatic" # automatic wins
def test_automatic_rules_never_appear_as_candidates():
got = candidates([], hits={})
assert got == {}
def test_candidates_are_in_evaluation_order_and_counts_add_up():
rows = [mvc("b", seq=1001, proto="tcp", dport="2"), mvc("a", seq=1000, proto="tcp", dport="1")]
hits = {"a": hit(), "b": hit()}
ctx = build(rows)
rules, _ = normalize(ctx)
out = cleanup.build(rules, ctx, hits=hits, states={}, managed={}, now=NOW)
assert [c["uuid"] for c in out] == ["a", "b"]
assert cleanup.summarize(out)["unused"] == 2
def test_the_threshold_is_clamped():
ctx = build([mvc("r", seq=1000)])
rules, _ = normalize(ctx)
assert cleanup.build(rules, ctx, hits={"r": hit(first_days=100)}, states={}, managed={}, now=NOW, unused_days=0) # 0 -> 1 day
+627
View File
@@ -0,0 +1,627 @@
"""Change history of live rules (1.30, backlog B4 part 1): observation loop logic, origin, idempotence, the
history endpoint, the usage sampling via ruleStats and the cleanup endpoint/MCP tool.
Database tests drive ``rule_history_service`` with searchRule answers shaped like the live ones; the HTTP tests
mock the firewall with respx.
"""
import asyncio
import json
import uuid as uuid_lib
from datetime import datetime, timedelta, timezone
from pathlib import Path
import httpx
import pytest
from sqlalchemy import select
from app.database import SessionLocal
from app.models.audit import AuditEvent
from app.models.host import OpnsenseHost
from app.models.rule_hit import RuleHit
from app.models.rule_history import LiveRuleState, RuleChangeEvent
from app.services import rule_hit_service, rule_history_service as rhs
from app.services.rule_analysis import context as ctx_mod
from tests.test_api_tokens import _bearer, _make
from tests.test_rule_analysis import FIXTURE, mvc
from tests.test_rule_analysis_api import FW, _mock_fw
from tests.test_rules import _create_host
LIVE = json.loads((Path(__file__).parent / "fixtures" / "rule_stats_live.json").read_text())
T0 = datetime(2026, 10, 1, 8, 0, tzinfo=timezone.utc)
R1, R2, R3 = "11111111-1111-4111-8111-111111111111", "22222222-2222-4222-8222-222222222222", "33333333-3333-4333-8333-333333333333"
def answer(*rules, automatic=True):
"""A searchRule answer: optional automatic rows plus MVC rows (internal helper keys removed)."""
rows = [dict(r) for r in FIXTURE["rules"]["rows"] if r.get("is_automatic")][:3] if automatic else []
for rule in rules:
row = dict(rule)
row.pop("_loaded", None)
rows.append(row)
return {"rows": rows}
def rule(uuid, **kw):
kw.setdefault("proto", "tcp")
kw.setdefault("dport", "443")
return mvc(uuid, **kw)
async def host_of(client, respx_mock, name="fw1", url="https://fw1.test"):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
return uuid_lib.UUID((await _create_host(client, name, url))["id"])
async def observe(host_id, search, at):
async with SessionLocal() as db:
host = await db.get(OpnsenseHost, host_id)
return await rhs.observe_rows(db, host, search, now=at)
async def events(host_id):
async with SessionLocal() as db:
rows = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.host_id == host_id)
.order_by(RuleChangeEvent.observed_at, RuleChangeEvent.seq))).all()
return [(e.rule_uuid, e.kind, e.origin, e.seq) for e in rows]
# ── pure helpers ──────────────────────────────────────────────────────────
def test_clean_row_drops_derived_display_fields_and_the_fingerprint_ignores_them():
row = {"uuid": R1, "enabled": "1", "description": "x", "%action": "Pass", "alias_meta_source_net": [{"value": "a"}],
"#priority": 400000, "category_colors": []}
other = dict(row, **{"%action": "Erlauben", "alias_meta_source_net": [{"value": "b"}]})
assert rhs.clean_row(row) == {"uuid": R1, "enabled": "1", "description": "x"}
assert rhs.fingerprint(rhs.clean_row(row)) == rhs.fingerprint(rhs.clean_row(other))
assert rhs.fingerprint(rhs.clean_row(dict(row, description="y"))) != rhs.fingerprint(rhs.clean_row(row))
def test_change_kind():
base = {"uuid": R1, "enabled": "1", "description": "x"}
assert rhs.change_kind(None, base) == "added" and rhs.change_kind(base, None) == "removed"
assert rhs.change_kind(base, {**base, "enabled": "0"}) == "disabled"
assert rhs.change_kind({**base, "enabled": "0"}, base) == "enabled"
assert rhs.change_kind(base, {**base, "enabled": "0", "description": "y"}) == "changed"
assert rhs.change_kind(base, {**base, "description": "y"}) == "changed"
def test_diff_fields_is_short_and_exact():
assert rhs.diff_fields({"a": 1, "b": 2}, {"a": 1, "b": 3, "c": 4}) == [
{"field": "b", "before": 2, "after": 3}, {"field": "c", "before": None, "after": 4}]
assert len(rhs.diff_fields({}, {str(i): i for i in range(40)})) == 12
# ── observation ───────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_first_observation_stores_the_starting_point_without_events(client, respx_mock):
host = await host_of(client, respx_mock)
result = await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0)
assert result == {"baseline": True, "events": 0, "touched": []}
assert await events(host) == []
async with SessionLocal() as db:
keys = {s.rule_uuid for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host))).all()}
assert keys == {R1, R2, rhs.BASELINE_KEY} # automatic rules are not tracked
@pytest.mark.asyncio
async def test_a_host_without_user_rules_still_gets_its_baseline(client, respx_mock):
"""The first rule created afterwards must show up as 'added', not vanish into a second baseline."""
host = await host_of(client, respx_mock)
await observe(host, answer(), T0)
await observe(host, answer(rule(R1)), T0 + timedelta(minutes=15))
assert await events(host) == [(R1, "added", "external", 1)]
@pytest.mark.asyncio
async def test_unchanged_rules_make_no_events_and_the_cycle_is_repeatable(client, respx_mock):
host = await host_of(client, respx_mock)
search = answer(rule(R1))
await observe(host, search, T0)
for n in (1, 2, 3):
assert (await observe(host, search, T0 + timedelta(minutes=15 * n)))["events"] == 0
assert await events(host) == []
@pytest.mark.asyncio
async def test_added_changed_disabled_enabled_removed_and_added_again(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
await observe(host, answer(rule(R1), rule(R2, seq=1001)), step(1)) # added
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(2)) # changed
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001, enabled="0")), step(3)) # disabled
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(4)) # enabled
await observe(host, answer(rule(R1, dport="8443")), step(5)) # removed
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(6)) # added again
assert await events(host) == [
(R2, "added", "external", 1), (R1, "changed", "external", 2), (R2, "disabled", "external", 2),
(R2, "enabled", "external", 3), (R2, "removed", "external", 4), (R2, "added", "external", 5)]
async with SessionLocal() as db:
changed = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.kind == "changed"))).one()
assert changed.before["destination_port"] == "443" and changed.after["destination_port"] == "8443"
gone = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.kind == "removed"))).one()
assert gone.before["uuid"] == R2 and gone.after is None
@pytest.mark.asyncio
async def test_derived_display_changes_are_not_rule_changes(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1, **{"%action": "Pass", "alias_meta_source_net": [{"value": "a"}]})), T0)
await observe(host, answer(rule(R1, **{"%action": "Erlauben", "alias_meta_source_net": [{"value": "b"}]})), T0 + timedelta(minutes=15))
assert await events(host) == []
@pytest.mark.asyncio
async def test_an_error_answer_does_not_remove_everything(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
for bad in ({"status": "failed"}, [], {"rows": "x"}, None):
assert (await observe(host, bad, T0 + timedelta(minutes=15)))["events"] == 0
assert await events(host) == []
await observe(host, answer(), T0 + timedelta(minutes=30)) # all rules gone: confirmed only by a second look
assert await events(host) == []
await observe(host, answer(), T0 + timedelta(minutes=45))
assert await events(host) == [(R1, "removed", "external", 2)]
@pytest.mark.asyncio
async def test_two_processes_seeing_the_same_change_record_it_once(client, respx_mock):
"""mutation: without the version claim / unique key both processes insert an event."""
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
changed = answer(rule(R1, dport="8443"), rule(R2, seq=1001))
await asyncio.gather(*(observe(host, changed, T0 + timedelta(minutes=15)) for _ in range(4)))
assert sorted(await events(host)) == sorted([(R1, "changed", "external", 2), (R2, "added", "external", 1)])
again = await observe(host, changed, T0 + timedelta(minutes=30))
assert again["events"] == 0 and len(await events(host)) == 2
@pytest.mark.asyncio
async def test_origin_is_opnexus_when_the_audit_trail_has_a_deployment_in_the_window(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
async with SessionLocal() as db:
db.add(AuditEvent(created_at=T0 + timedelta(minutes=5), correlation_id="c1", actor="alice", kind="rule", action="deploy",
status="pending", host_id=str(host), details={"remote_uuid": R2}))
db.add(AuditEvent(created_at=T0 + timedelta(minutes=6), correlation_id="c2", actor="mallory", kind="rule", action="deploy",
status="error", host_id=str(host), details={"remote_uuid": R3})) # failed: changed nothing
await db.commit()
await observe(host, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002)), T0 + timedelta(minutes=15))
async with SessionLocal() as db:
by_rule = {e.rule_uuid: e for e in (await db.scalars(select(RuleChangeEvent))).all()}
assert (by_rule[R2].origin, by_rule[R2].actor) == ("opnexus", "alice") and by_rule[R2].audit_id is not None
assert (by_rule[R3].origin, by_rule[R3].actor) == ("external", None)
@pytest.mark.asyncio
async def test_a_late_audit_entry_turns_an_external_event_into_an_opnexus_one(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
seen = T0 + timedelta(minutes=15)
await observe(host, answer(rule(R1), rule(R2, seq=1001)), seen)
assert await events(host) == [(R2, "added", "external", 1)]
async with SessionLocal() as db: # the deployment's audit row is committed a moment after the firewall changed
db.add(AuditEvent(created_at=seen + timedelta(seconds=3), correlation_id="c", actor="alice", kind="rule",
action="deploy", status="success", host_id=str(host), details={"remote_uuid": R2}))
await db.commit()
assert await rhs.reconcile_origins(db, await db.get(OpnsenseHost, host), now=seen + timedelta(minutes=15)) == 1
assert await events(host) == [(R2, "added", "opnexus", 1)]
async def _audit(host, rule_uuid, at, actor="alice", status="success"):
async with SessionLocal() as db:
db.add(AuditEvent(created_at=at, correlation_id=str(uuid_lib.uuid4()), actor=actor, kind="rule", action="deploy",
status=status, host_id=str(host), details={"remote_uuid": rule_uuid}))
await db.commit()
async def _reconcile(host, now):
async with SessionLocal() as db:
return await rhs.reconcile_origins(db, await db.get(OpnsenseHost, host), now=now)
@pytest.mark.asyncio
async def test_a_later_external_change_is_not_blamed_on_an_older_opnexus_deployment(client, respx_mock):
"""mutation: without the window start the re-check reaches back to the old deployment (here an unused one)."""
host = await host_of(client, respx_mock)
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0)
await _audit(host, R2, step(1) - timedelta(minutes=5)) # a deployment that changed nothing visible
await observe(host, answer(rule(R1), rule(R2, seq=1001)), step(1))
await observe(host, answer(rule(R1), rule(R2, seq=1001, dport="22")), step(2)) # changed in the firewall's GUI
assert await events(host) == [(R2, "changed", "external", 2)]
assert await _reconcile(host, step(3)) == 0
assert await events(host) == [(R2, "changed", "external", 2)]
@pytest.mark.asyncio
async def test_one_audit_entry_explains_only_one_event(client, respx_mock):
"""mutation: without the 'already claimed' filter both changes of the rule turn into OpNexus changes."""
host = await host_of(client, respx_mock)
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
await observe(host, answer(rule(R1)), T0)
await observe(host, answer(rule(R1, dport="1")), step(1))
await observe(host, answer(rule(R1, dport="2")), step(2))
await _audit(host, R1, step(1) + timedelta(seconds=3)) # arrives late; inside both windows
assert await _reconcile(host, step(3)) == 1
async with SessionLocal() as db:
origins = [e.origin for e in (await db.scalars(select(RuleChangeEvent).order_by(RuleChangeEvent.observed_at))).all()]
assert origins == ["opnexus", "external"] # the earlier change keeps it
@pytest.mark.asyncio
async def test_observe_host_settles_old_events_before_recording_new_ones(client, respx_mock):
"""mutation: reconciling after the observation lets the newer change of the same rule take the late entry."""
host = await host_of(client, respx_mock, url=FW)
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
await observe(host, answer(rule(R1)), T0)
await observe(host, answer(rule(R1, dport="1")), step(1))
await _audit(host, R1, step(1) + timedelta(seconds=3))
_mock_observe(respx_mock, FW, answer(rule(R1, dport="2")), httpx.Response(200, json={"status": "ok", "stats": {}}))
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=step(2))
assert [(e[1], e[2]) for e in await events(host)] == [("changed", "opnexus"), ("changed", "external")]
@pytest.mark.asyncio
async def test_pruning_removes_old_events_and_tombstones_only(client, respx_mock):
host = await host_of(client, respx_mock)
now = datetime.now(timezone.utc)
old = now - timedelta(days=400)
await observe(host, answer(rule(R1), rule(R2, seq=1001)), old)
await observe(host, answer(rule(R1, dport="1"), rule(R2, seq=1001)), old + timedelta(minutes=15))
await observe(host, answer(rule(R1, dport="1")), old + timedelta(minutes=30)) # R2 removed long ago
await observe(host, answer(rule(R1, dport="2")), now) # recent change
async with SessionLocal() as db:
assert await rhs.prune(db) >= 3
left = [(e.rule_uuid, e.kind) for e in (await db.scalars(select(RuleChangeEvent))).all()]
states = {s.rule_uuid for s in (await db.scalars(select(LiveRuleState))).all()}
assert left == [(R1, "changed")] and states == {R1, rhs.BASELINE_KEY}
# ── usage sampling via ruleStats ──────────────────────────────────────────
def test_parse_rule_stats_keeps_uuid_and_legacy_hex_labels():
got = rule_hit_service.parse_rule_stats(LIVE["ruleStats"])
assert got["db610dbc7c6467261cf08e8a72ade096"]["packets"] == 110719 # unmigrated legacy rule
assert got["2ec1c6db-61e1-4476-b3f6-0599a28990a2"]["bytes"] == 99582131
assert rule_hit_service.parse_rule_stats({"status": "failed"}) is None
assert rule_hit_service.parse_rule_stats({"stats": []}) is None and rule_hit_service.parse_rule_stats(None) is None
def test_the_legacy_user_rule_is_a_configured_rule_but_automatic_ones_are_not():
rows = {"rows": [LIVE["legacy_row"], {"uuid": "02f4bab031b57d1e30553ce08e0ec131", "is_automatic": True, "enabled": "1"}]}
assert set(rule_hit_service.configured_rules(rows)) == {"db610dbc7c6467261cf08e8a72ade096"}
assert set(rule_hit_service.configured_rules({"rows": [{"uuid": R1, "is_automatic": False}, {"uuid": R2, "is_automatic": "0"}]})) == {R1, R2}
def _mock_observe(respx_mock, base, search, stats):
respx_mock.get(f"{base}/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json=search))
respx_mock.get(f"{base}/api/firewall/filter_util/ruleStats").mock(return_value=stats)
@pytest.mark.asyncio
async def test_observe_host_reads_two_endpoints_and_feeds_usage_and_history(client, respx_mock):
host = await host_of(client, respx_mock, url=FW)
legacy = {**LIVE["legacy_row"], "enabled": "1"}
_mock_observe(respx_mock, FW, answer(legacy, rule(R1)), httpx.Response(200, json={
"status": "ok", "stats": {**LIVE["ruleStats"]["stats"], R1: {"packets": 0, "bytes": 0, "evaluations": 3, "pf_rules": 1}}}))
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
hits = {h.rule_uuid: h for h in (await db.scalars(select(RuleHit))).all()}
assert set(hits) == {"db610dbc7c6467261cf08e8a72ade096", R1}
assert hits["db610dbc7c6467261cf08e8a72ade096"].last_hit_at == T0 and hits[R1].last_hit_at is None
gets = [c.request.url.path for c in respx_mock.calls if "/api/firewall" in c.request.url.path]
assert sorted(gets) == ["/api/firewall/filter/searchRule", "/api/firewall/filter_util/ruleStats"]
assert len(await events(host)) == 0 # baseline
@pytest.mark.asyncio
async def test_usage_falls_back_to_pf_statistics_when_rule_stats_is_missing(client, respx_mock):
host = await host_of(client, respx_mock, url=FW)
_mock_observe(respx_mock, FW, answer(rule(R1)), httpx.Response(404, json={"message": "not found"}))
respx_mock.get(f"{FW}/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={
"rules": {"filter rules": {f'@5 pass in quick on vtnet1 inet all label "{R1}"': {"packets": 7, "bytes": 70, "evaluations": 9}}}}))
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
row = (await db.scalars(select(RuleHit))).one()
assert row.rule_uuid == R1 and row.last_hit_at == T0 and row.last_packets == 7
@pytest.mark.asyncio
async def test_a_failing_host_does_not_stop_the_cycle_for_the_others(client, respx_mock):
good = await host_of(client, respx_mock, "good", FW)
await host_of(client, respx_mock, "bad", "https://bad.test")
_mock_observe(respx_mock, FW, answer(rule(R1)), httpx.Response(200, json={"status": "ok", "stats": {}}))
respx_mock.route(url__startswith="https://bad.test").mock(side_effect=httpx.ConnectError("down"))
async with SessionLocal() as db:
summary = await rhs.observe_all(db, now=T0)
assert summary == {"hosts": 1, "failed": 1, "events": 0}
assert await events(good) == []
async with SessionLocal() as db:
assert (await db.get(LiveRuleState, (good, R1))) is not None
# ── answers that are not to be trusted ────────────────────────────────────
@pytest.mark.asyncio
async def test_an_answer_without_automatic_rules_is_rejected_and_nothing_is_lost(client, respx_mock):
"""mutation: without the completeness check a partial answer removes every rule and every usage row."""
host = await host_of(client, respx_mock, url=FW)
stats = httpx.Response(200, json={"status": "ok", "stats": {R1: {"packets": 3, "bytes": 1, "evaluations": 1}}})
_mock_observe(respx_mock, FW, answer(rule(R1), rule(R2, seq=1001)), stats)
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
respx_mock.clear()
partial = answer(rule(R1), automatic=False)
partial["rows"][0]["is_automatic"] = "0" # a firmware that flags automatic rows, but none is in the answer
_mock_observe(respx_mock, FW, partial, stats) # R2 and all automatic rows missing
async with SessionLocal() as db:
result = await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0 + timedelta(minutes=15))
assert result["skipped"] is True
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2}
assert await rule_hit_service.apply_sample(db, await db.get(OpnsenseHost, host), {}, {"rows": []}) == 0
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2}
assert await events(host) == []
@pytest.mark.asyncio
async def test_losing_most_rules_at_once_needs_a_second_look(client, respx_mock):
"""mutation: acting on the first look turns one broken answer into removed + added events for everything."""
host = await host_of(client, respx_mock)
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
full = answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002))
await observe(host, full, T0)
thin = answer(rule(R1)) # two of three gone
assert (await observe(host, thin, step(1)))["unconfirmed"] == 2
assert (await observe(host, full, step(2)))["events"] == 0 # it was a hiccup: nothing happened
assert await events(host) == []
assert (await observe(host, thin, step(3)))["skipped"] is True # the marker was cleared, so this is a first look again
await observe(host, thin, step(4)) # same picture twice: real
assert sorted(await events(host)) == sorted([(R2, "removed", "external", 2), (R3, "removed", "external", 2)])
@pytest.mark.asyncio
async def test_an_unconfirmed_answer_does_not_delete_usage_data_either(client, respx_mock):
"""mutation: sampling an unconfirmed answer drops the usage rows of the 'vanished' rules."""
host = await host_of(client, respx_mock, url=FW)
stats = httpx.Response(200, json={"status": "ok", "stats": {}})
for n, search in ((0, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002))), (1, answer(rule(R1)))):
respx_mock.clear()
_mock_observe(respx_mock, FW, search, stats)
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0 + timedelta(minutes=15 * n))
async with SessionLocal() as db:
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2, R3}
@pytest.mark.asyncio
async def test_losing_at_most_half_of_the_rules_is_recorded_at_once(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002)), T0)
await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0 + timedelta(minutes=15))
assert await events(host) == [(R3, "removed", "external", 2)]
# ── usage window ──────────────────────────────────────────────────────────
async def _hit(host, uuid=R1):
async with SessionLocal() as db:
return (await db.scalars(select(RuleHit).where(RuleHit.host_id == host, RuleHit.rule_uuid == uuid))).one()
async def _apply(host, packets, search, at):
async with SessionLocal() as db:
return await rule_hit_service.apply_sample(
db, await db.get(OpnsenseHost, host), {R1: {"packets": packets, "bytes": 0, "evaluations": 0}}, search, at)
@pytest.mark.asyncio
async def test_a_rule_that_was_switched_on_starts_its_observation_window_again(client, respx_mock):
"""mutation: an old first_seen_at makes a rule enabled yesterday 'unused, sure' today."""
host = await host_of(client, respx_mock)
await _apply(host, 0, answer(rule(R1, enabled="0")), T0)
await _apply(host, 0, answer(rule(R1, enabled="0")), T0 + timedelta(days=40))
assert rule_hit_service.classify(await _hit(host), T0 + timedelta(days=40)) == "disabled"
await _apply(host, 0, answer(rule(R1)), T0 + timedelta(days=41)) # switched on
row = await _hit(host)
assert row.first_seen_at == T0 + timedelta(days=41)
assert rule_hit_service.classify(row, T0 + timedelta(days=42)) == "collecting"
assert rule_hit_service.classify(row, T0 + timedelta(days=72)) == "unused"
@pytest.mark.asyncio
async def test_a_counter_reset_after_a_long_gap_restarts_the_window_but_not_after_a_short_one(client, respx_mock):
"""mutation: ignoring the gap treats hits lost to a reload during a long outage as 'no hits'."""
host = await host_of(client, respx_mock)
search = answer(rule(R1))
await _apply(host, 5, search, T0)
await _apply(host, 0, search, T0 + timedelta(minutes=15)) # reload, normal cadence
assert (await _hit(host)).first_seen_at == T0
await _apply(host, 4, search, T0 + timedelta(minutes=30)) # grew: a hit
await _apply(host, 0, search, T0 + timedelta(hours=5)) # reload after a gap of hours
row = await _hit(host)
assert row.first_seen_at == T0 + timedelta(hours=5) and row.last_hit_at == T0 + timedelta(minutes=30)
@pytest.mark.asyncio
async def test_an_edited_rule_restarts_its_window_and_a_move_does_not(client, respx_mock):
"""mutation: without restart_window an edited rule keeps its old observation window."""
host = await host_of(client, respx_mock, url=FW)
step = lambda n: T0 + timedelta(days=n) # noqa: E731
stats = httpx.Response(200, json={"status": "ok", "stats": {R1: {"packets": 0, "bytes": 0, "evaluations": 0}}})
for n, search in ((0, answer(rule(R1))), (40, answer(rule(R1, dport="8443"))), (41, answer(rule(R1, dport="8443", seq=1005)))):
respx_mock.clear()
_mock_observe(respx_mock, FW, search, stats)
async with SessionLocal() as db:
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=step(n))
if n == 40:
assert (await _hit(host)).first_seen_at == step(40) # edited: window starts over
assert (await _hit(host)).first_seen_at == step(40) # moved: unchanged
assert [(e[1]) for e in await events(host)] == ["changed", "moved"]
def test_a_move_is_a_move():
base = {"uuid": R1, "enabled": "1", "sequence": "5", "sort_order": "400000.0000005", "description": "x"}
assert rhs.change_kind(base, {**base, "sequence": "9", "sort_order": "400000.0000009"}) == "moved"
assert rhs.change_kind(base, {**base, "sequence": "9", "description": "y"}) == "changed"
assert rhs.change_kind(base, {**base, "sequence": "9", "enabled": "0"}) == "changed"
# ── endpoints ─────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_history_endpoint_merges_observed_changes_revisions_and_audit(client, respx_mock):
host = await host_of(client, respx_mock)
await observe(host, answer(rule(R1)), T0)
await observe(host, answer(rule(R1, dport="8443")), T0 + timedelta(minutes=15))
async with SessionLocal() as db:
db.add(AuditEvent(created_at=T0 + timedelta(minutes=2), correlation_id="c", actor="alice", kind="rule", action="deploy",
status="success", host_id=str(host), message="ok", details={"remote_uuid": R1}))
await db.commit()
body = (await client.get(f"/api/hosts/{host}/live-rules/{R1}/history")).json()
assert body["rule_uuid"] == R1 and body["watching_since"]
assert [(e["source"], e["kind"]) for e in body["events"]] == [("observed", "changed"), ("audit", "deploy")]
assert body["events"][0]["changes"] == [{"field": "destination_port", "before": "443", "after": "8443"}]
empty = (await client.get(f"/api/hosts/{host}/live-rules/{R3}/history")).json()
assert empty["events"] == []
missing = await client.get(f"/api/hosts/{uuid_lib.uuid4()}/live-rules/{R1}/history")
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "HOST_NICHT_GEFUNDEN"
@pytest.mark.asyncio
async def test_history_of_a_managed_rule_includes_its_saved_revisions(client, respx_mock):
from app.models.policy import RuleRevision
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
host = await host_of(client, respx_mock)
async with SessionLocal() as db:
rule_row = FirewallRule(description="Web in", interface="lan", action="pass", direction="in", protocol="tcp",
source_net="any", destination_net="any", destination_port="443")
db.add(rule_row)
await db.flush()
db.add(RuleDeployment(rule_id=rule_row.id, host_id=host, status="success", opnsense_rule_uuid=R1))
db.add(RuleRevision(rule_id=rule_row.id, note="angelegt", snapshot={"destination_port": "443"}, created_at=T0))
db.add(RuleRevision(rule_id=rule_row.id, note="Port geändert", snapshot={"destination_port": "8443"}, created_at=T0 + timedelta(hours=1)))
await db.commit()
body = (await client.get(f"/api/hosts/{host}/live-rules/{R1}/history")).json()
revisions = [e for e in body["events"] if e["source"] == "revision"]
assert [e["note"] for e in revisions] == ["Port geändert", "angelegt"]
assert revisions[0]["changes"] == [{"field": "destination_port", "before": "443", "after": "8443"}]
@pytest.fixture(autouse=True)
def _fresh_cache():
ctx_mod.clear_cache()
yield
ctx_mod.clear_cache()
async def _seed_hits(host_id, **hits):
async with SessionLocal() as db:
for uuid, (first_days, last_hit_days) in hits.items():
now = datetime.now(timezone.utc)
db.add(RuleHit(host_id=host_id, rule_uuid=uuid, description=uuid, enabled=True, first_seen_at=now - timedelta(days=first_days),
last_sampled_at=now - timedelta(minutes=10),
last_hit_at=None if last_hit_days is None else now - timedelta(days=last_hit_days)))
await db.commit()
@pytest.mark.asyncio
async def test_cleanup_endpoint_lists_candidates_with_confidence_and_removability(client, respx_mock):
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
data = {**FIXTURE["rules"], "rows": FIXTURE["rules"]["rows"] + [
{k: v for k, v in rule(R1, seq=1000, src="10.20.30.0/24", dport="").items() if k != "_loaded"},
{k: v for k, v in rule(R2, seq=1001, src="10.20.30.5", dport="").items() if k != "_loaded"},
{k: v for k, v in rule(R3, seq=1002, src="10.20.30.9", dport="22", enabled="0").items() if k != "_loaded"},
]}
_mock_fw(respx_mock)
respx_mock.get(f"{FW}/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json=data))
# pf only loads enabled rules: add lines for R1/R2 so they count as loaded
pf = json.loads(json.dumps(FIXTURE["pf"]))
for u, n in ((R1, 900), (R2, 901)):
pf["rules"]["filter rules"][f'@{n} pass in quick on vtnet1_vlan10 inet proto tcp from any to any label "{u}"'] = {}
respx_mock.get(f"{FW}/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json=pf))
host = await host_of(client, respx_mock, url=FW)
await _seed_hits(host, **{R2: (100, None)})
async with SessionLocal() as db:
rule_row = FirewallRule(description="Managed web", interface="lan", action="pass", direction="in", protocol="tcp",
source_net="any", destination_net="any", destination_port="443")
db.add(rule_row)
await db.flush()
db.add(RuleDeployment(rule_id=rule_row.id, host_id=host, status="success", opnsense_rule_uuid=R2))
db.add(LiveRuleState(host_id=host, rule_uuid=R3, fingerprint="x", row={}, present=True, version=1,
first_seen_at=T0, changed_at=datetime.now(timezone.utc) - timedelta(days=50),
last_seen_at=datetime.now(timezone.utc)))
await db.commit()
resp = await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 30})
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["unused_days"] == 30 and body["host_name"] == "fw1"
by_uuid = {c["uuid"]: c for c in body["candidates"]}
assert set(by_uuid) == {R2, R3}
r2 = {c["kind"]: c for c in by_uuid[R2]["categories"]}
assert r2["shadowed"]["by"]["uuid"] == R1 and r2["shadowed"]["confidence"] == "sure"
assert r2["unused"]["confidence"] == "sure"
assert by_uuid[R2]["removable"] is False and by_uuid[R2]["not_removable_reason"] == "managed"
assert by_uuid[R2]["managed_by"]["name"] == "Managed web"
assert [c["kind"] for c in by_uuid[R3]["categories"]] == ["disabled_long"] and by_uuid[R3]["removable"] is True
assert body["counts"] == {"unused": 1, "shadowed": 1, "duplicate": 0, "disabled_long": 1}
assert all(m == "GET" for m in {c.request.method for c in respx_mock.calls})
# a narrower window / the parameter is validated
assert (await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 0})).status_code == 422
short = (await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 365})).json()
assert {k["kind"] for c in short["candidates"] for k in c["categories"]} == {"shadowed"}
@pytest.mark.asyncio
async def test_cleanup_endpoint_roles_and_errors(client, respx_mock):
_mock_fw(respx_mock)
host = await host_of(client, respx_mock, url=FW)
unknown = await client.get(f"/api/hosts/{uuid_lib.uuid4()}/rule-analysis/cleanup")
assert unknown.status_code == 404 and unknown.json()["detail"]["code"] == "HOST_NICHT_GEFUNDEN"
token = await _make(client, "read")
async with _bearer(token["token"]) as api:
assert (await api.get(f"/api/hosts/{host}/rule-analysis/cleanup")).status_code == 200
assert (await api.get(f"/api/hosts/{host}/live-rules/{R1}/history")).status_code == 200
assert (await api.post(f"/api/hosts/{host}/rule-analysis/cleanup")).status_code in (403, 405)
ctx_mod.clear_cache()
respx_mock.clear()
respx_mock.route(url__startswith=FW).mock(side_effect=httpx.ConnectError("down"))
down = await client.get(f"/api/hosts/{host}/rule-analysis/cleanup")
assert down.status_code == 502 and down.json()["detail"]["code"] == "ANALYSE_FIREWALL_NICHT_LESBAR"
@pytest.mark.asyncio
async def test_anonymous_callers_get_nothing(anon_client):
hid = uuid_lib.uuid4()
assert (await anon_client.get(f"/api/hosts/{hid}/rule-analysis/cleanup")).status_code == 401
assert (await anon_client.get(f"/api/hosts/{hid}/live-rules/{R1}/history")).status_code == 401
@pytest.mark.asyncio
async def test_mcp_cleanup_tool_works_with_a_read_token_and_changes_nothing(client, respx_mock):
_mock_fw(respx_mock)
host = await host_of(client, respx_mock, url=FW)
token = await _make(client, "read")
async with _bearer(token["token"]) as api:
rpc = {"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": {"name": "rule_cleanup_candidates", "arguments": {"host_id": str(host), "unused_days": 90}}}
result = (await api.post("/api/mcp", json=rpc)).json()["result"]
tools = (await api.post("/api/mcp", json={"jsonrpc": "2.0", "id": 2, "method": "tools/list"})).json()["result"]["tools"]
assert result["isError"] is False
payload = json.loads(result["content"][0]["text"])
assert payload["unused_days"] == 90 and payload["candidates"] == [] and payload["host_name"] == "fw1"
tool = next(t for t in tools if t["name"] == "rule_cleanup_candidates")
assert tool["inputSchema"]["required"] == ["host_id"] and "path" not in tool
assert {c.request.method for c in respx_mock.calls if c.request.url.host != "test"} == {"GET"}
def test_firmware_without_the_automatic_flag_is_not_rejected_as_incomplete():
from app.services import rule_hit_service
assert rule_hit_service.looks_complete({"rows": [{"uuid": "u1"}, {"uuid": "u2"}]}) # no row carries the flag: non-empty is enough
assert not rule_hit_service.looks_complete({"rows": []})
assert not rule_hit_service.looks_complete({"rows": [{"uuid": "u1", "is_automatic": "0"}]}) # flagged firmware, no automatic rule: partial
assert rule_hit_service.looks_complete({"rows": [{"uuid": "u1", "is_automatic": "0"}, {"uuid": "a", "is_automatic": "1"}]})
+90 -7
View File
@@ -1106,11 +1106,12 @@ Sicherung) zeigt die Vorschau eine Warnung.
Auf der Seite **Sicherheitslage** zeigt die Tabelle „Regelnutzung“, wann jede Regel
zuletzt Verkehr getroffen hat. OPNexus liest dafür die Paketzähler der Firewall. Weil diese
bei jeder Änderung und jedem Neustart von vorn beginnen, merkt sich OPNexus den Zeitpunkt, an dem
ein Zähler zuletzt gewachsen ist (Auflösung: eine Prüfung, also bis zu 6 Stunden). Eine Regel gilt
ein Zähler zuletzt gewachsen ist (Auflösung: eine Beobachtung, alle 15 Minuten). Die Zähler kommen aus `ruleStats` der
Firewall (je Regel schon summiert; bei älteren OPNsense-Versionen aus dem geladenen Regelsatz). Eine Regel gilt
als **ungenutzt**, wenn OPNexus sie mindestens 30 Tage beobachtet hat, ohne einen Treffer zu sehen;
bis dahin steht sie auf „Daten werden gesammelt“, damit eine neue Installation keinen Fehlalarm
auslöst. Nur Regeln aus dem Regel-Editor bzw. der API werden erfasst (die automatischen Regeln von
OPNsense nicht). Ungenutzte Regeln fließen als Befund „Regeln in Benutzung“ in die Sicherheitslage ein.
bis dahin steht sie auf „Daten werden gesammelt“ (die Beobachtungszeit beginnt neu, wenn die Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern), damit eine neue Installation keinen Fehlalarm
auslöst. Erfasst werden die Regeln aus dem Regel-Editor bzw. der API und noch nicht migrierte Regeln im Altformat (die
automatischen Regeln von OPNsense nicht). Ungenutzte Regeln fließen als Befund „Regeln in Benutzung“ in die Sicherheitslage ein.
### Dashboard
@@ -1255,6 +1256,86 @@ vermutlich, unbekannt; Tooltip mit den Gründen) und der entscheidenden Regel. I
die offenen Punkte. Unter „Warum?“ steht jede geprüfte Regel in der Prüfreihenfolge mit ✓/✗/? je Feld.
Die Abfrage steht in der Adresszeile und lässt sich als Link teilen.
#### Bereinigung
Der Tab **Bereinigung** (Regelwerk › Regel-Analyse › Bereinigung) nennt je Firewall die Regeln, die einen Blick wert sind. Du wählst die Firewall
und die **Schwelle** (7, 30, 90 oder 180 Tage); die Auswahl steht in der Adresszeile (`?tab=cleanup&host=…&days=…`). Die Liste selbst wird nur gelesen; Ändern geht über die Auswahl (siehe unten):
| Befund | Bedeutung |
|---|---|
| **Ungenutzt** | OPNexus beobachtet die Regel seit mindestens der Schwelle und hat in dieser Zeit keinen Treffer gesehen (siehe „Regelnutzung“). Eine neue Regel erscheint erst nach der Beobachtungszeit. |
| **Verdeckt** | Eine frühere Quick-Regel trifft schon alles, was diese Regel trifft; die Regel entscheidet nie. Die verdeckende Regel steht daneben. |
| **Doppelt** | Wie verdeckt, aber die frühere Regel hat dieselbe Aktion und trifft genau dasselbe. |
| **Lange deaktiviert** | Die Regel ist deaktiviert und OPNexus hat sie seit mindestens der Schwelle unverändert gesehen (aus dem Änderungsverlauf). |
Jeder Befund trägt eine **Sicherheit**. **Sicher** gibt es nur mit vollständigen, frischen Belegen: bei „ungenutzt“ muss die Regel in pf geladen sein,
ohne Zeitplan, und die Zähler müssen frisch sein; bei „verdeckt/doppelt“ stammt die Reihenfolge von der Firewall, die verdeckende Regel ist geladen
und der Vergleich beruht auf den Regeln selbst. **Vermutlich** heißt: es beruht auf einer Annahme (der Inhalt von Aliasen, eine geschätzte
Reihenfolge, eine Regel mit Zeitplan, ein fehlender pf-Regelsatz). **Unbekannt** heißt: es gibt einen Hinweis ohne Beleg, etwa weil die Regel in pf nicht geladen
ist oder die Zähler veraltet sind. Der Tooltip erklärt es jeweils. Was nicht belegt ist, ist nie „sicher“.
Die Spalte **Aktion** sagt, was eine Auswahl der Regel bewirken würde (**Deaktivieren** oder, bei einer schon deaktivierten Regel, **Löschen**) oder warum
die Regel nicht angeboten wird: automatische Regeln (von OPNsense erzeugt), Regeln im Altformat (zuerst in OPNsense migrieren), von OPNexus ausgerollte Regeln
(die entfernst du über den normalen Ablauf der Regelseite, sonst legt OPNexus sie wieder an) und Regeln mit Befund „unbekannt“. Die Sicherheitslage verlinkt
ihre Befunde „ungenutzt“, „verdeckt“ und „doppelt“ direkt in diese Liste. Per API: `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP: `rule_cleanup_candidates`.
##### Verwaltungsquellen (Voraussetzung)
Bevor die Bereinigung etwas ändern darf, trägst du die **Verwaltungsquellen** ein: die IP-Adressen oder Netze (CIDR, keine Hostnamen), von denen aus deine Firewalls verwaltet
werden (OPNexus selbst, dein Arbeitsplatz, ein Jump-Host). Der Tab zeigt bis dahin einen Hinweis und (nur für Administratoren) ein kleines Formular; dasselbe steht unter
**Einstellungen › Regel-Bereinigung**, dort auch die **Obergrenze je Lauf** (1 bis 25, Standard 10) und die **Wartezeit bis zum Löschen** (1 bis 365 Tage, Standard 7).
Für jede gewählte Regel prüft OPNexus, ob sie ein Paket von einer Verwaltungsquelle an eine Adresse der Firewall auf einem Verwaltungsport treffen kann (API-Port aus
der Host-Adresse, 443, 80, 22 und die **weiteren Verwaltungsports**, die du unter Einstellungen › Regel-Bereinigung einträgst, z. B. SSH auf 2222 oder die Weboberfläche
auf einem eigenen Port; bis 20 Ports). Geprüft wird auf jedem Interface, auch für Pakete über nicht zugewiesene Geräte (WireGuard, OpenVPN, IPsec). Trifft die Regel oder
lässt sich das nicht ausschließen, wird sie nie deaktiviert. Dazu kommen Cluster-Regeln (CARP, pfsync), die nie angeboten werden. Eine Sperrregel wieder einzuschalten,
die den Verwaltungszugang treffen könnte, verlangt eine ausdrückliche Bestätigung je Regel.
> **Wichtig: OPNexus selbst gehört in die Verwaltungsquellen.** Trag die Adresse ein, mit der OPNexus bei den Firewalls ankommt (bei NAT die Absenderadresse nach der
> Übersetzung, über das Management-Overlay die Overlay-Adresse). Fehlt sie, kann eine Bereinigung genau die Regel abschalten, über die OPNexus die Firewall erreicht; dann
> scheitert auch das automatische Zurücksetzen. Im **Cluster** gilt das für jeden Knoten: OPNexus muss beide Knoten über eingetragene Quellen erreichen.
> Quellen, Ports und Firewall-Adressen sehen nur Administratoren; alle anderen sehen nur, ob etwas eingetragen ist.
##### Deaktivieren, Bestätigen, automatisches Zurücksetzen
1. **Auswählen.** Pro Lauf entweder nur Deaktivieren oder nur Löschen, höchstens so viele Regeln wie eingestellt. Bei Befunden „vermutlich“ fragt OPNexus vor der Vorschau je Regel
mit einem festen Text nach, ob du geprüft hast, dass die Regel nicht gebraucht wird (Zeitplan, Aliasinhalt, kurzes Beobachtungsfenster, Sperrregel ohne Treffer). „Ungenutzt“ ist unter
30 Tagen Beobachtung und bei Sperrregeln höchstens „vermutlich“.
2. **Vorschau.** OPNexus berechnet alles frisch von der Firewall (im Cluster von allen Knoten; die Regel muss auf jedem Knoten dieselbe UUID haben und es muss genau einen
Master geben). Je Regel stehen Aktion, Befund, Sicherheit, die Schutzprüfung zum Verwaltungszugang und Hinweise (z. B. „extern geändert“). Die Vorschau gilt 5 Minuten
und gehört dem, der sie erstellt hat. Offene Wartungen, Ausrollvorgänge oder eine andere Bereinigung im Cluster sperren sie.
3. **Ausrollen** (nur Administratoren, mit Eingabe des Firewall-Namens). OPNexus prüft den Stand erneut und schreibt nichts, wenn er von der Vorschau abweicht. Dann werden die Regeln
deaktiviert (im Cluster zuerst der Backup) und angewendet.
4. **Bestätigen oder zurücksetzen.** Der Kasten oben im Tab zeigt den Countdown. **Bestätigen** (Operator oder Administrator, **nur in der Browser-Sitzung, nie per API-Token**) liest
vorher jede Firewall noch einmal und verweigert, wenn sie nicht antwortet: das ist das Signal, dass du dich ausgesperrt haben könntest. Bestätigst du nicht
rechtzeitig, setzt OPNexus automatisch zurück; mit **Jetzt zurücksetzen** geht das sofort (Administrator, oder Operator, wenn dieser Schalter unter Einstellungen erlaubt ist). Ist
eine Regel fehlgeschlagen oder unklar (Status *Teilweise*), ist Bestätigen gesperrt. Schlägt das Zurücksetzen selbst fehl, hilft **Zurücksetzen wiederholen**. Nach einem Zurücksetzen beginnt
das Beobachtungsfenster der Regel neu.
##### Löschen und Wiederherstellen
Gelöscht werden nur Regeln, die schon deaktiviert sind: eine bestätigt deaktivierte Regel nach der Wartezeit (und nur, wenn sie seitdem unverändert und nicht extern geändert wurde)
oder eine von Hand deaktivierte Regel mit Befund „lange deaktiviert“ (sicher). Eine deaktivierte Regel ist in der Firewall nicht geladen, das Löschen ändert dort also nichts
und hat darum kein Bestätigungsfenster. OPNexus sichert vorher den vollständigen Regelinhalt samt Position; ohne Sicherung wird nicht gelöscht. Unter **Verlauf der Bereinigungen**
(am Seitenende des Tabs) zeigt jede Zeile ihre Schritte; bei gelöschten Regeln gibt es **Wiederherstellen** (nur Administratoren, mit Firewall-Namen): die Regel kommt aus der Sicherung neu zurück, **deaktiviert**,
mit neuer UUID (im Cluster auf allen Knoten) und möglichst direkt vor ihrer alten Nachfolgerin. Lässt sich die Position nicht herstellen, steht die Regel am Ende ihrer Gruppe und
OPNexus weist darauf hin. Fehlt der Firewall inzwischen ein Alias oder eine Kategorie der Regel, lehnt sie ab und ihre Meldung steht unverändert beim Ergebnis. Zum Einschalten gibt es
**Wieder aktivieren** (auch bei bestätigt deaktivierten Regeln): ein eigener Lauf mit Bestätigungsfenster, weil das Einschalten einer Sperrregel aussperren kann.
Rechte im Überblick: Vorschau, Abbrechen und Bestätigen dürfen Operatoren und Administratoren; Ausrollen, Wiederholen und Wiederherstellen nur Administratoren; Nur-Lese-Konten sehen die Liste und den Verlauf,
aber keine Knöpfe. Jeder Schritt steht je Regel im Audit-Protokoll und im Änderungsverlauf der Regel (Herkunft „über OPNexus“).
#### Verlauf einer Regel
OPNexus liest alle 15 Minuten die Regeln jeder Firewall (zwei Lesezugriffe) und vergleicht sie mit dem zuletzt gesehenen Stand – auch Regeln, die nicht von OPNexus
stammen. Jede Änderung wird ein Ereignis: **angelegt, geändert** (mit Vorher/Nachher der betroffenen Felder), **verschoben** (nur Reihenfolge oder Gruppe), **aktiviert, deaktiviert, entfernt**. Als **Herkunft**
steht „über OPNexus“, wenn das Audit seit der vorherigen Beobachtung (und noch für keine andere Änderung) ein Ausrollen oder Zurücksetzen dieser Regel auf dieser Firewall kennt (mit dem Benutzer), sonst „extern“ (in der
Oberfläche der Firewall, per Skript oder über einen anderen Zugang). Die erste Beobachtung einer Firewall speichert nur den Ausgangszustand, es entstehen
keine „angelegt“-Ereignisse für vorhandene Regeln; eine leere oder unvollständige Antwort der Firewall wird verworfen, und verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung; eine Änderung zwischen zwei Beobachtungen erscheint zum Zeitpunkt der zweiten. Ereignisse bleiben 365 Tage.
Zu sehen ist der Verlauf in der **Live-Ansicht** (Regelwerk › Live über alle Hosts) über die Zeilenaktion **Verlauf**, in der Bereinigungsliste am Symbol neben der Regel,
und per API (`GET /api/hosts/{id}/live-rules/{uuid}/history`). Bei einer von OPNexus verwalteten Regel stehen die gespeicherten Fassungen und die Audit-Einträge der
Regel dabei. Automatische Regeln werden nicht verfolgt.
### Sicherheitslage
Unter **Monitoring › Sicherheitslage** prüft OPNexus jede Firewall auf typische
@@ -1280,9 +1361,11 @@ Konfiguration nicht gespeichert. Alle sechs Stunden läuft die Prüfung von selb
Ein kritischer Befund kostet 30 Punkte, ein hoher 15, ein mittlerer 8, ein niedriger 3.
Nicht entscheidbare Prüfungen (etwa der Updatestatus einer gerade nicht erreichbaren
Firewall) kosten nichts. Verdeckt oder doppelt meldet OPNexus nur, was sich beweisen
lässt: Regeln werden Feld für Feld verglichen, Aliase werden nicht aufgelöst, also
bleibt eine Überschneidung über Aliase unentdeckt, aber es gibt keine falschen
Befunde. Beide Regelformate (OPNsense 26.x und ältere Installationen) werden gelesen.
lässt: Regeln werden in der Reihenfolge der Firewall (Floating, Interface-Gruppen, Interface-Regeln) Feld für Feld
verglichen, Netze und Portbereiche als Mengen; nur eine frühere Quick-Regel kann eine spätere verdecken, und
Regeln mit Zeitplan, Markierung oder Limits verdecken nichts. Aliase gelten nur bei gleichem Namen als gleich; so entstehen
keine falschen Befunde, aber eine Überschneidung über verschiedene Aliase bleibt hier unentdeckt (die Bereinigung vergleicht
auch statische Alias-Inhalte, dort als „vermutlich“). Beide Regelformate (OPNsense 26.x und ältere Installationen) werden gelesen.
Die Werte stehen auch als Prometheus-Metriken zur Verfügung (siehe unten).
Das **Dashboard** zeigt die Firewalls mit dem niedrigsten Wert. Der Knopf **Bericht** erzeugt eine druckbare Übersicht
+26 -6
View File
@@ -6,7 +6,9 @@ import { Button } from "@/components/ui/Button";
import { Card } from "@/components/ui/Card";
import { EmptyState } from "@/components/ui/EmptyState";
import { FieldGroup, Input, Label, Select } from "@/components/ui/Field";
import { RuleCleanup } from "@/components/RuleCleanup";
import { Loading } from "@/components/ui/Loading";
import { Tabs } from "@/components/ui/Tabs";
import { InfoTip } from "@/components/ui/Tooltip";
import { AnalysisResult, api, DeviceGroup, Host, InterfaceOption } from "@/lib/api";
import { Network, Search } from "lucide-react";
@@ -53,7 +55,30 @@ function toParams(form: FormState): URLSearchParams {
export default function AnalysisPage() {
const { t } = useLanguage();
return <Suspense fallback={<Loading>{t("common.loading")}</Loading>}><AnalysisContent /></Suspense>;
return <Suspense fallback={<Loading>{t("common.loading")}</Loading>}><AnalysisShell /></Suspense>;
}
/** Title and tabs (?tab=query|cleanup); each tab keeps its own state in the address bar. */
function AnalysisShell() {
const { t } = useLanguage();
const router = useRouter();
const params = useSearchParams();
const tab = params.get("tab") === "cleanup" ? "cleanup" : "query";
return (
<div>
<h1 className="mb-4 flex items-center gap-2 text-2xl font-semibold text-slate-900">
{t("analysis.title")}
<InfoTip text={t("analysis.intro")} size={18} />
</h1>
<Tabs
className="mb-6"
items={[{ key: "query", label: t("analysis.tabs.query") }, { key: "cleanup", label: t("analysis.tabs.cleanup") }]}
active={tab}
onChange={(key) => router.push(key === "cleanup" ? "/analysis?tab=cleanup" : "/analysis")}
/>
<div role="tabpanel">{tab === "cleanup" ? <RuleCleanup /> : <AnalysisContent />}</div>
</div>
);
}
function AnalysisContent() {
@@ -144,11 +169,6 @@ function AnalysisContent() {
return (
<div>
<h1 className="mb-6 flex items-center gap-2 text-2xl font-semibold text-slate-900">
{t("analysis.title")}
<InfoTip text={t("analysis.intro")} size={18} />
</h1>
<Card className="mb-6 p-4 sm:p-5">
<form onSubmit={submit} aria-label={t("analysis.form.title")} className="flex flex-col gap-4" noValidate>
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-4">
+1 -1
View File
@@ -5,7 +5,7 @@ import { api, AuditEvent, AuditPage } from "@/lib/api";
import { Button } from "@/components/ui/Button";
import { Input, Select } from "@/components/ui/Field";
import { useLanguage } from "@/components/LanguageProvider";
const kindKeys = ["rule","alias","port_forward","one_to_one","outbound_nat","host","user","auth","backup","ipsec_tunnel","openvpn_instance","wireguard_tunnel","certificate","acme","notification","gui_access","system"];
const kindKeys = ["rule","rule_cleanup","alias","port_forward","one_to_one","outbound_nat","host","user","auth","backup","ipsec_tunnel","openvpn_instance","wireguard_tunnel","certificate","acme","notification","gui_access","system"];
const actionKeys = ["deploy","confirm","rollback","remove","retry-rollback","delete","create","edit","reboot","update","ticket","redeem","end"];
const statusKeys = ["requested","accepted","rejected","interrupted","success","pending","error","reverted","revert_failed","removed","submitted","acknowledged","starting","unknown","denied"];
export default function AuditHistory() {
+12 -3
View File
@@ -10,7 +10,7 @@ import { useToast } from "@/components/ui/Toast";
import { api, ComplianceFinding, ComplianceHost, RuleHit, Severity } from "@/lib/api";
import { useLanguage } from "@/components/LanguageProvider";
import { useReadOnly } from "@/lib/useReadOnly";
import { CheckCircle2, CircleHelp, FileText, Info, RefreshCw, ShieldAlert, XCircle } from "lucide-react";
import { ArrowRight, CheckCircle2, CircleHelp, FileText, Info, RefreshCw, ShieldAlert, XCircle } from "lucide-react";
import Link from "next/link";
import { useEffect, useState } from "react";
@@ -149,7 +149,7 @@ export default function SecurityPage() {
{!detail.error && findings.length === 0 && <p className="px-5 py-6 text-sm text-slate-500">{t("security.neverChecked")}</p>}
<ul className="divide-y divide-slate-100">
{findings.map((finding) => (
<FindingRow key={finding.id} finding={finding} />
<FindingRow key={finding.id} finding={finding} hostId={detail.host_id} />
))}
</ul>
</Card>
@@ -193,7 +193,10 @@ export default function SecurityPage() {
);
}
function FindingRow({ finding }: { finding: ComplianceFinding }) {
// Findings about the rule set itself: the cleanup list shows which rules and how sure that is.
const CLEANUP_FINDINGS = new Set(["rules_unused", "rules_shadowed", "rules_duplicate"]);
function FindingRow({ finding, hostId }: { finding: ComplianceFinding; hostId: string }) {
const { t } = useLanguage();
const failed = finding.passed === false;
// A note (severity "info") is a suggestion, not an error: neutral icon instead of the red cross.
@@ -219,6 +222,12 @@ function FindingRow({ finding }: { finding: ComplianceFinding }) {
</ul>
)}
{failed && <p className="mt-1 text-sm text-slate-500">{t(`security.checks.${finding.id}.fix`)}</p>}
{failed && CLEANUP_FINDINGS.has(finding.id) && (
<Link href={`/analysis?tab=cleanup&host=${hostId}`} className="mt-1 inline-flex items-center gap-1 text-sm text-brand-600 hover:underline">
{t("security.cleanupLink")}
<ArrowRight size={14} aria-hidden="true" />
</Link>
)}
</div>
</li>
);
+2 -1
View File
@@ -11,6 +11,7 @@ import { Button } from "@/components/ui/Button";
import { FieldGroup, Input, Select } from "@/components/ui/Field";
import { useNotAdmin, useReadOnly } from "@/lib/useReadOnly";
import { OidcSection } from "@/components/OidcSection";
import { CleanupSettings } from "@/components/CleanupSettings";
import { BackupSettingsSection } from "@/components/BackupSettingsSection";
import { GuiAccessSection } from "@/components/GuiAccessSection";
import { AuditExportSection } from "@/components/AuditExportSection";
@@ -481,5 +482,5 @@ export default function SettingsPage(){
async function saveTheme(value:Theme){setSaving(true);setError('');setNotice('');const old=theme;setTheme(value);try{setSettings(await api.saveSettings({theme:value,language}));setNotice(t('settings.appearanceSaved'));}catch(e){setTheme(old);setError(errorMessage(e));}finally{setSaving(false);}}
async function saveLanguage(value:"de"|"en"){setSaving(true);setError('');setNotice('');const old=language;setLanguage(value);try{setSettings(await api.saveSettings({theme,language:value}));setNotice(translate(value,'settings.language.saved'));}catch(e){setLanguage(old);setError(errorMessage(e));}finally{setSaving(false);}}
async function changePassword(e:React.FormEvent){e.preventDefault();setError('');setNotice('');if(password!==repeat){setError(t('settings.passwordMismatch'));return;}setSaving(true);try{await api.changePassword(current,password);setCurrent('');setPassword('');setRepeat('');setNotice(t('settings.passwordChanged'));}catch(e){setError(errorMessage(e,'settings.passwordFailed'));}finally{setSaving(false);}}
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><section className="space-y-3 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.operational')}<InfoTip text={t('settings.serverManaged')} /></h2>{settings?<dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.deployWindow')}</dt><dd>{t('settings.seconds',{n:settings.deploy_confirm_window_seconds})}</dd><dt>{t('settings.sessionLifetime')}</dt><dd>{t('settings.days',{n:Math.round(settings.session_lifetime_seconds/86400)})}</dd></dl>:<p className="text-sm text-slate-500">{t('common.loading')}</p>}</section><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><section className="space-y-3 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.operational')}<InfoTip text={t('settings.serverManaged')} /></h2>{settings?<dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.deployWindow')}</dt><dd>{t('settings.seconds',{n:settings.deploy_confirm_window_seconds})}</dd><dt>{t('settings.sessionLifetime')}</dt><dd>{t('settings.days',{n:Math.round(settings.session_lifetime_seconds/86400)})}</dd></dl>:<p className="text-sm text-slate-500">{t('common.loading')}</p>}</section><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><CleanupSettings/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
}
+39
View File
@@ -0,0 +1,39 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { Badge, BadgeTone } from "@/components/ui/Badge";
import { Tooltip } from "@/components/ui/Tooltip";
import type { AnalysisConfidence, CleanupKind } from "@/lib/api";
import clsx from "clsx";
import { Check, CircleHelp } from "lucide-react";
export const KIND_TONE: Record<CleanupKind, BadgeTone> = { unused: "warning", shadowed: "accent", duplicate: "accent", disabled_long: "neutral" };
const CONFIDENCE_TONE: Record<AnalysisConfidence, BadgeTone> = { sure: "info", probable: "warning", unknown: "neutral" };
export function ConfidenceBadge({ confidence }: { confidence: AnalysisConfidence }) {
const { t } = useLanguage();
return (
<Tooltip content={t(`cleanup.confidence.tip.${confidence}`)} focusable>
<Badge
tone={CONFIDENCE_TONE[confidence]}
className={clsx(confidence === "unknown" && "border-dashed")}
aria-label={`${t("cleanup.confidence.label")}: ${t(`cleanup.confidence.${confidence}`)}`}
>
{confidence === "unknown" ? <CircleHelp size={12} aria-hidden="true" /> : <Check size={12} aria-hidden="true" />}
{t(`cleanup.confidence.${confidence}`)}
</Badge>
</Tooltip>
);
}
/** A note code of the analysis or the cleanup plan (cleanup.notes.*); unknown codes show raw. */
export function NoteBadge({ code }: { code: string }) {
const { t } = useLanguage();
const label = t(`cleanup.notes.${code}`);
const known = label !== `cleanup.notes.${code}`;
return (
<Tooltip content={known ? t(`cleanup.notes.${code}_hint`) : code} focusable>
<Badge tone="warning">{known ? label : code}</Badge>
</Tooltip>
);
}
+197
View File
@@ -0,0 +1,197 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { Badge, BadgeTone } from "@/components/ui/Badge";
import { Button } from "@/components/ui/Button";
import { Card } from "@/components/ui/Card";
import { Tooltip } from "@/components/ui/Tooltip";
import { CleanupItem, CleanupItemState, CleanupOperation, CleanupOperationStatus } from "@/lib/api";
import { cleanupMessage, COUNTDOWN_STATUSES, currentUuid, formatClock, latestItemIds, secondsUntil } from "@/lib/cleanup";
import { CircleCheck, Clock, RotateCcw, Undo2 } from "lucide-react";
import { forwardRef, ReactNode } from "react";
const CONFIRMABLE: CleanupOperationStatus[] = ["awaiting_confirm", "partial", "apply_failed"];
const STATUS_TONE: Record<CleanupOperationStatus, BadgeTone> = {
preview: "neutral", running: "info", awaiting_confirm: "warning", confirmed: "success", partial: "danger",
apply_failed: "danger", reverting: "info", reverted: "neutral", revert_failed: "danger", done: "success",
failed: "danger", cancelled: "neutral", expired: "neutral",
};
const ITEM_TONE: Record<CleanupItemState, BadgeTone> = {
queued: "neutral", disabling: "info", disabled: "warning", confirmed: "success", reverting: "info", reverted: "neutral",
deleting: "info", deleted: "neutral", restoring: "info", restored: "success", failed: "danger", unknown: "danger",
revert_failed: "danger", skipped: "neutral",
};
function useStateLabel() {
const { t } = useLanguage();
return (kind: string, state: CleanupItemState) => {
// For "enable" the item states mean switching on, not off.
if (kind === "enable" && (state === "disabling" || state === "disabled")) return t(`cleanup.stateEnable.${state}`);
return t(`cleanup.state.${state}`);
};
}
/** The operation's own message, unless a rule below already says the same (a restore warning sits on both). */
function ownMessage(t: (key: string) => string, operation: CleanupOperation): string {
const message = cleanupMessage(t, operation.message);
return message && !operation.items.some((i) => cleanupMessage(t, i.message) === message) ? message : "";
}
function ItemRows({ operation, actions }: { operation: CleanupOperation; actions?: (item: CleanupItem) => ReactNode }) {
const { t } = useLanguage();
const label = useStateLabel();
const clustered = new Set(operation.items.map((i) => i.host_id)).size > 1;
return (
<ul className="divide-y divide-slate-100 text-sm">
{operation.items.map((item) => {
const message = cleanupMessage(t, item.message);
return (
<li key={item.id} className="flex flex-wrap items-start gap-x-3 gap-y-1.5 py-2">
<div className="min-w-0 flex-1">
<p className="break-words font-medium text-slate-900">{item.description || item.rule_uuid}</p>
{clustered && item.host_name && (
<p className="text-xs text-slate-500">{item.host_name}{item.carp_role ? ` · ${item.carp_role}` : ""}</p>
)}
{message && <p className="mt-0.5 break-words text-xs text-slate-600">{message}</p>}
</div>
<Badge tone={ITEM_TONE[item.state]}>{label(operation.kind, item.state)}</Badge>
{item.restored_uuid && item.state === "deleted" && <Badge tone="success">{t("cleanup.hist.restoredBadge")}</Badge>}
{actions && <div className="flex flex-wrap gap-1.5">{actions(item)}</div>}
</li>
);
})}
</ul>
);
}
/**
* The one operation that is open on the host/cluster: what happened, how long the automatic revert still waits and the
* buttons for the decision (confirm = "I still have access", revert now, try again). Focusable so a roll-out can put
* the keyboard focus here.
*/
export const OperationBanner = forwardRef<HTMLElement, {
operation: CleanupOperation; now: number; busy: string | null; readOnly: boolean; notAdmin: boolean; operatorsMayDeploy: boolean;
onConfirm: () => void; onRevert: () => void; onRetry: () => void;
}>(function OperationBanner({ operation, now, busy, readOnly, notAdmin, operatorsMayDeploy, onConfirm, onRevert, onRetry }, ref) {
const { t, locale } = useLanguage();
const left = COUNTDOWN_STATUSES.includes(operation.status) ? secondsUntil(operation.revert_deadline, now) : null;
const message = ownMessage(t, operation);
const revertBlocked = notAdmin && !operatorsMayDeploy;
const confirmHint = operation.status === "partial" ? t("cleanup.op.confirmPartial") : operation.status === "apply_failed" ? t("cleanup.op.confirmApplyFailed") : null;
return (
<section
ref={ref}
tabIndex={-1}
aria-labelledby={`cleanup-op-${operation.id}`}
className="mb-6 rounded-2xl border border-flame-300 bg-[rgb(var(--tone-warning-bg))] p-4 outline-none focus-visible:ring-2 focus-visible:ring-brand-500 sm:p-5"
>
<div className="flex flex-wrap items-center gap-2">
<h3 id={`cleanup-op-${operation.id}`} className="text-base font-semibold text-slate-900">{t(`cleanup.op.title.${operation.kind}`)}</h3>
<Tooltip content={t(`cleanup.status.${operation.status}_hint`)} focusable>
<Badge tone={STATUS_TONE[operation.status]}>{t(`cleanup.status.${operation.status}`)}</Badge>
</Tooltip>
<span className="ml-auto text-xs text-slate-500">
{operation.actor}{operation.created_at ? ` · ${new Date(operation.created_at).toLocaleString(locale)}` : ""}
</span>
</div>
<p className="mt-2 text-sm text-slate-700">{t(`cleanup.op.state.${operation.status}`)}</p>
{left !== null && (
<p className="mt-2 flex items-center gap-1.5 text-sm font-medium text-flame-700">
<Clock size={15} aria-hidden="true" />
<span aria-hidden="true">{t("cleanup.op.countdown", { time: formatClock(left) })}</span>
<span className="sr-only" role="status" aria-live="polite">{t("cleanup.op.countdownMinutes", { n: Math.ceil(left / 60) })}</span>
</p>
)}
{message && <p role="status" className="mt-2 break-words rounded-lg bg-red-50 p-2.5 text-sm text-red-700">{message}</p>}
<div className="mt-3 rounded-xl bg-white/70 px-3"><ItemRows operation={operation} /></div>
{operation.warnings.map((code) => <p key={code} className="mt-2 text-sm text-flame-800">{cleanupMessage(t, code)}</p>)}
<div className="mt-4 flex flex-wrap items-center gap-2">
{CONFIRMABLE.includes(operation.status) && operation.kind !== "delete" && operation.kind !== "restore" && !readOnly && (
<Tooltip content={!operation.can_confirm ? (confirmHint ?? t("cleanup.op.confirmNotYet")) : null} focusable={!operation.can_confirm}>
<Button onClick={onConfirm} disabled={!operation.can_confirm || busy !== null} loading={busy === "confirm"}>
<CircleCheck size={15} aria-hidden="true" />{t("cleanup.op.confirm")}
</Button>
</Tooltip>
)}
{operation.can_revert && !readOnly && (
<Tooltip content={revertBlocked ? t("cleanup.op.revertNeedsAdmin") : null} focusable={revertBlocked}>
<Button variant="outline" onClick={onRevert} disabled={revertBlocked || busy !== null} loading={busy === "revert"}>
<Undo2 size={15} aria-hidden="true" />{t("cleanup.op.revert")}
</Button>
</Tooltip>
)}
{operation.can_retry && !readOnly && (
<Tooltip content={notAdmin ? t("cleanup.op.retryNeedsAdmin") : null} focusable={notAdmin}>
<Button variant="secondary" onClick={onRetry} disabled={notAdmin || busy !== null} loading={busy === "retry"}>
<RotateCcw size={15} aria-hidden="true" />{t(operation.kind === "disable" || operation.kind === "enable" ? "cleanup.op.retryRevert" : "cleanup.op.retryCheck")}
</Button>
</Tooltip>
)}
</div>
{operation.can_confirm && <p className="mt-3 text-xs text-slate-600">{t("cleanup.op.autoRevertNote")}</p>}
</section>
);
});
/** Finished operations of this host (and its cluster peers), newest first, with the follow-up action per rule. */
export function OperationHistory({ operations, hostId, now, minDays, readOnly, notAdmin, busy, onRestore, onFollowUp }: {
operations: CleanupOperation[]; hostId: string; now: number; minDays: number; readOnly: boolean; notAdmin: boolean; busy: boolean;
onRestore: (item: CleanupItem) => void;
onFollowUp: (kind: "enable" | "delete", item: CleanupItem, uuid: string) => void;
}) {
const { t, locale } = useLanguage();
const latest = latestItemIds(operations, hostId);
if (operations.length === 0) return <p className="py-2 text-sm text-slate-500">{t("cleanup.hist.empty")}</p>;
return (
<div className="space-y-3">
{operations.map((operation) => (
<Card key={operation.id} className="p-4">
<div className="flex flex-wrap items-center gap-2">
<h4 className="text-sm font-semibold text-slate-900">{t(`cleanup.op.title.${operation.kind}`)}</h4>
<Tooltip content={t(`cleanup.status.${operation.status}_hint`)} focusable>
<Badge tone={STATUS_TONE[operation.status]}>{t(`cleanup.status.${operation.status}`)}</Badge>
</Tooltip>
<span className="ml-auto text-xs text-slate-500">
{operation.actor}{operation.created_at ? ` · ${new Date(operation.created_at).toLocaleString(locale)}` : ""}
</span>
</div>
{ownMessage(t, operation) && <p className="mt-1.5 break-words text-xs text-slate-600">{ownMessage(t, operation)}</p>}
<ItemRows
operation={operation}
actions={readOnly ? undefined : (item) => {
const own = item.host_id === hostId && latest.has(item.id);
const buttons: ReactNode[] = [];
if (item.restorable && item.host_id === hostId) {
buttons.push(
<Tooltip key="restore" content={notAdmin ? t("cleanup.hist.adminOnly") : t("cleanup.hist.restoreHint")} focusable>
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={notAdmin || busy} onClick={() => onRestore(item)}>{t("cleanup.hist.restore")}</Button>
</Tooltip>,
);
}
const reenable = own && ((operation.kind === "disable" && item.state === "confirmed") || (operation.kind === "restore" && item.state === "restored"));
if (own && operation.kind === "disable" && item.state === "confirmed") {
const due = operation.confirmed_at ? new Date(operation.confirmed_at).getTime() + minDays * 86400000 : 0;
const waiting = due > now;
buttons.push(
<Tooltip key="delete" content={waiting ? t("cleanup.hist.deleteWait", { when: new Date(due).toLocaleDateString(locale) }) : t("cleanup.hist.deleteHint")} focusable>
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={waiting || busy} onClick={() => onFollowUp("delete", item, currentUuid(item, operation.kind))}>{t("cleanup.hist.delete")}</Button>
</Tooltip>,
);
}
if (reenable) {
buttons.push(
<Tooltip key="enable" content={t("cleanup.hist.enableHint")} focusable>
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={busy} onClick={() => onFollowUp("enable", item, currentUuid(item, operation.kind))}>{t("cleanup.hist.enable")}</Button>
</Tooltip>,
);
}
return buttons;
}}
/>
</Card>
))}
</div>
);
}
+338
View File
@@ -0,0 +1,338 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { ConfidenceBadge, KIND_TONE, NoteBadge } from "@/components/CleanupBadges";
import { RuleHistoryModal } from "@/components/RuleHistoryModal";
import { Badge } from "@/components/ui/Badge";
import { Button } from "@/components/ui/Button";
import { Input } from "@/components/ui/Field";
import { Loading } from "@/components/ui/Loading";
import { Modal } from "@/components/ui/Modal";
import { useToast } from "@/components/ui/Toast";
import { InfoTip, Tooltip } from "@/components/ui/Tooltip";
import { api, CleanupActionKind, CleanupOperation, CleanupPlanItem, CleanupPlanNode, CleanupPreview } from "@/lib/api";
import { ACK_TEXT_VERSION, formatClock, secondsUntil } from "@/lib/cleanup";
import { useNotAdmin } from "@/lib/useReadOnly";
import { History, RefreshCw, ShieldCheck, TriangleAlert } from "lucide-react";
import { useCallback, useEffect, useId, useRef, useState } from "react";
/** What the user picked: the kind of run and the rules (needsAck = "probable", an explicit acknowledgement each). */
export interface CleanupRequest {
kind: CleanupActionKind;
rules: { uuid: string; label: string; needsAck: boolean }[];
}
const ACTION_TONE = { disable: "warning", enable: "info", delete: "danger" } as const;
function duration(t: (key: string, params?: Record<string, string | number>) => string, seconds: number): string {
return seconds >= 60 ? t("cleanup.dur.minutes", { n: Math.round(seconds / 60) }) : t("cleanup.dur.seconds", { n: seconds });
}
function PlanItem({ item, kind, sources, coverName, acknowledged, onHistory }: {
item: CleanupPlanItem; kind: CleanupActionKind; sources: string[]; coverName: (uuid: string) => string | null;
acknowledged: boolean; onHistory: () => void;
}) {
const { t, locale } = useLanguage();
const protection = item.protection;
const cover = item.cover_uuid ? coverName(item.cover_uuid) : null;
const notes = Array.from(new Set([...item.notes, ...item.categories.flatMap((c) => c.notes ?? [])]));
return (
<li className="rounded-xl border border-slate-200 p-3 text-sm">
<div className="flex flex-wrap items-start gap-x-2 gap-y-1">
<span className="min-w-0 flex-1 break-words font-medium text-slate-900">{item.description}</span>
<button
type="button"
onClick={onHistory}
className="shrink-0 rounded p-1 text-slate-400 hover:bg-slate-100 hover:text-slate-700 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
aria-label={t("cleanup.history", { rule: item.description })}
title={t("cleanup.historyTitle")}
>
<History size={15} aria-hidden="true" />
</button>
<Badge tone={ACTION_TONE[kind]}>{t(`cleanup.action.${kind}`)}</Badge>
</div>
{item.categories.length > 0 && (
<div className="mt-2 flex flex-wrap items-center gap-1.5">
{item.categories.map((category) => (
<span key={category.kind} className="inline-flex items-center gap-1">
<Tooltip content={t(`cleanup.category.${category.kind}_hint`)} focusable>
<Badge tone={KIND_TONE[category.kind]}>{t(`cleanup.category.${category.kind}`)}</Badge>
</Tooltip>
<ConfidenceBadge confidence={category.confidence} />
</span>
))}
</div>
)}
{cover && <p className="mt-1.5 break-words text-xs text-slate-500">{t("cleanup.detail.by", { rule: cover })}</p>}
{item.basis?.type === "confirmed" && item.basis.confirmed_at && (
<p className="mt-1.5 text-xs text-slate-500">{t("cleanup.preview.basisConfirmed", { when: new Date(item.basis.confirmed_at).toLocaleDateString(locale) })}</p>
)}
{item.basis?.type === "disabled_long" && (
<p className="mt-1.5 text-xs text-slate-500">{item.basis.since && !Number.isNaN(Date.parse(item.basis.since))
? t("cleanup.detail.disabledSince", { when: new Date(item.basis.since).toLocaleDateString(locale) })
: t("cleanup.preview.basisDisabled")}</p>
)}
<div className="mt-2 flex flex-wrap items-center gap-1.5">
{protection && (
<Tooltip
content={t(`cleanup.protect.${protection.result}_hint`, { sources: sources.length ? sources.join(", ") : t("cleanup.protect.sourcesHidden"), queries: protection.queries })}
focusable
>
<Badge tone={protection.result === "false" ? "success" : "warning"}>
{protection.result === "false" ? <ShieldCheck size={12} aria-hidden="true" /> : <TriangleAlert size={12} aria-hidden="true" />}
{t(`cleanup.protect.${protection.result}`)}
</Badge>
</Tooltip>
)}
{item.external && (
<Tooltip content={t("cleanup.preview.externalHint")} focusable><Badge tone="warning">{t("cleanup.preview.external")}</Badge></Tooltip>
)}
{notes.map((code) => <NoteBadge key={code} code={code} />)}
{acknowledged && <Badge tone="neutral">{t("cleanup.preview.acknowledged")}</Badge>}
</div>
</li>
);
}
/**
* Select -> (acknowledge) -> preview -> roll out. The preview is recomputed from the live firewall(s) by the backend,
* valid for five minutes and bound to whoever created it; only an admin may roll it out (hostname typed). The
* confirmation and the automatic revert afterwards live in the operation banner of the tab.
*/
export function CleanupPreviewModal({ host, request, unusedDays, coverName, onClose, onDeployed }: {
host: { id: string; name: string };
request: CleanupRequest;
unusedDays: number;
coverName: (uuid: string) => string | null;
onClose: () => void;
onDeployed: (operation: CleanupOperation) => void;
}) {
const { t, errorMessage } = useLanguage();
const toast = useToast();
const notAdmin = useNotAdmin();
const [forced, setForced] = useState<string[]>([]);
const [acked, setAcked] = useState<string[]>([]);
const [phase, setPhase] = useState<"ack" | "loading" | "plan" | "deploying">(request.rules.some((r) => r.needsAck) ? "ack" : "loading");
const [preview, setPreview] = useState<CleanupPreview | null>(null);
const [error, setError] = useState<string | null>(null);
const [confirmName, setConfirmName] = useState("");
const [now, setNow] = useState(() => Date.now());
const [history, setHistory] = useState<CleanupPlanItem | null>(null);
const [historyNode, setHistoryNode] = useState<CleanupPlanNode | null>(null);
const spent = useRef(false);
const previewId = useRef<string | null>(null);
const nameErrorId = useId();
const needsAck = useCallback((uuid: string) => forced.includes(uuid) || request.rules.some((r) => r.uuid === uuid && r.needsAck), [forced, request.rules]);
const askAck = request.rules.filter((r) => needsAck(r.uuid));
const allAcked = askAck.every((r) => acked.includes(r.uuid));
const compute = useCallback(async (ackList: string[]) => {
setPhase("loading");
setError(null);
setPreview(null);
spent.current = false;
try {
const result = await api.previewCleanup(host.id, {
items: request.rules.map((r) => ({ uuid: r.uuid, kind: request.kind, ack: ackList.includes(r.uuid) })),
unused_days: unusedDays,
ack_text_version: ACK_TEXT_VERSION,
});
previewId.current = result.preview_id;
setPreview(result);
setNow(Date.now());
setPhase("plan");
} catch (e) {
const code = e && typeof e === "object" && "code" in e ? (e as { code?: string }).code : undefined;
const params = e && typeof e === "object" && "params" in e ? (e as { params?: Record<string, string> }).params : undefined;
if (code === "BEREINIGUNG_BESTAETIGUNG_FEHLT" && params?.uuids) {
// The backend sees more reasons for caution than the list did (e.g. an outside change): ask for those too.
setForced((current) => Array.from(new Set([...current, ...params.uuids.split(",").map((u) => u.trim())])));
setPhase("ack");
} else {
setPhase("plan");
}
setError(errorMessage(e, "cleanup.preview.failed"));
}
}, [host.id, request, unusedDays, errorMessage]);
useEffect(() => {
if (!request.rules.some((r) => r.needsAck)) void compute([]);
// Only once at open: later recomputation is the user's explicit action.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
useEffect(() => {
if (phase !== "plan" || !preview) return;
const timer = setInterval(() => setNow(Date.now()), 1000);
return () => clearInterval(timer);
}, [phase, preview]);
// An unused preview is dropped on the way out (a new one replaces it anyway; this keeps the list tidy).
useEffect(() => () => {
if (!spent.current && previewId.current) void api.cancelCleanup(host.id, previewId.current).catch(() => {});
}, [host.id]);
const left = preview ? secondsUntil(preview.expires_at, now) : 0;
const expired = phase === "plan" && preview !== null && left === 0;
const nodes = preview ? [...preview.plan.nodes].sort((a, b) => Number(b.role === "BACKUP") - Number(a.role === "BACKUP")) : [];
const clustered = nodes.length > 1;
const window = preview ? duration(t, preview.window_seconds) : "";
const nameOk = confirmName === host.name;
async function deploy() {
if (!preview || !nameOk || notAdmin) return;
setPhase("deploying");
setError(null);
try {
spent.current = true;
const operation = await api.deployCleanup(host.id, preview.preview_id, confirmName);
toast.show(t(operation.status === "awaiting_confirm" ? "cleanup.toast.deployedConfirm" : operation.status === "done" ? "cleanup.toast.deployedDone" : "cleanup.toast.deployedCheck"),
operation.status === "awaiting_confirm" || operation.status === "done" ? "success" : "info");
onDeployed(operation);
} catch (e) {
// The backend consumes the preview at the first moment it is looked at: a failed roll-out needs a new one.
setError(errorMessage(e, "cleanup.deploy.failed"));
toast.show(errorMessage(e, "cleanup.deploy.failed"), "danger");
setPreview(null);
setPhase("plan");
}
}
const title = t(`cleanup.preview.title.${request.kind}`);
const hasPlan = phase === "plan" && preview !== null;
const deployDisabledReason = notAdmin ? t("cleanup.preview.deployNotAdmin") : expired ? t("cleanup.preview.expired") : !nameOk ? t("cleanup.preview.deployDisabledName") : null;
return (
<>
<Modal title={title} wide onClose={() => { if (phase !== "deploying" && !history) onClose(); }}>
<div className="space-y-4">
<p className="text-sm text-slate-600">
{t(`cleanup.preview.intro.${request.kind}`, { host: host.name })}
<InfoTip text={t(`cleanup.preview.introHint.${request.kind}`)} size={13} className="ml-1 align-middle" />
</p>
{phase === "ack" && (
<div className="space-y-3">
<p className="text-sm font-medium text-slate-800">{t(request.kind === "enable" ? "cleanup.ack.enableTitle" : "cleanup.ack.title")}</p>
<ul className="space-y-2">
{askAck.map((rule) => (
<li key={rule.uuid} className="rounded-xl border border-slate-200 p-3">
<label className="flex items-start gap-2.5 text-sm">
<input
type="checkbox"
className="mt-0.5 h-4 w-4 shrink-0 rounded border-slate-300"
checked={acked.includes(rule.uuid)}
onChange={(e) => setAcked((current) => e.target.checked ? [...current, rule.uuid] : current.filter((u) => u !== rule.uuid))}
/>
<span className="min-w-0">
<span className="block break-words font-medium text-slate-900">{rule.label}</span>
<span className="block text-slate-600">{t(request.kind === "enable" ? "cleanup.ack.enableText" : "cleanup.ack.text", { rule: rule.label })}</span>
</span>
</label>
</li>
))}
</ul>
<div className="flex flex-wrap gap-2">
<Button onClick={() => void compute(acked)} disabled={!allAcked}>{t("cleanup.preview.compute")}</Button>
<Button variant="ghost" onClick={onClose}>{t("common.cancel")}</Button>
</div>
</div>
)}
{phase === "loading" && <Loading>{t("cleanup.preview.loading")}</Loading>}
{error && <p role="alert" className="break-words rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}
{phase === "plan" && !preview && (
<div className="flex flex-wrap gap-2">
<Button variant="outline" onClick={() => void compute(acked)}><RefreshCw size={14} aria-hidden="true" />{t("cleanup.preview.again")}</Button>
<Button variant="ghost" onClick={onClose}>{t("common.close")}</Button>
</div>
)}
{(hasPlan || phase === "deploying") && preview && (
<div className="space-y-4">
<div className="flex flex-wrap items-center justify-between gap-2 text-xs text-slate-500">
<span>{expired
? <span role="alert" className="font-medium text-red-600">{t("cleanup.preview.expired")}</span>
: t("cleanup.preview.validFor", { time: formatClock(left) })}</span>
{clustered && <span>{t("cleanup.preview.clusterOrder")}</span>}
</div>
{nodes.map((node, index) => (
<section key={node.host_id} aria-label={node.host_name} className="space-y-2">
<h3 className="flex flex-wrap items-center gap-2 text-sm font-semibold text-slate-800">
{clustered && <span className="text-slate-400">{index + 1}.</span>}
{node.host_name}
{node.role && <Badge tone={node.role === "MASTER" ? "info" : "neutral"}>{node.role}</Badge>}
</h3>
<ul className="space-y-2">
{node.items.map((item) => (
<PlanItem
key={item.uuid}
item={item}
kind={preview.kind}
sources={preview.plan.sources ?? []}
coverName={coverName}
acknowledged={preview.plan.acknowledged.includes(item.given)}
onHistory={() => { setHistory(item); setHistoryNode(node); }}
/>
))}
</ul>
</section>
))}
<div className="space-y-3 border-t border-slate-200 pt-4">
{request.kind !== "delete" && (
<p className="flex items-start gap-1.5 rounded-lg bg-flame-50 p-3 text-sm text-flame-800">
<TriangleAlert size={16} className="mt-0.5 shrink-0" aria-hidden="true" />
<span>{t("cleanup.preview.autoRevert", { window })}</span>
</p>
)}
{!notAdmin && (
<label className="block text-sm">
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.preview.confirmName")}<InfoTip text={t("cleanup.preview.confirmNameHint")} size={13} /></span>
<Input
autoComplete="off"
spellCheck={false}
aria-label={t("cleanup.preview.confirmNameAria", { host: host.name })}
aria-describedby={confirmName && !nameOk ? nameErrorId : undefined}
value={confirmName}
placeholder={host.name}
disabled={phase === "deploying"}
onChange={(e) => setConfirmName(e.target.value)}
onKeyDown={(e) => { if (e.key === "Enter" && nameOk && !expired && phase === "plan") void deploy(); }}
/>
{confirmName && !nameOk && <span id={nameErrorId} className="mt-1 block text-xs text-slate-500">{t("cleanup.preview.deployDisabledName")}</span>}
</label>
)}
<div className="flex flex-wrap items-center gap-2">
<Tooltip content={deployDisabledReason} focusable={deployDisabledReason !== null}>
<Button
variant={request.kind === "delete" ? "danger" : "primary"}
onClick={() => void deploy()}
disabled={deployDisabledReason !== null || phase === "deploying"}
loading={phase === "deploying"}
>
{t(`cleanup.deploy.${request.kind}`)}
</Button>
</Tooltip>
{expired && <Button variant="outline" onClick={() => void compute(acked)}><RefreshCw size={14} aria-hidden="true" />{t("cleanup.preview.again")}</Button>}
<Button variant="ghost" onClick={onClose} disabled={phase === "deploying"}>{t("common.cancel")}</Button>
{notAdmin && <InfoTip text={t("cleanup.preview.deployNotAdmin")} size={15} />}
</div>
</div>
</div>
)}
</div>
</Modal>
{history && historyNode && (
<RuleHistoryModal
title={history.description}
targets={[{ hostId: historyNode.host_id, hostName: historyNode.host_name, uuid: history.uuid }]}
onClose={() => { setHistory(null); setHistoryNode(null); }}
/>
)}
</>
);
}
@@ -0,0 +1,95 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { Button } from "@/components/ui/Button";
import { Input } from "@/components/ui/Field";
import { Modal } from "@/components/ui/Modal";
import { useToast } from "@/components/ui/Toast";
import { InfoTip } from "@/components/ui/Tooltip";
import { api, CleanupItem, CleanupOperation } from "@/lib/api";
import { cleanupMessage } from "@/lib/cleanup";
import { CircleCheck, TriangleAlert } from "lucide-react";
import { useId, useState } from "react";
/**
* Bring a deleted rule back from the content backup OPNexus took before deleting it. The rule returns switched OFF
* (a new UUID, at its old position where that still exists); switching it on is a separate step with a confirmation
* window. Admin only, hostname typed.
*/
export function CleanupRestoreModal({ host, item, onClose, onRestored }: {
host: { id: string; name: string }; item: CleanupItem; onClose: () => void; onRestored: (operation: CleanupOperation) => void;
}) {
const { t, errorMessage } = useLanguage();
const toast = useToast();
const [confirmName, setConfirmName] = useState("");
const [busy, setBusy] = useState(false);
const [error, setError] = useState<string | null>(null);
const [result, setResult] = useState<CleanupOperation | null>(null);
const hintId = useId();
const nameOk = confirmName === host.name;
async function restore() {
if (!nameOk) return;
setBusy(true);
setError(null);
try {
const operation = await api.restoreCleanupItem(host.id, item.id, confirmName);
setResult(operation);
onRestored(operation);
if (operation.status === "done") toast.show(t("cleanup.restore.done", { rule: item.description ?? item.rule_uuid }), "success");
else toast.show(cleanupMessage(t, operation.items.find((i) => i.message)?.message ?? operation.message) || t("cleanup.restore.failed"), "danger");
} catch (e) {
setError(errorMessage(e, "cleanup.restore.failed"));
toast.show(errorMessage(e, "cleanup.restore.failed"), "danger");
} finally {
setBusy(false);
}
}
return (
<Modal title={t("cleanup.restore.title")} onClose={() => { if (!busy) onClose(); }}>
<div className="space-y-4">
<p className="break-words font-medium text-slate-800">{item.description || item.rule_uuid}</p>
{!result && (
<>
<p className="text-sm text-slate-600">
{t("cleanup.restore.intro")}
<InfoTip text={t("cleanup.restore.introHint")} size={13} className="ml-1 align-middle" />
</p>
<label className="block text-sm">
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.preview.confirmName")}<InfoTip text={t("cleanup.preview.confirmNameHint")} size={13} /></span>
<Input
autoComplete="off"
spellCheck={false}
aria-label={t("cleanup.preview.confirmNameAria", { host: host.name })}
aria-describedby={confirmName && !nameOk ? hintId : undefined}
value={confirmName}
placeholder={host.name}
disabled={busy}
onChange={(e) => setConfirmName(e.target.value)}
onKeyDown={(e) => { if (e.key === "Enter" && nameOk && !busy) void restore(); }}
/>
{confirmName && !nameOk && <span id={hintId} className="mt-1 block text-xs text-slate-500">{t("cleanup.preview.deployDisabledName")}</span>}
</label>
</>
)}
{error && <p role="alert" className="break-words rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}
{result && (
<div role="status" className="space-y-2 text-sm">
{result.status === "done"
? <p className="flex items-start gap-1.5 text-emerald-800"><CircleCheck size={16} className="mt-0.5 shrink-0" aria-hidden="true" />{t("cleanup.restore.done", { rule: item.description ?? item.rule_uuid })}</p>
: <p className="break-words rounded-lg bg-red-50 p-3 text-red-700">{cleanupMessage(t, result.items.find((i) => i.message)?.message ?? result.message) || t("cleanup.restore.failed")}</p>}
{result.warnings.map((code) => (
<p key={code} className="flex items-start gap-1.5 rounded-lg bg-flame-50 p-3 text-flame-800"><TriangleAlert size={16} className="mt-0.5 shrink-0" aria-hidden="true" />{cleanupMessage(t, code)}</p>
))}
<p className="text-slate-600">{t("cleanup.restore.next")}</p>
</div>
)}
<div className="flex flex-wrap gap-2">
{!result && <Button onClick={() => void restore()} disabled={!nameOk} loading={busy}>{t("cleanup.restore.action")}</Button>}
<Button variant={result ? "primary" : "ghost"} onClick={onClose} disabled={busy}>{result ? t("common.close") : t("common.cancel")}</Button>
</div>
</div>
</Modal>
);
}
+164
View File
@@ -0,0 +1,164 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { Button } from "@/components/ui/Button";
import { Input } from "@/components/ui/Field";
import { InfoTip } from "@/components/ui/Tooltip";
import { useToast } from "@/components/ui/Toast";
import { api, GovernanceSettings } from "@/lib/api";
import { parsePorts, parseSources } from "@/lib/cleanup";
import { useNotAdmin } from "@/lib/useReadOnly";
import clsx from "clsx";
import { useEffect, useId, useState } from "react";
/**
* Settings of the rule cleanup: the management sources (addresses the firewalls are managed from; without them the
* cleanup is locked, because a rule that could cut them off is never switched off), the rule limit per run and the
* waiting time before a rule may be deleted. Admin only: renders nothing for other roles. `compact` is the inline
* variant of the cleanup tab, which only asks for the sources.
*/
export function CleanupSettings({ compact = false, onSaved }: { compact?: boolean; onSaved?: (settings: GovernanceSettings) => void }) {
const { t, errorMessage } = useLanguage();
const toast = useToast();
const notAdmin = useNotAdmin();
const [settings, setSettings] = useState<GovernanceSettings | null>(null);
const [sources, setSources] = useState("");
const [maxRules, setMaxRules] = useState("10");
const [minDays, setMinDays] = useState("7");
const [ports, setPorts] = useState("");
const [touched, setTouched] = useState(false);
const [busy, setBusy] = useState(false);
const [error, setError] = useState("");
const sourcesId = useId();
const portsId = useId();
const portsErrorId = useId();
const errorId = useId();
function apply(data: GovernanceSettings) {
setSettings(data);
setSources(data.cleanup_management_sources.join("\n"));
setMaxRules(String(data.cleanup_max_rules));
setMinDays(String(data.cleanup_delete_min_days));
setPorts((data.cleanup_management_ports ?? []).join(", "));
setTouched(false);
}
useEffect(() => {
api.getGovernance().then(apply).catch(() => {});
}, []);
// Reached from the cleanup tab via /settings#cleanup: the section only exists once the page has loaded.
useEffect(() => {
if (!compact && settings && !notAdmin && window.location.hash === "#cleanup") document.getElementById("cleanup")?.scrollIntoView({ block: "start" });
}, [compact, settings, notAdmin]);
const parsed = parseSources(sources);
const sourceError = parsed.error === "invalid" ? t("cleanup.settings.sourcesInvalid", { value: parsed.bad ?? "" })
: parsed.error === "everything" ? t("cleanup.settings.sourcesEverything", { value: parsed.bad ?? "" })
: parsed.error === "tooMany" ? t("cleanup.settings.sourcesTooMany") : "";
const maxValue = Number(maxRules);
const daysValue = Number(minDays);
const maxError = !Number.isInteger(maxValue) || maxValue < 1 || maxValue > 25 ? t("cleanup.settings.maxRulesInvalid") : "";
const daysError = !Number.isInteger(daysValue) || daysValue < 1 || daysValue > 365 ? t("cleanup.settings.minDaysInvalid") : "";
const parsedPorts = parsePorts(ports);
const portsError = parsedPorts.error === "invalid" ? t("cleanup.settings.portsInvalid", { value: parsedPorts.bad ?? "" })
: parsedPorts.error === "tooMany" ? t("cleanup.settings.portsTooMany") : "";
const invalid = Boolean(sourceError || (!compact && (maxError || daysError || portsError)));
const dirty = settings !== null && (parsed.entries.join("\n") !== settings.cleanup_management_sources.join("\n")
|| (!compact && (maxValue !== settings.cleanup_max_rules || daysValue !== settings.cleanup_delete_min_days
|| parsedPorts.ports.join(",") !== (settings.cleanup_management_ports ?? []).join(","))));
if (notAdmin) return null;
async function save(event: React.FormEvent) {
event.preventDefault();
setTouched(true);
if (invalid) return;
setBusy(true);
setError("");
try {
const saved = await api.updateCleanupSettings({
cleanup_management_sources: parsed.entries,
...(compact ? {} : { cleanup_max_rules: maxValue, cleanup_delete_min_days: daysValue, cleanup_management_ports: parsedPorts.ports }),
});
apply(saved);
toast.show(t("cleanup.settings.saved"), "success");
onSaved?.(saved);
} catch (e) {
setError(errorMessage(e, "common.saveFailed"));
} finally {
setBusy(false);
}
}
const form = (
<form onSubmit={save} className="space-y-4" noValidate>
<div>
<label htmlFor={sourcesId} className="mb-1 flex items-center gap-1 text-sm font-medium text-slate-700">
{t("cleanup.settings.sources")}
<InfoTip text={t("cleanup.settings.sourcesHint")} size={13} />
</label>
<textarea
id={sourcesId}
rows={compact ? 2 : 3}
value={sources}
onChange={(e) => setSources(e.target.value)}
onBlur={() => setTouched(true)}
placeholder={t("cleanup.settings.sourcesPlaceholder")}
spellCheck={false}
autoComplete="off"
aria-invalid={touched && Boolean(sourceError)}
aria-describedby={touched && sourceError ? errorId : undefined}
className={clsx(
"w-full rounded-lg border px-3 py-2 font-mono text-sm text-slate-900 shadow-sm focus:outline-none focus:ring-1",
touched && sourceError ? "border-red-400 focus:border-red-500 focus:ring-red-500" : "border-slate-300 focus:border-brand-500 focus:ring-brand-500",
)}
/>
{touched && sourceError && <p id={errorId} role="alert" className="mt-1 text-xs text-red-600">{sourceError}</p>}
</div>
{!compact && (
<div>
<label htmlFor={portsId} className="mb-1 flex items-center gap-1 text-sm font-medium text-slate-700">
{t("cleanup.settings.ports")}
<InfoTip text={t("cleanup.settings.portsHint")} size={13} />
</label>
<Input
id={portsId}
inputMode="numeric"
value={ports}
onChange={(e) => setPorts(e.target.value)}
placeholder={t("cleanup.settings.portsPlaceholder")}
autoComplete="off"
aria-invalid={Boolean(portsError)}
aria-describedby={portsError ? portsErrorId : undefined}
/>
{portsError && <p id={portsErrorId} role="alert" className="mt-1 text-xs text-red-600">{portsError}</p>}
</div>
)}
{!compact && (
<div className="grid gap-4 sm:grid-cols-2">
<label className="block text-sm">
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.settings.maxRules")}<InfoTip text={t("cleanup.settings.maxRulesHint")} size={13} /></span>
<Input type="number" inputMode="numeric" min={1} max={25} value={maxRules} onChange={(e) => setMaxRules(e.target.value)} aria-invalid={Boolean(maxError)} aria-label={t("cleanup.settings.maxRules")} />
{maxError && <span role="alert" className="mt-1 block text-xs text-red-600">{maxError}</span>}
</label>
<label className="block text-sm">
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.settings.minDays")}<InfoTip text={t("cleanup.settings.minDaysHint")} size={13} /></span>
<Input type="number" inputMode="numeric" min={1} max={365} value={minDays} onChange={(e) => setMinDays(e.target.value)} aria-invalid={Boolean(daysError)} aria-label={t("cleanup.settings.minDays")} />
{daysError && <span role="alert" className="mt-1 block text-xs text-red-600">{daysError}</span>}
</label>
</div>
)}
{error && <p role="alert" className="text-sm text-red-600">{error}</p>}
<Button type="submit" loading={busy} disabled={!dirty || (touched && invalid)}>{t("common.save")}</Button>
</form>
);
if (compact) return form;
return (
<section id="cleanup" className="space-y-4 rounded-xl border border-slate-200 bg-white p-5">
<h2 className="flex items-center gap-2 text-lg font-medium">{t("cleanup.settings.title")}<InfoTip text={t("cleanup.settings.intro")} /></h2>
{settings === null ? <p className="text-sm text-slate-500">{t("common.loading")}</p> : form}
</section>
);
}
+28 -4
View File
@@ -1,5 +1,6 @@
"use client";
import { HistoryTarget, RuleHistoryModal } from "@/components/RuleHistoryModal";
import { Card } from "@/components/ui/Card";
import { EmptyState } from "@/components/ui/EmptyState";
import { Skeleton } from "@/components/ui/Skeleton";
@@ -8,7 +9,7 @@ import { api, Host, LiveRule } from "@/lib/api";
import { useLanguage } from "@/components/LanguageProvider";
import { groupHostsByCluster } from "@/lib/clusters";
import clsx from "clsx";
import { Check, Network, X } from "lucide-react";
import { Check, History, Network, X } from "lucide-react";
import Link from "next/link";
import { useEffect, useState } from "react";
@@ -40,6 +41,8 @@ interface DriftRow {
fingerprint: string;
sample: LiveRule;
presentOn: Set<string>;
// The rule's id on each firewall (they differ between firewalls); the change history is kept per firewall and id.
uuids: Map<string, string>;
}
interface ClusterState {
@@ -61,18 +64,19 @@ async function computeClusterDrift(clusterHosts: Host[]): Promise<Omit<ClusterSt
);
const reachable = perHostRules.filter((r): r is { host: Host; rules: LiveRule[] } => r.rules !== null);
const byFingerprint = new Map<string, { sample: LiveRule; presentOn: Set<string> }>();
const byFingerprint = new Map<string, { sample: LiveRule; presentOn: Set<string>; uuids: Map<string, string> }>();
for (const { host, rules } of reachable) {
for (const rule of rules) {
const fp = fingerprint(rule);
const entry = byFingerprint.get(fp) ?? { sample: rule, presentOn: new Set<string>() };
const entry = byFingerprint.get(fp) ?? { sample: rule, presentOn: new Set<string>(), uuids: new Map<string, string>() };
entry.presentOn.add(host.id);
if (!entry.uuids.has(host.id)) entry.uuids.set(host.id, rule.uuid);
byFingerprint.set(fp, entry);
}
}
const allRows = Array.from(byFingerprint.entries())
.map(([fp, entry]) => ({ fingerprint: fp, sample: entry.sample, presentOn: entry.presentOn }))
.map(([fp, entry]) => ({ fingerprint: fp, sample: entry.sample, presentOn: entry.presentOn, uuids: entry.uuids }))
.sort((a, b) => a.sample.description.localeCompare(b.sample.description));
const driftedRows = allRows.filter((row) => row.presentOn.size < reachable.length);
@@ -85,6 +89,7 @@ export function LiveRulesDrift() {
const [ungrouped, setUngrouped] = useState<Host[]>([]);
const [error, setError] = useState<string | null>(null);
const [onlyDrift, setOnlyDrift] = useState(false);
const [historyOf, setHistoryOf] = useState<{ title: string; targets: HistoryTarget[] } | null>(null);
useEffect(() => {
api
@@ -178,6 +183,7 @@ export function LiveRulesDrift() {
{host.name}
</th>
))}
<th className="px-5 py-3 font-medium"><span className="sr-only">{t("common.actions")}</span></th>
</tr>
</thead>
<tbody>
@@ -209,6 +215,22 @@ export function LiveRulesDrift() {
)}
</td>
))}
<td className="px-5 py-3 text-right">
{!row.sample.is_automatic && (
<button
type="button"
onClick={() => setHistoryOf({
title: row.sample.description,
targets: cluster.hosts.filter((h) => row.uuids.has(h.id)).map((h) => ({ hostId: h.id, hostName: h.name, uuid: row.uuids.get(h.id) as string })),
})}
className="inline-flex items-center gap-1 rounded-lg px-2 py-1 text-xs text-slate-600 hover:bg-slate-100 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
aria-label={t("ruleHistory.open", { rule: row.sample.description })}
>
<History size={14} aria-hidden="true" />
{t("ruleHistory.action")}
</button>
)}
</td>
</tr>
))}
</tbody>
@@ -219,6 +241,8 @@ export function LiveRulesDrift() {
);
})}
{historyOf && <RuleHistoryModal title={historyOf.title} targets={historyOf.targets} onClose={() => setHistoryOf(null)} />}
{ungrouped.length > 0 && (
<div>
<h3 className="mb-2 text-lg font-semibold text-slate-900">{t("drift.noCluster")}</h3>
+1 -1
View File
@@ -69,7 +69,7 @@ export const navGroups: NavGroup[] = [
icon: Shield,
items: [
{ href: "/rules", label: "Regeln", icon: Shield, keywords: ["Firewall-Regeln", "Filter"] },
{ href: "/analysis", label: "Regel-Analyse", icon: ScanSearch, keywords: ["Welche Regel", "Welche Regel trifft", "Pfad", "Simulation", "Firewall-Regel prüfen", "Paket-Test", "Rule match", "Which rule", "Rule analysis", "Packet path", "Trace"] },
{ href: "/analysis", label: "Regel-Analyse", icon: ScanSearch, keywords: ["Welche Regel", "Welche Regel trifft", "Pfad", "Simulation", "Firewall-Regel prüfen", "Paket-Test", "Rule match", "Which rule", "Rule analysis", "Packet path", "Trace", "Bereinigung", "ungenutzte Regeln", "verdeckte Regeln", "doppelte Regeln", "Shadow", "Regelverlauf", "Cleanup", "Unused rules", "Shadowed rules", "Duplicate rules", "Rule history"] },
{ href: "/policy-hierarchy", label: "Policy-Hierarchie", icon: GitBranch, keywords: ["Pre-Rules", "Post-Rules", "Vererbung", "Inheritance", "Hierarchische Policies"] },
{ href: "/groups", label: "Gruppen", icon: FolderTree, keywords: ["Gerätegruppen", "Device Groups"] },
{ href: "/aliases", label: "Aliase", icon: Tags, keywords: ["Netzwerk-Objekte", "Netzwerkgruppen"] },
+490
View File
@@ -0,0 +1,490 @@
"use client";
import { ConfidenceBadge, KIND_TONE } from "@/components/CleanupBadges";
import { CleanupPreviewModal, CleanupRequest } from "@/components/CleanupPreviewModal";
import { CleanupRestoreModal } from "@/components/CleanupRestoreModal";
import { CleanupSettings } from "@/components/CleanupSettings";
import { OperationBanner, OperationHistory } from "@/components/CleanupOperations";
import { useLanguage } from "@/components/LanguageProvider";
import { RuleHistoryModal } from "@/components/RuleHistoryModal";
import { Badge } from "@/components/ui/Badge";
import { Button } from "@/components/ui/Button";
import { Card } from "@/components/ui/Card";
import { Collapsible } from "@/components/ui/Collapsible";
import { EmptyState } from "@/components/ui/EmptyState";
import { FieldGroup, Select } from "@/components/ui/Field";
import { Skeleton } from "@/components/ui/Skeleton";
import { useToast } from "@/components/ui/Toast";
import { InfoTip, Tooltip } from "@/components/ui/Tooltip";
import { api, CleanupActionKind, CleanupCandidate, CleanupCategory, CleanupItem, CleanupKind, CleanupOperation, GovernanceSettings, Host, RuleCleanupResult } from "@/lib/api";
import { activeOperation, historyOperations, isOpen, selectability } from "@/lib/cleanup";
import { useNotAdmin, useReadOnly } from "@/lib/useReadOnly";
import { History, Info, ListChecks, RefreshCw, Settings } from "lucide-react";
import Link from "next/link";
import { useRouter, useSearchParams } from "next/navigation";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
const THRESHOLDS = [7, 30, 90, 180];
const KINDS: CleanupKind[] = ["unused", "shadowed", "duplicate", "disabled_long"];
function CategoryLine({ category, days }: { category: CleanupCategory; days: number }) {
const { t, locale } = useLanguage();
let detail: string | null = null;
if (category.kind === "unused") detail = t("cleanup.detail.unused", { days });
else if (category.kind === "disabled_long") {
detail = category.since ? t("cleanup.detail.disabledSince", { when: new Date(category.since).toLocaleDateString(locale) }) : t("cleanup.detail.disabled", { days });
} else if (category.by) {
detail = t(category.by.automatic ? "cleanup.detail.byAutomatic" : "cleanup.detail.by", { rule: category.by.description || category.by.uuid || "?" });
}
return (
<div className="flex flex-wrap items-center gap-x-2 gap-y-1">
<Tooltip content={t(`cleanup.category.${category.kind}_hint`)} focusable>
<Badge tone={KIND_TONE[category.kind]}>{t(`cleanup.category.${category.kind}`)}</Badge>
</Tooltip>
<ConfidenceBadge confidence={category.confidence} />
{detail && <span className="min-w-0 break-words text-xs text-slate-500">{detail}</span>}
</div>
);
}
/** What the row offers: the action a selection would run (and whether it needs the rule's own acknowledgement), or why not. */
function ActionCell({ candidate, unusedDays }: { candidate: CleanupCandidate; unusedDays: number }) {
const { t } = useLanguage();
const sel = selectability(candidate, unusedDays);
if (sel.mode) {
return (
<div className="flex flex-col items-start gap-1">
<Badge tone={sel.mode === "delete" ? "danger" : "neutral"}>{t(`cleanup.action.${sel.mode}`)}</Badge>
{sel.mode === "disable" && sel.needsAck && (
<Tooltip content={t("cleanup.select.needsAckHint")} focusable><Badge tone="warning">{t("cleanup.select.needsAck")}</Badge></Tooltip>
)}
</div>
);
}
return (
<div className="flex flex-col items-start gap-1">
<Tooltip content={t(`cleanup.notRemovable.${sel.reason}_hint`)} focusable>
<Badge tone="neutral">{t(`cleanup.notRemovable.${sel.reason}`)}</Badge>
</Tooltip>
{sel.reason === "managed" && (
<Link href="/rules" className="text-xs text-brand-600 hover:underline">
{t("cleanup.notRemovable.managedLink", { rule: candidate.managed_by?.name ?? "" })}
</Link>
)}
</div>
);
}
const POLL_MS = 5000;
export function RuleCleanup() {
const { t, errorMessage, locale } = useLanguage();
const toast = useToast();
const readOnly = useReadOnly();
const notAdmin = useNotAdmin();
const router = useRouter();
const params = useSearchParams();
const [hosts, setHosts] = useState<Host[] | null>(null);
const [result, setResult] = useState<RuleCleanupResult | null>(null);
const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [tick, setTick] = useState(0);
const [refreshNext, setRefreshNext] = useState(false);
const [history, setHistory] = useState<CleanupCandidate | null>(null);
const [governance, setGovernance] = useState<GovernanceSettings | null>(null);
const [operations, setOperations] = useState<CleanupOperation[]>([]);
const [opsLoaded, setOpsLoaded] = useState(false);
const [selection, setSelection] = useState<{ kind: CleanupActionKind | null; uuids: string[] }>({ kind: null, uuids: [] });
const [request, setRequest] = useState<CleanupRequest | null>(null);
const [restoring, setRestoring] = useState<CleanupItem | null>(null);
const [busy, setBusy] = useState<string | null>(null);
const [now, setNow] = useState(() => Date.now());
const bannerRef = useRef<HTMLElement>(null);
const focusBanner = useRef(false);
const lastActive = useRef<string | null>(null);
const enabledHosts = useMemo(() => (hosts ?? []).filter((h) => h.enabled), [hosts]);
const days = THRESHOLDS.includes(Number(params.get("days"))) ? Number(params.get("days")) : 30;
const hostId = params.get("host") && enabledHosts.some((h) => h.id === params.get("host")) ? (params.get("host") as string) : enabledHosts[0]?.id ?? "";
const host = enabledHosts.find((h) => h.id === hostId) ?? null;
const active = activeOperation(operations);
const past = useMemo(() => historyOperations(operations), [operations]);
const configured = governance?.cleanup_management_sources_configured ?? true;
const maxRules = governance?.cleanup_max_rules ?? 10;
useEffect(() => {
api.listHosts().then(setHosts).catch((e) => { setHosts([]); setError(errorMessage(e, "cleanup.loadFailed")); });
api.getGovernance().then(setGovernance).catch(() => {});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
useEffect(() => {
if (!hostId) { setResult(null); return; }
let live = true;
setLoading(true);
setError(null);
api.getRuleCleanup(hostId, days, refreshNext)
.then((r) => live && setResult(r))
.catch((e) => { if (live) { setResult(null); setError(errorMessage(e, "cleanup.loadFailed")); } })
.finally(() => { if (live) { setLoading(false); setRefreshNext(false); } });
return () => { live = false; };
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [hostId, days, tick]);
useEffect(() => { setSelection({ kind: null, uuids: [] }); }, [result]);
const reloadCandidates = useCallback(() => { setRefreshNext(true); setTick((n) => n + 1); }, []);
const loadOperations = useCallback(async () => {
if (!hostId) { setOperations([]); setOpsLoaded(true); return; }
try {
const list = await api.listCleanupOperations(hostId);
setNow(Date.now());
setOperations(list);
// Somebody else (or the automatic revert) resolved the operation that was open a moment ago.
const open = activeOperation(list);
const before = lastActive.current;
if (before && before !== open?.id) {
const ended = list.find((op) => op.id === before);
if (ended && !isOpen(ended)) {
toast.show(t(`cleanup.toast.ended.${ended.status}`), ended.status === "reverted" ? "info" : "success");
reloadCandidates();
}
}
lastActive.current = open?.id ?? null;
} catch (e) {
setError(errorMessage(e, "cleanup.loadFailed"));
} finally {
setOpsLoaded(true);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [hostId]);
useEffect(() => { setOpsLoaded(false); lastActive.current = null; void loadOperations(); }, [loadOperations]);
// While something is open: poll, and tick the countdown once a second.
const activeId = active?.id;
useEffect(() => {
if (!activeId) return;
const poll = setInterval(() => void loadOperations(), POLL_MS);
const clock = setInterval(() => setNow(Date.now()), 1000);
return () => { clearInterval(poll); clearInterval(clock); };
}, [activeId, loadOperations]);
useEffect(() => {
if (activeId && focusBanner.current) { focusBanner.current = false; bannerRef.current?.focus(); }
}, [activeId]);
function update(next: { host?: string; days?: number }) {
const q = new URLSearchParams();
q.set("tab", "cleanup");
q.set("host", next.host ?? hostId);
q.set("days", String(next.days ?? days));
router.replace(`/analysis?${q.toString()}`);
}
const selectedKind = selection.kind;
function toggle(candidate: CleanupCandidate, mode: CleanupActionKind, on: boolean) {
const id = candidate.uuid ?? "";
setSelection((current) => {
const uuids = on ? [...current.uuids.filter((u) => u !== id), id] : current.uuids.filter((u) => u !== id);
return { kind: uuids.length ? mode : null, uuids };
});
}
function openPreview() {
if (!result || !selectedKind) return;
const rules = selection.uuids.flatMap((uuid) => {
const candidate = result.candidates.find((c) => c.uuid === uuid);
if (!candidate) return [];
const sel = selectability(candidate, result.unused_days);
return [{ uuid, label: candidate.description || uuid, needsAck: sel.mode === "disable" && sel.needsAck }];
});
setRequest({ kind: selectedKind, rules });
}
function coverName(uuid: string): string | null {
for (const candidate of result?.candidates ?? []) {
const by = candidate.categories.find((c) => c.by?.uuid === uuid)?.by;
if (by) return by.description || by.uuid;
}
return null;
}
async function act(name: "confirm" | "revert" | "retry") {
if (!active) return;
setBusy(name);
try {
const fn = name === "confirm" ? api.confirmCleanup : name === "revert" ? api.revertCleanup : api.retryCleanup;
// the operation's own anchor host: in a cluster it may be the other node than the one shown
const op = await fn(active.host_id, active.id);
const specific = t(`cleanup.toast.${name}.${op.status}`);
toast.show(specific === `cleanup.toast.${name}.${op.status}` ? t(`cleanup.toast.${name}.done`) : specific,
op.status === "revert_failed" || op.status === "failed" ? "danger" : "success");
lastActive.current = null;
await loadOperations();
reloadCandidates();
} catch (e) {
toast.show(errorMessage(e, `cleanup.${name}Failed`), "danger");
await loadOperations();
} finally {
setBusy(null);
}
}
function followUp(kind: "enable" | "delete", item: CleanupItem, uuid: string) {
setRequest({ kind, rules: [{ uuid, label: item.description || uuid, needsAck: false }] });
}
const total = result ? KINDS.reduce((sum, kind) => sum + result.counts[kind], 0) : 0;
const locked = !configured;
const limitReached = selection.uuids.length >= maxRules;
const previewBlockedReason = locked ? t("cleanup.select.locked") : active ? t("cleanup.select.busy") : selection.uuids.length === 0 ? t("cleanup.select.none") : null;
const checkboxLabel = (c: CleanupCandidate) => t("cleanup.select.row", { rule: c.description || c.uuid || "" });
return (
<div>
<Card className="mb-6 p-4 sm:p-5">
<form onSubmit={(e) => e.preventDefault()} aria-label={t("cleanup.form.title")} className="flex flex-wrap items-end gap-4">
<div className="w-full sm:w-64">
<FieldGroup label={t("cleanup.form.host")}>
<Select value={hostId} onChange={(e) => update({ host: e.target.value })} disabled={enabledHosts.length === 0}>
{enabledHosts.map((h) => <option key={h.id} value={h.id}>{h.name}</option>)}
</Select>
</FieldGroup>
</div>
<div className="w-full sm:w-48">
<FieldGroup label={t("cleanup.form.threshold")} hint={t("cleanup.form.thresholdHint")}>
<Select value={String(days)} onChange={(e) => update({ days: Number(e.target.value) })}>
{THRESHOLDS.map((n) => <option key={n} value={n}>{t("cleanup.form.days", { n })}</option>)}
</Select>
</FieldGroup>
</div>
<Button type="button" variant="outline" onClick={() => { reloadCandidates(); void loadOperations(); }} disabled={loading || !hostId} title={t("cleanup.form.refreshHint")}>
<RefreshCw size={14} aria-hidden="true" />
{t("common.refresh")}
</Button>
</form>
</Card>
{locked && (
<Card className="mb-6 p-4 sm:p-5">
<div className="flex items-start gap-3">
<Info size={18} className="mt-0.5 shrink-0 text-slate-400" aria-hidden="true" />
<div className="min-w-0 flex-1 space-y-3">
<div>
<p className="text-sm font-medium text-slate-800">{t("cleanup.mgmt.title")}</p>
<p className="mt-0.5 text-sm text-slate-600">
{t("cleanup.mgmt.text")}
<InfoTip text={t("cleanup.mgmt.hint")} size={13} className="ml-1 align-middle" />
</p>
</div>
{notAdmin ? <p className="text-sm text-slate-500">{t("cleanup.mgmt.askAdmin")}</p> : <CleanupSettings compact onSaved={setGovernance} />}
</div>
</div>
</Card>
)}
{active && host && (
<OperationBanner
ref={bannerRef}
operation={active}
now={now}
busy={busy}
readOnly={readOnly}
notAdmin={notAdmin}
operatorsMayDeploy={governance?.operators_may_deploy ?? true}
onConfirm={() => void act("confirm")}
onRevert={() => void act("revert")}
onRetry={() => void act("retry")}
/>
)}
{error && <p role="alert" className="mb-4 text-sm text-red-600">{error}</p>}
{hosts !== null && enabledHosts.length === 0 && <EmptyState icon={ListChecks} title={t("cleanup.noHosts")} />}
{loading && !result && <Skeleton className="h-40 w-full" />}
{result && (
<div aria-busy={loading}>
<div className="mb-3 flex flex-wrap items-center gap-2">
{KINDS.map((kind) => (
<Tooltip key={kind} content={t(`cleanup.category.${kind}_hint`)} focusable>
<Badge tone={result.counts[kind] > 0 ? KIND_TONE[kind] : "neutral"}>{t(`cleanup.category.${kind}`)}: {result.counts[kind]}</Badge>
</Tooltip>
))}
<span className="text-xs text-slate-500">{t("cleanup.rulesChecked", { count: result.rules_total })}</span>
<InfoTip
text={result.usage.watching_since
? t("cleanup.watching", { when: new Date(result.usage.watching_since).toLocaleDateString(locale), collecting: result.usage.collecting })
: t("cleanup.notWatching")}
size={14}
/>
{!notAdmin && !locked && (
<Link href="/settings#cleanup" className="ml-auto inline-flex items-center gap-1 text-xs text-brand-600 hover:underline">
<Settings size={13} aria-hidden="true" />{t("cleanup.settings.link")}
</Link>
)}
</div>
{result.notes.length > 0 && (
<div className="mb-3 flex flex-wrap gap-1.5">
{result.notes.map((code) => {
const label = t(`cleanup.notes.${code}`);
return (
<Tooltip key={code} content={label === `cleanup.notes.${code}` ? code : t(`cleanup.notes.${code}_hint`)} focusable>
<Badge tone="warning">{label === `cleanup.notes.${code}` ? code : label}</Badge>
</Tooltip>
);
})}
</div>
)}
{total === 0 ? (
<EmptyState icon={ListChecks} title={t("cleanup.empty.title")} description={t("cleanup.empty.description", { days })} />
) : (
<>
{!readOnly && (
<div className="mb-3 flex flex-wrap items-center gap-3">
<p role="status" aria-live="polite" className="text-sm text-slate-600">{t("cleanup.select.count", { n: selection.uuids.length, max: maxRules })}</p>
<Tooltip content={previewBlockedReason} focusable={previewBlockedReason !== null}>
<Button onClick={openPreview} disabled={previewBlockedReason !== null}>
<ListChecks size={15} aria-hidden="true" />
{t(selectedKind === "delete" ? "cleanup.select.previewDelete" : "cleanup.select.previewDisable")}
</Button>
</Tooltip>
{selection.uuids.length > 0 && <Button variant="ghost" onClick={() => setSelection({ kind: null, uuids: [] })}>{t("cleanup.select.clear")}</Button>}
</div>
)}
<Card className="overflow-x-auto p-0">
<table className="w-full text-left text-sm">
<caption className="sr-only">{t("cleanup.tableCaption", { host: result.host_name })}</caption>
<thead className="border-b border-slate-200 bg-slate-50 text-xs uppercase tracking-wide text-slate-500">
<tr>
{!readOnly && <th scope="col" className="w-10 px-4 py-3"><span className="sr-only">{t("cleanup.select.column")}</span></th>}
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.rule")}</th>
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.finding")}</th>
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.lastHit")}</th>
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.action")}</th>
</tr>
</thead>
<tbody>
{result.candidates.map((candidate) => {
const sel = selectability(candidate, result.unused_days);
const id = candidate.uuid ?? "";
const checked = selection.uuids.includes(id);
const reason = sel.mode === null ? t(`cleanup.notRemovable.${sel.reason}_hint`)
: locked ? t("cleanup.select.locked")
: active ? t("cleanup.select.busy")
: selectedKind && selectedKind !== sel.mode ? t("cleanup.select.otherKind")
: !checked && limitReached ? t("cleanup.select.limit", { max: maxRules })
: null;
return (
<tr key={candidate.uuid ?? candidate.label ?? candidate.description} className={`border-b border-slate-100 align-top last:border-0 ${checked ? "bg-brand-50/40" : ""}`}>
{!readOnly && (
<td className="px-4 py-3">
<Tooltip content={reason} focusable={reason !== null}>
<input
type="checkbox"
className="h-4 w-4 rounded border-slate-300"
checked={checked}
disabled={reason !== null && !checked}
aria-label={checkboxLabel(candidate)}
onChange={(e) => sel.mode && toggle(candidate, sel.mode, e.target.checked)}
/>
</Tooltip>
</td>
)}
<td className="max-w-[18rem] px-4 py-3">
<div className="flex items-start gap-1.5">
<span className="min-w-0 break-words font-medium text-slate-900">{candidate.description || candidate.uuid}</span>
<button
type="button"
onClick={() => setHistory(candidate)}
className="shrink-0 rounded p-1 text-slate-400 hover:bg-slate-100 hover:text-slate-700 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
aria-label={t("cleanup.history", { rule: candidate.description || candidate.uuid || "" })}
title={t("cleanup.historyTitle")}
>
<History size={15} aria-hidden="true" />
</button>
</div>
<p className="mt-0.5 break-words text-xs text-slate-500">
{candidate.action ? t(`analysis.verdict.${candidate.action}`) : "?"} · {candidate.interfaces.join(", ") || t("cleanup.allInterfaces")} · {candidate.source} → {candidate.destination}
{candidate.destination_port ? `:${candidate.destination_port}` : ""}
{!candidate.enabled && <span className="ml-1">({t("cleanup.disabledRule")})</span>}
</p>
</td>
<td className="min-w-[16rem] px-4 py-3">
<div className="space-y-1.5">
{candidate.categories.map((category) => <CategoryLine key={category.kind} category={category} days={result.unused_days} />)}
</div>
</td>
<td className="whitespace-nowrap px-4 py-3 text-slate-600">
{candidate.last_hit_at
? new Date(candidate.last_hit_at).toLocaleString(locale)
: candidate.watched_since
? <Tooltip content={t("cleanup.noHitSince", { when: new Date(candidate.watched_since).toLocaleDateString(locale) })} focusable><span>{t("cleanup.noHit")}</span></Tooltip>
: <span className="text-slate-300">–</span>}
</td>
<td className="min-w-[9rem] px-4 py-3"><ActionCell candidate={candidate} unusedDays={result.unused_days} /></td>
</tr>
);
})}
</tbody>
</table>
</Card>
</>
)}
</div>
)}
{host && opsLoaded && (
<Collapsible title={t("cleanup.hist.title")} count={past.length} className="mt-8">
<OperationHistory
operations={past}
hostId={hostId}
now={now}
minDays={governance?.cleanup_delete_min_days ?? 7}
readOnly={readOnly}
notAdmin={notAdmin}
busy={busy !== null || active !== null}
onRestore={setRestoring}
onFollowUp={followUp}
/>
</Collapsible>
)}
{request && host && (
<CleanupPreviewModal
host={{ id: host.id, name: host.name }}
request={request}
unusedDays={result?.unused_days ?? days}
coverName={coverName}
onClose={() => setRequest(null)}
onDeployed={(operation) => {
setRequest(null);
setSelection({ kind: null, uuids: [] });
if (isOpen(operation)) { focusBanner.current = true; lastActive.current = operation.id; }
setOperations((current) => [operation, ...current.filter((op) => op.id !== operation.id)]);
setNow(Date.now());
void loadOperations();
reloadCandidates();
}}
/>
)}
{restoring && host && (
<CleanupRestoreModal
host={{ id: host.id, name: host.name }}
item={restoring}
onClose={() => setRestoring(null)}
onRestored={() => { void loadOperations(); reloadCandidates(); }}
/>
)}
{history && result && (
<RuleHistoryModal
title={history.description || history.uuid || ""}
targets={[{ hostId: result.host_id, hostName: result.host_name, uuid: history.uuid ?? "" }]}
onClose={() => setHistory(null)}
/>
)}
</div>
);
}
+133
View File
@@ -0,0 +1,133 @@
"use client";
import { useLanguage } from "@/components/LanguageProvider";
import { Badge, BadgeTone } from "@/components/ui/Badge";
import { EmptyState } from "@/components/ui/EmptyState";
import { Select } from "@/components/ui/Field";
import { Loading } from "@/components/ui/Loading";
import { Modal } from "@/components/ui/Modal";
import { Tooltip } from "@/components/ui/Tooltip";
import { api, RuleHistory, RuleHistoryChange, RuleHistoryEvent } from "@/lib/api";
import { History } from "lucide-react";
import { useEffect, useState } from "react";
/** One firewall on which the rule exists, with the rule's id there (ids differ between firewalls). */
export interface HistoryTarget { hostId: string; hostName: string; uuid: string; }
const KIND_TONE: Record<string, BadgeTone> = {
added: "success", changed: "info", moved: "neutral", removed: "danger", enabled: "success", disabled: "warning",
};
function shorten(value: unknown): string {
const text = value === null || value === undefined || value === "" ? "–" : typeof value === "string" ? value : JSON.stringify(value);
return text.length > 60 ? `${text.slice(0, 57)}…` : text;
}
function useOptionalText() {
const { t } = useLanguage();
// Unknown codes (a newer backend, an audit action we have no wording for) show raw instead of breaking the list.
return (key: string, fallback: string) => {
const text = t(key);
return text === key ? fallback : text;
};
}
function ChangeList({ changes }: { changes: RuleHistoryChange[] }) {
const optional = useOptionalText();
if (changes.length === 0) return null;
return (
<ul className="mt-1 space-y-0.5 text-xs text-slate-600">
{changes.map((change) => (
<li key={change.field} className="break-words">
<span className="font-medium text-slate-700">{optional(`ruleHistory.field.${change.field}`, change.field)}</span>
{": "}
<span className="font-mono text-slate-500 line-through decoration-slate-300">{shorten(change.before)}</span>
{" → "}
<span className="font-mono text-slate-800">{shorten(change.after)}</span>
</li>
))}
</ul>
);
}
function EventRow({ event }: { event: RuleHistoryEvent }) {
const { t, locale } = useLanguage();
const optional = useOptionalText();
const label = event.source === "observed"
? optional(`ruleHistory.kind.${event.kind}`, event.kind)
: event.source === "revision"
? t("ruleHistory.kind.revision")
: optional(`ruleHistory.audit.${event.kind}`, event.kind);
const tone: BadgeTone = event.source === "observed" ? KIND_TONE[event.kind] ?? "neutral" : "neutral";
const showChanges = event.source === "revision" || event.kind === "changed" || event.kind === "moved";
return (
<li className="border-b border-slate-100 py-3 last:border-0">
<div className="flex flex-wrap items-center gap-x-2 gap-y-1">
<Badge tone={tone}>{label}</Badge>
{event.source === "audit" && event.status && <span className="text-xs text-slate-500">{event.status}</span>}
<Tooltip content={t(`ruleHistory.origin.${event.origin}_hint`)} focusable>
<Badge tone={event.origin === "opnexus" ? "accent" : "warning"}>{t(`ruleHistory.origin.${event.origin}`)}</Badge>
</Tooltip>
<time dateTime={event.at} className="text-xs text-slate-500">{new Date(event.at).toLocaleString(locale)}</time>
{event.actor && <span className="text-xs text-slate-500">{t("ruleHistory.by", { actor: event.actor })}</span>}
</div>
{event.note && <p className="mt-1 text-sm text-slate-600">{event.note}</p>}
{event.message && <p className="mt-1 break-words text-xs text-slate-500">{event.message}</p>}
{showChanges && <ChangeList changes={event.changes} />}
</li>
);
}
export function RuleHistoryModal({ title, targets, onClose }: { title: string; targets: HistoryTarget[]; onClose: () => void }) {
const { t, errorMessage, locale } = useLanguage();
const [index, setIndex] = useState(0);
const [history, setHistory] = useState<RuleHistory | null>(null);
const [error, setError] = useState<string | null>(null);
const target = targets[Math.min(index, targets.length - 1)];
useEffect(() => {
if (!target) return;
let active = true;
setHistory(null);
setError(null);
api.getRuleHistory(target.hostId, target.uuid)
.then((h) => active && setHistory(h))
.catch((e) => active && setError(errorMessage(e, "ruleHistory.loadFailed")));
return () => { active = false; };
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [target?.hostId, target?.uuid]);
return (
<Modal title={t("ruleHistory.title", { rule: title })} onClose={onClose} wide>
{targets.length > 1 && (
<div className="mb-3">
<label htmlFor="rule-history-host" className="mb-1 block text-sm font-medium text-slate-700">{t("ruleHistory.host")}</label>
<Select id="rule-history-host" value={String(index)} onChange={(e) => setIndex(Number(e.target.value))}>
{targets.map((item, i) => <option key={item.hostId} value={i}>{item.hostName}</option>)}
</Select>
</div>
)}
{error && <p role="alert" className="text-sm text-red-600">{error}</p>}
{!error && history === null && <Loading>{t("common.loading")}</Loading>}
{history && history.events.length === 0 && (
<EmptyState
icon={History}
title={t("ruleHistory.empty.title")}
description={history.watching_since
? t("ruleHistory.empty.since", { when: new Date(history.watching_since).toLocaleString(locale) })
: t("ruleHistory.empty.notYet")}
/>
)}
{history && history.events.length > 0 && (
<>
{history.watching_since && (
<p className="mb-1 text-xs text-slate-500">{t("ruleHistory.since", { when: new Date(history.watching_since).toLocaleString(locale) })}</p>
)}
<ul aria-label={t("ruleHistory.list")}>
{history.events.map((event, i) => <EventRow key={`${event.source}-${event.at}-${i}`} event={event} />)}
</ul>
</>
)}
</Modal>
);
}
+4 -2
View File
@@ -6,9 +6,11 @@ interface ModalProps {
title: string;
onClose: () => void;
children: ReactNode;
/** Wider dialog for lists and tables. */
wide?: boolean;
}
export function Modal({ title, onClose, children }: ModalProps) {
export function Modal({ title, onClose, children, wide = false }: ModalProps) {
const titleId = useId();
const { t } = useLanguage();
const dialogRef = useRef<HTMLDivElement>(null);
@@ -53,7 +55,7 @@ export function Modal({ title, onClose, children }: ModalProps) {
role="dialog"
aria-modal="true"
aria-labelledby={titleId}
className="flex max-h-[90vh] outline-none w-full max-w-lg flex-col rounded-2xl border border-slate-200 bg-white p-5 shadow-xl sm:p-6"
className={`flex max-h-[90vh] outline-none w-full ${wide ? "max-w-2xl" : "max-w-lg"} flex-col rounded-2xl border border-slate-200 bg-white p-5 shadow-xl sm:p-6`}
>
<div className="mb-4 flex shrink-0 items-center justify-between">
<h2 id={titleId} className="text-lg font-semibold text-slate-900">
+92 -2
View File
@@ -1574,6 +1574,21 @@ export const api = {
return request<AnalysisResult[]>(`/rule-analysis/match?${params.toString()}`);
},
getRuleHits: (id: string) => request<RuleHit[]>(`/hosts/${id}/rule-hits`),
getRuleCleanup: (id: string, unusedDays: number, refresh = false) =>
request<RuleCleanupResult>(`/hosts/${id}/rule-analysis/cleanup?unused_days=${unusedDays}${refresh ? "&refresh=true" : ""}`),
listCleanupOperations: (id: string) => request<CleanupOperation[]>(`/hosts/${id}/rule-cleanup`),
previewCleanup: (id: string, body: CleanupPreviewRequest) =>
request<CleanupPreview>(`/hosts/${id}/rule-cleanup/preview`, { method: "POST", body: JSON.stringify(body) }),
cancelCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/cancel`, { method: "POST" }),
deployCleanup: (id: string, op: string, confirmName: string) =>
request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/deploy`, { method: "POST", body: JSON.stringify({ confirm_name: confirmName }) }),
confirmCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/confirm`, { method: "POST" }),
revertCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/revert`, { method: "POST" }),
retryCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/retry`, { method: "POST" }),
restoreCleanupItem: (id: string, item: string, confirmName: string) =>
request<CleanupOperation>(`/hosts/${id}/rule-cleanup/items/${item}/restore`, { method: "POST", body: JSON.stringify({ confirm_name: confirmName }) }),
getRuleHistory: (id: string, ruleUuid: string) =>
request<RuleHistory>(`/hosts/${id}/live-rules/${encodeURIComponent(ruleUuid)}/history`),
checkHostCompliance: (id: string) => request<ComplianceHost>(`/hosts/${id}/compliance/check`, { method: "POST" }),
checkFleetCompliance: () => request<ComplianceHost[]>("/compliance/check", { method: "POST" }),
@@ -1581,8 +1596,9 @@ export const api = {
getOidcSettings: () => request<OidcSettings>("/settings/oidc"),
updateOidcSettings: (data: OidcSettingsUpdate) => request<OidcSettings>("/settings/oidc", { method: "PUT", body: JSON.stringify(data) }),
testOidc: () => request<{ success: boolean; authorization_endpoint: string }>("/settings/oidc/test", { method: "POST" }),
getGovernance: () => request<{ operators_may_deploy: boolean }>("/settings/governance"),
updateGovernance: (operators_may_deploy: boolean) => request<{ operators_may_deploy: boolean }>("/settings/governance", { method: "PUT", body: JSON.stringify({ operators_may_deploy }) }),
getGovernance: () => request<GovernanceSettings>("/settings/governance"),
updateGovernance: (operators_may_deploy: boolean) => request<GovernanceSettings>("/settings/governance", { method: "PUT", body: JSON.stringify({ operators_may_deploy }) }),
updateCleanupSettings: (data: CleanupSettingsUpdate) => request<GovernanceSettings>("/settings/governance", { method: "PUT", body: JSON.stringify(data) }),
getOverlay: () => request<{ settings: OverlaySettings; peers: OverlayPeer[] }>("/overlay"),
updateOverlay: (data: Partial<Pick<OverlaySettings, "enabled" | "endpoint" | "subnet" | "listen_port">>) =>
@@ -2135,3 +2151,77 @@ export interface AnalysisQuery {
src: string; dst: string; proto?: string; dport?: string; sport?: string; icmptype?: string;
iface?: string; egress?: string; host_ids?: string; group_id?: string; refresh?: boolean;
}
export type CleanupKind = "unused" | "shadowed" | "duplicate" | "disabled_long";
export type CleanupBlock = "automatic" | "legacy" | "managed";
export interface CleanupCategory {
kind: CleanupKind; confidence: AnalysisConfidence; days?: number; since?: string;
by?: { uuid: string | null; description: string; section: string; automatic: boolean };
}
export interface CleanupCandidate extends AnalysisRule {
enabled: boolean; categories: CleanupCategory[]; hit_state: "used" | "unused" | "collecting" | "disabled" | null;
last_hit_at: string | null; watched_since: string | null; removable: boolean; not_removable_reason: CleanupBlock | null;
managed_by: { kind: "rule" | "policy_hierarchy"; id: string; name: string } | null;
}
export interface RuleCleanupResult {
host_id: string; host_name: string; cluster_name: string | null; unused_days: number; generated_at: string;
rules_total: number; candidates: CleanupCandidate[]; counts: Record<CleanupKind, number>; notes: string[];
usage: { tracked: number; collecting: number; watching_since: string | null };
history: { watching_since: string | null };
}
export interface GovernanceSettings {
operators_may_deploy: boolean; cleanup_max_rules: number; cleanup_delete_min_days: number;
// The list itself only comes for admins; everybody else just learns whether it is set.
cleanup_management_sources: string[]; cleanup_management_sources_configured: boolean;
// Extra management ports (besides API port, 443, 80, 22); the list only for admins, too.
cleanup_management_ports: number[]; cleanup_management_ports_configured: boolean;
}
export interface CleanupSettingsUpdate {
cleanup_max_rules?: number; cleanup_delete_min_days?: number; cleanup_management_sources?: string[];
cleanup_management_ports?: number[];
}
export type CleanupActionKind = "disable" | "enable" | "delete";
export type CleanupOperationKind = CleanupActionKind | "restore";
export type CleanupOperationStatus =
| "preview" | "running" | "awaiting_confirm" | "confirmed" | "partial" | "apply_failed" | "reverting" | "reverted"
| "revert_failed" | "done" | "failed" | "cancelled" | "expired";
export type CleanupItemState =
| "queued" | "disabling" | "disabled" | "confirmed" | "reverting" | "reverted" | "deleting" | "deleted" | "restoring"
| "restored" | "failed" | "unknown" | "revert_failed" | "skipped";
export interface CleanupItem {
id: string; host_id: string; host_name: string | null; rule_uuid: string; seq: number; carp_role: string | null;
state: CleanupItemState; description: string | null; categories: CleanupCategory[] | null; confidence: AnalysisConfidence | null;
cover_uuid: string | null; restored_uuid: string | null; changed_at: string | null; message: string | null;
has_snapshot: boolean; restorable: boolean;
}
export interface CleanupPlanItem {
uuid: string; given: string; seq: number; description: string; action: string | null;
categories: { kind: CleanupKind; confidence: AnalysisConfidence; notes?: string[]; days?: number }[];
level: AnalysisConfidence | null; needs_ack: boolean; notes: string[]; cover_uuid: string | null;
protection: { result: "false" | "true" | "unknown"; queries: number; reasons: string[] } | null;
external: string | null;
basis?: { type: "confirmed" | "disabled_long"; confirmed_at?: string; since?: string };
}
export interface CleanupPlanNode { host_id: string; host_name: string; role: string | null; items: CleanupPlanItem[]; }
// sources/ports only for admins (the backend leaves them out for everybody else, sources_configured instead).
export interface CleanupPlan { kind: CleanupActionKind; uuids: string[]; unused_days: number; sources?: string[]; ports?: number[]; sources_configured?: boolean; acknowledged: string[]; nodes: CleanupPlanNode[]; }
export interface CleanupOperation {
id: string; host_id: string; host_name: string | null; cluster_name: string | null; kind: CleanupOperationKind;
status: CleanupOperationStatus; actor: string; created_at: string | null; expires_at: string | null;
revert_deadline: string | null; confirmed_at: string | null; finished_at: string | null; message: string | null;
plan: CleanupPlan | null; items: CleanupItem[]; can_confirm: boolean; can_revert: boolean; can_retry: boolean; warnings: string[];
}
export interface CleanupPreview {
preview_id: string; expires_at: string | null; kind: CleanupActionKind; plan: CleanupPlan; max_rules: number;
window_seconds: number; operation: CleanupOperation;
}
export interface CleanupPreviewRequest {
items: { uuid: string; kind: CleanupActionKind; ack: boolean }[]; unused_days: number; ack_text_version: string;
}
export interface RuleHistoryChange { field: string; before: unknown; after: unknown; }
export interface RuleHistoryEvent {
source: "observed" | "audit" | "revision"; at: string; kind: string; origin: "opnexus" | "external";
actor: string | null; changes: RuleHistoryChange[]; status?: string; message?: string | null; note?: string;
audit_id: string | null; managed_rule_id?: string;
}
export interface RuleHistory { host_id: string; rule_uuid: string; watching_since: string | null; events: RuleHistoryEvent[]; }
+151
View File
@@ -0,0 +1,151 @@
import type { AnalysisConfidence, CleanupCandidate, CleanupItem, CleanupOperation, CleanupOperationStatus } from "@/lib/api";
/** Version of the fixed acknowledgement text (cleanup.ack.text); the backend stores it with the audit entry. */
export const ACK_TEXT_VERSION = "1";
/** Same threshold as the backend: below a 30-day window "unused" is at most "probable" (monthly and quarterly jobs). */
export const MIN_SURE_UNUSED_DAYS = 30;
/** An operation in one of these states blocks everything else on the host/cluster until it is resolved. */
export const OPEN_STATUSES: CleanupOperationStatus[] = ["running", "awaiting_confirm", "partial", "apply_failed", "reverting", "revert_failed"];
/** States with an armed automatic revert (countdown). */
export const COUNTDOWN_STATUSES: CleanupOperationStatus[] = ["awaiting_confirm", "partial", "apply_failed"];
/** Hidden from the history: never executed, nothing happened. */
const NOISE_STATUSES: CleanupOperationStatus[] = ["preview", "cancelled", "expired"];
export type BlockReason = "automatic" | "legacy" | "managed" | "confidence" | "nothing" | "deleteWait" | "unreadable";
export type Selectability =
| { mode: "disable"; level: AnalysisConfidence; needsAck: boolean }
| { mode: "delete"; level: AnalysisConfidence; needsAck: false }
| { mode: null; reason: BlockReason };
const RANK: Record<AnalysisConfidence, number> = { unknown: 0, probable: 1, sure: 2 };
/**
* What the user may do with a candidate row. A client-side mirror of the backend rules (selectable_level) that only
* decides how the row looks: the backend recomputes everything at preview and may still refuse.
* - "unused" is at most "probable" for a block/reject rule and for a window shorter than 30 days,
* - "unknown" is never selectable, "probable" needs the rule's own acknowledgement,
* - an already disabled rule can only be deleted ("sure" disabled_long); everything else goes through "disable".
*/
export function selectability(candidate: CleanupCandidate, unusedDays: number): Selectability {
if (!candidate.removable) return { mode: null, reason: candidate.not_removable_reason ?? "automatic" };
if (!candidate.uuid) return { mode: null, reason: "unreadable" };
if (!candidate.enabled) {
const long = candidate.categories.find((c) => c.kind === "disabled_long");
if (long?.confidence === "sure") return { mode: "delete", level: "sure", needsAck: false };
return { mode: null, reason: long ? "deleteWait" : "nothing" };
}
let best: AnalysisConfidence | null = null;
for (const category of candidate.categories) {
if (category.kind === "disabled_long") continue;
let level = category.confidence;
if (category.kind === "unused" && (candidate.action === "block" || candidate.action === "reject" || unusedDays < MIN_SURE_UNUSED_DAYS)) {
if (RANK[level] > RANK.probable) level = "probable";
}
if (best === null || RANK[level] > RANK[best]) best = level;
}
if (best === null) return { mode: null, reason: "nothing" };
if (best === "unknown") return { mode: null, reason: "confidence" };
return { mode: "disable", level: best, needsAck: best === "probable" };
}
/** An ErrorCode name ("BEREINIGUNG_...", optionally followed by ": detail") is translated; the firewall's own text stays raw. */
export function cleanupMessage(t: (key: string) => string, message: string | null | undefined): string {
if (!message) return "";
const match = /^([A-Z][A-Z0-9_]{3,})(?::\s*([\s\S]*))?$/.exec(message);
if (match) {
const key = `errors.${match[1]}`;
const text = t(key);
if (text !== key) return match[2] ? `${text} ${match[2]}` : text;
}
return message;
}
/** The operation that blocks the host right now, if any (the newest one). */
export function isOpen(op: CleanupOperation): boolean {
// A delete/restore that ended with an unclear item is still unresolved (and blocks the host) until it is read again.
return OPEN_STATUSES.includes(op.status) || (op.can_retry && !NOISE_STATUSES.includes(op.status));
}
export function activeOperation(operations: CleanupOperation[]): CleanupOperation | null {
return operations.find(isOpen) ?? null;
}
export function historyOperations(operations: CleanupOperation[]): CleanupOperation[] {
return operations.filter((op) => !NOISE_STATUSES.includes(op.status) && !isOpen(op));
}
/** Rule identities (the UUID it has now and the one it had) for which a newer operation exists, per node. */
export function latestItemIds(operations: CleanupOperation[], hostId: string): Set<string> {
const seen = new Set<string>();
const latest = new Set<string>();
for (const op of operations) {
if (NOISE_STATUSES.includes(op.status)) continue;
for (const item of op.items) {
if (item.host_id !== hostId) continue;
const keys = [item.rule_uuid, item.restored_uuid].filter((k): k is string => !!k).map((k) => k.toLowerCase());
if (keys.some((k) => seen.has(k))) continue;
latest.add(item.id);
}
for (const item of op.items) {
if (item.host_id !== hostId) continue;
for (const key of [item.rule_uuid, item.restored_uuid]) if (key) seen.add(key.toLowerCase());
}
}
return latest;
}
/** The UUID under which a rule can be switched back on or deleted now (a restored rule has a new one). */
export function currentUuid(item: CleanupItem, operationKind: string): string {
return operationKind === "restore" && item.restored_uuid ? item.restored_uuid : item.rule_uuid;
}
export function secondsUntil(deadline: string | null | undefined, now: number): number {
if (!deadline) return 0;
return Math.max(0, Math.ceil((new Date(deadline).getTime() - now) / 1000));
}
export function formatClock(seconds: number): string {
const m = Math.floor(seconds / 60);
const s = seconds % 60;
return `${m}:${String(s).padStart(2, "0")}`;
}
/** Client-side check of the management sources, as strict as the backend (IP or CIDR, no names, not "everything"). */
/** Extra management ports: TCP ports 1-65535, separated by spaces/commas, at most `max`; sorted, no duplicates. */
export function parsePorts(text: string, max = 20): { ports: number[]; error: "invalid" | "tooMany" | null; bad?: string } {
const ports: number[] = [];
for (const raw of text.split(/[\s,;]+/).map((s) => s.trim()).filter(Boolean)) {
if (!/^\d{1,5}$/.test(raw) || Number(raw) < 1 || Number(raw) > 65535) return { ports, error: "invalid", bad: raw };
if (!ports.includes(Number(raw))) ports.push(Number(raw));
}
ports.sort((a, b) => a - b);
if (ports.length > max) return { ports, error: "tooMany" };
return { ports, error: null };
}
export function parseSources(text: string, max = 20): { entries: string[]; error: "empty" | "invalid" | "everything" | "tooMany" | null; bad?: string } {
const entries: string[] = [];
for (const raw of text.split(/[\s,;]+/).map((s) => s.trim()).filter(Boolean)) {
const [address, prefix, ...rest] = raw.split("/");
if (rest.length > 0) return { entries, error: "invalid", bad: raw };
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(address);
let max_prefix = 0;
if (v4) {
if (v4.slice(1).some((o) => Number(o) > 255 || (o.length > 1 && o.startsWith("0")))) return { entries, error: "invalid", bad: raw };
max_prefix = 32;
} else if (address.includes(":") && /^[0-9a-fA-F:.]+$/.test(address)) {
try { new URL(`http://[${address}]/`); } catch { return { entries, error: "invalid", bad: raw }; }
max_prefix = 128;
} else {
return { entries, error: "invalid", bad: raw };
}
if (prefix !== undefined) {
if (!/^\d{1,3}$/.test(prefix) || Number(prefix) > max_prefix) return { entries, error: "invalid", bad: raw };
if (Number(prefix) === 0) return { entries, error: "everything", bad: raw };
}
if (!entries.includes(raw)) entries.push(raw);
}
if (entries.length === 0) return { entries, error: "empty" };
if (entries.length > max) return { entries, error: "tooMany" };
return { entries, error: null };
}
+471 -2
View File
@@ -778,6 +778,7 @@
"OPEN_DEPLOYMENT_DRAFT_UNCHANGED": "Offene Bestätigung oder fehlgeschlagenen Rollback zuerst klären; Entwurf bleibt unverändert",
"OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY": "Offene Host-Wartung zuerst prüfen und abschließen, bevor Regeln ausgerollt werden.",
"OPEN_RESTORE_BLOCKS_DEPLOY": "Eine Wiederherstellung ist für diesen Host offen. Bitte zuerst bestätigen oder abbrechen.",
"OPEN_CLEANUP_BLOCKS_DEPLOY": "Für diesen Host läuft oder wartet eine Regel-Bereinigung. Bitte zuerst bestätigen, zurücknehmen oder abschließen.",
"READ_ONLY_ACCESS": "Nur-Lese-Zugriff: Diese Aktion erfordert die Rolle „Admin“.",
"CROSS_ORIGIN_BLOCKED": "Anfrage von einer fremden Herkunft blockiert",
"CROSS_SITE_REQUEST_BLOCKED": "Anfrage aus einer anderen Website blockiert (Sec-Fetch-Site). Öffne OPNexus direkt über seine eigene Adresse.",
@@ -868,7 +869,30 @@
"API_RIGHTS_MISSING": "Dem API-Benutzer fehlt das Recht, den Gateway-Status zu lesen",
"GATEWAY_NAME_INVALID": "Ungültiger Gateway-Name",
"GATEWAY_UNKNOWN": "Dieses Gateway kennt OPNexus auf der Firewall nicht (Name prüfen, Groß-/Kleinschreibung)",
"GATEWAY_MUTE_LIMIT": "Es können höchstens {limit} Gateways stummgeschaltet werden"
"GATEWAY_MUTE_LIMIT": "Es können höchstens {limit} Gateways stummgeschaltet werden",
"BEREINIGUNG_NICHT_GEFUNDEN": "Bereinigung nicht gefunden.",
"BEREINIGUNG_AUSWAHL_UNGUELTIG": "Ungültige Auswahl ({field}).",
"BEREINIGUNG_VORSCHAU_VERALTET": "Der Stand der Firewall hat sich seit der Vorschau geändert ({uuids}). Es wurde nichts geschrieben; Vorschau neu öffnen.",
"BEREINIGUNG_REGEL_NICHT_AENDERBAR": "Regel {uuid} auf {host} kann nicht bereinigt werden (Grund: {reason}).",
"BEREINIGUNG_OFFENE_OPERATION": "Im Host/Cluster ist noch etwas offen ({reason}). Erst bestätigen, zurücknehmen oder abschließen.",
"BEREINIGUNG_ZU_VIELE_REGELN": "Höchstens {max} Regeln je Bereinigung (gewählt: {count}).",
"BEREINIGUNG_BESTAETIGUNG_FEHLT": "Für diese Regeln fehlt die ausdrückliche Bestätigung: {uuids}.",
"BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN": "Zuerst in den Einstellungen die Verwaltungsquellen eintragen (Adressen, von denen aus die Firewalls verwaltet werden). Ohne sie ist die Bereinigung gesperrt.",
"BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG": "Cluster nicht vollständig oder Rollen unklar ({reason}). Die Bereinigung braucht alle Knoten, genau einen MASTER und dieselbe Regel-UUID auf jedem Knoten.",
"BEREINIGUNG_FRIST_ABGELAUFEN": "Die Bestätigungsfrist ist abgelaufen oder die Bereinigung ist nicht mehr offen.",
"BEREINIGUNG_NICHT_BESTAETIGBAR": "Diese Bereinigung kann nicht bestätigt werden ({status}): mindestens eine Regel ist fehlgeschlagen oder unklar. Jetzt zurücknehmen oder auf die automatische Rücknahme warten.",
"BEREINIGUNG_LIVESTAND_ABWEICHEND": "Auf der Firewall sehen diese Regeln anders aus als erwartet: {uuids}. Nicht bestätigt; prüfen oder zurücknehmen.",
"BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR": "{host} ist nicht erreichbar oder nicht lesbar. Nicht bestätigen, wenn die Firewall gerade nicht antwortet: die automatische Rücknahme stellt den alten Stand her.",
"BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT": "Regel {uuid} auf {host} ist noch nicht lange genug deaktiviert (noch {days} Tag(e)).",
"BEREINIGUNG_SNAPSHOT_FEHLT": "Für diese Regel gibt es keine Sicherung des Regelinhalts; sie wird nicht gelöscht bzw. kann nicht wiederhergestellt werden.",
"BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR": "Die Regel wurde wiederhergestellt, aber nicht an ihrer alten Position: sie steht am Ende ihrer Gruppe. Position in OPNsense prüfen.",
"BEREINIGUNG_AKTION_NICHT_MOEGLICH": "Diese Aktion ist im aktuellen Zustand nicht möglich ({status}).",
"BEREINIGUNG_NICHT_AUSGEFUEHRT": "Nicht ausgeführt: die Bereinigung wurde vorher abgebrochen.",
"BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN": "Die Änderung wurde gespeichert, aber das Anwenden auf der Firewall ist fehlgeschlagen.",
"BEREINIGUNG_AUSGANG_UNKLAR": "Die Firewall hat nicht eindeutig geantwortet; der Stand wurde nachgelesen.",
"BEREINIGUNG_ABGEBROCHEN_VERWAIST": "Der Vorgang wurde unterbrochen (OPNexus neu gestartet?). Stand auf der Firewall mit „Erneut prüfen“ nachlesen.",
"BEREINIGUNG_EINSTELLUNG_UNGUELTIG": "Ungültige Einstellung der Bereinigung ({field}: {reason}).",
"BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET": "Der Bestätigungstext dieser Seite ist veraltet. Seite neu laden und die Regeln erneut bestätigen."
},
"workflow": {
"attention": "Fehler prüfen",
@@ -2024,7 +2048,8 @@
"collecting": "Daten werden gesammelt",
"disabled": "deaktiviert"
}
}
},
"cleanupLink": "Kandidaten in der Bereinigung ansehen"
},
"tokens": {
"title": "API-Tokens",
@@ -2159,6 +2184,7 @@
"audit": {
"kind": {
"rule": "Regeln",
"rule_cleanup": "Regel-Bereinigung",
"alias": "Aliase",
"port_forward": "Port Forward",
"one_to_one": "1:1 NAT",
@@ -2178,6 +2204,11 @@
},
"action": {
"deploy": "Push",
"disable": "Deaktiviert",
"enable": "Aktiviert",
"revert": "Zurückgenommen",
"restore": "Wiederhergestellt",
"cleanup_settings": "Einstellungen geändert",
"confirm": "Bestätigung",
"rollback": "Rollback",
"remove": "Ziel entfernen",
@@ -3581,6 +3612,10 @@
"FIRST_PACKET_hint": "Bewertet wird das erste Paket einer neuen Verbindung. Antworten und bestehende Verbindungen folgen dem Zustand, nicht den Regeln.",
"SRC_PORT_EPHEMERAL": "Dynamischer Quellport",
"SRC_PORT_EPHEMERAL_hint": "Ohne Quellport wird ein beliebiger Port aus dem dynamischen Bereich angenommen."
},
"tabs": {
"query": "Abfrage",
"cleanup": "Bereinigung"
}
},
"guiAccess": {
@@ -3627,5 +3662,439 @@
},
"more": "Weitere fehlende Punkte: Einstellungen › Firewall-Oberfläche über OPNexus."
}
},
"cleanup": {
"loadFailed": "Die Bereinigungsliste konnte nicht geladen werden.",
"noHosts": "Keine aktivierte Firewall vorhanden.",
"form": {
"title": "Bereinigung: Firewall und Schwelle",
"host": "Firewall",
"threshold": "Schwelle",
"thresholdHint": "So lange darf eine Regel ohne Treffer sein (bzw. deaktiviert unverändert bleiben), bevor sie als Kandidat erscheint. OPNexus kann nur zurückschauen, seit es die Regel beobachtet.",
"days": "{n} Tage",
"refreshHint": "Liest Regeln, Interfaces und Aliase neu von der Firewall, statt den Zwischenspeicher (60 s) zu nutzen."
},
"rulesChecked": "{count} Regeln geprüft",
"watching": "OPNexus zählt die Treffer dieser Firewall seit {when}. {collecting} Regel(n) werden noch beobachtet und können noch nicht als ungenutzt gelten.",
"notWatching": "OPNexus hat noch keine Regelzähler dieser Firewall gesammelt. Ungenutzte Regeln erscheinen erst nach der Beobachtungszeit.",
"notes": {
"ORDER_GUESSED": "Reihenfolge geschätzt",
"ORDER_GUESSED_hint": "Diese OPNsense-Version liefert keine Sortierung; die Reihenfolge wurde aus Gruppe und Folgenummer abgeleitet. Befunde zu verdeckten und doppelten Regeln gelten höchstens als vermutlich.",
"ORDER_MISMATCH": "Reihenfolge weicht ab",
"ORDER_MISMATCH_hint": "Die geladene pf-Reihenfolge passt nicht zur Sortierung der Regeln. Befunde zu verdeckten Regeln mit Vorsicht lesen.",
"PF_RULESET_UNAVAILABLE": "Geladene Regeln unbekannt",
"PF_RULESET_UNAVAILABLE_hint": "Der von pf geladene Regelsatz war nicht lesbar. Ob eine Regel überhaupt aktiv ist, steht damit nicht fest; keine Aussage ist \"sicher\".",
"GROUPS_UNAVAILABLE": "Interface-Gruppen unbekannt",
"GROUPS_UNAVAILABLE_hint": "Die Interface-Gruppen waren nicht lesbar; Regeln auf Gruppen können nicht verglichen werden.",
"ALIASES_UNAVAILABLE": "Aliase unbekannt",
"ALIASES_UNAVAILABLE_hint": "Die Aliase waren nicht lesbar; Regeln mit Aliasen können nicht verglichen werden.",
"NAT_UNAVAILABLE": "NAT unbekannt",
"NAT_UNAVAILABLE_hint": "Die Port-Weiterleitungen waren nicht lesbar (für die Bereinigung ohne Belang).",
"CLUSTER_PEER_USAGE_UNKNOWN": "Nutzung im Cluster unvollständig",
"CLUSTER_PEER_USAGE_UNKNOWN_hint": "Für mindestens eine Regel fehlen die Zählerdaten eines Cluster-Knotens. \"Ungenutzt\" ist dann höchstens \"wahrscheinlich\".",
"UNUSED_BLOCK_RULE": "Sperrregel",
"UNUSED_BLOCK_RULE_hint": "Eine Sperrregel ohne Treffer kann trotzdem schützen (seltene Angriffe). Darum gilt „ungenutzt“ hier höchstens als vermutlich.",
"UNUSED_WINDOW_SHORT": "kurzes Fenster",
"UNUSED_WINDOW_SHORT_hint": "Unter 30 Tagen Beobachtung fehlen Monats- und Quartalsjobs. „Ungenutzt“ gilt höchstens als vermutlich.",
"EXTERNAL_CHANGE": "extern geändert",
"EXTERNAL_CHANGE_hint": "Die Regel oder die verdeckende Regel wurde im Beobachtungszeitraum außerhalb von OPNexus geändert. Der Befund gilt höchstens als vermutlich.",
"ENABLE_MAY_BLOCK_MANAGEMENT": "kann Verwaltung sperren",
"ENABLE_MAY_BLOCK_MANAGEMENT_hint": "Eine Sperrregel, für die nicht bewiesen ist, dass sie den Verwaltungszugang nicht trifft. Einschalten nur mit ausdrücklicher Bestätigung."
},
"empty": {
"title": "Keine Kandidaten",
"description": "In dieser Firewall ist keine Regel verdeckt, doppelt, seit {days} Tagen ungenutzt oder so lange deaktiviert."
},
"tableCaption": "Bereinigungskandidaten von {host}",
"col": {
"rule": "Regel",
"finding": "Befund",
"lastHit": "Letzter Treffer",
"removable": "Entfernbar",
"action": "Aktion"
},
"history": "Verlauf der Regel {rule} anzeigen",
"historyTitle": "Verlauf",
"allInterfaces": "alle Interfaces",
"disabledRule": "deaktiviert",
"category": {
"unused": "Ungenutzt",
"unused_hint": "Seit der Schwelle hat OPNexus keinen einzigen Treffer dieser Regel beobachtet (pf-Zähler, alle 15 Minuten gelesen).",
"shadowed": "Verdeckt",
"shadowed_hint": "Eine frühere Quick-Regel trifft schon alles, was diese Regel trifft. Diese Regel entscheidet nie.",
"duplicate": "Doppelt",
"duplicate_hint": "Eine frühere Regel trifft genau dasselbe und hat dieselbe Aktion.",
"disabled_long": "Lange deaktiviert",
"disabled_long_hint": "Die Regel ist deaktiviert und OPNexus hat sie seit der Schwelle unverändert so gesehen."
},
"detail": {
"unused": "kein Treffer in {days} Tagen",
"disabled": "seit mindestens {days} Tagen deaktiviert",
"disabledSince": "deaktiviert seit {when}",
"by": "durch \"{rule}\"",
"byAutomatic": "durch die automatische Regel \"{rule}\""
},
"confidence": {
"label": "Sicherheit",
"sure": "sicher",
"probable": "vermutlich",
"unknown": "unbekannt",
"tip": {
"sure": "Belegt: die Daten sind vollständig und aktuell, nichts davon beruht auf einer Annahme.",
"probable": "Beruht auf einer Annahme (z. B. Inhalt von Aliasen, geschätzte Reihenfolge, Zeitplan der Regel oder fehlender pf-Regelsatz). Vor dem Entfernen prüfen.",
"unknown": "Es gibt einen Hinweis, aber keinen Beleg (z. B. die Regel ist in pf nicht geladen oder die Zähler sind veraltet). Nicht darauf verlassen."
}
},
"noHit": "keiner beobachtet",
"noHitSince": "Seit {when} beobachtet OPNexus diese Regel und hat keinen Treffer gesehen.",
"notRemovable": {
"automatic": "automatisch",
"automatic_hint": "Diese Regel erzeugt OPNsense selbst und sie lässt sich nicht einzeln entfernen.",
"legacy": "Altformat",
"legacy_hint": "Die Regel ist noch im alten Format. Zuerst in OPNsense in das neue Regelformat migrieren.",
"managed": "von OPNexus verwaltet",
"managed_hint": "OPNexus hat diese Regel ausgerollt. Sie wird über den normalen Ablauf der Regelseite entfernt, sonst legt OPNexus sie wieder an.",
"managedLink": "Regel \"{rule}\" öffnen",
"confidence": "unbekannt",
"confidence_hint": "Es gibt nur einen Hinweis, aber keinen Beleg. Solche Regeln werden nie zum Entfernen angeboten.",
"nothing": "kein Befund",
"nothing_hint": "Für diese Regel liegt kein Befund vor, der ein Deaktivieren begründet.",
"deleteWait": "noch nicht",
"deleteWait_hint": "Gelöscht wird nur, was lange genug deaktiviert und unverändert war und als sicher gilt. Noch nicht erreicht.",
"unreadable": "nicht lesbar",
"unreadable_hint": "Die Regel hat keine UUID im neuen Regelformat und lässt sich nicht gezielt ändern."
},
"action": {
"disable": "Deaktivieren",
"enable": "Aktivieren",
"delete": "Löschen"
},
"select": {
"column": "Auswählen",
"row": "Regel {rule} auswählen",
"count": "{n} von {max} gewählt",
"previewDisable": "Vorschau: Deaktivieren",
"previewDelete": "Vorschau: Löschen",
"clear": "Auswahl aufheben",
"none": "Wählen Sie mindestens eine Regel aus.",
"locked": "Gesperrt, bis Verwaltungsquellen eingetragen sind.",
"busy": "Es ist noch eine Bereinigung offen: erst bestätigen oder zurücksetzen.",
"otherKind": "Eine Bereinigung kann nicht Deaktivieren und Löschen mischen. Zuerst die andere Auswahl aufheben.",
"limit": "Höchstens {max} Regeln je Bereinigung (Einstellung).",
"needsAck": "Bestätigung nötig",
"needsAckHint": "Der Befund beruht auf einer Annahme. Vor der Vorschau bestätigen Sie für diese Regel ausdrücklich, dass sie nicht gebraucht wird."
},
"settings": {
"link": "Einstellungen",
"title": "Regel-Bereinigung",
"intro": "Grenzen und Schutz für das Deaktivieren, Löschen und Wiederherstellen von Regeln im Bereich Analyse, Bereinigung.",
"sources": "Verwaltungsquellen",
"sourcesHint": "Adressen oder Netze, von denen aus die Firewalls verwaltet werden (OPNexus selbst, Ihr Arbeitsplatz, ein Jump-Host). Eine Regel, die den Zugang von dort zur Firewall treffen könnte, wird nie deaktiviert. Ohne Eintrag ist die Bereinigung gesperrt. Nur IP-Adressen oder Netze (CIDR), keine Namen.",
"sourcesPlaceholder": "203.0.113.10\n198.51.100.0/24",
"sourcesInvalid": "„{value}“ ist keine IP-Adresse und kein Netz (z. B. 203.0.113.10 oder 198.51.100.0/24). Hostnamen sind nicht erlaubt.",
"sourcesEverything": "„{value}“ umfasst den gesamten Adressraum und würde jede Regel schützen.",
"sourcesTooMany": "Höchstens 20 Einträge.",
"maxRules": "Regeln je Bereinigung",
"maxRulesHint": "So viele Regeln darf eine einzelne Bereinigung höchstens ändern (1 bis 25).",
"maxRulesInvalid": "Eine ganze Zahl von 1 bis 25.",
"minDays": "Wartezeit bis zum Löschen (Tage)",
"minDaysHint": "So lange muss eine bestätigt deaktivierte Regel unverändert bleiben, bevor sie gelöscht werden darf (1 bis 365).",
"minDaysInvalid": "Eine ganze Zahl von 1 bis 365.",
"saved": "Einstellungen der Bereinigung gespeichert.",
"ports": "Weitere Verwaltungsports",
"portsHint": "TCP-Ports, auf denen die Firewalls zusätzlich verwaltet werden (z. B. SSH oder Weboberfläche auf einem anderen Port). API-Port, 443, 80 und 22 sind immer geschützt.",
"portsPlaceholder": "z. B. 2222, 8443",
"portsInvalid": "Kein gültiger Port: {value} (1–65535).",
"portsTooMany": "Höchstens 20 Ports."
},
"mgmt": {
"title": "Bereinigung gesperrt: Verwaltungsquellen fehlen",
"text": "Tragen Sie die Adressen ein, von denen aus Ihre Firewalls verwaltet werden. Danach schaltet die Bereinigung keine Regel ab, die diesen Zugang treffen könnte.",
"hint": "OPNexus prüft für jede gewählte Regel, ob sie ein Paket von einer Verwaltungsquelle an die Verwaltungsadresse und den Port der Firewall treffen kann. Trifft sie oder ist das nicht sicher auszuschließen, bleibt sie unangetastet. Nur IP-Adressen oder Netze (CIDR).",
"askAdmin": "Ein Administrator muss die Verwaltungsquellen in den Einstellungen eintragen."
},
"dur": {
"minutes": "{n} Min.",
"seconds": "{n} Sek."
},
"preview": {
"title": {
"disable": "Vorschau: Regeln deaktivieren",
"enable": "Vorschau: Regeln wieder aktivieren",
"delete": "Vorschau: Regeln löschen"
},
"intro": {
"disable": "Die Regeln auf {host} werden nur deaktiviert, nicht gelöscht, und die Änderung wird angewendet.",
"enable": "Die Regeln auf {host} werden wieder eingeschaltet und die Änderung wird angewendet.",
"delete": "Die bereits deaktivierten Regeln auf {host} werden gelöscht. Ihr Inhalt wird vorher gesichert."
},
"introHint": {
"disable": "Die Liste wurde gerade neu von der Firewall berechnet. Beim Ausrollen prüft OPNexus alles noch einmal und schreibt nichts, wenn sich etwas geändert hat.",
"enable": "Das Einschalten einer Regel kann Verbindungen verändern oder sperren. Darum gibt es danach wieder ein Bestätigungsfenster. Die Liste wurde gerade neu von der Firewall berechnet.",
"delete": "Eine deaktivierte Regel ist in der Firewall nicht geladen, das Löschen ändert daran nichts und braucht kein Bestätigungsfenster. Unter „Verlauf der Bereinigungen“ lässt sich die Regel wiederherstellen (deaktiviert, an alter Position, soweit möglich)."
},
"loading": "Berechne die Vorschau von der Firewall …",
"failed": "Die Vorschau konnte nicht berechnet werden.",
"compute": "Vorschau berechnen",
"again": "Vorschau neu berechnen",
"validFor": "Vorschau noch {time} gültig",
"expired": "Die Vorschau ist abgelaufen. Bitte neu berechnen.",
"clusterOrder": "Cluster: zuerst Backup, dann Master. Zurücksetzen in umgekehrter Reihenfolge.",
"basisConfirmed": "Seit {when} bestätigt deaktiviert und unverändert.",
"basisDisabled": "Von Hand deaktiviert und lange unverändert.",
"external": "extern geändert",
"externalHint": "Seit der Beobachtung wurde die Regel außerhalb von OPNexus geändert.",
"acknowledged": "ausdrücklich bestätigt",
"autoRevert": "Danach haben Sie {window} Zeit zu prüfen, ob alles funktioniert und die Firewall erreichbar ist. Bestätigen Sie in dieser Zeit nicht, setzt OPNexus die Änderung automatisch zurück.",
"confirmName": "Name der Firewall zur Bestätigung",
"confirmNameHint": "Geben Sie den Namen der Firewall genau so ein. Das verhindert, dass versehentlich die falsche Firewall geändert wird.",
"confirmNameAria": "Name der Firewall zur Bestätigung eingeben: {host}",
"deployDisabledName": "Der Name muss genau dem der Firewall entsprechen.",
"deployNotAdmin": "Nur Administratoren dürfen ausrollen. Eine Vorschau gehört dem, der sie erstellt hat: ein Administrator muss selbst eine neue Vorschau öffnen."
},
"ack": {
"title": "Diese Befunde beruhen auf einer Annahme. Bestätigen Sie jede Regel einzeln.",
"text": "Nicht belegt: Zeitplan, Aliasinhalt oder Beobachtungszeitraum sind geschätzt. Ich habe geprüft, dass die Regel „{rule}“ nicht gebraucht wird.",
"enableTitle": "Das Einschalten dieser Sperrregel kann den Verwaltungszugang treffen. Bestätigen Sie jede Regel einzeln.",
"enableText": "Ich habe geprüft, dass das Einschalten von „{rule}“ den Zugang von OPNexus und der Verwaltung zur Firewall nicht sperrt."
},
"protect": {
"false": "Verwaltungszugang unberührt",
"false_hint": "Keine Verwaltungsquelle ({sources}) kann mit dieser Regel auf die Verwaltung der Firewall zugreifen (geprüft: {queries} Kombinationen aus Quelle, Ziel, Port und Interface).",
"true": "trifft den Verwaltungszugang",
"true_hint": "Die Regel kann den Zugang von einer Verwaltungsquelle ({sources}) zur Firewall treffen. Das Bestätigungsfenster ist hier die Absicherung.",
"unknown": "Verwaltungszugang unklar",
"unknown_hint": "Ob die Regel den Zugang von einer Verwaltungsquelle ({sources}) betrifft, ließ sich nicht sicher ausschließen. Das Bestätigungsfenster ist hier die Absicherung.",
"sourcesHidden": "die eingetragenen Verwaltungsquellen"
},
"deploy": {
"disable": "Deaktivieren und anwenden",
"enable": "Aktivieren und anwenden",
"delete": "Endgültig löschen",
"failed": "Das Ausrollen ist fehlgeschlagen."
},
"toast": {
"deployedConfirm": "Angewendet. Prüfen Sie jetzt die Firewall und bestätigen Sie, sonst wird zurückgesetzt.",
"deployedDone": "Erledigt.",
"deployedCheck": "Nicht vollständig: Details im Kasten oben.",
"confirm": {
"confirmed": "Bestätigt. Die Änderung bleibt bestehen.",
"done": "Bestätigt."
},
"revert": {
"reverted": "Zurückgesetzt. Die Regeln sind wieder wie vorher.",
"revert_failed": "Das Zurücksetzen ist fehlgeschlagen. Stand auf der Firewall prüfen.",
"done": "Zurückgesetzt."
},
"retry": {
"reverted": "Zurückgesetzt. Die Regeln sind wieder wie vorher.",
"revert_failed": "Das Zurücksetzen ist wieder fehlgeschlagen.",
"done": "Stand erneut geprüft.",
"failed": "Stand erneut geprüft, nicht alles ist abgeschlossen."
},
"ended": {
"reverted": "Die Bereinigung wurde automatisch zurückgesetzt.",
"confirmed": "Die Bereinigung wurde bestätigt.",
"done": "Die Bereinigung ist abgeschlossen.",
"failed": "Die Bereinigung ist beendet, aber nicht alles ist gelungen."
}
},
"confirmFailed": "Das Bestätigen ist fehlgeschlagen.",
"revertFailed": "Das Zurücksetzen ist fehlgeschlagen.",
"retryFailed": "Die erneute Prüfung ist fehlgeschlagen.",
"op": {
"title": {
"disable": "Regeln deaktivieren",
"enable": "Regeln wieder aktivieren",
"delete": "Regeln löschen",
"restore": "Regel wiederherstellen"
},
"state": {
"running": "OPNexus schreibt die Änderung gerade auf die Firewall. Die automatische Rücknahme ist bereits scharf.",
"awaiting_confirm": "Die Änderung ist auf der Firewall angewendet. Prüfen Sie, ob alles funktioniert und die Firewall erreichbar ist, und bestätigen Sie dann.",
"partial": "Nicht alle Regeln konnten geändert werden. Bestätigen ist gesperrt: jetzt zurücksetzen oder auf die automatische Rücknahme warten.",
"apply_failed": "Die Regeln sind geändert, aber das Anwenden auf der Firewall ist fehlgeschlagen. Jetzt zurücksetzen oder auf die automatische Rücknahme warten.",
"reverting": "OPNexus setzt die Änderung gerade zurück.",
"revert_failed": "Das Zurücksetzen ist fehlgeschlagen. Der Stand auf der Firewall ist unklar: in OPNsense prüfen und erneut versuchen.",
"confirmed": "Bestätigt.",
"reverted": "Zurückgesetzt.",
"done": "Abgeschlossen.",
"failed": "Bei mindestens einer Regel ist der Ausgang unklar. Mit „Erneut prüfen“ liest OPNexus den Stand der Firewall nach.",
"preview": "Vorschau.",
"cancelled": "Abgebrochen.",
"expired": "Abgelaufen."
},
"countdown": "Noch {time} bis zur automatischen Rücknahme",
"countdownMinutes": "Noch etwa {n} Min. bis zur automatischen Rücknahme.",
"confirm": "Bestätigen",
"confirmNotYet": "Bestätigen ist erst möglich, wenn die Änderung angewendet ist.",
"confirmPartial": "Bestätigen ist gesperrt, weil mindestens eine Regel fehlgeschlagen oder unklar ist.",
"confirmApplyFailed": "Bestätigen ist gesperrt, weil das Anwenden auf der Firewall fehlgeschlagen ist.",
"revert": "Jetzt zurücksetzen",
"revertNeedsAdmin": "Nur Administratoren dürfen zurücksetzen (Einstellung: Operatoren dürfen ausrollen).",
"retryRevert": "Zurücksetzen wiederholen",
"retryCheck": "Erneut prüfen",
"retryNeedsAdmin": "Nur Administratoren dürfen wiederholen.",
"autoRevertNote": "Bestätigen heißt: Sie erreichen die Firewall noch. Bestätigen Sie nicht, setzt OPNexus nach Ablauf der Zeit automatisch zurück. Bestätigen ist nur in der Browser-Sitzung möglich, nicht per API-Token."
},
"status": {
"preview": "Vorschau",
"preview_hint": "Berechnet, noch nichts geschrieben.",
"running": "Läuft",
"running_hint": "OPNexus schreibt gerade auf die Firewall.",
"awaiting_confirm": "Wartet auf Bestätigung",
"awaiting_confirm_hint": "Angewendet. Ohne Bestätigung setzt OPNexus nach Fristablauf automatisch zurück.",
"confirmed": "Bestätigt",
"confirmed_hint": "Vom Nutzer bestätigt. Die Änderung bleibt.",
"partial": "Teilweise",
"partial_hint": "Nicht alle Regeln konnten geändert werden. Zurücksetzen oder die automatische Rücknahme abwarten.",
"apply_failed": "Anwenden fehlgeschlagen",
"apply_failed_hint": "Die Regeln sind geändert, aber die Firewall hat die Änderung nicht angewendet.",
"reverting": "Wird zurückgesetzt",
"reverting_hint": "OPNexus stellt den alten Stand her.",
"reverted": "Zurückgesetzt",
"reverted_hint": "Der alte Stand ist wiederhergestellt. Das Beobachtungsfenster der Regeln beginnt neu.",
"revert_failed": "Zurücksetzen fehlgeschlagen",
"revert_failed_hint": "Der Stand auf der Firewall ist unklar. Prüfen und erneut versuchen.",
"done": "Abgeschlossen",
"done_hint": "Vollständig ausgeführt.",
"failed": "Fehlgeschlagen",
"failed_hint": "Nicht alles wurde ausgeführt. Details bei den Regeln.",
"cancelled": "Abgebrochen",
"cancelled_hint": "Die Vorschau wurde nicht ausgerollt.",
"expired": "Abgelaufen",
"expired_hint": "Die Vorschau war nicht mehr gültig."
},
"state": {
"queued": "wartet",
"disabling": "wird deaktiviert",
"disabled": "deaktiviert",
"confirmed": "bestätigt",
"reverting": "wird zurückgesetzt",
"reverted": "zurückgesetzt",
"deleting": "wird gelöscht",
"deleted": "gelöscht",
"restoring": "wird wiederhergestellt",
"restored": "wiederhergestellt (deaktiviert)",
"failed": "fehlgeschlagen",
"unknown": "Ausgang unklar",
"revert_failed": "Zurücksetzen fehlgeschlagen",
"skipped": "übersprungen"
},
"stateEnable": {
"disabling": "wird eingeschaltet",
"disabled": "eingeschaltet"
},
"hist": {
"title": "Verlauf der Bereinigungen",
"empty": "Auf dieser Firewall wurde noch nichts bereinigt.",
"restore": "Wiederherstellen …",
"restoreHint": "Legt die gelöschte Regel aus der Sicherung neu an, deaktiviert und möglichst an der alten Position.",
"adminOnly": "Nur Administratoren dürfen wiederherstellen.",
"delete": "Löschen …",
"deleteHint": "Öffnet die Vorschau zum Löschen dieser deaktivierten Regel.",
"deleteWait": "Löschen ist erst ab dem {when} möglich (Wartezeit nach der Bestätigung, einstellbar).",
"enable": "Wieder aktivieren …",
"enableHint": "Öffnet die Vorschau zum Einschalten der Regel, mit Bestätigungsfenster.",
"restoredBadge": "wiederhergestellt"
},
"restore": {
"title": "Regel wiederherstellen",
"intro": "Die gelöschte Regel wird aus der Sicherung neu angelegt, deaktiviert und möglichst an ihrer alten Position.",
"introHint": "Die Regel bekommt eine neue UUID. Im Cluster wird sie auf allen Knoten wiederhergestellt. Einschalten ist ein eigener Schritt („Wieder aktivieren“) mit Bestätigungsfenster. Sind Aliase oder Kategorien der Regel inzwischen weg, lehnt die Firewall ab und die Meldung steht beim Ergebnis.",
"action": "Wiederherstellen",
"done": "Regel „{rule}“ wiederhergestellt (deaktiviert).",
"failed": "Das Wiederherstellen ist fehlgeschlagen.",
"next": "Zum Einschalten: unter „Verlauf der Bereinigungen“ „Wieder aktivieren“ wählen."
},
"refuse": {
"managed": "von OPNexus verwaltet",
"automatic": "automatische Regel",
"legacy": "Altformat",
"management": "könnte den Verwaltungszugang treffen",
"infrastructure": "Cluster-/Infrastrukturregel",
"confidence": "Befund nicht belegt",
"not_candidate": "kein Kandidat mehr"
},
"openReason": {
"deployment": "Regel-Ausrollen offen",
"cleanup": "andere Bereinigung offen",
"rule": "Regel wird schon bearbeitet"
},
"clusterReason": {
"peer_disabled": "ein Knoten ist deaktiviert",
"peer_missing": "ein Knoten fehlt",
"roles": "Rollen unklar",
"uuid_missing": "Regel fehlt auf einem Knoten"
},
"settingsReason": {
"not_an_address": "keine IP-Adresse oder Netz",
"everything": "gesamter Adressraum",
"too_many": "zu viele Einträge"
}
},
"ruleHistory": {
"title": "Verlauf: {rule}",
"host": "Firewall",
"loadFailed": "Der Verlauf konnte nicht geladen werden.",
"list": "Ereignisse",
"action": "Verlauf",
"open": "Verlauf der Regel {rule} anzeigen",
"by": "von {actor}",
"since": "OPNexus beobachtet diese Firewall seit {when}.",
"empty": {
"title": "Keine Änderungen beobachtet",
"since": "Seit {when} hat sich an dieser Regel nichts geändert.",
"notYet": "OPNexus hat diese Firewall noch nicht beobachtet; die erste Beobachtung läuft kurz nach dem Start."
},
"origin": {
"opnexus": "über OPNexus",
"opnexus_hint": "Die Änderung passt zu einem Ausrollen oder Zurücksetzen durch OPNexus (Audit-Eintrag in diesem Zeitraum).",
"external": "extern",
"external_hint": "Kein OPNexus-Vorgang im Zeitraum: geändert in der Oberfläche der Firewall, per Skript oder durch einen anderen Zugang."
},
"kind": {
"added": "angelegt",
"changed": "geändert",
"removed": "entfernt",
"enabled": "aktiviert",
"disabled": "deaktiviert",
"revision": "in OPNexus gespeichert",
"moved": "verschoben"
},
"audit": {
"deploy": "ausgerollt",
"confirm": "bestätigt",
"rollback": "zurückgesetzt",
"retry-rollback": "Zurücksetzen wiederholt",
"state_change": "Zustand geändert"
},
"field": {
"description": "Beschreibung",
"enabled": "aktiv",
"action": "Aktion",
"interface": "Interface",
"interfacenot": "Interface invertiert",
"direction": "Richtung",
"ipprotocol": "IP-Version",
"protocol": "Protokoll",
"source_net": "Quelle",
"source_not": "Quelle invertiert",
"source_port": "Quellport",
"destination_net": "Ziel",
"destination_not": "Ziel invertiert",
"destination_port": "Zielport",
"quick": "Quick",
"gateway": "Gateway",
"sequence": "Reihenfolge",
"sort_order": "Sortierung",
"log": "Protokollieren",
"sched": "Zeitplan",
"statetype": "Statusart",
"tag": "Markierung",
"tagged": "Markiert mit",
"source_ports": "Quellport",
"prio_group": "Gruppe/Priorität"
}
}
}
+471 -2
View File
@@ -778,6 +778,7 @@
"OPEN_DEPLOYMENT_DRAFT_UNCHANGED": "Resolve the open confirmation or failed rollback first; the draft remains unchanged",
"OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY": "Check and complete the open host maintenance before deploying rules.",
"OPEN_RESTORE_BLOCKS_DEPLOY": "A restore is open for this host. Please confirm or cancel it first.",
"OPEN_CLEANUP_BLOCKS_DEPLOY": "A rule cleanup is running or waiting for this host. Please confirm, revert or finish it first.",
"READ_ONLY_ACCESS": "Read-only access: this action requires the “Admin” role.",
"CROSS_ORIGIN_BLOCKED": "Request from a foreign origin blocked",
"CROSS_SITE_REQUEST_BLOCKED": "Request from another site blocked (Sec-Fetch-Site). Open OPNexus directly at its own address.",
@@ -868,7 +869,30 @@
"API_RIGHTS_MISSING": "The API user lacks the right to read the gateway status",
"GATEWAY_NAME_INVALID": "Invalid gateway name",
"GATEWAY_UNKNOWN": "OPNexus does not know this gateway on the firewall (check the name and its case)",
"GATEWAY_MUTE_LIMIT": "At most {limit} gateways can be muted"
"GATEWAY_MUTE_LIMIT": "At most {limit} gateways can be muted",
"BEREINIGUNG_NICHT_GEFUNDEN": "Cleanup not found.",
"BEREINIGUNG_AUSWAHL_UNGUELTIG": "Invalid selection ({field}).",
"BEREINIGUNG_VORSCHAU_VERALTET": "The firewall changed since the preview ({uuids}). Nothing was written; open the preview again.",
"BEREINIGUNG_REGEL_NICHT_AENDERBAR": "Rule {uuid} on {host} cannot be cleaned up (reason: {reason}).",
"BEREINIGUNG_OFFENE_OPERATION": "Something is still open on the host/cluster ({reason}). Confirm, revert or finish it first.",
"BEREINIGUNG_ZU_VIELE_REGELN": "At most {max} rules per cleanup (selected: {count}).",
"BEREINIGUNG_BESTAETIGUNG_FEHLT": "These rules need an explicit acknowledgement: {uuids}.",
"BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN": "First enter the management sources in the settings (addresses the firewalls are managed from). Without them the cleanup is locked.",
"BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG": "Cluster incomplete or roles unclear ({reason}). The cleanup needs every node, exactly one MASTER and the same rule UUID on every node.",
"BEREINIGUNG_FRIST_ABGELAUFEN": "The confirmation window has passed or the cleanup is no longer open.",
"BEREINIGUNG_NICHT_BESTAETIGBAR": "This cleanup cannot be confirmed ({status}): at least one rule failed or is unclear. Revert now or wait for the automatic revert.",
"BEREINIGUNG_LIVESTAND_ABWEICHEND": "These rules look different on the firewall than expected: {uuids}. Not confirmed; check or revert.",
"BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR": "{host} cannot be reached or read. Do not confirm while the firewall does not answer: the automatic revert restores the previous state.",
"BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT": "Rule {uuid} on {host} has not been switched off long enough yet ({days} more day(s)).",
"BEREINIGUNG_SNAPSHOT_FEHLT": "There is no backup of this rule's content; it is not deleted or cannot be restored.",
"BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR": "The rule was restored, but not at its old position: it is at the end of its group. Check its position in OPNsense.",
"BEREINIGUNG_AKTION_NICHT_MOEGLICH": "This action is not possible in the current state ({status}).",
"BEREINIGUNG_NICHT_AUSGEFUEHRT": "Not executed: the cleanup stopped earlier.",
"BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN": "The change was saved, but applying it on the firewall failed.",
"BEREINIGUNG_AUSGANG_UNKLAR": "The firewall did not answer clearly; the state was read back.",
"BEREINIGUNG_ABGEBROCHEN_VERWAIST": "The run was interrupted (OPNexus restarted?). Read the firewall's state with “Check again”.",
"BEREINIGUNG_EINSTELLUNG_UNGUELTIG": "Invalid cleanup setting ({field}: {reason}).",
"BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET": "The acknowledgement text of this page is outdated. Reload the page and acknowledge the rules again."
},
"workflow": {
"attention": "Check errors",
@@ -2024,7 +2048,8 @@
"collecting": "collecting data",
"disabled": "disabled"
}
}
},
"cleanupLink": "Show the candidates in cleanup"
},
"tokens": {
"title": "API tokens",
@@ -2159,6 +2184,7 @@
"audit": {
"kind": {
"rule": "Rules",
"rule_cleanup": "Rule cleanup",
"alias": "Aliases",
"port_forward": "Port forward",
"one_to_one": "1:1 NAT",
@@ -2178,6 +2204,11 @@
},
"action": {
"deploy": "Push",
"disable": "Switched off",
"enable": "Switched on",
"revert": "Reverted",
"restore": "Restored",
"cleanup_settings": "Settings changed",
"confirm": "Confirmation",
"rollback": "Rollback",
"remove": "Remove target",
@@ -3581,6 +3612,10 @@
"FIRST_PACKET_hint": "The first packet of a new connection is evaluated. Replies and established connections follow state, not rules.",
"SRC_PORT_EPHEMERAL": "Ephemeral source port",
"SRC_PORT_EPHEMERAL_hint": "Without a source port, any port from the ephemeral range is assumed."
},
"tabs": {
"query": "Query",
"cleanup": "Cleanup"
}
},
"guiAccess": {
@@ -3627,5 +3662,439 @@
},
"more": "More missing items: Settings › Firewall interface through OPNexus."
}
},
"cleanup": {
"loadFailed": "The cleanup list could not be loaded.",
"noHosts": "No enabled firewall.",
"form": {
"title": "Cleanup: firewall and threshold",
"host": "Firewall",
"threshold": "Threshold",
"thresholdHint": "How long a rule may go without a hit (or stay disabled and unchanged) before it shows up as a candidate. OPNexus can only look back as far as it has been watching the rule.",
"days": "{n} days",
"refreshHint": "Re-reads rules, interfaces and aliases from the firewall instead of using the cache (60 s)."
},
"rulesChecked": "{count} rules checked",
"watching": "OPNexus has been counting this firewall's hits since {when}. {collecting} rule(s) are still being watched and cannot count as unused yet.",
"notWatching": "OPNexus has not collected rule counters from this firewall yet. Unused rules only show up after the observation period.",
"notes": {
"ORDER_GUESSED": "Order guessed",
"ORDER_GUESSED_hint": "This OPNsense version delivers no sort order; the order was derived from group and sequence number. Findings about shadowed and duplicate rules are at most probable.",
"ORDER_MISMATCH": "Order differs",
"ORDER_MISMATCH_hint": "The loaded pf order does not match the rules' sort order. Read findings about shadowed rules with care.",
"PF_RULESET_UNAVAILABLE": "Loaded rules unknown",
"PF_RULESET_UNAVAILABLE_hint": "The rule set pf has loaded could not be read, so it is not certain whether a rule is active at all. Nothing is \"sure\".",
"GROUPS_UNAVAILABLE": "Interface groups unknown",
"GROUPS_UNAVAILABLE_hint": "The interface groups could not be read; rules on groups cannot be compared.",
"ALIASES_UNAVAILABLE": "Aliases unknown",
"ALIASES_UNAVAILABLE_hint": "The aliases could not be read; rules using aliases cannot be compared.",
"NAT_UNAVAILABLE": "NAT unknown",
"NAT_UNAVAILABLE_hint": "The port forwards could not be read (irrelevant for cleanup).",
"CLUSTER_PEER_USAGE_UNKNOWN": "Cluster usage incomplete",
"CLUSTER_PEER_USAGE_UNKNOWN_hint": "For at least one rule a cluster node has no counter data. \"Unused\" is then \"probable\" at best.",
"UNUSED_BLOCK_RULE": "block rule",
"UNUSED_BLOCK_RULE_hint": "A block rule without hits can still protect (rare attacks). So “unused” counts as probable at most here.",
"UNUSED_WINDOW_SHORT": "short window",
"UNUSED_WINDOW_SHORT_hint": "Under 30 days of observation, monthly and quarterly jobs are missing. “Unused” counts as probable at most.",
"EXTERNAL_CHANGE": "changed externally",
"EXTERNAL_CHANGE_hint": "The rule or the rule that shadows it was changed outside OPNexus during the observation period. The finding counts as probable at most.",
"ENABLE_MAY_BLOCK_MANAGEMENT": "may block management",
"ENABLE_MAY_BLOCK_MANAGEMENT_hint": "A block rule that is not proven to leave the management access alone. Switching it on needs an explicit acknowledgement."
},
"empty": {
"title": "No candidates",
"description": "No rule on this firewall is shadowed, duplicate, unused for {days} days or disabled for that long."
},
"tableCaption": "Cleanup candidates of {host}",
"col": {
"rule": "Rule",
"finding": "Finding",
"lastHit": "Last hit",
"removable": "Removable",
"action": "Action"
},
"history": "Show the history of rule {rule}",
"historyTitle": "History",
"allInterfaces": "all interfaces",
"disabledRule": "disabled",
"category": {
"unused": "Unused",
"unused_hint": "OPNexus has not observed a single hit of this rule within the threshold (pf counters, read every 15 minutes).",
"shadowed": "Shadowed",
"shadowed_hint": "An earlier rule with \"quick\" already matches everything this rule matches. This rule never decides.",
"duplicate": "Duplicate",
"duplicate_hint": "An earlier rule matches exactly the same traffic and has the same action.",
"disabled_long": "Long disabled",
"disabled_long_hint": "The rule is disabled and OPNexus has seen it unchanged like that for the whole threshold."
},
"detail": {
"unused": "no hit in {days} days",
"disabled": "disabled for at least {days} days",
"disabledSince": "disabled since {when}",
"by": "by \"{rule}\"",
"byAutomatic": "by the automatic rule \"{rule}\""
},
"confidence": {
"label": "Confidence",
"sure": "sure",
"probable": "probable",
"unknown": "unknown",
"tip": {
"sure": "Proven: the data is complete and current, none of it rests on an assumption.",
"probable": "Rests on an assumption (e.g. alias content, guessed order, the rule's schedule or a missing pf rule set). Check before removing.",
"unknown": "There is a hint but no proof (e.g. the rule is not loaded in pf or the counters are stale). Do not rely on it."
}
},
"noHit": "none observed",
"noHitSince": "OPNexus has been watching this rule since {when} and has seen no hit.",
"notRemovable": {
"automatic": "automatic",
"automatic_hint": "OPNsense generates this rule itself and it cannot be removed on its own.",
"legacy": "legacy format",
"legacy_hint": "The rule is still in the old format. Migrate it to the new rule format in OPNsense first.",
"managed": "managed by OPNexus",
"managed_hint": "OPNexus deployed this rule. It is removed through the normal flow of the rules page, otherwise OPNexus would create it again.",
"managedLink": "Open rule \"{rule}\"",
"confidence": "unknown",
"confidence_hint": "There is a hint but no proof. Such rules are never offered for removal.",
"nothing": "no finding",
"nothing_hint": "There is no finding for this rule that would justify disabling it.",
"deleteWait": "not yet",
"deleteWait_hint": "Only rules that have been disabled long enough, unchanged, and count as sure are deleted. Not reached yet.",
"unreadable": "unreadable",
"unreadable_hint": "The rule has no UUID in the new rule format and cannot be changed individually."
},
"action": {
"disable": "Disable",
"enable": "Enable",
"delete": "Delete"
},
"select": {
"column": "Select",
"row": "Select rule {rule}",
"count": "{n} of {max} selected",
"previewDisable": "Preview: disable",
"previewDelete": "Preview: delete",
"clear": "Clear selection",
"none": "Select at least one rule.",
"locked": "Locked until management sources are set.",
"busy": "Another cleanup is still open: confirm or revert it first.",
"otherKind": "One cleanup cannot mix disabling and deleting. Clear the other selection first.",
"limit": "At most {max} rules per cleanup (setting).",
"needsAck": "Acknowledgement needed",
"needsAckHint": "The finding rests on an assumption. Before the preview you confirm for this rule explicitly that it is not needed."
},
"settings": {
"link": "Settings",
"title": "Rule cleanup",
"intro": "Limits and protection for disabling, deleting and restoring rules under Analysis, Cleanup.",
"sources": "Management sources",
"sourcesHint": "Addresses or networks the firewalls are managed from (OPNexus itself, your workstation, a jump host). A rule that could match access from there to the firewall is never disabled. Without an entry the cleanup is locked. IP addresses or networks (CIDR) only, no names.",
"sourcesPlaceholder": "203.0.113.10\n198.51.100.0/24",
"sourcesInvalid": "“{value}” is not an IP address or network (e.g. 203.0.113.10 or 198.51.100.0/24). Host names are not allowed.",
"sourcesEverything": "“{value}” covers the whole address space and would protect every rule.",
"sourcesTooMany": "At most 20 entries.",
"maxRules": "Rules per cleanup",
"maxRulesHint": "The most rules a single cleanup may change (1 to 25).",
"maxRulesInvalid": "A whole number from 1 to 25.",
"minDays": "Waiting time before deleting (days)",
"minDaysHint": "How long a rule that was confirmed as disabled has to stay unchanged before it may be deleted (1 to 365).",
"minDaysInvalid": "A whole number from 1 to 365.",
"saved": "Cleanup settings saved.",
"ports": "Additional management ports",
"portsHint": "TCP ports the firewalls are additionally managed on (e.g. SSH or the web GUI on another port). The API port, 443, 80 and 22 are always protected.",
"portsPlaceholder": "e.g. 2222, 8443",
"portsInvalid": "Not a valid port: {value} (1–65535).",
"portsTooMany": "At most 20 ports."
},
"mgmt": {
"title": "Cleanup locked: management sources missing",
"text": "Enter the addresses your firewalls are managed from. After that the cleanup never switches off a rule that could match this access.",
"hint": "For every selected rule OPNexus checks whether it can match a packet from a management source to the firewall's management address and port. If it matches, or this cannot be ruled out, the rule is left alone. IP addresses or networks (CIDR) only.",
"askAdmin": "An administrator has to enter the management sources in the settings."
},
"dur": {
"minutes": "{n} min",
"seconds": "{n} s"
},
"preview": {
"title": {
"disable": "Preview: disable rules",
"enable": "Preview: enable rules again",
"delete": "Preview: delete rules"
},
"intro": {
"disable": "The rules on {host} are only disabled, not deleted, and the change is applied.",
"enable": "The rules on {host} are switched on again and the change is applied.",
"delete": "The already disabled rules on {host} are deleted. Their content is backed up first."
},
"introHint": {
"disable": "The list was just recomputed from the firewall. When rolling out, OPNexus checks everything again and writes nothing if something changed.",
"enable": "Switching a rule on can change or block connections. That is why there is a confirmation window again. The list was just recomputed from the firewall.",
"delete": "A disabled rule is not loaded in the firewall, so deleting it changes nothing there and needs no confirmation window. Under “Cleanup history” the rule can be restored (disabled, at its old position where possible)."
},
"loading": "Computing the preview from the firewall …",
"failed": "The preview could not be computed.",
"compute": "Compute preview",
"again": "Recompute preview",
"validFor": "Preview valid for {time}",
"expired": "The preview has expired. Please recompute it.",
"clusterOrder": "Cluster: backup first, then master. Reverting runs in reverse order.",
"basisConfirmed": "Disabled and confirmed since {when}, unchanged.",
"basisDisabled": "Disabled by hand and unchanged for a long time.",
"external": "changed externally",
"externalHint": "The rule was changed outside OPNexus since it has been observed.",
"acknowledged": "explicitly acknowledged",
"autoRevert": "Afterwards you have {window} to check that everything works and the firewall is reachable. If you do not confirm in that time, OPNexus reverts the change automatically.",
"confirmName": "Firewall name to confirm",
"confirmNameHint": "Type the firewall's name exactly. This prevents changing the wrong firewall by accident.",
"confirmNameAria": "Type the firewall name to confirm: {host}",
"deployDisabledName": "The name has to match the firewall's name exactly.",
"deployNotAdmin": "Only administrators may roll out. A preview belongs to whoever created it: an administrator has to open a new preview themselves."
},
"ack": {
"title": "These findings rest on an assumption. Acknowledge each rule on its own.",
"text": "Not proven: schedule, alias content or observation period are estimated. I have checked that the rule “{rule}” is not needed.",
"enableTitle": "Switching these block rules on may hit the management access. Acknowledge each rule.",
"enableText": "I checked that switching on “{rule}” does not block OPNexus or the admins from reaching the firewall."
},
"protect": {
"false": "Management access untouched",
"false_hint": "No management source ({sources}) can use this rule to reach the firewall's management (checked: {queries} combinations of source, destination, port and interface).",
"true": "matches management access",
"true_hint": "The rule can match access from a management source ({sources}) to the firewall. The confirmation window is the safeguard here.",
"unknown": "Management access unclear",
"unknown_hint": "It could not be ruled out that the rule affects access from a management source ({sources}). The confirmation window is the safeguard here.",
"sourcesHidden": "the configured management sources"
},
"deploy": {
"disable": "Disable and apply",
"enable": "Enable and apply",
"delete": "Delete permanently",
"failed": "The roll-out failed."
},
"toast": {
"deployedConfirm": "Applied. Check the firewall now and confirm, otherwise it is reverted.",
"deployedDone": "Done.",
"deployedCheck": "Not complete: details in the box above.",
"confirm": {
"confirmed": "Confirmed. The change stays.",
"done": "Confirmed."
},
"revert": {
"reverted": "Reverted. The rules are as they were.",
"revert_failed": "Reverting failed. Check the firewall's state.",
"done": "Reverted."
},
"retry": {
"reverted": "Reverted. The rules are as they were.",
"revert_failed": "Reverting failed again.",
"done": "State checked again.",
"failed": "State checked again, not everything is complete."
},
"ended": {
"reverted": "The cleanup was reverted automatically.",
"confirmed": "The cleanup was confirmed.",
"done": "The cleanup is complete.",
"failed": "The cleanup has ended, but not everything worked."
}
},
"confirmFailed": "Confirming failed.",
"revertFailed": "Reverting failed.",
"retryFailed": "Checking again failed.",
"op": {
"title": {
"disable": "Disable rules",
"enable": "Enable rules again",
"delete": "Delete rules",
"restore": "Restore rule"
},
"state": {
"running": "OPNexus is writing the change to the firewall. The automatic revert is already armed.",
"awaiting_confirm": "The change is applied on the firewall. Check that everything works and the firewall is reachable, then confirm.",
"partial": "Not all rules could be changed. Confirming is locked: revert now or wait for the automatic revert.",
"apply_failed": "The rules are changed, but applying on the firewall failed. Revert now or wait for the automatic revert.",
"reverting": "OPNexus is reverting the change.",
"revert_failed": "Reverting failed. The firewall's state is unclear: check it in OPNsense and try again.",
"confirmed": "Confirmed.",
"reverted": "Reverted.",
"done": "Complete.",
"failed": "The outcome is unclear for at least one rule. “Check again” makes OPNexus read the firewall's state.",
"preview": "Preview.",
"cancelled": "Cancelled.",
"expired": "Expired."
},
"countdown": "{time} left until the automatic revert",
"countdownMinutes": "About {n} min left until the automatic revert.",
"confirm": "Confirm",
"confirmNotYet": "Confirming is possible once the change is applied.",
"confirmPartial": "Confirming is locked because at least one rule failed or is unclear.",
"confirmApplyFailed": "Confirming is locked because applying on the firewall failed.",
"revert": "Revert now",
"revertNeedsAdmin": "Only administrators may revert (setting: operators may roll out).",
"retryRevert": "Retry reverting",
"retryCheck": "Check again",
"retryNeedsAdmin": "Only administrators may retry.",
"autoRevertNote": "Confirming means: you can still reach the firewall. If you do not confirm, OPNexus reverts automatically when the time is up. Confirming works from the browser session only, not with an API token."
},
"status": {
"preview": "Preview",
"preview_hint": "Computed, nothing written yet.",
"running": "Running",
"running_hint": "OPNexus is writing to the firewall.",
"awaiting_confirm": "Waiting for confirmation",
"awaiting_confirm_hint": "Applied. Without confirmation OPNexus reverts automatically when the time is up.",
"confirmed": "Confirmed",
"confirmed_hint": "Confirmed by a user. The change stays.",
"partial": "Partial",
"partial_hint": "Not all rules could be changed. Revert or wait for the automatic revert.",
"apply_failed": "Apply failed",
"apply_failed_hint": "The rules are changed, but the firewall did not apply the change.",
"reverting": "Reverting",
"reverting_hint": "OPNexus is restoring the old state.",
"reverted": "Reverted",
"reverted_hint": "The old state is restored. The rules' observation window starts again.",
"revert_failed": "Revert failed",
"revert_failed_hint": "The firewall's state is unclear. Check it and try again.",
"done": "Complete",
"done_hint": "Fully carried out.",
"failed": "Failed",
"failed_hint": "Not everything was carried out. Details at the rules.",
"cancelled": "Cancelled",
"cancelled_hint": "The preview was not rolled out.",
"expired": "Expired",
"expired_hint": "The preview was no longer valid."
},
"state": {
"queued": "waiting",
"disabling": "being disabled",
"disabled": "disabled",
"confirmed": "confirmed",
"reverting": "being reverted",
"reverted": "reverted",
"deleting": "being deleted",
"deleted": "deleted",
"restoring": "being restored",
"restored": "restored (disabled)",
"failed": "failed",
"unknown": "outcome unclear",
"revert_failed": "revert failed",
"skipped": "skipped"
},
"stateEnable": {
"disabling": "being enabled",
"disabled": "enabled"
},
"hist": {
"title": "Cleanup history",
"empty": "Nothing has been cleaned up on this firewall yet.",
"restore": "Restore …",
"restoreHint": "Recreates the deleted rule from the backup, disabled and at its old position where possible.",
"adminOnly": "Only administrators may restore.",
"delete": "Delete …",
"deleteHint": "Opens the preview for deleting this disabled rule.",
"deleteWait": "Deleting is possible from {when} (waiting time after the confirmation, configurable).",
"enable": "Enable again …",
"enableHint": "Opens the preview for switching the rule on, with a confirmation window.",
"restoredBadge": "restored"
},
"restore": {
"title": "Restore rule",
"intro": "The deleted rule is recreated from the backup, disabled and at its old position where possible.",
"introHint": "The rule gets a new UUID. In a cluster it is restored on every node. Switching it on is a separate step (“Enable again”) with a confirmation window. If aliases or categories of the rule are gone meanwhile, the firewall refuses and its message appears in the result.",
"action": "Restore",
"done": "Rule “{rule}” restored (disabled).",
"failed": "Restoring failed.",
"next": "To switch it on: choose “Enable again” under “Cleanup history”."
},
"refuse": {
"managed": "managed by OPNexus",
"automatic": "automatic rule",
"legacy": "legacy format",
"management": "could match management access",
"infrastructure": "cluster/infrastructure rule",
"confidence": "finding not proven",
"not_candidate": "no longer a candidate"
},
"openReason": {
"deployment": "rule roll-out open",
"cleanup": "another cleanup open",
"rule": "rule already being handled"
},
"clusterReason": {
"peer_disabled": "a node is disabled",
"peer_missing": "a node is missing",
"roles": "roles unclear",
"uuid_missing": "rule missing on a node"
},
"settingsReason": {
"not_an_address": "not an IP address or network",
"everything": "whole address space",
"too_many": "too many entries"
}
},
"ruleHistory": {
"title": "History: {rule}",
"host": "Firewall",
"loadFailed": "The history could not be loaded.",
"list": "Events",
"action": "History",
"open": "Show the history of rule {rule}",
"by": "by {actor}",
"since": "OPNexus has been watching this firewall since {when}.",
"empty": {
"title": "No changes observed",
"since": "This rule has not changed since {when}.",
"notYet": "OPNexus has not observed this firewall yet; the first observation runs shortly after startup."
},
"origin": {
"opnexus": "via OPNexus",
"opnexus_hint": "The change matches a deployment or rollback by OPNexus (an audit entry in that period).",
"external": "external",
"external_hint": "No OPNexus operation in that period: changed in the firewall's own interface, by a script or through another access."
},
"kind": {
"added": "created",
"changed": "changed",
"removed": "removed",
"enabled": "enabled",
"disabled": "disabled",
"revision": "saved in OPNexus",
"moved": "moved"
},
"audit": {
"deploy": "deployed",
"confirm": "confirmed",
"rollback": "rolled back",
"retry-rollback": "rollback retried",
"state_change": "state changed"
},
"field": {
"description": "Description",
"enabled": "enabled",
"action": "Action",
"interface": "Interface",
"interfacenot": "Interface inverted",
"direction": "Direction",
"ipprotocol": "IP version",
"protocol": "Protocol",
"source_net": "Source",
"source_not": "Source inverted",
"source_port": "Source port",
"destination_net": "Destination",
"destination_not": "Destination inverted",
"destination_port": "Destination port",
"quick": "Quick",
"gateway": "Gateway",
"sequence": "Sequence",
"sort_order": "Sort order",
"log": "Log",
"sched": "Schedule",
"statetype": "State type",
"tag": "Tag",
"tagged": "Tagged",
"source_ports": "Source port",
"prio_group": "Group/priority"
}
}
}
+4
View File
@@ -67,6 +67,10 @@ const CODE_LIST_PARAMS: Record<string, { param: string; prefix: string }> = {
GUI_GATEWAY_NOT_CONFIGURED: { param: "reasons", prefix: "guiAccess.reason" },
NETWORK_CHANGE_BLOCKED: { param: "reasons", prefix: "network.blocks" },
NETWORK_ACK_REQUIRED: { param: "reasons", prefix: "network.risks" },
BEREINIGUNG_REGEL_NICHT_AENDERBAR: { param: "reason", prefix: "cleanup.refuse" },
BEREINIGUNG_OFFENE_OPERATION: { param: "reason", prefix: "cleanup.openReason" },
BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG: { param: "reason", prefix: "cleanup.clusterReason" },
BEREINIGUNG_EINSTELLUNG_UNGUELTIG: { param: "reason", prefix: "cleanup.settingsReason" },
};
/** `reasons: "name_exists, proto_set"` -> readable sentences joined by " · "; an unknown code stays as it is.
+2 -2
View File
@@ -1,6 +1,6 @@
// Manually bumped whenever a notable change ships — mirrors the dated
// entries already kept in the project README/CHANGELOG.md, no build
// tooling involved. Versioning scheme: see CHANGELOG.md ("Versionierung").
export const APP_VERSION = "1.29.0";
export const APP_CODENAME = "Caracara";
export const APP_VERSION = "1.31.0";
export const APP_CODENAME = "Vulture";
export const APP_MAINTAINER = "Julian Boldt";
+48
View File
@@ -275,6 +275,47 @@ preflight_proxy_config() {
say "Proxy-Konfiguration geprüft."
}
# B8 (1.30): Die CLI, die ein Update ausführt, ist die ALTE (im Updater-Image eingebaut bzw. im Bundle). Korrekturen an
# Vorprüfungen und Ablauf aus dem Zielrelease griffen deshalb erst beim übernächsten Update. Jetzt holt sie sich die CLI
# aus dem Updater-Image des Ziels und startet sich damit neu, aber nur unter diesen Bedingungen (Review: die CLI läuft
# mit Zugriff auf den Docker-Socket, also wie root): cosign und cosign.pub sind da, das Image wurde in DIESEM Lauf
# gezogen, seine Signatur wurde nach DIGEST geprüft (nicht nach Tag) und genau dieser Digest wird ausgeführt, und die
# Ziel-CLI kennt selbst refresh_cli (so wird nie eine alte, bekannt fehlerhafte CLI geholt). Jeder andere Fehler heißt
# nur: mit der vorhandenen CLI weitermachen. OPNEXUS_NO_CLI_REFRESH=1 schaltet es ab. Der Neustart-Schutz ist eine
# Shell-Variable plus internes Argument, keine Umgebungsvariable, die ein Aufrufer vererben könnte.
CLI_REFRESHED=0
refresh_cli() {
local target="$1" repo ref digest tmp keep=0
[ "$CLI_REFRESHED" != 1 ] && [ "${OPNEXUS_NO_CLI_REFRESH:-}" != 1 ] || return 0
repo=$(env_get OPNEXUS_IMAGE_REPO)
[ -n "$repo" ] || return 0
if ! command -v cosign >/dev/null 2>&1 || [ ! -f cosign.pub ]; then
say "CLI-Aktualisierung übersprungen (ohne cosign und cosign.pub wird keine CLI aus einem Image ausgeführt)."; return 0
fi
ref="$repo/opnexus-updater:$target"
if updater_enabled; then keep=1; fi
if ! docker pull -q "$ref" >/dev/null 2>&1; then say "CLI-Aktualisierung übersprungen (Updater-Image des Ziels nicht ladbar)."; return 0; fi
digest=$(docker image inspect --format '{{index .RepoDigests 0}}' "$ref" 2>/dev/null || true)
if [[ "$digest" != *@sha256:* ]]; then
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
say "CLI-Aktualisierung übersprungen (Digest des Updater-Images unbekannt)."; return 0
fi
cosign verify --key cosign.pub --insecure-ignore-tlog=true "$digest" >/dev/null 2>&1 \
|| die "Signatur des Updater-Images $ref ist ungültig oder fehlt — nichts wurde verändert."
tmp=$(mktemp "${TMPDIR:-/tmp}/opnexus-cli.XXXXXX") || return 0
if ! docker run --rm --pull never --entrypoint cat "$digest" /usr/local/bin/opnexus > "$tmp" 2>/dev/null \
|| [ ! -s "$tmp" ] || ! bash -n "$tmp" 2>/dev/null || ! grep -q '^refresh_cli() {' "$tmp"; then
rm -f "$tmp"; [ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
say "CLI-Aktualisierung übersprungen (CLI des Zielreleases nicht lesbar oder zu alt) — nutze die vorhandene."; return 0
fi
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
if cmp -s "$tmp" "$(readlink -f "$0")"; then rm -f "$tmp"; return 0; fi
say "Die CLI des Zielreleases (Signatur geprüft) ist neuer — starte sie."
chmod 0700 "$tmp"
export OPNEXUS_DIR="$PWD"
exec bash "$tmp" __refreshed update "$target"
}
cmd_update() {
require_release_mode
local target="${1:-$(env_get OPNEXUS_VERSION)}"
@@ -294,6 +335,7 @@ cmd_update() {
if updater_enabled; then pull_services+=(updater); fi
OPNEXUS_VERSION="$target" c pull "${pull_services[@]}" || die "Pull fehlgeschlagen — nichts wurde verändert."
verify_signatures "$target" "${pull_services[@]}"
refresh_cli "$target"
preflight_proxy_config "$target"
started=$(date -u +%Y%m%dT%H%M%SZ)
@@ -428,6 +470,12 @@ cmd_password() {
c exec -T backend python -m app.reset_password ${user:+"$user"}
}
# Interner Neustart durch refresh_cli: Marker setzen und die eigene Temp-Datei aufräumen (bash hat sie offen).
if [ "${1:-}" = __refreshed ]; then
CLI_REFRESHED=1; shift
case "$0" in */opnexus-cli.*) rm -f "$0" ;; esac
fi
case "${1:-}" in
password) shift; cmd_password "$@" ;;
status) if [ "${2:-}" = "--porcelain" ]; then cmd_status_porcelain; else cmd_status; fi ;;
+146
View File
@@ -0,0 +1,146 @@
"""refresh_cli in `opnexus` (B8): the CLI takes the newer copy from the target's updater image and restarts itself, but
only with a cosign check of the image DIGEST that succeeded in this run; every other failure continues with the present
CLI. Runs the real function with fake `docker` and `cosign` in front of PATH.
Run: cd updater && python3 -m unittest test_cli_refresh"""
import os
import shutil
import stat
import subprocess
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).with_name("opnexus").resolve()
DIGEST = "registry.example/opnexus/opnexus-updater@sha256:" + "a" * 64
FAKE_DOCKER = """#!/bin/sh
echo "$@" >> "$FAKE_LOG"
case "$1" in
pull) [ "${FAKE_PULL_RC:-0}" = 0 ] || exit 1 ;;
image) [ -z "${FAKE_NO_DIGEST:-}" ] && echo "$FAKE_DIGEST" ;;
run) [ "${FAKE_RC:-0}" = 0 ] || exit "$FAKE_RC"; cat "$FAKE_CLI" 2>/dev/null ;;
esac
"""
FAKE_COSIGN = """#!/bin/sh
echo "cosign $*" >> "$FAKE_LOG"
[ "${FAKE_COSIGN_RC:-0}" = 0 ]
"""
NEW_CLI = '#!/usr/bin/env bash\\nrefresh_cli() {\\n :\\n}\\necho "NEW-CLI $*"\\n'.replace("\\n", "\n")
def _exe(path: Path, text: str):
path.write_text(text)
path.chmod(path.stat().st_mode | stat.S_IXUSR)
class RefreshCli(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
(self.tmp / "bin").mkdir()
_exe(self.tmp / "bin" / "docker", FAKE_DOCKER)
_exe(self.tmp / "bin" / "cosign", FAKE_COSIGN)
(self.tmp / ".env").write_text("OPNEXUS_IMAGE_REPO=registry.example/opnexus\n")
(self.tmp / "cosign.pub").write_text("key")
self.log = self.tmp / "calls.log"
self.cli = self.tmp / "new-cli"
def run_refresh(self, new_cli=NEW_CLI, *, env_extra=None, env_file=None, pre="", no_cosign=False, no_pub=False):
self.cli.write_text(new_cli)
if env_file is not None:
(self.tmp / ".env").write_text(env_file)
if no_pub:
(self.tmp / "cosign.pub").unlink()
if no_cosign:
(self.tmp / "bin" / "cosign").unlink()
path = f"{self.tmp / 'bin'}:/usr/bin:/bin" if no_cosign else f"{self.tmp / 'bin'}:{os.environ['PATH']}" # a real cosign must not leak in
env = {**os.environ, "PATH": path, "OPNEXUS_DIR": str(self.tmp),
"FAKE_LOG": str(self.log), "FAKE_CLI": str(self.cli), "FAKE_DIGEST": DIGEST, "TMPDIR": str(self.tmp)}
for name in ("OPNEXUS_NO_CLI_REFRESH", "CLI_REFRESHED"):
env.pop(name, None)
env.update(env_extra or {})
body = f'source <(sed "/^case \\"\\${{1:-}}\\" in/,\\$d" "{SCRIPT}"); {pre} refresh_cli 9.9.9; echo CONTINUED'
return subprocess.run(["bash", "-c", body, str(SCRIPT)], env=env, capture_output=True, text=True, timeout=30)
def calls(self):
return self.log.read_text().splitlines() if self.log.exists() else []
def test_verified_newer_cli_runs_by_digest_with_the_internal_marker_argument(self):
r = self.run_refresh()
self.assertIn("NEW-CLI __refreshed update 9.9.9", r.stdout)
self.assertNotIn("CONTINUED", r.stdout) # exec: the old script does not go on
calls = self.calls()
self.assertTrue(any(c.startswith("cosign verify") and c.endswith(DIGEST) for c in calls)) # signature of the DIGEST
run = next(c for c in calls if c.startswith("run "))
self.assertTrue(run.endswith(f"{DIGEST} /usr/local/bin/opnexus")) # that same digest is executed
self.assertLess([c.split()[0] for c in calls].index("cosign"), [c.split()[0] for c in calls].index("run"))
def test_identical_cli_continues_and_removes_the_image_it_pulled(self):
r = self.run_refresh(SCRIPT.read_text())
self.assertIn("CONTINUED", r.stdout)
self.assertTrue(any(c.startswith("rmi registry.example/opnexus/opnexus-updater:9.9.9") for c in self.calls()))
def test_image_stays_when_the_updater_profile_is_enabled(self):
self.run_refresh(SCRIPT.read_text(), env_file="OPNEXUS_IMAGE_REPO=registry.example/opnexus\nCOMPOSE_PROFILES=updater\n")
self.assertFalse(any(c.startswith("rmi") for c in self.calls()))
def test_nothing_runs_without_a_real_signature_check(self):
for label, kwargs in {"no cosign": {"no_cosign": True}, "no cosign.pub": {"no_pub": True}}.items():
with self.subTest(label):
self.log.unlink(missing_ok=True)
r = self.run_refresh(**kwargs)
self.assertIn("CONTINUED", r.stdout)
self.assertFalse(any(c.startswith(("pull", "run")) for c in self.calls()))
def test_invalid_signature_aborts_the_update_and_runs_nothing(self):
r = self.run_refresh(env_extra={"FAKE_COSIGN_RC": "1"})
self.assertNotEqual(r.returncode, 0)
self.assertNotIn("CONTINUED", r.stdout)
self.assertNotIn("NEW-CLI", r.stdout)
self.assertFalse(any(c.startswith("run ") for c in self.calls()))
def test_pull_failure_never_falls_back_to_a_local_image(self):
r = self.run_refresh(env_extra={"FAKE_PULL_RC": "1"})
self.assertIn("CONTINUED", r.stdout)
self.assertFalse(any(c.startswith(("run ", "cosign")) for c in self.calls()))
def test_unknown_digest_missing_marker_broken_or_empty_copies_are_skipped(self):
for label, kwargs in {
"no digest": {"env_extra": {"FAKE_NO_DIGEST": "1"}},
"old CLI without refresh_cli": {"new_cli": "#!/usr/bin/env bash\necho OLD-CLI\n"},
"syntax error": {"new_cli": "refresh_cli() {\nif then fi (\n"},
"empty": {"new_cli": ""},
"docker run fails": {"env_extra": {"FAKE_RC": "1"}},
}.items():
with self.subTest(label):
r = self.run_refresh(**kwargs)
self.assertIn("CONTINUED", r.stdout)
self.assertNotIn("NEW-CLI", r.stdout)
self.assertNotIn("OLD-CLI", r.stdout)
def test_opt_out_and_missing_repo_do_not_even_ask_docker(self):
for label, kwargs in {"opt-out": {"env_extra": {"OPNEXUS_NO_CLI_REFRESH": "1"}},
"no repo": {"env_file": "POSTGRES_PASSWORD=x\n"}}.items():
with self.subTest(label):
self.log.unlink(missing_ok=True)
r = self.run_refresh(**kwargs)
self.assertIn("CONTINUED", r.stdout)
self.assertEqual(self.calls(), [])
def test_restart_guard_is_a_shell_variable_an_inherited_environment_value_cannot_switch_it_off(self):
inherited = self.run_refresh(env_extra={"CLI_REFRESHED": "1"})
self.assertIn("NEW-CLI __refreshed", inherited.stdout) # inheriting the name changes nothing
self.log.unlink(missing_ok=True)
again = self.run_refresh(pre="CLI_REFRESHED=1;") # the refreshed copy itself does not refresh again
self.assertIn("CONTINUED", again.stdout)
self.assertEqual(self.calls(), [])
def test_refreshed_copy_removes_its_own_temp_file(self):
copy = self.tmp / "opnexus-cli.abc123"
shutil.copy(SCRIPT, copy)
env = {**os.environ, "OPNEXUS_DIR": str(self.tmp)}
subprocess.run(["bash", str(copy), "__refreshed", "bogus"], env=env, capture_output=True, text=True, timeout=30)
self.assertFalse(copy.exists())
if __name__ == "__main__":
unittest.main()