Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b020e38d0a | ||
|
|
abd15318a3 |
No files matched your search
@@ -31,6 +31,49 @@ den Einstellungen.
|
||||
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
|
||||
ansteht): Merlin …
|
||||
|
||||
## [1.31.0] "Vulture" — 2026-10-22
|
||||
|
||||
- **Regel-Bereinigung: Entfernen, Bestätigen, Löschen, Wiederherstellen:** Im Tab **Bereinigung** (Regelwerk › Regel-Analyse) lassen sich Kandidaten jetzt auswählen
|
||||
(Checkbox, höchstens so viele Regeln je Lauf wie in den Einstellungen; „vermutlich“ nur mit einer ausdrücklichen Einzelbestätigung je Regel, „unbekannt“ nie).
|
||||
**Vorschau** zeigt je Firewall (im Cluster zuerst der Backup) die Regeln mit Befund, Sicherheit und Schutzprüfung; **Ausrollen** deaktiviert die Regeln und wendet an.
|
||||
Danach läuft ein Countdown: **Bestätigen** heißt „ich erreiche die Firewall noch“, ohne Bestätigung setzt OPNexus **automatisch zurück**; bis dahin geht auch
|
||||
**Jetzt zurücksetzen**. Bestätigte Regeln lassen sich nach der Wartezeit (einstellbar, Standard 7 Tage) **löschen**; vorher sichert OPNexus den Regelinhalt, und
|
||||
unter **Verlauf der Bereinigungen** kommt die Regel mit **Wiederherstellen** deaktiviert und möglichst an alter Position zurück (sonst mit Hinweis am Ende der Gruppe);
|
||||
**Wieder aktivieren** hat wieder ein Bestätigungsfenster. Wichtig: **Ausrollen, Wiederholen und Wiederherstellen darf nur ein Administrator**, eine Vorschau gehört
|
||||
dem, der sie erstellt hat (ein Admin öffnet selbst eine neue); **Bestätigen geht nur in der Browser-Sitzung**, nie per API-Token. **Die Bereinigung bleibt gesperrt,
|
||||
bis unter Einstellungen die Verwaltungsquellen** (IP-Adressen oder Netze, von denen aus verwaltet wird) **eingetragen sind**: Regeln, die diesen Zugang treffen
|
||||
könnten, deaktiviert OPNexus nie. Dort stehen auch die Obergrenze je Lauf, die Wartezeit bis zum Löschen und **weitere Verwaltungsports** (zusätzlich zu
|
||||
API-Port, 443, 80, 22). Tragen Sie die Adresse ein, mit der OPNexus selbst bei den Firewalls ankommt (im Cluster für jeden Knoten). Quellen, Ports und
|
||||
Firewall-Adressen sehen nur Administratoren. Der Hinweis „Entfernen folgt“ entfällt.
|
||||
- **Update: neuere CLI aus dem Zielrelease:** `opnexus update` (auch per Klick im Webinterface) holt sich nach dem Pull die CLI aus dem Updater-Image des
|
||||
Zielreleases und startet sich damit neu, damit Korrekturen am Update-Ablauf sofort greifen statt erst beim übernächsten Update. Das geschieht nur, wenn
|
||||
`cosign` und `cosign.pub` vorhanden sind und die Signatur des Image-**Digests** in diesem Lauf geprüft wurde (ausgeführt wird genau dieser Digest);
|
||||
sonst und bei jedem Fehler läuft das Update mit der vorhandenen CLI weiter. Abschalten: `OPNEXUS_NO_CLI_REFRESH=1`. Wirksam wird es ab dem Update
|
||||
*von* dieser Version.
|
||||
|
||||
## [1.30.0] "Owl" — 2026-10-22
|
||||
|
||||
- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
|
||||
Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
|
||||
trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
|
||||
dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
|
||||
höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
|
||||
verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
|
||||
Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
|
||||
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
|
||||
stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
|
||||
Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
|
||||
verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
|
||||
Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
|
||||
automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
|
||||
erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
|
||||
Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
|
||||
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
|
||||
15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
|
||||
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
|
||||
Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
|
||||
mehr verdeckte Regeln als bisher, nie weniger belegte.
|
||||
|
||||
## [1.29.0] "Caracara" — 2026-10-22
|
||||
|
||||
- **Fehlende API-Rechte werden sichtbar:** Darf der API-Schlüssel einer Firewall den Gateway- oder Dienststatus nicht lesen (OPNsense antwortet
|
||||
|
||||
+3
-3
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
|
||||
|
||||

|
||||
|
||||
> **Stand:** `1.29.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
|
||||
> **Stand:** `1.31.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
|
||||
|
||||
## Funktionen
|
||||
|
||||
@@ -26,7 +26,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
|
||||
- **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
|
||||
- **Automatisierung** — persönliche API-Tokens (Lesen / Ausrollen / Admin), Prometheus-Metriken, ein nur lesender MCP-Endpunkt für KI-Assistenten und ein OpenAPI-Schema.
|
||||
- **Sicherheitslage** — ein Wert von 0 bis 100 je Firewall aus Härtungs- und Regel-Hygiene-Prüfungen (offene WAN-Regeln, schwaches SSH/SNMP, verdeckte Regeln, alte Backups …), nur lesend.
|
||||
- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend.
|
||||
- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend. Eine Bereinigungsliste nennt ungenutzte, verdeckte, doppelte und lange deaktivierte Regeln (jeweils mit Sicherheit); ein Administrator kann ausgewählte Regeln abschalten (Vorschau, dann Ausrollen mit automatischer Rücknahme ohne Bestätigung), später löschen und aus einer Sicherung des Regelinhalts wiederherstellen. Jede Regel hat einen Änderungsverlauf – auch für Änderungen außerhalb von OPNexus.
|
||||
- **Backups & Wiederherstellung** — geplante Konfigurationssicherungen mit Aufbewahrung, verschlüsselte Offsite-Kopie auf S3-kompatiblen Speicher, Änderungsansicht zwischen Sicherungen und eine vollständige verschlüsselte Sicherung von OPNexus selbst (`opnexus backup --full`).
|
||||
- **Single Sign-on** — Anmeldung über deinen Identitätsanbieter (OIDC); die Rolle kommt bei jeder Anmeldung aus dessen Gruppen.
|
||||
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Operator / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
|
||||
@@ -65,7 +65,7 @@ Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelie
|
||||
|
||||
```bash
|
||||
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
|
||||
OPNEXUS_VERSION=1.29.0
|
||||
OPNEXUS_VERSION=1.31.0
|
||||
```
|
||||
|
||||
```bash
|
||||
|
||||
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
|
||||
|
||||

|
||||
|
||||
> **Status:** `1.29.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
|
||||
> **Status:** `1.31.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
|
||||
|
||||
## What it does
|
||||
|
||||
@@ -26,7 +26,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
|
||||
- **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
|
||||
- **Automation** — personal API tokens (read / deploy / admin), Prometheus metrics, a read-only MCP endpoint for AI assistants and an OpenAPI schema.
|
||||
- **Security posture** — a 0–100 score per firewall from hardening and rule-hygiene checks (open WAN rules, weak SSH/SNMP, shadowed rules, stale backups …), read-only.
|
||||
- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only.
|
||||
- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only. A cleanup list names unused, shadowed, duplicate and long-disabled rules (each with its confidence); an administrator can switch selected rules off (preview, then roll-out with an automatic revert unless confirmed), delete them later and restore them from a backup of their content. Every rule has a change history, including changes made outside OPNexus.
|
||||
- **Backups & recovery** — scheduled config backups with retention, an encrypted offsite copy to S3-compatible storage, a redacted change view between backups, and a full encrypted backup/restore of OPNexus itself (`opnexus backup --full`).
|
||||
- **Single sign-on** — sign in with your identity provider (OIDC); roles come from its groups at every login.
|
||||
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / operator / read-only), keyboard operation, dark mode, German and English UI.
|
||||
@@ -67,7 +67,7 @@ For access from other machines and for production use with TLS (bundled Caddy pr
|
||||
|
||||
```bash
|
||||
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
|
||||
OPNEXUS_VERSION=1.29.0
|
||||
OPNEXUS_VERSION=1.31.0
|
||||
```
|
||||
|
||||
```bash
|
||||
|
||||
@@ -160,6 +160,13 @@ class Settings(BaseSettings):
|
||||
# wireguard_key_store.py.
|
||||
wireguard_key_secret_dir: str = "/data/wireguard-key-secrets"
|
||||
|
||||
# ── Rule observation (usage counters + change history of the live rules) ─
|
||||
# Two read-only GETs per enabled firewall per cycle (searchRule, ruleStats). The interval is an operator
|
||||
# setting (env RULE_OBSERVE_INTERVAL_SECONDS); it can not go below the floor, which keeps a typo from
|
||||
# polling every firewall in a tight loop.
|
||||
rule_observe_interval_seconds: int = 900
|
||||
rule_observe_min_interval_seconds: int = 60
|
||||
|
||||
# ── Resource monitoring ──────────────────────────────────────────────
|
||||
# Fixed interval, no admin-editable settings row/UI for this — v1
|
||||
# deliberately kept minimal (see resource_monitor_service.py).
|
||||
|
||||
@@ -171,6 +171,7 @@ class ErrorCode(str, Enum):
|
||||
OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED"
|
||||
OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
|
||||
OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY"
|
||||
OPEN_CLEANUP_BLOCKS_DEPLOY = "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
UPSTREAM_REJECTED = "UPSTREAM_REJECTED"
|
||||
INVALID_CHARACTERS = "INVALID_CHARACTERS"
|
||||
SERVICE_NOT_FOUND = "SERVICE_NOT_FOUND"
|
||||
@@ -222,6 +223,30 @@ class ErrorCode(str, Enum):
|
||||
ANALYSE_INTERFACE_UNBEKANNT = "ANALYSE_INTERFACE_UNBEKANNT"
|
||||
ANALYSE_PARAMETER_UNGUELTIG = "ANALYSE_PARAMETER_UNGUELTIG"
|
||||
ANALYSE_FIREWALL_NICHT_LESBAR = "ANALYSE_FIREWALL_NICHT_LESBAR"
|
||||
# Rule cleanup (1.31, services/rule_cleanup_service.py)
|
||||
BEREINIGUNG_NICHT_GEFUNDEN = "BEREINIGUNG_NICHT_GEFUNDEN"
|
||||
BEREINIGUNG_AUSWAHL_UNGUELTIG = "BEREINIGUNG_AUSWAHL_UNGUELTIG"
|
||||
BEREINIGUNG_VORSCHAU_VERALTET = "BEREINIGUNG_VORSCHAU_VERALTET"
|
||||
BEREINIGUNG_REGEL_NICHT_AENDERBAR = "BEREINIGUNG_REGEL_NICHT_AENDERBAR"
|
||||
BEREINIGUNG_OFFENE_OPERATION = "BEREINIGUNG_OFFENE_OPERATION"
|
||||
BEREINIGUNG_ZU_VIELE_REGELN = "BEREINIGUNG_ZU_VIELE_REGELN"
|
||||
BEREINIGUNG_BESTAETIGUNG_FEHLT = "BEREINIGUNG_BESTAETIGUNG_FEHLT"
|
||||
BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN = "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
|
||||
BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG = "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG"
|
||||
BEREINIGUNG_FRIST_ABGELAUFEN = "BEREINIGUNG_FRIST_ABGELAUFEN"
|
||||
BEREINIGUNG_NICHT_BESTAETIGBAR = "BEREINIGUNG_NICHT_BESTAETIGBAR"
|
||||
BEREINIGUNG_LIVESTAND_ABWEICHEND = "BEREINIGUNG_LIVESTAND_ABWEICHEND"
|
||||
BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR = "BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR"
|
||||
BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT = "BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT"
|
||||
BEREINIGUNG_SNAPSHOT_FEHLT = "BEREINIGUNG_SNAPSHOT_FEHLT"
|
||||
BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR = "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
|
||||
BEREINIGUNG_AKTION_NICHT_MOEGLICH = "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
|
||||
BEREINIGUNG_NICHT_AUSGEFUEHRT = "BEREINIGUNG_NICHT_AUSGEFUEHRT"
|
||||
BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN = "BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN"
|
||||
BEREINIGUNG_AUSGANG_UNKLAR = "BEREINIGUNG_AUSGANG_UNKLAR"
|
||||
BEREINIGUNG_ABGEBROCHEN_VERWAIST = "BEREINIGUNG_ABGEBROCHEN_VERWAIST"
|
||||
BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET = "BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET"
|
||||
BEREINIGUNG_EINSTELLUNG_UNGUELTIG = "BEREINIGUNG_EINSTELLUNG_UNGUELTIG"
|
||||
|
||||
|
||||
def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:
|
||||
|
||||
+36
-1
@@ -53,6 +53,7 @@ from app.routers import scheduled_maintenance as scheduled_maintenance_router
|
||||
from app.routers import maintenance_windows as maintenance_windows_router
|
||||
from app.routers import rollouts as rollouts_router
|
||||
from app.routers import rule_analysis as rule_analysis_router
|
||||
from app.routers import rule_cleanup as rule_cleanup_router
|
||||
from app.routers import gui_access as gui_access_router
|
||||
from app.services import scheduled_maintenance_service
|
||||
from app.services import rollout_engine
|
||||
@@ -61,6 +62,8 @@ from app.services import audit_export_service
|
||||
from app.services import overlay_service, variables_service
|
||||
from app.services import backup_schedule_service
|
||||
from app.services import carp_event_service, carp_maintenance_window_service
|
||||
from app.services import rule_history_service
|
||||
from app.services import rule_cleanup_service
|
||||
|
||||
# Without this, Python's root logger defaults to WARNING with no handler,
|
||||
# so every logger.info() call in the 5 background loops below (CARP role
|
||||
@@ -141,6 +144,13 @@ async def _deploy_revert_sweep_loop():
|
||||
logger.info("Auto-reverted %d unconfirmed policy-hierarchy deployment(s)", len(reverted))
|
||||
except Exception:
|
||||
logger.exception("Policy-hierarchy deploy-revert sweep failed")
|
||||
try:
|
||||
async with SessionLocal() as db:
|
||||
reverted = await rule_cleanup_service.sweep_expired_cleanups(db)
|
||||
if reverted:
|
||||
logger.info("Regel-Bereinigung: %d Operation(en) automatisch zurückgenommen/abgeschlossen", len(reverted))
|
||||
except Exception:
|
||||
logger.exception("Rule-cleanup revert sweep failed")
|
||||
await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
@@ -234,6 +244,9 @@ async def _resource_poll_loop():
|
||||
deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
|
||||
if deleted:
|
||||
logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
|
||||
pruned = await rule_history_service.prune(db)
|
||||
if pruned:
|
||||
logger.info("Regelverlauf-Retention: %d alte Einträge gelöscht", pruned)
|
||||
except Exception:
|
||||
logger.exception("Resource-Poll fehlgeschlagen")
|
||||
await asyncio.sleep(settings.resource_poll_interval_seconds)
|
||||
@@ -322,6 +335,25 @@ async def _backup_poll_loop():
|
||||
await asyncio.sleep(_BACKUP_POLL_SECONDS)
|
||||
|
||||
|
||||
_RULE_OBSERVE_FIRST_RUN_SECONDS = 150
|
||||
|
||||
|
||||
async def _rule_observe_loop():
|
||||
"""Ninth background loop: reads the rules and their counters of every enabled firewall (two GETs each),
|
||||
keeps the usage sampling current and records what changed since the last look (services/rule_history_service.py).
|
||||
Writes only to the database. The first look at a firewall only stores the starting point."""
|
||||
await asyncio.sleep(_RULE_OBSERVE_FIRST_RUN_SECONDS)
|
||||
while True:
|
||||
try:
|
||||
async with SessionLocal() as db:
|
||||
result = await rule_history_service.observe_all(db)
|
||||
if result["events"] or result["failed"]:
|
||||
logger.info("Regelverlauf: %s", result)
|
||||
except Exception:
|
||||
logger.exception("Regelverlauf fehlgeschlagen")
|
||||
await asyncio.sleep(max(settings.rule_observe_interval_seconds, settings.rule_observe_min_interval_seconds))
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
conflicts = settings.gui_origin_conflicts()
|
||||
@@ -351,6 +383,7 @@ async def lifespan(app: FastAPI):
|
||||
compliance_task = asyncio.create_task(_compliance_poll_loop())
|
||||
audit_export_task = asyncio.create_task(_audit_export_poll_loop())
|
||||
backup_task = asyncio.create_task(_backup_poll_loop())
|
||||
rule_observe_task = asyncio.create_task(_rule_observe_loop())
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
@@ -362,7 +395,8 @@ async def lifespan(app: FastAPI):
|
||||
compliance_task.cancel()
|
||||
audit_export_task.cancel()
|
||||
backup_task.cancel()
|
||||
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task):
|
||||
rule_observe_task.cancel()
|
||||
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task, rule_observe_task):
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
@@ -536,5 +570,6 @@ app.include_router(scheduled_maintenance_router.router, prefix="/api")
|
||||
app.include_router(maintenance_windows_router.router, prefix="/api")
|
||||
app.include_router(rollouts_router.router, prefix="/api")
|
||||
app.include_router(rule_analysis_router.router, prefix="/api")
|
||||
app.include_router(rule_cleanup_router.router, prefix="/api")
|
||||
app.include_router(gui_access_router.router, prefix="/api")
|
||||
app.include_router(gui_access_router.internal_router)
|
||||
@@ -102,3 +102,9 @@ __all__ += ["RolloutPlan", "RolloutPlanWave", "RolloutRun", "RolloutRunHost", "R
|
||||
|
||||
from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
|
||||
__all__ += ["GuiAccessSettings", "GuiSession", "GuiTicket"]
|
||||
|
||||
from app.models.rule_history import LiveRuleState, RuleChangeEvent
|
||||
__all__ += ["LiveRuleState", "RuleChangeEvent"]
|
||||
|
||||
from app.models.rule_cleanup import RuleCleanupOperation, RuleCleanupItem
|
||||
__all__ += ["RuleCleanupOperation", "RuleCleanupItem"]
|
||||
@@ -1,5 +1,5 @@
|
||||
"""Separation of duties (singleton row): may operators roll changes out themselves?"""
|
||||
from sqlalchemy import Boolean, Integer
|
||||
from sqlalchemy import JSON, Boolean, Integer
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
@@ -11,3 +11,14 @@ class GovernanceSettings(Base):
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
# False: an operator prepares changes (edit, preview) but only an admin may deploy them.
|
||||
operators_may_deploy: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
|
||||
# Rule cleanup (1.31). At most this many rules per cleanup operation (default 10, ceiling 25).
|
||||
cleanup_max_rules: Mapped[int] = mapped_column(Integer, nullable=False, default=10)
|
||||
# A rule must have been switched off (and confirmed) this many days before it may be deleted (minimum 1).
|
||||
cleanup_delete_min_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7)
|
||||
# IPs/CIDRs the admin manages the firewalls from; no rule that could match them may be cleaned up. Empty list =
|
||||
# the cleanup is locked (fail closed). Only an admin may change it.
|
||||
cleanup_management_sources: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
# Extra TCP ports the firewalls are managed on (besides the API port of base_url, 443, 80 and 22): a rule that
|
||||
# could match a management source on one of them is protected too. Only an admin may change it.
|
||||
cleanup_management_ports: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Rule cleanup (1.31, B4 part 2): switching rules off or deleting them through preview -> deploy -> confirm.
|
||||
|
||||
``RuleCleanupOperation`` is one cleanup run on a host (or a cluster pair, anchored at one host); its id doubles as
|
||||
the preview token. ``RuleCleanupItem`` is one rule on one node of that run. The columns only carry what the design
|
||||
(DESIGN-1.31) needs to resume, revert and restore after a crash; the logic lives in services/rule_cleanup_service.py.
|
||||
"""
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy import JSON, DateTime, ForeignKey, Index, Integer, String, Text, text
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
# kind: disable | enable (re-enable after a restore) | delete | restore
|
||||
OPERATION_KINDS = ("disable", "enable", "delete", "restore")
|
||||
# preview -> running -> awaiting_confirm -> confirmed; the rest are the side exits.
|
||||
OPERATION_STATUSES = (
|
||||
"preview", "running", "awaiting_confirm", "confirmed", "partial", "apply_failed", "reverting", "reverted",
|
||||
"revert_failed", "done", "failed", "cancelled", "expired",
|
||||
)
|
||||
ITEM_STATES = (
|
||||
"queued", "disabling", "disabled", "confirmed", "reverting", "reverted", "deleting", "deleted", "restoring",
|
||||
"restored", "failed", "unknown", "revert_failed",
|
||||
# never attempted (the run stopped at an earlier error); not an open state, so the rule is free again
|
||||
"skipped",
|
||||
)
|
||||
# A rule in one of these states is being worked on or its outcome is unresolved: no second item may claim it.
|
||||
OPEN_ITEM_STATES = ("queued", "disabling", "disabled", "reverting", "deleting", "restoring", "unknown", "revert_failed")
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
class RuleCleanupOperation(Base):
|
||||
__tablename__ = "rule_cleanup_operations"
|
||||
__table_args__ = (
|
||||
Index("ix_rule_cleanup_operations_host", "host_id", "created_at"),
|
||||
Index("ix_rule_cleanup_operations_status", "status"),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
|
||||
cluster_name: Mapped[Optional[str]] = mapped_column(String(120), nullable=True)
|
||||
kind: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
status: Mapped[str] = mapped_column(String(20), nullable=False, default="preview")
|
||||
actor: Mapped[str] = mapped_column(String(150), nullable=False)
|
||||
# The stable account id next to the name (see migration 0026); no foreign key, a deleted account stays gone.
|
||||
actor_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=_now)
|
||||
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
# Written at every checkpoint, so another process can tell an abandoned ``running`` operation from a live one.
|
||||
heartbeat_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
# Auto-revert deadline (disable/enable only); NULL once confirmed or for kinds without a window.
|
||||
revert_deadline: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
confirmed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
finished_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
|
||||
# sha256 over everything the preview showed; the deploy recomputes it and refuses on any difference.
|
||||
preview_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
# unused_days, acknowledged rules (with the text version), protected management sources.
|
||||
options: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
|
||||
|
||||
|
||||
class RuleCleanupItem(Base):
|
||||
__tablename__ = "rule_cleanup_items"
|
||||
__table_args__ = (
|
||||
Index("ix_rule_cleanup_items_operation", "operation_id"),
|
||||
# Independent of any lock: one rule on one node can be in only one unfinished item at a time.
|
||||
Index(
|
||||
"uq_cleanup_rule_open", "host_id", "rule_uuid", unique=True,
|
||||
postgresql_where=text("state IN (" + ", ".join(f"'{s}'" for s in OPEN_ITEM_STATES) + ")"),
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||
operation_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rule_cleanup_operations.id", ondelete="CASCADE"), nullable=False)
|
||||
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
|
||||
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
seq: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
carp_role: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
state: Mapped[str] = mapped_column(String(16), nullable=False, default="queued")
|
||||
# What the preview showed.
|
||||
description: Mapped[str] = mapped_column(Text, nullable=False, default="")
|
||||
categories: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
confidence: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
row_before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
|
||||
fp_before: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
# shadowed/duplicate: the covering rule must stay unchanged until the deploy.
|
||||
cover_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
cover_fp: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
# Flattened getRule, raw answer and firmware; written BEFORE a rule is deleted.
|
||||
snapshot: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
|
||||
# Where the rule sat (neighbours in the same prio_group by sort_order), for the restore.
|
||||
anchor_next_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
anchor_prev_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
prio_group: Mapped[Optional[str]] = mapped_column(String(32), nullable=True)
|
||||
# delete -> the confirmed disable item it builds on; restore -> the delete item it undoes. Plain id, no foreign key.
|
||||
source_item_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
|
||||
restored_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
changed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Change history of the rules that are live on a firewall (all rules, also hand-made ones).
|
||||
|
||||
``live_rule_states`` is the last observed state per rule (a fingerprint plus the row itself), maintained by the
|
||||
``rule_observe`` loop; ``rule_change_events`` is what changed between two observations. See
|
||||
services/rule_history_service.py. Neither table holds anything the firewall itself would not show on its rule page.
|
||||
"""
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy import JSON, Boolean, DateTime, ForeignKey, Index, Integer, String, UniqueConstraint
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class LiveRuleState(Base):
|
||||
__tablename__ = "live_rule_states"
|
||||
|
||||
host_id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True)
|
||||
# The rule's UUID/label on the firewall; BASELINE_KEY marks "this host has been observed once".
|
||||
rule_uuid: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||
fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
row: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
# False = the rule is gone (kept as a marker so a returning UUID continues the numbering).
|
||||
present: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
# Counts the observed changes of this rule; the event of a change gets the new number, which makes the
|
||||
# observation idempotent when two processes see the same change.
|
||||
version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
changed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
|
||||
class RuleChangeEvent(Base):
|
||||
__tablename__ = "rule_change_events"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
|
||||
Index("ix_rule_change_events_rule", "host_id", "rule_uuid", "observed_at"),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||
host_id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
|
||||
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
seq: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
observed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
|
||||
kind: Mapped[str] = mapped_column(String(16), nullable=False) # added | changed | removed | enabled | disabled
|
||||
before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
|
||||
after: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
|
||||
origin: Mapped[str] = mapped_column(String(16), nullable=False) # opnexus | external
|
||||
actor: Mapped[Optional[str]] = mapped_column(String(150), nullable=True)
|
||||
# The audit entry that explains an OPNexus-made change; no foreign key, the audit trail outlives everything.
|
||||
audit_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
|
||||
# Start of the time since the previous observation of the host: only audit entries from then on can explain
|
||||
# this change (a later reconcile must not reach back to an older deployment).
|
||||
window_start: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
@@ -205,6 +205,8 @@ async def remove_previous_target(alias_id: uuid.UUID, host_id: uuid.UUID, approv
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in alias.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
await opnsense_client.del_alias_item(deployment.host, deployment.opnsense_alias_uuid)
|
||||
|
||||
@@ -1,14 +1,15 @@
|
||||
from datetime import datetime
|
||||
from fastapi import APIRouter, Depends, Query
|
||||
from fastapi import APIRouter, Depends, Query, Request
|
||||
from sqlalchemy import select, func, or_
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.database import get_db
|
||||
from app.models.audit import AuditEvent
|
||||
from app.services.audit_service import ADMIN_ONLY_DETAILS
|
||||
|
||||
router = APIRouter(prefix="/audit", tags=["audit"])
|
||||
|
||||
@router.get("")
|
||||
async def history(kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
|
||||
async def history(request: Request, kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
|
||||
status: str | None = None, action: str | None = None, actor: str | None = None,
|
||||
before: datetime | None = None, since: datetime | None = None,
|
||||
q: str = Query(default="", max_length=200), offset: int = Query(default=0, ge=0),
|
||||
@@ -24,5 +25,17 @@ async def history(kind: str | None = None, host_id: str | None = None, entity_id
|
||||
clauses.append(or_(*(c.ilike('%'+q+'%') for c in [AuditEvent.entity_name, AuditEvent.host_name, AuditEvent.actor, AuditEvent.message])))
|
||||
total = await db.scalar(select(func.count()).select_from(AuditEvent).where(*clauses))
|
||||
rows = (await db.scalars(select(AuditEvent).where(*clauses).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).offset(offset).limit(limit))).all()
|
||||
return {"total": total, "offset": offset, "limit": limit,
|
||||
"items": [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]}
|
||||
items = [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]
|
||||
if not full_view(request):
|
||||
for item in items:
|
||||
if isinstance(item.get("details"), dict) and ADMIN_ONLY_DETAILS in item["details"]:
|
||||
item["details"] = {**{k: v for k, v in item["details"].items() if k != ADMIN_ONLY_DETAILS},
|
||||
ADMIN_ONLY_DETAILS: "redacted"}
|
||||
return {"total": total, "offset": offset, "limit": limit, "items": items}
|
||||
|
||||
|
||||
def full_view(request: Request | None) -> bool:
|
||||
"""An admin account, and with a token only one of scope admin."""
|
||||
user = getattr(getattr(request, "state", None), "user", None)
|
||||
token = getattr(getattr(request, "state", None), "api_token", None)
|
||||
return getattr(user, "role", None) == "admin" and (token is None or token.scope == "admin")
|
||||
@@ -1,6 +1,15 @@
|
||||
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it)."""
|
||||
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it).
|
||||
|
||||
The rule-cleanup settings (1.31) live here too: how many rules one cleanup may touch, how long a rule must stay
|
||||
switched off before it may be deleted, the management sources (addresses the firewalls are managed from; no rule
|
||||
that could match them is ever cleaned up) and extra management ports (besides the API port, 443, 80 and 22). Only an admin writes any of it (middleware: ``/api/settings/...`` is
|
||||
admin-only for operators and tokens below scope admin; checked again here) and every change is audited with the
|
||||
old and the new values."""
|
||||
import ipaddress
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
@@ -8,12 +17,83 @@ from app.errors import ErrorCode, api_error
|
||||
from app.models.governance import GovernanceSettings
|
||||
from app.models.user import AdminUser
|
||||
from app.routers.auth import current_user
|
||||
from app.services import audit_service
|
||||
|
||||
router = APIRouter(prefix="/settings/governance", tags=["governance"])
|
||||
|
||||
MAX_CLEANUP_RULES = 25 # same ceiling as services/rule_cleanup_service.MAX_RULES_CEILING
|
||||
MAX_DELETE_MIN_DAYS = 365
|
||||
MAX_MANAGEMENT_SOURCES = 20
|
||||
MAX_MANAGEMENT_PORTS = 20
|
||||
|
||||
|
||||
class GovernanceRead(BaseModel):
|
||||
operators_may_deploy: bool
|
||||
cleanup_max_rules: int = 10
|
||||
cleanup_delete_min_days: int = 7
|
||||
# The list itself is shown to admins only; everybody else sees just whether it is set.
|
||||
cleanup_management_sources: list[str] = Field(default_factory=list)
|
||||
cleanup_management_sources_configured: bool = False
|
||||
# Extra management ports (besides API port, 443, 80, 22): also admins only, others see the flag.
|
||||
cleanup_management_ports: list[int] = Field(default_factory=list)
|
||||
cleanup_management_ports_configured: bool = False
|
||||
|
||||
|
||||
class GovernanceUpdate(BaseModel):
|
||||
operators_may_deploy: bool | None = None
|
||||
cleanup_max_rules: int | None = None
|
||||
cleanup_delete_min_days: int | None = None
|
||||
cleanup_management_sources: list[str] | None = None
|
||||
cleanup_management_ports: list[Any] | None = None # validated in clean_management_ports (no bool coercion)
|
||||
|
||||
|
||||
def _bad(field: str, reason: str):
|
||||
return api_error(422, ErrorCode.BEREINIGUNG_EINSTELLUNG_UNGUELTIG, field=field, reason=reason)
|
||||
|
||||
|
||||
def clean_management_sources(values: list[str]) -> list[str]:
|
||||
"""IP addresses and CIDR networks only (no host names: they could resolve differently later), canonical form,
|
||||
without duplicates; not the whole address space, which would protect every rule."""
|
||||
if len(values) > MAX_MANAGEMENT_SOURCES:
|
||||
raise _bad("cleanup_management_sources", "too_many")
|
||||
cleaned: list[str] = []
|
||||
for raw in values:
|
||||
text = str(raw).strip()
|
||||
try:
|
||||
net = ipaddress.ip_network(text, strict=False)
|
||||
except ValueError:
|
||||
raise _bad("cleanup_management_sources", "not_an_address") from None
|
||||
if net.prefixlen == 0:
|
||||
raise _bad("cleanup_management_sources", "everything")
|
||||
entry = str(net.network_address) if net.prefixlen == net.max_prefixlen else str(net)
|
||||
if entry not in cleaned:
|
||||
cleaned.append(entry)
|
||||
return cleaned
|
||||
|
||||
|
||||
def clean_management_ports(values) -> list[int]:
|
||||
"""TCP ports 1..65535, at most MAX_MANAGEMENT_PORTS, without duplicates, sorted."""
|
||||
if len(values) > MAX_MANAGEMENT_PORTS:
|
||||
raise _bad("cleanup_management_ports", "too_many")
|
||||
ports: set[int] = set()
|
||||
for raw in values:
|
||||
text = str(raw).strip()
|
||||
if isinstance(raw, bool) or not text.isdigit() or not 1 <= int(text) <= 65535:
|
||||
raise _bad("cleanup_management_ports", "not_a_port")
|
||||
ports.add(int(text))
|
||||
return sorted(ports)
|
||||
|
||||
|
||||
def _read(row: GovernanceSettings, user: AdminUser) -> GovernanceRead:
|
||||
sources = list(row.cleanup_management_sources or [])
|
||||
ports = list(row.cleanup_management_ports or [])
|
||||
return GovernanceRead(
|
||||
operators_may_deploy=row.operators_may_deploy, cleanup_max_rules=row.cleanup_max_rules or 10,
|
||||
cleanup_delete_min_days=row.cleanup_delete_min_days or 7,
|
||||
cleanup_management_sources=sources if user.role == "admin" else [],
|
||||
cleanup_management_sources_configured=bool(sources),
|
||||
cleanup_management_ports=ports if user.role == "admin" else [],
|
||||
cleanup_management_ports_configured=bool(ports))
|
||||
|
||||
|
||||
async def get_settings(db: AsyncSession) -> GovernanceSettings:
|
||||
@@ -28,14 +108,33 @@ async def get_settings(db: AsyncSession) -> GovernanceSettings:
|
||||
@router.get("", response_model=GovernanceRead)
|
||||
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
|
||||
"""Readable by everybody signed in (the UI explains why a deploy was refused)."""
|
||||
return await get_settings(db)
|
||||
return _read(await get_settings(db), user)
|
||||
|
||||
|
||||
@router.put("", response_model=GovernanceRead)
|
||||
async def update(payload: GovernanceRead, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
|
||||
async def update(payload: GovernanceUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
|
||||
if user.role != "admin":
|
||||
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
|
||||
row = await get_settings(db)
|
||||
row.operators_may_deploy = payload.operators_may_deploy
|
||||
changes: dict[str, dict] = {}
|
||||
if payload.cleanup_max_rules is not None:
|
||||
if not 1 <= payload.cleanup_max_rules <= MAX_CLEANUP_RULES:
|
||||
raise _bad("cleanup_max_rules", f"1..{MAX_CLEANUP_RULES}")
|
||||
if payload.cleanup_delete_min_days is not None:
|
||||
if not 1 <= payload.cleanup_delete_min_days <= MAX_DELETE_MIN_DAYS:
|
||||
raise _bad("cleanup_delete_min_days", f"1..{MAX_DELETE_MIN_DAYS}")
|
||||
sources = clean_management_sources(payload.cleanup_management_sources) if payload.cleanup_management_sources is not None else None
|
||||
ports = clean_management_ports(payload.cleanup_management_ports) if payload.cleanup_management_ports is not None else None
|
||||
for field, value in (("cleanup_max_rules", payload.cleanup_max_rules), ("cleanup_delete_min_days", payload.cleanup_delete_min_days),
|
||||
("cleanup_management_sources", sources), ("cleanup_management_ports", ports)):
|
||||
if value is not None and value != getattr(row, field):
|
||||
changes[field] = {"old": getattr(row, field), "new": value}
|
||||
setattr(row, field, value)
|
||||
if payload.operators_may_deploy is not None:
|
||||
row.operators_may_deploy = payload.operators_may_deploy
|
||||
if changes:
|
||||
db.add(audit_service.make_event(
|
||||
kind="settings", action="cleanup_settings", status="accepted", entity_id="governance",
|
||||
message="Einstellungen der Regel-Bereinigung geändert", details={"changes": changes}))
|
||||
await db.commit()
|
||||
return row
|
||||
return _read(row, user)
|
||||
@@ -147,6 +147,11 @@ async def get_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
|
||||
async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession = Depends(get_db)):
|
||||
await policy_lock(db)
|
||||
host = await _get_host_or_404(db, host_id)
|
||||
if (payload.base_url is not None or payload.enabled is not None or payload.clear_cluster_name
|
||||
or payload.cluster_name is not None):
|
||||
# An open rule cleanup (1.31) reverts through this address and this cluster pairing: not while it is open.
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [host])
|
||||
|
||||
if payload.name is not None:
|
||||
existing = await db.scalar(
|
||||
@@ -188,6 +193,9 @@ async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
if not host:
|
||||
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
|
||||
# An open rule cleanup (1.31) holds a revert deadline and snapshots for this host (and its cluster peers).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [host])
|
||||
# Both deployment tables FK to opnsense_hosts with ondelete=CASCADE, so
|
||||
# deleting the host would silently drop OPNexus's only record of what
|
||||
# it pushed there — the rule/alias itself stays live on the actual
|
||||
|
||||
@@ -193,6 +193,8 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
|
||||
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
if deployment.opnsense_child_uuid:
|
||||
|
||||
@@ -113,6 +113,23 @@ TOOLS = [
|
||||
# The per-rule trace is for the UI; the verdict, deciding rule and reasons are what matters here.
|
||||
"drop": ("trace",),
|
||||
},
|
||||
{
|
||||
"name": "rule_cleanup_candidates",
|
||||
"description": "Cleanup candidates of one firewall: rules that are unused (no hit for N days), shadowed or "
|
||||
"duplicate (an earlier quick rule already decides everything they match), or disabled for a "
|
||||
"long time. Each with its confidence (sure/probable/unknown), the last observed hit and "
|
||||
"whether it could be removed (not for automatic, legacy or OPNexus-managed rules). Reads only.",
|
||||
"inputSchema": {
|
||||
"type": "object", "required": ["host_id"],
|
||||
"properties": {
|
||||
"host_id": {"type": "string", "description": "Firewall id from list_hosts"},
|
||||
"unused_days": {"type": "integer", "minimum": 1, "maximum": 3650, "default": 30,
|
||||
"description": "observation window for unused / disabled rules"},
|
||||
},
|
||||
},
|
||||
"path": "/api/hosts/{host_id}/rule-analysis/cleanup",
|
||||
"query": ("unused_days",),
|
||||
},
|
||||
{
|
||||
"name": "api_get",
|
||||
"description": "Read any other OPNexus API endpoint (GET only), e.g. /api/rules, /api/aliases, /api/logs. "
|
||||
|
||||
@@ -231,6 +231,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in rule.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
await opnsense_client.del_one_to_one_rule(deployment.host, deployment.opnsense_one_to_one_uuid)
|
||||
|
||||
@@ -195,6 +195,8 @@ async def remove_previous_target(instance_id: uuid.UUID, host_id: uuid.UUID, app
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
|
||||
deployment = next(d for d in instance.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
if deployment.opnsense_instance_uuid:
|
||||
|
||||
@@ -238,6 +238,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in rule.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
await opnsense_client.del_outbound_nat_rule(deployment.host, deployment.opnsense_outbound_nat_uuid)
|
||||
|
||||
@@ -330,6 +330,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in rule.deployments if d.host_id == host_id)
|
||||
from app.services import opnsense_client
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
try:
|
||||
await opnsense_client.del_rule(deployment.host, deployment.opnsense_policy_hierarchy_uuid)
|
||||
await opnsense_client.apply(deployment.host)
|
||||
|
||||
@@ -233,6 +233,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in rule.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
await opnsense_client.del_port_forward_rule(deployment.host, deployment.opnsense_port_forward_uuid)
|
||||
|
||||
@@ -17,6 +17,8 @@ from app.database import get_db
|
||||
from app.errors import ErrorCode, api_error
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.policy import DeviceGroup
|
||||
from app.models.rule_history import LiveRuleState
|
||||
from app.services import rule_history_service
|
||||
from app.services.rule_analysis import service
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -89,3 +91,38 @@ async def fleet_match(
|
||||
return await _run(host, q, refresh)
|
||||
|
||||
return list(await asyncio.gather(*(one(h) for h in hosts)))
|
||||
|
||||
|
||||
@router.get("/hosts/{host_id}/rule-analysis/cleanup")
|
||||
async def host_cleanup(
|
||||
host_id: uuid.UUID, unused_days: int = Query(30, ge=1, le=3650, description="observation window for 'unused' / 'disabled for a long time'"),
|
||||
refresh: bool = False, db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Cleanup candidates (unused, shadowed, duplicate, disabled for a long time) with confidence and whether
|
||||
each rule could be offered for removal. Read-only; removal is not part of this endpoint."""
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
if host is None:
|
||||
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
|
||||
try:
|
||||
return await service.cleanup_candidates(db, host, unused_days=unused_days, refresh=refresh)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001 - an unreachable firewall is a 502, not a crash
|
||||
logger.info("cleanup candidates failed for %s: %s", host.name, exc)
|
||||
raise api_error(502, ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR) from exc
|
||||
|
||||
|
||||
@router.get("/hosts/{host_id}/live-rules/{rule_uuid}/history")
|
||||
async def live_rule_history(host_id: uuid.UUID, rule_uuid: str, limit: int = Query(100, ge=1, le=500),
|
||||
db: AsyncSession = Depends(get_db)):
|
||||
"""What happened to one rule on one firewall: observed changes (also hand-made ones), saved revisions of
|
||||
rules OPNexus manages and the audit entries, newest first. Reads the database only."""
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
if host is None:
|
||||
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
|
||||
baseline = await db.get(LiveRuleState, (host_id, rule_history_service.BASELINE_KEY))
|
||||
return {
|
||||
"host_id": str(host.id), "rule_uuid": rule_uuid.lower()[:64],
|
||||
"watching_since": baseline.first_seen_at if baseline else None,
|
||||
"events": await rule_history_service.rule_timeline(db, host, rule_uuid, limit),
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
"""Rule cleanup (1.31, B4 part 2): switch hand-made rules off, delete them later, restore them (DESIGN-1.31 §4).
|
||||
|
||||
POST /hosts/{id}/rule-cleanup/preview operator, admin prepare (writes nothing to a firewall)
|
||||
POST /hosts/{id}/rule-cleanup/{op}/cancel operator, admin the preview's own actor
|
||||
POST /hosts/{id}/rule-cleanup/{op}/deploy ADMIN, token scope admin
|
||||
POST /hosts/{id}/rule-cleanup/{op}/confirm operator, admin BROWSER SESSION ONLY, never a token
|
||||
POST /hosts/{id}/rule-cleanup/{op}/revert admin; operator only with operators_may_deploy
|
||||
POST /hosts/{id}/rule-cleanup/{op}/retry ADMIN, token scope admin
|
||||
POST /hosts/{id}/rule-cleanup/items/{item}/restore ADMIN, token scope admin
|
||||
GET /hosts/{id}/rule-cleanup, /{op} everybody signed in (read tokens too)
|
||||
|
||||
Who may do what is enforced twice: AuthMiddleware (services/api_token_service.py: _ADMIN_ONLY_CLEANUP,
|
||||
_SESSION_ONLY, _ROLLOUT_POST) and here, so a gap in one layer does not open the route. The audit trail is the
|
||||
service's (one ``rule_cleanup`` entry per step and rule) plus the middleware's request entry (kind ``host``,
|
||||
action = last path segment; ``/preview`` is not logged by it, the service writes its own entry).
|
||||
The services raise ``api_error``; nothing is translated here. MCP stays read-only (no tool, ``api_get`` is GET).
|
||||
"""
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, Request
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.errors import ErrorCode, api_error
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.user import AdminUser
|
||||
from app.routers import governance
|
||||
from app.routers.auth import resolve_user
|
||||
from app.schemas.rule_cleanup import CleanupOperationRead, CleanupPreviewRead, ConfirmName, PreviewRequest
|
||||
from app.services import rule_cleanup_service as service
|
||||
from app.services.rule_cleanup_service import ACK_TEXT_VERSION
|
||||
|
||||
router = APIRouter(prefix="/hosts/{host_id}/rule-cleanup", tags=["rule-cleanup"])
|
||||
|
||||
|
||||
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
if host is None:
|
||||
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
|
||||
return host
|
||||
|
||||
|
||||
def _token(request: Request):
|
||||
return getattr(request.state, "api_token", None)
|
||||
|
||||
|
||||
def _writer(user: AdminUser, request: Request) -> None:
|
||||
"""Any write: not for a read-only account, not with a read-scope token."""
|
||||
if user.role == "read_only":
|
||||
raise api_error(403, ErrorCode.READ_ONLY_ACCESS)
|
||||
token = _token(request)
|
||||
if token is not None and token.scope == "read":
|
||||
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
|
||||
|
||||
|
||||
def _admin_only(user: AdminUser, request: Request) -> None:
|
||||
"""Deploy, retry, restore: an admin account, and with a token only one of scope admin."""
|
||||
_writer(user, request)
|
||||
if user.role != "admin":
|
||||
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
|
||||
token = _token(request)
|
||||
if token is not None and token.scope != "admin":
|
||||
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
|
||||
|
||||
|
||||
async def _deployer(user: AdminUser, request: Request, db: AsyncSession) -> None:
|
||||
"""Revert: an admin, or an operator while the governance switch operators_may_deploy is on."""
|
||||
_writer(user, request)
|
||||
if user.role == "admin":
|
||||
return
|
||||
if user.role == "operator" and (await governance.get_settings(db)).operators_may_deploy:
|
||||
return
|
||||
raise api_error(403, ErrorCode.DEPLOY_NEEDS_ADMIN)
|
||||
|
||||
|
||||
def _session_only(user: AdminUser, request: Request) -> None:
|
||||
"""Confirm: "I still have access" is a human decision in a browser session, never a token's."""
|
||||
_writer(user, request)
|
||||
token = _token(request)
|
||||
if token is not None:
|
||||
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
|
||||
|
||||
|
||||
def _full_view(request: Request) -> bool:
|
||||
"""Admins (session or admin-scope token) see the management sources/ports and firewall addresses in a plan."""
|
||||
from app.routers.audit import full_view
|
||||
return full_view(request)
|
||||
|
||||
|
||||
async def _names(db: AsyncSession, operations: list[dict], request: Request) -> list[dict]:
|
||||
"""The firewall names the UI shows next to the ids; for non-admins without the security-relevant plan parts."""
|
||||
if not _full_view(request):
|
||||
operations = [service.redact(op) for op in operations]
|
||||
ids = {uuid.UUID(op["host_id"]) for op in operations} | {uuid.UUID(i["host_id"]) for op in operations for i in op["items"]}
|
||||
names = {h.id: h.name for h in (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(ids)))).all()} if ids else {}
|
||||
for op in operations:
|
||||
op["host_name"] = names.get(uuid.UUID(op["host_id"]))
|
||||
for item in op["items"]:
|
||||
item["host_name"] = names.get(uuid.UUID(item["host_id"]))
|
||||
return operations
|
||||
|
||||
|
||||
async def _one(db: AsyncSession, operation: dict, request: Request) -> dict:
|
||||
return (await _names(db, [operation], request))[0]
|
||||
|
||||
|
||||
@router.get("", response_model=list[CleanupOperationRead])
|
||||
async def list_operations(host_id: uuid.UUID, request: Request, limit: int = Query(50, ge=1, le=100),
|
||||
db: AsyncSession = Depends(get_db)):
|
||||
"""The newest operations of this host (and its cluster peers) with their items; unexpired previews included."""
|
||||
host = await _host(db, host_id)
|
||||
return await _names(db, await service.list_operations(db, host, limit), request)
|
||||
|
||||
|
||||
@router.post("/preview", response_model=CleanupPreviewRead)
|
||||
async def preview(host_id: uuid.UUID, payload: PreviewRequest, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
"""Recompute the plan from the live firewall(s) and store a 5-minute preview bound to the caller."""
|
||||
user = await resolve_user(request, db)
|
||||
_writer(user, request)
|
||||
host = await _host(db, host_id)
|
||||
kind, uuids, acknowledge = payload.resolved()
|
||||
if kind is None:
|
||||
raise api_error(422, ErrorCode.BEREINIGUNG_AUSWAHL_UNGUELTIG, field="kind")
|
||||
result = await service.preview(
|
||||
db, host, actor=user.username, actor_id=user.id, kind=kind, uuids=uuids, acknowledge=acknowledge,
|
||||
**({"unused_days": payload.unused_days} if payload.unused_days else {}),
|
||||
ack_text_version=payload.ack_text_version or ACK_TEXT_VERSION)
|
||||
result["operation"] = await _one(db, await service.get_operation(db, host, uuid.UUID(result["preview_id"])), request)
|
||||
return result if _full_view(request) else service.redact(result)
|
||||
|
||||
|
||||
@router.get("/{op_id}", response_model=CleanupOperationRead)
|
||||
async def get_operation(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.get_operation(db, host, op_id), request)
|
||||
|
||||
|
||||
@router.post("/{op_id}/cancel", response_model=CleanupOperationRead)
|
||||
async def cancel(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
"""Drop an unused preview (only its own actor can)."""
|
||||
user = await resolve_user(request, db)
|
||||
_writer(user, request)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.cancel(db, host, op_id, actor=user.username), request)
|
||||
|
||||
|
||||
@router.post("/{op_id}/deploy", response_model=CleanupOperationRead)
|
||||
async def deploy(host_id: uuid.UUID, op_id: uuid.UUID, payload: ConfirmName, request: Request,
|
||||
db: AsyncSession = Depends(get_db)):
|
||||
"""Execute a preview. Admin only; the live state must still give the fingerprint of the preview."""
|
||||
user = await resolve_user(request, db)
|
||||
_admin_only(user, request)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.deploy(db, host, op_id, actor=user.username, confirm_name=payload.confirm_name), request)
|
||||
|
||||
|
||||
@router.post("/{op_id}/confirm", response_model=CleanupOperationRead)
|
||||
async def confirm(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
""""I still have access": re-reads every node first. Browser session only, never with an API token."""
|
||||
user = await resolve_user(request, db)
|
||||
_session_only(user, request)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.confirm(db, host, op_id), request)
|
||||
|
||||
|
||||
@router.post("/{op_id}/revert", response_model=CleanupOperationRead)
|
||||
async def revert(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
"""Take the change back now (the same code the sweeper runs when the window passes)."""
|
||||
user = await resolve_user(request, db)
|
||||
await _deployer(user, request, db)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.revert_now(db, host, op_id), request)
|
||||
|
||||
|
||||
@router.post("/{op_id}/retry", response_model=CleanupOperationRead)
|
||||
async def retry(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
|
||||
"""Revert again after revert_failed, or read the firewall again for a delete/restore with an unclear item."""
|
||||
user = await resolve_user(request, db)
|
||||
_admin_only(user, request)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.retry(db, host, op_id), request)
|
||||
|
||||
|
||||
@router.post("/items/{item_id}/restore", response_model=CleanupOperationRead)
|
||||
async def restore(host_id: uuid.UUID, item_id: uuid.UUID, payload: ConfirmName, request: Request,
|
||||
db: AsyncSession = Depends(get_db)):
|
||||
"""Bring a deleted rule back, switched OFF, at its old position (a new UUID). The result is the restore run;
|
||||
``warnings`` can hold BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR."""
|
||||
user = await resolve_user(request, db)
|
||||
_admin_only(user, request)
|
||||
host = await _host(db, host_id)
|
||||
return await _one(db, await service.restore_item(
|
||||
db, host, item_id, actor=user.username, actor_id=user.id, confirm_name=payload.confirm_name), request)
|
||||
@@ -108,6 +108,10 @@ async def adopt_rule(payload: AdoptRequest, db: AsyncSession = Depends(get_db)):
|
||||
host = await db.get(OpnsenseHost, payload.host_id)
|
||||
if not host:
|
||||
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
|
||||
# Taking over a hand-made rule that a rule cleanup (1.31) is switching off/deleting would make it "managed"
|
||||
# under the cleanup's feet: not while one is open on this host or its cluster.
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [host])
|
||||
try:
|
||||
rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid, link_to=payload.link_to)
|
||||
except rule_adopt_service.RuleNotFoundOnHost as exc:
|
||||
@@ -276,6 +280,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
|
||||
deployment = next(d for d in rule.deployments if d.host_id == host_id)
|
||||
from app.services import opnsense_client
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
try:
|
||||
await opnsense_client.del_rule(deployment.host, deployment.opnsense_rule_uuid)
|
||||
await opnsense_client.apply(deployment.host)
|
||||
|
||||
@@ -207,6 +207,8 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
|
||||
if not entry["can_remove"]:
|
||||
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
|
||||
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
|
||||
from app.services import opnsense_client
|
||||
try:
|
||||
if deployment.opnsense_peer_uuid:
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Request and response shapes of the rule-cleanup API (routers/rule_cleanup.py, DESIGN-1.31 §4).
|
||||
|
||||
Responses mirror services/rule_cleanup_service.read_operation. Texts are not part of the API: ``message`` holds an
|
||||
ErrorCode name (``errors.<CODE>`` in the UI, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR) or the firewall's
|
||||
own wording; ``warnings`` is a list of ErrorCode names.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
CleanupKind = Literal["disable", "enable", "delete"]
|
||||
# Longest list the API accepts at all; the effective limit is the setting cleanup_max_rules (ceiling 25).
|
||||
MAX_SELECTION = 100
|
||||
|
||||
|
||||
class PreviewItem(BaseModel):
|
||||
"""One rule of the selection. ``kind`` (alias ``mode``) must be the same for all items of one preview."""
|
||||
uuid: str = Field(min_length=1, max_length=64)
|
||||
kind: CleanupKind | None = None
|
||||
mode: CleanupKind | None = None
|
||||
# The per-rule acknowledgement for a "probable" rule (the UI's checkbox).
|
||||
ack: bool = False
|
||||
|
||||
|
||||
class PreviewRequest(BaseModel):
|
||||
"""``{kind, uuids, acknowledge}`` or ``{items:[{uuid, kind|mode, ack}]}`` (or both: they are merged)."""
|
||||
kind: CleanupKind | None = None
|
||||
uuids: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
|
||||
items: list[PreviewItem] = Field(default_factory=list, max_length=MAX_SELECTION)
|
||||
acknowledge: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
|
||||
unused_days: int | None = Field(default=None, ge=1, le=3650)
|
||||
ack_text_version: str | None = Field(default=None, max_length=16)
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _single_kind(self):
|
||||
kinds = {k for k in [self.kind, *[i.kind or i.mode for i in self.items]] if k}
|
||||
if len(kinds) > 1:
|
||||
raise ValueError("kind must be the same for every item")
|
||||
return self
|
||||
|
||||
def resolved(self) -> tuple[str | None, list[str], list[str]]:
|
||||
kind = self.kind or next((i.kind or i.mode for i in self.items if i.kind or i.mode), None)
|
||||
uuids = [*self.uuids, *[i.uuid for i in self.items]]
|
||||
acknowledge = [*self.acknowledge, *[i.uuid for i in self.items if i.ack]]
|
||||
return kind, list(dict.fromkeys(u.strip() for u in uuids)), list(dict.fromkeys(a.strip() for a in acknowledge))
|
||||
|
||||
|
||||
class ConfirmName(BaseModel):
|
||||
"""Deploy and restore: the exact host name, typed by a human."""
|
||||
confirm_name: str = Field(max_length=255)
|
||||
|
||||
|
||||
class CleanupItemRead(BaseModel):
|
||||
id: str
|
||||
host_id: str
|
||||
host_name: str | None = None
|
||||
rule_uuid: str
|
||||
seq: int
|
||||
carp_role: str | None = None
|
||||
# queued, disabling, disabled, confirmed, reverting, reverted, deleting, deleted, restoring, restored, failed,
|
||||
# unknown, revert_failed, skipped (never attempted: the run stopped earlier)
|
||||
state: str
|
||||
description: str | None = None
|
||||
categories: Any = None
|
||||
confidence: str | None = None
|
||||
cover_uuid: str | None = None
|
||||
prio_group: str | None = None
|
||||
anchor_next_uuid: str | None = None
|
||||
anchor_prev_uuid: str | None = None
|
||||
source_item_id: str | None = None
|
||||
restored_uuid: str | None = None
|
||||
changed_at: str | None = None
|
||||
message: str | None = None
|
||||
has_snapshot: bool = False
|
||||
# A deleted rule with a stored snapshot that has not been restored yet (POST items/{id}/restore, admin).
|
||||
restorable: bool = False
|
||||
|
||||
|
||||
class CleanupOperationRead(BaseModel):
|
||||
id: str
|
||||
host_id: str
|
||||
host_name: str | None = None
|
||||
cluster_name: str | None = None
|
||||
kind: str # disable | enable | delete | restore
|
||||
# preview, running, awaiting_confirm, confirmed, partial, apply_failed, reverting, reverted, revert_failed, done,
|
||||
# failed, cancelled, expired
|
||||
status: str
|
||||
actor: str
|
||||
created_at: str | None = None
|
||||
expires_at: str | None = None
|
||||
started_at: str | None = None
|
||||
heartbeat_at: str | None = None
|
||||
revert_deadline: str | None = None
|
||||
confirmed_at: str | None = None
|
||||
finished_at: str | None = None
|
||||
message: str | None = None
|
||||
options: dict[str, Any] = Field(default_factory=dict)
|
||||
# Only while status == "preview".
|
||||
plan: dict[str, Any] | None = None
|
||||
items: list[CleanupItemRead] = Field(default_factory=list)
|
||||
can_confirm: bool = False
|
||||
can_revert: bool = False
|
||||
can_retry: bool = False
|
||||
# ErrorCode names, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR (restore only).
|
||||
warnings: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CleanupPreviewRead(BaseModel):
|
||||
preview_id: str
|
||||
expires_at: str | None = None
|
||||
kind: str
|
||||
plan: dict[str, Any]
|
||||
max_rules: int
|
||||
window_seconds: int
|
||||
operation: CleanupOperationRead
|
||||
@@ -130,6 +130,9 @@ async def sweep_expired_alias_deployments(db: AsyncSession) -> list[AliasDeploym
|
||||
|
||||
|
||||
async def undeploy_alias(db: AsyncSession, alias: FirewallAlias) -> None:
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, AliasDeployment, AliasDeployment.alias_id == alias.id)
|
||||
failures = []
|
||||
result = await db.execute(select(AliasDeployment).where(AliasDeployment.alias_id == alias.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -39,7 +39,9 @@ _LAST_USED_GRANULARITY = timedelta(minutes=1)
|
||||
|
||||
_SAFE_METHODS = ("GET", "HEAD", "OPTIONS")
|
||||
# GUI tickets open a firewall's web interface in the browser: an interactive admin session only.
|
||||
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$")
|
||||
# Rule cleanup (1.31): "I still have access" is a human decision, so confirming a cleanup is never possible with a
|
||||
# token, whatever its scope (a script must not be able to confirm blindly).
|
||||
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$|^/api/hosts/[^/]+/rule-cleanup/[^/]+/confirm/?$")
|
||||
# Reads that expose secrets (config.xml holds password hashes and VPN keys, the
|
||||
# notification settings the webhook URL/SMTP data): admin scope only.
|
||||
_ADMIN_SCOPE_READS = re.compile(r"^/api/hosts/[^/]+/backups/[^/]+$|^/api/notifications(/|$)|^/api/settings/|^/api/overlay/peers/[^/]+/instructions$")
|
||||
@@ -55,6 +57,13 @@ _ADMIN_ONLY_WRITES = re.compile(
|
||||
)
|
||||
|
||||
|
||||
# Rule cleanup: executing a cleanup, retrying a failed revert/reconcile and restoring a deleted rule are admin-only.
|
||||
# operators_may_deploy defaults to True, so _ROLLOUT_POST alone cannot express "admin only"; this pattern is checked
|
||||
# for operator accounts (operator_denies) and for tokens below scope admin (denies). Not part of _ADMIN_ONLY_WRITES:
|
||||
# that one means "who may access OPNexus / where data goes". The router checks the same rule once more.
|
||||
_ADMIN_ONLY_CLEANUP = re.compile(r"^/api/hosts/[^/]+/rule-cleanup/([^/]+/(deploy|retry)|items/[^/]+/restore)/?$")
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
@@ -73,7 +82,7 @@ def denies(scope: str, method: str, path: str) -> bool:
|
||||
return scope != "admin" and bool(_ADMIN_SCOPE_READS.match(path))
|
||||
if scope == "read":
|
||||
return True
|
||||
if scope == "deploy" and _ADMIN_ONLY_WRITES.match(path):
|
||||
if scope == "deploy" and (_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path)):
|
||||
return True
|
||||
return False
|
||||
|
||||
@@ -82,7 +91,8 @@ def operator_denies(method: str, path: str) -> bool:
|
||||
"""True if an `operator` account must not make this request: the writes that
|
||||
change who may access OPNexus or where it sends data. (Their own theme at
|
||||
PUT /api/settings stays open; account endpoints are not matched here.)"""
|
||||
return method not in _SAFE_METHODS and path != "/api/settings" and bool(_ADMIN_ONLY_WRITES.match(path))
|
||||
return method not in _SAFE_METHODS and path != "/api/settings" and bool(
|
||||
_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path))
|
||||
|
||||
|
||||
# Writes that put a change onto the fleet (as opposed to preparing one): deploy, remove/rollback a deployment,
|
||||
@@ -92,6 +102,9 @@ def operator_denies(method: str, path: str) -> bool:
|
||||
_ROLLOUT_POST = re.compile(
|
||||
r"/(deploy|push|sign|poll|register|reassert-ordering"
|
||||
r"|deployments/[^/]+/(remove|retry-rollback)"
|
||||
# Rule cleanup (1.31): switching rules off (/deploy matches above) and taking it back. Confirming and
|
||||
# previewing stay open, like confirm_rule.
|
||||
r"|rule-cleanup/[^/]+/(revert|retry)"
|
||||
r"|maintenance/(execute|carp-execute|schedule)"
|
||||
r"|network/changes"
|
||||
r"|services/[^/]+/(start|stop|restart)"
|
||||
|
||||
@@ -13,6 +13,9 @@ from app.database import SessionLocal
|
||||
from app.models.audit import AuditEvent
|
||||
|
||||
context = ContextVar("audit_context", default=None)
|
||||
# A key in ``details`` whose content only admins (session or admin-scope token) get to read (routers/audit.py):
|
||||
# firewall addresses and management sources of the rule cleanup.
|
||||
ADMIN_ONLY_DETAILS = "admin_only"
|
||||
|
||||
|
||||
# Column widths of audit_events. entity_id/host_id/action are taken from the
|
||||
|
||||
@@ -26,6 +26,7 @@ from app.models.host import OpnsenseHost
|
||||
from app.services import backup_store, rule_hit_service
|
||||
from app.services import opnsense_client as client
|
||||
from app.services.opnsense_client import OpnsenseAPIError
|
||||
from app.services.rule_analysis import coverage
|
||||
|
||||
MAX_CONFIG_BYTES = 32 * 1024 * 1024
|
||||
BACKUP_MAX_AGE_DAYS = 7
|
||||
@@ -83,10 +84,15 @@ def _mvc_rule(rule: Element) -> dict:
|
||||
"sport": _text(rule, "source_port"),
|
||||
"dst": _text(rule, "destination_net") or "any", "dst_not": _text(rule, "destination_not") == "1",
|
||||
"dport": _text(rule, "destination_port"),
|
||||
"floating": False,
|
||||
"if_not": _text(rule, "interfacenot") == "1",
|
||||
"floating": False, # decided in extract_rules (needs the list of interfaces)
|
||||
"quick": _text(rule, "quick") != "0",
|
||||
"descr": _text(rule, "description") or "",
|
||||
"order": int(_text(rule, "sequence") or 0),
|
||||
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
|
||||
"received_on": _text(rule, "received-on") or "",
|
||||
"icmptype": ",".join(filter(None, [_text(rule, "icmptype"), _text(rule, "icmp6type")])),
|
||||
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
|
||||
}
|
||||
|
||||
|
||||
@@ -101,17 +107,29 @@ def _legacy_rule(rule: Element, index: int) -> dict:
|
||||
"ipprotocol": _text(rule, "ipprotocol") or "inet",
|
||||
"protocol": (_text(rule, "protocol") or "any").lower(),
|
||||
"src": src, "src_not": src_not, "sport": sport, "dst": dst, "dst_not": dst_not, "dport": dport,
|
||||
"if_not": rule.find("interfacenot") is not None,
|
||||
"floating": _text(rule, "floating") == "yes",
|
||||
"quick": _text(rule, "quick") != "0",
|
||||
# Legacy rules: an interface rule is always evaluated "quick"; only a floating rule has the option,
|
||||
# and it is off unless <quick> says yes.
|
||||
"quick": _text(rule, "floating") != "yes" or (_text(rule, "quick") or "").lower() in ("yes", "1"),
|
||||
"descr": _text(rule, "descr") or "",
|
||||
"order": index,
|
||||
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
|
||||
"received_on": _text(rule, "received-on") or "", "icmptype": _text(rule, "icmptype") or "",
|
||||
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
|
||||
}
|
||||
|
||||
|
||||
def extract_rules(root: Element) -> list[list[dict]]:
|
||||
"""Rule lists in evaluation order, one per source (MVC, legacy)."""
|
||||
mvc = sorted((_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")), key=lambda r: r["order"])
|
||||
mvc = [_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")]
|
||||
known = {node.tag for node in root.findall("interfaces/*")}
|
||||
for rule in mvc: # same rule as OPNsense: not exactly one interface (or "not") = floating
|
||||
rule["floating"] = len(rule["interfaces"]) != 1 or rule["if_not"]
|
||||
rule["group"] = not rule["floating"] and rule["interfaces"][0] not in known
|
||||
mvc.sort(key=coverage.config_order)
|
||||
legacy = [_legacy_rule(r, i) for i, r in enumerate(root.findall("filter/rule"))]
|
||||
legacy.sort(key=coverage.config_order)
|
||||
return [group for group in (mvc, legacy) if group]
|
||||
|
||||
|
||||
@@ -133,41 +151,22 @@ def _is_any_any(rule: dict) -> bool:
|
||||
|
||||
|
||||
|
||||
def _covers(a: dict, b: dict) -> bool:
|
||||
"""Every packet matched by rule b is already matched by the earlier rule a.
|
||||
Only exact-or-any comparisons, so it never claims coverage it cannot prove."""
|
||||
if not (a["enabled"] and b["enabled"]) or a["floating"] or b["floating"] or not a["quick"]:
|
||||
return False
|
||||
if not a["interfaces"] or not set(b["interfaces"]) <= set(a["interfaces"]):
|
||||
return False
|
||||
if a["direction"] != b["direction"] or a["ipprotocol"] != b["ipprotocol"]:
|
||||
return False
|
||||
if a["protocol"] != "any" and a["protocol"] != b["protocol"]:
|
||||
return False
|
||||
for side in ("src", "dst"):
|
||||
if a[side + "_not"] or b[side + "_not"]:
|
||||
if (a[side], a[side + "_not"]) != (b[side], b[side + "_not"]):
|
||||
return False
|
||||
elif a[side] != "any" and a[side] != b[side]:
|
||||
return False
|
||||
for port in ("sport", "dport"):
|
||||
if a[port] and a[port] != b[port]:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _label(rule: dict) -> str:
|
||||
return rule["descr"] or f"{rule['action']} {rule['src']} → {rule['dst']}"
|
||||
|
||||
|
||||
def analyse_rules(groups: list[list[dict]]) -> dict:
|
||||
"""Shadowed and duplicate rules. The coverage logic lives in rule_analysis/coverage.py (shared with the
|
||||
cleanup candidates); the rules of one list are compared in evaluation order."""
|
||||
shadowed, duplicates = [], []
|
||||
for rules in groups:
|
||||
for j, later in enumerate(rules):
|
||||
for earlier in rules[:j]:
|
||||
if _covers(earlier, later):
|
||||
(duplicates if earlier["action"] == later["action"] and _covers(later, earlier) else shadowed).append(_label(later))
|
||||
break
|
||||
ordered = sorted(rules, key=coverage.config_order)
|
||||
norm = [coverage.from_config_rule(rule) for rule in ordered]
|
||||
by_identity = {id(n): r for n, r in zip(norm, ordered)}
|
||||
for overlap in coverage.analyse(norm):
|
||||
if overlap.by.order == overlap.rule.order:
|
||||
continue # equal position in config.xml: the real order is unknown, claim nothing
|
||||
(duplicates if overlap.kind == "duplicate" else shadowed).append(_label(by_identity[id(overlap.rule)]))
|
||||
return {"shadowed": shadowed, "duplicates": duplicates}
|
||||
|
||||
|
||||
|
||||
@@ -34,7 +34,27 @@ async def peers(db, host):
|
||||
return [host]
|
||||
|
||||
|
||||
async def ensure_no_maintenance(db, hosts):
|
||||
async def ensure_no_open_cleanup(db, hosts):
|
||||
"""A rule cleanup (1.31) that is running, waiting for its confirmation or unresolved owns the host and its
|
||||
cluster peers: nothing else may write to the firewall until it is confirmed, reverted or settled."""
|
||||
from app.services.rule_cleanup_service import has_open_cleanup
|
||||
ids = set()
|
||||
for host in hosts:
|
||||
ids.update(h.id for h in await peers(db, host))
|
||||
if ids and await has_open_cleanup(db, ids):
|
||||
raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)
|
||||
|
||||
|
||||
async def ensure_no_open_cleanup_on(db, model, condition):
|
||||
"""ensure_no_open_cleanup for every host that has a deployment row of ``model`` matching ``condition`` (the
|
||||
undeploy paths: deleting a central object removes it from all its firewalls)."""
|
||||
ids = set((await db.scalars(select(model.host_id).where(condition))).all())
|
||||
hosts = [h for h in [await db.get(OpnsenseHost, i) for i in ids] if h is not None]
|
||||
await ensure_no_open_cleanup(db, hosts)
|
||||
|
||||
|
||||
async def ensure_no_maintenance(db, hosts, *, cleanup=True):
|
||||
"""cleanup=False: only the cleanup service itself, which reports an open cleanup with its own error code."""
|
||||
ids = set()
|
||||
for host in hosts:
|
||||
ids.update(h.id for h in await peers(db, host))
|
||||
@@ -42,6 +62,8 @@ async def ensure_no_maintenance(db, hosts):
|
||||
raise api_error(409, ErrorCode.OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY)
|
||||
if ids and await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
|
||||
raise api_error(409, ErrorCode.OPEN_RESTORE_BLOCKS_DEPLOY)
|
||||
if cleanup and ids:
|
||||
await ensure_no_open_cleanup(db, hosts)
|
||||
|
||||
|
||||
async def _shared_change_blockers(db, fleet):
|
||||
@@ -56,6 +78,9 @@ async def _shared_change_blockers(db, fleet):
|
||||
blocked.append("Eine frühere Wartungsaktion im Host/Cluster muss zuerst geprüft und abgeschlossen werden.")
|
||||
if await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(fleet_ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
|
||||
blocked.append("Eine Wiederherstellung im Host/Cluster ist offen und muss zuerst bestätigt oder abgebrochen werden.")
|
||||
from app.services.rule_cleanup_service import has_open_cleanup
|
||||
if await has_open_cleanup(db, fleet_ids):
|
||||
blocked.append("Eine Regel-Bereinigung im Host/Cluster ist offen und muss zuerst bestätigt oder zurückgenommen werden.")
|
||||
return blocked
|
||||
|
||||
|
||||
|
||||
@@ -303,6 +303,9 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel) -> None:
|
||||
for this entity to gate on) — deletes Child/Local/Remote before their
|
||||
parent Connection (the ModelRelationField dependency direction), then
|
||||
the PreSharedKey (independent of the other 4), then reconfigures."""
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, IpsecTunnelDeployment, IpsecTunnelDeployment.tunnel_id == tunnel.id)
|
||||
failures = []
|
||||
result = await db.execute(select(IpsecTunnelDeployment).where(IpsecTunnelDeployment.tunnel_id == tunnel.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -201,6 +201,62 @@ async def check_revert_failed(db: AsyncSession) -> dict[str, dict]:
|
||||
"host_name": host.name if host else None,
|
||||
"message": deployment.message,
|
||||
}
|
||||
problems.update(await _cleanup_problems(db))
|
||||
return problems
|
||||
|
||||
|
||||
# Rule cleanup (1.31) states that need a human: the automatic revert failed, or a run ended half-way.
|
||||
_CLEANUP_STILL_CHANGED = ("revert_failed", "unknown", "disabling", "disabled", "failed")
|
||||
|
||||
|
||||
async def _cleanup_problems(db: AsyncSession) -> dict[str, dict]:
|
||||
"""Rule cleanups whose automatic revert failed (per rule that could not be switched back) and runs stuck in
|
||||
``partial``/``apply_failed`` (a decision is needed before the window ends), plus delete/restore runs with an
|
||||
unresolved item. Same ``revert_failed`` alert as the deployments. Keys: ``rule_cleanup:<item id>`` for one rule,
|
||||
``rule_cleanup_op:<operation id>`` for a whole run, so one open problem is one message."""
|
||||
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
|
||||
|
||||
problems: dict[str, dict] = {}
|
||||
operations = (await db.scalars(select(RuleCleanupOperation).where(
|
||||
RuleCleanupOperation.status.in_(("revert_failed", "partial", "apply_failed", "failed"))))).all()
|
||||
for op in operations:
|
||||
items = list((await db.scalars(select(RuleCleanupItem).where(RuleCleanupItem.operation_id == op.id)
|
||||
.order_by(RuleCleanupItem.seq))).all())
|
||||
host = await db.get(OpnsenseHost, op.host_id)
|
||||
base = {
|
||||
"deployment_type": "rule_cleanup", "deployment_id": str(op.id), "host_id": str(op.host_id),
|
||||
"host_name": host.name if host else None, "operation": op.kind, "operation_status": op.status,
|
||||
}
|
||||
if op.status == "revert_failed":
|
||||
stuck = [i for i in items if i.state in _CLEANUP_STILL_CHANGED]
|
||||
for item in stuck:
|
||||
member = await db.get(OpnsenseHost, item.host_id)
|
||||
problems[f"rule_cleanup:{item.id}"] = {
|
||||
**base, "entity_id": item.rule_uuid, "host_id": str(item.host_id),
|
||||
"host_name": member.name if member else None, "rule": item.description, "item_state": item.state,
|
||||
"message": item.message or op.message,
|
||||
"hinweis": "Bereinigung: Die Regel konnte nicht automatisch zurückgesetzt werden und ist möglicherweise "
|
||||
"noch geändert. Auf der Firewall prüfen und in OPNexus „Erneut zurücknehmen“ wählen.",
|
||||
}
|
||||
if not stuck:
|
||||
problems[f"rule_cleanup_op:{op.id}"] = {**base, "entity_id": str(op.id), "message": op.message,
|
||||
"hinweis": "Bereinigung: Das automatische Zurücknehmen ist fehlgeschlagen. Zustand der Firewall prüfen."}
|
||||
elif op.status in ("partial", "apply_failed"):
|
||||
problems[f"rule_cleanup_op:{op.id}"] = {
|
||||
**base, "entity_id": str(op.id), "message": op.message,
|
||||
"regeln": ", ".join(i.rule_uuid for i in items if i.state in ("failed", "unknown")) or None,
|
||||
"hinweis": "Bereinigung nur teilweise ausgeführt (oder das Anwenden ist fehlgeschlagen). Bestätigen ist gesperrt; "
|
||||
"jetzt zurücknehmen oder die automatische Rücknahme am Fristende abwarten.",
|
||||
}
|
||||
elif op.kind in ("delete", "restore"):
|
||||
for item in (i for i in items if i.state == "unknown"):
|
||||
member = await db.get(OpnsenseHost, item.host_id)
|
||||
problems[f"rule_cleanup:{item.id}"] = {
|
||||
**base, "entity_id": item.rule_uuid, "host_id": str(item.host_id),
|
||||
"host_name": member.name if member else None, "rule": item.description, "item_state": item.state,
|
||||
"message": item.message or op.message,
|
||||
"hinweis": "Bereinigung: Der Ausgang beim Löschen/Wiederherstellen ist unklar. Mit „Erneut prüfen“ nachlesen.",
|
||||
}
|
||||
return problems
|
||||
|
||||
|
||||
|
||||
@@ -135,6 +135,9 @@ async def sweep_expired_one_to_one_deployments(db: AsyncSession) -> list[OneToOn
|
||||
|
||||
|
||||
async def undeploy_one_to_one(db: AsyncSession, rule: OneToOneRule) -> None:
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, OneToOneDeployment, OneToOneDeployment.one_to_one_id == rule.id)
|
||||
failures = []
|
||||
result = await db.execute(select(OneToOneDeployment).where(OneToOneDeployment.one_to_one_id == rule.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -371,6 +371,9 @@ async def undeploy_instance(db: AsyncSession, instance: OpenVpnInstance) -> None
|
||||
for this entity to gate on) — deletes the Instance before its
|
||||
referenced StaticKey (the ModelRelationField dependency direction),
|
||||
then reconfigures."""
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, OpenVpnInstanceDeployment, OpenVpnInstanceDeployment.instance_id == instance.id)
|
||||
failures = []
|
||||
result = await db.execute(select(OpenVpnInstanceDeployment).where(OpenVpnInstanceDeployment.instance_id == instance.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -10,6 +10,7 @@ Auth is HTTP Basic: username = API key, password = API secret.
|
||||
"""
|
||||
import logging
|
||||
import json
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
@@ -33,6 +34,11 @@ class OpnsenseAmbiguousError(OpnsenseAPIError):
|
||||
outcome of a write is unknown and must be reconciled, not retried blindly."""
|
||||
|
||||
|
||||
class OpnsenseNotFoundError(OpnsenseAPIError):
|
||||
"""The firewall answered, but there is no such object (an empty `getRule` answer: unknown UUID, or a rule that is
|
||||
not part of the rules model - legacy or automatic). A refusal, never an unknown outcome."""
|
||||
|
||||
|
||||
class OpnsenseForbiddenError(OpnsenseAPIError):
|
||||
"""The firewall answered 403: the API key lacks the privilege for this endpoint. A refusal, not an unknown
|
||||
outcome; callers that only read can tell "no rights" from "unreachable". 401 (wrong key or secret) stays a
|
||||
@@ -241,6 +247,51 @@ def _mutation_result(data: Any, operation: str, results: set[str]) -> dict:
|
||||
return data
|
||||
|
||||
|
||||
# -- Filter rules (FilterController -> ApiMutableModelControllerBase, rules.rule) --------------------------------
|
||||
# Evidence for add/set/get/del/toggle/moveRuleBefore, checked 2026-10-22 against BOTH
|
||||
# source opnsense/core tags 26.7.7, 26.1.9 and 25.7.11, src/opnsense/mvc/app/controllers/OPNsense/:
|
||||
# Firewall/Api/FilterController.php (the *RuleAction methods, moveRuleBeforeAction, ittrRules),
|
||||
# Firewall/Api/FilterBaseController.php (applyAction),
|
||||
# Base/ApiMutableModelControllerBase.php (addBase, setBase, getBase, delBase, toggleBase)
|
||||
# live OPNsense 26.7.7 on two lab test firewalls (scratch rules whose description began with
|
||||
# `opnexus-b4-probe`, deleted again and applied afterwards; nothing was left behind).
|
||||
# Same in all three source lines unless said otherwise; the live checks cover 26.7.7 only.
|
||||
#
|
||||
# addRule POST, body {"rule": {...}}. Answer {"result": "saved", "uuid": "<new>"}; a validation failure is HTTP 200
|
||||
# {"result": "failed", "validations": {"rule.<field>": "<text>"}} (so _mutation_result raises).
|
||||
# `sequence` is a FilterSequenceField (models/OPNsense/Firewall/Filter.xml: 1..999999, default 1;
|
||||
# `sort_order` and `prio_group` are volatile, derived). Live: without `sequence` the rule gets the
|
||||
# highest sequence of ALL rules + 100 (566, 666, 766, ...), so a copy always lands at the END of its
|
||||
# prio_group; an explicit `sequence` IS honoured (sequence "3" -> sort_order 400000.0000003) and is not
|
||||
# checked for uniqueness. A restore can therefore either send the old number or call moveRuleBefore.
|
||||
# setRule POST /{uuid}, body {"rule": {...}}; answer {"result": "saved"}. It is an upsert (a well-formed unknown
|
||||
# UUID creates the rule) - never call it for a rule that may be gone.
|
||||
# delRule POST /{uuid}. {"result": "deleted"} or {"result": "not found"} (unknown, legacy and automatic UUIDs all
|
||||
# answer "not found", HTTP 200, nothing changes). 26.1 and 26.7 accept a comma-separated list; 25.7 a
|
||||
# single UUID. Consequence: "not found" proves nothing about a rule being gone - re-read it.
|
||||
# getRule GET /{uuid} -> {"rule": {...}} (every field of the rules model, select fields as
|
||||
# {"<option>": {"value": "<label>", "selected": 0|1}}, `categories` a select dict - or [] when the rule has
|
||||
# none -; no `uuid` key; read-only: `sequence`, `sort_order`, `prio_group` and the nested `audit`).
|
||||
# Live: an unknown, legacy (hex label) or automatic UUID answers HTTP 200 with a bare `[]`. 25.7 and 26.1
|
||||
# also know `?fetchmode=copy` (pre-fills `sequence`); 26.7.7 no longer has it - not used here.
|
||||
# Without a UUID it returns the template of a new rule (see get_rule_template above).
|
||||
# toggleRule POST /{uuid}/{enabled} - `enabled` is "0" or "1" and EXPLICIT; without it the rule flips (never do that).
|
||||
# Answer {"result": "Enabled"|"Disabled", "changed": true|false}. The explicit form is idempotent: repeating
|
||||
# it answers the same result with "changed": false (live: 0,0 -> true,false; 1,1 -> true,false). Any other
|
||||
# value answers {"result": "failed", "changed": false}; an unknown or legacy UUID, a GET, or a model that
|
||||
# has no such node answers {"result": "failed"} without "changed" - all HTTP 200. It only edits the
|
||||
# configuration; nothing reaches pf before apply() (the rule keeps its sort_order, searchRule shows
|
||||
# enabled "0"/"1" at once). 26.1/26.7 take a list of UUIDs (explicit value only); 25.7 one UUID.
|
||||
# moveRuleBefore POST /{selected}/{target}. {"status": "ok"} (no "result" key). Both rules must be in the same
|
||||
# `prio_group` - the sort_order prefix - which is NOT the interface: every single-interface rule of every
|
||||
# interface shares 400000 (lan and wan rules can be moved before each other), floating rules (several or
|
||||
# no interface) are 200000, interface-group rules 3000xx. Live: unknown UUID, same UUID twice and
|
||||
# different groups all answer HTTP 500 {"errorMessage": "Unexpected error, check log for details",
|
||||
# "errorTitle": "Filter"}, a GET answers 200 {"status": "error"}. A 500 is Ambiguous in _do_request, so a
|
||||
# caller that wants to tell "refused" from "unknown" must read sort_order back. The move only touches
|
||||
# `sequence` of the moved rule (and, when the gap is too small, of the rules behind the target); legacy
|
||||
# rules are not in the model and cannot be an anchor.
|
||||
# apply POST. {"status": "OK\n\n"} after `filter reload skip_alias`. FilterBaseController has no savepoint here.
|
||||
async def add_rule(host: OpnsenseHost, payload: dict) -> dict:
|
||||
return _mutation_result(await _request(host, "POST", "/firewall/filter/addRule", json={"rule": payload}), "Regel anlegen", {"saved"})
|
||||
|
||||
@@ -253,18 +304,93 @@ async def del_rule(host: OpnsenseHost, rule_uuid: str) -> dict:
|
||||
return _mutation_result(await _request(host, "POST", f"/firewall/filter/delRule/{rule_uuid}"), "Regel entfernen", {"deleted", "not found"})
|
||||
|
||||
|
||||
_RULE_REF = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
|
||||
|
||||
|
||||
def _rule_ref(rule_uuid: str) -> str:
|
||||
"""A rule reference ends up in the URL path: refuse anything that could leave it."""
|
||||
if not isinstance(rule_uuid, str) or not _RULE_REF.match(rule_uuid):
|
||||
raise OpnsenseAPIError("Ungültige Regel-UUID")
|
||||
return rule_uuid
|
||||
|
||||
|
||||
async def get_rule(host: OpnsenseHost, rule_uuid: str) -> dict:
|
||||
"""The full model record of one rule (`getRule/{uuid}`, the inner `rule` dict, select fields still in
|
||||
OPNsense's option form - reduce them with rule_write_payload()). A rule that is not in the rules model (unknown,
|
||||
legacy hex label, automatic) is answered with `[]`: raises OpnsenseNotFoundError."""
|
||||
data = await _request(host, "GET", f"/firewall/filter/getRule/{_rule_ref(rule_uuid)}")
|
||||
rule = data.get("rule") if isinstance(data, dict) else None
|
||||
if isinstance(rule, dict) and rule:
|
||||
return rule
|
||||
if data == [] or data == {}:
|
||||
raise OpnsenseNotFoundError(f"Regel {rule_uuid} gibt es auf {host.name} nicht (oder sie ist keine bearbeitbare Regel)")
|
||||
raise OpnsenseAPIError(f"Regel lesen: Keine auswertbare Antwort von {host.name}")
|
||||
|
||||
|
||||
async def toggle_rule(host: OpnsenseHost, rule_uuid: str, enabled: bool) -> dict:
|
||||
"""Sets the rule's `enabled` flag to exactly `enabled` (`toggleRule/{uuid}/1|0`; never the flipping form) and
|
||||
returns OPNsense's answer, e.g. {"result": "Disabled", "changed": False}. Repeating it is harmless. Only the
|
||||
configuration changes; call apply() afterwards. A refusal (unknown or legacy UUID) is a plain OpnsenseAPIError,
|
||||
a lost answer an OpnsenseAmbiguousError - then read the rule back before doing anything else."""
|
||||
if not isinstance(enabled, bool):
|
||||
raise TypeError("toggle_rule: 'enabled' must be a bool (an implicit flip would not be idempotent)")
|
||||
data = await _request(host, "POST", f"/firewall/filter/toggleRule/{_rule_ref(rule_uuid)}/{1 if enabled else 0}")
|
||||
return _mutation_result(data, "Regel ein-/ausschalten", {"enabled" if enabled else "disabled"})
|
||||
|
||||
|
||||
# Fields of a getRule record that a write must not send: the UUID is the rule's identity, `sequence` is assigned
|
||||
# on creation (sort_order and prio_group are derived from it), `audit` is metadata OPNsense maintains itself.
|
||||
RULE_READONLY_FIELDS = frozenset({"uuid", "sequence", "sort_order", "prio_group", "audit"})
|
||||
|
||||
|
||||
def rule_write_payload(rule: dict) -> dict[str, str]:
|
||||
"""Turns a getRule record into the flat form addRule/setRule accept: select fields become their selected
|
||||
option(s), comma-separated (`lan,wan`; none selected -> ""), the empty-list form of an empty multi-select
|
||||
becomes "", read-only fields are left out. Live round trip on 26.7.7 (getRule -> this -> addRule -> getRule)
|
||||
gave an identical record except uuid/sequence/sort_order/audit for single-interface, multi-interface (floating),
|
||||
no-interface (floating), interface-group, ICMP and any-protocol rules with categories, aliases, port ranges,
|
||||
TCP flags, tags and state limits."""
|
||||
out: dict[str, str] = {}
|
||||
for key, value in rule.items():
|
||||
if key in RULE_READONLY_FIELDS:
|
||||
continue
|
||||
if isinstance(value, dict):
|
||||
if value and all(isinstance(o, dict) and "selected" in o for o in value.values()):
|
||||
out[key] = ",".join(k for k, o in value.items() if o.get("selected") and k != "")
|
||||
# any other nested dict is metadata and is not sent
|
||||
elif isinstance(value, list):
|
||||
out[key] = ",".join(str(v) for v in value)
|
||||
else:
|
||||
out[key] = "" if value is None else str(value)
|
||||
return out
|
||||
|
||||
|
||||
async def apply(host: OpnsenseHost) -> dict:
|
||||
return _mutation_result(await _request(host, "POST", "/firewall/filter/apply"), "Änderungen anwenden", set())
|
||||
|
||||
|
||||
async def get_hasync(host: OpnsenseHost) -> dict:
|
||||
"""System > High Availability settings, read-only: `core/hasync/get` (Core/Api/HasyncController extends
|
||||
ApiMutableModelControllerBase, model Core/Hasync.xml; identical in opnsense/core 25.7.11, 26.1.9 and 26.7.7).
|
||||
Answer {"hasync": {"pfsyncinterface": {"": {"value": "Disabled", ...}, "lan": {"value": "LAN", "selected": 1},
|
||||
...}, "pfsyncpeerip": "<address or empty>", "synchronizetoip": ..., "syncitems": {...}}} - `pfsyncinterface` is an
|
||||
InterfaceField (the selected option, "" = disabled), `pfsyncpeerip` a NetworkField without mask. Live on 26.7.7
|
||||
(both nodes of the lab cluster): pfsyncinterface `lan`, a peer address, synchronizetoip empty. Used by the rule
|
||||
cleanup to keep pfsync rules out of reach (rule_analysis/protection.py); it reads nothing secret (the sync
|
||||
password field comes back, but is never stored or logged by the caller)."""
|
||||
return await _request(host, "GET", "/core/hasync/get")
|
||||
|
||||
|
||||
async def move_rule_before(host: OpnsenseHost, selected_uuid: str, target_uuid: str) -> dict:
|
||||
"""`firewall/filter/moveRuleBefore/{selected_uuid}/{target_uuid}`
|
||||
(FilterController::moveRuleBeforeAction) — repositions `selected_uuid`
|
||||
to sit directly before `target_uuid` via gap-numbered `sequence`
|
||||
floats, scoped to rules sharing the same `prio_group` (interface/type).
|
||||
Verified against OPNsense core source: no "move to top/bottom"
|
||||
sentinel exists, and both UUIDs must resolve to existing rules on the
|
||||
SAME interface/type or the controller throws. On success it returns
|
||||
floats, scoped to rules sharing the same `prio_group` (the sort_order
|
||||
prefix, see the evidence block above - not the interface). Verified against OPNsense core source and live
|
||||
(26.7.7): no "move to top/bottom"
|
||||
sentinel exists, and both UUIDs must resolve to existing rules in the
|
||||
SAME prio_group or the controller throws (HTTP 500, which this client
|
||||
reports as OpnsenseAmbiguousError). On success it returns
|
||||
`{"status": "ok"}` (confirmed against FilterController.php directly —
|
||||
unlike addRule/setRule/delRule this has no "result" key at all), so
|
||||
this is dispatched through _mutation_result with an empty `results`
|
||||
@@ -442,6 +568,11 @@ async def get_pf_rule_statistics(host: OpnsenseHost) -> dict:
|
||||
# ifname, members (comma separated identifiers), sequence; the built-in groups (enc0, openvpn,
|
||||
# wireguard) appear with their name as uuid and empty members.
|
||||
async def get_rule_stats(host: OpnsenseHost) -> dict:
|
||||
"""Per-label counters, already summed over every pf line of a rule. Used for the rule usage sampling
|
||||
(rule_hit_service). Live check 2026-10-22 on all three test firewalls (26.7, one with unmigrated legacy
|
||||
rules): HTTP 200, ``stats`` has exactly one key per label of ``pfStatistics/rules`` (30/30/25 labels,
|
||||
including the 32-hex label of the legacy user rule on the third), and each ``packets`` equals the sum over
|
||||
that label's pf lines (differences only by traffic between the two reads)."""
|
||||
return await _request(host, "GET", "/firewall/filter_util/ruleStats")
|
||||
|
||||
|
||||
|
||||
@@ -147,6 +147,9 @@ async def sweep_expired_outbound_nat_deployments(db: AsyncSession) -> list[Outbo
|
||||
|
||||
|
||||
async def undeploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule) -> None:
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, OutboundNatDeployment, OutboundNatDeployment.outbound_nat_id == rule.id)
|
||||
failures = []
|
||||
result = await db.execute(select(OutboundNatDeployment).where(OutboundNatDeployment.outbound_nat_id == rule.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -185,6 +185,9 @@ async def reassert_group_ordering(db: AsyncSession, group: PolicyHierarchyGroup)
|
||||
for h in g.hosts:
|
||||
hosts[h.id] = h
|
||||
errors: list[str] = []
|
||||
# moveRuleBefore changes rule positions: not while a rule cleanup (1.31) owns one of the firewalls.
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
await ensure_no_open_cleanup(db, [h for h in hosts.values() if h.enabled])
|
||||
for host in sorted(hosts.values(), key=lambda h: (h.name, str(h.id))):
|
||||
if not host.enabled:
|
||||
continue
|
||||
@@ -300,9 +303,13 @@ async def sweep_expired_hierarchy_deployments(db: AsyncSession) -> list[PolicyHi
|
||||
|
||||
|
||||
async def undeploy_hierarchy_rule(db: AsyncSession, rule: PolicyHierarchyRule) -> None:
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
failures = []
|
||||
result = await db.execute(select(PolicyHierarchyRuleDeployment).where(PolicyHierarchyRuleDeployment.policy_hierarchy_id == rule.id))
|
||||
for deployment in result.scalars().all():
|
||||
deployments = list(result.scalars().all())
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them.
|
||||
await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])
|
||||
for deployment in deployments:
|
||||
host = await db.get(OpnsenseHost, deployment.host_id)
|
||||
if host and deployment.opnsense_policy_hierarchy_uuid:
|
||||
try:
|
||||
|
||||
@@ -147,6 +147,9 @@ async def sweep_expired_port_forward_deployments(db: AsyncSession) -> list[PortF
|
||||
|
||||
|
||||
async def undeploy_port_forward(db: AsyncSession, rule: PortForwardRule) -> None:
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, PortForwardDeployment, PortForwardDeployment.port_forward_id == rule.id)
|
||||
failures = []
|
||||
result = await db.execute(select(PortForwardDeployment).where(PortForwardDeployment.port_forward_id == rule.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
"""Cleanup candidates of one firewall: unused, shadowed, duplicate and long-disabled rules (read-only).
|
||||
|
||||
Nothing here changes a firewall. The list says *which rules are worth a look and how sure that is*; removing
|
||||
them is a separate step (not part of this module).
|
||||
|
||||
Categories
|
||||
* ``unused`` OPNexus has watched the rule's pf counters for at least N days and saw no hit
|
||||
(services/rule_hit_service.py; N is a parameter, default 30).
|
||||
* ``shadowed`` an earlier quick rule already decides everything the rule matches (rule_analysis/coverage.py).
|
||||
* ``duplicate`` like shadowed, with the same action and the rule also covering the earlier one.
|
||||
* ``disabled_long`` disabled, and OPNexus has seen it unchanged for N days or more (rule history).
|
||||
|
||||
Confidence (never higher than what is proven; what is not proven is never "sure")
|
||||
* ``sure`` the evidence is complete: unused = the rule is loaded in pf, has no schedule, the counters were
|
||||
sampled recently and over the whole window; shadowed/duplicate = literal comparison, order from
|
||||
``sort_order``, the covering rule is loaded; disabled_long = observed unchanged for N days.
|
||||
* ``probable`` relies on a guess or on static alias content, or pf's loaded ruleset is not available, or the rule
|
||||
has a schedule (a hit might happen at another time), or (cluster) some node has no usage data for
|
||||
the rule. In a cluster "unused" is judged over all nodes (rule_hit_service.merge_cluster_hits):
|
||||
the CARP backup sees almost no traffic, so its zero alone never makes a rule unused.
|
||||
* ``unknown`` there is a hint but no support: the rule is not loaded in pf (no counters can exist), or the
|
||||
counters are stale.
|
||||
|
||||
``removable`` says whether the rule could be offered for removal at all; the reason is a code:
|
||||
``automatic`` (generated by OPNsense), ``legacy`` (not yet migrated, migrate it in OPNsense first),
|
||||
``managed`` (OPNexus manages the rule: remove it through the normal flow of the rules page).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Iterable
|
||||
|
||||
from app.services import rule_hit_service
|
||||
from app.services.rule_analysis import coverage
|
||||
from app.services.rule_analysis.context import Context
|
||||
from app.services.rule_analysis.model import SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO, NormRule
|
||||
|
||||
CATEGORIES = ("unused", "shadowed", "duplicate", "disabled_long")
|
||||
DEFAULT_UNUSED_DAYS = 30
|
||||
MIN_DAYS, MAX_DAYS = 1, 3650
|
||||
|
||||
|
||||
def removal_block(rule: NormRule, managed: dict | None) -> str | None:
|
||||
"""Why the rule could not be offered for removal (None = it could)."""
|
||||
if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_NAT_AUTO):
|
||||
return "automatic"
|
||||
if rule.legacy:
|
||||
return "legacy"
|
||||
if managed is not None:
|
||||
return "managed"
|
||||
return None
|
||||
|
||||
|
||||
def unused_confidence(rule: NormRule, hit, now: datetime) -> str:
|
||||
"""Confidence of an 'unused' verdict (the caller already knows the window passed without a hit)."""
|
||||
if rule.loaded is False:
|
||||
return "unknown" # pf did not load it (schedule, missing interface, gateway down): no counter can exist
|
||||
if hit.last_sampled_at is None or now - hit.last_sampled_at > rule_hit_service.SAMPLE_FRESH:
|
||||
return "unknown" # stale counters say nothing about the present
|
||||
if rule.loaded is None or rule.sched:
|
||||
return "probable" # pf's ruleset not available / the rule only works at certain times
|
||||
if getattr(hit, "peer_missing", False):
|
||||
return "probable" # cluster: some node has no usage data for this rule (rule_hit_service.ClusterHit)
|
||||
return "sure"
|
||||
|
||||
|
||||
def _name(rule: NormRule) -> str:
|
||||
return rule.description or rule.label or rule.uuid or (f"@{rule.pf_numbers[0]}" if rule.pf_numbers else "")
|
||||
|
||||
|
||||
def build(rules: list[NormRule], ctx: Context, *, hits: dict, states: dict, managed: dict[str, dict],
|
||||
now: datetime, unused_days: int = DEFAULT_UNUSED_DAYS, order_guessed: bool = False) -> list[dict]:
|
||||
"""Candidate rows, in evaluation order. ``hits``: RuleHit per rule label, ``states``: LiveRuleState per label,
|
||||
``managed``: {label: {"kind", "id", "name"}} of the rules OPNexus deployed to this firewall."""
|
||||
unused_days = max(MIN_DAYS, min(MAX_DAYS, int(unused_days)))
|
||||
overlaps = {id(o.rule): o for o in coverage.analyse(rules, ctx, order_guessed=order_guessed)}
|
||||
rows: list[dict] = []
|
||||
for rule in rules:
|
||||
if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
|
||||
continue
|
||||
categories: list[dict] = []
|
||||
key = rule.uuid.lower()
|
||||
hit = hits.get(key)
|
||||
if hit is not None and rule_hit_service.classify(hit, now, unused_days) == "unused":
|
||||
category = {"kind": "unused", "confidence": unused_confidence(rule, hit, now), "days": unused_days}
|
||||
if getattr(hit, "peer_missing", False):
|
||||
category["notes"] = ["CLUSTER_PEER_USAGE_UNKNOWN"]
|
||||
categories.append(category)
|
||||
overlap = overlaps.get(id(rule))
|
||||
if overlap is not None:
|
||||
categories.append({
|
||||
"kind": overlap.kind, "confidence": overlap.confidence,
|
||||
"by": {"uuid": overlap.by.uuid, "description": _name(overlap.by), "section": overlap.by.section,
|
||||
"automatic": overlap.by.automatic},
|
||||
})
|
||||
state = states.get(key)
|
||||
if (not rule.enabled and state is not None and state.present
|
||||
and now - state.changed_at >= timedelta(days=unused_days)):
|
||||
categories.append({"kind": "disabled_long", "confidence": "sure", "days": unused_days,
|
||||
"since": state.changed_at})
|
||||
if not categories:
|
||||
continue
|
||||
block = removal_block(rule, managed.get(key))
|
||||
rows.append({
|
||||
**rule.summary(), "enabled": rule.enabled, "categories": categories,
|
||||
"hit_state": rule_hit_service.classify(hit, now, unused_days) if hit is not None else None,
|
||||
"last_hit_at": hit.last_hit_at if hit is not None else None,
|
||||
"watched_since": hit.first_seen_at if hit is not None else None,
|
||||
"removable": block is None, "not_removable_reason": block,
|
||||
"managed_by": managed.get(key),
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def summarize(rows: Iterable[dict]) -> dict:
|
||||
counts = {kind: 0 for kind in CATEGORIES}
|
||||
for row in rows:
|
||||
for category in row["categories"]:
|
||||
counts[category["kind"]] += 1
|
||||
return counts
|
||||
@@ -0,0 +1,290 @@
|
||||
"""Does an earlier rule make a later one pointless? (shadowed / duplicate)
|
||||
|
||||
Pure functions on ``NormRule``; no I/O. Used by the security check (``compliance_service``, rules taken
|
||||
from config.xml) and by the cleanup candidates (rules taken from ``searchRule``).
|
||||
|
||||
pf evaluates all rules in order, the last matching rule wins unless a matching rule is ``quick``. So a later
|
||||
rule b can only be made dead by an earlier rule a when
|
||||
|
||||
* a is ``quick`` (a non-quick match is overridden by later rules and decides nothing by itself),
|
||||
* a is in force (enabled, not known to be unloaded) and has no condition we cannot compare
|
||||
(schedule, tags, ``received-on``, TCP flags, rate limits), and
|
||||
* every packet b matches is also matched by a: set coverage per field (interfaces, direction, address
|
||||
family, protocols, source/destination, ports, ICMP types).
|
||||
|
||||
The order is ``sort_order`` (floating 200000 before groups 300000+ before interfaces 400000, then the
|
||||
sequence); nothing is re-implemented here, ``NormRule.order`` already holds it.
|
||||
|
||||
Every comparison answers ``"sure"`` (literal values or identical names), ``"probable"`` (the result relies
|
||||
on the static content of aliases or on a guessed order) or ``None`` (not provable, so no claim is made).
|
||||
Whatever is not proven is never "sure": the weakest link decides.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from app.services.rule_analysis.context import Context
|
||||
from app.services.rule_analysis.model import (
|
||||
SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID, SECTION_NAT_AUTO,
|
||||
Endpoint, NormRule,
|
||||
)
|
||||
from app.services.rule_analysis.normalize import _af, _protos
|
||||
from app.services.rule_analysis.resolve import ADDRESS_TYPES, MAX_ALIAS_DEPTH, port_ranges
|
||||
|
||||
SURE, PROBABLE = "sure", "probable"
|
||||
_RANK = {None: 0, PROBABLE: 1, SURE: 2}
|
||||
|
||||
|
||||
def weakest(*values: str | None) -> str | None:
|
||||
"""The lowest confidence of the arguments (None beats everything: one unproven part spoils the claim)."""
|
||||
return min(values, key=lambda v: _RANK[v]) if values else SURE
|
||||
|
||||
|
||||
def _empty_context() -> Context:
|
||||
return Context(rows=[], pf=None, interfaces={}, groups=None, aliases={}, port_forwards=None)
|
||||
|
||||
|
||||
# ── Addresses ─────────────────────────────────────────────────────────────
|
||||
|
||||
Range = tuple[int, int, int] # (ip version, low, high)
|
||||
|
||||
|
||||
def _literal(token: str) -> Range | None:
|
||||
"""CIDR, single address or ``low-high`` range as an integer interval; None for names."""
|
||||
token = token.strip()
|
||||
try:
|
||||
if "-" in token and not token.startswith("-"):
|
||||
low, _, high = token.partition("-")
|
||||
lo, hi = ipaddress.ip_address(low.strip()), ipaddress.ip_address(high.strip())
|
||||
return (lo.version, int(lo), int(hi)) if lo.version == hi.version and lo <= hi else None
|
||||
net = ipaddress.ip_network(token, strict=False)
|
||||
except ValueError:
|
||||
return None
|
||||
return (net.version, int(net.network_address), int(net.broadcast_address))
|
||||
|
||||
|
||||
def _alias_ranges(name: str, ctx: Context, depth: int = 0, seen: tuple = ()) -> list[Range] | None:
|
||||
"""Static content of a host/network alias as intervals; None when anything about it is not static
|
||||
(tables, URLs, FQDNs, exclusions, unknown names, cycles)."""
|
||||
alias = ctx.aliases.get(name)
|
||||
if alias is None or name in seen or depth > MAX_ALIAS_DEPTH:
|
||||
return None
|
||||
if str(alias.get("type") or "").lower() not in ADDRESS_TYPES:
|
||||
return None
|
||||
if str(alias.get("enabled", "1")).strip().lower() in ("0", "false", "no", "off"):
|
||||
return None # a disabled alias is an empty table in pf: its configured content proves nothing
|
||||
result: list[Range] = []
|
||||
for entry in (e.strip() for e in re.split(r"[\n,]", str(alias.get("content") or "")) if e.strip()):
|
||||
if entry.startswith("!"):
|
||||
return None
|
||||
literal = _literal(entry)
|
||||
if literal is not None:
|
||||
result.append(literal)
|
||||
continue
|
||||
nested = _alias_ranges(entry, ctx, depth + 1, (*seen, name)) if entry in ctx.aliases else None
|
||||
if nested is None:
|
||||
return None
|
||||
result += nested
|
||||
return result or None
|
||||
|
||||
|
||||
def _ranges_of(token: str, ctx: Context) -> tuple[list[Range] | None, bool]:
|
||||
"""(intervals, came_from_alias)"""
|
||||
literal = _literal(token)
|
||||
if literal is not None:
|
||||
return [literal], False
|
||||
return _alias_ranges(token, ctx), True
|
||||
|
||||
|
||||
def _covered(inner: list[Range], outer: list[Range]) -> bool:
|
||||
return all(any(o[0] == i[0] and o[1] <= i[1] and i[2] <= o[2] for o in outer) for i in inner)
|
||||
|
||||
|
||||
def _item_covers(x: str, y: str, ctx: Context) -> str | None:
|
||||
"""Does the address item x contain everything the item y contains?"""
|
||||
x, y = x.strip(), y.strip()
|
||||
if x == y:
|
||||
return SURE
|
||||
outer, outer_alias = _ranges_of(x, ctx)
|
||||
inner, inner_alias = _ranges_of(y, ctx)
|
||||
if outer is None or inner is None or not _covered(inner, outer):
|
||||
return None
|
||||
return PROBABLE if (outer_alias or inner_alias) else SURE
|
||||
|
||||
|
||||
def endpoint_covers(a: Endpoint, b: Endpoint, ctx: Context) -> str | None:
|
||||
if a.is_any and not a.negated:
|
||||
return SURE
|
||||
if b.is_any and not b.negated:
|
||||
return None # a is narrower than "any"
|
||||
if a.negated != b.negated:
|
||||
return None
|
||||
if not a.negated: # every item of b lies inside some item of a
|
||||
parts = []
|
||||
for y in b.items:
|
||||
best = None
|
||||
for x in a.items:
|
||||
got = _item_covers(x, y, ctx)
|
||||
if _RANK[got] > _RANK[best]:
|
||||
best = got
|
||||
parts.append(best)
|
||||
return weakest(*parts)
|
||||
# "!A" covers "!B" when A lies inside B (outside of B is a subset of outside of A)
|
||||
parts = []
|
||||
for x in a.items:
|
||||
best = None
|
||||
for y in b.items:
|
||||
got = _item_covers(y, x, ctx)
|
||||
if _RANK[got] > _RANK[best]:
|
||||
best = got
|
||||
parts.append(best)
|
||||
return weakest(*parts)
|
||||
|
||||
|
||||
# ── Ports, interfaces, the rest ───────────────────────────────────────────
|
||||
|
||||
def _norm_ports(spec: str) -> str:
|
||||
return re.sub(r"\s+", "", spec or "").lower()
|
||||
|
||||
|
||||
def ports_cover(a: str, b: str, ctx: Context) -> str | None:
|
||||
if _norm_ports(a) in ("", "any"):
|
||||
return SURE
|
||||
if _norm_ports(b) in ("", "any"):
|
||||
return None
|
||||
if _norm_ports(a) == _norm_ports(b):
|
||||
return SURE
|
||||
outer, _ = port_ranges(a, ctx)
|
||||
inner, _ = port_ranges(b, ctx)
|
||||
if outer is None or inner is None:
|
||||
return None
|
||||
if not all(any(lo <= il and ih <= hi for lo, hi in outer) for il, ih in inner):
|
||||
return None
|
||||
alias_involved = any(part.strip() in ctx.aliases for spec in (a, b) for part in re.split(r"[\n,]", spec))
|
||||
return PROBABLE if alias_involved else SURE
|
||||
|
||||
|
||||
def _expand(names: set[str], ctx: Context) -> set[str]:
|
||||
result: set[str] = set()
|
||||
for name in names:
|
||||
result |= (ctx.groups[name] | {name}) if ctx.groups and name in ctx.groups else {name}
|
||||
return result
|
||||
|
||||
|
||||
def interfaces_cover(a: NormRule, b: NormRule, ctx: Context) -> str | None:
|
||||
if not a.interfaces and not a.if_not:
|
||||
return SURE # every interface
|
||||
if a.if_not or b.if_not:
|
||||
return SURE if (a.interfaces, a.if_not) == (b.interfaces, b.if_not) else None
|
||||
if not b.interfaces:
|
||||
return None # b applies everywhere, a only to some interfaces
|
||||
return SURE if _expand(b.interfaces, ctx) <= _expand(a.interfaces, ctx) else None
|
||||
|
||||
|
||||
def _conditions_ok(a: NormRule) -> bool:
|
||||
"""a must not depend on something we cannot compare: schedule, tags, received-on, TCP flags, limits."""
|
||||
return not (a.sched or a.tagged or a.received_on or a.extra_notes)
|
||||
|
||||
|
||||
def covers_fields(a: NormRule, b: NormRule, ctx: Context | None = None) -> str | None:
|
||||
"""Every packet rule b matches is already matched by rule a (order and enabled state not considered)."""
|
||||
ctx = ctx or _empty_context()
|
||||
if a.quick is not True or not _conditions_ok(a):
|
||||
return None
|
||||
if a.direction != "any" and a.direction != b.direction:
|
||||
return None
|
||||
if not b.af <= a.af:
|
||||
return None
|
||||
if a.protos is not None and (b.protos is None or not b.protos <= a.protos):
|
||||
return None
|
||||
if a.icmptypes is not None and (b.icmptypes is None or not b.icmptypes <= a.icmptypes):
|
||||
return None
|
||||
parts = [
|
||||
interfaces_cover(a, b, ctx),
|
||||
endpoint_covers(a.src, b.src, ctx), endpoint_covers(a.dst, b.dst, ctx),
|
||||
ports_cover(a.sport, b.sport, ctx), ports_cover(a.dport, b.dport, ctx),
|
||||
]
|
||||
return weakest(*parts)
|
||||
|
||||
|
||||
# ── Order-aware analysis ──────────────────────────────────────────────────
|
||||
|
||||
def _in_force(rule: NormRule) -> bool:
|
||||
return rule.enabled and rule.loaded is not False and rule.section not in (SECTION_INVALID, SECTION_NAT_AUTO)
|
||||
|
||||
|
||||
# normalize() notes after which the evaluation order is not trustworthy: findings stay at "probable" at most.
|
||||
ORDER_DOUBT = frozenset({"ORDER_GUESSED", "ORDER_MISMATCH"})
|
||||
|
||||
|
||||
def order_doubtful(notes) -> bool:
|
||||
return bool(ORDER_DOUBT & set(notes))
|
||||
|
||||
|
||||
@dataclass
|
||||
class Overlap:
|
||||
rule: NormRule # the later rule that is made pointless
|
||||
by: NormRule # the earlier rule that already decides everything the later one matches
|
||||
kind: str # "shadowed" | "duplicate"
|
||||
confidence: str # "sure" | "probable"
|
||||
|
||||
|
||||
def analyse(rules: list[NormRule], ctx: Context | None = None, *, order_guessed: bool = False) -> list[Overlap]:
|
||||
"""``rules`` in evaluation order. Each later rule is reported at most once, with the first earlier rule that
|
||||
covers it. Duplicate = same action and the later rule also covers the earlier one."""
|
||||
ctx = ctx or _empty_context()
|
||||
result: list[Overlap] = []
|
||||
for index, later in enumerate(rules):
|
||||
if not later.enabled or later.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
|
||||
continue
|
||||
if later.action is None:
|
||||
continue
|
||||
for earlier in rules[:index]:
|
||||
if not _in_force(earlier) or earlier.action is None:
|
||||
continue
|
||||
confidence = covers_fields(earlier, later, ctx)
|
||||
if confidence is None:
|
||||
continue
|
||||
# (Equal sort_order without pf line numbers always comes with loaded=None, i.e. "probable" here.)
|
||||
if order_guessed or not (earlier.order_known and later.order_known) or earlier.loaded is None:
|
||||
confidence = weakest(confidence, PROBABLE)
|
||||
duplicate = earlier.action == later.action and covers_fields(later, earlier, ctx) is not None
|
||||
if duplicate:
|
||||
confidence = weakest(confidence, covers_fields(later, earlier, ctx))
|
||||
result.append(Overlap(later, earlier, "duplicate" if duplicate else "shadowed", confidence or PROBABLE))
|
||||
break
|
||||
return result
|
||||
|
||||
|
||||
# ── Rules read from a config.xml (security check) ─────────────────────────
|
||||
|
||||
def config_order(rule: dict) -> tuple[int, int]:
|
||||
"""sort_order of a rule from config.xml in OPNsense's terms: floating 200000, interface group 300000,
|
||||
single interface 400000, each by sequence."""
|
||||
return (200000 if rule["floating"] else 300000 if rule.get("group") else 400000, rule["order"])
|
||||
|
||||
|
||||
def _config_endpoint(raw: str, negated: bool) -> Endpoint:
|
||||
raw = (raw or "").strip()
|
||||
items = [] if raw.lower() in ("", "any") else [i.strip() for i in raw.split(",") if i.strip()]
|
||||
return Endpoint(raw=raw, negated=negated, items=items)
|
||||
|
||||
|
||||
def from_config_rule(rule: dict) -> NormRule:
|
||||
"""Adapter for the rule dicts of compliance_service (MVC and legacy shapes already merged). Nothing is known
|
||||
about the loaded pf ruleset there, so ``loaded`` stays None (coverage then caps at "probable")."""
|
||||
order = config_order(rule)
|
||||
section = SECTION_FLOATING if rule["floating"] else SECTION_GROUP if rule.get("group") else SECTION_INTERFACE
|
||||
return NormRule(
|
||||
uuid="", label="", description=rule["descr"], order=order, order_known=True, section=section,
|
||||
enabled=rule["enabled"], action=rule["action"] if rule["action"] in ("pass", "block", "reject") else None,
|
||||
quick=bool(rule["quick"]), interfaces=set(rule["interfaces"]), if_not=bool(rule.get("if_not")),
|
||||
direction=rule["direction"] or "in", af=_af(rule["ipprotocol"]), protos=_protos(rule["protocol"]),
|
||||
src=_config_endpoint(rule["src"], rule["src_not"]), dst=_config_endpoint(rule["dst"], rule["dst_not"]),
|
||||
sport=rule["sport"] or "", dport=rule["dport"] or "",
|
||||
icmptypes={t.strip() for t in (rule.get("icmptype") or "").split(",") if t.strip()} or None,
|
||||
tagged=rule.get("tagged") or "", received_on=rule.get("received_on") or "", sched=rule.get("sched") or "",
|
||||
extra_notes=["RATE_LIMIT"] if rule.get("limited") else [],
|
||||
)
|
||||
@@ -117,6 +117,31 @@ def rule_fields(r: NormRule, q: Query, direction: str, iface: str, ctx: Context,
|
||||
return fields
|
||||
|
||||
|
||||
def match_rule(r: NormRule, q: Query, ctx: Context, *, iface: str | None = None,
|
||||
source: Outcome | None = None, overrides: dict[str, Outcome] | None = None) -> Outcome:
|
||||
"""Could the rule's own fields match the packet ``q`` arriving on ``iface``? Order, ``quick``, other rules and
|
||||
whether the rule is enabled or loaded play no part: this is the question "is this rule about such a packet at
|
||||
all", which the cleanup asks about the way the admin reaches the firewall (DESIGN-1.31 §3.2).
|
||||
|
||||
``iface`` defaults to ``q.iface`` and then to the interface guessed from the address; an interface that cannot
|
||||
be resolved is UNKNOWN. ``source`` replaces the source field (the caller judged it for a whole network, see
|
||||
rule_analysis/protection.py); ``overrides`` replaces further fields by name (e.g. ``interface`` for a packet
|
||||
arriving on a device that is not an assigned interface). Tags set by earlier rules are not known here, so ``tagged`` is UNKNOWN. Only
|
||||
``False`` is a proof that the rule does not match; ``None`` (unknown) must be treated like ``True``."""
|
||||
iface = iface or q.iface or guess_iface(q.src if q.direction == "in" else q.dst, ctx)
|
||||
if iface is None:
|
||||
return Outcome(None, ["IFACE_UNRESOLVED"])
|
||||
fields = rule_fields(r, q, q.direction, iface, ctx, set(), None)
|
||||
if source is not None:
|
||||
fields["source"] = source
|
||||
for name, outcome in (overrides or {}).items():
|
||||
if name in fields or name == "interface":
|
||||
fields[name] = outcome
|
||||
value = t_and(*(f.value for f in fields.values()))
|
||||
reasons = sorted({reason for f in fields.values() if f.value is None for reason in f.reasons}) if value is None else []
|
||||
return Outcome(value, reasons)
|
||||
|
||||
|
||||
def _step(r: NormRule, fields: dict[str, Outcome], result, effect: str) -> dict:
|
||||
return {
|
||||
"rule": r.summary(), "result": tri_str(result), "effect": effect,
|
||||
|
||||
@@ -13,6 +13,8 @@ NAT, 600000 = interface missing, never loaded).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from app.services.rule_analysis.context import Context
|
||||
from app.services.rule_analysis.model import (
|
||||
SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID,
|
||||
@@ -169,6 +171,11 @@ def normalize_row(row: dict, ctx: Context) -> NormRule:
|
||||
return rule
|
||||
|
||||
|
||||
# Words in an unlabeled pf line that restrict it in ways antispoof_rule() does not model (protocol, ports, flags,
|
||||
# tags, user/group, received-on, ICMP types, OS): such a line must never be used as a covering rule.
|
||||
_UNMODELLED = re.compile(r"\b(proto|port|flags|tagged|received-on|user|group|icmp6?-type|os|tag|label)\b")
|
||||
|
||||
|
||||
def antispoof_rule(line: PfLine, order: tuple[int, int]) -> NormRule:
|
||||
"""An unlabeled pf line OPNsense generated itself (antispoof / own-address block)."""
|
||||
return NormRule(
|
||||
@@ -179,6 +186,7 @@ def antispoof_rule(line: PfLine, order: tuple[int, int]) -> NormRule:
|
||||
src=Endpoint(raw="" if line.src == "any" else line.src, items=[] if line.src == "any" else [line.src]),
|
||||
dst=Endpoint(raw="" if line.dst == "any" else line.dst, items=[] if line.dst == "any" else [line.dst]),
|
||||
sport="", dport="", automatic=True, loaded=True, pf_numbers=[line.number], pf_line=line,
|
||||
extra_notes=["UNMODELLED_CONDITION"] if _UNMODELLED.search(line.text) else [],
|
||||
)
|
||||
|
||||
|
||||
@@ -188,7 +196,9 @@ def normalize(ctx: Context) -> tuple[list[NormRule], list[str]]:
|
||||
rules = [normalize_row(row, ctx) for row in ctx.rows]
|
||||
if any(not r.order_known for r in rules):
|
||||
notes.append("ORDER_GUESSED")
|
||||
rules.sort(key=lambda r: (r.order, r.uuid))
|
||||
# Equal sort_order (e.g. two rules with the same sequence): what pf actually loaded decides, the uuid only as
|
||||
# a last resort (then coverage.analyse() lowers its confidence, it cannot know the real order).
|
||||
rules.sort(key=lambda r: (r.order, min(r.pf_numbers) if r.pf_numbers else 10**9, r.uuid))
|
||||
if ctx.pf is not None:
|
||||
loaded = [r for r in rules if r.loaded and r.pf_numbers]
|
||||
firsts = [min(r.pf_numbers) for r in loaded]
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
"""May a rule be switched off without touching the way the admin reaches the firewall? (DESIGN-1.31 §3.2/§3.3)
|
||||
|
||||
Pure functions on ``NormRule`` and the analysis ``Context``; no I/O.
|
||||
|
||||
Management access (§3.2). For every configured management source (IP or network), every address of the
|
||||
firewall (the address in ``host.base_url`` plus every interface address and VIP) and every management port
|
||||
(the API port of ``base_url``, 443, 80, 22) and every interface the packet could arrive on, the rule's own
|
||||
fields are compared with that packet (``engine.match_rule``). A rule is safe only when EVERY such comparison is
|
||||
``False``. ``True`` or unknown means the rule is about management traffic: it is protected.
|
||||
|
||||
Why that is enough without simulating "the ruleset afterwards": a rule that does not match a packet takes no part
|
||||
in its evaluation at all - it neither decides it nor tags it (pf only tags on a match) - so removing it cannot
|
||||
change what happens to that packet. This holds whatever the verdict for the packet is, even when the analysis
|
||||
of the whole ruleset is only "probable". The packet's ingress interface is a routing question OPNexus cannot prove
|
||||
(asymmetric routes, VPNs, failover), so every interface is tried: that only ever protects more.
|
||||
|
||||
A management source that is a network is judged for all of its addresses at once: up to 256 addresses are
|
||||
enumerated (exact, with every resolver rule of the analysis), larger networks are compared as address ranges
|
||||
with the rule's literal/alias/interface addresses; anything that cannot be resolved is unknown = protected.
|
||||
|
||||
Infrastructure (§3.3, reason ``infrastructure``): CARP (IP protocol 112) and pfsync (240) rules, every rule that
|
||||
could match CARP advertisements (to 224.0.0.18 / ff02::12 from the interface's own networks) on an interface with
|
||||
a CARP VIP, and every rule that could match pfsync traffic on the pfsync interface (System > High Availability,
|
||||
``core/hasync/get``; when that cannot be read on a cluster node, on every interface). The second leash of the
|
||||
design ("(self) with port 22/80/443/API" when §3.2 is not False) is part of §3.2 already: such a rule is
|
||||
protected there.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from app.services.rule_analysis import coverage, resolve
|
||||
from app.services.rule_analysis.context import BUILTIN_GROUP_DEVICES, Context
|
||||
from app.services.rule_analysis.engine import Query, match_rule
|
||||
from app.services.rule_analysis.model import Endpoint, IPAddr, IPNet, NormRule, Outcome, o_or, t_not
|
||||
|
||||
MANAGEMENT_PORTS = (443, 80, 22)
|
||||
# A management network up to this size is checked address by address (exact); larger ones as address ranges.
|
||||
ENUMERATE_LIMIT = 256
|
||||
CARP_PROTO, PFSYNC_PROTO = "112", "240"
|
||||
CARP_GROUPS = {4: ipaddress.ip_address("224.0.0.18"), 6: ipaddress.ip_address("ff02::12")}
|
||||
PFSYNC_GROUP = ipaddress.ip_address("224.0.0.240")
|
||||
_LOOPBACK = {ipaddress.ip_address("127.0.0.1"), ipaddress.ip_address("::1")}
|
||||
|
||||
|
||||
def parse_sources(values) -> list[IPNet]:
|
||||
"""The configured management sources as networks (an address becomes a /32 or /128). Invalid entries are
|
||||
dropped; the caller treats an empty result as "not configured" (the cleanup is then locked)."""
|
||||
nets: list[IPNet] = []
|
||||
for value in values or []:
|
||||
try:
|
||||
net = ipaddress.ip_network(str(value).strip(), strict=False)
|
||||
except ValueError:
|
||||
continue
|
||||
if net not in nets:
|
||||
nets.append(net)
|
||||
return nets
|
||||
|
||||
|
||||
def management_targets(base_url: str, ctx: Context, extra_ports=()) -> tuple[list[IPAddr], list[int]]:
|
||||
"""Firewall addresses and ports the management traffic goes to; ``extra_ports`` = the admin's
|
||||
``cleanup_management_ports`` (an SSH or GUI port moved away from the defaults)."""
|
||||
parts = urlsplit(base_url or "")
|
||||
addrs: list[IPAddr] = []
|
||||
try:
|
||||
addrs.append(ipaddress.ip_address((parts.hostname or "").strip("[]")))
|
||||
except ValueError:
|
||||
pass # a DNS name: the interface addresses below cover the address it resolves to on the firewall
|
||||
for addr in ctx.self_addrs:
|
||||
if addr not in _LOOPBACK and addr not in addrs:
|
||||
addrs.append(addr)
|
||||
try:
|
||||
api_port = parts.port or 443
|
||||
except ValueError:
|
||||
api_port = 443
|
||||
ports = [api_port]
|
||||
for port in (*MANAGEMENT_PORTS, *extra_ports):
|
||||
if isinstance(port, int) and 1 <= port <= 65535 and port not in ports:
|
||||
ports.append(port)
|
||||
return addrs, ports
|
||||
|
||||
|
||||
def _interfaces(ctx: Context) -> list[str]:
|
||||
return sorted(i for i in ctx.interfaces if i != "lo0")
|
||||
|
||||
|
||||
# A packet can also arrive on a device that is not an assigned interface: a WireGuard instance (wg0), an OpenVPN
|
||||
# server (ovpns1), IPsec (enc0). They have no identifier, so they are not in ``ctx.interfaces``, and the built-in
|
||||
# groups that stand for them (``wireguard``, ``openvpn``, ``enc0``) list no members in ``firewall/group/searchItem``
|
||||
# (live fixture tests/fixtures/rule_analysis_fw.json: members ""). One extra, virtual ingress is checked per query:
|
||||
UNASSIGNED = "(unassigned)"
|
||||
|
||||
|
||||
def unassigned_interface(rule: NormRule, ctx: Context) -> Outcome:
|
||||
"""Could the rule's interface field match a packet arriving on an unassigned device? An assigned interface or a
|
||||
user-defined group (its members are assigned interfaces) cannot; a built-in group or an unknown name might
|
||||
(members unknown = protected); no interface (floating "any") always does."""
|
||||
if not rule.interfaces:
|
||||
return Outcome(True)
|
||||
results = []
|
||||
for name in rule.interfaces:
|
||||
if name in ctx.interfaces:
|
||||
results.append(Outcome(False))
|
||||
elif name in BUILTIN_GROUP_DEVICES or ctx.groups is None or name not in ctx.groups:
|
||||
results.append(Outcome(None, ["IFACE_MEMBERS_UNKNOWN"]))
|
||||
else:
|
||||
results.append(Outcome(False))
|
||||
result = o_or(results)
|
||||
return Outcome(t_not(result.value), result.reasons) if rule.if_not else result
|
||||
|
||||
|
||||
def _ingress(rule: NormRule, ctx: Context) -> list[tuple[str, dict | None]]:
|
||||
"""(interface, field overrides) for every ingress a management packet could take."""
|
||||
virtual = {"interface": unassigned_interface(rule, ctx)}
|
||||
if rule.received_on:
|
||||
virtual["received_on"] = Outcome(None, ["RECEIVED_ON"])
|
||||
return [(i, None) for i in _interfaces(ctx)] + [(UNASSIGNED, virtual)]
|
||||
|
||||
|
||||
def _item_ranges(token: str, ctx: Context) -> list[coverage.Range] | None:
|
||||
"""Address ranges an endpoint item stands for; None when that is not static."""
|
||||
ranges, _alias = coverage._ranges_of(token, ctx)
|
||||
if ranges is not None:
|
||||
return ranges
|
||||
name = token.strip().lower()
|
||||
if name in ("(self)", "self", "this firewall"):
|
||||
return [(a.version, int(a), int(a)) for a in ctx.self_addrs]
|
||||
if name.endswith("ip") and name[:-2] in ctx.interfaces:
|
||||
info = ctx.interfaces[name[:-2]]
|
||||
return [(a.version, int(a), int(a)) for a in (*info.addrs, *info.vips)]
|
||||
if name.startswith("__") and name.endswith("_network"):
|
||||
name = name[2:-len("_network")]
|
||||
if name in ctx.interfaces:
|
||||
return [(n.version, int(n.network_address), int(n.broadcast_address)) for n in ctx.interfaces[name].nets]
|
||||
return None
|
||||
|
||||
|
||||
def _covers_all(ranges: list[coverage.Range], version: int, low: int, high: int) -> bool:
|
||||
cursor = low
|
||||
for _v, a, b in sorted(r for r in ranges if r[0] == version):
|
||||
if a > cursor:
|
||||
return False
|
||||
cursor = max(cursor, b + 1)
|
||||
if cursor > high:
|
||||
return True
|
||||
return cursor > high
|
||||
|
||||
|
||||
def source_outcome(endpoint: Endpoint, net: IPNet, ctx: Context) -> Outcome:
|
||||
"""Does the endpoint match at least one address of ``net``? (False is a proof for every address.)"""
|
||||
if endpoint.is_any:
|
||||
return Outcome(True)
|
||||
if net.num_addresses <= ENUMERATE_LIMIT:
|
||||
return o_or([resolve.endpoint_match(endpoint, ip, ctx) for ip in net])
|
||||
ranges: list[coverage.Range] = []
|
||||
for item in endpoint.items:
|
||||
got = _item_ranges(item, ctx)
|
||||
if got is None:
|
||||
return Outcome(None, ["SOURCE_NOT_RESOLVABLE"])
|
||||
ranges += got
|
||||
low, high = int(net.network_address), int(net.broadcast_address)
|
||||
if endpoint.negated:
|
||||
return Outcome(not _covers_all(ranges, net.version, low, high))
|
||||
return Outcome(any(v == net.version and a <= high and b >= low for v, a, b in ranges))
|
||||
|
||||
|
||||
def _query(src: IPAddr, dst: IPAddr, proto: str, label: str, dport: int | None) -> Query:
|
||||
return Query(src=src, dst=dst, proto=proto, proto_label=label, dport=dport, direction="in", egress="none")
|
||||
|
||||
|
||||
def management_check(rule: NormRule, ctx: Context, sources: list[IPNet], targets: list[IPAddr],
|
||||
ports: list[int]) -> dict:
|
||||
"""{"result": "false" | "true" | "unknown", "queries": n, "reasons": [...], "match": {...} | None}.
|
||||
Only "false" makes the rule removable."""
|
||||
reasons: set[str] = set()
|
||||
queries = 0
|
||||
unknown_at = None
|
||||
interfaces = _ingress(rule, ctx)
|
||||
for net in sources:
|
||||
src = source_outcome(rule.src, net, ctx)
|
||||
if src.value is False:
|
||||
dsts = [d for d in targets if d.version == net.version]
|
||||
queries += len(dsts) * len(ports) * len(interfaces)
|
||||
continue
|
||||
for dst in (d for d in targets if d.version == net.version):
|
||||
for port in ports:
|
||||
q = _query(net.network_address, dst, "6", "tcp", port)
|
||||
for iface, overrides in interfaces:
|
||||
queries += 1
|
||||
out = match_rule(rule, q, ctx, iface=iface, source=src, overrides=overrides)
|
||||
where = {"source": str(net), "destination": str(dst), "port": port, "interface": iface}
|
||||
if out.value is True:
|
||||
return {"result": "true", "queries": queries, "reasons": [], "match": where}
|
||||
if out.value is None:
|
||||
reasons.update(out.reasons)
|
||||
unknown_at = unknown_at or where
|
||||
if not queries:
|
||||
# No address of the firewall in the family of any source: nothing could be proven.
|
||||
return {"result": "unknown", "queries": 0, "reasons": ["NO_MANAGEMENT_QUERY"], "match": None}
|
||||
if unknown_at is not None:
|
||||
return {"result": "unknown", "queries": queries, "reasons": sorted(reasons), "match": unknown_at}
|
||||
return {"result": "false", "queries": queries, "reasons": [], "match": None}
|
||||
|
||||
|
||||
def _hasync_interface(hasync: dict | None) -> str | None:
|
||||
"""The pfsync interface from ``core/hasync/get`` ("" = HA sync disabled); None when not readable."""
|
||||
node = (hasync or {}).get("hasync") if isinstance(hasync, dict) else None
|
||||
if not isinstance(node, dict) or "pfsyncinterface" not in node:
|
||||
return None
|
||||
value = node.get("pfsyncinterface")
|
||||
if isinstance(value, dict):
|
||||
return next((k for k, o in value.items() if isinstance(o, dict) and str(o.get("selected")) == "1"), "")
|
||||
return str(value or "")
|
||||
|
||||
|
||||
def _hasync_peer(hasync: dict | None) -> IPAddr | None:
|
||||
node = (hasync or {}).get("hasync") if isinstance(hasync, dict) else None
|
||||
try:
|
||||
return ipaddress.ip_address(str((node or {}).get("pfsyncpeerip") or "").strip())
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def infrastructure_reason(rule: NormRule, ctx: Context, *, hasync: dict | None, clustered: bool) -> str | None:
|
||||
"""'infrastructure' when the rule could carry CARP or pfsync traffic, else None."""
|
||||
if rule.protos is not None and rule.protos & {CARP_PROTO, PFSYNC_PROTO}:
|
||||
return "infrastructure"
|
||||
for info in ctx.interfaces.values():
|
||||
if info.identifier == "lo0" or not info.vips:
|
||||
continue
|
||||
for net in info.nets:
|
||||
group = CARP_GROUPS.get(net.version)
|
||||
out = match_rule(rule, _query(net.network_address, group, CARP_PROTO, "carp", None), ctx,
|
||||
iface=info.identifier, source=source_outcome(rule.src, net, ctx))
|
||||
if out.value is not False:
|
||||
return "infrastructure"
|
||||
pfsync_if = _hasync_interface(hasync)
|
||||
if pfsync_if == "" or (pfsync_if is None and not clustered):
|
||||
return None # HA sync off (or a single firewall whose HA settings are not readable)
|
||||
peer = _hasync_peer(hasync)
|
||||
candidates = [ctx.interfaces[pfsync_if]] if pfsync_if in ctx.interfaces else \
|
||||
[i for i in ctx.interfaces.values() if i.identifier != "lo0"] # unknown: every interface
|
||||
for info in candidates:
|
||||
for net in info.nets:
|
||||
if peer is not None and peer.version == net.version:
|
||||
src = Outcome(resolve.endpoint_match(rule.src, peer, ctx).value)
|
||||
src_ip = peer
|
||||
else:
|
||||
src, src_ip = source_outcome(rule.src, net, ctx), net.network_address
|
||||
for dst in [*(a for a in info.addrs if a.version == net.version)] + ([PFSYNC_GROUP] if net.version == 4 else []):
|
||||
out = match_rule(rule, _query(src_ip, dst, PFSYNC_PROTO, "pfsync", None), ctx,
|
||||
iface=info.identifier, source=src)
|
||||
if out.value is not False:
|
||||
return "infrastructure"
|
||||
return None
|
||||
@@ -3,11 +3,25 @@ from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.errors import ErrorCode, api_error
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
|
||||
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
|
||||
from app.models.rule import FirewallRule
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
from app.models.rule_hit import RuleHit
|
||||
from app.models.rule_history import LiveRuleState
|
||||
from app.services import rule_hit_service
|
||||
from app.services.rule_history_service import BASELINE_KEY
|
||||
from app.services.rule_analysis import cleanup, coverage
|
||||
from app.services.rule_analysis import context as ctx_mod
|
||||
from app.services.rule_analysis.engine import Query, evaluate
|
||||
from app.services.rule_analysis.normalize import normalize
|
||||
from app.services.rule_analysis.normalize import PROTO_NUMBERS, norm_proto
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -74,3 +88,75 @@ async def analyze_host(host: OpnsenseHost, q: Query, *, refresh: bool = False) -
|
||||
if any(ctx.refs.get(ip) is None for ip in (str(q.src), str(q.dst))):
|
||||
result["notes"] = sorted({*result["notes"], "ALIAS_REFERENCES_UNAVAILABLE"})
|
||||
return result
|
||||
|
||||
|
||||
async def _managed_rules(db: AsyncSession, host: OpnsenseHost) -> dict[str, dict]:
|
||||
"""{label on the firewall: what manages it} for every rule OPNexus deployed to this host."""
|
||||
managed: dict[str, dict] = {}
|
||||
for dep_uuid, rule_id, description in (await db.execute(
|
||||
select(RuleDeployment.opnsense_rule_uuid, FirewallRule.id, FirewallRule.description)
|
||||
.join(FirewallRule, FirewallRule.id == RuleDeployment.rule_id)
|
||||
.where(RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid.is_not(None)))).all():
|
||||
managed[dep_uuid.lower()] = {"kind": "rule", "id": str(rule_id), "name": description}
|
||||
for dep_uuid, rule_id, description in (await db.execute(
|
||||
select(PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid, PolicyHierarchyRule.id, PolicyHierarchyRule.description)
|
||||
.join(PolicyHierarchyRule, PolicyHierarchyRule.id == PolicyHierarchyRuleDeployment.policy_hierarchy_id)
|
||||
.where(PolicyHierarchyRuleDeployment.host_id == host.id,
|
||||
PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid.is_not(None)))).all():
|
||||
managed[dep_uuid.lower()] = {"kind": "policy_hierarchy", "id": str(rule_id), "name": description}
|
||||
return managed
|
||||
|
||||
|
||||
async def _hits(db: AsyncSession, host_id) -> dict:
|
||||
return {r.rule_uuid.lower(): r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host_id))).all()}
|
||||
|
||||
|
||||
async def cluster_hits(db: AsyncSession, host: OpnsenseHost) -> tuple[dict, dict]:
|
||||
"""(usage to judge "unused" with, the host's own hit rows). For a cluster member the usage is merged over all
|
||||
nodes (DESIGN-1.31 §3.4): "unused" only when it holds on every node."""
|
||||
own = await _hits(db, host.id)
|
||||
if not host.cluster_name:
|
||||
return own, own
|
||||
others = (await db.scalars(select(OpnsenseHost.id).where(
|
||||
OpnsenseHost.cluster_name == host.cluster_name, OpnsenseHost.id != host.id))).all()
|
||||
if not others:
|
||||
return own, own
|
||||
return rule_hit_service.merge_cluster_hits(own, [await _hits(db, other) for other in others]), own
|
||||
|
||||
|
||||
async def cleanup_view(db: AsyncSession, host: OpnsenseHost, *, unused_days: int = cleanup.DEFAULT_UNUSED_DAYS,
|
||||
refresh: bool = False, now: datetime | None = None):
|
||||
"""(context, normalized rules, candidate payload, managed rules) - the cleanup service needs all of them."""
|
||||
now = now or datetime.now(timezone.utc)
|
||||
ctx = await ctx_mod.load_context(host, use_cache=not refresh)
|
||||
rules, notes = normalize(ctx)
|
||||
usage, hits = await cluster_hits(db, host)
|
||||
states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
|
||||
managed = await _managed_rules(db, host)
|
||||
rows = cleanup.build(rules, ctx, hits=usage, states=states, managed=managed, now=now,
|
||||
unused_days=unused_days, order_guessed=coverage.order_doubtful(notes))
|
||||
return ctx, rules, _payload(host, rules, rows, hits, states, notes, ctx, unused_days, now), managed
|
||||
|
||||
|
||||
async def cleanup_candidates(db: AsyncSession, host: OpnsenseHost, *, unused_days: int = cleanup.DEFAULT_UNUSED_DAYS,
|
||||
refresh: bool = False, now: datetime | None = None) -> dict:
|
||||
"""Read-only. Raises when the firewall cannot be read (callers map that)."""
|
||||
return (await cleanup_view(db, host, unused_days=unused_days, refresh=refresh, now=now))[2]
|
||||
|
||||
|
||||
def _payload(host, rules, rows, hits, states, notes, ctx, unused_days, now) -> dict:
|
||||
all_notes = sorted({*notes, *ctx.notes})
|
||||
if host.cluster_name and any("CLUSTER_PEER_USAGE_UNKNOWN" in c.get("notes", ()) for r in rows for c in r["categories"]):
|
||||
all_notes = sorted({*all_notes, "CLUSTER_PEER_USAGE_UNKNOWN"})
|
||||
return {
|
||||
"host_id": str(host.id), "host_name": host.name, "cluster_name": host.cluster_name,
|
||||
"unused_days": max(cleanup.MIN_DAYS, min(cleanup.MAX_DAYS, int(unused_days))), "generated_at": now,
|
||||
"rules_total": sum(1 for r in rules if not r.automatic and r.section not in ("antispoof", "nat_auto", "invalid")),
|
||||
"candidates": rows, "counts": cleanup.summarize(rows), "notes": all_notes,
|
||||
"usage": {
|
||||
"tracked": len(hits),
|
||||
"collecting": sum(1 for h in hits.values() if rule_hit_service.classify(h, now, unused_days) == "collecting"),
|
||||
"watching_since": min((h.first_seen_at for h in hits.values()), default=None),
|
||||
},
|
||||
"history": {"watching_since": baseline.first_seen_at if (baseline := states.get(BASELINE_KEY)) else None},
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -155,9 +155,13 @@ async def sweep_expired_deployments(db: AsyncSession) -> list[RuleDeployment]:
|
||||
|
||||
|
||||
async def undeploy_rule(db: AsyncSession, rule: FirewallRule) -> None:
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup
|
||||
failures = []
|
||||
result = await db.execute(select(RuleDeployment).where(RuleDeployment.rule_id == rule.id))
|
||||
for deployment in result.scalars().all():
|
||||
deployments = list(result.scalars().all())
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them.
|
||||
await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])
|
||||
for deployment in deployments:
|
||||
host = await db.get(OpnsenseHost, deployment.host_id)
|
||||
if host and deployment.opnsense_rule_uuid:
|
||||
try:
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
"""Change history of the rules that are live on a firewall, also those OPNexus did not create.
|
||||
|
||||
Every ``rule_observe`` cycle reads ``searchRule`` (and the counters, for the usage sampling) of each enabled
|
||||
firewall and compares the user rules with the last observed state (``live_rule_states``). A difference becomes
|
||||
one ``rule_change_events`` row: added, changed, removed, enabled or disabled, with the rule as it was and as it
|
||||
is. The first observation of a firewall only records the starting point (no flood of "added" events).
|
||||
|
||||
Origin: a change is ``opnexus`` when the audit trail has a deployment event for that rule UUID on that host in
|
||||
the time since the previous observation (the actor is taken from it), else ``external`` (changed in the
|
||||
firewall's own GUI, by a script, ...). The audit entry is written a moment after the change reaches the firewall,
|
||||
so an event classified ``external`` is re-checked in the next cycles (``reconcile_origins``).
|
||||
|
||||
Two processes may run the loop at once (overlapping web updates, a replica): every change is claimed with an
|
||||
optimistic ``version`` on the state row and the event carries that version in a unique key, so a change is
|
||||
recorded exactly once.
|
||||
|
||||
Only the rows' own fields are kept; derived display fields (``%…``, ``alias_meta_*``, colours, priorities) are
|
||||
dropped so that a changed alias or colour does not look like a rule change.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import uuid as uuid_lib
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import delete, select, update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.audit import AuditEvent
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.policy import RuleRevision
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
from app.models.rule_history import LiveRuleState, RuleChangeEvent
|
||||
from app.services import opnsense_client as client
|
||||
from app.services import rule_hit_service
|
||||
from app.services.opnsense_client import OpnsenseAPIError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BASELINE_KEY = "__baseline__"
|
||||
RETENTION_DAYS = 365
|
||||
# Audit kinds whose events carry the UUID of a rule on a firewall in details["remote_uuid"]. "rule_cleanup": the
|
||||
# cleanup assistant (rule_cleanup_service) switching a hand-made rule off/on, deleting or restoring it.
|
||||
AUDIT_KINDS = ("rule", "policy_hierarchy", "rule_cleanup")
|
||||
# Audit statuses that mean the firewall's rule was NOT changed by that event (so it cannot explain a change).
|
||||
# "confirmed": the cleanup's confirmation only re-reads the firewall.
|
||||
AUDIT_STATUSES_NO_CHANGE = ("requested", "interrupted", "error", "confirmed")
|
||||
# The audit entry is written right after the firewall changed; allow for the observer running just before it.
|
||||
AUDIT_SLACK = timedelta(minutes=2)
|
||||
RECHECK_WINDOW = timedelta(minutes=45)
|
||||
_DROP_PREFIXES = ("%", "alias_meta", "#")
|
||||
_DROP_KEYS = {"category_colors", "categories_colors", "icons", "ref"}
|
||||
|
||||
|
||||
def clean_row(row: dict) -> dict:
|
||||
"""The part of a ``searchRule`` row that describes the rule itself."""
|
||||
return {k: v for k, v in row.items() if not k.startswith(_DROP_PREFIXES) and k not in _DROP_KEYS}
|
||||
|
||||
|
||||
def fingerprint(row: dict) -> str:
|
||||
return hashlib.sha256(json.dumps(row, sort_keys=True, separators=(",", ":"), default=str).encode()).hexdigest()
|
||||
|
||||
|
||||
def _enabled(row: dict) -> bool:
|
||||
return str(row.get("enabled", "1")).strip().lower() in ("1", "true", "yes", "on")
|
||||
|
||||
|
||||
# Fields that only say where the rule sits in the order. Another rule being inserted or a group being re-sorted
|
||||
# changes them for many rules at once; that is a "moved", not an edit of the rule.
|
||||
ORDER_FIELDS = frozenset({"sort_order", "prio_group", "sequence"})
|
||||
# Kinds after which what OPNexus learned about the rule's traffic no longer describes it.
|
||||
RESTARTS_USAGE = frozenset({"added", "changed", "enabled", "disabled"})
|
||||
|
||||
|
||||
def change_kind(before: dict | None, after: dict | None) -> str:
|
||||
if before is None:
|
||||
return "added"
|
||||
if after is None:
|
||||
return "removed"
|
||||
differing = {k for k in {*before, *after} if before.get(k) != after.get(k)}
|
||||
if differing == {"enabled"}:
|
||||
return "enabled" if _enabled(after) else "disabled"
|
||||
if differing and differing <= ORDER_FIELDS:
|
||||
return "moved"
|
||||
return "changed"
|
||||
|
||||
|
||||
def diff_fields(before: dict | None, after: dict | None, limit: int = 12) -> list[dict]:
|
||||
"""Short field-by-field difference: [{field, before, after}] (a missing side is None)."""
|
||||
before, after = before or {}, after or {}
|
||||
changes = [{"field": key, "before": before.get(key), "after": after.get(key)}
|
||||
for key in sorted({*before, *after}) if before.get(key) != after.get(key)]
|
||||
return changes[:limit]
|
||||
|
||||
|
||||
def _tracked(search: dict) -> dict[str, dict] | None:
|
||||
"""{rule label: cleaned row} of the user rules; None when the answer is not a rule list at all (an error
|
||||
page must never look like 'every rule was removed')."""
|
||||
rows = search.get("rows") if isinstance(search, dict) else None
|
||||
if not isinstance(rows, list):
|
||||
return None
|
||||
result: dict[str, dict] = {}
|
||||
for row in rows:
|
||||
if not isinstance(row, dict) or not row.get("uuid") or row.get("is_automatic") in (True, 1, "1"):
|
||||
continue
|
||||
result[str(row["uuid"]).lower()[:64]] = clean_row(row)
|
||||
return result
|
||||
|
||||
|
||||
async def _find_audit(db: AsyncSession, host_id, rule_uuid: str, start: datetime, end: datetime) -> AuditEvent | None:
|
||||
"""The newest deployment audit entry of this rule after ``start`` that no other event has claimed yet. The
|
||||
audit entry is written after the firewall changed, so it can only lie after the previous observation."""
|
||||
claimed = select(RuleChangeEvent.audit_id).where(RuleChangeEvent.audit_id.is_not(None))
|
||||
return await db.scalar(
|
||||
select(AuditEvent).where(
|
||||
AuditEvent.host_id == str(host_id), AuditEvent.kind.in_(AUDIT_KINDS),
|
||||
AuditEvent.created_at >= start, AuditEvent.created_at <= end + AUDIT_SLACK,
|
||||
AuditEvent.id.notin_(claimed),
|
||||
AuditEvent.status.notin_(AUDIT_STATUSES_NO_CHANGE),
|
||||
AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
|
||||
).order_by(AuditEvent.created_at.desc()).limit(1))
|
||||
|
||||
|
||||
async def observe_rows(db: AsyncSession, host: OpnsenseHost, search: dict, now: datetime | None = None) -> dict:
|
||||
"""Compare a ``searchRule`` answer with the stored state of the host. Returns {"baseline": bool, "events": n}."""
|
||||
now = now or datetime.now(timezone.utc)
|
||||
current = _tracked(search)
|
||||
if current is None or not rule_hit_service.looks_complete(search):
|
||||
# Not a full rule list (no automatic rules in it): never read an empty or partial answer as "all removed".
|
||||
return {"baseline": False, "events": 0, "skipped": True, "touched": []}
|
||||
states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
|
||||
baseline = states.get(BASELINE_KEY)
|
||||
since = baseline.last_seen_at if baseline else now
|
||||
events = 0
|
||||
touched: list[str] = []
|
||||
if baseline is not None:
|
||||
known = [u for u, st in states.items() if u != BASELINE_KEY and st.present]
|
||||
missing = sorted(u for u in known if u not in current)
|
||||
if known and len(missing) * 2 > len(known):
|
||||
# More than half of the known rules vanished at once: possible, but far more often a broken answer.
|
||||
# Only the same picture on two observations in a row counts.
|
||||
marker = hashlib.sha256("\n".join(missing).encode()).hexdigest()
|
||||
if (baseline.row or {}).get("pending") != marker:
|
||||
await db.execute(update(LiveRuleState).where(
|
||||
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY)
|
||||
.values(row={"pending": marker}).execution_options(synchronize_session=False))
|
||||
await db.commit()
|
||||
return {"baseline": False, "events": 0, "skipped": True, "touched": [], "unconfirmed": len(missing)}
|
||||
|
||||
async def claim(rule_uuid: str, state: LiveRuleState, new_row: dict | None) -> int | None:
|
||||
"""Move the state to its new version; None when another process already did."""
|
||||
values = {"version": LiveRuleState.version + 1, "changed_at": now, "last_seen_at": now,
|
||||
"present": new_row is not None}
|
||||
if new_row is not None:
|
||||
values.update(fingerprint=fingerprint(new_row), row=new_row)
|
||||
else:
|
||||
values.update(fingerprint="")
|
||||
return await db.scalar(update(LiveRuleState).where(
|
||||
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == rule_uuid, LiveRuleState.version == state.version,
|
||||
).values(**values).returning(LiveRuleState.version).execution_options(synchronize_session=False))
|
||||
|
||||
async def record(rule_uuid: str, seq: int, before: dict | None, after: dict | None) -> None:
|
||||
nonlocal events
|
||||
kind = change_kind(before, after)
|
||||
if kind in RESTARTS_USAGE:
|
||||
touched.append(rule_uuid)
|
||||
audit = await _find_audit(db, host.id, rule_uuid, since, now)
|
||||
stmt = pg_insert(RuleChangeEvent).values(
|
||||
id=uuid_lib.uuid4(), host_id=host.id, rule_uuid=rule_uuid, seq=seq, observed_at=now,
|
||||
kind=kind, before=before, after=after,
|
||||
origin="opnexus" if audit else "external", actor=audit.actor if audit else None,
|
||||
audit_id=audit.id if audit else None, window_start=since,
|
||||
).on_conflict_do_nothing(constraint="uq_rule_change_event")
|
||||
events += 1 if (await db.execute(stmt)).rowcount else 0
|
||||
|
||||
for rule_uuid, row in current.items():
|
||||
state = states.get(rule_uuid)
|
||||
if state is None:
|
||||
stmt = pg_insert(LiveRuleState).values(
|
||||
host_id=host.id, rule_uuid=rule_uuid, fingerprint=fingerprint(row), row=row, present=True, version=1,
|
||||
first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing()
|
||||
inserted = (await db.execute(stmt)).rowcount
|
||||
if inserted and baseline is not None:
|
||||
await record(rule_uuid, 1, None, row)
|
||||
elif not state.present:
|
||||
seq = await claim(rule_uuid, state, row)
|
||||
if seq is not None:
|
||||
await record(rule_uuid, seq, None, row)
|
||||
elif state.fingerprint != fingerprint(row):
|
||||
seq = await claim(rule_uuid, state, row)
|
||||
if seq is not None:
|
||||
await record(rule_uuid, seq, state.row, row)
|
||||
if baseline is not None:
|
||||
for rule_uuid, state in states.items():
|
||||
if rule_uuid == BASELINE_KEY or rule_uuid in current or not state.present:
|
||||
continue
|
||||
seq = await claim(rule_uuid, state, None)
|
||||
if seq is not None:
|
||||
await record(rule_uuid, seq, state.row, None)
|
||||
unchanged = [u for u in current if u in states and states[u].present]
|
||||
if unchanged:
|
||||
await db.execute(update(LiveRuleState).where(
|
||||
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid.in_(unchanged)).values(last_seen_at=now)
|
||||
.execution_options(synchronize_session=False))
|
||||
if baseline is None:
|
||||
await db.execute(pg_insert(LiveRuleState).values(
|
||||
host_id=host.id, rule_uuid=BASELINE_KEY, fingerprint="", row={}, present=False, version=1,
|
||||
first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing())
|
||||
else:
|
||||
await db.execute(update(LiveRuleState).where(
|
||||
LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY).values(last_seen_at=now, row={})
|
||||
.execution_options(synchronize_session=False))
|
||||
await db.commit()
|
||||
return {"baseline": baseline is None, "events": events, "touched": touched}
|
||||
|
||||
|
||||
async def reconcile_origins(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
|
||||
"""Recent 'external' events may only look so because the audit entry was written after the observation."""
|
||||
now = now or datetime.now(timezone.utc)
|
||||
recent = (await db.scalars(select(RuleChangeEvent).where(
|
||||
RuleChangeEvent.host_id == host.id, RuleChangeEvent.origin == "external",
|
||||
RuleChangeEvent.observed_at >= now - RECHECK_WINDOW).order_by(RuleChangeEvent.observed_at, RuleChangeEvent.seq))).all()
|
||||
fixed = 0
|
||||
for event in recent: # oldest first: an audit entry belongs to the earliest change it can explain
|
||||
start = event.window_start or (event.observed_at - RECHECK_WINDOW)
|
||||
audit = await _find_audit(db, host.id, event.rule_uuid, start, event.observed_at)
|
||||
if audit is not None:
|
||||
event.origin, event.actor, event.audit_id = "opnexus", audit.actor, audit.id
|
||||
# (autoflush makes the claim visible to the next event's query)
|
||||
fixed += 1
|
||||
if fixed:
|
||||
await db.commit()
|
||||
return fixed
|
||||
|
||||
|
||||
async def observe_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> dict:
|
||||
"""Two GETs: the rules and their counters. One read serves the usage sampling and the history."""
|
||||
# First settle the origin of earlier events (their audit entries may have arrived late), so that an entry is
|
||||
# not taken by a newer change of the same rule that happens to fall in its window.
|
||||
await reconcile_origins(db, host, now)
|
||||
search = await client.search_rules(host, all_rows=True)
|
||||
try:
|
||||
counters = await rule_hit_service.read_counters(host)
|
||||
except OpnsenseAPIError as exc:
|
||||
counters = None
|
||||
logger.info("Regelzähler von %s nicht lesbar: %s", host.name, exc)
|
||||
result = await observe_rows(db, host, search, now)
|
||||
if not result.get("skipped"): # a rejected or unconfirmed answer must not touch the usage data either
|
||||
if counters is not None:
|
||||
await rule_hit_service.apply_sample(db, host, counters, search, now)
|
||||
await rule_hit_service.restart_window(db, host.id, result["touched"], now)
|
||||
return result
|
||||
|
||||
|
||||
async def observe_all(db: AsyncSession, now: datetime | None = None) -> dict:
|
||||
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all()
|
||||
summary = {"hosts": 0, "failed": 0, "events": 0}
|
||||
for host in hosts:
|
||||
try:
|
||||
result = await observe_host(db, host, now)
|
||||
summary["hosts"] += 1
|
||||
summary["events"] += result["events"]
|
||||
except OpnsenseAPIError as exc:
|
||||
summary["failed"] += 1
|
||||
logger.warning("Regelverlauf von %s nicht lesbar: %s", host.name, exc)
|
||||
except Exception: # noqa: BLE001 - one host must never stop the cycle for the others
|
||||
summary["failed"] += 1
|
||||
await db.rollback()
|
||||
logger.exception("Regelverlauf von %s fehlgeschlagen", host.name)
|
||||
return summary
|
||||
|
||||
|
||||
async def prune(db: AsyncSession, retention_days: int = RETENTION_DAYS) -> int:
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
|
||||
events = await db.execute(delete(RuleChangeEvent).where(RuleChangeEvent.observed_at < cutoff))
|
||||
gone = await db.execute(delete(LiveRuleState).where(
|
||||
LiveRuleState.present.is_(False), LiveRuleState.rule_uuid != BASELINE_KEY, LiveRuleState.changed_at < cutoff))
|
||||
# Finished cleanup operations (with the snapshots of deleted rules) are kept as long as the history.
|
||||
from app.services.rule_cleanup_service import prune_operations
|
||||
cleanups = await prune_operations(db, cutoff)
|
||||
await db.commit()
|
||||
return (events.rowcount or 0) + (gone.rowcount or 0) + cleanups
|
||||
|
||||
|
||||
# ── Reading the history of one rule ───────────────────────────────────────
|
||||
|
||||
def _iso(value: datetime | None) -> str | None:
|
||||
return value.isoformat() if value else None
|
||||
|
||||
|
||||
async def rule_timeline(db: AsyncSession, host: OpnsenseHost, rule_uuid: str, limit: int = 200) -> list[dict]:
|
||||
"""Observed changes, saved revisions (OPNexus-managed rules) and audit entries of one rule, newest first."""
|
||||
rule_uuid = rule_uuid.lower()[:64]
|
||||
items: list[dict] = []
|
||||
for event in (await db.scalars(select(RuleChangeEvent).where(
|
||||
RuleChangeEvent.host_id == host.id, RuleChangeEvent.rule_uuid == rule_uuid)
|
||||
.order_by(RuleChangeEvent.observed_at.desc()).limit(limit))).all():
|
||||
items.append({
|
||||
"source": "observed", "at": event.observed_at, "kind": event.kind, "origin": event.origin, "actor": event.actor,
|
||||
"changes": diff_fields(event.before, event.after), "audit_id": event.audit_id,
|
||||
})
|
||||
audits = (await db.scalars(select(AuditEvent).where(
|
||||
AuditEvent.host_id == str(host.id), AuditEvent.kind.in_(AUDIT_KINDS),
|
||||
AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
|
||||
AuditEvent.status.notin_(("requested",))).order_by(AuditEvent.created_at.desc()).limit(limit))).all()
|
||||
explained = {item["audit_id"] for item in items if item["audit_id"]} # already shown as the observed change
|
||||
for audit in audits:
|
||||
if audit.id in explained:
|
||||
continue
|
||||
items.append({
|
||||
"source": "audit", "at": audit.created_at, "kind": audit.action, "status": audit.status, "actor": audit.actor,
|
||||
"origin": "opnexus", "message": audit.message, "audit_id": audit.id, "changes": [],
|
||||
})
|
||||
deployment = await db.scalar(select(RuleDeployment).where(
|
||||
RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid == rule_uuid))
|
||||
if deployment is not None:
|
||||
revisions = (await db.scalars(select(RuleRevision).where(RuleRevision.rule_id == deployment.rule_id)
|
||||
.order_by(RuleRevision.created_at.desc()).limit(limit))).all()
|
||||
ordered = sorted(revisions, key=lambda r: r.created_at)
|
||||
previous = None
|
||||
for revision in ordered:
|
||||
items.append({
|
||||
"source": "revision", "at": revision.created_at, "kind": "revision", "origin": "opnexus", "actor": None,
|
||||
"note": revision.note, "changes": diff_fields(previous, revision.snapshot) if previous is not None else [],
|
||||
"audit_id": None, "managed_rule_id": deployment.rule_id,
|
||||
})
|
||||
previous = revision.snapshot
|
||||
items.sort(key=lambda item: item["at"], reverse=True)
|
||||
return items[:limit]
|
||||
@@ -1,23 +1,31 @@
|
||||
"""Rule usage from pf's counters: which rules never match anything?
|
||||
|
||||
``sample_host`` reads the per-rule packet counters (``diagnostics/firewall/pfStatistics/rules``) and the
|
||||
configured rules (``searchRule``) and updates one row per rule: the time of the last observed hit.
|
||||
``sample_host`` reads the per-rule packet counters and the configured rules (``searchRule``) and updates one
|
||||
row per rule: the time of the last observed hit. The counters come from ``firewall/filter_util/ruleStats``
|
||||
(already summed per label by the firewall; verified live on OPNsense 26.7 on all three test firewalls, see
|
||||
the evidence block in opnsense_client.py, where its numbers equal the sum over ``pfStatistics/rules``). Older
|
||||
releases without that endpoint fall back to parsing ``diagnostics/firewall/pfStatistics/rules``.
|
||||
|
||||
* A counter that grew since the last sample is a hit "now" (at most one sampling interval late).
|
||||
* A counter that is *lower* than before means pf reloaded its ruleset (an apply or a reboot), which
|
||||
zeroes everything. Whatever it shows above zero was hit since the reload, so that also counts as a hit now;
|
||||
zero tells nothing, the old ``last_hit_at`` stays.
|
||||
zero tells nothing, the old ``last_hit_at`` stays. (Live, 2026-10-22, OPNsense 26.7.7: ``firewall/filter/apply``
|
||||
WITHOUT any pending change already zeroes every ``ruleStats`` counter - 16228 -> 0 packets on the busiest rule -
|
||||
so every apply is such a reload; writers sample right before their own apply, see rule_cleanup_service.)
|
||||
* A rule is called unused only after OPNexus has watched it for ``UNUSED_AFTER_DAYS`` without a hit.
|
||||
A new rule, or a brand-new installation, is "still collecting data" instead of a false alarm.
|
||||
|
||||
Only rules with a UUID label are tied to a configured rule (rules created in the rule editor or API; OPNsense's
|
||||
built-in automatic rules carry hash labels and are not user rules).
|
||||
A counter belongs to a configured rule by its label: the UUID of a rule created in the rule editor or API, or the
|
||||
32-hex label of a not yet migrated legacy rule (the ``searchRule`` row carries the same string as ``uuid``).
|
||||
OPNsense's built-in automatic rules also have hash labels but are flagged ``is_automatic`` there, so they are
|
||||
never tracked.
|
||||
"""
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy import delete, select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.host import OpnsenseHost
|
||||
@@ -28,7 +36,12 @@ from app.services.opnsense_client import OpnsenseAPIError
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
UNUSED_AFTER_DAYS = 30
|
||||
_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})"')
|
||||
_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}|[0-9a-fA-F]{32})"')
|
||||
# A sample counts as fresh for the "unused" confidence when it is at most this old.
|
||||
SAMPLE_FRESH = timedelta(hours=24)
|
||||
# A pf reload zeroes the counters; whatever was hit between the last sample and the reload is lost. After a gap
|
||||
# this long the loss is too big to call the rule unused on that evidence: the observation window starts over.
|
||||
SAMPLE_GAP = timedelta(hours=2)
|
||||
|
||||
|
||||
def parse_counters(stats: dict) -> dict[str, dict]:
|
||||
@@ -45,18 +58,65 @@ def parse_counters(stats: dict) -> dict[str, dict]:
|
||||
return result
|
||||
|
||||
|
||||
def parse_rule_stats(data: dict) -> dict[str, dict] | None:
|
||||
"""{label: {packets, bytes, evaluations}} from ``filter_util/ruleStats``; None when the answer is not
|
||||
that shape (older release, error page), so the caller can fall back to pfStatistics."""
|
||||
stats = (data or {}).get("stats") if isinstance(data, dict) else None
|
||||
if not isinstance(stats, dict) or (data or {}).get("status") != "ok":
|
||||
return None
|
||||
result: dict[str, dict] = {}
|
||||
for label, values in stats.items():
|
||||
if isinstance(values, dict):
|
||||
result[str(label).lower()] = {key: int(values.get(key) or 0) for key in ("packets", "bytes", "evaluations")}
|
||||
return result
|
||||
|
||||
|
||||
async def read_counters(host: OpnsenseHost) -> dict[str, dict]:
|
||||
"""Counters per rule label: ruleStats first, the pf ruleset text as fallback."""
|
||||
try:
|
||||
parsed = parse_rule_stats(await client.get_rule_stats(host))
|
||||
except OpnsenseAPIError:
|
||||
parsed = None
|
||||
if parsed is not None:
|
||||
return parsed
|
||||
return parse_counters(await client.get_pf_rule_statistics(host))
|
||||
|
||||
|
||||
def looks_complete(search) -> bool:
|
||||
"""A usable ``searchRule`` answer: a list that contains OPNsense's own automatic rules. A real firewall always
|
||||
has some, so an empty or partial answer (a firewall just restarting a service, a proxy error page) is rejected
|
||||
instead of being read as "every rule is gone"."""
|
||||
rows = search.get("rows") if isinstance(search, dict) else None
|
||||
if not isinstance(rows, list) or not rows:
|
||||
return False
|
||||
dicts = [r for r in rows if isinstance(r, dict)]
|
||||
if not any("is_automatic" in r for r in dicts):
|
||||
return bool(dicts) # firmware that does not flag automatic rules at all: a non-empty list is all we can ask for
|
||||
return any(r.get("is_automatic") in (True, 1, "1") for r in dicts)
|
||||
|
||||
|
||||
def configured_rules(search: dict) -> dict[str, dict]:
|
||||
rows = (search or {}).get("rows") or []
|
||||
return {
|
||||
str(r["uuid"]).lower(): {"description": r.get("description") or "", "enabled": r.get("enabled") in ("1", 1, True)}
|
||||
for r in rows if r.get("uuid") and not r.get("is_automatic")
|
||||
for r in rows if r.get("uuid") and r.get("is_automatic") not in (True, 1, "1")
|
||||
}
|
||||
|
||||
|
||||
async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
|
||||
"""Read the counters and the rules from the firewall and update the hit rows."""
|
||||
counters = await read_counters(host)
|
||||
return await apply_sample(db, host, counters, await client.search_rules(host, all_rows=True), now)
|
||||
|
||||
|
||||
async def apply_sample(db: AsyncSession, host: OpnsenseHost, counters: dict[str, dict], search: dict,
|
||||
now: datetime | None = None) -> int:
|
||||
"""Update the hit rows from counters (per label) and a ``searchRule`` answer that are already read
|
||||
(the observe loop shares one read between usage and change history)."""
|
||||
now = now or datetime.now(timezone.utc)
|
||||
counters = parse_counters(await client.get_pf_rule_statistics(host))
|
||||
rules = configured_rules(await client.search_rules(host, all_rows=True))
|
||||
if not looks_complete(search):
|
||||
return 0 # not a full rule list: an error or partial answer must not look like "every rule was deleted"
|
||||
rules = configured_rules(search)
|
||||
rows = {r.rule_uuid: r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host.id))).all()}
|
||||
for uuid_, meta in rules.items():
|
||||
counter = counters.get(uuid_, {"packets": 0, "bytes": 0})
|
||||
@@ -68,6 +128,10 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
|
||||
row.last_hit_at = now # already matching when first seen
|
||||
elif counter["packets"] > row.last_packets or (counter["packets"] < row.last_packets and counter["packets"] > 0):
|
||||
row.last_hit_at = now
|
||||
elif counter["packets"] < row.last_packets and now - row.last_sampled_at > SAMPLE_GAP:
|
||||
row.first_seen_at = now # reloaded after a long gap: the hits in between are unknown
|
||||
if meta["enabled"] and not row.enabled:
|
||||
row.first_seen_at = now # a rule that was off has not been "waiting for traffic" while it was off
|
||||
row.description = meta["description"][:255]
|
||||
row.enabled = meta["enabled"]
|
||||
row.last_packets, row.last_bytes, row.last_sampled_at = counter["packets"], counter["bytes"], now
|
||||
@@ -78,6 +142,18 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
|
||||
return len(rules)
|
||||
|
||||
|
||||
async def restart_window(db: AsyncSession, host_id, rule_uuids, now: datetime | None = None) -> int:
|
||||
"""The rules were edited, enabled or created: what OPNexus saw before does not describe them any more, so the
|
||||
observation window for "unused" starts over (an earlier last hit is kept)."""
|
||||
keys = [u.lower() for u in rule_uuids]
|
||||
if not keys:
|
||||
return 0
|
||||
result = await db.execute(update(RuleHit).where(RuleHit.host_id == host_id, RuleHit.rule_uuid.in_(keys))
|
||||
.values(first_seen_at=now or datetime.now(timezone.utc)).execution_options(synchronize_session=False))
|
||||
await db.commit()
|
||||
return result.rowcount or 0
|
||||
|
||||
|
||||
async def sample_all(db: AsyncSession) -> int:
|
||||
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)))).all()
|
||||
sampled = 0
|
||||
@@ -92,14 +168,49 @@ async def sample_all(db: AsyncSession) -> int:
|
||||
return sampled
|
||||
|
||||
|
||||
def classify(row: RuleHit, now: datetime) -> str:
|
||||
"""'used' | 'unused' | 'collecting' | 'disabled'"""
|
||||
@dataclass
|
||||
class ClusterHit:
|
||||
"""The usage of one rule over all nodes of a cluster (DESIGN-1.31 §3.4). Has the attributes ``classify`` and
|
||||
``cleanup.unused_confidence`` read from a ``RuleHit``. Traffic goes through the CARP master, a backup sees
|
||||
almost nothing: a rule is only unused when it is unused everywhere, so the newest hit of any node counts, the
|
||||
window starts at the latest start of any node and the counters are only as fresh as the oldest node's."""
|
||||
rule_uuid: str
|
||||
enabled: bool
|
||||
first_seen_at: datetime
|
||||
last_hit_at: datetime | None
|
||||
last_sampled_at: datetime
|
||||
# The rule is not tracked on some node (missing there, or the node was never sampled): nothing is known
|
||||
# about its traffic there, so "unused" is never better than "probable".
|
||||
peer_missing: bool = False
|
||||
|
||||
|
||||
def merge_cluster_hits(own: dict[str, RuleHit], peers: list[dict[str, RuleHit]]) -> dict[str, ClusterHit]:
|
||||
"""Per rule over the union of all nodes' rows (a rule this node has no row for is still judged, as
|
||||
``peer_missing``: what this node saw of it is unknown)."""
|
||||
merged: dict[str, ClusterHit] = {}
|
||||
for key in {*own, *(k for p in peers for k in p)}:
|
||||
rows = [own.get(key), *(p.get(key) for p in peers)]
|
||||
present = [r for r in rows if r is not None]
|
||||
hits = [r.last_hit_at for r in present if r.last_hit_at is not None]
|
||||
mine = own.get(key) or present[0]
|
||||
merged[key] = ClusterHit(
|
||||
rule_uuid=mine.rule_uuid, enabled=mine.enabled,
|
||||
first_seen_at=max(r.first_seen_at for r in present),
|
||||
last_hit_at=max(hits) if hits else None,
|
||||
last_sampled_at=min(r.last_sampled_at for r in present),
|
||||
peer_missing=len(present) < len(rows),
|
||||
)
|
||||
return merged
|
||||
|
||||
|
||||
def classify(row: RuleHit, now: datetime, days: int = UNUSED_AFTER_DAYS) -> str:
|
||||
"""'used' | 'unused' | 'collecting' | 'disabled'; ``days`` is the observation window."""
|
||||
if not row.enabled:
|
||||
return "disabled"
|
||||
if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=UNUSED_AFTER_DAYS):
|
||||
if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=days):
|
||||
return "used"
|
||||
watched = now - row.first_seen_at
|
||||
if watched < timedelta(days=UNUSED_AFTER_DAYS):
|
||||
if watched < timedelta(days=days):
|
||||
return "used" if row.last_hit_at is not None else "collecting"
|
||||
return "unused"
|
||||
|
||||
|
||||
@@ -374,6 +374,9 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: WireguardTunnel) -> None:
|
||||
dependency direction; deleting the Peer first also auto-unlinks it
|
||||
from the Server's `peers` list server-side, per delClientAction's own
|
||||
behavior — see module docstring), then reconfigures."""
|
||||
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
|
||||
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
|
||||
await ensure_no_open_cleanup_on(db, WireguardTunnelDeployment, WireguardTunnelDeployment.tunnel_id == tunnel.id)
|
||||
failures = []
|
||||
result = await db.execute(select(WireguardTunnelDeployment).where(WireguardTunnelDeployment.tunnel_id == tunnel.id))
|
||||
for deployment in result.scalars().all():
|
||||
|
||||
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
|
||||
Versioning scheme: see CHANGELOG.md ("Versionierung").
|
||||
"""
|
||||
|
||||
APP_VERSION = "1.29.0"
|
||||
APP_CODENAME = "Caracara"
|
||||
APP_VERSION = "1.31.0"
|
||||
APP_CODENAME = "Vulture"
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Änderungsverlauf der Regeln auf den Firewalls (1.30): letzter beobachteter Stand und Änderungsereignisse.
|
||||
|
||||
Revision ID: 0027
|
||||
Revises: 0026
|
||||
Create Date: 2026-10-22
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision = "0027"
|
||||
down_revision = "0026"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
UUID = postgresql.UUID(as_uuid=True)
|
||||
TS = sa.DateTime(timezone=True)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
|
||||
inspector = sa.inspect(op.get_bind())
|
||||
if not inspector.has_table("live_rule_states"):
|
||||
op.create_table(
|
||||
"live_rule_states",
|
||||
sa.Column("host_id", UUID, nullable=False),
|
||||
sa.Column("rule_uuid", sa.String(64), nullable=False),
|
||||
sa.Column("fingerprint", sa.String(64), nullable=False),
|
||||
sa.Column("row", sa.JSON(), nullable=False),
|
||||
sa.Column("present", sa.Boolean(), nullable=False),
|
||||
sa.Column("version", sa.Integer(), nullable=False),
|
||||
sa.Column("first_seen_at", TS, nullable=False),
|
||||
sa.Column("changed_at", TS, nullable=False),
|
||||
sa.Column("last_seen_at", TS, nullable=False),
|
||||
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("host_id", "rule_uuid"),
|
||||
)
|
||||
if not inspector.has_table("rule_change_events"):
|
||||
op.create_table(
|
||||
"rule_change_events",
|
||||
sa.Column("id", UUID, nullable=False),
|
||||
sa.Column("host_id", UUID, nullable=False),
|
||||
sa.Column("rule_uuid", sa.String(64), nullable=False),
|
||||
sa.Column("seq", sa.Integer(), nullable=False),
|
||||
sa.Column("observed_at", TS, nullable=False),
|
||||
sa.Column("kind", sa.String(16), nullable=False),
|
||||
sa.Column("before", sa.JSON(), nullable=True),
|
||||
sa.Column("after", sa.JSON(), nullable=True),
|
||||
sa.Column("origin", sa.String(16), nullable=False),
|
||||
sa.Column("actor", sa.String(150), nullable=True),
|
||||
sa.Column("audit_id", UUID, nullable=True),
|
||||
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
|
||||
)
|
||||
op.create_index("ix_rule_change_events_observed_at", "rule_change_events", ["observed_at"])
|
||||
op.create_index("ix_rule_change_events_rule", "rule_change_events", ["host_id", "rule_uuid", "observed_at"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute("DROP TABLE IF EXISTS rule_change_events")
|
||||
op.execute("DROP TABLE IF EXISTS live_rule_states")
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Änderungsverlauf: Fensteranfang je Ereignis für die Herkunftsbestimmung (1.30).
|
||||
|
||||
Revision ID: 0028
|
||||
Revises: 0027
|
||||
Create Date: 2026-10-22
|
||||
"""
|
||||
from alembic import op
|
||||
|
||||
revision = "0028"
|
||||
down_revision = "0027"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have the column.
|
||||
op.execute("ALTER TABLE rule_change_events ADD COLUMN IF NOT EXISTS window_start timestamptz")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute("ALTER TABLE rule_change_events DROP COLUMN IF EXISTS window_start")
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Regel-Bereinigung (1.31, B4 Teil 2): Operationen, Einträge, Einstellungen.
|
||||
|
||||
Revision ID: 0029
|
||||
Revises: 0028
|
||||
Create Date: 2026-10-22
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision = "0029"
|
||||
down_revision = "0028"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
UUID = postgresql.UUID(as_uuid=True)
|
||||
TS = sa.DateTime(timezone=True)
|
||||
|
||||
# Must equal OPEN_ITEM_STATES in app/models/rule_cleanup.py (the schema comparison of the tests notices a drift).
|
||||
_OPEN_STATES = ("queued", "disabling", "disabled", "reverting", "deleting", "restoring", "unknown", "revert_failed")
|
||||
|
||||
_SETTINGS = [
|
||||
("cleanup_max_rules", "integer NOT NULL DEFAULT 10"),
|
||||
("cleanup_delete_min_days", "integer NOT NULL DEFAULT 7"),
|
||||
("cleanup_management_sources", "json NOT NULL DEFAULT '[]'::json"),
|
||||
("cleanup_management_ports", "json NOT NULL DEFAULT '[]'::json"),
|
||||
]
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# A pre-Alembic install is bridged with create_all() first (app/migrate.py), which may already have made all of it.
|
||||
inspector = sa.inspect(op.get_bind())
|
||||
if not inspector.has_table("rule_cleanup_operations"):
|
||||
op.create_table(
|
||||
"rule_cleanup_operations",
|
||||
sa.Column("id", UUID, nullable=False),
|
||||
sa.Column("host_id", UUID, nullable=False),
|
||||
sa.Column("cluster_name", sa.String(120), nullable=True),
|
||||
sa.Column("kind", sa.String(16), nullable=False),
|
||||
sa.Column("status", sa.String(20), nullable=False),
|
||||
sa.Column("actor", sa.String(150), nullable=False),
|
||||
sa.Column("actor_id", UUID, nullable=True),
|
||||
sa.Column("created_at", TS, nullable=False),
|
||||
sa.Column("expires_at", TS, nullable=False),
|
||||
sa.Column("started_at", TS, nullable=True),
|
||||
sa.Column("heartbeat_at", TS, nullable=True),
|
||||
sa.Column("revert_deadline", TS, nullable=True),
|
||||
sa.Column("confirmed_at", TS, nullable=True),
|
||||
sa.Column("finished_at", TS, nullable=True),
|
||||
sa.Column("message", sa.String(1000), nullable=True),
|
||||
sa.Column("preview_fingerprint", sa.String(64), nullable=False),
|
||||
sa.Column("options", sa.JSON(), nullable=False),
|
||||
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index("ix_rule_cleanup_operations_host", "rule_cleanup_operations", ["host_id", "created_at"])
|
||||
op.create_index("ix_rule_cleanup_operations_status", "rule_cleanup_operations", ["status"])
|
||||
if not inspector.has_table("rule_cleanup_items"):
|
||||
op.create_table(
|
||||
"rule_cleanup_items",
|
||||
sa.Column("id", UUID, nullable=False),
|
||||
sa.Column("operation_id", UUID, nullable=False),
|
||||
sa.Column("host_id", UUID, nullable=False),
|
||||
sa.Column("rule_uuid", sa.String(64), nullable=False),
|
||||
sa.Column("seq", sa.Integer(), nullable=False),
|
||||
sa.Column("carp_role", sa.String(16), nullable=True),
|
||||
sa.Column("state", sa.String(16), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=False),
|
||||
sa.Column("categories", sa.JSON(), nullable=False),
|
||||
sa.Column("confidence", sa.String(16), nullable=True),
|
||||
sa.Column("row_before", sa.JSON(), nullable=True),
|
||||
sa.Column("fp_before", sa.String(64), nullable=True),
|
||||
sa.Column("cover_uuid", sa.String(64), nullable=True),
|
||||
sa.Column("cover_fp", sa.String(64), nullable=True),
|
||||
sa.Column("snapshot", sa.JSON(), nullable=True),
|
||||
sa.Column("anchor_next_uuid", sa.String(64), nullable=True),
|
||||
sa.Column("anchor_prev_uuid", sa.String(64), nullable=True),
|
||||
sa.Column("prio_group", sa.String(32), nullable=True),
|
||||
sa.Column("source_item_id", UUID, nullable=True),
|
||||
sa.Column("restored_uuid", sa.String(64), nullable=True),
|
||||
sa.Column("changed_at", TS, nullable=True),
|
||||
sa.Column("message", sa.String(1000), nullable=True),
|
||||
sa.ForeignKeyConstraint(["operation_id"], ["rule_cleanup_operations.id"], ondelete="CASCADE"),
|
||||
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index("ix_rule_cleanup_items_operation", "rule_cleanup_items", ["operation_id"])
|
||||
op.create_index(
|
||||
"uq_cleanup_rule_open", "rule_cleanup_items", ["host_id", "rule_uuid"], unique=True,
|
||||
postgresql_where=sa.text("state IN (" + ", ".join(f"'{s}'" for s in _OPEN_STATES) + ")"),
|
||||
)
|
||||
for name, ddl in _SETTINGS:
|
||||
op.execute(f"ALTER TABLE governance_settings ADD COLUMN IF NOT EXISTS {name} {ddl}")
|
||||
# The default only fills the existing row; the model supplies it for new ones (and the schema comparison
|
||||
# cannot compare a JSON server default).
|
||||
op.execute(f"ALTER TABLE governance_settings ALTER COLUMN {name} DROP DEFAULT")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
for name, _ddl in reversed(_SETTINGS):
|
||||
op.execute(f"ALTER TABLE governance_settings DROP COLUMN IF EXISTS {name}")
|
||||
op.execute("DROP TABLE IF EXISTS rule_cleanup_items")
|
||||
op.execute("DROP TABLE IF EXISTS rule_cleanup_operations")
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
{
|
||||
"_comment": "Anonymized from live OPNsense 26.7 test firewalls (2026-10-22): firewall/filter_util/ruleStats (3 automatic labels, 3 MVC rules, 1 unmigrated legacy rule with a 32-hex label; counters as read) and the matching searchRule row of the legacy rule. No addresses involved.",
|
||||
"ruleStats": {
|
||||
"status": "ok",
|
||||
"stats": {
|
||||
"02f4bab031b57d1e30553ce08e0ec131": {
|
||||
"pf_rules": 2,
|
||||
"evaluations": 17734,
|
||||
"packets": 250,
|
||||
"bytes": 28670,
|
||||
"states": 0
|
||||
},
|
||||
"7355b5da99f40893221b6c4140de1006": {
|
||||
"pf_rules": 6,
|
||||
"evaluations": 860,
|
||||
"packets": 0,
|
||||
"bytes": 0,
|
||||
"states": 0
|
||||
},
|
||||
"80f90f0d4c5aba0c28d1c539e1e35766": {
|
||||
"pf_rules": 12,
|
||||
"evaluations": 1219,
|
||||
"packets": 0,
|
||||
"bytes": 0,
|
||||
"states": 0
|
||||
},
|
||||
"fccf1b9d-2461-43d6-938f-2c3c39b3ddee": {
|
||||
"pf_rules": 1,
|
||||
"evaluations": 171,
|
||||
"packets": 458,
|
||||
"bytes": 35302,
|
||||
"states": 0
|
||||
},
|
||||
"920a850f-68c3-42f6-a5a4-5d06984e417b": {
|
||||
"pf_rules": 2,
|
||||
"evaluations": 8,
|
||||
"packets": 4,
|
||||
"bytes": 756,
|
||||
"states": 0
|
||||
},
|
||||
"2ec1c6db-61e1-4476-b3f6-0599a28990a2": {
|
||||
"pf_rules": 1,
|
||||
"evaluations": 9625,
|
||||
"packets": 234816,
|
||||
"bytes": 99582131,
|
||||
"states": 40
|
||||
},
|
||||
"db610dbc7c6467261cf08e8a72ade096": {
|
||||
"pf_rules": 1,
|
||||
"evaluations": 4712,
|
||||
"packets": 110719,
|
||||
"bytes": 55995167,
|
||||
"states": 24
|
||||
}
|
||||
}
|
||||
},
|
||||
"legacy_row": {
|
||||
"@attributes": {
|
||||
"uuid": "a00d53d2-8107-4d87-a71d-53ac85c1e9da"
|
||||
},
|
||||
"interface": "WAN",
|
||||
"ipprotocol": "inet",
|
||||
"statetype": "keep state",
|
||||
"direction": "in",
|
||||
"quick": "1",
|
||||
"updated": {
|
||||
"username": "[email protected]",
|
||||
"time": "1791536720.85",
|
||||
"description": "/firewall_rules_edit.php made changes"
|
||||
},
|
||||
"created": {
|
||||
"username": "[email protected]",
|
||||
"time": "1791536720.85",
|
||||
"description": "/firewall_rules_edit.php made changes"
|
||||
},
|
||||
"seq": 0,
|
||||
"#priority": 400000,
|
||||
"enabled": "1",
|
||||
"action": "pass",
|
||||
"replyto": "",
|
||||
"description": "ALLOW ANY",
|
||||
"source_net": "any",
|
||||
"source_port": "",
|
||||
"destination_net": "any",
|
||||
"destination_port": "",
|
||||
"ref": "ui/firewall/migration",
|
||||
"uuid": "db610dbc7c6467261cf08e8a72ade096",
|
||||
"source_not": "0",
|
||||
"destination_not": "0",
|
||||
"%direction": "In",
|
||||
"%action": "Pass",
|
||||
"%ipprotocol": "IPv4",
|
||||
"protocol": "",
|
||||
"%protocol": "",
|
||||
"sort_order": "400000.1000023",
|
||||
"legacy": true,
|
||||
"is_automatic": false,
|
||||
"category_colors": [],
|
||||
"alias_meta_source_net": [
|
||||
{
|
||||
"value": "any",
|
||||
"%value": "any",
|
||||
"isAlias": false,
|
||||
"description": ""
|
||||
}
|
||||
],
|
||||
"alias_meta_destination_net": [
|
||||
{
|
||||
"value": "any",
|
||||
"%value": "any",
|
||||
"isAlias": false,
|
||||
"description": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ Each mutation: {"file": ..., "old": ..., "new": ..., "tests": "pytest selection"
|
||||
The file is patched, the selected tests must FAIL, then the file is restored.
|
||||
"""
|
||||
import json
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
@@ -23,7 +24,7 @@ def main(spec_path: str) -> int:
|
||||
continue
|
||||
try:
|
||||
path.write_text(original.replace(m["old"], m["new"]))
|
||||
result = subprocess.run([sys.executable, "-m", "pytest", "-q", "-x", "-p", "no:cacheprovider", *m["tests"].split()],
|
||||
result = subprocess.run([sys.executable, "-m", "pytest", "-q", "-x", "-p", "no:cacheprovider", *shlex.split(m["tests"])],
|
||||
capture_output=True, text=True)
|
||||
finally:
|
||||
path.write_text(original)
|
||||
|
||||
@@ -0,0 +1,380 @@
|
||||
[
|
||||
{
|
||||
"name": "fingerprint not compared at deploy",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if plan_fingerprint(plan) != op.preview_fingerprint:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_deploy_refuses_a_stale_preview_without_writing"
|
||||
},
|
||||
{
|
||||
"name": "management UNKNOWN treated as FALSE (service)",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if check[\"result\"] != \"false\":",
|
||||
"new": " if check[\"result\"] == \"true\":",
|
||||
"tests": "tests/test_rule_cleanup.py::test_rules_that_must_not_be_switched_off_are_refused"
|
||||
},
|
||||
{
|
||||
"name": "management UNKNOWN treated as FALSE (protection)",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " return {\"result\": \"unknown\", \"queries\": queries, \"reasons\": sorted(reasons), \"match\": unknown_at}",
|
||||
"new": " return {\"result\": \"false\", \"queries\": queries, \"reasons\": [], \"match\": None}",
|
||||
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_rules_that_must_not_be_switched_off_are_refused"
|
||||
},
|
||||
{
|
||||
"name": "network source: ranges ignored",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " return Outcome(any(v == net.version and a <= high and b >= low for v, a, b in ranges))",
|
||||
"new": " return Outcome(False)",
|
||||
"tests": "tests/test_rule_cleanup_protection.py"
|
||||
},
|
||||
{
|
||||
"name": "CARP/pfsync protocol not excluded",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " if rule.protos is not None and rule.protos & {CARP_PROTO, PFSYNC_PROTO}:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_protection.py"
|
||||
},
|
||||
{
|
||||
"name": "pfsync interface not checked",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " if pfsync_if == \"\" or (pfsync_if is None and not clustered):",
|
||||
"new": " if True:",
|
||||
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_cluster_pfsync_interface_rules_are_infrastructure"
|
||||
},
|
||||
{
|
||||
"name": "no management sources not locked",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if not sources:\n raise api_error(409, ErrorCode.BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN)\n clustered",
|
||||
"new": " if False:\n raise api_error(409, ErrorCode.BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN)\n clustered",
|
||||
"tests": "tests/test_rule_cleanup.py::test_without_management_sources_everything_is_locked"
|
||||
},
|
||||
{
|
||||
"name": "unused sure below 30 days",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if unused_days < MIN_SURE_UNUSED_DAYS:",
|
||||
"new": " if unused_days < 7:",
|
||||
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_short_window_and_external_change_cap_at_probable"
|
||||
},
|
||||
{
|
||||
"name": "unused block rule not capped",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if rule.action in (\"block\", \"reject\"):",
|
||||
"new": " if rule.action in (\"reject\",):",
|
||||
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_probable_needs_an_explicit_acknowledgement_per_rule"
|
||||
},
|
||||
{
|
||||
"name": "external change not capped",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if external and best is not None:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_protection.py tests/test_rule_cleanup.py::test_short_window_and_external_change_cap_at_probable"
|
||||
},
|
||||
{
|
||||
"name": "probable without acknowledgement",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if level == \"probable\" and given not in acknowledge:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_probable_needs_an_explicit_acknowledgement_per_rule"
|
||||
},
|
||||
{
|
||||
"name": "unknown confidence selectable",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if level == \"unknown\":\n refuse(\"confidence\")",
|
||||
"new": " if False:\n refuse(\"confidence\")",
|
||||
"tests": "tests/test_rule_cleanup.py::test_managed_legacy_and_unknown_confidence_are_refused"
|
||||
},
|
||||
{
|
||||
"name": "cluster usage merge removed",
|
||||
"file": "app/services/rule_analysis/service.py",
|
||||
"old": " return rule_hit_service.merge_cluster_hits(own, [await _hits(db, other) for other in others]), own",
|
||||
"new": " return own, own",
|
||||
"tests": "tests/test_rule_cleanup.py::test_cluster_usage_is_judged_over_all_nodes"
|
||||
},
|
||||
{
|
||||
"name": "write order master first",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " return sorted(nodes, key=lambda n: (n.role == \"MASTER\", n.host.name))",
|
||||
"new": " return sorted(nodes, key=lambda n: (n.role != \"MASTER\", n.host.name))",
|
||||
"tests": "tests/test_rule_cleanup.py::test_cluster_writes_backup_first_and_reverts_master_first"
|
||||
},
|
||||
{
|
||||
"name": "revert order backup first",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " for item in sorted(items, key=lambda i: (i.carp_role != \"MASTER\", str(i.host_id))):",
|
||||
"new": " for item in sorted(items, key=lambda i: (i.carp_role == \"MASTER\", str(i.host_id))):",
|
||||
"tests": "tests/test_rule_cleanup.py::test_cluster_writes_backup_first_and_reverts_master_first"
|
||||
},
|
||||
{
|
||||
"name": "no apply after a partial failure",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if any(i.state in (\"disabled\", \"unknown\") for i in node_items):\n # Also after an error",
|
||||
"new": " if not stop and any(i.state in (\"disabled\", \"unknown\") for i in node_items):\n # Also after an error",
|
||||
"tests": "tests/test_rule_cleanup.py::test_refused_toggle_stops_writing_but_applies_what_is_switched"
|
||||
},
|
||||
{
|
||||
"name": "write continues after a refusal",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " item.state, item.message = \"failed\", str(exc)[:1000]\n _audit(db, op, item, node.host, action=op.kind, status=\"error\", message=item.message)\n stop = True",
|
||||
"new": " item.state, item.message = \"failed\", str(exc)[:1000]\n _audit(db, op, item, node.host, action=op.kind, status=\"error\", message=item.message)",
|
||||
"tests": "tests/test_rule_cleanup.py::test_refused_toggle_stops_writing_but_applies_what_is_switched"
|
||||
},
|
||||
{
|
||||
"name": "unclear toggle counted as done",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " item.state = \"disabled\" if seen is target else (\"failed\" if seen is not None else \"unknown\")",
|
||||
"new": " item.state = \"disabled\"",
|
||||
"tests": "tests/test_rule_cleanup.py::test_unclear_toggle_is_settled_by_reading"
|
||||
},
|
||||
{
|
||||
"name": "confirm with unreachable firewall",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " except OpnsenseAPIError as exc:\n raise api_error(502, ErrorCode.BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR, host=member.name) from exc\n if not rule_hit_service.looks_complete(search):",
|
||||
"new": " except OpnsenseAPIError as exc:\n continue\n if not rule_hit_service.looks_complete(search):",
|
||||
"tests": "tests/test_rule_cleanup.py::test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different"
|
||||
},
|
||||
{
|
||||
"name": "confirm ignores re-enabled rule",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if row is None or _truthy(row.get(\"enabled\", \"1\")) is not target \\\n or _content_fp",
|
||||
"new": " if row is None \\\n or _content_fp",
|
||||
"tests": "tests/test_rule_cleanup.py::test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different"
|
||||
},
|
||||
{
|
||||
"name": "confirm after the deadline",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " RuleCleanupOperation.revert_deadline > now).values(\n status=\"confirmed\"",
|
||||
"new": " RuleCleanupOperation.revert_deadline > now - timedelta(days=1)).values(\n status=\"confirmed\"",
|
||||
"tests": "tests/test_rule_cleanup.py::test_deadline_passing_while_confirm_reads_the_firewall_refuses tests/test_rule_cleanup.py::test_confirm_and_sweeper_at_the_deadline_decide_once"
|
||||
},
|
||||
{
|
||||
"name": "sweeper does not revert expired",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " or_(and_(RuleCleanupOperation.status.in_(SWEEP_STATUSES), RuleCleanupOperation.revert_deadline <= now),",
|
||||
"new": " or_(and_(RuleCleanupOperation.status.in_(SWEEP_STATUSES), RuleCleanupOperation.revert_deadline <= now - timedelta(days=1)),",
|
||||
"tests": "tests/test_rule_cleanup.py::test_confirm_after_the_deadline_is_refused_and_the_sweeper_reverts tests/test_rule_cleanup.py::test_failed_apply_keeps_the_deadline_and_the_sweeper_reverts"
|
||||
},
|
||||
{
|
||||
"name": "sweeper retries revert_failed",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": "SWEEP_STATUSES = (\"awaiting_confirm\", \"partial\", \"apply_failed\")",
|
||||
"new": "SWEEP_STATUSES = (\"awaiting_confirm\", \"partial\", \"apply_failed\", \"revert_failed\")",
|
||||
"tests": "tests/test_rule_cleanup.py::test_revert_now_and_failed_revert_is_not_retried_automatically"
|
||||
},
|
||||
{
|
||||
"name": "heartbeat not checked",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if not _stale(op, len(items), nodes, now):\n # alive",
|
||||
"new": " if False:\n # alive",
|
||||
"tests": "tests/test_rule_cleanup.py::test_sweeper_skips_a_live_run_and_takes_over_an_abandoned_one"
|
||||
},
|
||||
{
|
||||
"name": "delete before the waiting time",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if now < due:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change"
|
||||
},
|
||||
{
|
||||
"name": "delete of a changed rule",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if _content_fp(row, False) != _content_fp(item.row_before or {}, False):",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change"
|
||||
},
|
||||
{
|
||||
"name": "snapshot not committed before delRule",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " item.state, item.changed_at = \"deleting\", _now()\n await _commit(db) # heartbeat + snapshot + intent, durable before delRule\n",
|
||||
"new": " item.state, item.changed_at = \"deleting\", _now()\n",
|
||||
"tests": "tests/test_rule_cleanup.py::test_delete_commits_the_snapshot_before_delrule"
|
||||
},
|
||||
{
|
||||
"name": "unclear delete counted as deleted",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " return \"failed\" # still there",
|
||||
"new": " return \"deleted\"",
|
||||
"tests": "tests/test_rule_cleanup.py::test_unclear_delete_is_settled_with_getrule"
|
||||
},
|
||||
{
|
||||
"name": "restore anchor ignored",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " await client.move_rule_before(member, new_uuid, anchor)",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup.py::test_restore_adds_disabled_at_the_anchor_with_a_new_uuid"
|
||||
},
|
||||
{
|
||||
"name": "restore position not verified",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " return i > 0 and position[i - 1] == new_uuid.lower()",
|
||||
"new": " return True",
|
||||
"tests": "tests/test_rule_cleanup.py::test_restore_with_a_refused_move_is_verified_by_reading_and_only_warns"
|
||||
},
|
||||
{
|
||||
"name": "R1 unassigned ingress not checked",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " return [(i, None) for i in _interfaces(ctx)] + [(UNASSIGNED, virtual)]",
|
||||
"new": " return [(i, None) for i in _interfaces(ctx)]",
|
||||
"tests": "tests/test_rule_cleanup_protection.py"
|
||||
},
|
||||
{
|
||||
"name": "R1 builtin group members treated as known",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " elif name in BUILTIN_GROUP_DEVICES or ctx.groups is None or name not in ctx.groups:",
|
||||
"new": " elif ctx.groups is None or name not in ctx.groups:",
|
||||
"tests": "tests/test_rule_cleanup_protection.py"
|
||||
},
|
||||
{
|
||||
"name": "R2 sweeper takes over a live run past its deadline",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if not _stale(op, len(items), nodes, now):",
|
||||
"new": " if not (op.revert_deadline is not None and op.revert_deadline <= now) and not _stale(op, len(items), nodes, now):",
|
||||
"tests": "tests/test_rule_cleanup.py::test_sweeper_leaves_a_live_run_alone_even_past_its_deadline"
|
||||
},
|
||||
{
|
||||
"name": "R2 heartbeat before a write not conditional",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " RuleCleanupOperation.id == op.id, RuleCleanupOperation.status == expect).values(heartbeat_at=now)",
|
||||
"new": " RuleCleanupOperation.id == op.id).values(heartbeat_at=now)",
|
||||
"tests": "tests/test_rule_cleanup.py::test_a_run_taken_over_by_another_backend_writes_nothing_more"
|
||||
},
|
||||
{
|
||||
"name": "R3 extra management ports ignored",
|
||||
"file": "app/services/rule_analysis/protection.py",
|
||||
"old": " for port in (*MANAGEMENT_PORTS, *extra_ports):",
|
||||
"new": " for port in MANAGEMENT_PORTS:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_extra_management_ports_protect_rules_on_them"
|
||||
},
|
||||
{
|
||||
"name": "R3 management ports shown to non-admins",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " cleanup_management_ports=ports if user.role == \"admin\" else [],",
|
||||
"new": " cleanup_management_ports=ports,",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_management_ports_are_validated_normalised_audited_and_hidden"
|
||||
},
|
||||
{
|
||||
"name": "R3 port range not validated",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if isinstance(raw, bool) or not text.isdigit() or not 1 <= int(text) <= 65535:",
|
||||
"new": " if not text.isdigit():",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_management_ports_are_validated_normalised_audited_and_hidden"
|
||||
},
|
||||
{
|
||||
"name": "R4 any host may act on any operation",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if op.host_id != host.id and not (cluster and host.cluster_name and op.host_id in {h.id for h in await _fleet(db, host)}):",
|
||||
"new": " if op.host_id != host.id and not cluster:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_cluster_peer_may_confirm_and_read_but_strangers_may_not"
|
||||
},
|
||||
{
|
||||
"name": "R4 peers refused",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if op.host_id != host.id and not (cluster and host.cluster_name and op.host_id in {h.id for h in await _fleet(db, host)}):",
|
||||
"new": " if op.host_id != host.id:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_cluster_peer_may_confirm_and_read_but_strangers_may_not"
|
||||
},
|
||||
{
|
||||
"name": "R5 half-deleted node not skipped",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if clustered and kind == \"delete\" and await _already_deleted(db, node.host.id, u):",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
|
||||
},
|
||||
{
|
||||
"name": "R5 failed delete counts as basis",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if not (o.kind == \"delete\" and i.state in (\"failed\", \"skipped\"))]",
|
||||
"new": " ]",
|
||||
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
|
||||
},
|
||||
{
|
||||
"name": "R5 nothing left still planned",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if not any(i[\"given\"] == given for n in plan_nodes for i in n[\"items\"]):",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_finished"
|
||||
},
|
||||
{
|
||||
"name": "R5 restore needs all siblings deleted",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if s.state == \"deleted\" and not s.restored_uuid]",
|
||||
"new": " ]\n if any(s.state != \"deleted\" or s.restored_uuid for s in siblings):\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=\"partial\")",
|
||||
"tests": "tests/test_rule_cleanup.py::test_half_done_cluster_delete_can_be_restored_where_it_was_deleted"
|
||||
},
|
||||
{
|
||||
"name": "R7 enabling a block rule without acknowledgement",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] != \"false\":",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless"
|
||||
},
|
||||
{
|
||||
"name": "R7 unknown protection counted as harmless",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] != \"false\":",
|
||||
"new": " if rule.action != \"pass\" and item[\"protection\"][\"result\"] == \"true\":",
|
||||
"tests": "tests/test_rule_cleanup.py::test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless"
|
||||
},
|
||||
{
|
||||
"name": "R8 router does not redact for non-admins",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if not _full_view(request):\n operations = [service.redact(op) for op in operations]",
|
||||
"new": " if False:\n operations = [service.redact(op) for op in operations]",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R8 preview answer not redacted",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " return result if _full_view(request) else service.redact(result)",
|
||||
"new": " return result",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R8 audit admin_only not stripped",
|
||||
"file": "app/routers/audit.py",
|
||||
"old": " if not full_view(request):",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R8 read token of an admin sees everything",
|
||||
"file": "app/routers/audit.py",
|
||||
"old": " return getattr(user, \"role\", None) == \"admin\" and (token is None or token.scope == \"admin\")",
|
||||
"new": " return getattr(user, \"role\", None) == \"admin\"",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R8 base_url kept in plan",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " out[\"nodes\"] = [{**{k: v for k, v in n.items() if k != \"base_url\"},",
|
||||
"new": " out[\"nodes\"] = [{**{k: v for k, v in n.items()},",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R8 protection match kept",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " return {k: v for k, v in check.items() if k != \"match\"}",
|
||||
"new": " return check",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_sources_and_firewall_addresses_are_for_admins_only tests/test_rule_cleanup_protection.py::test_redacted_plan_names_no_source_port_or_firewall_address"
|
||||
},
|
||||
{
|
||||
"name": "R10 stored restored_uuid ignored",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if item.restored_uuid:\n try:\n await client.get_rule(member, item.restored_uuid)",
|
||||
"new": " if False:\n try:\n await client.get_rule(member, item.restored_uuid)",
|
||||
"tests": "tests/test_rule_cleanup.py::test_unclear_restore_uses_the_stored_uuid_before_searching_by_content"
|
||||
},
|
||||
{
|
||||
"name": "R10 twins resolved to the first",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if len(found) == 1:\n return \"found\", found[0]",
|
||||
"new": " if found:\n return \"found\", found[0]",
|
||||
"tests": "tests/test_rule_cleanup.py::test_unclear_restore_with_twins_and_no_stored_uuid_stays_unknown"
|
||||
},
|
||||
{
|
||||
"name": "R11 any acknowledgement text version accepted",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " if str(ack_text_version) != ACK_TEXT_VERSION:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup.py::test_an_outdated_acknowledgement_text_is_refused"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,373 @@
|
||||
[
|
||||
{
|
||||
"name": "middleware: confirm not session-only",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "|^/api/hosts/[^/]+/rule-cleanup/[^/]+/confirm/?$\")",
|
||||
"new": "\")",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: retry not admin-only",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "(deploy|retry)|items",
|
||||
"new": "(deploy)|items",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: restore not admin-only",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "|items/[^/]+/restore)/?$\")",
|
||||
"new": ")/?$\")",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: deploy-scope token passes admin-only cleanup routes",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "if scope == \"deploy\" and (_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path)):",
|
||||
"new": "if scope == \"deploy\" and (_ADMIN_ONLY_WRITES.match(path)):",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: operator passes admin-only cleanup routes",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": " _ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path))",
|
||||
"new": " _ADMIN_ONLY_WRITES.match(path))",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: revert not a rollout (operators_may_deploy ignored)",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "r\"|rule-cleanup/[^/]+/(revert|retry)\"",
|
||||
"new": "r\"|rule-cleanup/[^/]+/(retry)\"",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "middleware: retry not a rollout",
|
||||
"file": "app/services/api_token_service.py",
|
||||
"old": "r\"|rule-cleanup/[^/]+/(revert|retry)\"",
|
||||
"new": "r\"|rule-cleanup/[^/]+/(revert)\"",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: deploy only needs a writer",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.deploy(",
|
||||
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.deploy(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: retry only needs a writer",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.retry(",
|
||||
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.retry(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: restore only needs a writer",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " _admin_only(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.restore_item(",
|
||||
"new": " _writer(user, request)\n host = await _host(db, host_id)\n return await _one(db, await service.restore_item(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: confirm only needs a writer (tokens may confirm)",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " _session_only(user, request)\n",
|
||||
"new": " _writer(user, request)\n",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: revert needs no deployer",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " await _deployer(user, request, db)\n",
|
||||
"new": " _writer(user, request)\n",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: admin role not checked",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if user.role != \"admin\":\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n token = _token(request)",
|
||||
"new": " if False:\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n token = _token(request)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: admin token scope not checked",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if token is not None and token.scope != \"admin\":",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: session-only token check removed",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if token is not None:\n raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)",
|
||||
"new": " if False:\n raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: operator revert ignores operators_may_deploy",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": "if user.role == \"operator\" and (await governance.get_settings(db)).operators_may_deploy:",
|
||||
"new": "if user.role == \"operator\":",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: read-only account may write",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if user.role == \"read_only\":\n raise api_error(403, ErrorCode.READ_ONLY_ACCESS)",
|
||||
"new": " if False:\n raise api_error(403, ErrorCode.READ_ONLY_ACCESS)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: read-scope token may write",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " if token is not None and token.scope == \"read\":",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: preview needs no writer",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": " user = await resolve_user(request, db)\n _writer(user, request)\n host = await _host(db, host_id)\n kind, uuids",
|
||||
"new": " user = await resolve_user(request, db)\n host = await _host(db, host_id)\n kind, uuids",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'every_route_for_every_caller or pattern_table or governance_settings_are_admin'"
|
||||
},
|
||||
{
|
||||
"name": "router: actor is not the account name",
|
||||
"file": "app/routers/rule_cleanup.py",
|
||||
"old": "actor=user.username, actor_id=user.id, kind=kind",
|
||||
"new": "actor='x', actor_id=user.id, kind=kind",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'preview_belongs_to_its_actor or shapes_and_audit'"
|
||||
},
|
||||
{
|
||||
"name": "settings: any role may write",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if user.role != \"admin\":\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n row = await get_settings(db)\n changes",
|
||||
"new": " if False:\n raise api_error(403, ErrorCode.ADMIN_REQUIRED)\n row = await get_settings(db)\n changes",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k only_an_admin_writes"
|
||||
},
|
||||
{
|
||||
"name": "settings: whole address space accepted",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if net.prefixlen == 0:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: host names accepted",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " raise _bad(\"cleanup_management_sources\", \"not_an_address\") from None",
|
||||
"new": " continue",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: unlimited list",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if len(values) > MAX_MANAGEMENT_SOURCES:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: sources not canonical",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " entry = str(net.network_address) if net.prefixlen == net.max_prefixlen else str(net)",
|
||||
"new": " entry = text",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: sources shown to everybody",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": "cleanup_management_sources=sources if user.role == \"admin\" else [],",
|
||||
"new": "cleanup_management_sources=sources,",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: change not audited",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if changes:\n db.add(audit_service.make_event(",
|
||||
"new": " if False:\n db.add(audit_service.make_event(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: max rules unbounded",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if not 1 <= payload.cleanup_max_rules <= MAX_CLEANUP_RULES:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "settings: min days unbounded",
|
||||
"file": "app/routers/governance.py",
|
||||
"old": " if not 1 <= payload.cleanup_delete_min_days <= MAX_DELETE_MIN_DAYS:",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'cleanup_settings or management_sources or only_see_whether'"
|
||||
},
|
||||
{
|
||||
"name": "lock: ensure_no_maintenance ignores a cleanup",
|
||||
"file": "app/services/host_maintenance_service.py",
|
||||
"old": " if cleanup and ids:\n await ensure_no_open_cleanup(db, hosts)",
|
||||
"new": " if False:\n await ensure_no_open_cleanup(db, hosts)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: ensure_no_open_cleanup never raises",
|
||||
"file": "app/services/host_maintenance_service.py",
|
||||
"old": " if ids and await has_open_cleanup(db, ids):\n raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)",
|
||||
"new": " if False:\n raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: maintenance inspection ignores a cleanup",
|
||||
"file": "app/services/host_maintenance_service.py",
|
||||
"old": " if await has_open_cleanup(db, fleet_ids):\n blocked.append(",
|
||||
"new": " if False:\n blocked.append(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: host delete ignores a cleanup",
|
||||
"file": "app/routers/hosts.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [host])\n # Both deployment tables FK",
|
||||
"new": " # Both deployment tables FK",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: host re-addressing ignores a cleanup",
|
||||
"file": "app/routers/hosts.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [host])\n\n if payload.name is not None:",
|
||||
"new": " pass\n\n if payload.name is not None:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: rule deletion ignores a cleanup",
|
||||
"file": "app/services/rule_deploy_service.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: hierarchy rule deletion ignores a cleanup",
|
||||
"file": "app/services/policy_hierarchy_deploy_service.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [h for h in [await db.get(OpnsenseHost, d.host_id) for d in deployments] if h])",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: hierarchy reordering ignores a cleanup",
|
||||
"file": "app/services/policy_hierarchy_deploy_service.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [h for h in hosts.values() if h.enabled])",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: removing a rule target ignores a cleanup",
|
||||
"file": "app/routers/rules.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: removing a hierarchy target ignores a cleanup",
|
||||
"file": "app/routers/policy_hierarchy.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: adoption ignores a cleanup",
|
||||
"file": "app/routers/rules.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [host])\n try:\n rule = await rule_adopt_service.adopt(",
|
||||
"new": " try:\n rule = await rule_adopt_service.adopt(",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or covers_the_cluster or host_delete_and_readdressing or rule_deletion_and_removal or hierarchy_rule_deletion or former_hierarchy or adopting_a_rule or confirmed_cleanup_blocks_nothing'"
|
||||
},
|
||||
{
|
||||
"name": "lock: cleanup service trips over itself",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": "await ensure_no_maintenance(db, [host], cleanup=False)",
|
||||
"new": "await ensure_no_maintenance(db, [host])",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'open_cleanup_blocks_other or preview_input_forms or shapes_and_audit or other_open_work'"
|
||||
},
|
||||
{
|
||||
"name": "service: lost revert claim hits an expired row",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": "\n await db.refresh(op) # _revert rolled back when it lost the claim: the status is expired\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=op.status)\n await db.commit()",
|
||||
"new": "\n raise api_error(409, ErrorCode.BEREINIGUNG_AKTION_NICHT_MOEGLICH, status=op.status)\n await db.commit()",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k revert_is_for_admins"
|
||||
},
|
||||
{
|
||||
"name": "service: lost retry claim hits an expired row",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": "\n await db.refresh(op) # _revert rolled back when it lost the claim: the status is expired",
|
||||
"new": "",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k retry_after_a_failed"
|
||||
},
|
||||
{
|
||||
"name": "notify: failed revert not reported",
|
||||
"file": "app/services/notification_checks.py",
|
||||
"old": " if op.status == \"revert_failed\":",
|
||||
"new": " if False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
|
||||
},
|
||||
{
|
||||
"name": "notify: partial run not reported",
|
||||
"file": "app/services/notification_checks.py",
|
||||
"old": " elif op.status in (\"partial\", \"apply_failed\"):",
|
||||
"new": " elif False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
|
||||
},
|
||||
{
|
||||
"name": "notify: unclear delete not reported",
|
||||
"file": "app/services/notification_checks.py",
|
||||
"old": " elif op.kind in (\"delete\", \"restore\"):",
|
||||
"new": " elif False:",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
|
||||
},
|
||||
{
|
||||
"name": "notify: every item of a failed revert is reported",
|
||||
"file": "app/services/notification_checks.py",
|
||||
"old": " stuck = [i for i in items if i.state in _CLEANUP_STILL_CHANGED]",
|
||||
"new": " stuck = list(items)",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
|
||||
},
|
||||
{
|
||||
"name": "notify: cleanup problems not part of the revert alert",
|
||||
"file": "app/services/notification_checks.py",
|
||||
"old": " problems.update(await _cleanup_problems(db))",
|
||||
"new": " pass",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k 'failed_reverts_and_unclear or unclear_delete_is_reported'"
|
||||
},
|
||||
{
|
||||
"name": "preview writes no own audit entry",
|
||||
"file": "app/services/rule_cleanup_service.py",
|
||||
"old": " db.add(audit_service.make_event(\n kind=\"rule_cleanup\", action=\"preview\"",
|
||||
"new": " (lambda *a, **k: None)(audit_service.make_event(\n kind=\"rule_cleanup\", action=\"preview\"",
|
||||
"tests": "tests/test_rule_cleanup_api.py -k shapes_and_audit"
|
||||
},
|
||||
{
|
||||
"name": "R9 alias undeploy without cleanup lock",
|
||||
"file": "app/services/alias_deploy_service.py",
|
||||
"old": " await ensure_no_open_cleanup_on(db, AliasDeployment, AliasDeployment.alias_id == alias.id)\n",
|
||||
"new": "",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
|
||||
},
|
||||
{
|
||||
"name": "R9 alias target removal without cleanup lock",
|
||||
"file": "app/routers/aliases.py",
|
||||
"old": " await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes\n",
|
||||
"new": "",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
|
||||
},
|
||||
{
|
||||
"name": "R9 wireguard undeploy without cleanup lock",
|
||||
"file": "app/services/wireguard_deploy_service.py",
|
||||
"old": " await ensure_no_open_cleanup_on(db, WireguardTunnelDeployment, WireguardTunnelDeployment.tunnel_id == tunnel.id)\n",
|
||||
"new": "",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_every_undeploy_and_target_removal_checks_for_an_open_cleanup_first"
|
||||
},
|
||||
{
|
||||
"name": "R9 helper checks nothing",
|
||||
"file": "app/services/host_maintenance_service.py",
|
||||
"old": " ids = set((await db.scalars(select(model.host_id).where(condition))).all())",
|
||||
"new": " ids = set()",
|
||||
"tests": "tests/test_rule_cleanup_api.py::test_alias_delete_and_target_removal_wait_for_the_cleanup"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,241 @@
|
||||
"""A small stateful OPNsense for the rule-cleanup tests (respx side effects).
|
||||
|
||||
It keeps its rules in memory and answers searchRule, getRule, toggleRule, delRule, addRule, moveRuleBefore, apply,
|
||||
ruleStats, pfStatistics (generated from the enabled rules, like pf after an apply), interfacesInfo, groups, aliases,
|
||||
port forwards, getVipStatus and core/hasync/get in the shapes seen live (OPNsense 26.7, see opnsense_client.py).
|
||||
``fail`` injects errors per call: {("toggle", uuid): httpx.ReadTimeout | httpx.Response | callable}.
|
||||
Addresses: documentation/private ranges only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import re
|
||||
import uuid as uuid_lib
|
||||
|
||||
import httpx
|
||||
|
||||
R_BASE = "aaaaaaaa-0000-4000-8000-000000000100"
|
||||
R_DUP = "aaaaaaaa-0000-4000-8000-000000000200"
|
||||
R_UNUSED = "aaaaaaaa-0000-4000-8000-000000000300"
|
||||
R_BLOCK = "aaaaaaaa-0000-4000-8000-000000000400"
|
||||
R_MGMT = "aaaaaaaa-0000-4000-8000-000000000500"
|
||||
R_CARP = "aaaaaaaa-0000-4000-8000-000000000600"
|
||||
R_ALIAS = "aaaaaaaa-0000-4000-8000-000000000700"
|
||||
R_TAIL = "aaaaaaaa-0000-4000-8000-000000000800"
|
||||
R_LEGACY = "0123456789abcdef0123456789abcdef"
|
||||
AUTO = "02f4bab031b57d1e30553ce08e0ec131"
|
||||
MGMT_SOURCE = "203.0.113.10"
|
||||
DEVICES = {"wan": "vtnet0", "lan": "vtnet1"}
|
||||
|
||||
|
||||
def rule(uuid, seq, *, action="pass", interface="lan", proto="TCP", src="any", dst="any", dport="", enabled="1",
|
||||
description=None, **extra) -> dict:
|
||||
row = {
|
||||
"uuid": uuid, "sort_order": f"400000.{seq:07d}", "prio_group": "400000", "sequence": str(seq),
|
||||
"enabled": enabled, "action": action, "quick": "1", "interface": interface, "interfacenot": "0",
|
||||
"direction": "in", "ipprotocol": "inet", "protocol": proto, "source_net": src, "source_not": "0",
|
||||
"source_port": "", "destination_net": dst, "destination_not": "0", "destination_port": dport,
|
||||
"gateway": "", "sched": "", "statetype": "keep", "tagged": "", "log": "0", "categories": "",
|
||||
"description": description or f"cleanup-{seq}",
|
||||
}
|
||||
row.update(extra)
|
||||
return row
|
||||
|
||||
|
||||
def default_rules() -> list[dict]:
|
||||
return [
|
||||
{"uuid": AUTO, "sort_order": "000001.1000001", "legacy": True, "is_automatic": True, "enabled": "1",
|
||||
"action": "block", "quick": False, "direction": "in", "ipprotocol": "inet46", "protocol": "",
|
||||
"source_net": "", "source_port": "", "destination_net": "", "destination_port": "",
|
||||
"description": "Default deny / state violation rule"},
|
||||
rule(R_BASE, 100, src="10.20.30.0/24", dport="9000", description="base 9000"),
|
||||
rule(R_DUP, 200, src="10.20.30.0/24", dport="9000", description="dup 9000"),
|
||||
rule(R_UNUSED, 300, src="10.20.30.0/24", dport="8081", description="unused 8081"),
|
||||
rule(R_BLOCK, 400, action="block", proto="UDP", dport="9999", description="block 9999"),
|
||||
rule(R_MGMT, 500, interface="wan", src="203.0.113.0/24", dport="443", description="mgmt 443"),
|
||||
rule(R_CARP, 600, proto="CARP", description="carp"),
|
||||
rule(R_ALIAS, 700, interface="wan", src="tbl", dport="443", description="alias 443"),
|
||||
rule(R_TAIL, 800, src="10.20.30.7", dport="7000", description="tail 7000"),
|
||||
]
|
||||
|
||||
|
||||
class FakeFirewall:
|
||||
def __init__(self, base: str, *, role: str | None = None, rules: list[dict] | None = None,
|
||||
pfsync: str = "", peer: str = "", lan_addr: str = "10.20.30.2", wan_addr: str = "203.0.113.57",
|
||||
log: list | None = None):
|
||||
self.base = base
|
||||
self.log = log # shared across firewalls: the order of calls over a cluster
|
||||
self.role = role
|
||||
self.rules: list[dict] = copy.deepcopy(rules if rules is not None else default_rules())
|
||||
self.pfsync, self.peer = pfsync, peer
|
||||
self.lan_addr, self.wan_addr = lan_addr, wan_addr
|
||||
self.fail: dict = {}
|
||||
self.calls: list[tuple[str, str]] = []
|
||||
self.applied = 0
|
||||
self.down = False
|
||||
self.on_call = None # callable(kind, uuid) run before a write is answered (e.g. look at the database)
|
||||
|
||||
# ── state helpers ──
|
||||
def get(self, uuid):
|
||||
return next((r for r in self.rules if r["uuid"].lower() == uuid.lower()), None)
|
||||
|
||||
def _inject(self, key):
|
||||
effect = self.fail.get(key)
|
||||
if effect is None:
|
||||
return None
|
||||
if isinstance(effect, list):
|
||||
effect = effect.pop(0) if effect else None
|
||||
if effect is None:
|
||||
return None
|
||||
if isinstance(effect, type) and issubclass(effect, Exception):
|
||||
raise effect("injected")
|
||||
if isinstance(effect, Exception):
|
||||
raise effect
|
||||
if callable(effect) and not isinstance(effect, httpx.Response):
|
||||
return effect()
|
||||
return effect
|
||||
|
||||
def pf_lines(self) -> dict:
|
||||
lines = {"@0 scrub in all fragment reassemble": {}}
|
||||
n = 1
|
||||
for r in sorted(self.rules, key=lambda r: r["sort_order"]):
|
||||
if str(r.get("enabled")) != "1":
|
||||
continue
|
||||
if r.get("is_automatic"):
|
||||
lines[f'@{n} block drop in log inet all label "{r["uuid"]}"'] = {"evaluations": 0}
|
||||
else:
|
||||
words = ["block" if r["action"] == "block" else "pass", "in", "quick", f"on {DEVICES.get(r['interface'], r['interface'])}", "inet"]
|
||||
if r["protocol"] not in ("", "any"):
|
||||
words.append(f"proto {r['protocol'].lower()}")
|
||||
words.append(f'all label "{r["uuid"]}"')
|
||||
lines[f"@{n} " + " ".join(words)] = {"evaluations": 0, "packets": 0}
|
||||
n += 1
|
||||
return {"rules": {"filter rules": lines}}
|
||||
|
||||
def model(self, row: dict) -> dict:
|
||||
"""getRule shape: select fields as option dicts, no uuid key."""
|
||||
out = {k: v for k, v in row.items() if k not in ("uuid", "is_automatic", "legacy")}
|
||||
out["action"] = {a: {"value": a, "selected": 1 if row["action"] == a else 0} for a in ("pass", "block", "reject")}
|
||||
out["interface"] = {i: {"value": i.upper(), "selected": 1 if i in str(row["interface"]).split(",") else 0} for i in ("lan", "wan")}
|
||||
out["categories"] = []
|
||||
out["audit"] = {"created": "x", "updated": "y"}
|
||||
return out
|
||||
|
||||
# ── respx wiring ──
|
||||
def mount(self, respx_mock):
|
||||
respx_mock.route(url__startswith=self.base).mock(side_effect=self.handle)
|
||||
return self
|
||||
|
||||
def handle(self, request: httpx.Request) -> httpx.Response:
|
||||
path = request.url.path.removeprefix("/api")
|
||||
method = request.method
|
||||
self.calls.append((method, path))
|
||||
if self.log is not None:
|
||||
self.log.append((self.base, method, path))
|
||||
if self.down:
|
||||
raise httpx.ConnectError("down")
|
||||
if method == "GET" and path == "/diagnostics/interface/getVipStatus":
|
||||
rows = [] if self.role is None else [{"interface": "LAN", "vhid": "10", "status": self.role}]
|
||||
return httpx.Response(200, json={"rows": rows, "carp": {"allow": "1", "maintenancemode": False}})
|
||||
if method == "GET" and path == "/firewall/filter/searchRule":
|
||||
got = self._inject(("search", ""))
|
||||
if got is not None:
|
||||
return got
|
||||
return httpx.Response(200, json={"rows": copy.deepcopy(sorted(self.rules, key=lambda r: r["sort_order"]))})
|
||||
if method == "GET" and path == "/diagnostics/firewall/pfStatistics/rules":
|
||||
return httpx.Response(200, json=self.pf_lines())
|
||||
if method == "GET" and path == "/firewall/filter_util/ruleStats":
|
||||
return httpx.Response(200, json={"status": "ok", "stats": {}})
|
||||
if method == "GET" and path == "/interfaces/overview/interfacesInfo":
|
||||
return httpx.Response(200, json={"rows": [
|
||||
{"identifier": "wan", "device": "vtnet0", "description": "WAN", "ipv4": [{"ipaddr": f"{self.wan_addr}/24"}],
|
||||
"ipv6": [], "gateways": ["203.0.113.1"], "routes": ["default"]},
|
||||
{"identifier": "lan", "device": "vtnet1", "description": "LAN",
|
||||
"ipv4": [{"ipaddr": f"{self.lan_addr}/24"}, {"ipaddr": "10.20.30.1/24", "vhid": "10"}],
|
||||
"ipv6": [], "gateways": [], "routes": ["10.20.30.0/24"]},
|
||||
{"identifier": "lo0", "device": "lo0", "description": "Loopback", "ipv4": [{"ipaddr": "127.0.0.1/8"}], "ipv6": []},
|
||||
]})
|
||||
if method == "GET" and path == "/firewall/group/searchItem":
|
||||
return httpx.Response(200, json={"rows": []})
|
||||
if method == "GET" and path == "/firewall/alias/searchItem":
|
||||
return httpx.Response(200, json={"rows": [{"name": "tbl", "type": "urltable", "content": "https://203.0.113.9/list"}]})
|
||||
if method == "GET" and path == "/firewall/d_nat/searchRule":
|
||||
return httpx.Response(200, json={"rows": []})
|
||||
if method == "GET" and path == "/core/hasync/get":
|
||||
sel = {"": {"value": "Disabled", "selected": 1 if not self.pfsync else 0},
|
||||
"lan": {"value": "LAN", "selected": 1 if self.pfsync == "lan" else 0}}
|
||||
return httpx.Response(200, json={"hasync": {"pfsyncinterface": sel, "pfsyncpeerip": self.peer, "password": "x"}})
|
||||
m = re.fullmatch(r"/firewall/filter/getRule/([^/]+)", path)
|
||||
if method == "GET" and m:
|
||||
got = self._inject(("get", m.group(1)))
|
||||
if got is not None:
|
||||
return got
|
||||
row = self.get(m.group(1))
|
||||
if row is None or row.get("is_automatic") or row.get("legacy"):
|
||||
return httpx.Response(200, json=[])
|
||||
return httpx.Response(200, json={"rule": self.model(row)})
|
||||
m = re.fullmatch(r"/firewall/filter/toggleRule/([^/]+)/([01])", path)
|
||||
if method == "POST" and m:
|
||||
if self.on_call:
|
||||
self.on_call("toggle", m.group(1))
|
||||
row = self.get(m.group(1))
|
||||
if row is not None and not row.get("legacy"):
|
||||
pending = ("toggle", m.group(1))
|
||||
if pending in self.fail and self.fail.get(("toggle_applies", m.group(1))):
|
||||
row["enabled"] = m.group(2) # applied on the firewall, but the answer is lost
|
||||
got = self._inject(pending)
|
||||
if got is not None:
|
||||
return got
|
||||
changed = row["enabled"] != m.group(2)
|
||||
row["enabled"] = m.group(2)
|
||||
return httpx.Response(200, json={"result": "Enabled" if m.group(2) == "1" else "Disabled", "changed": changed})
|
||||
return httpx.Response(200, json={"result": "failed"})
|
||||
m = re.fullmatch(r"/firewall/filter/delRule/([^/]+)", path)
|
||||
if method == "POST" and m:
|
||||
if self.on_call:
|
||||
self.on_call("delete", m.group(1))
|
||||
row = self.get(m.group(1))
|
||||
applies = self.fail.get(("delete_applies", m.group(1)))
|
||||
if applies and row is not None:
|
||||
self.rules.remove(row)
|
||||
got = self._inject(("delete", m.group(1)))
|
||||
if got is not None:
|
||||
return got
|
||||
if row is None or row.get("legacy"):
|
||||
return httpx.Response(200, json={"result": "not found"})
|
||||
self.rules.remove(row)
|
||||
return httpx.Response(200, json={"result": "deleted"})
|
||||
if method == "POST" and path == "/firewall/filter/addRule":
|
||||
got = self._inject(("add", ""))
|
||||
if got is not None:
|
||||
return got
|
||||
import json as _json
|
||||
body = _json.loads(request.content)["rule"]
|
||||
new = str(uuid_lib.uuid4())
|
||||
top = max(int(float(r.get("sequence") or 0)) for r in self.rules if not r.get("is_automatic"))
|
||||
row = {**body, "uuid": new, "sequence": str(top + 100), "sort_order": f"400000.{top + 100:07d}", "prio_group": "400000"}
|
||||
self.rules.append(row)
|
||||
return httpx.Response(200, json={"result": "saved", "uuid": new})
|
||||
m = re.fullmatch(r"/firewall/filter/moveRuleBefore/([^/]+)/([^/]+)", path)
|
||||
if method == "POST" and m:
|
||||
got = self._inject(("move", m.group(1))) or self._inject(("move", "*"))
|
||||
if got is not None:
|
||||
return got
|
||||
moving, target = self.get(m.group(1)), self.get(m.group(2))
|
||||
if moving is None or target is None or moving is target:
|
||||
return httpx.Response(500, json={"errorMessage": "Unexpected error, check log for details"})
|
||||
seq = int(target["sequence"]) - 1
|
||||
moving["sequence"], moving["sort_order"] = str(seq), f"400000.{seq:07d}"
|
||||
return httpx.Response(200, json={"status": "ok"})
|
||||
if method == "POST" and path == "/firewall/filter/apply":
|
||||
if self.on_call:
|
||||
self.on_call("apply", "")
|
||||
got = self._inject(("apply", ""))
|
||||
if got is not None:
|
||||
return got
|
||||
self.applied += 1
|
||||
return httpx.Response(200, json={"status": "OK\n\n"})
|
||||
return httpx.Response(404, json={"message": f"unexpected {method} {path}"})
|
||||
|
||||
def writes(self) -> list[tuple[str, str]]:
|
||||
return [(m, p) for m, p in self.calls if m == "POST"]
|
||||
@@ -12,12 +12,12 @@ from tests.test_rules import _create_host
|
||||
|
||||
|
||||
def _mvc_rule(*, interface="lan", action="pass", src="any", dst="any", protocol="any", descr="r", seq=1,
|
||||
enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0"):
|
||||
enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0", extra=""):
|
||||
return (f"<rule><enabled>{enabled}</enabled><action>{action}</action><quick>{quick}</quick><interface>{interface}</interface>"
|
||||
f"<direction>{direction}</direction><ipprotocol>{ipprotocol}</ipprotocol><protocol>{protocol}</protocol>"
|
||||
f"<source_net>{src}</source_net><source_not>{src_not}</source_not><destination_net>{dst}</destination_net>"
|
||||
f"<destination_not>0</destination_not><destination_port>{dport}</destination_port>"
|
||||
f"<description>{descr}</description><sequence>{seq}</sequence></rule>")
|
||||
f"<description>{descr}</description><sequence>{seq}</sequence>{extra}</rule>")
|
||||
|
||||
|
||||
def _config(*, rules="", legacy="", protocol="https", ssh="", snmpd="", timeservers="0.pool.ntp.org", syslog="",
|
||||
@@ -105,6 +105,52 @@ def test_rules_are_ordered_by_sequence_not_file_order():
|
||||
assert _run(_config(rules=rules))["rules_shadowed"]["params"]["examples"] == ["narrow"]
|
||||
|
||||
|
||||
def test_coverage_uses_networks_and_the_floating_section_like_the_firewall_does():
|
||||
"""1.30: a CIDR inside a CIDR is shadowed, a floating quick rule is evaluated before every interface rule,
|
||||
and an interface rule never shadows a floating one that comes first."""
|
||||
shadow = _run(_config(rules=(
|
||||
_mvc_rule(src="10.0.0.0/8", action="block", descr="wide", seq=1)
|
||||
+ _mvc_rule(src="10.1.0.0/16", descr="inside", seq=2))))
|
||||
assert shadow["rules_shadowed"]["params"]["examples"] == ["inside"]
|
||||
floating = _run(_config(rules=(
|
||||
_mvc_rule(interface="lan", src="10.1.0.0/16", descr="interface rule", seq=1)
|
||||
+ _mvc_rule(interface="lan,wan", src="10.0.0.0/8", action="block", descr="floating block", seq=9))))
|
||||
assert floating["rules_shadowed"]["params"]["examples"] == ["interface rule"]
|
||||
reverse = _run(_config(rules=(
|
||||
_mvc_rule(interface="lan", src="10.0.0.0/8", action="block", descr="interface rule", seq=1)
|
||||
+ _mvc_rule(interface="lan,wan", src="10.1.0.0/16", descr="floating narrow", seq=9))))
|
||||
assert reverse["rules_shadowed"]["passed"] is True
|
||||
|
||||
|
||||
def test_a_legacy_floating_rule_is_only_quick_when_it_says_so():
|
||||
"""mutation: defaulting legacy rules to quick makes a plain floating rule 'cover' everything after it."""
|
||||
def legacy_rule(descr, *, floating, quick=None, src="any"):
|
||||
return (f"<rule><type>block</type><interface>lan,wan</interface><ipprotocol>inet</ipprotocol>"
|
||||
f"{'<floating>yes</floating>' if floating else ''}{f'<quick>{quick}</quick>' if quick else ''}"
|
||||
f"<descr>{descr}</descr><source><any>1</any></source><destination><any>1</any></destination></rule>"
|
||||
if src == "any" else
|
||||
f"<rule><type>pass</type><interface>lan</interface><ipprotocol>inet</ipprotocol><descr>{descr}</descr>"
|
||||
f"<source><address>{src}</address></source><destination><any/></destination></rule>")
|
||||
later = legacy_rule("later", floating=False, src="10.0.0.0/8")
|
||||
assert _run(_config(legacy=legacy_rule("not quick", floating=True) + later))["rules_shadowed"]["passed"] is True
|
||||
quick = _run(_config(legacy=legacy_rule("quick", floating=True, quick="yes") + later))
|
||||
assert quick["rules_shadowed"]["params"]["examples"] == ["later"]
|
||||
# a non-floating legacy rule is always quick, whatever <quick> says
|
||||
plain = _run(_config(legacy=legacy_rule("interface rule", floating=False) .replace("lan,wan", "lan") + later))
|
||||
assert plain["rules_shadowed"]["params"]["examples"] == ["later"]
|
||||
|
||||
|
||||
def test_rules_at_the_same_position_in_the_config_are_not_compared():
|
||||
rules = _mvc_rule(src="any", action="block", descr="first", seq=5) + _mvc_rule(src="10.0.0.0/8", descr="second", seq=5)
|
||||
assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True
|
||||
|
||||
|
||||
def test_a_scheduled_or_tagged_rule_does_not_shadow_anything():
|
||||
for extra in ("<sched>office</sched>", "<tagged>mark</tagged>", "<received-on>wan</received-on>"):
|
||||
rules = _mvc_rule(src="any", action="block", descr="timed", seq=1, extra=extra) + _mvc_rule(src="10.0.0.0/8", descr="later", seq=2)
|
||||
assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True, extra
|
||||
|
||||
|
||||
def test_ssh_levels():
|
||||
on = "<ssh><enabled>enabled</enabled><permitrootlogin>{}</permitrootlogin><passwordauth>{}</passwordauth></ssh>"
|
||||
worst = _run(_config(ssh=on.format(1, 1)))["ssh_hardened"]
|
||||
@@ -159,6 +205,8 @@ def _mock_host(respx_mock, xml, firmware_status="none"):
|
||||
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
|
||||
respx_mock.get(url__regex=r".*/core/backup/download/this").mock(return_value=httpx.Response(200, content=xml))
|
||||
respx_mock.get(url__regex=r".*/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": firmware_status}))
|
||||
# An older release without ruleStats: the usage sampling falls back to the pf ruleset text.
|
||||
respx_mock.get(url__regex=r".*/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={"message": "not found"}))
|
||||
respx_mock.get(url__regex=r".*/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
|
||||
respx_mock.get(url__regex=r".*/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
|
||||
|
||||
@@ -246,7 +294,10 @@ def test_pf_counters_are_tied_to_rule_uuids_and_summed_over_expanded_lines():
|
||||
stats["rules"]["filter rules"][f'@7 pass in quick on vtnet0 inet6 all label "{U1}"'] = {"evaluations": 2, "packets": 3, "bytes": 30}
|
||||
got = rh.parse_counters(stats)
|
||||
assert got[U1] == {"packets": 8, "bytes": 530, "evaluations": 12}
|
||||
assert "2f2c37025cd02b5cd29fd330f8e48954" not in got # md5 labels are automatic rules, not user rules
|
||||
# An md5 label is either an automatic rule or an unmigrated legacy rule: only searchRule (is_automatic) tells them apart.
|
||||
assert got["2f2c37025cd02b5cd29fd330f8e48954"]["packets"] == 9
|
||||
assert "2f2c37025cd02b5cd29fd330f8e48954" not in rh.configured_rules(
|
||||
{"rows": [{"uuid": "2f2c37025cd02b5cd29fd330f8e48954", "is_automatic": True}]})
|
||||
assert rh.parse_counters({}) == {} and rh.parse_counters({"rules": None}) == {}
|
||||
|
||||
|
||||
@@ -259,7 +310,8 @@ async def _sample(host_id, stats, search, at):
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, __import__("uuid").UUID(host_id))
|
||||
from unittest.mock import AsyncMock, patch
|
||||
with patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
|
||||
with patch.object(rh.client, "get_rule_stats", AsyncMock(side_effect=rh.OpnsenseAPIError("404"))), \
|
||||
patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
|
||||
patch.object(rh.client, "search_rules", AsyncMock(return_value=search)):
|
||||
return await rh.sample_host(db, host, now=at)
|
||||
|
||||
@@ -373,6 +425,7 @@ async def test_one_truncated_config_does_not_stop_the_others(client, respx_mock)
|
||||
for base, body in (("bad", b"<opnsense><system><webgui>"), ("good", _config())):
|
||||
respx_mock.get(f"https://{base}.test/api/core/backup/download/this").mock(return_value=httpx.Response(200, content=body))
|
||||
respx_mock.get(f"https://{base}.test/api/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": "none"}))
|
||||
respx_mock.get(f"https://{base}.test/api/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={}))
|
||||
respx_mock.get(f"https://{base}.test/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
|
||||
respx_mock.get(f"https://{base}.test/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
|
||||
resp = await client.post("/api/compliance/check")
|
||||
|
||||
@@ -331,5 +331,123 @@ async def test_0026_adds_account_ids_matches_existing_names_and_is_idempotent(sc
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
await _alembic(scratch, "downgrade", "0025")
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
assert await _version(scratch) == _head() == "0026"
|
||||
assert await _version(scratch) == _head()
|
||||
assert await _schema_diff(scratch) == []
|
||||
|
||||
|
||||
async def test_0027_creates_the_rule_history_tables_and_is_idempotent(scratch):
|
||||
"""1.29 -> 1.30: two new tables, the downgrade drops them; running the upgrade over tables that a pre-Alembic
|
||||
bridge (create_all) already made changes nothing."""
|
||||
await _alembic(scratch, "upgrade", "0026")
|
||||
async with scratch.connect() as conn:
|
||||
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
|
||||
assert "live_rule_states" not in tables and "rule_change_events" not in tables
|
||||
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
assert await _version(scratch) == _head()
|
||||
assert await _schema_diff(scratch) == []
|
||||
host = "11111111-1111-1111-1111-111111111111"
|
||||
async with scratch.begin() as conn:
|
||||
await conn.execute(text(
|
||||
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
|
||||
"VALUES (:h, 'fw', 'https://fw.test', 'k', true, true, now())"), {"h": host})
|
||||
await conn.execute(text(
|
||||
"INSERT INTO rule_change_events (id, host_id, rule_uuid, seq, observed_at, kind, origin) "
|
||||
"VALUES (gen_random_uuid(), :h, 'r', 1, now(), 'added', 'external')"), {"h": host})
|
||||
with pytest.raises(DBAPIError): # the same change can only be recorded once
|
||||
async with scratch.begin() as conn:
|
||||
await conn.execute(text(
|
||||
"INSERT INTO rule_change_events (id, host_id, rule_uuid, seq, observed_at, kind, origin) "
|
||||
"VALUES (gen_random_uuid(), :h, 'r', 1, now(), 'added', 'external')"), {"h": host})
|
||||
async with scratch.begin() as conn: # deleting the host takes its history with it
|
||||
await conn.execute(text("DELETE FROM opnsense_hosts"))
|
||||
assert (await conn.execute(text("SELECT count(*) FROM rule_change_events"))).scalar() == 0
|
||||
|
||||
await _alembic(scratch, "downgrade", "0026")
|
||||
async with scratch.connect() as conn:
|
||||
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
|
||||
assert "live_rule_states" not in tables and "rule_change_events" not in tables
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
await _alembic(scratch, "upgrade", "head") # second run: nothing to do
|
||||
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
|
||||
|
||||
|
||||
async def test_0028_adds_the_event_window_and_is_idempotent(scratch):
|
||||
await _alembic(scratch, "upgrade", "0027")
|
||||
async with scratch.connect() as conn:
|
||||
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'rule_change_events'"))).scalars().all())
|
||||
assert "window_start" not in cols
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
|
||||
await _alembic(scratch, "downgrade", "0027")
|
||||
async with scratch.connect() as conn:
|
||||
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'rule_change_events'"))).scalars().all())
|
||||
assert "window_start" not in cols
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
|
||||
|
||||
|
||||
async def test_0029_creates_the_rule_cleanup_tables_and_settings_and_is_idempotent(scratch):
|
||||
"""1.30 -> 1.31: two tables with a partial unique index (one unfinished item per rule and node) and three
|
||||
columns on governance_settings; the downgrade removes all of it again."""
|
||||
await _alembic(scratch, "upgrade", "0028")
|
||||
async with scratch.begin() as conn:
|
||||
await conn.execute(text("INSERT INTO governance_settings (id, operators_may_deploy) VALUES (1, true)"))
|
||||
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
|
||||
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'governance_settings'"))).scalars().all())
|
||||
assert "rule_cleanup_operations" not in tables and "rule_cleanup_items" not in tables
|
||||
assert "cleanup_max_rules" not in cols
|
||||
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
assert await _version(scratch) == _head() and await _schema_diff(scratch) == []
|
||||
async with scratch.connect() as conn: # the existing settings row got the defaults of the design
|
||||
row = (await conn.execute(text(
|
||||
"SELECT cleanup_max_rules, cleanup_delete_min_days, cleanup_management_sources, cleanup_management_ports "
|
||||
"FROM governance_settings"))).one()
|
||||
assert tuple(row) == (10, 7, [], [])
|
||||
|
||||
host, op_id = "11111111-1111-1111-1111-111111111111", "22222222-2222-2222-2222-222222222222"
|
||||
async with scratch.begin() as conn:
|
||||
await conn.execute(text(
|
||||
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
|
||||
"VALUES (:h, 'fw', 'https://fw.test', 'k', true, true, now())"), {"h": host})
|
||||
await conn.execute(text(
|
||||
"INSERT INTO rule_cleanup_operations (id, host_id, kind, status, actor, created_at, expires_at, preview_fingerprint, options) "
|
||||
"VALUES (:o, :h, 'disable', 'preview', 'admin', now(), now(), 'f', '{}')"), {"o": op_id, "h": host})
|
||||
|
||||
async def item(conn, rule: str, state: str):
|
||||
await conn.execute(text(
|
||||
"INSERT INTO rule_cleanup_items (id, operation_id, host_id, rule_uuid, seq, state, description, categories) "
|
||||
f"VALUES (gen_random_uuid(), :o, :h, '{rule}', 0, '{state}', 'r', '[]')"), {"o": op_id, "h": host})
|
||||
|
||||
async with scratch.begin() as conn:
|
||||
await item(conn, "r1", "disabled")
|
||||
await item(conn, "r2", "disabled") # another rule is fine
|
||||
await item(conn, "r1", "confirmed") # finished items do not count
|
||||
await item(conn, "r1", "reverted")
|
||||
for blocked in ("queued", "unknown", "revert_failed", "deleting"): # r1 is still open as "disabled"
|
||||
with pytest.raises(DBAPIError):
|
||||
async with scratch.begin() as conn:
|
||||
await item(conn, "r1", blocked)
|
||||
async with scratch.begin() as conn: # on another node the same UUID is a different rule
|
||||
await conn.execute(text(
|
||||
"INSERT INTO opnsense_hosts (id, name, base_url, api_key, verify_tls, enabled, created_at) "
|
||||
"VALUES ('33333333-3333-3333-3333-333333333333', 'fw2', 'https://fw2.test', 'k', true, true, now())"))
|
||||
await conn.execute(text(
|
||||
"INSERT INTO rule_cleanup_items (id, operation_id, host_id, rule_uuid, seq, state, description, categories) "
|
||||
"VALUES (gen_random_uuid(), :o, '33333333-3333-3333-3333-333333333333', 'r1', 0, 'disabled', 'r', '[]')"), {"o": op_id})
|
||||
async with scratch.begin() as conn: # the host takes its operations and their items with it
|
||||
await conn.execute(text("DELETE FROM opnsense_hosts WHERE id = :h"), {"h": host})
|
||||
assert (await conn.execute(text("SELECT count(*) FROM rule_cleanup_operations"))).scalar() == 0
|
||||
assert (await conn.execute(text("SELECT count(*) FROM rule_cleanup_items"))).scalar() == 0
|
||||
|
||||
await _alembic(scratch, "downgrade", "0028")
|
||||
async with scratch.connect() as conn:
|
||||
tables = set((await conn.execute(text("SELECT tablename FROM pg_tables WHERE schemaname = 'public'"))).scalars().all())
|
||||
cols = set((await conn.execute(text("SELECT column_name FROM information_schema.columns WHERE table_name = 'governance_settings'"))).scalars().all())
|
||||
assert "rule_cleanup_operations" not in tables and "rule_cleanup_items" not in tables
|
||||
assert not {"cleanup_max_rules", "cleanup_delete_min_days", "cleanup_management_sources", "cleanup_management_ports"} & cols
|
||||
await _alembic(scratch, "upgrade", "head")
|
||||
await _alembic(scratch, "upgrade", "head") # second run: nothing to do
|
||||
assert await _version(scratch) == _head() == "0029" and await _schema_diff(scratch) == []
|
||||
@@ -146,9 +146,9 @@ def test_what_counts_as_a_rollout(method, path, rollout):
|
||||
async def test_with_separation_of_duties_operators_prepare_but_only_admins_deploy(client):
|
||||
op = await _operator(client, "dutyop")
|
||||
try:
|
||||
assert (await op.get("/api/settings/governance")).json() == {"operators_may_deploy": True}
|
||||
assert (await op.get("/api/settings/governance")).json()["operators_may_deploy"] is True
|
||||
assert (await op.put("/api/settings/governance", json={"operators_may_deploy": False})).status_code == 403
|
||||
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": False})).json() == {"operators_may_deploy": False}
|
||||
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": False})).json()["operators_may_deploy"] is False
|
||||
# Preparing still works ...
|
||||
assert (await op.post("/api/groups", json={"name": "prepared"})).status_code == 201
|
||||
# ... rolling out does not, whatever the target (the 403 comes before any lookup).
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
"""Client calls for filter rules that the cleanup assistant (1.31) builds on: toggleRule, getRule, the
|
||||
getRule -> addRule payload, and the request shape / failure classification of add/del/move/apply.
|
||||
|
||||
The JSON fixtures are shaped like live answers of OPNsense 26.7.7 (checked 2026-10-22 on a test firewall) with
|
||||
documentation addresses and made-up UUIDs. Evidence for each endpoint: comment block in services/opnsense_client.py.
|
||||
"""
|
||||
import json
|
||||
import uuid
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.services import opnsense_client as c
|
||||
|
||||
BASE = "https://fw.test/api/firewall/filter"
|
||||
RULE = "5c6f3bfa-1718-4216-81c8-04a75527960f"
|
||||
OTHER = "74190f58-f35f-4402-8794-5410e99ea364"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host(monkeypatch):
|
||||
monkeypatch.setattr(c.opnsense_secret_store, "read_secret", lambda host_id: "secret")
|
||||
return OpnsenseHost(id=uuid.uuid4(), name="fw-a", base_url="https://fw.test", api_key="key", verify_tls=False)
|
||||
|
||||
|
||||
def _opts(selected: list[str], options: dict[str, str]) -> dict:
|
||||
return {k: {"value": v, "selected": 1 if k in selected else 0} for k, v in options.items()}
|
||||
|
||||
|
||||
def live_rule() -> dict:
|
||||
"""A `getRule/<uuid>` record: select fields as option dicts, `categories` an option dict, read-only extras."""
|
||||
return {
|
||||
"enabled": "1", "sequence": "566", "sort_order": "200000.0000566", "prio_group": "200000", "quick": "1",
|
||||
"action": _opts(["pass"], {"pass": "Pass", "block": "Block", "reject": "Reject"}),
|
||||
"interface": _opts(["lan", "wan"], {"lan": "LAN", "wan": "WAN", "wireguard": "WireGuard (Group)"}),
|
||||
"direction": _opts(["in"], {"in": "In", "out": "Out", "any": "Both"}),
|
||||
"protocol": _opts(["TCP"], {"any": "any", "TCP": "TCP", "UDP": "UDP"}),
|
||||
"gateway": _opts([""], {"": "None", "WAN_GW": "WAN_GW - 203.0.113.1"}),
|
||||
"categories": _opts(["c1", "c2"], {"c1": "one", "c2": "two", "c3": "three"}),
|
||||
"tcpflags1": _opts([], {"syn": "SYN", "ack": "ACK"}),
|
||||
"source_net": "bogons", "destination_net": "203.0.113.9", "destination_port": "8080-8443", "source_port": "",
|
||||
"description": "opnexus-b4-probe", "statetimeout": None, "log": 1,
|
||||
"audit": {"created": {"username": "", "time": "", "description": ""}, "userdata": {"note": ""}},
|
||||
}
|
||||
|
||||
|
||||
def _body(route) -> dict:
|
||||
return json.loads(route.calls.last.request.content)
|
||||
|
||||
|
||||
# ── toggle_rule ────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("enabled, tail, answer", [(False, "0", "Disabled"), (True, "1", "Enabled")])
|
||||
async def test_toggle_rule_sends_the_explicit_value_and_returns_the_answer(host, respx_mock, enabled, tail, answer):
|
||||
route = respx_mock.post(f"{BASE}/toggleRule/{RULE}/{tail}").mock(
|
||||
return_value=httpx.Response(200, json={"result": answer, "changed": True}))
|
||||
assert await c.toggle_rule(host, RULE, enabled) == {"result": answer, "changed": True}
|
||||
assert route.call_count == 1 and not route.calls.last.request.content # never the flipping form, no body
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_toggle_rule_is_idempotent_an_unchanged_answer_is_success(host, respx_mock):
|
||||
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(
|
||||
return_value=httpx.Response(200, json={"result": "Disabled", "changed": False}))
|
||||
assert (await c.toggle_rule(host, RULE, False))["changed"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("bad", [0, 1, "0", "1", None, "false"])
|
||||
async def test_toggle_rule_refuses_anything_but_a_bool_without_a_request(host, respx_mock, bad):
|
||||
route = respx_mock.post(url__regex=r".*/toggleRule/.*").mock(return_value=httpx.Response(200, json={}))
|
||||
with pytest.raises(TypeError):
|
||||
await c.toggle_rule(host, RULE, bad)
|
||||
assert route.call_count == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_toggle_rule_answer_for_the_other_state_is_not_accepted(host, respx_mock):
|
||||
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(return_value=httpx.Response(200, json={"result": "Enabled", "changed": True}))
|
||||
with pytest.raises(c.OpnsenseAPIError) as info:
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
assert not isinstance(info.value, c.OpnsenseAmbiguousError)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("answer", [{"result": "failed"}, {"result": "failed", "changed": False}])
|
||||
async def test_toggle_rule_unknown_uuid_is_a_plain_refusal(host, respx_mock, answer):
|
||||
"""Live: an unknown or legacy UUID answers HTTP 200 {"result": "failed"} (no "changed" key)."""
|
||||
respx_mock.post(f"{BASE}/toggleRule/{RULE}/0").mock(return_value=httpx.Response(200, json=answer))
|
||||
with pytest.raises(c.OpnsenseAPIError) as info:
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
assert type(info.value) is c.OpnsenseAPIError
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_toggle_rule_failure_classes(host, respx_mock):
|
||||
route = respx_mock.post(f"{BASE}/toggleRule/{RULE}/0")
|
||||
route.mock(return_value=httpx.Response(403, json={"status": 403, "message": "Forbidden"}))
|
||||
with pytest.raises(c.OpnsenseForbiddenError):
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
route.mock(return_value=httpx.Response(500, text="boom"))
|
||||
with pytest.raises(c.OpnsenseAmbiguousError): # the change may have been saved
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
route.mock(side_effect=httpx.ReadTimeout("slow"))
|
||||
with pytest.raises(c.OpnsenseAmbiguousError):
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
route.mock(side_effect=httpx.ConnectError("down")) # never sent: a plain error, not an unknown outcome
|
||||
with pytest.raises(c.OpnsenseAPIError) as info:
|
||||
await c.toggle_rule(host, RULE, False)
|
||||
assert not isinstance(info.value, c.OpnsenseAmbiguousError)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("bad", ["../x", "a/b", "a?b", "", "x" * 65, "a b"])
|
||||
async def test_rule_references_cannot_leave_the_url_path(host, respx_mock, bad):
|
||||
route = respx_mock.route().mock(return_value=httpx.Response(200, json={}))
|
||||
with pytest.raises(c.OpnsenseAPIError):
|
||||
await c.toggle_rule(host, bad, False)
|
||||
with pytest.raises(c.OpnsenseAPIError):
|
||||
await c.get_rule(host, bad)
|
||||
assert route.call_count == 0
|
||||
|
||||
|
||||
# ── get_rule ───────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_rule_returns_the_inner_record(host, respx_mock):
|
||||
route = respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json={"rule": live_rule()}))
|
||||
rule = await c.get_rule(host, RULE)
|
||||
assert route.call_count == 1 and rule["description"] == "opnexus-b4-probe" and rule["interface"]["lan"]["selected"] == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("answer", [[], {}])
|
||||
async def test_get_rule_empty_answer_means_not_found(host, respx_mock, answer):
|
||||
"""Live: unknown, legacy (hex label) and automatic UUIDs answer HTTP 200 with a bare `[]`."""
|
||||
respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json=answer))
|
||||
with pytest.raises(c.OpnsenseNotFoundError):
|
||||
await c.get_rule(host, RULE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_rule_failure_classes(host, respx_mock):
|
||||
route = respx_mock.get(f"{BASE}/getRule/{RULE}")
|
||||
route.mock(return_value=httpx.Response(200, json={"rule": "nonsense"}))
|
||||
with pytest.raises(c.OpnsenseAPIError) as info:
|
||||
await c.get_rule(host, RULE)
|
||||
assert not isinstance(info.value, c.OpnsenseNotFoundError) # an unreadable answer is not proof that the rule is gone
|
||||
route.mock(return_value=httpx.Response(403, text="no"))
|
||||
with pytest.raises(c.OpnsenseForbiddenError):
|
||||
await c.get_rule(host, RULE)
|
||||
route.mock(return_value=httpx.Response(502, text="bad gateway"))
|
||||
with pytest.raises(c.OpnsenseAmbiguousError):
|
||||
await c.get_rule(host, RULE)
|
||||
|
||||
|
||||
# ── rule_write_payload (getRule -> addRule) ───────────────────────────────
|
||||
|
||||
def test_rule_write_payload_reduces_selects_and_drops_read_only_fields():
|
||||
payload = c.rule_write_payload(live_rule())
|
||||
assert payload["interface"] == "lan,wan" # multi-interface: comma separated
|
||||
assert payload["categories"] == "c1,c2"
|
||||
assert payload["action"] == "pass" and payload["protocol"] == "TCP" and payload["direction"] == "in"
|
||||
assert payload["gateway"] == "" # the "None" option has the key ""
|
||||
assert payload["tcpflags1"] == "" # nothing selected
|
||||
assert payload["statetimeout"] == "" and payload["log"] == "1" and payload["source_port"] == ""
|
||||
assert payload["enabled"] == "1" and payload["destination_port"] == "8080-8443"
|
||||
assert not set(payload) & {"uuid", "sequence", "sort_order", "prio_group", "audit"}
|
||||
assert all(isinstance(v, str) for v in payload.values())
|
||||
|
||||
|
||||
def test_rule_write_payload_empty_category_list_becomes_an_empty_string():
|
||||
"""Live: a rule without categories answers `"categories": []` (PHP's empty array), not an option dict."""
|
||||
rule = live_rule() | {"categories": []}
|
||||
assert c.rule_write_payload(rule)["categories"] == ""
|
||||
|
||||
|
||||
def test_rule_write_payload_does_not_mutate_its_input():
|
||||
rule = live_rule()
|
||||
before = json.dumps(rule, sort_keys=True)
|
||||
c.rule_write_payload(rule)
|
||||
assert json.dumps(rule, sort_keys=True) == before
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_copy_is_added_with_exactly_the_payload_and_stays_disabled(host, respx_mock):
|
||||
"""The restore path: getRule -> payload -> addRule(enabled=0). The body must carry the payload unchanged."""
|
||||
respx_mock.get(f"{BASE}/getRule/{RULE}").mock(return_value=httpx.Response(200, json={"rule": live_rule()}))
|
||||
add = respx_mock.post(f"{BASE}/addRule").mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": OTHER}))
|
||||
payload = c.rule_write_payload(await c.get_rule(host, RULE)) | {"enabled": "0"}
|
||||
assert (await c.add_rule(host, payload))["uuid"] == OTHER
|
||||
sent = _body(add)["rule"]
|
||||
assert sent == payload and sent["enabled"] == "0" and "sequence" not in sent and "uuid" not in sent
|
||||
|
||||
|
||||
# ── add / set / del / move / apply: request shape and classification ─────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_rule_validation_failure_is_a_plain_refusal(host, respx_mock):
|
||||
respx_mock.post(f"{BASE}/addRule").mock(return_value=httpx.Response(
|
||||
200, json={"result": "failed", "validations": {"rule.destination_port": "Please specify a valid portnumber"}}))
|
||||
with pytest.raises(c.OpnsenseAPIError) as info:
|
||||
await c.add_rule(host, {"destination_port": "1,2"})
|
||||
assert type(info.value) is c.OpnsenseAPIError and "destination_port" in str(info.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("answer", ["deleted", "not found"])
|
||||
async def test_del_rule_accepts_deleted_and_not_found(host, respx_mock, answer):
|
||||
route = respx_mock.post(f"{BASE}/delRule/{RULE}").mock(return_value=httpx.Response(200, json={"result": answer}))
|
||||
assert (await c.del_rule(host, RULE))["result"] == answer and route.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_del_rule_failed_answer_and_5xx(host, respx_mock):
|
||||
route = respx_mock.post(f"{BASE}/delRule/{RULE}")
|
||||
route.mock(return_value=httpx.Response(200, json={"result": "failed"}))
|
||||
with pytest.raises(c.OpnsenseAPIError):
|
||||
await c.del_rule(host, RULE)
|
||||
route.mock(return_value=httpx.Response(500, text="x"))
|
||||
with pytest.raises(c.OpnsenseAmbiguousError):
|
||||
await c.del_rule(host, RULE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_rule_before_url_and_answer(host, respx_mock):
|
||||
route = respx_mock.post(f"{BASE}/moveRuleBefore/{RULE}/{OTHER}").mock(return_value=httpx.Response(200, json={"status": "ok"}))
|
||||
assert await c.move_rule_before(host, RULE, OTHER) == {"status": "ok"} and route.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_rule_before_refusal_is_http_500_and_therefore_ambiguous(host, respx_mock):
|
||||
"""Live: unknown UUID, the same UUID twice and rules of different prio_groups all answer HTTP 500
|
||||
{"errorMessage": "Unexpected error, check log for details"}. Callers must read the position back."""
|
||||
respx_mock.post(f"{BASE}/moveRuleBefore/{RULE}/{OTHER}").mock(return_value=httpx.Response(
|
||||
500, json={"errorMessage": "Unexpected error, check log for details", "errorTitle": "Filter", "errorLevel": "warning"}))
|
||||
with pytest.raises(c.OpnsenseAmbiguousError):
|
||||
await c.move_rule_before(host, RULE, OTHER)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_url_and_answer(host, respx_mock):
|
||||
route = respx_mock.post(f"{BASE}/apply").mock(return_value=httpx.Response(200, json={"status": "OK\n\n"}))
|
||||
assert (await c.apply(host))["status"].strip() == "OK" and route.call_count == 1
|
||||
@@ -0,0 +1,994 @@
|
||||
"""Rule cleanup (1.31, B4 part 2): preview, deploy, confirm, revert/sweeper, delete and restore against a stateful
|
||||
fake firewall (tests/rule_cleanup_fake.py) and the test database. Numbers refer to DESIGN-1.31 §6. Roles and API
|
||||
tokens are checked in the router (package 6), not here."""
|
||||
import asyncio
|
||||
import uuid as uuid_lib
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from fastapi import HTTPException
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from app.config import settings
|
||||
from app.database import SessionLocal
|
||||
from app.models.audit import AuditEvent
|
||||
from app.models.governance import GovernanceSettings
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.host_operation import HostOperation
|
||||
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
|
||||
from app.models.rule_hit import RuleHit
|
||||
from app.models.rule_history import LiveRuleState, RuleChangeEvent
|
||||
from app.services import rule_cleanup_service as svc
|
||||
from app.services import rule_history_service as rhs
|
||||
from app.services.rule_analysis import context as ctx_mod
|
||||
from tests.rule_cleanup_fake import (
|
||||
AUTO, MGMT_SOURCE, R_ALIAS, R_BASE, R_BLOCK, R_CARP, R_DUP, R_LEGACY, R_MGMT, R_TAIL, R_UNUSED, FakeFirewall,
|
||||
default_rules, rule,
|
||||
)
|
||||
from tests.test_rules import _create_host
|
||||
|
||||
FW1, FW2 = "https://203.0.113.57", "https://203.0.113.58"
|
||||
UNUSED_SEEDED = (R_UNUSED, R_BLOCK, R_MGMT, R_CARP, R_ALIAS, R_TAIL)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _fresh_cache():
|
||||
ctx_mod.clear_cache()
|
||||
yield
|
||||
ctx_mod.clear_cache()
|
||||
|
||||
|
||||
def _now():
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
async def _settings(sources=(MGMT_SOURCE,), max_rules=10, min_days=7, ports=()):
|
||||
async with SessionLocal() as db:
|
||||
row = await db.get(GovernanceSettings, 1)
|
||||
if row is None:
|
||||
row = GovernanceSettings(id=1, operators_may_deploy=True)
|
||||
db.add(row)
|
||||
row.cleanup_management_sources = list(sources)
|
||||
row.cleanup_management_ports = list(ports)
|
||||
row.cleanup_max_rules, row.cleanup_delete_min_days = max_rules, min_days
|
||||
await db.commit()
|
||||
|
||||
|
||||
async def _seed_hits(host_id, uuids, *, days=100, hit_days=None, sampled_minutes=10):
|
||||
async with SessionLocal() as db:
|
||||
now = _now()
|
||||
for u in uuids:
|
||||
db.add(RuleHit(host_id=host_id, rule_uuid=u.lower(), description=u, enabled=True,
|
||||
first_seen_at=now - timedelta(days=days), last_packets=0, last_bytes=0,
|
||||
last_sampled_at=now - timedelta(minutes=sampled_minutes),
|
||||
last_hit_at=None if hit_days is None else now - timedelta(days=hit_days)))
|
||||
await db.commit()
|
||||
|
||||
|
||||
async def call(host_id, fn, *args, **kw):
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
return await fn(db, host, *args, **kw)
|
||||
|
||||
|
||||
async def refused(host_id, fn, *args, **kw) -> dict:
|
||||
with pytest.raises(HTTPException) as info:
|
||||
await call(host_id, fn, *args, **kw)
|
||||
return {"status": info.value.status_code, **info.value.detail}
|
||||
|
||||
|
||||
async def preview(host_id, uuids, actor="admin", **kw):
|
||||
kw.setdefault("kind", "disable")
|
||||
return await call(host_id, svc.preview, actor=actor, uuids=uuids, **kw)
|
||||
|
||||
|
||||
async def deploy(host_id, preview_id, actor="admin", name=None):
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
return await svc.deploy(db, host, uuid_lib.UUID(preview_id), actor=actor, confirm_name=name or host.name)
|
||||
|
||||
|
||||
async def _items(op_id):
|
||||
async with SessionLocal() as db:
|
||||
return {i.rule_uuid: i for i in (await db.scalars(select(RuleCleanupItem).where(
|
||||
RuleCleanupItem.operation_id == uuid_lib.UUID(str(op_id))))).all()}
|
||||
|
||||
|
||||
async def _op(op_id):
|
||||
async with SessionLocal() as db:
|
||||
return await db.get(RuleCleanupOperation, uuid_lib.UUID(str(op_id)))
|
||||
|
||||
|
||||
async def _backdate(op_id, **fields):
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleCleanupOperation).where(RuleCleanupOperation.id == uuid_lib.UUID(str(op_id))).values(**fields))
|
||||
await db.commit()
|
||||
|
||||
|
||||
async def _sweep():
|
||||
async with SessionLocal() as db:
|
||||
return await svc.sweep_expired_cleanups(db)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def env(client, respx_mock):
|
||||
log: list = []
|
||||
fw = FakeFirewall(FW1, log=log).mount(respx_mock)
|
||||
host_id = uuid_lib.UUID((await _create_host(client, "fw-a", FW1))["id"])
|
||||
await _settings()
|
||||
await _seed_hits(host_id, UNUSED_SEEDED)
|
||||
fw.calls.clear()
|
||||
log.clear()
|
||||
return SimpleNamespace(fw=fw, host_id=host_id, log=log)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def cluster(client, respx_mock):
|
||||
log: list = []
|
||||
backup = FakeFirewall(FW2, role="BACKUP", pfsync="lan", peer="10.20.30.2", lan_addr="10.20.30.3",
|
||||
wan_addr="203.0.113.58", log=log).mount(respx_mock)
|
||||
master = FakeFirewall(FW1, role="MASTER", pfsync="lan", peer="10.20.30.3", log=log).mount(respx_mock)
|
||||
a = uuid_lib.UUID((await _create_host(client, "fw-a", FW1))["id"])
|
||||
b = uuid_lib.UUID((await _create_host(client, "fw-b", FW2))["id"])
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(OpnsenseHost).where(OpnsenseHost.id.in_([a, b])).values(cluster_name="ha-test"))
|
||||
await db.commit()
|
||||
await _settings()
|
||||
await _seed_hits(a, UNUSED_SEEDED)
|
||||
await _seed_hits(b, UNUSED_SEEDED)
|
||||
for f in (master, backup):
|
||||
f.calls.clear()
|
||||
log.clear()
|
||||
return SimpleNamespace(master=master, backup=backup, a=a, b=b, log=log)
|
||||
|
||||
|
||||
def _writes(log):
|
||||
return [(base, path) for base, method, path in log if method == "POST"]
|
||||
|
||||
|
||||
# ── Preview ───────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_preview_recomputes_from_the_live_firewall_and_stores_a_bound_token(env):
|
||||
result = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
plan = result["plan"]
|
||||
items = {i["uuid"]: i for i in plan["nodes"][0]["items"]}
|
||||
assert set(items) == {R_UNUSED, R_DUP}
|
||||
assert items[R_UNUSED]["level"] == "sure" and items[R_UNUSED]["protection"]["result"] == "false"
|
||||
assert items[R_DUP]["cover_uuid"] == R_BASE and items[R_DUP]["cover_fp"]
|
||||
assert env.fw.writes() == [] # nothing is written by a preview
|
||||
assert ("GET", "/firewall/filter/searchRule") in env.fw.calls # read live, not from a cache or the request
|
||||
op = await _op(result["preview_id"])
|
||||
assert op.status == "preview" and op.actor == "admin" and len(op.preview_fingerprint) == 64
|
||||
assert timedelta(minutes=4) < op.expires_at - _now() <= timedelta(minutes=5)
|
||||
async with SessionLocal() as db:
|
||||
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "rule_cleanup"))).all()
|
||||
assert [(a.action, a.status) for a in audit] == [("preview", "requested")]
|
||||
assert "remote_uuid" not in audit[0].details # a preview never appears as a change in the rule history
|
||||
# a second preview of the same actor replaces the first
|
||||
again = await preview(env.host_id, [R_UNUSED])
|
||||
assert (await _op(result["preview_id"])).status == "cancelled" and (await _op(again["preview_id"])).status == "preview"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_without_management_sources_everything_is_locked(env):
|
||||
await _settings(sources=())
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
|
||||
await _settings(sources=("not-an-address",))
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("uuid,reason", [
|
||||
(R_MGMT, "management"), # allows the management source to the firewall's 443
|
||||
(R_ALIAS, "management"), # source is a URL table: match unknown = protected (case 18)
|
||||
(R_CARP, "infrastructure"), # CARP protocol
|
||||
(R_BASE, "not_candidate"), # nothing says it is removable
|
||||
])
|
||||
async def test_rules_that_must_not_be_switched_off_are_refused(env, uuid, reason):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[uuid], acknowledge=[uuid])
|
||||
assert got["status"] == 409 and got["code"] == "BEREINIGUNG_REGEL_NICHT_AENDERBAR"
|
||||
assert got["params"]["reason"] == reason and got["params"]["uuid"] == uuid
|
||||
assert env.fw.writes() == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_managed_legacy_and_unknown_confidence_are_refused(env):
|
||||
from app.models.rule import FirewallRule
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
env.fw.rules.append(rule(R_LEGACY, 900, src="10.20.30.0/24", dport="6000", legacy=True))
|
||||
await _seed_hits(env.host_id, [R_LEGACY])
|
||||
async with SessionLocal() as db:
|
||||
managed = FirewallRule(description="Managed", interface="lan", action="pass", direction="in", protocol="tcp",
|
||||
source_net="any", destination_net="any", destination_port="8081")
|
||||
db.add(managed)
|
||||
await db.flush()
|
||||
db.add(RuleDeployment(rule_id=managed.id, host_id=env.host_id, status="success", opnsense_rule_uuid=R_UNUSED))
|
||||
# stale counters (sampled two days ago): "unused" is unknown, never selectable
|
||||
await db.execute(update(RuleHit).where(RuleHit.rule_uuid == R_TAIL).values(last_sampled_at=_now() - timedelta(days=2)))
|
||||
await db.commit()
|
||||
for uuid, reason in ((R_UNUSED, "managed"), (R_LEGACY, "legacy"), (R_TAIL, "confidence")):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[uuid], acknowledge=[uuid])
|
||||
assert got["params"]["reason"] == reason, (uuid, got)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probable_needs_an_explicit_acknowledgement_per_rule(env):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_BLOCK, R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT" and got["params"]["uuids"] == R_BLOCK
|
||||
ok = await preview(env.host_id, [R_BLOCK, R_UNUSED], acknowledge=[R_BLOCK], ack_text_version="1")
|
||||
item = next(i for i in ok["plan"]["nodes"][0]["items"] if i["uuid"] == R_BLOCK)
|
||||
assert item["level"] == "probable" and item["needs_ack"] and "UNUSED_BLOCK_RULE" in item["notes"]
|
||||
op = await _op(ok["preview_id"])
|
||||
assert op.options["acknowledged"] == [{"uuid": R_BLOCK, "text_version": "1"}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_short_window_and_external_change_cap_at_probable(env):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED], unused_days=14)
|
||||
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
|
||||
async with SessionLocal() as db:
|
||||
db.add(RuleChangeEvent(host_id=env.host_id, rule_uuid=R_UNUSED, seq=1, observed_at=_now() - timedelta(days=2),
|
||||
kind="changed", before={}, after={}, origin="external"))
|
||||
await db.commit()
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
|
||||
ok = await preview(env.host_id, [R_UNUSED], acknowledge=[R_UNUSED])
|
||||
assert "EXTERNAL_CHANGE" in ok["plan"]["nodes"][0]["items"][0]["notes"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rule_limit_and_selection_validation(env):
|
||||
await _settings(max_rules=2)
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED, R_DUP, R_TAIL])
|
||||
assert got["status"] == 422 and got["code"] == "BEREINIGUNG_ZU_VIELE_REGELN" and got["params"]["max"] == 2
|
||||
for uuids in ([], ["../apply"]):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=uuids)
|
||||
assert got["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG"
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="wipe", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_maintenance_open_deployments_and_open_cleanups_block(env):
|
||||
from app.models.rule import FirewallRule
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
async with SessionLocal() as db:
|
||||
op = HostOperation(host_id=env.host_id, host_name="fw-a", actor="admin", action="reboot", status="submitted",
|
||||
expires_at=_now(), fingerprint="x", snapshot={})
|
||||
db.add(op)
|
||||
await db.commit()
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(HostOperation).values(status="done"))
|
||||
managed = FirewallRule(description="x", interface="lan", action="pass", direction="in", protocol="tcp",
|
||||
source_net="any", destination_net="any")
|
||||
db.add(managed)
|
||||
await db.flush()
|
||||
dep = RuleDeployment(rule_id=managed.id, host_id=env.host_id, status="pending", opnsense_rule_uuid=R_TAIL)
|
||||
db.add(dep)
|
||||
await db.commit()
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" and got["params"]["reason"] == "deployment"
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleDeployment).values(status="success"))
|
||||
await db.commit()
|
||||
first = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, first["preview_id"])
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_DUP])
|
||||
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" and got["params"]["reason"] == "cleanup"
|
||||
async with SessionLocal() as db:
|
||||
assert await svc.has_open_cleanup(db, [env.host_id])
|
||||
|
||||
|
||||
# ── Deploy ────────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_deploy_switches_off_samples_before_apply_and_waits_for_confirmation(env):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
env.fw.calls.clear()
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
assert result["status"] == "awaiting_confirm" and result["can_confirm"]
|
||||
writes = env.fw.writes()
|
||||
assert sorted(writes[:2]) == sorted([("POST", f"/firewall/filter/toggleRule/{R_DUP}/0"),
|
||||
("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/0")])
|
||||
assert writes[2:] == [("POST", "/firewall/filter/apply")] # one apply per node, after the toggles
|
||||
apply_at = env.fw.calls.index(("POST", "/firewall/filter/apply"))
|
||||
last_toggle = max(i for i, c in enumerate(env.fw.calls) if "toggleRule" in c[1])
|
||||
assert ("GET", "/firewall/filter_util/ruleStats") in env.fw.calls[last_toggle:apply_at] # sampled before the apply
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "0" and env.fw.get(R_DUP)["enabled"] == "0"
|
||||
op = await _op(p["preview_id"])
|
||||
window = timedelta(seconds=settings.deploy_confirm_window_seconds)
|
||||
assert window - timedelta(seconds=10) < op.revert_deadline - _now() <= window
|
||||
items = await _items(p["preview_id"])
|
||||
assert {i.state for i in items.values()} == {"disabled"}
|
||||
assert items[R_UNUSED].fp_before and items[R_UNUSED].row_before["enabled"] == "1"
|
||||
async with SessionLocal() as db:
|
||||
audits = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "rule_cleanup", AuditEvent.action == "disable"))).all()
|
||||
assert {(a.details["remote_uuid"], a.status) for a in audits} == {(R_UNUSED, "accepted"), (R_DUP, "accepted")}
|
||||
assert all(a.host_id == str(env.host_id) and a.details["preview_fingerprint"] == op.preview_fingerprint for a in audits)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rule_history_shows_the_cleanup_as_opnexus(env):
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, env.host_id)
|
||||
await rhs.observe_rows(db, host, {"rows": env.fw.rules}) # baseline
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, env.host_id)
|
||||
await rhs.observe_rows(db, host, {"rows": env.fw.rules})
|
||||
event = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.rule_uuid == R_UNUSED))).one()
|
||||
assert event.kind == "disabled" and event.origin == "opnexus" and event.audit_id is not None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("change", ["edited", "moved", "cover_edited", "hit"])
|
||||
async def test_deploy_refuses_a_stale_preview_without_writing(env, change):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
if change == "edited":
|
||||
env.fw.get(R_UNUSED)["destination_port"] = "8082"
|
||||
elif change == "moved":
|
||||
env.fw.get(R_UNUSED)["sort_order"] = "400000.0000301"
|
||||
elif change == "cover_edited":
|
||||
env.fw.get(R_BASE)["description"] = "base changed"
|
||||
else: # a hit since the preview: the fresh sample at deploy sees the counter grow
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleHit).where(RuleHit.rule_uuid == R_UNUSED).values(last_hit_at=_now()))
|
||||
await db.commit()
|
||||
env.fw.calls.clear()
|
||||
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(p["preview_id"]), actor="admin", confirm_name="fw-a")
|
||||
expected = R_DUP if change == "cover_edited" else R_UNUSED
|
||||
if change == "hit":
|
||||
assert got["code"] == "BEREINIGUNG_REGEL_NICHT_AENDERBAR" and got["params"]["uuid"] == R_UNUSED
|
||||
else:
|
||||
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET" and got["params"]["uuids"] == expected
|
||||
assert env.fw.writes() == []
|
||||
assert (await _op(p["preview_id"])).status == "expired"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_preview_token_is_single_use_bound_to_actor_and_name(env):
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
pid = uuid_lib.UUID(p["preview_id"])
|
||||
got = await refused(env.host_id, svc.deploy, pid, actor="someone-else", confirm_name="fw-a")
|
||||
assert got["status"] == 404
|
||||
got = await refused(env.host_id, svc.deploy, pid, actor="admin", confirm_name="FW-A")
|
||||
assert got["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
got = await refused(env.host_id, svc.deploy, pid, actor="admin", confirm_name="fw-a")
|
||||
assert got["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
|
||||
# an expired preview
|
||||
await call(env.host_id, svc.revert_now, pid)
|
||||
q = await preview(env.host_id, [R_DUP])
|
||||
await _backdate(q["preview_id"], expires_at=_now() - timedelta(seconds=1))
|
||||
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(q["preview_id"]), actor="admin", confirm_name="fw-a")
|
||||
assert got["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_refused_toggle_stops_writing_but_applies_what_is_switched(env):
|
||||
uuids = [R_DUP, R_UNUSED, R_BLOCK, R_TAIL]
|
||||
p = await preview(env.host_id, uuids, acknowledge=[R_BLOCK])
|
||||
env.fw.fail[("toggle", R_BLOCK)] = httpx.Response(200, json={"result": "failed"})
|
||||
env.fw.calls.clear()
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
states = {i.rule_uuid: i.state for i in (await _items(p["preview_id"])).values()}
|
||||
assert states == {R_DUP: "disabled", R_UNUSED: "disabled", R_BLOCK: "failed", R_TAIL: "queued"}
|
||||
assert ("POST", f"/firewall/filter/toggleRule/{R_TAIL}/0") not in env.fw.calls # nothing after the error
|
||||
assert env.fw.writes()[-1] == ("POST", "/firewall/filter/apply") # but the switched ones are applied
|
||||
assert result["status"] == "partial" and not result["can_confirm"] and result["can_revert"]
|
||||
assert (await _op(p["preview_id"])).revert_deadline > _now()
|
||||
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
assert got["code"] == "BEREINIGUNG_NICHT_BESTAETIGBAR"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("applied,readable,state", [(True, True, "disabled"), (False, True, "failed"), (False, False, "unknown")])
|
||||
async def test_unclear_toggle_is_settled_by_reading(env, applied, readable, state):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
items_order = [i["uuid"] for i in p["plan"]["nodes"][0]["items"]]
|
||||
first = items_order[0]
|
||||
env.fw.fail[("toggle", first)] = httpx.ReadTimeout
|
||||
env.fw.fail[("toggle_applies", first)] = applied
|
||||
if not readable:
|
||||
env.fw.fail[("get", first)] = httpx.ConnectError
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
items = await _items(p["preview_id"])
|
||||
assert items[first].state == state
|
||||
# no further write after an unclear answer (never attempted: "skipped" once nothing changed at all)
|
||||
assert items[items_order[1]].state == ("skipped" if state == "failed" else "queued")
|
||||
expect_apply = state in ("disabled", "unknown")
|
||||
assert (("POST", "/firewall/filter/apply") in env.fw.calls) is expect_apply
|
||||
assert result["status"] == {"disabled": "partial", "failed": "failed", "unknown": "partial"}[state]
|
||||
assert not result["can_confirm"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_failed_apply_keeps_the_deadline_and_the_sweeper_reverts(env):
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
env.fw.fail[("apply", "")] = [httpx.Response(500, json={}), None]
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
assert result["status"] == "apply_failed" and result["revert_deadline"]
|
||||
await _sweep()
|
||||
assert (await _op(p["preview_id"])).status == "apply_failed" # deadline not reached yet
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
|
||||
env.fw.calls.clear()
|
||||
await _sweep()
|
||||
assert (await _op(p["preview_id"])).status == "reverted"
|
||||
assert env.fw.writes() == [("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/1"), ("POST", "/firewall/filter/apply")]
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "1"
|
||||
|
||||
|
||||
# ── Confirm ───────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_confirm_reads_the_firewall_and_refuses_when_unreachable_or_different(env):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
pid = uuid_lib.UUID(p["preview_id"])
|
||||
env.fw.down = True
|
||||
got = await refused(env.host_id, svc.confirm, pid)
|
||||
assert got["status"] == 502 and got["code"] == "BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR"
|
||||
assert (await _op(pid)).status == "awaiting_confirm" # stays open: the sweeper reverts it
|
||||
env.fw.down = False
|
||||
env.fw.get(R_DUP)["enabled"] = "1" # someone switched it back on in the GUI
|
||||
got = await refused(env.host_id, svc.confirm, pid)
|
||||
assert got["code"] == "BEREINIGUNG_LIVESTAND_ABWEICHEND" and got["params"]["uuids"] == R_DUP
|
||||
env.fw.get(R_DUP)["enabled"] = "0"
|
||||
env.fw.calls.clear()
|
||||
result = await call(env.host_id, svc.confirm, pid)
|
||||
assert result["status"] == "confirmed" and {i["state"] for i in result["items"]} == {"confirmed"}
|
||||
assert env.fw.writes() == [] # confirming only reads
|
||||
op = await _op(pid)
|
||||
assert op.revert_deadline is None and op.confirmed_at is not None
|
||||
assert await _sweep() == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_confirm_after_the_deadline_is_refused_and_the_sweeper_reverts(env):
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
|
||||
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
assert got["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
|
||||
assert len(await _sweep()) == 1
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "1"
|
||||
assert {i.state for i in (await _items(p["preview_id"])).values()} == {"reverted"}
|
||||
async with SessionLocal() as db:
|
||||
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "revert"))).one()
|
||||
assert audit.actor == "system:rollback" and audit.details["remote_uuid"] == R_UNUSED
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_revert_now_and_failed_revert_is_not_retried_automatically(env):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
|
||||
result = await call(env.host_id, svc.revert_now, uuid_lib.UUID(p["preview_id"]))
|
||||
assert result["status"] == "revert_failed" and result["can_retry"]
|
||||
states = {i["rule_uuid"]: i["state"] for i in result["items"]}
|
||||
assert states == {R_UNUSED: "reverted", R_DUP: "revert_failed"} # the others are still reverted
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "1" and env.fw.writes()[-1] == ("POST", "/firewall/filter/apply")
|
||||
env.fw.calls.clear()
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(minutes=5))
|
||||
await _sweep()
|
||||
assert env.fw.writes() == [] # never retried by the sweeper
|
||||
del env.fw.fail[("toggle", R_DUP)]
|
||||
result = await call(env.host_id, svc.retry, uuid_lib.UUID(p["preview_id"]))
|
||||
assert result["status"] == "reverted" and env.fw.get(R_DUP)["enabled"] == "1"
|
||||
|
||||
|
||||
# ── Sweeper and two backends ──────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sweeper_skips_a_live_run_and_takes_over_an_abandoned_one(env):
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
# simulate a backend that died mid-run: running, item half-way
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleCleanupOperation).values(status="running", heartbeat_at=_now(),
|
||||
revert_deadline=_now() + timedelta(minutes=10)))
|
||||
await db.execute(update(RuleCleanupItem).values(state="disabling"))
|
||||
await db.commit()
|
||||
env.fw.calls.clear()
|
||||
assert await _sweep() == [] # (b) fresh heartbeat: someone is still working on it
|
||||
assert env.fw.writes() == []
|
||||
await _backdate(p["preview_id"], heartbeat_at=_now() - timedelta(hours=1))
|
||||
assert await _sweep() == [p["preview_id"]] # (c) stale heartbeat: taken over and reverted
|
||||
assert env.fw.writes() == [("POST", f"/firewall/filter/toggleRule/{R_UNUSED}/1"), ("POST", "/firewall/filter/apply")]
|
||||
assert (await _op(p["preview_id"])).status == "reverted"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_two_deploys_of_one_preview_write_once(env):
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
outcomes = await asyncio.gather(deploy(env.host_id, p["preview_id"]), deploy(env.host_id, p["preview_id"]),
|
||||
return_exceptions=True)
|
||||
errors = [o for o in outcomes if isinstance(o, HTTPException)]
|
||||
assert len(errors) == 1 and errors[0].detail["code"] == "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
|
||||
assert sum(1 for c in env.fw.calls if "toggleRule" in c[1]) == 2 # (a) each rule exactly once
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_second_operation_on_the_same_rule_is_refused(env):
|
||||
a = await preview(env.host_id, [R_UNUSED], actor="admin")
|
||||
b = await preview(env.host_id, [R_UNUSED], actor="admin2")
|
||||
await deploy(env.host_id, a["preview_id"])
|
||||
got = await refused(env.host_id, svc.deploy, uuid_lib.UUID(b["preview_id"]), actor="admin2", confirm_name="fw-a")
|
||||
assert got["code"] == "BEREINIGUNG_OFFENE_OPERATION" # (d)
|
||||
# and the partial unique index holds even without the service's checks
|
||||
async with SessionLocal() as db:
|
||||
op = await db.get(RuleCleanupOperation, uuid_lib.UUID(b["preview_id"]))
|
||||
db.add(RuleCleanupItem(operation_id=op.id, host_id=env.host_id, rule_uuid=R_UNUSED, state="queued", description=""))
|
||||
with pytest.raises(Exception):
|
||||
await db.commit()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_confirm_and_sweeper_at_the_deadline_decide_once(env):
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() + timedelta(milliseconds=300))
|
||||
|
||||
async def late_confirm():
|
||||
await asyncio.sleep(0.4)
|
||||
return await call(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
|
||||
async def late_sweep():
|
||||
await asyncio.sleep(0.4)
|
||||
return await _sweep()
|
||||
|
||||
outcomes = await asyncio.gather(late_confirm(), late_sweep(), return_exceptions=True)
|
||||
op = await _op(p["preview_id"])
|
||||
assert op.status == "reverted" # (e) the deadline passed: confirm refused, exactly one revert
|
||||
assert isinstance(outcomes[0], HTTPException) and outcomes[0].detail["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
|
||||
assert sum(1 for c in env.fw.calls if c[1].endswith(f"{R_UNUSED}/1")) == 1
|
||||
|
||||
|
||||
# ── Cluster ───────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_writes_backup_first_and_reverts_master_first(cluster):
|
||||
p = await preview(cluster.a, [R_UNUSED])
|
||||
assert [n["role"] for n in p["plan"]["nodes"]] == ["BACKUP", "MASTER"]
|
||||
await deploy(cluster.a, p["preview_id"])
|
||||
writes = _writes(cluster.log)
|
||||
assert writes == [(FW2, f"/firewall/filter/toggleRule/{R_UNUSED}/0"), (FW2, "/firewall/filter/apply"),
|
||||
(FW1, f"/firewall/filter/toggleRule/{R_UNUSED}/0"), (FW1, "/firewall/filter/apply")]
|
||||
items = await _items(p["preview_id"])
|
||||
assert len(items) == 1 # (same rule_uuid on both nodes: keyed by uuid here)
|
||||
async with SessionLocal() as db:
|
||||
roles = {i.host_id: i.carp_role for i in (await db.scalars(select(RuleCleanupItem))).all()}
|
||||
assert roles == {cluster.a: "MASTER", cluster.b: "BACKUP"}
|
||||
cluster.log.clear()
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
|
||||
await _sweep()
|
||||
assert _writes(cluster.log) == [(FW1, f"/firewall/filter/toggleRule/{R_UNUSED}/1"), (FW1, "/firewall/filter/apply"),
|
||||
(FW2, f"/firewall/filter/toggleRule/{R_UNUSED}/1"), (FW2, "/firewall/filter/apply")]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_master_failure_leaves_backup_switched_and_unconfirmable(cluster):
|
||||
p = await preview(cluster.a, [R_UNUSED])
|
||||
cluster.master.fail[("toggle", R_UNUSED)] = httpx.Response(200, json={"result": "failed"})
|
||||
result = await deploy(cluster.a, p["preview_id"])
|
||||
assert result["status"] == "partial" and not result["can_confirm"]
|
||||
assert cluster.backup.get(R_UNUSED)["enabled"] == "0" and cluster.master.get(R_UNUSED)["enabled"] == "1"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_refuses_missing_uuid_offline_peer_and_two_masters(cluster):
|
||||
cluster.backup.rules = [r for r in cluster.backup.rules if r["uuid"] != R_UNUSED]
|
||||
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG" and got["params"]["reason"] == "uuid_missing"
|
||||
cluster.backup.rules = default_rules()
|
||||
cluster.backup.role = "MASTER"
|
||||
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG" and got["params"]["reason"] == "roles"
|
||||
cluster.backup.role = "BACKUP"
|
||||
cluster.backup.down = True
|
||||
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["status"] == 502 and got["params"]["host"] == "fw-b"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_usage_is_judged_over_all_nodes(cluster):
|
||||
# case 16: the backup saw no hit, the master did: not unused
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleHit).where(RuleHit.host_id == cluster.a, RuleHit.rule_uuid == R_UNUSED)
|
||||
.values(last_hit_at=_now() - timedelta(days=1)))
|
||||
await db.commit()
|
||||
got = await refused(cluster.b, svc.preview, actor="admin", kind="disable", uuids=[R_UNUSED])
|
||||
assert got["params"]["reason"] == "not_candidate"
|
||||
# no usage data on one node: at best probable
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(RuleHit.__table__.delete().where(RuleHit.host_id == cluster.a, RuleHit.rule_uuid == R_TAIL))
|
||||
await db.commit()
|
||||
got = await refused(cluster.b, svc.preview, actor="admin", kind="disable", uuids=[R_TAIL])
|
||||
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_pfsync_interface_rules_are_infrastructure(cluster):
|
||||
# "any" from the LAN net on the pfsync interface (LAN) would also carry pfsync
|
||||
for fw in (cluster.master, cluster.backup):
|
||||
fw.rules.append(rule("aaaaaaaa-0000-4000-8000-000000000900", 900, proto="any", src="10.20.30.0/24", description="lan any"))
|
||||
await _seed_hits(cluster.a, ["aaaaaaaa-0000-4000-8000-000000000900"])
|
||||
await _seed_hits(cluster.b, ["aaaaaaaa-0000-4000-8000-000000000900"])
|
||||
got = await refused(cluster.a, svc.preview, actor="admin", kind="disable", uuids=["aaaaaaaa-0000-4000-8000-000000000900"])
|
||||
assert got["params"]["reason"] == "infrastructure"
|
||||
|
||||
|
||||
# ── Delete and restore ────────────────────────────────────────────────────
|
||||
|
||||
async def _confirmed_disable(host_id, uuids, *, days_ago=8):
|
||||
p = await preview(host_id, uuids)
|
||||
await deploy(host_id, p["preview_id"])
|
||||
await call(host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=days_ago))
|
||||
return p["preview_id"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_needs_the_waiting_time_an_unchanged_rule_and_no_external_change(env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED], days_ago=2)
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT" and got["params"]["days"] == 5
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleCleanupOperation).values(confirmed_at=_now() - timedelta(days=8)))
|
||||
await db.commit()
|
||||
ok = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
assert ok["plan"]["nodes"][0]["items"][0]["basis"]["type"] == "confirmed"
|
||||
env.fw.get(R_UNUSED)["destination_port"] = "8099"
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET"
|
||||
env.fw.get(R_UNUSED)["destination_port"] = "8081"
|
||||
async with SessionLocal() as db:
|
||||
db.add(RuleChangeEvent(host_id=env.host_id, rule_uuid=R_UNUSED, seq=5, observed_at=_now() - timedelta(days=1),
|
||||
kind="changed", before={}, after={}, origin="external"))
|
||||
await db.commit()
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED])
|
||||
assert got["code"] == "BEREINIGUNG_VORSCHAU_VERALTET"
|
||||
env.fw.get(R_DUP)["enabled"] = "1"
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="delete", uuids=[R_DUP])
|
||||
assert got["params"]["reason"] == "not_candidate" # an enabled rule is never deleted
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_commits_the_snapshot_before_delrule(env, monkeypatch):
|
||||
from app.services import opnsense_client as client_mod
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
seen: list = []
|
||||
real_del = client_mod.del_rule
|
||||
|
||||
async def spy(host, uuid): # what another session sees in the database at the moment delRule goes out
|
||||
async with SessionLocal() as db:
|
||||
item = (await db.scalars(select(RuleCleanupItem).where(
|
||||
RuleCleanupItem.operation_id == uuid_lib.UUID(p["preview_id"])))).one()
|
||||
seen.append((item.state, bool(item.snapshot), item.anchor_next_uuid))
|
||||
return await real_del(host, uuid)
|
||||
monkeypatch.setattr(client_mod, "del_rule", spy)
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
assert seen == [("deleting", True, R_BLOCK)] # durable before the remote delete
|
||||
assert result["status"] == "done"
|
||||
item = next(iter((await _items(p["preview_id"])).values()))
|
||||
assert item.state == "deleted" and item.snapshot["rule"]["destination_port"] == "8081"
|
||||
assert item.snapshot["rule"]["action"] == "pass" and "uuid" not in item.snapshot["rule"]
|
||||
assert item.prio_group == "400000" and item.anchor_prev_uuid == R_DUP
|
||||
assert env.fw.get(R_UNUSED) is None and env.fw.writes()[-1] == ("POST", "/firewall/filter/apply")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("applied,state", [(True, "deleted"), (False, "failed")])
|
||||
async def test_unclear_delete_is_settled_with_getrule(env, applied, state):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
env.fw.fail[("delete", R_UNUSED)] = httpx.ReadTimeout
|
||||
env.fw.fail[("delete_applies", R_UNUSED)] = applied
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
assert next(iter((await _items(p["preview_id"])).values())).state == state
|
||||
assert result["status"] == ("done" if state == "deleted" else "failed")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_adds_disabled_at_the_anchor_with_a_new_uuid(env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
deleted = next(iter((await _items(p["preview_id"])).values()))
|
||||
env.fw.calls.clear()
|
||||
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["status"] == "done" and result["warnings"] == []
|
||||
item = result["items"][0]
|
||||
new = item["restored_uuid"]
|
||||
assert new and new != R_UNUSED and item["state"] == "restored"
|
||||
restored = env.fw.get(new)
|
||||
assert restored["enabled"] == "0" and restored["destination_port"] == "8081"
|
||||
assert ("POST", f"/firewall/filter/moveRuleBefore/{new}/{R_BLOCK}") in env.fw.calls
|
||||
order = [r["uuid"] for r in sorted(env.fw.rules, key=lambda r: r["sort_order"]) if not r.get("is_automatic")]
|
||||
assert order.index(new) == order.index(R_BLOCK) - 1 and order.index(new) == order.index(R_DUP) + 1
|
||||
async with SessionLocal() as db:
|
||||
audit = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "restore"))).one()
|
||||
source = await db.get(RuleCleanupItem, deleted.id)
|
||||
assert audit.details["remote_uuid"] == new and audit.details["restored_from"] == R_UNUSED
|
||||
assert source.restored_uuid == new
|
||||
got = await refused(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert got["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH" # restored once only
|
||||
# re-enabling the restored rule is an operation with a window
|
||||
e = await preview(env.host_id, [new], kind="enable")
|
||||
env.fw.calls.clear()
|
||||
r = await deploy(env.host_id, e["preview_id"])
|
||||
assert r["status"] == "awaiting_confirm" and ("POST", f"/firewall/filter/toggleRule/{new}/1") in env.fw.calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_without_anchor_warns_and_keeps_the_rule(env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
deleted = next(iter((await _items(p["preview_id"])).values()))
|
||||
env.fw.rules = [r for r in env.fw.rules if r["uuid"] != R_BLOCK] # the anchor is gone meanwhile
|
||||
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["status"] == "done" and result["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
|
||||
assert result["items"][0]["message"] == "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
|
||||
assert env.fw.get(result["items"][0]["restored_uuid"])["enabled"] == "0"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_refused_by_the_firewall_is_failed_with_its_message(env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
deleted = next(iter((await _items(p["preview_id"])).values()))
|
||||
env.fw.fail[("add", "")] = httpx.Response(200, json={"result": "failed", "validations": {"rule.source_net": "alias missing"}})
|
||||
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["status"] == "failed" and "alias missing" in result["items"][0]["message"]
|
||||
async with SessionLocal() as db:
|
||||
assert (await db.get(RuleCleanupItem, deleted.id)).restored_uuid is None # can be tried again
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_delete_restore_and_reenable_follow_the_restore_link(cluster):
|
||||
p = await preview(cluster.a, [R_UNUSED])
|
||||
await deploy(cluster.a, p["preview_id"])
|
||||
await call(cluster.a, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=8))
|
||||
d = await preview(cluster.a, [R_UNUSED], kind="delete")
|
||||
cluster.log.clear()
|
||||
assert (await deploy(cluster.a, d["preview_id"]))["status"] == "done"
|
||||
assert [w for w in _writes(cluster.log) if "delRule" in w[1]] == [(FW2, f"/firewall/filter/delRule/{R_UNUSED}"),
|
||||
(FW1, f"/firewall/filter/delRule/{R_UNUSED}")]
|
||||
async with SessionLocal() as db:
|
||||
deleted = (await db.scalars(select(RuleCleanupItem).where(
|
||||
RuleCleanupItem.operation_id == uuid_lib.UUID(d["preview_id"]), RuleCleanupItem.host_id == cluster.a))).one()
|
||||
restored = await call(cluster.a, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert restored["status"] == "done" and len(restored["items"]) == 2 # both nodes, backup first
|
||||
new = {i["host_id"]: i["restored_uuid"] for i in restored["items"]}
|
||||
assert new[str(cluster.a)] != new[str(cluster.b)] # addRule: a new UUID on every node
|
||||
assert cluster.backup.get(new[str(cluster.b)])["enabled"] == "0" and cluster.master.get(new[str(cluster.a)])["enabled"] == "0"
|
||||
# re-enable through the master's UUID: the backup's copy is found through the restore operation
|
||||
e = await preview(cluster.a, [new[str(cluster.a)]], kind="enable")
|
||||
assert {n["host_name"]: n["items"][0]["uuid"] for n in e["plan"]["nodes"]} == {"fw-a": new[str(cluster.a)], "fw-b": new[str(cluster.b)]}
|
||||
cluster.log.clear()
|
||||
r = await deploy(cluster.a, e["preview_id"])
|
||||
assert r["status"] == "awaiting_confirm"
|
||||
assert [w for w in _writes(cluster.log) if "toggleRule" in w[1]] == [
|
||||
(FW2, f"/firewall/filter/toggleRule/{new[str(cluster.b)]}/1"), (FW1, f"/firewall/filter/toggleRule/{new[str(cluster.a)]}/1")]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_with_a_refused_move_is_verified_by_reading_and_only_warns(env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
deleted = next(iter((await _items(p["preview_id"])).values()))
|
||||
# moveRuleBefore answers HTTP 500 (an Ambiguous error for the client, P3): the position is read back
|
||||
env.fw.fail[("move", "*")] = httpx.Response(500, json={"errorMessage": "Unexpected error, check log for details"})
|
||||
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["status"] == "done" and result["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_deadline_passing_while_confirm_reads_the_firewall_refuses(env):
|
||||
import time
|
||||
p = await preview(env.host_id, [R_UNUSED])
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() + timedelta(milliseconds=300))
|
||||
env.fw.fail[("search", "")] = lambda: time.sleep(0.6) # a slow firewall: the window closes during the read
|
||||
got = await refused(env.host_id, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
assert got["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
|
||||
assert (await _op(p["preview_id"])).status == "awaiting_confirm" # left to the sweeper
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sweeper_leaves_a_live_run_alone_even_past_its_deadline(env, monkeypatch): # review finding 2
|
||||
from app.services import opnsense_client as client_mod
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
real = client_mod.toggle_rule
|
||||
swept: list = []
|
||||
|
||||
async def toggle(host, uuid, enabled): # a second backend's sweeper starts while the first toggle is on the wire
|
||||
if not swept:
|
||||
await _backdate(p["preview_id"], revert_deadline=_now() - timedelta(seconds=1))
|
||||
swept.append(asyncio.create_task(_sweep())) # it gets the lock in the deploy's next checkpoint gap
|
||||
await asyncio.sleep(0.05)
|
||||
return await real(host, uuid, enabled)
|
||||
monkeypatch.setattr(client_mod, "toggle_rule", toggle)
|
||||
result = await deploy(env.host_id, p["preview_id"])
|
||||
assert await swept[0] == [] # fresh heartbeat: not taken over although the deadline passed
|
||||
assert result["status"] == "awaiting_confirm"
|
||||
assert {i.state for i in (await _items(p["preview_id"])).values()} == {"disabled"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_run_taken_over_by_another_backend_writes_nothing_more(env, monkeypatch): # review finding 2
|
||||
from app.services import opnsense_client as client_mod
|
||||
p = await preview(env.host_id, [R_UNUSED, R_DUP])
|
||||
real = client_mod.toggle_rule
|
||||
calls: list = []
|
||||
|
||||
async def toggle(host, uuid, enabled):
|
||||
calls.append(uuid)
|
||||
if len(calls) == 1: # another backend took the operation over while this toggle was on the wire
|
||||
await _backdate(p["preview_id"], status="reverting")
|
||||
return await real(host, uuid, enabled)
|
||||
monkeypatch.setattr(client_mod, "toggle_rule", toggle)
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
assert len(calls) == 1 # the conditional heartbeat before the second toggle found the operation gone
|
||||
assert (await _op(p["preview_id"])).status == "reverting" # left to the new owner, not overwritten
|
||||
states = sorted(i.state for i in (await _items(p["preview_id"])).values())
|
||||
assert states == ["disabled", "queued"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_extra_management_ports_protect_rules_on_them(env): # review finding 3
|
||||
custom = "aaaaaaaa-0000-4000-8000-000000000910"
|
||||
env.fw.rules.append(rule(custom, 910, interface="wan", src="203.0.113.0/24", dport="8443", description="gui 8443"))
|
||||
await _seed_hits(env.host_id, [custom])
|
||||
ok = await preview(env.host_id, [custom]) # 8443 is not one of the fixed ports: not about management
|
||||
assert ok["plan"]["ports"] == []
|
||||
await _settings(ports=(8443,))
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[custom])
|
||||
assert got["params"]["reason"] == "management"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cluster_peer_may_confirm_and_read_but_strangers_may_not(cluster, client, respx_mock): # review finding 4
|
||||
p = await preview(cluster.a, [R_UNUSED])
|
||||
await deploy(cluster.a, p["preview_id"])
|
||||
pid = uuid_lib.UUID(p["preview_id"])
|
||||
assert (await call(cluster.b, svc.get_operation, pid))["id"] == p["preview_id"]
|
||||
FakeFirewall("https://203.0.113.59").mount(respx_mock)
|
||||
other = uuid_lib.UUID((await _create_host(client, "fw-c", "https://203.0.113.59"))["id"])
|
||||
for fn in (svc.get_operation, svc.confirm, svc.revert_now, svc.retry):
|
||||
assert (await refused(other, fn, pid))["status"] == 404
|
||||
got = await refused(cluster.b, svc.cancel, pid, actor="admin") # previews/deploys stay bound to the anchor
|
||||
assert got["status"] == 404
|
||||
result = await call(cluster.b, svc.confirm, pid)
|
||||
assert result["status"] == "confirmed"
|
||||
|
||||
|
||||
async def _half_deleted_cluster(cluster):
|
||||
p = await preview(cluster.a, [R_UNUSED])
|
||||
await deploy(cluster.a, p["preview_id"])
|
||||
await call(cluster.a, svc.confirm, uuid_lib.UUID(p["preview_id"]))
|
||||
await _backdate(p["preview_id"], confirmed_at=_now() - timedelta(days=8))
|
||||
d = await preview(cluster.a, [R_UNUSED], kind="delete")
|
||||
cluster.master.fail[("delete", R_UNUSED)] = httpx.Response(200, json={"result": "failed"})
|
||||
result = await deploy(cluster.a, d["preview_id"])
|
||||
assert result["status"] == "failed"
|
||||
assert cluster.backup.get(R_UNUSED) is None and cluster.master.get(R_UNUSED) is not None
|
||||
del cluster.master.fail[("delete", R_UNUSED)]
|
||||
return d["preview_id"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_half_done_cluster_delete_can_be_finished(cluster): # review finding 5
|
||||
await _half_deleted_cluster(cluster)
|
||||
again = await preview(cluster.a, [R_UNUSED], kind="delete")
|
||||
assert [(n["host_name"], len(n["items"])) for n in again["plan"]["nodes"]] == [("fw-b", 0), ("fw-a", 1)]
|
||||
cluster.log.clear()
|
||||
assert (await deploy(cluster.a, again["preview_id"]))["status"] == "done"
|
||||
assert [w for w in _writes(cluster.log) if "delRule" in w[1]] == [(FW1, f"/firewall/filter/delRule/{R_UNUSED}")]
|
||||
assert cluster.master.get(R_UNUSED) is None
|
||||
got = await refused(cluster.a, svc.preview, actor="admin", kind="delete", uuids=[R_UNUSED]) # nothing left
|
||||
assert got["params"]["reason"] == "not_candidate"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_half_done_cluster_delete_can_be_restored_where_it_was_deleted(cluster): # review finding 5
|
||||
delete_op = await _half_deleted_cluster(cluster)
|
||||
async with SessionLocal() as db:
|
||||
deleted = (await db.scalars(select(RuleCleanupItem).where(
|
||||
RuleCleanupItem.operation_id == uuid_lib.UUID(delete_op), RuleCleanupItem.state == "deleted"))).one()
|
||||
result = await call(cluster.a, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["status"] == "done" and [i["host_id"] for i in result["items"]] == [str(cluster.b)]
|
||||
assert cluster.backup.get(result["items"][0]["restored_uuid"])["enabled"] == "0"
|
||||
assert cluster.master.get(R_UNUSED)["enabled"] == "0" # the master's copy was never deleted: untouched
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_switching_a_block_rule_back_on_needs_an_acknowledgement_unless_proven_harmless(env): # review finding 7
|
||||
lock_out = "aaaaaaaa-0000-4000-8000-000000000920"
|
||||
env.fw.rules.append(rule(lock_out, 920, action="block", interface="wan", src="203.0.113.0/24", dport="443",
|
||||
enabled="0", description="block mgmt"))
|
||||
unknown = "aaaaaaaa-0000-4000-8000-000000000940" # source is a URL table: match unknown
|
||||
env.fw.rules.append(rule(unknown, 940, action="block", interface="wan", src="tbl", dport="443", enabled="0",
|
||||
description="block table"))
|
||||
harmless = "aaaaaaaa-0000-4000-8000-000000000930"
|
||||
env.fw.rules.append(rule(harmless, 930, action="block", proto="UDP", dport="5353", enabled="0", description="block mdns"))
|
||||
async with SessionLocal() as db: # both were switched off by a confirmed cleanup earlier
|
||||
op = RuleCleanupOperation(host_id=env.host_id, kind="disable", status="confirmed", actor="admin",
|
||||
expires_at=_now(), preview_fingerprint="x", options={}, confirmed_at=_now())
|
||||
db.add(op)
|
||||
await db.flush()
|
||||
for u in (lock_out, harmless, unknown):
|
||||
db.add(RuleCleanupItem(operation_id=op.id, host_id=env.host_id, rule_uuid=u, state="confirmed", description=u))
|
||||
await db.commit()
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="enable", uuids=[lock_out, harmless, unknown])
|
||||
assert got["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT" and got["params"]["uuids"] == ", ".join(sorted([lock_out, unknown]))
|
||||
ok = await preview(env.host_id, [lock_out, harmless, unknown], kind="enable", acknowledge=[lock_out, unknown])
|
||||
items = {i["uuid"]: i for i in ok["plan"]["nodes"][0]["items"]}
|
||||
assert items[lock_out]["needs_ack"] and items[lock_out]["notes"] == ["ENABLE_MAY_BLOCK_MANAGEMENT"]
|
||||
assert not items[harmless]["needs_ack"] and items[harmless]["protection"]["result"] == "false"
|
||||
|
||||
|
||||
async def _unknown_restore(env):
|
||||
"""A restore whose addRule answer was lost: the item stays unknown."""
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
p = await preview(env.host_id, [R_UNUSED], kind="delete")
|
||||
await deploy(env.host_id, p["preview_id"])
|
||||
deleted = next(iter((await _items(p["preview_id"])).values()))
|
||||
snapshot_row = dict(deleted.snapshot["row"])
|
||||
env.fw.fail[("add", "")] = httpx.ReadTimeout
|
||||
result = await call(env.host_id, svc.restore_item, deleted.id, actor="admin", confirm_name="fw-a")
|
||||
assert result["items"][0]["state"] == "unknown"
|
||||
del env.fw.fail[("add", "")]
|
||||
return result, snapshot_row
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unclear_restore_uses_the_stored_uuid_before_searching_by_content(env): # review finding 10
|
||||
result, row = await _unknown_restore(env)
|
||||
# on the firewall: the real copy and a content-identical twin someone made by hand
|
||||
real, twin = "cccccccc-0000-4000-8000-000000000001", "cccccccc-0000-4000-8000-000000000002"
|
||||
for u, seq in ((real, 950), (twin, 960)):
|
||||
env.fw.rules.append({**row, "uuid": u, "sequence": str(seq), "sort_order": f"400000.{seq:07d}"})
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleCleanupItem).where(RuleCleanupItem.id == uuid_lib.UUID(result["items"][0]["id"]))
|
||||
.values(restored_uuid=real))
|
||||
await db.commit()
|
||||
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"]))
|
||||
assert done["items"][0]["state"] == "restored" and done["items"][0]["restored_uuid"] == real
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unclear_restore_with_twins_and_no_stored_uuid_stays_unknown(env): # review finding 10
|
||||
result, row = await _unknown_restore(env)
|
||||
for u, seq in (("cccccccc-0000-4000-8000-000000000003", 950), ("cccccccc-0000-4000-8000-000000000004", 960)):
|
||||
env.fw.rules.append({**row, "uuid": u, "sequence": str(seq), "sort_order": f"400000.{seq:07d}"})
|
||||
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"]))
|
||||
assert done["items"][0]["state"] == "unknown" and done["items"][0]["message"] == "BEREINIGUNG_AUSGANG_UNKLAR"
|
||||
env.fw.rules = [r for r in env.fw.rules if not r["uuid"].startswith("cccccccc")]
|
||||
done = await call(env.host_id, svc.retry, uuid_lib.UUID(result["id"])) # nothing there: it was never created
|
||||
assert done["items"][0]["state"] == "failed"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_outdated_acknowledgement_text_is_refused(env): # review finding 11
|
||||
for version in ("0", "2", ""):
|
||||
got = await refused(env.host_id, svc.preview, actor="admin", kind="disable", uuids=[R_BLOCK],
|
||||
acknowledge=[R_BLOCK], ack_text_version=version)
|
||||
assert got["status"] == 409 and got["code"] == "BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET", version
|
||||
assert env.fw.calls == [] # refused before anything is read
|
||||
ok = await preview(env.host_id, [R_BLOCK], acknowledge=[R_BLOCK], ack_text_version=svc.ACK_TEXT_VERSION)
|
||||
assert ok["plan"]["acknowledged"] == [R_BLOCK]
|
||||
@@ -0,0 +1,910 @@
|
||||
"""Rule cleanup API (1.31, package 6): router, rights, locks in both directions, notification, settings.
|
||||
|
||||
Who may do what is enforced by TWO layers (AuthMiddleware via services/api_token_service.py, and the router itself).
|
||||
The role matrix therefore runs three times: with both layers, with the middleware's rights checks switched off (the
|
||||
router alone must refuse) and with the router's checks switched off (the middleware alone must refuse). A mutation
|
||||
of either layer is then caught by one of the variants (tests/mutations_cleanup_1_31_api.json).
|
||||
Addresses: documentation/private ranges only."""
|
||||
import uuid as uuid_lib
|
||||
from datetime import timedelta
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from app import auth_middleware
|
||||
from app.database import SessionLocal
|
||||
from app.models.audit import AuditEvent
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.host_operation import HostOperation
|
||||
from app.models.rule_cleanup import RuleCleanupItem, RuleCleanupOperation
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
from app.routers import rule_cleanup as router_mod
|
||||
from app.schemas.rule_cleanup import CleanupItemRead, CleanupOperationRead
|
||||
from app.services import api_token_service, notification_checks
|
||||
from app.services import rule_cleanup_service as svc
|
||||
from tests.rule_cleanup_fake import MGMT_SOURCE, R_ALIAS, R_BASE, R_BLOCK, R_DUP, R_MGMT, R_TAIL, R_UNUSED
|
||||
from tests.test_api_tokens import _bearer, _login, _make
|
||||
from tests.test_rule_cleanup import ( # noqa: F401 (fixtures are used by name)
|
||||
FW1, FW2, _backdate, _confirmed_disable, _fresh_cache, _items, _now, _op, _settings, cluster, env, refused,
|
||||
)
|
||||
|
||||
PW = "cleanup-api-test-pw-1"
|
||||
NOHOST = "00000000-0000-0000-0000-0000000000aa"
|
||||
NOID = "00000000-0000-0000-0000-0000000000bb"
|
||||
|
||||
|
||||
# ── Principals ────────────────────────────────────────────────────────────
|
||||
|
||||
async def _user(client, name: str, role: str):
|
||||
resp = await client.post("/api/users", json={"username": name, "password": PW, "role": role})
|
||||
assert resp.status_code in (201, 409), resp.text # the account survives between tests
|
||||
return await _login(name, PW)
|
||||
|
||||
|
||||
async def _token_of(session, name: str, scope: str):
|
||||
resp = await session.post("/api/tokens", json={"name": name, "scope": scope, "current_password": PW})
|
||||
assert resp.status_code == 201, resp.text
|
||||
return _bearer(resp.json()["token"])
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def who(client):
|
||||
"""Every kind of caller: sessions of an admin, an operator and a read-only account, and tokens of each scope
|
||||
(an admin's read/deploy/admin, an operator's deploy, a read-only account's read)."""
|
||||
operator = await _user(client, "cl-operator", "operator")
|
||||
read_only = await _user(client, "cl-readonly", "read_only")
|
||||
callers = {
|
||||
"admin": client, "operator": operator, "read_only": read_only,
|
||||
"tok_read": _bearer((await _make(client, "read", name="cl-read"))["token"]),
|
||||
"tok_deploy": _bearer((await _make(client, "deploy", name="cl-deploy"))["token"]),
|
||||
"tok_admin": _bearer((await _make(client, "admin", name="cl-admin"))["token"]),
|
||||
"tok_op_deploy": await _token_of(operator, "cl-op-deploy", "deploy"),
|
||||
"tok_ro_read": await _token_of(read_only, "cl-ro-read", "read"),
|
||||
}
|
||||
yield callers
|
||||
for name, session in callers.items():
|
||||
if name != "admin":
|
||||
await session.aclose()
|
||||
|
||||
|
||||
async def _governance(client, may_deploy: bool):
|
||||
assert (await client.put("/api/settings/governance", json={"operators_may_deploy": may_deploy})).status_code == 200
|
||||
|
||||
|
||||
# ── The role matrix ───────────────────────────────────────────────────────
|
||||
|
||||
ROUTES = {
|
||||
"list": ("GET", "", None),
|
||||
"get": ("GET", "/{op}", None),
|
||||
"preview": ("POST", "/preview", {"kind": "disable", "uuids": ["a"]}),
|
||||
"cancel": ("POST", "/{op}/cancel", None),
|
||||
"deploy": ("POST", "/{op}/deploy", {"confirm_name": "x"}),
|
||||
"confirm": ("POST", "/{op}/confirm", None),
|
||||
"revert": ("POST", "/{op}/revert", None),
|
||||
"retry": ("POST", "/{op}/retry", None),
|
||||
"restore": ("POST", "/items/{op}/restore", {"confirm_name": "x"}),
|
||||
}
|
||||
READ_ONLY_CODES = {"READ_ONLY_ACCESS", "API_TOKEN_SCOPE_FORBIDDEN"}
|
||||
|
||||
|
||||
def verdict(route: str, caller: str, may_deploy: bool):
|
||||
"""None: the caller passes the gate (the unknown host then answers 404). Otherwise the acceptable 403 codes."""
|
||||
if route in ("list", "get"):
|
||||
return None
|
||||
if caller in ("read_only", "tok_ro_read"):
|
||||
return READ_ONLY_CODES
|
||||
if caller == "tok_read":
|
||||
return {"API_TOKEN_SCOPE_FORBIDDEN"}
|
||||
if route in ("preview", "cancel"):
|
||||
return None
|
||||
if route == "confirm": # a human in a browser session; no token, whatever its scope
|
||||
return None if caller in ("admin", "operator") else {"API_TOKEN_SCOPE_FORBIDDEN"}
|
||||
if route in ("deploy", "retry", "restore"): # admin only, even though operators_may_deploy is on
|
||||
if caller in ("admin", "tok_admin"):
|
||||
return None
|
||||
return {"admin": None, "operator": {"ADMIN_REQUIRED"}, "tok_deploy": {"API_TOKEN_SCOPE_FORBIDDEN"},
|
||||
"tok_op_deploy": {"API_TOKEN_SCOPE_FORBIDDEN", "ADMIN_REQUIRED"}}[caller]
|
||||
assert route == "revert" # an admin, or an operator while operators_may_deploy is on
|
||||
if caller in ("admin", "tok_deploy", "tok_admin"):
|
||||
return None
|
||||
return None if may_deploy else {"DEPLOY_NEEDS_ADMIN"}
|
||||
|
||||
|
||||
async def _gate(session, route: str):
|
||||
method, path, body = ROUTES[route]
|
||||
url = f"/api/hosts/{NOHOST}/rule-cleanup" + path.format(op=NOID)
|
||||
resp = await session.request(method, url, **({"json": body} if body is not None else {}))
|
||||
code = (resp.json().get("detail") or {}).get("code") if resp.headers.get("content-type", "").startswith("application/json") else None
|
||||
return resp.status_code, code
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("may_deploy", [True, False])
|
||||
@pytest.mark.parametrize("variant", ["both", "router_only", "middleware_only"])
|
||||
async def test_every_route_for_every_caller(client, who, monkeypatch, variant, may_deploy):
|
||||
await _governance(client, may_deploy)
|
||||
try:
|
||||
if variant == "router_only": # switch the middleware's rights checks off: the router must still refuse
|
||||
monkeypatch.setattr(api_token_service, "denies", lambda *a, **k: False)
|
||||
monkeypatch.setattr(api_token_service, "operator_denies", lambda *a, **k: False)
|
||||
monkeypatch.setattr(api_token_service, "is_rollout", lambda *a, **k: False)
|
||||
|
||||
class _Everything: # the read-only gate of the middleware lets this request through
|
||||
def __contains__(self, _):
|
||||
return True
|
||||
monkeypatch.setattr(auth_middleware, "_READ_ONLY_ALLOWED_WRITES", _Everything())
|
||||
if variant == "middleware_only": # switch the router's checks off: the middleware must still refuse
|
||||
async def _nothing_async(*a, **k):
|
||||
return None
|
||||
for name in ("_writer", "_admin_only", "_session_only"):
|
||||
monkeypatch.setattr(router_mod, name, lambda *a, **k: None)
|
||||
monkeypatch.setattr(router_mod, "_deployer", _nothing_async)
|
||||
problems = []
|
||||
for route in ROUTES:
|
||||
for caller, session in who.items():
|
||||
want = verdict(route, caller, may_deploy)
|
||||
got = await _gate(session, route)
|
||||
ok = got == (404, "HOST_NICHT_GEFUNDEN") if want is None else (got[0] == 403 and got[1] in want)
|
||||
if not ok:
|
||||
problems.append((route, caller, got, want))
|
||||
assert not problems, problems
|
||||
finally:
|
||||
await _governance(client, True)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("method,path,session_only,admin_only,rollout", [
|
||||
("POST", "/api/hosts/h/rule-cleanup/preview", False, False, False),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/cancel", False, False, False),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/confirm", True, False, False),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/confirm/", True, False, False),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/deploy", False, True, True),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/deploy/", False, True, False),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/revert", False, False, True),
|
||||
("POST", "/api/hosts/h/rule-cleanup/o/retry", False, True, True),
|
||||
("POST", "/api/hosts/h/rule-cleanup/items/i/restore", False, True, False),
|
||||
("GET", "/api/hosts/h/rule-cleanup", False, False, False),
|
||||
("GET", "/api/hosts/h/rule-cleanup/o", False, False, False),
|
||||
])
|
||||
def test_pattern_table(method, path, session_only, admin_only, rollout):
|
||||
denied_for_read = method == "POST"
|
||||
assert api_token_service.denies("read", method, path) is denied_for_read
|
||||
# a token below scope admin: admin-only routes and confirm are closed
|
||||
assert api_token_service.denies("deploy", method, path) is (session_only or admin_only)
|
||||
# an admin-scope token reaches everything except what only a session may do
|
||||
assert api_token_service.denies("admin", method, path) is session_only
|
||||
assert api_token_service.operator_denies(method, path) is admin_only
|
||||
assert api_token_service.is_rollout(method, path) is rollout
|
||||
|
||||
|
||||
def test_governance_settings_are_admin_writes_for_operators_and_deploy_tokens():
|
||||
assert api_token_service.operator_denies("PUT", "/api/settings/governance") is True
|
||||
assert api_token_service.denies("deploy", "PUT", "/api/settings/governance") is True
|
||||
assert api_token_service.denies("admin", "PUT", "/api/settings/governance") is False
|
||||
|
||||
|
||||
# ── A full run over the API: shapes, audit, input forms ──────────────────
|
||||
|
||||
def _base(env_or_id) -> str:
|
||||
return f"/api/hosts/{getattr(env_or_id, 'host_id', env_or_id)}/rule-cleanup"
|
||||
|
||||
|
||||
async def _audit(**filters):
|
||||
async with SessionLocal() as db:
|
||||
rows = (await db.scalars(select(AuditEvent).where(*[getattr(AuditEvent, k) == v for k, v in filters.items()])
|
||||
.order_by(AuditEvent.created_at))).all()
|
||||
return rows
|
||||
|
||||
|
||||
async def _open(session, env, uuids=(R_UNUSED, R_DUP), name="fw-a"):
|
||||
"""Preview + deploy over the API; the run waits for its confirmation."""
|
||||
pv = await session.post(_base(env) + "/preview", json={"kind": "disable", "uuids": list(uuids)})
|
||||
assert pv.status_code == 200, pv.text
|
||||
op = await session.post(f"{_base(env)}/{pv.json()['preview_id']}/deploy", json={"confirm_name": name})
|
||||
assert op.status_code == 200, op.text
|
||||
return op.json()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_preview_deploy_confirm_over_the_api_with_shapes_and_audit(client, env):
|
||||
pv = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED, R_DUP]})
|
||||
assert pv.status_code == 200, pv.text
|
||||
body = pv.json()
|
||||
assert {"preview_id", "expires_at", "kind", "plan", "max_rules", "window_seconds", "operation"} <= set(body)
|
||||
assert body["max_rules"] == 10 and body["kind"] == "disable" and body["window_seconds"] > 0
|
||||
assert {i["uuid"] for i in body["plan"]["nodes"][0]["items"]} == {R_UNUSED, R_DUP}
|
||||
operation = body["operation"]
|
||||
assert set(operation) == set(CleanupOperationRead.model_fields) # the UI gets exactly the documented fields
|
||||
assert operation["status"] == "preview" and operation["plan"] and operation["items"] == []
|
||||
assert operation["actor"] == "admin" and operation["host_name"] == "fw-a"
|
||||
# The middleware does not log /preview; the service writes the one entry (and it is no history change).
|
||||
assert [(a.action, a.status) for a in await _audit(kind="rule_cleanup")] == [("preview", "requested")]
|
||||
assert await _audit(kind="host", action="preview") == []
|
||||
assert env.fw.writes() == []
|
||||
|
||||
pid = body["preview_id"]
|
||||
wrong = await client.post(f"{_base(env)}/{pid}/deploy", json={"confirm_name": "fw-x"})
|
||||
assert wrong.status_code == 409 and wrong.json()["detail"]["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
|
||||
deployed = await client.post(f"{_base(env)}/{pid}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert deployed.status_code == 200, deployed.text
|
||||
op = deployed.json()
|
||||
assert set(op) == set(CleanupOperationRead.model_fields) and all(set(i) == set(CleanupItemRead.model_fields) for i in op["items"])
|
||||
assert op["status"] == "awaiting_confirm" and op["revert_deadline"] and op["plan"] is None
|
||||
assert (op["can_confirm"], op["can_revert"], op["can_retry"]) == (True, True, False)
|
||||
assert {(i["rule_uuid"], i["state"], i["host_name"]) for i in op["items"]} == {(R_UNUSED, "disabled", "fw-a"), (R_DUP, "disabled", "fw-a")}
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "0"
|
||||
# Audit: one request pair from the middleware (kind host), one entry per rule from the service - no duplicates.
|
||||
# the wrong-name attempt and the real one: a request entry and an outcome each, from the middleware only
|
||||
assert sorted(a.status for a in await _audit(kind="host", action="deploy")) == ["accepted", "rejected", "requested", "requested"]
|
||||
assert [a.status for a in await _audit(kind="rule_cleanup", action="disable")] == ["accepted", "accepted"]
|
||||
assert await _audit(kind="rule_cleanup", action="deploy") == [] and await _audit(kind="host", action="disable") == []
|
||||
|
||||
listing = (await client.get(_base(env))).json()
|
||||
assert [o["id"] for o in listing] == [pid] and listing[0]["status"] == "awaiting_confirm"
|
||||
one = (await client.get(f"{_base(env)}/{pid}")).json()
|
||||
assert one == listing[0]
|
||||
confirmed = await client.post(f"{_base(env)}/{pid}/confirm")
|
||||
assert confirmed.status_code == 200, confirmed.text
|
||||
c = confirmed.json()
|
||||
assert c["status"] == "confirmed" and c["revert_deadline"] is None and {i["state"] for i in c["items"]} == {"confirmed"}
|
||||
assert (c["can_confirm"], c["can_revert"], c["can_retry"]) == (False, False, False)
|
||||
assert [a.status for a in await _audit(kind="rule_cleanup", action="confirm")] == ["confirmed", "confirmed"]
|
||||
assert sorted(a.status for a in await _audit(kind="host", action="confirm")) == ["accepted", "requested"]
|
||||
again = await client.post(f"{_base(env)}/{pid}/confirm")
|
||||
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_FRIST_ABGELAUFEN"
|
||||
missing = await client.get(f"{_base(env)}/{NOID}")
|
||||
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
|
||||
assert (await client.get(f"/api/hosts/{NOHOST}/rule-cleanup")).status_code == 404
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_preview_input_forms_and_validation(client, env):
|
||||
url = _base(env) + "/preview"
|
||||
# items[] with per-rule kind/mode and acknowledgement; a "probable" rule needs the acknowledgement
|
||||
plain = {"items": [{"uuid": R_UNUSED, "mode": "disable"}, {"uuid": R_BLOCK, "kind": "disable"}]}
|
||||
got = await client.post(url, json=plain)
|
||||
assert got.status_code == 409 and got.json()["detail"]["code"] == "BEREINIGUNG_BESTAETIGUNG_FEHLT"
|
||||
assert R_BLOCK in got.json()["detail"]["params"]["uuids"]
|
||||
ack = {"items": [{"uuid": R_UNUSED, "mode": "disable"}, {"uuid": R_BLOCK, "kind": "disable", "ack": True}]}
|
||||
done = await client.post(url, json=ack)
|
||||
assert done.status_code == 200, done.text
|
||||
assert {i["uuid"] for i in done.json()["plan"]["nodes"][0]["items"]} == {R_UNUSED, R_BLOCK}
|
||||
assert done.json()["plan"]["acknowledged"] == [R_BLOCK]
|
||||
# the old form: kind + uuids + acknowledge
|
||||
old = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED, R_BLOCK], "acknowledge": [R_BLOCK]})
|
||||
assert old.status_code == 200 and (await _op(done.json()["preview_id"])).status == "cancelled" # one preview per actor
|
||||
# bad input
|
||||
for bad in ({"uuids": [R_UNUSED]}, {"items": []}):
|
||||
r = await client.post(url, json=bad)
|
||||
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG", bad
|
||||
for bad in ({"kind": "disable", "uuids": []}, {"kind": "disable", "uuids": ["../x"]}):
|
||||
r = await client.post(url, json=bad)
|
||||
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_AUSWAHL_UNGUELTIG", bad
|
||||
mixed = {"items": [{"uuid": R_UNUSED, "kind": "disable"}, {"uuid": R_DUP, "kind": "delete"}]}
|
||||
assert (await client.post(url, json=mixed)).status_code == 422
|
||||
assert (await client.post(url, json={"kind": "restore", "uuids": [R_UNUSED]})).status_code == 422
|
||||
assert (await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED], "unused_days": 0})).status_code == 422
|
||||
assert (await client.post(f"/api/hosts/{NOHOST}/rule-cleanup/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).status_code == 404
|
||||
# the settings reach the service: the limit and the fail-closed lock without management sources
|
||||
assert (await client.put("/api/settings/governance", json={"cleanup_max_rules": 1})).status_code == 200
|
||||
many = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED, R_DUP]})
|
||||
assert many.status_code == 422 and many.json()["detail"]["code"] == "BEREINIGUNG_ZU_VIELE_REGELN" and many.json()["detail"]["params"]["max"] == 1
|
||||
assert (await client.put("/api/settings/governance", json={"cleanup_management_sources": []})).status_code == 200
|
||||
locked = await client.post(url, json={"kind": "disable", "uuids": [R_UNUSED]})
|
||||
assert locked.status_code == 409 and locked.json()["detail"]["code"] == "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_preview_belongs_to_its_actor_and_can_be_cancelled(client, env, who):
|
||||
operator = who["operator"]
|
||||
mine = (await operator.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).json()
|
||||
assert mine["operation"]["actor"] == "cl-operator"
|
||||
# the admin neither sees it as his own nor deploys it; the operator cannot deploy at all
|
||||
stranger = await client.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert stranger.status_code == 404 and stranger.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
|
||||
refused_op = await operator.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert refused_op.status_code == 403 and refused_op.json()["detail"]["code"] == "ADMIN_REQUIRED"
|
||||
assert (await client.post(f"{_base(env)}/{mine['preview_id']}/cancel")).status_code == 404 # not the admin's
|
||||
cancelled = await operator.post(f"{_base(env)}/{mine['preview_id']}/cancel")
|
||||
assert cancelled.status_code == 200 and cancelled.json()["status"] == "cancelled"
|
||||
# the admin's refused attempt (not his preview) and the operator's real cancel: a request entry and an outcome each
|
||||
assert sorted(a.status for a in await _audit(kind="host", action="cancel")) == ["accepted", "rejected", "requested", "requested"]
|
||||
again = await operator.post(f"{_base(env)}/{mine['preview_id']}/cancel")
|
||||
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
|
||||
gone = await client.post(f"{_base(env)}/{mine['preview_id']}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert gone.status_code == 404 # (not the admin's preview)
|
||||
assert env.fw.writes() == []
|
||||
# an operator prepares, the admin prepares his own and executes; the operator may confirm afterwards
|
||||
own = (await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})).json()
|
||||
deployed = await client.post(f"{_base(env)}/{own['preview_id']}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert deployed.status_code == 200
|
||||
confirmed = await operator.post(f"{_base(env)}/{own['preview_id']}/confirm")
|
||||
assert confirmed.status_code == 200 and confirmed.json()["status"] == "confirmed"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_confirm_never_works_with_a_token_and_changes_nothing(client, env, who):
|
||||
op = await _open(client, env)
|
||||
for name in ("tok_admin", "tok_deploy", "tok_op_deploy"):
|
||||
r = await who[name].post(f"{_base(env)}/{op['id']}/confirm")
|
||||
assert r.status_code == 403 and r.json()["detail"]["code"] == "API_TOKEN_SCOPE_FORBIDDEN", name
|
||||
state = await _op(op["id"])
|
||||
assert state.status == "awaiting_confirm" and state.revert_deadline is not None
|
||||
assert {i.state for i in (await _items(op["id"])).values()} == {"disabled"}
|
||||
# the same token may read the run, and the browser session confirms
|
||||
assert (await who["tok_read"].get(f"{_base(env)}/{op['id']}")).json()["can_confirm"] is True
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/confirm")).json()["status"] == "confirmed"
|
||||
denied = await _audit(kind="host", action="confirm", status="rejected")
|
||||
assert not denied # the middleware refused before the router: no request entry for a refused token confirm
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_revert_is_for_admins_and_for_operators_only_while_they_may_deploy(client, env, who):
|
||||
op = await _open(client, env)
|
||||
url = f"{_base(env)}/{op['id']}/revert"
|
||||
await _governance(client, False)
|
||||
try:
|
||||
assert (await who["operator"].post(url)).status_code == 403
|
||||
assert (await who["tok_op_deploy"].post(url)).status_code == 403
|
||||
assert (await who["read_only"].post(url)).status_code == 403
|
||||
assert (await _op(op["id"])).status == "awaiting_confirm" and env.fw.get(R_UNUSED)["enabled"] == "0"
|
||||
finally:
|
||||
await _governance(client, True)
|
||||
reverted = await who["operator"].post(url)
|
||||
assert reverted.status_code == 200, reverted.text
|
||||
r = reverted.json()
|
||||
assert r["status"] == "reverted" and {i["state"] for i in r["items"]} == {"reverted"} and not r["can_revert"]
|
||||
assert env.fw.get(R_UNUSED)["enabled"] == "1" and env.fw.get(R_DUP)["enabled"] == "1"
|
||||
again = await client.post(url)
|
||||
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_after_a_failed_revert_is_admin_only(client, env, who):
|
||||
op = await _open(client, env)
|
||||
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
|
||||
failed = await client.post(f"{_base(env)}/{op['id']}/revert")
|
||||
assert failed.status_code == 200 and failed.json()["status"] == "revert_failed" and failed.json()["can_retry"] is True
|
||||
del env.fw.fail[("toggle", R_DUP)]
|
||||
url = f"{_base(env)}/{op['id']}/retry"
|
||||
for name, code in (("operator", "ADMIN_REQUIRED"), ("tok_deploy", "API_TOKEN_SCOPE_FORBIDDEN"), ("read_only", "READ_ONLY_ACCESS")):
|
||||
r = await who[name].post(url)
|
||||
assert r.status_code == 403 and r.json()["detail"]["code"] == code, name
|
||||
assert (await _op(op["id"])).status == "revert_failed"
|
||||
done = await who["tok_admin"].post(url)
|
||||
assert done.status_code == 200 and done.json()["status"] == "reverted" and env.fw.get(R_DUP)["enabled"] == "1"
|
||||
nothing = await client.post(url)
|
||||
assert nothing.status_code == 409 and nothing.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_and_restore_over_the_api(client, env, who):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
|
||||
assert pv["operation"]["kind"] == "delete"
|
||||
done = await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})
|
||||
assert done.status_code == 200, done.text
|
||||
op = done.json()
|
||||
assert op["status"] == "done" and op["kind"] == "delete" and (op["can_confirm"], op["can_revert"], op["can_retry"]) == (False,) * 3
|
||||
item = op["items"][0]
|
||||
assert item["state"] == "deleted" and item["restorable"] is True and item["has_snapshot"] is True
|
||||
assert env.fw.get(R_UNUSED) is None
|
||||
assert [a.status for a in await _audit(kind="rule_cleanup", action="delete")] == ["accepted"]
|
||||
url = f"{_base(env)}/items/{item['id']}/restore"
|
||||
for name, code in (("operator", "ADMIN_REQUIRED"), ("tok_deploy", "API_TOKEN_SCOPE_FORBIDDEN"), ("read_only", "READ_ONLY_ACCESS")):
|
||||
r = await who[name].post(url, json={"confirm_name": "fw-a"})
|
||||
assert r.status_code == 403 and r.json()["detail"]["code"] == code, name
|
||||
wrong = await client.post(url, json={"confirm_name": "nope"})
|
||||
assert wrong.status_code == 409 and wrong.json()["detail"]["code"] == "HOSTNAMEN_EXAKT_EINGEBEN"
|
||||
assert env.fw.get(R_UNUSED) is None
|
||||
restored = await who["tok_admin"].post(url, json={"confirm_name": "fw-a"})
|
||||
assert restored.status_code == 200, restored.text
|
||||
r = restored.json()
|
||||
assert r["kind"] == "restore" and r["status"] == "done" and r["warnings"] == []
|
||||
new = r["items"][0]["restored_uuid"]
|
||||
assert new and r["items"][0]["state"] == "restored" and env.fw.get(new)["enabled"] == "0"
|
||||
listed = {o["id"]: o for o in (await client.get(_base(env))).json()}
|
||||
source = next(i for i in listed[op["id"]]["items"] if i["id"] == item["id"])
|
||||
assert source["restorable"] is False and source["restored_uuid"] == new
|
||||
assert [a.status for a in await _audit(kind="rule_cleanup", action="restore")] == ["accepted"]
|
||||
again = await client.post(url, json={"confirm_name": "fw-a"})
|
||||
assert again.status_code == 409 and again.json()["detail"]["code"] == "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
|
||||
nothing = await client.post(f"{_base(env)}/{op['id']}/retry")
|
||||
assert nothing.status_code == 409 # nothing unclear in a finished delete
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_restore_reports_a_lost_position_as_a_warning(client, env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
|
||||
op = (await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})).json()
|
||||
env.fw.rules = [r for r in env.fw.rules if r["uuid"] != R_BLOCK] # the anchor is gone
|
||||
r = await client.post(f"{_base(env)}/items/{op['items'][0]['id']}/restore", json={"confirm_name": "fw-a"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["warnings"] == ["BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"]
|
||||
assert r.json()["items"][0]["message"] == "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
|
||||
missing = await client.post(f"{_base(env)}/items/{NOID}/restore", json={"confirm_name": "fw-a"})
|
||||
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "BEREINIGUNG_NICHT_GEFUNDEN"
|
||||
|
||||
|
||||
# ── Settings ──────────────────────────────────────────────────────────────
|
||||
|
||||
GOV = "/api/settings/governance"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_management_sources_are_validated_normalised_and_audited(client):
|
||||
put = await client.put(GOV, json={"cleanup_management_sources": [" 203.0.113.10 ", "10.20.30.5/24", "203.0.113.10", "2001:db8::1"],
|
||||
"cleanup_max_rules": 25, "cleanup_delete_min_days": 30})
|
||||
assert put.status_code == 200, put.text
|
||||
got = put.json()
|
||||
assert got["cleanup_management_sources"] == ["203.0.113.10", "10.20.30.0/24", "2001:db8::1"]
|
||||
assert got["cleanup_management_sources_configured"] is True and got["cleanup_max_rules"] == 25 and got["cleanup_delete_min_days"] == 30
|
||||
assert (await client.get(GOV)).json() == got
|
||||
audit = await _audit(kind="settings", action="cleanup_settings")
|
||||
assert len(audit) == 1 and audit[0].actor == "admin"
|
||||
changes = audit[0].details["changes"]
|
||||
assert changes["cleanup_management_sources"]["new"] == got["cleanup_management_sources"]
|
||||
assert changes["cleanup_max_rules"] == {"old": 10, "new": 25}
|
||||
# an unchanged value writes no further entry; a partial update leaves the rest alone
|
||||
assert (await client.put(GOV, json={"cleanup_max_rules": 25, "operators_may_deploy": True})).json()["cleanup_delete_min_days"] == 30
|
||||
assert len(await _audit(kind="settings", action="cleanup_settings")) == 1
|
||||
bad_sources = [["fw.example.test"], ["203.0.113.999"], ["0.0.0.0/0"], ["::/0"], ["203.0.113.1", ""], [f"10.20.30.{i}" for i in range(21)], ["10.20.30.0/33"]]
|
||||
for sources in bad_sources:
|
||||
r = await client.put(GOV, json={"cleanup_management_sources": sources})
|
||||
assert r.status_code == 422 and r.json()["detail"]["code"] == "BEREINIGUNG_EINSTELLUNG_UNGUELTIG", sources
|
||||
assert r.json()["detail"]["params"]["field"] == "cleanup_management_sources"
|
||||
for field, value in (("cleanup_max_rules", 0), ("cleanup_max_rules", 26), ("cleanup_delete_min_days", 0), ("cleanup_delete_min_days", 366)):
|
||||
r = await client.put(GOV, json={field: value})
|
||||
assert r.status_code == 422 and r.json()["detail"]["params"]["field"] == field, (field, value)
|
||||
assert (await client.get(GOV)).json() == got # nothing of the refused requests was stored
|
||||
assert len(await _audit(kind="settings", action="cleanup_settings")) == 1
|
||||
assert (await client.put(GOV, json={"cleanup_management_sources": [f"10.20.30.{i}" for i in range(20)]})).status_code == 200
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_management_ports_are_validated_normalised_audited_and_hidden(client, who): # review finding 3
|
||||
put = await client.put(GOV, json={"cleanup_management_ports": [8443, "2222", 8443]})
|
||||
assert put.status_code == 200, put.text
|
||||
assert put.json()["cleanup_management_ports"] == [2222, 8443] and put.json()["cleanup_management_ports_configured"] is True
|
||||
audit = await _audit(kind="settings", action="cleanup_settings")
|
||||
assert audit[-1].details["changes"]["cleanup_management_ports"] == {"old": [], "new": [2222, 8443]}
|
||||
for ports in ([0], [65536], ["ssh"], [True], [-1], list(range(1, 22))):
|
||||
r = await client.put(GOV, json={"cleanup_management_ports": ports})
|
||||
assert r.status_code == 422 and r.json()["detail"]["params"]["field"] == "cleanup_management_ports", ports
|
||||
assert (await client.get(GOV)).json()["cleanup_management_ports"] == [2222, 8443]
|
||||
for name in ("operator", "read_only"):
|
||||
seen = (await who[name].get(GOV)).json()
|
||||
assert seen["cleanup_management_ports"] == [] and seen["cleanup_management_ports_configured"] is True, name
|
||||
r = await who[name].put(GOV, json={"cleanup_management_ports": [1]})
|
||||
assert r.status_code == 403, name
|
||||
assert (await client.put(GOV, json={"cleanup_management_ports": []})).json()["cleanup_management_ports_configured"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("variant", ["both", "router_only"])
|
||||
async def test_only_an_admin_writes_the_cleanup_settings(client, who, monkeypatch, variant):
|
||||
await client.put(GOV, json={"cleanup_management_sources": ["203.0.113.10"]})
|
||||
if variant == "router_only":
|
||||
monkeypatch.setattr(api_token_service, "denies", lambda *a, **k: False)
|
||||
monkeypatch.setattr(api_token_service, "operator_denies", lambda *a, **k: False)
|
||||
# router_only: an admin-owned deploy/read token IS an admin account: only the token layer (switched off) stops it
|
||||
attempts = {"operator": {"ADMIN_REQUIRED"}, "tok_op_deploy": {"API_TOKEN_SCOPE_FORBIDDEN", "ADMIN_REQUIRED"},
|
||||
"read_only": {"READ_ONLY_ACCESS", "ADMIN_REQUIRED"}}
|
||||
if variant == "both":
|
||||
attempts.update({"tok_deploy": {"API_TOKEN_SCOPE_FORBIDDEN"}, "tok_read": {"API_TOKEN_SCOPE_FORBIDDEN"}})
|
||||
for name, codes in attempts.items():
|
||||
r = await who[name].put(GOV, json={"cleanup_management_sources": ["198.51.100.7"], "cleanup_max_rules": 3})
|
||||
assert r.status_code == 403 and r.json()["detail"]["code"] in codes, (name, r.text)
|
||||
assert (await client.get(GOV)).json()["cleanup_management_sources"] == ["203.0.113.10"]
|
||||
assert (await client.get(GOV)).json()["cleanup_max_rules"] == 10
|
||||
# an admin-scope token acts as the admin and the audit names the token
|
||||
ok = await who["tok_admin"].put(GOV, json={"cleanup_management_sources": ["198.51.100.7"]})
|
||||
assert ok.status_code == 200, ok.text
|
||||
assert [a.actor for a in await _audit(kind="settings", action="cleanup_settings")][-1] == "admin (API: cl-admin)"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_others_only_see_whether_management_sources_are_set(client, who):
|
||||
await client.put(GOV, json={"cleanup_management_sources": ["203.0.113.10"]})
|
||||
assert (await client.get(GOV)).json()["cleanup_management_sources"] == ["203.0.113.10"]
|
||||
for name in ("operator", "read_only"):
|
||||
seen = (await who[name].get(GOV)).json()
|
||||
assert seen["cleanup_management_sources"] == [] and seen["cleanup_management_sources_configured"] is True, name
|
||||
assert seen["cleanup_max_rules"] == 10 and seen["operators_may_deploy"] is True
|
||||
|
||||
|
||||
# ── Locks in both directions ──────────────────────────────────────────────
|
||||
|
||||
async def _central_rule(client, env, **extra):
|
||||
resp = await client.post("/api/rules", json={
|
||||
"description": "managed ssh", "interface": "lan", "direction": "in", "protocol": "tcp", "action": "pass",
|
||||
"source_net": "any", "destination_net": "lan", "destination_port": "22",
|
||||
"target_host_ids": [str(env.host_id)], **extra})
|
||||
assert resp.status_code == 201, resp.text
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def _deployed_row(rule_id, host_id, firewall_uuid, status="success"):
|
||||
async with SessionLocal() as db:
|
||||
dep = RuleDeployment(rule_id=uuid_lib.UUID(rule_id), host_id=host_id, status=status, opnsense_rule_uuid=firewall_uuid)
|
||||
db.add(dep)
|
||||
await db.commit()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_open_cleanup_blocks_other_writes_to_the_firewall(client, env):
|
||||
rule = await _central_rule(client, env)
|
||||
alias = await client.post("/api/aliases", json={"name": "cl_alias", "type": "host", "content": "203.0.113.5",
|
||||
"target_host_ids": [str(env.host_id)]})
|
||||
assert alias.status_code == 201, alias.text
|
||||
hid = env.host_id
|
||||
await _open(client, env)
|
||||
writes = list(env.fw.writes())
|
||||
blocked = [
|
||||
("POST", f"/api/rules/{rule['id']}/deploy", None),
|
||||
("POST", f"/api/aliases/{alias.json()['id']}/deploy", None),
|
||||
("POST", f"/api/hosts/{hid}/snapshots", {"name": "x"}),
|
||||
("POST", f"/api/hosts/{hid}/services/dhcpd/restart", None),
|
||||
("POST", f"/api/hosts/{hid}/maintenance/check-updates", None),
|
||||
("POST", f"/api/hosts/{hid}/network/changes", {"kind": "vlan", "action": "add", "fields": {}, "preview_token": "0" * 64}),
|
||||
("DELETE", f"/api/hosts/{hid}/snapshots/abc", None),
|
||||
]
|
||||
for method, path, body in blocked:
|
||||
r = await client.request(method, path, **({"json": body} if body is not None else {}))
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", (path, r.text)
|
||||
assert env.fw.writes() == writes # nothing reached the firewall
|
||||
# the cleanup's own follow-up steps are not blocked by itself, and a second cleanup is refused with its own code
|
||||
second = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_BLOCK], "acknowledge": [R_BLOCK]})
|
||||
assert second.status_code == 409 and second.json()["detail"]["code"] == "BEREINIGUNG_OFFENE_OPERATION"
|
||||
assert second.json()["detail"]["params"]["reason"] == "cleanup"
|
||||
op = (await client.get(_base(env))).json()[0]
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/revert")).status_code == 200
|
||||
again = await client.post(f"/api/hosts/{hid}/maintenance/check-updates")
|
||||
assert again.status_code != 409 or again.json()["detail"]["code"] != "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_block_covers_the_cluster_and_the_maintenance_inspection(cluster, client):
|
||||
from app.services.host_maintenance_service import _shared_change_blockers, ensure_no_maintenance, ensure_no_open_cleanup, peers
|
||||
pv = await client.post(_base(cluster.a) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})
|
||||
assert pv.status_code == 200, pv.text
|
||||
assert (await client.post(f"{_base(cluster.a)}/{pv.json()['preview_id']}/deploy", json={"confirm_name": "fw-a"})).status_code == 200
|
||||
async with SessionLocal() as db:
|
||||
a, b = await db.get(OpnsenseHost, cluster.a), await db.get(OpnsenseHost, cluster.b)
|
||||
for host in (a, b): # the partner node is just as busy
|
||||
for call in (ensure_no_maintenance, ensure_no_open_cleanup):
|
||||
with pytest.raises(Exception) as info:
|
||||
await call(db, [host])
|
||||
assert info.value.detail["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
blockers = await _shared_change_blockers(db, await peers(db, b))
|
||||
assert any("Bereinigung" in text for text in blockers)
|
||||
await ensure_no_maintenance(db, [a], cleanup=False) # the cleanup service's own check does not trip over itself
|
||||
# deleting or re-addressing either node is refused while it is open
|
||||
for host_id in (cluster.a, cluster.b):
|
||||
r = await client.delete(f"/api/hosts/{host_id}")
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", host_id
|
||||
op = (await client.get(_base(cluster.a))).json()[0]
|
||||
assert (await client.post(f"{_base(cluster.a)}/{op['id']}/revert")).status_code == 200
|
||||
async with SessionLocal() as db:
|
||||
assert await _shared_change_blockers(db, await peers(db, await db.get(OpnsenseHost, cluster.b))) == []
|
||||
await ensure_no_maintenance(db, [await db.get(OpnsenseHost, cluster.b)])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_host_delete_and_readdressing_are_refused_while_a_cleanup_is_open(client, env):
|
||||
hid = env.host_id
|
||||
await _open(client, env)
|
||||
gone = await client.delete(f"/api/hosts/{hid}")
|
||||
assert gone.status_code == 409 and gone.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
for body in ({"base_url": "https://203.0.113.99"}, {"enabled": False}, {"cluster_name": "ha-x"}, {"clear_cluster_name": True}):
|
||||
r = await client.patch(f"/api/hosts/{hid}", json=body)
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", body
|
||||
assert (await client.patch(f"/api/hosts/{hid}", json={"name": "fw-a2"})).status_code == 200 # a rename is harmless
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, hid)
|
||||
assert host.base_url == FW1 and host.enabled and host.name == "fw-a2"
|
||||
op = (await client.get(_base(env))).json()[0]
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/revert")).status_code == 200
|
||||
assert (await client.delete(f"/api/hosts/{hid}")).status_code == 204 # settled: the host may go (its runs go with it)
|
||||
async with SessionLocal() as db:
|
||||
assert (await db.scalars(select(RuleCleanupOperation))).all() == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_confirmed_cleanup_blocks_nothing(client, env):
|
||||
op = await _open(client, env)
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/confirm")).status_code == 200
|
||||
async with SessionLocal() as db:
|
||||
assert not await svc.has_open_cleanup(db, [env.host_id])
|
||||
r = await client.post(f"/api/hosts/{env.host_id}/maintenance/check-updates")
|
||||
assert r.status_code != 409 or r.json()["detail"]["code"] != "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rule_deletion_and_removal_of_targets_wait_for_the_cleanup(client, env):
|
||||
rule = await _central_rule(client, env, target_host_ids=[])
|
||||
await _deployed_row(rule["id"], env.host_id, R_TAIL)
|
||||
token = (await client.get(f"/api/rules/{rule['id']}/preview")).json()["token"]
|
||||
await _open(client, env)
|
||||
env.fw.calls.clear()
|
||||
deleted = await client.delete(f"/api/rules/{rule['id']}")
|
||||
assert deleted.status_code == 409 and deleted.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY"
|
||||
removed = await client.post(f"/api/rules/{rule['id']}/deployments/{env.host_id}/remove", json={"preview_token": token})
|
||||
assert removed.status_code == 409 and removed.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", removed.text
|
||||
assert env.fw.writes() == [] and env.fw.get(R_TAIL) is not None
|
||||
op = (await client.get(_base(env))).json()[0]
|
||||
await client.post(f"{_base(env)}/{op['id']}/revert")
|
||||
assert (await client.delete(f"/api/rules/{rule['id']}")).status_code == 204 # the same call works afterwards
|
||||
assert env.fw.get(R_TAIL) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_hierarchy_rule_deletion_and_reordering_wait_for_the_cleanup(client, env):
|
||||
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
|
||||
from tests.test_policy_hierarchy import _create_group, _hrule_payload
|
||||
group = await _create_group(client, "Cleanup-Root")
|
||||
assert (await client.patch(f"/api/hosts/{env.host_id}", json={"policy_hierarchy_group_id": group["id"]})).status_code == 200
|
||||
rule = (await client.post("/api/policy-hierarchy/rules", json=_hrule_payload(group["id"], description="guard"))).json()
|
||||
async with SessionLocal() as db:
|
||||
db.add(PolicyHierarchyRuleDeployment(policy_hierarchy_id=uuid_lib.UUID(rule["id"]), host_id=env.host_id, status="success",
|
||||
opnsense_policy_hierarchy_uuid=R_TAIL))
|
||||
await db.commit()
|
||||
await _open(client, env)
|
||||
env.fw.calls.clear()
|
||||
deleted = await client.delete(f"/api/policy-hierarchy/rules/{rule['id']}")
|
||||
assert deleted.status_code == 409 and deleted.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", deleted.text
|
||||
reorder = await client.post(f"/api/policy-hierarchy/groups/{group['id']}/reassert-ordering")
|
||||
assert reorder.status_code == 409 and reorder.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", reorder.text
|
||||
assert env.fw.writes() == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_removing_a_former_hierarchy_target_waits_for_the_cleanup(client, env):
|
||||
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
|
||||
from tests.test_policy_hierarchy import _create_group, _hrule_payload
|
||||
group = await _create_group(client, "Cleanup-Former")
|
||||
rule = (await client.post("/api/policy-hierarchy/rules", json=_hrule_payload(group["id"], description="former"))).json()
|
||||
async with SessionLocal() as db: # deployed on the host, which is not in the group (any more): "retained"
|
||||
db.add(PolicyHierarchyRuleDeployment(policy_hierarchy_id=uuid_lib.UUID(rule["id"]), host_id=env.host_id, status="success",
|
||||
opnsense_policy_hierarchy_uuid=R_TAIL))
|
||||
await db.commit()
|
||||
plan = (await client.get(f"/api/policy-hierarchy/rules/{rule['id']}/preview")).json()
|
||||
assert [h["can_remove"] for h in plan["hosts"]] == [True]
|
||||
await _open(client, env)
|
||||
env.fw.calls.clear()
|
||||
r = await client.post(f"/api/policy-hierarchy/rules/{rule['id']}/deployments/{env.host_id}/remove", json={"preview_token": plan["token"]})
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
|
||||
assert env.fw.writes() == [] and env.fw.get(R_TAIL) is not None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_adopting_a_rule_waits_for_the_cleanup(client, env):
|
||||
await _open(client, env)
|
||||
r = await client.post("/api/rules/adopt", json={"host_id": str(env.host_id), "rule_uuid": R_BASE})
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_other_open_work_blocks_a_new_cleanup_over_the_api(client, env):
|
||||
url = _base(env) + "/preview"
|
||||
body = {"kind": "disable", "uuids": [R_UNUSED]}
|
||||
async with SessionLocal() as db:
|
||||
db.add(HostOperation(host_id=env.host_id, host_name="fw-a", actor="admin", action="reboot", status="submitted",
|
||||
expires_at=_now(), fingerprint="x", snapshot={}))
|
||||
await db.commit()
|
||||
r = await client.post(url, json=body)
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(HostOperation).values(status="done"))
|
||||
await db.commit()
|
||||
rule = await _central_rule(client, env)
|
||||
await _deployed_row(rule["id"], env.host_id, R_TAIL, status="pending")
|
||||
r = await client.post(url, json=body)
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "BEREINIGUNG_OFFENE_OPERATION" and r.json()["detail"]["params"]["reason"] == "deployment"
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleDeployment).values(status="success"))
|
||||
await db.commit()
|
||||
assert (await client.post(url, json=body)).status_code == 200
|
||||
assert env.fw.writes() == []
|
||||
|
||||
|
||||
# ── Notification ──────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_failed_reverts_and_unclear_runs_are_reported_with_stable_keys(client, env):
|
||||
op = await _open(client, env)
|
||||
async with SessionLocal() as db:
|
||||
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
|
||||
# a run that ended half-way (decision needed): one problem for the run
|
||||
async with SessionLocal() as db:
|
||||
await db.execute(update(RuleCleanupOperation).where(RuleCleanupOperation.id == uuid_lib.UUID(op["id"])).values(status="partial"))
|
||||
await db.commit()
|
||||
problems = await notification_checks.check_revert_failed(db)
|
||||
key = f"rule_cleanup_op:{op['id']}"
|
||||
assert set(problems) == {key} and problems[key]["deployment_type"] == "rule_cleanup"
|
||||
assert problems[key]["host_name"] == "fw-a" and "teilweise" in problems[key]["hinweis"]
|
||||
# the automatic revert failed: one problem per rule that is still changed (the key is the item, not the run)
|
||||
env.fw.fail[("toggle", R_DUP)] = httpx.Response(200, json={"result": "failed"})
|
||||
await client.post(f"{_base(env)}/{op['id']}/revert")
|
||||
items = await _items(op["id"])
|
||||
assert (await _op(op["id"])).status == "revert_failed"
|
||||
async with SessionLocal() as db:
|
||||
first = await notification_checks.check_revert_failed(db)
|
||||
second = await notification_checks.check_revert_failed(db)
|
||||
stuck = f"rule_cleanup:{items[R_DUP].id}"
|
||||
assert set(first) == {stuck} and set(second) == set(first) # stable: the same problem is not reported twice
|
||||
assert first[stuck]["entity_id"] == R_DUP and first[stuck]["host_name"] == "fw-a" and "Erneut zurücknehmen" in first[stuck]["hinweis"]
|
||||
assert first[stuck]["operation_status"] == "revert_failed" and first[stuck]["item_state"] == "revert_failed"
|
||||
# solved: gone
|
||||
del env.fw.fail[("toggle", R_DUP)]
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/retry")).json()["status"] == "reverted"
|
||||
async with SessionLocal() as db:
|
||||
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_unclear_delete_is_reported_until_it_is_read_again(client, env):
|
||||
await _confirmed_disable(env.host_id, [R_UNUSED])
|
||||
pv = (await client.post(_base(env) + "/preview", json={"kind": "delete", "uuids": [R_UNUSED]})).json()
|
||||
env.fw.fail[("delete", R_UNUSED)] = httpx.ReadTimeout
|
||||
|
||||
def _then_unreadable(kind, rule_uuid): # the firewall stops answering once delRule went out
|
||||
if kind == "delete":
|
||||
env.fw.fail[("get", R_UNUSED)] = httpx.ReadTimeout
|
||||
env.fw.on_call = _then_unreadable
|
||||
op = (await client.post(f"{_base(env)}/{pv['preview_id']}/deploy", json={"confirm_name": "fw-a"})).json()
|
||||
assert op["status"] == "failed" and op["items"][0]["state"] == "unknown" and op["can_retry"] is True
|
||||
async with SessionLocal() as db:
|
||||
problems = await notification_checks.check_revert_failed(db)
|
||||
key = f"rule_cleanup:{op['items'][0]['id']}"
|
||||
assert set(problems) == {key} and problems[key]["operation"] == "delete" and "unklar" in problems[key]["hinweis"]
|
||||
env.fw.on_call = None
|
||||
del env.fw.fail[("get", R_UNUSED)]
|
||||
del env.fw.fail[("delete", R_UNUSED)]
|
||||
assert (await client.post(f"{_base(env)}/{op['id']}/retry")).status_code == 200
|
||||
async with SessionLocal() as db:
|
||||
assert not [k for k in await notification_checks.check_revert_failed(db) if k.startswith("rule_cleanup")]
|
||||
|
||||
|
||||
# ── MCP and the schema ────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mcp_stays_read_only_for_the_cleanup(client, env, who):
|
||||
from app.routers.mcp import TOOLS
|
||||
names = {t["name"] for t in TOOLS}
|
||||
assert not [n for n in names if "cleanup" in n and n != "rule_cleanup_candidates"]
|
||||
assert all(t.get("path", "").startswith("/api/") for t in TOOLS if "path" in t)
|
||||
op = await _open(client, env)
|
||||
|
||||
def call(path):
|
||||
return {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "api_get", "arguments": {"path": path}}}
|
||||
|
||||
async with _bearer((await _make(client, "admin", name="mcp-admin"))["token"]) as api:
|
||||
for tail in ("/preview", f"/{op['id']}/confirm", f"/{op['id']}/revert", f"/{op['id']}/deploy", f"/{op['id']}/retry"):
|
||||
result = (await api.post("/api/mcp", json=call(f"/api/hosts/{env.host_id}/rule-cleanup{tail}"))).json()["result"]
|
||||
assert result["isError"] is True, tail # a GET on a POST route: refused, nothing runs
|
||||
listing = (await api.post("/api/mcp", json=call(f"/api/hosts/{env.host_id}/rule-cleanup"))).json()["result"]
|
||||
assert listing["isError"] is False and op["id"] in listing["content"][0]["text"]
|
||||
assert (await _op(op["id"])).status == "awaiting_confirm"
|
||||
# a plain POST of a tool name that would write does not exist
|
||||
async with _bearer((await _make(client, "admin", name="mcp-admin2"))["token"]) as api:
|
||||
r = (await api.post("/api/mcp", json={"jsonrpc": "2.0", "id": 2, "method": "tools/call",
|
||||
"params": {"name": "rule_cleanup_deploy", "arguments": {}}})).json()
|
||||
assert "error" in r or r["result"]["isError"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_openapi_knows_the_routes_and_their_models(client):
|
||||
schema = (await client.get("/api/openapi.json")).json()
|
||||
base = "/api/hosts/{host_id}/rule-cleanup"
|
||||
paths = schema["paths"]
|
||||
assert set(paths[base]) == {"get"} and set(paths[base + "/preview"]) == {"post"} and set(paths[base + "/{op_id}"]) == {"get"}
|
||||
for tail in ("cancel", "deploy", "confirm", "revert", "retry"):
|
||||
assert set(paths[f"{base}/{{op_id}}/{tail}"]) == {"post"}, tail
|
||||
assert set(paths[base + "/items/{item_id}/restore"]) == {"post"}
|
||||
assert {"CleanupOperationRead", "CleanupItemRead", "CleanupPreviewRead", "PreviewRequest", "ConfirmName"} <= set(schema["components"]["schemas"])
|
||||
ops = schema["components"]["schemas"]["CleanupOperationRead"]["properties"]
|
||||
assert {"can_confirm", "can_revert", "can_retry", "plan", "warnings", "items", "revert_deadline", "message"} <= set(ops)
|
||||
assert {"restorable", "state", "host_name", "has_snapshot"} <= set(schema["components"]["schemas"]["CleanupItemRead"]["properties"])
|
||||
|
||||
|
||||
def test_every_new_error_code_has_a_text_in_both_languages():
|
||||
import json
|
||||
import pathlib
|
||||
from app.errors import ErrorCode
|
||||
root = pathlib.Path(__file__).resolve().parents[2] / "frontend" / "lib" / "i18n"
|
||||
if not root.is_dir():
|
||||
pytest.skip("the frontend is not part of this image (scripts/check_i18n.py checks the same from the repo root)")
|
||||
for lang in ("de", "en"):
|
||||
errors = json.loads((root / f"{lang}.json").read_text(encoding="utf-8"))["errors"]
|
||||
for code in ErrorCode:
|
||||
if code.name.startswith("BEREINIGUNG_") or code.name == "OPEN_CLEANUP_BLOCKS_DEPLOY":
|
||||
assert errors.get(code.value), (lang, code.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_refused_operator_attempt_is_audited_and_writes_nothing(client, env, who):
|
||||
op = await _open(client, env)
|
||||
r = await who["operator"].post(f"{_base(env)}/{op['id']}/retry")
|
||||
assert r.status_code == 403
|
||||
denied = await _audit(kind="host", action="retry", status="denied")
|
||||
assert [a.actor for a in denied] == ["cl-operator"] and denied[0].host_id == str(env.host_id)
|
||||
assert (await _op(op["id"])).status == "awaiting_confirm"
|
||||
|
||||
|
||||
def _leaks(value) -> bool:
|
||||
"""Anything that names a management source or a firewall's base URL."""
|
||||
import json as _json
|
||||
text = _json.dumps(value)
|
||||
return MGMT_SOURCE in text or "203.0.113.57" in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sources_and_firewall_addresses_are_for_admins_only(client, env, who): # review finding 8
|
||||
mine = await client.post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_UNUSED]})
|
||||
assert mine.json()["plan"]["sources"] == [f"{MGMT_SOURCE}/32"] and mine.json()["plan"]["nodes"][0]["base_url"]
|
||||
theirs = await who["operator"].post(_base(env) + "/preview", json={"kind": "disable", "uuids": [R_DUP]})
|
||||
assert theirs.status_code == 200, theirs.text
|
||||
body = theirs.json()
|
||||
assert "sources" not in body["plan"] and body["plan"]["sources_configured"] is True and not _leaks(body)
|
||||
assert body["plan"]["nodes"][0]["items"][0]["protection"]["result"] == "false"
|
||||
for name in ("operator", "read_only", "tok_read", "tok_deploy", "tok_ro_read"):
|
||||
listed = await who[name].get(_base(env))
|
||||
assert listed.status_code == 200 and listed.json() and not _leaks(listed.json()), name
|
||||
one = await who[name].get(f"{_base(env)}/{mine.json()['preview_id']}")
|
||||
assert one.status_code == 200 and not _leaks(one.json()), name
|
||||
audit = await who[name].get("/api/audit", params={"kind": "rule_cleanup"})
|
||||
assert audit.status_code == 200 and audit.json()["items"] and not _leaks(audit.json()), name
|
||||
assert {i["details"]["admin_only"] for i in audit.json()["items"]} == {"redacted"}, name
|
||||
for name in ("admin", "tok_admin"):
|
||||
assert _leaks((await who[name].get(_base(env))).json()), name
|
||||
audit = (await who[name].get("/api/audit", params={"kind": "rule_cleanup"})).json()
|
||||
assert all(isinstance(i["details"]["admin_only"], dict) for i in audit["items"]), name
|
||||
|
||||
|
||||
# Review finding 9: undeploying a central object and removing a former target write to the firewall too.
|
||||
_UNDEPLOY_SITES = [
|
||||
("app.services.alias_deploy_service", "undeploy_alias"), ("app.services.openvpn_deploy_service", "undeploy_instance"),
|
||||
("app.services.wireguard_deploy_service", "undeploy_tunnel"), ("app.services.port_forward_deploy_service", "undeploy_port_forward"),
|
||||
("app.services.outbound_nat_deploy_service", "undeploy_outbound_nat"), ("app.services.one_to_one_deploy_service", "undeploy_one_to_one"),
|
||||
("app.services.ipsec_deploy_service", "undeploy_tunnel"),
|
||||
] + [(f"app.routers.{r}", "remove_previous_target") for r in
|
||||
("aliases", "port_forward", "one_to_one", "outbound_nat", "wireguard", "openvpn", "ipsec")]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("module,name", _UNDEPLOY_SITES)
|
||||
def test_every_undeploy_and_target_removal_checks_for_an_open_cleanup_first(module, name):
|
||||
import importlib
|
||||
import inspect
|
||||
source = inspect.getsource(getattr(importlib.import_module(module), name))
|
||||
lock = min(i for i in (source.find("ensure_no_open_cleanup_on("), source.find("ensure_no_open_cleanup(")) if i >= 0)
|
||||
first_write = source.find("opnsense_client.")
|
||||
first_write = min(i for i in (first_write, source.find("client.del_"), source.find("_client.del")) if i >= 0)
|
||||
assert 0 <= lock < first_write, (module, name)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_alias_delete_and_target_removal_wait_for_the_cleanup(client, env):
|
||||
from app.models.alias_deployment import AliasDeployment
|
||||
deployed = await client.post("/api/aliases", json={"name": "cl_gone", "type": "host", "content": "203.0.113.5",
|
||||
"target_host_ids": [str(env.host_id)]})
|
||||
former = await client.post("/api/aliases", json={"name": "cl_former", "type": "host", "content": "203.0.113.6",
|
||||
"target_host_ids": []})
|
||||
assert deployed.status_code == 201 and former.status_code == 201, (deployed.text, former.text)
|
||||
async with SessionLocal() as db:
|
||||
for alias in (deployed.json(), former.json()):
|
||||
db.add(AliasDeployment(alias_id=uuid_lib.UUID(alias["id"]), host_id=env.host_id, status="success",
|
||||
opnsense_alias_uuid="aaaaaaaa-1111-4111-8111-111111111111", last_payload={"name": alias["name"]}))
|
||||
await db.commit()
|
||||
preview = (await client.get(f"/api/aliases/{former.json()['id']}/preview")).json()
|
||||
assert next(h for h in preview["hosts"] if h["host_id"] == str(env.host_id))["can_remove"], preview
|
||||
await _open(client, env)
|
||||
writes = list(env.fw.writes())
|
||||
r = await client.delete(f"/api/aliases/{deployed.json()['id']}")
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
|
||||
preview = (await client.get(f"/api/aliases/{former.json()['id']}/preview")).json()
|
||||
r = await client.post(f"/api/aliases/{former.json()['id']}/deployments/{env.host_id}/remove", json={"preview_token": preview["token"]})
|
||||
assert r.status_code == 409 and r.json()["detail"]["code"] == "OPEN_CLEANUP_BLOCKS_DEPLOY", r.text
|
||||
assert env.fw.writes() == writes
|
||||
@@ -0,0 +1,193 @@
|
||||
"""Pure parts of the rule cleanup (1.31): engine.match_rule, the management-access protection, the infrastructure
|
||||
exclusion, the confidence caps and the cluster usage merge. No database, no firewall."""
|
||||
import ipaddress
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from app.services import rule_hit_service
|
||||
from app.services.rule_analysis import cleanup, protection
|
||||
from app.services.rule_analysis.context import Context, IfInfo
|
||||
from app.services.rule_analysis.engine import Query, match_rule
|
||||
from app.services.rule_analysis.model import Endpoint, NormRule
|
||||
from app.services.rule_cleanup_service import selectable_level
|
||||
|
||||
NOW = datetime(2026, 10, 22, 12, 0, tzinfo=timezone.utc)
|
||||
ip = ipaddress.ip_address
|
||||
net = ipaddress.ip_network
|
||||
|
||||
|
||||
def ctx(**kw) -> Context:
|
||||
interfaces = {
|
||||
"wan": IfInfo("wan", "vtnet0", "WAN", nets=[net("203.0.113.0/24")], addrs=[ip("203.0.113.57")], has_gateway=True),
|
||||
"lan": IfInfo("lan", "vtnet1", "LAN", nets=[net("10.20.30.0/24")], addrs=[ip("10.20.30.2")], vips=[ip("10.20.30.1")]),
|
||||
}
|
||||
return Context(rows=[], pf=None, interfaces=interfaces, groups={}, aliases=kw.get("aliases", {}), port_forwards=[])
|
||||
|
||||
|
||||
def nrule(*, src="", dst="", dport="", protos=frozenset({"6"}), interfaces=frozenset({"wan"}), action="pass", negated=False,
|
||||
tagged="", uuid="u1") -> NormRule:
|
||||
items = [i for i in src.split(",") if i]
|
||||
return NormRule(uuid=uuid, label=uuid, description=uuid, order=(400000, 1), order_known=True, section="interface",
|
||||
enabled=True, action=action, quick=True, interfaces=set(interfaces), if_not=False, direction="in",
|
||||
af={4}, protos=None if protos is None else set(protos), src=Endpoint(raw=src, negated=negated, items=items),
|
||||
dst=Endpoint(raw=dst, items=[i for i in dst.split(",") if i]), sport="", dport=dport, tagged=tagged)
|
||||
|
||||
|
||||
SOURCES = [net("203.0.113.10/32")]
|
||||
|
||||
|
||||
def check(rule, c=None, sources=SOURCES):
|
||||
c = c or ctx()
|
||||
targets, ports = protection.management_targets("https://203.0.113.57:4443", c)
|
||||
return protection.management_check(rule, c, sources, targets, ports)
|
||||
|
||||
|
||||
def test_targets_are_the_api_address_and_port_plus_every_firewall_address():
|
||||
targets, ports = protection.management_targets("https://203.0.113.57:4443", ctx())
|
||||
assert targets[0] == ip("203.0.113.57") and ip("10.20.30.1") in targets and ip("127.0.0.1") not in targets
|
||||
assert ports == [4443, 443, 80, 22]
|
||||
|
||||
|
||||
def test_a_rule_for_the_management_path_is_protected():
|
||||
assert check(nrule(src="203.0.113.0/24", dport="443"))["result"] == "true"
|
||||
assert check(nrule(src="", dport="4443"))["result"] == "true" # any source, API port
|
||||
assert check(nrule(src="203.0.113.10", dport="", protos=None))["result"] == "true"
|
||||
|
||||
|
||||
def test_only_a_proven_non_match_is_removable():
|
||||
assert check(nrule(src="10.20.30.0/24", dport="443"))["result"] == "false" # other source
|
||||
assert check(nrule(src="203.0.113.0/24", dport="8081"))["result"] == "false" # other port
|
||||
assert check(nrule(src="203.0.113.0/24", dport="443", protos={"17"}))["result"] == "false" # udp
|
||||
|
||||
|
||||
def test_unknown_is_protected(): # case 18
|
||||
unknown = check(nrule(src="tbl", dport="443"), ctx(aliases={"tbl": {"name": "tbl", "type": "urltable"}}))
|
||||
assert unknown["result"] == "unknown" and unknown["match"]["port"] in (4443, 443)
|
||||
tagged = check(nrule(src="", dport="443", tagged="mark"))
|
||||
assert tagged["result"] == "unknown" and "TAGGED" in tagged["reasons"]
|
||||
q = Query(src=ip("203.0.113.10"), dst=ip("203.0.113.57"), proto="6", proto_label="tcp", dport=443)
|
||||
assert match_rule(nrule(interfaces={"nosuchgroup"}), q, ctx(), iface="wan").value is None
|
||||
|
||||
|
||||
def test_network_sources_are_judged_for_every_address():
|
||||
small = [net("203.0.113.0/28")]
|
||||
assert check(nrule(src="203.0.113.14", dport="443"), sources=small)["result"] == "true" # one address inside
|
||||
assert check(nrule(src="203.0.113.20", dport="443"), sources=small)["result"] == "false"
|
||||
big = [net("10.0.0.0/8")]
|
||||
assert check(nrule(src="10.20.30.0/24", dport="443"), sources=big)["result"] == "true" # ranges overlap
|
||||
assert check(nrule(src="192.0.2.0/24", dport="443"), sources=big)["result"] == "false"
|
||||
assert check(nrule(src="lan", dport="443"), sources=big)["result"] == "true" # interface network
|
||||
assert check(nrule(src="10.0.0.0/8", dport="443", negated=True), sources=big)["result"] == "false" # !whole net
|
||||
assert check(nrule(src="tbl", dport="443"), ctx(aliases={"tbl": {"type": "urltable"}}), sources=big)["result"] == "unknown"
|
||||
|
||||
|
||||
def test_no_query_possible_is_protected():
|
||||
got = check(nrule(src="", dport="443"), sources=[net("2001:db8::/64")]) # no IPv6 address on the firewall
|
||||
assert got["result"] == "unknown" and got["reasons"] == ["NO_MANAGEMENT_QUERY"]
|
||||
|
||||
|
||||
def test_carp_and_pfsync_rules_are_infrastructure():
|
||||
c = ctx()
|
||||
assert protection.infrastructure_reason(nrule(protos={"112"}), c, hasync=None, clustered=False) == "infrastructure"
|
||||
assert protection.infrastructure_reason(nrule(protos={"240"}), c, hasync=None, clustered=False) == "infrastructure"
|
||||
lan_any = nrule(src="10.20.30.0/24", protos=None, interfaces={"lan"})
|
||||
assert protection.infrastructure_reason(lan_any, c, hasync=None, clustered=False) == "infrastructure" # CARP adverts
|
||||
lan_tcp = nrule(src="10.20.30.0/24", dport="8081", interfaces={"lan"})
|
||||
assert protection.infrastructure_reason(lan_tcp, c, hasync=None, clustered=True) is None
|
||||
wan_any = nrule(src="", protos=None, interfaces={"wan"})
|
||||
off = {"hasync": {"pfsyncinterface": {"": {"selected": 1}, "wan": {"selected": 0}}, "pfsyncpeerip": ""}}
|
||||
on = {"hasync": {"pfsyncinterface": {"": {"selected": 0}, "wan": {"selected": 1}}, "pfsyncpeerip": "203.0.113.58"}}
|
||||
assert protection.infrastructure_reason(wan_any, c, hasync=off, clustered=True) is None
|
||||
assert protection.infrastructure_reason(wan_any, c, hasync=on, clustered=True) == "infrastructure"
|
||||
# HA settings not readable on a cluster node: every interface counts as a possible pfsync interface
|
||||
assert protection.infrastructure_reason(wan_any, c, hasync=None, clustered=True) == "infrastructure"
|
||||
|
||||
|
||||
def test_confidence_caps():
|
||||
rule = nrule()
|
||||
unused = {"categories": [{"kind": "unused", "confidence": "sure"}]}
|
||||
assert selectable_level(unused, rule, unused_days=30, external=False)[0] == "sure"
|
||||
assert selectable_level(unused, rule, unused_days=29, external=False)[0] == "probable"
|
||||
assert selectable_level(unused, nrule(action="block"), unused_days=90, external=False)[0] == "probable"
|
||||
assert selectable_level(unused, nrule(action="reject"), unused_days=90, external=False)[0] == "probable"
|
||||
assert selectable_level(unused, rule, unused_days=90, external=True)[0] == "probable"
|
||||
both = {"categories": [{"kind": "unused", "confidence": "unknown"}, {"kind": "duplicate", "confidence": "sure"}]}
|
||||
assert selectable_level(both, nrule(action="block"), unused_days=7, external=False)[0] == "sure" # best category
|
||||
assert selectable_level({"categories": [{"kind": "shadowed", "confidence": "unknown"}]}, rule, unused_days=30, external=False)[0] == "unknown"
|
||||
assert selectable_level({"categories": [{"kind": "disabled_long", "confidence": "sure"}]}, rule, unused_days=30, external=False)[0] is None
|
||||
|
||||
|
||||
class Hit:
|
||||
def __init__(self, uuid, first_days, hit_days=None, sampled_minutes=10, enabled=True):
|
||||
self.rule_uuid, self.enabled = uuid, enabled
|
||||
self.first_seen_at = NOW - timedelta(days=first_days)
|
||||
self.last_hit_at = None if hit_days is None else NOW - timedelta(days=hit_days)
|
||||
self.last_sampled_at = NOW - timedelta(minutes=sampled_minutes)
|
||||
|
||||
|
||||
def test_cluster_usage_merge(): # case 16
|
||||
master = {"r": Hit("r", 100, hit_days=1)}
|
||||
backup = {"r": Hit("r", 100)}
|
||||
merged = rule_hit_service.merge_cluster_hits(backup, [master])
|
||||
assert rule_hit_service.classify(merged["r"], NOW, 30) == "used"
|
||||
both_idle = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{"r": Hit("r", 20)}])
|
||||
assert rule_hit_service.classify(both_idle["r"], NOW, 30) == "collecting" # window = latest start
|
||||
stale = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{"r": Hit("r", 100, sampled_minutes=60 * 48)}])
|
||||
assert stale["r"].last_sampled_at == NOW - timedelta(hours=48) # as fresh as the oldest node
|
||||
missing = rule_hit_service.merge_cluster_hits({"r": Hit("r", 100)}, [{}])
|
||||
assert missing["r"].peer_missing and rule_hit_service.classify(missing["r"], NOW, 30) == "unused"
|
||||
rule = nrule()
|
||||
rule.loaded = True
|
||||
assert cleanup.unused_confidence(rule, missing["r"], NOW) == "probable"
|
||||
assert cleanup.unused_confidence(rule, both_idle["r"], NOW) == "sure"
|
||||
|
||||
|
||||
def _live_rule(**kw):
|
||||
"""A rule normalized inside the live fixture (tests/fixtures/rule_analysis_fw.json: built-in groups wireguard,
|
||||
openvpn, enc0 with members "" and unassigned devices without identifier)."""
|
||||
from app.services.rule_analysis.normalize import normalize
|
||||
from tests.test_rule_analysis import build, mvc
|
||||
row = mvc("bbbbbbbb-0000-4000-8000-000000000001", seq=900, proto="tcp", **kw)
|
||||
c = build([row])
|
||||
rules, _ = normalize(c)
|
||||
return next(r for r in rules if r.uuid == row["uuid"]), c
|
||||
|
||||
|
||||
def test_rules_on_builtin_groups_without_known_members_are_protected(): # review finding 1
|
||||
for group in ("wireguard", "openvpn", "enc0"):
|
||||
rule, c = _live_rule(interface=group, src="203.0.113.10", dst="(self)", dport="443")
|
||||
got = check(rule, c)
|
||||
assert got["result"] == "unknown" and "IFACE_MEMBERS_UNKNOWN" in got["reasons"], (group, got)
|
||||
assert got["match"]["interface"] == protection.UNASSIGNED
|
||||
# the virtual ingress does not make every rule protected
|
||||
rule, c = _live_rule(interface="wireguard", src="203.0.113.10", dst="(self)", dport="8081")
|
||||
assert check(rule, c)["result"] == "false"
|
||||
rule, c = _live_rule(interface="lan", src="10.20.30.0/24", dst="(self)", dport="443")
|
||||
assert check(rule, c)["result"] == "false"
|
||||
# floating "any interface" and "not lan" also match a packet on wg0
|
||||
rule, c = _live_rule(interface="", src="203.0.113.10", dst="(self)", dport="22")
|
||||
assert check(rule, c)["result"] == "true"
|
||||
rule, c = _live_rule(interface="lan", interfacenot="1", src="203.0.113.10", dst="(self)", dport="22")
|
||||
assert check(rule, c)["result"] == "true"
|
||||
|
||||
|
||||
def test_unassigned_interface_outcome():
|
||||
c = ctx()
|
||||
c.groups = {"mygroup": {"lan"}, "wireguard": set()}
|
||||
assert protection.unassigned_interface(nrule(interfaces={"lan"}), c).value is False
|
||||
assert protection.unassigned_interface(nrule(interfaces={"mygroup"}), c).value is False
|
||||
assert protection.unassigned_interface(nrule(interfaces={"wireguard"}), c).value is None
|
||||
assert protection.unassigned_interface(nrule(interfaces={"nosuch"}), c).value is None
|
||||
assert protection.unassigned_interface(nrule(interfaces=set()), c).value is True
|
||||
|
||||
|
||||
def test_redacted_plan_names_no_source_port_or_firewall_address(): # review finding 8
|
||||
from app.services.rule_cleanup_service import redact
|
||||
plan = {"kind": "enable", "sources": ["203.0.113.10/32"], "ports": [2222], "nodes": [
|
||||
{"host_id": "h", "base_url": "https://203.0.113.57", "items": [
|
||||
{"uuid": "u", "protection": {"result": "true", "queries": 3, "reasons": [],
|
||||
"match": {"source": "203.0.113.10/32", "destination": "203.0.113.57"}}}]}]}
|
||||
out = redact({"plan": plan, "operation": {"plan": plan}})
|
||||
for p in (out["plan"], out["operation"]["plan"]):
|
||||
assert "203.0.113" not in str(p) and "2222" not in str(p)
|
||||
assert p["sources_configured"] is True and p["nodes"][0]["items"][0]["protection"]["result"] == "true"
|
||||
assert plan["sources"] and plan["nodes"][0]["base_url"] # the stored plan itself is untouched
|
||||
@@ -0,0 +1,391 @@
|
||||
"""Coverage (shadowed/duplicate) and cleanup candidates: pure functions on NormRule (1.30, backlog B4 part 1).
|
||||
|
||||
Rules are built the way the live ``searchRule`` delivers them (tests/test_rule_analysis.py: fixture of a real
|
||||
26.7 firewall plus synthetic MVC rows with matching pf lines). Cases that must stay red when the logic is broken
|
||||
are marked "mutation" in their docstring.
|
||||
"""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from app.services.rule_analysis import cleanup, coverage
|
||||
from app.services.rule_analysis.normalize import normalize
|
||||
from tests.test_rule_analysis import build, mvc
|
||||
|
||||
NOW = datetime(2026, 10, 22, 12, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def overlaps(rows, **kw):
|
||||
ctx = build(rows, **kw)
|
||||
rules, notes = normalize(ctx)
|
||||
found = coverage.analyse(rules, ctx, order_guessed=coverage.order_doubtful(notes))
|
||||
return {o.rule.uuid: o for o in found}
|
||||
|
||||
|
||||
# ── coverage ──────────────────────────────────────────────────────────────
|
||||
|
||||
def test_a_narrower_rule_behind_a_broad_quick_rule_is_shadowed_and_sure():
|
||||
found = overlaps([
|
||||
mvc("broad", seq=1000, action="block", src="10.20.30.0/24"),
|
||||
mvc("narrow", seq=1001, src="10.20.30.0/25", dport="443", proto="tcp"),
|
||||
])
|
||||
assert found["narrow"].kind == "shadowed" and found["narrow"].by.uuid == "broad" and found["narrow"].confidence == "sure"
|
||||
assert "broad" not in found
|
||||
|
||||
|
||||
def test_an_identical_rule_with_the_same_action_is_a_duplicate_a_different_action_is_only_shadowed():
|
||||
found = overlaps([
|
||||
mvc("first", seq=1000, src="10.20.30.0/24", dport="443", proto="tcp"),
|
||||
mvc("second", seq=1001, src="10.20.30.0/24", dport="443", proto="tcp"),
|
||||
mvc("other", seq=1002, src="10.20.30.0/24", dport="443", proto="tcp", action="block"),
|
||||
])
|
||||
assert (found["second"].kind, found["second"].by.uuid) == ("duplicate", "first")
|
||||
assert found["other"].kind == "shadowed"
|
||||
|
||||
|
||||
def test_the_wider_later_rule_is_not_covered_by_the_narrower_earlier_one():
|
||||
found = overlaps([
|
||||
mvc("narrow", seq=1000, src="10.20.30.64/26"),
|
||||
mvc("wide", seq=1001, src="10.20.30.0/24"),
|
||||
])
|
||||
assert found == {}
|
||||
|
||||
|
||||
def test_only_a_quick_rule_covers():
|
||||
"""mutation: ignoring `quick` makes the last-match-wins rule look like a cover."""
|
||||
found = overlaps([
|
||||
mvc("floaty", seq=1000, quick="0", action="block"),
|
||||
mvc("later", seq=1001, src="10.20.30.0/24"),
|
||||
])
|
||||
assert found == {}
|
||||
|
||||
|
||||
def test_sets_ranges_and_protocol_families():
|
||||
rows = [
|
||||
mvc("all-ports", seq=1000, proto="tcp/udp", ipprotocol="inet46", dport="1000:2000", src="10.20.30.0/24"),
|
||||
mvc("inside", seq=1001, proto="tcp", ipprotocol="inet", dport="1500", src="10.20.30.5"),
|
||||
mvc("udp-edge", seq=1002, proto="udp", ipprotocol="inet", dport="2000", src="10.20.30.0/24"),
|
||||
mvc("outside", seq=1003, proto="tcp", ipprotocol="inet", dport="2001", src="10.20.30.5"),
|
||||
mvc("v6-inside-v46", seq=1004, proto="tcp", ipprotocol="inet6", dport="1500", src="2001:db8::5"),
|
||||
mvc("no-port", seq=1005, proto="tcp", ipprotocol="inet", src="10.20.30.5"),
|
||||
mvc("icmp", seq=1006, proto="icmp", ipprotocol="inet", src="10.20.30.5"),
|
||||
]
|
||||
found = overlaps(rows)
|
||||
assert set(found) == {"inside", "udp-edge"} # "v6-inside-v46": an IPv6 address is outside the 10.20.30.0/24 source
|
||||
assert all(o.by.uuid == "all-ports" for o in found.values())
|
||||
|
||||
|
||||
def test_inet_does_not_cover_inet46_but_inet46_covers_inet6():
|
||||
found = overlaps([mvc("v4", seq=1000, ipprotocol="inet"), mvc("both", seq=1001, ipprotocol="inet46")])
|
||||
assert found == {}
|
||||
found = overlaps([mvc("both", seq=1000, ipprotocol="inet46"), mvc("v6", seq=1001, ipprotocol="inet6")])
|
||||
assert found["v6"].by.uuid == "both"
|
||||
|
||||
|
||||
def test_a_narrow_earlier_rule_never_covers_an_any_later_one():
|
||||
found = overlaps([mvc("web", seq=1000, proto="tcp", dport="443"), mvc("any", seq=1001)])
|
||||
assert found == {}
|
||||
|
||||
|
||||
def test_interface_sets_and_floating_order():
|
||||
rows = [
|
||||
mvc("float-quick", prio=200000, seq=5, interface="lan,wan", action="block", src="10.20.30.0/24", loaded=True),
|
||||
mvc("on-lan", seq=1000, interface="lan", src="10.20.30.7"),
|
||||
mvc("on-wan", seq=1001, interface="wan", src="10.20.30.7"),
|
||||
mvc("on-all", seq=1002, interface="", src="10.20.30.7"),
|
||||
]
|
||||
found = overlaps(rows)
|
||||
assert found["on-lan"].by.uuid == "float-quick" and found["on-wan"].by.uuid == "float-quick"
|
||||
assert "on-all" not in found # applies to interfaces the floating rule does not name
|
||||
|
||||
|
||||
def test_a_later_floating_rule_cannot_be_covered_by_an_interface_rule():
|
||||
"""mutation: sorting by `sequence` instead of `sort_order` would put the interface rule first."""
|
||||
found = overlaps([
|
||||
mvc("interface-rule", prio=400000, seq=1, interface="lan", src="any"),
|
||||
mvc("floating-rule", prio=200000, seq=9, interface="lan,wan", src="10.20.30.7"),
|
||||
])
|
||||
assert "floating-rule" not in found
|
||||
# and the other way round: the floating rule is evaluated first and wins
|
||||
found = overlaps([
|
||||
mvc("interface-rule", prio=400000, seq=9, interface="lan", src="10.20.30.7"),
|
||||
mvc("floating-rule", prio=200000, seq=1, interface="lan,wan", src="10.20.30.0/24"),
|
||||
])
|
||||
assert found["interface-rule"].by.uuid == "floating-rule"
|
||||
|
||||
|
||||
def test_a_group_rule_comes_before_the_interface_rule():
|
||||
found = overlaps([
|
||||
mvc("plain", prio=400000, seq=1, interface="lan", src="10.20.30.7"),
|
||||
mvc("grouped", prio=300005, seq=900, interface="lan", src="10.20.30.0/24"),
|
||||
])
|
||||
assert found["plain"].by.uuid == "grouped"
|
||||
|
||||
|
||||
def test_conditions_the_engine_cannot_compare_prevent_a_claim():
|
||||
for extra in ({"sched": "work-hours"}, {"tagged": "mark"}, {"received-on": "wan"}):
|
||||
found = overlaps([mvc("cond", seq=1000, **extra), mvc("later", seq=1001, src="10.20.30.5")])
|
||||
assert found == {}, extra
|
||||
|
||||
|
||||
def test_disabled_and_unloaded_rules_do_not_cover():
|
||||
found = overlaps([mvc("off", seq=1000, enabled="0"), mvc("later", seq=1001, src="10.20.30.5")])
|
||||
assert found == {}
|
||||
found = overlaps([mvc("not-loaded", seq=1000, loaded=False), mvc("later", seq=1001, src="10.20.30.5")])
|
||||
assert found == {}
|
||||
|
||||
|
||||
def test_negation_is_compared_exactly():
|
||||
rows = [
|
||||
mvc("not-a", seq=1000, source_not="1", src="10.20.30.0/24"),
|
||||
mvc("same", seq=1001, source_not="1", src="10.20.30.0/24"),
|
||||
mvc("plain", seq=1002, src="10.20.30.5"),
|
||||
]
|
||||
found = overlaps(rows)
|
||||
assert set(found) == {"same"} and found["same"].kind == "duplicate"
|
||||
|
||||
|
||||
def test_a_negated_source_covers_a_negated_source_that_excludes_more():
|
||||
"""mutation: using plain containment for '!' lists claims the wrong direction."""
|
||||
narrow_exclusion = mvc("not-narrow", seq=1000, source_not="1", src="10.1.0.0/16")
|
||||
wide_exclusion = mvc("not-wide", seq=1001, source_not="1", src="10.0.0.0/8")
|
||||
assert overlaps([narrow_exclusion, wide_exclusion])["not-wide"].by.uuid == "not-narrow"
|
||||
assert overlaps([mvc("not-wide", seq=1000, source_not="1", src="10.0.0.0/8"),
|
||||
mvc("not-narrow", seq=1001, source_not="1", src="10.1.0.0/16")]) == {}
|
||||
|
||||
|
||||
def test_a_negated_rule_does_not_cover_a_plain_one_and_the_reverse():
|
||||
"""mutation: dropping the negation guard lets '!host' appear to cover a network that contains the host."""
|
||||
assert overlaps([mvc("not-host", seq=1000, source_not="1", src="10.20.30.5"), mvc("net", seq=1001, src="10.20.30.0/24")]) == {}
|
||||
assert overlaps([mvc("net", seq=1000, src="10.20.30.0/24"), mvc("not-host", seq=1001, source_not="1", src="10.20.30.5")]) == {}
|
||||
|
||||
|
||||
def test_order_comes_from_sort_order_with_and_without_the_pf_ruleset():
|
||||
"""mutation: sorting by sequence alone puts the interface rule (sequence 1) before the group rule."""
|
||||
for pf in (True, False):
|
||||
found = overlaps([
|
||||
mvc("plain", prio=400000, seq=1, interface="lan", src="10.20.30.7"),
|
||||
mvc("grouped", prio=300005, seq=900, interface="lan", src="10.20.30.0/24"),
|
||||
], pf=pf)
|
||||
assert found["plain"].by.uuid == "grouped", pf
|
||||
|
||||
|
||||
def test_equal_sort_order_is_decided_by_the_pf_line_numbers_not_the_uuid():
|
||||
"""mutation: sorting ties by uuid turns the effective block rule into a 'sure' shadow of the pass rule."""
|
||||
rows = [mvc("zzz-narrow", seq=1000, src="10.20.30.0/24", dport="443", proto="tcp"),
|
||||
mvc("aaa-broad", seq=1000, action="block")]
|
||||
assert overlaps(rows) == {} # pf evaluates zzz-narrow first; the broad rule is not covered by the narrow one
|
||||
rows.reverse() # pf now evaluates the broad quick block first
|
||||
found = overlaps(rows)
|
||||
assert found["zzz-narrow"].by.uuid == "aaa-broad" and found["zzz-narrow"].confidence == "sure"
|
||||
|
||||
|
||||
def test_equal_sort_order_without_pf_data_is_at_most_probable():
|
||||
rows = [mvc("aaa-broad", seq=1000, action="block"), mvc("zzz-narrow", seq=1000, src="10.20.30.0/24")]
|
||||
assert overlaps(rows, pf=False)["zzz-narrow"].confidence == "probable"
|
||||
|
||||
|
||||
def test_a_pf_order_mismatch_or_guessed_order_caps_confidence():
|
||||
"""mutation: removing ORDER_MISMATCH from the doubt set."""
|
||||
assert coverage.order_doubtful(["ORDER_MISMATCH"]) and coverage.order_doubtful(["ORDER_GUESSED", "X"])
|
||||
assert not coverage.order_doubtful(["PF_RULESET_UNAVAILABLE"]) and not coverage.order_doubtful([])
|
||||
ctx = build([mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")])
|
||||
rules, _ = normalize(ctx)
|
||||
assert coverage.analyse(rules, ctx, order_guessed=True)[0].confidence == "probable"
|
||||
|
||||
|
||||
def test_an_unlabeled_pf_line_with_conditions_is_never_a_cover():
|
||||
"""mutation: ignoring proto/port/... in the unlabeled line lets a 'tcp only' block shadow every protocol."""
|
||||
from app.services.rule_analysis import pf_ruleset
|
||||
from app.services.rule_analysis.normalize import antispoof_rule
|
||||
plain = antispoof_rule(pf_ruleset.parse_line(5, "block drop in quick inet from 10.20.30.0/24 to any"), (400000, 0))
|
||||
assert plain.extra_notes == []
|
||||
for text in ("block drop in quick inet proto tcp from 10.20.30.0/24 to any",
|
||||
"block drop in quick inet from 10.20.30.0/24 port 53 to any",
|
||||
"block drop in quick inet from 10.20.30.0/24 to any flags S/SA",
|
||||
"block drop in quick inet from 10.20.30.0/24 to any user 0",
|
||||
"block drop in quick inet from 10.20.30.0/24 to any tagged x",
|
||||
"block drop in quick inet from 10.20.30.0/24 to any received-on vtnet0"):
|
||||
assert antispoof_rule(pf_ruleset.parse_line(6, text), (400000, 0)).extra_notes == ["UNMODELLED_CONDITION"], text
|
||||
victim = antispoof_rule(pf_ruleset.parse_line(7, "block drop in quick inet from 10.20.30.5 to any"), (400000, 1))
|
||||
victim.extra_notes, victim.interfaces = [], set()
|
||||
assert coverage.covers_fields(plain, victim) == "sure"
|
||||
conditional = antispoof_rule(pf_ruleset.parse_line(8, "block drop in quick inet proto tcp from 10.20.30.0/24 to any"), (400000, 0))
|
||||
assert coverage.covers_fields(conditional, victim) is None
|
||||
|
||||
|
||||
def test_a_disabled_alias_is_an_empty_table_and_never_evaluated():
|
||||
"""mutation: using the content of a disabled alias claims coverage by something pf does not have."""
|
||||
aliases = {"OFF": {"type": "host", "content": "10.20.30.0/24", "enabled": "0"},
|
||||
"ON": {"type": "host", "content": "10.20.30.0/24", "enabled": "1"}}
|
||||
rows = [mvc("a", seq=1000, src="OFF", dport="80", proto="tcp"), mvc("b", seq=1001, src="10.20.30.5", dport="80", proto="tcp")]
|
||||
assert overlaps(rows, aliases=aliases) == {}
|
||||
rows[0] = mvc("a", seq=1000, src="ON", dport="80", proto="tcp")
|
||||
assert overlaps(rows, aliases=aliases)["b"].confidence == "probable"
|
||||
|
||||
|
||||
def test_aliases_same_name_sure_other_name_probable_unresolvable_never():
|
||||
aliases = {
|
||||
"WEB": {"type": "host", "content": "10.20.30.5\n10.20.30.6", "enabled": "1"},
|
||||
"WEB_ONE": {"type": "host", "content": "10.20.30.5", "enabled": "1"},
|
||||
"NESTED": {"type": "network", "content": "WEB_ONE", "enabled": "1"},
|
||||
"FEED": {"type": "urltable", "content": "https://example.invalid/list", "enabled": "1"},
|
||||
}
|
||||
rows = [
|
||||
mvc("by-name", seq=1000, src="WEB", dport="80", proto="tcp"),
|
||||
mvc("same", seq=1001, src="WEB", dport="80", proto="tcp"),
|
||||
mvc("static", seq=1002, src="WEB_ONE", dport="80", proto="tcp"),
|
||||
mvc("nested", seq=1003, src="NESTED", dport="80", proto="tcp"),
|
||||
mvc("table-a", seq=1004, src="FEED", dport="81", proto="tcp"),
|
||||
mvc("table-b", seq=1005, src="WEB_ONE", dport="81", proto="tcp"),
|
||||
]
|
||||
found = overlaps(rows, aliases=aliases)
|
||||
assert (found["same"].kind, found["same"].confidence) == ("duplicate", "sure")
|
||||
assert (found["static"].kind, found["static"].confidence) == ("shadowed", "probable")
|
||||
assert found["nested"].confidence == "probable"
|
||||
assert "table-b" not in found # FEED has no static content: nothing is claimed
|
||||
|
||||
|
||||
def test_port_alias_comparison():
|
||||
aliases = {"P1": {"type": "port", "content": "80\n443", "enabled": "1"}, "P2": {"type": "port", "content": "443", "enabled": "1"}}
|
||||
rows = [mvc("a", seq=1000, proto="tcp", dport="P1"), mvc("b", seq=1001, proto="tcp", dport="P2"),
|
||||
mvc("c", seq=1002, proto="tcp", dport="P1")]
|
||||
found = overlaps(rows, aliases=aliases)
|
||||
assert (found["b"].kind, found["b"].confidence) == ("shadowed", "probable")
|
||||
assert (found["c"].kind, found["c"].confidence) == ("duplicate", "sure")
|
||||
|
||||
|
||||
def test_without_the_loaded_pf_ruleset_nothing_is_sure():
|
||||
"""What is not proven is never sure."""
|
||||
rows = [mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")]
|
||||
assert overlaps(rows, pf=False)["b"].confidence == "probable"
|
||||
assert overlaps(rows, pf=True)["b"].confidence == "sure"
|
||||
|
||||
|
||||
def test_a_guessed_order_lowers_the_confidence():
|
||||
rows = [mvc("a", seq=1000, src="10.20.30.0/24"), mvc("b", seq=1001, src="10.20.30.5")]
|
||||
for row in rows:
|
||||
row.pop("sort_order")
|
||||
found = overlaps(rows)
|
||||
assert found["b"].confidence == "probable"
|
||||
|
||||
|
||||
def test_the_unmodified_live_fixture_has_no_overlap_among_user_rules():
|
||||
ctx = build([])
|
||||
rules, _ = normalize(ctx)
|
||||
assert [o.rule.description for o in coverage.analyse(rules, ctx) if not o.rule.automatic] == []
|
||||
|
||||
|
||||
# ── candidates ────────────────────────────────────────────────────────────
|
||||
|
||||
def hit(*, enabled=True, first_days=100, last_hit_days=None, sampled_hours=1):
|
||||
return SimpleNamespace(
|
||||
enabled=enabled, first_seen_at=NOW - timedelta(days=first_days),
|
||||
last_hit_at=None if last_hit_days is None else NOW - timedelta(days=last_hit_days),
|
||||
last_sampled_at=NOW - timedelta(hours=sampled_hours), last_packets=0, last_bytes=0)
|
||||
|
||||
|
||||
def candidates(rows, *, hits=None, states=None, managed=None, days=30, **kw):
|
||||
ctx = build(rows, **kw)
|
||||
rules, notes = normalize(ctx)
|
||||
out = cleanup.build(rules, ctx, hits=hits or {}, states=states or {}, managed=managed or {}, now=NOW,
|
||||
unused_days=days, order_guessed=coverage.order_doubtful(notes))
|
||||
return {c["uuid"]: c for c in out}
|
||||
|
||||
|
||||
def kinds(candidate):
|
||||
return {c["kind"]: c["confidence"] for c in candidate["categories"]}
|
||||
|
||||
|
||||
def test_unused_needs_the_whole_window_without_a_hit():
|
||||
rows = [mvc("quiet", seq=1000, proto="tcp", dport="1"), mvc("busy", seq=1001, proto="tcp", dport="2"),
|
||||
mvc("young", seq=1002, proto="tcp", dport="3"), mvc("never", seq=1003, proto="tcp", dport="4")]
|
||||
hits = {"quiet": hit(last_hit_days=60), "busy": hit(last_hit_days=3), "young": hit(first_days=5), "never": hit()}
|
||||
got = candidates(rows, hits=hits)
|
||||
assert set(got) == {"quiet", "never"}
|
||||
assert kinds(got["quiet"]) == {"unused": "sure"}
|
||||
|
||||
|
||||
def test_the_threshold_is_a_parameter():
|
||||
rows = [mvc("r", seq=1000)]
|
||||
hits = {"r": hit(first_days=100, last_hit_days=45)}
|
||||
assert "r" not in candidates(rows, hits=hits, days=90)
|
||||
assert "r" in candidates(rows, hits=hits, days=30)
|
||||
assert "r" not in candidates(rows, hits={"r": hit(first_days=20)}, days=30) # watched less than the window
|
||||
|
||||
|
||||
def test_unused_is_never_sure_without_evidence_the_rule_can_be_hit():
|
||||
"""mutation: dropping the loaded/schedule/freshness checks lets every unused rule through as sure."""
|
||||
rows = [mvc("plain", seq=1000, proto="tcp", dport="1"), mvc("timed", seq=1001, sched="work-hours", proto="tcp", dport="2"),
|
||||
mvc("unloaded", seq=1002, loaded=False, proto="tcp", dport="3"), mvc("stale", seq=1003, proto="tcp", dport="4")]
|
||||
hits = {u: hit() for u in ("plain", "timed", "unloaded")} | {"stale": hit(sampled_hours=72)}
|
||||
got = candidates(rows, hits=hits)
|
||||
assert kinds(got["plain"])["unused"] == "sure"
|
||||
assert kinds(got["timed"])["unused"] == "probable"
|
||||
assert kinds(got["unloaded"])["unused"] == "unknown"
|
||||
assert kinds(got["stale"])["unused"] == "unknown"
|
||||
without_pf = candidates(rows, hits=hits, pf=False)
|
||||
assert kinds(without_pf["plain"])["unused"] == "probable"
|
||||
|
||||
|
||||
def test_a_rule_can_be_in_several_categories():
|
||||
rows = [mvc("broad", seq=1000, src="10.20.30.0/24"), mvc("narrow", seq=1001, src="10.20.30.5")]
|
||||
got = candidates(rows, hits={"narrow": hit()})
|
||||
assert kinds(got["narrow"]) == {"unused": "sure", "shadowed": "sure"}
|
||||
assert got["narrow"]["categories"][1]["by"]["uuid"] == "broad"
|
||||
|
||||
|
||||
def test_disabled_for_a_long_time_comes_from_the_observed_history():
|
||||
rows = [mvc("old-off", seq=1000, enabled="0"), mvc("new-off", seq=1001, enabled="0"), mvc("never-seen", seq=1002, enabled="0")]
|
||||
states = {
|
||||
"old-off": SimpleNamespace(present=True, changed_at=NOW - timedelta(days=40)),
|
||||
"new-off": SimpleNamespace(present=True, changed_at=NOW - timedelta(days=3)),
|
||||
}
|
||||
got = candidates(rows, states=states)
|
||||
assert set(got) == {"old-off"} and kinds(got["old-off"]) == {"disabled_long": "sure"}
|
||||
|
||||
|
||||
def test_disabled_rules_are_neither_unused_nor_shadowing_candidates_of_the_unused_kind():
|
||||
got = candidates([mvc("off", seq=1000, enabled="0")], hits={"off": hit(enabled=False)})
|
||||
assert got == {}
|
||||
|
||||
|
||||
def test_not_removable_reasons():
|
||||
rows = [mvc("plain", seq=1000, proto="tcp", dport="1"), mvc("managed-one", seq=1001, proto="tcp", dport="2"),
|
||||
mvc("legacy-one", seq=1002, legacy=True, proto="tcp", dport="3")]
|
||||
hits = {u: hit() for u in ("plain", "managed-one", "legacy-one")}
|
||||
got = candidates(rows, hits=hits, managed={"managed-one": {"kind": "rule", "id": "x", "name": "Managed"}})
|
||||
assert (got["plain"]["removable"], got["plain"]["not_removable_reason"]) == (True, None)
|
||||
assert (got["managed-one"]["removable"], got["managed-one"]["not_removable_reason"]) == (False, "managed")
|
||||
assert got["managed-one"]["managed_by"]["name"] == "Managed"
|
||||
assert (got["legacy-one"]["removable"], got["legacy-one"]["not_removable_reason"]) == (False, "legacy")
|
||||
|
||||
|
||||
def test_removal_block_for_automatic_rules():
|
||||
ctx = build([])
|
||||
rules, _ = normalize(ctx)
|
||||
automatic = next(r for r in rules if r.automatic and r.section != "antispoof")
|
||||
assert cleanup.removal_block(automatic, None) == "automatic"
|
||||
assert cleanup.removal_block(automatic, {"kind": "rule"}) == "automatic" # automatic wins
|
||||
|
||||
|
||||
def test_automatic_rules_never_appear_as_candidates():
|
||||
got = candidates([], hits={})
|
||||
assert got == {}
|
||||
|
||||
|
||||
def test_candidates_are_in_evaluation_order_and_counts_add_up():
|
||||
rows = [mvc("b", seq=1001, proto="tcp", dport="2"), mvc("a", seq=1000, proto="tcp", dport="1")]
|
||||
hits = {"a": hit(), "b": hit()}
|
||||
ctx = build(rows)
|
||||
rules, _ = normalize(ctx)
|
||||
out = cleanup.build(rules, ctx, hits=hits, states={}, managed={}, now=NOW)
|
||||
assert [c["uuid"] for c in out] == ["a", "b"]
|
||||
assert cleanup.summarize(out)["unused"] == 2
|
||||
|
||||
|
||||
def test_the_threshold_is_clamped():
|
||||
ctx = build([mvc("r", seq=1000)])
|
||||
rules, _ = normalize(ctx)
|
||||
assert cleanup.build(rules, ctx, hits={"r": hit(first_days=100)}, states={}, managed={}, now=NOW, unused_days=0) # 0 -> 1 day
|
||||
@@ -0,0 +1,627 @@
|
||||
"""Change history of live rules (1.30, backlog B4 part 1): observation loop logic, origin, idempotence, the
|
||||
history endpoint, the usage sampling via ruleStats and the cleanup endpoint/MCP tool.
|
||||
|
||||
Database tests drive ``rule_history_service`` with searchRule answers shaped like the live ones; the HTTP tests
|
||||
mock the firewall with respx.
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import uuid as uuid_lib
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.database import SessionLocal
|
||||
from app.models.audit import AuditEvent
|
||||
from app.models.host import OpnsenseHost
|
||||
from app.models.rule_hit import RuleHit
|
||||
from app.models.rule_history import LiveRuleState, RuleChangeEvent
|
||||
from app.services import rule_hit_service, rule_history_service as rhs
|
||||
from app.services.rule_analysis import context as ctx_mod
|
||||
from tests.test_api_tokens import _bearer, _make
|
||||
from tests.test_rule_analysis import FIXTURE, mvc
|
||||
from tests.test_rule_analysis_api import FW, _mock_fw
|
||||
from tests.test_rules import _create_host
|
||||
|
||||
LIVE = json.loads((Path(__file__).parent / "fixtures" / "rule_stats_live.json").read_text())
|
||||
T0 = datetime(2026, 10, 1, 8, 0, tzinfo=timezone.utc)
|
||||
R1, R2, R3 = "11111111-1111-4111-8111-111111111111", "22222222-2222-4222-8222-222222222222", "33333333-3333-4333-8333-333333333333"
|
||||
|
||||
|
||||
def answer(*rules, automatic=True):
|
||||
"""A searchRule answer: optional automatic rows plus MVC rows (internal helper keys removed)."""
|
||||
rows = [dict(r) for r in FIXTURE["rules"]["rows"] if r.get("is_automatic")][:3] if automatic else []
|
||||
for rule in rules:
|
||||
row = dict(rule)
|
||||
row.pop("_loaded", None)
|
||||
rows.append(row)
|
||||
return {"rows": rows}
|
||||
|
||||
|
||||
def rule(uuid, **kw):
|
||||
kw.setdefault("proto", "tcp")
|
||||
kw.setdefault("dport", "443")
|
||||
return mvc(uuid, **kw)
|
||||
|
||||
|
||||
async def host_of(client, respx_mock, name="fw1", url="https://fw1.test"):
|
||||
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
|
||||
return uuid_lib.UUID((await _create_host(client, name, url))["id"])
|
||||
|
||||
|
||||
async def observe(host_id, search, at):
|
||||
async with SessionLocal() as db:
|
||||
host = await db.get(OpnsenseHost, host_id)
|
||||
return await rhs.observe_rows(db, host, search, now=at)
|
||||
|
||||
|
||||
async def events(host_id):
|
||||
async with SessionLocal() as db:
|
||||
rows = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.host_id == host_id)
|
||||
.order_by(RuleChangeEvent.observed_at, RuleChangeEvent.seq))).all()
|
||||
return [(e.rule_uuid, e.kind, e.origin, e.seq) for e in rows]
|
||||
|
||||
|
||||
# ── pure helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
def test_clean_row_drops_derived_display_fields_and_the_fingerprint_ignores_them():
|
||||
row = {"uuid": R1, "enabled": "1", "description": "x", "%action": "Pass", "alias_meta_source_net": [{"value": "a"}],
|
||||
"#priority": 400000, "category_colors": []}
|
||||
other = dict(row, **{"%action": "Erlauben", "alias_meta_source_net": [{"value": "b"}]})
|
||||
assert rhs.clean_row(row) == {"uuid": R1, "enabled": "1", "description": "x"}
|
||||
assert rhs.fingerprint(rhs.clean_row(row)) == rhs.fingerprint(rhs.clean_row(other))
|
||||
assert rhs.fingerprint(rhs.clean_row(dict(row, description="y"))) != rhs.fingerprint(rhs.clean_row(row))
|
||||
|
||||
|
||||
def test_change_kind():
|
||||
base = {"uuid": R1, "enabled": "1", "description": "x"}
|
||||
assert rhs.change_kind(None, base) == "added" and rhs.change_kind(base, None) == "removed"
|
||||
assert rhs.change_kind(base, {**base, "enabled": "0"}) == "disabled"
|
||||
assert rhs.change_kind({**base, "enabled": "0"}, base) == "enabled"
|
||||
assert rhs.change_kind(base, {**base, "enabled": "0", "description": "y"}) == "changed"
|
||||
assert rhs.change_kind(base, {**base, "description": "y"}) == "changed"
|
||||
|
||||
|
||||
def test_diff_fields_is_short_and_exact():
|
||||
assert rhs.diff_fields({"a": 1, "b": 2}, {"a": 1, "b": 3, "c": 4}) == [
|
||||
{"field": "b", "before": 2, "after": 3}, {"field": "c", "before": None, "after": 4}]
|
||||
assert len(rhs.diff_fields({}, {str(i): i for i in range(40)})) == 12
|
||||
|
||||
|
||||
# ── observation ───────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_first_observation_stores_the_starting_point_without_events(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
result = await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0)
|
||||
assert result == {"baseline": True, "events": 0, "touched": []}
|
||||
assert await events(host) == []
|
||||
async with SessionLocal() as db:
|
||||
keys = {s.rule_uuid for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host))).all()}
|
||||
assert keys == {R1, R2, rhs.BASELINE_KEY} # automatic rules are not tracked
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_host_without_user_rules_still_gets_its_baseline(client, respx_mock):
|
||||
"""The first rule created afterwards must show up as 'added', not vanish into a second baseline."""
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(), T0)
|
||||
await observe(host, answer(rule(R1)), T0 + timedelta(minutes=15))
|
||||
assert await events(host) == [(R1, "added", "external", 1)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unchanged_rules_make_no_events_and_the_cycle_is_repeatable(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
search = answer(rule(R1))
|
||||
await observe(host, search, T0)
|
||||
for n in (1, 2, 3):
|
||||
assert (await observe(host, search, T0 + timedelta(minutes=15 * n)))["events"] == 0
|
||||
assert await events(host) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_added_changed_disabled_enabled_removed_and_added_again(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), step(1)) # added
|
||||
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(2)) # changed
|
||||
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001, enabled="0")), step(3)) # disabled
|
||||
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(4)) # enabled
|
||||
await observe(host, answer(rule(R1, dport="8443")), step(5)) # removed
|
||||
await observe(host, answer(rule(R1, dport="8443"), rule(R2, seq=1001)), step(6)) # added again
|
||||
assert await events(host) == [
|
||||
(R2, "added", "external", 1), (R1, "changed", "external", 2), (R2, "disabled", "external", 2),
|
||||
(R2, "enabled", "external", 3), (R2, "removed", "external", 4), (R2, "added", "external", 5)]
|
||||
async with SessionLocal() as db:
|
||||
changed = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.kind == "changed"))).one()
|
||||
assert changed.before["destination_port"] == "443" and changed.after["destination_port"] == "8443"
|
||||
gone = (await db.scalars(select(RuleChangeEvent).where(RuleChangeEvent.kind == "removed"))).one()
|
||||
assert gone.before["uuid"] == R2 and gone.after is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_derived_display_changes_are_not_rule_changes(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1, **{"%action": "Pass", "alias_meta_source_net": [{"value": "a"}]})), T0)
|
||||
await observe(host, answer(rule(R1, **{"%action": "Erlauben", "alias_meta_source_net": [{"value": "b"}]})), T0 + timedelta(minutes=15))
|
||||
assert await events(host) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_error_answer_does_not_remove_everything(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
for bad in ({"status": "failed"}, [], {"rows": "x"}, None):
|
||||
assert (await observe(host, bad, T0 + timedelta(minutes=15)))["events"] == 0
|
||||
assert await events(host) == []
|
||||
await observe(host, answer(), T0 + timedelta(minutes=30)) # all rules gone: confirmed only by a second look
|
||||
assert await events(host) == []
|
||||
await observe(host, answer(), T0 + timedelta(minutes=45))
|
||||
assert await events(host) == [(R1, "removed", "external", 2)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_two_processes_seeing_the_same_change_record_it_once(client, respx_mock):
|
||||
"""mutation: without the version claim / unique key both processes insert an event."""
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
changed = answer(rule(R1, dport="8443"), rule(R2, seq=1001))
|
||||
await asyncio.gather(*(observe(host, changed, T0 + timedelta(minutes=15)) for _ in range(4)))
|
||||
assert sorted(await events(host)) == sorted([(R1, "changed", "external", 2), (R2, "added", "external", 1)])
|
||||
again = await observe(host, changed, T0 + timedelta(minutes=30))
|
||||
assert again["events"] == 0 and len(await events(host)) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_origin_is_opnexus_when_the_audit_trail_has_a_deployment_in_the_window(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
async with SessionLocal() as db:
|
||||
db.add(AuditEvent(created_at=T0 + timedelta(minutes=5), correlation_id="c1", actor="alice", kind="rule", action="deploy",
|
||||
status="pending", host_id=str(host), details={"remote_uuid": R2}))
|
||||
db.add(AuditEvent(created_at=T0 + timedelta(minutes=6), correlation_id="c2", actor="mallory", kind="rule", action="deploy",
|
||||
status="error", host_id=str(host), details={"remote_uuid": R3})) # failed: changed nothing
|
||||
await db.commit()
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002)), T0 + timedelta(minutes=15))
|
||||
async with SessionLocal() as db:
|
||||
by_rule = {e.rule_uuid: e for e in (await db.scalars(select(RuleChangeEvent))).all()}
|
||||
assert (by_rule[R2].origin, by_rule[R2].actor) == ("opnexus", "alice") and by_rule[R2].audit_id is not None
|
||||
assert (by_rule[R3].origin, by_rule[R3].actor) == ("external", None)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_late_audit_entry_turns_an_external_event_into_an_opnexus_one(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
seen = T0 + timedelta(minutes=15)
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), seen)
|
||||
assert await events(host) == [(R2, "added", "external", 1)]
|
||||
async with SessionLocal() as db: # the deployment's audit row is committed a moment after the firewall changed
|
||||
db.add(AuditEvent(created_at=seen + timedelta(seconds=3), correlation_id="c", actor="alice", kind="rule",
|
||||
action="deploy", status="success", host_id=str(host), details={"remote_uuid": R2}))
|
||||
await db.commit()
|
||||
assert await rhs.reconcile_origins(db, await db.get(OpnsenseHost, host), now=seen + timedelta(minutes=15)) == 1
|
||||
assert await events(host) == [(R2, "added", "opnexus", 1)]
|
||||
|
||||
|
||||
async def _audit(host, rule_uuid, at, actor="alice", status="success"):
|
||||
async with SessionLocal() as db:
|
||||
db.add(AuditEvent(created_at=at, correlation_id=str(uuid_lib.uuid4()), actor=actor, kind="rule", action="deploy",
|
||||
status=status, host_id=str(host), details={"remote_uuid": rule_uuid}))
|
||||
await db.commit()
|
||||
|
||||
|
||||
async def _reconcile(host, now):
|
||||
async with SessionLocal() as db:
|
||||
return await rhs.reconcile_origins(db, await db.get(OpnsenseHost, host), now=now)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_later_external_change_is_not_blamed_on_an_older_opnexus_deployment(client, respx_mock):
|
||||
"""mutation: without the window start the re-check reaches back to the old deployment (here an unused one)."""
|
||||
host = await host_of(client, respx_mock)
|
||||
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0)
|
||||
await _audit(host, R2, step(1) - timedelta(minutes=5)) # a deployment that changed nothing visible
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), step(1))
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001, dport="22")), step(2)) # changed in the firewall's GUI
|
||||
assert await events(host) == [(R2, "changed", "external", 2)]
|
||||
assert await _reconcile(host, step(3)) == 0
|
||||
assert await events(host) == [(R2, "changed", "external", 2)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_one_audit_entry_explains_only_one_event(client, respx_mock):
|
||||
"""mutation: without the 'already claimed' filter both changes of the rule turn into OpNexus changes."""
|
||||
host = await host_of(client, respx_mock)
|
||||
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
await observe(host, answer(rule(R1, dport="1")), step(1))
|
||||
await observe(host, answer(rule(R1, dport="2")), step(2))
|
||||
await _audit(host, R1, step(1) + timedelta(seconds=3)) # arrives late; inside both windows
|
||||
assert await _reconcile(host, step(3)) == 1
|
||||
async with SessionLocal() as db:
|
||||
origins = [e.origin for e in (await db.scalars(select(RuleChangeEvent).order_by(RuleChangeEvent.observed_at))).all()]
|
||||
assert origins == ["opnexus", "external"] # the earlier change keeps it
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_observe_host_settles_old_events_before_recording_new_ones(client, respx_mock):
|
||||
"""mutation: reconciling after the observation lets the newer change of the same rule take the late entry."""
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
await observe(host, answer(rule(R1, dport="1")), step(1))
|
||||
await _audit(host, R1, step(1) + timedelta(seconds=3))
|
||||
_mock_observe(respx_mock, FW, answer(rule(R1, dport="2")), httpx.Response(200, json={"status": "ok", "stats": {}}))
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=step(2))
|
||||
assert [(e[1], e[2]) for e in await events(host)] == [("changed", "opnexus"), ("changed", "external")]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pruning_removes_old_events_and_tombstones_only(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
now = datetime.now(timezone.utc)
|
||||
old = now - timedelta(days=400)
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), old)
|
||||
await observe(host, answer(rule(R1, dport="1"), rule(R2, seq=1001)), old + timedelta(minutes=15))
|
||||
await observe(host, answer(rule(R1, dport="1")), old + timedelta(minutes=30)) # R2 removed long ago
|
||||
await observe(host, answer(rule(R1, dport="2")), now) # recent change
|
||||
async with SessionLocal() as db:
|
||||
assert await rhs.prune(db) >= 3
|
||||
left = [(e.rule_uuid, e.kind) for e in (await db.scalars(select(RuleChangeEvent))).all()]
|
||||
states = {s.rule_uuid for s in (await db.scalars(select(LiveRuleState))).all()}
|
||||
assert left == [(R1, "changed")] and states == {R1, rhs.BASELINE_KEY}
|
||||
|
||||
|
||||
# ── usage sampling via ruleStats ──────────────────────────────────────────
|
||||
|
||||
def test_parse_rule_stats_keeps_uuid_and_legacy_hex_labels():
|
||||
got = rule_hit_service.parse_rule_stats(LIVE["ruleStats"])
|
||||
assert got["db610dbc7c6467261cf08e8a72ade096"]["packets"] == 110719 # unmigrated legacy rule
|
||||
assert got["2ec1c6db-61e1-4476-b3f6-0599a28990a2"]["bytes"] == 99582131
|
||||
assert rule_hit_service.parse_rule_stats({"status": "failed"}) is None
|
||||
assert rule_hit_service.parse_rule_stats({"stats": []}) is None and rule_hit_service.parse_rule_stats(None) is None
|
||||
|
||||
|
||||
def test_the_legacy_user_rule_is_a_configured_rule_but_automatic_ones_are_not():
|
||||
rows = {"rows": [LIVE["legacy_row"], {"uuid": "02f4bab031b57d1e30553ce08e0ec131", "is_automatic": True, "enabled": "1"}]}
|
||||
assert set(rule_hit_service.configured_rules(rows)) == {"db610dbc7c6467261cf08e8a72ade096"}
|
||||
assert set(rule_hit_service.configured_rules({"rows": [{"uuid": R1, "is_automatic": False}, {"uuid": R2, "is_automatic": "0"}]})) == {R1, R2}
|
||||
|
||||
|
||||
def _mock_observe(respx_mock, base, search, stats):
|
||||
respx_mock.get(f"{base}/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json=search))
|
||||
respx_mock.get(f"{base}/api/firewall/filter_util/ruleStats").mock(return_value=stats)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_observe_host_reads_two_endpoints_and_feeds_usage_and_history(client, respx_mock):
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
legacy = {**LIVE["legacy_row"], "enabled": "1"}
|
||||
_mock_observe(respx_mock, FW, answer(legacy, rule(R1)), httpx.Response(200, json={
|
||||
"status": "ok", "stats": {**LIVE["ruleStats"]["stats"], R1: {"packets": 0, "bytes": 0, "evaluations": 3, "pf_rules": 1}}}))
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
|
||||
hits = {h.rule_uuid: h for h in (await db.scalars(select(RuleHit))).all()}
|
||||
assert set(hits) == {"db610dbc7c6467261cf08e8a72ade096", R1}
|
||||
assert hits["db610dbc7c6467261cf08e8a72ade096"].last_hit_at == T0 and hits[R1].last_hit_at is None
|
||||
gets = [c.request.url.path for c in respx_mock.calls if "/api/firewall" in c.request.url.path]
|
||||
assert sorted(gets) == ["/api/firewall/filter/searchRule", "/api/firewall/filter_util/ruleStats"]
|
||||
assert len(await events(host)) == 0 # baseline
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_usage_falls_back_to_pf_statistics_when_rule_stats_is_missing(client, respx_mock):
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
_mock_observe(respx_mock, FW, answer(rule(R1)), httpx.Response(404, json={"message": "not found"}))
|
||||
respx_mock.get(f"{FW}/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={
|
||||
"rules": {"filter rules": {f'@5 pass in quick on vtnet1 inet all label "{R1}"': {"packets": 7, "bytes": 70, "evaluations": 9}}}}))
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
|
||||
row = (await db.scalars(select(RuleHit))).one()
|
||||
assert row.rule_uuid == R1 and row.last_hit_at == T0 and row.last_packets == 7
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_failing_host_does_not_stop_the_cycle_for_the_others(client, respx_mock):
|
||||
good = await host_of(client, respx_mock, "good", FW)
|
||||
await host_of(client, respx_mock, "bad", "https://bad.test")
|
||||
_mock_observe(respx_mock, FW, answer(rule(R1)), httpx.Response(200, json={"status": "ok", "stats": {}}))
|
||||
respx_mock.route(url__startswith="https://bad.test").mock(side_effect=httpx.ConnectError("down"))
|
||||
async with SessionLocal() as db:
|
||||
summary = await rhs.observe_all(db, now=T0)
|
||||
assert summary == {"hosts": 1, "failed": 1, "events": 0}
|
||||
assert await events(good) == []
|
||||
async with SessionLocal() as db:
|
||||
assert (await db.get(LiveRuleState, (good, R1))) is not None
|
||||
|
||||
|
||||
# ── answers that are not to be trusted ────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_answer_without_automatic_rules_is_rejected_and_nothing_is_lost(client, respx_mock):
|
||||
"""mutation: without the completeness check a partial answer removes every rule and every usage row."""
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
stats = httpx.Response(200, json={"status": "ok", "stats": {R1: {"packets": 3, "bytes": 1, "evaluations": 1}}})
|
||||
_mock_observe(respx_mock, FW, answer(rule(R1), rule(R2, seq=1001)), stats)
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0)
|
||||
respx_mock.clear()
|
||||
partial = answer(rule(R1), automatic=False)
|
||||
partial["rows"][0]["is_automatic"] = "0" # a firmware that flags automatic rows, but none is in the answer
|
||||
_mock_observe(respx_mock, FW, partial, stats) # R2 and all automatic rows missing
|
||||
async with SessionLocal() as db:
|
||||
result = await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0 + timedelta(minutes=15))
|
||||
assert result["skipped"] is True
|
||||
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2}
|
||||
assert await rule_hit_service.apply_sample(db, await db.get(OpnsenseHost, host), {}, {"rows": []}) == 0
|
||||
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2}
|
||||
assert await events(host) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_losing_most_rules_at_once_needs_a_second_look(client, respx_mock):
|
||||
"""mutation: acting on the first look turns one broken answer into removed + added events for everything."""
|
||||
host = await host_of(client, respx_mock)
|
||||
step = lambda n: T0 + timedelta(minutes=15 * n) # noqa: E731
|
||||
full = answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002))
|
||||
await observe(host, full, T0)
|
||||
thin = answer(rule(R1)) # two of three gone
|
||||
assert (await observe(host, thin, step(1)))["unconfirmed"] == 2
|
||||
assert (await observe(host, full, step(2)))["events"] == 0 # it was a hiccup: nothing happened
|
||||
assert await events(host) == []
|
||||
assert (await observe(host, thin, step(3)))["skipped"] is True # the marker was cleared, so this is a first look again
|
||||
await observe(host, thin, step(4)) # same picture twice: real
|
||||
assert sorted(await events(host)) == sorted([(R2, "removed", "external", 2), (R3, "removed", "external", 2)])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_unconfirmed_answer_does_not_delete_usage_data_either(client, respx_mock):
|
||||
"""mutation: sampling an unconfirmed answer drops the usage rows of the 'vanished' rules."""
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
stats = httpx.Response(200, json={"status": "ok", "stats": {}})
|
||||
for n, search in ((0, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002))), (1, answer(rule(R1)))):
|
||||
respx_mock.clear()
|
||||
_mock_observe(respx_mock, FW, search, stats)
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=T0 + timedelta(minutes=15 * n))
|
||||
async with SessionLocal() as db:
|
||||
assert {h.rule_uuid for h in (await db.scalars(select(RuleHit))).all()} == {R1, R2, R3}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_losing_at_most_half_of_the_rules_is_recorded_at_once(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001), rule(R3, seq=1002)), T0)
|
||||
await observe(host, answer(rule(R1), rule(R2, seq=1001)), T0 + timedelta(minutes=15))
|
||||
assert await events(host) == [(R3, "removed", "external", 2)]
|
||||
|
||||
|
||||
# ── usage window ──────────────────────────────────────────────────────────
|
||||
|
||||
async def _hit(host, uuid=R1):
|
||||
async with SessionLocal() as db:
|
||||
return (await db.scalars(select(RuleHit).where(RuleHit.host_id == host, RuleHit.rule_uuid == uuid))).one()
|
||||
|
||||
|
||||
async def _apply(host, packets, search, at):
|
||||
async with SessionLocal() as db:
|
||||
return await rule_hit_service.apply_sample(
|
||||
db, await db.get(OpnsenseHost, host), {R1: {"packets": packets, "bytes": 0, "evaluations": 0}}, search, at)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_rule_that_was_switched_on_starts_its_observation_window_again(client, respx_mock):
|
||||
"""mutation: an old first_seen_at makes a rule enabled yesterday 'unused, sure' today."""
|
||||
host = await host_of(client, respx_mock)
|
||||
await _apply(host, 0, answer(rule(R1, enabled="0")), T0)
|
||||
await _apply(host, 0, answer(rule(R1, enabled="0")), T0 + timedelta(days=40))
|
||||
assert rule_hit_service.classify(await _hit(host), T0 + timedelta(days=40)) == "disabled"
|
||||
await _apply(host, 0, answer(rule(R1)), T0 + timedelta(days=41)) # switched on
|
||||
row = await _hit(host)
|
||||
assert row.first_seen_at == T0 + timedelta(days=41)
|
||||
assert rule_hit_service.classify(row, T0 + timedelta(days=42)) == "collecting"
|
||||
assert rule_hit_service.classify(row, T0 + timedelta(days=72)) == "unused"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_counter_reset_after_a_long_gap_restarts_the_window_but_not_after_a_short_one(client, respx_mock):
|
||||
"""mutation: ignoring the gap treats hits lost to a reload during a long outage as 'no hits'."""
|
||||
host = await host_of(client, respx_mock)
|
||||
search = answer(rule(R1))
|
||||
await _apply(host, 5, search, T0)
|
||||
await _apply(host, 0, search, T0 + timedelta(minutes=15)) # reload, normal cadence
|
||||
assert (await _hit(host)).first_seen_at == T0
|
||||
await _apply(host, 4, search, T0 + timedelta(minutes=30)) # grew: a hit
|
||||
await _apply(host, 0, search, T0 + timedelta(hours=5)) # reload after a gap of hours
|
||||
row = await _hit(host)
|
||||
assert row.first_seen_at == T0 + timedelta(hours=5) and row.last_hit_at == T0 + timedelta(minutes=30)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_edited_rule_restarts_its_window_and_a_move_does_not(client, respx_mock):
|
||||
"""mutation: without restart_window an edited rule keeps its old observation window."""
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
step = lambda n: T0 + timedelta(days=n) # noqa: E731
|
||||
stats = httpx.Response(200, json={"status": "ok", "stats": {R1: {"packets": 0, "bytes": 0, "evaluations": 0}}})
|
||||
for n, search in ((0, answer(rule(R1))), (40, answer(rule(R1, dport="8443"))), (41, answer(rule(R1, dport="8443", seq=1005)))):
|
||||
respx_mock.clear()
|
||||
_mock_observe(respx_mock, FW, search, stats)
|
||||
async with SessionLocal() as db:
|
||||
await rhs.observe_host(db, await db.get(OpnsenseHost, host), now=step(n))
|
||||
if n == 40:
|
||||
assert (await _hit(host)).first_seen_at == step(40) # edited: window starts over
|
||||
assert (await _hit(host)).first_seen_at == step(40) # moved: unchanged
|
||||
assert [(e[1]) for e in await events(host)] == ["changed", "moved"]
|
||||
|
||||
|
||||
def test_a_move_is_a_move():
|
||||
base = {"uuid": R1, "enabled": "1", "sequence": "5", "sort_order": "400000.0000005", "description": "x"}
|
||||
assert rhs.change_kind(base, {**base, "sequence": "9", "sort_order": "400000.0000009"}) == "moved"
|
||||
assert rhs.change_kind(base, {**base, "sequence": "9", "description": "y"}) == "changed"
|
||||
assert rhs.change_kind(base, {**base, "sequence": "9", "enabled": "0"}) == "changed"
|
||||
|
||||
|
||||
# ── endpoints ─────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_history_endpoint_merges_observed_changes_revisions_and_audit(client, respx_mock):
|
||||
host = await host_of(client, respx_mock)
|
||||
await observe(host, answer(rule(R1)), T0)
|
||||
await observe(host, answer(rule(R1, dport="8443")), T0 + timedelta(minutes=15))
|
||||
async with SessionLocal() as db:
|
||||
db.add(AuditEvent(created_at=T0 + timedelta(minutes=2), correlation_id="c", actor="alice", kind="rule", action="deploy",
|
||||
status="success", host_id=str(host), message="ok", details={"remote_uuid": R1}))
|
||||
await db.commit()
|
||||
body = (await client.get(f"/api/hosts/{host}/live-rules/{R1}/history")).json()
|
||||
assert body["rule_uuid"] == R1 and body["watching_since"]
|
||||
assert [(e["source"], e["kind"]) for e in body["events"]] == [("observed", "changed"), ("audit", "deploy")]
|
||||
assert body["events"][0]["changes"] == [{"field": "destination_port", "before": "443", "after": "8443"}]
|
||||
empty = (await client.get(f"/api/hosts/{host}/live-rules/{R3}/history")).json()
|
||||
assert empty["events"] == []
|
||||
missing = await client.get(f"/api/hosts/{uuid_lib.uuid4()}/live-rules/{R1}/history")
|
||||
assert missing.status_code == 404 and missing.json()["detail"]["code"] == "HOST_NICHT_GEFUNDEN"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_history_of_a_managed_rule_includes_its_saved_revisions(client, respx_mock):
|
||||
from app.models.policy import RuleRevision
|
||||
from app.models.rule import FirewallRule
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
host = await host_of(client, respx_mock)
|
||||
async with SessionLocal() as db:
|
||||
rule_row = FirewallRule(description="Web in", interface="lan", action="pass", direction="in", protocol="tcp",
|
||||
source_net="any", destination_net="any", destination_port="443")
|
||||
db.add(rule_row)
|
||||
await db.flush()
|
||||
db.add(RuleDeployment(rule_id=rule_row.id, host_id=host, status="success", opnsense_rule_uuid=R1))
|
||||
db.add(RuleRevision(rule_id=rule_row.id, note="angelegt", snapshot={"destination_port": "443"}, created_at=T0))
|
||||
db.add(RuleRevision(rule_id=rule_row.id, note="Port geändert", snapshot={"destination_port": "8443"}, created_at=T0 + timedelta(hours=1)))
|
||||
await db.commit()
|
||||
body = (await client.get(f"/api/hosts/{host}/live-rules/{R1}/history")).json()
|
||||
revisions = [e for e in body["events"] if e["source"] == "revision"]
|
||||
assert [e["note"] for e in revisions] == ["Port geändert", "angelegt"]
|
||||
assert revisions[0]["changes"] == [{"field": "destination_port", "before": "443", "after": "8443"}]
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _fresh_cache():
|
||||
ctx_mod.clear_cache()
|
||||
yield
|
||||
ctx_mod.clear_cache()
|
||||
|
||||
|
||||
async def _seed_hits(host_id, **hits):
|
||||
async with SessionLocal() as db:
|
||||
for uuid, (first_days, last_hit_days) in hits.items():
|
||||
now = datetime.now(timezone.utc)
|
||||
db.add(RuleHit(host_id=host_id, rule_uuid=uuid, description=uuid, enabled=True, first_seen_at=now - timedelta(days=first_days),
|
||||
last_sampled_at=now - timedelta(minutes=10),
|
||||
last_hit_at=None if last_hit_days is None else now - timedelta(days=last_hit_days)))
|
||||
await db.commit()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cleanup_endpoint_lists_candidates_with_confidence_and_removability(client, respx_mock):
|
||||
from app.models.rule import FirewallRule
|
||||
from app.models.rule_deployment import RuleDeployment
|
||||
data = {**FIXTURE["rules"], "rows": FIXTURE["rules"]["rows"] + [
|
||||
{k: v for k, v in rule(R1, seq=1000, src="10.20.30.0/24", dport="").items() if k != "_loaded"},
|
||||
{k: v for k, v in rule(R2, seq=1001, src="10.20.30.5", dport="").items() if k != "_loaded"},
|
||||
{k: v for k, v in rule(R3, seq=1002, src="10.20.30.9", dport="22", enabled="0").items() if k != "_loaded"},
|
||||
]}
|
||||
_mock_fw(respx_mock)
|
||||
respx_mock.get(f"{FW}/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json=data))
|
||||
# pf only loads enabled rules: add lines for R1/R2 so they count as loaded
|
||||
pf = json.loads(json.dumps(FIXTURE["pf"]))
|
||||
for u, n in ((R1, 900), (R2, 901)):
|
||||
pf["rules"]["filter rules"][f'@{n} pass in quick on vtnet1_vlan10 inet proto tcp from any to any label "{u}"'] = {}
|
||||
respx_mock.get(f"{FW}/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json=pf))
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
await _seed_hits(host, **{R2: (100, None)})
|
||||
async with SessionLocal() as db:
|
||||
rule_row = FirewallRule(description="Managed web", interface="lan", action="pass", direction="in", protocol="tcp",
|
||||
source_net="any", destination_net="any", destination_port="443")
|
||||
db.add(rule_row)
|
||||
await db.flush()
|
||||
db.add(RuleDeployment(rule_id=rule_row.id, host_id=host, status="success", opnsense_rule_uuid=R2))
|
||||
db.add(LiveRuleState(host_id=host, rule_uuid=R3, fingerprint="x", row={}, present=True, version=1,
|
||||
first_seen_at=T0, changed_at=datetime.now(timezone.utc) - timedelta(days=50),
|
||||
last_seen_at=datetime.now(timezone.utc)))
|
||||
await db.commit()
|
||||
resp = await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 30})
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["unused_days"] == 30 and body["host_name"] == "fw1"
|
||||
by_uuid = {c["uuid"]: c for c in body["candidates"]}
|
||||
assert set(by_uuid) == {R2, R3}
|
||||
r2 = {c["kind"]: c for c in by_uuid[R2]["categories"]}
|
||||
assert r2["shadowed"]["by"]["uuid"] == R1 and r2["shadowed"]["confidence"] == "sure"
|
||||
assert r2["unused"]["confidence"] == "sure"
|
||||
assert by_uuid[R2]["removable"] is False and by_uuid[R2]["not_removable_reason"] == "managed"
|
||||
assert by_uuid[R2]["managed_by"]["name"] == "Managed web"
|
||||
assert [c["kind"] for c in by_uuid[R3]["categories"]] == ["disabled_long"] and by_uuid[R3]["removable"] is True
|
||||
assert body["counts"] == {"unused": 1, "shadowed": 1, "duplicate": 0, "disabled_long": 1}
|
||||
assert all(m == "GET" for m in {c.request.method for c in respx_mock.calls})
|
||||
# a narrower window / the parameter is validated
|
||||
assert (await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 0})).status_code == 422
|
||||
short = (await client.get(f"/api/hosts/{host}/rule-analysis/cleanup", params={"unused_days": 365})).json()
|
||||
assert {k["kind"] for c in short["candidates"] for k in c["categories"]} == {"shadowed"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cleanup_endpoint_roles_and_errors(client, respx_mock):
|
||||
_mock_fw(respx_mock)
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
unknown = await client.get(f"/api/hosts/{uuid_lib.uuid4()}/rule-analysis/cleanup")
|
||||
assert unknown.status_code == 404 and unknown.json()["detail"]["code"] == "HOST_NICHT_GEFUNDEN"
|
||||
token = await _make(client, "read")
|
||||
async with _bearer(token["token"]) as api:
|
||||
assert (await api.get(f"/api/hosts/{host}/rule-analysis/cleanup")).status_code == 200
|
||||
assert (await api.get(f"/api/hosts/{host}/live-rules/{R1}/history")).status_code == 200
|
||||
assert (await api.post(f"/api/hosts/{host}/rule-analysis/cleanup")).status_code in (403, 405)
|
||||
ctx_mod.clear_cache()
|
||||
respx_mock.clear()
|
||||
respx_mock.route(url__startswith=FW).mock(side_effect=httpx.ConnectError("down"))
|
||||
down = await client.get(f"/api/hosts/{host}/rule-analysis/cleanup")
|
||||
assert down.status_code == 502 and down.json()["detail"]["code"] == "ANALYSE_FIREWALL_NICHT_LESBAR"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_anonymous_callers_get_nothing(anon_client):
|
||||
hid = uuid_lib.uuid4()
|
||||
assert (await anon_client.get(f"/api/hosts/{hid}/rule-analysis/cleanup")).status_code == 401
|
||||
assert (await anon_client.get(f"/api/hosts/{hid}/live-rules/{R1}/history")).status_code == 401
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mcp_cleanup_tool_works_with_a_read_token_and_changes_nothing(client, respx_mock):
|
||||
_mock_fw(respx_mock)
|
||||
host = await host_of(client, respx_mock, url=FW)
|
||||
token = await _make(client, "read")
|
||||
async with _bearer(token["token"]) as api:
|
||||
rpc = {"jsonrpc": "2.0", "id": 1, "method": "tools/call",
|
||||
"params": {"name": "rule_cleanup_candidates", "arguments": {"host_id": str(host), "unused_days": 90}}}
|
||||
result = (await api.post("/api/mcp", json=rpc)).json()["result"]
|
||||
tools = (await api.post("/api/mcp", json={"jsonrpc": "2.0", "id": 2, "method": "tools/list"})).json()["result"]["tools"]
|
||||
assert result["isError"] is False
|
||||
payload = json.loads(result["content"][0]["text"])
|
||||
assert payload["unused_days"] == 90 and payload["candidates"] == [] and payload["host_name"] == "fw1"
|
||||
tool = next(t for t in tools if t["name"] == "rule_cleanup_candidates")
|
||||
assert tool["inputSchema"]["required"] == ["host_id"] and "path" not in tool
|
||||
assert {c.request.method for c in respx_mock.calls if c.request.url.host != "test"} == {"GET"}
|
||||
|
||||
|
||||
def test_firmware_without_the_automatic_flag_is_not_rejected_as_incomplete():
|
||||
from app.services import rule_hit_service
|
||||
assert rule_hit_service.looks_complete({"rows": [{"uuid": "u1"}, {"uuid": "u2"}]}) # no row carries the flag: non-empty is enough
|
||||
assert not rule_hit_service.looks_complete({"rows": []})
|
||||
assert not rule_hit_service.looks_complete({"rows": [{"uuid": "u1", "is_automatic": "0"}]}) # flagged firmware, no automatic rule: partial
|
||||
assert rule_hit_service.looks_complete({"rows": [{"uuid": "u1", "is_automatic": "0"}, {"uuid": "a", "is_automatic": "1"}]})
|
||||
+90
-7
@@ -1106,11 +1106,12 @@ Sicherung) zeigt die Vorschau eine Warnung.
|
||||
Auf der Seite **Sicherheitslage** zeigt die Tabelle „Regelnutzung“, wann jede Regel
|
||||
zuletzt Verkehr getroffen hat. OPNexus liest dafür die Paketzähler der Firewall. Weil diese
|
||||
bei jeder Änderung und jedem Neustart von vorn beginnen, merkt sich OPNexus den Zeitpunkt, an dem
|
||||
ein Zähler zuletzt gewachsen ist (Auflösung: eine Prüfung, also bis zu 6 Stunden). Eine Regel gilt
|
||||
ein Zähler zuletzt gewachsen ist (Auflösung: eine Beobachtung, alle 15 Minuten). Die Zähler kommen aus `ruleStats` der
|
||||
Firewall (je Regel schon summiert; bei älteren OPNsense-Versionen aus dem geladenen Regelsatz). Eine Regel gilt
|
||||
als **ungenutzt**, wenn OPNexus sie mindestens 30 Tage beobachtet hat, ohne einen Treffer zu sehen;
|
||||
bis dahin steht sie auf „Daten werden gesammelt“, damit eine neue Installation keinen Fehlalarm
|
||||
auslöst. Nur Regeln aus dem Regel-Editor bzw. der API werden erfasst (die automatischen Regeln von
|
||||
OPNsense nicht). Ungenutzte Regeln fließen als Befund „Regeln in Benutzung“ in die Sicherheitslage ein.
|
||||
bis dahin steht sie auf „Daten werden gesammelt“ (die Beobachtungszeit beginnt neu, wenn die Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern), damit eine neue Installation keinen Fehlalarm
|
||||
auslöst. Erfasst werden die Regeln aus dem Regel-Editor bzw. der API und noch nicht migrierte Regeln im Altformat (die
|
||||
automatischen Regeln von OPNsense nicht). Ungenutzte Regeln fließen als Befund „Regeln in Benutzung“ in die Sicherheitslage ein.
|
||||
|
||||
### Dashboard
|
||||
|
||||
@@ -1255,6 +1256,86 @@ vermutlich, unbekannt; Tooltip mit den Gründen) und der entscheidenden Regel. I
|
||||
die offenen Punkte. Unter „Warum?“ steht jede geprüfte Regel in der Prüfreihenfolge mit ✓/✗/? je Feld.
|
||||
Die Abfrage steht in der Adresszeile und lässt sich als Link teilen.
|
||||
|
||||
#### Bereinigung
|
||||
|
||||
Der Tab **Bereinigung** (Regelwerk › Regel-Analyse › Bereinigung) nennt je Firewall die Regeln, die einen Blick wert sind. Du wählst die Firewall
|
||||
und die **Schwelle** (7, 30, 90 oder 180 Tage); die Auswahl steht in der Adresszeile (`?tab=cleanup&host=…&days=…`). Die Liste selbst wird nur gelesen; Ändern geht über die Auswahl (siehe unten):
|
||||
|
||||
| Befund | Bedeutung |
|
||||
|---|---|
|
||||
| **Ungenutzt** | OPNexus beobachtet die Regel seit mindestens der Schwelle und hat in dieser Zeit keinen Treffer gesehen (siehe „Regelnutzung“). Eine neue Regel erscheint erst nach der Beobachtungszeit. |
|
||||
| **Verdeckt** | Eine frühere Quick-Regel trifft schon alles, was diese Regel trifft; die Regel entscheidet nie. Die verdeckende Regel steht daneben. |
|
||||
| **Doppelt** | Wie verdeckt, aber die frühere Regel hat dieselbe Aktion und trifft genau dasselbe. |
|
||||
| **Lange deaktiviert** | Die Regel ist deaktiviert und OPNexus hat sie seit mindestens der Schwelle unverändert gesehen (aus dem Änderungsverlauf). |
|
||||
|
||||
Jeder Befund trägt eine **Sicherheit**. **Sicher** gibt es nur mit vollständigen, frischen Belegen: bei „ungenutzt“ muss die Regel in pf geladen sein,
|
||||
ohne Zeitplan, und die Zähler müssen frisch sein; bei „verdeckt/doppelt“ stammt die Reihenfolge von der Firewall, die verdeckende Regel ist geladen
|
||||
und der Vergleich beruht auf den Regeln selbst. **Vermutlich** heißt: es beruht auf einer Annahme (der Inhalt von Aliasen, eine geschätzte
|
||||
Reihenfolge, eine Regel mit Zeitplan, ein fehlender pf-Regelsatz). **Unbekannt** heißt: es gibt einen Hinweis ohne Beleg, etwa weil die Regel in pf nicht geladen
|
||||
ist oder die Zähler veraltet sind. Der Tooltip erklärt es jeweils. Was nicht belegt ist, ist nie „sicher“.
|
||||
|
||||
Die Spalte **Aktion** sagt, was eine Auswahl der Regel bewirken würde (**Deaktivieren** oder, bei einer schon deaktivierten Regel, **Löschen**) oder warum
|
||||
die Regel nicht angeboten wird: automatische Regeln (von OPNsense erzeugt), Regeln im Altformat (zuerst in OPNsense migrieren), von OPNexus ausgerollte Regeln
|
||||
(die entfernst du über den normalen Ablauf der Regelseite, sonst legt OPNexus sie wieder an) und Regeln mit Befund „unbekannt“. Die Sicherheitslage verlinkt
|
||||
ihre Befunde „ungenutzt“, „verdeckt“ und „doppelt“ direkt in diese Liste. Per API: `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP: `rule_cleanup_candidates`.
|
||||
|
||||
##### Verwaltungsquellen (Voraussetzung)
|
||||
|
||||
Bevor die Bereinigung etwas ändern darf, trägst du die **Verwaltungsquellen** ein: die IP-Adressen oder Netze (CIDR, keine Hostnamen), von denen aus deine Firewalls verwaltet
|
||||
werden (OPNexus selbst, dein Arbeitsplatz, ein Jump-Host). Der Tab zeigt bis dahin einen Hinweis und (nur für Administratoren) ein kleines Formular; dasselbe steht unter
|
||||
**Einstellungen › Regel-Bereinigung**, dort auch die **Obergrenze je Lauf** (1 bis 25, Standard 10) und die **Wartezeit bis zum Löschen** (1 bis 365 Tage, Standard 7).
|
||||
Für jede gewählte Regel prüft OPNexus, ob sie ein Paket von einer Verwaltungsquelle an eine Adresse der Firewall auf einem Verwaltungsport treffen kann (API-Port aus
|
||||
der Host-Adresse, 443, 80, 22 und die **weiteren Verwaltungsports**, die du unter Einstellungen › Regel-Bereinigung einträgst, z. B. SSH auf 2222 oder die Weboberfläche
|
||||
auf einem eigenen Port; bis 20 Ports). Geprüft wird auf jedem Interface, auch für Pakete über nicht zugewiesene Geräte (WireGuard, OpenVPN, IPsec). Trifft die Regel oder
|
||||
lässt sich das nicht ausschließen, wird sie nie deaktiviert. Dazu kommen Cluster-Regeln (CARP, pfsync), die nie angeboten werden. Eine Sperrregel wieder einzuschalten,
|
||||
die den Verwaltungszugang treffen könnte, verlangt eine ausdrückliche Bestätigung je Regel.
|
||||
|
||||
> **Wichtig: OPNexus selbst gehört in die Verwaltungsquellen.** Trag die Adresse ein, mit der OPNexus bei den Firewalls ankommt (bei NAT die Absenderadresse nach der
|
||||
> Übersetzung, über das Management-Overlay die Overlay-Adresse). Fehlt sie, kann eine Bereinigung genau die Regel abschalten, über die OPNexus die Firewall erreicht; dann
|
||||
> scheitert auch das automatische Zurücksetzen. Im **Cluster** gilt das für jeden Knoten: OPNexus muss beide Knoten über eingetragene Quellen erreichen.
|
||||
> Quellen, Ports und Firewall-Adressen sehen nur Administratoren; alle anderen sehen nur, ob etwas eingetragen ist.
|
||||
|
||||
##### Deaktivieren, Bestätigen, automatisches Zurücksetzen
|
||||
|
||||
1. **Auswählen.** Pro Lauf entweder nur Deaktivieren oder nur Löschen, höchstens so viele Regeln wie eingestellt. Bei Befunden „vermutlich“ fragt OPNexus vor der Vorschau je Regel
|
||||
mit einem festen Text nach, ob du geprüft hast, dass die Regel nicht gebraucht wird (Zeitplan, Aliasinhalt, kurzes Beobachtungsfenster, Sperrregel ohne Treffer). „Ungenutzt“ ist unter
|
||||
30 Tagen Beobachtung und bei Sperrregeln höchstens „vermutlich“.
|
||||
2. **Vorschau.** OPNexus berechnet alles frisch von der Firewall (im Cluster von allen Knoten; die Regel muss auf jedem Knoten dieselbe UUID haben und es muss genau einen
|
||||
Master geben). Je Regel stehen Aktion, Befund, Sicherheit, die Schutzprüfung zum Verwaltungszugang und Hinweise (z. B. „extern geändert“). Die Vorschau gilt 5 Minuten
|
||||
und gehört dem, der sie erstellt hat. Offene Wartungen, Ausrollvorgänge oder eine andere Bereinigung im Cluster sperren sie.
|
||||
3. **Ausrollen** (nur Administratoren, mit Eingabe des Firewall-Namens). OPNexus prüft den Stand erneut und schreibt nichts, wenn er von der Vorschau abweicht. Dann werden die Regeln
|
||||
deaktiviert (im Cluster zuerst der Backup) und angewendet.
|
||||
4. **Bestätigen oder zurücksetzen.** Der Kasten oben im Tab zeigt den Countdown. **Bestätigen** (Operator oder Administrator, **nur in der Browser-Sitzung, nie per API-Token**) liest
|
||||
vorher jede Firewall noch einmal und verweigert, wenn sie nicht antwortet: das ist das Signal, dass du dich ausgesperrt haben könntest. Bestätigst du nicht
|
||||
rechtzeitig, setzt OPNexus automatisch zurück; mit **Jetzt zurücksetzen** geht das sofort (Administrator, oder Operator, wenn dieser Schalter unter Einstellungen erlaubt ist). Ist
|
||||
eine Regel fehlgeschlagen oder unklar (Status *Teilweise*), ist Bestätigen gesperrt. Schlägt das Zurücksetzen selbst fehl, hilft **Zurücksetzen wiederholen**. Nach einem Zurücksetzen beginnt
|
||||
das Beobachtungsfenster der Regel neu.
|
||||
|
||||
##### Löschen und Wiederherstellen
|
||||
|
||||
Gelöscht werden nur Regeln, die schon deaktiviert sind: eine bestätigt deaktivierte Regel nach der Wartezeit (und nur, wenn sie seitdem unverändert und nicht extern geändert wurde)
|
||||
oder eine von Hand deaktivierte Regel mit Befund „lange deaktiviert“ (sicher). Eine deaktivierte Regel ist in der Firewall nicht geladen, das Löschen ändert dort also nichts
|
||||
und hat darum kein Bestätigungsfenster. OPNexus sichert vorher den vollständigen Regelinhalt samt Position; ohne Sicherung wird nicht gelöscht. Unter **Verlauf der Bereinigungen**
|
||||
(am Seitenende des Tabs) zeigt jede Zeile ihre Schritte; bei gelöschten Regeln gibt es **Wiederherstellen** (nur Administratoren, mit Firewall-Namen): die Regel kommt aus der Sicherung neu zurück, **deaktiviert**,
|
||||
mit neuer UUID (im Cluster auf allen Knoten) und möglichst direkt vor ihrer alten Nachfolgerin. Lässt sich die Position nicht herstellen, steht die Regel am Ende ihrer Gruppe und
|
||||
OPNexus weist darauf hin. Fehlt der Firewall inzwischen ein Alias oder eine Kategorie der Regel, lehnt sie ab und ihre Meldung steht unverändert beim Ergebnis. Zum Einschalten gibt es
|
||||
**Wieder aktivieren** (auch bei bestätigt deaktivierten Regeln): ein eigener Lauf mit Bestätigungsfenster, weil das Einschalten einer Sperrregel aussperren kann.
|
||||
|
||||
Rechte im Überblick: Vorschau, Abbrechen und Bestätigen dürfen Operatoren und Administratoren; Ausrollen, Wiederholen und Wiederherstellen nur Administratoren; Nur-Lese-Konten sehen die Liste und den Verlauf,
|
||||
aber keine Knöpfe. Jeder Schritt steht je Regel im Audit-Protokoll und im Änderungsverlauf der Regel (Herkunft „über OPNexus“).
|
||||
|
||||
#### Verlauf einer Regel
|
||||
|
||||
OPNexus liest alle 15 Minuten die Regeln jeder Firewall (zwei Lesezugriffe) und vergleicht sie mit dem zuletzt gesehenen Stand – auch Regeln, die nicht von OPNexus
|
||||
stammen. Jede Änderung wird ein Ereignis: **angelegt, geändert** (mit Vorher/Nachher der betroffenen Felder), **verschoben** (nur Reihenfolge oder Gruppe), **aktiviert, deaktiviert, entfernt**. Als **Herkunft**
|
||||
steht „über OPNexus“, wenn das Audit seit der vorherigen Beobachtung (und noch für keine andere Änderung) ein Ausrollen oder Zurücksetzen dieser Regel auf dieser Firewall kennt (mit dem Benutzer), sonst „extern“ (in der
|
||||
Oberfläche der Firewall, per Skript oder über einen anderen Zugang). Die erste Beobachtung einer Firewall speichert nur den Ausgangszustand, es entstehen
|
||||
keine „angelegt“-Ereignisse für vorhandene Regeln; eine leere oder unvollständige Antwort der Firewall wird verworfen, und verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung; eine Änderung zwischen zwei Beobachtungen erscheint zum Zeitpunkt der zweiten. Ereignisse bleiben 365 Tage.
|
||||
|
||||
Zu sehen ist der Verlauf in der **Live-Ansicht** (Regelwerk › Live über alle Hosts) über die Zeilenaktion **Verlauf**, in der Bereinigungsliste am Symbol neben der Regel,
|
||||
und per API (`GET /api/hosts/{id}/live-rules/{uuid}/history`). Bei einer von OPNexus verwalteten Regel stehen die gespeicherten Fassungen und die Audit-Einträge der
|
||||
Regel dabei. Automatische Regeln werden nicht verfolgt.
|
||||
|
||||
### Sicherheitslage
|
||||
|
||||
Unter **Monitoring › Sicherheitslage** prüft OPNexus jede Firewall auf typische
|
||||
@@ -1280,9 +1361,11 @@ Konfiguration nicht gespeichert. Alle sechs Stunden läuft die Prüfung von selb
|
||||
Ein kritischer Befund kostet 30 Punkte, ein hoher 15, ein mittlerer 8, ein niedriger 3.
|
||||
Nicht entscheidbare Prüfungen (etwa der Updatestatus einer gerade nicht erreichbaren
|
||||
Firewall) kosten nichts. Verdeckt oder doppelt meldet OPNexus nur, was sich beweisen
|
||||
lässt: Regeln werden Feld für Feld verglichen, Aliase werden nicht aufgelöst, also
|
||||
bleibt eine Überschneidung über Aliase unentdeckt, aber es gibt keine falschen
|
||||
Befunde. Beide Regelformate (OPNsense 26.x und ältere Installationen) werden gelesen.
|
||||
lässt: Regeln werden in der Reihenfolge der Firewall (Floating, Interface-Gruppen, Interface-Regeln) Feld für Feld
|
||||
verglichen, Netze und Portbereiche als Mengen; nur eine frühere Quick-Regel kann eine spätere verdecken, und
|
||||
Regeln mit Zeitplan, Markierung oder Limits verdecken nichts. Aliase gelten nur bei gleichem Namen als gleich; so entstehen
|
||||
keine falschen Befunde, aber eine Überschneidung über verschiedene Aliase bleibt hier unentdeckt (die Bereinigung vergleicht
|
||||
auch statische Alias-Inhalte, dort als „vermutlich“). Beide Regelformate (OPNsense 26.x und ältere Installationen) werden gelesen.
|
||||
Die Werte stehen auch als Prometheus-Metriken zur Verfügung (siehe unten).
|
||||
|
||||
Das **Dashboard** zeigt die Firewalls mit dem niedrigsten Wert. Der Knopf **Bericht** erzeugt eine druckbare Übersicht
|
||||
|
||||
@@ -6,7 +6,9 @@ import { Button } from "@/components/ui/Button";
|
||||
import { Card } from "@/components/ui/Card";
|
||||
import { EmptyState } from "@/components/ui/EmptyState";
|
||||
import { FieldGroup, Input, Label, Select } from "@/components/ui/Field";
|
||||
import { RuleCleanup } from "@/components/RuleCleanup";
|
||||
import { Loading } from "@/components/ui/Loading";
|
||||
import { Tabs } from "@/components/ui/Tabs";
|
||||
import { InfoTip } from "@/components/ui/Tooltip";
|
||||
import { AnalysisResult, api, DeviceGroup, Host, InterfaceOption } from "@/lib/api";
|
||||
import { Network, Search } from "lucide-react";
|
||||
@@ -53,7 +55,30 @@ function toParams(form: FormState): URLSearchParams {
|
||||
|
||||
export default function AnalysisPage() {
|
||||
const { t } = useLanguage();
|
||||
return <Suspense fallback={<Loading>{t("common.loading")}</Loading>}><AnalysisContent /></Suspense>;
|
||||
return <Suspense fallback={<Loading>{t("common.loading")}</Loading>}><AnalysisShell /></Suspense>;
|
||||
}
|
||||
|
||||
/** Title and tabs (?tab=query|cleanup); each tab keeps its own state in the address bar. */
|
||||
function AnalysisShell() {
|
||||
const { t } = useLanguage();
|
||||
const router = useRouter();
|
||||
const params = useSearchParams();
|
||||
const tab = params.get("tab") === "cleanup" ? "cleanup" : "query";
|
||||
return (
|
||||
<div>
|
||||
<h1 className="mb-4 flex items-center gap-2 text-2xl font-semibold text-slate-900">
|
||||
{t("analysis.title")}
|
||||
<InfoTip text={t("analysis.intro")} size={18} />
|
||||
</h1>
|
||||
<Tabs
|
||||
className="mb-6"
|
||||
items={[{ key: "query", label: t("analysis.tabs.query") }, { key: "cleanup", label: t("analysis.tabs.cleanup") }]}
|
||||
active={tab}
|
||||
onChange={(key) => router.push(key === "cleanup" ? "/analysis?tab=cleanup" : "/analysis")}
|
||||
/>
|
||||
<div role="tabpanel">{tab === "cleanup" ? <RuleCleanup /> : <AnalysisContent />}</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function AnalysisContent() {
|
||||
@@ -144,11 +169,6 @@ function AnalysisContent() {
|
||||
|
||||
return (
|
||||
<div>
|
||||
<h1 className="mb-6 flex items-center gap-2 text-2xl font-semibold text-slate-900">
|
||||
{t("analysis.title")}
|
||||
<InfoTip text={t("analysis.intro")} size={18} />
|
||||
</h1>
|
||||
|
||||
<Card className="mb-6 p-4 sm:p-5">
|
||||
<form onSubmit={submit} aria-label={t("analysis.form.title")} className="flex flex-col gap-4" noValidate>
|
||||
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-4">
|
||||
|
||||
@@ -5,7 +5,7 @@ import { api, AuditEvent, AuditPage } from "@/lib/api";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Input, Select } from "@/components/ui/Field";
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
const kindKeys = ["rule","alias","port_forward","one_to_one","outbound_nat","host","user","auth","backup","ipsec_tunnel","openvpn_instance","wireguard_tunnel","certificate","acme","notification","gui_access","system"];
|
||||
const kindKeys = ["rule","rule_cleanup","alias","port_forward","one_to_one","outbound_nat","host","user","auth","backup","ipsec_tunnel","openvpn_instance","wireguard_tunnel","certificate","acme","notification","gui_access","system"];
|
||||
const actionKeys = ["deploy","confirm","rollback","remove","retry-rollback","delete","create","edit","reboot","update","ticket","redeem","end"];
|
||||
const statusKeys = ["requested","accepted","rejected","interrupted","success","pending","error","reverted","revert_failed","removed","submitted","acknowledged","starting","unknown","denied"];
|
||||
export default function AuditHistory() {
|
||||
|
||||
@@ -10,7 +10,7 @@ import { useToast } from "@/components/ui/Toast";
|
||||
import { api, ComplianceFinding, ComplianceHost, RuleHit, Severity } from "@/lib/api";
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { useReadOnly } from "@/lib/useReadOnly";
|
||||
import { CheckCircle2, CircleHelp, FileText, Info, RefreshCw, ShieldAlert, XCircle } from "lucide-react";
|
||||
import { ArrowRight, CheckCircle2, CircleHelp, FileText, Info, RefreshCw, ShieldAlert, XCircle } from "lucide-react";
|
||||
import Link from "next/link";
|
||||
import { useEffect, useState } from "react";
|
||||
|
||||
@@ -149,7 +149,7 @@ export default function SecurityPage() {
|
||||
{!detail.error && findings.length === 0 && <p className="px-5 py-6 text-sm text-slate-500">{t("security.neverChecked")}</p>}
|
||||
<ul className="divide-y divide-slate-100">
|
||||
{findings.map((finding) => (
|
||||
<FindingRow key={finding.id} finding={finding} />
|
||||
<FindingRow key={finding.id} finding={finding} hostId={detail.host_id} />
|
||||
))}
|
||||
</ul>
|
||||
</Card>
|
||||
@@ -193,7 +193,10 @@ export default function SecurityPage() {
|
||||
);
|
||||
}
|
||||
|
||||
function FindingRow({ finding }: { finding: ComplianceFinding }) {
|
||||
// Findings about the rule set itself: the cleanup list shows which rules and how sure that is.
|
||||
const CLEANUP_FINDINGS = new Set(["rules_unused", "rules_shadowed", "rules_duplicate"]);
|
||||
|
||||
function FindingRow({ finding, hostId }: { finding: ComplianceFinding; hostId: string }) {
|
||||
const { t } = useLanguage();
|
||||
const failed = finding.passed === false;
|
||||
// A note (severity "info") is a suggestion, not an error: neutral icon instead of the red cross.
|
||||
@@ -219,6 +222,12 @@ function FindingRow({ finding }: { finding: ComplianceFinding }) {
|
||||
</ul>
|
||||
)}
|
||||
{failed && <p className="mt-1 text-sm text-slate-500">{t(`security.checks.${finding.id}.fix`)}</p>}
|
||||
{failed && CLEANUP_FINDINGS.has(finding.id) && (
|
||||
<Link href={`/analysis?tab=cleanup&host=${hostId}`} className="mt-1 inline-flex items-center gap-1 text-sm text-brand-600 hover:underline">
|
||||
{t("security.cleanupLink")}
|
||||
<ArrowRight size={14} aria-hidden="true" />
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
</li>
|
||||
);
|
||||
|
||||
@@ -11,6 +11,7 @@ import { Button } from "@/components/ui/Button";
|
||||
import { FieldGroup, Input, Select } from "@/components/ui/Field";
|
||||
import { useNotAdmin, useReadOnly } from "@/lib/useReadOnly";
|
||||
import { OidcSection } from "@/components/OidcSection";
|
||||
import { CleanupSettings } from "@/components/CleanupSettings";
|
||||
import { BackupSettingsSection } from "@/components/BackupSettingsSection";
|
||||
import { GuiAccessSection } from "@/components/GuiAccessSection";
|
||||
import { AuditExportSection } from "@/components/AuditExportSection";
|
||||
@@ -481,5 +482,5 @@ export default function SettingsPage(){
|
||||
async function saveTheme(value:Theme){setSaving(true);setError('');setNotice('');const old=theme;setTheme(value);try{setSettings(await api.saveSettings({theme:value,language}));setNotice(t('settings.appearanceSaved'));}catch(e){setTheme(old);setError(errorMessage(e));}finally{setSaving(false);}}
|
||||
async function saveLanguage(value:"de"|"en"){setSaving(true);setError('');setNotice('');const old=language;setLanguage(value);try{setSettings(await api.saveSettings({theme,language:value}));setNotice(translate(value,'settings.language.saved'));}catch(e){setLanguage(old);setError(errorMessage(e));}finally{setSaving(false);}}
|
||||
async function changePassword(e:React.FormEvent){e.preventDefault();setError('');setNotice('');if(password!==repeat){setError(t('settings.passwordMismatch'));return;}setSaving(true);try{await api.changePassword(current,password);setCurrent('');setPassword('');setRepeat('');setNotice(t('settings.passwordChanged'));}catch(e){setError(errorMessage(e,'settings.passwordFailed'));}finally{setSaving(false);}}
|
||||
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><section className="space-y-3 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.operational')}<InfoTip text={t('settings.serverManaged')} /></h2>{settings?<dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.deployWindow')}</dt><dd>{t('settings.seconds',{n:settings.deploy_confirm_window_seconds})}</dd><dt>{t('settings.sessionLifetime')}</dt><dd>{t('settings.days',{n:Math.round(settings.session_lifetime_seconds/86400)})}</dd></dl>:<p className="text-sm text-slate-500">{t('common.loading')}</p>}</section><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
|
||||
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><section className="space-y-3 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.operational')}<InfoTip text={t('settings.serverManaged')} /></h2>{settings?<dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.deployWindow')}</dt><dd>{t('settings.seconds',{n:settings.deploy_confirm_window_seconds})}</dd><dt>{t('settings.sessionLifetime')}</dt><dd>{t('settings.days',{n:Math.round(settings.session_lifetime_seconds/86400)})}</dd></dl>:<p className="text-sm text-slate-500">{t('common.loading')}</p>}</section><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><CleanupSettings/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { Badge, BadgeTone } from "@/components/ui/Badge";
|
||||
import { Tooltip } from "@/components/ui/Tooltip";
|
||||
import type { AnalysisConfidence, CleanupKind } from "@/lib/api";
|
||||
import clsx from "clsx";
|
||||
import { Check, CircleHelp } from "lucide-react";
|
||||
|
||||
export const KIND_TONE: Record<CleanupKind, BadgeTone> = { unused: "warning", shadowed: "accent", duplicate: "accent", disabled_long: "neutral" };
|
||||
const CONFIDENCE_TONE: Record<AnalysisConfidence, BadgeTone> = { sure: "info", probable: "warning", unknown: "neutral" };
|
||||
|
||||
export function ConfidenceBadge({ confidence }: { confidence: AnalysisConfidence }) {
|
||||
const { t } = useLanguage();
|
||||
return (
|
||||
<Tooltip content={t(`cleanup.confidence.tip.${confidence}`)} focusable>
|
||||
<Badge
|
||||
tone={CONFIDENCE_TONE[confidence]}
|
||||
className={clsx(confidence === "unknown" && "border-dashed")}
|
||||
aria-label={`${t("cleanup.confidence.label")}: ${t(`cleanup.confidence.${confidence}`)}`}
|
||||
>
|
||||
{confidence === "unknown" ? <CircleHelp size={12} aria-hidden="true" /> : <Check size={12} aria-hidden="true" />}
|
||||
{t(`cleanup.confidence.${confidence}`)}
|
||||
</Badge>
|
||||
</Tooltip>
|
||||
);
|
||||
}
|
||||
|
||||
/** A note code of the analysis or the cleanup plan (cleanup.notes.*); unknown codes show raw. */
|
||||
export function NoteBadge({ code }: { code: string }) {
|
||||
const { t } = useLanguage();
|
||||
const label = t(`cleanup.notes.${code}`);
|
||||
const known = label !== `cleanup.notes.${code}`;
|
||||
return (
|
||||
<Tooltip content={known ? t(`cleanup.notes.${code}_hint`) : code} focusable>
|
||||
<Badge tone="warning">{known ? label : code}</Badge>
|
||||
</Tooltip>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { Badge, BadgeTone } from "@/components/ui/Badge";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Card } from "@/components/ui/Card";
|
||||
import { Tooltip } from "@/components/ui/Tooltip";
|
||||
import { CleanupItem, CleanupItemState, CleanupOperation, CleanupOperationStatus } from "@/lib/api";
|
||||
import { cleanupMessage, COUNTDOWN_STATUSES, currentUuid, formatClock, latestItemIds, secondsUntil } from "@/lib/cleanup";
|
||||
import { CircleCheck, Clock, RotateCcw, Undo2 } from "lucide-react";
|
||||
import { forwardRef, ReactNode } from "react";
|
||||
|
||||
const CONFIRMABLE: CleanupOperationStatus[] = ["awaiting_confirm", "partial", "apply_failed"];
|
||||
|
||||
const STATUS_TONE: Record<CleanupOperationStatus, BadgeTone> = {
|
||||
preview: "neutral", running: "info", awaiting_confirm: "warning", confirmed: "success", partial: "danger",
|
||||
apply_failed: "danger", reverting: "info", reverted: "neutral", revert_failed: "danger", done: "success",
|
||||
failed: "danger", cancelled: "neutral", expired: "neutral",
|
||||
};
|
||||
|
||||
const ITEM_TONE: Record<CleanupItemState, BadgeTone> = {
|
||||
queued: "neutral", disabling: "info", disabled: "warning", confirmed: "success", reverting: "info", reverted: "neutral",
|
||||
deleting: "info", deleted: "neutral", restoring: "info", restored: "success", failed: "danger", unknown: "danger",
|
||||
revert_failed: "danger", skipped: "neutral",
|
||||
};
|
||||
|
||||
function useStateLabel() {
|
||||
const { t } = useLanguage();
|
||||
return (kind: string, state: CleanupItemState) => {
|
||||
// For "enable" the item states mean switching on, not off.
|
||||
if (kind === "enable" && (state === "disabling" || state === "disabled")) return t(`cleanup.stateEnable.${state}`);
|
||||
return t(`cleanup.state.${state}`);
|
||||
};
|
||||
}
|
||||
|
||||
/** The operation's own message, unless a rule below already says the same (a restore warning sits on both). */
|
||||
function ownMessage(t: (key: string) => string, operation: CleanupOperation): string {
|
||||
const message = cleanupMessage(t, operation.message);
|
||||
return message && !operation.items.some((i) => cleanupMessage(t, i.message) === message) ? message : "";
|
||||
}
|
||||
|
||||
function ItemRows({ operation, actions }: { operation: CleanupOperation; actions?: (item: CleanupItem) => ReactNode }) {
|
||||
const { t } = useLanguage();
|
||||
const label = useStateLabel();
|
||||
const clustered = new Set(operation.items.map((i) => i.host_id)).size > 1;
|
||||
return (
|
||||
<ul className="divide-y divide-slate-100 text-sm">
|
||||
{operation.items.map((item) => {
|
||||
const message = cleanupMessage(t, item.message);
|
||||
return (
|
||||
<li key={item.id} className="flex flex-wrap items-start gap-x-3 gap-y-1.5 py-2">
|
||||
<div className="min-w-0 flex-1">
|
||||
<p className="break-words font-medium text-slate-900">{item.description || item.rule_uuid}</p>
|
||||
{clustered && item.host_name && (
|
||||
<p className="text-xs text-slate-500">{item.host_name}{item.carp_role ? ` · ${item.carp_role}` : ""}</p>
|
||||
)}
|
||||
{message && <p className="mt-0.5 break-words text-xs text-slate-600">{message}</p>}
|
||||
</div>
|
||||
<Badge tone={ITEM_TONE[item.state]}>{label(operation.kind, item.state)}</Badge>
|
||||
{item.restored_uuid && item.state === "deleted" && <Badge tone="success">{t("cleanup.hist.restoredBadge")}</Badge>}
|
||||
{actions && <div className="flex flex-wrap gap-1.5">{actions(item)}</div>}
|
||||
</li>
|
||||
);
|
||||
})}
|
||||
</ul>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The one operation that is open on the host/cluster: what happened, how long the automatic revert still waits and the
|
||||
* buttons for the decision (confirm = "I still have access", revert now, try again). Focusable so a roll-out can put
|
||||
* the keyboard focus here.
|
||||
*/
|
||||
export const OperationBanner = forwardRef<HTMLElement, {
|
||||
operation: CleanupOperation; now: number; busy: string | null; readOnly: boolean; notAdmin: boolean; operatorsMayDeploy: boolean;
|
||||
onConfirm: () => void; onRevert: () => void; onRetry: () => void;
|
||||
}>(function OperationBanner({ operation, now, busy, readOnly, notAdmin, operatorsMayDeploy, onConfirm, onRevert, onRetry }, ref) {
|
||||
const { t, locale } = useLanguage();
|
||||
const left = COUNTDOWN_STATUSES.includes(operation.status) ? secondsUntil(operation.revert_deadline, now) : null;
|
||||
const message = ownMessage(t, operation);
|
||||
const revertBlocked = notAdmin && !operatorsMayDeploy;
|
||||
const confirmHint = operation.status === "partial" ? t("cleanup.op.confirmPartial") : operation.status === "apply_failed" ? t("cleanup.op.confirmApplyFailed") : null;
|
||||
return (
|
||||
<section
|
||||
ref={ref}
|
||||
tabIndex={-1}
|
||||
aria-labelledby={`cleanup-op-${operation.id}`}
|
||||
className="mb-6 rounded-2xl border border-flame-300 bg-[rgb(var(--tone-warning-bg))] p-4 outline-none focus-visible:ring-2 focus-visible:ring-brand-500 sm:p-5"
|
||||
>
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<h3 id={`cleanup-op-${operation.id}`} className="text-base font-semibold text-slate-900">{t(`cleanup.op.title.${operation.kind}`)}</h3>
|
||||
<Tooltip content={t(`cleanup.status.${operation.status}_hint`)} focusable>
|
||||
<Badge tone={STATUS_TONE[operation.status]}>{t(`cleanup.status.${operation.status}`)}</Badge>
|
||||
</Tooltip>
|
||||
<span className="ml-auto text-xs text-slate-500">
|
||||
{operation.actor}{operation.created_at ? ` · ${new Date(operation.created_at).toLocaleString(locale)}` : ""}
|
||||
</span>
|
||||
</div>
|
||||
<p className="mt-2 text-sm text-slate-700">{t(`cleanup.op.state.${operation.status}`)}</p>
|
||||
{left !== null && (
|
||||
<p className="mt-2 flex items-center gap-1.5 text-sm font-medium text-flame-700">
|
||||
<Clock size={15} aria-hidden="true" />
|
||||
<span aria-hidden="true">{t("cleanup.op.countdown", { time: formatClock(left) })}</span>
|
||||
<span className="sr-only" role="status" aria-live="polite">{t("cleanup.op.countdownMinutes", { n: Math.ceil(left / 60) })}</span>
|
||||
</p>
|
||||
)}
|
||||
{message && <p role="status" className="mt-2 break-words rounded-lg bg-red-50 p-2.5 text-sm text-red-700">{message}</p>}
|
||||
<div className="mt-3 rounded-xl bg-white/70 px-3"><ItemRows operation={operation} /></div>
|
||||
{operation.warnings.map((code) => <p key={code} className="mt-2 text-sm text-flame-800">{cleanupMessage(t, code)}</p>)}
|
||||
<div className="mt-4 flex flex-wrap items-center gap-2">
|
||||
{CONFIRMABLE.includes(operation.status) && operation.kind !== "delete" && operation.kind !== "restore" && !readOnly && (
|
||||
<Tooltip content={!operation.can_confirm ? (confirmHint ?? t("cleanup.op.confirmNotYet")) : null} focusable={!operation.can_confirm}>
|
||||
<Button onClick={onConfirm} disabled={!operation.can_confirm || busy !== null} loading={busy === "confirm"}>
|
||||
<CircleCheck size={15} aria-hidden="true" />{t("cleanup.op.confirm")}
|
||||
</Button>
|
||||
</Tooltip>
|
||||
)}
|
||||
{operation.can_revert && !readOnly && (
|
||||
<Tooltip content={revertBlocked ? t("cleanup.op.revertNeedsAdmin") : null} focusable={revertBlocked}>
|
||||
<Button variant="outline" onClick={onRevert} disabled={revertBlocked || busy !== null} loading={busy === "revert"}>
|
||||
<Undo2 size={15} aria-hidden="true" />{t("cleanup.op.revert")}
|
||||
</Button>
|
||||
</Tooltip>
|
||||
)}
|
||||
{operation.can_retry && !readOnly && (
|
||||
<Tooltip content={notAdmin ? t("cleanup.op.retryNeedsAdmin") : null} focusable={notAdmin}>
|
||||
<Button variant="secondary" onClick={onRetry} disabled={notAdmin || busy !== null} loading={busy === "retry"}>
|
||||
<RotateCcw size={15} aria-hidden="true" />{t(operation.kind === "disable" || operation.kind === "enable" ? "cleanup.op.retryRevert" : "cleanup.op.retryCheck")}
|
||||
</Button>
|
||||
</Tooltip>
|
||||
)}
|
||||
</div>
|
||||
{operation.can_confirm && <p className="mt-3 text-xs text-slate-600">{t("cleanup.op.autoRevertNote")}</p>}
|
||||
</section>
|
||||
);
|
||||
});
|
||||
|
||||
/** Finished operations of this host (and its cluster peers), newest first, with the follow-up action per rule. */
|
||||
export function OperationHistory({ operations, hostId, now, minDays, readOnly, notAdmin, busy, onRestore, onFollowUp }: {
|
||||
operations: CleanupOperation[]; hostId: string; now: number; minDays: number; readOnly: boolean; notAdmin: boolean; busy: boolean;
|
||||
onRestore: (item: CleanupItem) => void;
|
||||
onFollowUp: (kind: "enable" | "delete", item: CleanupItem, uuid: string) => void;
|
||||
}) {
|
||||
const { t, locale } = useLanguage();
|
||||
const latest = latestItemIds(operations, hostId);
|
||||
if (operations.length === 0) return <p className="py-2 text-sm text-slate-500">{t("cleanup.hist.empty")}</p>;
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
{operations.map((operation) => (
|
||||
<Card key={operation.id} className="p-4">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<h4 className="text-sm font-semibold text-slate-900">{t(`cleanup.op.title.${operation.kind}`)}</h4>
|
||||
<Tooltip content={t(`cleanup.status.${operation.status}_hint`)} focusable>
|
||||
<Badge tone={STATUS_TONE[operation.status]}>{t(`cleanup.status.${operation.status}`)}</Badge>
|
||||
</Tooltip>
|
||||
<span className="ml-auto text-xs text-slate-500">
|
||||
{operation.actor}{operation.created_at ? ` · ${new Date(operation.created_at).toLocaleString(locale)}` : ""}
|
||||
</span>
|
||||
</div>
|
||||
{ownMessage(t, operation) && <p className="mt-1.5 break-words text-xs text-slate-600">{ownMessage(t, operation)}</p>}
|
||||
<ItemRows
|
||||
operation={operation}
|
||||
actions={readOnly ? undefined : (item) => {
|
||||
const own = item.host_id === hostId && latest.has(item.id);
|
||||
const buttons: ReactNode[] = [];
|
||||
if (item.restorable && item.host_id === hostId) {
|
||||
buttons.push(
|
||||
<Tooltip key="restore" content={notAdmin ? t("cleanup.hist.adminOnly") : t("cleanup.hist.restoreHint")} focusable>
|
||||
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={notAdmin || busy} onClick={() => onRestore(item)}>{t("cleanup.hist.restore")}</Button>
|
||||
</Tooltip>,
|
||||
);
|
||||
}
|
||||
const reenable = own && ((operation.kind === "disable" && item.state === "confirmed") || (operation.kind === "restore" && item.state === "restored"));
|
||||
if (own && operation.kind === "disable" && item.state === "confirmed") {
|
||||
const due = operation.confirmed_at ? new Date(operation.confirmed_at).getTime() + minDays * 86400000 : 0;
|
||||
const waiting = due > now;
|
||||
buttons.push(
|
||||
<Tooltip key="delete" content={waiting ? t("cleanup.hist.deleteWait", { when: new Date(due).toLocaleDateString(locale) }) : t("cleanup.hist.deleteHint")} focusable>
|
||||
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={waiting || busy} onClick={() => onFollowUp("delete", item, currentUuid(item, operation.kind))}>{t("cleanup.hist.delete")}</Button>
|
||||
</Tooltip>,
|
||||
);
|
||||
}
|
||||
if (reenable) {
|
||||
buttons.push(
|
||||
<Tooltip key="enable" content={t("cleanup.hist.enableHint")} focusable>
|
||||
<Button variant="outline" className="!px-2.5 !py-1 text-xs" disabled={busy} onClick={() => onFollowUp("enable", item, currentUuid(item, operation.kind))}>{t("cleanup.hist.enable")}</Button>
|
||||
</Tooltip>,
|
||||
);
|
||||
}
|
||||
return buttons;
|
||||
}}
|
||||
/>
|
||||
</Card>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { ConfidenceBadge, KIND_TONE, NoteBadge } from "@/components/CleanupBadges";
|
||||
import { RuleHistoryModal } from "@/components/RuleHistoryModal";
|
||||
import { Badge } from "@/components/ui/Badge";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Input } from "@/components/ui/Field";
|
||||
import { Loading } from "@/components/ui/Loading";
|
||||
import { Modal } from "@/components/ui/Modal";
|
||||
import { useToast } from "@/components/ui/Toast";
|
||||
import { InfoTip, Tooltip } from "@/components/ui/Tooltip";
|
||||
import { api, CleanupActionKind, CleanupOperation, CleanupPlanItem, CleanupPlanNode, CleanupPreview } from "@/lib/api";
|
||||
import { ACK_TEXT_VERSION, formatClock, secondsUntil } from "@/lib/cleanup";
|
||||
import { useNotAdmin } from "@/lib/useReadOnly";
|
||||
import { History, RefreshCw, ShieldCheck, TriangleAlert } from "lucide-react";
|
||||
import { useCallback, useEffect, useId, useRef, useState } from "react";
|
||||
|
||||
/** What the user picked: the kind of run and the rules (needsAck = "probable", an explicit acknowledgement each). */
|
||||
export interface CleanupRequest {
|
||||
kind: CleanupActionKind;
|
||||
rules: { uuid: string; label: string; needsAck: boolean }[];
|
||||
}
|
||||
|
||||
const ACTION_TONE = { disable: "warning", enable: "info", delete: "danger" } as const;
|
||||
|
||||
function duration(t: (key: string, params?: Record<string, string | number>) => string, seconds: number): string {
|
||||
return seconds >= 60 ? t("cleanup.dur.minutes", { n: Math.round(seconds / 60) }) : t("cleanup.dur.seconds", { n: seconds });
|
||||
}
|
||||
|
||||
function PlanItem({ item, kind, sources, coverName, acknowledged, onHistory }: {
|
||||
item: CleanupPlanItem; kind: CleanupActionKind; sources: string[]; coverName: (uuid: string) => string | null;
|
||||
acknowledged: boolean; onHistory: () => void;
|
||||
}) {
|
||||
const { t, locale } = useLanguage();
|
||||
const protection = item.protection;
|
||||
const cover = item.cover_uuid ? coverName(item.cover_uuid) : null;
|
||||
const notes = Array.from(new Set([...item.notes, ...item.categories.flatMap((c) => c.notes ?? [])]));
|
||||
return (
|
||||
<li className="rounded-xl border border-slate-200 p-3 text-sm">
|
||||
<div className="flex flex-wrap items-start gap-x-2 gap-y-1">
|
||||
<span className="min-w-0 flex-1 break-words font-medium text-slate-900">{item.description}</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onHistory}
|
||||
className="shrink-0 rounded p-1 text-slate-400 hover:bg-slate-100 hover:text-slate-700 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
|
||||
aria-label={t("cleanup.history", { rule: item.description })}
|
||||
title={t("cleanup.historyTitle")}
|
||||
>
|
||||
<History size={15} aria-hidden="true" />
|
||||
</button>
|
||||
<Badge tone={ACTION_TONE[kind]}>{t(`cleanup.action.${kind}`)}</Badge>
|
||||
</div>
|
||||
{item.categories.length > 0 && (
|
||||
<div className="mt-2 flex flex-wrap items-center gap-1.5">
|
||||
{item.categories.map((category) => (
|
||||
<span key={category.kind} className="inline-flex items-center gap-1">
|
||||
<Tooltip content={t(`cleanup.category.${category.kind}_hint`)} focusable>
|
||||
<Badge tone={KIND_TONE[category.kind]}>{t(`cleanup.category.${category.kind}`)}</Badge>
|
||||
</Tooltip>
|
||||
<ConfidenceBadge confidence={category.confidence} />
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
{cover && <p className="mt-1.5 break-words text-xs text-slate-500">{t("cleanup.detail.by", { rule: cover })}</p>}
|
||||
{item.basis?.type === "confirmed" && item.basis.confirmed_at && (
|
||||
<p className="mt-1.5 text-xs text-slate-500">{t("cleanup.preview.basisConfirmed", { when: new Date(item.basis.confirmed_at).toLocaleDateString(locale) })}</p>
|
||||
)}
|
||||
{item.basis?.type === "disabled_long" && (
|
||||
<p className="mt-1.5 text-xs text-slate-500">{item.basis.since && !Number.isNaN(Date.parse(item.basis.since))
|
||||
? t("cleanup.detail.disabledSince", { when: new Date(item.basis.since).toLocaleDateString(locale) })
|
||||
: t("cleanup.preview.basisDisabled")}</p>
|
||||
)}
|
||||
<div className="mt-2 flex flex-wrap items-center gap-1.5">
|
||||
{protection && (
|
||||
<Tooltip
|
||||
content={t(`cleanup.protect.${protection.result}_hint`, { sources: sources.length ? sources.join(", ") : t("cleanup.protect.sourcesHidden"), queries: protection.queries })}
|
||||
focusable
|
||||
>
|
||||
<Badge tone={protection.result === "false" ? "success" : "warning"}>
|
||||
{protection.result === "false" ? <ShieldCheck size={12} aria-hidden="true" /> : <TriangleAlert size={12} aria-hidden="true" />}
|
||||
{t(`cleanup.protect.${protection.result}`)}
|
||||
</Badge>
|
||||
</Tooltip>
|
||||
)}
|
||||
{item.external && (
|
||||
<Tooltip content={t("cleanup.preview.externalHint")} focusable><Badge tone="warning">{t("cleanup.preview.external")}</Badge></Tooltip>
|
||||
)}
|
||||
{notes.map((code) => <NoteBadge key={code} code={code} />)}
|
||||
{acknowledged && <Badge tone="neutral">{t("cleanup.preview.acknowledged")}</Badge>}
|
||||
</div>
|
||||
</li>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Select -> (acknowledge) -> preview -> roll out. The preview is recomputed from the live firewall(s) by the backend,
|
||||
* valid for five minutes and bound to whoever created it; only an admin may roll it out (hostname typed). The
|
||||
* confirmation and the automatic revert afterwards live in the operation banner of the tab.
|
||||
*/
|
||||
export function CleanupPreviewModal({ host, request, unusedDays, coverName, onClose, onDeployed }: {
|
||||
host: { id: string; name: string };
|
||||
request: CleanupRequest;
|
||||
unusedDays: number;
|
||||
coverName: (uuid: string) => string | null;
|
||||
onClose: () => void;
|
||||
onDeployed: (operation: CleanupOperation) => void;
|
||||
}) {
|
||||
const { t, errorMessage } = useLanguage();
|
||||
const toast = useToast();
|
||||
const notAdmin = useNotAdmin();
|
||||
const [forced, setForced] = useState<string[]>([]);
|
||||
const [acked, setAcked] = useState<string[]>([]);
|
||||
const [phase, setPhase] = useState<"ack" | "loading" | "plan" | "deploying">(request.rules.some((r) => r.needsAck) ? "ack" : "loading");
|
||||
const [preview, setPreview] = useState<CleanupPreview | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [confirmName, setConfirmName] = useState("");
|
||||
const [now, setNow] = useState(() => Date.now());
|
||||
const [history, setHistory] = useState<CleanupPlanItem | null>(null);
|
||||
const [historyNode, setHistoryNode] = useState<CleanupPlanNode | null>(null);
|
||||
const spent = useRef(false);
|
||||
const previewId = useRef<string | null>(null);
|
||||
const nameErrorId = useId();
|
||||
|
||||
const needsAck = useCallback((uuid: string) => forced.includes(uuid) || request.rules.some((r) => r.uuid === uuid && r.needsAck), [forced, request.rules]);
|
||||
const askAck = request.rules.filter((r) => needsAck(r.uuid));
|
||||
const allAcked = askAck.every((r) => acked.includes(r.uuid));
|
||||
|
||||
const compute = useCallback(async (ackList: string[]) => {
|
||||
setPhase("loading");
|
||||
setError(null);
|
||||
setPreview(null);
|
||||
spent.current = false;
|
||||
try {
|
||||
const result = await api.previewCleanup(host.id, {
|
||||
items: request.rules.map((r) => ({ uuid: r.uuid, kind: request.kind, ack: ackList.includes(r.uuid) })),
|
||||
unused_days: unusedDays,
|
||||
ack_text_version: ACK_TEXT_VERSION,
|
||||
});
|
||||
previewId.current = result.preview_id;
|
||||
setPreview(result);
|
||||
setNow(Date.now());
|
||||
setPhase("plan");
|
||||
} catch (e) {
|
||||
const code = e && typeof e === "object" && "code" in e ? (e as { code?: string }).code : undefined;
|
||||
const params = e && typeof e === "object" && "params" in e ? (e as { params?: Record<string, string> }).params : undefined;
|
||||
if (code === "BEREINIGUNG_BESTAETIGUNG_FEHLT" && params?.uuids) {
|
||||
// The backend sees more reasons for caution than the list did (e.g. an outside change): ask for those too.
|
||||
setForced((current) => Array.from(new Set([...current, ...params.uuids.split(",").map((u) => u.trim())])));
|
||||
setPhase("ack");
|
||||
} else {
|
||||
setPhase("plan");
|
||||
}
|
||||
setError(errorMessage(e, "cleanup.preview.failed"));
|
||||
}
|
||||
}, [host.id, request, unusedDays, errorMessage]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!request.rules.some((r) => r.needsAck)) void compute([]);
|
||||
// Only once at open: later recomputation is the user's explicit action.
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (phase !== "plan" || !preview) return;
|
||||
const timer = setInterval(() => setNow(Date.now()), 1000);
|
||||
return () => clearInterval(timer);
|
||||
}, [phase, preview]);
|
||||
|
||||
// An unused preview is dropped on the way out (a new one replaces it anyway; this keeps the list tidy).
|
||||
useEffect(() => () => {
|
||||
if (!spent.current && previewId.current) void api.cancelCleanup(host.id, previewId.current).catch(() => {});
|
||||
}, [host.id]);
|
||||
|
||||
const left = preview ? secondsUntil(preview.expires_at, now) : 0;
|
||||
const expired = phase === "plan" && preview !== null && left === 0;
|
||||
const nodes = preview ? [...preview.plan.nodes].sort((a, b) => Number(b.role === "BACKUP") - Number(a.role === "BACKUP")) : [];
|
||||
const clustered = nodes.length > 1;
|
||||
const window = preview ? duration(t, preview.window_seconds) : "";
|
||||
const nameOk = confirmName === host.name;
|
||||
|
||||
async function deploy() {
|
||||
if (!preview || !nameOk || notAdmin) return;
|
||||
setPhase("deploying");
|
||||
setError(null);
|
||||
try {
|
||||
spent.current = true;
|
||||
const operation = await api.deployCleanup(host.id, preview.preview_id, confirmName);
|
||||
toast.show(t(operation.status === "awaiting_confirm" ? "cleanup.toast.deployedConfirm" : operation.status === "done" ? "cleanup.toast.deployedDone" : "cleanup.toast.deployedCheck"),
|
||||
operation.status === "awaiting_confirm" || operation.status === "done" ? "success" : "info");
|
||||
onDeployed(operation);
|
||||
} catch (e) {
|
||||
// The backend consumes the preview at the first moment it is looked at: a failed roll-out needs a new one.
|
||||
setError(errorMessage(e, "cleanup.deploy.failed"));
|
||||
toast.show(errorMessage(e, "cleanup.deploy.failed"), "danger");
|
||||
setPreview(null);
|
||||
setPhase("plan");
|
||||
}
|
||||
}
|
||||
|
||||
const title = t(`cleanup.preview.title.${request.kind}`);
|
||||
const hasPlan = phase === "plan" && preview !== null;
|
||||
const deployDisabledReason = notAdmin ? t("cleanup.preview.deployNotAdmin") : expired ? t("cleanup.preview.expired") : !nameOk ? t("cleanup.preview.deployDisabledName") : null;
|
||||
|
||||
return (
|
||||
<>
|
||||
<Modal title={title} wide onClose={() => { if (phase !== "deploying" && !history) onClose(); }}>
|
||||
<div className="space-y-4">
|
||||
<p className="text-sm text-slate-600">
|
||||
{t(`cleanup.preview.intro.${request.kind}`, { host: host.name })}
|
||||
<InfoTip text={t(`cleanup.preview.introHint.${request.kind}`)} size={13} className="ml-1 align-middle" />
|
||||
</p>
|
||||
|
||||
{phase === "ack" && (
|
||||
<div className="space-y-3">
|
||||
<p className="text-sm font-medium text-slate-800">{t(request.kind === "enable" ? "cleanup.ack.enableTitle" : "cleanup.ack.title")}</p>
|
||||
<ul className="space-y-2">
|
||||
{askAck.map((rule) => (
|
||||
<li key={rule.uuid} className="rounded-xl border border-slate-200 p-3">
|
||||
<label className="flex items-start gap-2.5 text-sm">
|
||||
<input
|
||||
type="checkbox"
|
||||
className="mt-0.5 h-4 w-4 shrink-0 rounded border-slate-300"
|
||||
checked={acked.includes(rule.uuid)}
|
||||
onChange={(e) => setAcked((current) => e.target.checked ? [...current, rule.uuid] : current.filter((u) => u !== rule.uuid))}
|
||||
/>
|
||||
<span className="min-w-0">
|
||||
<span className="block break-words font-medium text-slate-900">{rule.label}</span>
|
||||
<span className="block text-slate-600">{t(request.kind === "enable" ? "cleanup.ack.enableText" : "cleanup.ack.text", { rule: rule.label })}</span>
|
||||
</span>
|
||||
</label>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button onClick={() => void compute(acked)} disabled={!allAcked}>{t("cleanup.preview.compute")}</Button>
|
||||
<Button variant="ghost" onClick={onClose}>{t("common.cancel")}</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{phase === "loading" && <Loading>{t("cleanup.preview.loading")}</Loading>}
|
||||
|
||||
{error && <p role="alert" className="break-words rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}
|
||||
|
||||
{phase === "plan" && !preview && (
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button variant="outline" onClick={() => void compute(acked)}><RefreshCw size={14} aria-hidden="true" />{t("cleanup.preview.again")}</Button>
|
||||
<Button variant="ghost" onClick={onClose}>{t("common.close")}</Button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{(hasPlan || phase === "deploying") && preview && (
|
||||
<div className="space-y-4">
|
||||
<div className="flex flex-wrap items-center justify-between gap-2 text-xs text-slate-500">
|
||||
<span>{expired
|
||||
? <span role="alert" className="font-medium text-red-600">{t("cleanup.preview.expired")}</span>
|
||||
: t("cleanup.preview.validFor", { time: formatClock(left) })}</span>
|
||||
{clustered && <span>{t("cleanup.preview.clusterOrder")}</span>}
|
||||
</div>
|
||||
{nodes.map((node, index) => (
|
||||
<section key={node.host_id} aria-label={node.host_name} className="space-y-2">
|
||||
<h3 className="flex flex-wrap items-center gap-2 text-sm font-semibold text-slate-800">
|
||||
{clustered && <span className="text-slate-400">{index + 1}.</span>}
|
||||
{node.host_name}
|
||||
{node.role && <Badge tone={node.role === "MASTER" ? "info" : "neutral"}>{node.role}</Badge>}
|
||||
</h3>
|
||||
<ul className="space-y-2">
|
||||
{node.items.map((item) => (
|
||||
<PlanItem
|
||||
key={item.uuid}
|
||||
item={item}
|
||||
kind={preview.kind}
|
||||
sources={preview.plan.sources ?? []}
|
||||
coverName={coverName}
|
||||
acknowledged={preview.plan.acknowledged.includes(item.given)}
|
||||
onHistory={() => { setHistory(item); setHistoryNode(node); }}
|
||||
/>
|
||||
))}
|
||||
</ul>
|
||||
</section>
|
||||
))}
|
||||
|
||||
<div className="space-y-3 border-t border-slate-200 pt-4">
|
||||
{request.kind !== "delete" && (
|
||||
<p className="flex items-start gap-1.5 rounded-lg bg-flame-50 p-3 text-sm text-flame-800">
|
||||
<TriangleAlert size={16} className="mt-0.5 shrink-0" aria-hidden="true" />
|
||||
<span>{t("cleanup.preview.autoRevert", { window })}</span>
|
||||
</p>
|
||||
)}
|
||||
{!notAdmin && (
|
||||
<label className="block text-sm">
|
||||
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.preview.confirmName")}<InfoTip text={t("cleanup.preview.confirmNameHint")} size={13} /></span>
|
||||
<Input
|
||||
autoComplete="off"
|
||||
spellCheck={false}
|
||||
aria-label={t("cleanup.preview.confirmNameAria", { host: host.name })}
|
||||
aria-describedby={confirmName && !nameOk ? nameErrorId : undefined}
|
||||
value={confirmName}
|
||||
placeholder={host.name}
|
||||
disabled={phase === "deploying"}
|
||||
onChange={(e) => setConfirmName(e.target.value)}
|
||||
onKeyDown={(e) => { if (e.key === "Enter" && nameOk && !expired && phase === "plan") void deploy(); }}
|
||||
/>
|
||||
{confirmName && !nameOk && <span id={nameErrorId} className="mt-1 block text-xs text-slate-500">{t("cleanup.preview.deployDisabledName")}</span>}
|
||||
</label>
|
||||
)}
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Tooltip content={deployDisabledReason} focusable={deployDisabledReason !== null}>
|
||||
<Button
|
||||
variant={request.kind === "delete" ? "danger" : "primary"}
|
||||
onClick={() => void deploy()}
|
||||
disabled={deployDisabledReason !== null || phase === "deploying"}
|
||||
loading={phase === "deploying"}
|
||||
>
|
||||
{t(`cleanup.deploy.${request.kind}`)}
|
||||
</Button>
|
||||
</Tooltip>
|
||||
{expired && <Button variant="outline" onClick={() => void compute(acked)}><RefreshCw size={14} aria-hidden="true" />{t("cleanup.preview.again")}</Button>}
|
||||
<Button variant="ghost" onClick={onClose} disabled={phase === "deploying"}>{t("common.cancel")}</Button>
|
||||
{notAdmin && <InfoTip text={t("cleanup.preview.deployNotAdmin")} size={15} />}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</Modal>
|
||||
{history && historyNode && (
|
||||
<RuleHistoryModal
|
||||
title={history.description}
|
||||
targets={[{ hostId: historyNode.host_id, hostName: historyNode.host_name, uuid: history.uuid }]}
|
||||
onClose={() => { setHistory(null); setHistoryNode(null); }}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Input } from "@/components/ui/Field";
|
||||
import { Modal } from "@/components/ui/Modal";
|
||||
import { useToast } from "@/components/ui/Toast";
|
||||
import { InfoTip } from "@/components/ui/Tooltip";
|
||||
import { api, CleanupItem, CleanupOperation } from "@/lib/api";
|
||||
import { cleanupMessage } from "@/lib/cleanup";
|
||||
import { CircleCheck, TriangleAlert } from "lucide-react";
|
||||
import { useId, useState } from "react";
|
||||
|
||||
/**
|
||||
* Bring a deleted rule back from the content backup OPNexus took before deleting it. The rule returns switched OFF
|
||||
* (a new UUID, at its old position where that still exists); switching it on is a separate step with a confirmation
|
||||
* window. Admin only, hostname typed.
|
||||
*/
|
||||
export function CleanupRestoreModal({ host, item, onClose, onRestored }: {
|
||||
host: { id: string; name: string }; item: CleanupItem; onClose: () => void; onRestored: (operation: CleanupOperation) => void;
|
||||
}) {
|
||||
const { t, errorMessage } = useLanguage();
|
||||
const toast = useToast();
|
||||
const [confirmName, setConfirmName] = useState("");
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [result, setResult] = useState<CleanupOperation | null>(null);
|
||||
const hintId = useId();
|
||||
const nameOk = confirmName === host.name;
|
||||
|
||||
async function restore() {
|
||||
if (!nameOk) return;
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
try {
|
||||
const operation = await api.restoreCleanupItem(host.id, item.id, confirmName);
|
||||
setResult(operation);
|
||||
onRestored(operation);
|
||||
if (operation.status === "done") toast.show(t("cleanup.restore.done", { rule: item.description ?? item.rule_uuid }), "success");
|
||||
else toast.show(cleanupMessage(t, operation.items.find((i) => i.message)?.message ?? operation.message) || t("cleanup.restore.failed"), "danger");
|
||||
} catch (e) {
|
||||
setError(errorMessage(e, "cleanup.restore.failed"));
|
||||
toast.show(errorMessage(e, "cleanup.restore.failed"), "danger");
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Modal title={t("cleanup.restore.title")} onClose={() => { if (!busy) onClose(); }}>
|
||||
<div className="space-y-4">
|
||||
<p className="break-words font-medium text-slate-800">{item.description || item.rule_uuid}</p>
|
||||
{!result && (
|
||||
<>
|
||||
<p className="text-sm text-slate-600">
|
||||
{t("cleanup.restore.intro")}
|
||||
<InfoTip text={t("cleanup.restore.introHint")} size={13} className="ml-1 align-middle" />
|
||||
</p>
|
||||
<label className="block text-sm">
|
||||
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.preview.confirmName")}<InfoTip text={t("cleanup.preview.confirmNameHint")} size={13} /></span>
|
||||
<Input
|
||||
autoComplete="off"
|
||||
spellCheck={false}
|
||||
aria-label={t("cleanup.preview.confirmNameAria", { host: host.name })}
|
||||
aria-describedby={confirmName && !nameOk ? hintId : undefined}
|
||||
value={confirmName}
|
||||
placeholder={host.name}
|
||||
disabled={busy}
|
||||
onChange={(e) => setConfirmName(e.target.value)}
|
||||
onKeyDown={(e) => { if (e.key === "Enter" && nameOk && !busy) void restore(); }}
|
||||
/>
|
||||
{confirmName && !nameOk && <span id={hintId} className="mt-1 block text-xs text-slate-500">{t("cleanup.preview.deployDisabledName")}</span>}
|
||||
</label>
|
||||
</>
|
||||
)}
|
||||
{error && <p role="alert" className="break-words rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}
|
||||
{result && (
|
||||
<div role="status" className="space-y-2 text-sm">
|
||||
{result.status === "done"
|
||||
? <p className="flex items-start gap-1.5 text-emerald-800"><CircleCheck size={16} className="mt-0.5 shrink-0" aria-hidden="true" />{t("cleanup.restore.done", { rule: item.description ?? item.rule_uuid })}</p>
|
||||
: <p className="break-words rounded-lg bg-red-50 p-3 text-red-700">{cleanupMessage(t, result.items.find((i) => i.message)?.message ?? result.message) || t("cleanup.restore.failed")}</p>}
|
||||
{result.warnings.map((code) => (
|
||||
<p key={code} className="flex items-start gap-1.5 rounded-lg bg-flame-50 p-3 text-flame-800"><TriangleAlert size={16} className="mt-0.5 shrink-0" aria-hidden="true" />{cleanupMessage(t, code)}</p>
|
||||
))}
|
||||
<p className="text-slate-600">{t("cleanup.restore.next")}</p>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{!result && <Button onClick={() => void restore()} disabled={!nameOk} loading={busy}>{t("cleanup.restore.action")}</Button>}
|
||||
<Button variant={result ? "primary" : "ghost"} onClick={onClose} disabled={busy}>{result ? t("common.close") : t("common.cancel")}</Button>
|
||||
</div>
|
||||
</div>
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Input } from "@/components/ui/Field";
|
||||
import { InfoTip } from "@/components/ui/Tooltip";
|
||||
import { useToast } from "@/components/ui/Toast";
|
||||
import { api, GovernanceSettings } from "@/lib/api";
|
||||
import { parsePorts, parseSources } from "@/lib/cleanup";
|
||||
import { useNotAdmin } from "@/lib/useReadOnly";
|
||||
import clsx from "clsx";
|
||||
import { useEffect, useId, useState } from "react";
|
||||
|
||||
/**
|
||||
* Settings of the rule cleanup: the management sources (addresses the firewalls are managed from; without them the
|
||||
* cleanup is locked, because a rule that could cut them off is never switched off), the rule limit per run and the
|
||||
* waiting time before a rule may be deleted. Admin only: renders nothing for other roles. `compact` is the inline
|
||||
* variant of the cleanup tab, which only asks for the sources.
|
||||
*/
|
||||
export function CleanupSettings({ compact = false, onSaved }: { compact?: boolean; onSaved?: (settings: GovernanceSettings) => void }) {
|
||||
const { t, errorMessage } = useLanguage();
|
||||
const toast = useToast();
|
||||
const notAdmin = useNotAdmin();
|
||||
const [settings, setSettings] = useState<GovernanceSettings | null>(null);
|
||||
const [sources, setSources] = useState("");
|
||||
const [maxRules, setMaxRules] = useState("10");
|
||||
const [minDays, setMinDays] = useState("7");
|
||||
const [ports, setPorts] = useState("");
|
||||
const [touched, setTouched] = useState(false);
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [error, setError] = useState("");
|
||||
const sourcesId = useId();
|
||||
const portsId = useId();
|
||||
const portsErrorId = useId();
|
||||
const errorId = useId();
|
||||
|
||||
function apply(data: GovernanceSettings) {
|
||||
setSettings(data);
|
||||
setSources(data.cleanup_management_sources.join("\n"));
|
||||
setMaxRules(String(data.cleanup_max_rules));
|
||||
setMinDays(String(data.cleanup_delete_min_days));
|
||||
setPorts((data.cleanup_management_ports ?? []).join(", "));
|
||||
setTouched(false);
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
api.getGovernance().then(apply).catch(() => {});
|
||||
}, []);
|
||||
|
||||
// Reached from the cleanup tab via /settings#cleanup: the section only exists once the page has loaded.
|
||||
useEffect(() => {
|
||||
if (!compact && settings && !notAdmin && window.location.hash === "#cleanup") document.getElementById("cleanup")?.scrollIntoView({ block: "start" });
|
||||
}, [compact, settings, notAdmin]);
|
||||
|
||||
const parsed = parseSources(sources);
|
||||
const sourceError = parsed.error === "invalid" ? t("cleanup.settings.sourcesInvalid", { value: parsed.bad ?? "" })
|
||||
: parsed.error === "everything" ? t("cleanup.settings.sourcesEverything", { value: parsed.bad ?? "" })
|
||||
: parsed.error === "tooMany" ? t("cleanup.settings.sourcesTooMany") : "";
|
||||
const maxValue = Number(maxRules);
|
||||
const daysValue = Number(minDays);
|
||||
const maxError = !Number.isInteger(maxValue) || maxValue < 1 || maxValue > 25 ? t("cleanup.settings.maxRulesInvalid") : "";
|
||||
const daysError = !Number.isInteger(daysValue) || daysValue < 1 || daysValue > 365 ? t("cleanup.settings.minDaysInvalid") : "";
|
||||
const parsedPorts = parsePorts(ports);
|
||||
const portsError = parsedPorts.error === "invalid" ? t("cleanup.settings.portsInvalid", { value: parsedPorts.bad ?? "" })
|
||||
: parsedPorts.error === "tooMany" ? t("cleanup.settings.portsTooMany") : "";
|
||||
const invalid = Boolean(sourceError || (!compact && (maxError || daysError || portsError)));
|
||||
const dirty = settings !== null && (parsed.entries.join("\n") !== settings.cleanup_management_sources.join("\n")
|
||||
|| (!compact && (maxValue !== settings.cleanup_max_rules || daysValue !== settings.cleanup_delete_min_days
|
||||
|| parsedPorts.ports.join(",") !== (settings.cleanup_management_ports ?? []).join(","))));
|
||||
|
||||
if (notAdmin) return null;
|
||||
|
||||
async function save(event: React.FormEvent) {
|
||||
event.preventDefault();
|
||||
setTouched(true);
|
||||
if (invalid) return;
|
||||
setBusy(true);
|
||||
setError("");
|
||||
try {
|
||||
const saved = await api.updateCleanupSettings({
|
||||
cleanup_management_sources: parsed.entries,
|
||||
...(compact ? {} : { cleanup_max_rules: maxValue, cleanup_delete_min_days: daysValue, cleanup_management_ports: parsedPorts.ports }),
|
||||
});
|
||||
apply(saved);
|
||||
toast.show(t("cleanup.settings.saved"), "success");
|
||||
onSaved?.(saved);
|
||||
} catch (e) {
|
||||
setError(errorMessage(e, "common.saveFailed"));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
const form = (
|
||||
<form onSubmit={save} className="space-y-4" noValidate>
|
||||
<div>
|
||||
<label htmlFor={sourcesId} className="mb-1 flex items-center gap-1 text-sm font-medium text-slate-700">
|
||||
{t("cleanup.settings.sources")}
|
||||
<InfoTip text={t("cleanup.settings.sourcesHint")} size={13} />
|
||||
</label>
|
||||
<textarea
|
||||
id={sourcesId}
|
||||
rows={compact ? 2 : 3}
|
||||
value={sources}
|
||||
onChange={(e) => setSources(e.target.value)}
|
||||
onBlur={() => setTouched(true)}
|
||||
placeholder={t("cleanup.settings.sourcesPlaceholder")}
|
||||
spellCheck={false}
|
||||
autoComplete="off"
|
||||
aria-invalid={touched && Boolean(sourceError)}
|
||||
aria-describedby={touched && sourceError ? errorId : undefined}
|
||||
className={clsx(
|
||||
"w-full rounded-lg border px-3 py-2 font-mono text-sm text-slate-900 shadow-sm focus:outline-none focus:ring-1",
|
||||
touched && sourceError ? "border-red-400 focus:border-red-500 focus:ring-red-500" : "border-slate-300 focus:border-brand-500 focus:ring-brand-500",
|
||||
)}
|
||||
/>
|
||||
{touched && sourceError && <p id={errorId} role="alert" className="mt-1 text-xs text-red-600">{sourceError}</p>}
|
||||
</div>
|
||||
{!compact && (
|
||||
<div>
|
||||
<label htmlFor={portsId} className="mb-1 flex items-center gap-1 text-sm font-medium text-slate-700">
|
||||
{t("cleanup.settings.ports")}
|
||||
<InfoTip text={t("cleanup.settings.portsHint")} size={13} />
|
||||
</label>
|
||||
<Input
|
||||
id={portsId}
|
||||
inputMode="numeric"
|
||||
value={ports}
|
||||
onChange={(e) => setPorts(e.target.value)}
|
||||
placeholder={t("cleanup.settings.portsPlaceholder")}
|
||||
autoComplete="off"
|
||||
aria-invalid={Boolean(portsError)}
|
||||
aria-describedby={portsError ? portsErrorId : undefined}
|
||||
/>
|
||||
{portsError && <p id={portsErrorId} role="alert" className="mt-1 text-xs text-red-600">{portsError}</p>}
|
||||
</div>
|
||||
)}
|
||||
{!compact && (
|
||||
<div className="grid gap-4 sm:grid-cols-2">
|
||||
<label className="block text-sm">
|
||||
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.settings.maxRules")}<InfoTip text={t("cleanup.settings.maxRulesHint")} size={13} /></span>
|
||||
<Input type="number" inputMode="numeric" min={1} max={25} value={maxRules} onChange={(e) => setMaxRules(e.target.value)} aria-invalid={Boolean(maxError)} aria-label={t("cleanup.settings.maxRules")} />
|
||||
{maxError && <span role="alert" className="mt-1 block text-xs text-red-600">{maxError}</span>}
|
||||
</label>
|
||||
<label className="block text-sm">
|
||||
<span className="mb-1 flex items-center gap-1 font-medium text-slate-700">{t("cleanup.settings.minDays")}<InfoTip text={t("cleanup.settings.minDaysHint")} size={13} /></span>
|
||||
<Input type="number" inputMode="numeric" min={1} max={365} value={minDays} onChange={(e) => setMinDays(e.target.value)} aria-invalid={Boolean(daysError)} aria-label={t("cleanup.settings.minDays")} />
|
||||
{daysError && <span role="alert" className="mt-1 block text-xs text-red-600">{daysError}</span>}
|
||||
</label>
|
||||
</div>
|
||||
)}
|
||||
{error && <p role="alert" className="text-sm text-red-600">{error}</p>}
|
||||
<Button type="submit" loading={busy} disabled={!dirty || (touched && invalid)}>{t("common.save")}</Button>
|
||||
</form>
|
||||
);
|
||||
|
||||
if (compact) return form;
|
||||
return (
|
||||
<section id="cleanup" className="space-y-4 rounded-xl border border-slate-200 bg-white p-5">
|
||||
<h2 className="flex items-center gap-2 text-lg font-medium">{t("cleanup.settings.title")}<InfoTip text={t("cleanup.settings.intro")} /></h2>
|
||||
{settings === null ? <p className="text-sm text-slate-500">{t("common.loading")}</p> : form}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
"use client";
|
||||
|
||||
import { HistoryTarget, RuleHistoryModal } from "@/components/RuleHistoryModal";
|
||||
import { Card } from "@/components/ui/Card";
|
||||
import { EmptyState } from "@/components/ui/EmptyState";
|
||||
import { Skeleton } from "@/components/ui/Skeleton";
|
||||
@@ -8,7 +9,7 @@ import { api, Host, LiveRule } from "@/lib/api";
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { groupHostsByCluster } from "@/lib/clusters";
|
||||
import clsx from "clsx";
|
||||
import { Check, Network, X } from "lucide-react";
|
||||
import { Check, History, Network, X } from "lucide-react";
|
||||
import Link from "next/link";
|
||||
import { useEffect, useState } from "react";
|
||||
|
||||
@@ -40,6 +41,8 @@ interface DriftRow {
|
||||
fingerprint: string;
|
||||
sample: LiveRule;
|
||||
presentOn: Set<string>;
|
||||
// The rule's id on each firewall (they differ between firewalls); the change history is kept per firewall and id.
|
||||
uuids: Map<string, string>;
|
||||
}
|
||||
|
||||
interface ClusterState {
|
||||
@@ -61,18 +64,19 @@ async function computeClusterDrift(clusterHosts: Host[]): Promise<Omit<ClusterSt
|
||||
);
|
||||
|
||||
const reachable = perHostRules.filter((r): r is { host: Host; rules: LiveRule[] } => r.rules !== null);
|
||||
const byFingerprint = new Map<string, { sample: LiveRule; presentOn: Set<string> }>();
|
||||
const byFingerprint = new Map<string, { sample: LiveRule; presentOn: Set<string>; uuids: Map<string, string> }>();
|
||||
for (const { host, rules } of reachable) {
|
||||
for (const rule of rules) {
|
||||
const fp = fingerprint(rule);
|
||||
const entry = byFingerprint.get(fp) ?? { sample: rule, presentOn: new Set<string>() };
|
||||
const entry = byFingerprint.get(fp) ?? { sample: rule, presentOn: new Set<string>(), uuids: new Map<string, string>() };
|
||||
entry.presentOn.add(host.id);
|
||||
if (!entry.uuids.has(host.id)) entry.uuids.set(host.id, rule.uuid);
|
||||
byFingerprint.set(fp, entry);
|
||||
}
|
||||
}
|
||||
|
||||
const allRows = Array.from(byFingerprint.entries())
|
||||
.map(([fp, entry]) => ({ fingerprint: fp, sample: entry.sample, presentOn: entry.presentOn }))
|
||||
.map(([fp, entry]) => ({ fingerprint: fp, sample: entry.sample, presentOn: entry.presentOn, uuids: entry.uuids }))
|
||||
.sort((a, b) => a.sample.description.localeCompare(b.sample.description));
|
||||
const driftedRows = allRows.filter((row) => row.presentOn.size < reachable.length);
|
||||
|
||||
@@ -85,6 +89,7 @@ export function LiveRulesDrift() {
|
||||
const [ungrouped, setUngrouped] = useState<Host[]>([]);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [onlyDrift, setOnlyDrift] = useState(false);
|
||||
const [historyOf, setHistoryOf] = useState<{ title: string; targets: HistoryTarget[] } | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
api
|
||||
@@ -178,6 +183,7 @@ export function LiveRulesDrift() {
|
||||
{host.name}
|
||||
</th>
|
||||
))}
|
||||
<th className="px-5 py-3 font-medium"><span className="sr-only">{t("common.actions")}</span></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
@@ -209,6 +215,22 @@ export function LiveRulesDrift() {
|
||||
)}
|
||||
</td>
|
||||
))}
|
||||
<td className="px-5 py-3 text-right">
|
||||
{!row.sample.is_automatic && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setHistoryOf({
|
||||
title: row.sample.description,
|
||||
targets: cluster.hosts.filter((h) => row.uuids.has(h.id)).map((h) => ({ hostId: h.id, hostName: h.name, uuid: row.uuids.get(h.id) as string })),
|
||||
})}
|
||||
className="inline-flex items-center gap-1 rounded-lg px-2 py-1 text-xs text-slate-600 hover:bg-slate-100 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
|
||||
aria-label={t("ruleHistory.open", { rule: row.sample.description })}
|
||||
>
|
||||
<History size={14} aria-hidden="true" />
|
||||
{t("ruleHistory.action")}
|
||||
</button>
|
||||
)}
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
@@ -219,6 +241,8 @@ export function LiveRulesDrift() {
|
||||
);
|
||||
})}
|
||||
|
||||
{historyOf && <RuleHistoryModal title={historyOf.title} targets={historyOf.targets} onClose={() => setHistoryOf(null)} />}
|
||||
|
||||
{ungrouped.length > 0 && (
|
||||
<div>
|
||||
<h3 className="mb-2 text-lg font-semibold text-slate-900">{t("drift.noCluster")}</h3>
|
||||
|
||||
@@ -69,7 +69,7 @@ export const navGroups: NavGroup[] = [
|
||||
icon: Shield,
|
||||
items: [
|
||||
{ href: "/rules", label: "Regeln", icon: Shield, keywords: ["Firewall-Regeln", "Filter"] },
|
||||
{ href: "/analysis", label: "Regel-Analyse", icon: ScanSearch, keywords: ["Welche Regel", "Welche Regel trifft", "Pfad", "Simulation", "Firewall-Regel prüfen", "Paket-Test", "Rule match", "Which rule", "Rule analysis", "Packet path", "Trace"] },
|
||||
{ href: "/analysis", label: "Regel-Analyse", icon: ScanSearch, keywords: ["Welche Regel", "Welche Regel trifft", "Pfad", "Simulation", "Firewall-Regel prüfen", "Paket-Test", "Rule match", "Which rule", "Rule analysis", "Packet path", "Trace", "Bereinigung", "ungenutzte Regeln", "verdeckte Regeln", "doppelte Regeln", "Shadow", "Regelverlauf", "Cleanup", "Unused rules", "Shadowed rules", "Duplicate rules", "Rule history"] },
|
||||
{ href: "/policy-hierarchy", label: "Policy-Hierarchie", icon: GitBranch, keywords: ["Pre-Rules", "Post-Rules", "Vererbung", "Inheritance", "Hierarchische Policies"] },
|
||||
{ href: "/groups", label: "Gruppen", icon: FolderTree, keywords: ["Gerätegruppen", "Device Groups"] },
|
||||
{ href: "/aliases", label: "Aliase", icon: Tags, keywords: ["Netzwerk-Objekte", "Netzwerkgruppen"] },
|
||||
|
||||
@@ -0,0 +1,490 @@
|
||||
"use client";
|
||||
|
||||
import { ConfidenceBadge, KIND_TONE } from "@/components/CleanupBadges";
|
||||
import { CleanupPreviewModal, CleanupRequest } from "@/components/CleanupPreviewModal";
|
||||
import { CleanupRestoreModal } from "@/components/CleanupRestoreModal";
|
||||
import { CleanupSettings } from "@/components/CleanupSettings";
|
||||
import { OperationBanner, OperationHistory } from "@/components/CleanupOperations";
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { RuleHistoryModal } from "@/components/RuleHistoryModal";
|
||||
import { Badge } from "@/components/ui/Badge";
|
||||
import { Button } from "@/components/ui/Button";
|
||||
import { Card } from "@/components/ui/Card";
|
||||
import { Collapsible } from "@/components/ui/Collapsible";
|
||||
import { EmptyState } from "@/components/ui/EmptyState";
|
||||
import { FieldGroup, Select } from "@/components/ui/Field";
|
||||
import { Skeleton } from "@/components/ui/Skeleton";
|
||||
import { useToast } from "@/components/ui/Toast";
|
||||
import { InfoTip, Tooltip } from "@/components/ui/Tooltip";
|
||||
import { api, CleanupActionKind, CleanupCandidate, CleanupCategory, CleanupItem, CleanupKind, CleanupOperation, GovernanceSettings, Host, RuleCleanupResult } from "@/lib/api";
|
||||
import { activeOperation, historyOperations, isOpen, selectability } from "@/lib/cleanup";
|
||||
import { useNotAdmin, useReadOnly } from "@/lib/useReadOnly";
|
||||
import { History, Info, ListChecks, RefreshCw, Settings } from "lucide-react";
|
||||
import Link from "next/link";
|
||||
import { useRouter, useSearchParams } from "next/navigation";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
|
||||
const THRESHOLDS = [7, 30, 90, 180];
|
||||
const KINDS: CleanupKind[] = ["unused", "shadowed", "duplicate", "disabled_long"];
|
||||
|
||||
function CategoryLine({ category, days }: { category: CleanupCategory; days: number }) {
|
||||
const { t, locale } = useLanguage();
|
||||
let detail: string | null = null;
|
||||
if (category.kind === "unused") detail = t("cleanup.detail.unused", { days });
|
||||
else if (category.kind === "disabled_long") {
|
||||
detail = category.since ? t("cleanup.detail.disabledSince", { when: new Date(category.since).toLocaleDateString(locale) }) : t("cleanup.detail.disabled", { days });
|
||||
} else if (category.by) {
|
||||
detail = t(category.by.automatic ? "cleanup.detail.byAutomatic" : "cleanup.detail.by", { rule: category.by.description || category.by.uuid || "?" });
|
||||
}
|
||||
return (
|
||||
<div className="flex flex-wrap items-center gap-x-2 gap-y-1">
|
||||
<Tooltip content={t(`cleanup.category.${category.kind}_hint`)} focusable>
|
||||
<Badge tone={KIND_TONE[category.kind]}>{t(`cleanup.category.${category.kind}`)}</Badge>
|
||||
</Tooltip>
|
||||
<ConfidenceBadge confidence={category.confidence} />
|
||||
{detail && <span className="min-w-0 break-words text-xs text-slate-500">{detail}</span>}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
/** What the row offers: the action a selection would run (and whether it needs the rule's own acknowledgement), or why not. */
|
||||
function ActionCell({ candidate, unusedDays }: { candidate: CleanupCandidate; unusedDays: number }) {
|
||||
const { t } = useLanguage();
|
||||
const sel = selectability(candidate, unusedDays);
|
||||
if (sel.mode) {
|
||||
return (
|
||||
<div className="flex flex-col items-start gap-1">
|
||||
<Badge tone={sel.mode === "delete" ? "danger" : "neutral"}>{t(`cleanup.action.${sel.mode}`)}</Badge>
|
||||
{sel.mode === "disable" && sel.needsAck && (
|
||||
<Tooltip content={t("cleanup.select.needsAckHint")} focusable><Badge tone="warning">{t("cleanup.select.needsAck")}</Badge></Tooltip>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<div className="flex flex-col items-start gap-1">
|
||||
<Tooltip content={t(`cleanup.notRemovable.${sel.reason}_hint`)} focusable>
|
||||
<Badge tone="neutral">{t(`cleanup.notRemovable.${sel.reason}`)}</Badge>
|
||||
</Tooltip>
|
||||
{sel.reason === "managed" && (
|
||||
<Link href="/rules" className="text-xs text-brand-600 hover:underline">
|
||||
{t("cleanup.notRemovable.managedLink", { rule: candidate.managed_by?.name ?? "" })}
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const POLL_MS = 5000;
|
||||
|
||||
export function RuleCleanup() {
|
||||
const { t, errorMessage, locale } = useLanguage();
|
||||
const toast = useToast();
|
||||
const readOnly = useReadOnly();
|
||||
const notAdmin = useNotAdmin();
|
||||
const router = useRouter();
|
||||
const params = useSearchParams();
|
||||
const [hosts, setHosts] = useState<Host[] | null>(null);
|
||||
const [result, setResult] = useState<RuleCleanupResult | null>(null);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [tick, setTick] = useState(0);
|
||||
const [refreshNext, setRefreshNext] = useState(false);
|
||||
const [history, setHistory] = useState<CleanupCandidate | null>(null);
|
||||
const [governance, setGovernance] = useState<GovernanceSettings | null>(null);
|
||||
const [operations, setOperations] = useState<CleanupOperation[]>([]);
|
||||
const [opsLoaded, setOpsLoaded] = useState(false);
|
||||
const [selection, setSelection] = useState<{ kind: CleanupActionKind | null; uuids: string[] }>({ kind: null, uuids: [] });
|
||||
const [request, setRequest] = useState<CleanupRequest | null>(null);
|
||||
const [restoring, setRestoring] = useState<CleanupItem | null>(null);
|
||||
const [busy, setBusy] = useState<string | null>(null);
|
||||
const [now, setNow] = useState(() => Date.now());
|
||||
const bannerRef = useRef<HTMLElement>(null);
|
||||
const focusBanner = useRef(false);
|
||||
const lastActive = useRef<string | null>(null);
|
||||
|
||||
const enabledHosts = useMemo(() => (hosts ?? []).filter((h) => h.enabled), [hosts]);
|
||||
const days = THRESHOLDS.includes(Number(params.get("days"))) ? Number(params.get("days")) : 30;
|
||||
const hostId = params.get("host") && enabledHosts.some((h) => h.id === params.get("host")) ? (params.get("host") as string) : enabledHosts[0]?.id ?? "";
|
||||
const host = enabledHosts.find((h) => h.id === hostId) ?? null;
|
||||
const active = activeOperation(operations);
|
||||
const past = useMemo(() => historyOperations(operations), [operations]);
|
||||
const configured = governance?.cleanup_management_sources_configured ?? true;
|
||||
const maxRules = governance?.cleanup_max_rules ?? 10;
|
||||
|
||||
useEffect(() => {
|
||||
api.listHosts().then(setHosts).catch((e) => { setHosts([]); setError(errorMessage(e, "cleanup.loadFailed")); });
|
||||
api.getGovernance().then(setGovernance).catch(() => {});
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (!hostId) { setResult(null); return; }
|
||||
let live = true;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
api.getRuleCleanup(hostId, days, refreshNext)
|
||||
.then((r) => live && setResult(r))
|
||||
.catch((e) => { if (live) { setResult(null); setError(errorMessage(e, "cleanup.loadFailed")); } })
|
||||
.finally(() => { if (live) { setLoading(false); setRefreshNext(false); } });
|
||||
return () => { live = false; };
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [hostId, days, tick]);
|
||||
|
||||
useEffect(() => { setSelection({ kind: null, uuids: [] }); }, [result]);
|
||||
|
||||
const reloadCandidates = useCallback(() => { setRefreshNext(true); setTick((n) => n + 1); }, []);
|
||||
|
||||
const loadOperations = useCallback(async () => {
|
||||
if (!hostId) { setOperations([]); setOpsLoaded(true); return; }
|
||||
try {
|
||||
const list = await api.listCleanupOperations(hostId);
|
||||
setNow(Date.now());
|
||||
setOperations(list);
|
||||
// Somebody else (or the automatic revert) resolved the operation that was open a moment ago.
|
||||
const open = activeOperation(list);
|
||||
const before = lastActive.current;
|
||||
if (before && before !== open?.id) {
|
||||
const ended = list.find((op) => op.id === before);
|
||||
if (ended && !isOpen(ended)) {
|
||||
toast.show(t(`cleanup.toast.ended.${ended.status}`), ended.status === "reverted" ? "info" : "success");
|
||||
reloadCandidates();
|
||||
}
|
||||
}
|
||||
lastActive.current = open?.id ?? null;
|
||||
} catch (e) {
|
||||
setError(errorMessage(e, "cleanup.loadFailed"));
|
||||
} finally {
|
||||
setOpsLoaded(true);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [hostId]);
|
||||
|
||||
useEffect(() => { setOpsLoaded(false); lastActive.current = null; void loadOperations(); }, [loadOperations]);
|
||||
|
||||
// While something is open: poll, and tick the countdown once a second.
|
||||
const activeId = active?.id;
|
||||
useEffect(() => {
|
||||
if (!activeId) return;
|
||||
const poll = setInterval(() => void loadOperations(), POLL_MS);
|
||||
const clock = setInterval(() => setNow(Date.now()), 1000);
|
||||
return () => { clearInterval(poll); clearInterval(clock); };
|
||||
}, [activeId, loadOperations]);
|
||||
|
||||
useEffect(() => {
|
||||
if (activeId && focusBanner.current) { focusBanner.current = false; bannerRef.current?.focus(); }
|
||||
}, [activeId]);
|
||||
|
||||
function update(next: { host?: string; days?: number }) {
|
||||
const q = new URLSearchParams();
|
||||
q.set("tab", "cleanup");
|
||||
q.set("host", next.host ?? hostId);
|
||||
q.set("days", String(next.days ?? days));
|
||||
router.replace(`/analysis?${q.toString()}`);
|
||||
}
|
||||
|
||||
const selectedKind = selection.kind;
|
||||
function toggle(candidate: CleanupCandidate, mode: CleanupActionKind, on: boolean) {
|
||||
const id = candidate.uuid ?? "";
|
||||
setSelection((current) => {
|
||||
const uuids = on ? [...current.uuids.filter((u) => u !== id), id] : current.uuids.filter((u) => u !== id);
|
||||
return { kind: uuids.length ? mode : null, uuids };
|
||||
});
|
||||
}
|
||||
|
||||
function openPreview() {
|
||||
if (!result || !selectedKind) return;
|
||||
const rules = selection.uuids.flatMap((uuid) => {
|
||||
const candidate = result.candidates.find((c) => c.uuid === uuid);
|
||||
if (!candidate) return [];
|
||||
const sel = selectability(candidate, result.unused_days);
|
||||
return [{ uuid, label: candidate.description || uuid, needsAck: sel.mode === "disable" && sel.needsAck }];
|
||||
});
|
||||
setRequest({ kind: selectedKind, rules });
|
||||
}
|
||||
|
||||
function coverName(uuid: string): string | null {
|
||||
for (const candidate of result?.candidates ?? []) {
|
||||
const by = candidate.categories.find((c) => c.by?.uuid === uuid)?.by;
|
||||
if (by) return by.description || by.uuid;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
async function act(name: "confirm" | "revert" | "retry") {
|
||||
if (!active) return;
|
||||
setBusy(name);
|
||||
try {
|
||||
const fn = name === "confirm" ? api.confirmCleanup : name === "revert" ? api.revertCleanup : api.retryCleanup;
|
||||
// the operation's own anchor host: in a cluster it may be the other node than the one shown
|
||||
const op = await fn(active.host_id, active.id);
|
||||
const specific = t(`cleanup.toast.${name}.${op.status}`);
|
||||
toast.show(specific === `cleanup.toast.${name}.${op.status}` ? t(`cleanup.toast.${name}.done`) : specific,
|
||||
op.status === "revert_failed" || op.status === "failed" ? "danger" : "success");
|
||||
lastActive.current = null;
|
||||
await loadOperations();
|
||||
reloadCandidates();
|
||||
} catch (e) {
|
||||
toast.show(errorMessage(e, `cleanup.${name}Failed`), "danger");
|
||||
await loadOperations();
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
function followUp(kind: "enable" | "delete", item: CleanupItem, uuid: string) {
|
||||
setRequest({ kind, rules: [{ uuid, label: item.description || uuid, needsAck: false }] });
|
||||
}
|
||||
|
||||
const total = result ? KINDS.reduce((sum, kind) => sum + result.counts[kind], 0) : 0;
|
||||
const locked = !configured;
|
||||
const limitReached = selection.uuids.length >= maxRules;
|
||||
const previewBlockedReason = locked ? t("cleanup.select.locked") : active ? t("cleanup.select.busy") : selection.uuids.length === 0 ? t("cleanup.select.none") : null;
|
||||
const checkboxLabel = (c: CleanupCandidate) => t("cleanup.select.row", { rule: c.description || c.uuid || "" });
|
||||
|
||||
return (
|
||||
<div>
|
||||
<Card className="mb-6 p-4 sm:p-5">
|
||||
<form onSubmit={(e) => e.preventDefault()} aria-label={t("cleanup.form.title")} className="flex flex-wrap items-end gap-4">
|
||||
<div className="w-full sm:w-64">
|
||||
<FieldGroup label={t("cleanup.form.host")}>
|
||||
<Select value={hostId} onChange={(e) => update({ host: e.target.value })} disabled={enabledHosts.length === 0}>
|
||||
{enabledHosts.map((h) => <option key={h.id} value={h.id}>{h.name}</option>)}
|
||||
</Select>
|
||||
</FieldGroup>
|
||||
</div>
|
||||
<div className="w-full sm:w-48">
|
||||
<FieldGroup label={t("cleanup.form.threshold")} hint={t("cleanup.form.thresholdHint")}>
|
||||
<Select value={String(days)} onChange={(e) => update({ days: Number(e.target.value) })}>
|
||||
{THRESHOLDS.map((n) => <option key={n} value={n}>{t("cleanup.form.days", { n })}</option>)}
|
||||
</Select>
|
||||
</FieldGroup>
|
||||
</div>
|
||||
<Button type="button" variant="outline" onClick={() => { reloadCandidates(); void loadOperations(); }} disabled={loading || !hostId} title={t("cleanup.form.refreshHint")}>
|
||||
<RefreshCw size={14} aria-hidden="true" />
|
||||
{t("common.refresh")}
|
||||
</Button>
|
||||
</form>
|
||||
</Card>
|
||||
|
||||
{locked && (
|
||||
<Card className="mb-6 p-4 sm:p-5">
|
||||
<div className="flex items-start gap-3">
|
||||
<Info size={18} className="mt-0.5 shrink-0 text-slate-400" aria-hidden="true" />
|
||||
<div className="min-w-0 flex-1 space-y-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium text-slate-800">{t("cleanup.mgmt.title")}</p>
|
||||
<p className="mt-0.5 text-sm text-slate-600">
|
||||
{t("cleanup.mgmt.text")}
|
||||
<InfoTip text={t("cleanup.mgmt.hint")} size={13} className="ml-1 align-middle" />
|
||||
</p>
|
||||
</div>
|
||||
{notAdmin ? <p className="text-sm text-slate-500">{t("cleanup.mgmt.askAdmin")}</p> : <CleanupSettings compact onSaved={setGovernance} />}
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{active && host && (
|
||||
<OperationBanner
|
||||
ref={bannerRef}
|
||||
operation={active}
|
||||
now={now}
|
||||
busy={busy}
|
||||
readOnly={readOnly}
|
||||
notAdmin={notAdmin}
|
||||
operatorsMayDeploy={governance?.operators_may_deploy ?? true}
|
||||
onConfirm={() => void act("confirm")}
|
||||
onRevert={() => void act("revert")}
|
||||
onRetry={() => void act("retry")}
|
||||
/>
|
||||
)}
|
||||
|
||||
{error && <p role="alert" className="mb-4 text-sm text-red-600">{error}</p>}
|
||||
{hosts !== null && enabledHosts.length === 0 && <EmptyState icon={ListChecks} title={t("cleanup.noHosts")} />}
|
||||
{loading && !result && <Skeleton className="h-40 w-full" />}
|
||||
|
||||
{result && (
|
||||
<div aria-busy={loading}>
|
||||
<div className="mb-3 flex flex-wrap items-center gap-2">
|
||||
{KINDS.map((kind) => (
|
||||
<Tooltip key={kind} content={t(`cleanup.category.${kind}_hint`)} focusable>
|
||||
<Badge tone={result.counts[kind] > 0 ? KIND_TONE[kind] : "neutral"}>{t(`cleanup.category.${kind}`)}: {result.counts[kind]}</Badge>
|
||||
</Tooltip>
|
||||
))}
|
||||
<span className="text-xs text-slate-500">{t("cleanup.rulesChecked", { count: result.rules_total })}</span>
|
||||
<InfoTip
|
||||
text={result.usage.watching_since
|
||||
? t("cleanup.watching", { when: new Date(result.usage.watching_since).toLocaleDateString(locale), collecting: result.usage.collecting })
|
||||
: t("cleanup.notWatching")}
|
||||
size={14}
|
||||
/>
|
||||
{!notAdmin && !locked && (
|
||||
<Link href="/settings#cleanup" className="ml-auto inline-flex items-center gap-1 text-xs text-brand-600 hover:underline">
|
||||
<Settings size={13} aria-hidden="true" />{t("cleanup.settings.link")}
|
||||
</Link>
|
||||
)}
|
||||
</div>
|
||||
{result.notes.length > 0 && (
|
||||
<div className="mb-3 flex flex-wrap gap-1.5">
|
||||
{result.notes.map((code) => {
|
||||
const label = t(`cleanup.notes.${code}`);
|
||||
return (
|
||||
<Tooltip key={code} content={label === `cleanup.notes.${code}` ? code : t(`cleanup.notes.${code}_hint`)} focusable>
|
||||
<Badge tone="warning">{label === `cleanup.notes.${code}` ? code : label}</Badge>
|
||||
</Tooltip>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{total === 0 ? (
|
||||
<EmptyState icon={ListChecks} title={t("cleanup.empty.title")} description={t("cleanup.empty.description", { days })} />
|
||||
) : (
|
||||
<>
|
||||
{!readOnly && (
|
||||
<div className="mb-3 flex flex-wrap items-center gap-3">
|
||||
<p role="status" aria-live="polite" className="text-sm text-slate-600">{t("cleanup.select.count", { n: selection.uuids.length, max: maxRules })}</p>
|
||||
<Tooltip content={previewBlockedReason} focusable={previewBlockedReason !== null}>
|
||||
<Button onClick={openPreview} disabled={previewBlockedReason !== null}>
|
||||
<ListChecks size={15} aria-hidden="true" />
|
||||
{t(selectedKind === "delete" ? "cleanup.select.previewDelete" : "cleanup.select.previewDisable")}
|
||||
</Button>
|
||||
</Tooltip>
|
||||
{selection.uuids.length > 0 && <Button variant="ghost" onClick={() => setSelection({ kind: null, uuids: [] })}>{t("cleanup.select.clear")}</Button>}
|
||||
</div>
|
||||
)}
|
||||
<Card className="overflow-x-auto p-0">
|
||||
<table className="w-full text-left text-sm">
|
||||
<caption className="sr-only">{t("cleanup.tableCaption", { host: result.host_name })}</caption>
|
||||
<thead className="border-b border-slate-200 bg-slate-50 text-xs uppercase tracking-wide text-slate-500">
|
||||
<tr>
|
||||
{!readOnly && <th scope="col" className="w-10 px-4 py-3"><span className="sr-only">{t("cleanup.select.column")}</span></th>}
|
||||
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.rule")}</th>
|
||||
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.finding")}</th>
|
||||
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.lastHit")}</th>
|
||||
<th scope="col" className="px-4 py-3 font-medium">{t("cleanup.col.action")}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{result.candidates.map((candidate) => {
|
||||
const sel = selectability(candidate, result.unused_days);
|
||||
const id = candidate.uuid ?? "";
|
||||
const checked = selection.uuids.includes(id);
|
||||
const reason = sel.mode === null ? t(`cleanup.notRemovable.${sel.reason}_hint`)
|
||||
: locked ? t("cleanup.select.locked")
|
||||
: active ? t("cleanup.select.busy")
|
||||
: selectedKind && selectedKind !== sel.mode ? t("cleanup.select.otherKind")
|
||||
: !checked && limitReached ? t("cleanup.select.limit", { max: maxRules })
|
||||
: null;
|
||||
return (
|
||||
<tr key={candidate.uuid ?? candidate.label ?? candidate.description} className={`border-b border-slate-100 align-top last:border-0 ${checked ? "bg-brand-50/40" : ""}`}>
|
||||
{!readOnly && (
|
||||
<td className="px-4 py-3">
|
||||
<Tooltip content={reason} focusable={reason !== null}>
|
||||
<input
|
||||
type="checkbox"
|
||||
className="h-4 w-4 rounded border-slate-300"
|
||||
checked={checked}
|
||||
disabled={reason !== null && !checked}
|
||||
aria-label={checkboxLabel(candidate)}
|
||||
onChange={(e) => sel.mode && toggle(candidate, sel.mode, e.target.checked)}
|
||||
/>
|
||||
</Tooltip>
|
||||
</td>
|
||||
)}
|
||||
<td className="max-w-[18rem] px-4 py-3">
|
||||
<div className="flex items-start gap-1.5">
|
||||
<span className="min-w-0 break-words font-medium text-slate-900">{candidate.description || candidate.uuid}</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setHistory(candidate)}
|
||||
className="shrink-0 rounded p-1 text-slate-400 hover:bg-slate-100 hover:text-slate-700 focus-visible:outline focus-visible:outline-2 focus-visible:outline-brand-500"
|
||||
aria-label={t("cleanup.history", { rule: candidate.description || candidate.uuid || "" })}
|
||||
title={t("cleanup.historyTitle")}
|
||||
>
|
||||
<History size={15} aria-hidden="true" />
|
||||
</button>
|
||||
</div>
|
||||
<p className="mt-0.5 break-words text-xs text-slate-500">
|
||||
{candidate.action ? t(`analysis.verdict.${candidate.action}`) : "?"} · {candidate.interfaces.join(", ") || t("cleanup.allInterfaces")} · {candidate.source} → {candidate.destination}
|
||||
{candidate.destination_port ? `:${candidate.destination_port}` : ""}
|
||||
{!candidate.enabled && <span className="ml-1">({t("cleanup.disabledRule")})</span>}
|
||||
</p>
|
||||
</td>
|
||||
<td className="min-w-[16rem] px-4 py-3">
|
||||
<div className="space-y-1.5">
|
||||
{candidate.categories.map((category) => <CategoryLine key={category.kind} category={category} days={result.unused_days} />)}
|
||||
</div>
|
||||
</td>
|
||||
<td className="whitespace-nowrap px-4 py-3 text-slate-600">
|
||||
{candidate.last_hit_at
|
||||
? new Date(candidate.last_hit_at).toLocaleString(locale)
|
||||
: candidate.watched_since
|
||||
? <Tooltip content={t("cleanup.noHitSince", { when: new Date(candidate.watched_since).toLocaleDateString(locale) })} focusable><span>{t("cleanup.noHit")}</span></Tooltip>
|
||||
: <span className="text-slate-300">–</span>}
|
||||
</td>
|
||||
<td className="min-w-[9rem] px-4 py-3"><ActionCell candidate={candidate} unusedDays={result.unused_days} /></td>
|
||||
</tr>
|
||||
);
|
||||
})}
|
||||
</tbody>
|
||||
</table>
|
||||
</Card>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{host && opsLoaded && (
|
||||
<Collapsible title={t("cleanup.hist.title")} count={past.length} className="mt-8">
|
||||
<OperationHistory
|
||||
operations={past}
|
||||
hostId={hostId}
|
||||
now={now}
|
||||
minDays={governance?.cleanup_delete_min_days ?? 7}
|
||||
readOnly={readOnly}
|
||||
notAdmin={notAdmin}
|
||||
busy={busy !== null || active !== null}
|
||||
onRestore={setRestoring}
|
||||
onFollowUp={followUp}
|
||||
/>
|
||||
</Collapsible>
|
||||
)}
|
||||
|
||||
{request && host && (
|
||||
<CleanupPreviewModal
|
||||
host={{ id: host.id, name: host.name }}
|
||||
request={request}
|
||||
unusedDays={result?.unused_days ?? days}
|
||||
coverName={coverName}
|
||||
onClose={() => setRequest(null)}
|
||||
onDeployed={(operation) => {
|
||||
setRequest(null);
|
||||
setSelection({ kind: null, uuids: [] });
|
||||
if (isOpen(operation)) { focusBanner.current = true; lastActive.current = operation.id; }
|
||||
setOperations((current) => [operation, ...current.filter((op) => op.id !== operation.id)]);
|
||||
setNow(Date.now());
|
||||
void loadOperations();
|
||||
reloadCandidates();
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{restoring && host && (
|
||||
<CleanupRestoreModal
|
||||
host={{ id: host.id, name: host.name }}
|
||||
item={restoring}
|
||||
onClose={() => setRestoring(null)}
|
||||
onRestored={() => { void loadOperations(); reloadCandidates(); }}
|
||||
/>
|
||||
)}
|
||||
{history && result && (
|
||||
<RuleHistoryModal
|
||||
title={history.description || history.uuid || ""}
|
||||
targets={[{ hostId: result.host_id, hostName: result.host_name, uuid: history.uuid ?? "" }]}
|
||||
onClose={() => setHistory(null)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
"use client";
|
||||
|
||||
import { useLanguage } from "@/components/LanguageProvider";
|
||||
import { Badge, BadgeTone } from "@/components/ui/Badge";
|
||||
import { EmptyState } from "@/components/ui/EmptyState";
|
||||
import { Select } from "@/components/ui/Field";
|
||||
import { Loading } from "@/components/ui/Loading";
|
||||
import { Modal } from "@/components/ui/Modal";
|
||||
import { Tooltip } from "@/components/ui/Tooltip";
|
||||
import { api, RuleHistory, RuleHistoryChange, RuleHistoryEvent } from "@/lib/api";
|
||||
import { History } from "lucide-react";
|
||||
import { useEffect, useState } from "react";
|
||||
|
||||
/** One firewall on which the rule exists, with the rule's id there (ids differ between firewalls). */
|
||||
export interface HistoryTarget { hostId: string; hostName: string; uuid: string; }
|
||||
|
||||
const KIND_TONE: Record<string, BadgeTone> = {
|
||||
added: "success", changed: "info", moved: "neutral", removed: "danger", enabled: "success", disabled: "warning",
|
||||
};
|
||||
|
||||
function shorten(value: unknown): string {
|
||||
const text = value === null || value === undefined || value === "" ? "–" : typeof value === "string" ? value : JSON.stringify(value);
|
||||
return text.length > 60 ? `${text.slice(0, 57)}…` : text;
|
||||
}
|
||||
|
||||
function useOptionalText() {
|
||||
const { t } = useLanguage();
|
||||
// Unknown codes (a newer backend, an audit action we have no wording for) show raw instead of breaking the list.
|
||||
return (key: string, fallback: string) => {
|
||||
const text = t(key);
|
||||
return text === key ? fallback : text;
|
||||
};
|
||||
}
|
||||
|
||||
function ChangeList({ changes }: { changes: RuleHistoryChange[] }) {
|
||||
const optional = useOptionalText();
|
||||
if (changes.length === 0) return null;
|
||||
return (
|
||||
<ul className="mt-1 space-y-0.5 text-xs text-slate-600">
|
||||
{changes.map((change) => (
|
||||
<li key={change.field} className="break-words">
|
||||
<span className="font-medium text-slate-700">{optional(`ruleHistory.field.${change.field}`, change.field)}</span>
|
||||
{": "}
|
||||
<span className="font-mono text-slate-500 line-through decoration-slate-300">{shorten(change.before)}</span>
|
||||
{" → "}
|
||||
<span className="font-mono text-slate-800">{shorten(change.after)}</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
);
|
||||
}
|
||||
|
||||
function EventRow({ event }: { event: RuleHistoryEvent }) {
|
||||
const { t, locale } = useLanguage();
|
||||
const optional = useOptionalText();
|
||||
const label = event.source === "observed"
|
||||
? optional(`ruleHistory.kind.${event.kind}`, event.kind)
|
||||
: event.source === "revision"
|
||||
? t("ruleHistory.kind.revision")
|
||||
: optional(`ruleHistory.audit.${event.kind}`, event.kind);
|
||||
const tone: BadgeTone = event.source === "observed" ? KIND_TONE[event.kind] ?? "neutral" : "neutral";
|
||||
const showChanges = event.source === "revision" || event.kind === "changed" || event.kind === "moved";
|
||||
return (
|
||||
<li className="border-b border-slate-100 py-3 last:border-0">
|
||||
<div className="flex flex-wrap items-center gap-x-2 gap-y-1">
|
||||
<Badge tone={tone}>{label}</Badge>
|
||||
{event.source === "audit" && event.status && <span className="text-xs text-slate-500">{event.status}</span>}
|
||||
<Tooltip content={t(`ruleHistory.origin.${event.origin}_hint`)} focusable>
|
||||
<Badge tone={event.origin === "opnexus" ? "accent" : "warning"}>{t(`ruleHistory.origin.${event.origin}`)}</Badge>
|
||||
</Tooltip>
|
||||
<time dateTime={event.at} className="text-xs text-slate-500">{new Date(event.at).toLocaleString(locale)}</time>
|
||||
{event.actor && <span className="text-xs text-slate-500">{t("ruleHistory.by", { actor: event.actor })}</span>}
|
||||
</div>
|
||||
{event.note && <p className="mt-1 text-sm text-slate-600">{event.note}</p>}
|
||||
{event.message && <p className="mt-1 break-words text-xs text-slate-500">{event.message}</p>}
|
||||
{showChanges && <ChangeList changes={event.changes} />}
|
||||
</li>
|
||||
);
|
||||
}
|
||||
|
||||
export function RuleHistoryModal({ title, targets, onClose }: { title: string; targets: HistoryTarget[]; onClose: () => void }) {
|
||||
const { t, errorMessage, locale } = useLanguage();
|
||||
const [index, setIndex] = useState(0);
|
||||
const [history, setHistory] = useState<RuleHistory | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const target = targets[Math.min(index, targets.length - 1)];
|
||||
|
||||
useEffect(() => {
|
||||
if (!target) return;
|
||||
let active = true;
|
||||
setHistory(null);
|
||||
setError(null);
|
||||
api.getRuleHistory(target.hostId, target.uuid)
|
||||
.then((h) => active && setHistory(h))
|
||||
.catch((e) => active && setError(errorMessage(e, "ruleHistory.loadFailed")));
|
||||
return () => { active = false; };
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [target?.hostId, target?.uuid]);
|
||||
|
||||
return (
|
||||
<Modal title={t("ruleHistory.title", { rule: title })} onClose={onClose} wide>
|
||||
{targets.length > 1 && (
|
||||
<div className="mb-3">
|
||||
<label htmlFor="rule-history-host" className="mb-1 block text-sm font-medium text-slate-700">{t("ruleHistory.host")}</label>
|
||||
<Select id="rule-history-host" value={String(index)} onChange={(e) => setIndex(Number(e.target.value))}>
|
||||
{targets.map((item, i) => <option key={item.hostId} value={i}>{item.hostName}</option>)}
|
||||
</Select>
|
||||
</div>
|
||||
)}
|
||||
{error && <p role="alert" className="text-sm text-red-600">{error}</p>}
|
||||
{!error && history === null && <Loading>{t("common.loading")}</Loading>}
|
||||
{history && history.events.length === 0 && (
|
||||
<EmptyState
|
||||
icon={History}
|
||||
title={t("ruleHistory.empty.title")}
|
||||
description={history.watching_since
|
||||
? t("ruleHistory.empty.since", { when: new Date(history.watching_since).toLocaleString(locale) })
|
||||
: t("ruleHistory.empty.notYet")}
|
||||
/>
|
||||
)}
|
||||
{history && history.events.length > 0 && (
|
||||
<>
|
||||
{history.watching_since && (
|
||||
<p className="mb-1 text-xs text-slate-500">{t("ruleHistory.since", { when: new Date(history.watching_since).toLocaleString(locale) })}</p>
|
||||
)}
|
||||
<ul aria-label={t("ruleHistory.list")}>
|
||||
{history.events.map((event, i) => <EventRow key={`${event.source}-${event.at}-${i}`} event={event} />)}
|
||||
</ul>
|
||||
</>
|
||||
)}
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
@@ -6,9 +6,11 @@ interface ModalProps {
|
||||
title: string;
|
||||
onClose: () => void;
|
||||
children: ReactNode;
|
||||
/** Wider dialog for lists and tables. */
|
||||
wide?: boolean;
|
||||
}
|
||||
|
||||
export function Modal({ title, onClose, children }: ModalProps) {
|
||||
export function Modal({ title, onClose, children, wide = false }: ModalProps) {
|
||||
const titleId = useId();
|
||||
const { t } = useLanguage();
|
||||
const dialogRef = useRef<HTMLDivElement>(null);
|
||||
@@ -53,7 +55,7 @@ export function Modal({ title, onClose, children }: ModalProps) {
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
aria-labelledby={titleId}
|
||||
className="flex max-h-[90vh] outline-none w-full max-w-lg flex-col rounded-2xl border border-slate-200 bg-white p-5 shadow-xl sm:p-6"
|
||||
className={`flex max-h-[90vh] outline-none w-full ${wide ? "max-w-2xl" : "max-w-lg"} flex-col rounded-2xl border border-slate-200 bg-white p-5 shadow-xl sm:p-6`}
|
||||
>
|
||||
<div className="mb-4 flex shrink-0 items-center justify-between">
|
||||
<h2 id={titleId} className="text-lg font-semibold text-slate-900">
|
||||
|
||||
+92
-2
@@ -1574,6 +1574,21 @@ export const api = {
|
||||
return request<AnalysisResult[]>(`/rule-analysis/match?${params.toString()}`);
|
||||
},
|
||||
getRuleHits: (id: string) => request<RuleHit[]>(`/hosts/${id}/rule-hits`),
|
||||
getRuleCleanup: (id: string, unusedDays: number, refresh = false) =>
|
||||
request<RuleCleanupResult>(`/hosts/${id}/rule-analysis/cleanup?unused_days=${unusedDays}${refresh ? "&refresh=true" : ""}`),
|
||||
listCleanupOperations: (id: string) => request<CleanupOperation[]>(`/hosts/${id}/rule-cleanup`),
|
||||
previewCleanup: (id: string, body: CleanupPreviewRequest) =>
|
||||
request<CleanupPreview>(`/hosts/${id}/rule-cleanup/preview`, { method: "POST", body: JSON.stringify(body) }),
|
||||
cancelCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/cancel`, { method: "POST" }),
|
||||
deployCleanup: (id: string, op: string, confirmName: string) =>
|
||||
request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/deploy`, { method: "POST", body: JSON.stringify({ confirm_name: confirmName }) }),
|
||||
confirmCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/confirm`, { method: "POST" }),
|
||||
revertCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/revert`, { method: "POST" }),
|
||||
retryCleanup: (id: string, op: string) => request<CleanupOperation>(`/hosts/${id}/rule-cleanup/${op}/retry`, { method: "POST" }),
|
||||
restoreCleanupItem: (id: string, item: string, confirmName: string) =>
|
||||
request<CleanupOperation>(`/hosts/${id}/rule-cleanup/items/${item}/restore`, { method: "POST", body: JSON.stringify({ confirm_name: confirmName }) }),
|
||||
getRuleHistory: (id: string, ruleUuid: string) =>
|
||||
request<RuleHistory>(`/hosts/${id}/live-rules/${encodeURIComponent(ruleUuid)}/history`),
|
||||
checkHostCompliance: (id: string) => request<ComplianceHost>(`/hosts/${id}/compliance/check`, { method: "POST" }),
|
||||
checkFleetCompliance: () => request<ComplianceHost[]>("/compliance/check", { method: "POST" }),
|
||||
|
||||
@@ -1581,8 +1596,9 @@ export const api = {
|
||||
getOidcSettings: () => request<OidcSettings>("/settings/oidc"),
|
||||
updateOidcSettings: (data: OidcSettingsUpdate) => request<OidcSettings>("/settings/oidc", { method: "PUT", body: JSON.stringify(data) }),
|
||||
testOidc: () => request<{ success: boolean; authorization_endpoint: string }>("/settings/oidc/test", { method: "POST" }),
|
||||
getGovernance: () => request<{ operators_may_deploy: boolean }>("/settings/governance"),
|
||||
updateGovernance: (operators_may_deploy: boolean) => request<{ operators_may_deploy: boolean }>("/settings/governance", { method: "PUT", body: JSON.stringify({ operators_may_deploy }) }),
|
||||
getGovernance: () => request<GovernanceSettings>("/settings/governance"),
|
||||
updateGovernance: (operators_may_deploy: boolean) => request<GovernanceSettings>("/settings/governance", { method: "PUT", body: JSON.stringify({ operators_may_deploy }) }),
|
||||
updateCleanupSettings: (data: CleanupSettingsUpdate) => request<GovernanceSettings>("/settings/governance", { method: "PUT", body: JSON.stringify(data) }),
|
||||
|
||||
getOverlay: () => request<{ settings: OverlaySettings; peers: OverlayPeer[] }>("/overlay"),
|
||||
updateOverlay: (data: Partial<Pick<OverlaySettings, "enabled" | "endpoint" | "subnet" | "listen_port">>) =>
|
||||
@@ -2135,3 +2151,77 @@ export interface AnalysisQuery {
|
||||
src: string; dst: string; proto?: string; dport?: string; sport?: string; icmptype?: string;
|
||||
iface?: string; egress?: string; host_ids?: string; group_id?: string; refresh?: boolean;
|
||||
}
|
||||
|
||||
export type CleanupKind = "unused" | "shadowed" | "duplicate" | "disabled_long";
|
||||
export type CleanupBlock = "automatic" | "legacy" | "managed";
|
||||
export interface CleanupCategory {
|
||||
kind: CleanupKind; confidence: AnalysisConfidence; days?: number; since?: string;
|
||||
by?: { uuid: string | null; description: string; section: string; automatic: boolean };
|
||||
}
|
||||
export interface CleanupCandidate extends AnalysisRule {
|
||||
enabled: boolean; categories: CleanupCategory[]; hit_state: "used" | "unused" | "collecting" | "disabled" | null;
|
||||
last_hit_at: string | null; watched_since: string | null; removable: boolean; not_removable_reason: CleanupBlock | null;
|
||||
managed_by: { kind: "rule" | "policy_hierarchy"; id: string; name: string } | null;
|
||||
}
|
||||
export interface RuleCleanupResult {
|
||||
host_id: string; host_name: string; cluster_name: string | null; unused_days: number; generated_at: string;
|
||||
rules_total: number; candidates: CleanupCandidate[]; counts: Record<CleanupKind, number>; notes: string[];
|
||||
usage: { tracked: number; collecting: number; watching_since: string | null };
|
||||
history: { watching_since: string | null };
|
||||
}
|
||||
export interface GovernanceSettings {
|
||||
operators_may_deploy: boolean; cleanup_max_rules: number; cleanup_delete_min_days: number;
|
||||
// The list itself only comes for admins; everybody else just learns whether it is set.
|
||||
cleanup_management_sources: string[]; cleanup_management_sources_configured: boolean;
|
||||
// Extra management ports (besides API port, 443, 80, 22); the list only for admins, too.
|
||||
cleanup_management_ports: number[]; cleanup_management_ports_configured: boolean;
|
||||
}
|
||||
export interface CleanupSettingsUpdate {
|
||||
cleanup_max_rules?: number; cleanup_delete_min_days?: number; cleanup_management_sources?: string[];
|
||||
cleanup_management_ports?: number[];
|
||||
}
|
||||
export type CleanupActionKind = "disable" | "enable" | "delete";
|
||||
export type CleanupOperationKind = CleanupActionKind | "restore";
|
||||
export type CleanupOperationStatus =
|
||||
| "preview" | "running" | "awaiting_confirm" | "confirmed" | "partial" | "apply_failed" | "reverting" | "reverted"
|
||||
| "revert_failed" | "done" | "failed" | "cancelled" | "expired";
|
||||
export type CleanupItemState =
|
||||
| "queued" | "disabling" | "disabled" | "confirmed" | "reverting" | "reverted" | "deleting" | "deleted" | "restoring"
|
||||
| "restored" | "failed" | "unknown" | "revert_failed" | "skipped";
|
||||
export interface CleanupItem {
|
||||
id: string; host_id: string; host_name: string | null; rule_uuid: string; seq: number; carp_role: string | null;
|
||||
state: CleanupItemState; description: string | null; categories: CleanupCategory[] | null; confidence: AnalysisConfidence | null;
|
||||
cover_uuid: string | null; restored_uuid: string | null; changed_at: string | null; message: string | null;
|
||||
has_snapshot: boolean; restorable: boolean;
|
||||
}
|
||||
export interface CleanupPlanItem {
|
||||
uuid: string; given: string; seq: number; description: string; action: string | null;
|
||||
categories: { kind: CleanupKind; confidence: AnalysisConfidence; notes?: string[]; days?: number }[];
|
||||
level: AnalysisConfidence | null; needs_ack: boolean; notes: string[]; cover_uuid: string | null;
|
||||
protection: { result: "false" | "true" | "unknown"; queries: number; reasons: string[] } | null;
|
||||
external: string | null;
|
||||
basis?: { type: "confirmed" | "disabled_long"; confirmed_at?: string; since?: string };
|
||||
}
|
||||
export interface CleanupPlanNode { host_id: string; host_name: string; role: string | null; items: CleanupPlanItem[]; }
|
||||
// sources/ports only for admins (the backend leaves them out for everybody else, sources_configured instead).
|
||||
export interface CleanupPlan { kind: CleanupActionKind; uuids: string[]; unused_days: number; sources?: string[]; ports?: number[]; sources_configured?: boolean; acknowledged: string[]; nodes: CleanupPlanNode[]; }
|
||||
export interface CleanupOperation {
|
||||
id: string; host_id: string; host_name: string | null; cluster_name: string | null; kind: CleanupOperationKind;
|
||||
status: CleanupOperationStatus; actor: string; created_at: string | null; expires_at: string | null;
|
||||
revert_deadline: string | null; confirmed_at: string | null; finished_at: string | null; message: string | null;
|
||||
plan: CleanupPlan | null; items: CleanupItem[]; can_confirm: boolean; can_revert: boolean; can_retry: boolean; warnings: string[];
|
||||
}
|
||||
export interface CleanupPreview {
|
||||
preview_id: string; expires_at: string | null; kind: CleanupActionKind; plan: CleanupPlan; max_rules: number;
|
||||
window_seconds: number; operation: CleanupOperation;
|
||||
}
|
||||
export interface CleanupPreviewRequest {
|
||||
items: { uuid: string; kind: CleanupActionKind; ack: boolean }[]; unused_days: number; ack_text_version: string;
|
||||
}
|
||||
export interface RuleHistoryChange { field: string; before: unknown; after: unknown; }
|
||||
export interface RuleHistoryEvent {
|
||||
source: "observed" | "audit" | "revision"; at: string; kind: string; origin: "opnexus" | "external";
|
||||
actor: string | null; changes: RuleHistoryChange[]; status?: string; message?: string | null; note?: string;
|
||||
audit_id: string | null; managed_rule_id?: string;
|
||||
}
|
||||
export interface RuleHistory { host_id: string; rule_uuid: string; watching_since: string | null; events: RuleHistoryEvent[]; }
|
||||
@@ -0,0 +1,151 @@
|
||||
import type { AnalysisConfidence, CleanupCandidate, CleanupItem, CleanupOperation, CleanupOperationStatus } from "@/lib/api";
|
||||
|
||||
/** Version of the fixed acknowledgement text (cleanup.ack.text); the backend stores it with the audit entry. */
|
||||
export const ACK_TEXT_VERSION = "1";
|
||||
/** Same threshold as the backend: below a 30-day window "unused" is at most "probable" (monthly and quarterly jobs). */
|
||||
export const MIN_SURE_UNUSED_DAYS = 30;
|
||||
/** An operation in one of these states blocks everything else on the host/cluster until it is resolved. */
|
||||
export const OPEN_STATUSES: CleanupOperationStatus[] = ["running", "awaiting_confirm", "partial", "apply_failed", "reverting", "revert_failed"];
|
||||
/** States with an armed automatic revert (countdown). */
|
||||
export const COUNTDOWN_STATUSES: CleanupOperationStatus[] = ["awaiting_confirm", "partial", "apply_failed"];
|
||||
/** Hidden from the history: never executed, nothing happened. */
|
||||
const NOISE_STATUSES: CleanupOperationStatus[] = ["preview", "cancelled", "expired"];
|
||||
|
||||
export type BlockReason = "automatic" | "legacy" | "managed" | "confidence" | "nothing" | "deleteWait" | "unreadable";
|
||||
export type Selectability =
|
||||
| { mode: "disable"; level: AnalysisConfidence; needsAck: boolean }
|
||||
| { mode: "delete"; level: AnalysisConfidence; needsAck: false }
|
||||
| { mode: null; reason: BlockReason };
|
||||
|
||||
const RANK: Record<AnalysisConfidence, number> = { unknown: 0, probable: 1, sure: 2 };
|
||||
|
||||
/**
|
||||
* What the user may do with a candidate row. A client-side mirror of the backend rules (selectable_level) that only
|
||||
* decides how the row looks: the backend recomputes everything at preview and may still refuse.
|
||||
* - "unused" is at most "probable" for a block/reject rule and for a window shorter than 30 days,
|
||||
* - "unknown" is never selectable, "probable" needs the rule's own acknowledgement,
|
||||
* - an already disabled rule can only be deleted ("sure" disabled_long); everything else goes through "disable".
|
||||
*/
|
||||
export function selectability(candidate: CleanupCandidate, unusedDays: number): Selectability {
|
||||
if (!candidate.removable) return { mode: null, reason: candidate.not_removable_reason ?? "automatic" };
|
||||
if (!candidate.uuid) return { mode: null, reason: "unreadable" };
|
||||
if (!candidate.enabled) {
|
||||
const long = candidate.categories.find((c) => c.kind === "disabled_long");
|
||||
if (long?.confidence === "sure") return { mode: "delete", level: "sure", needsAck: false };
|
||||
return { mode: null, reason: long ? "deleteWait" : "nothing" };
|
||||
}
|
||||
let best: AnalysisConfidence | null = null;
|
||||
for (const category of candidate.categories) {
|
||||
if (category.kind === "disabled_long") continue;
|
||||
let level = category.confidence;
|
||||
if (category.kind === "unused" && (candidate.action === "block" || candidate.action === "reject" || unusedDays < MIN_SURE_UNUSED_DAYS)) {
|
||||
if (RANK[level] > RANK.probable) level = "probable";
|
||||
}
|
||||
if (best === null || RANK[level] > RANK[best]) best = level;
|
||||
}
|
||||
if (best === null) return { mode: null, reason: "nothing" };
|
||||
if (best === "unknown") return { mode: null, reason: "confidence" };
|
||||
return { mode: "disable", level: best, needsAck: best === "probable" };
|
||||
}
|
||||
|
||||
/** An ErrorCode name ("BEREINIGUNG_...", optionally followed by ": detail") is translated; the firewall's own text stays raw. */
|
||||
export function cleanupMessage(t: (key: string) => string, message: string | null | undefined): string {
|
||||
if (!message) return "";
|
||||
const match = /^([A-Z][A-Z0-9_]{3,})(?::\s*([\s\S]*))?$/.exec(message);
|
||||
if (match) {
|
||||
const key = `errors.${match[1]}`;
|
||||
const text = t(key);
|
||||
if (text !== key) return match[2] ? `${text} ${match[2]}` : text;
|
||||
}
|
||||
return message;
|
||||
}
|
||||
|
||||
/** The operation that blocks the host right now, if any (the newest one). */
|
||||
export function isOpen(op: CleanupOperation): boolean {
|
||||
// A delete/restore that ended with an unclear item is still unresolved (and blocks the host) until it is read again.
|
||||
return OPEN_STATUSES.includes(op.status) || (op.can_retry && !NOISE_STATUSES.includes(op.status));
|
||||
}
|
||||
|
||||
export function activeOperation(operations: CleanupOperation[]): CleanupOperation | null {
|
||||
return operations.find(isOpen) ?? null;
|
||||
}
|
||||
|
||||
export function historyOperations(operations: CleanupOperation[]): CleanupOperation[] {
|
||||
return operations.filter((op) => !NOISE_STATUSES.includes(op.status) && !isOpen(op));
|
||||
}
|
||||
|
||||
/** Rule identities (the UUID it has now and the one it had) for which a newer operation exists, per node. */
|
||||
export function latestItemIds(operations: CleanupOperation[], hostId: string): Set<string> {
|
||||
const seen = new Set<string>();
|
||||
const latest = new Set<string>();
|
||||
for (const op of operations) {
|
||||
if (NOISE_STATUSES.includes(op.status)) continue;
|
||||
for (const item of op.items) {
|
||||
if (item.host_id !== hostId) continue;
|
||||
const keys = [item.rule_uuid, item.restored_uuid].filter((k): k is string => !!k).map((k) => k.toLowerCase());
|
||||
if (keys.some((k) => seen.has(k))) continue;
|
||||
latest.add(item.id);
|
||||
}
|
||||
for (const item of op.items) {
|
||||
if (item.host_id !== hostId) continue;
|
||||
for (const key of [item.rule_uuid, item.restored_uuid]) if (key) seen.add(key.toLowerCase());
|
||||
}
|
||||
}
|
||||
return latest;
|
||||
}
|
||||
|
||||
/** The UUID under which a rule can be switched back on or deleted now (a restored rule has a new one). */
|
||||
export function currentUuid(item: CleanupItem, operationKind: string): string {
|
||||
return operationKind === "restore" && item.restored_uuid ? item.restored_uuid : item.rule_uuid;
|
||||
}
|
||||
|
||||
export function secondsUntil(deadline: string | null | undefined, now: number): number {
|
||||
if (!deadline) return 0;
|
||||
return Math.max(0, Math.ceil((new Date(deadline).getTime() - now) / 1000));
|
||||
}
|
||||
|
||||
export function formatClock(seconds: number): string {
|
||||
const m = Math.floor(seconds / 60);
|
||||
const s = seconds % 60;
|
||||
return `${m}:${String(s).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
/** Client-side check of the management sources, as strict as the backend (IP or CIDR, no names, not "everything"). */
|
||||
/** Extra management ports: TCP ports 1-65535, separated by spaces/commas, at most `max`; sorted, no duplicates. */
|
||||
export function parsePorts(text: string, max = 20): { ports: number[]; error: "invalid" | "tooMany" | null; bad?: string } {
|
||||
const ports: number[] = [];
|
||||
for (const raw of text.split(/[\s,;]+/).map((s) => s.trim()).filter(Boolean)) {
|
||||
if (!/^\d{1,5}$/.test(raw) || Number(raw) < 1 || Number(raw) > 65535) return { ports, error: "invalid", bad: raw };
|
||||
if (!ports.includes(Number(raw))) ports.push(Number(raw));
|
||||
}
|
||||
ports.sort((a, b) => a - b);
|
||||
if (ports.length > max) return { ports, error: "tooMany" };
|
||||
return { ports, error: null };
|
||||
}
|
||||
|
||||
export function parseSources(text: string, max = 20): { entries: string[]; error: "empty" | "invalid" | "everything" | "tooMany" | null; bad?: string } {
|
||||
const entries: string[] = [];
|
||||
for (const raw of text.split(/[\s,;]+/).map((s) => s.trim()).filter(Boolean)) {
|
||||
const [address, prefix, ...rest] = raw.split("/");
|
||||
if (rest.length > 0) return { entries, error: "invalid", bad: raw };
|
||||
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(address);
|
||||
let max_prefix = 0;
|
||||
if (v4) {
|
||||
if (v4.slice(1).some((o) => Number(o) > 255 || (o.length > 1 && o.startsWith("0")))) return { entries, error: "invalid", bad: raw };
|
||||
max_prefix = 32;
|
||||
} else if (address.includes(":") && /^[0-9a-fA-F:.]+$/.test(address)) {
|
||||
try { new URL(`http://[${address}]/`); } catch { return { entries, error: "invalid", bad: raw }; }
|
||||
max_prefix = 128;
|
||||
} else {
|
||||
return { entries, error: "invalid", bad: raw };
|
||||
}
|
||||
if (prefix !== undefined) {
|
||||
if (!/^\d{1,3}$/.test(prefix) || Number(prefix) > max_prefix) return { entries, error: "invalid", bad: raw };
|
||||
if (Number(prefix) === 0) return { entries, error: "everything", bad: raw };
|
||||
}
|
||||
if (!entries.includes(raw)) entries.push(raw);
|
||||
}
|
||||
if (entries.length === 0) return { entries, error: "empty" };
|
||||
if (entries.length > max) return { entries, error: "tooMany" };
|
||||
return { entries, error: null };
|
||||
}
|
||||
+471
-2
@@ -778,6 +778,7 @@
|
||||
"OPEN_DEPLOYMENT_DRAFT_UNCHANGED": "Offene Bestätigung oder fehlgeschlagenen Rollback zuerst klären; Entwurf bleibt unverändert",
|
||||
"OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY": "Offene Host-Wartung zuerst prüfen und abschließen, bevor Regeln ausgerollt werden.",
|
||||
"OPEN_RESTORE_BLOCKS_DEPLOY": "Eine Wiederherstellung ist für diesen Host offen. Bitte zuerst bestätigen oder abbrechen.",
|
||||
"OPEN_CLEANUP_BLOCKS_DEPLOY": "Für diesen Host läuft oder wartet eine Regel-Bereinigung. Bitte zuerst bestätigen, zurücknehmen oder abschließen.",
|
||||
"READ_ONLY_ACCESS": "Nur-Lese-Zugriff: Diese Aktion erfordert die Rolle „Admin“.",
|
||||
"CROSS_ORIGIN_BLOCKED": "Anfrage von einer fremden Herkunft blockiert",
|
||||
"CROSS_SITE_REQUEST_BLOCKED": "Anfrage aus einer anderen Website blockiert (Sec-Fetch-Site). Öffne OPNexus direkt über seine eigene Adresse.",
|
||||
@@ -868,7 +869,30 @@
|
||||
"API_RIGHTS_MISSING": "Dem API-Benutzer fehlt das Recht, den Gateway-Status zu lesen",
|
||||
"GATEWAY_NAME_INVALID": "Ungültiger Gateway-Name",
|
||||
"GATEWAY_UNKNOWN": "Dieses Gateway kennt OPNexus auf der Firewall nicht (Name prüfen, Groß-/Kleinschreibung)",
|
||||
"GATEWAY_MUTE_LIMIT": "Es können höchstens {limit} Gateways stummgeschaltet werden"
|
||||
"GATEWAY_MUTE_LIMIT": "Es können höchstens {limit} Gateways stummgeschaltet werden",
|
||||
"BEREINIGUNG_NICHT_GEFUNDEN": "Bereinigung nicht gefunden.",
|
||||
"BEREINIGUNG_AUSWAHL_UNGUELTIG": "Ungültige Auswahl ({field}).",
|
||||
"BEREINIGUNG_VORSCHAU_VERALTET": "Der Stand der Firewall hat sich seit der Vorschau geändert ({uuids}). Es wurde nichts geschrieben; Vorschau neu öffnen.",
|
||||
"BEREINIGUNG_REGEL_NICHT_AENDERBAR": "Regel {uuid} auf {host} kann nicht bereinigt werden (Grund: {reason}).",
|
||||
"BEREINIGUNG_OFFENE_OPERATION": "Im Host/Cluster ist noch etwas offen ({reason}). Erst bestätigen, zurücknehmen oder abschließen.",
|
||||
"BEREINIGUNG_ZU_VIELE_REGELN": "Höchstens {max} Regeln je Bereinigung (gewählt: {count}).",
|
||||
"BEREINIGUNG_BESTAETIGUNG_FEHLT": "Für diese Regeln fehlt die ausdrückliche Bestätigung: {uuids}.",
|
||||
"BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN": "Zuerst in den Einstellungen die Verwaltungsquellen eintragen (Adressen, von denen aus die Firewalls verwaltet werden). Ohne sie ist die Bereinigung gesperrt.",
|
||||
"BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG": "Cluster nicht vollständig oder Rollen unklar ({reason}). Die Bereinigung braucht alle Knoten, genau einen MASTER und dieselbe Regel-UUID auf jedem Knoten.",
|
||||
"BEREINIGUNG_FRIST_ABGELAUFEN": "Die Bestätigungsfrist ist abgelaufen oder die Bereinigung ist nicht mehr offen.",
|
||||
"BEREINIGUNG_NICHT_BESTAETIGBAR": "Diese Bereinigung kann nicht bestätigt werden ({status}): mindestens eine Regel ist fehlgeschlagen oder unklar. Jetzt zurücknehmen oder auf die automatische Rücknahme warten.",
|
||||
"BEREINIGUNG_LIVESTAND_ABWEICHEND": "Auf der Firewall sehen diese Regeln anders aus als erwartet: {uuids}. Nicht bestätigt; prüfen oder zurücknehmen.",
|
||||
"BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR": "{host} ist nicht erreichbar oder nicht lesbar. Nicht bestätigen, wenn die Firewall gerade nicht antwortet: die automatische Rücknahme stellt den alten Stand her.",
|
||||
"BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT": "Regel {uuid} auf {host} ist noch nicht lange genug deaktiviert (noch {days} Tag(e)).",
|
||||
"BEREINIGUNG_SNAPSHOT_FEHLT": "Für diese Regel gibt es keine Sicherung des Regelinhalts; sie wird nicht gelöscht bzw. kann nicht wiederhergestellt werden.",
|
||||
"BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR": "Die Regel wurde wiederhergestellt, aber nicht an ihrer alten Position: sie steht am Ende ihrer Gruppe. Position in OPNsense prüfen.",
|
||||
"BEREINIGUNG_AKTION_NICHT_MOEGLICH": "Diese Aktion ist im aktuellen Zustand nicht möglich ({status}).",
|
||||
"BEREINIGUNG_NICHT_AUSGEFUEHRT": "Nicht ausgeführt: die Bereinigung wurde vorher abgebrochen.",
|
||||
"BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN": "Die Änderung wurde gespeichert, aber das Anwenden auf der Firewall ist fehlgeschlagen.",
|
||||
"BEREINIGUNG_AUSGANG_UNKLAR": "Die Firewall hat nicht eindeutig geantwortet; der Stand wurde nachgelesen.",
|
||||
"BEREINIGUNG_ABGEBROCHEN_VERWAIST": "Der Vorgang wurde unterbrochen (OPNexus neu gestartet?). Stand auf der Firewall mit „Erneut prüfen“ nachlesen.",
|
||||
"BEREINIGUNG_EINSTELLUNG_UNGUELTIG": "Ungültige Einstellung der Bereinigung ({field}: {reason}).",
|
||||
"BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET": "Der Bestätigungstext dieser Seite ist veraltet. Seite neu laden und die Regeln erneut bestätigen."
|
||||
},
|
||||
"workflow": {
|
||||
"attention": "Fehler prüfen",
|
||||
@@ -2024,7 +2048,8 @@
|
||||
"collecting": "Daten werden gesammelt",
|
||||
"disabled": "deaktiviert"
|
||||
}
|
||||
}
|
||||
},
|
||||
"cleanupLink": "Kandidaten in der Bereinigung ansehen"
|
||||
},
|
||||
"tokens": {
|
||||
"title": "API-Tokens",
|
||||
@@ -2159,6 +2184,7 @@
|
||||
"audit": {
|
||||
"kind": {
|
||||
"rule": "Regeln",
|
||||
"rule_cleanup": "Regel-Bereinigung",
|
||||
"alias": "Aliase",
|
||||
"port_forward": "Port Forward",
|
||||
"one_to_one": "1:1 NAT",
|
||||
@@ -2178,6 +2204,11 @@
|
||||
},
|
||||
"action": {
|
||||
"deploy": "Push",
|
||||
"disable": "Deaktiviert",
|
||||
"enable": "Aktiviert",
|
||||
"revert": "Zurückgenommen",
|
||||
"restore": "Wiederhergestellt",
|
||||
"cleanup_settings": "Einstellungen geändert",
|
||||
"confirm": "Bestätigung",
|
||||
"rollback": "Rollback",
|
||||
"remove": "Ziel entfernen",
|
||||
@@ -3581,6 +3612,10 @@
|
||||
"FIRST_PACKET_hint": "Bewertet wird das erste Paket einer neuen Verbindung. Antworten und bestehende Verbindungen folgen dem Zustand, nicht den Regeln.",
|
||||
"SRC_PORT_EPHEMERAL": "Dynamischer Quellport",
|
||||
"SRC_PORT_EPHEMERAL_hint": "Ohne Quellport wird ein beliebiger Port aus dem dynamischen Bereich angenommen."
|
||||
},
|
||||
"tabs": {
|
||||
"query": "Abfrage",
|
||||
"cleanup": "Bereinigung"
|
||||
}
|
||||
},
|
||||
"guiAccess": {
|
||||
@@ -3627,5 +3662,439 @@
|
||||
},
|
||||
"more": "Weitere fehlende Punkte: Einstellungen › Firewall-Oberfläche über OPNexus."
|
||||
}
|
||||
},
|
||||
"cleanup": {
|
||||
"loadFailed": "Die Bereinigungsliste konnte nicht geladen werden.",
|
||||
"noHosts": "Keine aktivierte Firewall vorhanden.",
|
||||
"form": {
|
||||
"title": "Bereinigung: Firewall und Schwelle",
|
||||
"host": "Firewall",
|
||||
"threshold": "Schwelle",
|
||||
"thresholdHint": "So lange darf eine Regel ohne Treffer sein (bzw. deaktiviert unverändert bleiben), bevor sie als Kandidat erscheint. OPNexus kann nur zurückschauen, seit es die Regel beobachtet.",
|
||||
"days": "{n} Tage",
|
||||
"refreshHint": "Liest Regeln, Interfaces und Aliase neu von der Firewall, statt den Zwischenspeicher (60 s) zu nutzen."
|
||||
},
|
||||
"rulesChecked": "{count} Regeln geprüft",
|
||||
"watching": "OPNexus zählt die Treffer dieser Firewall seit {when}. {collecting} Regel(n) werden noch beobachtet und können noch nicht als ungenutzt gelten.",
|
||||
"notWatching": "OPNexus hat noch keine Regelzähler dieser Firewall gesammelt. Ungenutzte Regeln erscheinen erst nach der Beobachtungszeit.",
|
||||
"notes": {
|
||||
"ORDER_GUESSED": "Reihenfolge geschätzt",
|
||||
"ORDER_GUESSED_hint": "Diese OPNsense-Version liefert keine Sortierung; die Reihenfolge wurde aus Gruppe und Folgenummer abgeleitet. Befunde zu verdeckten und doppelten Regeln gelten höchstens als vermutlich.",
|
||||
"ORDER_MISMATCH": "Reihenfolge weicht ab",
|
||||
"ORDER_MISMATCH_hint": "Die geladene pf-Reihenfolge passt nicht zur Sortierung der Regeln. Befunde zu verdeckten Regeln mit Vorsicht lesen.",
|
||||
"PF_RULESET_UNAVAILABLE": "Geladene Regeln unbekannt",
|
||||
"PF_RULESET_UNAVAILABLE_hint": "Der von pf geladene Regelsatz war nicht lesbar. Ob eine Regel überhaupt aktiv ist, steht damit nicht fest; keine Aussage ist \"sicher\".",
|
||||
"GROUPS_UNAVAILABLE": "Interface-Gruppen unbekannt",
|
||||
"GROUPS_UNAVAILABLE_hint": "Die Interface-Gruppen waren nicht lesbar; Regeln auf Gruppen können nicht verglichen werden.",
|
||||
"ALIASES_UNAVAILABLE": "Aliase unbekannt",
|
||||
"ALIASES_UNAVAILABLE_hint": "Die Aliase waren nicht lesbar; Regeln mit Aliasen können nicht verglichen werden.",
|
||||
"NAT_UNAVAILABLE": "NAT unbekannt",
|
||||
"NAT_UNAVAILABLE_hint": "Die Port-Weiterleitungen waren nicht lesbar (für die Bereinigung ohne Belang).",
|
||||
"CLUSTER_PEER_USAGE_UNKNOWN": "Nutzung im Cluster unvollständig",
|
||||
"CLUSTER_PEER_USAGE_UNKNOWN_hint": "Für mindestens eine Regel fehlen die Zählerdaten eines Cluster-Knotens. \"Ungenutzt\" ist dann höchstens \"wahrscheinlich\".",
|
||||
"UNUSED_BLOCK_RULE": "Sperrregel",
|
||||
"UNUSED_BLOCK_RULE_hint": "Eine Sperrregel ohne Treffer kann trotzdem schützen (seltene Angriffe). Darum gilt „ungenutzt“ hier höchstens als vermutlich.",
|
||||
"UNUSED_WINDOW_SHORT": "kurzes Fenster",
|
||||
"UNUSED_WINDOW_SHORT_hint": "Unter 30 Tagen Beobachtung fehlen Monats- und Quartalsjobs. „Ungenutzt“ gilt höchstens als vermutlich.",
|
||||
"EXTERNAL_CHANGE": "extern geändert",
|
||||
"EXTERNAL_CHANGE_hint": "Die Regel oder die verdeckende Regel wurde im Beobachtungszeitraum außerhalb von OPNexus geändert. Der Befund gilt höchstens als vermutlich.",
|
||||
"ENABLE_MAY_BLOCK_MANAGEMENT": "kann Verwaltung sperren",
|
||||
"ENABLE_MAY_BLOCK_MANAGEMENT_hint": "Eine Sperrregel, für die nicht bewiesen ist, dass sie den Verwaltungszugang nicht trifft. Einschalten nur mit ausdrücklicher Bestätigung."
|
||||
},
|
||||
"empty": {
|
||||
"title": "Keine Kandidaten",
|
||||
"description": "In dieser Firewall ist keine Regel verdeckt, doppelt, seit {days} Tagen ungenutzt oder so lange deaktiviert."
|
||||
},
|
||||
"tableCaption": "Bereinigungskandidaten von {host}",
|
||||
"col": {
|
||||
"rule": "Regel",
|
||||
"finding": "Befund",
|
||||
"lastHit": "Letzter Treffer",
|
||||
"removable": "Entfernbar",
|
||||
"action": "Aktion"
|
||||
},
|
||||
"history": "Verlauf der Regel {rule} anzeigen",
|
||||
"historyTitle": "Verlauf",
|
||||
"allInterfaces": "alle Interfaces",
|
||||
"disabledRule": "deaktiviert",
|
||||
"category": {
|
||||
"unused": "Ungenutzt",
|
||||
"unused_hint": "Seit der Schwelle hat OPNexus keinen einzigen Treffer dieser Regel beobachtet (pf-Zähler, alle 15 Minuten gelesen).",
|
||||
"shadowed": "Verdeckt",
|
||||
"shadowed_hint": "Eine frühere Quick-Regel trifft schon alles, was diese Regel trifft. Diese Regel entscheidet nie.",
|
||||
"duplicate": "Doppelt",
|
||||
"duplicate_hint": "Eine frühere Regel trifft genau dasselbe und hat dieselbe Aktion.",
|
||||
"disabled_long": "Lange deaktiviert",
|
||||
"disabled_long_hint": "Die Regel ist deaktiviert und OPNexus hat sie seit der Schwelle unverändert so gesehen."
|
||||
},
|
||||
"detail": {
|
||||
"unused": "kein Treffer in {days} Tagen",
|
||||
"disabled": "seit mindestens {days} Tagen deaktiviert",
|
||||
"disabledSince": "deaktiviert seit {when}",
|
||||
"by": "durch \"{rule}\"",
|
||||
"byAutomatic": "durch die automatische Regel \"{rule}\""
|
||||
},
|
||||
"confidence": {
|
||||
"label": "Sicherheit",
|
||||
"sure": "sicher",
|
||||
"probable": "vermutlich",
|
||||
"unknown": "unbekannt",
|
||||
"tip": {
|
||||
"sure": "Belegt: die Daten sind vollständig und aktuell, nichts davon beruht auf einer Annahme.",
|
||||
"probable": "Beruht auf einer Annahme (z. B. Inhalt von Aliasen, geschätzte Reihenfolge, Zeitplan der Regel oder fehlender pf-Regelsatz). Vor dem Entfernen prüfen.",
|
||||
"unknown": "Es gibt einen Hinweis, aber keinen Beleg (z. B. die Regel ist in pf nicht geladen oder die Zähler sind veraltet). Nicht darauf verlassen."
|
||||
}
|
||||
},
|
||||
"noHit": "keiner beobachtet",
|
||||
"noHitSince": "Seit {when} beobachtet OPNexus diese Regel und hat keinen Treffer gesehen.",
|
||||
"notRemovable": {
|
||||
"automatic": "automatisch",
|
||||
"automatic_hint": "Diese Regel erzeugt OPNsense selbst und sie lässt sich nicht einzeln entfernen.",
|
||||
"legacy": "Altformat",
|
||||
"legacy_hint": "Die Regel ist noch im alten Format. Zuerst in OPNsense in das neue Regelformat migrieren.",
|
||||
"managed": "von OPNexus verwaltet",
|
||||
"managed_hint": "OPNexus hat diese Regel ausgerollt. Sie wird über den normalen Ablauf der Regelseite entfernt, sonst legt OPNexus sie wieder an.",
|
||||
"managedLink": "Regel \"{rule}\" öffnen",
|
||||
"confidence": "unbekannt",
|
||||
"confidence_hint": "Es gibt nur einen Hinweis, aber keinen Beleg. Solche Regeln werden nie zum Entfernen angeboten.",
|
||||
"nothing": "kein Befund",
|
||||
"nothing_hint": "Für diese Regel liegt kein Befund vor, der ein Deaktivieren begründet.",
|
||||
"deleteWait": "noch nicht",
|
||||
"deleteWait_hint": "Gelöscht wird nur, was lange genug deaktiviert und unverändert war und als sicher gilt. Noch nicht erreicht.",
|
||||
"unreadable": "nicht lesbar",
|
||||
"unreadable_hint": "Die Regel hat keine UUID im neuen Regelformat und lässt sich nicht gezielt ändern."
|
||||
},
|
||||
"action": {
|
||||
"disable": "Deaktivieren",
|
||||
"enable": "Aktivieren",
|
||||
"delete": "Löschen"
|
||||
},
|
||||
"select": {
|
||||
"column": "Auswählen",
|
||||
"row": "Regel {rule} auswählen",
|
||||
"count": "{n} von {max} gewählt",
|
||||
"previewDisable": "Vorschau: Deaktivieren",
|
||||
"previewDelete": "Vorschau: Löschen",
|
||||
"clear": "Auswahl aufheben",
|
||||
"none": "Wählen Sie mindestens eine Regel aus.",
|
||||
"locked": "Gesperrt, bis Verwaltungsquellen eingetragen sind.",
|
||||
"busy": "Es ist noch eine Bereinigung offen: erst bestätigen oder zurücksetzen.",
|
||||
"otherKind": "Eine Bereinigung kann nicht Deaktivieren und Löschen mischen. Zuerst die andere Auswahl aufheben.",
|
||||
"limit": "Höchstens {max} Regeln je Bereinigung (Einstellung).",
|
||||
"needsAck": "Bestätigung nötig",
|
||||
"needsAckHint": "Der Befund beruht auf einer Annahme. Vor der Vorschau bestätigen Sie für diese Regel ausdrücklich, dass sie nicht gebraucht wird."
|
||||
},
|
||||
"settings": {
|
||||
"link": "Einstellungen",
|
||||
"title": "Regel-Bereinigung",
|
||||
"intro": "Grenzen und Schutz für das Deaktivieren, Löschen und Wiederherstellen von Regeln im Bereich Analyse, Bereinigung.",
|
||||
"sources": "Verwaltungsquellen",
|
||||
"sourcesHint": "Adressen oder Netze, von denen aus die Firewalls verwaltet werden (OPNexus selbst, Ihr Arbeitsplatz, ein Jump-Host). Eine Regel, die den Zugang von dort zur Firewall treffen könnte, wird nie deaktiviert. Ohne Eintrag ist die Bereinigung gesperrt. Nur IP-Adressen oder Netze (CIDR), keine Namen.",
|
||||
"sourcesPlaceholder": "203.0.113.10\n198.51.100.0/24",
|
||||
"sourcesInvalid": "„{value}“ ist keine IP-Adresse und kein Netz (z. B. 203.0.113.10 oder 198.51.100.0/24). Hostnamen sind nicht erlaubt.",
|
||||
"sourcesEverything": "„{value}“ umfasst den gesamten Adressraum und würde jede Regel schützen.",
|
||||
"sourcesTooMany": "Höchstens 20 Einträge.",
|
||||
"maxRules": "Regeln je Bereinigung",
|
||||
"maxRulesHint": "So viele Regeln darf eine einzelne Bereinigung höchstens ändern (1 bis 25).",
|
||||
"maxRulesInvalid": "Eine ganze Zahl von 1 bis 25.",
|
||||
"minDays": "Wartezeit bis zum Löschen (Tage)",
|
||||
"minDaysHint": "So lange muss eine bestätigt deaktivierte Regel unverändert bleiben, bevor sie gelöscht werden darf (1 bis 365).",
|
||||
"minDaysInvalid": "Eine ganze Zahl von 1 bis 365.",
|
||||
"saved": "Einstellungen der Bereinigung gespeichert.",
|
||||
"ports": "Weitere Verwaltungsports",
|
||||
"portsHint": "TCP-Ports, auf denen die Firewalls zusätzlich verwaltet werden (z. B. SSH oder Weboberfläche auf einem anderen Port). API-Port, 443, 80 und 22 sind immer geschützt.",
|
||||
"portsPlaceholder": "z. B. 2222, 8443",
|
||||
"portsInvalid": "Kein gültiger Port: {value} (1–65535).",
|
||||
"portsTooMany": "Höchstens 20 Ports."
|
||||
},
|
||||
"mgmt": {
|
||||
"title": "Bereinigung gesperrt: Verwaltungsquellen fehlen",
|
||||
"text": "Tragen Sie die Adressen ein, von denen aus Ihre Firewalls verwaltet werden. Danach schaltet die Bereinigung keine Regel ab, die diesen Zugang treffen könnte.",
|
||||
"hint": "OPNexus prüft für jede gewählte Regel, ob sie ein Paket von einer Verwaltungsquelle an die Verwaltungsadresse und den Port der Firewall treffen kann. Trifft sie oder ist das nicht sicher auszuschließen, bleibt sie unangetastet. Nur IP-Adressen oder Netze (CIDR).",
|
||||
"askAdmin": "Ein Administrator muss die Verwaltungsquellen in den Einstellungen eintragen."
|
||||
},
|
||||
"dur": {
|
||||
"minutes": "{n} Min.",
|
||||
"seconds": "{n} Sek."
|
||||
},
|
||||
"preview": {
|
||||
"title": {
|
||||
"disable": "Vorschau: Regeln deaktivieren",
|
||||
"enable": "Vorschau: Regeln wieder aktivieren",
|
||||
"delete": "Vorschau: Regeln löschen"
|
||||
},
|
||||
"intro": {
|
||||
"disable": "Die Regeln auf {host} werden nur deaktiviert, nicht gelöscht, und die Änderung wird angewendet.",
|
||||
"enable": "Die Regeln auf {host} werden wieder eingeschaltet und die Änderung wird angewendet.",
|
||||
"delete": "Die bereits deaktivierten Regeln auf {host} werden gelöscht. Ihr Inhalt wird vorher gesichert."
|
||||
},
|
||||
"introHint": {
|
||||
"disable": "Die Liste wurde gerade neu von der Firewall berechnet. Beim Ausrollen prüft OPNexus alles noch einmal und schreibt nichts, wenn sich etwas geändert hat.",
|
||||
"enable": "Das Einschalten einer Regel kann Verbindungen verändern oder sperren. Darum gibt es danach wieder ein Bestätigungsfenster. Die Liste wurde gerade neu von der Firewall berechnet.",
|
||||
"delete": "Eine deaktivierte Regel ist in der Firewall nicht geladen, das Löschen ändert daran nichts und braucht kein Bestätigungsfenster. Unter „Verlauf der Bereinigungen“ lässt sich die Regel wiederherstellen (deaktiviert, an alter Position, soweit möglich)."
|
||||
},
|
||||
"loading": "Berechne die Vorschau von der Firewall …",
|
||||
"failed": "Die Vorschau konnte nicht berechnet werden.",
|
||||
"compute": "Vorschau berechnen",
|
||||
"again": "Vorschau neu berechnen",
|
||||
"validFor": "Vorschau noch {time} gültig",
|
||||
"expired": "Die Vorschau ist abgelaufen. Bitte neu berechnen.",
|
||||
"clusterOrder": "Cluster: zuerst Backup, dann Master. Zurücksetzen in umgekehrter Reihenfolge.",
|
||||
"basisConfirmed": "Seit {when} bestätigt deaktiviert und unverändert.",
|
||||
"basisDisabled": "Von Hand deaktiviert und lange unverändert.",
|
||||
"external": "extern geändert",
|
||||
"externalHint": "Seit der Beobachtung wurde die Regel außerhalb von OPNexus geändert.",
|
||||
"acknowledged": "ausdrücklich bestätigt",
|
||||
"autoRevert": "Danach haben Sie {window} Zeit zu prüfen, ob alles funktioniert und die Firewall erreichbar ist. Bestätigen Sie in dieser Zeit nicht, setzt OPNexus die Änderung automatisch zurück.",
|
||||
"confirmName": "Name der Firewall zur Bestätigung",
|
||||
"confirmNameHint": "Geben Sie den Namen der Firewall genau so ein. Das verhindert, dass versehentlich die falsche Firewall geändert wird.",
|
||||
"confirmNameAria": "Name der Firewall zur Bestätigung eingeben: {host}",
|
||||
"deployDisabledName": "Der Name muss genau dem der Firewall entsprechen.",
|
||||
"deployNotAdmin": "Nur Administratoren dürfen ausrollen. Eine Vorschau gehört dem, der sie erstellt hat: ein Administrator muss selbst eine neue Vorschau öffnen."
|
||||
},
|
||||
"ack": {
|
||||
"title": "Diese Befunde beruhen auf einer Annahme. Bestätigen Sie jede Regel einzeln.",
|
||||
"text": "Nicht belegt: Zeitplan, Aliasinhalt oder Beobachtungszeitraum sind geschätzt. Ich habe geprüft, dass die Regel „{rule}“ nicht gebraucht wird.",
|
||||
"enableTitle": "Das Einschalten dieser Sperrregel kann den Verwaltungszugang treffen. Bestätigen Sie jede Regel einzeln.",
|
||||
"enableText": "Ich habe geprüft, dass das Einschalten von „{rule}“ den Zugang von OPNexus und der Verwaltung zur Firewall nicht sperrt."
|
||||
},
|
||||
"protect": {
|
||||
"false": "Verwaltungszugang unberührt",
|
||||
"false_hint": "Keine Verwaltungsquelle ({sources}) kann mit dieser Regel auf die Verwaltung der Firewall zugreifen (geprüft: {queries} Kombinationen aus Quelle, Ziel, Port und Interface).",
|
||||
"true": "trifft den Verwaltungszugang",
|
||||
"true_hint": "Die Regel kann den Zugang von einer Verwaltungsquelle ({sources}) zur Firewall treffen. Das Bestätigungsfenster ist hier die Absicherung.",
|
||||
"unknown": "Verwaltungszugang unklar",
|
||||
"unknown_hint": "Ob die Regel den Zugang von einer Verwaltungsquelle ({sources}) betrifft, ließ sich nicht sicher ausschließen. Das Bestätigungsfenster ist hier die Absicherung.",
|
||||
"sourcesHidden": "die eingetragenen Verwaltungsquellen"
|
||||
},
|
||||
"deploy": {
|
||||
"disable": "Deaktivieren und anwenden",
|
||||
"enable": "Aktivieren und anwenden",
|
||||
"delete": "Endgültig löschen",
|
||||
"failed": "Das Ausrollen ist fehlgeschlagen."
|
||||
},
|
||||
"toast": {
|
||||
"deployedConfirm": "Angewendet. Prüfen Sie jetzt die Firewall und bestätigen Sie, sonst wird zurückgesetzt.",
|
||||
"deployedDone": "Erledigt.",
|
||||
"deployedCheck": "Nicht vollständig: Details im Kasten oben.",
|
||||
"confirm": {
|
||||
"confirmed": "Bestätigt. Die Änderung bleibt bestehen.",
|
||||
"done": "Bestätigt."
|
||||
},
|
||||
"revert": {
|
||||
"reverted": "Zurückgesetzt. Die Regeln sind wieder wie vorher.",
|
||||
"revert_failed": "Das Zurücksetzen ist fehlgeschlagen. Stand auf der Firewall prüfen.",
|
||||
"done": "Zurückgesetzt."
|
||||
},
|
||||
"retry": {
|
||||
"reverted": "Zurückgesetzt. Die Regeln sind wieder wie vorher.",
|
||||
"revert_failed": "Das Zurücksetzen ist wieder fehlgeschlagen.",
|
||||
"done": "Stand erneut geprüft.",
|
||||
"failed": "Stand erneut geprüft, nicht alles ist abgeschlossen."
|
||||
},
|
||||
"ended": {
|
||||
"reverted": "Die Bereinigung wurde automatisch zurückgesetzt.",
|
||||
"confirmed": "Die Bereinigung wurde bestätigt.",
|
||||
"done": "Die Bereinigung ist abgeschlossen.",
|
||||
"failed": "Die Bereinigung ist beendet, aber nicht alles ist gelungen."
|
||||
}
|
||||
},
|
||||
"confirmFailed": "Das Bestätigen ist fehlgeschlagen.",
|
||||
"revertFailed": "Das Zurücksetzen ist fehlgeschlagen.",
|
||||
"retryFailed": "Die erneute Prüfung ist fehlgeschlagen.",
|
||||
"op": {
|
||||
"title": {
|
||||
"disable": "Regeln deaktivieren",
|
||||
"enable": "Regeln wieder aktivieren",
|
||||
"delete": "Regeln löschen",
|
||||
"restore": "Regel wiederherstellen"
|
||||
},
|
||||
"state": {
|
||||
"running": "OPNexus schreibt die Änderung gerade auf die Firewall. Die automatische Rücknahme ist bereits scharf.",
|
||||
"awaiting_confirm": "Die Änderung ist auf der Firewall angewendet. Prüfen Sie, ob alles funktioniert und die Firewall erreichbar ist, und bestätigen Sie dann.",
|
||||
"partial": "Nicht alle Regeln konnten geändert werden. Bestätigen ist gesperrt: jetzt zurücksetzen oder auf die automatische Rücknahme warten.",
|
||||
"apply_failed": "Die Regeln sind geändert, aber das Anwenden auf der Firewall ist fehlgeschlagen. Jetzt zurücksetzen oder auf die automatische Rücknahme warten.",
|
||||
"reverting": "OPNexus setzt die Änderung gerade zurück.",
|
||||
"revert_failed": "Das Zurücksetzen ist fehlgeschlagen. Der Stand auf der Firewall ist unklar: in OPNsense prüfen und erneut versuchen.",
|
||||
"confirmed": "Bestätigt.",
|
||||
"reverted": "Zurückgesetzt.",
|
||||
"done": "Abgeschlossen.",
|
||||
"failed": "Bei mindestens einer Regel ist der Ausgang unklar. Mit „Erneut prüfen“ liest OPNexus den Stand der Firewall nach.",
|
||||
"preview": "Vorschau.",
|
||||
"cancelled": "Abgebrochen.",
|
||||
"expired": "Abgelaufen."
|
||||
},
|
||||
"countdown": "Noch {time} bis zur automatischen Rücknahme",
|
||||
"countdownMinutes": "Noch etwa {n} Min. bis zur automatischen Rücknahme.",
|
||||
"confirm": "Bestätigen",
|
||||
"confirmNotYet": "Bestätigen ist erst möglich, wenn die Änderung angewendet ist.",
|
||||
"confirmPartial": "Bestätigen ist gesperrt, weil mindestens eine Regel fehlgeschlagen oder unklar ist.",
|
||||
"confirmApplyFailed": "Bestätigen ist gesperrt, weil das Anwenden auf der Firewall fehlgeschlagen ist.",
|
||||
"revert": "Jetzt zurücksetzen",
|
||||
"revertNeedsAdmin": "Nur Administratoren dürfen zurücksetzen (Einstellung: Operatoren dürfen ausrollen).",
|
||||
"retryRevert": "Zurücksetzen wiederholen",
|
||||
"retryCheck": "Erneut prüfen",
|
||||
"retryNeedsAdmin": "Nur Administratoren dürfen wiederholen.",
|
||||
"autoRevertNote": "Bestätigen heißt: Sie erreichen die Firewall noch. Bestätigen Sie nicht, setzt OPNexus nach Ablauf der Zeit automatisch zurück. Bestätigen ist nur in der Browser-Sitzung möglich, nicht per API-Token."
|
||||
},
|
||||
"status": {
|
||||
"preview": "Vorschau",
|
||||
"preview_hint": "Berechnet, noch nichts geschrieben.",
|
||||
"running": "Läuft",
|
||||
"running_hint": "OPNexus schreibt gerade auf die Firewall.",
|
||||
"awaiting_confirm": "Wartet auf Bestätigung",
|
||||
"awaiting_confirm_hint": "Angewendet. Ohne Bestätigung setzt OPNexus nach Fristablauf automatisch zurück.",
|
||||
"confirmed": "Bestätigt",
|
||||
"confirmed_hint": "Vom Nutzer bestätigt. Die Änderung bleibt.",
|
||||
"partial": "Teilweise",
|
||||
"partial_hint": "Nicht alle Regeln konnten geändert werden. Zurücksetzen oder die automatische Rücknahme abwarten.",
|
||||
"apply_failed": "Anwenden fehlgeschlagen",
|
||||
"apply_failed_hint": "Die Regeln sind geändert, aber die Firewall hat die Änderung nicht angewendet.",
|
||||
"reverting": "Wird zurückgesetzt",
|
||||
"reverting_hint": "OPNexus stellt den alten Stand her.",
|
||||
"reverted": "Zurückgesetzt",
|
||||
"reverted_hint": "Der alte Stand ist wiederhergestellt. Das Beobachtungsfenster der Regeln beginnt neu.",
|
||||
"revert_failed": "Zurücksetzen fehlgeschlagen",
|
||||
"revert_failed_hint": "Der Stand auf der Firewall ist unklar. Prüfen und erneut versuchen.",
|
||||
"done": "Abgeschlossen",
|
||||
"done_hint": "Vollständig ausgeführt.",
|
||||
"failed": "Fehlgeschlagen",
|
||||
"failed_hint": "Nicht alles wurde ausgeführt. Details bei den Regeln.",
|
||||
"cancelled": "Abgebrochen",
|
||||
"cancelled_hint": "Die Vorschau wurde nicht ausgerollt.",
|
||||
"expired": "Abgelaufen",
|
||||
"expired_hint": "Die Vorschau war nicht mehr gültig."
|
||||
},
|
||||
"state": {
|
||||
"queued": "wartet",
|
||||
"disabling": "wird deaktiviert",
|
||||
"disabled": "deaktiviert",
|
||||
"confirmed": "bestätigt",
|
||||
"reverting": "wird zurückgesetzt",
|
||||
"reverted": "zurückgesetzt",
|
||||
"deleting": "wird gelöscht",
|
||||
"deleted": "gelöscht",
|
||||
"restoring": "wird wiederhergestellt",
|
||||
"restored": "wiederhergestellt (deaktiviert)",
|
||||
"failed": "fehlgeschlagen",
|
||||
"unknown": "Ausgang unklar",
|
||||
"revert_failed": "Zurücksetzen fehlgeschlagen",
|
||||
"skipped": "übersprungen"
|
||||
},
|
||||
"stateEnable": {
|
||||
"disabling": "wird eingeschaltet",
|
||||
"disabled": "eingeschaltet"
|
||||
},
|
||||
"hist": {
|
||||
"title": "Verlauf der Bereinigungen",
|
||||
"empty": "Auf dieser Firewall wurde noch nichts bereinigt.",
|
||||
"restore": "Wiederherstellen …",
|
||||
"restoreHint": "Legt die gelöschte Regel aus der Sicherung neu an, deaktiviert und möglichst an der alten Position.",
|
||||
"adminOnly": "Nur Administratoren dürfen wiederherstellen.",
|
||||
"delete": "Löschen …",
|
||||
"deleteHint": "Öffnet die Vorschau zum Löschen dieser deaktivierten Regel.",
|
||||
"deleteWait": "Löschen ist erst ab dem {when} möglich (Wartezeit nach der Bestätigung, einstellbar).",
|
||||
"enable": "Wieder aktivieren …",
|
||||
"enableHint": "Öffnet die Vorschau zum Einschalten der Regel, mit Bestätigungsfenster.",
|
||||
"restoredBadge": "wiederhergestellt"
|
||||
},
|
||||
"restore": {
|
||||
"title": "Regel wiederherstellen",
|
||||
"intro": "Die gelöschte Regel wird aus der Sicherung neu angelegt, deaktiviert und möglichst an ihrer alten Position.",
|
||||
"introHint": "Die Regel bekommt eine neue UUID. Im Cluster wird sie auf allen Knoten wiederhergestellt. Einschalten ist ein eigener Schritt („Wieder aktivieren“) mit Bestätigungsfenster. Sind Aliase oder Kategorien der Regel inzwischen weg, lehnt die Firewall ab und die Meldung steht beim Ergebnis.",
|
||||
"action": "Wiederherstellen",
|
||||
"done": "Regel „{rule}“ wiederhergestellt (deaktiviert).",
|
||||
"failed": "Das Wiederherstellen ist fehlgeschlagen.",
|
||||
"next": "Zum Einschalten: unter „Verlauf der Bereinigungen“ „Wieder aktivieren“ wählen."
|
||||
},
|
||||
"refuse": {
|
||||
"managed": "von OPNexus verwaltet",
|
||||
"automatic": "automatische Regel",
|
||||
"legacy": "Altformat",
|
||||
"management": "könnte den Verwaltungszugang treffen",
|
||||
"infrastructure": "Cluster-/Infrastrukturregel",
|
||||
"confidence": "Befund nicht belegt",
|
||||
"not_candidate": "kein Kandidat mehr"
|
||||
},
|
||||
"openReason": {
|
||||
"deployment": "Regel-Ausrollen offen",
|
||||
"cleanup": "andere Bereinigung offen",
|
||||
"rule": "Regel wird schon bearbeitet"
|
||||
},
|
||||
"clusterReason": {
|
||||
"peer_disabled": "ein Knoten ist deaktiviert",
|
||||
"peer_missing": "ein Knoten fehlt",
|
||||
"roles": "Rollen unklar",
|
||||
"uuid_missing": "Regel fehlt auf einem Knoten"
|
||||
},
|
||||
"settingsReason": {
|
||||
"not_an_address": "keine IP-Adresse oder Netz",
|
||||
"everything": "gesamter Adressraum",
|
||||
"too_many": "zu viele Einträge"
|
||||
}
|
||||
},
|
||||
"ruleHistory": {
|
||||
"title": "Verlauf: {rule}",
|
||||
"host": "Firewall",
|
||||
"loadFailed": "Der Verlauf konnte nicht geladen werden.",
|
||||
"list": "Ereignisse",
|
||||
"action": "Verlauf",
|
||||
"open": "Verlauf der Regel {rule} anzeigen",
|
||||
"by": "von {actor}",
|
||||
"since": "OPNexus beobachtet diese Firewall seit {when}.",
|
||||
"empty": {
|
||||
"title": "Keine Änderungen beobachtet",
|
||||
"since": "Seit {when} hat sich an dieser Regel nichts geändert.",
|
||||
"notYet": "OPNexus hat diese Firewall noch nicht beobachtet; die erste Beobachtung läuft kurz nach dem Start."
|
||||
},
|
||||
"origin": {
|
||||
"opnexus": "über OPNexus",
|
||||
"opnexus_hint": "Die Änderung passt zu einem Ausrollen oder Zurücksetzen durch OPNexus (Audit-Eintrag in diesem Zeitraum).",
|
||||
"external": "extern",
|
||||
"external_hint": "Kein OPNexus-Vorgang im Zeitraum: geändert in der Oberfläche der Firewall, per Skript oder durch einen anderen Zugang."
|
||||
},
|
||||
"kind": {
|
||||
"added": "angelegt",
|
||||
"changed": "geändert",
|
||||
"removed": "entfernt",
|
||||
"enabled": "aktiviert",
|
||||
"disabled": "deaktiviert",
|
||||
"revision": "in OPNexus gespeichert",
|
||||
"moved": "verschoben"
|
||||
},
|
||||
"audit": {
|
||||
"deploy": "ausgerollt",
|
||||
"confirm": "bestätigt",
|
||||
"rollback": "zurückgesetzt",
|
||||
"retry-rollback": "Zurücksetzen wiederholt",
|
||||
"state_change": "Zustand geändert"
|
||||
},
|
||||
"field": {
|
||||
"description": "Beschreibung",
|
||||
"enabled": "aktiv",
|
||||
"action": "Aktion",
|
||||
"interface": "Interface",
|
||||
"interfacenot": "Interface invertiert",
|
||||
"direction": "Richtung",
|
||||
"ipprotocol": "IP-Version",
|
||||
"protocol": "Protokoll",
|
||||
"source_net": "Quelle",
|
||||
"source_not": "Quelle invertiert",
|
||||
"source_port": "Quellport",
|
||||
"destination_net": "Ziel",
|
||||
"destination_not": "Ziel invertiert",
|
||||
"destination_port": "Zielport",
|
||||
"quick": "Quick",
|
||||
"gateway": "Gateway",
|
||||
"sequence": "Reihenfolge",
|
||||
"sort_order": "Sortierung",
|
||||
"log": "Protokollieren",
|
||||
"sched": "Zeitplan",
|
||||
"statetype": "Statusart",
|
||||
"tag": "Markierung",
|
||||
"tagged": "Markiert mit",
|
||||
"source_ports": "Quellport",
|
||||
"prio_group": "Gruppe/Priorität"
|
||||
}
|
||||
}
|
||||
}
|
||||
+471
-2
@@ -778,6 +778,7 @@
|
||||
"OPEN_DEPLOYMENT_DRAFT_UNCHANGED": "Resolve the open confirmation or failed rollback first; the draft remains unchanged",
|
||||
"OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY": "Check and complete the open host maintenance before deploying rules.",
|
||||
"OPEN_RESTORE_BLOCKS_DEPLOY": "A restore is open for this host. Please confirm or cancel it first.",
|
||||
"OPEN_CLEANUP_BLOCKS_DEPLOY": "A rule cleanup is running or waiting for this host. Please confirm, revert or finish it first.",
|
||||
"READ_ONLY_ACCESS": "Read-only access: this action requires the “Admin” role.",
|
||||
"CROSS_ORIGIN_BLOCKED": "Request from a foreign origin blocked",
|
||||
"CROSS_SITE_REQUEST_BLOCKED": "Request from another site blocked (Sec-Fetch-Site). Open OPNexus directly at its own address.",
|
||||
@@ -868,7 +869,30 @@
|
||||
"API_RIGHTS_MISSING": "The API user lacks the right to read the gateway status",
|
||||
"GATEWAY_NAME_INVALID": "Invalid gateway name",
|
||||
"GATEWAY_UNKNOWN": "OPNexus does not know this gateway on the firewall (check the name and its case)",
|
||||
"GATEWAY_MUTE_LIMIT": "At most {limit} gateways can be muted"
|
||||
"GATEWAY_MUTE_LIMIT": "At most {limit} gateways can be muted",
|
||||
"BEREINIGUNG_NICHT_GEFUNDEN": "Cleanup not found.",
|
||||
"BEREINIGUNG_AUSWAHL_UNGUELTIG": "Invalid selection ({field}).",
|
||||
"BEREINIGUNG_VORSCHAU_VERALTET": "The firewall changed since the preview ({uuids}). Nothing was written; open the preview again.",
|
||||
"BEREINIGUNG_REGEL_NICHT_AENDERBAR": "Rule {uuid} on {host} cannot be cleaned up (reason: {reason}).",
|
||||
"BEREINIGUNG_OFFENE_OPERATION": "Something is still open on the host/cluster ({reason}). Confirm, revert or finish it first.",
|
||||
"BEREINIGUNG_ZU_VIELE_REGELN": "At most {max} rules per cleanup (selected: {count}).",
|
||||
"BEREINIGUNG_BESTAETIGUNG_FEHLT": "These rules need an explicit acknowledgement: {uuids}.",
|
||||
"BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN": "First enter the management sources in the settings (addresses the firewalls are managed from). Without them the cleanup is locked.",
|
||||
"BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG": "Cluster incomplete or roles unclear ({reason}). The cleanup needs every node, exactly one MASTER and the same rule UUID on every node.",
|
||||
"BEREINIGUNG_FRIST_ABGELAUFEN": "The confirmation window has passed or the cleanup is no longer open.",
|
||||
"BEREINIGUNG_NICHT_BESTAETIGBAR": "This cleanup cannot be confirmed ({status}): at least one rule failed or is unclear. Revert now or wait for the automatic revert.",
|
||||
"BEREINIGUNG_LIVESTAND_ABWEICHEND": "These rules look different on the firewall than expected: {uuids}. Not confirmed; check or revert.",
|
||||
"BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR": "{host} cannot be reached or read. Do not confirm while the firewall does not answer: the automatic revert restores the previous state.",
|
||||
"BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT": "Rule {uuid} on {host} has not been switched off long enough yet ({days} more day(s)).",
|
||||
"BEREINIGUNG_SNAPSHOT_FEHLT": "There is no backup of this rule's content; it is not deleted or cannot be restored.",
|
||||
"BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR": "The rule was restored, but not at its old position: it is at the end of its group. Check its position in OPNsense.",
|
||||
"BEREINIGUNG_AKTION_NICHT_MOEGLICH": "This action is not possible in the current state ({status}).",
|
||||
"BEREINIGUNG_NICHT_AUSGEFUEHRT": "Not executed: the cleanup stopped earlier.",
|
||||
"BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN": "The change was saved, but applying it on the firewall failed.",
|
||||
"BEREINIGUNG_AUSGANG_UNKLAR": "The firewall did not answer clearly; the state was read back.",
|
||||
"BEREINIGUNG_ABGEBROCHEN_VERWAIST": "The run was interrupted (OPNexus restarted?). Read the firewall's state with “Check again”.",
|
||||
"BEREINIGUNG_EINSTELLUNG_UNGUELTIG": "Invalid cleanup setting ({field}: {reason}).",
|
||||
"BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET": "The acknowledgement text of this page is outdated. Reload the page and acknowledge the rules again."
|
||||
},
|
||||
"workflow": {
|
||||
"attention": "Check errors",
|
||||
@@ -2024,7 +2048,8 @@
|
||||
"collecting": "collecting data",
|
||||
"disabled": "disabled"
|
||||
}
|
||||
}
|
||||
},
|
||||
"cleanupLink": "Show the candidates in cleanup"
|
||||
},
|
||||
"tokens": {
|
||||
"title": "API tokens",
|
||||
@@ -2159,6 +2184,7 @@
|
||||
"audit": {
|
||||
"kind": {
|
||||
"rule": "Rules",
|
||||
"rule_cleanup": "Rule cleanup",
|
||||
"alias": "Aliases",
|
||||
"port_forward": "Port forward",
|
||||
"one_to_one": "1:1 NAT",
|
||||
@@ -2178,6 +2204,11 @@
|
||||
},
|
||||
"action": {
|
||||
"deploy": "Push",
|
||||
"disable": "Switched off",
|
||||
"enable": "Switched on",
|
||||
"revert": "Reverted",
|
||||
"restore": "Restored",
|
||||
"cleanup_settings": "Settings changed",
|
||||
"confirm": "Confirmation",
|
||||
"rollback": "Rollback",
|
||||
"remove": "Remove target",
|
||||
@@ -3581,6 +3612,10 @@
|
||||
"FIRST_PACKET_hint": "The first packet of a new connection is evaluated. Replies and established connections follow state, not rules.",
|
||||
"SRC_PORT_EPHEMERAL": "Ephemeral source port",
|
||||
"SRC_PORT_EPHEMERAL_hint": "Without a source port, any port from the ephemeral range is assumed."
|
||||
},
|
||||
"tabs": {
|
||||
"query": "Query",
|
||||
"cleanup": "Cleanup"
|
||||
}
|
||||
},
|
||||
"guiAccess": {
|
||||
@@ -3627,5 +3662,439 @@
|
||||
},
|
||||
"more": "More missing items: Settings › Firewall interface through OPNexus."
|
||||
}
|
||||
},
|
||||
"cleanup": {
|
||||
"loadFailed": "The cleanup list could not be loaded.",
|
||||
"noHosts": "No enabled firewall.",
|
||||
"form": {
|
||||
"title": "Cleanup: firewall and threshold",
|
||||
"host": "Firewall",
|
||||
"threshold": "Threshold",
|
||||
"thresholdHint": "How long a rule may go without a hit (or stay disabled and unchanged) before it shows up as a candidate. OPNexus can only look back as far as it has been watching the rule.",
|
||||
"days": "{n} days",
|
||||
"refreshHint": "Re-reads rules, interfaces and aliases from the firewall instead of using the cache (60 s)."
|
||||
},
|
||||
"rulesChecked": "{count} rules checked",
|
||||
"watching": "OPNexus has been counting this firewall's hits since {when}. {collecting} rule(s) are still being watched and cannot count as unused yet.",
|
||||
"notWatching": "OPNexus has not collected rule counters from this firewall yet. Unused rules only show up after the observation period.",
|
||||
"notes": {
|
||||
"ORDER_GUESSED": "Order guessed",
|
||||
"ORDER_GUESSED_hint": "This OPNsense version delivers no sort order; the order was derived from group and sequence number. Findings about shadowed and duplicate rules are at most probable.",
|
||||
"ORDER_MISMATCH": "Order differs",
|
||||
"ORDER_MISMATCH_hint": "The loaded pf order does not match the rules' sort order. Read findings about shadowed rules with care.",
|
||||
"PF_RULESET_UNAVAILABLE": "Loaded rules unknown",
|
||||
"PF_RULESET_UNAVAILABLE_hint": "The rule set pf has loaded could not be read, so it is not certain whether a rule is active at all. Nothing is \"sure\".",
|
||||
"GROUPS_UNAVAILABLE": "Interface groups unknown",
|
||||
"GROUPS_UNAVAILABLE_hint": "The interface groups could not be read; rules on groups cannot be compared.",
|
||||
"ALIASES_UNAVAILABLE": "Aliases unknown",
|
||||
"ALIASES_UNAVAILABLE_hint": "The aliases could not be read; rules using aliases cannot be compared.",
|
||||
"NAT_UNAVAILABLE": "NAT unknown",
|
||||
"NAT_UNAVAILABLE_hint": "The port forwards could not be read (irrelevant for cleanup).",
|
||||
"CLUSTER_PEER_USAGE_UNKNOWN": "Cluster usage incomplete",
|
||||
"CLUSTER_PEER_USAGE_UNKNOWN_hint": "For at least one rule a cluster node has no counter data. \"Unused\" is then \"probable\" at best.",
|
||||
"UNUSED_BLOCK_RULE": "block rule",
|
||||
"UNUSED_BLOCK_RULE_hint": "A block rule without hits can still protect (rare attacks). So “unused” counts as probable at most here.",
|
||||
"UNUSED_WINDOW_SHORT": "short window",
|
||||
"UNUSED_WINDOW_SHORT_hint": "Under 30 days of observation, monthly and quarterly jobs are missing. “Unused” counts as probable at most.",
|
||||
"EXTERNAL_CHANGE": "changed externally",
|
||||
"EXTERNAL_CHANGE_hint": "The rule or the rule that shadows it was changed outside OPNexus during the observation period. The finding counts as probable at most.",
|
||||
"ENABLE_MAY_BLOCK_MANAGEMENT": "may block management",
|
||||
"ENABLE_MAY_BLOCK_MANAGEMENT_hint": "A block rule that is not proven to leave the management access alone. Switching it on needs an explicit acknowledgement."
|
||||
},
|
||||
"empty": {
|
||||
"title": "No candidates",
|
||||
"description": "No rule on this firewall is shadowed, duplicate, unused for {days} days or disabled for that long."
|
||||
},
|
||||
"tableCaption": "Cleanup candidates of {host}",
|
||||
"col": {
|
||||
"rule": "Rule",
|
||||
"finding": "Finding",
|
||||
"lastHit": "Last hit",
|
||||
"removable": "Removable",
|
||||
"action": "Action"
|
||||
},
|
||||
"history": "Show the history of rule {rule}",
|
||||
"historyTitle": "History",
|
||||
"allInterfaces": "all interfaces",
|
||||
"disabledRule": "disabled",
|
||||
"category": {
|
||||
"unused": "Unused",
|
||||
"unused_hint": "OPNexus has not observed a single hit of this rule within the threshold (pf counters, read every 15 minutes).",
|
||||
"shadowed": "Shadowed",
|
||||
"shadowed_hint": "An earlier rule with \"quick\" already matches everything this rule matches. This rule never decides.",
|
||||
"duplicate": "Duplicate",
|
||||
"duplicate_hint": "An earlier rule matches exactly the same traffic and has the same action.",
|
||||
"disabled_long": "Long disabled",
|
||||
"disabled_long_hint": "The rule is disabled and OPNexus has seen it unchanged like that for the whole threshold."
|
||||
},
|
||||
"detail": {
|
||||
"unused": "no hit in {days} days",
|
||||
"disabled": "disabled for at least {days} days",
|
||||
"disabledSince": "disabled since {when}",
|
||||
"by": "by \"{rule}\"",
|
||||
"byAutomatic": "by the automatic rule \"{rule}\""
|
||||
},
|
||||
"confidence": {
|
||||
"label": "Confidence",
|
||||
"sure": "sure",
|
||||
"probable": "probable",
|
||||
"unknown": "unknown",
|
||||
"tip": {
|
||||
"sure": "Proven: the data is complete and current, none of it rests on an assumption.",
|
||||
"probable": "Rests on an assumption (e.g. alias content, guessed order, the rule's schedule or a missing pf rule set). Check before removing.",
|
||||
"unknown": "There is a hint but no proof (e.g. the rule is not loaded in pf or the counters are stale). Do not rely on it."
|
||||
}
|
||||
},
|
||||
"noHit": "none observed",
|
||||
"noHitSince": "OPNexus has been watching this rule since {when} and has seen no hit.",
|
||||
"notRemovable": {
|
||||
"automatic": "automatic",
|
||||
"automatic_hint": "OPNsense generates this rule itself and it cannot be removed on its own.",
|
||||
"legacy": "legacy format",
|
||||
"legacy_hint": "The rule is still in the old format. Migrate it to the new rule format in OPNsense first.",
|
||||
"managed": "managed by OPNexus",
|
||||
"managed_hint": "OPNexus deployed this rule. It is removed through the normal flow of the rules page, otherwise OPNexus would create it again.",
|
||||
"managedLink": "Open rule \"{rule}\"",
|
||||
"confidence": "unknown",
|
||||
"confidence_hint": "There is a hint but no proof. Such rules are never offered for removal.",
|
||||
"nothing": "no finding",
|
||||
"nothing_hint": "There is no finding for this rule that would justify disabling it.",
|
||||
"deleteWait": "not yet",
|
||||
"deleteWait_hint": "Only rules that have been disabled long enough, unchanged, and count as sure are deleted. Not reached yet.",
|
||||
"unreadable": "unreadable",
|
||||
"unreadable_hint": "The rule has no UUID in the new rule format and cannot be changed individually."
|
||||
},
|
||||
"action": {
|
||||
"disable": "Disable",
|
||||
"enable": "Enable",
|
||||
"delete": "Delete"
|
||||
},
|
||||
"select": {
|
||||
"column": "Select",
|
||||
"row": "Select rule {rule}",
|
||||
"count": "{n} of {max} selected",
|
||||
"previewDisable": "Preview: disable",
|
||||
"previewDelete": "Preview: delete",
|
||||
"clear": "Clear selection",
|
||||
"none": "Select at least one rule.",
|
||||
"locked": "Locked until management sources are set.",
|
||||
"busy": "Another cleanup is still open: confirm or revert it first.",
|
||||
"otherKind": "One cleanup cannot mix disabling and deleting. Clear the other selection first.",
|
||||
"limit": "At most {max} rules per cleanup (setting).",
|
||||
"needsAck": "Acknowledgement needed",
|
||||
"needsAckHint": "The finding rests on an assumption. Before the preview you confirm for this rule explicitly that it is not needed."
|
||||
},
|
||||
"settings": {
|
||||
"link": "Settings",
|
||||
"title": "Rule cleanup",
|
||||
"intro": "Limits and protection for disabling, deleting and restoring rules under Analysis, Cleanup.",
|
||||
"sources": "Management sources",
|
||||
"sourcesHint": "Addresses or networks the firewalls are managed from (OPNexus itself, your workstation, a jump host). A rule that could match access from there to the firewall is never disabled. Without an entry the cleanup is locked. IP addresses or networks (CIDR) only, no names.",
|
||||
"sourcesPlaceholder": "203.0.113.10\n198.51.100.0/24",
|
||||
"sourcesInvalid": "“{value}” is not an IP address or network (e.g. 203.0.113.10 or 198.51.100.0/24). Host names are not allowed.",
|
||||
"sourcesEverything": "“{value}” covers the whole address space and would protect every rule.",
|
||||
"sourcesTooMany": "At most 20 entries.",
|
||||
"maxRules": "Rules per cleanup",
|
||||
"maxRulesHint": "The most rules a single cleanup may change (1 to 25).",
|
||||
"maxRulesInvalid": "A whole number from 1 to 25.",
|
||||
"minDays": "Waiting time before deleting (days)",
|
||||
"minDaysHint": "How long a rule that was confirmed as disabled has to stay unchanged before it may be deleted (1 to 365).",
|
||||
"minDaysInvalid": "A whole number from 1 to 365.",
|
||||
"saved": "Cleanup settings saved.",
|
||||
"ports": "Additional management ports",
|
||||
"portsHint": "TCP ports the firewalls are additionally managed on (e.g. SSH or the web GUI on another port). The API port, 443, 80 and 22 are always protected.",
|
||||
"portsPlaceholder": "e.g. 2222, 8443",
|
||||
"portsInvalid": "Not a valid port: {value} (1–65535).",
|
||||
"portsTooMany": "At most 20 ports."
|
||||
},
|
||||
"mgmt": {
|
||||
"title": "Cleanup locked: management sources missing",
|
||||
"text": "Enter the addresses your firewalls are managed from. After that the cleanup never switches off a rule that could match this access.",
|
||||
"hint": "For every selected rule OPNexus checks whether it can match a packet from a management source to the firewall's management address and port. If it matches, or this cannot be ruled out, the rule is left alone. IP addresses or networks (CIDR) only.",
|
||||
"askAdmin": "An administrator has to enter the management sources in the settings."
|
||||
},
|
||||
"dur": {
|
||||
"minutes": "{n} min",
|
||||
"seconds": "{n} s"
|
||||
},
|
||||
"preview": {
|
||||
"title": {
|
||||
"disable": "Preview: disable rules",
|
||||
"enable": "Preview: enable rules again",
|
||||
"delete": "Preview: delete rules"
|
||||
},
|
||||
"intro": {
|
||||
"disable": "The rules on {host} are only disabled, not deleted, and the change is applied.",
|
||||
"enable": "The rules on {host} are switched on again and the change is applied.",
|
||||
"delete": "The already disabled rules on {host} are deleted. Their content is backed up first."
|
||||
},
|
||||
"introHint": {
|
||||
"disable": "The list was just recomputed from the firewall. When rolling out, OPNexus checks everything again and writes nothing if something changed.",
|
||||
"enable": "Switching a rule on can change or block connections. That is why there is a confirmation window again. The list was just recomputed from the firewall.",
|
||||
"delete": "A disabled rule is not loaded in the firewall, so deleting it changes nothing there and needs no confirmation window. Under “Cleanup history” the rule can be restored (disabled, at its old position where possible)."
|
||||
},
|
||||
"loading": "Computing the preview from the firewall …",
|
||||
"failed": "The preview could not be computed.",
|
||||
"compute": "Compute preview",
|
||||
"again": "Recompute preview",
|
||||
"validFor": "Preview valid for {time}",
|
||||
"expired": "The preview has expired. Please recompute it.",
|
||||
"clusterOrder": "Cluster: backup first, then master. Reverting runs in reverse order.",
|
||||
"basisConfirmed": "Disabled and confirmed since {when}, unchanged.",
|
||||
"basisDisabled": "Disabled by hand and unchanged for a long time.",
|
||||
"external": "changed externally",
|
||||
"externalHint": "The rule was changed outside OPNexus since it has been observed.",
|
||||
"acknowledged": "explicitly acknowledged",
|
||||
"autoRevert": "Afterwards you have {window} to check that everything works and the firewall is reachable. If you do not confirm in that time, OPNexus reverts the change automatically.",
|
||||
"confirmName": "Firewall name to confirm",
|
||||
"confirmNameHint": "Type the firewall's name exactly. This prevents changing the wrong firewall by accident.",
|
||||
"confirmNameAria": "Type the firewall name to confirm: {host}",
|
||||
"deployDisabledName": "The name has to match the firewall's name exactly.",
|
||||
"deployNotAdmin": "Only administrators may roll out. A preview belongs to whoever created it: an administrator has to open a new preview themselves."
|
||||
},
|
||||
"ack": {
|
||||
"title": "These findings rest on an assumption. Acknowledge each rule on its own.",
|
||||
"text": "Not proven: schedule, alias content or observation period are estimated. I have checked that the rule “{rule}” is not needed.",
|
||||
"enableTitle": "Switching these block rules on may hit the management access. Acknowledge each rule.",
|
||||
"enableText": "I checked that switching on “{rule}” does not block OPNexus or the admins from reaching the firewall."
|
||||
},
|
||||
"protect": {
|
||||
"false": "Management access untouched",
|
||||
"false_hint": "No management source ({sources}) can use this rule to reach the firewall's management (checked: {queries} combinations of source, destination, port and interface).",
|
||||
"true": "matches management access",
|
||||
"true_hint": "The rule can match access from a management source ({sources}) to the firewall. The confirmation window is the safeguard here.",
|
||||
"unknown": "Management access unclear",
|
||||
"unknown_hint": "It could not be ruled out that the rule affects access from a management source ({sources}). The confirmation window is the safeguard here.",
|
||||
"sourcesHidden": "the configured management sources"
|
||||
},
|
||||
"deploy": {
|
||||
"disable": "Disable and apply",
|
||||
"enable": "Enable and apply",
|
||||
"delete": "Delete permanently",
|
||||
"failed": "The roll-out failed."
|
||||
},
|
||||
"toast": {
|
||||
"deployedConfirm": "Applied. Check the firewall now and confirm, otherwise it is reverted.",
|
||||
"deployedDone": "Done.",
|
||||
"deployedCheck": "Not complete: details in the box above.",
|
||||
"confirm": {
|
||||
"confirmed": "Confirmed. The change stays.",
|
||||
"done": "Confirmed."
|
||||
},
|
||||
"revert": {
|
||||
"reverted": "Reverted. The rules are as they were.",
|
||||
"revert_failed": "Reverting failed. Check the firewall's state.",
|
||||
"done": "Reverted."
|
||||
},
|
||||
"retry": {
|
||||
"reverted": "Reverted. The rules are as they were.",
|
||||
"revert_failed": "Reverting failed again.",
|
||||
"done": "State checked again.",
|
||||
"failed": "State checked again, not everything is complete."
|
||||
},
|
||||
"ended": {
|
||||
"reverted": "The cleanup was reverted automatically.",
|
||||
"confirmed": "The cleanup was confirmed.",
|
||||
"done": "The cleanup is complete.",
|
||||
"failed": "The cleanup has ended, but not everything worked."
|
||||
}
|
||||
},
|
||||
"confirmFailed": "Confirming failed.",
|
||||
"revertFailed": "Reverting failed.",
|
||||
"retryFailed": "Checking again failed.",
|
||||
"op": {
|
||||
"title": {
|
||||
"disable": "Disable rules",
|
||||
"enable": "Enable rules again",
|
||||
"delete": "Delete rules",
|
||||
"restore": "Restore rule"
|
||||
},
|
||||
"state": {
|
||||
"running": "OPNexus is writing the change to the firewall. The automatic revert is already armed.",
|
||||
"awaiting_confirm": "The change is applied on the firewall. Check that everything works and the firewall is reachable, then confirm.",
|
||||
"partial": "Not all rules could be changed. Confirming is locked: revert now or wait for the automatic revert.",
|
||||
"apply_failed": "The rules are changed, but applying on the firewall failed. Revert now or wait for the automatic revert.",
|
||||
"reverting": "OPNexus is reverting the change.",
|
||||
"revert_failed": "Reverting failed. The firewall's state is unclear: check it in OPNsense and try again.",
|
||||
"confirmed": "Confirmed.",
|
||||
"reverted": "Reverted.",
|
||||
"done": "Complete.",
|
||||
"failed": "The outcome is unclear for at least one rule. “Check again” makes OPNexus read the firewall's state.",
|
||||
"preview": "Preview.",
|
||||
"cancelled": "Cancelled.",
|
||||
"expired": "Expired."
|
||||
},
|
||||
"countdown": "{time} left until the automatic revert",
|
||||
"countdownMinutes": "About {n} min left until the automatic revert.",
|
||||
"confirm": "Confirm",
|
||||
"confirmNotYet": "Confirming is possible once the change is applied.",
|
||||
"confirmPartial": "Confirming is locked because at least one rule failed or is unclear.",
|
||||
"confirmApplyFailed": "Confirming is locked because applying on the firewall failed.",
|
||||
"revert": "Revert now",
|
||||
"revertNeedsAdmin": "Only administrators may revert (setting: operators may roll out).",
|
||||
"retryRevert": "Retry reverting",
|
||||
"retryCheck": "Check again",
|
||||
"retryNeedsAdmin": "Only administrators may retry.",
|
||||
"autoRevertNote": "Confirming means: you can still reach the firewall. If you do not confirm, OPNexus reverts automatically when the time is up. Confirming works from the browser session only, not with an API token."
|
||||
},
|
||||
"status": {
|
||||
"preview": "Preview",
|
||||
"preview_hint": "Computed, nothing written yet.",
|
||||
"running": "Running",
|
||||
"running_hint": "OPNexus is writing to the firewall.",
|
||||
"awaiting_confirm": "Waiting for confirmation",
|
||||
"awaiting_confirm_hint": "Applied. Without confirmation OPNexus reverts automatically when the time is up.",
|
||||
"confirmed": "Confirmed",
|
||||
"confirmed_hint": "Confirmed by a user. The change stays.",
|
||||
"partial": "Partial",
|
||||
"partial_hint": "Not all rules could be changed. Revert or wait for the automatic revert.",
|
||||
"apply_failed": "Apply failed",
|
||||
"apply_failed_hint": "The rules are changed, but the firewall did not apply the change.",
|
||||
"reverting": "Reverting",
|
||||
"reverting_hint": "OPNexus is restoring the old state.",
|
||||
"reverted": "Reverted",
|
||||
"reverted_hint": "The old state is restored. The rules' observation window starts again.",
|
||||
"revert_failed": "Revert failed",
|
||||
"revert_failed_hint": "The firewall's state is unclear. Check it and try again.",
|
||||
"done": "Complete",
|
||||
"done_hint": "Fully carried out.",
|
||||
"failed": "Failed",
|
||||
"failed_hint": "Not everything was carried out. Details at the rules.",
|
||||
"cancelled": "Cancelled",
|
||||
"cancelled_hint": "The preview was not rolled out.",
|
||||
"expired": "Expired",
|
||||
"expired_hint": "The preview was no longer valid."
|
||||
},
|
||||
"state": {
|
||||
"queued": "waiting",
|
||||
"disabling": "being disabled",
|
||||
"disabled": "disabled",
|
||||
"confirmed": "confirmed",
|
||||
"reverting": "being reverted",
|
||||
"reverted": "reverted",
|
||||
"deleting": "being deleted",
|
||||
"deleted": "deleted",
|
||||
"restoring": "being restored",
|
||||
"restored": "restored (disabled)",
|
||||
"failed": "failed",
|
||||
"unknown": "outcome unclear",
|
||||
"revert_failed": "revert failed",
|
||||
"skipped": "skipped"
|
||||
},
|
||||
"stateEnable": {
|
||||
"disabling": "being enabled",
|
||||
"disabled": "enabled"
|
||||
},
|
||||
"hist": {
|
||||
"title": "Cleanup history",
|
||||
"empty": "Nothing has been cleaned up on this firewall yet.",
|
||||
"restore": "Restore …",
|
||||
"restoreHint": "Recreates the deleted rule from the backup, disabled and at its old position where possible.",
|
||||
"adminOnly": "Only administrators may restore.",
|
||||
"delete": "Delete …",
|
||||
"deleteHint": "Opens the preview for deleting this disabled rule.",
|
||||
"deleteWait": "Deleting is possible from {when} (waiting time after the confirmation, configurable).",
|
||||
"enable": "Enable again …",
|
||||
"enableHint": "Opens the preview for switching the rule on, with a confirmation window.",
|
||||
"restoredBadge": "restored"
|
||||
},
|
||||
"restore": {
|
||||
"title": "Restore rule",
|
||||
"intro": "The deleted rule is recreated from the backup, disabled and at its old position where possible.",
|
||||
"introHint": "The rule gets a new UUID. In a cluster it is restored on every node. Switching it on is a separate step (“Enable again”) with a confirmation window. If aliases or categories of the rule are gone meanwhile, the firewall refuses and its message appears in the result.",
|
||||
"action": "Restore",
|
||||
"done": "Rule “{rule}” restored (disabled).",
|
||||
"failed": "Restoring failed.",
|
||||
"next": "To switch it on: choose “Enable again” under “Cleanup history”."
|
||||
},
|
||||
"refuse": {
|
||||
"managed": "managed by OPNexus",
|
||||
"automatic": "automatic rule",
|
||||
"legacy": "legacy format",
|
||||
"management": "could match management access",
|
||||
"infrastructure": "cluster/infrastructure rule",
|
||||
"confidence": "finding not proven",
|
||||
"not_candidate": "no longer a candidate"
|
||||
},
|
||||
"openReason": {
|
||||
"deployment": "rule roll-out open",
|
||||
"cleanup": "another cleanup open",
|
||||
"rule": "rule already being handled"
|
||||
},
|
||||
"clusterReason": {
|
||||
"peer_disabled": "a node is disabled",
|
||||
"peer_missing": "a node is missing",
|
||||
"roles": "roles unclear",
|
||||
"uuid_missing": "rule missing on a node"
|
||||
},
|
||||
"settingsReason": {
|
||||
"not_an_address": "not an IP address or network",
|
||||
"everything": "whole address space",
|
||||
"too_many": "too many entries"
|
||||
}
|
||||
},
|
||||
"ruleHistory": {
|
||||
"title": "History: {rule}",
|
||||
"host": "Firewall",
|
||||
"loadFailed": "The history could not be loaded.",
|
||||
"list": "Events",
|
||||
"action": "History",
|
||||
"open": "Show the history of rule {rule}",
|
||||
"by": "by {actor}",
|
||||
"since": "OPNexus has been watching this firewall since {when}.",
|
||||
"empty": {
|
||||
"title": "No changes observed",
|
||||
"since": "This rule has not changed since {when}.",
|
||||
"notYet": "OPNexus has not observed this firewall yet; the first observation runs shortly after startup."
|
||||
},
|
||||
"origin": {
|
||||
"opnexus": "via OPNexus",
|
||||
"opnexus_hint": "The change matches a deployment or rollback by OPNexus (an audit entry in that period).",
|
||||
"external": "external",
|
||||
"external_hint": "No OPNexus operation in that period: changed in the firewall's own interface, by a script or through another access."
|
||||
},
|
||||
"kind": {
|
||||
"added": "created",
|
||||
"changed": "changed",
|
||||
"removed": "removed",
|
||||
"enabled": "enabled",
|
||||
"disabled": "disabled",
|
||||
"revision": "saved in OPNexus",
|
||||
"moved": "moved"
|
||||
},
|
||||
"audit": {
|
||||
"deploy": "deployed",
|
||||
"confirm": "confirmed",
|
||||
"rollback": "rolled back",
|
||||
"retry-rollback": "rollback retried",
|
||||
"state_change": "state changed"
|
||||
},
|
||||
"field": {
|
||||
"description": "Description",
|
||||
"enabled": "enabled",
|
||||
"action": "Action",
|
||||
"interface": "Interface",
|
||||
"interfacenot": "Interface inverted",
|
||||
"direction": "Direction",
|
||||
"ipprotocol": "IP version",
|
||||
"protocol": "Protocol",
|
||||
"source_net": "Source",
|
||||
"source_not": "Source inverted",
|
||||
"source_port": "Source port",
|
||||
"destination_net": "Destination",
|
||||
"destination_not": "Destination inverted",
|
||||
"destination_port": "Destination port",
|
||||
"quick": "Quick",
|
||||
"gateway": "Gateway",
|
||||
"sequence": "Sequence",
|
||||
"sort_order": "Sort order",
|
||||
"log": "Log",
|
||||
"sched": "Schedule",
|
||||
"statetype": "State type",
|
||||
"tag": "Tag",
|
||||
"tagged": "Tagged",
|
||||
"source_ports": "Source port",
|
||||
"prio_group": "Group/priority"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -67,6 +67,10 @@ const CODE_LIST_PARAMS: Record<string, { param: string; prefix: string }> = {
|
||||
GUI_GATEWAY_NOT_CONFIGURED: { param: "reasons", prefix: "guiAccess.reason" },
|
||||
NETWORK_CHANGE_BLOCKED: { param: "reasons", prefix: "network.blocks" },
|
||||
NETWORK_ACK_REQUIRED: { param: "reasons", prefix: "network.risks" },
|
||||
BEREINIGUNG_REGEL_NICHT_AENDERBAR: { param: "reason", prefix: "cleanup.refuse" },
|
||||
BEREINIGUNG_OFFENE_OPERATION: { param: "reason", prefix: "cleanup.openReason" },
|
||||
BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG: { param: "reason", prefix: "cleanup.clusterReason" },
|
||||
BEREINIGUNG_EINSTELLUNG_UNGUELTIG: { param: "reason", prefix: "cleanup.settingsReason" },
|
||||
};
|
||||
|
||||
/** `reasons: "name_exists, proto_set"` -> readable sentences joined by " · "; an unknown code stays as it is.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Manually bumped whenever a notable change ships — mirrors the dated
|
||||
// entries already kept in the project README/CHANGELOG.md, no build
|
||||
// tooling involved. Versioning scheme: see CHANGELOG.md ("Versionierung").
|
||||
export const APP_VERSION = "1.29.0";
|
||||
export const APP_CODENAME = "Caracara";
|
||||
export const APP_VERSION = "1.31.0";
|
||||
export const APP_CODENAME = "Vulture";
|
||||
export const APP_MAINTAINER = "Julian Boldt";
|
||||
@@ -275,6 +275,47 @@ preflight_proxy_config() {
|
||||
say "Proxy-Konfiguration geprüft."
|
||||
}
|
||||
|
||||
# B8 (1.30): Die CLI, die ein Update ausführt, ist die ALTE (im Updater-Image eingebaut bzw. im Bundle). Korrekturen an
|
||||
# Vorprüfungen und Ablauf aus dem Zielrelease griffen deshalb erst beim übernächsten Update. Jetzt holt sie sich die CLI
|
||||
# aus dem Updater-Image des Ziels und startet sich damit neu, aber nur unter diesen Bedingungen (Review: die CLI läuft
|
||||
# mit Zugriff auf den Docker-Socket, also wie root): cosign und cosign.pub sind da, das Image wurde in DIESEM Lauf
|
||||
# gezogen, seine Signatur wurde nach DIGEST geprüft (nicht nach Tag) und genau dieser Digest wird ausgeführt, und die
|
||||
# Ziel-CLI kennt selbst refresh_cli (so wird nie eine alte, bekannt fehlerhafte CLI geholt). Jeder andere Fehler heißt
|
||||
# nur: mit der vorhandenen CLI weitermachen. OPNEXUS_NO_CLI_REFRESH=1 schaltet es ab. Der Neustart-Schutz ist eine
|
||||
# Shell-Variable plus internes Argument, keine Umgebungsvariable, die ein Aufrufer vererben könnte.
|
||||
CLI_REFRESHED=0
|
||||
refresh_cli() {
|
||||
local target="$1" repo ref digest tmp keep=0
|
||||
[ "$CLI_REFRESHED" != 1 ] && [ "${OPNEXUS_NO_CLI_REFRESH:-}" != 1 ] || return 0
|
||||
repo=$(env_get OPNEXUS_IMAGE_REPO)
|
||||
[ -n "$repo" ] || return 0
|
||||
if ! command -v cosign >/dev/null 2>&1 || [ ! -f cosign.pub ]; then
|
||||
say "CLI-Aktualisierung übersprungen (ohne cosign und cosign.pub wird keine CLI aus einem Image ausgeführt)."; return 0
|
||||
fi
|
||||
ref="$repo/opnexus-updater:$target"
|
||||
if updater_enabled; then keep=1; fi
|
||||
if ! docker pull -q "$ref" >/dev/null 2>&1; then say "CLI-Aktualisierung übersprungen (Updater-Image des Ziels nicht ladbar)."; return 0; fi
|
||||
digest=$(docker image inspect --format '{{index .RepoDigests 0}}' "$ref" 2>/dev/null || true)
|
||||
if [[ "$digest" != *@sha256:* ]]; then
|
||||
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
|
||||
say "CLI-Aktualisierung übersprungen (Digest des Updater-Images unbekannt)."; return 0
|
||||
fi
|
||||
cosign verify --key cosign.pub --insecure-ignore-tlog=true "$digest" >/dev/null 2>&1 \
|
||||
|| die "Signatur des Updater-Images $ref ist ungültig oder fehlt — nichts wurde verändert."
|
||||
tmp=$(mktemp "${TMPDIR:-/tmp}/opnexus-cli.XXXXXX") || return 0
|
||||
if ! docker run --rm --pull never --entrypoint cat "$digest" /usr/local/bin/opnexus > "$tmp" 2>/dev/null \
|
||||
|| [ ! -s "$tmp" ] || ! bash -n "$tmp" 2>/dev/null || ! grep -q '^refresh_cli() {' "$tmp"; then
|
||||
rm -f "$tmp"; [ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
|
||||
say "CLI-Aktualisierung übersprungen (CLI des Zielreleases nicht lesbar oder zu alt) — nutze die vorhandene."; return 0
|
||||
fi
|
||||
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
|
||||
if cmp -s "$tmp" "$(readlink -f "$0")"; then rm -f "$tmp"; return 0; fi
|
||||
say "Die CLI des Zielreleases (Signatur geprüft) ist neuer — starte sie."
|
||||
chmod 0700 "$tmp"
|
||||
export OPNEXUS_DIR="$PWD"
|
||||
exec bash "$tmp" __refreshed update "$target"
|
||||
}
|
||||
|
||||
cmd_update() {
|
||||
require_release_mode
|
||||
local target="${1:-$(env_get OPNEXUS_VERSION)}"
|
||||
@@ -294,6 +335,7 @@ cmd_update() {
|
||||
if updater_enabled; then pull_services+=(updater); fi
|
||||
OPNEXUS_VERSION="$target" c pull "${pull_services[@]}" || die "Pull fehlgeschlagen — nichts wurde verändert."
|
||||
verify_signatures "$target" "${pull_services[@]}"
|
||||
refresh_cli "$target"
|
||||
preflight_proxy_config "$target"
|
||||
|
||||
started=$(date -u +%Y%m%dT%H%M%SZ)
|
||||
@@ -428,6 +470,12 @@ cmd_password() {
|
||||
c exec -T backend python -m app.reset_password ${user:+"$user"}
|
||||
}
|
||||
|
||||
# Interner Neustart durch refresh_cli: Marker setzen und die eigene Temp-Datei aufräumen (bash hat sie offen).
|
||||
if [ "${1:-}" = __refreshed ]; then
|
||||
CLI_REFRESHED=1; shift
|
||||
case "$0" in */opnexus-cli.*) rm -f "$0" ;; esac
|
||||
fi
|
||||
|
||||
case "${1:-}" in
|
||||
password) shift; cmd_password "$@" ;;
|
||||
status) if [ "${2:-}" = "--porcelain" ]; then cmd_status_porcelain; else cmd_status; fi ;;
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
"""refresh_cli in `opnexus` (B8): the CLI takes the newer copy from the target's updater image and restarts itself, but
|
||||
only with a cosign check of the image DIGEST that succeeded in this run; every other failure continues with the present
|
||||
CLI. Runs the real function with fake `docker` and `cosign` in front of PATH.
|
||||
Run: cd updater && python3 -m unittest test_cli_refresh"""
|
||||
import os
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT = Path(__file__).with_name("opnexus").resolve()
|
||||
DIGEST = "registry.example/opnexus/opnexus-updater@sha256:" + "a" * 64
|
||||
FAKE_DOCKER = """#!/bin/sh
|
||||
echo "$@" >> "$FAKE_LOG"
|
||||
case "$1" in
|
||||
pull) [ "${FAKE_PULL_RC:-0}" = 0 ] || exit 1 ;;
|
||||
image) [ -z "${FAKE_NO_DIGEST:-}" ] && echo "$FAKE_DIGEST" ;;
|
||||
run) [ "${FAKE_RC:-0}" = 0 ] || exit "$FAKE_RC"; cat "$FAKE_CLI" 2>/dev/null ;;
|
||||
esac
|
||||
"""
|
||||
FAKE_COSIGN = """#!/bin/sh
|
||||
echo "cosign $*" >> "$FAKE_LOG"
|
||||
[ "${FAKE_COSIGN_RC:-0}" = 0 ]
|
||||
"""
|
||||
NEW_CLI = '#!/usr/bin/env bash\\nrefresh_cli() {\\n :\\n}\\necho "NEW-CLI $*"\\n'.replace("\\n", "\n")
|
||||
|
||||
|
||||
def _exe(path: Path, text: str):
|
||||
path.write_text(text)
|
||||
path.chmod(path.stat().st_mode | stat.S_IXUSR)
|
||||
|
||||
|
||||
class RefreshCli(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
|
||||
(self.tmp / "bin").mkdir()
|
||||
_exe(self.tmp / "bin" / "docker", FAKE_DOCKER)
|
||||
_exe(self.tmp / "bin" / "cosign", FAKE_COSIGN)
|
||||
(self.tmp / ".env").write_text("OPNEXUS_IMAGE_REPO=registry.example/opnexus\n")
|
||||
(self.tmp / "cosign.pub").write_text("key")
|
||||
self.log = self.tmp / "calls.log"
|
||||
self.cli = self.tmp / "new-cli"
|
||||
|
||||
def run_refresh(self, new_cli=NEW_CLI, *, env_extra=None, env_file=None, pre="", no_cosign=False, no_pub=False):
|
||||
self.cli.write_text(new_cli)
|
||||
if env_file is not None:
|
||||
(self.tmp / ".env").write_text(env_file)
|
||||
if no_pub:
|
||||
(self.tmp / "cosign.pub").unlink()
|
||||
if no_cosign:
|
||||
(self.tmp / "bin" / "cosign").unlink()
|
||||
path = f"{self.tmp / 'bin'}:/usr/bin:/bin" if no_cosign else f"{self.tmp / 'bin'}:{os.environ['PATH']}" # a real cosign must not leak in
|
||||
env = {**os.environ, "PATH": path, "OPNEXUS_DIR": str(self.tmp),
|
||||
"FAKE_LOG": str(self.log), "FAKE_CLI": str(self.cli), "FAKE_DIGEST": DIGEST, "TMPDIR": str(self.tmp)}
|
||||
for name in ("OPNEXUS_NO_CLI_REFRESH", "CLI_REFRESHED"):
|
||||
env.pop(name, None)
|
||||
env.update(env_extra or {})
|
||||
body = f'source <(sed "/^case \\"\\${{1:-}}\\" in/,\\$d" "{SCRIPT}"); {pre} refresh_cli 9.9.9; echo CONTINUED'
|
||||
return subprocess.run(["bash", "-c", body, str(SCRIPT)], env=env, capture_output=True, text=True, timeout=30)
|
||||
|
||||
def calls(self):
|
||||
return self.log.read_text().splitlines() if self.log.exists() else []
|
||||
|
||||
def test_verified_newer_cli_runs_by_digest_with_the_internal_marker_argument(self):
|
||||
r = self.run_refresh()
|
||||
self.assertIn("NEW-CLI __refreshed update 9.9.9", r.stdout)
|
||||
self.assertNotIn("CONTINUED", r.stdout) # exec: the old script does not go on
|
||||
calls = self.calls()
|
||||
self.assertTrue(any(c.startswith("cosign verify") and c.endswith(DIGEST) for c in calls)) # signature of the DIGEST
|
||||
run = next(c for c in calls if c.startswith("run "))
|
||||
self.assertTrue(run.endswith(f"{DIGEST} /usr/local/bin/opnexus")) # that same digest is executed
|
||||
self.assertLess([c.split()[0] for c in calls].index("cosign"), [c.split()[0] for c in calls].index("run"))
|
||||
|
||||
def test_identical_cli_continues_and_removes_the_image_it_pulled(self):
|
||||
r = self.run_refresh(SCRIPT.read_text())
|
||||
self.assertIn("CONTINUED", r.stdout)
|
||||
self.assertTrue(any(c.startswith("rmi registry.example/opnexus/opnexus-updater:9.9.9") for c in self.calls()))
|
||||
|
||||
def test_image_stays_when_the_updater_profile_is_enabled(self):
|
||||
self.run_refresh(SCRIPT.read_text(), env_file="OPNEXUS_IMAGE_REPO=registry.example/opnexus\nCOMPOSE_PROFILES=updater\n")
|
||||
self.assertFalse(any(c.startswith("rmi") for c in self.calls()))
|
||||
|
||||
def test_nothing_runs_without_a_real_signature_check(self):
|
||||
for label, kwargs in {"no cosign": {"no_cosign": True}, "no cosign.pub": {"no_pub": True}}.items():
|
||||
with self.subTest(label):
|
||||
self.log.unlink(missing_ok=True)
|
||||
r = self.run_refresh(**kwargs)
|
||||
self.assertIn("CONTINUED", r.stdout)
|
||||
self.assertFalse(any(c.startswith(("pull", "run")) for c in self.calls()))
|
||||
|
||||
def test_invalid_signature_aborts_the_update_and_runs_nothing(self):
|
||||
r = self.run_refresh(env_extra={"FAKE_COSIGN_RC": "1"})
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertNotIn("CONTINUED", r.stdout)
|
||||
self.assertNotIn("NEW-CLI", r.stdout)
|
||||
self.assertFalse(any(c.startswith("run ") for c in self.calls()))
|
||||
|
||||
def test_pull_failure_never_falls_back_to_a_local_image(self):
|
||||
r = self.run_refresh(env_extra={"FAKE_PULL_RC": "1"})
|
||||
self.assertIn("CONTINUED", r.stdout)
|
||||
self.assertFalse(any(c.startswith(("run ", "cosign")) for c in self.calls()))
|
||||
|
||||
def test_unknown_digest_missing_marker_broken_or_empty_copies_are_skipped(self):
|
||||
for label, kwargs in {
|
||||
"no digest": {"env_extra": {"FAKE_NO_DIGEST": "1"}},
|
||||
"old CLI without refresh_cli": {"new_cli": "#!/usr/bin/env bash\necho OLD-CLI\n"},
|
||||
"syntax error": {"new_cli": "refresh_cli() {\nif then fi (\n"},
|
||||
"empty": {"new_cli": ""},
|
||||
"docker run fails": {"env_extra": {"FAKE_RC": "1"}},
|
||||
}.items():
|
||||
with self.subTest(label):
|
||||
r = self.run_refresh(**kwargs)
|
||||
self.assertIn("CONTINUED", r.stdout)
|
||||
self.assertNotIn("NEW-CLI", r.stdout)
|
||||
self.assertNotIn("OLD-CLI", r.stdout)
|
||||
|
||||
def test_opt_out_and_missing_repo_do_not_even_ask_docker(self):
|
||||
for label, kwargs in {"opt-out": {"env_extra": {"OPNEXUS_NO_CLI_REFRESH": "1"}},
|
||||
"no repo": {"env_file": "POSTGRES_PASSWORD=x\n"}}.items():
|
||||
with self.subTest(label):
|
||||
self.log.unlink(missing_ok=True)
|
||||
r = self.run_refresh(**kwargs)
|
||||
self.assertIn("CONTINUED", r.stdout)
|
||||
self.assertEqual(self.calls(), [])
|
||||
|
||||
def test_restart_guard_is_a_shell_variable_an_inherited_environment_value_cannot_switch_it_off(self):
|
||||
inherited = self.run_refresh(env_extra={"CLI_REFRESHED": "1"})
|
||||
self.assertIn("NEW-CLI __refreshed", inherited.stdout) # inheriting the name changes nothing
|
||||
self.log.unlink(missing_ok=True)
|
||||
again = self.run_refresh(pre="CLI_REFRESHED=1;") # the refreshed copy itself does not refresh again
|
||||
self.assertIn("CONTINUED", again.stdout)
|
||||
self.assertEqual(self.calls(), [])
|
||||
|
||||
def test_refreshed_copy_removes_its_own_temp_file(self):
|
||||
copy = self.tmp / "opnexus-cli.abc123"
|
||||
shutil.copy(SCRIPT, copy)
|
||||
env = {**os.environ, "OPNEXUS_DIR": str(self.tmp)}
|
||||
subprocess.run(["bash", str(copy), "__refreshed", "bogus"], env=env, capture_output=True, text=True, timeout=30)
|
||||
self.assertFalse(copy.exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user