Compare commits

...
8 Commits
Author SHA1 Message Date
Julian Boldt c662a58426 Release v1.15.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Updates aus dem Webinterface. Auf Klick, mit Fortschritt und automatischem Rollback; ein eigener,
optionaler Container erledigt die Arbeit. Ende-zu-Ende mit echten Images getestet, siehe Handbuch
„Update aus dem Webinterface (optional)“:

- **Updates aus dem Webinterface (optional):** Einstellungen › „OPNexus aktualisieren“ zeigt die
  neueste Version samt Release-Notes und aktualisiert auf Klick — mit Fortschrittsanzeige,
  automatischem Rollback bei einem fehlgeschlagenen Update und „Auf Vorversion zurücksetzen“
  (mit Hinweis, ob dabei Daten verloren gehen). Dahinter arbeitet ein eigener, kleiner
  **Updater-Container** (`updater/`, Compose-Profil `updater`, mit `COMPOSE_PROFILES=updater`
  in `.env` aktivieren), der dieselbe Logik wie `opnexus update` ausführt. Er ist standardmäßig
  **aus** und der einzige Container mit Docker-Socket; das Backend bekommt ihn nie. Der Updater
  hat keinen veröffentlichten Port, ist nur mit einem selbst erzeugten Token ansprechbar und kann
  ausschließlich „aktualisieren auf die vom Release-Feed gemeldete neueste stabile Version“,
  „zurückrollen“ und „Status“. Hauptversions-Sprünge bleiben der Kommandozeile vorbehalten.
  Anfragen stehen im Audit-Log (neuer Typ „OPNexus-System“). Gilt nur für Release-Images.
- `opnexus` (jetzt in `updater/opnexus`, `scripts/opnexus` ist ein Symlink): `status --porcelain`,
  Exit-Code 3 für „fehlgeschlagen und sauber zurückgerollt“. Release-Images und -Bundle enthalten
  zusätzlich `opnexus-updater`.
- Der Banner „Neue Version verfügbar“ verlinkt auf den neuen Abschnitt in den Einstellungen.
2026-10-10 00:24:01 +02:00
Julian Boldt 3b76f9f5fa Release v1.14.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Diagnose von der Firewall aus sowie VXLAN und GIF. Gegen die OPNsense-Core-Quellen
geprüft und live auf den Test-Firewalls getestet, siehe Handbuch „Erweiterung 1.14.0“:

- **Host-System › Diagnose:** Ping, Traceroute, Port-Prüfung und DNS-Abfrage, ausgeführt von der
  ausgewählten Firewall. Nur für Admins, strenge Eingabeprüfung, jeder Lauf im Audit-Log,
  Ping-Jobs werden immer wieder aufgeräumt, der Banner der Gegenstelle wird nicht weitergegeben.
  Packet-Capture gibt es bewusst nicht (Mitschnitte enthalten fremde Nutzdaten).
- **Netzwerk › Geräte:** VXLAN und GIF-Tunnel (anlegen, ändern, löschen) über dieselbe abgesicherte
  Engine wie die übrigen Geräte.
- **Fehlerkorrektur:** Auf den Host-System-Seiten war „Übersicht › Hosts“ markiert und der Seitentitel
  falsch, weil `/hostsystem` mit `/hosts` beginnt.
2026-10-09 22:49:10 +02:00
Julian Boldt b341ccd49f Release v1.13.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Mehr Verwaltung pro Firewall und ein Abgleich der Cluster-Knoten. Alles über
dieselbe abgesicherte Engine wie in 1.12.0 und live auf drei Test-Firewalls
geprüft, siehe Handbuch „Erweiterung 1.13.0“:

- **Netzwerk › Geräte:** LAGG, Bridge, Loopback und GRE. Dazu Gateway-Gruppen,
  DNS-Weiterleitungen (Unbound) und DHCP-Bereiche (dnsmasq).
- **Neue Gruppe „Host-System“:** Tunables (immer mit Bestätigung, Vorgaben nur per
  „Überschreiben“), Cron, Syslog-Ziele und ZFS-Snapshots (anlegen und löschen; der
  aktive Snapshot ist geschützt, das Aktivieren bleibt OPNsense selbst überlassen).
- **Netzwerk › Abgleich:** zeigt Unterschiede in der Netzwerk-Konfiguration zwischen den
  Knoten eines Clusters. Neuer Alarm „Netzwerk-Abweichung im Cluster“ (Einstellungen ›
  Benachrichtigungen, Migration 0004).
- **Auf den Partner übertragen:** Nach dem Anlegen eines Objekts lässt es sich mit
  einem Klick auch auf dem anderen Cluster-Knoten anlegen (eigene Vorschau, eigenes Backup).
- **Kopfzeile:** Mit acht Gruppen zeigt die Navigation die Namen ab 1150 px, darunter
  nur Icons, damit Suche und Abmelden in einer Zeile bleiben.
2026-10-09 22:08:39 +02:00
Julian Boldt 20fd575420 Release v1.12.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Netzwerk-Verwaltung und Übernahme von Hand angelegter Regeln. Alles gegen die
OPNsense-Core-Quellen geprüft und live auf drei Test-Firewalls (26.7.1 und
26.7.6) getestet, siehe Handbuch „Netzwerk, DNS und DHCP“:

- **Neuer Bereich „Netzwerk“:** Inventar je Firewall (Schnittstellen mit
  Adressen und CARP-Zustand, VLANs, virtuelle IPs, Gateways, Routen, DNS-Einträge,
  DHCP) und Änderungen daran: anlegen, bearbeiten, löschen für VLANs, virtuelle
  IPs, Gateways, statische Routen, Schnittstellen-Zuweisung samt IP-Konfiguration,
  Unbound-DNS-Einträge, dnsmasq-Hosts sowie Kea-Subnetze und -Reservierungen.
- **Abgesichert:** Jede Änderung läuft über eine Vorschau mit Diff und einem Token,
  der den Firewall-Stand erfasst. Vor dem Schreiben wird ein Konfig-Backup
  gespeichert (ohne Backup keine Änderung), danach angewendet und zurückgelesen.
  OPNexus erkennt, über welche Schnittstelle es die Firewall erreicht; Änderungen,
  die diese Verbindung kappen können (Verwaltungs-Schnittstelle, Standard-Gateway,
  Standardroute, Route über die Firewall-Adresse), brauchen eine ausdrückliche
  Bestätigung mit Eingabe des Firewall-Namens. Es gibt bewusst **keinen**
  automatischen Rollback: OPNexus kann eine Änderung nicht zurücknehmen, wenn sie
  seine eigene Verbindung trennt. Das Backup vor der Änderung ist der Rückweg.
- **Schnittstellen-Löschen:** OPNsense entfernt dabei selbst alle Regeln, die die
  Schnittstelle nutzen. Die Vorschau listet sie, die Bestätigung ist Pflicht.
  Bereits auf der Firewall vorgemerkte Änderungen blockieren eine neue Änderung.
- **IP-Konfiguration per API gibt es erst ab OPNsense 26.7.6.** Auf 26.7.1 ist die
  Zuweisungsliste nur lesbar; OPNexus erkennt das an den Daten und sperrt die
  Änderung mit Begründung.
- **Von der Firewall übernehmen:** Regeln, die direkt in OPNsense angelegt wurden,
  lassen sich in OPNexus übernehmen, bearbeiten und ausrollen. Die Übernahme schreibt
  nichts auf die Firewall. Regeln, die das Modell verfälschen würde (invertierte
  Quelle/Ziel, mehrere Schnittstellen, keine Quick-Regel, …), werden mit Grund
  abgelehnt. Neu: Regeln haben ein Protokollieren-Flag (`log`, Migration 0003),
  damit es beim Bearbeiten nicht still verloren geht; nicht modellierte Felder
  (Gateway, Zustandsoptionen, …) bleiben auf der Firewall unverändert.
- **Geheimnisse:** Das CARP-Passwort (liefert OPNsense im Klartext) und der
  Kea-DDNS-Schlüssel werden nie zurückgegeben, gespeichert oder auditiert.
- **Host-Adresse änderbar:** Hosts › Bearbeiten hat jetzt das Feld „Adresse“. Nach
  einer Änderung der Verwaltungs-IP muss der Host nicht mehr gelöscht und neu
  angelegt werden; Verlauf und Zuordnungen bleiben erhalten. Meldet eine Änderung
  „angewendet, aber nicht prüfbar“, weist die Oberfläche darauf hin.
- **Fehler aus den Live-Tests, die Mocks nicht gezeigt haben:** Jede Änderung einer
  virtuellen IP muss die Adresse (`network`) wiederholen, sonst leert OPNsense sie
  bzw. lehnt ab; die Prüfung nach dem Anwenden vergleicht jetzt den vollständigen
  Soll-Zustand statt nur der gesendeten Felder. Dazu: Gateway-Priorität als Zahl,
  Routen mit `enabled` statt `disabled`, Beschreibungsmuster der Schnittstellen,
  gesperrte Schnittstellen, Pflicht-Backup vor jeder Änderung.
- **Hinweis zu den Versionen:** Die IP-Konfiguration von Schnittstellen per API gibt
  es erst ab OPNsense 26.7.6. Mit 26.7.5 (die Prod-Firewalls) sind VLANs, virtuelle
  IPs, Gateways, Routen, DNS und DHCP verwaltbar, Schnittstellen nur lesbar.
2026-10-09 21:23:04 +02:00
Julian Boldt 1fc2d2c87c Release v1.11.1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Fehler aus dem ersten Live-Test der bisher nur gemockten Funktionen auf drei
echten OPNsense-Firewalls (alle in den Tests mit Mocks unsichtbar geblieben).
Live bestätigt sind jetzt auch das Major-Upgrade über den Update Center
(25.7 → 26.1 → 26.7, je mit Backup und Abschluss-Bestätigung), WireGuard,
OpenVPN, IPsec, Zertifikate, ACME (Staging), hierarchische Policies,
Benachrichtigungen und der HA-Config-Sync:

- **Zertifikate (Trust-Store) und ACME ließen sich nicht pushen:** Die URLs
  endeten auf `…Action` (`/trust/cert/addAction`), OPNsense kennt aber nur
  `/trust/cert/add`, `set`, `del` bzw. `/acmeclient/…/add`, `update`, `del` (404).
  Die Mocks hatten denselben Tippfehler. Ein neuer Test prüft den Client-Quelltext
  gegen dieses Muster.
- **OpenVPN-Deploy scheiterte immer:** Die Firewall verlangt `vpnid` und vergibt
  es nicht selbst. OPNexus wählt jetzt die nächste freie. Außerdem erwarten
  `cert`/`ca` (OpenVPN) und `caref` (Zertifikat signieren) die **refid** der
  Trust-Store-Einträge, der Picker lieferte die Zeilen-UUID. Die Listen
  `/hosts/{id}/certificates` und `/cas` liefern jetzt `refid`; im
  Zertifikatsformular gibt es ein CA-Dropdown statt eines Freitextfelds.
- **WireGuard meldete „Erfolg“ ohne laufenden Tunnel:** Ist der globale Dienst
  (VPN › WireGuard › Allgemein) aus, entsteht trotz geschriebener Konfiguration
  kein Interface. Der Deploy schaltet den Dienst jetzt ein.
- **Dienststeuerung:** Das Backend prüfte weder, ob der Dienst existiert, noch ob
  er gesperrt ist (pf, webgui, routing …). Beides war nur in der UI gesperrt;
  unbekannte Namen meldete die Firewall trotzdem als „ok“. Jetzt 404
  `SERVICE_NOT_FOUND` bzw. 409 `SERVICE_LOCKED`.
- **Fehlertexte der Firewall unleserlich:** Die Redaktion maskierte jede
  Validierungsmeldung, deren Feldname `cert`, `crt` oder `auth` enthält
  („instance.cert: [redacted]“). In `validations` lösen diese drei Namen die
  Maskierung nicht mehr aus; echte Geheimnisfelder bleiben maskiert. Das
  Löschen eines benutzten Zertifikats nennt jetzt den Grund der Firewall
  („Item in use by OpenVPN …“) statt eines allgemeinen Fehlers.
- **Drift-Alarme flackerten beim Neustart eines Cluster-Mitglieds:** Mit nur
  einem erreichbaren Host gab es nichts zu vergleichen, offene Regel-/NAT-Drift-
  und Zertifikatsprobleme wurden „gelöst“ und danach neu geöffnet (je ein
  Meldungspaar pro Problem). Offene Probleme bleiben jetzt unverändert, solange
  sie nicht beurteilbar sind.
- **IPsec lief nach dem Deploy nicht:** Der globale IPsec-Schalter war aus, die
  Verbindungen wurden gespeichert, `reconfigure` antwortete „ok“, OPNexus meldete
  „Erfolg“ (gleiche Klasse wie bei WireGuard). Der Deploy schaltet ihn jetzt ein;
  zwischen zwei Test-FWs steht der Tunnel (Phase 1 verbunden).
- **Löschen scheiterte ohne Grund:** Wenn die Firewall ein Löschen ablehnt (Alias
  noch in einer Regel, Zertifikat in Benutzung), nennt die Meldung jetzt den Grund
  der Firewall (`UPSTREAM_REJECTED`) statt „Anfrage fehlgeschlagen“.
- **Kaputte Eingaben:** Ein NUL-Zeichen in einem Textfeld ergab einen 500er samt
  abgebrochener Verbindung, ebenso ein langes oder NUL-haltiges Pfadsegment (der
  Audit-Eintrag übernahm es ungeprüft in `VARCHAR(36)`). Jetzt 422 bzw. 404/422;
  ein Fuzz-Lauf über alle 188 Routen findet keine 5xx mehr.
- **`opnexus rollback` landete auf der falschen Version:** Zurückgesetzt wurde auf
  die vom Backend gemeldete Programmversion statt auf den vorher eingestellten
  Image-Tag. Bei Release-Candidates (rc1-Images melden schon `1.11.0`) führte das
  auf das neuere Release. Jetzt wird der vorherige exakte Tag wiederhergestellt.
- **Handy-Ansicht (375px) zoomte aus:** Die ausgeblendete Überschrift der
  Aktionsspalte (`sr-only`, absolut positioniert) lag außerhalb der scrollbaren
  Tabellenkarte und machte die Seite bis zu doppelt so breit (Hosts, Benutzer,
  Host-Details). Scroll-Container sind jetzt der Bezugsrahmen.
- **Überschriften-Reihenfolge:** Kartentitel in den Listen (OpenVPN, IPsec,
  WireGuard, NAT) sind `h2` statt `h3`, sichtbar erst mit Daten; axe meldet auf
  allen Seiten, hell und dunkel, nichts mehr.

- **CARP-Wartungsfenster mit Auto-Revert:** Nach dem automatischen Zurückschalten
  blieb die Aktion „CARP-Übergabe“ (`carp_enter`) offen. Sie ließ sich nicht mehr
  bestätigen (der Host ist ja nicht mehr im Wartungsmodus) und blockierte alle
  Deploys und Updates im Cluster (`OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY`), bis
  jemand sie von Hand abbrach. Der Auto-Revert schließt die Aktion jetzt selbst
  (Status „abgebrochen“, Audit-Eintrag). Schlägt das Zurückschalten fehl, bleibt
  sie bewusst offen. Gefunden beim Live-Test gegen zwei OPNsense-26.7-Firewalls.
- **Update Center, Protokoll-Fenster:** Nach einem Update mit Neustart ist das
  Protokoll der Firewall leer; angezeigt wurde dann nur das Wort „error“, obwohl
  das Update erfolgreich war. Jetzt steht dort ein erklärender Hinweis.
2026-10-09 18:35:53 +02:00
Julian Boldt 9889be64ef Release v1.11.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Sicherheitsüberprüfung vom 2026-10-07, umgesetzte Korrekturen.
**Verhaltensänderungen**, die beim Update auffallen können:

- **Backup-Rohdownload nur für Admins**. Nur-Lese-Konten sehen weiter die
  Liste der Snapshots, bekommen die `config.xml` (enthält Passwort-Hashes, VPN-Schlüssel)
  aber nicht mehr; jeder Versuch steht im Audit-Verlauf (neuer Status „abgelehnt“).
- **Passwortwechsel/-reset beendet alle Sitzungen** des Kontos; ebenso
  2FA-Reset durch einen Admin. 2FA aktivieren/deaktivieren beendet die *anderen* Sitzungen.
- **SMTP über SSL/STARTTLS prüft Zertifikat und Hostname**. Mailserver mit
  privater CA: `SMTP_CA_FILE` setzen. SMTP-Anmeldung ohne Verschlüsselung ist gesperrt
  (`ALLOW_INSECURE_SMTP_AUTH=true` hebt das bewusst auf).
- **Firewalls nur noch per `https://`** — beim Anlegen und bei jeder Anfrage;
  ein vorhandener `http://`-Host zeigt „offline“, statt Zugangsdaten im Klartext zu senden
  (`ALLOW_INSECURE_FIREWALL_URLS=true` nur für Labor-Setups).
- **VPN-Push (IPsec/OpenVPN/WireGuard) verlangt das Vorschau-Token** und das
  Token ändert sich, sobald nur das Geheimnis (PSK/Schlüssel) rotiert wurde. Die Vorschau
  zeigt „unverändert“ jetzt korrekt, statt jeden Tunnel dauerhaft als geändert.
- **Host löschen** ist auch bei VPN-, Zertifikats- und ACME-Objekten auf der Firewall
  gesperrt, bis diese entfernt sind; Schlüsseldateien nie bereitgestellter
  Entwürfe werden mit entfernt.
- Benachrichtigungs-Geheimnisse liegen jetzt im Volume `notification_secrets`
 . Ohne lesbares Signatur-/SMTP-Geheimnis wird **nichts** mehr gesendet
  (vorher: unsigniert/ohne Login). Wer Geheimnisse gesetzt hat, muss sie nach dem
  Update einmal neu eingeben, falls der Container zwischenzeitlich ersetzt wurde.

Weitere Korrekturen:

- Oberfläche: Der „Bestätigen“-Button steht jetzt am Ende der Deploy-Ergebnis-Zeile statt
  in der Aktionszeile (Regeln, Aliase, Port-Forwards, 1:1-NAT, Outbound-NAT,
  Hierarchie-Regeln); die Aktionszeile springt nicht mehr, wenn ein Deployment offen wird.

- **Deployments hinterlassen vor dem ersten Firewall-Aufruf eine dauerhafte Spur**
  (gemeinsamer Ablauf `deploy_flow.py` für Regeln, Aliase, Port-Forwards, 1:1-NAT,
  Outbound-NAT und Hierarchie-Regeln): Bei Absturz, verlorener Antwort oder Timeout bleibt
  die Rollback-Frist erhalten und der Sweeper macht die Änderung rückgängig. Wo die Firewall
  eine neue Regel möglicherweise angelegt hat, ohne dass eine UUID bekannt ist, steht der
  Eintrag auf „Rollback fehlgeschlagen“ mit dem Hinweis „Ausgang unklar“ (blockiert
  erneutes Deployen, löst die Benachrichtigung aus); „Rollback wiederholen“ entfernt ihn.
  Von der Firewall beantwortete Ablehnungen und Verbindungsfehler bleiben einfache Fehler.
- TOTP- und Recovery-Codes sind auch bei gleichzeitigen Anfragen genau einmal
  verwendbar (Zeilensperre).
- Login-Sperre: begrenzter Speicher, Argon2 läuft außerhalb des Event-Loops mit
  Parallelitätsgrenze, 2FA-Einrichtung/-Aktivierung/-Deaktivierung und Passwortwechsel
  haben ein Versuchslimit; offene 2FA-Einrichtung läuft nach 15 Minuten ab; Proxy
  begrenzt API-Anfragen auf 2 MB.
- Fehlermeldungen der Firewall werden vor Speicherung/Anzeige von Geheimnissen
  bereinigt und gekürzt; Validierungsfehler (422) geben Eingaben nicht mehr
  zurück; Webhook-URLs (oft mit Token) sieht ein Nur-Lese-Konto nur noch als
  Host.
- Audit: MFA/Passwort, VPN, Zertifikate/ACME, Benachrichtigungen und Backup-Exporte
  werden aufgezeichnet, ebenso abgelehnte Schreibversuche von Nur-Lese-Konten.
- Same-Origin-Prüfung berücksichtigt das Schema bei `COOKIE_SECURE=true`;
  Warnung beim Start, wenn `FRONTEND_ORIGIN` https ist, das Cookie aber nicht `Secure`
 . Geheimnisdateien werden atomar mit Modus 0600 angelegt.
- Frontend-Abhängigkeiten (`sharp`, `source-map-js`, `brace-expansion`) aktualisiert.
  Offen: `braces`/`postcss-selector-parser` nur in Build-Werkzeugen (Tailwind 3 /
  ESLint), nicht im Laufzeit-Image; für `braces` gibt es keine gepatchte Version.
2026-10-07 13:34:54 +02:00
Julian Boldt 5c092fabac Release v1.11.0-rc2
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Siehe CHANGELOG.md
2026-10-07 12:50:51 +02:00
Julian Boldt e51df4ba04 Release v1.11.0-rc1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Siehe CHANGELOG.md
2026-10-07 12:24:38 +02:00
155 changed files with 11596 additions and 894 deletions

No files matched your search

+20 -1
View File
@@ -71,6 +71,25 @@ TRUST_PROXY_HEADERS=false
# Release-Feed (GitHub/Forgejo "releases/latest"). Leer = aus: dann macht # Release-Feed (GitHub/Forgejo "releases/latest"). Leer = aus: dann macht
# OPNexus keine ausgehende Anfrage. Gesendet wird nur ein GET mit # OPNexus keine ausgehende Anfrage. Gesendet wird nur ein GET mit
# User-Agent "OPNexus/<Version>". Admins sehen bei einer neueren Version einen # User-Agent "OPNexus/<Version>". Admins sehen bei einer neueren Version einen
# Hinweis; aktualisiert wird nie automatisch, sondern mit `opnexus update`. # Hinweis; aktualisiert wird nie automatisch, sondern mit `opnexus update` oder
# (optional, siehe unten) per Klick im Webinterface.
# UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json # UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json
# UPDATE_CHECK_INTERVAL_HOURS=12 # UPDATE_CHECK_INTERVAL_HOURS=12
# ── Updates aus dem Webinterface (optional, braucht Release-Images) ───
# Startet den Updater-Container, mit dem Admins unter Einstellungen › „OPNexus
# aktualisieren“ ein Update auslösen können. Er ist der einzige Container mit
# Docker-Socket-Zugriff (= root-gleiche Rechte auf diesem Host) und standardmäßig
# AUS. Benötigt außerdem UPDATE_CHECK_URL (oben). Siehe Handbuch.
# COMPOSE_PROFILES=updater
# ── Transportsicherheit (nur ändern, wenn nötig) ──────────────────────
# Firewall-API-Zugangsdaten gehen per HTTP-Basic-Auth raus, deshalb sind
# http://-Adressen für Firewalls abgelehnt (beim Anlegen UND bei jeder
# Anfrage). Nur für isolierte Labor-Setups auf "true" setzen.
# ALLOW_INSECURE_FIREWALL_URLS=false
# SMTP über SSL/STARTTLS prüft Zertifikat UND Hostname. Mailserver mit
# privater CA: CA-PEM in den Backend-Container einbinden und hier angeben.
# SMTP_CA_FILE=/data/smtp-ca.pem
# SMTP-Anmeldung ohne Verschlüsselung (Sicherheit "none") bleibt gesperrt.
# ALLOW_INSECURE_SMTP_AUTH=false
+2
View File
@@ -59,3 +59,5 @@ jobs:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- run: python3 scripts/check_i18n.py - run: python3 scripts/check_i18n.py
- run: bash -n scripts/opnexus && bash -n scripts/check_version.sh && bash -n scripts/make_release_bundle.sh - run: bash -n scripts/opnexus && bash -n scripts/check_version.sh && bash -n scripts/make_release_bundle.sh
# Updater-Container (Zustandsübergänge, Ablehnungen, Auth) ohne Docker-Daemon.
- run: python3 -m unittest discover -s updater -p 'test_*.py'
+2 -2
View File
@@ -12,7 +12,7 @@
# Berechtigung "write:package" eintragen) # Berechtigung "write:package" eintragen)
# Wer auf mehrere Registries veröffentlichen will, pusht den Tag einfach zu # Wer auf mehrere Registries veröffentlichen will, pusht den Tag einfach zu
# mehreren Remotes — jeder Remote baut mit seiner eigenen Konfiguration. # mehreren Remotes — jeder Remote baut mit seiner eigenen Konfiguration.
# Ergebnis: <REGISTRY>/<IMAGE_NAMESPACE>/opnexus-{backend,frontend,proxy}:<version> # Ergebnis: <REGISTRY>/<IMAGE_NAMESPACE>/opnexus-{backend,frontend,proxy,updater}:<version>
# (flache Namen, weil verschachtelte Pfade je Registry unterschiedlich gehen). # (flache Namen, weil verschachtelte Pfade je Registry unterschiedlich gehen).
# #
# Der Publish-Job braucht einen Runner mit Docker-Zugriff, der QEMU- # Der Publish-Job braucht einen Runner mit Docker-Zugriff, der QEMU-
@@ -42,7 +42,7 @@ jobs:
strategy: strategy:
fail-fast: true fail-fast: true
matrix: matrix:
service: [backend, frontend, proxy] service: [backend, frontend, proxy, updater]
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
+236
View File
@@ -31,6 +31,242 @@ den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin … ansteht): Merlin …
## [1.15.0] "Gyrfalcon" — 2026-10-10
Updates aus dem Webinterface. Auf Klick, mit Fortschritt und automatischem Rollback; ein eigener,
optionaler Container erledigt die Arbeit. Ende-zu-Ende mit echten Images getestet, siehe Handbuch
„Update aus dem Webinterface (optional)“:
- **Updates aus dem Webinterface (optional):** Einstellungen › „OPNexus aktualisieren“ zeigt die
neueste Version samt Release-Notes und aktualisiert auf Klick — mit Fortschrittsanzeige,
automatischem Rollback bei einem fehlgeschlagenen Update und „Auf Vorversion zurücksetzen“
(mit Hinweis, ob dabei Daten verloren gehen). Dahinter arbeitet ein eigener, kleiner
**Updater-Container** (`updater/`, Compose-Profil `updater`, mit `COMPOSE_PROFILES=updater`
in `.env` aktivieren), der dieselbe Logik wie `opnexus update` ausführt. Er ist standardmäßig
**aus** und der einzige Container mit Docker-Socket; das Backend bekommt ihn nie. Der Updater
hat keinen veröffentlichten Port, ist nur mit einem selbst erzeugten Token ansprechbar und kann
ausschließlich „aktualisieren auf die vom Release-Feed gemeldete neueste stabile Version“,
„zurückrollen“ und „Status“. Hauptversions-Sprünge bleiben der Kommandozeile vorbehalten.
Anfragen stehen im Audit-Log (neuer Typ „OPNexus-System“). Gilt nur für Release-Images.
- `opnexus` (jetzt in `updater/opnexus`, `scripts/opnexus` ist ein Symlink): `status --porcelain`,
Exit-Code 3 für „fehlgeschlagen und sauber zurückgerollt“. Release-Images und -Bundle enthalten
zusätzlich `opnexus-updater`.
- Der Banner „Neue Version verfügbar“ verlinkt auf den neuen Abschnitt in den Einstellungen.
## [1.14.0] "Eagle" — 2026-10-09
Diagnose von der Firewall aus sowie VXLAN und GIF. Gegen die OPNsense-Core-Quellen
geprüft und live auf den Test-Firewalls getestet, siehe Handbuch „Erweiterung 1.14.0“:
- **Host-System › Diagnose:** Ping, Traceroute, Port-Prüfung und DNS-Abfrage, ausgeführt von der
ausgewählten Firewall. Nur für Admins, strenge Eingabeprüfung, jeder Lauf im Audit-Log,
Ping-Jobs werden immer wieder aufgeräumt, der Banner der Gegenstelle wird nicht weitergegeben.
Packet-Capture gibt es bewusst nicht (Mitschnitte enthalten fremde Nutzdaten).
- **Netzwerk › Geräte:** VXLAN und GIF-Tunnel (anlegen, ändern, löschen) über dieselbe abgesicherte
Engine wie die übrigen Geräte.
- **Fehlerkorrektur:** Auf den Host-System-Seiten war „Übersicht › Hosts“ markiert und der Seitentitel
falsch, weil `/hostsystem` mit `/hosts` beginnt.
## [1.13.0] "Buzzard" — 2026-10-09
Mehr Verwaltung pro Firewall und ein Abgleich der Cluster-Knoten. Alles über
dieselbe abgesicherte Engine wie in 1.12.0 und live auf drei Test-Firewalls
geprüft, siehe Handbuch „Erweiterung 1.13.0“:
- **Netzwerk › Geräte:** LAGG, Bridge, Loopback und GRE. Dazu Gateway-Gruppen,
DNS-Weiterleitungen (Unbound) und DHCP-Bereiche (dnsmasq).
- **Neue Gruppe „Host-System“:** Tunables (immer mit Bestätigung, Vorgaben nur per
„Überschreiben“), Cron, Syslog-Ziele und ZFS-Snapshots (anlegen und löschen; der
aktive Snapshot ist geschützt, das Aktivieren bleibt OPNsense selbst überlassen).
- **Netzwerk › Abgleich:** zeigt Unterschiede in der Netzwerk-Konfiguration zwischen den
Knoten eines Clusters. Neuer Alarm „Netzwerk-Abweichung im Cluster“ (Einstellungen ›
Benachrichtigungen, Migration 0004).
- **Auf den Partner übertragen:** Nach dem Anlegen eines Objekts lässt es sich mit
einem Klick auch auf dem anderen Cluster-Knoten anlegen (eigene Vorschau, eigenes Backup).
- **Kopfzeile:** Mit acht Gruppen zeigt die Navigation die Namen ab 1150 px, darunter
nur Icons, damit Suche und Abmelden in einer Zeile bleiben.
## [1.12.0] "Condor" — 2026-10-09
Netzwerk-Verwaltung und Übernahme von Hand angelegter Regeln. Alles gegen die
OPNsense-Core-Quellen geprüft und live auf drei Test-Firewalls (26.7.1 und
26.7.6) getestet, siehe Handbuch „Netzwerk, DNS und DHCP“:
- **Neuer Bereich „Netzwerk“:** Inventar je Firewall (Schnittstellen mit
Adressen und CARP-Zustand, VLANs, virtuelle IPs, Gateways, Routen, DNS-Einträge,
DHCP) und Änderungen daran: anlegen, bearbeiten, löschen für VLANs, virtuelle
IPs, Gateways, statische Routen, Schnittstellen-Zuweisung samt IP-Konfiguration,
Unbound-DNS-Einträge, dnsmasq-Hosts sowie Kea-Subnetze und -Reservierungen.
- **Abgesichert:** Jede Änderung läuft über eine Vorschau mit Diff und einem Token,
der den Firewall-Stand erfasst. Vor dem Schreiben wird ein Konfig-Backup
gespeichert (ohne Backup keine Änderung), danach angewendet und zurückgelesen.
OPNexus erkennt, über welche Schnittstelle es die Firewall erreicht; Änderungen,
die diese Verbindung kappen können (Verwaltungs-Schnittstelle, Standard-Gateway,
Standardroute, Route über die Firewall-Adresse), brauchen eine ausdrückliche
Bestätigung mit Eingabe des Firewall-Namens. Es gibt bewusst **keinen**
automatischen Rollback: OPNexus kann eine Änderung nicht zurücknehmen, wenn sie
seine eigene Verbindung trennt. Das Backup vor der Änderung ist der Rückweg.
- **Schnittstellen-Löschen:** OPNsense entfernt dabei selbst alle Regeln, die die
Schnittstelle nutzen. Die Vorschau listet sie, die Bestätigung ist Pflicht.
Bereits auf der Firewall vorgemerkte Änderungen blockieren eine neue Änderung.
- **IP-Konfiguration per API gibt es erst ab OPNsense 26.7.6.** Auf 26.7.1 ist die
Zuweisungsliste nur lesbar; OPNexus erkennt das an den Daten und sperrt die
Änderung mit Begründung.
- **Von der Firewall übernehmen:** Regeln, die direkt in OPNsense angelegt wurden,
lassen sich in OPNexus übernehmen, bearbeiten und ausrollen. Die Übernahme schreibt
nichts auf die Firewall. Regeln, die das Modell verfälschen würde (invertierte
Quelle/Ziel, mehrere Schnittstellen, keine Quick-Regel, …), werden mit Grund
abgelehnt. Neu: Regeln haben ein Protokollieren-Flag (`log`, Migration 0003),
damit es beim Bearbeiten nicht still verloren geht; nicht modellierte Felder
(Gateway, Zustandsoptionen, …) bleiben auf der Firewall unverändert.
- **Geheimnisse:** Das CARP-Passwort (liefert OPNsense im Klartext) und der
Kea-DDNS-Schlüssel werden nie zurückgegeben, gespeichert oder auditiert.
- **Host-Adresse änderbar:** Hosts › Bearbeiten hat jetzt das Feld „Adresse“. Nach
einer Änderung der Verwaltungs-IP muss der Host nicht mehr gelöscht und neu
angelegt werden; Verlauf und Zuordnungen bleiben erhalten. Meldet eine Änderung
„angewendet, aber nicht prüfbar“, weist die Oberfläche darauf hin.
- **Fehler aus den Live-Tests, die Mocks nicht gezeigt haben:** Jede Änderung einer
virtuellen IP muss die Adresse (`network`) wiederholen, sonst leert OPNsense sie
bzw. lehnt ab; die Prüfung nach dem Anwenden vergleicht jetzt den vollständigen
Soll-Zustand statt nur der gesendeten Felder. Dazu: Gateway-Priorität als Zahl,
Routen mit `enabled` statt `disabled`, Beschreibungsmuster der Schnittstellen,
gesperrte Schnittstellen, Pflicht-Backup vor jeder Änderung.
- **Hinweis zu den Versionen:** Die IP-Konfiguration von Schnittstellen per API gibt
es erst ab OPNsense 26.7.6. Mit 26.7.5 (die Prod-Firewalls) sind VLANs, virtuelle
IPs, Gateways, Routen, DNS und DHCP verwaltbar, Schnittstellen nur lesbar.
## [1.11.1] "Merlin" — 2026-10-09
Fehler aus dem ersten Live-Test der bisher nur gemockten Funktionen auf drei
echten OPNsense-Firewalls (alle in den Tests mit Mocks unsichtbar geblieben).
Live bestätigt sind jetzt auch das Major-Upgrade über den Update Center
(25.7 → 26.1 → 26.7, je mit Backup und Abschluss-Bestätigung), WireGuard,
OpenVPN, IPsec, Zertifikate, ACME (Staging), hierarchische Policies,
Benachrichtigungen und der HA-Config-Sync:
- **Zertifikate (Trust-Store) und ACME ließen sich nicht pushen:** Die URLs
endeten auf `…Action` (`/trust/cert/addAction`), OPNsense kennt aber nur
`/trust/cert/add`, `set`, `del` bzw. `/acmeclient/…/add`, `update`, `del` (404).
Die Mocks hatten denselben Tippfehler. Ein neuer Test prüft den Client-Quelltext
gegen dieses Muster.
- **OpenVPN-Deploy scheiterte immer:** Die Firewall verlangt `vpnid` und vergibt
es nicht selbst. OPNexus wählt jetzt die nächste freie. Außerdem erwarten
`cert`/`ca` (OpenVPN) und `caref` (Zertifikat signieren) die **refid** der
Trust-Store-Einträge, der Picker lieferte die Zeilen-UUID. Die Listen
`/hosts/{id}/certificates` und `/cas` liefern jetzt `refid`; im
Zertifikatsformular gibt es ein CA-Dropdown statt eines Freitextfelds.
- **WireGuard meldete „Erfolg“ ohne laufenden Tunnel:** Ist der globale Dienst
(VPN › WireGuard › Allgemein) aus, entsteht trotz geschriebener Konfiguration
kein Interface. Der Deploy schaltet den Dienst jetzt ein.
- **Dienststeuerung:** Das Backend prüfte weder, ob der Dienst existiert, noch ob
er gesperrt ist (pf, webgui, routing …). Beides war nur in der UI gesperrt;
unbekannte Namen meldete die Firewall trotzdem als „ok“. Jetzt 404
`SERVICE_NOT_FOUND` bzw. 409 `SERVICE_LOCKED`.
- **Fehlertexte der Firewall unleserlich:** Die Redaktion maskierte jede
Validierungsmeldung, deren Feldname `cert`, `crt` oder `auth` enthält
(„instance.cert: [redacted]“). In `validations` lösen diese drei Namen die
Maskierung nicht mehr aus; echte Geheimnisfelder bleiben maskiert. Das
Löschen eines benutzten Zertifikats nennt jetzt den Grund der Firewall
(„Item in use by OpenVPN …“) statt eines allgemeinen Fehlers.
- **Drift-Alarme flackerten beim Neustart eines Cluster-Mitglieds:** Mit nur
einem erreichbaren Host gab es nichts zu vergleichen, offene Regel-/NAT-Drift-
und Zertifikatsprobleme wurden „gelöst“ und danach neu geöffnet (je ein
Meldungspaar pro Problem). Offene Probleme bleiben jetzt unverändert, solange
sie nicht beurteilbar sind.
- **IPsec lief nach dem Deploy nicht:** Der globale IPsec-Schalter war aus, die
Verbindungen wurden gespeichert, `reconfigure` antwortete „ok“, OPNexus meldete
„Erfolg“ (gleiche Klasse wie bei WireGuard). Der Deploy schaltet ihn jetzt ein;
zwischen zwei Test-FWs steht der Tunnel (Phase 1 verbunden).
- **Löschen scheiterte ohne Grund:** Wenn die Firewall ein Löschen ablehnt (Alias
noch in einer Regel, Zertifikat in Benutzung), nennt die Meldung jetzt den Grund
der Firewall (`UPSTREAM_REJECTED`) statt „Anfrage fehlgeschlagen“.
- **Kaputte Eingaben:** Ein NUL-Zeichen in einem Textfeld ergab einen 500er samt
abgebrochener Verbindung, ebenso ein langes oder NUL-haltiges Pfadsegment (der
Audit-Eintrag übernahm es ungeprüft in `VARCHAR(36)`). Jetzt 422 bzw. 404/422;
ein Fuzz-Lauf über alle 188 Routen findet keine 5xx mehr.
- **`opnexus rollback` landete auf der falschen Version:** Zurückgesetzt wurde auf
die vom Backend gemeldete Programmversion statt auf den vorher eingestellten
Image-Tag. Bei Release-Candidates (rc1-Images melden schon `1.11.0`) führte das
auf das neuere Release. Jetzt wird der vorherige exakte Tag wiederhergestellt.
- **Handy-Ansicht (375px) zoomte aus:** Die ausgeblendete Überschrift der
Aktionsspalte (`sr-only`, absolut positioniert) lag außerhalb der scrollbaren
Tabellenkarte und machte die Seite bis zu doppelt so breit (Hosts, Benutzer,
Host-Details). Scroll-Container sind jetzt der Bezugsrahmen.
- **Überschriften-Reihenfolge:** Kartentitel in den Listen (OpenVPN, IPsec,
WireGuard, NAT) sind `h2` statt `h3`, sichtbar erst mit Daten; axe meldet auf
allen Seiten, hell und dunkel, nichts mehr.
- **CARP-Wartungsfenster mit Auto-Revert:** Nach dem automatischen Zurückschalten
blieb die Aktion „CARP-Übergabe“ (`carp_enter`) offen. Sie ließ sich nicht mehr
bestätigen (der Host ist ja nicht mehr im Wartungsmodus) und blockierte alle
Deploys und Updates im Cluster (`OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY`), bis
jemand sie von Hand abbrach. Der Auto-Revert schließt die Aktion jetzt selbst
(Status „abgebrochen“, Audit-Eintrag). Schlägt das Zurückschalten fehl, bleibt
sie bewusst offen. Gefunden beim Live-Test gegen zwei OPNsense-26.7-Firewalls.
- **Update Center, Protokoll-Fenster:** Nach einem Update mit Neustart ist das
Protokoll der Firewall leer; angezeigt wurde dann nur das Wort „error“, obwohl
das Update erfolgreich war. Jetzt steht dort ein erklärender Hinweis.
## [1.11.0] "Merlin" — 2026-10-07
Sicherheitsüberprüfung vom 2026-10-07, umgesetzte Korrekturen.
**Verhaltensänderungen**, die beim Update auffallen können:
- **Backup-Rohdownload nur für Admins**. Nur-Lese-Konten sehen weiter die
Liste der Snapshots, bekommen die `config.xml` (enthält Passwort-Hashes, VPN-Schlüssel)
aber nicht mehr; jeder Versuch steht im Audit-Verlauf (neuer Status „abgelehnt“).
- **Passwortwechsel/-reset beendet alle Sitzungen** des Kontos; ebenso
2FA-Reset durch einen Admin. 2FA aktivieren/deaktivieren beendet die *anderen* Sitzungen.
- **SMTP über SSL/STARTTLS prüft Zertifikat und Hostname**. Mailserver mit
privater CA: `SMTP_CA_FILE` setzen. SMTP-Anmeldung ohne Verschlüsselung ist gesperrt
(`ALLOW_INSECURE_SMTP_AUTH=true` hebt das bewusst auf).
- **Firewalls nur noch per `https://`** — beim Anlegen und bei jeder Anfrage;
ein vorhandener `http://`-Host zeigt „offline“, statt Zugangsdaten im Klartext zu senden
(`ALLOW_INSECURE_FIREWALL_URLS=true` nur für Labor-Setups).
- **VPN-Push (IPsec/OpenVPN/WireGuard) verlangt das Vorschau-Token** und das
Token ändert sich, sobald nur das Geheimnis (PSK/Schlüssel) rotiert wurde. Die Vorschau
zeigt „unverändert“ jetzt korrekt, statt jeden Tunnel dauerhaft als geändert.
- **Host löschen** ist auch bei VPN-, Zertifikats- und ACME-Objekten auf der Firewall
gesperrt, bis diese entfernt sind; Schlüsseldateien nie bereitgestellter
Entwürfe werden mit entfernt.
- Benachrichtigungs-Geheimnisse liegen jetzt im Volume `notification_secrets`
. Ohne lesbares Signatur-/SMTP-Geheimnis wird **nichts** mehr gesendet
(vorher: unsigniert/ohne Login). Wer Geheimnisse gesetzt hat, muss sie nach dem
Update einmal neu eingeben, falls der Container zwischenzeitlich ersetzt wurde.
Weitere Korrekturen:
- Oberfläche: Der „Bestätigen“-Button steht jetzt am Ende der Deploy-Ergebnis-Zeile statt
in der Aktionszeile (Regeln, Aliase, Port-Forwards, 1:1-NAT, Outbound-NAT,
Hierarchie-Regeln); die Aktionszeile springt nicht mehr, wenn ein Deployment offen wird.
- **Deployments hinterlassen vor dem ersten Firewall-Aufruf eine dauerhafte Spur**
(gemeinsamer Ablauf `deploy_flow.py` für Regeln, Aliase, Port-Forwards, 1:1-NAT,
Outbound-NAT und Hierarchie-Regeln): Bei Absturz, verlorener Antwort oder Timeout bleibt
die Rollback-Frist erhalten und der Sweeper macht die Änderung rückgängig. Wo die Firewall
eine neue Regel möglicherweise angelegt hat, ohne dass eine UUID bekannt ist, steht der
Eintrag auf „Rollback fehlgeschlagen“ mit dem Hinweis „Ausgang unklar“ (blockiert
erneutes Deployen, löst die Benachrichtigung aus); „Rollback wiederholen“ entfernt ihn.
Von der Firewall beantwortete Ablehnungen und Verbindungsfehler bleiben einfache Fehler.
- TOTP- und Recovery-Codes sind auch bei gleichzeitigen Anfragen genau einmal
verwendbar (Zeilensperre).
- Login-Sperre: begrenzter Speicher, Argon2 läuft außerhalb des Event-Loops mit
Parallelitätsgrenze, 2FA-Einrichtung/-Aktivierung/-Deaktivierung und Passwortwechsel
haben ein Versuchslimit; offene 2FA-Einrichtung läuft nach 15 Minuten ab; Proxy
begrenzt API-Anfragen auf 2 MB.
- Fehlermeldungen der Firewall werden vor Speicherung/Anzeige von Geheimnissen
bereinigt und gekürzt; Validierungsfehler (422) geben Eingaben nicht mehr
zurück; Webhook-URLs (oft mit Token) sieht ein Nur-Lese-Konto nur noch als
Host.
- Audit: MFA/Passwort, VPN, Zertifikate/ACME, Benachrichtigungen und Backup-Exporte
werden aufgezeichnet, ebenso abgelehnte Schreibversuche von Nur-Lese-Konten.
- Same-Origin-Prüfung berücksichtigt das Schema bei `COOKIE_SECURE=true`;
Warnung beim Start, wenn `FRONTEND_ORIGIN` https ist, das Cookie aber nicht `Secure`
. Geheimnisdateien werden atomar mit Modus 0600 angelegt.
- Frontend-Abhängigkeiten (`sharp`, `source-map-js`, `brace-expansion`) aktualisiert.
Offen: `braces`/`postcss-selector-parser` nur in Build-Werkzeugen (Tailwind 3 /
ESLint), nicht im Laufzeit-Image; für `braces` gibt es keine gepatchte Version.
## [1.10.1] "Lanner" — 2026-10-06 ## [1.10.1] "Lanner" — 2026-10-06
PATCH-Release (Bugfixes, Dokumentation, Lizenz/Repo-Umstellung) nach dem ersten Live-Test PATCH-Release (Bugfixes, Dokumentation, Lizenz/Repo-Umstellung) nach dem ersten Live-Test
+11 -2
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](website/assets/screenshots/de/rules-light-desktop.webp) ![Zentrale Firewall-Regeln](website/assets/screenshots/de/rules-light-desktop.webp)
> **Stand:** `1.10.1-rc2` ist der aktuelle öffentliche Release-Candidate (er löst `1.10.0-rc1`/`-rc2` ab). Entwickelt wurde gegen echte OPNsense-Instanzen, eine frische Installation aus den veröffentlichten Images ist aber noch nicht im Feld erprobt — bitte Ecken und Kanten melden. > **Stand:** `1.15.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen ## Funktionen
@@ -16,13 +16,22 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
- **Sichere Änderungen** — jede Änderung läuft *Prüfen → Ausrollen → Bestätigen*; bleibt die Bestätigung im Zeitfenster aus, rollt OPNexus zurück. - **Sichere Änderungen** — jede Änderung läuft *Prüfen → Ausrollen → Bestätigen*; bleibt die Bestätigung im Zeitfenster aus, rollt OPNexus zurück.
- **Hierarchische Policies** — Pre-/Post-Rulebases mit Vererbung über Gruppenbäume. - **Hierarchische Policies** — Pre-/Post-Rulebases mit Vererbung über Gruppenbäume.
- **VPN** — IPsec-Site-to-Site, OpenVPN und WireGuard, einmal definiert und ausgerollt. - **VPN** — IPsec-Site-to-Site, OpenVPN und WireGuard, einmal definiert und ausgerollt.
- **Netzwerk** — Schnittstellen, VLANs, virtuelle IPs, Gateways, Routen, LAGG/Bridge/GRE, DNS und DHCP je Firewall, dazu Tunables, Cron, Syslog, Snapshots und Diagnose (Ping, Traceroute, Port-Prüfung, DNS), ein Abgleich der Cluster-Knoten, mit Diff-Vorschau, Backup vor jeder Änderung und Schutz davor, die eigene Verbindung zu kappen; von Hand angelegte Regeln lassen sich übernehmen.
- **Zertifikate** — Trust-Store für alle Firewalls und Ausstellung per ACME / Let's Encrypt. - **Zertifikate** — Trust-Store für alle Firewalls und Ausstellung per ACME / Let's Encrypt.
- **Logs & Audit** — zentrale Log-Suche mit Korrelation, Dashboards und lückenloser Deploy-Verlauf. - **Logs & Audit** — zentrale Log-Suche mit Korrelation, Dashboards und lückenloser Deploy-Verlauf.
- **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus. - **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus.
- **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch. - **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
Alle Screenshots zeigen fiktive Beispieldaten. Alle Screenshots zeigen fiktive Beispieldaten.
## Ratgeber
- [OPNsense-Firewalls zentral verwalten](https://opnexus.dev/opnsense-zentral-verwalten/) – wann sich zentrale Verwaltung lohnt und wie sie funktioniert
- [OPNsense-HA-Cluster mit CARP verwalten](https://opnexus.dev/opnsense-ha-cluster-carp/) – Rollen, Alarme, geordnete Updates, Abgleich beider Knoten
- [OPNsense-Regeln sicher ausrollen](https://opnexus.dev/opnsense-regeln-sicher-ausrollen/) – Vorschau, Bestätigungsfenster und automatischer Rollback
- [Das Kommandozeilen-Werkzeug `opnexus`](https://opnexus.dev/docs/cli/) – Installation aktualisieren, zurückrollen und sichern
## Schnellstart ## Schnellstart
Voraussetzung: Docker mit Compose-Plugin. Voraussetzung: Docker mit Compose-Plugin.
@@ -46,7 +55,7 @@ Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelie
```bash ```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.10.1-rc2 OPNEXUS_VERSION=1.15.0
``` ```
```bash ```bash
+13 -2
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](website/assets/screenshots/en/rules-light-desktop.webp) ![Central firewall rules](website/assets/screenshots/en/rules-light-desktop.webp)
> **Status:** `1.10.1-rc2` is the current public release candidate (it replaces `1.10.0-rc1`/`-rc2`). It has been developed against real OPNsense instances, but a fresh install from the published images has not been field-tested yet — please expect rough edges and report them. > **Status:** `1.15.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does ## What it does
@@ -16,15 +16,24 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
- **Safe changes** — every change goes *preview → deploy → confirm*; without your confirmation inside the time window OPNexus rolls it back. - **Safe changes** — every change goes *preview → deploy → confirm*; without your confirmation inside the time window OPNexus rolls it back.
- **Hierarchical policies** — pre/post rulebases with inheritance across group trees. - **Hierarchical policies** — pre/post rulebases with inheritance across group trees.
- **VPN** — IPsec site-to-site, OpenVPN and WireGuard, defined once and rolled out. - **VPN** — IPsec site-to-site, OpenVPN and WireGuard, defined once and rolled out.
- **Network** — interfaces, VLANs, virtual IPs, gateways, routes, LAGG/bridge/GRE, DNS and DHCP per firewall, plus tunables, cron, syslog, snapshots and diagnostics (ping, traceroute, port check, DNS), a comparison of cluster nodes, with a diff preview, a backup before every change and a guard against cutting your own connection; rules created by hand on a firewall can be adopted.
- **Certificates** — trust store for all firewalls and ACME / Let's Encrypt issuance. - **Certificates** — trust store for all firewalls and ACME / Let's Encrypt issuance.
- **Logs & audit** — central log search with correlation, dashboards and a complete deploy audit trail. - **Logs & audit** — central log search with correlation, dashboards and a complete deploy audit trail.
- **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode. - **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode.
- **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / read-only), keyboard operation, dark mode, German and English UI. - **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / read-only), keyboard operation, dark mode, German and English UI.
![Cluster view](website/assets/screenshots/en/cluster-light-desktop.webp) ![Cluster view](website/assets/screenshots/en/cluster-light-desktop.webp)
All screenshots show fictitious sample data. All screenshots show fictitious sample data.
## Guides
- [Manage OPNsense firewalls centrally](https://opnexus.dev/en/manage-opnsense-centrally/) – when central management pays off and how it works
- [Manage OPNsense HA clusters with CARP](https://opnexus.dev/en/opnsense-ha-cluster-carp/) – roles, alerts, ordered updates, comparing both nodes
- [Roll out OPNsense rules safely](https://opnexus.dev/en/roll-out-opnsense-rules-safely/) – preview, confirmation window and automatic rollback
- [The `opnexus` command-line tool](https://opnexus.dev/en/docs/cli/) – update, roll back and back up an installation
## Quick start ## Quick start
Requires Docker with the Compose plugin. Requires Docker with the Compose plugin.
@@ -48,7 +57,7 @@ For access from other machines and for production use with TLS (bundled Caddy pr
```bash ```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.10.1-rc2 OPNEXUS_VERSION=1.15.0
``` ```
```bash ```bash
@@ -57,6 +66,8 @@ docker compose pull && docker compose up -d --no-build
`scripts/opnexus update` / `rollback` then update the installation (pull first, database backup before migrations, automatic rollback if the services do not become healthy). `scripts/opnexus update` / `rollback` then update the installation (pull first, database backup before migrations, automatic rollback if the services do not become healthy).
**Updating from the web interface** is available as an opt-in: set `COMPOSE_PROFILES=updater` in `.env` and an administrator can update (and roll back) under *Settings › Update OPNexus*. It runs as a separate small container that is the only one with Docker access — see the [handbook](docs/HANDBUCH.md#update-aus-dem-webinterface-optional) (German) for the security model.
## Privacy ## Privacy
OPNexus is self-hosted and has no telemetry. It only connects to your firewalls and to notification targets you configure yourself. The optional new-version notice (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) is **off** unless you set it; when on, it makes one plain `GET` at most every 12 hours with the user agent `OPNexus/<version>` — no instance or user data. OPNexus is self-hosted and has no telemetry. It only connects to your firewalls and to notification targets you configure yourself. The optional new-version notice (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) is **off** unless you set it; when on, it makes one plain `GET` at most every 12 hours with the user agent `OPNexus/<version>` — no instance or user data.
+61 -9
View File
@@ -42,6 +42,45 @@ _READ_ONLY_ALLOWED_WRITES = {
} }
# Resource families whose URLs are /api/<family>/<collection>/<id>/<action>
# (or /api/<family>/<action> for the account/notification endpoints). The
# older kinds below keep their original /api/<kind>/<id>/<action> parsing.
_SECURITY_KINDS = {
"ipsec": "ipsec_tunnel", "openvpn": "openvpn_instance", "wireguard": "wireguard_tunnel",
"certificates": "certificate", "acme": "acme", "notifications": "notification", "auth": "auth",
}
_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings"}
def _is_uuid(value: str) -> bool:
try:
uuid.UUID(value)
return True
except ValueError:
return False
def _security_event(parts: list[str], method: str) -> dict:
"""kind/action/entity/host for VPN, certificate, notification and account-security
requests. Only path segments are used — never query strings or bodies."""
ids = [p for p in parts[2:] if _is_uuid(p)]
tail = parts[-1]
if parts[1] == "auth":
action = "/".join(parts[2:])
elif method == "DELETE":
action = "delete"
elif ids and not _is_uuid(tail):
action = tail # deploy / push / remove / register / sign / poll
elif method in ("PUT", "PATCH"):
action = "edit"
elif not ids and tail not in _COLLECTIONS and len(parts) > 2:
action = tail # e.g. notifications/test
else:
action = "create"
return {"kind": _SECURITY_KINDS[parts[1]], "action": action[:40], "entity_id": ids[0] if ids else None,
"host_id": ids[1] if len(ids) > 1 else None}
class AuthMiddleware(BaseHTTPMiddleware): class AuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next): async def dispatch(self, request: Request, call_next):
if request.url.path in _PUBLIC_PATHS: if request.url.path in _PUBLIC_PATHS:
@@ -52,25 +91,38 @@ class AuthMiddleware(BaseHTTPMiddleware):
async with SessionLocal() as db: async with SessionLocal() as db:
user = await auth_service.resolve_session(db, token) user = await auth_service.resolve_session(db, token)
if user is not None: if user is not None:
parts = request.url.path.strip("/").split("/")
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system"}
writing = request.method in ("POST", "PUT", "PATCH", "DELETE")
if len(parts) > 1 and parts[1] in _SECURITY_KINDS:
event = {**_security_event(parts, request.method), "details": {"method": request.method, "path": request.url.path}}
kind = event["kind"]
else:
kind = kinds.get(parts[1]) if len(parts) > 1 else None
action = ("delete" if request.method == "DELETE" else parts[-1] if len(parts) > 3 else "edit" if request.method in ("PUT", "PATCH") else "create")
event = {"kind": kind, "action": action, "entity_id": parts[2] if len(parts) > 2 else None,
"host_id": parts[2] if kind == "host" and len(parts) > 2 else None,
"details": {"method": request.method, "path": request.url.path}}
tracked = kind and writing and not request.url.path.endswith(("/preview", "/carp-preview", "/revisions"))
if ( if (
user.role == "read_only" user.role == "read_only"
and request.method not in ("GET", "HEAD", "OPTIONS") and request.method not in ("GET", "HEAD", "OPTIONS")
and request.url.path not in _READ_ONLY_ALLOWED_WRITES and request.url.path not in _READ_ONLY_ALLOWED_WRITES
): ):
if tracked:
# Refused attempts at security-relevant writes stay attributable.
ctx = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": event["action"]})
try:
await audit_service.persist(**event, status="denied", message="Schreibzugriff für Nur-Lese-Konto verweigert")
finally:
audit_service.context.reset(ctx)
return JSONResponse( return JSONResponse(
status_code=403, status_code=403,
content={"detail": {"code": ErrorCode.READ_ONLY_ACCESS.value, "params": {}}}, content={"detail": {"code": ErrorCode.READ_ONLY_ACCESS.value, "params": {}}},
) )
request.state.user = user request.state.user = user
parts = request.url.path.strip("/").split("/") token = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": event["action"]})
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user"}
kind = kinds.get(parts[1]) if len(parts) > 1 else None
tracked = kind and request.method in ("POST", "PUT", "PATCH", "DELETE") and not request.url.path.endswith(("/preview", "/carp-preview", "/revisions"))
action = ("delete" if request.method == "DELETE" else parts[-1] if len(parts) > 3 else "edit" if request.method in ("PUT", "PATCH") else "create")
token = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": action})
event = {"kind": kind, "action": action, "entity_id": parts[2] if len(parts) > 2 else None,
"host_id": parts[2] if kind == "host" and len(parts) > 2 else None,
"details": {"method": request.method, "path": request.url.path}}
try: try:
if tracked: if tracked:
await audit_service.persist(**event, status="requested") await audit_service.persist(**event, status="requested")
+19
View File
@@ -28,6 +28,14 @@ class Settings(BaseSettings):
update_check_url: str = "" update_check_url: str = ""
update_check_interval_hours: int = 12 update_check_interval_hours: int = 12
# ── Updates aus dem Webinterface (app/services/updater_client.py) ─────
# Optionaler Updater-Container (Compose-Profil "updater"). Das Backend hat
# nie Zugriff auf den Docker-Socket; es reicht nur "aktualisieren" /
# "zurückrollen" an diesen Dienst weiter, authentifiziert mit dem Token,
# den der Updater in sein Zustands-Volume schreibt.
updater_url: str = "http://updater:8080"
updater_token_file: str = "/updater/token"
# Cookies now carry the session, so this can no longer be a bare "*" # Cookies now carry the session, so this can no longer be a bare "*"
# (the Fetch spec forbids credentialed requests against a wildcard # (the Fetch spec forbids credentialed requests against a wildcard
# response) — see main.py's allow_origin_regex. Default only matches # response) — see main.py's allow_origin_regex. Default only matches
@@ -47,6 +55,10 @@ class Settings(BaseSettings):
opnsense_secret_dir: str = "/data/opnsense-secrets" opnsense_secret_dir: str = "/data/opnsense-secrets"
opnsense_backup_dir: str = "/data/opnsense-backups" opnsense_backup_dir: str = "/data/opnsense-backups"
opnsense_timeout: float = 10.0 opnsense_timeout: float = 10.0
# Firewall API credentials are sent as HTTP Basic auth. Plain http://
# addresses are therefore refused (on create AND on every request) unless
# this is switched on — only meant for isolated lab setups.
allow_insecure_firewall_urls: bool = False
# ── Auth ────────────────────────────────────────────────────────── # ── Auth ──────────────────────────────────────────────────────────
admin_username: str = "admin" admin_username: str = "admin"
@@ -86,6 +98,13 @@ class Settings(BaseSettings):
# calls every cycle. # calls every cycle.
notification_poll_min_interval_seconds: int = 60 notification_poll_min_interval_seconds: int = 60
notification_smtp_timeout_seconds: float = 10.0 notification_smtp_timeout_seconds: float = 10.0
# SMTP over SSL/STARTTLS always verifies the server certificate and host
# name against the container's trust store. A mail server with a private
# CA needs that CA's PEM file mounted and referenced here.
smtp_ca_file: str = ""
# Sending SMTP credentials over an unencrypted connection (security "none")
# is refused unless this is switched on explicitly.
allow_insecure_smtp_auth: bool = False
# ── Log ingest ──────────────────────────────────────────────────── # ── Log ingest ────────────────────────────────────────────────────
# Floor against an admin setting an absurdly low poll interval — same # Floor against an admin setting an absurdly low poll interval — same
+18
View File
@@ -6,6 +6,12 @@ from fastapi import HTTPException
class ErrorCode(str, Enum): class ErrorCode(str, Enum):
UPDATER_UNAVAILABLE = "UPDATER_UNAVAILABLE"
UPDATER_BUSY = "UPDATER_BUSY"
UPDATER_REJECTED = "UPDATER_REJECTED"
UPDATE_NOT_OFFERED = "UPDATE_NOT_OFFERED"
UPDATE_MAJOR_CHANGE = "UPDATE_MAJOR_CHANGE"
ADMIN_REQUIRED = "ADMIN_REQUIRED"
BACKUP_EMPTY = "BACKUP_EMPTY" BACKUP_EMPTY = "BACKUP_EMPTY"
BACKUP_INVALID_XML = "BACKUP_INVALID_XML" BACKUP_INVALID_XML = "BACKUP_INVALID_XML"
BACKUP_TOO_LARGE = "BACKUP_TOO_LARGE" BACKUP_TOO_LARGE = "BACKUP_TOO_LARGE"
@@ -59,6 +65,7 @@ class ErrorCode(str, Enum):
HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845 = "HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845" HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845 = "HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845"
HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721 = "HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721" HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721 = "HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721"
HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85 = "HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85" HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85 = "HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85"
HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN = "HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN"
HOST_IST_DEAKTIVIERT = "HOST_IST_DEAKTIVIERT" HOST_IST_DEAKTIVIERT = "HOST_IST_DEAKTIVIERT"
HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS = "HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS" HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS = "HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS"
HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B = "HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B" HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B = "HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B"
@@ -98,6 +105,13 @@ class ErrorCode(str, Enum):
REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203 = "REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203" REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203 = "REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203"
REGEL_HAT_KEINE_ZIEL_HOSTS = "REGEL_HAT_KEINE_ZIEL_HOSTS" REGEL_HAT_KEINE_ZIEL_HOSTS = "REGEL_HAT_KEINE_ZIEL_HOSTS"
REGEL_NICHT_GEFUNDEN = "REGEL_NICHT_GEFUNDEN" REGEL_NICHT_GEFUNDEN = "REGEL_NICHT_GEFUNDEN"
NETWORK_ACK_REQUIRED = "NETWORK_ACK_REQUIRED"
NETWORK_APPLY_UNCERTAIN = "NETWORK_APPLY_UNCERTAIN"
NETWORK_BACKUP_FAILED = "NETWORK_BACKUP_FAILED"
NETWORK_CHANGE_BLOCKED = "NETWORK_CHANGE_BLOCKED"
NETWORK_FIELD_INVALID = "NETWORK_FIELD_INVALID"
NETWORK_ITEM_NOT_FOUND = "NETWORK_ITEM_NOT_FOUND"
RULE_NOT_ADOPTABLE = "RULE_NOT_ADOPTABLE"
REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0 = "REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0" REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0 = "REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0"
RESTORE_VORGANG_NICHT_GEFUNDEN = "RESTORE_VORGANG_NICHT_GEFUNDEN" RESTORE_VORGANG_NICHT_GEFUNDEN = "RESTORE_VORGANG_NICHT_GEFUNDEN"
ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL = "ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL" ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL = "ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL"
@@ -133,6 +147,10 @@ class ErrorCode(str, Enum):
OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED" OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED"
OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY" OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY" OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY"
UPSTREAM_REJECTED = "UPSTREAM_REJECTED"
INVALID_CHARACTERS = "INVALID_CHARACTERS"
SERVICE_NOT_FOUND = "SERVICE_NOT_FOUND"
SERVICE_LOCKED = "SERVICE_LOCKED"
READ_ONLY_ACCESS = "READ_ONLY_ACCESS" READ_ONLY_ACCESS = "READ_ONLY_ACCESS"
+46 -1
View File
@@ -3,8 +3,13 @@ import logging
import re import re
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from fastapi import FastAPI from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from sqlalchemy.exc import DBAPIError
from app.errors import ErrorCode
from app import models # noqa: F401 - registers models with Base.metadata from app import models # noqa: F401 - registers models with Base.metadata
from app.auth_middleware import AuthMiddleware from app.auth_middleware import AuthMiddleware
@@ -31,6 +36,10 @@ from app.routers import ipsec as ipsec_router
from app.routers import openvpn as openvpn_router from app.routers import openvpn as openvpn_router
from app.routers import wireguard as wireguard_router from app.routers import wireguard as wireguard_router
from app.routers import certificates as certificates_router from app.routers import certificates as certificates_router
from app.routers import network as network_router
from app.routers import snapshots as snapshots_router
from app.routers import network_drift as network_drift_router
from app.routers import diagnostics as diagnostics_router
from app.routers import restore as restore_router from app.routers import restore as restore_router
from app.version import APP_VERSION from app.version import APP_VERSION
from app.routers import dashboard as dashboard_router from app.routers import dashboard as dashboard_router
@@ -245,6 +254,13 @@ async def lifespan(app: FastAPI):
await migrate.run() await migrate.run()
async with SessionLocal() as db: async with SessionLocal() as db:
await auth_service.bootstrap_admin(db) await auth_service.bootstrap_admin(db)
if any(o.startswith("https://") for o in settings.allowed_origins) and not settings.cookie_secure:
logger.warning(
"FRONTEND_ORIGIN is https:// but COOKIE_SECURE is false — the session cookie "
"would also be sent over plain HTTP. Set COOKIE_SECURE=true."
)
if settings.allow_insecure_firewall_urls:
logger.warning("ALLOW_INSECURE_FIREWALL_URLS is on: firewall API credentials may travel over plain HTTP.")
sweep_task = asyncio.create_task(_deploy_revert_sweep_loop()) sweep_task = asyncio.create_task(_deploy_revert_sweep_loop())
notification_task = asyncio.create_task(_notification_poll_loop()) notification_task = asyncio.create_task(_notification_poll_loop())
@@ -268,6 +284,31 @@ async def lifespan(app: FastAPI):
app = FastAPI(title="OPNexus", version=APP_VERSION, lifespan=lifespan) app = FastAPI(title="OPNexus", version=APP_VERSION, lifespan=lifespan)
@app.exception_handler(RequestValidationError)
async def _safe_validation_error(request: Request, exc: RequestValidationError):
"""FastAPI's default 422 body echoes the offending `input` (and validator
`ctx`) back — for password/key/PSK fields that puts the secret into
responses that browsers, proxies and error telemetry may record. Only
where the problem is and what it is are returned."""
return JSONResponse(
status_code=422,
content={"detail": [{"loc": list(err.get("loc", ())), "msg": err.get("msg", ""), "type": err.get("type", "")} for err in exc.errors()]},
)
@app.exception_handler(DBAPIError)
async def _database_text_errors(request: Request, exc: DBAPIError):
"""A NUL byte in any text field cannot be stored by Postgres and used to
surface as a 500 plus a dropped connection (found by fuzzing). That is bad
input, not a server fault. Anything else stays a normal 500."""
if type(exc.orig).__name__.endswith("CharacterNotInRepertoireError") or "0x00" in str(exc.orig):
return JSONResponse(
status_code=422,
content={"detail": {"code": ErrorCode.INVALID_CHARACTERS.value, "params": {}}},
)
raise exc
app.add_middleware(AuthMiddleware) app.add_middleware(AuthMiddleware)
# Inside CORS so preflights are answered first; outside Auth so a forged # Inside CORS so preflights are answered first; outside Auth so a forged
# cross-site write is rejected before any session lookup happens. # cross-site write is rejected before any session lookup happens.
@@ -353,5 +394,9 @@ app.include_router(restore_router.router, prefix="/api")
app.include_router(dashboard_router.router, prefix="/api") app.include_router(dashboard_router.router, prefix="/api")
app.include_router(cluster_router.router, prefix="/api") app.include_router(cluster_router.router, prefix="/api")
app.include_router(services_router.router, prefix="/api") app.include_router(services_router.router, prefix="/api")
app.include_router(network_router.router, prefix="/api")
app.include_router(snapshots_router.router, prefix="/api")
app.include_router(network_drift_router.router, prefix="/api")
app.include_router(diagnostics_router.router, prefix="/api")
app.include_router(resource_samples_router.router, prefix="/api") app.include_router(resource_samples_router.router, prefix="/api")
app.include_router(scheduled_maintenance_router.router, prefix="/api") app.include_router(scheduled_maintenance_router.router, prefix="/api")
+1
View File
@@ -48,6 +48,7 @@ class NotificationSettings(Base):
maintenance_unknown_enabled: Mapped[bool] = mapped_column(Boolean, default=True) maintenance_unknown_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
carp_health_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true") carp_health_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
nat_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true") nat_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
network_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
carp_auto_revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true") carp_auto_revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300) poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300)
+2 -1
View File
@@ -1,7 +1,7 @@
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, String from sqlalchemy import Boolean, DateTime, String, false
from sqlalchemy.dialects.postgresql import UUID as PGUUID from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.orm import Mapped, mapped_column, relationship
@@ -29,6 +29,7 @@ class FirewallRule(Base):
destination_net: Mapped[str] = mapped_column(String(128), default="any") destination_net: Mapped[str] = mapped_column(String(128), default="any")
destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True) destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
enabled: Mapped[bool] = mapped_column(Boolean, default=True) enabled: Mapped[bool] = mapped_column(Boolean, default=True)
log: Mapped[bool] = mapped_column(Boolean, default=False, server_default=false())
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc) DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
) )
+1 -1
View File
@@ -146,7 +146,7 @@ async def delete_alias(alias_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
await undeploy_alias(db, alias) await undeploy_alias(db, alias)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(alias, attribute_names=["deployments"]) await db.refresh(alias, attribute_names=["deployments"])
await db.delete(alias) await db.delete(alias)
await db.commit() await db.commit()
+72 -17
View File
@@ -2,6 +2,7 @@ import logging
from fastapi import APIRouter, Depends, Request, Response, status from fastapi import APIRouter, Depends, Request, Response, status
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error from app.errors import ErrorCode, api_error
@@ -45,6 +46,22 @@ def _clear_session_cookie(response: Response) -> None:
response.delete_cookie(key=COOKIE_NAME, path="/") response.delete_cookie(key=COOKIE_NAME, path="/")
def _check_account_budget(user: AdminUser, action: str) -> None:
"""429 once too many failed password/code attempts piled up for this account
and action (see auth_service.account_throttle)."""
wait = auth_service.account_throttle.retry_after(str(user.id), action)
if wait > 0:
raise api_error(status.HTTP_429_TOO_MANY_REQUESTS, ErrorCode.TOO_MANY_LOGIN_ATTEMPTS, wait=wait)
def _account_failure(user: AdminUser, action: str) -> None:
auth_service.account_throttle.record_failure(str(user.id), action)
def _account_success(user: AdminUser, action: str) -> None:
auth_service.account_throttle.reset(str(user.id), action)
async def resolve_user(request: Request, db: AsyncSession) -> AdminUser: async def resolve_user(request: Request, db: AsyncSession) -> AdminUser:
token = request.cookies.get(COOKIE_NAME, "") token = request.cookies.get(COOKIE_NAME, "")
user = await auth_service.resolve_session(db, token) user = await auth_service.resolve_session(db, token)
@@ -84,6 +101,7 @@ async def login(payload: LoginRequest, request: Request, response: Response, db:
logger.info("Failed login for '%s' from %s", payload.username, ip) logger.info("Failed login for '%s' from %s", payload.username, ip)
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
verified_hash = user.password_hash
if user.totp_enabled: if user.totp_enabled:
# Only revealed after the password was correct, so it cannot be used # Only revealed after the password was correct, so it cannot be used
# to find out which usernames exist or have 2FA. # to find out which usernames exist or have 2FA.
@@ -98,7 +116,7 @@ async def login(payload: LoginRequest, request: Request, response: Response, db:
auth_service.throttle.reset(ip, payload.username) auth_service.throttle.reset(ip, payload.username)
failed_since_last_login = auth_service.throttle.pop_failures_since_success(payload.username) failed_since_last_login = auth_service.throttle.pop_failures_since_success(payload.username)
token, expires_at = await auth_service.create_session( token, expires_at = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip, verified_hash=verified_hash
) )
_set_session_cookie(response, token) _set_session_cookie(response, token)
logger.info("Successful login for '%s' from %s", user.username, ip) logger.info("Successful login for '%s' from %s", user.username, ip)
@@ -114,20 +132,33 @@ async def login(payload: LoginRequest, request: Request, response: Response, db:
async def _accept_second_factor(db: AsyncSession, user: AdminUser, code: str) -> bool: async def _accept_second_factor(db: AsyncSession, user: AdminUser, code: str) -> bool:
"""Authenticator code (replay-protected) or, failing that, a one-time recovery code.""" """Authenticator code (replay-protected) or, failing that, a one-time recovery code.
secret = totp_service.read_secret(user.id)
A code is single-use only if "check, then mark as used" cannot interleave
with a second request carrying the same code. The user row is therefore
re-read under a row lock (FOR UPDATE, fresh values) and the used step /
consumed recovery hash is committed while holding it; a concurrent attempt
blocks here and then sees the updated state.
"""
locked = await db.scalar(
select(AdminUser).where(AdminUser.id == user.id).with_for_update().execution_options(populate_existing=True)
)
if locked is None:
return False
secret = totp_service.read_secret(locked.id)
if secret: if secret:
step = totp_service.verify_code(secret, code, user.totp_last_step) step = totp_service.verify_code(secret, code, locked.totp_last_step)
if step is not None: if step is not None:
user.totp_last_step = step locked.totp_last_step = step
await db.commit() await db.commit()
return True return True
remaining = totp_service.consume_recovery(user.totp_recovery_hashes, code) remaining = totp_service.consume_recovery(locked.totp_recovery_hashes, code)
if remaining is not None: if remaining is not None:
user.totp_recovery_hashes = remaining locked.totp_recovery_hashes = remaining
await db.commit() await db.commit()
logger.warning("Recovery code used by '%s' (%d left)", user.username, len(remaining)) logger.warning("Recovery code used by '%s' (%d left)", locked.username, len(remaining))
return True return True
await db.commit() # nothing changed — just release the row lock
return False return False
@@ -156,45 +187,66 @@ async def totp_setup(payload: TotpSetupRequest, user: AdminUser = Depends(curren
active until /2fa/enable proves the authenticator works.""" active until /2fa/enable proves the authenticator works."""
if user.totp_enabled: if user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED) raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED)
if not auth_service.verify_password(user.password_hash, payload.password): _check_account_budget(user, "2fa-setup")
if not await auth_service.verify_password_async(user.password_hash, payload.password):
_account_failure(user, "2fa-setup")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
_account_success(user, "2fa-setup")
secret = totp_service.generate_secret() secret = totp_service.generate_secret()
totp_service.save_secret(user.id, secret, pending=True) totp_service.save_secret(user.id, secret, pending=True)
return {"secret": secret, "otpauth_uri": totp_service.otpauth_uri(user.username, secret)} return {"secret": secret, "otpauth_uri": totp_service.otpauth_uri(user.username, secret)}
@router.post("/2fa/enable") @router.post("/2fa/enable")
async def totp_enable(payload: TotpEnableRequest, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)): async def totp_enable(
payload: TotpEnableRequest, request: Request,
db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user),
):
if user.totp_enabled: if user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED) raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED)
_check_account_budget(user, "2fa-enable")
pending = totp_service.read_secret(user.id, pending=True) pending = totp_service.read_secret(user.id, pending=True)
if not pending: if not pending:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_SETUP_MISSING) raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_SETUP_MISSING)
step = totp_service.verify_code(pending, payload.code) step = totp_service.verify_code(pending, payload.code)
if step is None: if step is None:
# Not reset by a fresh /2fa/setup: recreating the pending enrolment
# must not hand out a new guessing budget.
_account_failure(user, "2fa-enable")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
_account_success(user, "2fa-enable")
plain, hashes = totp_service.new_recovery_codes() plain, hashes = totp_service.new_recovery_codes()
totp_service.promote_pending(user.id) totp_service.promote_pending(user.id)
user.totp_enabled = True user.totp_enabled = True
user.totp_last_step = step user.totp_last_step = step
user.totp_recovery_hashes = hashes user.totp_recovery_hashes = hashes
# Other sessions were established without the second factor — end them.
await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
await db.commit() await db.commit()
logger.info("Two-factor enabled for '%s'", user.username) logger.info("Two-factor enabled for '%s'", user.username)
return {"enabled": True, "recovery_codes": plain} return {"enabled": True, "recovery_codes": plain}
@router.post("/2fa/disable") @router.post("/2fa/disable")
async def totp_disable(payload: TotpDisableRequest, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)): async def totp_disable(
payload: TotpDisableRequest, request: Request,
db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user),
):
if not user.totp_enabled: if not user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_NOT_ENABLED) raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_NOT_ENABLED)
if not auth_service.verify_password(user.password_hash, payload.password): _check_account_budget(user, "2fa-disable")
if not await auth_service.verify_password_async(user.password_hash, payload.password):
_account_failure(user, "2fa-disable")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
if not await _accept_second_factor(db, user, payload.code): if not await _accept_second_factor(db, user, payload.code):
_account_failure(user, "2fa-disable")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
_account_success(user, "2fa-disable")
totp_service.delete_secrets(user.id) totp_service.delete_secrets(user.id)
user.totp_enabled = False user.totp_enabled = False
user.totp_recovery_hashes = None user.totp_recovery_hashes = None
user.totp_last_step = 0 user.totp_last_step = 0
await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
await db.commit() await db.commit()
logger.info("Two-factor disabled for '%s'", user.username) logger.info("Two-factor disabled for '%s'", user.username)
return {"enabled": False} return {"enabled": False}
@@ -225,15 +277,18 @@ async def change_password(
db: AsyncSession = Depends(get_db), db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user), user: AdminUser = Depends(current_user),
): ):
if not auth_service.verify_password(user.password_hash, payload.current_password): _check_account_budget(user, "change-password")
if not await auth_service.verify_password_async(user.password_hash, payload.current_password):
_account_failure(user, "change-password")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH) raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
_account_success(user, "change-password")
if payload.new_password == payload.current_password: if payload.new_password == payload.current_password:
raise api_error(status.HTTP_400_BAD_REQUEST, ErrorCode.DAS_NEUE_PASSWORT_MUSS_SICH_VOM_AKTUELLEN_UNTERSCHEIDEN) raise api_error(status.HTTP_400_BAD_REQUEST, ErrorCode.DAS_NEUE_PASSWORT_MUSS_SICH_VOM_AKTUELLEN_UNTERSCHEIDEN)
user.password_hash = auth_service.hash_password(payload.new_password) # New hash, revocation of EVERY existing session (including other devices
await db.commit() # an attacker may hold) and the replacement session commit as one unit.
user.password_hash = await auth_service.hash_password_async(payload.new_password)
await auth_service.revoke_session(db, request.cookies.get(COOKIE_NAME, "")) await auth_service.revoke_sessions(db, user.id, commit=False)
token, _ = await auth_service.create_session( token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=client_ip(request) db, user, user_agent=request.headers.get("User-Agent"), ip_address=client_ip(request)
) )
+3 -3
View File
@@ -112,7 +112,7 @@ async def delete_managed_certificate(cert_id: uuid.UUID, db: AsyncSession = Depe
try: try:
await managed_certificate_service.delete_certificate(db, cert, host) await managed_certificate_service.delete_certificate(db, cert, host)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
# ── ACME accounts ────────────────────────────────────────────────────── # ── ACME accounts ──────────────────────────────────────────────────────
@@ -197,7 +197,7 @@ async def delete_acme_account(account_id: uuid.UUID, db: AsyncSession = Depends(
try: try:
await acme_service.delete_account(db, account, host) await acme_service.delete_account(db, account, host)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
# ── ACME certificate requests ───────────────────────────────────────── # ── ACME certificate requests ─────────────────────────────────────────
@@ -298,4 +298,4 @@ async def delete_acme_certificate_request(request_id: uuid.UUID, db: AsyncSessio
try: try:
await acme_service.delete_certificate_request(db, req, host) await acme_service.delete_certificate_request(db, req, host)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
+68
View File
@@ -0,0 +1,68 @@
import uuid
from typing import Literal
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.services import diagnostics_service
router = APIRouter(prefix="/hosts/{host_id}/diagnostics", tags=["diagnostics"])
IpProto = Literal["inet", "inet6"]
class PingRequest(BaseModel):
target: str = Field(min_length=1, max_length=253)
ipproto: IpProto = "inet"
count: int = Field(default=4, ge=1, le=10)
packetsize: int | None = Field(default=None, ge=1, le=9000)
source: str | None = Field(default=None, max_length=64)
class TracerouteRequest(BaseModel):
target: str = Field(min_length=1, max_length=253)
ipproto: IpProto = "inet"
protocol: Literal["udp", "icmp"] = "icmp"
source: str | None = Field(default=None, max_length=64)
class PortprobeRequest(BaseModel):
target: str = Field(min_length=1, max_length=253)
port: int = Field(ge=1, le=65535)
ipproto: IpProto = "inet"
source: str | None = Field(default=None, max_length=64)
class DnsRequest(BaseModel):
name: str = Field(min_length=1, max_length=253)
server: str | None = Field(default=None, max_length=64)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.post("/ping")
async def ping(host_id: uuid.UUID, p: PingRequest, db: AsyncSession = Depends(get_db)):
return await diagnostics_service.ping(await _host(db, host_id), p.target, p.ipproto, p.count, p.packetsize, p.source)
@router.post("/traceroute")
async def traceroute(host_id: uuid.UUID, p: TracerouteRequest, db: AsyncSession = Depends(get_db)):
return await diagnostics_service.traceroute(await _host(db, host_id), p.target, p.ipproto, p.protocol, p.source)
@router.post("/portprobe")
async def portprobe(host_id: uuid.UUID, p: PortprobeRequest, db: AsyncSession = Depends(get_db)):
return await diagnostics_service.portprobe(await _host(db, host_id), p.target, p.port, p.ipproto, p.source)
@router.post("/dns")
async def dns(host_id: uuid.UUID, p: DnsRequest, db: AsyncSession = Depends(get_db)):
return await diagnostics_service.dns_lookup(await _host(db, host_id), p.name, p.server)
+47 -2
View File
@@ -10,11 +10,20 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db from app.database import get_db
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.services import audit_service
from app.services.policy_service import policy_lock from app.services.policy_service import policy_lock
from app.models.host import OpnsenseHost from app.models.host import OpnsenseHost
from app.models.policy_hierarchy_group import PolicyHierarchyGroup from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.models.rule_deployment import RuleDeployment from app.models.rule_deployment import RuleDeployment
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.models.openvpn_instance_deployment import OpenVpnInstanceDeployment
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
from app.models.managed_certificate import ManagedCertificate
from app.models.acme_account import AcmeAccount
from app.models.acme_certificate_request import AcmeCertificateRequest
from app.models.alias_deployment import AliasDeployment from app.models.alias_deployment import AliasDeployment
from app.schemas.host import AliasOption, FirmwareInfo, HostBackup, HostCa, HostCertificate, HostCreate, HostRead, HostUpdate, TlsProbeRequest, TlsProbeResult, InterfaceOption, LiveRule from app.schemas.host import AliasOption, FirmwareInfo, HostBackup, HostCa, HostCertificate, HostCreate, HostRead, HostUpdate, TlsProbeRequest, TlsProbeResult, InterfaceOption, LiveRule
from app.services import backup_store, opnsense_client, opnsense_secret_store, tls_pin_service from app.services import backup_store, opnsense_client, opnsense_secret_store, tls_pin_service
@@ -146,6 +155,8 @@ async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession
if existing: if existing:
raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS) raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
host.name = payload.name host.name = payload.name
if payload.base_url is not None:
host.base_url = payload.base_url
if payload.verify_tls is not None: if payload.verify_tls is not None:
host.verify_tls = payload.verify_tls host.verify_tls = payload.verify_tls
if payload.clear_tls_pin: if payload.clear_tls_pin:
@@ -192,13 +203,35 @@ async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85) raise api_error(409, ErrorCode.HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85)
if await db.scalar(select(PolicyHierarchyRuleDeployment.id).where(PolicyHierarchyRuleDeployment.host_id == host_id).limit(1)): if await db.scalar(select(PolicyHierarchyRuleDeployment.id).where(PolicyHierarchyRuleDeployment.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845) raise api_error(409, ErrorCode.HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845)
# VPN tunnels, certificates and ACME objects likewise cascade away with the host
# while staying live on the firewall: block until they are removed there.
for deployment_model in (IpsecTunnelDeployment, OpenVpnInstanceDeployment, WireguardTunnelDeployment):
if await db.scalar(select(deployment_model.id).where(deployment_model.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN)
for cert_model in (ManagedCertificate, AcmeAccount, AcmeCertificateRequest):
# A never-deployed draft only exists locally and may go with the host.
if await db.scalar(
select(cert_model.id).where(
cert_model.host_id == host_id,
(cert_model.opnsense_uuid.is_not(None)) | (cert_model.status == "success"),
).limit(1)
):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN)
from app.models.host_operation import HostOperation from app.models.host_operation import HostOperation
from app.services.host_maintenance_service import ACTIVE from app.services.host_maintenance_service import ACTIVE
if await db.scalar(select(HostOperation.id).where(HostOperation.host_id == host_id, HostOperation.status.in_(ACTIVE)).limit(1)): if await db.scalar(select(HostOperation.id).where(HostOperation.host_id == host_id, HostOperation.status.in_(ACTIVE)).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0) raise api_error(409, ErrorCode.HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0)
# Local drafts cascade away with the host; their file-backed secrets must not outlive them.
from app.services import acme_secret_store, cert_key_store
draft_certs = (await db.scalars(select(ManagedCertificate.id).where(ManagedCertificate.host_id == host_id))).all()
draft_accounts = (await db.scalars(select(AcmeAccount.id).where(AcmeAccount.host_id == host_id))).all()
await db.delete(host) await db.delete(host)
await db.commit() await db.commit()
opnsense_secret_store.delete_secret(host_id) opnsense_secret_store.delete_secret(host_id)
for cert_id in draft_certs:
cert_key_store.delete_secret(cert_id)
for account_id in draft_accounts:
acme_secret_store.delete_secret(account_id)
@router.get("/{host_id}/live-rules", response_model=list[LiveRule]) @router.get("/{host_id}/live-rules", response_model=list[LiveRule])
@@ -246,11 +279,23 @@ async def list_host_backups(host_id: uuid.UUID, db: AsyncSession = Depends(get_d
@router.get("/{host_id}/backups/{backup_id}") @router.get("/{host_id}/backups/{backup_id}")
async def download_host_backup(host_id: uuid.UUID, backup_id: str, db: AsyncSession = Depends(get_db)): async def download_host_backup(
await _get_host_or_404(db, host_id) host_id: uuid.UUID, backup_id: str,
db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user),
):
"""Raw config.xml snapshot — contains password hashes, VPN keys and other
firewall secrets, so unlike the (metadata-only) listing it is not part of
the "read_only may GET everything" allowance. Every attempt is audited
(who/which snapshot, never the content)."""
host = await _get_host_or_404(db, host_id)
event = {"kind": "backup", "action": "download", "entity_id": backup_id[:36], "host_id": str(host_id), "host_name": host.name}
if user.role != "admin":
await audit_service.persist(**event, status="denied", message="Rohdownload nur für Administratoren")
raise api_error(403, ErrorCode.READ_ONLY_ACCESS)
content = backup_store.read_snapshot(host_id, backup_id) content = backup_store.read_snapshot(host_id, backup_id)
if content is None: if content is None:
raise api_error(404, ErrorCode.BACKUP_NICHT_GEFUNDEN) raise api_error(404, ErrorCode.BACKUP_NICHT_GEFUNDEN)
await audit_service.persist(**event, status="accepted")
return Response( return Response(
content=content, content=content,
media_type="application/xml", media_type="application/xml",
+4 -4
View File
@@ -1,6 +1,6 @@
import uuid import uuid
from fastapi import APIRouter, Depends, Body from fastapi import APIRouter, Depends
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
@@ -151,10 +151,10 @@ async def list_revisions(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db
@router.post("/{tunnel_id}/deploy", response_model=IpsecTunnelRead) @router.post("/{tunnel_id}/deploy", response_model=IpsecTunnelRead)
async def deploy(tunnel_id: uuid.UUID, approval: IpsecDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)): async def deploy(tunnel_id: uuid.UUID, approval: IpsecDeployApproval, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id) tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel) plan = preview(tunnel)
if approval and approval.preview_token != plan["token"]: if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN) raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if not tunnel.pre_shared_key_set: if not tunnel.pre_shared_key_set:
raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_PRE_SHARED_4AE1FC63) raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_PRE_SHARED_4AE1FC63)
@@ -174,7 +174,7 @@ async def delete_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)
try: try:
await undeploy_tunnel(db, tunnel) await undeploy_tunnel(db, tunnel)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(tunnel, attribute_names=["deployments"]) await db.refresh(tunnel, attribute_names=["deployments"])
ipsec_psk_store.delete_secret(tunnel.id) ipsec_psk_store.delete_secret(tunnel.id)
await db.delete(tunnel) await db.delete(tunnel)
+90
View File
@@ -0,0 +1,90 @@
import re
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.routers.hosts import _upstream_error
from app.schemas.network import (
HostNetwork, NetworkChangeExecute, NetworkChangePreview, NetworkChangeRequest, NetworkChangeResult,
)
from app.services import network_change_service, network_service
from app.services.policy_service import policy_lock
from app.services.opnsense_client import OpnsenseAPIError
router = APIRouter(prefix="/hosts/{host_id}/network", tags=["network"])
@router.get("", response_model=HostNetwork)
async def host_network(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return await network_service.get_host_network(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
async def _host_or_404(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.post("/changes/preview", response_model=NetworkChangePreview)
async def preview_change(host_id: uuid.UUID, payload: NetworkChangeRequest, db: AsyncSession = Depends(get_db)):
host = await _host_or_404(db, host_id)
try:
plan = await network_change_service.preview(db, host, payload.kind, payload.action, payload.uuid, payload.fields)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
return {k: v for k, v in plan.items() if not k.startswith("_")}
@router.post("/changes", response_model=NetworkChangeResult)
async def execute_change(host_id: uuid.UUID, payload: NetworkChangeExecute, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host_or_404(db, host_id)
return await network_change_service.execute(
db, host, payload.kind, payload.action, payload.uuid, payload.fields, payload.preview_token, payload.acknowledge)
@router.get("/template/{kind}")
async def change_template(host_id: uuid.UUID, kind: str, db: AsyncSession = Depends(get_db)):
"""Blank form for one kind, with the choices this firewall accepts for its select fields."""
if kind not in network_change_service.FIELDS:
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
host = await _host_or_404(db, host_id)
try:
return await network_change_service.template(host, kind)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/item/{kind}/{item_uuid}")
async def current_item(host_id: uuid.UUID, kind: str, item_uuid: str, db: AsyncSession = Depends(get_db)):
"""The object's current values on the firewall, for pre-filling an edit form."""
if kind not in network_change_service.FIELDS or not re.fullmatch(r"[A-Za-z0-9_.:-]{1,64}", item_uuid):
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
host = await _host_or_404(db, host_id)
try:
return await network_change_service.item(host, kind, item_uuid)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/objects/{kind}")
async def list_objects(host_id: uuid.UUID, kind: str, db: AsyncSession = Depends(get_db)):
"""Rows of a generic object kind (tunables, cron jobs, LAGG, ...), reduced to their display columns."""
if kind not in network_change_service.OBJECT_COLUMNS:
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
host = await _host_or_404(db, host_id)
try:
return await network_change_service.list_objects(host, kind)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
+29
View File
@@ -0,0 +1,29 @@
from collections import defaultdict
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.host import OpnsenseHost
from app.services import network_drift_service
router = APIRouter(prefix="/network", tags=["network"])
@router.get("/drift")
async def network_drift(db: AsyncSession = Depends(get_db)):
"""Per cluster: differences in the network configuration between its nodes (live read of every node)."""
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all()
by_cluster: dict[str, list[OpnsenseHost]] = defaultdict(list)
for host in hosts:
if host.cluster_name:
by_cluster[host.cluster_name].append(host)
clusters = []
for name, members in sorted(by_cluster.items()):
if len(members) < 2:
continue
differences, compared, unreachable = await network_drift_service.compute_cluster(members)
clusters.append({"name": name, "hosts": [h.name for h in members], "compared": compared,
"unreachable": unreachable, "differences": differences})
return {"clusters": clusters}
+22 -2
View File
@@ -1,9 +1,13 @@
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, Query from fastapi import APIRouter, Depends, Query
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error from app.errors import ErrorCode, api_error
from app.database import get_db from app.database import get_db
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.models.notification import NotificationProblem from app.models.notification import NotificationProblem
from app.schemas.notification import ( from app.schemas.notification import (
NotificationProblemRead, NotificationProblemRead,
@@ -16,9 +20,25 @@ from app.services import notification_delivery, notification_settings_service
router = APIRouter(prefix="/notifications", tags=["notifications"]) router = APIRouter(prefix="/notifications", tags=["notifications"])
def _redact_url(url: str | None) -> str | None:
"""Webhook URLs very often carry the credential itself (a token in the path or
query), so anyone who can read the URL can post to the receiver. Readers see
only where it points."""
if not url:
return url
parts = urlsplit(url)
if not parts.hostname:
return "…"
return f"{parts.scheme}://{parts.hostname}{f':{parts.port}' if parts.port else ''}/…"
@router.get("/settings", response_model=NotificationSettingsRead) @router.get("/settings", response_model=NotificationSettingsRead)
async def read_settings(db: AsyncSession = Depends(get_db)): async def read_settings(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
return await notification_settings_service.get_settings(db) row = await notification_settings_service.get_settings(db)
result = NotificationSettingsRead.model_validate(row)
if user.role != "admin":
result = result.model_copy(update={"webhook_url": _redact_url(result.webhook_url)})
return result
@router.put("/settings", response_model=NotificationSettingsRead) @router.put("/settings", response_model=NotificationSettingsRead)
+2 -2
View File
@@ -168,7 +168,7 @@ async def delete_one_to_one_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(
try: try:
await undeploy_one_to_one(db, rule) await undeploy_one_to_one(db, rule)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(rule, attribute_names=["deployments"]) await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule) await db.delete(rule)
await db.commit() await db.commit()
@@ -268,4 +268,4 @@ async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
return split_rows(await opnsense_client.search_one_to_one_rules(host)) return split_rows(await opnsense_client.search_one_to_one_rules(host))
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
+4 -4
View File
@@ -1,6 +1,6 @@
import uuid import uuid
from fastapi import APIRouter, Depends, Body from fastapi import APIRouter, Depends
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
@@ -153,10 +153,10 @@ async def list_revisions(instance_id: uuid.UUID, db: AsyncSession = Depends(get_
@router.post("/{instance_id}/deploy", response_model=OpenVpnInstanceRead) @router.post("/{instance_id}/deploy", response_model=OpenVpnInstanceRead)
async def deploy(instance_id: uuid.UUID, approval: OpenVpnDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)): async def deploy(instance_id: uuid.UUID, approval: OpenVpnDeployApproval, db: AsyncSession = Depends(get_db)):
instance = await _load_instance(db, instance_id) instance = await _load_instance(db, instance_id)
plan = preview(instance) plan = preview(instance)
if approval and approval.preview_token != plan["token"]: if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN) raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if instance.tls_key_enabled and not instance.tls_key_material_set: if instance.tls_key_enabled and not instance.tls_key_material_set:
raise api_error(400, ErrorCode.TLS_HARDENING_IST_AKTIVIERT_ABER_ES_WURDE_NOCH_F53C81DD) raise api_error(400, ErrorCode.TLS_HARDENING_IST_AKTIVIERT_ABER_ES_WURDE_NOCH_F53C81DD)
@@ -176,7 +176,7 @@ async def delete_instance(instance_id: uuid.UUID, db: AsyncSession = Depends(get
try: try:
await undeploy_instance(db, instance) await undeploy_instance(db, instance)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(instance, attribute_names=["deployments"]) await db.refresh(instance, attribute_names=["deployments"])
openvpn_key_store.delete_secret(instance.id) openvpn_key_store.delete_secret(instance.id)
await db.delete(instance) await db.delete(instance)
+2 -2
View File
@@ -175,7 +175,7 @@ async def delete_outbound_nat_rule(rule_id: uuid.UUID, db: AsyncSession = Depend
try: try:
await undeploy_outbound_nat(db, rule) await undeploy_outbound_nat(db, rule)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(rule, attribute_names=["deployments"]) await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule) await db.delete(rule)
await db.commit() await db.commit()
@@ -275,7 +275,7 @@ async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
return split_rows(await opnsense_client.search_outbound_nat_rules(host)) return split_rows(await opnsense_client.search_outbound_nat_rules(host))
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
@router.get("/hosts/{host_id}/mode") @router.get("/hosts/{host_id}/mode")
+1 -1
View File
@@ -294,7 +294,7 @@ async def delete_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
await undeploy_hierarchy_rule(db, rule) await undeploy_hierarchy_rule(db, rule)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(rule, attribute_names=["deployments"]) await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule) await db.delete(rule)
await db.commit() await db.commit()
+2 -2
View File
@@ -170,7 +170,7 @@ async def delete_port_forward_rule(rule_id: uuid.UUID, db: AsyncSession = Depend
try: try:
await undeploy_port_forward(db, rule) await undeploy_port_forward(db, rule)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(rule, attribute_names=["deployments"]) await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule) await db.delete(rule)
await db.commit() await db.commit()
@@ -270,4 +270,4 @@ async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
return split_rows(await opnsense_client.search_port_forward_rules(host)) return split_rows(await opnsense_client.search_port_forward_rules(host))
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
+41 -1
View File
@@ -17,6 +17,8 @@ from app.services.alias_service import check_alias_consistency, describe_alias_c
from app.services.interface_service import check_interface_consistency from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError from app.services.opnsense_client import OpnsenseAPIError
from app.services.rule_deploy_service import confirm_rule, deploy_rule, undeploy_rule, revert_deployment from app.services.rule_deploy_service import confirm_rule, deploy_rule, undeploy_rule, revert_deployment
from app.services import rule_adopt_service
from app.schemas.rule import AdoptRequest, UnmanagedRule
router = APIRouter(prefix="/rules", tags=["rules"]) router = APIRouter(prefix="/rules", tags=["rules"])
@@ -59,6 +61,7 @@ def _to_read(
destination_net=rule.destination_net, destination_net=rule.destination_net,
destination_port=rule.destination_port, destination_port=rule.destination_port,
enabled=rule.enabled, enabled=rule.enabled,
log=rule.log,
created_at=rule.created_at, created_at=rule.created_at,
updated_at=rule.updated_at, updated_at=rule.updated_at,
target_host_ids=[host.id for host in rule.targets], target_host_ids=[host.id for host in rule.targets],
@@ -84,6 +87,42 @@ async def list_rules(db: AsyncSession = Depends(get_db)):
return [_to_read(rule, rule.targets, rule.deployments) for rule in rules] return [_to_read(rule, rule.targets, rule.deployments) for rule in rules]
@router.get("/unmanaged", response_model=list[UnmanagedRule])
async def list_unmanaged_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""Rules found on the host that OPNexus does not manage yet (live read)."""
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return await rule_adopt_service.list_unmanaged(db, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
@router.post("/adopt", response_model=RuleRead, status_code=201)
async def adopt_rule(payload: AdoptRequest, db: AsyncSession = Depends(get_db)):
"""Takes over a hand-made rule. Writes nothing to the firewall."""
await policy_lock(db)
host = await db.get(OpnsenseHost, payload.host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid)
except rule_adopt_service.RuleNotFoundOnHost as exc:
raise api_error(404, ErrorCode.REGEL_NICHT_GEFUNDEN) from exc
except rule_adopt_service.NotAdoptable as exc:
raise api_error(409, ErrorCode.RULE_NOT_ADOPTABLE, reasons=", ".join(exc.reasons)) from exc
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
await db.commit()
# The new object sits in the session without its relationships loaded; db.get() would return it as-is.
rule = (await db.execute(
select(FirewallRule).where(FirewallRule.id == rule.id).options(*_load_options())
.execution_options(populate_existing=True)
)).scalar_one()
return _to_read(rule, rule.targets, rule.deployments)
@router.post("", response_model=RuleRead, status_code=201) @router.post("", response_model=RuleRead, status_code=201)
async def create_rule(payload: RuleCreate, db: AsyncSession = Depends(get_db)): async def create_rule(payload: RuleCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db) await policy_lock(db)
@@ -101,6 +140,7 @@ async def create_rule(payload: RuleCreate, db: AsyncSession = Depends(get_db)):
destination_net=payload.destination_net, destination_net=payload.destination_net,
destination_port=payload.destination_port, destination_port=payload.destination_port,
enabled=payload.enabled, enabled=payload.enabled,
log=payload.log,
targets=hosts, targets=hosts,
groups=groups, groups=groups,
) )
@@ -172,7 +212,7 @@ async def delete_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
try: try:
await undeploy_rule(db, rule) await undeploy_rule(db, rule)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(rule, attribute_names=["deployments"]) await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule) await db.delete(rule)
await db.commit() await db.commit()
+13
View File
@@ -31,6 +31,19 @@ async def _control(host_id: uuid.UUID, name: str, action: str, fn, db: AsyncSess
await policy_lock(db) await policy_lock(db)
host = await host_or_404(db, host_id) host = await host_or_404(db, host_id)
await ensure_no_maintenance(db, [host]) await ensure_no_maintenance(db, [host])
# The UI greys out locked services, but the API must not rely on that:
# unknown names are accepted by the firewall with "ok", and locked ones
# (pf, webgui, routing, ...) can be stopped through it.
try:
listing = await client.search_services(host)
except client.OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
rows = listing.get("rows", []) if isinstance(listing, dict) else []
row = next((r for r in rows if name in (r.get("id"), r.get("name"))), None)
if row is None:
raise api_error(404, ErrorCode.SERVICE_NOT_FOUND, service=name)
if row.get("locked") in (1, "1", True):
raise api_error(409, ErrorCode.SERVICE_LOCKED, service=name)
await audit_service.persist( await audit_service.persist(
kind="host", action=action, status="requested", kind="host", action=action, status="requested",
host_id=str(host.id), host_name=host.name, host_id=str(host.id), host_name=host.name,
+55
View File
@@ -0,0 +1,55 @@
import uuid
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.routers.hosts import _upstream_error
from app.services import snapshot_service
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/snapshots", tags=["snapshots"])
class SnapshotCreate(BaseModel):
name: str = Field(min_length=1, max_length=64)
note: str = Field(default="", max_length=200)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.get("")
async def list_snapshots(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
try:
return await snapshot_service.list_snapshots(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.post("", status_code=201)
async def create_snapshot(host_id: uuid.UUID, payload: SnapshotCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host(db, host_id)
await ensure_no_maintenance(db, [host])
return await snapshot_service.create(host, payload.name, payload.note)
@router.delete("/{snapshot_uuid}", status_code=204)
async def delete_snapshot(host_id: uuid.UUID, snapshot_uuid: str, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host(db, host_id)
await ensure_no_maintenance(db, [host])
if not snapshot_uuid.replace("-", "").isalnum() or len(snapshot_uuid) > 64:
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
await snapshot_service.delete(host, snapshot_uuid)
+96 -2
View File
@@ -1,6 +1,10 @@
from fastapi import APIRouter from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from app.services import update_check_service from app.errors import ErrorCode, api_error
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.services import update_check_service, updater_client
router = APIRouter(prefix="/system", tags=["system"]) router = APIRouter(prefix="/system", tags=["system"])
@@ -10,3 +14,93 @@ async def version_check() -> dict:
"""Ist eine neuere OPNexus-Version veröffentlicht? Aus, solange """Ist eine neuere OPNexus-Version veröffentlicht? Aus, solange
UPDATE_CHECK_URL nicht gesetzt ist (dann ohne ausgehende Anfrage).""" UPDATE_CHECK_URL nicht gesetzt ist (dann ohne ausgehende Anfrage)."""
return await update_check_service.check() return await update_check_service.check()
def _admin(user: AdminUser) -> AdminUser:
# Der Updater ist Administration der Installation selbst — auch das reine
# Lesen (Logs, Rollback-Optionen) bleibt Admins vorbehalten.
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
return user
def _major(version: str | None) -> int | None:
try:
return int((version or "").split(".")[0])
except ValueError:
return None
class UpdateRequest(BaseModel):
version: str = Field(min_length=5, max_length=20)
@router.get("/updater")
async def updater_status(user: AdminUser = Depends(current_user)) -> dict:
"""Zustand für den Abschnitt "OPNexus aktualisieren" in den Einstellungen:
Versionshinweis des Release-Feeds plus (falls aktiviert) der Updater."""
_admin(user)
check = await update_check_service.check()
result: dict = {
"available": False, "reason": None, "mode": None,
"current": check["current"], "latest": check["latest"], "update_available": check["update_available"],
"release_url": check["release_url"], "feed_enabled": check["enabled"],
"feed_error": check["error"], "checked_at": check["checked_at"],
"can_update": False, "major_change": False,
"job": None, "phases": [], "log": [], "services": {},
"rollback_available": False, "rollback_loses_data": False, "rollback_version": None,
}
try:
updater = await updater_client.status()
except updater_client.UpdaterUnavailable as exc:
result["reason"] = str(exc)
return result
except updater_client.UpdaterRejected:
result["reason"] = "unreachable"
return result
mode = updater.get("mode")
running = (updater.get("job") or {}).get("state") == "running"
result.update(
available=True, mode=mode, job=updater.get("job"), phases=updater.get("phases") or [],
log=updater.get("log") or [], services=updater.get("services") or {},
rollback_available=bool(updater.get("rollback_available")),
rollback_loses_data=bool(updater.get("rollback_loses_data")),
rollback_version=((updater.get("last_update") or {}).get("prev_version")),
)
# Hauptversionssprünge nur bewusst über die Kommandozeile (Release-Notes lesen).
result["major_change"] = bool(check["latest"]) and _major(check["latest"]) != _major(check["current"])
result["can_update"] = (
mode == "release" and not running and bool(check["update_available"]) and not result["major_change"]
)
return result
@router.post("/updater/update", status_code=202)
async def updater_update(payload: UpdateRequest, user: AdminUser = Depends(current_user)) -> dict:
"""Aktualisiert OPNexus auf die vom Release-Feed gemeldete neueste stabile
Version. Beliebige Versionen werden bewusst nicht angenommen."""
_admin(user)
check = await update_check_service.check()
if not check["update_available"] or payload.version.lstrip("v") != check["latest"]:
raise api_error(409, ErrorCode.UPDATE_NOT_OFFERED)
if _major(check["latest"]) != _major(check["current"]):
raise api_error(409, ErrorCode.UPDATE_MAJOR_CHANGE)
return await _forward(updater_client.start_update(check["latest"]))
@router.post("/updater/rollback", status_code=202)
async def updater_rollback(user: AdminUser = Depends(current_user)) -> dict:
_admin(user)
return await _forward(updater_client.start_rollback())
async def _forward(call) -> dict:
try:
return await call
except updater_client.UpdaterUnavailable:
raise api_error(503, ErrorCode.UPDATER_UNAVAILABLE)
except updater_client.UpdaterRejected as rejected:
if rejected.code == "busy":
raise api_error(409, ErrorCode.UPDATER_BUSY)
raise api_error(409, ErrorCode.UPDATER_REJECTED, reason=rejected.code)
+7 -2
View File
@@ -34,7 +34,7 @@ async def create_user(payload: UserCreate, db: AsyncSession = Depends(get_db)):
raise api_error(409, ErrorCode.BENUTZERNAME_BEREITS_VERGEBEN) raise api_error(409, ErrorCode.BENUTZERNAME_BEREITS_VERGEBEN)
user = AdminUser( user = AdminUser(
username=payload.username, username=payload.username,
password_hash=auth_service.hash_password(payload.password), password_hash=await auth_service.hash_password_async(payload.password),
role=payload.role.value, role=payload.role.value,
) )
db.add(user) db.add(user)
@@ -64,6 +64,8 @@ async def reset_two_factor(user_id: uuid.UUID, db: AsyncSession = Depends(get_db
target.totp_enabled = False target.totp_enabled = False
target.totp_recovery_hashes = None target.totp_recovery_hashes = None
target.totp_last_step = 0 target.totp_last_step = 0
# The account lost its second factor (maybe to someone else): force a fresh login.
await auth_service.revoke_sessions(db, target.id, commit=False)
await db.commit() await db.commit()
return target return target
@@ -73,7 +75,10 @@ async def reset_password(user_id: uuid.UUID, payload: UserPasswordReset, db: Asy
target = await db.get(AdminUser, user_id) target = await db.get(AdminUser, user_id)
if target is None: if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN) raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
target.password_hash = auth_service.hash_password(payload.new_password) target.password_hash = await auth_service.hash_password_async(payload.new_password)
# A reset is the answer to a suspected compromise: every session of the
# account ends with the old password, atomically with the new hash.
await auth_service.revoke_sessions(db, target.id, commit=False)
await db.commit() await db.commit()
return target return target
+4 -4
View File
@@ -1,6 +1,6 @@
import uuid import uuid
from fastapi import APIRouter, Depends, Body from fastapi import APIRouter, Depends
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
@@ -162,10 +162,10 @@ async def list_revisions(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db
@router.post("/{tunnel_id}/deploy", response_model=WireguardTunnelRead) @router.post("/{tunnel_id}/deploy", response_model=WireguardTunnelRead)
async def deploy(tunnel_id: uuid.UUID, approval: WireguardDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)): async def deploy(tunnel_id: uuid.UUID, approval: WireguardDeployApproval, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id) tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel) plan = preview(tunnel)
if approval and approval.preview_token != plan["token"]: if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN) raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if not tunnel.server_private_key_set: if not tunnel.server_private_key_set:
raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998) raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998)
@@ -187,7 +187,7 @@ async def delete_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)
try: try:
await undeploy_tunnel(db, tunnel) await undeploy_tunnel(db, tunnel)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)) from exc
await db.refresh(tunnel, attribute_names=["deployments"]) await db.refresh(tunnel, attribute_names=["deployments"])
wireguard_key_store.delete_private_key(tunnel.id) wireguard_key_store.delete_private_key(tunnel.id)
wireguard_key_store.delete_psk(tunnel.id) wireguard_key_store.delete_psk(tunnel.id)
+19 -1
View File
@@ -2,7 +2,9 @@ import uuid
from datetime import datetime from datetime import datetime
from typing import Literal from typing import Literal
from pydantic import BaseModel, ConfigDict, Field from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.services.firewall_url import validate_base_url
class HostCreate(BaseModel): class HostCreate(BaseModel):
@@ -17,6 +19,11 @@ class HostCreate(BaseModel):
cluster_name: str | None = Field(default=None, max_length=120) cluster_name: str | None = Field(default=None, max_length=120)
policy_hierarchy_group_id: uuid.UUID | None = None policy_hierarchy_group_id: uuid.UUID | None = None
@field_validator("base_url")
@classmethod
def _base_url_is_safe(cls, value: str) -> str:
return validate_base_url(value)
class TlsProbeRequest(BaseModel): class TlsProbeRequest(BaseModel):
base_url: str = Field(min_length=1, max_length=255) base_url: str = Field(min_length=1, max_length=255)
@@ -40,6 +47,8 @@ class HostUpdate(BaseModel):
recreate is still the way to rotate a host's API key for now.""" recreate is still the way to rotate a host's API key for now."""
name: str | None = Field(default=None, min_length=1, max_length=120) name: str | None = Field(default=None, min_length=1, max_length=120)
# New address of the firewall, e.g. after its management IP was changed.
base_url: str | None = Field(default=None, min_length=1, max_length=255)
verify_tls: bool | None = None verify_tls: bool | None = None
tls_ca_pem: str | None = Field(default=None, max_length=32768) tls_ca_pem: str | None = Field(default=None, max_length=32768)
clear_tls_pin: bool = False clear_tls_pin: bool = False
@@ -49,6 +58,11 @@ class HostUpdate(BaseModel):
policy_hierarchy_group_id: uuid.UUID | None = None policy_hierarchy_group_id: uuid.UUID | None = None
clear_policy_hierarchy_group_id: bool = False # same ambiguity as clear_cluster_name above clear_policy_hierarchy_group_id: bool = False # same ambiguity as clear_cluster_name above
@field_validator("base_url")
@classmethod
def _base_url_is_safe(cls, value: str | None) -> str | None:
return validate_base_url(value) if value is not None else None
class HostRead(BaseModel): class HostRead(BaseModel):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
@@ -116,6 +130,9 @@ CertStatus = Literal["valid", "expiring_soon", "expired", "unknown"]
class HostCertificate(BaseModel): class HostCertificate(BaseModel):
uuid: str uuid: str
# OPNsense's own cross-references (OpenVPN cert/ca, Trust-Store caref) use
# this id, not the row UUID.
refid: str = ""
descr: str descr: str
common_name: str | None common_name: str | None
issuer: str issuer: str
@@ -128,6 +145,7 @@ class HostCertificate(BaseModel):
class HostCa(BaseModel): class HostCa(BaseModel):
uuid: str uuid: str
refid: str = ""
descr: str descr: str
common_name: str | None common_name: str | None
valid_from: datetime | None valid_from: datetime | None
+183
View File
@@ -0,0 +1,183 @@
from typing import Any, Literal
from pydantic import BaseModel, Field
class NetAddress(BaseModel):
address: str # CIDR, e.g. 192.168.1.2/24
# Set for CARP addresses only
vhid: str | None = None
carp_status: str | None = None
class NetInterface(BaseModel):
device: str
identifier: str | None # OPNsense's own name (wan, lan, opt3) — None when unassigned
description: str
status: str
enabled: bool
is_physical: bool
mtu: str | None
mac: str | None
media: str | None
vlan_tag: str | None
vlan_parent: str | None
ipv4_type: str | None # "Static IPv4" / "DHCP" ... (assigned interfaces only)
ipv6_type: str | None
addresses: list[NetAddress]
pending_action: str | None = None
# Listed in the firewall's interface assignment (wan, lan, optN) — only those can be edited or removed.
assigned: bool = False
# This is the interface the firewall answers OPNexus on. Changing it can cut the connection.
is_management: bool = False
class NetVlan(BaseModel):
uuid: str
parent: str
tag: str
pcp: str | None
description: str
device: str | None
class NetVip(BaseModel):
"""No CARP password here, by design: OPNsense returns it in plain text."""
uuid: str
interface: str
interface_label: str
mode: str
address: str
vhid: str | None
advbase: str | None
advskew: str | None
description: str
class NetGateway(BaseModel):
uuid: str
name: str
description: str
interface: str
interface_label: str
ip_protocol: str
address: str
is_default: bool
disabled: bool
priority: str | None
status: str
class NetRoute(BaseModel):
uuid: str
network: str
gateway: str
description: str
disabled: bool
class NetDnsHost(BaseModel):
uuid: str
hostname: str
domain: str
record_type: str
server: str
enabled: bool
description: str
class NetDhcpHost(BaseModel):
uuid: str
host: str
domain: str
ip: str
hwaddr: str
lease_time: str
description: str
class NetKeaSubnet(BaseModel):
uuid: str
subnet: str
pools: str
routers: str
dns_servers: str
description: str
class NetKeaReservation(BaseModel):
uuid: str
subnet: str
ip_address: str
hw_address: str
hostname: str
description: str
class HostNetwork(BaseModel):
host_id: str
host_name: str
interfaces: list[NetInterface]
vlans: list[NetVlan]
vips: list[NetVip]
gateways: list[NetGateway]
routes: list[NetRoute]
dns_hosts: list[NetDnsHost] = []
dhcp_hosts: list[NetDhcpHost] = []
kea_subnets: list[NetKeaSubnet] = []
kea_reservations: list[NetKeaReservation] = []
management_device: str | None
# The firewall's own API can set interface IPs/types (OPNsense >= ~26.7.6). Older 26.7.x
# only lists assignments. Detected from the data, not from a version number.
can_configure_interfaces: bool
# Sections the firewall could not deliver (e.g. older OPNsense without that API) — never silently empty.
unavailable: list[str]
class NetworkChangeRequest(BaseModel):
kind: Literal["vlan", "vip", "gateway", "route", "interface", "dns_host", "dhcp_host", "kea_subnet", "kea_reservation",
"tunable", "cron_job", "syslog_destination", "gateway_group", "lagg", "bridge", "loopback", "gre", "gif", "vxlan", "dns_forward", "dhcp_range"]
action: Literal["add", "set", "del"]
# Goes into a URL path, so only plain identifiers.
uuid: str | None = Field(default=None, pattern=r"^[A-Za-z0-9_.:-]{1,64}$")
fields: dict[str, Any] = Field(default_factory=dict)
class NetworkChangeExecute(NetworkChangeRequest):
preview_token: str = Field(min_length=64, max_length=64)
acknowledge: bool = False
class NetworkChangeRisk(BaseModel):
code: str
params: dict[str, Any] = Field(default_factory=dict)
class NetworkFieldChange(BaseModel):
field: str
before: str | None
after: str | None
class NetworkChangePreview(BaseModel):
token: str
kind: str
action: str
uuid: str | None
host_id: str
host_name: str
changes: list[NetworkFieldChange]
risks: list[NetworkChangeRisk]
warnings: list[NetworkChangeRisk]
blocks: list[NetworkChangeRisk]
requires_acknowledgement: bool
management_device: str | None
can_execute: bool
class NetworkChangeResult(BaseModel):
status: str
uuid: str | None
verified: bool
mismatches: list[str]
backup_id: str
+3 -1
View File
@@ -9,7 +9,7 @@ from app.config import settings
SmtpSecurity = Literal["none", "starttls", "ssl"] SmtpSecurity = Literal["none", "starttls", "ssl"]
ProblemType = Literal[ ProblemType = Literal[
"host_offline", "cert_expiry", "rule_drift", "revert_failed", "maintenance_unknown", "host_offline", "cert_expiry", "rule_drift", "revert_failed", "maintenance_unknown",
"carp_split_brain", "nat_drift", "carp_auto_revert_failed", "carp_split_brain", "nat_drift", "carp_auto_revert_failed", "network_drift",
] ]
@@ -36,6 +36,7 @@ class NotificationSettingsRead(BaseModel):
maintenance_unknown_enabled: bool maintenance_unknown_enabled: bool
carp_health_enabled: bool carp_health_enabled: bool
nat_drift_enabled: bool nat_drift_enabled: bool
network_drift_enabled: bool = True
carp_auto_revert_failed_enabled: bool carp_auto_revert_failed_enabled: bool
poll_interval_seconds: int poll_interval_seconds: int
@@ -71,6 +72,7 @@ class NotificationSettingsUpdate(BaseModel):
maintenance_unknown_enabled: bool | None = None maintenance_unknown_enabled: bool | None = None
carp_health_enabled: bool | None = None carp_health_enabled: bool | None = None
nat_drift_enabled: bool | None = None nat_drift_enabled: bool | None = None
network_drift_enabled: bool | None = None
carp_auto_revert_failed_enabled: bool | None = None carp_auto_revert_failed_enabled: bool | None = None
poll_interval_seconds: int | None = Field( poll_interval_seconds: int | None = Field(
+26
View File
@@ -17,6 +17,7 @@ class RuleCreate(BaseModel):
destination_net: str = Field(default="any", min_length=1, max_length=128) destination_net: str = Field(default="any", min_length=1, max_length=128)
destination_port: str | None = Field(default=None, max_length=32) destination_port: str | None = Field(default=None, max_length=32)
enabled: bool = True enabled: bool = True
log: bool = False
target_host_ids: list[uuid.UUID] = Field(default_factory=list) target_host_ids: list[uuid.UUID] = Field(default_factory=list)
target_group_ids: list[uuid.UUID] = Field(default_factory=list) target_group_ids: list[uuid.UUID] = Field(default_factory=list)
@@ -56,6 +57,7 @@ class RuleRead(BaseModel):
destination_net: str destination_net: str
destination_port: str | None destination_port: str | None
enabled: bool enabled: bool
log: bool = False
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
target_host_ids: list[uuid.UUID] target_host_ids: list[uuid.UUID]
@@ -72,3 +74,27 @@ class RuleEdit(RuleCreate):
class DeployApproval(BaseModel): class DeployApproval(BaseModel):
preview_token: str = Field(min_length=64, max_length=64) preview_token: str = Field(min_length=64, max_length=64)
class AdoptRequest(BaseModel):
host_id: uuid.UUID
rule_uuid: str = Field(min_length=1, max_length=64)
class UnmanagedRule(BaseModel):
uuid: str
description: str
interface: str
interface_label: str
direction: str
action: str
protocol: str
source_net: str
source_port: str
destination_net: str
destination_port: str
enabled: bool
log: bool
adoptable: bool
# Codes such as "negated_source" or "not_quick"; the UI turns them into text.
reasons: list[str]
+5 -1
View File
@@ -27,7 +27,11 @@ def _origin_allowed(origin: str, host_header: str) -> bool:
if not parsed.scheme or not parsed.netloc: if not parsed.scheme or not parsed.netloc:
return False return False
if parsed.netloc.lower() == host_header.lower(): if parsed.netloc.lower() == host_header.lower():
return True # Same host is not enough on an HTTPS deployment: an http:// page on that
# very host (plain-HTTP port, downgraded first visit) is a different
# origin. The backend only sees the proxy's plain-HTTP hop, so the
# deployment's own COOKIE_SECURE switch says which scheme is legitimate.
return parsed.scheme == "https" or not settings.cookie_secure
return f"{parsed.scheme}://{parsed.netloc}".lower() in {o.lower() for o in settings.allowed_origins} return f"{parsed.scheme}://{parsed.netloc}".lower() in {o.lower() for o in settings.allowed_origins}
+2 -3
View File
@@ -10,6 +10,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".eabhmac" _SUFFIX = ".eabhmac"
@@ -20,9 +21,7 @@ def _path(account_id: uuid.UUID) -> Path:
def save_secret(account_id: uuid.UUID, secret: str) -> None: def save_secret(account_id: uuid.UUID, secret: str) -> None:
path = _path(account_id) path = _path(account_id)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(account_id: uuid.UUID) -> str | None: def read_secret(account_id: uuid.UUID) -> str | None:
+19 -43
View File
@@ -19,7 +19,8 @@ from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment from app.models.alias_deployment import AliasDeployment
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
def _now() -> datetime: def _now() -> datetime:
@@ -54,48 +55,23 @@ async def _deploy_to_host(db: AsyncSession, alias: FirewallAlias, host: Opnsense
prior_uuid = existing.opnsense_alias_uuid if existing else None prior_uuid = existing.opnsense_alias_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_alias_item(host, prior_uuid, payload)
else:
result = await opnsense_client.add_alias_item(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Alias-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.reconfigure_aliases(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_alias_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = AliasDeployment(
alias_id=alias.id, host_id=host.id, status="error", message=str(exc),
opnsense_alias_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds) async def write(uuid_):
if existing: if uuid_:
existing.status = "pending" await opnsense_client.set_alias_item(host, uuid_, payload)
existing.message = None return uuid_
existing.opnsense_alias_uuid = new_uuid result = await opnsense_client.add_alias_item(host, payload)
existing.deployed_at = _now() new_uuid = result.get("uuid") if isinstance(result, dict) else None
existing.revert_deadline = deadline if not new_uuid:
existing.previous_payload = prior_live_payload raise OpnsenseAmbiguousError("Firewall hat keine Alias-UUID zurückgegeben; Stand manuell prüfen")
existing.last_payload = payload return new_uuid
return existing
deployment = AliasDeployment( deployment = await push_to_host(
alias_id=alias.id, host_id=host.id, status="pending", message=None, db, existing=existing, uuid_attr="opnsense_alias_uuid",
opnsense_alias_uuid=new_uuid, deployed_at=_now(), revert_deadline=deadline, make=lambda **fields: AliasDeployment(alias_id=alias.id, host_id=host.id, **fields),
previous_payload=None, last_payload=payload, prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.reconfigure_aliases(host),
) )
db.add(deployment)
return deployment return deployment
@@ -113,7 +89,7 @@ async def deploy_alias(db: AsyncSession, alias: FirewallAlias, hosts: list[Opnse
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = f"Alias wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}" deployment.message = f"Alias wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}"
deployment.revert_deadline = None # The change is live: keep the revert deadline so an unconfirmed result is still undone.
await db.commit() await db.commit()
return deployments return deployments
@@ -134,7 +110,7 @@ async def sweep_expired_alias_deployments(db: AsyncSession) -> list[AliasDeploym
from app.services.policy_service import policy_lock from app.services.policy_service import policy_lock
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(AliasDeployment).where(AliasDeployment.status == "pending", AliasDeployment.revert_deadline <= _now()) select(AliasDeployment).where(AliasDeployment.status.in_(("pending", "error")), AliasDeployment.revert_deadline <= _now())
) )
expired = list(result.scalars().all()) expired = list(result.scalars().all())
touched: list[AliasDeployment] = [] touched: list[AliasDeployment] = []
@@ -159,7 +135,7 @@ async def undeploy_alias(db: AsyncSession, alias: FirewallAlias) -> None:
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+20 -3
View File
@@ -15,10 +15,27 @@ from app.models.audit import AuditEvent
context = ContextVar("audit_context", default=None) context = ContextVar("audit_context", default=None)
# Column widths of audit_events. entity_id/host_id/action are taken from the
# request path BEFORE the router validated it, so a junk segment (300 chars,
# or a NUL byte which Postgres text cannot store) used to turn into a 500 or a
# dropped connection instead of a plain 404/422 (found by fuzzing, 2026-10-09).
_LIMITS = {"correlation_id": 36, "actor": 150, "kind": 32, "action": 40, "status": 32,
"entity_id": 36, "entity_name": 255, "host_id": 36, "host_name": 255}
def _clean(key, value):
if not isinstance(value, str):
return value
value = value.replace("\x00", "")
limit = _LIMITS.get(key)
return value[:limit] if limit else value
def make_event(**values): def make_event(**values):
ctx = context.get() or {} ctx = context.get() or {}
return AuditEvent(actor=ctx.get("actor", "system:rollback"), values = {k: _clean(k, v) for k, v in values.items()}
correlation_id=ctx.get("id", str(uuid.uuid4())), **values) return AuditEvent(actor=_clean("actor", ctx.get("actor", "system:rollback")),
correlation_id=_clean("correlation_id", ctx.get("id", str(uuid.uuid4()))), **values)
async def persist(**values): async def persist(**values):
@@ -42,7 +59,7 @@ def deployment_history(session, flush_context, instances):
PolicyHierarchyRuleDeployment: ("policy_hierarchy", PolicyHierarchyRule, "policy_hierarchy_id", "opnsense_policy_hierarchy_uuid"), PolicyHierarchyRuleDeployment: ("policy_hierarchy", PolicyHierarchyRule, "policy_hierarchy_id", "opnsense_policy_hierarchy_uuid"),
} }
for dep in list(session.new) + list(session.dirty) + list(session.deleted): for dep in list(session.new) + list(session.dirty) + list(session.deleted):
if type(dep) not in mapping: if type(dep) not in mapping or getattr(dep, "_skip_audit", False):
continue continue
is_new, deleted = dep in session.new, dep in session.deleted is_new, deleted = dep in session.new, dep in session.deleted
if not is_new and not deleted and not session.is_modified(dep, include_collections=False): if not is_new and not deleted and not session.is_modified(dep, include_collections=False):
+116 -12
View File
@@ -20,6 +20,7 @@ Design notes
not reveal whether the account exists. not reveal whether the account exists.
""" """
import asyncio
import hashlib import hashlib
import logging import logging
import secrets import secrets
@@ -33,6 +34,7 @@ from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings from app.config import settings
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser, Session from app.models.user import AdminUser, Session
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -60,6 +62,36 @@ def verify_password(stored_hash: str, password: str) -> bool:
return False return False
# Argon2 is deliberately expensive (CPU and ~64 MiB each). Run on the default
# executor so the single event loop (health checks, deploy/rollback sweeps)
# keeps running, and cap how many run — and how many may queue — at once so a
# flood of login attempts cannot exhaust CPU/memory or the thread pool.
_HASH_CONCURRENCY = 4
_HASH_MAX_WAITING = 32
_hash_slots = asyncio.Semaphore(_HASH_CONCURRENCY)
_hash_waiting = 0
async def _bounded(function, *args):
global _hash_waiting
if _hash_waiting >= _HASH_MAX_WAITING:
raise api_error(503, ErrorCode.TOO_MANY_LOGIN_ATTEMPTS, wait=5)
_hash_waiting += 1
try:
async with _hash_slots:
return await asyncio.to_thread(function, *args)
finally:
_hash_waiting -= 1
async def hash_password_async(password: str) -> str:
return await _bounded(hash_password, password)
async def verify_password_async(stored_hash: str, password: str) -> bool:
return await _bounded(verify_password, stored_hash, password)
def needs_rehash(stored_hash: str) -> bool: def needs_rehash(stored_hash: str) -> bool:
try: try:
return _ph.check_needs_rehash(stored_hash) return _ph.check_needs_rehash(stored_hash)
@@ -76,21 +108,32 @@ class _AttemptRecord:
class _LoginThrottle: class _LoginThrottle:
"""Failed-login limiter with two independent buckets per window: """Failed-attempt limiter with two independent buckets per window:
* ``ip::username`` — targeted guessing against the one account * ``ip::username`` — targeted guessing against the one account
* ``ip`` — spraying across many usernames from one source * ``ip`` — spraying across many usernames from one source
A login is blocked while *either* bucket is locked out. A login is blocked while *either* bucket is locked out.
All state is bounded: records whose window and lockout have both passed
are dropped, and a hard cap evicts the least recently active entries, so
a stream of made-up usernames/sources cannot grow memory without limit.
""" """
MAX_RECORDS = 20_000
MAX_COUNTERS = 5_000
COUNTER_TTL_SECONDS = 24 * 3600
_PURGE_EVERY_SECONDS = 30.0
def __init__(self) -> None: def __init__(self) -> None:
self._records: dict[str, _AttemptRecord] = {} self._records: dict[str, _AttemptRecord] = {}
# Total failures per username since their last successful login — # Total failures per username since their last successful login —
# informational only (shown to the user after login), separate from # informational only (shown to the user after login), separate from
# the rolling lockout windows above. In-memory/per-process like the # the rolling lockout windows above: username -> (count, last_failure).
# rest of this throttle, so it resets on a backend restart. # In-memory/per-process like the rest of this throttle, so it resets
self._failures_since_success: dict[str, int] = {} # on a backend restart.
self._failures_since_success: dict[str, tuple[int, float]] = {}
self._last_purge = 0.0
@staticmethod @staticmethod
def _user_key(ip: str, username: str) -> str: def _user_key(ip: str, username: str) -> str:
@@ -107,6 +150,32 @@ class _LoginThrottle:
remaining = rec.locked_until - time.monotonic() remaining = rec.locked_until - time.monotonic()
return int(remaining) + 1 if remaining > 0 else 0 return int(remaining) + 1 if remaining > 0 else 0
@staticmethod
def _last_activity(rec: _AttemptRecord) -> float:
return max([rec.locked_until, *rec.failures]) if rec.failures or rec.locked_until else 0.0
def _purge(self, now: float, *, force: bool = False) -> None:
if not force and now - self._last_purge < self._PURGE_EVERY_SECONDS and len(self._records) < self.MAX_RECORDS:
return
self._last_purge = now
window = settings.login_window_seconds
for key in [
k for k, r in self._records.items()
if r.locked_until <= now and all(now - t >= window for t in r.failures)
]:
del self._records[key]
if len(self._records) > self.MAX_RECORDS:
# Still over the cap with live entries: drop the stalest unlocked ones first.
stale = sorted(self._records, key=lambda k: (self._records[k].locked_until > now, self._last_activity(self._records[k])))
for key in stale[: len(self._records) - self.MAX_RECORDS]:
del self._records[key]
for name in [n for n, (_, ts) in self._failures_since_success.items() if now - ts > self.COUNTER_TTL_SECONDS]:
del self._failures_since_success[name]
if len(self._failures_since_success) > self.MAX_COUNTERS:
oldest = sorted(self._failures_since_success, key=lambda n: self._failures_since_success[n][1])
for name in oldest[: len(self._failures_since_success) - self.MAX_COUNTERS]:
del self._failures_since_success[name]
def _record_key(self, key: str, max_attempts: int) -> None: def _record_key(self, key: str, max_attempts: int) -> None:
now = time.monotonic() now = time.monotonic()
rec = self._records.setdefault(key, _AttemptRecord()) rec = self._records.setdefault(key, _AttemptRecord())
@@ -119,16 +188,19 @@ class _LoginThrottle:
def retry_after(self, ip: str, username: str) -> int: def retry_after(self, ip: str, username: str) -> int:
"""Seconds the caller must wait, or 0 if not currently locked out.""" """Seconds the caller must wait, or 0 if not currently locked out."""
self._purge(time.monotonic())
return max( return max(
self._retry_after_key(self._user_key(ip, username)), self._retry_after_key(self._user_key(ip, username)),
self._retry_after_key(self._ip_key(ip)), self._retry_after_key(self._ip_key(ip)),
) )
def record_failure(self, ip: str, username: str) -> None: def record_failure(self, ip: str, username: str) -> None:
self._purge(time.monotonic())
self._record_key(self._user_key(ip, username), settings.login_max_attempts) self._record_key(self._user_key(ip, username), settings.login_max_attempts)
self._record_key(self._ip_key(ip), settings.login_ip_max_attempts) self._record_key(self._ip_key(ip), settings.login_ip_max_attempts)
key = username.lower() key = username.lower()
self._failures_since_success[key] = self._failures_since_success.get(key, 0) + 1 count, _ = self._failures_since_success.get(key, (0, 0.0))
self._failures_since_success[key] = (count + 1, time.monotonic())
def reset(self, ip: str, username: str) -> None: def reset(self, ip: str, username: str) -> None:
self._records.pop(self._user_key(ip, username), None) self._records.pop(self._user_key(ip, username), None)
@@ -136,10 +208,15 @@ class _LoginThrottle:
def pop_failures_since_success(self, username: str) -> int: def pop_failures_since_success(self, username: str) -> int:
"""Read and clear the failure count for a username, called on a """Read and clear the failure count for a username, called on a
successful login so the count reflects only attempts since then.""" successful login so the count reflects only attempts since then."""
return self._failures_since_success.pop(username.lower(), 0) return self._failures_since_success.pop(username.lower(), (0, 0.0))[0]
throttle = _LoginThrottle() throttle = _LoginThrottle()
# Separate budget for the self-service security endpoints (2FA setup/enable/
# disable, password change): keyed by (user id, action) instead of (ip,
# username), so repeating the request from a stolen session cannot be used to
# guess the password or a second-factor code without limit.
account_throttle = _LoginThrottle()
# ── Sessions ─────────────────────────────────────────────────────────── # ── Sessions ───────────────────────────────────────────────────────────
@@ -158,12 +235,27 @@ async def create_session(
*, *,
user_agent: str | None = None, user_agent: str | None = None,
ip_address: str | None = None, ip_address: str | None = None,
verified_hash: str | None = None,
) -> tuple[str, datetime]: ) -> tuple[str, datetime]:
"""Create a session, returning the raw token (to be set as a cookie) and its expiry.""" """Create a session, returning the raw token (to be set as a cookie) and its expiry.
``verified_hash`` is the password hash the caller just authenticated
against. The user row is re-read under a row lock first: a password
change/reset that committed while the (slow) password check was running
changed the hash, and the login must not outlive that revocation.
"""
user_id = user.id
if verified_hash is not None:
locked = await db.scalar(
select(AdminUser.password_hash).where(AdminUser.id == user_id).with_for_update()
)
if locked != verified_hash:
await db.rollback()
raise api_error(401, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
token = secrets.token_urlsafe(SESSION_TOKEN_BYTES) token = secrets.token_urlsafe(SESSION_TOKEN_BYTES)
expires_at = _now() + timedelta(seconds=settings.session_lifetime) expires_at = _now() + timedelta(seconds=settings.session_lifetime)
session = Session( session = Session(
user_id=user.id, user_id=user_id,
token_hash=_hash_token(token), token_hash=_hash_token(token),
expires_at=expires_at, expires_at=expires_at,
last_used_at=_now(), last_used_at=_now(),
@@ -171,7 +263,7 @@ async def create_session(
ip_address=ip_address, ip_address=ip_address,
) )
db.add(session) db.add(session)
await db.execute(delete(Session).where(Session.user_id == user.id, Session.expires_at < _now())) await db.execute(delete(Session).where(Session.user_id == user_id, Session.expires_at < _now()))
await db.commit() await db.commit()
return token, expires_at return token, expires_at
@@ -193,6 +285,18 @@ async def revoke_session(db: AsyncSession, token: str) -> None:
await db.commit() await db.commit()
async def revoke_sessions(db: AsyncSession, user_id, *, keep_token: str | None = None, commit: bool = True) -> None:
"""Delete every session of an account (optionally sparing the caller's own).
With ``commit=False`` the deletion joins the caller's transaction, so a
credential change and the revocation succeed or fail together."""
query = delete(Session).where(Session.user_id == user_id)
if keep_token:
query = query.where(Session.token_hash != _hash_token(keep_token))
await db.execute(query)
if commit:
await db.commit()
# ── Authentication ──────────────────────────────────────────────────── # ── Authentication ────────────────────────────────────────────────────
async def authenticate( async def authenticate(
@@ -211,14 +315,14 @@ async def authenticate(
user = result.scalar_one_or_none() user = result.scalar_one_or_none()
if user is None: if user is None:
verify_password(_DUMMY_HASH, password) await verify_password_async(_DUMMY_HASH, password)
return None return None
if not verify_password(user.password_hash, password): if not await verify_password_async(user.password_hash, password):
return None return None
if needs_rehash(user.password_hash): if needs_rehash(user.password_hash):
user.password_hash = hash_password(password) user.password_hash = await hash_password_async(password)
if finalize: if finalize:
await finalize_login(db, user, ip) await finalize_login(db, user, ip)
else: else:
+3 -3
View File
@@ -8,6 +8,7 @@ from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".xml" _SUFFIX = ".xml"
@@ -21,11 +22,10 @@ def save_snapshot(host_id: uuid.UUID, content: bytes) -> str:
microsecond precision to avoid collisions on rapid consecutive saves. microsecond precision to avoid collisions on rapid consecutive saves.
Returns the filename, used as the backup's id.""" Returns the filename, used as the backup's id."""
host_dir = _host_dir(host_id) host_dir = _host_dir(host_id)
host_dir.mkdir(parents=True, exist_ok=True) host_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
filename = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S%f") + _SUFFIX filename = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S%f") + _SUFFIX
path = host_dir / filename path = host_dir / filename
path.write_bytes(content) secure_files.write_private(path, content)
path.chmod(0o600)
return filename return filename
@@ -19,6 +19,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.models.carp_maintenance_window import CarpMaintenanceWindow from app.models.carp_maintenance_window import CarpMaintenanceWindow
from app.models.host import OpnsenseHost from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.services import audit_service, host_status_service from app.services import audit_service, host_status_service
from app.services import opnsense_client as client from app.services import opnsense_client as client
@@ -63,6 +64,26 @@ async def close_windows_for_manual_leave(db: AsyncSession, host: OpnsenseHost) -
await db.commit() await db.commit()
async def _close_enter_operations(db: AsyncSession, host: OpnsenseHost, message: str) -> None:
"""Closes the host's still-open `carp_enter` maintenance record once the
window has been resolved by the timer. Without this the record stays
"submitted" forever: acknowledge cannot succeed (the host is no longer in
maintenance) and `ensure_no_maintenance` keeps blocking every deploy and
update in the cluster until someone cancels it by hand."""
result = await db.execute(
select(HostOperation).where(
HostOperation.host_id == host.id, HostOperation.action == "carp_enter", HostOperation.status == "submitted",
)
)
for op in result.scalars().all():
op.status = "cancelled"
op.message = message
await audit_service.persist(
kind="host", action="carp_enter", status="cancelled",
host_id=str(host.id), host_name=host.name, entity_id=str(op.id), message=message,
)
async def sweep_expired(db: AsyncSession) -> list[CarpMaintenanceWindow]: async def sweep_expired(db: AsyncSession) -> list[CarpMaintenanceWindow]:
result = await db.execute( result = await db.execute(
select(CarpMaintenanceWindow).where( select(CarpMaintenanceWindow).where(
@@ -88,6 +109,7 @@ async def sweep_expired(db: AsyncSession) -> list[CarpMaintenanceWindow]:
window.status = "cancelled_manual" window.status = "cancelled_manual"
window.resolved_at = _now() window.resolved_at = _now()
window.message = "Host war beim Ablauf bereits nicht mehr im CARP-Wartungsmodus." window.message = "Host war beim Ablauf bereits nicht mehr im CARP-Wartungsmodus."
await _close_enter_operations(db, host, window.message)
resolved.append(window) resolved.append(window)
continue continue
result_payload = await client.toggle_carp_maintenance(host) result_payload = await client.toggle_carp_maintenance(host)
@@ -96,6 +118,7 @@ async def sweep_expired(db: AsyncSession) -> list[CarpMaintenanceWindow]:
window.status = "reverted" window.status = "reverted"
window.resolved_at = _now() window.resolved_at = _now()
window.message = "Automatisch nach Ablauf des Zeitfensters in den Normalbetrieb zurückgeschaltet." window.message = "Automatisch nach Ablauf des Zeitfensters in den Normalbetrieb zurückgeschaltet."
await _close_enter_operations(db, host, window.message)
await audit_service.persist( await audit_service.persist(
kind="host", action="carp_auto_revert", status="submitted", kind="host", action="carp_auto_revert", status="submitted",
host_id=str(host.id), host_name=host.name, entity_id=str(window.id), host_id=str(host.id), host_name=host.name, entity_id=str(window.id),
+2 -3
View File
@@ -22,6 +22,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".certkey" _SUFFIX = ".certkey"
@@ -32,9 +33,7 @@ def _path(cert_id: uuid.UUID) -> Path:
def save_secret(cert_id: uuid.UUID, secret: str) -> None: def save_secret(cert_id: uuid.UUID, secret: str) -> None:
path = _path(cert_id) path = _path(cert_id)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(cert_id: uuid.UUID) -> str | None: def read_secret(cert_id: uuid.UUID) -> str | None:
+125
View File
@@ -0,0 +1,125 @@
"""The shared write path of every "push to a firewall, then confirm or auto-revert"
feature (rules, aliases, port forwards, 1:1 NAT, outbound NAT, hierarchy rules).
Safety property: from the moment OPNexus may have changed something on a
firewall, a durable record exists that lets the revert sweeper (or an operator)
find and undo it — even if the reply is lost, the apply fails or the process
dies in between.
1. *Intent* is committed BEFORE the first remote call:
- existing deployment: status "pending" with the revert deadline armed and
the previous payload as rollback target (a crash leaves a normal,
sweepable pending change);
- first deployment (no remote UUID known yet): status "revert_failed" =
"outcome unknown, check the firewall" — the state that already blocks
retries/edits and raises a notification; "Rollback wiederholen" just
forgets the row when no UUID was ever recorded.
2. The write. A refusal the firewall answered restores the previous state; a
lost/ambiguous reply keeps the intent record (existing: deadline stays armed,
new: stays "unknown").
3. After a confirmed write the UUID is committed at once and the row becomes a
normal armed "pending" change. If applying it fails the row is "error" but
keeps its deadline, so the sweeper still reverts a possibly live change.
4. Only after a successful apply the confirmation window starts.
policy_lock (a transaction-scoped advisory lock) is released by the intermediate
commits and taken again right away; the intent row itself keeps other deploys
and the sweeper away in the gap (pending/unknown rows are never re-deployed).
"""
from datetime import datetime, timedelta, timezone
from typing import Any, Awaitable, Callable
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
UNKNOWN_OUTCOME = (
"Ausgang unklar: Die Firewall hat die Anfrage möglicherweise ausgeführt. Stand dort prüfen; "
"„Rollback wiederholen“ entfernt diesen Eintrag, ohne die Firewall anzusprechen."
)
_FIELDS = ("status", "message", "last_payload", "previous_payload", "revert_deadline", "deployed_at")
def _now() -> datetime:
return datetime.now(timezone.utc)
async def _checkpoint(db: AsyncSession) -> None:
from app.services.policy_service import policy_lock
await db.commit()
await policy_lock(db)
async def push_to_host(
db: AsyncSession,
*,
existing: Any | None,
make: Callable[..., Any],
uuid_attr: str,
prior_uuid: str | None,
prior_live_payload: dict | None,
payload: dict,
write: Callable[[str | None], Awaitable[str]],
apply: Callable[[], Awaitable[Any]],
) -> Any:
window = timedelta(seconds=settings.deploy_confirm_window_seconds)
before = {name: getattr(existing, name) for name in _FIELDS} if existing else None
# 1. durable intent
if existing is not None:
row = existing
row.status, row.message = "pending", None
row.previous_payload, row.last_payload = prior_live_payload, payload
row.revert_deadline, row.deployed_at = _now() + window, _now()
else:
row = make(
status="revert_failed", message=UNKNOWN_OUTCOME, last_payload=payload, previous_payload=None,
revert_deadline=None, deployed_at=_now(), **{uuid_attr: None},
)
db.add(row)
# The intent is a technical safety record, not a state change worth a history
# entry of its own (the request itself is already audited as "requested").
row._skip_audit = True
await _checkpoint(db)
row._skip_audit = False
wrote = False
try:
new_uuid = await write(prior_uuid)
wrote = True
# 3. recoverable by UUID from here on
setattr(row, uuid_attr, new_uuid)
row.status, row.message = "pending", None
row.previous_payload = prior_live_payload if existing is not None else None
row.last_payload, row.revert_deadline, row.deployed_at = payload, _now() + window, _now()
await _checkpoint(db)
await apply()
except OpnsenseAPIError as exc:
ambiguous = isinstance(exc, OpnsenseAmbiguousError)
if wrote:
# written, apply unconfirmed: possibly live — status "error" but the deadline stays armed
row.status, row.message = "error", str(exc)
row.revert_deadline, row.deployed_at = _now() + window, _now()
elif existing is not None and ambiguous:
# the update may or may not have been applied: keep the armed rollback
row.status, row.message = "error", f"{exc} (Ausgang unklar, wird zurückgesetzt)"
row.deployed_at = _now()
elif existing is not None:
# a refusal the firewall answered: nothing changed, restore what was there
for name, value in before.items():
setattr(row, name, value)
row.status, row.message, row.deployed_at = "error", str(exc), _now()
elif ambiguous:
row.status, row.message = "revert_failed", f"{exc} — {UNKNOWN_OUTCOME}"
else:
row.status, row.message = "error", str(exc)
row.last_payload, row.revert_deadline = None, None
return row
# 4. confirmed: the confirmation window starts now
row.status, row.message = "pending", None
row.revert_deadline, row.deployed_at = _now() + window, _now()
return row
+142
View File
@@ -0,0 +1,142 @@
"""Network probes run FROM a firewall: ping, traceroute, TCP port probe, DNS lookup.
Admin-only by construction (every call is a POST, which the read-only role is
refused for) and audited with the target, because a firewall that probes
arbitrary addresses on request is also a scanning tool. Inputs are checked
here before they are sent (the firewall validates them again); results are
shown and never stored. Ping runs as a short job that is always stopped and
removed again, even when polling fails.
"""
import asyncio
import ipaddress
import re
import time
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.services import audit_service, opnsense_client
from app.services.opnsense_client import OpnsenseAPIError, OpnsenseDiagnosticRejected
TARGET = re.compile(r"^[A-Za-z0-9]([A-Za-z0-9.:_-]{0,251}[A-Za-z0-9])?$")
# At most a few probes at a time per OPNexus process, so the page cannot be used to hammer a firewall.
_SLOTS = asyncio.Semaphore(4)
def clean_target(value: str, field: str = "target") -> str:
value = (value or "").strip()
if not TARGET.match(value):
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field=field)
return value
def clean_source(value: str | None) -> str:
value = (value or "").strip()
if not value:
return ""
try:
return str(ipaddress.ip_address(value))
except ValueError:
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field="source") from None
async def _audit(host: OpnsenseHost, tool: str, target: str, ok: bool, details: dict, message: str | None = None) -> None:
await audit_service.persist(kind="diagnostic", action=f"diag.{tool}", status="ok" if ok else "failed",
entity_name=target, host_id=str(host.id), host_name=host.name, message=message, details=details)
def _rejected(exc: OpnsenseDiagnosticRejected):
field = next(iter(exc.validations), "target").split(".")[-1]
return api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field=field)
async def _guarded(host: OpnsenseHost, tool: str, target: str, params: dict, call):
async with _SLOTS:
try:
result = await call()
except OpnsenseDiagnosticRejected as exc:
await _audit(host, tool, target, False, params, "rejected: " + ", ".join(exc.validations))
raise _rejected(exc) from exc
except OpnsenseAPIError as exc:
await _audit(host, tool, target, False, params, str(exc)[:300])
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
await _audit(host, tool, target, True, params)
return result
async def traceroute(host: OpnsenseHost, target: str, ipproto: str, protocol: str, source: str | None) -> dict:
target, source = clean_target(target), clean_source(source)
params = {"target": target, "ipproto": ipproto, "protocol": protocol, "source": source}
async def call():
data = await opnsense_client.diagnostic_set(
host, "/diagnostics/traceroute/set", "traceroute",
{"hostname": target, "ipproto": ipproto, "protocol": protocol, "source_address": source})
resp = data.get("response") or {}
hops = [{"ttl": str(r.get("ttl", "")), "address": str(r.get("address", "")), "host": str(r.get("host", "")),
"as": str(r.get("AS", "")), "probes": str(r.get("probes", ""))} for r in resp.get("rows", []) if isinstance(r, dict)]
return {"target": target, "notice": str(resp.get("notice", "")), "hops": hops}
return await _guarded(host, "traceroute", target, params, call)
async def portprobe(host: OpnsenseHost, target: str, port: int, ipproto: str, source: str | None) -> dict:
target, source = clean_target(target), clean_source(source)
params = {"target": target, "port": port, "ipproto": ipproto, "source": source}
async def call():
data = await opnsense_client.diagnostic_set(
host, "/diagnostics/portprobe/set", "portprobe",
{"hostname": target, "port": str(port), "ipproto": ipproto, "source_address": source, "source_port": "", "showtext": "0"})
message = str((data.get("response") or {}).get("message", ""))
# Never forward the remote payload (a service banner): only whether the connection worked.
return {"target": target, "port": port, "open": "succeeded" in message.lower(), "message": message}
return await _guarded(host, "portprobe", target, params, call)
async def dns_lookup(host: OpnsenseHost, name: str, server: str | None) -> dict:
name = clean_target(name, "name")
server = clean_source(server)
params = {"name": name, "server": server}
async def call():
data = await opnsense_client.diagnostic_set(host, "/diagnostics/dns_diagnostics/set", "dns", {"hostname": name, "server": server})
out = {}
for rtype, rec in (data.get("response") or {}).items():
if isinstance(rec, dict):
out[str(rtype)] = {"answers": [str(a) for a in rec.get("answers", [])], "query_time": str(rec.get("query_time", "")),
"server": str(rec.get("server", ""))}
return {"name": name, "records": out}
return await _guarded(host, "dns", name, params, call)
async def ping(host: OpnsenseHost, target: str, ipproto: str, count: int, packetsize: int | None, source: str | None) -> dict:
target, source = clean_target(target), clean_source(source)
params = {"target": target, "ipproto": ipproto, "count": count, "packetsize": packetsize, "source": source}
async def call():
job = await opnsense_client.ping_job_start(host, {
"hostname": target, "fam": "ip6" if ipproto == "inet6" else "ip", "source_address": source,
"packetsize": str(packetsize or ""), "disable_frag": "0", "interval": "1", "description": "OPNexus"})
try:
state = None
deadline = time.monotonic() + count + 6
while time.monotonic() < deadline:
await asyncio.sleep(1)
state = await opnsense_client.ping_job_state(host, job)
if state and int(state.get("send") or 0) >= count:
break
await opnsense_client.ping_job_stop(host, job)
await asyncio.sleep(0.5)
state = await opnsense_client.ping_job_state(host, job) or state
finally:
# Never leave a job running on the firewall.
try:
await opnsense_client.ping_job_stop(host, job)
await opnsense_client.ping_job_remove(host, job)
except OpnsenseAPIError:
pass
if not state:
raise OpnsenseAPIError("Ping: kein Ergebnis von der Firewall")
return {"target": target, "sent": int(state.get("send") or 0), "received": int(state.get("received") or 0),
"loss": str(state.get("loss", "")), "min": state.get("min"), "avg": state.get("avg"), "max": state.get("max"),
"stddev": state.get("std-dev"), "error": state.get("last_error")}
return await _guarded(host, "ping", target, params, call)
+41
View File
@@ -0,0 +1,41 @@
"""Policy for the address under which a firewall's API is reached.
The API key/secret travel as HTTP Basic credentials, so a plain ``http://``
address would hand them to anyone on the path. ``verify_tls`` is a different
control (does the certificate get checked) and does not upgrade or reject an
``http://`` URL, hence the scheme is enforced here — when a host is created
and again, for rows that already exist, right before a request is built.
``ALLOW_INSECURE_FIREWALL_URLS=true`` is the explicit, deployment-level
escape hatch for lab setups.
"""
from urllib.parse import urlsplit
from app.config import settings
def check_base_url(value: str) -> str | None:
"""Return a reason (German, user-facing) if the URL must not be used, else None."""
try:
parts = urlsplit(value.strip())
parts.port # noqa: B018 - raises ValueError on an invalid port
except ValueError:
return "Ungültige Adresse"
if parts.scheme not in ("https", "http") or not parts.hostname:
return "Die Adresse muss mit https:// beginnen und einen Host enthalten"
if parts.scheme == "http" and not settings.allow_insecure_firewall_urls:
return (
"Unverschlüsseltes http:// ist nicht erlaubt (API-Zugangsdaten würden im Klartext "
"übertragen) – bitte https:// verwenden"
)
if parts.username is not None or parts.password is not None:
return "Zugangsdaten gehören nicht in die Adresse"
if parts.query or parts.fragment:
return "Die Adresse darf keine Parameter oder Fragmente enthalten"
return None
def validate_base_url(value: str) -> str:
reason = check_base_url(value)
if reason:
raise ValueError(reason)
return value
+4 -1
View File
@@ -254,6 +254,9 @@ async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseH
try: try:
await _write_config(host, payloads, deployment) await _write_config(host, payloads, deployment)
# Same trap as WireGuard: with the global IPsec switch off the config
# is stored but nothing runs, and reconfigure still answers ok.
await opnsense_client.ensure_ipsec_enabled(host)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
@@ -324,7 +327,7 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel) -> None:
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+11 -3
View File
@@ -11,6 +11,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".psk" _SUFFIX = ".psk"
@@ -21,9 +22,7 @@ def _path(tunnel_id: uuid.UUID) -> Path:
def save_secret(tunnel_id: uuid.UUID, secret: str) -> None: def save_secret(tunnel_id: uuid.UUID, secret: str) -> None:
path = _path(tunnel_id) path = _path(tunnel_id)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(tunnel_id: uuid.UUID) -> str | None: def read_secret(tunnel_id: uuid.UUID) -> str | None:
@@ -35,3 +34,12 @@ def read_secret(tunnel_id: uuid.UUID) -> str | None:
def delete_secret(tunnel_id: uuid.UUID) -> None: def delete_secret(tunnel_id: uuid.UUID) -> None:
_path(tunnel_id).unlink(missing_ok=True) _path(tunnel_id).unlink(missing_ok=True)
def version(tunnel_id: uuid.UUID) -> str | None:
"""Opaque change marker (last write time) — lets an approval token notice a
rotated secret without containing, or being derivable from, the secret."""
try:
return str(_path(tunnel_id).stat().st_mtime_ns)
except FileNotFoundError:
return None
+13 -6
View File
@@ -8,7 +8,8 @@ import hashlib
import json import json
from app.models.policy import IpsecTunnelRevision from app.models.policy import IpsecTunnelRevision
from app.services.ipsec_deploy_service import build_payloads from app.services import ipsec_psk_store
from app.services.ipsec_deploy_service import _redact_for_storage, build_payloads
from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/ipsec.py from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/ipsec.py
IPSEC_FIELDS = ( IPSEC_FIELDS = (
@@ -33,6 +34,9 @@ def revision(db, tunnel, note):
def preview(tunnel): def preview(tunnel):
payloads = build_payloads(tunnel) payloads = build_payloads(tunnel)
# What deployments persist/compare is the redacted form (secret -> hash);
# comparing the raw payload with it would flag an untouched secret as changed.
comparable = _redact_for_storage(payloads)
deployments = {d.host_id: d for d in tunnel.deployments} deployments = {d.host_id: d for d in tunnel.deployments}
hosts = effective_hosts(tunnel) hosts = effective_hosts(tunnel)
rows = [] rows = []
@@ -40,7 +44,7 @@ def preview(tunnel):
dep = deployments.get(host.id) dep = deployments.get(host.id)
before = dep.last_payload if dep else None before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_connection_uuid else "update" action = "create" if not dep or not dep.opnsense_connection_uuid else "update"
if before == payloads and dep and dep.status == "success": if before == comparable and dep and dep.status == "success":
action = "unchanged" action = "unchanged"
if not host.enabled: if not host.enabled:
action = "disabled" action = "disabled"
@@ -48,7 +52,7 @@ def preview(tunnel):
if action not in ("disabled",): if action not in ("disabled",):
for section in ("connection", "local", "remote", "child", "psk"): for section in ("connection", "local", "remote", "child", "psk"):
before_section = (before or {}).get(section, {}) before_section = (before or {}).get(section, {})
after_section = payloads[section] after_section = comparable[section]
for key, value in after_section.items(): for key, value in after_section.items():
if key == "connection": if key == "connection":
continue # UUID filled in at deploy time, not a user-visible field continue # UUID filled in at deploy time, not a user-visible field
@@ -57,7 +61,7 @@ def preview(tunnel):
label = "Pre-Shared-Key" if section == "psk" and key == "Key" else f"{section}.{key}" label = "Pre-Shared-Key" if section == "psk" and key == "Key" else f"{section}.{key}"
# Never surface the PSK's own value in a diff view. # Never surface the PSK's own value in a diff view.
shown_before = "•••" if section == "psk" and key == "Key" and prior else prior shown_before = "•••" if section == "psk" and key == "Key" and prior else prior
shown_after = "•••" if section == "psk" and key == "Key" else value shown_after = "•••" if section == "psk" and key == "Key" and value else value
changes.append({"field": label, "before": shown_before, "after": shown_after}) changes.append({"field": label, "before": shown_before, "after": shown_after})
rows.append({ rows.append({
"host_id": str(host.id), "host_name": host.name, "action": action, "host_id": str(host.id), "host_name": host.name, "action": action,
@@ -89,6 +93,9 @@ def preview(tunnel):
warnings.append("Für diesen Tunnel ist noch kein Pre-Shared-Key hinterlegt.") warnings.append("Für diesen Tunnel ist noch kein Pre-Shared-Key hinterlegt.")
state = { state = {
"snapshot": snapshot(tunnel), "rows": rows, "snapshot": snapshot(tunnel), "rows": rows,
# Rotating only the secret changes nothing else in the snapshot — bind the
# approval to the stored secret's version so an old approval cannot cover it.
"secret_version": ipsec_psk_store.version(tunnel.id),
"hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts], "hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts],
"deployments": sorted( "deployments": sorted(
[(str(d.host_id), d.opnsense_connection_uuid, d.opnsense_local_uuid, d.opnsense_remote_uuid, [(str(d.host_id), d.opnsense_connection_uuid, d.opnsense_local_uuid, d.opnsense_remote_uuid,
@@ -109,12 +116,12 @@ def workflow(tunnel, deployments):
hosts = effective_hosts(tunnel) hosts = effective_hosts(tunnel)
active = [h for h in hosts if h.enabled] active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments} by_host = {d.host_id: d for d in deployments}
payloads = build_payloads(tunnel) comparable = _redact_for_storage(build_payloads(tunnel))
target_ids = {h.id for h in hosts} target_ids = {h.id for h in hosts}
counts = { counts = {
"targets": len(hosts), "targets": len(hosts),
"active_targets": len(active), "active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payloads), "changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != comparable),
"errors": sum(d.status == "error" for d in deployments), "errors": sum(d.status == "error" for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments), "retained": sum(d.host_id not in target_ids for d in deployments),
} }
@@ -44,6 +44,7 @@ def _row_to_certificate(row: dict) -> HostCertificate:
return HostCertificate( return HostCertificate(
uuid=row.get("uuid", ""), uuid=row.get("uuid", ""),
refid=row.get("refid", ""),
descr=row.get("descr") or "(ohne Beschreibung)", descr=row.get("descr") or "(ohne Beschreibung)",
common_name=row.get("commonname") or None, common_name=row.get("commonname") or None,
issuer=row.get("%caref") or "Selbstsigniert", issuer=row.get("%caref") or "Selbstsigniert",
@@ -68,6 +69,7 @@ def _row_to_ca(row: dict) -> HostCa:
return HostCa( return HostCa(
uuid=row.get("uuid", ""), uuid=row.get("uuid", ""),
refid=row.get("refid", ""),
descr=row.get("descr") or "(ohne Beschreibung)", descr=row.get("descr") or "(ohne Beschreibung)",
common_name=row.get("commonname") or None, common_name=row.get("commonname") or None,
valid_from=valid_from, valid_from=valid_from,
+5 -1
View File
@@ -62,6 +62,10 @@ async def _live_rows_or_none(host: OpnsenseHost, nat_type: str) -> list[dict] |
async def compute_cluster_drift(hosts: list[OpnsenseHost], nat_type: str) -> list[dict]: async def compute_cluster_drift(hosts: list[OpnsenseHost], nat_type: str) -> list[dict]:
return (await compute_cluster_drift_with_reachable(hosts, nat_type))[0]
async def compute_cluster_drift_with_reachable(hosts: list[OpnsenseHost], nat_type: str) -> tuple[list[dict], set]:
"""Same shape/contract as rule_drift_service.compute_cluster_drift: """Same shape/contract as rule_drift_service.compute_cluster_drift:
one row per fingerprint missing from at least one reachable host.""" one row per fingerprint missing from at least one reachable host."""
_, fingerprint_fn = _SPECS[nat_type] _, fingerprint_fn = _SPECS[nat_type]
@@ -78,4 +82,4 @@ async def compute_cluster_drift(hosts: list[OpnsenseHost], nat_type: str) -> lis
for entry in by_fingerprint.values(): for entry in by_fingerprint.values():
entry["reachable_host_ids"] = reachable_ids entry["reachable_host_ids"] = reachable_ids
return [entry for entry in by_fingerprint.values() if len(entry["present_on"]) < len(reachable_ids)] return [entry for entry in by_fingerprint.values() if len(entry["present_on"]) < len(reachable_ids)], reachable_ids
@@ -0,0 +1,453 @@
"""Changes to VLANs, virtual IPs, gateways and static routes of ONE firewall.
Unlike rules, these objects are specific to a firewall (own IPs, own VLAN
layout), so there is no central copy and no multi-host push: the firewall is the
source of truth and every change goes straight to it, but only through a
preview/execute pair:
preview reads the live state, computes the diff and the risks, and returns a
token that covers exactly that state.
execute re-reads the live state, refuses if it no longer matches the token,
refuses risky changes without an explicit acknowledgement, saves a
fresh config backup, writes the object, applies it, and reads it back
to check that the firewall really holds what was asked for.
The main risk is cutting the connection OPNexus itself uses (the interface or
address the firewall is reached on, the default route, a route that swallows the
management address). Those cases are detected from the firewall's own
interface list and need the acknowledgement. OPNexus cannot undo a change that
cuts it off, so there is deliberately NO automatic rollback here — the backup
taken before the change is the way back.
Secrets (the CARP password) are never returned, hashed into the token only, and
never written to the audit log.
"""
import hashlib
import ipaddress
import json
import re
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.services import audit_service, backup_store, network_service, opnsense_client
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.opnsense_client import OpnsenseAPIError
# Only these fields can be sent; anything else is rejected, never forwarded.
FIELDS: dict[str, set[str]] = {
"vlan": {"if", "tag", "pcp", "proto", "descr"},
"vip": {"interface", "mode", "network", "vhid", "advbase", "advskew", "password", "descr", "nosync", "peer", "peer6"},
"gateway": {"disabled", "name", "descr", "interface", "ipprotocol", "gateway", "defaultgw", "fargw",
"monitor_disable", "monitor", "priority", "weight"},
"route": {"network", "gateway", "descr", "enabled"},
"dns_host": {"enabled", "hostname", "domain", "rr", "server", "mx", "mxprio", "addptr", "description"},
"dhcp_host": {"host", "domain", "local", "ip", "hwaddr", "client_id", "lease_time", "ignore", "descr"},
"kea_subnet": {"subnet", "pools", "valid_lifetime", "next_server", "description", "option_data.routers",
"option_data.domain_name_servers", "option_data.domain_name", "option_data.ntp_servers"},
"kea_reservation": {"subnet", "ip_address", "hw_address", "client_id", "hostname", "description", "next_server"},
"tunable": {"tunable", "value", "descr"},
"cron_job": {"enabled", "minutes", "hours", "days", "months", "weekdays", "who", "command", "parameters", "description"},
"syslog_destination": {"enabled", "transport", "program", "level", "facility", "hostname", "port", "rfc5424", "description"},
"gateway_group": {"name", "item", "item2", "item3", "item4", "item5", "trigger", "poolopts", "descr"},
"lagg": {"members", "primary_member", "proto", "lacp_fast_timeout", "use_flowid", "lagghash", "lacp_strict", "mtu", "descr"},
"bridge": {"members", "linklocal", "enablestp", "proto", "descr"},
"loopback": {"deviceId", "description"},
"gre": {"local-addr", "remote-addr", "tunnel-local-addr", "tunnel-remote-addr", "tunnel-remote-net", "descr"},
"gif": {"local-addr", "remote-addr", "tunnel-local-addr", "tunnel-remote-addr", "tunnel-remote-net", "descr"},
"vxlan": {"deviceId", "vxlanid", "vxlanlocal", "vxlanlocalport", "vxlanremote", "vxlanremoteport", "vxlangroup", "vxlandev"},
"dns_forward": {"enabled", "type", "domain", "server", "port", "verify", "forward_tcp_upstream", "forward_first", "description"},
"dhcp_range": {"interface", "start_addr", "end_addr", "subnet_mask", "lease_time", "domain_type", "domain", "description"},
"interface": {"if", "descr", "enable", "type4", "ipaddr", "gateway", "type6", "ipaddrv6", "gatewayv6", "mtu",
"blockpriv", "blockbogons", "lock"},
}
SECRET_FIELDS: dict[str, set[str]] = {"vip": {"password"}}
# Section of HostNetwork that must be readable for the kind to be managed on this firewall.
SECTION = {"vlan": "vlans", "vip": "vips", "gateway": "gateways", "route": "routes", "interface": "assignments",
"dns_host": "dns_hosts", "dhcp_host": "dhcp_hosts", "kea_subnet": "kea_subnets",
"kea_reservation": "kea_reservations"}
# Columns the generic object lists show (the firewall's own field names). Lists never carry fields outside FIELDS.
OBJECT_COLUMNS: dict[str, list[str]] = {
"tunable": ["tunable", "value", "default_value", "descr"],
"cron_job": ["enabled", "minutes", "hours", "days", "months", "weekdays", "command", "description"],
"syslog_destination": ["enabled", "transport", "hostname", "port", "program", "level", "description"],
"gateway_group": ["name", "item", "item2", "item3", "trigger", "descr"],
"lagg": ["laggif", "members", "proto", "descr"],
"bridge": ["bridgeif", "members", "proto", "descr"],
"loopback": ["deviceId", "description"],
"gre": ["greif", "local-addr", "remote-addr", "tunnel-local-addr", "tunnel-remote-addr", "descr"],
"gif": ["gifif", "local-addr", "remote-addr", "tunnel-local-addr", "tunnel-remote-addr", "descr"],
"vxlan": ["deviceId", "vxlanid", "vxlanlocal", "vxlanremote", "vxlangroup", "vxlandev"],
"dns_forward": ["enabled", "type", "domain", "server", "port", "description"],
"dhcp_range": ["interface", "start_addr", "end_addr", "subnet_mask", "lease_time", "description"],
}
# These change how the network is wired; the others (DNS/DHCP data) cannot cut the connection to the firewall.
WIRING_KINDS = {"vlan", "vip", "gateway", "route", "interface", "lagg", "bridge", "gre", "gif", "vxlan"}
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
def validate_fields(kind: str, fields: dict) -> dict[str, str]:
allowed = FIELDS[kind]
clean: dict[str, str] = {}
for key, value in fields.items():
if key not in allowed:
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field=str(key)[:40])
if isinstance(value, bool):
value = "1" if value else "0"
if value is None:
value = ""
if not isinstance(value, (str, int)) or isinstance(value, bool):
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field=key)
text = str(value).strip()
if len(text) > 255 or _CONTROL.search(text):
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field=key)
clean[key] = text
return clean
def _explode(node: dict) -> dict:
"""Kea nests its DHCP options one level deep; treat them as "option_data.<name>" like the whitelist does."""
out: dict = {}
for key, value in node.items():
if key == "option_data" and isinstance(value, dict):
for child, inner in value.items():
out[f"option_data.{child}"] = inner
else:
out[key] = value
return out
def flatten(kind: str, node: dict) -> dict[str, str]:
"""OPNsense answers select fields as {option: {selected: 0|1}}; reduce to the selected value(s)."""
out: dict[str, str] = {}
for key, value in _explode(node).items():
if key in SECRET_FIELDS.get(kind, set()):
continue
if isinstance(value, dict):
selected = [k for k, o in value.items() if isinstance(o, dict) and o.get("selected") and k != ""]
if value and all(isinstance(o, dict) and "selected" in o for o in value.values()):
out[key] = ",".join(selected)
continue
out[key] = "" if value is None else str(value)
return out
def _ip(text: str):
try:
return ipaddress.ip_interface(text.strip()).ip
except ValueError:
return None
def _network(text: str):
try:
return ipaddress.ip_network(text.strip(), strict=False)
except ValueError:
return None
def _risk(code: str, **params) -> dict:
return {"code": code, "params": params}
def assess(kind: str, action: str, uuid: str | None, before: dict, after: dict,
net, mgmt_addresses: set, peers: list[str], context: dict | None = None) -> tuple[list[dict], list[dict], list[dict]]:
"""Returns (risks needing acknowledgement, plain warnings, hard blocks)."""
risks: list[dict] = []
warnings: list[dict] = []
blocks: list[dict] = []
mgmt_device = net.management_device
mgmt_iface = next((i for i in net.interfaces if i.device == mgmt_device), None)
mgmt_ident = mgmt_iface.identifier if mgmt_iface else None
if SECTION.get(kind) in net.unavailable:
blocks.append(_risk("api_unavailable", section=SECTION[kind]))
if mgmt_device is None and kind in WIRING_KINDS:
# We cannot tell what this change could cut, so treat every change to existing objects as risky.
if action in ("set", "del"):
risks.append(_risk("management_unknown"))
context = context or {}
if kind == "interface":
if not net.can_configure_interfaces:
blocks.append(_risk("interface_api_unavailable"))
if context.get("pending"):
blocks.append(_risk("interface_changes_pending"))
if action in ("set", "del") and mgmt_ident and uuid == mgmt_ident:
risks.append(_risk("management_interface", device=mgmt_device))
if action == "del" and before.get("lock") == "1":
# OPNsense refuses to remove a locked interface; say so now instead of failing at apply time.
blocks.append(_risk("interface_locked"))
if action == "del":
rules = context.get("rules_using") or []
if rules:
risks.append(_risk("interface_rules_removed", count=len(rules), names=", ".join(rules[:5])))
if action in ("set", "del") and uuid in ("wan", "lan") and uuid != mgmt_ident:
warnings.append(_risk("core_interface", name=uuid))
if kind in ("lagg", "bridge"):
members = {m for side in (before, after) for m in side.get("members", "").split(",") if m}
if mgmt_device and mgmt_device in members:
risks.append(_risk("management_member", device=mgmt_device))
elif mgmt_device is None and action in ("set", "del"):
pass # already covered by management_unknown above
if kind == "tunable":
# sysctl values can change how the whole system behaves (and some only apply after a reboot).
risks.append(_risk("tunable_change"))
if action in ("set", "del") and uuid and "-" not in uuid:
# A built-in default (its id is not a UUID): the firewall would create a new entry instead of
# changing it, so overriding one is an "add", not an edit.
blocks.append(_risk("tunable_default_row"))
if kind == "gateway_group" and action == "del":
warnings.append(_risk("gateway_group_in_use"))
if kind == "vlan" and action in ("set", "del"):
device = next((v.device for v in net.vlans if v.uuid == uuid), None)
if device and device == mgmt_device:
risks.append(_risk("management_interface", device=device))
if kind == "vip":
for side in (before, after) if action != "add" else (after,):
ip = _ip(side.get("network", "")) if side.get("network") else None
if ip is not None and ip in mgmt_addresses:
risks.append(_risk("management_address", address=str(ip)))
break
if "carp" in (before.get("mode"), after.get("mode")):
warnings.append(_risk("carp_change"))
if kind == "gateway":
current = next((g for g in net.gateways if g.uuid == uuid), None)
if current is not None and action in ("set", "del"):
if current.is_default:
risks.append(_risk("default_gateway", name=current.name))
if mgmt_ident and current.interface == mgmt_ident:
risks.append(_risk("management_gateway", name=current.name))
if action in ("add", "set") and after.get("defaultgw") == "1" and (current is None or not current.is_default):
risks.append(_risk("default_gateway_change"))
if kind == "route":
for side in (before, after) if action != "add" else (after,):
network = _network(side.get("network", "")) if side.get("network") else None
if network is None:
continue
if network.prefixlen == 0:
risks.append(_risk("default_route", network=str(network)))
break
if any(a.version == network.version and a in network for a in mgmt_addresses):
risks.append(_risk("route_covers_management", network=str(network)))
break
if peers:
warnings.append(_risk("cluster_peers", hosts=", ".join(peers)))
# De-duplicate while keeping order.
seen, unique = set(), []
for r in risks:
key = json.dumps(r, sort_keys=True)
if key not in seen:
seen.add(key)
unique.append(r)
return unique, warnings, blocks
def _token(host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict, before: dict, mgmt_device) -> str:
state = {
"host": str(host.id), "url": host.base_url, "kind": kind, "action": action, "uuid": uuid,
# hash, so a secret never sits in the token's input in the clear
"fields": hashlib.sha256(json.dumps(fields, sort_keys=True).encode()).hexdigest(),
"before": before, "mgmt": mgmt_device,
}
return hashlib.sha256(json.dumps(state, sort_keys=True).encode()).hexdigest()
def _public(kind: str, fields: dict) -> dict:
return {k: ("(gesetzt)" if k in SECRET_FIELDS.get(kind, set()) and v else v) for k, v in fields.items()
if not (k in SECRET_FIELDS.get(kind, set()) and not v)}
async def _peers(db: AsyncSession, host: OpnsenseHost) -> list[str]:
if not host.cluster_name:
return []
rows = await db.scalars(select(OpnsenseHost.name).where(
OpnsenseHost.cluster_name == host.cluster_name, OpnsenseHost.id != host.id).order_by(OpnsenseHost.name))
return list(rows.all())
async def list_objects(host: OpnsenseHost, kind: str) -> dict:
"""Rows of one generic object kind, reduced to the columns above (plus uuid) — never whole rows,
so nothing the list does not need (or must not show) leaves the server."""
columns = OBJECT_COLUMNS[kind]
try:
rows = await opnsense_client.search_network_objects(host, kind)
except OpnsenseAPIError as exc:
if "404" in str(exc):
return {"available": False, "columns": columns, "rows": []} # this OPNsense version lacks the API
raise
out = []
for row in rows:
item = {"uuid": str(row.get("uuid", ""))}
for col in columns:
value = row.get(col)
item[col] = "" if value is None or isinstance(value, (dict, list)) else str(value)
out.append(item)
return {"available": True, "columns": columns, "rows": out}
async def _rules_using(host: OpnsenseHost, identifier: str) -> list[str]:
"""Descriptions of the user-made rules that name this interface (OPNsense deletes them with it)."""
data = await opnsense_client.search_rules(host, all_rows=True)
rows = data.get("rows", []) if isinstance(data, dict) else []
return [str(r.get("description") or r.get("uuid")) for r in rows
if isinstance(r, dict) and not r.get("is_automatic")
and identifier in str(r.get("interface") or "").split(",")]
async def preview(db: AsyncSession, host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict) -> dict:
fields = validate_fields(kind, fields)
if action in ("set", "del") and not uuid:
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
net = await network_service.get_host_network(host)
before: dict[str, str] = {}
if action in ("set", "del"):
node = await opnsense_client.get_network_item(host, kind, uuid)
if not node or not any(v not in ("", None) for v in node.values() if not isinstance(v, dict)):
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
# Only the fields that may be sent: the rest (e.g. Kea's DDNS key secret) never reaches the token or the audit log.
before = {k: v for k, v in flatten(kind, node).items() if k in FIELDS[kind]}
after = {} if action == "del" else ({**before, **fields} if action == "set" else dict(fields))
mgmt_addresses = await network_service.management_addresses(host)
context: dict = {}
if kind == "interface":
if net.can_configure_interfaces:
context["pending"] = await opnsense_client.interface_changes_pending(host)
if action == "del":
context["rules_using"] = await _rules_using(host, uuid)
risks, warnings, blocks = assess(kind, action, uuid, before, after, net, mgmt_addresses, await _peers(db, host), context)
secret = SECRET_FIELDS.get(kind, set())
changes = []
for key in sorted(set(before) | set(fields)):
old, new = before.get(key), fields.get(key) if action != "del" else None
if action == "del":
if old not in (None, "") and key in FIELDS[kind]:
changes.append({"field": key, "before": old, "after": None})
elif key in fields and (action == "add" or old != new):
if key in secret:
changes.append({"field": key, "before": None, "after": "(gesetzt)" if new else None})
else:
changes.append({"field": key, "before": old, "after": new})
token = _token(host, kind, action, uuid, fields, before, net.management_device)
return {
"token": token, "kind": kind, "action": action, "uuid": uuid, "host_id": str(host.id), "host_name": host.name,
"changes": changes, "risks": risks, "warnings": warnings, "blocks": blocks,
"requires_acknowledgement": bool(risks), "management_device": net.management_device,
"can_execute": not blocks and (action == "del" or bool(changes)),
"_fields": fields, "_before": before,
}
def _matches(expected: dict[str, str], actual: dict[str, str], secret: set[str]) -> list[str]:
return [k for k, v in expected.items() if k not in secret and actual.get(k, "") != v]
async def _verify(host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict) -> tuple[bool, list[str]]:
try:
if action == "del":
present = await opnsense_client.list_network_item_uuids(host, kind)
return (uuid not in present), ([] if uuid not in present else ["still_present"])
actual = flatten(kind, await opnsense_client.get_network_item(host, kind, uuid))
mismatches = _matches(fields, actual, SECRET_FIELDS.get(kind, set()))
return (not mismatches), mismatches
except OpnsenseAPIError:
return False, ["unreadable"]
async def execute(db: AsyncSession, host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict,
token: str, acknowledged: bool) -> dict:
await ensure_no_maintenance(db, [host])
try:
plan = await preview(db, host, kind, action, uuid, fields)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
if plan["token"] != token:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if plan["blocks"]:
raise api_error(409, ErrorCode.NETWORK_CHANGE_BLOCKED, reasons=", ".join(b["code"] for b in plan["blocks"]))
if not plan["can_execute"]:
raise api_error(409, ErrorCode.NETWORK_CHANGE_BLOCKED, reasons="nothing_to_change")
if plan["requires_acknowledgement"] and not acknowledged:
raise api_error(409, ErrorCode.NETWORK_ACK_REQUIRED, reasons=", ".join(r["code"] for r in plan["risks"]))
fields = plan["_fields"]
send = dict(fields)
if kind == "vip" and action == "set" and plan["_before"].get("network"):
# VipSettingsController::getVipOverlay blanks address and prefix whenever "network" is missing from
# the request, so every VIP update must repeat it (found live: a partial set was rejected / could wipe it).
send.setdefault("network", plan["_before"]["network"])
# What the object must look like afterwards: for an edit that is everything it had, plus the change.
expected = {**plan["_before"], **send} if action == "set" else dict(send)
entity = fields.get("name") or fields.get("descr") or plan["_before"].get("name") or plan["_before"].get("descr") or kind
audit = {"kind": "network", "action": f"{kind}.{action}"[:40], "entity_id": uuid, "entity_name": entity,
"host_id": str(host.id), "host_name": host.name}
details = {"before": plan["_before"], "after": _public(kind, fields) if action != "del" else None,
"risks": [r["code"] for r in plan["risks"]], "acknowledged": acknowledged}
# Way back: a config backup taken right before the change. No backup, no change.
try:
backup_id = backup_store.save_snapshot(host.id, await opnsense_client.download_backup(host))
except OpnsenseAPIError as exc:
await audit_service.persist(**audit, status="aborted", message="Backup vor der Änderung fehlgeschlagen", details=details)
raise api_error(502, ErrorCode.NETWORK_BACKUP_FAILED, host=host.name) from exc
details["backup_id"] = backup_id
await audit_service.persist(**audit, status="started", message=None, details=details)
try:
result = await opnsense_client.write_network_item(host, kind, action, uuid, send)
except OpnsenseAPIError as exc:
await audit_service.persist(**audit, status="rejected", message=str(exc)[:500], details=details)
raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)[:300]) from exc
new_uuid = uuid or (result.get("uuid") if isinstance(result, dict) else None)
try:
await opnsense_client.reconfigure_network(host, kind)
except OpnsenseAPIError as exc:
# Written but not (provably) applied: say so instead of reporting success.
await audit_service.persist(**audit, status="apply_uncertain", message=str(exc)[:500], details=details)
raise api_error(502, ErrorCode.NETWORK_APPLY_UNCERTAIN, host=host.name, backup_id=backup_id) from exc
verified, mismatches = await _verify(host, kind, action, new_uuid, expected)
await audit_service.persist(**audit, status="applied" if verified else "unverified",
message=None if verified else ", ".join(mismatches)[:500], details=details)
return {"status": "applied", "uuid": new_uuid, "verified": verified, "mismatches": mismatches, "backup_id": backup_id}
def _options(value: dict) -> list[dict]:
return [{"value": k, "label": str(o.get("value", k)) if isinstance(o, dict) else str(k), "selected": bool(isinstance(o, dict) and o.get("selected"))}
for k, o in value.items()]
async def template(host: OpnsenseHost, kind: str) -> dict:
"""The firewall's own blank form for this kind: field defaults and, for select fields, the choices it
accepts right now (e.g. only the gateways it considers valid for a route)."""
node = _explode(await opnsense_client.get_network_item(host, kind, None))
out: dict[str, dict] = {}
for key in sorted(FIELDS[kind]):
if key not in node:
continue
value = node[key]
if isinstance(value, dict) and value and all(isinstance(o, dict) and "selected" in o for o in value.values()):
options = _options(value)
# Free-text list fields (IPs, MACs, DNS servers) arrive in this shape too; the form decides how to show them.
out[key] = {"value": ",".join(o["value"] for o in options if o["selected"] and o["value"] != ""), "options": options}
elif key in SECRET_FIELDS.get(kind, set()):
out[key] = {"value": "", "options": None}
else:
out[key] = {"value": "" if value is None or isinstance(value, dict) else str(value), "options": None}
return out
async def item(host: OpnsenseHost, kind: str, uuid: str) -> dict[str, str]:
"""One object as the firewall holds it right now, reduced to the fields that may be sent (no secrets)."""
node = await opnsense_client.get_network_item(host, kind, uuid)
if not node or not any(v not in ("", None) for v in node.values() if not isinstance(v, dict)):
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
return {k: v for k, v in flatten(kind, node).items() if k in FIELDS[kind]}
@@ -0,0 +1,86 @@
"""Compares the network configuration of the nodes of an HA cluster.
Interfaces, VLANs, virtual IPs, gateways, routes, DNS and DHCP objects should
be the same on both nodes of a pair; a VLAN or gateway that exists on one node
only is exactly the kind of difference that breaks a failover months later.
Per-node values are deliberately NOT compared: interface addresses, CARP
advskew/password and the active default gateway legitimately differ between
the nodes. What is compared is the identity of each object (its key) and the
settings that must agree (its signature). A category is skipped when any
reachable node cannot deliver it (older OPNsense), instead of reporting that
as a difference.
"""
from app.models.host import OpnsenseHost
from app.schemas.network import HostNetwork
from app.services import network_service
from app.services.opnsense_client import OpnsenseAPIError
CATEGORIES = ["interfaces", "vlans", "vips", "gateways", "routes", "dns_hosts", "dhcp_hosts", "kea_subnets", "kea_reservations"]
def signatures(net: HostNetwork) -> dict[str, dict[str, str]]:
out: dict[str, dict[str, str]] = {c: {} for c in CATEGORIES}
for i in net.interfaces:
if i.assigned and i.identifier:
out["interfaces"][i.identifier] = f"{i.description}|{i.vlan_tag or ''}|{i.ipv4_type or ''}"
for v in net.vlans:
out["vlans"][f"{v.tag}@{v.parent}"] = v.description
for v in net.vips:
if v.mode == "carp":
out["vips"][f"carp:{v.vhid}"] = f"{v.address}|{v.advbase or ''}|{v.description}"
else:
out["vips"][f"{v.mode}:{v.address}"] = v.description
for g in net.gateways:
out["gateways"][g.name] = f"{g.interface}|{g.ip_protocol}|{g.address}|{g.disabled}"
for r in net.routes:
out["routes"][r.network] = f"{r.gateway}|{r.disabled}"
for d in net.dns_hosts:
out["dns_hosts"][f"{d.hostname}.{d.domain}|{d.record_type}"] = f"{d.server}|{d.enabled}"
for h in net.dhcp_hosts:
out["dhcp_hosts"][f"{h.host}.{h.domain}"] = f"{h.ip}|{h.hwaddr}"
for s in net.kea_subnets:
out["kea_subnets"][s.subnet] = f"{s.pools}|{s.routers}|{s.dns_servers}"
for k in net.kea_reservations:
out["kea_reservations"][f"{k.ip_address}|{k.hw_address}"] = k.hostname
return out
# Which HostNetwork.unavailable entry makes a category incomparable.
_UNAVAILABLE_KEY = {"interfaces": "assignments"}
def compare(per_host: dict[str, HostNetwork]) -> list[dict]:
"""per_host: host name -> its network. Returns one row per object that is missing on a node or differs."""
names = sorted(per_host)
sigs = {n: signatures(per_host[n]) for n in names}
rows: list[dict] = []
for category in CATEGORIES:
skip_key = _UNAVAILABLE_KEY.get(category, category)
if any(skip_key in per_host[n].unavailable for n in names):
continue
keys = sorted({k for n in names for k in sigs[n][category]})
for key in keys:
present = {n: sigs[n][category][key] for n in names if key in sigs[n][category]}
if len(present) < len(names):
rows.append({"category": category, "key": key, "status": "missing",
"present_on": sorted(present), "missing_from": sorted(set(names) - set(present)),
"signatures": present})
elif len(set(present.values())) > 1:
rows.append({"category": category, "key": key, "status": "different",
"present_on": names, "missing_from": [], "signatures": present})
return rows
async def compute_cluster(hosts: list[OpnsenseHost]) -> tuple[list[dict], list[str], list[str]]:
"""Returns (differences, compared host names, unreachable host names)."""
per_host: dict[str, HostNetwork] = {}
unreachable: list[str] = []
for host in hosts:
try:
per_host[host.name] = await network_service.get_host_network(host)
except OpnsenseAPIError:
unreachable.append(host.name)
if len(per_host) < 2:
return [], sorted(per_host), sorted(unreachable)
return compare(per_host), sorted(per_host), sorted(unreachable)
+250
View File
@@ -0,0 +1,250 @@
"""Read-only network inventory of one firewall: interfaces, VLANs, virtual
IPs (CARP), gateways and static routes, straight from the firewall.
Also works out which interface the firewall answers OPNexus on. Every later
write feature in this area (VLAN/IP/gateway changes) must refuse or escalate
changes to that interface — an apply there can cut the very connection OPNexus
would need to undo it.
Shapes verified live on OPNsense 26.7.1 and 26.7.6 (see opnsense_client).
"""
import asyncio
import ipaddress
import socket
from urllib.parse import urlparse
from app.models.host import OpnsenseHost
from app.schemas.network import (
HostNetwork,
NetAddress,
NetGateway,
NetDhcpHost,
NetDnsHost,
NetInterface,
NetKeaReservation,
NetKeaSubnet,
NetRoute,
NetVip,
NetVlan,
)
from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError
def _rows(data) -> list[dict]:
rows = data.get("rows", []) if isinstance(data, dict) else []
return [r for r in rows if isinstance(r, dict)]
def _flag(value) -> bool:
return value in ("1", 1, True)
def _text(value) -> str:
return "" if value is None else str(value)
def _opt(value) -> str | None:
"""OPNsense is inconsistent about number vs. string (a gateway priority came back as 254, not "254")."""
return None if value in (None, "") else str(value)
async def management_addresses(host: OpnsenseHost) -> set:
"""IP addresses OPNexus uses to reach this firewall (literal or resolved)."""
hostname = urlparse(host.base_url).hostname
if not hostname:
return set()
try:
return {ipaddress.ip_address(hostname)}
except ValueError:
pass
try:
infos = await asyncio.get_running_loop().getaddrinfo(hostname, None, type=socket.SOCK_STREAM)
except OSError:
return set()
found = set()
for info in infos:
try:
found.add(ipaddress.ip_address(info[4][0].split("%")[0]))
except ValueError:
continue
return found
def _addresses(row: dict) -> list[NetAddress]:
result = []
for key in ("ipv4", "ipv6"):
for entry in row.get(key) or []:
if isinstance(entry, dict) and entry.get("ipaddr"):
result.append(NetAddress(
address=str(entry["ipaddr"]),
vhid=_opt(entry.get("vhid")),
carp_status=_opt(entry.get("status")),
))
return result
def _address_ip(address: str):
try:
return ipaddress.ip_interface(address).ip
except ValueError:
return None
def _interface(row: dict, assignments: dict[str, dict], management: set) -> NetInterface:
identifier = row.get("identifier") or None
assigned = assignments.get(identifier or "") or {}
addresses = _addresses(row)
vlan = row.get("vlan") or {}
return NetInterface(
device=_text(row.get("device")),
identifier=identifier,
description=_text(row.get("description")),
status=_text(row.get("status")),
enabled=_flag(row.get("enabled", True)) if "enabled" in row else True,
is_physical=bool(row.get("is_physical")),
mtu=_opt(row.get("mtu")),
mac=_opt(row.get("macaddr")),
media=_opt(row.get("media")),
vlan_tag=_opt((vlan.get("tag") if isinstance(vlan, dict) else None) or row.get("vlan_tag")),
vlan_parent=_opt(vlan.get("parent")) if isinstance(vlan, dict) else None,
ipv4_type=assigned.get("%type4") or None,
ipv6_type=assigned.get("%type6") or None,
addresses=addresses,
pending_action=assigned.get("pending_action") or None,
assigned=bool(identifier and identifier in assignments),
is_management=any(_address_ip(a.address) in management for a in addresses if not a.vhid),
)
def _vlan(row: dict) -> NetVlan:
return NetVlan(
uuid=_text(row.get("uuid")),
parent=_text(row.get("if")),
tag=_text(row.get("tag")),
pcp=_opt(row.get("pcp")),
description=_text(row.get("descr")),
device=(row.get("vlanif") or "").split(" ")[0] or None,
)
def _vip(row: dict) -> NetVip:
# Deliberately picks fields by name: the row also carries the CARP password.
return NetVip(
uuid=_text(row.get("uuid")),
interface=_text(row.get("interface")),
interface_label=_text(row.get("%interface") or row.get("interface")),
mode=_text(row.get("mode")),
address=_text(row.get("address") or row.get("subnet")),
vhid=_opt(row.get("vhid")),
advbase=_opt(row.get("advbase")),
advskew=_opt(row.get("advskew")),
description=_text(row.get("descr")),
)
def _gateway(row: dict) -> NetGateway:
return NetGateway(
uuid=_text(row.get("uuid")),
name=_text(row.get("name")),
description=_text(row.get("descr")),
interface=_text(row.get("interface")),
interface_label=_text(row.get("interface_descr") or row.get("interface")),
ip_protocol=_text(row.get("ipprotocol")),
address=_text(row.get("gateway")),
is_default=bool(row.get("defaultgw")),
disabled=bool(row.get("disabled")),
priority=_opt(row.get("priority")),
status=_text(row.get("status")),
)
def _route(row: dict) -> NetRoute:
return NetRoute(
uuid=_text(row.get("uuid")),
network=_text(row.get("network")),
gateway=_text(row.get("gateway")),
description=_text(row.get("descr")),
# list rows carry "enabled" ("1"/"0"); older shapes a "disabled" flag
disabled=_flag(row.get("disabled")) or row.get("enabled") in ("0", 0, False),
)
def _dns_host(row: dict) -> NetDnsHost:
return NetDnsHost(
uuid=_text(row.get("uuid")), hostname=_text(row.get("hostname")), domain=_text(row.get("domain")),
record_type=_text(row.get("rr")), server=_text(row.get("server")), enabled=_flag(row.get("enabled")),
description=_text(row.get("description")),
)
def _dhcp_host(row: dict) -> NetDhcpHost:
return NetDhcpHost(
uuid=_text(row.get("uuid")), host=_text(row.get("host")), domain=_text(row.get("domain")),
ip=_text(row.get("ip")), hwaddr=_text(row.get("hwaddr")), lease_time=_text(row.get("lease_time")),
description=_text(row.get("descr")),
)
def _kea_subnet(row: dict) -> NetKeaSubnet:
# Picked by name on purpose: the row also carries the DDNS key secret.
return NetKeaSubnet(
uuid=_text(row.get("uuid")), subnet=_text(row.get("subnet")), pools=_text(row.get("pools")),
routers=_text(row.get("option_data.routers")), dns_servers=_text(row.get("option_data.domain_name_servers")),
description=_text(row.get("description")),
)
def _kea_reservation(row: dict) -> NetKeaReservation:
return NetKeaReservation(
uuid=_text(row.get("uuid")), subnet=_text(row.get("%subnet") or row.get("subnet")),
ip_address=_text(row.get("ip_address")), hw_address=_text(row.get("hw_address")),
hostname=_text(row.get("hostname")), description=_text(row.get("description")),
)
async def _optional(name: str, call, unavailable: list[str]) -> list[dict]:
try:
return _rows(await call)
except OpnsenseAPIError:
unavailable.append(name)
return []
async def get_host_network(host: OpnsenseHost) -> HostNetwork:
"""Raises OpnsenseAPIError when the interface list itself is unavailable
(host down); every other section degrades to `unavailable` instead."""
info_rows = _rows(await opnsense_client.get_interfaces_info(host))
unavailable: list[str] = []
(assignment_rows, vlan_rows, vip_rows, gateway_rows, route_rows,
dns_rows, dhcp_rows, kea_subnet_rows, kea_reservation_rows) = await asyncio.gather(
_optional("assignments", opnsense_client.search_interface_assignments(host), unavailable),
_optional("vlans", opnsense_client.search_vlans(host), unavailable),
_optional("vips", opnsense_client.search_vips(host), unavailable),
_optional("gateways", opnsense_client.search_gateways(host), unavailable),
_optional("routes", opnsense_client.search_static_routes(host), unavailable),
_optional("dns_hosts", opnsense_client.search_dns_hosts(host), unavailable),
_optional("dhcp_hosts", opnsense_client.search_dhcp_hosts(host), unavailable),
_optional("kea_subnets", opnsense_client.search_kea_subnets(host), unavailable),
_optional("kea_reservations", opnsense_client.search_kea_reservations(host), unavailable),
)
management = await management_addresses(host)
assignments = {_text(r.get("identifier")): r for r in assignment_rows if r.get("identifier")}
interfaces = [_interface(r, assignments, management) for r in info_rows]
management_device = next((i.device for i in interfaces if i.is_management), None)
return HostNetwork(
host_id=str(host.id),
host_name=host.name,
interfaces=interfaces,
vlans=[_vlan(r) for r in vlan_rows],
vips=[_vip(r) for r in vip_rows],
gateways=[_gateway(r) for r in gateway_rows],
routes=[_route(r) for r in route_rows],
dns_hosts=[_dns_host(r) for r in dns_rows if not r.get("isAlias")],
dhcp_hosts=[_dhcp_host(r) for r in dhcp_rows],
kea_subnets=[_kea_subnet(r) for r in kea_subnet_rows],
kea_reservations=[_kea_reservation(r) for r in kea_reservation_rows],
management_device=management_device,
can_configure_interfaces=any("type4" in r for r in assignment_rows),
unavailable=sorted(unavailable),
)
+54 -7
View File
@@ -14,7 +14,8 @@ from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.outbound_nat_deployment import OutboundNatDeployment from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.models.port_forward_deployment import PortForwardDeployment from app.models.port_forward_deployment import PortForwardDeployment
from app.models.rule_deployment import RuleDeployment from app.models.rule_deployment import RuleDeployment
from app.services import nat_drift_service, rule_drift_service from app.services import nat_drift_service, network_drift_service, rule_drift_service
from app.services.notification_dedup import Problems
from app.services.host_status_service import get_statuses from app.services.host_status_service import get_statuses
from app.services.live_certificates_service import get_live_certificates from app.services.live_certificates_service import get_live_certificates
from app.services.opnsense_client import OpnsenseAPIError from app.services.opnsense_client import OpnsenseAPIError
@@ -53,13 +54,15 @@ async def check_host_offline(db: AsyncSession) -> dict[str, dict]:
async def check_certificates(db: AsyncSession) -> dict[str, dict]: async def check_certificates(db: AsyncSession) -> dict[str, dict]:
hosts = await _enabled_hosts(db) hosts = await _enabled_hosts(db)
problems: dict[str, dict] = {} problems = Problems()
for host in hosts: for host in hosts:
try: try:
certs = await get_live_certificates(host) certs = await get_live_certificates(host)
except OpnsenseAPIError: except OpnsenseAPIError:
# Unreachable — already reported by check_host_offline, no need # Unreachable — already reported by check_host_offline, no need
# to double-report it here as a certificate problem. # to double-report it here as a certificate problem. Its open
# certificate problems stay as they are (not evaluable now).
problems.preserve_prefixes.add(f"{host.id}:")
continue continue
for cert in certs: for cert in certs:
if cert.status not in ("expiring_soon", "expired"): if cert.status not in ("expiring_soon", "expired"):
@@ -83,12 +86,18 @@ async def check_rule_drift(db: AsyncSession) -> dict[str, dict]:
if host.cluster_name: if host.cluster_name:
by_cluster[host.cluster_name].append(host) by_cluster[host.cluster_name].append(host)
problems: dict[str, dict] = {} problems = Problems()
for cluster_name, cluster_hosts in by_cluster.items(): for cluster_name, cluster_hosts in by_cluster.items():
if len(cluster_hosts) < 2: if len(cluster_hosts) < 2:
continue continue
names_by_id = {h.id: h.name for h in cluster_hosts} names_by_id = {h.id: h.name for h in cluster_hosts}
for row in await rule_drift_service.compute_cluster_drift(cluster_hosts): rows, reachable = await rule_drift_service.compute_cluster_drift_with_reachable(cluster_hosts)
if len(reachable) < 2:
# A member is down (reboot/update): drift cannot be judged, so keep
# what is open instead of resolving it and re-opening it later.
problems.preserve_prefixes.add(f"{cluster_name}:")
continue
for row in rows:
key = f"{cluster_name}:{row['fingerprint']}" key = f"{cluster_name}:{row['fingerprint']}"
present_names = sorted(names_by_id[hid] for hid in row["present_on"]) present_names = sorted(names_by_id[hid] for hid in row["present_on"])
missing_names = sorted( missing_names = sorted(
@@ -113,13 +122,17 @@ async def check_nat_drift(db: AsyncSession) -> dict[str, dict]:
if host.cluster_name: if host.cluster_name:
by_cluster[host.cluster_name].append(host) by_cluster[host.cluster_name].append(host)
problems: dict[str, dict] = {} problems = Problems()
for cluster_name, cluster_hosts in by_cluster.items(): for cluster_name, cluster_hosts in by_cluster.items():
if len(cluster_hosts) < 2: if len(cluster_hosts) < 2:
continue continue
names_by_id = {h.id: h.name for h in cluster_hosts} names_by_id = {h.id: h.name for h in cluster_hosts}
for nat_type in _NAT_TYPE_LABELS: for nat_type in _NAT_TYPE_LABELS:
for row in await nat_drift_service.compute_cluster_drift(cluster_hosts, nat_type): rows, reachable = await nat_drift_service.compute_cluster_drift_with_reachable(cluster_hosts, nat_type)
if len(reachable) < 2:
problems.preserve_prefixes.add(f"{cluster_name}:{nat_type}:")
continue
for row in rows:
key = f"{cluster_name}:{nat_type}:{row['fingerprint']}" key = f"{cluster_name}:{nat_type}:{row['fingerprint']}"
present_names = sorted(names_by_id[hid] for hid in row["present_on"]) present_names = sorted(names_by_id[hid] for hid in row["present_on"])
missing_names = sorted( missing_names = sorted(
@@ -135,6 +148,40 @@ async def check_nat_drift(db: AsyncSession) -> dict[str, dict]:
return problems return problems
_NETWORK_LABELS = {
"interfaces": "Schnittstelle", "vlans": "VLAN", "vips": "Virtuelle IP", "gateways": "Gateway", "routes": "Route",
"dns_hosts": "DNS-Eintrag", "dhcp_hosts": "DHCP-Host", "kea_subnets": "DHCP-Subnetz", "kea_reservations": "DHCP-Reservierung",
}
async def check_network_drift(db: AsyncSession) -> dict[str, dict]:
hosts = await _enabled_hosts(db)
by_cluster: dict[str, list[OpnsenseHost]] = defaultdict(list)
for host in hosts:
if host.cluster_name:
by_cluster[host.cluster_name].append(host)
problems = Problems()
for cluster_name, cluster_hosts in by_cluster.items():
if len(cluster_hosts) < 2:
continue
differences, compared, _unreachable = await network_drift_service.compute_cluster(cluster_hosts)
if len(compared) < 2:
# Not evaluable right now (a node is down, which check_host_offline reports): keep what is open.
problems.preserve_prefixes.add(f"{cluster_name}:")
continue
for row in differences:
key = f"{cluster_name}:{row['category']}:{row['key']}"
problems[key] = {
"cluster_name": cluster_name,
"object": f"{_NETWORK_LABELS.get(row['category'], row['category'])} {row['key']}",
"status": "fehlt auf einem Knoten" if row["status"] == "missing" else "Einstellungen weichen ab",
"present_on": row["present_on"],
"missing_from": row["missing_from"],
}
return problems
async def check_revert_failed(db: AsyncSession) -> dict[str, dict]: async def check_revert_failed(db: AsyncSession) -> dict[str, dict]:
problems: dict[str, dict] = {} problems: dict[str, dict] = {}
for deployment_type, model, id_attr in _DEPLOYMENT_MODELS: for deployment_type, model, id_attr in _DEPLOYMENT_MODELS:
+13 -1
View File
@@ -13,6 +13,17 @@ def _now() -> datetime:
return datetime.now(timezone.utc) return datetime.now(timezone.utc)
class Problems(dict):
"""The result of a check ({dedup_key: context}) plus the key prefixes the
check could NOT evaluate this round (e.g. a cluster member is offline, so
drift is not computable). Open problems under such a prefix are left alone
instead of being resolved and re-opened on the next round."""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.preserve_prefixes: set[str] = set()
async def diff_and_update( async def diff_and_update(
db: AsyncSession, problem_type: str, current: dict[str, dict] db: AsyncSession, problem_type: str, current: dict[str, dict]
) -> tuple[list[NotificationProblem], list[NotificationProblem]]: ) -> tuple[list[NotificationProblem], list[NotificationProblem]]:
@@ -74,8 +85,9 @@ async def diff_and_update(
existing.last_seen_at = now existing.last_seen_at = now
existing.context = context existing.context = context
preserve = tuple(getattr(current, "preserve_prefixes", ()))
for key, row in open_rows.items(): for key, row in open_rows.items():
if key not in current: if key not in current and not (preserve and key.startswith(preserve)):
row.status = "resolved" row.status = "resolved"
row.resolved_at = now row.resolved_at = now
resolved.append(row) resolved.append(row)
+40 -8
View File
@@ -9,6 +9,7 @@ import hashlib
import hmac import hmac
import json import json
import smtplib import smtplib
import ssl
from email.message import EmailMessage from email.message import EmailMessage
import httpx import httpx
@@ -30,6 +31,7 @@ _LABELS = {
"maintenance_unknown": "Wartungsergebnis unklar", "maintenance_unknown": "Wartungsergebnis unklar",
"carp_split_brain": "CARP-Zustand kritisch", "carp_split_brain": "CARP-Zustand kritisch",
"nat_drift": "NAT-Drift im Cluster", "nat_drift": "NAT-Drift im Cluster",
"network_drift": "Netzwerk-Abweichung im Cluster",
"carp_auto_revert_failed": "CARP-Auto-Revert fehlgeschlagen", "carp_auto_revert_failed": "CARP-Auto-Revert fehlgeschlagen",
} }
@@ -67,20 +69,43 @@ async def send_webhook(settings_row: NotificationSettings, problem: Notification
headers = {"Content-Type": "application/json"} headers = {"Content-Type": "application/json"}
if settings_row.webhook_signing_secret_set: if settings_row.webhook_signing_secret_set:
secret = notification_secret_store.read_secret("webhook_signing_secret") secret = notification_secret_store.read_secret("webhook_signing_secret")
if secret: if not secret:
signature = hmac.new(secret.encode("utf-8"), raw, hashlib.sha256).hexdigest() # Fail closed: silently sending an UNSIGNED message to a receiver that
headers["X-OPNexus-Signature"] = f"sha256={signature}" # was told to expect signatures would change the authentication
# behaviour without anyone noticing.
raise NotificationDeliveryError(
"Signaturschlüssel für den Webhook ist konfiguriert, aber nicht lesbar – bitte neu setzen"
)
signature = hmac.new(secret.encode("utf-8"), raw, hashlib.sha256).hexdigest()
headers["X-OPNexus-Signature"] = f"sha256={signature}"
try: try:
async with httpx.AsyncClient(timeout=settings.opnsense_timeout) as client: async with httpx.AsyncClient(timeout=settings.opnsense_timeout) as client:
resp = await client.post(settings_row.webhook_url, content=raw, headers=headers) resp = await client.post(settings_row.webhook_url, content=raw, headers=headers)
resp.raise_for_status() resp.raise_for_status()
except httpx.HTTPError as exc: except httpx.HTTPError as exc:
raise NotificationDeliveryError(f"Webhook-Zustellung fehlgeschlagen: {exc}") from exc # str(exc) names the full URL, which often is the credential itself
# (token in path/query) — keep it out of messages and logs.
reason = f"HTTP {exc.response.status_code}" if isinstance(exc, httpx.HTTPStatusError) else type(exc).__name__
raise NotificationDeliveryError(f"Webhook-Zustellung fehlgeschlagen ({reason})") from None
def _tls_context() -> ssl.SSLContext:
"""Verifying context (certificate chain AND hostname). smtplib's own default
when no context is passed is an *unverifying* one — the connection would be
encrypted but could be impersonated."""
return ssl.create_default_context(cafile=settings.smtp_ca_file or None)
def _send_email_sync(settings_row: NotificationSettings, subject: str, body: str) -> None: def _send_email_sync(settings_row: NotificationSettings, subject: str, body: str) -> None:
password = notification_secret_store.read_secret("smtp_password") password = notification_secret_store.read_secret("smtp_password")
authenticated = bool(settings_row.smtp_username and settings_row.smtp_password_set)
if authenticated and not password:
raise NotificationDeliveryError("SMTP-Passwort ist konfiguriert, aber nicht lesbar – bitte neu setzen")
if authenticated and settings_row.smtp_security == "none" and not settings.allow_insecure_smtp_auth:
raise NotificationDeliveryError(
"SMTP-Anmeldung ohne Verschlüsselung ist deaktiviert – STARTTLS oder SSL/TLS wählen"
)
message = EmailMessage() message = EmailMessage()
message["Subject"] = subject message["Subject"] = subject
message["From"] = settings_row.smtp_from_address message["From"] = settings_row.smtp_from_address
@@ -89,17 +114,22 @@ def _send_email_sync(settings_row: NotificationSettings, subject: str, body: str
timeout = settings.notification_smtp_timeout_seconds timeout = settings.notification_smtp_timeout_seconds
if settings_row.smtp_security == "ssl": if settings_row.smtp_security == "ssl":
client = smtplib.SMTP_SSL(settings_row.smtp_host, settings_row.smtp_port, timeout=timeout) client = smtplib.SMTP_SSL(
settings_row.smtp_host, settings_row.smtp_port, timeout=timeout, context=_tls_context()
)
else: else:
client = smtplib.SMTP(settings_row.smtp_host, settings_row.smtp_port, timeout=timeout) client = smtplib.SMTP(settings_row.smtp_host, settings_row.smtp_port, timeout=timeout)
try: try:
if settings_row.smtp_security == "starttls": if settings_row.smtp_security == "starttls":
client.starttls() client.starttls(context=_tls_context())
if settings_row.smtp_username and password: if authenticated:
client.login(settings_row.smtp_username, password) client.login(settings_row.smtp_username, password)
client.send_message(message) client.send_message(message)
finally: finally:
client.quit() try:
client.quit()
except (smtplib.SMTPException, OSError):
client.close()
async def send_email(settings_row: NotificationSettings, subject: str, body: str) -> None: async def send_email(settings_row: NotificationSettings, subject: str, body: str) -> None:
@@ -107,5 +137,7 @@ async def send_email(settings_row: NotificationSettings, subject: str, body: str
raise NotificationDeliveryError("SMTP-Konfiguration unvollständig") raise NotificationDeliveryError("SMTP-Konfiguration unvollständig")
try: try:
await asyncio.to_thread(_send_email_sync, settings_row, subject, body) await asyncio.to_thread(_send_email_sync, settings_row, subject, body)
except NotificationDeliveryError:
raise
except (OSError, smtplib.SMTPException) as exc: except (OSError, smtplib.SMTPException) as exc:
raise NotificationDeliveryError(f"E-Mail-Zustellung fehlgeschlagen: {exc}") from exc raise NotificationDeliveryError(f"E-Mail-Zustellung fehlgeschlagen: {exc}") from exc
@@ -8,6 +8,7 @@ webhook config for the whole install).
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".secret" _SUFFIX = ".secret"
@@ -18,9 +19,7 @@ def _path(name: str) -> Path:
def save_secret(name: str, secret: str) -> None: def save_secret(name: str, secret: str) -> None:
path = _path(name) path = _path(name)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(name: str) -> str | None: def read_secret(name: str) -> str | None:
@@ -28,6 +28,7 @@ _CHECKS = [
("maintenance_unknown", "maintenance_unknown_enabled", notification_checks.check_maintenance_unknown), ("maintenance_unknown", "maintenance_unknown_enabled", notification_checks.check_maintenance_unknown),
("carp_split_brain", "carp_health_enabled", notification_checks.check_carp_health), ("carp_split_brain", "carp_health_enabled", notification_checks.check_carp_health),
("nat_drift", "nat_drift_enabled", notification_checks.check_nat_drift), ("nat_drift", "nat_drift_enabled", notification_checks.check_nat_drift),
("network_drift", "network_drift_enabled", notification_checks.check_network_drift),
("carp_auto_revert_failed", "carp_auto_revert_failed_enabled", notification_checks.check_carp_auto_revert_failed), ("carp_auto_revert_failed", "carp_auto_revert_failed_enabled", notification_checks.check_carp_auto_revert_failed),
] ]
@@ -25,7 +25,7 @@ async def update_settings(db: AsyncSession, payload: NotificationSettingsUpdate)
"smtp_from_address", "smtp_to_addresses", "smtp_from_address", "smtp_to_addresses",
"host_offline_enabled", "cert_expiring_enabled", "rule_drift_enabled", "host_offline_enabled", "cert_expiring_enabled", "rule_drift_enabled",
"revert_failed_enabled", "maintenance_unknown_enabled", "carp_health_enabled", "nat_drift_enabled", "revert_failed_enabled", "maintenance_unknown_enabled", "carp_health_enabled", "nat_drift_enabled",
"carp_auto_revert_failed_enabled", "network_drift_enabled", "carp_auto_revert_failed_enabled",
"poll_interval_seconds", "poll_interval_seconds",
] ]
for field in simple_fields: for field in simple_fields:
@@ -10,7 +10,8 @@ from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment from app.models.one_to_one_deployment import OneToOneDeployment
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
def _now() -> datetime: def _now() -> datetime:
@@ -53,48 +54,23 @@ async def _deploy_to_host(db: AsyncSession, rule: OneToOneRule, host: OpnsenseHo
prior_uuid = existing.opnsense_one_to_one_uuid if existing else None prior_uuid = existing.opnsense_one_to_one_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_one_to_one_rule(host, prior_uuid, payload)
else:
result = await opnsense_client.add_one_to_one_rule(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.apply_one_to_one(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_one_to_one_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = OneToOneDeployment(
one_to_one_id=rule.id, host_id=host.id, status="error", message=str(exc),
opnsense_one_to_one_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds) async def write(uuid_):
if existing: if uuid_:
existing.status = "pending" await opnsense_client.set_one_to_one_rule(host, uuid_, payload)
existing.message = None return uuid_
existing.opnsense_one_to_one_uuid = new_uuid result = await opnsense_client.add_one_to_one_rule(host, payload)
existing.deployed_at = _now() new_uuid = result.get("uuid") if isinstance(result, dict) else None
existing.revert_deadline = deadline if not new_uuid:
existing.previous_payload = prior_live_payload raise OpnsenseAmbiguousError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
existing.last_payload = payload return new_uuid
return existing
deployment = OneToOneDeployment( deployment = await push_to_host(
one_to_one_id=rule.id, host_id=host.id, status="pending", message=None, db, existing=existing, uuid_attr="opnsense_one_to_one_uuid",
opnsense_one_to_one_uuid=new_uuid, deployed_at=_now(), revert_deadline=deadline, make=lambda **fields: OneToOneDeployment(one_to_one_id=rule.id, host_id=host.id, **fields),
previous_payload=None, last_payload=payload, prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.apply_one_to_one(host),
) )
db.add(deployment)
return deployment return deployment
@@ -112,7 +88,7 @@ async def deploy_one_to_one(db: AsyncSession, rule: OneToOneRule, hosts: list[Op
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = f"1:1 NAT wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}" deployment.message = f"1:1 NAT wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}"
deployment.revert_deadline = None # The change is live: keep the revert deadline so an unconfirmed result is still undone.
await db.commit() await db.commit()
return deployments return deployments
@@ -138,7 +114,7 @@ async def sweep_expired_one_to_one_deployments(db: AsyncSession) -> list[OneToOn
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(OneToOneDeployment).where( select(OneToOneDeployment).where(
OneToOneDeployment.status == "pending", OneToOneDeployment.revert_deadline <= _now() OneToOneDeployment.status.in_(("pending", "error")), OneToOneDeployment.revert_deadline <= _now()
) )
) )
expired = list(result.scalars().all()) expired = list(result.scalars().all())
@@ -164,7 +140,7 @@ async def undeploy_one_to_one(db: AsyncSession, rule: OneToOneRule) -> None:
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+21 -3
View File
@@ -15,7 +15,8 @@ citations):
Instances.Instance: vpnid (VPNIdField, Required, MinimumValue 1, NO Instances.Instance: vpnid (VPNIdField, Required, MinimumValue 1, NO
<Default> — never sent by this app; OPNsense's own UI never prompts for <Default> — never sent by this app; OPNsense's own UI never prompts for
one either, so its field-type presumably auto-assigns one the same way one either, so its field-type presumably auto-assigns one the same way
on addAction — NOT independently verified, flagged rather than guessed), on addAction — VERIFIED LIVE 2026-10-09 on 26.7.5: it is NOT auto-assigned, add() fails
with "instance.vpnid: A value is required." — see _next_vpnid),
enabled (BooleanField, Default 1), dev_type (OptionValues enabled (BooleanField, Default 1), dev_type (OptionValues
tun/tap/ovpn(DCO), Default "tun" — never sent, relies on the model tun/tap/ovpn(DCO), Default "tun" — never sent, relies on the model
default exactly like IPsec relies on Child.mode defaulting to "tunnel"), default exactly like IPsec relies on Child.mode defaulting to "tunnel"),
@@ -27,7 +28,8 @@ citations):
section below, this is the key finding that reshaped v1), server section below, this is the key finding that reshaped v1), server
(NetworkField, NetMaskRequired Y — server-role tunnel network), cert / (NetworkField, NetMaskRequired Y — server-role tunnel network), cert /
ca (CertificateField, i.e. ModelRelationField into OPNsense's own ca (CertificateField, i.e. ModelRelationField into OPNsense's own
Trust\\Cert store), tls_key (ModelRelationField -> this SAME model's Trust\\Cert store; values are the Trust-Store REFIDs, not row UUIDs —
verified live 2026-10-09), tls_key (ModelRelationField -> this SAME model's
StaticKeys.StaticKey, display "description,mode"), auth/data-ciphers StaticKeys.StaticKey, display "description,mode"), auth/data-ciphers
(cipher suite OptionFields — v1 always omits these, relying on (cipher suite OptionFields — v1 always omits these, relying on
OPNsense's own defaults, exactly like IPsec's proposals default to OPNsense's own defaults, exactly like IPsec's proposals default to
@@ -238,6 +240,18 @@ def build_payloads(instance: OpenVpnInstance, host_id: uuid.UUID) -> dict:
} }
async def _next_vpnid(host: OpnsenseHost) -> str:
data = await opnsense_client.search_openvpn_instances(host)
rows = data.get("rows", []) if isinstance(data, dict) else []
used = []
for row in rows:
try:
used.append(int(row.get("vpnid")))
except (TypeError, ValueError):
continue
return str(max(used, default=0) + 1)
async def _write_config(host: OpnsenseHost, payloads: dict, deployment: OpenVpnInstanceDeployment) -> None: async def _write_config(host: OpnsenseHost, payloads: dict, deployment: OpenVpnInstanceDeployment) -> None:
"""Writes the StaticKey (if enabled) then the Instance, in that """Writes the StaticKey (if enabled) then the Instance, in that
dependency order, persisting each UUID onto `deployment` as soon as dependency order, persisting each UUID onto `deployment` as soon as
@@ -270,6 +284,10 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: OpenVpnI
if deployment.opnsense_instance_uuid: if deployment.opnsense_instance_uuid:
await opnsense_client.set_openvpn_instance(host, deployment.opnsense_instance_uuid, instance_wire) await opnsense_client.set_openvpn_instance(host, deployment.opnsense_instance_uuid, instance_wire)
else: else:
# vpnid is Required and the firewall does NOT assign it (verified live
# on 26.7: "instance.vpnid: A value is required."), so pick the next
# free one. It stays untouched on later set() calls.
instance_wire["vpnid"] = await _next_vpnid(host)
result = await opnsense_client.add_openvpn_instance(host, instance_wire) result = await opnsense_client.add_openvpn_instance(host, instance_wire)
deployment.opnsense_instance_uuid = result.get("uuid") if isinstance(result, dict) else None deployment.opnsense_instance_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_instance_uuid: if not deployment.opnsense_instance_uuid:
@@ -370,7 +388,7 @@ async def undeploy_instance(db: AsyncSession, instance: OpenVpnInstance) -> None
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+11 -3
View File
@@ -21,6 +21,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".ovpnkey" _SUFFIX = ".ovpnkey"
@@ -31,9 +32,7 @@ def _path(instance_id: uuid.UUID) -> Path:
def save_secret(instance_id: uuid.UUID, secret: str) -> None: def save_secret(instance_id: uuid.UUID, secret: str) -> None:
path = _path(instance_id) path = _path(instance_id)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(instance_id: uuid.UUID) -> str | None: def read_secret(instance_id: uuid.UUID) -> str | None:
@@ -45,3 +44,12 @@ def read_secret(instance_id: uuid.UUID) -> str | None:
def delete_secret(instance_id: uuid.UUID) -> None: def delete_secret(instance_id: uuid.UUID) -> None:
_path(instance_id).unlink(missing_ok=True) _path(instance_id).unlink(missing_ok=True)
def version(instance_id: uuid.UUID) -> str | None:
"""Opaque change marker (last write time) — lets an approval token notice a
rotated secret without containing, or being derivable from, the secret."""
try:
return str(_path(instance_id).stat().st_mtime_ns)
except FileNotFoundError:
return None
+6 -2
View File
@@ -7,7 +7,8 @@ import hashlib
import json import json
from app.models.policy import OpenVpnInstanceRevision from app.models.policy import OpenVpnInstanceRevision
from app.services.openvpn_deploy_service import MissingCertError, build_payloads from app.services import openvpn_key_store
from app.services.openvpn_deploy_service import MissingCertError, _redact_for_storage, build_payloads
from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/openvpn.py from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/openvpn.py
OPENVPN_FIELDS = ( OPENVPN_FIELDS = (
@@ -56,6 +57,8 @@ def preview(instance):
"changes": [], "changes": [],
}) })
continue continue
# Compare in the stored (secret -> hash) form, see ipsec_service.preview().
payloads = _redact_for_storage(payloads)
action = "create" if not dep or not dep.opnsense_instance_uuid else "update" action = "create" if not dep or not dep.opnsense_instance_uuid else "update"
if before == payloads and dep and dep.status == "success": if before == payloads and dep and dep.status == "success":
action = "unchanged" action = "unchanged"
@@ -108,6 +111,7 @@ def preview(instance):
blocked = instance.tls_key_enabled and not instance.tls_key_material_set blocked = instance.tls_key_enabled and not instance.tls_key_material_set
state = { state = {
"snapshot": snapshot(instance), "rows": rows, "snapshot": snapshot(instance), "rows": rows,
"secret_version": openvpn_key_store.version(instance.id),
"hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts], "hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts],
"deployments": sorted( "deployments": sorted(
[(str(d.host_id), d.opnsense_statickey_uuid, d.opnsense_instance_uuid, d.last_payload, [(str(d.host_id), d.opnsense_statickey_uuid, d.opnsense_instance_uuid, d.last_payload,
@@ -130,7 +134,7 @@ def _host_changed(instance, by_host, host):
except MissingCertError: except MissingCertError:
return True return True
dep = by_host.get(host.id) dep = by_host.get(host.id)
return dep is None or dep.last_payload != payloads return dep is None or dep.last_payload != _redact_for_storage(payloads)
def workflow(instance, deployments): def workflow(instance, deployments):
+333 -29
View File
@@ -16,7 +16,8 @@ import httpx
from app.config import settings from app.config import settings
from app.models.host import OpnsenseHost from app.models.host import OpnsenseHost
from app.services import opnsense_secret_store, tls_pin_service from app.services import firewall_url, opnsense_secret_store, tls_pin_service
from app.services.redaction import redact_data, redact_text
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -25,7 +26,18 @@ class OpnsenseAPIError(Exception):
pass pass
class OpnsenseAmbiguousError(OpnsenseAPIError):
"""The request may have been executed although no usable answer arrived (lost
reply, timeout while waiting, server error, no UUID returned). A refusal the
firewall actually answered is a plain OpnsenseAPIError; this one means the
outcome of a write is unknown and must be reconciled, not retried blindly."""
def _client(host: OpnsenseHost) -> httpx.AsyncClient: def _client(host: OpnsenseHost) -> httpx.AsyncClient:
# Before the secret is even read: an http:// host must never receive credentials.
reason = firewall_url.check_base_url(host.base_url)
if reason:
raise OpnsenseAPIError(f"Host '{host.name}': {reason}")
secret = opnsense_secret_store.read_secret(host.id) secret = opnsense_secret_store.read_secret(host.id)
if not secret: if not secret:
raise OpnsenseAPIError(f"Kein API-Secret für Host '{host.name}' hinterlegt") raise OpnsenseAPIError(f"Kein API-Secret für Host '{host.name}' hinterlegt")
@@ -44,7 +56,12 @@ async def _do_request(host: OpnsenseHost, method: str, path: str, **kwargs) -> h
resp.raise_for_status() resp.raise_for_status()
return resp return resp
except httpx.HTTPStatusError as exc: except httpx.HTTPStatusError as exc:
raise OpnsenseAPIError(f"{exc.response.status_code} {path}: {exc.response.text}") from exc # The body may reflect submitted secrets: keep only a scrubbed, bounded excerpt.
secret = opnsense_secret_store.read_secret(host.id)
error = OpnsenseAmbiguousError if exc.response.status_code >= 500 else OpnsenseAPIError
raise error(
f"{exc.response.status_code} {path}: {redact_text(exc.response.text, host.api_key, secret)}"
) from exc
except httpx.HTTPError as exc: except httpx.HTTPError as exc:
if host.tls_ca_pem and "CERTIFICATE_VERIFY_FAILED" in str(exc): if host.tls_ca_pem and "CERTIFICATE_VERIFY_FAILED" in str(exc):
raise OpnsenseAPIError( raise OpnsenseAPIError(
@@ -54,12 +71,17 @@ async def _do_request(host: OpnsenseHost, method: str, path: str, **kwargs) -> h
if isinstance(exc, httpx.RemoteProtocolError): if isinstance(exc, httpx.RemoteProtocolError):
# The firewall answered, but the HTTP framing is broken (seen with # The firewall answered, but the HTTP framing is broken (seen with
# OPNsense 26.7: kernel TLS corrupts responses above ~64 KiB). # OPNsense 26.7: kernel TLS corrupts responses above ~64 KiB).
raise OpnsenseAPIError( raise OpnsenseAmbiguousError(
f"{host.name} hat eine beschädigte HTTP-Antwort geliefert ({exc}). " f"{host.name} hat eine beschädigte HTTP-Antwort geliefert ({exc}). "
"Bei OPNsense 26.x hilft meist `kern.ipc.tls.enable=0` (System ▸ Einstellungen ▸ " "Bei OPNsense 26.x hilft meist `kern.ipc.tls.enable=0` (System ▸ Einstellungen ▸ "
"Tunables), siehe Handbuch „Fehlerbehebung“." "Tunables), siehe Handbuch „Fehlerbehebung“."
) from exc ) from exc
raise OpnsenseAPIError(f"Verbindung zu {host.name} fehlgeschlagen: {exc}") from exc # Once the request was (partly) sent, a failed read/write says nothing about
# whether the firewall executed it; a failed connect means it never did.
error = OpnsenseAmbiguousError if isinstance(
exc, (httpx.ReadTimeout, httpx.ReadError, httpx.WriteTimeout, httpx.WriteError)
) else OpnsenseAPIError
raise error(f"Verbindung zu {host.name} fehlgeschlagen: {exc}") from exc
async def _request(host: OpnsenseHost, method: str, path: str, **kwargs) -> Any: async def _request(host: OpnsenseHost, method: str, path: str, **kwargs) -> Any:
@@ -167,8 +189,7 @@ def _mutation_result(data: Any, operation: str, results: set[str]) -> dict:
status = str(data.get("status", "")).strip().lower() status = str(data.get("status", "")).strip().lower()
accepted = result in results if "result" in data else status in {"ok", "done"} accepted = result in results if "result" in data else status in {"ok", "done"}
if data.get("validations") or not accepted or status in {"error", "failed"}: if data.get("validations") or not accepted or status in {"error", "failed"}:
detail = json.dumps(data, ensure_ascii=False)[:350] raise OpnsenseAPIError(f"{operation} von Firewall nicht bestätigt: {redact_data(data)}")
raise OpnsenseAPIError(f"{operation} von Firewall nicht bestätigt: {detail}")
return data return data
@@ -536,6 +557,19 @@ async def del_ipsec_psk(host: OpnsenseHost, uuid_: str) -> dict:
return _mutation_result(await _request(host, "POST", f"/ipsec/preSharedKeys/delItem/{uuid_}"), "IPsec Pre-Shared-Key entfernen", {"deleted", "not found"}) return _mutation_result(await _request(host, "POST", f"/ipsec/preSharedKeys/delItem/{uuid_}"), "IPsec Pre-Shared-Key entfernen", {"deleted", "not found"})
async def ensure_ipsec_enabled(host: OpnsenseHost) -> bool:
"""Switch the service-wide IPsec flag on (`ipsec/connections/isEnabled` /
`toggle/1`, ConnectionsController — it is NOT part of the service
controller). Without it the connections are stored but no daemon runs
(live 26.7.6: is_enabled=false, service status "disabled", yet
reconfigure answered ok). Returns True when it had to be switched on."""
data = await _request(host, "GET", "/ipsec/connections/is_enabled")
if isinstance(data, dict) and data.get("enabled") is True:
return False
_mutation_result(await _request(host, "POST", "/ipsec/connections/toggle/1"), "IPsec aktivieren", {"ok"})
return True
async def reconfigure_ipsec(host: OpnsenseHost) -> dict: async def reconfigure_ipsec(host: OpnsenseHost) -> dict:
"""`ipsec/service/reconfigure` (ServiceController::reconfigureAction, """`ipsec/service/reconfigure` (ServiceController::reconfigureAction,
inherited from ApiMutableServiceControllerBase) — soft-reloads a inherited from ApiMutableServiceControllerBase) — soft-reloads a
@@ -735,19 +769,12 @@ async def reconfigure_openvpn(host: OpnsenseHost) -> dict:
# ServiceController's own `$internalServiceEnabled = 'enabled'` / # ServiceController's own `$internalServiceEnabled = 'enabled'` /
# `$internalServiceClass = General::class` tie together for the # `$internalServiceClass = General::class` tie together for the
# framework's generic service-status plumbing. NOT INDEPENDENTLY # framework's generic service-status plumbing. NOT INDEPENDENTLY
# VERIFIED: whether `wireguard configure` (the configd action # VERIFIED LIVE (26.7.5, 2026-10-09): with General.enabled=0 a deploy
# reconfigureAction shells out to) actually brings up any wg-quick # writes Server+Peer correctly but NO wg interface comes up and
# interface end-to-end while General.enabled=0 — reconfigureAction's # `wireguard/service/show` stays empty, while OPNexus used to report
# OWN PHP body never reads $internalServiceEnabled at all (it always # "success". Enabling it (wireguard/general/set + reconfigure) brings
# runs unconditionally), so this depends on configd's own action # wg0 up. wireguard_deploy_service therefore switches it on before the
# script content, which is out of the controller/model source layer # reconfigure (ensure_wireguard_enabled below); undeploy never touches it.
# this app's verification standard covers (same boundary IPsec/
# OpenVPN draw). Flagged as a real, source-confirmed structural
# difference from IPsec/OpenVPN (neither of which have an equivalent
# separate master switch) rather than silently ignored — see README:
# operators should confirm "VPN > WireGuard > General" is enabled at
# least once via the OPNsense UI before relying on OPNexus-pushed
# tunnels.
# #
# Field names verified against Server.xml/Client.xml (fetched directly, # Field names verified against Server.xml/Client.xml (fetched directly,
# not summarized) — see wireguard_deploy_service.py for the exact # not summarized) — see wireguard_deploy_service.py for the exact
@@ -800,6 +827,21 @@ async def del_wireguard_peer(host: OpnsenseHost, uuid_: str) -> dict:
return _mutation_result(await _request(host, "POST", f"/wireguard/client/delClient/{uuid_}"), "WireGuard-Peer entfernen", {"deleted", "not found"}) return _mutation_result(await _request(host, "POST", f"/wireguard/client/delClient/{uuid_}"), "WireGuard-Peer entfernen", {"deleted", "not found"})
async def ensure_wireguard_enabled(host: OpnsenseHost) -> bool:
"""Turn the service-wide WireGuard switch (General.enabled, default 0) on
if it is off. Returns True when it had to be switched on."""
data = await _request(host, "GET", "/wireguard/general/get")
general = data.get("general", {}) if isinstance(data, dict) else {}
if str(general.get("enabled", "0")) == "1":
return False
_mutation_result(
await _request(host, "POST", "/wireguard/general/set", json={"general": {"enabled": "1"}}),
"WireGuard-Dienst aktivieren",
{"saved"},
)
return True
async def reconfigure_wireguard(host: OpnsenseHost) -> dict: async def reconfigure_wireguard(host: OpnsenseHost) -> dict:
"""`wireguard/service/reconfigure` (ServiceController::reconfigureAction) """`wireguard/service/reconfigure` (ServiceController::reconfigureAction)
— runs interface registration + template reload + `wireguard configure`. — runs interface registration + template reload + `wireguard configure`.
@@ -817,6 +859,10 @@ async def reconfigure_wireguard(host: OpnsenseHost) -> dict:
# CertController.php directly) — see managed_certificate_service.py's # CertController.php directly) — see managed_certificate_service.py's
# module docstring for the full field-mapping writeup. # module docstring for the full field-mapping writeup.
# #
# URL SEGMENTS ARE NOT PHP METHOD NAMES: the router strips the `Action`
# suffix, so addAction() is POST /trust/cert/add (the `...Action` URL is a
# 404 — found live 2026-10-09 on 26.7.5, the respx mocks had hidden it).
#
# CertController extends ApiMutableModelControllerBase exactly like # CertController extends ApiMutableModelControllerBase exactly like
# FilterController/ConnectionsController/etc: searchAction (already used # FilterController/ConnectionsController/etc: searchAction (already used
# via search_certificates() above) / getAction / addAction / setAction / # via search_certificates() above) / getAction / addAction / setAction /
@@ -845,11 +891,11 @@ def _trust_cert_result(data: Any, operation: str) -> dict:
async def add_trust_cert(host: OpnsenseHost, payload: dict) -> dict: async def add_trust_cert(host: OpnsenseHost, payload: dict) -> dict:
return _trust_cert_result(await _request(host, "POST", "/trust/cert/addAction", json={"cert": payload}), "Zertifikat anlegen") return _trust_cert_result(await _request(host, "POST", "/trust/cert/add", json={"cert": payload}), "Zertifikat anlegen")
async def set_trust_cert(host: OpnsenseHost, uuid_: str, payload: dict) -> dict: async def set_trust_cert(host: OpnsenseHost, uuid_: str, payload: dict) -> dict:
return _trust_cert_result(await _request(host, "POST", f"/trust/cert/setAction/{uuid_}", json={"cert": payload}), "Zertifikat ändern") return _trust_cert_result(await _request(host, "POST", f"/trust/cert/set/{uuid_}", json={"cert": payload}), "Zertifikat ändern")
async def del_trust_cert(host: OpnsenseHost, uuid_: str) -> dict: async def del_trust_cert(host: OpnsenseHost, uuid_: str) -> dict:
@@ -858,7 +904,7 @@ async def del_trust_cert(host: OpnsenseHost, uuid_: str) -> dict:
# matching every other del* call in this module (add_alias_item's # matching every other del* call in this module (add_alias_item's
# del_alias_item sibling etc.), so this reuses the same # del_alias_item sibling etc.), so this reuses the same
# {"deleted", "not found"} results set they all share. # {"deleted", "not found"} results set they all share.
return _mutation_result(await _request(host, "POST", f"/trust/cert/delAction/{uuid_}"), "Zertifikat entfernen", {"deleted", "not found"}) return _mutation_result(await _request(host, "POST", f"/trust/cert/del/{uuid_}"), "Zertifikat entfernen", {"deleted", "not found"})
async def get_trust_cert(host: OpnsenseHost, uuid_: str) -> dict: async def get_trust_cert(host: OpnsenseHost, uuid_: str) -> dict:
@@ -919,15 +965,15 @@ def _acme_result(data: Any, operation: str) -> dict:
async def add_acme_account(host: OpnsenseHost, payload: dict) -> dict: async def add_acme_account(host: OpnsenseHost, payload: dict) -> dict:
return _acme_result(await _request(host, "POST", "/acmeclient/accounts/addAction", json={"account": payload}), "ACME-Account anlegen") return _acme_result(await _request(host, "POST", "/acmeclient/accounts/add", json={"account": payload}), "ACME-Account anlegen")
async def update_acme_account(host: OpnsenseHost, uuid_: str, payload: dict) -> dict: async def update_acme_account(host: OpnsenseHost, uuid_: str, payload: dict) -> dict:
return _acme_result(await _request(host, "POST", f"/acmeclient/accounts/updateAction/{uuid_}", json={"account": payload}), "ACME-Account ändern") return _acme_result(await _request(host, "POST", f"/acmeclient/accounts/update/{uuid_}", json={"account": payload}), "ACME-Account ändern")
async def del_acme_account(host: OpnsenseHost, uuid_: str) -> dict: async def del_acme_account(host: OpnsenseHost, uuid_: str) -> dict:
return _mutation_result(await _request(host, "POST", f"/acmeclient/accounts/delAction/{uuid_}"), "ACME-Account entfernen", {"deleted", "not found"}) return _mutation_result(await _request(host, "POST", f"/acmeclient/accounts/del/{uuid_}"), "ACME-Account entfernen", {"deleted", "not found"})
async def get_acme_account(host: OpnsenseHost, uuid_: str) -> dict: async def get_acme_account(host: OpnsenseHost, uuid_: str) -> dict:
@@ -935,15 +981,15 @@ async def get_acme_account(host: OpnsenseHost, uuid_: str) -> dict:
async def add_acme_certificate(host: OpnsenseHost, payload: dict) -> dict: async def add_acme_certificate(host: OpnsenseHost, payload: dict) -> dict:
return _acme_result(await _request(host, "POST", "/acmeclient/certificates/addAction", json={"certificate": payload}), "ACME-Zertifikatsanfrage anlegen") return _acme_result(await _request(host, "POST", "/acmeclient/certificates/add", json={"certificate": payload}), "ACME-Zertifikatsanfrage anlegen")
async def update_acme_certificate(host: OpnsenseHost, uuid_: str, payload: dict) -> dict: async def update_acme_certificate(host: OpnsenseHost, uuid_: str, payload: dict) -> dict:
return _acme_result(await _request(host, "POST", f"/acmeclient/certificates/updateAction/{uuid_}", json={"certificate": payload}), "ACME-Zertifikatsanfrage ändern") return _acme_result(await _request(host, "POST", f"/acmeclient/certificates/update/{uuid_}", json={"certificate": payload}), "ACME-Zertifikatsanfrage ändern")
async def del_acme_certificate(host: OpnsenseHost, uuid_: str) -> dict: async def del_acme_certificate(host: OpnsenseHost, uuid_: str) -> dict:
return _mutation_result(await _request(host, "POST", f"/acmeclient/certificates/delAction/{uuid_}"), "ACME-Zertifikatsanfrage entfernen", {"deleted", "not found"}) return _mutation_result(await _request(host, "POST", f"/acmeclient/certificates/del/{uuid_}"), "ACME-Zertifikatsanfrage entfernen", {"deleted", "not found"})
async def get_acme_certificate(host: OpnsenseHost, uuid_: str) -> dict: async def get_acme_certificate(host: OpnsenseHost, uuid_: str) -> dict:
@@ -962,7 +1008,7 @@ def _acme_action_result(data: Any, operation: str) -> dict:
return data return data
if str(data.get("result", "")).strip().lower() == "failed": if str(data.get("result", "")).strip().lower() == "failed":
raise OpnsenseAPIError(f"{operation}: von der Firewall abgelehnt (Objekt nicht gefunden oder keine POST-Anfrage)") raise OpnsenseAPIError(f"{operation}: von der Firewall abgelehnt (Objekt nicht gefunden oder keine POST-Anfrage)")
raise OpnsenseAPIError(f"{operation}: unerwartete Antwortform der Firewall: {json.dumps(data, ensure_ascii=False)[:350]}") raise OpnsenseAPIError(f"{operation}: unerwartete Antwortform der Firewall: {redact_data(data)}")
async def register_acme_account(host: OpnsenseHost, uuid_: str) -> dict: async def register_acme_account(host: OpnsenseHost, uuid_: str) -> dict:
@@ -981,3 +1027,261 @@ async def sign_acme_certificate(host: OpnsenseHost, uuid_: str) -> dict:
against anything but a respx mock / fully-mocked write path in against anything but a respx mock / fully-mocked write path in
tests/verification.""" tests/verification."""
return _acme_action_result(await _request(host, "POST", f"/acmeclient/certificates/sign/{uuid_}"), "ACME-Zertifikat ausstellen") return _acme_action_result(await _request(host, "POST", f"/acmeclient/certificates/sign/{uuid_}"), "ACME-Zertifikat ausstellen")
# ── Network inventory (read-only) ────────────────────────────────────────
# Interfaces\Api\{Overview,Assignment,VlanSettings,VipSettings}Controller,
# Routing\Api\SettingsController and Routes\Api\RoutesController, verified
# against core source (github.com/opnsense/core, master) AND live with real
# read-only GETs on OPNsense 26.7.1/26.7.6 (2026-10-09). Shapes: the search*
# calls return {"rows": [...]} (flat dicts with "%"-prefixed display
# counterparts); interfacesInfo rows carry device/identifier/status/ipv4[].
# Caution: vip_settings rows contain the CARP password in plain text — callers
# must drop it (see network_service).
async def get_interfaces_info(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/interfaces/overview/interfacesInfo", params={"rowCount": -1})
async def search_interface_assignments(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/interfaces/assignment/searchItem", params={"rowCount": -1})
async def search_vlans(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/interfaces/vlan_settings/searchItem", params={"rowCount": -1})
async def search_vips(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/interfaces/vip_settings/searchItem", params={"rowCount": -1})
async def search_gateways(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/routing/settings/searchGateway", params={"rowCount": -1})
async def search_static_routes(host: OpnsenseHost) -> dict:
return await _request(host, "GET", "/routes/routes/searchroute", params={"rowCount": -1})
# ── Network objects (write) ──────────────────────────────────────────────
# One table for the four config objects handled by network_change_service.
# Sources (core master, read 2026-10-09): Interfaces/Api/VlanSettingsController,
# VipSettingsController, Routing/Api/SettingsController (gateways) and
# Routes/Api/RoutesController. Each add/set takes {<body_key>: {fields}}; set/del
# address the item by UUID; nothing takes effect before the object's reconfigure
# call, which answers {"status": "ok"}. The VIP form posts address+prefix as ONE
# field called "network".
# The interface assignment (Interfaces/Api/AssignmentController, model NetworkInterface) is two-step:
# add/set/del only stage the change (a todo file on the firewall); reconfigure ("interface apply") makes
# it real. Its items are addressed by the interface identifier (wan, lan, opt1), not a UUID, and
# deleting one makes OPNsense itself drop every rule/NAT entry that references it.
NETWORK_ITEM_APIS = {
"vlan": {"body_key": "vlan", "base": "/interfaces/vlan_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem",
"reconfigure": "/interfaces/vlan_settings/reconfigure"},
"vip": {"body_key": "vip", "base": "/interfaces/vip_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem",
"reconfigure": "/interfaces/vip_settings/reconfigure"},
"gateway": {"body_key": "gateway_item", "base": "/routing/settings", "add": "addGateway", "set": "setGateway",
"del": "delGateway", "get": "getGateway", "search": "searchGateway",
"reconfigure": "/routing/settings/reconfigure"},
"route": {"body_key": "route", "base": "/routes/routes", "add": "addroute", "set": "setroute",
"del": "delroute", "get": "getroute", "search": "searchroute",
"reconfigure": "/routes/routes/reconfigure"},
"interface": {"body_key": "interface", "base": "/interfaces/assignment", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem",
"reconfigure": "/interfaces/assignment/reconfigure"},
# DNS / DHCP (Unbound/Api/SettingsController, Dnsmasq/Api/SettingsController, Kea/Api/Dhcpv4Controller;
# reconfigure is the service controller's inherited action). Read 2026-10-09.
"dns_host": {"body_key": "host", "base": "/unbound/settings", "add": "addHostOverride", "set": "setHostOverride",
"del": "delHostOverride", "get": "getHostOverride", "search": "searchHostOverride",
"reconfigure": "/unbound/service/reconfigure"},
"dhcp_host": {"body_key": "host", "base": "/dnsmasq/settings", "add": "addHost", "set": "setHost",
"del": "delHost", "get": "getHost", "search": "searchHost",
"reconfigure": "/dnsmasq/service/reconfigure"},
"kea_subnet": {"body_key": "subnet4", "base": "/kea/dhcpv4", "add": "addSubnet", "set": "setSubnet",
"del": "delSubnet", "get": "getSubnet", "search": "searchSubnet",
"reconfigure": "/kea/service/reconfigure"},
# System / interface devices / DNS+DHCP extras (same controllers pattern; read 2026-10-09).
"tunable": {"body_key": "sysctl", "base": "/core/tunables", "add": "addItem", "set": "setItem", "del": "delItem",
"get": "getItem", "search": "searchItem", "reconfigure": "/core/tunables/reconfigure"},
"cron_job": {"body_key": "job", "base": "/cron/settings", "add": "addJob", "set": "setJob", "del": "delJob",
"get": "getJob", "search": "searchJobs", "reconfigure": "/cron/service/reconfigure"},
"syslog_destination": {"body_key": "destination", "base": "/syslog/settings", "add": "addDestination",
"set": "setDestination", "del": "delDestination", "get": "getDestination",
"search": "searchDestinations", "reconfigure": "/syslog/service/reconfigure"},
"gateway_group": {"body_key": "gateway_group", "base": "/routing/group_settings", "add": "add", "set": "set",
"del": "del", "get": "get", "search": "search", "reconfigure": "/routing/group_settings/reconfigure"},
"lagg": {"body_key": "lagg", "base": "/interfaces/lagg_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/lagg_settings/reconfigure"},
"bridge": {"body_key": "bridge", "base": "/interfaces/bridge_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/bridge_settings/reconfigure"},
"loopback": {"body_key": "loopback", "base": "/interfaces/loopback_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/loopback_settings/reconfigure"},
"gre": {"body_key": "gre", "base": "/interfaces/gre_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/gre_settings/reconfigure"},
"vxlan": {"body_key": "vxlan", "base": "/interfaces/vxlan_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/vxlan_settings/reconfigure"},
"gif": {"body_key": "gif", "base": "/interfaces/gif_settings", "add": "addItem", "set": "setItem",
"del": "delItem", "get": "getItem", "search": "searchItem", "reconfigure": "/interfaces/gif_settings/reconfigure"},
"dns_forward": {"body_key": "dot", "base": "/unbound/settings", "add": "addForward", "set": "setForward",
"del": "delForward", "get": "getForward", "search": "searchForward",
"reconfigure": "/unbound/service/reconfigure"},
"dhcp_range": {"body_key": "range", "base": "/dnsmasq/settings", "add": "addRange", "set": "setRange",
"del": "delRange", "get": "getRange", "search": "searchRange",
"reconfigure": "/dnsmasq/service/reconfigure"},
"kea_reservation": {"body_key": "reservation", "base": "/kea/dhcpv4", "add": "addReservation",
"set": "setReservation", "del": "delReservation", "get": "getReservation",
"search": "searchReservation", "reconfigure": "/kea/service/reconfigure"},
}
def _nest(fields: dict) -> dict:
"""{"option_data.routers": "x"} -> {"option_data": {"routers": "x"}} (the wire form of Kea's option block)."""
out: dict = {}
for key, value in fields.items():
head, _, tail = key.partition(".")
if tail:
out.setdefault(head, {})[tail] = value
else:
out[key] = value
return out
async def get_network_item(host: OpnsenseHost, kind: str, uuid: str | None) -> dict:
api = NETWORK_ITEM_APIS[kind]
path = f"{api['base']}/{api['get']}" + (f"/{uuid}" if uuid else "")
data = await _request(host, "GET", path)
item = data.get(api["body_key"]) if isinstance(data, dict) else None
return item if isinstance(item, dict) else {}
async def write_network_item(host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict) -> dict:
api = NETWORK_ITEM_APIS[kind]
if action == "add":
return _mutation_result(
await _request(host, "POST", f"{api['base']}/{api['add']}", json={api["body_key"]: _nest(fields)}),
"Objekt anlegen", {"saved"})
if action == "set":
return _mutation_result(
await _request(host, "POST", f"{api['base']}/{api['set']}/{uuid}", json={api["body_key"]: _nest(fields)}),
"Objekt ändern", {"saved"})
return _mutation_result(
await _request(host, "POST", f"{api['base']}/{api['del']}/{uuid}"), "Objekt entfernen", {"deleted"})
async def reconfigure_network(host: OpnsenseHost, kind: str) -> dict:
return _mutation_result(
await _request(host, "POST", NETWORK_ITEM_APIS[kind]["reconfigure"]), "Änderung anwenden", set())
async def list_network_item_uuids(host: OpnsenseHost, kind: str) -> list[str]:
api = NETWORK_ITEM_APIS[kind]
data = await _request(host, "GET", f"{api['base']}/{api['search']}", params={"rowCount": -1})
rows = data.get("rows", []) if isinstance(data, dict) else []
return [str(r.get("uuid")) for r in rows if isinstance(r, dict) and r.get("uuid")]
async def interface_changes_pending(host: OpnsenseHost) -> bool:
"""True while staged interface changes wait for an apply (AssignmentController::pendingAction).
Our own apply would apply those too, so a change must not start on top of them."""
data = await _request(host, "GET", "/interfaces/assignment/pending")
return isinstance(data, dict) and data.get("status") == "pending"
async def _search_all(host: OpnsenseHost, path: str) -> dict:
return await _request(host, "GET", path, params={"rowCount": -1})
async def search_dns_hosts(host: OpnsenseHost) -> dict:
return await _search_all(host, "/unbound/settings/searchHostOverride")
async def search_dhcp_hosts(host: OpnsenseHost) -> dict:
return await _search_all(host, "/dnsmasq/settings/searchHost")
async def search_kea_subnets(host: OpnsenseHost) -> dict:
return await _search_all(host, "/kea/dhcpv4/searchSubnet")
async def search_kea_reservations(host: OpnsenseHost) -> dict:
return await _search_all(host, "/kea/dhcpv4/searchReservation")
async def search_network_objects(host: OpnsenseHost, kind: str) -> list[dict]:
api = NETWORK_ITEM_APIS[kind]
data = await _request(host, "GET", f"{api['base']}/{api['search']}", params={"rowCount": -1})
rows = data.get("rows", []) if isinstance(data, dict) else []
return [r for r in rows if isinstance(r, dict)]
# ── ZFS snapshots (Core/Api/SnapshotsController, read 2026-10-09) ────────
# Not a config object: flat form fields instead of {body_key: {...}}, no reconfigure, and the answers are
# free-form ("OK" strings / {"status": ...}), so callers confirm by listing again.
async def list_snapshots(host: OpnsenseHost) -> list[dict]:
data = await _request(host, "GET", "/core/snapshots/search", params={"rowCount": -1})
rows = data.get("rows", []) if isinstance(data, dict) else []
return [r for r in rows if isinstance(r, dict)]
async def create_snapshot(host: OpnsenseHost, name: str, note: str = "") -> None:
data = await _request(host, "POST", "/core/snapshots/add", json={"name": name, "note": note})
if isinstance(data, dict) and (data.get("validations") or str(data.get("status", "")).lower() == "failed"):
raise OpnsenseAPIError(f"Snapshot anlegen von Firewall nicht bestätigt: {redact_data(data)}")
async def delete_snapshot(host: OpnsenseHost, uuid: str) -> None:
data = await _request(host, "POST", f"/core/snapshots/del/{uuid}")
if isinstance(data, dict) and (data.get("validations") or str(data.get("status", "")).lower() == "failed"):
raise OpnsenseAPIError(f"Snapshot löschen von Firewall nicht bestätigt: {redact_data(data)}")
# ── Diagnostics (Diagnostics/Api/{Ping,Traceroute,Portprobe,DnsDiagnostics}Controller, read + live 2026-10-09) ──
# Traceroute/portprobe/DNS answer synchronously to the same "set" call that validates the form:
# {"result": "ok", "response": {...}} or {"result": "failed", "validations": {"<model>.settings.<field>": "..."}}.
# Ping is a job: set (returns a uuid) -> start -> searchJobs shows live counters -> stop -> remove.
# A probe to a closed/blackholed port takes the firewall 10 s, so these calls need a longer timeout.
DIAGNOSTIC_TIMEOUT = 40.0
async def diagnostic_set(host: OpnsenseHost, path: str, model: str, settings: dict) -> dict:
data = await _request(host, "POST", path, json={model: {"settings": settings}}, timeout=DIAGNOSTIC_TIMEOUT)
if not isinstance(data, dict):
raise OpnsenseAPIError("Diagnose: keine auswertbare Antwort der Firewall")
if data.get("result") != "ok":
raise OpnsenseDiagnosticRejected(data.get("validations") or {})
return data
class OpnsenseDiagnosticRejected(OpnsenseAPIError):
"""The firewall refused the input (its own validation); carries {field: message}."""
def __init__(self, validations: dict):
super().__init__("Eingabe von der Firewall abgelehnt")
self.validations = validations
async def ping_job_start(host: OpnsenseHost, settings: dict) -> str:
data = await diagnostic_set(host, "/diagnostics/ping/set", "ping", settings)
job = data.get("uuid")
if not job:
raise OpnsenseAPIError("Ping: die Firewall hat keine Job-ID geliefert")
started = await _request(host, "POST", f"/diagnostics/ping/start/{job}", timeout=DIAGNOSTIC_TIMEOUT)
if not isinstance(started, dict) or started.get("status") != "ok":
await ping_job_remove(host, job)
raise OpnsenseAPIError("Ping: der Job konnte nicht gestartet werden")
return job
async def ping_job_state(host: OpnsenseHost, job: str) -> dict | None:
data = await _request(host, "GET", "/diagnostics/ping/searchJobs", params={"rowCount": -1})
rows = data.get("rows", []) if isinstance(data, dict) else []
return next((r for r in rows if isinstance(r, dict) and r.get("id") == job), None)
async def ping_job_stop(host: OpnsenseHost, job: str) -> None:
await _request(host, "POST", f"/diagnostics/ping/stop/{job}")
async def ping_job_remove(host: OpnsenseHost, job: str) -> None:
await _request(host, "POST", f"/diagnostics/ping/remove/{job}")
@@ -8,6 +8,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_SUFFIX = ".secret" _SUFFIX = ".secret"
@@ -18,9 +19,7 @@ def _path(host_id: uuid.UUID) -> Path:
def save_secret(host_id: uuid.UUID, secret: str) -> None: def save_secret(host_id: uuid.UUID, secret: str) -> None:
path = _path(host_id) path = _path(host_id)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_secret(host_id: uuid.UUID) -> str | None: def read_secret(host_id: uuid.UUID) -> str | None:
@@ -11,7 +11,8 @@ from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.services.outbound_nat_mode import require_manual_mode from app.services.outbound_nat_mode import require_manual_mode
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
def _now() -> datetime: def _now() -> datetime:
@@ -57,48 +58,23 @@ async def _deploy_to_host(db: AsyncSession, rule: OutboundNatRule, host: Opnsens
prior_uuid = existing.opnsense_outbound_nat_uuid if existing else None prior_uuid = existing.opnsense_outbound_nat_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_outbound_nat_rule(host, prior_uuid, payload)
else:
result = await opnsense_client.add_outbound_nat_rule(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.apply_outbound_nat(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_outbound_nat_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = OutboundNatDeployment(
outbound_nat_id=rule.id, host_id=host.id, status="error", message=str(exc),
opnsense_outbound_nat_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds) async def write(uuid_):
if existing: if uuid_:
existing.status = "pending" await opnsense_client.set_outbound_nat_rule(host, uuid_, payload)
existing.message = None return uuid_
existing.opnsense_outbound_nat_uuid = new_uuid result = await opnsense_client.add_outbound_nat_rule(host, payload)
existing.deployed_at = _now() new_uuid = result.get("uuid") if isinstance(result, dict) else None
existing.revert_deadline = deadline if not new_uuid:
existing.previous_payload = prior_live_payload raise OpnsenseAmbiguousError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
existing.last_payload = payload return new_uuid
return existing
deployment = OutboundNatDeployment( deployment = await push_to_host(
outbound_nat_id=rule.id, host_id=host.id, status="pending", message=None, db, existing=existing, uuid_attr="opnsense_outbound_nat_uuid",
opnsense_outbound_nat_uuid=new_uuid, deployed_at=_now(), revert_deadline=deadline, make=lambda **fields: OutboundNatDeployment(outbound_nat_id=rule.id, host_id=host.id, **fields),
previous_payload=None, last_payload=payload, prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.apply_outbound_nat(host),
) )
db.add(deployment)
return deployment return deployment
@@ -119,7 +95,7 @@ async def deploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule, hosts: li
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = f"Outbound NAT wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}" deployment.message = f"Outbound NAT wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}"
deployment.revert_deadline = None # The change is live: keep the revert deadline so an unconfirmed result is still undone.
except Exception: except Exception:
# A mode can change after fleet preflight. Preserve already applied # A mode can change after fleet preflight. Preserve already applied
# hosts and their rollback deadlines even if a later host now blocks. # hosts and their rollback deadlines even if a later host now blocks.
@@ -150,7 +126,7 @@ async def sweep_expired_outbound_nat_deployments(db: AsyncSession) -> list[Outbo
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(OutboundNatDeployment).where( select(OutboundNatDeployment).where(
OutboundNatDeployment.status == "pending", OutboundNatDeployment.revert_deadline <= _now() OutboundNatDeployment.status.in_(("pending", "error")), OutboundNatDeployment.revert_deadline <= _now()
) )
) )
expired = list(result.scalars().all()) expired = list(result.scalars().all())
@@ -176,7 +152,7 @@ async def undeploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule) -> None
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
@@ -37,7 +37,8 @@ from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
from app.services.policy_hierarchy_service import ancestor_chain, descendants, ordered_hierarchy_rules from app.services.policy_hierarchy_service import ancestor_chain, descendants, ordered_hierarchy_rules
@@ -219,62 +220,36 @@ async def _deploy_to_host(db: AsyncSession, rule: PolicyHierarchyRule, host: Opn
prior_uuid = existing.opnsense_policy_hierarchy_uuid if existing else None prior_uuid = existing.opnsense_policy_hierarchy_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_rule(host, prior_uuid, payload)
else:
result = await opnsense_client.add_rule(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.apply(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_policy_hierarchy_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = PolicyHierarchyRuleDeployment(
policy_hierarchy_id=rule.id, host_id=host.id, status="error", message=str(exc),
opnsense_policy_hierarchy_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
# Rule content is live. Persist the uuid now (flushed) so the async def write(uuid_):
# positioning pass below sees this deployment as an anchor too. if uuid_:
if existing: await opnsense_client.set_rule(host, uuid_, payload)
deployment = existing return uuid_
deployment.opnsense_policy_hierarchy_uuid = new_uuid result = await opnsense_client.add_rule(host, payload)
else: new_uuid = result.get("uuid") if isinstance(result, dict) else None
deployment = PolicyHierarchyRuleDeployment( if not new_uuid:
policy_hierarchy_id=rule.id, host_id=host.id, status="pending", raise OpnsenseAmbiguousError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
opnsense_policy_hierarchy_uuid=new_uuid, deployed_at=_now(), return new_uuid
)
db.add(deployment) deployment = await push_to_host(
await db.flush() db, existing=existing, uuid_attr="opnsense_policy_hierarchy_uuid",
make=lambda **fields: PolicyHierarchyRuleDeployment(policy_hierarchy_id=rule.id, host_id=host.id, **fields),
prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.apply(host),
)
if deployment.status != "pending":
return deployment
try: try:
await reassert_ordering(db, host, rule.interface) await reassert_ordering(db, host, rule.interface)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
# Live but not provably in the right position — this must not read # Live but not provably in the right position — this must not read
# as a merely-unconfirmed pending change (see module docstring). # as a merely-unconfirmed pending change (see module docstring), yet
# it stays recoverable: the deadline remains armed.
deployment.status = "error" deployment.status = "error"
deployment.message = f"Regel wurde übernommen, Positionierung ('quick') aber nicht bestätigt: {exc}" deployment.message = f"Regel wurde übernommen, Positionierung ('quick') aber nicht bestätigt: {exc}"
deployment.revert_deadline = None
deployment.deployed_at = _now() deployment.deployed_at = _now()
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds)
deployment.status = "pending"
deployment.message = None
deployment.deployed_at = _now()
deployment.revert_deadline = deadline
deployment.previous_payload = prior_live_payload
deployment.last_payload = payload
return deployment return deployment
@@ -309,7 +284,7 @@ async def sweep_expired_hierarchy_deployments(db: AsyncSession) -> list[PolicyHi
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(PolicyHierarchyRuleDeployment).where( select(PolicyHierarchyRuleDeployment).where(
PolicyHierarchyRuleDeployment.status == "pending", PolicyHierarchyRuleDeployment.status.in_(("pending", "error")),
PolicyHierarchyRuleDeployment.revert_deadline <= _now(), PolicyHierarchyRuleDeployment.revert_deadline <= _now(),
) )
) )
@@ -336,7 +311,7 @@ async def undeploy_hierarchy_rule(db: AsyncSession, rule: PolicyHierarchyRule) -
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+1 -1
View File
@@ -7,7 +7,7 @@ from sqlalchemy import text
from app.models.policy import RuleRevision from app.models.policy import RuleRevision
from app.services.rule_deploy_service import _build_payload from app.services.rule_deploy_service import _build_payload
RULE_FIELDS = ("description", "interface", "direction", "ipprotocol", "protocol", "action", "source_net", "source_port", "destination_net", "destination_port", "enabled") RULE_FIELDS = ("description", "interface", "direction", "ipprotocol", "protocol", "action", "source_net", "source_port", "destination_net", "destination_port", "enabled", "log")
async def policy_lock(db): async def policy_lock(db):
@@ -22,7 +22,8 @@ from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment from app.models.port_forward_deployment import PortForwardDeployment
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
def _now() -> datetime: def _now() -> datetime:
@@ -65,48 +66,23 @@ async def _deploy_to_host(db: AsyncSession, rule: PortForwardRule, host: Opnsens
prior_uuid = existing.opnsense_port_forward_uuid if existing else None prior_uuid = existing.opnsense_port_forward_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_port_forward_rule(host, prior_uuid, payload)
else:
result = await opnsense_client.add_port_forward_rule(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.apply_port_forward(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_port_forward_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = PortForwardDeployment(
port_forward_id=rule.id, host_id=host.id, status="error", message=str(exc),
opnsense_port_forward_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds) async def write(uuid_):
if existing: if uuid_:
existing.status = "pending" await opnsense_client.set_port_forward_rule(host, uuid_, payload)
existing.message = None return uuid_
existing.opnsense_port_forward_uuid = new_uuid result = await opnsense_client.add_port_forward_rule(host, payload)
existing.deployed_at = _now() new_uuid = result.get("uuid") if isinstance(result, dict) else None
existing.revert_deadline = deadline if not new_uuid:
existing.previous_payload = prior_live_payload raise OpnsenseAmbiguousError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
existing.last_payload = payload return new_uuid
return existing
deployment = PortForwardDeployment( deployment = await push_to_host(
port_forward_id=rule.id, host_id=host.id, status="pending", message=None, db, existing=existing, uuid_attr="opnsense_port_forward_uuid",
opnsense_port_forward_uuid=new_uuid, deployed_at=_now(), revert_deadline=deadline, make=lambda **fields: PortForwardDeployment(port_forward_id=rule.id, host_id=host.id, **fields),
previous_payload=None, last_payload=payload, prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.apply_port_forward(host),
) )
db.add(deployment)
return deployment return deployment
@@ -124,7 +100,7 @@ async def deploy_port_forward(db: AsyncSession, rule: PortForwardRule, hosts: li
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = f"Port Forward wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}" deployment.message = f"Port Forward wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}"
deployment.revert_deadline = None # The change is live: keep the revert deadline so an unconfirmed result is still undone.
await db.commit() await db.commit()
return deployments return deployments
@@ -150,7 +126,7 @@ async def sweep_expired_port_forward_deployments(db: AsyncSession) -> list[PortF
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(PortForwardDeployment).where( select(PortForwardDeployment).where(
PortForwardDeployment.status == "pending", PortForwardDeployment.revert_deadline <= _now() PortForwardDeployment.status.in_(("pending", "error")), PortForwardDeployment.revert_deadline <= _now()
) )
) )
expired = list(result.scalars().all()) expired = list(result.scalars().all())
@@ -176,7 +152,7 @@ async def undeploy_port_forward(db: AsyncSession, rule: PortForwardRule) -> None
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+79
View File
@@ -0,0 +1,79 @@
"""Scrub text that came back from a firewall before it is stored, logged or
shown to a user.
An upstream error body can echo the request that caused it — including the
password, pre-shared key or private key we just sent — or contain secret-bearing
configuration. OPNexus keeps these messages (deployment status, maintenance
results, host status detail) and read-only users can see them, so they must not
carry such values. Redaction is applied where the exception is built
(opnsense_client), so every consumer gets the safe version.
"""
import json
import re
from typing import Any
MAX_LENGTH = 300
_REDACTED = "[redacted]"
# Field names whose value is (or may be) secret material.
_SECRET_KEY = re.compile(
r"pass|secret|psk|pre_?shared|private|prv|token|api_?key|apikey|credential|auth|crt|cert|\bkey\b|_key|key_|tls_?key",
re.IGNORECASE,
)
_PEM = re.compile(r"-----BEGIN [A-Z0-9 ]+-----.*?(?:-----END [A-Z0-9 ]+-----|$)", re.DOTALL)
# key=value / key: value pairs in non-JSON text
_PAIR = re.compile(r"(?i)\b([\w.\-]*(?:pass|secret|psk|token|private|api_?key|key)[\w.\-]*)\b([\"']?\s*[=:]\s*)(\"[^\"]*\"|'[^']*'|[^\s,;&]+)")
# Keys under which OPNsense lists per-field validation errors
# ({"validations": {"instance.cert": "Please select a valid certificate"}}).
# There the field name says WHICH field failed. "cert", "crt" and "auth" in it
# do not make the message secret, so those three alone stop triggering the
# redaction (otherwise every certificate/auth error from the firewall reads
# "[redacted]"). Real secret names (psk, key, password, ...) still do.
_VALIDATION_KEYS = {"validations", "validation"}
_SECRET_KEY_IN_VALIDATIONS = re.compile(
r"pass|secret|psk|pre_?shared|private|prv|token|api_?key|apikey|credential|\bkey\b|_key|key_|tls_?key",
re.IGNORECASE,
)
def _scrub_json(value: Any, key: str = "", in_validations: bool = False) -> Any:
if isinstance(value, dict):
nested = in_validations or key in _VALIDATION_KEYS
return {k: _scrub_json(v, str(k), nested) for k, v in value.items()}
if isinstance(value, list):
return [_scrub_json(v, key, in_validations) for v in value]
pattern = _SECRET_KEY_IN_VALIDATIONS if in_validations else _SECRET_KEY
if isinstance(value, str) and key and pattern.search(key):
return _REDACTED
if isinstance(value, str):
return _scrub_text(value)
return value
def _scrub_text(text: str) -> str:
return _PAIR.sub(lambda m: f"{m.group(1)}{m.group(2)}{_REDACTED}", _PEM.sub(_REDACTED, text))
def redact_text(text: str, *known_secrets: str | None) -> str:
"""Return a bounded, secret-free rendering of an upstream response body."""
text = text or ""
try:
parsed = json.loads(text)
except ValueError:
parsed = None
if isinstance(parsed, (dict, list)):
text = json.dumps(_scrub_json(parsed), ensure_ascii=False)
else:
text = _scrub_text(text)
for secret in known_secrets:
if secret and len(secret) >= 4:
text = text.replace(secret, _REDACTED)
text = " ".join(text.split())
return text if len(text) <= MAX_LENGTH else text[:MAX_LENGTH] + "…"
def redact_data(data: Any, *known_secrets: str | None) -> str:
"""Same for an already-parsed JSON structure."""
return redact_text(json.dumps(data, ensure_ascii=False, default=str), *known_secrets)
+158
View File
@@ -0,0 +1,158 @@
"""Takes over a firewall rule that was created by hand on a host, so it can be
edited, versioned and deployed from OPNexus like any other rule.
Adoption itself writes NOTHING to the firewall: it records the rule as it is
right now (read fresh from the host at adoption time, never from what the
browser sent) together with its OPNsense UUID. The first change afterwards goes
through the normal preview → push → confirm/auto-revert flow.
Only rules OPNexus can describe without distortion are adopted. A rule using
something the model cannot express (several interfaces, an inverted source, a
non-quick rule, ...) is refused with the reason instead of being imported in a
way that would silently change its meaning on the next edit. Fields the model
does not know about (gateway, state options, schedules, tags, ...) are left
alone on the firewall: setRule only overwrites the fields it is sent.
"""
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.host import OpnsenseHost
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.services import opnsense_client
from app.services.policy_service import revision
from app.services.rule_deploy_service import _build_payload
class RuleNotFoundOnHost(Exception):
pass
class NotAdoptable(Exception):
def __init__(self, reasons: list[str]):
super().__init__(", ".join(reasons))
self.reasons = reasons
def _on(value) -> bool:
return value in ("1", 1, True)
def _clean(value) -> str:
return "" if value is None else str(value).strip()
def reasons_not_adoptable(row: dict, managed_uuids: set[str]) -> list[str]:
"""Machine-readable reasons (the UI translates them); empty = adoptable."""
reasons = []
if row.get("is_automatic"):
reasons.append("automatic")
if _clean(row.get("uuid")) in managed_uuids:
reasons.append("already_managed")
interface = _clean(row.get("interface"))
if not interface or "," in interface:
reasons.append("interface_not_single")
if _on(row.get("interfacenot")):
reasons.append("negated_interface")
if _on(row.get("source_not")):
reasons.append("negated_source")
if _on(row.get("destination_not")):
reasons.append("negated_destination")
if _clean(row.get("received-on")):
reasons.append("received_on")
if _clean(row.get("quick")) != "1":
reasons.append("not_quick")
if _clean(row.get("direction")) not in ("in", "out"):
reasons.append("direction_unsupported")
if _clean(row.get("action")) not in ("pass", "block", "reject"):
reasons.append("action_unsupported")
if _clean(row.get("ipprotocol")) not in ("inet", "inet6"):
reasons.append("ipprotocol_unsupported")
if not _clean(row.get("description")):
reasons.append("no_description")
return reasons
async def _managed_uuids(db: AsyncSession, host: OpnsenseHost) -> set[str]:
rows = await db.scalars(
select(RuleDeployment.opnsense_rule_uuid).where(
RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid.is_not(None)
)
)
return set(rows.all())
async def _live_rows(host: OpnsenseHost) -> list[dict]:
data = await opnsense_client.search_rules(host, all_rows=True)
rows = data.get("rows", []) if isinstance(data, dict) else []
return [r for r in rows if isinstance(r, dict) and r.get("uuid")]
async def list_unmanaged(db: AsyncSession, host: OpnsenseHost) -> list[dict]:
managed = await _managed_uuids(db, host)
result = []
for row in await _live_rows(host):
if row.get("is_automatic"):
continue # built-in/system rules never show up here at all
reasons = reasons_not_adoptable(row, managed)
if "already_managed" in reasons:
continue
result.append({
"uuid": _clean(row.get("uuid")),
"description": _clean(row.get("description")),
"interface": _clean(row.get("interface")),
"interface_label": _clean(row.get("%interface")) or _clean(row.get("interface")),
"direction": _clean(row.get("direction")),
"action": _clean(row.get("action")),
"protocol": _clean(row.get("protocol")) or "any",
"source_net": _clean(row.get("source_net")) or "any",
"source_port": _clean(row.get("source_port")),
"destination_net": _clean(row.get("destination_net")) or "any",
"destination_port": _clean(row.get("destination_port")),
"enabled": _on(row.get("enabled")),
"log": _on(row.get("log")),
"adoptable": not reasons,
"reasons": reasons,
})
return result
async def adopt(db: AsyncSession, host: OpnsenseHost, rule_uuid: str) -> FirewallRule:
managed = await _managed_uuids(db, host)
row = next((r for r in await _live_rows(host) if _clean(r.get("uuid")) == rule_uuid), None)
if row is None:
raise RuleNotFoundOnHost(rule_uuid)
reasons = reasons_not_adoptable(row, managed)
if reasons:
raise NotAdoptable(reasons)
rule = FirewallRule(
description=_clean(row.get("description")),
interface=_clean(row.get("interface")),
direction=_clean(row.get("direction")),
ipprotocol=_clean(row.get("ipprotocol")),
protocol=_clean(row.get("protocol")) or "any",
action=_clean(row.get("action")),
source_net=_clean(row.get("source_net")) or "any",
source_port=_clean(row.get("source_port")) or None,
destination_net=_clean(row.get("destination_net")) or "any",
destination_port=_clean(row.get("destination_port")) or None,
enabled=_on(row.get("enabled")),
log=_on(row.get("log")),
targets=[host],
groups=[],
)
db.add(rule)
await db.flush()
revision(db, rule, "Von Firewall übernommen")
db.add(RuleDeployment(
rule_id=rule.id,
host_id=host.id,
status="success",
message="Von der Firewall übernommen",
opnsense_rule_uuid=rule_uuid,
last_payload=_build_payload(rule),
previous_payload=None,
))
await db.flush()
return rule
+20 -44
View File
@@ -28,7 +28,8 @@ from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment from app.models.rule_deployment import RuleDeployment
from app.services.audit_service import tracked_rollback from app.services.audit_service import tracked_rollback
from app.services import opnsense_client from app.services import opnsense_client
from app.services.opnsense_client import OpnsenseAPIError from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
def _now() -> datetime: def _now() -> datetime:
@@ -49,7 +50,7 @@ def _build_payload(rule: FirewallRule) -> dict:
"destination_net": rule.destination_net, "destination_net": rule.destination_net,
"destination_port": rule.destination_port or "", "destination_port": rule.destination_port or "",
"description": rule.description, "description": rule.description,
"log": "0", "log": "1" if rule.log else "0",
} }
@@ -73,48 +74,23 @@ async def _deploy_to_host(db: AsyncSession, rule: FirewallRule, host: OpnsenseHo
prior_uuid = existing.opnsense_rule_uuid if existing else None prior_uuid = existing.opnsense_rule_uuid if existing else None
prior_live_payload = existing.last_payload if existing else None prior_live_payload = existing.last_payload if existing else None
new_uuid = prior_uuid
try:
if prior_uuid:
await opnsense_client.set_rule(host, prior_uuid, payload)
else:
result = await opnsense_client.add_rule(host, payload)
new_uuid = result.get("uuid") if isinstance(result, dict) else None
if not new_uuid:
raise OpnsenseAPIError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
await opnsense_client.apply(host)
except OpnsenseAPIError as exc:
if existing:
existing.status = "error"
existing.opnsense_rule_uuid = new_uuid
existing.message = str(exc)
existing.revert_deadline = None
existing.deployed_at = _now()
return existing
deployment = RuleDeployment(
rule_id=rule.id, host_id=host.id, status="error", message=str(exc),
opnsense_rule_uuid=new_uuid, deployed_at=_now(),
)
db.add(deployment)
return deployment
deadline = _now() + timedelta(seconds=settings.deploy_confirm_window_seconds) async def write(uuid_):
if existing: if uuid_:
existing.status = "pending" await opnsense_client.set_rule(host, uuid_, payload)
existing.message = None return uuid_
existing.opnsense_rule_uuid = new_uuid result = await opnsense_client.add_rule(host, payload)
existing.deployed_at = _now() new_uuid = result.get("uuid") if isinstance(result, dict) else None
existing.revert_deadline = deadline if not new_uuid:
existing.previous_payload = prior_live_payload raise OpnsenseAmbiguousError("Firewall hat keine Regel-UUID zurückgegeben; Stand manuell prüfen")
existing.last_payload = payload return new_uuid
return existing
deployment = RuleDeployment( deployment = await push_to_host(
rule_id=rule.id, host_id=host.id, status="pending", message=None, db, existing=existing, uuid_attr="opnsense_rule_uuid",
opnsense_rule_uuid=new_uuid, deployed_at=_now(), revert_deadline=deadline, make=lambda **fields: RuleDeployment(rule_id=rule.id, host_id=host.id, **fields),
previous_payload=None, last_payload=payload, prior_uuid=prior_uuid, prior_live_payload=prior_live_payload, payload=payload,
write=write, apply=lambda: opnsense_client.apply(host),
) )
db.add(deployment)
return deployment return deployment
@@ -132,7 +108,7 @@ async def deploy_rule(db: AsyncSession, rule: FirewallRule, hosts: list[Opnsense
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = f"Regel wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}" deployment.message = f"Regel wurde übernommen, Hierarchie-Reihenfolge aber nicht bestätigt: {exc}"
deployment.revert_deadline = None # The change is live: keep the revert deadline so an unconfirmed result is still undone.
await db.commit() await db.commit()
return deployments return deployments
@@ -159,7 +135,7 @@ async def sweep_expired_deployments(db: AsyncSession) -> list[RuleDeployment]:
from app.services.policy_service import policy_lock from app.services.policy_service import policy_lock
await policy_lock(db) await policy_lock(db)
result = await db.execute( result = await db.execute(
select(RuleDeployment).where(RuleDeployment.status == "pending", RuleDeployment.revert_deadline <= _now()) select(RuleDeployment).where(RuleDeployment.status.in_(("pending", "error")), RuleDeployment.revert_deadline <= _now())
) )
expired = list(result.scalars().all()) expired = list(result.scalars().all())
touched: list[RuleDeployment] = [] touched: list[RuleDeployment] = []
@@ -184,7 +160,7 @@ async def undeploy_rule(db: AsyncSession, rule: FirewallRule) -> None:
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+5 -1
View File
@@ -38,6 +38,10 @@ async def _live_rules_or_none(host: OpnsenseHost) -> list[LiveRule] | None:
async def compute_cluster_drift(hosts: list[OpnsenseHost]) -> list[dict]: async def compute_cluster_drift(hosts: list[OpnsenseHost]) -> list[dict]:
return (await compute_cluster_drift_with_reachable(hosts))[0]
async def compute_cluster_drift_with_reachable(hosts: list[OpnsenseHost]) -> tuple[list[dict], set]:
"""Returns one row per fingerprint that isn't present on every reachable """Returns one row per fingerprint that isn't present on every reachable
host in the cluster: {"fingerprint": str, "sample": LiveRule, host in the cluster: {"fingerprint": str, "sample": LiveRule,
"present_on": set[host_id], "reachable_host_ids": set[host_id]}. "present_on": set[host_id], "reachable_host_ids": set[host_id]}.
@@ -58,4 +62,4 @@ async def compute_cluster_drift(hosts: list[OpnsenseHost]) -> list[dict]:
for entry in by_fingerprint.values(): for entry in by_fingerprint.values():
entry["reachable_host_ids"] = reachable_ids entry["reachable_host_ids"] = reachable_ids
return [entry for entry in by_fingerprint.values() if len(entry["present_on"]) < len(reachable_ids)] return [entry for entry in by_fingerprint.values() if len(entry["present_on"]) < len(reachable_ids)], reachable_ids
+29
View File
@@ -0,0 +1,29 @@
"""Writing secret material to disk without a window in which it is readable.
``Path.write_text()`` followed by ``chmod(0o600)`` creates the file with the
process umask (typically 0644) first. Here the file is created with mode 0600
from the start, under a fresh unpredictable name (``O_EXCL`` + ``O_NOFOLLOW``,
so nothing planted there is followed), flushed to disk and then renamed over
the destination — readers see either the complete old or the complete new
content, never a partial write, and a symlink at the destination is replaced
rather than written through.
"""
import os
import secrets
from pathlib import Path
def write_private(path: Path, data: str | bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
payload = data.encode() if isinstance(data, str) else data
tmp = path.with_name(f".{path.name}.{secrets.token_hex(8)}.tmp")
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
try:
with os.fdopen(fd, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp, path)
except BaseException:
tmp.unlink(missing_ok=True)
raise
+73
View File
@@ -0,0 +1,73 @@
"""ZFS snapshots (boot environments) of one firewall: list, create, delete.
Snapshots are a safety net before bigger changes, not a rollback button:
activating one needs a reboot and discards everything since, so that action is
deliberately NOT offered here. Deleting refuses the active/next-boot snapshot
(the firewall refuses too; we say so up front). Every step is audited.
"""
import re
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.services import audit_service, opnsense_client
from app.services.opnsense_client import OpnsenseAPIError
NAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$")
def _row(row: dict) -> dict:
return {
"uuid": str(row.get("uuid", "")), "name": str(row.get("name", "")), "active": str(row.get("active", "")),
"size": str(row.get("size", "")), "created": str(row.get("created_str", "")),
"mountpoint": str(row.get("mountpoint", "")),
}
async def list_snapshots(host: OpnsenseHost) -> dict:
try:
supported = (await opnsense_client._request(host, "GET", "/core/snapshots/isSupported")).get("supported")
except OpnsenseAPIError as exc:
if "404" in str(exc):
return {"supported": False, "rows": []}
raise
if not supported:
return {"supported": False, "rows": []}
return {"supported": True, "rows": [_row(r) for r in await opnsense_client.list_snapshots(host)]}
async def create(host: OpnsenseHost, name: str, note: str) -> dict:
if not NAME.match(name):
raise api_error(422, ErrorCode.NETWORK_FIELD_INVALID, field="name")
audit = {"kind": "snapshot", "action": "snapshot.add", "entity_name": name, "host_id": str(host.id), "host_name": host.name}
await audit_service.persist(**audit, status="started", message=None, details={"note": note[:200]})
try:
await opnsense_client.create_snapshot(host, name, note[:200])
created = next((r for r in await opnsense_client.list_snapshots(host) if r.get("name") == name), None)
except OpnsenseAPIError as exc:
await audit_service.persist(**audit, status="rejected", message=str(exc)[:500], details={})
raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)[:300]) from exc
await audit_service.persist(**audit, status="applied" if created else "unverified", message=None, details={})
if not created:
raise api_error(502, ErrorCode.NETWORK_APPLY_UNCERTAIN, host=host.name, backup_id="-")
return _row(created)
async def delete(host: OpnsenseHost, uuid: str) -> None:
rows = await opnsense_client.list_snapshots(host)
target = next((r for r in rows if str(r.get("uuid")) == uuid), None)
if target is None:
raise api_error(404, ErrorCode.NETWORK_ITEM_NOT_FOUND)
if str(target.get("active", "")).strip() not in ("", "-"):
# N = active now, R = active after reboot: removing either would break the boot environment.
raise api_error(409, ErrorCode.NETWORK_CHANGE_BLOCKED, reasons="snapshot_active")
audit = {"kind": "snapshot", "action": "snapshot.del", "entity_name": str(target.get("name")), "host_id": str(host.id), "host_name": host.name}
await audit_service.persist(**audit, status="started", message=None, details={})
try:
await opnsense_client.delete_snapshot(host, uuid)
still = any(str(r.get("uuid")) == uuid for r in await opnsense_client.list_snapshots(host))
except OpnsenseAPIError as exc:
await audit_service.persist(**audit, status="rejected", message=str(exc)[:500], details={})
raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)[:300]) from exc
await audit_service.persist(**audit, status="unverified" if still else "applied", message=None, details={})
if still:
raise api_error(502, ErrorCode.NETWORK_APPLY_UNCERTAIN, host=host.name, backup_id="-")
+13 -4
View File
@@ -17,6 +17,7 @@ from pathlib import Path
from urllib.parse import quote from urllib.parse import quote
from app.config import settings from app.config import settings
from app.services import secure_files
STEP_SECONDS = 30 STEP_SECONDS = 30
DIGITS = 6 DIGITS = 6
@@ -94,14 +95,22 @@ def _path(user_id: uuid.UUID, pending: bool) -> Path:
def save_secret(user_id: uuid.UUID, secret: str, pending: bool = False) -> None: def save_secret(user_id: uuid.UUID, secret: str, pending: bool = False) -> None:
path = _path(user_id, pending) path = _path(user_id, pending)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret)
path.write_text(secret)
path.chmod(0o600)
# An enrolment that was started but never completed expires, so a forgotten
# QR code (or a stolen session's half-finished attempt) does not stay valid.
PENDING_TTL_SECONDS = 15 * 60
def read_secret(user_id: uuid.UUID, pending: bool = False) -> str | None: def read_secret(user_id: uuid.UUID, pending: bool = False) -> str | None:
path = _path(user_id, pending) path = _path(user_id, pending)
return path.read_text().strip() if path.exists() else None if not path.exists():
return None
if pending and time.time() - path.stat().st_mtime > PENDING_TTL_SECONDS:
path.unlink(missing_ok=True)
return None
return path.read_text().strip()
def delete_secrets(user_id: uuid.UUID) -> None: def delete_secrets(user_id: uuid.UUID) -> None:
+70
View File
@@ -0,0 +1,70 @@
"""Client für den optionalen OPNexus-Updater-Container (updater/server.py).
Das Backend kann selbst nichts am Host ändern und soll das auch nicht können:
Es fragt den Updater-Dienst nach dem Zustand und reicht eine bereits
geprüfte Update-/Rollback-Anfrage weiter. Ist der Dienst nicht aktiviert oder
nicht erreichbar, ist das ein normaler Zustand ("nicht verfügbar"), kein Fehler.
"""
import logging
from pathlib import Path
import httpx
from app.config import settings
logger = logging.getLogger(__name__)
_STATUS_TIMEOUT = 50.0 # der Updater fragt docker ab (bis zu 45 s)
_ACTION_TIMEOUT = 15.0
class UpdaterUnavailable(Exception):
"""Updater nicht aktiviert (kein Token) oder nicht erreichbar."""
class UpdaterRejected(Exception):
def __init__(self, code: str):
super().__init__(code)
self.code = code
def _token() -> str:
try:
token = Path(settings.updater_token_file).read_text(encoding="utf-8").strip()
except OSError:
token = ""
if not token:
raise UpdaterUnavailable("not_enabled")
return token
async def _call(method: str, path: str, payload: dict | None, timeout: float) -> dict:
headers = {"Authorization": f"Bearer {_token()}"}
try:
async with httpx.AsyncClient(timeout=timeout) as http:
response = await http.request(method, settings.updater_url.rstrip("/") + path, headers=headers, json=payload)
except httpx.HTTPError as exc:
logger.info("Updater nicht erreichbar: %s", exc.__class__.__name__)
raise UpdaterUnavailable("unreachable") from exc
try:
body = response.json()
except ValueError:
body = {}
if response.status_code == 401:
# Token passt nicht (Volume neu angelegt?) — wie "nicht verfügbar" behandeln.
raise UpdaterUnavailable("unauthorized")
if response.status_code >= 400:
raise UpdaterRejected(str(body.get("error") or "rejected"))
return body
async def status() -> dict:
return await _call("GET", "/status", None, _STATUS_TIMEOUT)
async def start_update(version: str) -> dict:
return await _call("POST", "/update", {"version": version}, _ACTION_TIMEOUT)
async def start_rollback() -> dict:
return await _call("POST", "/rollback", {}, _ACTION_TIMEOUT)
@@ -324,6 +324,9 @@ async def _deploy_to_host(db: AsyncSession, tunnel: WireguardTunnel, host: Opnse
try: try:
await _write_config(host, payloads, deployment) await _write_config(host, payloads, deployment)
# Without the global switch the config is stored but no interface
# ever comes up (live-verified), so a "success" would be a lie.
await opnsense_client.ensure_wireguard_enabled(host)
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
@@ -388,7 +391,7 @@ async def undeploy_tunnel(db: AsyncSession, tunnel: WireguardTunnel) -> None:
except OpnsenseAPIError as exc: except OpnsenseAPIError as exc:
deployment.status = "error" deployment.status = "error"
deployment.message = str(exc) deployment.message = str(exc)
failures.append(host.name) failures.append(f"{host.name} ({str(exc)[:200]})")
continue continue
await db.delete(deployment) await db.delete(deployment)
await db.flush() await db.flush()
+15 -6
View File
@@ -28,6 +28,7 @@ import uuid
from pathlib import Path from pathlib import Path
from app.config import settings from app.config import settings
from app.services import secure_files
_PRIVATE_KEY_SUFFIX = ".wgkey" _PRIVATE_KEY_SUFFIX = ".wgkey"
_PSK_SUFFIX = ".wgpsk" _PSK_SUFFIX = ".wgpsk"
@@ -39,9 +40,7 @@ def _path(tunnel_id: uuid.UUID, suffix: str) -> Path:
def save_private_key(tunnel_id: uuid.UUID, secret: str) -> None: def save_private_key(tunnel_id: uuid.UUID, secret: str) -> None:
path = _path(tunnel_id, _PRIVATE_KEY_SUFFIX) path = _path(tunnel_id, _PRIVATE_KEY_SUFFIX)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_private_key(tunnel_id: uuid.UUID) -> str | None: def read_private_key(tunnel_id: uuid.UUID) -> str | None:
@@ -57,9 +56,7 @@ def delete_private_key(tunnel_id: uuid.UUID) -> None:
def save_psk(tunnel_id: uuid.UUID, secret: str) -> None: def save_psk(tunnel_id: uuid.UUID, secret: str) -> None:
path = _path(tunnel_id, _PSK_SUFFIX) path = _path(tunnel_id, _PSK_SUFFIX)
path.parent.mkdir(parents=True, exist_ok=True) secure_files.write_private(path, secret.strip())
path.write_text(secret.strip())
path.chmod(0o600)
def read_psk(tunnel_id: uuid.UUID) -> str | None: def read_psk(tunnel_id: uuid.UUID) -> str | None:
@@ -71,3 +68,15 @@ def read_psk(tunnel_id: uuid.UUID) -> str | None:
def delete_psk(tunnel_id: uuid.UUID) -> None: def delete_psk(tunnel_id: uuid.UUID) -> None:
_path(tunnel_id, _PSK_SUFFIX).unlink(missing_ok=True) _path(tunnel_id, _PSK_SUFFIX).unlink(missing_ok=True)
def version(tunnel_id: uuid.UUID) -> str:
"""Opaque change marker for both secrets (last write times) — lets an approval
token notice a rotated key/PSK without containing, or being derivable from, them."""
marks = []
for suffix in (_PRIVATE_KEY_SUFFIX, _PSK_SUFFIX):
try:
marks.append(str(_path(tunnel_id, suffix).stat().st_mtime_ns))
except FileNotFoundError:
marks.append("-")
return ":".join(marks)
+6 -3
View File
@@ -8,7 +8,8 @@ import hashlib
import json import json
from app.models.policy import WireguardTunnelRevision from app.models.policy import WireguardTunnelRevision
from app.services.wireguard_deploy_service import build_payloads from app.services import wireguard_key_store
from app.services.wireguard_deploy_service import _redact_for_storage, build_payloads
from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/wireguard.py from app.services.policy_service import effective_hosts # noqa: F401 - re-exported for routers/wireguard.py
WIREGUARD_FIELDS = ( WIREGUARD_FIELDS = (
@@ -43,7 +44,8 @@ def revision(db, tunnel, note):
def preview(tunnel): def preview(tunnel):
payloads = build_payloads(tunnel) # Compare in the stored (secret -> hash) form, see ipsec_service.preview().
payloads = _redact_for_storage(build_payloads(tunnel))
deployments = {d.host_id: d for d in tunnel.deployments} deployments = {d.host_id: d for d in tunnel.deployments}
hosts = effective_hosts(tunnel) hosts = effective_hosts(tunnel)
rows = [] rows = []
@@ -103,6 +105,7 @@ def preview(tunnel):
blocked = not tunnel.server_private_key_set or (tunnel.peer_psk_enabled and not tunnel.peer_psk_set) blocked = not tunnel.server_private_key_set or (tunnel.peer_psk_enabled and not tunnel.peer_psk_set)
state = { state = {
"snapshot": snapshot(tunnel), "rows": rows, "snapshot": snapshot(tunnel), "rows": rows,
"secret_version": wireguard_key_store.version(tunnel.id),
"hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts], "hosts": [(str(h.id), h.base_url, h.enabled, h.verify_tls) for h in hosts],
"deployments": sorted( "deployments": sorted(
[(str(d.host_id), d.opnsense_server_uuid, d.opnsense_peer_uuid, d.last_payload, [(str(d.host_id), d.opnsense_server_uuid, d.opnsense_peer_uuid, d.last_payload,
@@ -123,7 +126,7 @@ def workflow(tunnel, deployments):
hosts = effective_hosts(tunnel) hosts = effective_hosts(tunnel)
active = [h for h in hosts if h.enabled] active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments} by_host = {d.host_id: d for d in deployments}
payloads = build_payloads(tunnel) payloads = _redact_for_storage(build_payloads(tunnel))
target_ids = {h.id for h in hosts} target_ids = {h.id for h in hosts}
counts = { counts = {
"targets": len(hosts), "targets": len(hosts),
+2 -2
View File
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
Versioning scheme: see CHANGELOG.md ("Versionierung"). Versioning scheme: see CHANGELOG.md ("Versionierung").
""" """
APP_VERSION = "1.10.1" APP_VERSION = "1.15.0"
APP_CODENAME = "Lanner" APP_CODENAME = "Gyrfalcon"
@@ -0,0 +1,24 @@
"""Regeln: Protokollieren-Flag (log).
Bisher schrieb jeder Push fest log=0. Damit eine von Hand angelegte Regel mit
Protokollierung übernommen und danach bearbeitet werden kann, ohne dass das
Flag still verloren geht, ist es jetzt ein Feld der Regel.
Revision ID: 0003
Revises: 0002
Create Date: 2026-10-09
"""
from alembic import op
revision = "0003"
down_revision = "0002"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("ALTER TABLE firewall_rules ADD COLUMN IF NOT EXISTS log boolean NOT NULL DEFAULT false")
def downgrade() -> None:
op.execute("ALTER TABLE firewall_rules DROP COLUMN IF EXISTS log")
@@ -0,0 +1,20 @@
"""Benachrichtigung bei Netzwerk-Abweichungen zwischen den Knoten eines Clusters.
Revision ID: 0004
Revises: 0003
Create Date: 2026-10-09
"""
from alembic import op
revision = "0004"
down_revision = "0003"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS network_drift_enabled boolean NOT NULL DEFAULT true")
def downgrade() -> None:
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS network_drift_enabled")
+4 -2
View File
@@ -76,9 +76,11 @@ def _reset_login_throttle():
# across requests/sessions in production), so it leaks between tests # across requests/sessions in production), so it leaks between tests
# unless explicitly cleared. Without this, a lockout test running before # unless explicitly cleared. Without this, a lockout test running before
# a login-success test would spuriously 429 it. # a login-success test would spuriously 429 it.
auth_service.throttle._records.clear() for throttle in (auth_service.throttle, auth_service.account_throttle):
throttle._records.clear()
yield yield
auth_service.throttle._records.clear() for throttle in (auth_service.throttle, auth_service.account_throttle):
throttle._records.clear()
@pytest_asyncio.fixture @pytest_asyncio.fixture
+9
View File
@@ -0,0 +1,9 @@
"""Shared test helpers."""
async def deploy_approved(client, deploy_url: str):
"""VPN deploys require the token of a fresh preview (like the UI's review
step): fetch it, then POST the deploy with it."""
preview = await client.get(deploy_url.rsplit("/deploy", 1)[0] + "/preview")
assert preview.status_code == 200, preview.text
return await client.post(deploy_url, json={"preview_token": preview.json()["token"]})
+14 -14
View File
@@ -75,7 +75,7 @@ async def test_push_acme_account_with_eab(client, respx_mock):
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]), ) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]), )
account_id = resp.json()["id"] account_id = resp.json()["id"]
add_route = respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( add_route = respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
resp = await client.post(f"/api/acme/accounts/{account_id}/push") resp = await client.post(f"/api/acme/accounts/{account_id}/push")
@@ -97,7 +97,7 @@ async def test_register_account_regression_response_shape(client, respx_mock):
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
@@ -118,7 +118,7 @@ async def test_register_account_failed_precheck_shape(client, respx_mock):
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
@@ -175,7 +175,7 @@ async def test_push_certificate_request_injects_account_uuid(client, respx_mock)
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
@@ -183,7 +183,7 @@ async def test_push_certificate_request_injects_account_uuid(client, respx_mock)
resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id)) resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id))
request_id = resp.json()["id"] request_id = resp.json()["id"]
add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/addAction$").mock( add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
resp = await client.post(f"/api/acme/certificates/{request_id}/push") resp = await client.post(f"/api/acme/certificates/{request_id}/push")
@@ -202,14 +202,14 @@ async def test_push_certificate_request_is_noop_when_unchanged(client, respx_moc
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id)) resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id))
request_id = resp.json()["id"] request_id = resp.json()["id"]
add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/addAction$").mock( add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
resp1 = await client.post(f"/api/acme/certificates/{request_id}/push") resp1 = await client.post(f"/api/acme/certificates/{request_id}/push")
@@ -230,7 +230,7 @@ async def test_push_certificate_request_repushes_after_account_switch(client, re
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"], name="Account A")) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"], name="Account A"))
account_a = resp.json()["id"] account_a = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
side_effect=[ side_effect=[
httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-a"}), httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-a"}),
httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-b"}), httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-b"}),
@@ -243,7 +243,7 @@ async def test_push_certificate_request_repushes_after_account_switch(client, re
resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_a)) resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_a))
request = resp.json() request = resp.json()
add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/addAction$").mock( add_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
push_resp = await client.post(f"/api/acme/certificates/{request['id']}/push") push_resp = await client.post(f"/api/acme/certificates/{request['id']}/push")
@@ -257,7 +257,7 @@ async def test_push_certificate_request_repushes_after_account_switch(client, re
resp = await client.put(f"/api/acme/certificates/{request['id']}", json=edit_payload) resp = await client.put(f"/api/acme/certificates/{request['id']}", json=edit_payload)
assert resp.status_code == 200, resp.text assert resp.status_code == 200, resp.text
set_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/updateAction/.*").mock( set_route = respx_mock.post(url__regex=r".*/acmeclient/certificates/update/.*").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
resp = await client.post(f"/api/acme/certificates/{request['id']}/push") resp = await client.post(f"/api/acme/certificates/{request['id']}/push")
@@ -272,14 +272,14 @@ async def test_sign_is_async_trigger_not_a_success_claim(client, respx_mock):
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id)) resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id))
request_id = resp.json()["id"] request_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/certificates/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/certificates/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
await client.post(f"/api/acme/certificates/{request_id}/push") await client.post(f"/api/acme/certificates/{request_id}/push")
@@ -302,14 +302,14 @@ async def test_poll_certificate_status_updates_remote_fields(client, respx_mock)
host = await _create_host(client, "fw1", "https://fw1.test") host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"])) resp = await client.post("/api/acme/accounts", json=_account_payload(host["id"]))
account_id = resp.json()["id"] account_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/accounts/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/accounts/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "acct-uuid-1"})
) )
await client.post(f"/api/acme/accounts/{account_id}/push") await client.post(f"/api/acme/accounts/{account_id}/push")
resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id)) resp = await client.post("/api/acme/certificates", json=_cert_request_payload(host["id"], account_id))
request_id = resp.json()["id"] request_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/acmeclient/certificates/addAction$").mock( respx_mock.post(url__regex=r".*/acmeclient/certificates/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-req-uuid-1"})
) )
await client.post(f"/api/acme/certificates/{request_id}/push") await client.post(f"/api/acme/certificates/{request_id}/push")
+21
View File
@@ -128,6 +128,27 @@ async def test_delete_alias(client: AsyncClient, respx_mock):
assert (await client.get("/api/aliases")).json() == [] assert (await client.get("/api/aliases")).json() == []
@pytest.mark.asyncio
async def test_delete_alias_refused_by_firewall_names_the_reason(client: AsyncClient, respx_mock):
"""Live 26.7: an alias still used by a rule cannot be removed; the user must
see why (it used to be a bare "request failed")."""
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
host = await _create_host(client, "fw1", "https://fw1.test")
r = (await client.post("/api/aliases", json=_alias_payload([host["id"]]))).json()
_mock_alias_push(respx_mock, "https://fw1.test")
assert (await client.post(f"/api/aliases/{r['id']}/deploy")).status_code == 200
assert (await client.post(f"/api/aliases/{r['id']}/confirm")).status_code == 200
respx_mock.post(url__regex=r".*/firewall/alias/delItem/.*").mock(
return_value=httpx.Response(200, json={"status": "failed", "message": "Alias is used by rule FW3 test block"})
)
resp = await client.delete(f"/api/aliases/{r['id']}")
assert resp.status_code == 502, resp.text
detail = resp.json()["detail"]
assert detail["code"] == "UPSTREAM_REJECTED"
assert "used by rule" in detail["params"]["reason"]
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_deploy_alias_without_targets_rejected(client: AsyncClient): async def test_deploy_alias_without_targets_rejected(client: AsyncClient):
r = (await client.post("/api/aliases", json=_alias_payload([]))).json() r = (await client.post("/api/aliases", json=_alias_payload([]))).json()
+51
View File
@@ -157,6 +157,57 @@ async def test_sweep_reverts_expired_window(client, respx_mock):
assert await carp_maintenance_window_service.active_window(db, uuid.UUID(primary["id"])) is None assert await carp_maintenance_window_service.active_window(db, uuid.UUID(primary["id"])) is None
async def _enter_with_expired_window(client, primary):
plan = (await client.post(f"/api/hosts/{primary['id']}/maintenance/carp-preview")).json()
await client.post(
f"/api/hosts/{primary['id']}/maintenance/carp-execute",
json={"preview_id": plan["preview_id"], "confirm_name": "fw-primary", "acknowledge_downtime": True, "auto_revert_minutes": 30},
)
async with SessionLocal() as db:
window = await carp_maintenance_window_service.active_window(db, uuid.UUID(primary["id"]))
window.auto_revert_at = datetime.now(timezone.utc) - timedelta(seconds=1)
await db.commit()
return plan["preview_id"]
async def test_sweep_revert_closes_the_enter_operation_and_unblocks_deploys(client, respx_mock):
"""Nach dem Auto-Revert darf die carp_enter-Aktion nicht offen bleiben:
sie ließe sich nie mehr bestätigen und blockiert sonst alle Deploys im Cluster."""
from app.services.host_maintenance_service import ACTIVE, ensure_no_maintenance
roles, switch = _stateful_cluster(respx_mock)
primary, secondary = await _make_cluster(client)
op_id = await _enter_with_expired_window(client, primary)
async with SessionLocal() as db:
op = await db.get(HostOperation, uuid.UUID(op_id))
assert op.status == "submitted"
await carp_maintenance_window_service.sweep_expired(db)
async with SessionLocal() as db:
op = await db.get(HostOperation, uuid.UUID(op_id))
assert op.status == "cancelled" and op.status not in ACTIVE
assert "Automatisch" in op.message
host = await db.get(__import__("app.models.host", fromlist=["OpnsenseHost"]).OpnsenseHost, uuid.UUID(primary["id"]))
await ensure_no_maintenance(db, [host]) # raises if still blocked
async def test_sweep_failed_revert_keeps_the_enter_operation_open(client, respx_mock):
"""Schlägt der automatische Rückschalt-Versuch fehl, muss ein Mensch nachsehen:
die Aktion bleibt offen."""
roles, switch = _stateful_cluster(respx_mock)
primary, secondary = await _make_cluster(client)
op_id = await _enter_with_expired_window(client, primary)
switch.mock(side_effect=httpx.ConnectError("boom"))
async with SessionLocal() as db:
resolved = await carp_maintenance_window_service.sweep_expired(db)
assert resolved[0].status == "revert_failed"
async with SessionLocal() as db:
assert (await db.get(HostOperation, uuid.UUID(op_id))).status == "submitted"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_sweep_marks_revert_failed_on_toggle_error(client, respx_mock): async def test_sweep_marks_revert_failed_on_toggle_error(client, respx_mock):
respx_mock.get("https://fw-a.test/api/diagnostics/interface/getVipStatus").mock( respx_mock.get("https://fw-a.test/api/diagnostics/interface/getVipStatus").mock(
+13 -6
View File
@@ -95,7 +95,7 @@ async def test_push_managed_certificate(client, respx_mock):
resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"])) resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"]))
cert_id = resp.json()["id"] cert_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/trust/cert/addAction$").mock( respx_mock.post(url__regex=r".*/trust/cert/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"})
) )
resp = await client.post(f"/api/certificates/managed/{cert_id}/push") resp = await client.post(f"/api/certificates/managed/{cert_id}/push")
@@ -112,7 +112,7 @@ async def test_push_is_noop_when_nothing_changed(client, respx_mock):
resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"])) resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"]))
cert_id = resp.json()["id"] cert_id = resp.json()["id"]
add_route = respx_mock.post(url__regex=r".*/trust/cert/addAction$").mock( add_route = respx_mock.post(url__regex=r".*/trust/cert/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"})
) )
resp1 = await client.post(f"/api/certificates/managed/{cert_id}/push") resp1 = await client.post(f"/api/certificates/managed/{cert_id}/push")
@@ -148,14 +148,21 @@ async def test_delete_in_use_certificate_returns_stable_error_code(client, respx
resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"])) resp = await client.post("/api/certificates/managed", json=_cert_payload(host["id"]))
cert_id = resp.json()["id"] cert_id = resp.json()["id"]
respx_mock.post(url__regex=r".*/trust/cert/addAction$").mock( respx_mock.post(url__regex=r".*/trust/cert/add$").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "cert-uuid-1"})
) )
await client.post(f"/api/certificates/managed/{cert_id}/push") await client.post(f"/api/certificates/managed/{cert_id}/push")
respx_mock.post(url__regex=r".*/trust/cert/delAction/.*").mock( respx_mock.post(url__regex=r".*/trust/cert/del/.*").mock(
return_value=httpx.Response(500, json={"message": "OpenVPN - roadwarrior {openvpn.instance.abc}"}) # shape captured from a real OPNsense 26.7.5 (cert used by an OpenVPN instance)
return_value=httpx.Response(
500,
json={"errorMessage": "OpenVPN - roadwarrior {OpenVPN.Instances.Instance.abc}\n", "errorTitle": "Item in use by", "errorLevel": "error"},
)
) )
resp = await client.delete(f"/api/certificates/managed/{cert_id}") resp = await client.delete(f"/api/certificates/managed/{cert_id}")
assert resp.status_code == 502 assert resp.status_code == 502
assert resp.json()["detail"] == {"code": "UPSTREAM_REQUEST_FAILED", "params": {}} detail = resp.json()["detail"]
assert detail["code"] == "UPSTREAM_REJECTED"
assert "Item in use by" in detail["params"]["reason"]
assert "roadwarrior" in detail["params"]["reason"]
+14
View File
@@ -0,0 +1,14 @@
"""OPNsense routes drop the `Action` suffix of the PHP method name
(addAction() -> /add). A URL such as /trust/cert/addAction is a 404 on a real
firewall, but a respx mock written with the same typo happily accepts it
(live-found 2026-10-09), so guard the client source itself."""
import re
from pathlib import Path
import app.services.opnsense_client as client
def test_no_request_path_ends_in_action_suffix():
source = Path(client.__file__).read_text()
offenders = re.findall(r'"/[A-Za-z0-9_/{}$.]*?[a-z]Action(?:/[^"]*)?"', source)
assert offenders == [], offenders
+143
View File
@@ -0,0 +1,143 @@
"""Ping / traceroute / port probe / DNS lookup run from a firewall."""
import asyncio
import json
import httpx
import pytest
from sqlalchemy import select
from app.database import SessionLocal
from app.models.audit import AuditEvent
BASE = "https://10.20.30.85"
async def _host(client, respx_mock, name="fw-diag"):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
return (await client.post("/api/hosts", json={"name": name, "base_url": BASE, "api_key": "k", "api_secret": "s"})).json()["id"]
def _url(path):
return f"{BASE}/api{path}"
async def _events(host_id):
async with SessionLocal() as db:
rows = await db.scalars(select(AuditEvent).where(AuditEvent.host_id == host_id, AuditEvent.kind == "diagnostic").order_by(AuditEvent.created_at))
return list(rows.all())
@pytest.fixture
def fast_sleep(monkeypatch):
async def instant(_seconds):
return None
monkeypatch.setattr(asyncio, "sleep", instant)
@pytest.mark.asyncio
async def test_traceroute_is_parsed_and_audited(client, respx_mock):
host_id = await _host(client, respx_mock)
sent = respx_mock.post(_url("/diagnostics/traceroute/set")).mock(return_value=httpx.Response(200, json={
"result": "ok", "response": {"notice": "traceroute to x", "rows": [
{"address": "10.0.0.1", "ttl": "1", "AS": "AS0", "host": "gw.lan", "probes": "0.4 ms"}]}}))
resp = await client.post(f"/api/hosts/{host_id}/diagnostics/traceroute", json={"target": "example.com", "protocol": "icmp"})
assert resp.status_code == 200, resp.text
assert resp.json()["hops"] == [{"ttl": "1", "address": "10.0.0.1", "host": "gw.lan", "as": "AS0", "probes": "0.4 ms"}]
assert json.loads(sent.calls.last.request.content)["traceroute"]["settings"]["hostname"] == "example.com"
events = await _events(host_id)
assert [(e.action, e.status, e.entity_name) for e in events] == [("diag.traceroute", "ok", "example.com")]
@pytest.mark.asyncio
async def test_bad_targets_never_reach_the_firewall(client, respx_mock):
host_id = await _host(client, respx_mock, "fw-diag2")
route = respx_mock.post(_url("/diagnostics/traceroute/set")).mock(return_value=httpx.Response(200, json={"result": "ok", "response": {}}))
for bad in ("a;b", "x y", "$(reboot)", "-h", "a" * 300, "host/../x"):
resp = await client.post(f"/api/hosts/{host_id}/diagnostics/traceroute", json={"target": bad})
assert resp.status_code == 422, bad
assert not route.called
assert (await client.post(f"/api/hosts/{host_id}/diagnostics/portprobe", json={"target": "x.test", "port": 70000})).status_code == 422
assert (await client.post(f"/api/hosts/{host_id}/diagnostics/ping", json={"target": "x.test", "count": 500})).status_code == 422
assert (await client.post(f"/api/hosts/{host_id}/diagnostics/ping", json={"target": "x.test", "source": "not-an-ip"})).status_code == 422
@pytest.mark.asyncio
async def test_the_firewalls_own_validation_is_reported_as_422_and_audited(client, respx_mock):
host_id = await _host(client, respx_mock, "fw-diag3")
respx_mock.post(_url("/diagnostics/dns_diagnostics/set")).mock(return_value=httpx.Response(200, json={
"result": "failed", "validations": {"dns.settings.server": "Provide a valid server."}}))
resp = await client.post(f"/api/hosts/{host_id}/diagnostics/dns", json={"name": "example.com", "server": "1.2.3.4"})
assert resp.status_code == 422 and resp.json()["detail"]["params"]["field"] == "server"
assert [(e.status) for e in await _events(host_id)] == ["failed"]
@pytest.mark.asyncio
async def test_portprobe_reports_open_or_closed_and_never_the_banner(client, respx_mock):
host_id = await _host(client, respx_mock, "fw-diag4")
answers = iter([
{"result": "ok", "response": {"message": "Connection to 10.0.0.1 443 port [tcp/https] succeeded!", "payload": "SSH-2.0-SECRET-BANNER"}},
{"result": "ok", "response": {"message": "nc: connect to 10.0.0.1 port 1 (tcp) failed: Operation timed out", "payload": ""}},
])
respx_mock.post(_url("/diagnostics/portprobe/set")).mock(side_effect=lambda r: httpx.Response(200, json=next(answers)))
opened = await client.post(f"/api/hosts/{host_id}/diagnostics/portprobe", json={"target": "10.0.0.1", "port": 443})
closed = await client.post(f"/api/hosts/{host_id}/diagnostics/portprobe", json={"target": "10.0.0.1", "port": 1})
assert opened.json()["open"] is True and closed.json()["open"] is False
assert "SECRET-BANNER" not in opened.text
@pytest.mark.asyncio
async def test_dns_lookup(client, respx_mock):
host_id = await _host(client, respx_mock, "fw-diag5")
respx_mock.post(_url("/diagnostics/dns_diagnostics/set")).mock(return_value=httpx.Response(200, json={"result": "ok", "response": {
"A": {"answers": ["example.com.\t300\tIN\tA\t192.0.2.1"], "query_time": "42 msec", "server": "10.0.0.1"}}}))
body = (await client.post(f"/api/hosts/{host_id}/diagnostics/dns", json={"name": "example.com"})).json()
assert body["records"]["A"]["answers"][0].endswith("192.0.2.1") and body["records"]["A"]["query_time"] == "42 msec"
@pytest.mark.asyncio
async def test_ping_runs_a_job_and_always_cleans_it_up(client, respx_mock, fast_sleep):
host_id = await _host(client, respx_mock, "fw-diag6")
respx_mock.post(_url("/diagnostics/ping/set")).mock(return_value=httpx.Response(200, json={"result": "ok", "uuid": "job-1"}))
start = respx_mock.post(_url("/diagnostics/ping/start/job-1")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
row = {"id": "job-1", "status": "running", "send": 4, "received": 4, "loss": "0.0%", "min": 0.3, "avg": 0.4, "max": 0.5, "std-dev": 0.05, "last_error": None}
respx_mock.get(_url("/diagnostics/ping/searchJobs")).mock(return_value=httpx.Response(200, json={"rows": [row]}))
stop = respx_mock.post(_url("/diagnostics/ping/stop/job-1")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
remove = respx_mock.post(_url("/diagnostics/ping/remove/job-1")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
resp = await client.post(f"/api/hosts/{host_id}/diagnostics/ping", json={"target": "10.0.0.1", "count": 4})
assert resp.status_code == 200, resp.text
body = resp.json()
assert (body["sent"], body["received"], body["loss"], body["avg"]) == (4, 4, "0.0%", 0.4)
assert start.called and stop.called and remove.called
@pytest.mark.asyncio
async def test_ping_job_is_removed_even_when_polling_fails(client, respx_mock, fast_sleep):
host_id = await _host(client, respx_mock, "fw-diag7")
respx_mock.post(_url("/diagnostics/ping/set")).mock(return_value=httpx.Response(200, json={"result": "ok", "uuid": "job-2"}))
respx_mock.post(_url("/diagnostics/ping/start/job-2")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
respx_mock.get(_url("/diagnostics/ping/searchJobs")).mock(side_effect=httpx.ConnectError("gone"))
stop = respx_mock.post(_url("/diagnostics/ping/stop/job-2")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
remove = respx_mock.post(_url("/diagnostics/ping/remove/job-2")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
resp = await client.post(f"/api/hosts/{host_id}/diagnostics/ping", json={"target": "10.0.0.1"})
assert resp.status_code == 502
assert stop.called and remove.called # no orphaned job on the firewall
assert [e.status for e in await _events(host_id)] == ["failed"]
@pytest.mark.asyncio
async def test_only_admins_can_run_probes(anon_client, client, respx_mock):
from httpx import ASGITransport, AsyncClient
from app.main import app
host_id = await _host(client, respx_mock, "fw-diag8")
assert (await anon_client.post(f"/api/hosts/{host_id}/diagnostics/dns", json={"name": "example.com"})).status_code == 401
created = await client.post("/api/users", json={"username": "diag_ro", "password": "Diag-Read-Only-2026", "role": "read_only"})
assert created.status_code == 201, created.text
try:
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ro:
assert (await ro.post("/api/auth/login", json={"username": "diag_ro", "password": "Diag-Read-Only-2026"})).status_code == 200
probe = respx_mock.post(_url("/diagnostics/dns_diagnostics/set")).mock(return_value=httpx.Response(200, json={"result": "ok", "response": {}}))
for tool, body in (("dns", {"name": "example.com"}), ("ping", {"target": "10.0.0.1"}), ("traceroute", {"target": "10.0.0.1"}), ("portprobe", {"target": "10.0.0.1", "port": 22})):
assert (await ro.post(f"/api/hosts/{host_id}/diagnostics/{tool}", json=body)).status_code == 403, tool
assert not probe.called
finally:
await client.delete(f"/api/users/{created.json()['id']}")
+15
View File
@@ -467,6 +467,8 @@ async def test_cas_parses_real_shape(client, respx_mock):
"rows": [ "rows": [
{ {
"uuid": "ca-uuid-1", "uuid": "ca-uuid-1",
# OpenVPN cert/ca and Trust-Store caref reference this id, not the uuid
"refid": "6ac8a49fa1589",
"descr": "Internal CA", "descr": "Internal CA",
"commonname": "Internal Root CA", "commonname": "Internal Root CA",
"valid_from": str(now - 300 * 86400), "valid_from": str(now - 300 * 86400),
@@ -482,6 +484,7 @@ async def test_cas_parses_real_shape(client, respx_mock):
cas = resp.json() cas = resp.json()
assert len(cas) == 1 assert len(cas) == 1
assert cas[0]["uuid"] == "ca-uuid-1" assert cas[0]["uuid"] == "ca-uuid-1"
assert cas[0]["refid"] == "6ac8a49fa1589"
assert cas[0]["common_name"] == "Internal Root CA" assert cas[0]["common_name"] == "Internal Root CA"
assert cas[0]["status"] == "valid" assert cas[0]["status"] == "valid"
@@ -572,3 +575,15 @@ async def test_firmware_info_broken_http_framing_gives_actionable_message(client
await opnsense_client.get_firmware_info(host) await opnsense_client.get_firmware_info(host)
assert "kern.ipc.tls.enable=0" in str(info.value) assert "kern.ipc.tls.enable=0" in str(info.value)
assert "malformed chunk footer" in str(info.value) assert "malformed chunk footer" in str(info.value)
@pytest.mark.asyncio
async def test_host_address_can_be_changed_after_the_firewall_moved(client, respx_mock):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
host = (await client.post("/api/hosts", json={"name": "fw-moved", "base_url": "https://10.9.9.1", "api_key": "k", "api_secret": "s"})).json()
resp = await client.patch(f"/api/hosts/{host['id']}", json={"base_url": "https://10.9.9.2"})
assert resp.status_code == 200, resp.text
assert resp.json()["base_url"] == "https://10.9.9.2" and resp.json()["id"] == host["id"]
for bad in ("ftp://x", "http://10.9.9.3", "https://user:[email protected]", "https://10.9.9.3/?a=b"):
assert (await client.patch(f"/api/hosts/{host['id']}", json={"base_url": bad})).status_code == 422, bad
assert next(h for h in (await client.get("/api/hosts")).json() if h["id"] == host["id"])["base_url"] == "https://10.9.9.2"
+2 -1
View File
@@ -2,6 +2,7 @@ import httpx
import pytest import pytest
from httpx import AsyncClient from httpx import AsyncClient
from tests.helpers import deploy_approved
async def _create_host(client: AsyncClient, name: str, base_url: str) -> dict: async def _create_host(client: AsyncClient, name: str, base_url: str) -> dict:
@@ -84,7 +85,7 @@ async def test_deploy_without_targets_rejected(client):
resp = await client.post("/api/ipsec/tunnels", json=_tunnel_payload([])) resp = await client.post("/api/ipsec/tunnels", json=_tunnel_payload([]))
tunnel_id = resp.json()["id"] tunnel_id = resp.json()["id"]
resp = await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 400 assert resp.status_code == 400
+35 -7
View File
@@ -15,9 +15,19 @@ from sqlalchemy import select
from app.database import SessionLocal from app.database import SessionLocal
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from tests.test_ipsec import _create_host, _tunnel_payload from tests.test_ipsec import _create_host, _tunnel_payload
from tests.helpers import deploy_approved
def _mock_full_success(respx_mock, base_url: str): def _mock_ipsec_enabled(respx_mock, base_url: str, enabled: bool = True):
# The service-wide switch is checked before every deploy (live 26.7.6:
# off by default, and reconfigure still answers ok while it is off).
respx_mock.get(f"{base_url}/api/ipsec/connections/is_enabled").mock(
return_value=httpx.Response(200, json={"enabled": enabled})
)
def _mock_full_success(respx_mock, base_url: str, enabled: bool = True):
_mock_ipsec_enabled(respx_mock, base_url, enabled)
respx_mock.post(f"{base_url}/api/ipsec/connections/addConnection").mock( respx_mock.post(f"{base_url}/api/ipsec/connections/addConnection").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "conn-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "conn-1"})
) )
@@ -47,7 +57,7 @@ async def test_deploy_success_persists_all_five_uuids(client, respx_mock):
_mock_full_success(respx_mock, "https://fw1.test") _mock_full_success(respx_mock, "https://fw1.test")
resp = await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 200, resp.text assert resp.status_code == 200, resp.text
dep = resp.json()["deployments"][0] dep = resp.json()["deployments"][0]
assert dep["status"] == "success" assert dep["status"] == "success"
@@ -90,7 +100,7 @@ async def test_deploy_config_write_failure_surfaces_as_error_without_rollback(cl
return_value=httpx.Response(200, json={"status": "ok"}) return_value=httpx.Response(200, json={"status": "ok"})
) )
resp = await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 200, resp.text assert resp.status_code == 200, resp.text
dep = resp.json()["deployments"][0] dep = resp.json()["deployments"][0]
assert dep["status"] == "error" assert dep["status"] == "error"
@@ -133,11 +143,12 @@ async def test_deploy_reconfigure_failure_uses_exact_specified_message(client, r
respx_mock.post("https://fw1.test/api/ipsec/preSharedKeys/addItem").mock( respx_mock.post("https://fw1.test/api/ipsec/preSharedKeys/addItem").mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "psk-1"}) return_value=httpx.Response(200, json={"result": "saved", "uuid": "psk-1"})
) )
_mock_ipsec_enabled(respx_mock, "https://fw1.test")
respx_mock.post("https://fw1.test/api/ipsec/service/reconfigure").mock( respx_mock.post("https://fw1.test/api/ipsec/service/reconfigure").mock(
side_effect=httpx.ConnectError("connection refused") side_effect=httpx.ConnectError("connection refused")
) )
resp = await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 200, resp.text assert resp.status_code == 200, resp.text
dep = resp.json()["deployments"][0] dep = resp.json()["deployments"][0]
assert dep["status"] == "error" assert dep["status"] == "error"
@@ -162,11 +173,11 @@ async def test_redeploy_unchanged_skips_opnsense_entirely(client, respx_mock):
tunnel_id = resp.json()["id"] tunnel_id = resp.json()["id"]
_mock_full_success(respx_mock, "https://fw1.test") _mock_full_success(respx_mock, "https://fw1.test")
await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
add_route = respx_mock.post("https://fw1.test/api/ipsec/connections/addConnection") add_route = respx_mock.post("https://fw1.test/api/ipsec/connections/addConnection")
resp = await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 200, resp.text assert resp.status_code == 200, resp.text
dep = resp.json()["deployments"][0] dep = resp.json()["deployments"][0]
assert dep["status"] == "success" assert dep["status"] == "success"
@@ -181,7 +192,7 @@ async def test_undeploy_deletes_child_local_remote_connection_then_psk(client, r
tunnel_id = resp.json()["id"] tunnel_id = resp.json()["id"]
_mock_full_success(respx_mock, "https://fw1.test") _mock_full_success(respx_mock, "https://fw1.test")
await client.post(f"/api/ipsec/tunnels/{tunnel_id}/deploy") await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
del_child = respx_mock.post("https://fw1.test/api/ipsec/connections/delChild/child-1").mock( del_child = respx_mock.post("https://fw1.test/api/ipsec/connections/delChild/child-1").mock(
return_value=httpx.Response(200, json={"result": "deleted"}) return_value=httpx.Response(200, json={"result": "deleted"})
@@ -202,3 +213,20 @@ async def test_undeploy_deletes_child_local_remote_connection_then_psk(client, r
resp = await client.delete(f"/api/ipsec/tunnels/{tunnel_id}") resp = await client.delete(f"/api/ipsec/tunnels/{tunnel_id}")
assert resp.status_code == 204, resp.text assert resp.status_code == 204, resp.text
assert del_child.called and del_local.called and del_remote.called and del_conn.called and del_psk.called assert del_child.called and del_local.called and del_remote.called and del_conn.called and del_psk.called
@pytest.mark.asyncio
async def test_deploy_switches_on_the_global_ipsec_service_when_disabled(client, respx_mock):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
host = await _create_host(client, "fw1", "https://fw1.test")
resp = await client.post("/api/ipsec/tunnels", json=_tunnel_payload([host["id"]]))
tunnel_id = resp.json()["id"]
_mock_full_success(respx_mock, "https://fw1.test", enabled=False)
toggle = respx_mock.post("https://fw1.test/api/ipsec/connections/toggle/1").mock(
return_value=httpx.Response(200, json={"status": "ok"})
)
resp = await deploy_approved(client, f"/api/ipsec/tunnels/{tunnel_id}/deploy")
assert resp.status_code == 200, resp.text
assert resp.json()["deployments"][0]["status"] == "success"
assert toggle.call_count == 1
+178
View File
@@ -0,0 +1,178 @@
"""Read-only network inventory. Response shapes copied from real OPNsense
26.7.x test firewalls (CARP password replaced)."""
import httpx
import pytest
BASE = "https://fw-net.test"
INTERFACES_INFO = {"total": 3, "rowCount": 3, "current": 1, "rows": [
{"device": "vtnet0", "identifier": "wan", "description": "WAN", "status": "up", "is_physical": True,
"mtu": "1500", "macaddr": "bc:24:11:25:7e:b8", "media": "10Gbase-T <full-duplex>",
"ipv4": [{"ipaddr": "10.20.30.57/24"}], "ipv6": []},
{"device": "vtnet1_vlan10", "identifier": "lan", "description": "LAN", "status": "up", "is_physical": False,
"mtu": "1500", "ipv4": [{"ipaddr": "192.168.1.2/24"},
{"ipaddr": "192.168.1.1/24", "vhid": "10", "status": "MASTER"}],
"vlan": {"tag": "10", "proto": "802.1q", "pcp": "0", "parent": "vtnet1"}},
{"device": "vtnet1", "identifier": "", "description": "Unassigned Interface", "status": "up",
"is_physical": True, "ipv4": []},
]}
ASSIGNMENTS = {"rows": [
{"uuid": "wan", "identifier": "wan", "pending_action": "", "type4": "staticv4", "%type4": "Static IPv4", "%type6": "DHCPv6"},
{"uuid": "lan", "identifier": "lan", "pending_action": "", "type4": "staticv4", "%type4": "Static IPv4", "%type6": "Identity association"},
]}
VLANS = {"rows": [{"uuid": "v1", "if": "vtnet1", "tag": "10", "pcp": "0", "descr": "", "vlanif": "vtnet1_vlan10 [LAN]"}]}
VIPS = {"rows": [{"uuid": "vip1", "interface": "lan", "%interface": "LAN", "mode": "carp", "address": "192.168.1.1/24",
"vhid": "10", "advbase": "1", "advskew": "0", "descr": "CARP LAN", "password": "SUPER-SECRET-CARP"}]}
GATEWAYS = {"rows": [{"uuid": "g1", "name": "WAN_GW", "descr": "", "interface": "wan", "interface_descr": "WAN",
"ipprotocol": "inet", "gateway": "10.20.30.1", "defaultgw": True, "disabled": False,
"priority": "255", "status": "Online"}]}
DNS_HOSTS = {"rows": [
{"uuid": "d1", "enabled": "1", "hostname": "nas", "domain": "lan.example", "rr": "A", "server": "192.168.1.5", "description": "NAS", "isAlias": False},
{"uuid": "d1-a", "enabled": "1", "hostname": "files", "domain": "lan.example", "rr": "A", "server": "192.168.1.5", "isAlias": True}]}
DHCP_HOSTS = {"rows": [{"uuid": "h1", "host": "printer", "domain": "lan.example", "ip": "192.168.1.30", "hwaddr": "02:00:00:aa:bb:cc",
"lease_time": "", "descr": "Printer"}]}
KEA_SUBNETS = {"rows": [{"uuid": "s1", "subnet": "192.168.77.0/24", "pools": "192.168.77.100-192.168.77.150",
"option_data.routers": "192.168.77.1", "option_data.domain_name_servers": "",
"description": "lab", "ddns_domain_key_secret": "KEA-DDNS-SECRET"}]}
KEA_RESERVATIONS = {"rows": [{"uuid": "k1", "subnet": "s1", "%subnet": "192.168.77.0/24", "ip_address": "192.168.77.120",
"hw_address": "02:00:00:aa:bb:dd", "hostname": "kearsv", "description": "Reservation"}]}
ROUTES = {"rows": [{"uuid": "r1", "network": "10.9.0.0/16", "gateway": "WAN_GW", "descr": "lab", "disabled": "0"}]}
async def _host(client, respx_mock, base=BASE, name="fw-net"):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
resp = await client.post("/api/hosts", json={"name": name, "base_url": base, "api_key": "k", "api_secret": "s"})
return resp.json()["id"]
def _mock(respx_mock, base=BASE, info=INTERFACES_INFO, **overrides):
routes = {
"/interfaces/overview/interfacesInfo": info,
"/interfaces/assignment/searchItem": ASSIGNMENTS,
"/interfaces/vlan_settings/searchItem": VLANS,
"/interfaces/vip_settings/searchItem": VIPS,
"/routing/settings/searchGateway": GATEWAYS,
"/routes/routes/searchroute": ROUTES,
"/unbound/settings/searchHostOverride": DNS_HOSTS,
"/dnsmasq/settings/searchHost": DHCP_HOSTS,
"/kea/dhcpv4/searchSubnet": KEA_SUBNETS,
"/kea/dhcpv4/searchReservation": KEA_RESERVATIONS,
}
for path, body in routes.items():
reply = overrides.get(path, httpx.Response(200, json=body))
respx_mock.get(f"{base}/api{path}").mock(return_value=reply)
@pytest.mark.asyncio
async def test_network_inventory_parses_real_shapes(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.57")
_mock(respx_mock, base="https://10.20.30.57")
resp = await client.get(f"/api/hosts/{host_id}/network")
assert resp.status_code == 200, resp.text
data = resp.json()
by_device = {i["device"]: i for i in data["interfaces"]}
assert by_device["vtnet0"]["identifier"] == "wan"
assert by_device["vtnet0"]["ipv4_type"] == "Static IPv4"
assert by_device["vtnet1"]["identifier"] is None
assert by_device["vtnet0"]["assigned"] is True and by_device["vtnet1"]["assigned"] is False
lan = by_device["vtnet1_vlan10"]
assert lan["vlan_tag"] == "10" and lan["vlan_parent"] == "vtnet1"
assert [a["carp_status"] for a in lan["addresses"]] == [None, "MASTER"]
assert data["vlans"][0]["device"] == "vtnet1_vlan10"
assert data["gateways"][0]["is_default"] is True
assert data["routes"][0]["network"] == "10.9.0.0/16"
assert data["unavailable"] == []
assert data["can_configure_interfaces"] is True
@pytest.mark.asyncio
async def test_network_marks_management_interface(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.57")
_mock(respx_mock, base="https://10.20.30.57")
data = (await client.get(f"/api/hosts/{host_id}/network")).json()
assert data["management_device"] == "vtnet0"
assert [i["device"] for i in data["interfaces"] if i["is_management"]] == ["vtnet0"]
@pytest.mark.asyncio
async def test_network_carp_address_is_not_management(client, respx_mock):
# Reached over the shared CARP address: that is not one interface's own IP, so no guess.
host_id = await _host(client, respx_mock, base="https://192.168.1.1", name="fw-carp")
_mock(respx_mock, base="https://192.168.1.1")
data = (await client.get(f"/api/hosts/{host_id}/network")).json()
assert data["management_device"] is None
@pytest.mark.asyncio
async def test_network_never_returns_carp_password(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.58", name="fw-pw")
_mock(respx_mock, base="https://10.20.30.58")
resp = await client.get(f"/api/hosts/{host_id}/network")
assert "SUPER-SECRET-CARP" not in resp.text
assert "password" not in resp.json()["vips"][0]
@pytest.mark.asyncio
async def test_network_missing_section_is_reported_not_hidden(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.59", name="fw-old")
_mock(respx_mock, base="https://10.20.30.59", **{
"/interfaces/assignment/searchItem": httpx.Response(404, json={"errorMessage": "Endpoint not found"}),
})
resp = await client.get(f"/api/hosts/{host_id}/network")
assert resp.status_code == 200, resp.text
data = resp.json()
assert data["unavailable"] == ["assignments"]
assert len(data["interfaces"]) == 3
@pytest.mark.asyncio
async def test_network_host_unreachable_returns_502(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.60", name="fw-down")
respx_mock.get(url__regex=r".*/interfaces/overview/interfacesInfo").mock(side_effect=httpx.ConnectError("refused"))
assert (await client.get(f"/api/hosts/{host_id}/network")).status_code == 502
@pytest.mark.asyncio
async def test_network_unknown_host_404(client):
resp = await client.get("/api/hosts/00000000-0000-0000-0000-000000000000/network")
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_network_tolerates_numbers_where_strings_were_expected(client, respx_mock):
# Seen live on 26.7.x: a gateway's priority arrived as the integer 254.
host_id = await _host(client, respx_mock, base="https://10.20.30.61", name="fw-int")
gateways = {"rows": [{**GATEWAYS["rows"][0], "priority": 254}]}
_mock(respx_mock, base="https://10.20.30.61", **{
"/routing/settings/searchGateway": httpx.Response(200, json=gateways),
})
resp = await client.get(f"/api/hosts/{host_id}/network")
assert resp.status_code == 200, resp.text
assert resp.json()["gateways"][0]["priority"] == "254"
@pytest.mark.asyncio
async def test_network_older_26_7_assignment_list_cannot_configure(client, respx_mock):
# OPNsense 26.7.1: the assignment list only has identifier/descr/if, no type4/ipaddr.
host_id = await _host(client, respx_mock, base="https://10.20.30.62", name="fw-2671")
thin = {"rows": [{"uuid": "wan", "identifier": "wan", "descr": "", "if": "vtnet0"}]}
_mock(respx_mock, base="https://10.20.30.62", **{
"/interfaces/assignment/searchItem": httpx.Response(200, json=thin),
})
data = (await client.get(f"/api/hosts/{host_id}/network")).json()
assert data["can_configure_interfaces"] is False
assert data["unavailable"] == []
@pytest.mark.asyncio
async def test_network_lists_dns_and_dhcp_objects_without_secrets(client, respx_mock):
host_id = await _host(client, respx_mock, base="https://10.20.30.63", name="fw-dns")
_mock(respx_mock, base="https://10.20.30.63")
resp = await client.get(f"/api/hosts/{host_id}/network")
assert "KEA-DDNS-SECRET" not in resp.text
data = resp.json()
assert [(d["hostname"], d["server"]) for d in data["dns_hosts"]] == [("nas", "192.168.1.5")] # alias children are not rows of their own
assert data["dhcp_hosts"][0]["hwaddr"] == "02:00:00:aa:bb:cc"
assert (data["kea_subnets"][0]["subnet"], data["kea_subnets"][0]["routers"]) == ("192.168.77.0/24", "192.168.77.1")
assert data["kea_reservations"][0]["subnet"] == "192.168.77.0/24"
assert data["unavailable"] == []
+559
View File
@@ -0,0 +1,559 @@
"""Writes to VLANs / virtual IPs / gateways / routes, and the safety around them."""
import json
import httpx
import pytest
from sqlalchemy import select
from app.database import SessionLocal
from app.models.audit import AuditEvent
from tests.test_network import GATEWAYS, INTERFACES_INFO, _host, _mock
MGMT = "https://10.20.30.57"
def _url(path, base=MGMT):
return f"{base}/api{path}"
def _backup(respx_mock, base=MGMT, status=200):
return respx_mock.get(_url("/core/backup/download/this", base)).mock(
return_value=httpx.Response(status, content=b"<opnsense/>"))
def _item(respx_mock, path, key, node, base=MGMT):
respx_mock.get(_url(path, base)).mock(return_value=httpx.Response(200, json={key: node}))
async def _setup(client, respx_mock, name, base=MGMT, **overrides):
host_id = await _host(client, respx_mock, base=base, name=name)
_mock(respx_mock, base=base, **overrides)
return host_id
def _body(call):
return json.loads(call.request.content.decode())
async def _preview(client, host_id, **payload):
return await client.post(f"/api/hosts/{host_id}/network/changes/preview", json=payload)
async def _execute(client, host_id, token, ack=False, **payload):
return await client.post(f"/api/hosts/{host_id}/network/changes",
json={**payload, "preview_token": token, "acknowledge": ack})
async def _audit(host_id):
async with SessionLocal() as db:
rows = await db.scalars(select(AuditEvent).where(AuditEvent.host_id == host_id, AuditEvent.kind == "network")
.order_by(AuditEvent.created_at))
return list(rows.all())
VLAN_ADD = {"kind": "vlan", "action": "add", "fields": {"if": "vtnet1", "tag": "20", "descr": "guests"}}
@pytest.mark.asyncio
async def test_preview_add_is_read_only_and_shows_diff(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c1")
resp = await _preview(client, host_id, **VLAN_ADD)
assert resp.status_code == 200, resp.text
plan = resp.json()
assert plan["can_execute"] is True and plan["requires_acknowledgement"] is False
assert {c["field"] for c in plan["changes"]} == {"if", "tag", "descr"}
assert len(plan["token"]) == 64
assert not [c for c in respx_mock.calls if c.request.method == "POST" and "/api/hosts" not in str(c.request.url)
and "vlan_settings" in str(c.request.url)]
@pytest.mark.asyncio
async def test_execute_add_backs_up_writes_applies_and_verifies(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c2")
backup = _backup(respx_mock)
add = respx_mock.post(_url("/interfaces/vlan_settings/addItem")).mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "new-vlan"}))
reconf = respx_mock.post(_url("/interfaces/vlan_settings/reconfigure")).mock(
return_value=httpx.Response(200, json={"status": "ok"}))
_item(respx_mock, "/interfaces/vlan_settings/getItem/new-vlan", "vlan",
{"if": {"vtnet1": {"selected": 1}}, "tag": "20", "descr": "guests"})
token = (await _preview(client, host_id, **VLAN_ADD)).json()["token"]
resp = await _execute(client, host_id, token, **VLAN_ADD)
assert resp.status_code == 200, resp.text
result = resp.json()
assert result["uuid"] == "new-vlan" and result["verified"] is True and result["backup_id"].endswith(".xml")
assert backup.called and reconf.called
assert _body(add.calls.last) == {"vlan": {"if": "vtnet1", "tag": "20", "descr": "guests"}}
assert [e.status for e in await _audit(host_id)] == ["started", "applied"]
@pytest.mark.asyncio
async def test_stale_token_is_refused_and_nothing_is_written(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c3")
add = respx_mock.post(_url("/interfaces/vlan_settings/addItem")).mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "x"}))
resp = await _execute(client, host_id, "0" * 64, **VLAN_ADD)
assert resp.status_code == 409
assert not add.called
@pytest.mark.asyncio
async def test_token_changes_when_the_fields_change(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c4")
one = (await _preview(client, host_id, **VLAN_ADD)).json()["token"]
other = dict(VLAN_ADD, fields={**VLAN_ADD["fields"], "tag": "21"})
two = (await _preview(client, host_id, **other)).json()["token"]
assert one != two
# a token for the first request cannot be used for the second
assert (await _execute(client, host_id, one, **other)).status_code == 409
@pytest.mark.asyncio
async def test_backup_failure_aborts_before_any_write(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c5")
_backup(respx_mock, status=500)
add = respx_mock.post(_url("/interfaces/vlan_settings/addItem")).mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "x"}))
token = (await _preview(client, host_id, **VLAN_ADD)).json()["token"]
resp = await _execute(client, host_id, token, **VLAN_ADD)
assert resp.status_code == 502
assert resp.json()["detail"]["code"] == "NETWORK_BACKUP_FAILED"
assert not add.called
assert [e.status for e in await _audit(host_id)] == ["aborted"]
@pytest.mark.asyncio
async def test_firewall_rejection_is_reported_and_not_applied(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c6")
_backup(respx_mock)
respx_mock.post(_url("/interfaces/vlan_settings/addItem")).mock(
return_value=httpx.Response(200, json={"result": "failed", "validations": {"vlan.tag": "bad"}}))
reconf = respx_mock.post(_url("/interfaces/vlan_settings/reconfigure")).mock(
return_value=httpx.Response(200, json={"status": "ok"}))
token = (await _preview(client, host_id, **VLAN_ADD)).json()["token"]
resp = await _execute(client, host_id, token, **VLAN_ADD)
assert resp.status_code == 502 and resp.json()["detail"]["code"] == "UPSTREAM_REJECTED"
assert not reconf.called
assert (await _audit(host_id))[-1].status == "rejected"
@pytest.mark.asyncio
async def test_apply_failure_is_never_reported_as_success(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c7")
_backup(respx_mock)
respx_mock.post(_url("/interfaces/vlan_settings/addItem")).mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "x"}))
respx_mock.post(_url("/interfaces/vlan_settings/reconfigure")).mock(side_effect=httpx.ReadTimeout("slow"))
token = (await _preview(client, host_id, **VLAN_ADD)).json()["token"]
resp = await _execute(client, host_id, token, **VLAN_ADD)
assert resp.status_code == 502 and resp.json()["detail"]["code"] == "NETWORK_APPLY_UNCERTAIN"
assert resp.json()["detail"]["params"]["backup_id"].endswith(".xml")
assert (await _audit(host_id))[-1].status == "apply_uncertain"
@pytest.mark.asyncio
async def test_unknown_field_is_rejected_not_forwarded(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c8")
resp = await _preview(client, host_id, kind="vlan", action="add", fields={"if": "vtnet1", "tag": "9", "evil": "x"})
assert resp.status_code == 422 and resp.json()["detail"]["code"] == "NETWORK_FIELD_INVALID"
@pytest.mark.asyncio
async def test_changing_the_management_vlan_needs_acknowledgement(client, respx_mock):
# fw reached on 192.168.1.2 = the LAN VLAN device
base = "https://192.168.1.2"
vlans = {"rows": [{"uuid": "v-mgmt", "if": "vtnet1", "tag": "10", "pcp": "0", "descr": "", "vlanif": "vtnet1_vlan10 [LAN]"}]}
host_id = await _setup(client, respx_mock, "fw-c9", base=base, **{
"/interfaces/vlan_settings/searchItem": httpx.Response(200, json=vlans)})
_item(respx_mock, "/interfaces/vlan_settings/getItem/v-mgmt", "vlan",
{"if": {"vtnet1": {"selected": 1}}, "tag": "10", "descr": "", "vlanif": "vtnet1_vlan10"}, base=base)
_backup(respx_mock, base)
respx_mock.post(_url("/interfaces/vlan_settings/setItem/v-mgmt", base)).mock(
return_value=httpx.Response(200, json={"result": "saved"}))
respx_mock.post(_url("/interfaces/vlan_settings/reconfigure", base)).mock(return_value=httpx.Response(200, json={"status": "ok"}))
change = {"kind": "vlan", "action": "set", "uuid": "v-mgmt", "fields": {"descr": "renamed"}}
plan = (await _preview(client, host_id, **change)).json()
assert plan["requires_acknowledgement"] is True
assert plan["risks"][0]["code"] == "management_interface"
refused = await _execute(client, host_id, plan["token"], **change)
assert refused.status_code == 409 and refused.json()["detail"]["code"] == "NETWORK_ACK_REQUIRED"
allowed = await _execute(client, host_id, plan["token"], ack=True, **change)
assert allowed.status_code == 200, allowed.text
assert (await _audit(host_id))[0].details["acknowledged"] is True
@pytest.mark.asyncio
async def test_vip_password_never_leaves_the_server(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c10")
_item(respx_mock, "/interfaces/vip_settings/getItem/vip1", "vip", {
"interface": {"lan": {"selected": 1}}, "mode": {"carp": {"selected": 1}}, "network": "192.168.1.1/24",
"vhid": "10", "password": "STORED-SECRET", "descr": "CARP LAN", "advskew": "0"})
_backup(respx_mock)
set_route = respx_mock.post(_url("/interfaces/vip_settings/setItem/vip1")).mock(
return_value=httpx.Response(200, json={"result": "saved"}))
respx_mock.post(_url("/interfaces/vip_settings/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
change = {"kind": "vip", "action": "set", "uuid": "vip1", "fields": {"password": "NEW-SECRET", "advskew": "100"}}
resp = await _preview(client, host_id, **change)
assert "STORED-SECRET" not in resp.text and "NEW-SECRET" not in resp.text
plan = resp.json()
assert {"field": "password", "before": None, "after": "(gesetzt)"} in plan["changes"]
assert [w["code"] for w in plan["warnings"]] == ["carp_change"]
done = await _execute(client, host_id, plan["token"], **change)
assert done.status_code == 200, done.text
assert "NEW-SECRET" not in done.text
assert _body(set_route.calls.last)["vip"]["password"] == "NEW-SECRET" # but it does reach the firewall
for event in await _audit(host_id):
assert "NEW-SECRET" not in json.dumps(event.details) and "STORED-SECRET" not in json.dumps(event.details)
@pytest.mark.asyncio
async def test_default_gateway_and_default_route_need_acknowledgement(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c11")
_item(respx_mock, "/routing/settings/getGateway/g1", "gateway_item",
{"name": "WAN_GW", "interface": {"wan": {"selected": 1}}, "gateway": "10.20.30.1", "priority": "255"})
plan = (await _preview(client, host_id, kind="gateway", action="del", uuid="g1", fields={})).json()
assert {r["code"] for r in plan["risks"]} >= {"default_gateway", "management_gateway"}
route = (await _preview(client, host_id, kind="route", action="add",
fields={"network": "0.0.0.0/0", "gateway": "WAN_GW"})).json()
assert [r["code"] for r in route["risks"]] == ["default_route"]
covers = (await _preview(client, host_id, kind="route", action="add",
fields={"network": "10.20.0.0/16", "gateway": "WAN_GW"})).json()
assert [r["code"] for r in covers["risks"]] == ["route_covers_management"]
harmless = (await _preview(client, host_id, kind="route", action="add",
fields={"network": "172.16.0.0/16", "gateway": "WAN_GW"})).json()
assert harmless["risks"] == [] and harmless["requires_acknowledgement"] is False
@pytest.mark.asyncio
async def test_delete_is_verified_by_absence(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c12")
_item(respx_mock, "/routes/routes/getroute/r1", "route", {"network": "10.9.0.0/16", "gateway": {"WAN_GW": {"selected": 1}}, "descr": "lab", "enabled": "1"})
_backup(respx_mock)
respx_mock.post(_url("/routes/routes/delroute/r1")).mock(return_value=httpx.Response(200, json={"result": "deleted"}))
respx_mock.post(_url("/routes/routes/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
# after the delete the route list no longer contains it
respx_mock.get(_url("/routes/routes/searchroute")).mock(return_value=httpx.Response(200, json={"rows": []}))
change = {"kind": "route", "action": "del", "uuid": "r1", "fields": {}}
plan = (await _preview(client, host_id, **change)).json()
assert plan["risks"] == [] # 10.9.0.0/16 does not touch the management address
done = await _execute(client, host_id, plan["token"], **change)
assert done.status_code == 200 and done.json()["verified"] is True
@pytest.mark.asyncio
async def test_missing_object_is_404(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c13")
respx_mock.get(_url("/interfaces/vlan_settings/getItem/ghost")).mock(return_value=httpx.Response(200, json={"vlan": {}}))
resp = await _preview(client, host_id, kind="vlan", action="del", uuid="ghost", fields={})
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_unknown_management_path_is_treated_as_risky(client, respx_mock):
# The address OPNexus uses belongs to no interface (NAT, or a CARP address): be careful, not optimistic.
base = "https://10.20.30.99"
host_id = await _setup(client, respx_mock, "fw-c14", base=base)
_item(respx_mock, "/routes/routes/getroute/r2", "route", {"network": "172.16.0.0/16", "gateway": {"WAN_GW": {"selected": 1}}, "enabled": "1"}, base=base)
plan = (await _preview(client, host_id, kind="route", action="set", uuid="r2", fields={"descr": "x"}, )).json()
assert [r["code"] for r in plan["risks"]] == ["management_unknown"]
add = (await _preview(client, host_id, kind="route", action="add", fields={"network": "172.17.0.0/16", "gateway": "WAN_GW"})).json()
assert add["risks"] == [] # a new, harmless object cannot cut anything
@pytest.mark.asyncio
async def test_template_returns_the_firewalls_own_choices(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c15")
_item(respx_mock, "/routes/routes/getroute", "route", {
"network": "", "descr": "", "enabled": "1", "secretless": "x",
"gateway": {"Null4": {"value": "Null4", "selected": 0}, "WAN_GW": {"value": "WAN_GW - 10.20.30.1", "selected": 1}}})
resp = await client.get(f"/api/hosts/{host_id}/network/template/route")
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["gateway"]["value"] == "WAN_GW"
assert [o["value"] for o in body["gateway"]["options"]] == ["Null4", "WAN_GW"]
assert "secretless" not in body # only fields that may be sent
assert (await client.get(f"/api/hosts/{host_id}/network/template/bogus")).status_code == 404
@pytest.mark.asyncio
async def test_inventory_reads_route_enabled_flag(client, respx_mock):
# Real 26.7.x route rows say enabled "1"/"0", not disabled.
rows = {"rows": [{"uuid": "r1", "network": "10.9.0.0/16", "gateway": "WAN_GW", "descr": "on", "enabled": "1"},
{"uuid": "r2", "network": "10.8.0.0/16", "gateway": "WAN_GW", "descr": "off", "enabled": "0"}]}
host_id = await _setup(client, respx_mock, "fw-c16", **{"/routes/routes/searchroute": httpx.Response(200, json=rows)})
routes = (await client.get(f"/api/hosts/{host_id}/network")).json()["routes"]
assert [(r["network"], r["disabled"]) for r in routes] == [("10.9.0.0/16", False), ("10.8.0.0/16", True)]
@pytest.mark.asyncio
async def test_delete_preview_lists_only_meaningful_fields(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c17")
_item(respx_mock, "/interfaces/vlan_settings/getItem/v9", "vlan",
{"if": {"vtnet1": {"selected": 1}}, "tag": "40", "descr": "x", "vlanif": "vlan01", "pcp": ""})
plan = (await _preview(client, host_id, kind="vlan", action="del", uuid="v9", fields={})).json()
assert {c["field"] for c in plan["changes"]} == {"if", "tag", "descr"}
@pytest.mark.asyncio
async def test_item_endpoint_returns_current_values_without_secrets(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-c18")
_item(respx_mock, "/interfaces/vip_settings/getItem/vip1", "vip", {
"interface": {"lan": {"selected": 1}}, "mode": {"carp": {"selected": 1}}, "network": "192.168.1.1/24",
"vhid": "10", "password": "STORED-SECRET", "address": "192.168.1.1/24", "descr": "CARP LAN"})
resp = await client.get(f"/api/hosts/{host_id}/network/item/vip/vip1")
assert resp.status_code == 200, resp.text
assert "STORED-SECRET" not in resp.text
assert resp.json() == {"interface": "lan", "mode": "carp", "network": "192.168.1.1/24", "vhid": "10", "descr": "CARP LAN"}
assert (await client.get(f"/api/hosts/{host_id}/network/item/vip/..%2Fx")).status_code == 404
# ── interface assignment / IP configuration ──────────────────────────────
def _pending(respx_mock, status="ok", base=MGMT):
respx_mock.get(_url("/interfaces/assignment/pending", base)).mock(return_value=httpx.Response(200, json={"status": status}))
IF_ADD = {"kind": "interface", "action": "add", "fields": {"if": "vtnet1_vlan20", "descr": "GUEST", "enable": "1",
"type4": "staticv4", "ipaddr": "10.20.0.1/24"}}
@pytest.mark.asyncio
async def test_interface_add_stages_applies_and_verifies(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-i1")
_pending(respx_mock)
_backup(respx_mock)
add = respx_mock.post(_url("/interfaces/assignment/addItem")).mock(
return_value=httpx.Response(200, json={"result": "saved", "uuid": "opt1"}))
reconf = respx_mock.post(_url("/interfaces/assignment/reconfigure")).mock(
return_value=httpx.Response(200, json={"status": "ok"}))
_item(respx_mock, "/interfaces/assignment/getItem/opt1", "interface", {
"if": {"vtnet1_vlan20": {"selected": 1}}, "descr": "GUEST", "enable": "1",
"type4": {"staticv4": {"selected": 1}}, "ipaddr": "10.20.0.1/24"})
plan = (await _preview(client, host_id, **IF_ADD)).json()
assert plan["can_execute"] is True and plan["risks"] == []
done = await _execute(client, host_id, plan["token"], **IF_ADD)
assert done.status_code == 200, done.text
assert done.json()["uuid"] == "opt1" and done.json()["verified"] is True
assert _body(add.calls.last)["interface"]["ipaddr"] == "10.20.0.1/24"
assert reconf.called
@pytest.mark.asyncio
async def test_interface_changes_are_blocked_on_firewalls_without_the_api(client, respx_mock):
thin = {"rows": [{"uuid": "wan", "identifier": "wan", "descr": "", "if": "vtnet0"}]} # 26.7.1: no type4
host_id = await _setup(client, respx_mock, "fw-i2", **{"/interfaces/assignment/searchItem": httpx.Response(200, json=thin)})
plan = (await _preview(client, host_id, **IF_ADD)).json()
assert plan["can_execute"] is False
assert [b["code"] for b in plan["blocks"]] == ["interface_api_unavailable"]
resp = await _execute(client, host_id, plan["token"], **IF_ADD)
assert resp.status_code == 409 and resp.json()["detail"]["code"] == "NETWORK_CHANGE_BLOCKED"
@pytest.mark.asyncio
async def test_staged_changes_from_someone_else_block_a_new_change(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-i3")
_pending(respx_mock, status="pending")
plan = (await _preview(client, host_id, **IF_ADD)).json()
assert [b["code"] for b in plan["blocks"]] == ["interface_changes_pending"]
add = respx_mock.post(_url("/interfaces/assignment/addItem")).mock(return_value=httpx.Response(200, json={"result": "saved"}))
assert (await _execute(client, host_id, plan["token"], **IF_ADD)).status_code == 409
assert not add.called
@pytest.mark.asyncio
async def test_deleting_an_interface_lists_the_rules_that_will_vanish(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-i4")
_pending(respx_mock)
_item(respx_mock, "/interfaces/assignment/getItem/opt1", "interface", {
"if": {"vtnet1_vlan20": {"selected": 1}}, "descr": "GUEST", "enable": "1"})
respx_mock.get(_url("/firewall/filter/searchRule")).mock(return_value=httpx.Response(200, json={"rows": [
{"uuid": "a", "description": "guest out", "interface": "opt1", "is_automatic": False},
{"uuid": "b", "description": "multi", "interface": "lan,opt1", "is_automatic": False},
{"uuid": "c", "description": "other", "interface": "lan", "is_automatic": False},
{"uuid": "d", "description": "system", "interface": "opt1", "is_automatic": True}]}))
plan = (await _preview(client, host_id, kind="interface", action="del", uuid="opt1", fields={})).json()
risk = next(r for r in plan["risks"] if r["code"] == "interface_rules_removed")
assert risk["params"]["count"] == 2 and "guest out" in risk["params"]["names"] and "other" not in risk["params"]["names"]
assert plan["requires_acknowledgement"] is True
@pytest.mark.asyncio
async def test_changing_the_management_interface_needs_acknowledgement(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-i5") # reached on 10.20.30.57 = wan
_pending(respx_mock)
_item(respx_mock, "/interfaces/assignment/getItem/wan", "interface", {
"if": {"vtnet0": {"selected": 1}}, "descr": "WAN", "enable": "1", "type4": {"staticv4": {"selected": 1}},
"ipaddr": "10.20.30.57/24"})
plan = (await _preview(client, host_id, kind="interface", action="set", uuid="wan", fields={"ipaddr": "10.20.30.77/24"})).json()
assert [r["code"] for r in plan["risks"]] == ["management_interface"]
assert plan["requires_acknowledgement"] is True
@pytest.mark.asyncio
async def test_locked_interface_cannot_be_deleted_and_says_why(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-i6")
_pending(respx_mock)
_item(respx_mock, "/interfaces/assignment/getItem/opt1", "interface",
{"if": {"vtnet1_vlan20": {"selected": 1}}, "descr": "GUEST", "enable": "1", "lock": "1"})
respx_mock.get(_url("/firewall/filter/searchRule")).mock(return_value=httpx.Response(200, json={"rows": []}))
plan = (await _preview(client, host_id, kind="interface", action="del", uuid="opt1", fields={})).json()
assert [b["code"] for b in plan["blocks"]] == ["interface_locked"] and plan["can_execute"] is False
# ── DNS / DHCP objects ───────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_kea_subnet_options_are_sent_nested_and_read_back_flat(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-d1")
_backup(respx_mock)
add = respx_mock.post(_url("/kea/dhcpv4/addSubnet")).mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": "sn1"}))
respx_mock.post(_url("/kea/service/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
_item(respx_mock, "/kea/dhcpv4/getSubnet/sn1", "subnet4", {
"subnet": "192.168.50.0/24", "pools": "192.168.50.100-192.168.50.200", "description": "lab",
"option_data": {"routers": {"192.168.50.1": {"value": "192.168.50.1", "selected": 1}, "": {"value": "", "selected": 0}},
"domain_name": "lab.example"}})
change = {"kind": "kea_subnet", "action": "add", "fields": {
"subnet": "192.168.50.0/24", "pools": "192.168.50.100-192.168.50.200", "description": "lab",
"option_data.routers": "192.168.50.1", "option_data.domain_name": "lab.example"}}
plan = (await _preview(client, host_id, **change)).json()
assert plan["risks"] == [] # DHCP data cannot cut the connection, even with an unknown management path
done = await _execute(client, host_id, plan["token"], **change)
assert done.status_code == 200 and done.json()["verified"] is True, done.text
assert _body(add.calls.last) == {"subnet4": {
"subnet": "192.168.50.0/24", "pools": "192.168.50.100-192.168.50.200", "description": "lab",
"option_data": {"routers": "192.168.50.1", "domain_name": "lab.example"}}}
@pytest.mark.asyncio
async def test_dns_override_on_unknown_management_path_is_not_treated_as_risky(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-d2", base="https://10.20.30.98")
_item(respx_mock, "/unbound/settings/getHostOverride/h1", "host",
{"hostname": "nas", "domain": "lan.example", "server": "192.168.1.5", "rr": {"A": {"value": "A", "selected": 1}}},
base="https://10.20.30.98")
plan = (await _preview(client, host_id, kind="dns_host", action="set", uuid="h1", fields={"server": "192.168.1.6"})).json()
assert plan["risks"] == [] and plan["can_execute"] is True
assert plan["changes"] == [{"field": "server", "before": "192.168.1.5", "after": "192.168.1.6"}]
@pytest.mark.asyncio
async def test_template_turns_tokenizer_fields_into_comma_text(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-d3")
_item(respx_mock, "/dnsmasq/settings/getHost", "host", {
"host": "", "domain": "", "local": "0",
"ip": {"": {"value": "", "selected": 1}},
"hwaddr": {"aa:bb:cc:dd:ee:ff": {"value": "aa:bb:cc:dd:ee:ff", "selected": 1}, "11:22:33:44:55:66": {"value": "11:22:33:44:55:66", "selected": 1}}})
body = (await client.get(f"/api/hosts/{host_id}/network/template/dhcp_host")).json()
assert body["ip"]["value"] == "" and body["hwaddr"]["value"] == "aa:bb:cc:dd:ee:ff,11:22:33:44:55:66"
@pytest.mark.asyncio
async def test_unlisted_secret_fields_never_reach_the_audit_log(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-d4")
_backup(respx_mock)
_item(respx_mock, "/kea/dhcpv4/getSubnet/sn9", "subnet4", {
"subnet": "192.168.9.0/24", "description": "old", "ddns_domain_key_secret": "KEA-DDNS-SECRET"})
respx_mock.post(_url("/kea/dhcpv4/setSubnet/sn9")).mock(return_value=httpx.Response(200, json={"result": "saved"}))
respx_mock.post(_url("/kea/service/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
change = {"kind": "kea_subnet", "action": "set", "uuid": "sn9", "fields": {"description": "new"}}
plan = (await _preview(client, host_id, **change)).json()
assert (await _execute(client, host_id, plan["token"], **change)).status_code == 200
assert "KEA-DDNS-SECRET" not in json.dumps([e.details for e in await _audit(host_id)])
@pytest.mark.asyncio
async def test_vip_edit_always_repeats_the_address(client, respx_mock):
# OPNsense blanks a VIP's address when an update leaves out "network" (seen live on 26.7.6).
host_id = await _setup(client, respx_mock, "fw-v1")
_item(respx_mock, "/interfaces/vip_settings/getItem/vip1", "vip", {
"interface": {"lan": {"selected": 1}}, "mode": {"carp": {"selected": 1}}, "network": "192.168.1.50/24",
"vhid": "11", "advskew": "0", "descr": "x"})
_backup(respx_mock)
put = respx_mock.post(_url("/interfaces/vip_settings/setItem/vip1")).mock(return_value=httpx.Response(200, json={"result": "saved"}))
respx_mock.post(_url("/interfaces/vip_settings/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
change = {"kind": "vip", "action": "set", "uuid": "vip1", "fields": {"advskew": "20"}}
plan = (await _preview(client, host_id, **change)).json()
done = await _execute(client, host_id, plan["token"], **change)
assert done.status_code == 200, done.text
assert _body(put.calls.last)["vip"] == {"advskew": "20", "network": "192.168.1.50/24"}
@pytest.mark.asyncio
async def test_verification_catches_a_field_that_changed_without_being_asked(client, respx_mock):
# The firewall dropped an untouched field while saving: that must not be reported as a clean success.
host_id = await _setup(client, respx_mock, "fw-v2")
state = iter([
{"descr": "old", "network": "172.16.0.0/16", "gateway": {"WAN_GW": {"selected": 1}}, "enabled": "1"}, # preview (before)
{"descr": "old", "network": "172.16.0.0/16", "gateway": {"WAN_GW": {"selected": 1}}, "enabled": "1"}, # execute re-preview
{"descr": "new", "network": "", "gateway": {"WAN_GW": {"selected": 1}}, "enabled": "1"}, # read-back: network wiped
])
respx_mock.get(_url("/routes/routes/getroute/r5")).mock(side_effect=lambda req: httpx.Response(200, json={"route": next(state)}))
_backup(respx_mock)
respx_mock.post(_url("/routes/routes/setroute/r5")).mock(return_value=httpx.Response(200, json={"result": "saved"}))
respx_mock.post(_url("/routes/routes/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
change = {"kind": "route", "action": "set", "uuid": "r5", "fields": {"descr": "new"}}
plan = (await _preview(client, host_id, **change)).json()
done = (await _execute(client, host_id, plan["token"], **change)).json()
assert done["verified"] is False and done["mismatches"] == ["network"]
# ── generic object kinds ─────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_objects_list_returns_only_the_display_columns(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-o1")
respx_mock.get(_url("/syslog/settings/searchDestinations")).mock(return_value=httpx.Response(200, json={"rows": [
{"uuid": "d1", "enabled": "1", "transport": "udp4", "hostname": "192.0.2.10", "port": "514", "program": "", "level": "info",
"facility": "daemon", "certificate": "SECRET-CERT-REF", "description": "central"}]}))
body = (await client.get(f"/api/hosts/{host_id}/network/objects/syslog_destination")).json()
assert body["available"] is True and body["rows"][0]["hostname"] == "192.0.2.10"
assert "certificate" not in body["rows"][0] and "facility" not in body["rows"][0]
assert (await client.get(f"/api/hosts/{host_id}/network/objects/vlan")).status_code == 404 # typed kinds have their own lists
@pytest.mark.asyncio
async def test_objects_list_reports_a_missing_api_instead_of_failing(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-o2")
respx_mock.get(_url("/interfaces/gre_settings/searchItem")).mock(return_value=httpx.Response(404, json={"errorMessage": "Endpoint not found"}))
body = (await client.get(f"/api/hosts/{host_id}/network/objects/gre")).json()
assert body == {"available": False, "columns": body["columns"], "rows": []}
@pytest.mark.asyncio
async def test_tunables_always_need_acknowledgement_and_defaults_cannot_be_edited(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-o3")
add = (await _preview(client, host_id, kind="tunable", action="add", fields={"tunable": "vfs.read_max", "value": "128"})).json()
assert [r["code"] for r in add["risks"]] == ["tunable_change"] and add["requires_acknowledgement"] is True
_item(respx_mock, "/core/tunables/getItem/0add3c58db981d0d24d5d4ab322e3695", "sysctl",
{"tunable": "net.inet.tcp.drop_synfin", "value": "1", "descr": "built in"})
default = (await _preview(client, host_id, kind="tunable", action="set", uuid="0add3c58db981d0d24d5d4ab322e3695", fields={"value": "0"})).json()
assert [b["code"] for b in default["blocks"]] == ["tunable_default_row"] and default["can_execute"] is False
@pytest.mark.asyncio
async def test_lagg_or_bridge_containing_the_management_device_needs_acknowledgement(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-o4") # reached on vtnet0
mine = (await _preview(client, host_id, kind="lagg", action="add", fields={"members": "vtnet0,vtnet2", "proto": "failover"})).json()
assert [r["code"] for r in mine["risks"]] == ["management_member"]
other = (await _preview(client, host_id, kind="lagg", action="add", fields={"members": "vtnet2,vtnet3", "proto": "failover"})).json()
assert other["risks"] == []
@pytest.mark.asyncio
async def test_vxlan_and_gif_are_listed_and_changed_like_the_other_devices(client, respx_mock):
host_id = await _setup(client, respx_mock, "fw-o5")
respx_mock.get(_url("/interfaces/vxlan_settings/searchItem")).mock(return_value=httpx.Response(200, json={"rows": [
{"uuid": "x1", "vxlanif": "vxlan1", "vxlanid": "100", "vxlanlocal": "10.20.30.57", "vxlanremote": "10.20.30.58",
"vxlangroup": "", "vxlandev": "lan", "secret_like": "dropped"}]}))
body = (await client.get(f"/api/hosts/{host_id}/network/objects/vxlan")).json()
assert body["rows"][0]["vxlanid"] == "100" and "secret_like" not in body["rows"][0]
_backup(respx_mock)
add = respx_mock.post(_url("/interfaces/gif_settings/addItem")).mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": "g1"}))
respx_mock.post(_url("/interfaces/gif_settings/reconfigure")).mock(return_value=httpx.Response(200, json={"status": "ok"}))
_item(respx_mock, "/interfaces/gif_settings/getItem/g1", "gif", {"local-addr": "10.20.30.57", "remote-addr": "198.51.100.9", "descr": "t"})
change = {"kind": "gif", "action": "add", "fields": {"local-addr": "10.20.30.57", "remote-addr": "198.51.100.9", "descr": "t"}}
plan = (await _preview(client, host_id, **change)).json()
done = await _execute(client, host_id, plan["token"], **change)
assert done.status_code == 200 and done.json()["verified"] is True, done.text
assert _body(add.calls.last) == {"gif": change["fields"]}
+94
View File
@@ -0,0 +1,94 @@
"""Network configuration differences between the nodes of an HA cluster."""
import httpx
import pytest
from app.database import SessionLocal
from app.schemas.network import HostNetwork
from app.services import network_drift_service, notification_checks
from tests.test_network import DHCP_HOSTS, DNS_HOSTS, GATEWAYS, INTERFACES_INFO, KEA_RESERVATIONS, KEA_SUBNETS, ROUTES, VIPS, VLANS, ASSIGNMENTS
ENDPOINTS = {
"/interfaces/overview/interfacesInfo": INTERFACES_INFO, "/interfaces/assignment/searchItem": ASSIGNMENTS,
"/interfaces/vlan_settings/searchItem": VLANS, "/interfaces/vip_settings/searchItem": VIPS,
"/routing/settings/searchGateway": GATEWAYS, "/routes/routes/searchroute": ROUTES,
"/unbound/settings/searchHostOverride": DNS_HOSTS, "/dnsmasq/settings/searchHost": DHCP_HOSTS,
"/kea/dhcpv4/searchSubnet": KEA_SUBNETS, "/kea/dhcpv4/searchReservation": KEA_RESERVATIONS,
}
async def _host(client, name, base, cluster="ha"):
r = await client.post("/api/hosts", json={"name": name, "base_url": base, "api_key": "k", "api_secret": "s",
"verify_tls": False, "cluster_name": cluster})
assert r.status_code == 201, r.text
def _serve(respx_mock, base, **replace):
for path, body in ENDPOINTS.items():
respx_mock.get(f"{base}/api{path}").mock(return_value=httpx.Response(200, json=replace.get(path, body)))
def _net(**replace) -> HostNetwork:
data = {
"host_id": "x", "host_name": "x", "interfaces": [], "vlans": [], "vips": [], "gateways": [], "routes": [],
"management_device": None, "can_configure_interfaces": True, "unavailable": [],
}
data.update(replace)
return HostNetwork(**data)
def test_identical_configuration_has_no_differences_even_with_per_node_values():
vip_a = {"uuid": "1", "interface": "lan", "interface_label": "LAN", "mode": "carp", "address": "192.168.1.1/24",
"vhid": "10", "advbase": "1", "advskew": "0", "description": "x"}
vip_b = {**vip_a, "uuid": "2", "advskew": "100"} # advskew legitimately differs between master and backup
a, b = _net(vips=[vip_a]), _net(vips=[vip_b])
assert network_drift_service.compare({"a": a, "b": b}) == []
def test_missing_and_different_objects_are_reported_with_who_has_what():
vlan = {"uuid": "1", "parent": "vtnet1", "tag": "10", "pcp": "0", "description": "LAN", "device": "vtnet1_vlan10"}
gw_a = {"uuid": "g", "name": "WAN_GW", "description": "", "interface": "wan", "interface_label": "WAN", "ip_protocol": "inet",
"address": "10.0.0.1", "is_default": True, "disabled": False, "priority": "255", "status": "Online"}
a = _net(vlans=[vlan, {**vlan, "uuid": "2", "tag": "20"}], gateways=[gw_a])
b = _net(vlans=[vlan], gateways=[{**gw_a, "address": "10.0.0.9"}])
rows = {(r["category"], r["key"]): r for r in network_drift_service.compare({"a": a, "b": b})}
assert rows[("vlans", "20@vtnet1")]["status"] == "missing" and rows[("vlans", "20@vtnet1")]["missing_from"] == ["b"]
assert rows[("gateways", "WAN_GW")]["status"] == "different"
assert rows[("gateways", "WAN_GW")]["signatures"]["a"] != rows[("gateways", "WAN_GW")]["signatures"]["b"]
assert ("vlans", "10@vtnet1") not in rows
def test_a_category_one_node_cannot_deliver_is_not_reported_as_a_difference():
vlan = {"uuid": "1", "parent": "vtnet1", "tag": "10", "pcp": "0", "description": "", "device": "d"}
a, b = _net(vlans=[vlan]), _net(vlans=[], unavailable=["vlans"])
assert network_drift_service.compare({"a": a, "b": b}) == []
@pytest.mark.asyncio
async def test_endpoint_and_alert_report_a_route_missing_on_one_node(client, respx_mock):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
await _host(client, "fw-da", "https://10.20.30.91")
await _host(client, "fw-db", "https://10.20.30.92")
_serve(respx_mock, "https://10.20.30.91")
_serve(respx_mock, "https://10.20.30.92", **{"/routes/routes/searchroute": {"rows": []}})
body = (await client.get("/api/network/drift")).json()
cluster = next(c for c in body["clusters"] if c["name"] == "ha")
assert cluster["unreachable"] == []
diff = [d for d in cluster["differences"] if d["category"] == "routes"]
assert [(d["key"], d["status"], d["missing_from"]) for d in diff] == [("10.9.0.0/16", "missing", ["fw-db"])]
async with SessionLocal() as db:
problems = await notification_checks.check_network_drift(db)
mine = [v for k, v in problems.items() if k.startswith("ha:routes:")]
assert mine and mine[0]["missing_from"] == ["fw-db"] and "Route" in mine[0]["object"]
@pytest.mark.asyncio
async def test_an_unreachable_node_keeps_open_alerts_instead_of_resolving_them(client, respx_mock):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
await _host(client, "fw-dc", "https://10.20.30.93", cluster="ha2")
await _host(client, "fw-dd", "https://10.20.30.94", cluster="ha2")
_serve(respx_mock, "https://10.20.30.93")
respx_mock.get(url__regex=r"https://10\.20\.30\.94/.*").mock(side_effect=httpx.ConnectError("down"))
async with SessionLocal() as db:
problems = await notification_checks.check_network_drift(db)
assert "ha2:" in problems.preserve_prefixes
+35
View File
@@ -177,6 +177,41 @@ async def test_check_rule_drift_no_drift_when_identical(client, respx_mock):
assert problems == {} assert problems == {}
@pytest.mark.asyncio
async def test_drift_stays_open_while_a_cluster_member_is_unreachable(client, respx_mock):
"""Live 26.7.5: rebooting one firewall made every open drift problem
resolve (only one host reachable => nothing to compare) and re-open
afterwards, sending a resolved+opened pair per problem."""
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
await _create_host(client, "fw-a", "https://fw-a.test", cluster_name="ha")
await _create_host(client, "fw-b", "https://fw-b.test", cluster_name="ha")
respx_mock.get("https://fw-a.test/api/firewall/filter/searchRule").mock(
return_value=httpx.Response(200, json={"rows": [_live_rule_row("r1", "Allow SSH"), _live_rule_row("r2", "Only A")]})
)
b = respx_mock.get("https://fw-b.test/api/firewall/filter/searchRule").mock(
return_value=httpx.Response(200, json={"rows": [_live_rule_row("r1", "Allow SSH")]})
)
async with SessionLocal() as db:
current = await notification_checks.check_rule_drift(db)
opened, resolved = await notification_dedup.diff_and_update(db, "rule_drift", current)
await db.commit()
assert len(opened) == 1 and not resolved
b.mock(side_effect=httpx.ConnectError("down")) # fw-b reboots
current = await notification_checks.check_rule_drift(db)
assert dict(current) == {}
opened, resolved = await notification_dedup.diff_and_update(db, "rule_drift", current)
await db.commit()
assert not opened and not resolved # still open, nothing sent
b.mock(return_value=httpx.Response(200, json={"rows": [_live_rule_row("r1", "Allow SSH")]}))
current = await notification_checks.check_rule_drift(db)
opened, resolved = await notification_dedup.diff_and_update(db, "rule_drift", current)
await db.commit()
assert not opened and not resolved # same problem, no duplicate "opened"
# ── check_revert_failed / check_maintenance_unknown ───────────────────── # ── check_revert_failed / check_maintenance_unknown ─────────────────────
@pytest.mark.asyncio @pytest.mark.asyncio
Loaded 100 of 155 files, more files were not shown because too many files have changed in this diff. Show more