Compare commits

..
27 Commits
Author SHA1 Message Date
Julian Boldt d93b154660 Release v1.33.0
Release / verify (push) Waiting to run
Release / test (push) Blocked by required conditions
CI / backend (push) Blocked by required conditions
CI / frontend (push) Blocked by required conditions
CI / checks (push) Blocked by required conditions
- **Aus dem Testlauf 1.31.0 behoben (T2 bis T6, B15):**
  - **Benutzerliste:** `GET /api/users` zeigte jedem angemeldeten Konto (auch Nur-Lese, auch per Token) alle Benutzernamen, Rollen, letzte Anmeldung und 2FA-Status.
    Jetzt sehen **nur Admins** die ganze Liste, alle anderen nur das eigene Konto.
  - **Hosts:** Dieselbe Adresse zweimal einzutragen wird abgelehnt (`HOST_URL_EXISTS`, auch beim Ändern der Adresse). Die Hosts-Seite und der Einrichtungsassistent
    prüfen die Verbindung beim Anlegen (Schlüssel abgelehnt / zu wenig Rechte / Zertifikat / keine Verbindung), speichern bei einem Fehler nichts und bieten
    „Trotzdem anlegen“. Über die API ist das `verify_connection: true` (Standard aus). **Löschen verlangt jetzt auch am Server den Hostnamen:**
    `DELETE /api/hosts/{id}?confirm_name=<Name>` (sonst 409); Skripte, die Hosts löschen, müssen das anpassen.
  - **Zertifikate:** `cert_type: v3_ca` wird nicht mehr angenommen (es entstand ein Zertifikat mit CA-Erweiterung, keine nutzbare CA); die Oberfläche bot es nie an.
  - **Traceroute:** Zwischenstationen ohne Antwort stehen als Zeile „keine Antwort“ in der Liste (vorher sprang die Zählung 1 → 3); ein Weg ganz ohne Antwort wird erklärt.
  - **Netzwerk:** Die Vorschau prüft Zahlenbereiche vorab (VLAN-Tag 1–4094, PCP 0–7, VHID 1–255, Basis 1–254, Skew 0–254, gegen das Modell der OPNsense gelesen): `422` mit dem
    Feldnamen statt `502` von der Firewall.
  - **Gruppen:** Eine Gruppe, die ein VPN-Objekt als Ziel hat, lässt sich nicht mehr löschen (die Verknüpfung fiel still weg).
  - Das Skript `scripts/verify_i18n.mjs` läuft jetzt auch gegen die HTTPS-Test-Instanz.
- **Regel-Bereinigung: OPNexus selbst ist automatisch geschützt (B10).** Beim Planen fragt die Bereinigung jede Firewall, von welcher Adresse sie OPNexus kommen sieht
  (die eigene API-Verbindung in der Zustandstabelle), und schützt diese Adresse zusätzlich zu den Verwaltungsquellen. Vorher konnte eine Regel abgeschaltet werden, über
  die nur OPNexus die Firewall erreicht, wenn in den Quellen nur Arbeitsplätze standen; dann scheiterte auch das automatische Zurücksetzen. Die Vorschau zeigt die Adresse
  je Firewall (nur Admins) oder sagt, dass sie sich nicht ermitteln ließ (API-Recht „Diagnose: Zustände“, Management-Tunnel).
- **Keine verlorenen Regeltreffer mehr (B11):** Jedes Anwenden setzt die Regelzähler der Firewall zurück. OPNexus liest sie jetzt unmittelbar vor jedem Anwenden, das es
  selbst auslöst (Regeln, NAT, Aliase, IPsec/OpenVPN/WireGuard, Netzwerk); vorher konnte eine Regel deshalb fälschlich als „ungenutzt“ gelten.
- **Einstellungen › Regel-Bereinigung:** schlägt die eigene Browser-Adresse als Verwaltungsquelle vor (B12). Geprüft: OPNsense 26.7 hat kein „savepoint“ zum Zurücknehmen
  durch die Firewall selbst (Quellcode und live 404), das automatische Zurücksetzen bleibt bei OPNexus.

- **Installation unter Windows:** Neue Anleitung auf der Website (`/docs/windows/`, auch auf Englisch) und im Handbuch: Docker Desktop mit WSL 2, Zertifizierungsstelle importieren, Zugriff aus dem Netz (Windows-Firewall), Updates mit `opnexus` in WSL und Fehlerbehebung. Entwickelt und getestet wird weiterhin unter Linux; die Anleitung sagt das offen.
- **Sicherheit und Stabilität (aus einer externen Prüfung):**
  - **Rückweg bleibt erhalten:** Wer nach „Konfiguration geschrieben, Anwenden fehlgeschlagen“ erneut ausrollte, überschrieb den ursprünglichen Rückweg; die automatische
    Rücknahme stellte dann den unbestätigten Stand wieder her. Jetzt bleibt der Rückweg der Stand vor dem ersten Versuch. Wartung, Regel-Bereinigung und
    „Konfiguration als Code“ behandeln eine solche Zeile wie eine offene Bestätigung. Gilt für Regeln, Aliase, Port-Forwards, 1:1-NAT, Outbound-NAT und Hierarchie-Regeln.
  - **Anmeldung und 2FA:** Eine Passwort-Anmeldung, die genau während der Aktivierung von 2FA lief, bekam eine Sitzung ohne zweiten Faktor. Jetzt wird sie mit `TOTP_REQUIRED` abgewiesen.
  - **Letzter Administrator:** Zwei Administratoren, die sich gleichzeitig herabstuften oder gegenseitig löschten, konnten alle Administratoren entfernen. Beides ist jetzt serialisiert.
  - **Operatoren:** Mit ausgeschaltetem „Operatoren dürfen ausrollen“ konnten sie noch Snapshots (Boot-Umgebungen) anlegen. Ein Test hält künftig jede schreibende Route
    fest, die keiner Rechte-Regel zugeordnet ist.
  - **Zertifikate und ACME:** Der Ziel-Host ist nach dem ersten Push fest (`409 ZIEL_HOST_NACH_AUSROLLEN_FEST`); vorher meldete ein Host-Wechsel Erfolg für einen Host, der nie
    etwas erhielt. Ein ACME-Konto mit Anfragen lässt sich nicht auf einen anderen Host verschieben (`ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET`).
  - **VPN-Ausrollen:** IPsec, OpenVPN und WireGuard sichern jede von der Firewall vergebene Kennung sofort; bei einem Abbruch entstehen beim nächsten Versuch keine Duplikate.
    Pro VPN läuft nur ein Ausrollen oder Entfernen zugleich (`409 VPN_DEPLOY_LAEUFT_BEREITS`).
  - **Konfiguration als Code:** Gültiges YAML mit falsch geformtem Abschnitt (z. B. `aliases: 42`) ergibt `422 CONFIG_INVALID` statt eines Serverfehlers.
  - **`.env`:** Die Anleitungen legen sie mit `install -m 600` an; `opnexus update` und `opnexus status` setzen eine zu offene Datei auf 0600. `opnexus rollback` liest seinen
    Update-Zustand nur noch als Text und führt ihn nie aus.
  - **Abhängigkeiten:** Next.js 15.5.28, PyJWT 2.15.1.
2026-10-24 00:22:22 +02:00
Julian Boldt cf9981c6eb Release v1.32.1
Release / verify (push) Waiting to run
Release / test (push) Blocked by required conditions
CI / backend (push) Blocked by required conditions
CI / frontend (push) Blocked by required conditions
CI / checks (push) Blocked by required conditions
- **Korrektur: Rollouts erkannten die Zielversion bei anderen Release-Typen falsch.** OPNexus las die neue Version nur am Kernpaket `opnsense`. Bei der
  Business-Edition (`opnsense-business`) und beim Wechsel des Release-Typs (z. B. auf `opnsense-devel`, dann als neues Paket angeboten) fiel es auf
  `product_latest` zurück; das Gate hielt den Rollout dann fälschlich mit „Version passt nicht“ an. Live gefunden und belegt (Rollout mit Cluster in einer Welle).
- **Plan-Hinweis zu Alias-Namen ehrlich formuliert:** „kein zentraler Alias in OPNexus“ statt „kein Alias dieses Namens“ (von Hand auf der Firewall
  angelegte Aliase sind in Ordnung; der Planer fragt bewusst keine Firewall).
2026-10-23 12:40:49 +02:00
Julian Boldt 4eea6424e7 Release v1.32.0
Release / verify (push) Waiting to run
Release / test (push) Blocked by required conditions
CI / backend (push) Blocked by required conditions
CI / frontend (push) Blocked by required conditions
CI / checks (push) Blocked by required conditions
- **Konfiguration als Code:** Unter **System › Konfiguration als Code** (und per `opnexus export` / `opnexus apply`) lassen sich Gerätegruppen, Hierarchiegruppen,
  Aliase, Filterregeln, Port-Forwards, 1:1-NAT, Outbound-NAT, Hierarchie-Regeln und Standort-Variablen als **eine YAML-Datei exportieren** (sortiert, ohne Zeitstempel,
  ohne Zugänge oder Schlüssel, für Git und Reviews) und wieder **importieren**: erst ein **Plan** (neu, geändert, entfernt, unverändert, blockiert, mit Hinweisen), dann
  das **Übernehmen** durch einen Administrator in einer Transaktion mit dem Token genau dieses Plans. **Der Import ändert nur die Entwürfe in OPNexus, nie eine Firewall**;
  ausgerollt wird wie immer über Vorschau → Ausrollen → Bestätigen. Ausgerollte oder noch verwendete Objekte entfernt er nie (Prune blockiert den Plan). Die Datei wird
  nur als Daten gelesen (keine Anker, Tags, Merge-Schlüssel, Mehrfach-Dokumente; 1 MiB, strenges Schema mit Zeile und Pfad bei Fehlern). Plan und Export gehen auch mit „Lesen“-Token
  und Nur-Lese-Konten, `opnexus apply --dry-run` liefert Exit-Codes für CI (0 / 2 / 3 / 4). Regeln und NAT-Objekte tragen dafür einen stabilen `key` (Migration, wird für
  Bestand automatisch vergeben). Gruppenänderungen stehen jetzt im Audit.
- **Rückweg zur vorherigen Firmware:** Vor jedem Firmware-Update und Major-Upgrade (manuell, geplant, Rollout, CARP-Übergabe) legt OPNexus auf der Firewall ein
  **ZFS-Boot-Environment** `opnexus-pre-<Version>-<Zeit>` als Rückweg-Punkt an (ohne den Punkt startet das Update nicht; ohne ZFS mit Hinweis). Neue Aktion **Zur vorherigen Firmware**
  (Host-Wartung): Vorschau, Hostname, Downtime-Bestätigung, vorher lokale Sicherung der Konfiguration, dann Aktivieren, Prüfung und Neustart (scheitert ein Schritt, wird das
  vorherige Boot-Environment wieder eingeschaltet). **Der Rückweg setzt auch die Konfiguration zurück** (live belegt): Ausrollvorgänge seit dem Punkt werden als unklar markiert,
  damit das nächste Ausrollen wirklich schreibt. Ein Rollout macht nie von selbst einen Rückweg.
- **Rollout-Gate: Erreichbarkeitsproben.** Bis zu zehn Ping-/TCP-Proben je Plan, von der Firewall selbst ausgeführt, vor der Welle und nach der Beobachtungszeit:
  vorher ok und jetzt nicht mehr hält den Rollout an, schon vorher gestört ist nur ein Hinweis. (Erreichbarkeit von der Firewall aus, nicht der Verkehr durch sie hindurch.)
- Migrationen 0030 (`config_key`) und 0031 (`gate_probes`).
2026-10-23 11:27:47 +02:00
Julian Boldt a8ca0a14ef Release v1.31.1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Korrektur: `opnexus password` und `reset_2fa` stürzten ab.** Seit den Standort-Variablen (1.22) brach das Zurücksetzen des Admin-Passworts
  (`./opnexus password`) bzw. der Zwei-Faktor-Anmeldung (`docker compose exec backend python -m app.reset_2fa <benutzer>`) mit einem
  SQLAlchemy-Fehler ab, weil ein Datenmodell nicht registriert war. Der Weg bei vergessenem Passwort oder verlorenem 2FA-Gerät funktioniert wieder;
  ein Regressionstest sichert ab, dass jedes Hilfsskript alle Modelle kennt.
2026-10-22 23:45:57 +02:00
Julian Boldt b020e38d0a Release v1.31.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Regel-Bereinigung: Entfernen, Bestätigen, Löschen, Wiederherstellen:** Im Tab **Bereinigung** (Regelwerk › Regel-Analyse) lassen sich Kandidaten jetzt auswählen
  (Checkbox, höchstens so viele Regeln je Lauf wie in den Einstellungen; „vermutlich“ nur mit einer ausdrücklichen Einzelbestätigung je Regel, „unbekannt“ nie).
  **Vorschau** zeigt je Firewall (im Cluster zuerst der Backup) die Regeln mit Befund, Sicherheit und Schutzprüfung; **Ausrollen** deaktiviert die Regeln und wendet an.
  Danach läuft ein Countdown: **Bestätigen** heißt „ich erreiche die Firewall noch“, ohne Bestätigung setzt OPNexus **automatisch zurück**; bis dahin geht auch
  **Jetzt zurücksetzen**. Bestätigte Regeln lassen sich nach der Wartezeit (einstellbar, Standard 7 Tage) **löschen**; vorher sichert OPNexus den Regelinhalt, und
  unter **Verlauf der Bereinigungen** kommt die Regel mit **Wiederherstellen** deaktiviert und möglichst an alter Position zurück (sonst mit Hinweis am Ende der Gruppe);
  **Wieder aktivieren** hat wieder ein Bestätigungsfenster. Wichtig: **Ausrollen, Wiederholen und Wiederherstellen darf nur ein Administrator**, eine Vorschau gehört
  dem, der sie erstellt hat (ein Admin öffnet selbst eine neue); **Bestätigen geht nur in der Browser-Sitzung**, nie per API-Token. **Die Bereinigung bleibt gesperrt,
  bis unter Einstellungen die Verwaltungsquellen** (IP-Adressen oder Netze, von denen aus verwaltet wird) **eingetragen sind**: Regeln, die diesen Zugang treffen
  könnten, deaktiviert OPNexus nie. Dort stehen auch die Obergrenze je Lauf, die Wartezeit bis zum Löschen und **weitere Verwaltungsports** (zusätzlich zu
  API-Port, 443, 80, 22). Tragen Sie die Adresse ein, mit der OPNexus selbst bei den Firewalls ankommt (im Cluster für jeden Knoten). Quellen, Ports und
  Firewall-Adressen sehen nur Administratoren. Der Hinweis „Entfernen folgt“ entfällt.
- **Update: neuere CLI aus dem Zielrelease:** `opnexus update` (auch per Klick im Webinterface) holt sich nach dem Pull die CLI aus dem Updater-Image des
  Zielreleases und startet sich damit neu, damit Korrekturen am Update-Ablauf sofort greifen statt erst beim übernächsten Update. Das geschieht nur, wenn
  `cosign` und `cosign.pub` vorhanden sind und die Signatur des Image-**Digests** in diesem Lauf geprüft wurde (ausgeführt wird genau dieser Digest);
  sonst und bei jedem Fehler läuft das Update mit der vorhandenen CLI weiter. Abschalten: `OPNEXUS_NO_CLI_REFRESH=1`. Wirksam wird es ab dem Update
  *von* dieser Version.
2026-10-22 21:04:32 +02:00
Julian Boldt abd15318a3 Release v1.30.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
  Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
  trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
  dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
  höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
  verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
  Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
  stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
  Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
  verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
  Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
  automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
  erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
  Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
  15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
  Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
  mehr verdeckte Regeln als bisher, nie weniger belegte.
2026-10-22 10:27:02 +02:00
Julian Boldt ec20940139 Release v1.29.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fehlende API-Rechte werden sichtbar:** Darf der API-Schlüssel einer Firewall den Gateway- oder Dienststatus nicht lesen (OPNsense antwortet
  mit 403), fiel das bisher niemandem auf, und der standardmäßig aktive Gateway-Alarm konnte nie auslösen. Jetzt zeigen die Host-Seite
  (Abschnitt „Gateways“) und die Dashboard-Karte „Fleet-/CARP-Gesundheit“ ein Warn-Badge „API-Rechte fehlen: Gateways, Dienste“; der Tooltip nennt die
  OPNsense-Berechtigung, die beim Benutzer oder bei der Gruppe fehlt (System: Gateways, Status: Services, Lobby: Dashboard). Zusätzlich gibt es
  den neuen Alarm **API-Rechte fehlen** (standardmäßig an, abschaltbar unter Einstellungen › Benachrichtigungen, gleiche Entprellung wie die
  anderen Messungs-Alarme) und die Prometheus-Metrik `opnexus_host_api_rights_missing{host,read}`. Nur eine ausdrückliche Ablehnung zählt;
  Zeitüberschreitungen oder Serverfehler bleiben „unbekannt“.
- **Gateway-Alarm pro Gateway abschaltbar:** In der Gateway-Liste der Host-Seite schaltet eine Glocke den Alarm für ein einzelnes Gateway aus
  (z. B. ruhendes Zweit-WAN oder tote IPv6-Gateways); ein „Alarm aus“-Badge zeigt es an. Ein offener Alarm dieses Gateways löst sich dabei
  auf. Nur Administratoren, die Änderung steht im Audit-Log. Das Dashboard zeigt bei mehr als acht Gateways „+ N weitere“.
- **Firewall-Oberfläche (GUI-Zugriff) härter:** Ein frei gewordener Platz wird für eine **andere** Firewall erst 5 Minuten nach dem Ende der letzten
  Sitzung vergeben (Schutz, falls dort noch ein Tab offen ist, der die Cookies des Platzes teilt); dieselbe Firewall bekommt ihn sofort wieder.
  Sind alle Plätze belegt oder noch in der Abkühlzeit, nennt die Meldung den Grund. Ändern sich Adresse, festgesetztes Zertifikat oder Tunnelweg
  einer Firewall, während ihre Oberfläche offen ist, endet die Sitzung spätestens nach 20 Sekunden. Das Limit von 64 gleichzeitigen Sitzungen
  verdrängt keine laufende Sitzung mehr: eine neue wird abgelehnt (mit Meldung), bis eine endet. **Hinweis:** Das GUI-Gateway-Image muss zur
  Backend-Version passen (wie bisher beim Update beider Container).
- **Anmeldung (Cookies) härter:** Das alte Sitzungs-Cookie `opnexus-session` wird bei HTTPS **nicht mehr akzeptiert**. **Alle, die sich seit der
  Umstellung auf 1.27 nicht neu angemeldet haben, müssen sich einmal neu anmelden**; Skripte mit rohem Sitzungs-Cookie brauchen den Namen
  `__Host-opnexus-session` (API-Tokens sind nicht betroffen, ohne HTTPS bleibt `opnexus-session` der normale Name). Beim Abmelden wird ein altes
  Cookie weiterhin gelöscht. Das Cookie, das die SSO-Anmeldung an den Browser bindet, heißt bei HTTPS jetzt `__Host-opnexus-oidc-state`; der
  Rückweg vom Anbieter wird nur noch als Seitenaufruf (Navigation) von einer fremden Seite zugelassen, und ohne dieses Cookie scheitert er immer.
- **Webhook, Benachrichtigungskanäle und SSO-Anbieter: Schutz vor DNS-Rebinding:** Bisher prüfte OPNexus die Adresse, auf die ein Name zeigt, und
  der HTTP-Client löste den Namen danach noch einmal auf; ein Server unter Fremdkontrolle konnte beim zweiten Mal eine interne Adresse liefern. Jetzt
  verbindet OPNexus genau zu der geprüften Adresse (Host-Name, TLS-Zertifikatsprüfung und Anfrage bleiben beim Namen); zeigt eine Antwort auch nur
  teilweise auf Loopback, Link-Local oder einen Cloud-Metadaten-Dienst (auch AWS-IPv6 und Alibaba), kommt keine Verbindung zustande und die Zustellung meldet „Zieladresse nicht
  erlaubt“. Private Netze bleiben erlaubt (eigener ntfy, Gotify, Matrix). Proxy-Umgebungsvariablen werden für diese Aufrufe nicht mehr beachtet.
- **Benachrichtigungen parallel zugestellt:** Webhook, E-Mail und alle Kanäle werden gleichzeitig bedient, jeder mit eigener Zeitgrenze (20 Sekunden),
  und ein Prüfdurchlauf sendet insgesamt höchstens 90 Sekunden. Ein hängender Kanal verzögert weder die anderen noch die nächsten Prüfungen; im Log
  steht dann „timed out“ (ohne Adresse oder Token). Wie bisher wird einmal je Ereignis zugestellt, ohne Wiederholung, **außer** der Durchlauf war so voll, dass
  für ein Ereignis noch gar kein Versuch stattfand: es wird im nächsten Durchlauf zugestellt. Die Zustellung beginnt jetzt nach allen Prüfungen
  eines Durchlaufs statt mitten darin.
- **Rollouts: Fingerabdruck, verantwortliche Person, überlappende Termine:** Der Fingerabdruck eines aktivierten Plans umfasst jetzt auch die
  ausgelassenen und verschobenen Termine seines Fensters: **wer einen Termin auslässt oder verschiebt, muss einen automatischen Plan danach neu
  aktivieren** (ohne solche Ausnahmen bleiben bestehende Aktivierungen gültig; **Pläne, deren Fenster schon Ausnahmen haben, nach dem Update
  einmal neu aktivieren**, sonst meldet der Plan „Plan geändert“ und startet nicht automatisch). Die verantwortliche Person (Aktivierende, Startende, Freigebende)
  wird zusätzlich als Konto-ID gespeichert (Migration 0026, bestehende Einträge werden über den Namen zugeordnet): ein gelöschtes und unter gleichem
  Namen neu angelegtes Konto gilt nicht mehr als dieselbe Person, der Lauf pausiert bzw. der automatische Start unterbleibt mit „Person darf nicht mehr
  ausrollen“. Angezeigt wird weiter der Benutzername. Überlappen sich zwei Termine eines Wartungsfensters, zählt der zuletzt begonnene (bisher der
  früheste), sodass ein älterer, längst abgeschnittener Termin keinen neuen Start mehr blockiert.
- **Kleinigkeiten:** Ressourcen-Alarme zeigen bei fehlendem Messwert keine Zahl mehr („None %“ entfällt). Der Dialog „Von Firewall übernehmen“ zeigt ohne aktive
  Firewall einen Hinweis statt dauerhaft „Lädt…“. Fehlermeldungen beim Übernehmen und Verknüpfen nennen die Gründe in Klartext (Deutsch/Englisch) statt
  roher Codes wie `name_exists`; dasselbe gilt für „GUI-Zugriff nicht eingerichtet“ und blockierte Netzwerkänderungen. Der Test-Knopf der
  Benachrichtigungskanäle ist für Nicht-Admins ausgeblendet (der Server verweigert den Test ohnehin). Der Tooltip der aktiven Plätze nennt keine
  Umgebungsvariable mehr. Eine Gateway-Sitzung, die der Gateway-Container nicht mehr bestätigt (Absturz), zählt für die Abkühlzeit der Plätze als
  beendet.
- **Grafana-Dashboard erweitert:** neue Felder für Gateways (erreichbar, Paketverlust, Latenz), Swap, gestoppte Dienste, fehlende API-Rechte und
  Rollouts nach Status; dazu die Metrik `opnexus_rollout_runs{status}`.
2026-10-22 01:58:06 +02:00
Julian Boldt 8128d14da4 Release v1.28.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Übernehmen: mit vorhandenem zentralem Objekt verknüpfen:** Dieselbe von Hand angelegte Regel, derselbe Alias oder dieselbe NAT-Regel auf mehreren
  Firewalls (z. B. beide HA-Knoten) wird nicht mehr als zweites Objekt übernommen. Im Dialog „Von Firewall übernehmen“ erscheint bei exakter
  Übereinstimmung (nach Einsetzen der Host-Variablen, bei Aliasen inkl. Verweisen) die Aktion **Verknüpfen**: die Firewall wird weiteres Ziel des
  vorhandenen Objekts, die Vorschau zeigt „keine Änderung“, und der nächste Rollout aktualisiert alle Firewalls an einem Objekt. Auf der Firewall wird
  dabei nichts geändert. Unterscheidet sich ein gleichnamiges Objekt, zeigt der Dialog die abweichenden Felder; verknüpft wird dann nie. „Alle
  übernehmbaren“ fragt vorher nach („3 verknüpfen, 2 neu übernehmen, 1 nicht übernehmbar“). Aliase, die bisher wegen „Name existiert“ abgelehnt wurden,
  lassen sich so übernehmen.
  Wartet das vorhandene Objekt gerade auf eine Bestätigung, wird nicht verknüpft; „Alle übernehmbaren“ lässt den Eintrag dann aus. Verknüpfen
  macht ein gerade geöffnetes Bearbeiten-Formular des Objekts ungültig, damit die neue Firewall nicht versehentlich wieder aus den Zielen fällt.

- **Tooltips überarbeitet:** Der deaktivierte Button „Oberfläche öffnen“ nennt jetzt den Grund im Klartext samt nächstem Schritt (statt nur einer
  Variablen wie `GUI_GATEWAY_TOKEN`); dasselbe gilt für ausgeschaltete Funktion, deaktivierte Firewall, Nicht-HTTPS-Adresse und ausgeschaltete
  Zertifikatsprüfung. Texte durchgehend in Du-Form, kürzere VPN-Einleitungen, verständlichere Hinweise bei gesperrten Buttons (Zertifikate, Benutzer,
  Dienste). Icon-Buttons (Bearbeiten, Löschen, Schließen, Reihenfolge) zeigen beim Darüberfahren ihren Namen.
2026-10-21 23:05:16 +02:00
Julian Boldt 95f297d3f2 Release v1.27.2
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **`opnexus update` prüft den Proxy vor dem Wechsel:** Nach dem Ziehen der Images validiert das Werkzeug die Caddy-Konfiguration des neuen
  Proxy-Images mit den Umgebungsvariablen *dieser* Installation. Ist sie ungültig, bricht das Update ab, bevor etwas gewechselt wurde.
  Der Fall aus 1.27.0 (Compose-Datei mit leerem `OPNEXUS_GUI_SITES`) wird dabei automatisch repariert: Das Werkzeug trägt den Platzhalter
  `OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` ein und sagt es. Installationen, die noch die Compose-Datei aus 1.27.0 haben,
  müssen also nur einmal `opnexus update` ausführen.
- **`opnexus update` erkennt einen abstürzenden Proxy:** Ein Dienst ohne Healthcheck (der Proxy) gilt jetzt erst nach 10 Sekunden
  durchgehender Laufzeit als gesund. Bisher meldete das Update „Fertig“, obwohl der Proxy in einer Neustart-Schleife hing, und das
  automatische Zurückrollen griff nicht.
2026-10-21 21:23:53 +02:00
Julian Boldt ad17dcad25 Release v1.27.1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fix (1.27.1): Proxy startete mit der neuen Compose-Datei nicht.** Mit der `docker-compose.yml` aus 1.27.0 gab Compose die Variable
  `OPNEXUS_GUI_SITES` als leeren Text an den Proxy weiter; Caddy nutzt seinen Standardwert aber nur bei *nicht gesetzter* Variable und
  brach mit „server block without any key“ ab, OPNexus war dann nicht mehr erreichbar. Die Compose-Datei setzt jetzt einen gültigen
  Platzhalter als Standard. Betroffen waren nur Installationen, die die neue Compose-Datei aus 1.27.0 übernommen hatten (ohne
  `OPNEXUS_GUI_SITES` in der `.env`); wer nur die Images aktualisiert hatte, war nicht betroffen. Sofort-Abhilfe ohne Update:
  `OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` eintragen und `docker compose up -d proxy` ausführen.
2026-10-21 20:44:14 +02:00
Julian Boldt 5cb2ebe1c6 Release v1.27.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Vorgesehen als MINOR-Release **1.27.0** „Firewall-Oberfläche über OPNexus“.

- **Firewall-Oberfläche über OPNexus:** Auf der Host-Seite öffnet **Oberfläche öffnen** die Weboberfläche der Firewall
  in einem neuen Tab, auch bei Firewalls hinter NAT im Management-Tunnel. Nur für Admins; jede Firewall läuft in einer
  eigenen Browser-Origin (Platz = Port), Cookies bleiben auf dem Server, das Ticket gilt einmalig 30 Sekunden,
  Inaktivitäts- und Gesamtlimit sind einstellbar, Ticket, Start und Ende stehen im Audit (Art „GUI-Zugriff“).
  Standardmäßig **aus**: braucht das Compose-Profil `gui`, `OPNEXUS_GUI_SITES`, `GUI_PORTS`, `GUI_GATEWAY_TOKEN`,
  `COOKIE_SECURE=true` und den Schalter unter Einstellungen (Handbuch, „Firewall-Oberfläche über OPNexus“).
  Live getestet auf OPNsense 26.7.
- **Einschränkungen:** kein WebSocket, keine automatische Anmeldung, höchstens 4 Plätze gleichzeitig. OPNexus-Adresse
  vorher in die Whitelist des Anmeldeschutzes (sshguard) der Firewall eintragen, sonst können Fehlanmeldungen die
  API-Zugriffe von OPNexus aussperren. Nach einem Neustart des wg-hub braucht der Tunnel 2 bis 4 Minuten.
- **Hinweis für Betreiber mit eigenen Skripten:** Bei HTTPS heißt das Sitzungs-Cookie jetzt `__Host-opnexus-session`
  (bestehende Sitzungen werden ohne Abmelden übernommen), und Cookie-Anfragen an `/api` mit `Sec-Fetch-Site: same-site/cross-site`
  werden abgelehnt. Zugriffe mit API-Token sind nicht betroffen.
2026-10-21 20:14:06 +02:00
Julian Boldt 18d489693d Release v1.26.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Geplant als MINOR-Release **1.26.0** (Regel-Analyse). Backend, MCP und Oberfläche.

- **Regel-Analyse „Welche Regel greift?“ (API):** `GET /api/hosts/{id}/rule-analysis/match` und
  `GET /api/rule-analysis/match` (ganze Flotte, Auswahl oder Gerätegruppe) beantworten für Quelle, Ziel, Protokoll
  und Port, ob eine Firewall das erste Paket einer neuen Verbindung durchlässt oder blockiert, welche Regel das
  entscheidet und warum (Prüfprotokoll Regel für Regel). Grundlage ist der tatsächlich geladene pf-Regelsatz
  inklusive automatischer Regeln, Floating- und Gruppenregeln, `quick`/letzter Treffer, Negation, verschachtelter
  Aliase (pf-Tabellentest der Firewall), Port-Aliase und Adressfamilien. Jede Antwort trägt eine ehrliche
  Konfidenz **sicher / vermutlich / unbekannt** mit Begründungscodes; was nicht belegbar ist (z. B. `tagged`,
  nicht auflösbare Aliase, geratenes Interface, NAT in der Nähe), wird nie als „sicher“ gemeldet. Nur lesend,
  auch mit `read`-Token nutzbar; es wird nichts gespeichert und nichts an der Firewall geändert.
- **MCP:** neues Werkzeug `rule_match` für KI-Assistenten (gleiche Abfrage, ohne Prüfprotokoll).
- **Seite „Regel-Analyse“ (Regelwerk):** Formular für Hosts oder Gerätegruppe, Quell-/Ziel-IP, Protokoll, Ports,
  Eingangs-Interface („automatisch“ oder Liste) und optional die Ausgangsseite. Die Abfrage steht in der Adresszeile
  (teilbar, wird beim Öffnen sofort ausgeführt). Je Firewall eine Karte mit Erlaubt/Blockiert/Abgewiesen, Sicherheit
  des Urteils, entscheidender Regel und Hinweisen; ein „unbekannt“-Ergebnis erscheint bewusst nicht als Urteil.
  Aufklappbares „Warum?“ zeigt jede geprüfte Regel mit ✓/✗/? je Feld, überstimmte Regeln durchgestrichen. Auch mit
  Nur-Lesen-Rolle nutzbar.
2026-10-21 13:12:11 +02:00
Julian Boldt a9f562eec6 Release v1.25.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Geplant als MINOR-Release **1.25.0** (Flotten-Rollout).

- **Rollouts in Wellen (System › Rollouts):** Firmware-Updates für die ganze Flotte in Ringen, z. B. erst
  `ring-canary`, dann `ring-pilot`, dann `ring-prod` (Wellen aus Gerätegruppen). Jeder Host läuft über die
  bestehende geplante Wartung (Backup vor dem Update, CARP-Übergabe für den MASTER, Audit), immer nur ein Host
  gleichzeitig und nur ein Rollout zur selben Zeit; im Cluster zuerst der BACKUP, dann der MASTER. Nach jeder Welle
  folgen eine Beobachtungszeit und eine **Gesundheitsprüfung** (erreichbar, Zielversion installiert, CARP je VHID,
  Gateways und Dienste wie vorher, keine offenen Sperren, optional Sicherheits-Score). Rot hält den Rollout an
  (nach einer Nachprüfung zwei Minuten später) und meldet sich. Anhalten, Fortsetzen (prüft zuerst alles bisher
  Aktualisierte erneut), Welle freigeben, Host überspringen und Abbrechen auf der Lauf-Seite; die Seite zeigt
  Wellen, Hosts (Version vorher → nachher, aktueller Schritt) und die Prüfmatrix.
- **Sicherheitsnetz:** Weicht die angebotene Version in einem späteren Ring von der des ersten Rings ab, hält der
  Rollout an (OPNsense installiert immer die neueste). Ein Host ohne Update wird übersprungen, ein fehlgeschlagener
  nie von selbst wiederholt. Angehalten wird auch, wenn die verantwortliche Person das Recht zum Ausrollen verliert,
  eine fremde Wartung im Cluster länger als 15 Minuten offen ist, der MASTER ohne Übergabe dran wäre oder ein Lauf
  älter als 14 Tage ist. Höchstens 10 Wellen und 100 Hosts je Plan.
- **Wiederkehrende Wartungsfenster:** wöchentlich oder monatlich (n-ter Wochentag, letzter, fester Tag) in einer
  Zeitzone, mit Sommerzeit-Regel, Abschneidezeit („neue Hosts nur in den ersten N Minuten“) und einzelnen Terminen
  zum Auslassen oder Verschieben. Ein Plan mit Fenster startet neue Hosts nur darin; mit **automatischem Start**
  (nur stabile Updates) beginnt er zu jedem Termin von selbst, aber nur, solange Plan, Gruppen und Fenster seit
  der Aktivierung unverändert sind. Jede Änderung deaktiviert ihn.
- **Geplante Wartung prüft sich selbst:** Neue Option „Ergebnis automatisch prüfen und abschließen“ (im Dialog
  vorausgewählt, bei Rollouts immer an). Nach dem Auftrag prüft OPNexus, ob die Firmware wieder bereit ist, der
  erwartete Neustart stattfand und die Version lesbar ist, und schließt die Wartung dann selbst ab. Bisher blieb
  sie auf „übergeben“ und sperrte Host und Cluster, bis jemand sie von Hand abschloss.
- **Benachrichtigungen:** „Rollout angehalten“ und „Rollout nicht automatisch gestartet“ (ein Schalter, standardmäßig an).
- **Rechte:** Rollout starten/aktivieren/fortsetzen/freigeben/überspringen/abbrechen und Wartungsfenster ändern
  gelten als Ausrollen (Aufgabentrennung); **Anhalten** darf jeder mit Schreibrecht. Update Center: Hosts im
  laufenden Rollout sind für manuelle Firmware-Aktionen gesperrt. Ein OPNexus-Update während eines Rollouts ist
  sicher, wird aber angekündigt.
- **Getestet:** live auf den Test-Firewalls (OPNsense 26.7): automatischer Start im Fenster, Canary-Update
  26.7.1_1 → 26.7.7, absichtlich gestoppter Dienst → Gate rot → angehalten (zweiter Ring unberührt,
  Benachrichtigung), Fortsetzen, Pilot-Update 26.7.6 → 26.7.7 auf dem damaligen MASTER mit CARP-Übergabe und einem
  Neustart des OPNexus-Backends mitten im Auftrag (genau ein Update-Auftrag), Freigabe, Host ohne Update
  übersprungen, Anhalten/Fortsetzen/Abbrechen von Hand, Termin auslassen. **Nur gemockt:** Major-Upgrade als
  Rollout, ein Cluster in *einer* Welle, Cluster-Prüfung vor dem zweiten Knoten, abweichende Zielversion,
  Zeitüberschreitungen, Rechteverlust, Fenster schließt mitten im Lauf; die Sommerzeit-Regeln sind Rechentests.
2026-10-21 11:59:41 +02:00
Julian Boldt a3274f64ae Release v1.24.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Alarme für Gateways, Ressourcen und Dienste:** Neue Ereignisse in Einstellungen › Benachrichtigungen.
  **Gateway gestört** (standardmäßig **an**) meldet ein Gateway, das OPNsense als „down“ führt oder dessen Paketverlust (ab 20 %) bzw.
  Latenz (ab 500 ms) über der Schwelle liegt; bewusst abgeschaltete und nicht überwachte Gateways bleiben still. **Ressourcen-Schwelle**
  (standardmäßig aus) prüft Festplatte (ab 90 %), Arbeitsspeicher (ab 95 %), Swap (ab 50 %) und CPU-Last (ab 2 pro Thread), **Dienst ausgefallen**
  (standardmäßig aus) eine einstellbare Liste (Unbound, Dnsmasq, DHCP, NTP, OpenVPN, WireGuard, IPsec …) und meldet nur Dienste, die zuvor
  liefen. Alle drei stützen sich auf die Minutenmessung und lösen erst aus, wenn die letzten 3 (Ressourcen: 5) Messungen übereinstimmen, und
  enden erst, wenn alle wieder in Ordnung sind: Einzelspitzen und flackernde Verbindungen schicken nichts. Der Gateway-Status (Verlust, Latenz,
  Abweichung) steht außerdem auf der Host-Seite, als Kachel auf dem Dashboard und als Prometheus-Metriken (`opnexus_host_gateway_*`,
  `opnexus_host_swap_used_percent`, `opnexus_host_service_running`). Die Messung speichert dafür zusätzlich Swap, CPU-Threads, Gateways
  und Dienste in der vorhandenen Messreihe (keine neue Tabelle).
- **Weitere Benachrichtigungskanäle:** ntfy, Gotify, Telegram, Microsoft Teams (Workflows-/Incoming-Webhook) und Matrix, beliebig viele
  je Typ, mit „Testen“-Knopf pro Kanal. Zugangsdaten (Tokens, die Teams-Adresse) liegen wie das SMTP-Passwort in der Geheimnis-Ablage, nie
  in der Datenbank oder in Antworten, und tauchen nicht in Fehlermeldungen oder Logs auf. Webhook und neue Kanäle prüfen die Zieladresse:
  nur http(s), keine Zugangsdaten in der Adresse, keine lokalen/Link-Local-/Metadaten-Adressen; Weiterleitungen werden nicht verfolgt.
  Betrifft die bestehende Webhook-Adresse: `localhost`, Loopback, 169.254.x.x und Zugangsdaten in der Adresse (`https://user:pass@…`)
  werden jetzt abgelehnt. Eine solche gespeicherte Adresse wird nicht mehr beliefert (nur ein Log-Eintrag), und die
  Benachrichtigungs-Einstellungen lassen sich erst wieder speichern, wenn sie korrigiert oder geleert ist (die Meldung nennt den Grund).
  Neue Einstellungen kommen mit Migration 0020.

- **Bestand übernehmen für Aliase und NAT:** Auf den Seiten Aliase und NAT (Port Forward, 1:1, Outbound) gibt es jetzt **Von Firewall übernehmen**
  wie bei den Regeln. Der Dialog zeigt, was von Hand in OPNsense angelegt wurde, einzeln zum Übernehmen oder mit „Alle übernehmbaren“ in einem Schritt;
  verschachtelte Aliase werden in der richtigen Reihenfolge übernommen. Die Übernahme schreibt nichts auf die Firewall: Die Vorschau zeigt direkt
  danach „keine Änderung“, der nächste Rollout aktualisiert dasselbe Objekt (kein Duplikat). Nicht übernommen werden OPNsense-eigene Aliase,
  Typen, die OPNexus nicht kennt (z. B. URL, GeoIP), Aliase mit Verweis auf nicht übernommene Aliase sowie NAT-Regeln, die das Modell verfälschen würden
  (z. B. „Kein Redirect“, verknüpfte Filterregel, mehrere Schnittstellen, „Kein NAT“); die Oberfläche nennt den Grund je Objekt.
  Outbound-NAT lässt sich nur übernehmen, wenn der Modus der Firewall Hybrid oder Manuell ist. Einstellungen, die OPNexus nicht verwaltet (Protokollierung,
  Pool-Optionen, Tags, Kategorien), bleiben unverändert und werden angezeigt.

- **Images für ARM:** Alle Release-Images (backend, frontend, proxy, updater) gibt es jetzt für `linux/amd64` und `linux/arm64`
  (Raspberry Pi 4/5, ARM-Server, Apple-Silicon-Docker).
2026-10-21 01:57:30 +02:00
Julian Boldt 9ba6357193 Release v1.23.11
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Updates per Klick prüfen jetzt ebenfalls die Signatur:** Der Updater-Container enthält `cosign` (feste Version, Prüfsumme beim Bau
  geprüft) und verifiziert die neuen Images vor dem Wechsel, wenn `cosign.pub` im Projektverzeichnis liegt. Das Image wird dadurch größer
  (rund 266 statt 123 MB).
2026-10-20 15:46:34 +02:00
Julian Boldt 23d3a00ae1 Release v1.23.10
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Signierte Release-Images:** Die Images (backend, frontend, proxy, updater) werden mit cosign signiert, beginnend mit 1.23.9. Der öffentliche
  Schlüssel liegt als `cosign.pub` im Repository und im Release-Bundle. `opnexus update` prüft die Signaturen vor dem Wechsel, wenn `cosign`
  installiert ist. Anleitung: Handbuch, „Images prüfen“.
2026-10-20 15:38:35 +02:00
Julian Boldt 4379a17b15 Release v1.23.9
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- Update-Karte: Der Hinweis für Installationen ohne Release-Images nennt jetzt die tatsächliche Ursache („OPNEXUS_IMAGE_REPO ist nicht gesetzt“)
  statt „baut aus dem Quellcode“.
- `cosign.pub`: öffentlicher Schlüssel für die Signaturen der Release-Images (das Release-Skript kann Images jetzt signieren).
2026-10-20 15:16:51 +02:00
Julian Boldt 983e05a816 Release v1.23.8
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Einstellungen › OPNexus aktualisieren:** Auf einer Installation, die aus dem Quellcode gebaut wird (kein `OPNEXUS_IMAGE_REPO`), erscheint nicht mehr
  die Anleitung zum Aktivieren des Updaters – Updates per Klick gibt es dort ohnehin nicht –, sondern nur der Hinweis auf `git pull` und
  `docker compose up -d --build`. Das Backend erkennt den Modus jetzt selbst (neue Umgebungsvariable im Compose-File).
2026-10-20 14:52:10 +02:00
Julian Boldt dd6a64d953 Release v1.23.7
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- Hinweistext der Versionsprüfung verweist auf den Schalter „oben“ statt „unten“.
2026-10-20 14:43:39 +02:00
Julian Boldt 6964ec6537 Release v1.23.6
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Versionsprüfung im Webinterface:** Unter Einstellungen › OPNexus aktualisieren lässt sich die Prüfung auf neue Versionen jetzt per Schalter
  und Feed-Adresse einstellen – kein Eingriff in die `.env` mehr nötig (Migration `0019`). Standardmäßig bleibt sie aus. Ein gesetztes
  `UPDATE_CHECK_URL` in der `.env` hat weiter Vorrang und sperrt den Schalter (mit Hinweis).
2026-10-20 14:38:19 +02:00
Julian Boldt 5e5d51b57f Release v1.23.5
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fix (Ursache des Vorfalls vom 20.10. in 1.23.4):** OPNsense merkt ein CARP-VIP zum Löschen vor, wenn eine Änderung kein `interface` mitschickt,
  und der anschließende „Anwenden“-Schritt entfernt es dann live, ohne es wieder anzulegen. OPNexus schickt bei jeder VIP-Änderung jetzt die
  bisherige Schnittstelle mit (wie schon die Adresse). Auf dem Test-Cluster mit echten Firewalls nachgestellt und bestätigt.
- **Einstellungen › OPNexus aktualisieren:** Die Anleitung zum Aktivieren nennt jetzt alles Nötige in einem Block (auch `UPDATE_CHECK_URL`) und weist
  darauf hin, dass Updates per Klick Release-Images brauchen; wer aus dem Quellcode baut, aktualisiert mit `git pull` und `docker compose up -d --build`.
2026-10-20 14:22:12 +02:00
Julian Boldt 1aba618ad5 Release v1.23.4
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fix (Produktiv-Vorfall 2026-10-20):** Das Ändern der Beschreibung eines CARP-VIP wendete das VIP neu an und ließ es auf beiden Firewalls
  im Zustand INIT hängen – OPNexus zeigte trotzdem „alles in Ordnung“. Nur die Beschreibung zu ändern, wendet jetzt nichts mehr an. Nach jeder
  anderen CARP-VIP-Änderung prüft OPNexus den Live-Zustand (MASTER/BACKUP), wendet bei Bedarf einmal erneut an und meldet sonst „Angewendet,
  Prüfung weicht ab“.
- **CARP-Überwachung je VHID:** Neue Warnungen `vip_down` (ein VIP ist auf einem Node nicht aktiv) und `role_mismatch` (ein Node ist nur für
  einen Teil der VIPs MASTER). Bisher zählte der Host schon als MASTER, sobald ein einziges VIP MASTER war.
2026-10-20 13:56:25 +02:00
Julian Boldt ba954c05bf Release v1.23.3
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **SSO:** Liefert der Anbieter `preferred_username`/`email` nicht im ID-Token (Authelia), fragt OPNexus sie beim Userinfo-Endpunkt ab; der neue
  Benutzer heißt dann `alice` statt wie die kryptische Benutzerkennung des Anbieters. Geprüft gegen Keycloak 26 und Authelia 4.39; S3-Offsite gegen
  Garage, MinIO, RustFS und Versity S3 Gateway.
2026-10-20 12:56:39 +02:00
Julian Boldt 63d014b14c Release v1.23.2
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **SSO-Fix (gefunden beim Test gegen ein echtes Keycloak):** Die Standard-Scopes enthalten nicht mehr `groups` – Keycloak und Entra ID
  lehnten die Anmeldung damit mit `invalid_scope` ab. Bestehende Einstellungen mit dem alten Standardwert werden umgestellt (Migration
  `0018`); Anbieter wie Okta tragen `groups` bei Bedarf selbst ein (Hinweis in den Einstellungen).
2026-10-20 09:17:31 +02:00
Julian Boldt 4838dad588 Release v1.23.1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Sicherheit (Ergebnis eines unabhängigen Reviews der Versionen 1.18–1.23):**
  - Aufgabentrennung: Mit „nur Admins rollen aus“ ist jetzt auch das Löschen verwalteter Objekte (entfernt sie von den Firewalls), Zertifikat-/ACME-Push, Dienste starten/stoppen und „Reihenfolge erneut durchsetzen“ den Admins vorbehalten, nicht nur `/deploy`.
  - Backup-Änderungsansicht: weitere Geheimnis-Felder (`otp_seed`, `pass`, `pwd`, …) werden ausgeblendet, die Platzhalter sind pro Abruf gesalzen (kein Wörterbuchabgleich, kein Vergleich über Abrufe/Firewalls).
  - SSO: der Anmeldeversuch ist per Cookie an den Browser gebunden (kein Login-CSRF, abgefangene Code/State-Paare sind nutzlos); SSO-Sitzungen dauern 12 Stunden statt 30 Tage; Gruppen-Strings werden nur an Komma/Zeilenumbruch getrennt (Gruppen mit Leerzeichen); Fehlertext des Anbieters wird für das Log bereinigt.
  - Das Ändern des eigenen Passworts widerruft jetzt auch die eigenen API-Tokens.
  - Management-Tunnel: der Hub routet nicht mehr zwischen Firewalls oder ins Compose-Netz (`FORWARD` verworfen, `ip_forward=0`); loopback-/link-local-Netze sind als Tunnel-Netz nicht erlaubt.
  - Sicherheitslage: eine defekte `config.xml` stoppt die Prüfung der übrigen Firewalls nicht mehr; die Auswertung läuft außerhalb der Ereignisschleife.
  - Behoben: S3-Offsite mit Port 443 im Endpunkt (Signatur passte nicht); Variablenwerte mit angehängtem Zeilenumbruch werden abgelehnt.
- Fertiges **Grafana-Dashboard** (`docs/grafana-opnexus.json`) für die Prometheus-Metriken von OPNexus.
2026-10-20 02:24:15 +02:00
Julian Boldt c4c8c4ff3a Release v1.23.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Sicherheitslage im Alltag**: Dashboard-Karte mit den Firewalls mit dem niedrigsten Wert; Benachrichtigung bei kritischen und hohen
  Befunden (je Firewall und Prüfung, mit „behoben“-Meldung; standardmäßig aus; Migration `0017`); druckbarer **Sicherheitsbericht**
  (Knopf „Bericht“, Drucken/PDF über den Browser).
2026-10-20 02:02:04 +02:00
Julian Boldt 295555cfd6 Release v1.22.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
  (Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
  Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
  Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
2026-10-20 01:42:30 +02:00
316 changed files with 44180 additions and 640 deletions

No files matched your search

+26 -1
View File
@@ -73,6 +73,8 @@ TRUST_PROXY_HEADERS=false
# User-Agent "OPNexus/<Version>". Admins sehen bei einer neueren Version einen
# Hinweis; aktualisiert wird nie automatisch, sondern mit `opnexus update` oder
# (optional, siehe unten) per Klick im Webinterface.
# Einfacher: im Webinterface unter Einstellungen › OPNexus aktualisieren einschalten.
# Ein Wert hier hat Vorrang und sperrt den Schalter dort.
# UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json
# UPDATE_CHECK_INTERVAL_HOURS=12
@@ -80,7 +82,7 @@ TRUST_PROXY_HEADERS=false
# Startet den Updater-Container, mit dem Admins unter Einstellungen › „OPNexus
# aktualisieren“ ein Update auslösen können. Er ist der einzige Container mit
# Docker-Socket-Zugriff (= root-gleiche Rechte auf diesem Host) und standardmäßig
# AUS. Benötigt außerdem UPDATE_CHECK_URL (oben). Siehe Handbuch, "Updates per Klick".
# AUS. Braucht außerdem die Versionsprüfung (Webinterface oder UPDATE_CHECK_URL, oben). Siehe Handbuch, "Updates per Klick".
# COMPOSE_PROFILES=updater
# ── Transportsicherheit (nur ändern, wenn nötig) ──────────────────────
@@ -98,3 +100,26 @@ TRUST_PROXY_HEADERS=false
# Zum Aktivieren (Handbuch „Firewalls hinter NAT“): COMPOSE_PROFILES=overlay
# und den UDP-Port aus dem Internet erreichbar machen.
#OVERLAY_PORT=51820
# ── GUI-Fernzugriff auf verwaltete Firewalls (optional, Standard AUS) ──
# Öffnet die Weboberfläche einer Firewall über OPNexus, auch hinter NAT
# (Management-Tunnel), jeweils in einem eigenen Browser-Ursprung (Slot-Port).
# Dreifach aus, bis alles hier gesetzt ist UND ein Admin den Schalter unter
# Einstellungen einschaltet. Voraussetzungen: HTTPS über den Proxy und
# COOKIE_SECURE=true. Siehe Handbuch, „Firewall-Oberfläche über OPNexus“.
# 1. Profil aktivieren (mit anderen Profilen kommagetrennt, z. B. overlay,gui):
# COMPOSE_PROFILES=gui
# 2. Slot-Ursprünge: dieselbe Adresse wie in OPNEXUS_SITES, je Slot ein Port
# aus GUI_PORTS. Darf NICHT in FRONTEND_ORIGIN stehen (Start wird sonst verweigert).
# OPNEXUS_GUI_SITES=https://10.0.0.5:9441, https://10.0.0.5:9442, https://10.0.0.5:9443, https://10.0.0.5:9444
# 3. Gemeinsames Geheimnis Backend <-> Gateway, mind. 32 Zeichen, z. B.
# `openssl rand -hex 32`:
# GUI_GATEWAY_TOKEN=
# 4. Wo die Slot-Ports lauschen. Standard nur lokal (127.0.0.1); für den
# Zugriff aus dem Netz die Adresse dieses Hosts eintragen (wie bei
# PROXY_BIND_ADDRESS). GUI_PORTS ist ein Bereich, Host- und Container-Port
# sind gleich. GUI_PORTS zum Einschalten SETZEN: ohne den Eintrag nimmt
# Docker zufällige freie Ports (damit ein Update nie an belegten Ports wie
# 9443 scheitert), und die Slot-Adressen oben wären nicht erreichbar.
# GUI_BIND_ADDRESS=127.0.0.1
# GUI_PORTS=9441-9444
+407 -1
View File
@@ -29,7 +29,413 @@ in der App auf der Login-Seite, im Footer jeder Seite, auf `/about` und in
den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin …
ansteht): Saker …
## [1.33.0] "Saker" — 2026-10-24
- **Aus dem Testlauf 1.31.0 behoben (T2 bis T6, B15):**
- **Benutzerliste:** `GET /api/users` zeigte jedem angemeldeten Konto (auch Nur-Lese, auch per Token) alle Benutzernamen, Rollen, letzte Anmeldung und 2FA-Status.
Jetzt sehen **nur Admins** die ganze Liste, alle anderen nur das eigene Konto.
- **Hosts:** Dieselbe Adresse zweimal einzutragen wird abgelehnt (`HOST_URL_EXISTS`, auch beim Ändern der Adresse). Die Hosts-Seite und der Einrichtungsassistent
prüfen die Verbindung beim Anlegen (Schlüssel abgelehnt / zu wenig Rechte / Zertifikat / keine Verbindung), speichern bei einem Fehler nichts und bieten
„Trotzdem anlegen“. Über die API ist das `verify_connection: true` (Standard aus). **Löschen verlangt jetzt auch am Server den Hostnamen:**
`DELETE /api/hosts/{id}?confirm_name=<Name>` (sonst 409); Skripte, die Hosts löschen, müssen das anpassen.
- **Zertifikate:** `cert_type: v3_ca` wird nicht mehr angenommen (es entstand ein Zertifikat mit CA-Erweiterung, keine nutzbare CA); die Oberfläche bot es nie an.
- **Traceroute:** Zwischenstationen ohne Antwort stehen als Zeile „keine Antwort“ in der Liste (vorher sprang die Zählung 1 → 3); ein Weg ganz ohne Antwort wird erklärt.
- **Netzwerk:** Die Vorschau prüft Zahlenbereiche vorab (VLAN-Tag 1–4094, PCP 0–7, VHID 1–255, Basis 1–254, Skew 0–254, gegen das Modell der OPNsense gelesen): `422` mit dem
Feldnamen statt `502` von der Firewall.
- **Gruppen:** Eine Gruppe, die ein VPN-Objekt als Ziel hat, lässt sich nicht mehr löschen (die Verknüpfung fiel still weg).
- Das Skript `scripts/verify_i18n.mjs` läuft jetzt auch gegen die HTTPS-Test-Instanz.
- **Regel-Bereinigung: OPNexus selbst ist automatisch geschützt (B10).** Beim Planen fragt die Bereinigung jede Firewall, von welcher Adresse sie OPNexus kommen sieht
(die eigene API-Verbindung in der Zustandstabelle), und schützt diese Adresse zusätzlich zu den Verwaltungsquellen. Vorher konnte eine Regel abgeschaltet werden, über
die nur OPNexus die Firewall erreicht, wenn in den Quellen nur Arbeitsplätze standen; dann scheiterte auch das automatische Zurücksetzen. Die Vorschau zeigt die Adresse
je Firewall (nur Admins) oder sagt, dass sie sich nicht ermitteln ließ (API-Recht „Diagnose: Zustände“, Management-Tunnel).
- **Keine verlorenen Regeltreffer mehr (B11):** Jedes Anwenden setzt die Regelzähler der Firewall zurück. OPNexus liest sie jetzt unmittelbar vor jedem Anwenden, das es
selbst auslöst (Regeln, NAT, Aliase, IPsec/OpenVPN/WireGuard, Netzwerk); vorher konnte eine Regel deshalb fälschlich als „ungenutzt“ gelten.
- **Einstellungen › Regel-Bereinigung:** schlägt die eigene Browser-Adresse als Verwaltungsquelle vor (B12). Geprüft: OPNsense 26.7 hat kein „savepoint“ zum Zurücknehmen
durch die Firewall selbst (Quellcode und live 404), das automatische Zurücksetzen bleibt bei OPNexus.
- **Installation unter Windows:** Neue Anleitung auf der Website (`/docs/windows/`, auch auf Englisch) und im Handbuch: Docker Desktop mit WSL 2, Zertifizierungsstelle importieren, Zugriff aus dem Netz (Windows-Firewall), Updates mit `opnexus` in WSL und Fehlerbehebung. Entwickelt und getestet wird weiterhin unter Linux; die Anleitung sagt das offen.
- **Sicherheit und Stabilität (aus einer externen Prüfung):**
- **Rückweg bleibt erhalten:** Wer nach „Konfiguration geschrieben, Anwenden fehlgeschlagen“ erneut ausrollte, überschrieb den ursprünglichen Rückweg; die automatische
Rücknahme stellte dann den unbestätigten Stand wieder her. Jetzt bleibt der Rückweg der Stand vor dem ersten Versuch. Wartung, Regel-Bereinigung und
„Konfiguration als Code“ behandeln eine solche Zeile wie eine offene Bestätigung. Gilt für Regeln, Aliase, Port-Forwards, 1:1-NAT, Outbound-NAT und Hierarchie-Regeln.
- **Anmeldung und 2FA:** Eine Passwort-Anmeldung, die genau während der Aktivierung von 2FA lief, bekam eine Sitzung ohne zweiten Faktor. Jetzt wird sie mit `TOTP_REQUIRED` abgewiesen.
- **Letzter Administrator:** Zwei Administratoren, die sich gleichzeitig herabstuften oder gegenseitig löschten, konnten alle Administratoren entfernen. Beides ist jetzt serialisiert.
- **Operatoren:** Mit ausgeschaltetem „Operatoren dürfen ausrollen“ konnten sie noch Snapshots (Boot-Umgebungen) anlegen. Ein Test hält künftig jede schreibende Route
fest, die keiner Rechte-Regel zugeordnet ist.
- **Zertifikate und ACME:** Der Ziel-Host ist nach dem ersten Push fest (`409 ZIEL_HOST_NACH_AUSROLLEN_FEST`); vorher meldete ein Host-Wechsel Erfolg für einen Host, der nie
etwas erhielt. Ein ACME-Konto mit Anfragen lässt sich nicht auf einen anderen Host verschieben (`ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET`).
- **VPN-Ausrollen:** IPsec, OpenVPN und WireGuard sichern jede von der Firewall vergebene Kennung sofort; bei einem Abbruch entstehen beim nächsten Versuch keine Duplikate.
Pro VPN läuft nur ein Ausrollen oder Entfernen zugleich (`409 VPN_DEPLOY_LAEUFT_BEREITS`).
- **Konfiguration als Code:** Gültiges YAML mit falsch geformtem Abschnitt (z. B. `aliases: 42`) ergibt `422 CONFIG_INVALID` statt eines Serverfehlers.
- **`.env`:** Die Anleitungen legen sie mit `install -m 600` an; `opnexus update` und `opnexus status` setzen eine zu offene Datei auf 0600. `opnexus rollback` liest seinen
Update-Zustand nur noch als Text und führt ihn nie aus.
- **Abhängigkeiten:** Next.js 15.5.28, PyJWT 2.15.1.
## [1.32.1] "Merlin" — 2026-10-23
- **Korrektur: Rollouts erkannten die Zielversion bei anderen Release-Typen falsch.** OPNexus las die neue Version nur am Kernpaket `opnsense`. Bei der
Business-Edition (`opnsense-business`) und beim Wechsel des Release-Typs (z. B. auf `opnsense-devel`, dann als neues Paket angeboten) fiel es auf
`product_latest` zurück; das Gate hielt den Rollout dann fälschlich mit „Version passt nicht“ an. Live gefunden und belegt (Rollout mit Cluster in einer Welle).
- **Plan-Hinweis zu Alias-Namen ehrlich formuliert:** „kein zentraler Alias in OPNexus“ statt „kein Alias dieses Namens“ (von Hand auf der Firewall
angelegte Aliase sind in Ordnung; der Planer fragt bewusst keine Firewall).
## [1.32.0] "Merlin" — 2026-10-23
- **Konfiguration als Code:** Unter **System › Konfiguration als Code** (und per `opnexus export` / `opnexus apply`) lassen sich Gerätegruppen, Hierarchiegruppen,
Aliase, Filterregeln, Port-Forwards, 1:1-NAT, Outbound-NAT, Hierarchie-Regeln und Standort-Variablen als **eine YAML-Datei exportieren** (sortiert, ohne Zeitstempel,
ohne Zugänge oder Schlüssel, für Git und Reviews) und wieder **importieren**: erst ein **Plan** (neu, geändert, entfernt, unverändert, blockiert, mit Hinweisen), dann
das **Übernehmen** durch einen Administrator in einer Transaktion mit dem Token genau dieses Plans. **Der Import ändert nur die Entwürfe in OPNexus, nie eine Firewall**;
ausgerollt wird wie immer über Vorschau → Ausrollen → Bestätigen. Ausgerollte oder noch verwendete Objekte entfernt er nie (Prune blockiert den Plan). Die Datei wird
nur als Daten gelesen (keine Anker, Tags, Merge-Schlüssel, Mehrfach-Dokumente; 1 MiB, strenges Schema mit Zeile und Pfad bei Fehlern). Plan und Export gehen auch mit „Lesen“-Token
und Nur-Lese-Konten, `opnexus apply --dry-run` liefert Exit-Codes für CI (0 / 2 / 3 / 4). Regeln und NAT-Objekte tragen dafür einen stabilen `key` (Migration, wird für
Bestand automatisch vergeben). Gruppenänderungen stehen jetzt im Audit.
- **Rückweg zur vorherigen Firmware:** Vor jedem Firmware-Update und Major-Upgrade (manuell, geplant, Rollout, CARP-Übergabe) legt OPNexus auf der Firewall ein
**ZFS-Boot-Environment** `opnexus-pre-<Version>-<Zeit>` als Rückweg-Punkt an (ohne den Punkt startet das Update nicht; ohne ZFS mit Hinweis). Neue Aktion **Zur vorherigen Firmware**
(Host-Wartung): Vorschau, Hostname, Downtime-Bestätigung, vorher lokale Sicherung der Konfiguration, dann Aktivieren, Prüfung und Neustart (scheitert ein Schritt, wird das
vorherige Boot-Environment wieder eingeschaltet). **Der Rückweg setzt auch die Konfiguration zurück** (live belegt): Ausrollvorgänge seit dem Punkt werden als unklar markiert,
damit das nächste Ausrollen wirklich schreibt. Ein Rollout macht nie von selbst einen Rückweg.
- **Rollout-Gate: Erreichbarkeitsproben.** Bis zu zehn Ping-/TCP-Proben je Plan, von der Firewall selbst ausgeführt, vor der Welle und nach der Beobachtungszeit:
vorher ok und jetzt nicht mehr hält den Rollout an, schon vorher gestört ist nur ein Hinweis. (Erreichbarkeit von der Firewall aus, nicht der Verkehr durch sie hindurch.)
- Migrationen 0030 (`config_key`) und 0031 (`gate_probes`).
## [1.31.1] "Vulture" — 2026-10-22
- **Korrektur: `opnexus password` und `reset_2fa` stürzten ab.** Seit den Standort-Variablen (1.22) brach das Zurücksetzen des Admin-Passworts
(`./opnexus password`) bzw. der Zwei-Faktor-Anmeldung (`docker compose exec backend python -m app.reset_2fa <benutzer>`) mit einem
SQLAlchemy-Fehler ab, weil ein Datenmodell nicht registriert war. Der Weg bei vergessenem Passwort oder verlorenem 2FA-Gerät funktioniert wieder;
ein Regressionstest sichert ab, dass jedes Hilfsskript alle Modelle kennt.
## [1.31.0] "Vulture" — 2026-10-22
- **Regel-Bereinigung: Entfernen, Bestätigen, Löschen, Wiederherstellen:** Im Tab **Bereinigung** (Regelwerk › Regel-Analyse) lassen sich Kandidaten jetzt auswählen
(Checkbox, höchstens so viele Regeln je Lauf wie in den Einstellungen; „vermutlich“ nur mit einer ausdrücklichen Einzelbestätigung je Regel, „unbekannt“ nie).
**Vorschau** zeigt je Firewall (im Cluster zuerst der Backup) die Regeln mit Befund, Sicherheit und Schutzprüfung; **Ausrollen** deaktiviert die Regeln und wendet an.
Danach läuft ein Countdown: **Bestätigen** heißt „ich erreiche die Firewall noch“, ohne Bestätigung setzt OPNexus **automatisch zurück**; bis dahin geht auch
**Jetzt zurücksetzen**. Bestätigte Regeln lassen sich nach der Wartezeit (einstellbar, Standard 7 Tage) **löschen**; vorher sichert OPNexus den Regelinhalt, und
unter **Verlauf der Bereinigungen** kommt die Regel mit **Wiederherstellen** deaktiviert und möglichst an alter Position zurück (sonst mit Hinweis am Ende der Gruppe);
**Wieder aktivieren** hat wieder ein Bestätigungsfenster. Wichtig: **Ausrollen, Wiederholen und Wiederherstellen darf nur ein Administrator**, eine Vorschau gehört
dem, der sie erstellt hat (ein Admin öffnet selbst eine neue); **Bestätigen geht nur in der Browser-Sitzung**, nie per API-Token. **Die Bereinigung bleibt gesperrt,
bis unter Einstellungen die Verwaltungsquellen** (IP-Adressen oder Netze, von denen aus verwaltet wird) **eingetragen sind**: Regeln, die diesen Zugang treffen
könnten, deaktiviert OPNexus nie. Dort stehen auch die Obergrenze je Lauf, die Wartezeit bis zum Löschen und **weitere Verwaltungsports** (zusätzlich zu
API-Port, 443, 80, 22). Tragen Sie die Adresse ein, mit der OPNexus selbst bei den Firewalls ankommt (im Cluster für jeden Knoten). Quellen, Ports und
Firewall-Adressen sehen nur Administratoren. Der Hinweis „Entfernen folgt“ entfällt.
- **Update: neuere CLI aus dem Zielrelease:** `opnexus update` (auch per Klick im Webinterface) holt sich nach dem Pull die CLI aus dem Updater-Image des
Zielreleases und startet sich damit neu, damit Korrekturen am Update-Ablauf sofort greifen statt erst beim übernächsten Update. Das geschieht nur, wenn
`cosign` und `cosign.pub` vorhanden sind und die Signatur des Image-**Digests** in diesem Lauf geprüft wurde (ausgeführt wird genau dieser Digest);
sonst und bei jedem Fehler läuft das Update mit der vorhandenen CLI weiter. Abschalten: `OPNEXUS_NO_CLI_REFRESH=1`. Wirksam wird es ab dem Update
*von* dieser Version.
## [1.30.0] "Owl" — 2026-10-22
- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
mehr verdeckte Regeln als bisher, nie weniger belegte.
## [1.29.0] "Caracara" — 2026-10-22
- **Fehlende API-Rechte werden sichtbar:** Darf der API-Schlüssel einer Firewall den Gateway- oder Dienststatus nicht lesen (OPNsense antwortet
mit 403), fiel das bisher niemandem auf, und der standardmäßig aktive Gateway-Alarm konnte nie auslösen. Jetzt zeigen die Host-Seite
(Abschnitt „Gateways“) und die Dashboard-Karte „Fleet-/CARP-Gesundheit“ ein Warn-Badge „API-Rechte fehlen: Gateways, Dienste“; der Tooltip nennt die
OPNsense-Berechtigung, die beim Benutzer oder bei der Gruppe fehlt (System: Gateways, Status: Services, Lobby: Dashboard). Zusätzlich gibt es
den neuen Alarm **API-Rechte fehlen** (standardmäßig an, abschaltbar unter Einstellungen › Benachrichtigungen, gleiche Entprellung wie die
anderen Messungs-Alarme) und die Prometheus-Metrik `opnexus_host_api_rights_missing{host,read}`. Nur eine ausdrückliche Ablehnung zählt;
Zeitüberschreitungen oder Serverfehler bleiben „unbekannt“.
- **Gateway-Alarm pro Gateway abschaltbar:** In der Gateway-Liste der Host-Seite schaltet eine Glocke den Alarm für ein einzelnes Gateway aus
(z. B. ruhendes Zweit-WAN oder tote IPv6-Gateways); ein „Alarm aus“-Badge zeigt es an. Ein offener Alarm dieses Gateways löst sich dabei
auf. Nur Administratoren, die Änderung steht im Audit-Log. Das Dashboard zeigt bei mehr als acht Gateways „+ N weitere“.
- **Firewall-Oberfläche (GUI-Zugriff) härter:** Ein frei gewordener Platz wird für eine **andere** Firewall erst 5 Minuten nach dem Ende der letzten
Sitzung vergeben (Schutz, falls dort noch ein Tab offen ist, der die Cookies des Platzes teilt); dieselbe Firewall bekommt ihn sofort wieder.
Sind alle Plätze belegt oder noch in der Abkühlzeit, nennt die Meldung den Grund. Ändern sich Adresse, festgesetztes Zertifikat oder Tunnelweg
einer Firewall, während ihre Oberfläche offen ist, endet die Sitzung spätestens nach 20 Sekunden. Das Limit von 64 gleichzeitigen Sitzungen
verdrängt keine laufende Sitzung mehr: eine neue wird abgelehnt (mit Meldung), bis eine endet. **Hinweis:** Das GUI-Gateway-Image muss zur
Backend-Version passen (wie bisher beim Update beider Container).
- **Anmeldung (Cookies) härter:** Das alte Sitzungs-Cookie `opnexus-session` wird bei HTTPS **nicht mehr akzeptiert**. **Alle, die sich seit der
Umstellung auf 1.27 nicht neu angemeldet haben, müssen sich einmal neu anmelden**; Skripte mit rohem Sitzungs-Cookie brauchen den Namen
`__Host-opnexus-session` (API-Tokens sind nicht betroffen, ohne HTTPS bleibt `opnexus-session` der normale Name). Beim Abmelden wird ein altes
Cookie weiterhin gelöscht. Das Cookie, das die SSO-Anmeldung an den Browser bindet, heißt bei HTTPS jetzt `__Host-opnexus-oidc-state`; der
Rückweg vom Anbieter wird nur noch als Seitenaufruf (Navigation) von einer fremden Seite zugelassen, und ohne dieses Cookie scheitert er immer.
- **Webhook, Benachrichtigungskanäle und SSO-Anbieter: Schutz vor DNS-Rebinding:** Bisher prüfte OPNexus die Adresse, auf die ein Name zeigt, und
der HTTP-Client löste den Namen danach noch einmal auf; ein Server unter Fremdkontrolle konnte beim zweiten Mal eine interne Adresse liefern. Jetzt
verbindet OPNexus genau zu der geprüften Adresse (Host-Name, TLS-Zertifikatsprüfung und Anfrage bleiben beim Namen); zeigt eine Antwort auch nur
teilweise auf Loopback, Link-Local oder einen Cloud-Metadaten-Dienst (auch AWS-IPv6 und Alibaba), kommt keine Verbindung zustande und die Zustellung meldet „Zieladresse nicht
erlaubt“. Private Netze bleiben erlaubt (eigener ntfy, Gotify, Matrix). Proxy-Umgebungsvariablen werden für diese Aufrufe nicht mehr beachtet.
- **Benachrichtigungen parallel zugestellt:** Webhook, E-Mail und alle Kanäle werden gleichzeitig bedient, jeder mit eigener Zeitgrenze (20 Sekunden),
und ein Prüfdurchlauf sendet insgesamt höchstens 90 Sekunden. Ein hängender Kanal verzögert weder die anderen noch die nächsten Prüfungen; im Log
steht dann „timed out“ (ohne Adresse oder Token). Wie bisher wird einmal je Ereignis zugestellt, ohne Wiederholung, **außer** der Durchlauf war so voll, dass
für ein Ereignis noch gar kein Versuch stattfand: es wird im nächsten Durchlauf zugestellt. Die Zustellung beginnt jetzt nach allen Prüfungen
eines Durchlaufs statt mitten darin.
- **Rollouts: Fingerabdruck, verantwortliche Person, überlappende Termine:** Der Fingerabdruck eines aktivierten Plans umfasst jetzt auch die
ausgelassenen und verschobenen Termine seines Fensters: **wer einen Termin auslässt oder verschiebt, muss einen automatischen Plan danach neu
aktivieren** (ohne solche Ausnahmen bleiben bestehende Aktivierungen gültig; **Pläne, deren Fenster schon Ausnahmen haben, nach dem Update
einmal neu aktivieren**, sonst meldet der Plan „Plan geändert“ und startet nicht automatisch). Die verantwortliche Person (Aktivierende, Startende, Freigebende)
wird zusätzlich als Konto-ID gespeichert (Migration 0026, bestehende Einträge werden über den Namen zugeordnet): ein gelöschtes und unter gleichem
Namen neu angelegtes Konto gilt nicht mehr als dieselbe Person, der Lauf pausiert bzw. der automatische Start unterbleibt mit „Person darf nicht mehr
ausrollen“. Angezeigt wird weiter der Benutzername. Überlappen sich zwei Termine eines Wartungsfensters, zählt der zuletzt begonnene (bisher der
früheste), sodass ein älterer, längst abgeschnittener Termin keinen neuen Start mehr blockiert.
- **Kleinigkeiten:** Ressourcen-Alarme zeigen bei fehlendem Messwert keine Zahl mehr („None %“ entfällt). Der Dialog „Von Firewall übernehmen“ zeigt ohne aktive
Firewall einen Hinweis statt dauerhaft „Lädt…“. Fehlermeldungen beim Übernehmen und Verknüpfen nennen die Gründe in Klartext (Deutsch/Englisch) statt
roher Codes wie `name_exists`; dasselbe gilt für „GUI-Zugriff nicht eingerichtet“ und blockierte Netzwerkänderungen. Der Test-Knopf der
Benachrichtigungskanäle ist für Nicht-Admins ausgeblendet (der Server verweigert den Test ohnehin). Der Tooltip der aktiven Plätze nennt keine
Umgebungsvariable mehr. Eine Gateway-Sitzung, die der Gateway-Container nicht mehr bestätigt (Absturz), zählt für die Abkühlzeit der Plätze als
beendet.
- **Grafana-Dashboard erweitert:** neue Felder für Gateways (erreichbar, Paketverlust, Latenz), Swap, gestoppte Dienste, fehlende API-Rechte und
Rollouts nach Status; dazu die Metrik `opnexus_rollout_runs{status}`.
## [1.28.0] "Sparrowhawk" — 2026-10-21
- **Übernehmen: mit vorhandenem zentralem Objekt verknüpfen:** Dieselbe von Hand angelegte Regel, derselbe Alias oder dieselbe NAT-Regel auf mehreren
Firewalls (z. B. beide HA-Knoten) wird nicht mehr als zweites Objekt übernommen. Im Dialog „Von Firewall übernehmen“ erscheint bei exakter
Übereinstimmung (nach Einsetzen der Host-Variablen, bei Aliasen inkl. Verweisen) die Aktion **Verknüpfen**: die Firewall wird weiteres Ziel des
vorhandenen Objekts, die Vorschau zeigt „keine Änderung“, und der nächste Rollout aktualisiert alle Firewalls an einem Objekt. Auf der Firewall wird
dabei nichts geändert. Unterscheidet sich ein gleichnamiges Objekt, zeigt der Dialog die abweichenden Felder; verknüpft wird dann nie. „Alle
übernehmbaren“ fragt vorher nach („3 verknüpfen, 2 neu übernehmen, 1 nicht übernehmbar“). Aliase, die bisher wegen „Name existiert“ abgelehnt wurden,
lassen sich so übernehmen.
Wartet das vorhandene Objekt gerade auf eine Bestätigung, wird nicht verknüpft; „Alle übernehmbaren“ lässt den Eintrag dann aus. Verknüpfen
macht ein gerade geöffnetes Bearbeiten-Formular des Objekts ungültig, damit die neue Firewall nicht versehentlich wieder aus den Zielen fällt.
- **Tooltips überarbeitet:** Der deaktivierte Button „Oberfläche öffnen“ nennt jetzt den Grund im Klartext samt nächstem Schritt (statt nur einer
Variablen wie `GUI_GATEWAY_TOKEN`); dasselbe gilt für ausgeschaltete Funktion, deaktivierte Firewall, Nicht-HTTPS-Adresse und ausgeschaltete
Zertifikatsprüfung. Texte durchgehend in Du-Form, kürzere VPN-Einleitungen, verständlichere Hinweise bei gesperrten Buttons (Zertifikate, Benutzer,
Dienste). Icon-Buttons (Bearbeiten, Löschen, Schließen, Reihenfolge) zeigen beim Darüberfahren ihren Namen.
## [1.27.2] "Buzzard" — 2026-10-21
- **`opnexus update` prüft den Proxy vor dem Wechsel:** Nach dem Ziehen der Images validiert das Werkzeug die Caddy-Konfiguration des neuen
Proxy-Images mit den Umgebungsvariablen *dieser* Installation. Ist sie ungültig, bricht das Update ab, bevor etwas gewechselt wurde.
Der Fall aus 1.27.0 (Compose-Datei mit leerem `OPNEXUS_GUI_SITES`) wird dabei automatisch repariert: Das Werkzeug trägt den Platzhalter
`OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` ein und sagt es. Installationen, die noch die Compose-Datei aus 1.27.0 haben,
müssen also nur einmal `opnexus update` ausführen.
- **`opnexus update` erkennt einen abstürzenden Proxy:** Ein Dienst ohne Healthcheck (der Proxy) gilt jetzt erst nach 10 Sekunden
durchgehender Laufzeit als gesund. Bisher meldete das Update „Fertig“, obwohl der Proxy in einer Neustart-Schleife hing, und das
automatische Zurückrollen griff nicht.
## [1.27.1] "Buzzard" — 2026-10-21
- **Fix (1.27.1): Proxy startete mit der neuen Compose-Datei nicht.** Mit der `docker-compose.yml` aus 1.27.0 gab Compose die Variable
`OPNEXUS_GUI_SITES` als leeren Text an den Proxy weiter; Caddy nutzt seinen Standardwert aber nur bei *nicht gesetzter* Variable und
brach mit „server block without any key“ ab, OPNexus war dann nicht mehr erreichbar. Die Compose-Datei setzt jetzt einen gültigen
Platzhalter als Standard. Betroffen waren nur Installationen, die die neue Compose-Datei aus 1.27.0 übernommen hatten (ohne
`OPNEXUS_GUI_SITES` in der `.env`); wer nur die Images aktualisiert hatte, war nicht betroffen. Sofort-Abhilfe ohne Update:
`OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` eintragen und `docker compose up -d proxy` ausführen.
## [1.27.0] "Buzzard" — 2026-10-21
Vorgesehen als MINOR-Release **1.27.0** „Firewall-Oberfläche über OPNexus“.
- **Firewall-Oberfläche über OPNexus:** Auf der Host-Seite öffnet **Oberfläche öffnen** die Weboberfläche der Firewall
in einem neuen Tab, auch bei Firewalls hinter NAT im Management-Tunnel. Nur für Admins; jede Firewall läuft in einer
eigenen Browser-Origin (Platz = Port), Cookies bleiben auf dem Server, das Ticket gilt einmalig 30 Sekunden,
Inaktivitäts- und Gesamtlimit sind einstellbar, Ticket, Start und Ende stehen im Audit (Art „GUI-Zugriff“).
Standardmäßig **aus**: braucht das Compose-Profil `gui`, `OPNEXUS_GUI_SITES`, `GUI_PORTS`, `GUI_GATEWAY_TOKEN`,
`COOKIE_SECURE=true` und den Schalter unter Einstellungen (Handbuch, „Firewall-Oberfläche über OPNexus“).
Live getestet auf OPNsense 26.7.
- **Einschränkungen:** kein WebSocket, keine automatische Anmeldung, höchstens 4 Plätze gleichzeitig. OPNexus-Adresse
vorher in die Whitelist des Anmeldeschutzes (sshguard) der Firewall eintragen, sonst können Fehlanmeldungen die
API-Zugriffe von OPNexus aussperren. Nach einem Neustart des wg-hub braucht der Tunnel 2 bis 4 Minuten.
- **Hinweis für Betreiber mit eigenen Skripten:** Bei HTTPS heißt das Sitzungs-Cookie jetzt `__Host-opnexus-session`
(bestehende Sitzungen werden ohne Abmelden übernommen), und Cookie-Anfragen an `/api` mit `Sec-Fetch-Site: same-site/cross-site`
werden abgelehnt. Zugriffe mit API-Token sind nicht betroffen.
## [1.26.0] "Kite" — 2026-10-21
Geplant als MINOR-Release **1.26.0** (Regel-Analyse). Backend, MCP und Oberfläche.
- **Regel-Analyse „Welche Regel greift?“ (API):** `GET /api/hosts/{id}/rule-analysis/match` und
`GET /api/rule-analysis/match` (ganze Flotte, Auswahl oder Gerätegruppe) beantworten für Quelle, Ziel, Protokoll
und Port, ob eine Firewall das erste Paket einer neuen Verbindung durchlässt oder blockiert, welche Regel das
entscheidet und warum (Prüfprotokoll Regel für Regel). Grundlage ist der tatsächlich geladene pf-Regelsatz
inklusive automatischer Regeln, Floating- und Gruppenregeln, `quick`/letzter Treffer, Negation, verschachtelter
Aliase (pf-Tabellentest der Firewall), Port-Aliase und Adressfamilien. Jede Antwort trägt eine ehrliche
Konfidenz **sicher / vermutlich / unbekannt** mit Begründungscodes; was nicht belegbar ist (z. B. `tagged`,
nicht auflösbare Aliase, geratenes Interface, NAT in der Nähe), wird nie als „sicher“ gemeldet. Nur lesend,
auch mit `read`-Token nutzbar; es wird nichts gespeichert und nichts an der Firewall geändert.
- **MCP:** neues Werkzeug `rule_match` für KI-Assistenten (gleiche Abfrage, ohne Prüfprotokoll).
- **Seite „Regel-Analyse“ (Regelwerk):** Formular für Hosts oder Gerätegruppe, Quell-/Ziel-IP, Protokoll, Ports,
Eingangs-Interface („automatisch“ oder Liste) und optional die Ausgangsseite. Die Abfrage steht in der Adresszeile
(teilbar, wird beim Öffnen sofort ausgeführt). Je Firewall eine Karte mit Erlaubt/Blockiert/Abgewiesen, Sicherheit
des Urteils, entscheidender Regel und Hinweisen; ein „unbekannt“-Ergebnis erscheint bewusst nicht als Urteil.
Aufklappbares „Warum?“ zeigt jede geprüfte Regel mit ✓/✗/? je Feld, überstimmte Regeln durchgestrichen. Auch mit
Nur-Lesen-Rolle nutzbar.
## [1.25.0] "Goshawk" — 2026-10-21
Geplant als MINOR-Release **1.25.0** (Flotten-Rollout).
- **Rollouts in Wellen (System › Rollouts):** Firmware-Updates für die ganze Flotte in Ringen, z. B. erst
`ring-canary`, dann `ring-pilot`, dann `ring-prod` (Wellen aus Gerätegruppen). Jeder Host läuft über die
bestehende geplante Wartung (Backup vor dem Update, CARP-Übergabe für den MASTER, Audit), immer nur ein Host
gleichzeitig und nur ein Rollout zur selben Zeit; im Cluster zuerst der BACKUP, dann der MASTER. Nach jeder Welle
folgen eine Beobachtungszeit und eine **Gesundheitsprüfung** (erreichbar, Zielversion installiert, CARP je VHID,
Gateways und Dienste wie vorher, keine offenen Sperren, optional Sicherheits-Score). Rot hält den Rollout an
(nach einer Nachprüfung zwei Minuten später) und meldet sich. Anhalten, Fortsetzen (prüft zuerst alles bisher
Aktualisierte erneut), Welle freigeben, Host überspringen und Abbrechen auf der Lauf-Seite; die Seite zeigt
Wellen, Hosts (Version vorher → nachher, aktueller Schritt) und die Prüfmatrix.
- **Sicherheitsnetz:** Weicht die angebotene Version in einem späteren Ring von der des ersten Rings ab, hält der
Rollout an (OPNsense installiert immer die neueste). Ein Host ohne Update wird übersprungen, ein fehlgeschlagener
nie von selbst wiederholt. Angehalten wird auch, wenn die verantwortliche Person das Recht zum Ausrollen verliert,
eine fremde Wartung im Cluster länger als 15 Minuten offen ist, der MASTER ohne Übergabe dran wäre oder ein Lauf
älter als 14 Tage ist. Höchstens 10 Wellen und 100 Hosts je Plan.
- **Wiederkehrende Wartungsfenster:** wöchentlich oder monatlich (n-ter Wochentag, letzter, fester Tag) in einer
Zeitzone, mit Sommerzeit-Regel, Abschneidezeit („neue Hosts nur in den ersten N Minuten“) und einzelnen Terminen
zum Auslassen oder Verschieben. Ein Plan mit Fenster startet neue Hosts nur darin; mit **automatischem Start**
(nur stabile Updates) beginnt er zu jedem Termin von selbst, aber nur, solange Plan, Gruppen und Fenster seit
der Aktivierung unverändert sind. Jede Änderung deaktiviert ihn.
- **Geplante Wartung prüft sich selbst:** Neue Option „Ergebnis automatisch prüfen und abschließen“ (im Dialog
vorausgewählt, bei Rollouts immer an). Nach dem Auftrag prüft OPNexus, ob die Firmware wieder bereit ist, der
erwartete Neustart stattfand und die Version lesbar ist, und schließt die Wartung dann selbst ab. Bisher blieb
sie auf „übergeben“ und sperrte Host und Cluster, bis jemand sie von Hand abschloss.
- **Benachrichtigungen:** „Rollout angehalten“ und „Rollout nicht automatisch gestartet“ (ein Schalter, standardmäßig an).
- **Rechte:** Rollout starten/aktivieren/fortsetzen/freigeben/überspringen/abbrechen und Wartungsfenster ändern
gelten als Ausrollen (Aufgabentrennung); **Anhalten** darf jeder mit Schreibrecht. Update Center: Hosts im
laufenden Rollout sind für manuelle Firmware-Aktionen gesperrt. Ein OPNexus-Update während eines Rollouts ist
sicher, wird aber angekündigt.
- **Getestet:** live auf den Test-Firewalls (OPNsense 26.7): automatischer Start im Fenster, Canary-Update
26.7.1_1 → 26.7.7, absichtlich gestoppter Dienst → Gate rot → angehalten (zweiter Ring unberührt,
Benachrichtigung), Fortsetzen, Pilot-Update 26.7.6 → 26.7.7 auf dem damaligen MASTER mit CARP-Übergabe und einem
Neustart des OPNexus-Backends mitten im Auftrag (genau ein Update-Auftrag), Freigabe, Host ohne Update
übersprungen, Anhalten/Fortsetzen/Abbrechen von Hand, Termin auslassen. **Nur gemockt:** Major-Upgrade als
Rollout, ein Cluster in *einer* Welle, Cluster-Prüfung vor dem zweiten Knoten, abweichende Zielversion,
Zeitüberschreitungen, Rechteverlust, Fenster schließt mitten im Lauf; die Sommerzeit-Regeln sind Rechentests.
## [1.24.0] "Merlin" — 2026-10-21
- **Alarme für Gateways, Ressourcen und Dienste:** Neue Ereignisse in Einstellungen › Benachrichtigungen.
**Gateway gestört** (standardmäßig **an**) meldet ein Gateway, das OPNsense als „down“ führt oder dessen Paketverlust (ab 20 %) bzw.
Latenz (ab 500 ms) über der Schwelle liegt; bewusst abgeschaltete und nicht überwachte Gateways bleiben still. **Ressourcen-Schwelle**
(standardmäßig aus) prüft Festplatte (ab 90 %), Arbeitsspeicher (ab 95 %), Swap (ab 50 %) und CPU-Last (ab 2 pro Thread), **Dienst ausgefallen**
(standardmäßig aus) eine einstellbare Liste (Unbound, Dnsmasq, DHCP, NTP, OpenVPN, WireGuard, IPsec …) und meldet nur Dienste, die zuvor
liefen. Alle drei stützen sich auf die Minutenmessung und lösen erst aus, wenn die letzten 3 (Ressourcen: 5) Messungen übereinstimmen, und
enden erst, wenn alle wieder in Ordnung sind: Einzelspitzen und flackernde Verbindungen schicken nichts. Der Gateway-Status (Verlust, Latenz,
Abweichung) steht außerdem auf der Host-Seite, als Kachel auf dem Dashboard und als Prometheus-Metriken (`opnexus_host_gateway_*`,
`opnexus_host_swap_used_percent`, `opnexus_host_service_running`). Die Messung speichert dafür zusätzlich Swap, CPU-Threads, Gateways
und Dienste in der vorhandenen Messreihe (keine neue Tabelle).
- **Weitere Benachrichtigungskanäle:** ntfy, Gotify, Telegram, Microsoft Teams (Workflows-/Incoming-Webhook) und Matrix, beliebig viele
je Typ, mit „Testen“-Knopf pro Kanal. Zugangsdaten (Tokens, die Teams-Adresse) liegen wie das SMTP-Passwort in der Geheimnis-Ablage, nie
in der Datenbank oder in Antworten, und tauchen nicht in Fehlermeldungen oder Logs auf. Webhook und neue Kanäle prüfen die Zieladresse:
nur http(s), keine Zugangsdaten in der Adresse, keine lokalen/Link-Local-/Metadaten-Adressen; Weiterleitungen werden nicht verfolgt.
Betrifft die bestehende Webhook-Adresse: `localhost`, Loopback, 169.254.x.x und Zugangsdaten in der Adresse (`https://user:pass@…`)
werden jetzt abgelehnt. Eine solche gespeicherte Adresse wird nicht mehr beliefert (nur ein Log-Eintrag), und die
Benachrichtigungs-Einstellungen lassen sich erst wieder speichern, wenn sie korrigiert oder geleert ist (die Meldung nennt den Grund).
Neue Einstellungen kommen mit Migration 0020.
- **Bestand übernehmen für Aliase und NAT:** Auf den Seiten Aliase und NAT (Port Forward, 1:1, Outbound) gibt es jetzt **Von Firewall übernehmen**
wie bei den Regeln. Der Dialog zeigt, was von Hand in OPNsense angelegt wurde, einzeln zum Übernehmen oder mit „Alle übernehmbaren“ in einem Schritt;
verschachtelte Aliase werden in der richtigen Reihenfolge übernommen. Die Übernahme schreibt nichts auf die Firewall: Die Vorschau zeigt direkt
danach „keine Änderung“, der nächste Rollout aktualisiert dasselbe Objekt (kein Duplikat). Nicht übernommen werden OPNsense-eigene Aliase,
Typen, die OPNexus nicht kennt (z. B. URL, GeoIP), Aliase mit Verweis auf nicht übernommene Aliase sowie NAT-Regeln, die das Modell verfälschen würden
(z. B. „Kein Redirect“, verknüpfte Filterregel, mehrere Schnittstellen, „Kein NAT“); die Oberfläche nennt den Grund je Objekt.
Outbound-NAT lässt sich nur übernehmen, wenn der Modus der Firewall Hybrid oder Manuell ist. Einstellungen, die OPNexus nicht verwaltet (Protokollierung,
Pool-Optionen, Tags, Kategorien), bleiben unverändert und werden angezeigt.
- **Images für ARM:** Alle Release-Images (backend, frontend, proxy, updater) gibt es jetzt für `linux/amd64` und `linux/arm64`
(Raspberry Pi 4/5, ARM-Server, Apple-Silicon-Docker).
## [1.23.11] "Harrier" — 2026-10-20
- **Updates per Klick prüfen jetzt ebenfalls die Signatur:** Der Updater-Container enthält `cosign` (feste Version, Prüfsumme beim Bau
geprüft) und verifiziert die neuen Images vor dem Wechsel, wenn `cosign.pub` im Projektverzeichnis liegt. Das Image wird dadurch größer
(rund 266 statt 123 MB).
## [1.23.10] "Harrier" — 2026-10-20
- **Signierte Release-Images:** Die Images (backend, frontend, proxy, updater) werden mit cosign signiert, beginnend mit 1.23.9. Der öffentliche
Schlüssel liegt als `cosign.pub` im Repository und im Release-Bundle. `opnexus update` prüft die Signaturen vor dem Wechsel, wenn `cosign`
installiert ist. Anleitung: Handbuch, „Images prüfen“.
## [1.23.9] "Harrier" — 2026-10-20
- Update-Karte: Der Hinweis für Installationen ohne Release-Images nennt jetzt die tatsächliche Ursache („OPNEXUS_IMAGE_REPO ist nicht gesetzt“)
statt „baut aus dem Quellcode“.
- `cosign.pub`: öffentlicher Schlüssel für die Signaturen der Release-Images (das Release-Skript kann Images jetzt signieren).
## [1.23.8] "Harrier" — 2026-10-20
- **Einstellungen › OPNexus aktualisieren:** Auf einer Installation, die aus dem Quellcode gebaut wird (kein `OPNEXUS_IMAGE_REPO`), erscheint nicht mehr
die Anleitung zum Aktivieren des Updaters – Updates per Klick gibt es dort ohnehin nicht –, sondern nur der Hinweis auf `git pull` und
`docker compose up -d --build`. Das Backend erkennt den Modus jetzt selbst (neue Umgebungsvariable im Compose-File).
## [1.23.7] "Harrier" — 2026-10-20
- Hinweistext der Versionsprüfung verweist auf den Schalter „oben“ statt „unten“.
## [1.23.6] "Harrier" — 2026-10-20
- **Versionsprüfung im Webinterface:** Unter Einstellungen › OPNexus aktualisieren lässt sich die Prüfung auf neue Versionen jetzt per Schalter
und Feed-Adresse einstellen – kein Eingriff in die `.env` mehr nötig (Migration `0019`). Standardmäßig bleibt sie aus. Ein gesetztes
`UPDATE_CHECK_URL` in der `.env` hat weiter Vorrang und sperrt den Schalter (mit Hinweis).
## [1.23.5] "Harrier" — 2026-10-20
- **Fix (Ursache des Vorfalls vom 20.10. in 1.23.4):** OPNsense merkt ein CARP-VIP zum Löschen vor, wenn eine Änderung kein `interface` mitschickt,
und der anschließende „Anwenden“-Schritt entfernt es dann live, ohne es wieder anzulegen. OPNexus schickt bei jeder VIP-Änderung jetzt die
bisherige Schnittstelle mit (wie schon die Adresse). Auf dem Test-Cluster mit echten Firewalls nachgestellt und bestätigt.
- **Einstellungen › OPNexus aktualisieren:** Die Anleitung zum Aktivieren nennt jetzt alles Nötige in einem Block (auch `UPDATE_CHECK_URL`) und weist
darauf hin, dass Updates per Klick Release-Images brauchen; wer aus dem Quellcode baut, aktualisiert mit `git pull` und `docker compose up -d --build`.
## [1.23.4] "Harrier" — 2026-10-20
- **Fix (Produktiv-Vorfall 2026-10-20):** Das Ändern der Beschreibung eines CARP-VIP wendete das VIP neu an und ließ es auf beiden Firewalls
im Zustand INIT hängen – OPNexus zeigte trotzdem „alles in Ordnung“. Nur die Beschreibung zu ändern, wendet jetzt nichts mehr an. Nach jeder
anderen CARP-VIP-Änderung prüft OPNexus den Live-Zustand (MASTER/BACKUP), wendet bei Bedarf einmal erneut an und meldet sonst „Angewendet,
Prüfung weicht ab“.
- **CARP-Überwachung je VHID:** Neue Warnungen `vip_down` (ein VIP ist auf einem Node nicht aktiv) und `role_mismatch` (ein Node ist nur für
einen Teil der VIPs MASTER). Bisher zählte der Host schon als MASTER, sobald ein einziges VIP MASTER war.
## [1.23.3] "Harrier" — 2026-10-20
- **SSO:** Liefert der Anbieter `preferred_username`/`email` nicht im ID-Token (Authelia), fragt OPNexus sie beim Userinfo-Endpunkt ab; der neue
Benutzer heißt dann `alice` statt wie die kryptische Benutzerkennung des Anbieters. Geprüft gegen Keycloak 26 und Authelia 4.39; S3-Offsite gegen
Garage, MinIO, RustFS und Versity S3 Gateway.
## [1.23.2] "Harrier" — 2026-10-20
- **SSO-Fix (gefunden beim Test gegen ein echtes Keycloak):** Die Standard-Scopes enthalten nicht mehr `groups` – Keycloak und Entra ID
lehnten die Anmeldung damit mit `invalid_scope` ab. Bestehende Einstellungen mit dem alten Standardwert werden umgestellt (Migration
`0018`); Anbieter wie Okta tragen `groups` bei Bedarf selbst ein (Hinweis in den Einstellungen).
## [1.23.1] "Harrier" — 2026-10-20
- **Sicherheit (Ergebnis eines unabhängigen Reviews der Versionen 1.18–1.23):**
- Aufgabentrennung: Mit „nur Admins rollen aus“ ist jetzt auch das Löschen verwalteter Objekte (entfernt sie von den Firewalls), Zertifikat-/ACME-Push, Dienste starten/stoppen und „Reihenfolge erneut durchsetzen“ den Admins vorbehalten, nicht nur `/deploy`.
- Backup-Änderungsansicht: weitere Geheimnis-Felder (`otp_seed`, `pass`, `pwd`, …) werden ausgeblendet, die Platzhalter sind pro Abruf gesalzen (kein Wörterbuchabgleich, kein Vergleich über Abrufe/Firewalls).
- SSO: der Anmeldeversuch ist per Cookie an den Browser gebunden (kein Login-CSRF, abgefangene Code/State-Paare sind nutzlos); SSO-Sitzungen dauern 12 Stunden statt 30 Tage; Gruppen-Strings werden nur an Komma/Zeilenumbruch getrennt (Gruppen mit Leerzeichen); Fehlertext des Anbieters wird für das Log bereinigt.
- Das Ändern des eigenen Passworts widerruft jetzt auch die eigenen API-Tokens.
- Management-Tunnel: der Hub routet nicht mehr zwischen Firewalls oder ins Compose-Netz (`FORWARD` verworfen, `ip_forward=0`); loopback-/link-local-Netze sind als Tunnel-Netz nicht erlaubt.
- Sicherheitslage: eine defekte `config.xml` stoppt die Prüfung der übrigen Firewalls nicht mehr; die Auswertung läuft außerhalb der Ereignisschleife.
- Behoben: S3-Offsite mit Port 443 im Endpunkt (Signatur passte nicht); Variablenwerte mit angehängtem Zeilenumbruch werden abgelehnt.
- Fertiges **Grafana-Dashboard** (`docs/grafana-opnexus.json`) für die Prometheus-Metriken von OPNexus.
## [1.23.0] "Harrier" — 2026-10-20
- **Sicherheitslage im Alltag**: Dashboard-Karte mit den Firewalls mit dem niedrigsten Wert; Benachrichtigung bei kritischen und hohen
Befunden (je Firewall und Prüfung, mit „behoben“-Meldung; standardmäßig aus; Migration `0017`); druckbarer **Sicherheitsbericht**
(Knopf „Bericht“, Drucken/PDF über den Browser).
## [1.22.0] "Peregrine" — 2026-10-20
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
(Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
## [1.21.0] "Osprey" — 2026-10-20
+13 -7
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
> **Stand:** `1.21.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
> **Stand:** `1.33.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen
@@ -16,14 +16,18 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
- **Sichere Änderungen** — jede Änderung läuft *Prüfen → Ausrollen → Bestätigen*; bleibt die Bestätigung im Zeitfenster aus, rollt OPNexus zurück.
- **Hierarchische Policies** — Pre-/Post-Rulebases mit Vererbung über Gruppenbäume.
- **VPN** — IPsec-Site-to-Site, OpenVPN und WireGuard, einmal definiert und ausgerollt.
- **Netzwerk** — Schnittstellen, VLANs, virtuelle IPs, Gateways, Routen, LAGG/Bridge/GRE, DNS und DHCP je Firewall, dazu Tunables, Cron, Syslog, Snapshots und Diagnose (Ping, Traceroute, Port-Prüfung, DNS), ein Abgleich der Cluster-Knoten, mit Diff-Vorschau, Backup vor jeder Änderung und Schutz davor, die eigene Verbindung zu kappen; von Hand angelegte Regeln lassen sich übernehmen.
- **Netzwerk** — Schnittstellen, VLANs, virtuelle IPs, Gateways, Routen, LAGG/Bridge/GRE, DNS und DHCP je Firewall, dazu Tunables, Cron, Syslog, Snapshots und Diagnose (Ping, Traceroute, Port-Prüfung, DNS), ein Abgleich der Cluster-Knoten, mit Diff-Vorschau, Backup vor jeder Änderung und Schutz davor, die eigene Verbindung zu kappen; von Hand angelegte Regeln, Aliase und NAT-Regeln lassen sich übernehmen.
- **Zertifikate** — Trust-Store für alle Firewalls und Ausstellung per ACME / Let's Encrypt.
- **Alarme** — Gateway-/WAN-Ausfall (down, Paketverlust, Latenz), Schwellen für Platte, Speicher, Swap und CPU sowie ausgefallene Dienste; über mehrere Messungen entprellt, damit Einzelspitzen und flackernde Verbindungen schweigen. Zustellung per E-Mail, Webhook, ntfy, Gotify, Telegram, Microsoft Teams oder Matrix, jedes Ereignis einzeln schaltbar.
- **Firewall-Oberfläche über OPNexus** — die Weboberfläche einer Firewall in einem eigenen Browser-Tab über OPNexus öffnen, auch hinter NAT; nur für Admins, Einmal-Ticket, eigene Origin, Zeitlimits und Audit (standardmäßig aus).
- **Logs & Audit** — zentrale Log-Suche mit Korrelation, Dashboards und lückenloser Deploy-Verlauf.
- **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus.
- **Update-Center und Rollouts** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus; Rollouts in Wellen (Canary → Pilot → Prod) mit wiederkehrenden Wartungsfenstern, Gesundheitsprüfung zwischen den Wellen (auf Wunsch mit Erreichbarkeitsproben, die die Firewall selbst ausführt) und automatischem Halt bei jeder Auffälligkeit. Vor jedem Update legt OPNexus ein ZFS-Boot-Environment als Rückweg-Punkt an; „Zur vorherigen Firmware“ stellt es geführt wieder her.
- **Konfiguration als Code** — Regeln, Aliase, NAT, Gruppen, Policy-Hierarchie und Standort-Variablen als eine YAML-Datei exportieren (für Git und Review) und wieder importieren: erst ein Plan (neu / geändert / entfernt / blockiert), dann das Übernehmen in einer Transaktion, das nur die Entwürfe in OPNexus ändert (nie eine Firewall). `opnexus export` / `opnexus apply --dry-run` für Konsole und CI.
- **Einrichtungsassistent** — geführter Erststart nach der Installation: eigenes Passwort, Zwei-Faktor und die erste Firewall verbinden.
- **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
- **Automatisierung** — persönliche API-Tokens (Lesen / Ausrollen / Admin), Prometheus-Metriken, ein nur lesender MCP-Endpunkt für KI-Assistenten und ein OpenAPI-Schema.
- **Sicherheitslage** — ein Wert von 0 bis 100 je Firewall aus Härtungs- und Regel-Hygiene-Prüfungen (offene WAN-Regeln, schwaches SSH/SNMP, verdeckte Regeln, alte Backups …), nur lesend.
- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend. Eine Bereinigungsliste nennt ungenutzte, verdeckte, doppelte und lange deaktivierte Regeln (jeweils mit Sicherheit); ein Administrator kann ausgewählte Regeln abschalten (Vorschau, dann Ausrollen mit automatischer Rücknahme ohne Bestätigung), später löschen und aus einer Sicherung des Regelinhalts wiederherstellen. Jede Regel hat einen Änderungsverlauf – auch für Änderungen außerhalb von OPNexus.
- **Backups & Wiederherstellung** — geplante Konfigurationssicherungen mit Aufbewahrung, verschlüsselte Offsite-Kopie auf S3-kompatiblen Speicher, Änderungsansicht zwischen Sicherungen und eine vollständige verschlüsselte Sicherung von OPNexus selbst (`opnexus backup --full`).
- **Single Sign-on** — Anmeldung über deinen Identitätsanbieter (OIDC); die Rolle kommt bei jeder Anmeldung aus dessen Gruppen.
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Operator / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
@@ -44,7 +48,7 @@ Voraussetzung: Docker mit Compose-Plugin.
```bash
git clone https://git.opnexus.dev/opnexus/opnexus.git
cd opnexus
cp .env.example .env # POSTGRES_PASSWORD (und ADMIN_PASSWORD) setzen
install -m 600 .env.example .env # POSTGRES_PASSWORD (und ADMIN_PASSWORD) setzen; nur für dich lesbar
docker compose up -d --build
```
@@ -56,13 +60,15 @@ scripts/opnexus password
Nach der ersten Anmeldung öffnet sich von selbst ein kurzer **Einrichtungsassistent**: eigenes Passwort setzen, optional Zwei-Faktor-Anmeldung einschalten, die erste Firewall verbinden (samt Anleitung für den API-Key in OPNsense). Er lässt sich überspringen und jederzeit unter System › Einrichtung wieder öffnen.
**Windows:** läuft in Docker Desktop mit WSL 2, siehe die [Installationsanleitung für Windows](https://opnexus.dev/docs/windows/). Entwickelt und getestet wird unter Linux; für den Dauerbetrieb ist ein Linux-Server oder eine Linux-VM besser.
Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelieferter Caddy-Proxy) siehe das [Handbuch](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls).
**Fertige Images** (amd64) liegen in der Registry dieses Gitea. In der `.env`:
**Fertige Images** (amd64 und arm64) liegen in der Registry dieses Gitea. In der `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.21.0
OPNEXUS_VERSION=1.33.0
```
```bash
@@ -73,7 +79,7 @@ Aktualisiert wird danach mit `scripts/opnexus update` / `rollback` (erst ziehen,
## Datenschutz
OPNexus wird selbst betrieben und hat keine Telemetrie. Es verbindet sich nur mit deinen Firewalls und mit Benachrichtigungszielen, die du selbst einrichtest. Der optionale Hinweis auf neue Versionen (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) ist **aus**, solange du ihn nicht setzt; eingeschaltet geht höchstens alle 12 Stunden ein einfaches `GET` mit dem User-Agent `OPNexus/<Version>` hinaus — ohne Instanz- oder Nutzerdaten.
OPNexus wird selbst betrieben und hat keine Telemetrie. Es verbindet sich nur mit deinen Firewalls und mit Benachrichtigungszielen, die du selbst einrichtest. Der optionale Hinweis auf neue Versionen (Schalter unter Einstellungen › OPNexus aktualisieren, oder `UPDATE_CHECK_URL` in der `.env`) ist **aus**, solange du ihn nicht einschaltest; eingeschaltet geht höchstens alle 12 Stunden ein einfaches `GET` mit dem User-Agent `OPNexus/<Version>` hinaus — ohne Instanz- oder Nutzerdaten.
## Lizenz
+13 -7
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](docs/screenshots/readme-rules-en.webp)
> **Status:** `1.21.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
> **Status:** `1.33.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does
@@ -16,14 +16,18 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
- **Safe changes** — every change goes *preview → deploy → confirm*; without your confirmation inside the time window OPNexus rolls it back.
- **Hierarchical policies** — pre/post rulebases with inheritance across group trees.
- **VPN** — IPsec site-to-site, OpenVPN and WireGuard, defined once and rolled out.
- **Network** — interfaces, VLANs, virtual IPs, gateways, routes, LAGG/bridge/GRE, DNS and DHCP per firewall, plus tunables, cron, syslog, snapshots and diagnostics (ping, traceroute, port check, DNS), a comparison of cluster nodes, with a diff preview, a backup before every change and a guard against cutting your own connection; rules created by hand on a firewall can be adopted.
- **Network** — interfaces, VLANs, virtual IPs, gateways, routes, LAGG/bridge/GRE, DNS and DHCP per firewall, plus tunables, cron, syslog, snapshots and diagnostics (ping, traceroute, port check, DNS), a comparison of cluster nodes, with a diff preview, a backup before every change and a guard against cutting your own connection; rules, aliases and NAT rules created by hand on a firewall can be adopted.
- **Certificates** — trust store for all firewalls and ACME / Let's Encrypt issuance.
- **Alerts** — gateway/WAN outage (down, packet loss, latency), disk/memory/swap/CPU thresholds and stopped services, debounced over several measurements so spikes and flapping links stay quiet; delivered by email, webhook, ntfy, Gotify, Telegram, Microsoft Teams or Matrix, all switchable per event.
- **Firewall interface through OPNexus** — open a firewall's own web interface in a separate browser tab through OPNexus, even behind NAT; admins only, one-time ticket, isolated origin, time limits and audit (off by default).
- **Logs & audit** — central log search with correlation, dashboards and a complete deploy audit trail.
- **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode.
- **Update center & rollouts** — firmware updates for the fleet, aware of HA order and maintenance mode; rollouts in waves (canary → pilot → prod) with recurring maintenance windows, a health gate between the waves (including optional reachability probes run from the firewall) and an automatic stop on anything unusual. Before every update OPNexus creates a ZFS boot environment as a rollback point, and a guided "back to previous firmware" restores it.
- **Configuration as code** — export rules, aliases, NAT, groups, the policy hierarchy and site variables as one YAML file for Git and review, and import it again: a plan first (new / changed / removed / blocked), then a one-transaction apply that only changes OPNexus' drafts (never a firewall). `opnexus export` / `opnexus apply --dry-run` for the console and CI.
- **Setup assistant** — a guided first run after installation: your own password, two-factor, and connecting the first firewall.
- **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
- **Automation** — personal API tokens (read / deploy / admin), Prometheus metrics, a read-only MCP endpoint for AI assistants and an OpenAPI schema.
- **Security posture** — a 0–100 score per firewall from hardening and rule-hygiene checks (open WAN rules, weak SSH/SNMP, shadowed rules, stale backups …), read-only.
- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only. A cleanup list names unused, shadowed, duplicate and long-disabled rules (each with its confidence); an administrator can switch selected rules off (preview, then roll-out with an automatic revert unless confirmed), delete them later and restore them from a backup of their content. Every rule has a change history, including changes made outside OPNexus.
- **Backups & recovery** — scheduled config backups with retention, an encrypted offsite copy to S3-compatible storage, a redacted change view between backups, and a full encrypted backup/restore of OPNexus itself (`opnexus backup --full`).
- **Single sign-on** — sign in with your identity provider (OIDC); roles come from its groups at every login.
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / operator / read-only), keyboard operation, dark mode, German and English UI.
@@ -46,7 +50,7 @@ Requires Docker with the Compose plugin.
```bash
git clone https://git.opnexus.dev/opnexus/opnexus.git
cd opnexus
cp .env.example .env # set POSTGRES_PASSWORD (and ADMIN_PASSWORD)
install -m 600 .env.example .env # set POSTGRES_PASSWORD (and ADMIN_PASSWORD); readable by you only
docker compose up -d --build
```
@@ -58,13 +62,15 @@ scripts/opnexus password
After the first sign-in a short **setup assistant** opens on its own: set your own password, optionally enable two-factor sign-in, connect your first firewall (including how to create the API key in OPNsense). You can skip it and reopen it any time under System › Setup.
**Windows:** run it in Docker Desktop with WSL 2 – see the [installation guide for Windows](https://opnexus.dev/en/docs/windows/). OPNexus is developed and tested on Linux; for permanent operation a Linux server or VM is the better choice.
For access from other machines and for production use with TLS (bundled Caddy proxy), see the [handbook](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls) (German).
**Prebuilt images** (amd64) are published to this Gitea's registry. In `.env`:
**Prebuilt images** (amd64 and arm64) are published to this Gitea's registry. In `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.21.0
OPNEXUS_VERSION=1.33.0
```
```bash
@@ -77,7 +83,7 @@ docker compose pull && docker compose up -d --no-build
## Privacy
OPNexus is self-hosted and has no telemetry. It only connects to your firewalls and to notification targets you configure yourself. The optional new-version notice (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) is **off** unless you set it; when on, it makes one plain `GET` at most every 12 hours with the user agent `OPNexus/<version>` — no instance or user data.
OPNexus is self-hosted and has no telemetry. It only connects to your firewalls and to notification targets you configure yourself. The optional new-version notice (switch under Settings › Update OPNexus, or `UPDATE_CHECK_URL` in the `.env`) is **off** unless you turn it on; when on, it makes one plain `GET` at most every 12 hours with the user agent `OPNexus/<version>` — no instance or user data.
## License
+14 -7
View File
@@ -7,7 +7,7 @@ from starlette.responses import JSONResponse
from app.database import SessionLocal
from app.errors import ErrorCode
from app.routers.auth import COOKIE_NAME
from app.routers.auth import session_token
from app.routers import governance
from app.routers.auth import client_ip
from app.services import api_token_service, auth_service
@@ -43,6 +43,8 @@ _READ_ONLY_ALLOWED_WRITES = {
"/api/settings",
# Read-only JSON-RPC tools (see routers/mcp.py).
"/api/mcp",
# Configuration as code: the plan only compares a file with the database (routers/config_code.py).
"/api/config/import/plan",
# Self-scoped: a user managing their own second factor.
"/api/auth/2fa/setup",
"/api/auth/2fa/enable",
@@ -56,9 +58,9 @@ _READ_ONLY_ALLOWED_WRITES = {
_SECURITY_KINDS = {
"ipsec": "ipsec_tunnel", "openvpn": "openvpn_instance", "wireguard": "wireguard_tunnel",
"certificates": "certificate", "acme": "acme", "notifications": "notification", "auth": "auth",
"tokens": "api_token",
"tokens": "api_token", "rollouts": "rollout",
}
_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings"}
_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings", "plans", "runs"}
def _is_uuid(value: str) -> bool:
@@ -94,10 +96,14 @@ class AuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
if request.url.path in _PUBLIC_PATHS:
return await call_next(request)
if request.url.path.startswith("/internal/gui/"):
# GUI gateway only: never routed by the proxy, authenticated by the
# shared gateway token inside routers/gui_access.py.
return await call_next(request)
api_token = None
bearer = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
token = request.cookies.get(COOKIE_NAME, "")
token = session_token(request)
if bearer is not None:
# An API token wins over a cookie; a bad one never falls back to it.
async with SessionLocal() as db:
@@ -117,7 +123,7 @@ class AuthMiddleware(BaseHTTPMiddleware):
)
if user is not None:
parts = request.url.path.strip("/").split("/")
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system", "backups": "backup"}
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system", "backups": "backup", "maintenance-windows": "maintenance_window", "groups": "group", "config": "config"}
writing = request.method in ("POST", "PUT", "PATCH", "DELETE")
if len(parts) > 1 and parts[1] in _SECURITY_KINDS:
event = {**_security_event(parts, request.method), "details": {"method": request.method, "path": request.url.path}}
@@ -127,10 +133,11 @@ class AuthMiddleware(BaseHTTPMiddleware):
if len(parts) > 2 and parts[1] == "settings":
kind = "settings" # where OPNexus stores/sends things (own prefs at /api/settings stay unaudited)
action = ("delete" if request.method == "DELETE" else parts[-1] if len(parts) > 3 else "edit" if request.method in ("PUT", "PATCH") else "create")
event = {"kind": kind, "action": action, "entity_id": parts[2] if len(parts) > 2 else None,
entity = parts[2] if len(parts) > 2 and kind != "config" else None
event = {"kind": kind, "action": action, "entity_id": entity,
"host_id": parts[2] if kind == "host" and len(parts) > 2 else None,
"details": {"method": request.method, "path": request.url.path}}
tracked = kind and writing and not request.url.path.endswith(("/preview", "/carp-preview", "/revisions"))
tracked = kind and writing and not request.url.path.endswith(("/preview", "/carp-preview", "/revisions", "/import/plan"))
if (
user.role == "read_only"
+29
View File
@@ -6,6 +6,17 @@ class Settings(BaseSettings):
def allowed_origins(self) -> list[str]:
return [o.strip().rstrip("/") for o in self.frontend_origin.split(",") if o.strip()]
@property
def gui_site_origins(self) -> list[str]:
return [o.strip().rstrip("/") for o in self.opnexus_gui_sites.split(",") if o.strip()]
def gui_origin_conflicts(self) -> list[str]:
"""Slot origins of the GUI gateway that are also listed in FRONTEND_ORIGIN.
Such an origin would get CORS with credentials and pass the Sec-Fetch-Site
exception — a firewall GUI could then read OPNexus' API. Startup is refused."""
allowed = {o.lower() for o in self.allowed_origins}
return [o for o in self.gui_site_origins if o.lower() in allowed]
database_url: str = (
"postgresql+asyncpg://opnexus:opnexus@postgres:5432/opnexus"
)
@@ -27,6 +38,9 @@ class Settings(BaseSettings):
# keine Instanz- oder Nutzerdaten.
update_check_url: str = ""
update_check_interval_hours: int = 12
# Registry der Release-Images (OPNEXUS_IMAGE_REPO). Leer = OPNexus wird lokal aus dem
# Quellcode gebaut; dann gibt es keine Updates per Klick (nur der Hinweis in den Einstellungen).
opnexus_image_repo: str = ""
# ── Updates aus dem Webinterface (app/services/updater_client.py) ─────
# Optionaler Updater-Container (Compose-Profil "updater"). Das Backend hat
@@ -89,6 +103,14 @@ class Settings(BaseSettings):
# per-IP lockout entirely.
trust_proxy_headers: bool = False
# ── GUI gateway (1.27, optional container "gui-gateway") ─────────
# Slot origins the gateway is served on, comma-separated, e.g.
# "https://192.0.2.10:9441,https://192.0.2.10:9442". Empty = feature unavailable.
opnexus_gui_sites: str = ""
# Shared secret between backend and gateway for /internal/gui/* (at least
# 32 characters). Empty = the internal endpoints answer 404.
gui_gateway_token: str = ""
# ── Deploy lockout protection ────────────────────────────────────
deploy_confirm_window_seconds: int = 180
@@ -138,6 +160,13 @@ class Settings(BaseSettings):
# wireguard_key_store.py.
wireguard_key_secret_dir: str = "/data/wireguard-key-secrets"
# ── Rule observation (usage counters + change history of the live rules) ─
# Two read-only GETs per enabled firewall per cycle (searchRule, ruleStats). The interval is an operator
# setting (env RULE_OBSERVE_INTERVAL_SECONDS); it can not go below the floor, which keeps a typo from
# polling every firewall in a tight loop.
rule_observe_interval_seconds: int = 900
rule_observe_min_interval_seconds: int = 60
# ── Resource monitoring ──────────────────────────────────────────────
# Fixed interval, no admin-editable settings row/UI for this — v1
# deliberately kept minimal (see resource_monitor_service.py).
+5
View File
@@ -0,0 +1,5 @@
"""Konfiguration als Code (1.32): YAML-Export und -Import der zentralen Objekte.
Leitsatz: Der Import ändert nur die OPNexus-Datenbank (Entwürfe). Auf Firewalls kommt weiterhin nur etwas über
Vorschau, Ausrollen und Bestätigen. Entwurf: docs/schema/opnexus-config-v1.schema.json, Handbuch „Konfiguration als Code“.
"""
+228
View File
@@ -0,0 +1,228 @@
"""Host-Konsole: `python -m app.config_code {export|plan|apply|schema}`.
Normalerweise über `opnexus export` und `opnexus apply` auf dem Docker-Host aufgerufen. Wer die Konsole des Hosts
bedient, ist ohnehin allmächtig, deshalb braucht das CLI kein Token; der Akteur im Audit ist `cli (Host-Konsole)`.
Die Datei kommt über stdin (das Backend öffnet nie eine Datei), das Ergebnis geht nach stdout.
Exit-Codes (für CI): 0 angewendet bzw. keine Unterschiede, 2 Unterschiede (nur `plan`), 3 Datei ungültig,
4 blockiert/veraltet/Selbstprüfung, 1 sonstiger Fehler.
"""
import argparse
import asyncio
import json
import re
import sys
import uuid
from app.config_code import apply as applier
from app.config_code import plan as planner
from app.config_code import schema
from app.config_code.export import export_yaml
from app.config_code.loader import MAX_BYTES, ConfigError, display, parse
ACTOR = "cli (Host-Konsole)"
TOKEN_MARK = "# plan-token: "
_ANSI = re.compile(r"[\x00-\x08\x0b-\x1f\x7f]")
ERRORS_DE = {
"yaml_alias_forbidden": "Aliase (*name) sind nicht erlaubt",
"yaml_anchor_forbidden": "Anker (&name) sind nicht erlaubt",
"yaml_merge_forbidden": "Merge-Schlüssel (<<) sind nicht erlaubt",
"yaml_tag_forbidden": "Explizite Tags (!, !!) sind nicht erlaubt",
"yaml_directive_forbidden": "Direktiven (%YAML, %TAG) sind nicht erlaubt",
"yaml_multiple_documents": "Mehr als ein YAML-Dokument",
"yaml_empty": "Die Datei ist leer",
"yaml_syntax": "YAML-Syntaxfehler",
"duplicate_key": "Doppelter Schlüssel",
"non_string_key": "Schlüssel muss Text sein",
"too_deep": "Zu tief verschachtelt",
"too_many_nodes": "Zu viele Elemente",
"scalar_too_long": "Wert zu lang",
"too_many_objects": "Zu viele Objekte",
"too_many_entries": "Zu viele Einträge",
"too_many_variables": "Zu viele Variablen für diesen Host",
"too_large": "Datei zu groß",
"bad_encoding": "Kein gültiges UTF-8",
"not_a_mapping": "Die Datei muss ein Mapping sein",
"missing_version": "opnexus_config: 1 fehlt",
"unsupported_version": "Diese Version des Dateiformats ist unbekannt",
"unknown_field": "Unbekanntes Feld",
"missing_field": "Pflichtfeld fehlt",
"bad_type": "Falscher Typ",
"bad_value": "Ungültiger Wert",
"bad_format": "Falsches Format",
"too_short": "Zu kurz",
"too_long": "Zu lang",
"control_character": "Steuerzeichen nicht erlaubt",
"surrounding_whitespace": "Leerzeichen am Rand nicht erlaubt",
"bad_variable_name": "Ungültiger Variablenname",
"bad_variable_value": "Ungültiger Variablenwert",
"duplicate_object_key": "Schlüssel kommt zweimal in der Datei vor",
"duplicate_entry": "Eintrag kommt doppelt vor",
"unknown_host": "Unbekannter Host",
"unknown_group": "Unbekannte Gerätegruppe",
"unknown_hierarchy_group": "Unbekannte Hierarchiegruppe",
"hierarchy_cycle": "Zyklus im Hierarchiebaum",
"host_in_two_hierarchy_groups": "Host in zwei Hierarchiegruppen",
"placeholder_not_supported": "Platzhalter {{…}} sind in Hierarchie-Regeln nicht möglich",
}
REASONS_DE = {
"busy_deployment": "offenes Deployment (erst bestätigen oder zurückrollen)",
"prune_deployed": "war ausgerollt (zuerst in der Oberfläche entfernen)",
"prune_referenced": "wird noch verwendet",
}
WARNINGS_DE = {
"redeploy_needed": "{count} geänderte Objekte sind ausgerollt: danach über die Vorschau neu ausrollen",
"targets_without_link": "{count} neue Objekte haben Ziele, aber keine Verknüpfung zu Objekten auf den Firewalls (Duplikate beim Ausrollen möglich)",
"unknown_alias_reference": "{value}: kein zentraler Alias in OPNexus (auf der Firewall von Hand angelegt? dann in Ordnung)",
"variable_removed_in_use": "Variable {name} von {host} wird entfernt, ein Objekt nutzt sie noch",
"target_host_disabled": "Ziel-Host {host} ist deaktiviert",
}
SYMBOL = {"create": "+", "update": "~", "delete": "-", "blocked": "!"}
LABEL = {"create": "(neu)", "delete": "(entfernen)"}
def safe(text) -> str:
"""Nothing from a foreign file reaches the terminal as a control sequence."""
return _ANSI.sub("?", str(text))
def _short(value) -> str:
return safe(display(json.dumps(value, ensure_ascii=False), 60))
def format_errors(errors: list[dict]) -> str:
lines = []
for e in errors:
where = f"Zeile {e['line']}: " if e.get("line") else ""
path = f"{safe(e['path'])}: " if e.get("path") else ""
detail = ""
params = {k: v for k, v in (e.get("params") or {}).items() if k in ("value", "field", "key", "host", "group", "max", "probes")}
if params:
detail = " (" + ", ".join(f"{k}={safe(v)}" for k, v in params.items()) + ")"
lines.append(f" {where}{path}{ERRORS_DE.get(e['code'], e['code'])}{detail}")
return "\n".join(lines)
def format_plan(public: dict) -> str:
lines = []
for item in public["items"]:
action = item["action"]
planned = item["planned_action"]
name = f"{item['kind']} {safe(item['key'])}"
suffix = "" if item["deployed_hosts"] == 0 else f" [auf {item['deployed_hosts']} Firewall(s) ausgerollt]"
if action == "blocked":
reasons = "; ".join(REASONS_DE.get(r, r) for r in item["reasons"])
lines.append(f"! {name} blockiert {LABEL.get(planned, '(ändern)')}: {reasons}")
elif planned in LABEL:
lines.append(f"{SYMBOL[planned]} {name} {LABEL[planned]}{suffix}")
else:
for change in item["changes"] or [{"field": "", "before": None, "after": None}]:
if change["field"]:
lines.append(f"~ {name} {safe(change['field'])}: {_short(change['before'])} → {_short(change['after'])}{suffix}")
else:
lines.append(f"~ {name}{suffix}")
if item.get("implicit"):
lines[-1] += " (Folge der Reihenfolge)"
for warning in public["warnings"]:
template = WARNINGS_DE.get(warning["code"])
params = {k: safe(v) for k, v in (warning.get("params") or {}).items()}
text = template.format(**params) if template else warning["code"]
where = f"{safe(warning['path'])}: " if warning.get("path") else ""
lines.append(f"? {where}{text}")
s = public["summary"]
lines.append(f"Plan: {s['create']} neu, {s['update']} geändert, {s['delete']} entfernt, {s['unchanged']} unverändert, {s['blocked']} blockiert")
return "\n".join(lines)
def _read_stdin() -> bytes:
data = sys.stdin.buffer.read(MAX_BYTES + 1)
return data
async def run(argv: list[str], stdin: bytes | None = None) -> tuple[int, str, str]:
"""(exit code, stdout, stderr). Separate from main() so tests can call it inside their event loop."""
parser = argparse.ArgumentParser(prog="python -m app.config_code", add_help=False)
parser.add_argument("command", choices=["export", "plan", "apply", "schema"])
parser.add_argument("--kinds", default=None)
parser.add_argument("--prune", action="store_true")
parser.add_argument("--json", action="store_true")
parser.add_argument("--plan-token", default=None)
try:
args = parser.parse_args(argv)
except SystemExit:
return 1, "", "Aufruf: python -m app.config_code {export|plan|apply|schema}\n"
if args.command == "schema":
return 0, schema.json_schema_text(), ""
from app.database import SessionLocal
from app.services import audit_service
context = audit_service.context.set({"actor": ACTOR, "id": str(uuid.uuid4()), "action": args.command})
try:
async with SessionLocal() as db:
if args.command == "export":
wanted = None
if args.kinds:
wanted = {k.strip() for k in args.kinds.split(",") if k.strip()}
if not wanted <= set(schema.KINDS):
return 1, "", f"Unbekannte Art in --kinds. Möglich: {', '.join(schema.KINDS)}\n"
return 0, await export_yaml(db, wanted), ""
raw = stdin if stdin is not None else _read_stdin()
try:
parsed = parse(raw)
if args.command == "plan":
plan = await planner.build_plan(db, parsed, args.prune)
await db.rollback()
public = plan.public()
out = json.dumps(public, ensure_ascii=False, indent=2) + "\n" if args.json else (
format_plan(public) + f"\n{TOKEN_MARK}{plan.token}\n")
s = public["summary"]
if not plan.applicable:
return 4, out, ""
return (2 if s["create"] + s["update"] + s["delete"] else 0), out, ""
if not args.plan_token:
return 1, "", "apply braucht --plan-token (opnexus apply zeigt erst den Plan und fragt dann).\n"
try:
plan = await applier.apply_config(db, parsed, args.prune, args.plan_token)
except applier.PlanStale:
await db.rollback()
return 4, "", "Seit dem Plan hat sich etwas geändert. Nichts wurde übernommen. Bitte neu planen.\n"
except applier.PlanBlocked as exc:
await db.rollback()
return 4, format_plan(exc.plan.public()) + "\n", "Der Plan enthält blockierte Objekte. Nichts wurde übernommen.\n"
except applier.SelfCheckFailed as exc:
await db.rollback()
return 4, "", f"Selbstprüfung fehlgeschlagen, zurückgenommen: {', '.join(exc.differences[:10])}\n"
except Exception:
await db.rollback()
raise
await db.commit()
s = plan.summary()
if args.json:
return 0, json.dumps({"summary": s, "sha256": parsed.sha256}, ensure_ascii=False) + "\n", ""
return 0, (f"Übernommen: {s['create']} neu, {s['update']} geändert, {s['delete']} entfernt. "
"Es wurde nichts auf Firewalls geschrieben: die Entwürfe jetzt über Vorschau → Ausrollen → Bestätigen ausrollen.\n"), ""
except ConfigError as exc:
if args.json:
return 3, json.dumps({"code": exc.code, "errors": exc.errors}, ensure_ascii=False, indent=2) + "\n", ""
return 3, "", f"Die Datei ist ungültig:\n{format_errors(exc.errors)}\n"
finally:
audit_service.context.reset(context)
def main(argv: list[str]) -> int:
try:
code, out, err = asyncio.run(run(argv))
except Exception as exc: # noqa: BLE001 the host CLI shows one line, details are in the container log
sys.stderr.write(f"Fehler: {safe(exc)[:300]}\n")
return 1
sys.stdout.write(out)
sys.stderr.write(err)
return code
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+312
View File
@@ -0,0 +1,312 @@
"""Apply: einen geprüften Plan in einer Transaktion in die Datenbank schreiben. Nie auf eine Firewall.
Ablauf: `policy_lock`, Plan neu berechnen, Token vergleichen (sonst PlanStale), blockierte Pläne ablehnen, schreiben
(Hierarchiegruppen → Gerätegruppen → Aliase → Regeln → NAT → Hierarchie-Regeln → Host-Zuordnung → Variablen,
Löschungen umgekehrt), Revisionen und Audit wie in der UI, dann Selbstprüfung: die Datenbank muss danach genau dem
erwarteten Zustand des Plans entsprechen, sonst SelfCheckFailed (der Aufrufer rollt zurück).
Dieses Modul importiert bewusst weder `opnsense_client` noch einen `*_deploy_service` (test_config_code_apply prüft das).
Es legt keine Deployment-Zeile an und ruft kein `undeploy_*`: gelöscht werden nur Objekte ohne jedes Deployment.
Den Commit macht der Aufrufer.
"""
from datetime import datetime, timezone
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.config_code import plan as planner
from app.config_code.export import canonical
from app.config_code.loader import Parsed
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.host_variable import HostVariable
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import (AliasRevision, DeviceGroup, OneToOneRevision, OutboundNatRevision,
PolicyHierarchyRuleRevision, PortForwardRevision, RuleRevision)
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.services import audit_service
NOTE = "Import (Konfiguration als Code)"
AUDIT_KIND = {
"device_groups": "group", "policy_hierarchy_groups": "policy_hierarchy_group", "aliases": "alias", "rules": "rule",
"port_forwards": "port_forward", "one_to_one": "one_to_one", "outbound_nat": "outbound_nat",
"policy_hierarchy_rules": "policy_hierarchy", "variables": "variable",
}
class PlanStale(Exception):
pass
class PlanBlocked(Exception):
def __init__(self, plan: planner.Plan):
self.plan = plan
super().__init__("plan blocked")
class SelfCheckFailed(Exception):
def __init__(self, differences: list[str]):
self.differences = differences
super().__init__("; ".join(differences[:5]))
def _revision_writer(kind: str):
"""(revision model, foreign key, snapshot function) as the UI uses them, imported lazily per kind.
The *_policy_service modules import their deploy services (for previews); importing them only here keeps the
import of this module free of any firewall client, and nothing here calls into them except `snapshot`.
"""
if kind == "rules":
from app.services.policy_service import snapshot
return RuleRevision, "rule_id", snapshot
if kind == "aliases":
from app.services.alias_policy_service import snapshot
return AliasRevision, "alias_id", snapshot
if kind == "port_forwards":
from app.services.port_forward_policy_service import snapshot
return PortForwardRevision, "port_forward_id", snapshot
if kind == "one_to_one":
from app.services.one_to_one_policy_service import snapshot
return OneToOneRevision, "one_to_one_id", snapshot
if kind == "outbound_nat":
from app.services.outbound_nat_policy_service import snapshot
return OutboundNatRevision, "outbound_nat_id", snapshot
from app.services.policy_hierarchy_service import snapshot
return PolicyHierarchyRuleRevision, "policy_hierarchy_id", snapshot
MODELS = {"aliases": FirewallAlias, "rules": FirewallRule, "port_forwards": PortForwardRule, "one_to_one": OneToOneRule,
"outbound_nat": OutboundNatRule, "policy_hierarchy_rules": PolicyHierarchyRule}
SCALARS = {
"aliases": ("name", "type", "description", "enabled"),
"rules": ("description", "interface", "direction", "ipprotocol", "protocol", "action", "source_net", "source_port",
"destination_net", "destination_port", "enabled", "log"),
"port_forwards": ("description", "interface", "ipprotocol", "protocol", "source_net", "source_port", "destination_net",
"destination_port", "target", "local_port", "natreflection", "enabled"),
"one_to_one": ("description", "interface", "type", "external", "source_net", "destination_net", "source_not",
"destination_not", "natreflection", "log", "enabled"),
"outbound_nat": ("description", "interface", "ipprotocol", "protocol", "source_net", "source_port", "destination_net",
"destination_port", "source_not", "destination_not", "target", "target_port", "enabled"),
"policy_hierarchy_rules": ("rulebase", "description", "interface", "direction", "ipprotocol", "protocol", "action",
"source_net", "source_port", "destination_net", "destination_port", "enabled"),
}
class _Writer:
def __init__(self, db: AsyncSession, plan: planner.Plan):
self.db = db
self.plan = plan
self.now = datetime.now(timezone.utc)
self.events: list[dict] = []
async def load(self) -> None:
db = self.db
self.hosts = {h.name: h for h in (await db.execute(select(OpnsenseHost))).scalars().all()}
self.groups = {g.name: g for g in (await db.execute(
select(DeviceGroup).options(selectinload(DeviceGroup.hosts)))).scalars().all()}
self.hierarchy = {g.name: g for g in (await db.execute(select(PolicyHierarchyGroup))).scalars().all()}
self.rows: dict[str, dict] = {}
for kind, model in MODELS.items():
options = [selectinload(model.deployments)]
if kind != "policy_hierarchy_rules":
options += [selectinload(model.targets), selectinload(model.groups)]
rows = (await db.execute(select(model).options(*options))).scalars().all()
self.rows[kind] = {(r.name if kind == "aliases" else r.config_key): r for r in rows}
def items(self, kind: str, *actions: str) -> list[planner.Item]:
return [i for i in self.plan.items if i.kind == kind and i.action in actions]
def event(self, item: planner.Item, entity_id, name: str) -> None:
self.events.append({"kind": AUDIT_KIND[item.kind], "action": f"import_{item.action}", "status": "success",
"entity_id": str(entity_id) if entity_id else None, "entity_name": name,
"details": {"key": item.key, "fields": [c["field"] for c in item.changes]}})
# -- groups ----------------------------------------------------------------------------------------------------------
async def hierarchy_groups(self) -> None:
items = self.items("policy_hierarchy_groups", "create", "update")
for item in items:
if item.action == "create":
group = PolicyHierarchyGroup(name=item.key, description=item.after["description"])
self.db.add(group)
self.hierarchy[item.key] = group
await self.db.flush()
for item in items:
group = self.hierarchy[item.key]
group.description = item.after["description"]
parent = item.after["parent"]
group.parent_id = self.hierarchy[parent].id if parent else None
self.event(item, group.id, group.name)
await self.db.flush()
async def device_groups(self) -> None:
for item in self.items("device_groups", "create", "update"):
group = self.groups.get(item.key)
if group is None:
group = DeviceGroup(name=item.key)
self.db.add(group)
self.groups[item.key] = group
group.description = item.after["description"]
group.hosts = [self.hosts[n] for n in item.after["hosts"]]
await self.db.flush()
self.event(item, group.id, group.name)
# -- objects ---------------------------------------------------------------------------------------------------------
async def objects(self, kind: str) -> None:
model = MODELS[kind]
revision_model, fk, snapshot = _revision_writer(kind)
for item in self.items(kind, "create", "update"):
after = item.after
row = self.rows[kind].get(item.key)
if row is None:
row = model(targets=[], groups=[]) if kind != "policy_hierarchy_rules" else model()
if kind != "aliases":
row.config_key = item.key
self.db.add(row)
self.rows[kind][item.key] = row
elif not await self.db.scalar(select(revision_model.id).where(getattr(revision_model, fk) == row.id).limit(1)):
self.db.add(revision_model(**{fk: row.id}, note="Vor erster Bearbeitung", snapshot=snapshot(row)))
for name in SCALARS[kind]:
setattr(row, name, after[name])
if kind == "aliases":
row.content = "\n".join(after["entries"])
if kind == "policy_hierarchy_rules":
row.hierarchy_group_id = self.hierarchy[after["group"]].id
else:
row.targets = [self.hosts[n] for n in after["targets"]["hosts"]]
row.groups = [self.groups[n] for n in after["targets"]["groups"]]
if "hierarchy_group" in after["targets"]:
hierarchy = after["targets"]["hierarchy_group"]
row.policy_hierarchy_group_id = self.hierarchy[hierarchy].id if hierarchy else None
row.updated_at = self.now # also for target-only changes: invalidates optimistic edits open in the UI
await self.db.flush()
self.db.add(revision_model(**{fk: row.id}, note=NOTE, snapshot=snapshot(row)))
self.event(item, row.id, row.name if kind == "aliases" else row.description)
async def hierarchy_positions(self) -> None:
"""sequence = rank for every rule in a list the file touches (contiguous 0..n-1, same order as the plan)."""
if "policy_hierarchy_rules" not in self.plan.touched:
return
deleted = {i.key for i in self.items("policy_hierarchy_rules", "delete")}
for key, rank in self.plan.positions.items():
row = self.rows["policy_hierarchy_rules"].get(key)
if row is not None and key not in deleted and row.sequence != rank:
row.sequence = rank
await self.db.flush()
async def host_hierarchy(self) -> None:
for host_name, group_name in self.plan.host_hierarchy.items():
host = self.hosts[host_name]
wanted = self.hierarchy[group_name].id if group_name else None
if host.policy_hierarchy_group_id != wanted:
host.policy_hierarchy_group_id = wanted
await self.db.flush()
async def variables(self) -> None:
if "variables" not in self.plan.touched:
return
existing = {(v.host_id, v.name): v for v in (await self.db.execute(select(HostVariable))).scalars().all()}
for item in self.items("variables", "create", "update"):
host = self.hosts[item.after["host"]]
row = existing.get((host.id, item.after["name"]))
if row is None:
self.db.add(HostVariable(host_id=host.id, name=item.after["name"], value=item.after["value"]))
else:
row.value = item.after["value"]
self.event(item, None, item.key)
for item in self.items("variables", "delete"):
host = self.hosts[item.before["host"]]
await self.db.execute(delete(HostVariable).where(HostVariable.host_id == host.id, HostVariable.name == item.before["name"]))
self.event(item, None, item.key)
await self.db.flush()
# -- deletions -------------------------------------------------------------------------------------------------------
async def deletions(self) -> None:
for kind in ("policy_hierarchy_rules", "outbound_nat", "one_to_one", "port_forwards", "rules", "aliases"):
for item in self.items(kind, "delete"):
row = self.rows[kind][item.key]
if row.deployments: # the planner blocks this; never undeploy from here
raise RuntimeError(f"{kind} {item.key}: delete with deployments")
self.event(item, row.id, row.name if kind == "aliases" else row.description)
await self.db.delete(row)
await self.db.flush()
for item in self.items("device_groups", "delete"):
row = self.groups[item.key]
self.event(item, row.id, row.name)
await self.db.delete(row)
await self.db.flush()
doomed = [self.hierarchy[i.key] for i in self.items("policy_hierarchy_groups", "delete")]
by_id = {g.id: g for g in self.hierarchy.values()}
def depth(group) -> int:
n, current = 0, group
while current.parent_id is not None and current.parent_id in by_id and n < 1000:
n, current = n + 1, by_id[current.parent_id]
return n
for row in sorted(doomed, key=depth, reverse=True): # children before parents (parent_id is RESTRICT)
item = next(i for i in self.items("policy_hierarchy_groups", "delete") if i.key == row.name)
self.event(item, row.id, row.name)
await self.db.delete(row)
await self.db.flush()
async def _self_check(db: AsyncSession, plan: planner.Plan) -> None:
db.expire_all() # compare what is in the database, not what the session believes
actual = await canonical(db, set(plan.touched))
differences: list[str] = []
for kind in plan.touched:
if kind == "variables":
got = {f"{h}/{n}": {"key": f"{h}/{n}", "host": h, "name": n, "value": v}
for h, values in actual["variables"].items() for n, v in values.items()}
else:
ident = planner.KIND_SPECS[kind].ident
got = {item[ident]: item for item in actual[kind]}
want = plan.expected[kind]
for key in sorted(set(got) | set(want)):
if got.get(key) != want.get(key):
differences.append(f"{kind}:{key}")
if "policy_hierarchy_rules" in plan.touched:
rows = (await db.execute(select(PolicyHierarchyRule).options(selectinload(PolicyHierarchyRule.group)))).scalars().all()
ranks = planner._ranks(rows, lambda r: r.config_key, lambda r: (r.group.name, r.rulebase), lambda r: (r.sequence, str(r.id)))
for key, rank in ranks.items():
if plan.positions.get(key) != rank:
differences.append(f"policy_hierarchy_rules:{key}:position")
if differences:
raise SelfCheckFailed(differences)
async def apply_config(db: AsyncSession, parsed: Parsed, prune: bool, token: str) -> planner.Plan:
"""Writes the plan into the session (flushed, not committed). Raises PlanStale, PlanBlocked, SelfCheckFailed or
ConfigError; on any exception the caller must roll back."""
from app.services.policy_service import policy_lock
await policy_lock(db)
plan = await planner.build_plan(db, parsed, prune)
if plan.token != token:
raise PlanStale()
if not plan.applicable:
raise PlanBlocked(plan)
writer = _Writer(db, plan)
await writer.load()
await writer.hierarchy_groups()
await writer.device_groups()
for kind in ("aliases", "rules", "port_forwards", "one_to_one", "outbound_nat", "policy_hierarchy_rules"):
if kind in plan.touched:
await writer.objects(kind)
await writer.hierarchy_positions()
await writer.host_hierarchy()
await writer.variables()
await writer.deletions()
await _self_check(db, plan)
db.add(audit_service.make_event(kind="config", action="import", status="success", entity_name=parsed.sha256[:16],
details={"sha256": parsed.sha256, "prune": prune, "kinds": plan.touched,
"summary": plan.summary()}))
for values in writer.events:
db.add(audit_service.make_event(**values))
await db.flush()
return plan
+154
View File
@@ -0,0 +1,154 @@
"""Kanonische Darstellung der zentralen Objekte und deren YAML-Text.
`canonical(db)` liefert dieselbe Form, die der Lader aus einer Datei erzeugt (Feldnamen und -folge des Schemas). Der
Planer vergleicht beide; der Export schreibt sie nur aus. Deterministisch: feste Abschnittsfolge, Objekte nach Schlüssel,
Ziel- und Mitgliederlisten sortiert, alle Felder immer (auch Defaults und `null`), kein Zeitstempel.
"""
import yaml
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.config_code import schema
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.version import APP_VERSION
HEADER = (
"# OPNexus-Konfiguration (Schema v{schema}), exportiert von OPNexus {version}.\n"
"# Enthält keine Zugänge zu Firewalls, Benutzer, Tokens oder Schlüssel. Prüfen: opnexus apply DATEI --dry-run\n"
)
def _ordered(doc: type[schema._Doc], values: dict) -> dict:
"""Values in the field order of the schema model; a missing or extra field is a bug in the exporter."""
fields = list(doc.model_fields)
if set(values) != set(fields):
raise RuntimeError(f"{doc.__name__}: exporter fields differ from schema ({sorted(set(values) ^ set(fields))})")
return {name: values[name] for name in fields}
def _targets(doc: type[schema._Doc], obj, hierarchy_names: dict) -> dict:
values = {"hosts": sorted(h.name for h in obj.targets), "groups": sorted(g.name for g in obj.groups)}
if "hierarchy_group" in doc.model_fields:
values["hierarchy_group"] = hierarchy_names.get(obj.policy_hierarchy_group_id)
return _ordered(doc, values)
def _flat(doc: type[schema._Doc], obj, **extra) -> dict:
"""Scalar columns of `obj` for every schema field that is not computed here (`key` comes from `config_key`)."""
values = dict(extra)
for name in doc.model_fields:
if name in values:
continue
values[name] = obj.config_key if name == "key" else getattr(obj, name)
return _ordered(doc, values)
def _hierarchy_order(groups: list[PolicyHierarchyGroup]) -> list[PolicyHierarchyGroup]:
"""Parents before children, siblings by name; the same rule the loader expects."""
children: dict = {}
for group in groups:
children.setdefault(group.parent_id, []).append(group)
ordered: list[PolicyHierarchyGroup] = []
def walk(parent_id) -> None:
for group in sorted(children.get(parent_id, []), key=lambda g: g.name):
ordered.append(group)
walk(group.id)
walk(None)
return ordered
async def canonical(db: AsyncSession, kinds: set[str] | None = None) -> dict[str, list | dict]:
"""{kind: content} for the requested kinds (default all), in file order."""
wanted = set(schema.KINDS) if kinds is None else kinds
out: dict[str, list | dict] = {}
all_groups = (await db.execute(select(PolicyHierarchyGroup).options(selectinload(PolicyHierarchyGroup.hosts)))).scalars().all()
hierarchy_names = {g.id: g.name for g in all_groups}
if "device_groups" in wanted:
groups = (await db.execute(select(DeviceGroup).options(selectinload(DeviceGroup.hosts)))).scalars().all()
out["device_groups"] = [
_ordered(schema.DeviceGroupDoc, {"name": g.name, "description": g.description or "", "hosts": sorted(h.name for h in g.hosts)})
for g in sorted(groups, key=lambda g: g.name)
]
if "policy_hierarchy_groups" in wanted:
out["policy_hierarchy_groups"] = [
_ordered(schema.HierarchyGroupDoc, {
"name": g.name,
"parent": hierarchy_names.get(g.parent_id),
"description": g.description or "",
"hosts": sorted(h.name for h in g.hosts),
})
for g in _hierarchy_order(list(all_groups))
]
if "aliases" in wanted:
aliases = (await db.execute(
select(FirewallAlias).options(selectinload(FirewallAlias.targets), selectinload(FirewallAlias.groups))
)).scalars().all()
out["aliases"] = [
_ordered(schema.AliasDoc, {
"name": a.name,
"type": a.type,
"description": a.description or "",
"enabled": a.enabled,
"entries": [line for line in (a.content or "").split("\n") if line],
"targets": _targets(schema.TargetsWithHierarchy, a, hierarchy_names),
})
for a in sorted(aliases, key=lambda a: a.name)
]
simple = (
("rules", FirewallRule, schema.RuleDoc, schema.Targets),
("port_forwards", PortForwardRule, schema.PortForwardDoc, schema.TargetsWithHierarchy),
("one_to_one", OneToOneRule, schema.OneToOneDoc, schema.TargetsWithHierarchy),
("outbound_nat", OutboundNatRule, schema.OutboundNatDoc, schema.TargetsWithHierarchy),
)
for kind, model, doc, targets_doc in simple:
if kind not in wanted:
continue
rows = (await db.execute(select(model).options(selectinload(model.targets), selectinload(model.groups)))).scalars().all()
out[kind] = [
_flat(doc, row, targets=_targets(targets_doc, row, hierarchy_names))
for row in sorted(rows, key=lambda r: r.config_key)
]
if "policy_hierarchy_rules" in wanted:
rules = (await db.execute(select(PolicyHierarchyRule))).scalars().all()
group_order = {g.id: i for i, g in enumerate(_hierarchy_order(list(all_groups)))}
out["policy_hierarchy_rules"] = [
_flat(schema.HierarchyRuleDoc, r, group=hierarchy_names[r.hierarchy_group_id])
for r in sorted(rules, key=lambda r: (group_order[r.hierarchy_group_id], r.rulebase != "pre", r.sequence, str(r.id))) # deploy order
]
if "variables" in wanted:
hosts = (await db.execute(select(OpnsenseHost).options(selectinload(OpnsenseHost.variables)))).scalars().all()
out["variables"] = {
h.name: {v.name: v.value for v in sorted(h.variables, key=lambda v: v.name)}
for h in sorted(hosts, key=lambda h: h.name)
if h.variables
}
return out
def dump_yaml(content: dict[str, list | dict]) -> str:
"""The file text: fixed header, `opnexus_config: 1`, then the sections in schema order."""
document = {"opnexus_config": schema.SCHEMA_VERSION, **{kind: content[kind] for kind in schema.KINDS if kind in content}}
body = yaml.safe_dump(document, sort_keys=False, default_flow_style=False, allow_unicode=True, width=4096)
return HEADER.format(schema=schema.SCHEMA_VERSION, version=APP_VERSION) + body
async def export_yaml(db: AsyncSession, kinds: set[str] | None = None) -> str:
return dump_yaml(await canonical(db, kinds))
+307
View File
@@ -0,0 +1,307 @@
"""Strikter YAML-Lader für Konfigurationsdateien (Sicherheitsfläche: die Datei kommt von außen).
Erlaubt ist nur das, was der Exporter selbst schreibt: Mappings, Listen und Skalare. Verboten und mit eigenem Code
gemeldet: Anker/Aliase (Billion Laughs), Merge-Keys, explizite Tags (`!!python/...`, `!!binary`, `!`), Direktiven,
mehrere Dokumente, doppelte oder nicht-textuelle Mapping-Schlüssel. Implizite Typen nur wie im YAML-1.2-Core-Schema
eingeschränkt: `true/false`, `null/~`/leer und Dezimal-Ganzzahlen; `on`, `yes`, `1.5`, `2026-10-22`, `0755`, `1:30` bleiben
Text. Danach prüft das Schema (schema.ConfigDoc) Felder, Typen, Längen und Steuerzeichen.
Reines Python (`yaml.SafeLoader`, nicht `CSafeLoader`): nur dort greifen die Hooks unten.
"""
import hashlib
import re
from typing import NamedTuple
import yaml
from pydantic import ValidationError
from yaml.composer import Composer
from yaml.constructor import SafeConstructor
from yaml.events import AliasEvent, DocumentStartEvent, StreamEndEvent
from yaml.nodes import MappingNode, ScalarNode, SequenceNode
from yaml.parser import Parser
from yaml.reader import Reader
from yaml.resolver import BaseResolver
from yaml.scanner import Scanner
from app.config_code import schema
MAX_BYTES = 1024 * 1024
MAX_DEPTH = 8
MAX_NODES = 100_000
MAX_SCALAR = schema.MAX_SCALAR
MAX_OBJECTS_PER_KIND = 5000
MAX_VARIABLE_HOSTS = 500
MAX_VARIABLES_PER_HOST = 200 # wie routers/variables.py
MAX_ALIAS_ENTRIES = 2000
MAX_ALIAS_CONTENT = 8000 # wie schemas/alias.py `content`
_STR = "tag:yaml.org,2002:str"
class ConfigError(Exception):
"""The file is not acceptable. `errors` is a list of {code, path, line, params}; `code` is the overall reason."""
def __init__(self, code: str, errors: list[dict]):
self.code = code
self.errors = errors
super().__init__(f"{code}: {errors[:3]}")
def _err(code: str, path: str = "", line: int | None = None, **params) -> dict:
return {"code": code, "path": path, "line": line, "params": params}
def display(value, limit: int = 80) -> str:
"""A value for an error message: shortened and with every control character escaped (no ANSI from foreign files)."""
text = value if isinstance(value, str) else repr(value)
text = text.encode("unicode_escape").decode("ascii")
return text if len(text) <= limit else text[: limit - 1] + "…"
def _path(parts) -> str:
out = ""
for part in parts:
if isinstance(part, int):
out += f"[{part}]"
else:
out += ("." if out else "") + str(part)
return out
class _Rejected(yaml.YAMLError):
def __init__(self, code: str, mark=None, path=(), **params):
self.code, self.mark, self.path, self.params = code, mark, tuple(path), params
super().__init__(code)
class StrictResolver(BaseResolver):
yaml_implicit_resolvers: dict = {}
StrictResolver.add_implicit_resolver(
"tag:yaml.org,2002:bool", re.compile(r"^(?:true|True|TRUE|false|False|FALSE)$"), list("tTfF"))
# Dezimal ohne führende Null und höchstens 18 Stellen: int() großer Zahlen kostet Rechenzeit (und Python begrenzt die Länge).
StrictResolver.add_implicit_resolver(
"tag:yaml.org,2002:int", re.compile(r"^(?:0|-?[1-9][0-9]{0,17})$"), list("-0123456789"))
StrictResolver.add_implicit_resolver(
"tag:yaml.org,2002:null", re.compile(r"^(?:~|null|Null|NULL|)$"), ["~", "n", "N", ""])
class StrictConstructor(SafeConstructor):
yaml_constructors: dict = {}
for _tag, _fn in (
("tag:yaml.org,2002:null", SafeConstructor.construct_yaml_null),
("tag:yaml.org,2002:bool", SafeConstructor.construct_yaml_bool),
("tag:yaml.org,2002:int", SafeConstructor.construct_yaml_int),
(_STR, SafeConstructor.construct_yaml_str),
("tag:yaml.org,2002:seq", SafeConstructor.construct_yaml_seq),
("tag:yaml.org,2002:map", SafeConstructor.construct_yaml_map),
):
StrictConstructor.add_constructor(_tag, _fn)
class StrictLoader(Reader, Scanner, Parser, Composer, StrictConstructor, StrictResolver):
def __init__(self, stream: str):
Reader.__init__(self, stream)
Scanner.__init__(self)
Parser.__init__(self)
Composer.__init__(self)
StrictConstructor.__init__(self)
StrictResolver.__init__(self)
self._depth = 0
self._nodes = 0
self._path: list = []
self.lines: dict[tuple, int] = {} # path -> 1-based line, for messages from the schema check
# -- document level ---------------------------------------------------------------------------------------------
def get_single_node(self):
self.get_event() # StreamStart
if self.check_event(StreamEndEvent):
raise _Rejected("yaml_empty")
start = self.peek_event()
if isinstance(start, DocumentStartEvent) and (start.version is not None or start.tags):
raise _Rejected("yaml_directive_forbidden", start.start_mark)
node = self.compose_document()
if not self.check_event(StreamEndEvent):
raise _Rejected("yaml_multiple_documents", self.peek_event().start_mark)
self.get_event()
return node
# -- node level ---------------------------------------------------------------------------------------------------
def compose_node(self, parent, index):
event = self.peek_event()
if isinstance(event, AliasEvent):
raise _Rejected("yaml_alias_forbidden", event.start_mark, self._path)
if getattr(event, "anchor", None) is not None:
raise _Rejected("yaml_anchor_forbidden", event.start_mark, self._path)
if getattr(event, "tag", None) is not None:
raise _Rejected("yaml_tag_forbidden", event.start_mark, self._path, tag=display(event.tag))
self._nodes += 1
if self._nodes > MAX_NODES:
raise _Rejected("too_many_nodes", event.start_mark, self._path, max=MAX_NODES)
if isinstance(index, ScalarNode):
segment = index.value
elif isinstance(index, int) and not isinstance(index, bool):
segment = index
else:
segment = None # a mapping key
if segment is not None:
self._path.append(segment)
self.lines.setdefault(tuple(self._path), event.start_mark.line + 1)
self._depth += 1
try:
if self._depth > MAX_DEPTH:
raise _Rejected("too_deep", event.start_mark, self._path, max=MAX_DEPTH)
return super().compose_node(parent, index)
finally:
self._depth -= 1
if segment is not None:
self._path.pop()
def compose_scalar_node(self, anchor):
event = self.peek_event()
if len(event.value) > MAX_SCALAR:
raise _Rejected("scalar_too_long", event.start_mark, self._path, max=MAX_SCALAR)
return super().compose_scalar_node(anchor)
def compose_mapping_node(self, anchor):
node = super().compose_mapping_node(anchor)
seen: set[str] = set()
for key, _value in node.value:
if not isinstance(key, ScalarNode) or key.tag != _STR:
raise _Rejected("non_string_key", key.start_mark, self._path)
if key.value == "<<":
raise _Rejected("yaml_merge_forbidden", key.start_mark, self._path)
if key.value in seen:
raise _Rejected("duplicate_key", key.start_mark, self._path, key=display(key.value))
seen.add(key.value)
return node
def load_yaml(raw: bytes | str) -> tuple[object, dict[tuple, int]]:
"""Raw bytes -> plain Python data (dict/list/str/int/bool/None) plus a path->line map. Raises ConfigError."""
if isinstance(raw, str):
raw = raw.encode("utf-8")
if len(raw) > MAX_BYTES:
raise ConfigError("CONFIG_TOO_LARGE", [_err("too_large", max=MAX_BYTES)])
try:
text = raw.decode("utf-8", errors="strict")
except UnicodeDecodeError as exc:
raise ConfigError("CONFIG_INVALID", [_err("bad_encoding", offset=exc.start)]) from None
if text.startswith(""):
text = text[1:]
loader = None
try:
loader = StrictLoader(text) # the Reader already rejects unprintable characters here
node = loader.get_single_node()
data = loader.construct_document(node)
except _Rejected as exc:
line = exc.mark.line + 1 if exc.mark is not None else None
raise ConfigError("CONFIG_INVALID", [_err(exc.code, _path(exc.path), line, **exc.params)]) from None
except yaml.MarkedYAMLError as exc:
mark = exc.problem_mark or exc.context_mark
line = mark.line + 1 if mark is not None else None
raise ConfigError("CONFIG_INVALID", [_err("yaml_syntax", "", line, problem=display(exc.problem or ""))]) from None
except yaml.YAMLError as exc: # e.g. ReaderError for unprintable characters
raise ConfigError("CONFIG_INVALID", [_err("yaml_syntax", problem=display(str(exc)))]) from None
finally:
if loader is not None:
loader.dispose()
return data, loader.lines
def _line_for(lines: dict[tuple, int], loc: tuple) -> int | None:
for end in range(len(loc), 0, -1):
if loc[:end] in lines:
return lines[loc[:end]]
return None
_PYDANTIC_CODES = {
"extra_forbidden": "unknown_field",
"missing": "missing_field",
"literal_error": "bad_value",
"string_too_short": "too_short",
"string_too_long": "too_long",
"string_pattern_mismatch": "bad_format",
}
_OWN_CODES = {"control_character", "surrounding_whitespace", "bad_variable_name", "bad_variable_value"}
def _schema_errors(exc: ValidationError, lines: dict[tuple, int]) -> list[dict]:
out = []
for error in exc.errors(include_url=False):
loc = tuple(error["loc"])
code = _PYDANTIC_CODES.get(error["type"])
if code is None and error["type"] == "value_error":
message = str(error.get("ctx", {}).get("error", ""))
code = message if message in _OWN_CODES else "bad_value"
if code is None:
code = "bad_type"
params = {}
if error["type"] == "extra_forbidden":
params["field"] = display(loc[-1]) if loc else ""
elif code not in ("missing_field",) and "input" in error and not isinstance(error["input"], (dict, list)):
params["value"] = display(error["input"])
out.append(_err(code, _path(loc), _line_for(lines, loc), **params))
return out
def _check_counts(data: dict, lines: dict[tuple, int]) -> list[dict]:
errors = []
for kind in schema.KINDS:
section = data.get(kind)
if kind == "variables":
if isinstance(section, dict):
if len(section) > MAX_VARIABLE_HOSTS:
errors.append(_err("too_many_objects", kind, _line_for(lines, (kind,)), max=MAX_VARIABLE_HOSTS))
for host, values in section.items():
if isinstance(values, dict) and len(values) > MAX_VARIABLES_PER_HOST:
errors.append(_err("too_many_objects", _path((kind, host)), _line_for(lines, (kind, host)),
max=MAX_VARIABLES_PER_HOST))
continue
if isinstance(section, list) and len(section) > MAX_OBJECTS_PER_KIND:
errors.append(_err("too_many_objects", kind, _line_for(lines, (kind,)), max=MAX_OBJECTS_PER_KIND))
aliases = data.get("aliases")
for i, alias in enumerate(aliases if isinstance(aliases, list) else []): # any other shape is for the schema check
entries = alias.get("entries") if isinstance(alias, dict) else None
if isinstance(entries, list):
loc = ("aliases", i, "entries")
if len(entries) > MAX_ALIAS_ENTRIES:
errors.append(_err("too_many_entries", _path(loc), _line_for(lines, loc), max=MAX_ALIAS_ENTRIES))
elif len("\n".join(str(e) for e in entries)) > MAX_ALIAS_CONTENT:
errors.append(_err("too_long", _path(loc), _line_for(lines, loc), max=MAX_ALIAS_CONTENT))
return errors
class Parsed(NamedTuple):
doc: schema.ConfigDoc
sha256: str # of the raw bytes, shown in the plan and stored in the audit
lines: dict[tuple, int] # path -> line, for messages of the planner
def parse(raw: bytes | str) -> Parsed:
"""Load, limit and validate a file. Raises ConfigError."""
if isinstance(raw, str):
raw = raw.encode("utf-8")
sha256 = hashlib.sha256(raw).hexdigest()
data, lines = load_yaml(raw)
if not isinstance(data, dict):
raise ConfigError("CONFIG_INVALID", [_err("not_a_mapping", line=1)])
version = data.get("opnexus_config")
if type(version) is not int or version != schema.SCHEMA_VERSION: # `true == 1` in Python
code = "CONFIG_UNSUPPORTED_VERSION" if isinstance(version, int) and not isinstance(version, bool) else "CONFIG_INVALID"
raise ConfigError(code, [_err("unsupported_version" if code != "CONFIG_INVALID" else "missing_version",
"opnexus_config", _line_for(lines, ("opnexus_config",)),
**({"value": display(version)} if version is not None else {}))])
errors = _check_counts(data, lines)
if errors:
raise ConfigError("CONFIG_INVALID", errors)
try:
doc = schema.ConfigDoc.model_validate(data)
except ValidationError as exc:
raise ConfigError("CONFIG_INVALID", _schema_errors(exc, lines)[:100]) from None
return Parsed(doc, sha256, lines)
@@ -0,0 +1,933 @@
{
"$defs": {
"AliasDoc": {
"additionalProperties": false,
"properties": {
"name": {
"maxLength": 64,
"minLength": 1,
"pattern": "^[A-Za-z0-9_]+$",
"title": "Name",
"type": "string"
},
"type": {
"default": "host",
"enum": [
"host",
"network",
"port",
"networkgroup",
"mac",
"asn"
],
"title": "Type",
"type": "string"
},
"description": {
"default": "",
"maxLength": 255,
"title": "Description",
"type": "string"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
},
"entries": {
"items": {
"maxLength": 8000,
"minLength": 1,
"type": "string"
},
"title": "Entries",
"type": "array"
},
"targets": {
"$ref": "#/$defs/TargetsWithHierarchy"
}
},
"required": [
"name"
],
"title": "AliasDoc",
"type": "object"
},
"DeviceGroupDoc": {
"additionalProperties": false,
"properties": {
"name": {
"maxLength": 120,
"minLength": 1,
"title": "Name",
"type": "string"
},
"description": {
"default": "",
"maxLength": 2000,
"title": "Description",
"type": "string"
},
"hosts": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Hosts",
"type": "array"
}
},
"required": [
"name"
],
"title": "DeviceGroupDoc",
"type": "object"
},
"HierarchyGroupDoc": {
"additionalProperties": false,
"properties": {
"name": {
"maxLength": 120,
"minLength": 1,
"title": "Name",
"type": "string"
},
"parent": {
"anyOf": [
{
"maxLength": 120,
"minLength": 1,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Parent"
},
"description": {
"default": "",
"maxLength": 2000,
"title": "Description",
"type": "string"
},
"hosts": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Hosts",
"type": "array"
}
},
"required": [
"name"
],
"title": "HierarchyGroupDoc",
"type": "object"
},
"HierarchyRuleDoc": {
"additionalProperties": false,
"properties": {
"key": {
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"title": "Key",
"type": "string"
},
"group": {
"maxLength": 120,
"minLength": 1,
"title": "Group",
"type": "string"
},
"rulebase": {
"default": "pre",
"enum": [
"pre",
"post"
],
"title": "Rulebase",
"type": "string"
},
"description": {
"maxLength": 255,
"minLength": 1,
"title": "Description",
"type": "string"
},
"interface": {
"maxLength": 64,
"minLength": 1,
"title": "Interface",
"type": "string"
},
"direction": {
"default": "in",
"enum": [
"in",
"out"
],
"title": "Direction",
"type": "string"
},
"ipprotocol": {
"default": "inet",
"enum": [
"inet",
"inet6"
],
"title": "Ipprotocol",
"type": "string"
},
"protocol": {
"default": "any",
"maxLength": 16,
"minLength": 1,
"title": "Protocol",
"type": "string"
},
"action": {
"default": "pass",
"enum": [
"pass",
"block",
"reject"
],
"title": "Action",
"type": "string"
},
"source_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Source Net",
"type": "string"
},
"source_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Port"
},
"destination_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Destination Net",
"type": "string"
},
"destination_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Destination Port"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
}
},
"required": [
"key",
"group",
"description",
"interface"
],
"title": "HierarchyRuleDoc",
"type": "object"
},
"OneToOneDoc": {
"additionalProperties": false,
"properties": {
"key": {
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"title": "Key",
"type": "string"
},
"description": {
"maxLength": 255,
"minLength": 1,
"title": "Description",
"type": "string"
},
"interface": {
"maxLength": 64,
"minLength": 1,
"title": "Interface",
"type": "string"
},
"type": {
"default": "binat",
"enum": [
"binat",
"nat"
],
"title": "Type",
"type": "string"
},
"external": {
"maxLength": 128,
"minLength": 1,
"title": "External",
"type": "string"
},
"source_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Source Net",
"type": "string"
},
"destination_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Destination Net",
"type": "string"
},
"source_not": {
"default": false,
"title": "Source Not",
"type": "boolean"
},
"destination_not": {
"default": false,
"title": "Destination Not",
"type": "boolean"
},
"natreflection": {
"default": "",
"enum": [
"",
"enable",
"disable"
],
"title": "Natreflection",
"type": "string"
},
"log": {
"default": false,
"title": "Log",
"type": "boolean"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
},
"targets": {
"$ref": "#/$defs/TargetsWithHierarchy"
}
},
"required": [
"key",
"description",
"interface",
"external"
],
"title": "OneToOneDoc",
"type": "object"
},
"OutboundNatDoc": {
"additionalProperties": false,
"properties": {
"key": {
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"title": "Key",
"type": "string"
},
"description": {
"maxLength": 255,
"minLength": 1,
"title": "Description",
"type": "string"
},
"interface": {
"maxLength": 64,
"minLength": 1,
"title": "Interface",
"type": "string"
},
"ipprotocol": {
"default": "inet",
"enum": [
"inet",
"inet6"
],
"title": "Ipprotocol",
"type": "string"
},
"protocol": {
"default": "any",
"maxLength": 16,
"minLength": 1,
"title": "Protocol",
"type": "string"
},
"source_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Source Net",
"type": "string"
},
"source_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Port"
},
"destination_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Destination Net",
"type": "string"
},
"destination_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Destination Port"
},
"source_not": {
"default": false,
"title": "Source Not",
"type": "boolean"
},
"destination_not": {
"default": false,
"title": "Destination Not",
"type": "boolean"
},
"target": {
"default": "",
"maxLength": 128,
"title": "Target",
"type": "string"
},
"target_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Target Port"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
},
"targets": {
"$ref": "#/$defs/TargetsWithHierarchy"
}
},
"required": [
"key",
"description",
"interface"
],
"title": "OutboundNatDoc",
"type": "object"
},
"PortForwardDoc": {
"additionalProperties": false,
"properties": {
"key": {
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"title": "Key",
"type": "string"
},
"description": {
"maxLength": 255,
"minLength": 1,
"title": "Description",
"type": "string"
},
"interface": {
"maxLength": 64,
"minLength": 1,
"title": "Interface",
"type": "string"
},
"ipprotocol": {
"default": "inet",
"enum": [
"inet",
"inet6"
],
"title": "Ipprotocol",
"type": "string"
},
"protocol": {
"default": "tcp",
"maxLength": 16,
"minLength": 1,
"title": "Protocol",
"type": "string"
},
"source_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Source Net",
"type": "string"
},
"source_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Port"
},
"destination_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Destination Net",
"type": "string"
},
"destination_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Destination Port"
},
"target": {
"maxLength": 128,
"minLength": 1,
"title": "Target",
"type": "string"
},
"local_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Local Port"
},
"natreflection": {
"default": "",
"enum": [
"",
"purenat",
"disable"
],
"title": "Natreflection",
"type": "string"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
},
"targets": {
"$ref": "#/$defs/TargetsWithHierarchy"
}
},
"required": [
"key",
"description",
"interface",
"target"
],
"title": "PortForwardDoc",
"type": "object"
},
"RuleDoc": {
"additionalProperties": false,
"properties": {
"key": {
"pattern": "^[a-z0-9][a-z0-9._-]{0,63}$",
"title": "Key",
"type": "string"
},
"description": {
"maxLength": 255,
"minLength": 1,
"title": "Description",
"type": "string"
},
"interface": {
"maxLength": 64,
"minLength": 1,
"title": "Interface",
"type": "string"
},
"direction": {
"default": "in",
"enum": [
"in",
"out"
],
"title": "Direction",
"type": "string"
},
"ipprotocol": {
"default": "inet",
"enum": [
"inet",
"inet6"
],
"title": "Ipprotocol",
"type": "string"
},
"protocol": {
"default": "any",
"maxLength": 16,
"minLength": 1,
"title": "Protocol",
"type": "string"
},
"action": {
"default": "pass",
"enum": [
"pass",
"block",
"reject"
],
"title": "Action",
"type": "string"
},
"source_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Source Net",
"type": "string"
},
"source_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Source Port"
},
"destination_net": {
"default": "any",
"maxLength": 128,
"minLength": 1,
"title": "Destination Net",
"type": "string"
},
"destination_port": {
"anyOf": [
{
"maxLength": 32,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Destination Port"
},
"enabled": {
"default": true,
"title": "Enabled",
"type": "boolean"
},
"log": {
"default": false,
"title": "Log",
"type": "boolean"
},
"targets": {
"$ref": "#/$defs/Targets"
}
},
"required": [
"key",
"description",
"interface"
],
"title": "RuleDoc",
"type": "object"
},
"Targets": {
"additionalProperties": false,
"properties": {
"hosts": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Hosts",
"type": "array"
},
"groups": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Groups",
"type": "array"
}
},
"title": "Targets",
"type": "object"
},
"TargetsWithHierarchy": {
"additionalProperties": false,
"properties": {
"hosts": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Hosts",
"type": "array"
},
"groups": {
"items": {
"maxLength": 120,
"minLength": 1,
"type": "string"
},
"title": "Groups",
"type": "array"
},
"hierarchy_group": {
"anyOf": [
{
"maxLength": 120,
"minLength": 1,
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"title": "Hierarchy Group"
}
},
"title": "TargetsWithHierarchy",
"type": "object"
}
},
"additionalProperties": false,
"description": "Eine Datei. Ein fehlender Abschnitt lässt die Art unberührt (None); ein leerer Abschnitt ist ein leerer Sollzustand.",
"properties": {
"opnexus_config": {
"const": 1,
"title": "Opnexus Config",
"type": "integer"
},
"device_groups": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/DeviceGroupDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Device Groups"
},
"policy_hierarchy_groups": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/HierarchyGroupDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Policy Hierarchy Groups"
},
"aliases": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/AliasDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Aliases"
},
"rules": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/RuleDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Rules"
},
"port_forwards": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/PortForwardDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Port Forwards"
},
"one_to_one": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/OneToOneDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "One To One"
},
"outbound_nat": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/OutboundNatDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Outbound Nat"
},
"policy_hierarchy_rules": {
"anyOf": [
{
"items": {
"$ref": "#/$defs/HierarchyRuleDoc"
},
"type": "array"
},
{
"type": "null"
}
],
"default": null,
"title": "Policy Hierarchy Rules"
},
"variables": {
"anyOf": [
{
"additionalProperties": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"propertyNames": {
"maxLength": 120,
"minLength": 1
},
"type": "object"
},
{
"type": "null"
}
],
"default": null,
"title": "Variables"
}
},
"required": [
"opnexus_config"
],
"title": "OPNexus-Konfiguration (Schema v1)",
"type": "object",
"$schema": "https://json-schema.org/draft/2020-12/schema"
}
+570
View File
@@ -0,0 +1,570 @@
"""Planer: Datei gegen Datenbank, ohne etwas zu schreiben und ohne Firewall-Kontakt.
Der Plan ist ein reiner Inhaltsvergleich der kanonischen Formen (export.canonical und die Datei) je Art und Schlüssel:
`create`, `update` (Feldänderungen vorher/nachher), `delete` (nur mit `prune`), `unchanged`, und Gründe, die ein Objekt
blockieren. Ein Abschnitt, der in der Datei fehlt, lässt die Art unberührt; ein vorhandener, auch leerer Abschnitt plus
`prune` gleicht die Art ab.
Fehler in der Datei (unbekannter Host, Zyklus, doppelter Schlüssel, Platzhalter in Hierarchie-Regeln …) sind
ConfigError (422). Zustände in der Datenbank, die das Übernehmen verhindern (offene Deployments, Prune von
ausgerollten oder noch referenzierten Objekten), sind `blocked`: der Plan wird gezeigt, ist aber nicht anwendbar.
Das Plan-Token ist ein SHA-256 über Datei, `prune` und alles in der Datenbank, wovon das Ergebnis abhängt. Apply
rechnet den Plan unter `policy_lock` neu und vergleicht das Token (`CONFIG_PLAN_STALE`).
"""
import hashlib
import json
import re
import uuid
from dataclasses import dataclass, field
from pydantic import ValidationError
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.config_code import schema
from app.config_code.export import canonical
from app.config_code.loader import MAX_VARIABLES_PER_HOST, ConfigError, Parsed, _line_for, _path, display
from app.database import Base
from app.services.deploy_flow import is_unsettled
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.schemas.alias import AliasCreate
from app.schemas.one_to_one import OneToOneCreate
from app.schemas.outbound_nat import OutboundNatCreate
from app.schemas.policy_hierarchy import PolicyHierarchyRuleCreate
from app.schemas.port_forward import PortForwardCreate
from app.schemas.rule import RuleCreate
from app.services.variables_service import names_in
DEPLOYED = ("success", "pending", "revert_failed")
@dataclass(frozen=True)
class Kind:
section: str
model: type | None
ident: str # identity field in the file ("key"/"name")
api: type | None = None # API create schema, run on every file object (same validators as the UI)
deployable: bool = False
KIND_SPECS = {
"device_groups": Kind("device_groups", DeviceGroup, "name"),
"policy_hierarchy_groups": Kind("policy_hierarchy_groups", PolicyHierarchyGroup, "name"),
"aliases": Kind("aliases", FirewallAlias, "name", AliasCreate, True),
"rules": Kind("rules", FirewallRule, "key", RuleCreate, True),
"port_forwards": Kind("port_forwards", PortForwardRule, "key", PortForwardCreate, True),
"one_to_one": Kind("one_to_one", OneToOneRule, "key", OneToOneCreate, True),
"outbound_nat": Kind("outbound_nat", OutboundNatRule, "key", OutboundNatCreate, True),
"policy_hierarchy_rules": Kind("policy_hierarchy_rules", PolicyHierarchyRule, "key", PolicyHierarchyRuleCreate, True),
"variables": Kind("variables", None, "key"),
}
TARGET_KINDS = ("aliases", "rules", "port_forwards", "one_to_one", "outbound_nat")
HIERARCHY_TARGET_KINDS = ("aliases", "port_forwards", "one_to_one", "outbound_nat")
RULE_KINDS = ("rules", "port_forwards", "one_to_one", "outbound_nat", "policy_hierarchy_rules")
# Fields whose value may name an alias (OPNsense resolves aliases by name in these).
ALIAS_REF_FIELDS = ("source_net", "destination_net", "source_port", "destination_port", "target", "local_port", "target_port", "external")
# Values in those fields that are not aliases: keywords, interface names/addresses, OPNsense's built-in aliases.
_NOT_ALIAS = re.compile(r"^(?:any|\(self\)|(?:lan|wan|lo0|opt\d+)(?:ip|net)?|wanip|lanip|bogons|bogonsv6|sshlockout|virusprot|__.*)$", re.I)
_ALIAS_LIKE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
@dataclass
class Item:
kind: str
key: str
action: str # create | update | delete | unchanged
before: dict | None = None
after: dict | None = None
changes: list = field(default_factory=list)
reasons: list = field(default_factory=list)
deployed_hosts: int = 0
implicit: bool = False # not in the file, changed only as a consequence (position, hierarchy members)
db_id: uuid.UUID | None = None
def public(self) -> dict:
return {
"kind": self.kind,
"key": self.key,
"action": "blocked" if self.reasons else self.action,
"planned_action": self.action,
"changes": self.changes,
"deployed_hosts": self.deployed_hosts,
"reasons": self.reasons,
"implicit": self.implicit,
}
@dataclass
class Plan:
sha256: str
prune: bool
touched: list[str]
items: list[Item]
warnings: list[dict]
token: str = ""
# For apply and its self-check: the complete expected state of every touched kind, keyed by identity.
expected: dict = field(default_factory=dict)
positions: dict = field(default_factory=dict) # hierarchy rule key -> rank in its (group, rulebase)
host_hierarchy: dict = field(default_factory=dict) # host name -> hierarchy group name (only when that section is touched)
file_objects: dict = field(default_factory=dict) # kind -> {identity: file dict}
@property
def applicable(self) -> bool:
return not any(i.reasons for i in self.items)
def summary(self) -> dict:
out = {"create": 0, "update": 0, "delete": 0, "unchanged": 0, "blocked": 0}
for item in self.items:
out["blocked" if item.reasons else item.action] += 1
return out
def public(self) -> dict:
changed = [i for i in self.items if i.action != "unchanged" or i.reasons]
return {
"token": self.token,
"sha256": self.sha256,
"prune": self.prune,
"kinds": self.touched,
"summary": self.summary(),
"items": [i.public() for i in changed],
"warnings": self.warnings,
"applicable": self.applicable,
}
def _err(errors: list, code: str, loc: tuple, lines: dict, **params) -> None:
errors.append({"code": code, "path": _path(loc), "line": _line_for(lines, loc), "params": params})
def _file_items(doc: schema.ConfigDoc, kind: str) -> list[dict]:
section = getattr(doc, kind)
if kind == "variables":
return [{"key": f"{host}/{name}", "host": host, "name": name, "value": value}
for host, values in section.items() for name, value in values.items()]
return [item.model_dump(mode="python") for item in section]
def _normalize(kind: str, item: dict, loc: tuple, errors: list, lines: dict) -> dict:
"""Sorted target/member lists like the exporter writes them; duplicates are an error, not silently merged."""
def sorted_unique(values: list, where: tuple) -> list:
seen = set()
for i, value in enumerate(values):
if value in seen:
_err(errors, "duplicate_entry", where + (i,), lines, value=display(value))
seen.add(value)
return sorted(seen)
item = dict(item)
if "targets" in item:
targets = dict(item["targets"])
targets["hosts"] = sorted_unique(targets["hosts"], loc + ("targets", "hosts"))
targets["groups"] = sorted_unique(targets["groups"], loc + ("targets", "groups"))
item["targets"] = targets
if kind in ("device_groups", "policy_hierarchy_groups"):
item["hosts"] = sorted_unique(item["hosts"], loc + ("hosts",))
return item
def _changes(before: dict, after: dict) -> list[dict]:
out = []
for name in after:
if name == "targets":
for sub in after["targets"]:
if before["targets"].get(sub) != after["targets"][sub]:
out.append({"field": f"targets.{sub}", "before": before["targets"].get(sub), "after": after["targets"][sub]})
elif before.get(name) != after[name]:
out.append({"field": name, "before": before.get(name), "after": after[name]})
return out
def _ranks(rows, key_of, bucket_of, order_of) -> dict:
buckets: dict = {}
for row in rows:
buckets.setdefault(bucket_of(row), []).append(row)
out = {}
for members in buckets.values():
for rank, row in enumerate(sorted(members, key=order_of)):
out[key_of(row)] = rank
return out
async def _generic_references(db: AsyncSession, target_table: str, skip: set[str], ids: list[uuid.UUID]) -> dict:
"""{id: [table, ...]} for rows in tables outside the import that point at `ids` (VPN, rollout waves, future tables)."""
found: dict = {}
if not ids:
return found
for table in Base.metadata.sorted_tables:
if table.name in skip:
continue
for fk in table.foreign_keys:
if fk.column.table.name != target_table:
continue
column = fk.parent
rows = (await db.execute(
select(column, func.count()).where(column.in_(ids)).group_by(column)
)).all()
for ref_id, _count in rows:
found.setdefault(ref_id, []).append(table.name)
return found
async def _load(db: AsyncSession) -> dict:
loaded: dict = {}
for kind in TARGET_KINDS:
model = KIND_SPECS[kind].model
rows = (await db.execute(select(model).options(selectinload(model.deployments)))).scalars().all()
loaded[kind] = {(r.name if kind == "aliases" else r.config_key): r for r in rows}
rows = (await db.execute(select(PolicyHierarchyRule).options(selectinload(PolicyHierarchyRule.deployments)))).scalars().all()
loaded["policy_hierarchy_rules"] = {r.config_key: r for r in rows}
loaded["device_groups"] = {g.name: g for g in (await db.execute(select(DeviceGroup))).scalars().all()}
loaded["policy_hierarchy_groups"] = {g.name: g for g in (await db.execute(select(PolicyHierarchyGroup))).scalars().all()}
loaded["hosts"] = {h.name: h for h in (await db.execute(select(OpnsenseHost))).scalars().all()}
return loaded
async def build_plan(db: AsyncSession, parsed: Parsed, prune: bool) -> Plan:
"""The plan for `parsed` against the current database. Raises ConfigError for problems in the file itself."""
doc, lines = parsed.doc, parsed.lines
touched = [k for k in schema.KINDS if getattr(doc, k) is not None]
errors: list[dict] = []
warnings: list[dict] = []
db_canon = await canonical(db)
loaded = await _load(db)
hosts = loaded["hosts"]
hierarchy_by_id = {g.id: g.name for g in loaded["policy_hierarchy_groups"].values()}
# -- file objects: identity, duplicates, normalization -------------------------------------------------------------
file_objects: dict[str, dict[str, dict]] = {}
file_locs: dict[tuple, tuple] = {}
for kind in touched:
spec = KIND_SPECS[kind]
objects: dict[str, dict] = {}
for index, raw in enumerate(_file_items(doc, kind)):
if kind == "variables":
ident, loc = raw["key"], ("variables", raw["host"], raw["name"])
else:
ident, loc = raw[spec.ident], (kind, index)
if ident in objects:
_err(errors, "duplicate_object_key", loc + ((spec.ident,) if kind != "variables" else ()), lines, key=display(ident))
continue
objects[ident] = raw if kind == "variables" else _normalize(kind, raw, loc, errors, lines)
file_locs[(kind, ident)] = loc
file_objects[kind] = objects
def db_items(kind: str) -> dict[str, dict]:
if kind == "variables":
return {f"{h}/{n}": {"key": f"{h}/{n}", "host": h, "name": n, "value": v}
for h, values in db_canon["variables"].items() for n, v in values.items()}
ident = KIND_SPECS[kind].ident
return {item[ident]: item for item in db_canon[kind]}
db_objects = {kind: db_items(kind) for kind in schema.KINDS}
# -- post-import state of every kind (file objects + what stays in the database) -----------------------------------
deleted: dict[str, set[str]] = {kind: set() for kind in schema.KINDS}
post: dict[str, dict[str, dict]] = {}
for kind in schema.KINDS:
if kind in file_objects:
remaining = {k: v for k, v in db_objects[kind].items() if k not in file_objects[kind]}
if prune:
deleted[kind] = set(remaining)
remaining = {}
post[kind] = {**remaining, **file_objects[kind]}
else:
post[kind] = dict(db_objects[kind])
host_names = set(hosts)
group_names = set(post["device_groups"])
hierarchy_names = set(post["policy_hierarchy_groups"])
# -- references in the file ------------------------------------------------------------------------------------------
for kind in touched:
for ident, item in file_objects[kind].items():
loc = file_locs[(kind, ident)]
if kind == "variables":
if item["host"] not in host_names:
_err(errors, "unknown_host", ("variables", item["host"]), lines, host=display(item["host"]))
continue
member_hosts = item.get("hosts", []) + item.get("targets", {}).get("hosts", [])
for name in member_hosts:
if name not in host_names:
_err(errors, "unknown_host", loc, lines, host=display(name))
for name in item.get("targets", {}).get("groups", []):
if name not in group_names:
_err(errors, "unknown_group", loc + ("targets", "groups"), lines, group=display(name))
hierarchy_ref = item.get("targets", {}).get("hierarchy_group") if kind in HIERARCHY_TARGET_KINDS else None
for ref, where in ((hierarchy_ref, ("targets", "hierarchy_group")), (item.get("parent"), ("parent",)),
(item.get("group") if kind == "policy_hierarchy_rules" else None, ("group",))):
if ref is not None and ref not in hierarchy_names:
_err(errors, "unknown_hierarchy_group", loc + where, lines, group=display(ref))
if kind == "policy_hierarchy_rules":
for name, value in item.items():
if names_in(value):
_err(errors, "placeholder_not_supported", loc + (name,), lines)
spec = KIND_SPECS[kind]
if spec.api is not None:
item = file_objects[kind][ident] = post[kind][ident] = _check_api_model(kind, item, loc, errors, lines)
for name in member_hosts:
host = hosts.get(name)
if host is not None and not host.enabled:
warnings.append({"code": "target_host_disabled", "path": _path(loc), "params": {"host": display(name)}})
# -- hierarchy tree and host membership ------------------------------------------------------------------------------
host_hierarchy: dict[str, str | None] = {h.name: hierarchy_by_id.get(h.policy_hierarchy_group_id) for h in hosts.values()}
if "policy_hierarchy_groups" in file_objects:
file_groups = file_objects["policy_hierarchy_groups"]
for host_name, group in list(host_hierarchy.items()):
if prune or group in file_groups:
# The file lists the complete membership of its groups; with prune it is the whole hierarchy.
host_hierarchy[host_name] = None
listed: dict[str, str] = {}
for name, group in file_groups.items():
for host_name in group["hosts"]:
if host_name in listed:
_err(errors, "host_in_two_hierarchy_groups", file_locs[("policy_hierarchy_groups", name)] + ("hosts",),
lines, host=display(host_name), first=display(listed[host_name]))
listed[host_name] = name
if host_name in host_hierarchy:
host_hierarchy[host_name] = name
parents = {name: g["parent"] for name, g in post["policy_hierarchy_groups"].items()}
for start in parents:
seen, current = set(), start
while current is not None and current not in seen:
seen.add(current)
current = parents.get(current)
if current is not None and current == start:
_err(errors, "hierarchy_cycle", file_locs.get(("policy_hierarchy_groups", start), ("policy_hierarchy_groups",)),
lines, group=display(start))
for name, group in post["policy_hierarchy_groups"].items(): # membership as it will be after the import
post["policy_hierarchy_groups"][name] = {**group, "hosts": sorted(h for h, g in host_hierarchy.items() if g == name)}
if "variables" in file_objects: # the host page allows MAX_VARIABLES_PER_HOST; a merged import must not get past it
per_host: dict[str, int] = {}
for item in post["variables"].values():
per_host[item["host"]] = per_host.get(item["host"], 0) + 1
for host_name, count in sorted(per_host.items()):
if count > MAX_VARIABLES_PER_HOST:
_err(errors, "too_many_variables", ("variables", host_name), lines, host=display(host_name),
max=MAX_VARIABLES_PER_HOST, count=count)
if errors:
raise ConfigError("CONFIG_INVALID", errors[:100])
# -- hierarchy rule positions ----------------------------------------------------------------------------------------
db_rule_rows = loaded["policy_hierarchy_rules"]
db_positions = _ranks(db_rule_rows.values(), lambda r: r.config_key,
lambda r: (hierarchy_by_id[r.hierarchy_group_id], r.rulebase), lambda r: (r.sequence, str(r.id))) # = deploy order
positions = dict(db_positions)
if "policy_hierarchy_rules" in file_objects:
buckets: dict[tuple, list[str]] = {}
for key, item in file_objects["policy_hierarchy_rules"].items(): # file order = order in the file
buckets.setdefault((item["group"], item["rulebase"]), []).append(key)
for key, item in post["policy_hierarchy_rules"].items():
if key not in file_objects["policy_hierarchy_rules"]: # stays in the database: after the file's rules
buckets.setdefault((item["group"], item["rulebase"]), [])
positions = {}
extras: dict[tuple, list[str]] = {}
for key in sorted((k for k in post["policy_hierarchy_rules"] if k not in file_objects["policy_hierarchy_rules"]),
key=lambda k: db_positions.get(k, 0)):
item = post["policy_hierarchy_rules"][key]
extras.setdefault((item["group"], item["rulebase"]), []).append(key)
for bucket in set(buckets) | set(extras):
for rank, key in enumerate(buckets.get(bucket, []) + extras.get(bucket, [])):
positions[key] = rank
# -- items -----------------------------------------------------------------------------------------------------------
items: list[Item] = []
for kind in touched:
rows = loaded.get(kind, {})
for ident in sorted(set(file_objects[kind]) | deleted[kind] | _implicit(kind, post, db_objects, positions, db_positions)):
row = rows.get(ident) if kind != "variables" else None
before = db_objects[kind].get(ident)
after = post[kind].get(ident) if ident not in deleted[kind] else None
if before is None:
item = Item(kind, ident, "create", None, after)
elif after is None:
item = Item(kind, ident, "delete", before, None)
else:
changes = _changes(before, after)
if kind == "policy_hierarchy_rules" and db_positions.get(ident) != positions.get(ident):
changes.append({"field": "position", "before": db_positions.get(ident), "after": positions.get(ident)})
item = Item(kind, ident, "update" if changes else "unchanged", before, after, changes)
item.implicit = ident not in file_objects[kind] and ident not in deleted[kind]
if row is not None:
item.db_id = row.id
deployments = getattr(row, "deployments", None) or []
item.deployed_hosts = sum(1 for d in deployments if d.status in DEPLOYED)
if item.action in ("update", "delete") and any(is_unsettled(d) for d in deployments):
item.reasons.append("busy_deployment")
if item.action == "delete" and deployments:
item.reasons.append("prune_deployed")
items.append(item)
await _prune_references(db, items, post, loaded, deleted, host_hierarchy)
redeploy = sum(1 for i in items if i.action == "update" and i.deployed_hosts)
if redeploy:
warnings.append({"code": "redeploy_needed", "path": "", "params": {"count": redeploy}})
creates_with_targets = sum(1 for i in items if i.action == "create" and i.kind in TARGET_KINDS
and (i.after["targets"]["hosts"] or i.after["targets"]["groups"] or i.after["targets"].get("hierarchy_group")))
if creates_with_targets and not any(loaded[k] for k in TARGET_KINDS + ("policy_hierarchy_rules",)):
warnings.append({"code": "targets_without_link", "path": "", "params": {"count": creates_with_targets}})
warnings += _alias_reference_warnings(post, file_objects, file_locs)
warnings += _variable_warnings(post, deleted)
plan = Plan(parsed.sha256, prune, touched, items, warnings, positions=positions,
host_hierarchy=host_hierarchy if "policy_hierarchy_groups" in file_objects else {},
file_objects=file_objects)
plan.expected = {kind: post[kind] for kind in touched}
plan.token = _token(plan, db_canon, db_positions, loaded)
return plan
def _implicit(kind: str, post: dict, db_objects: dict, positions: dict, db_positions: dict) -> set[str]:
"""Objects the file does not name but whose stored state still changes (rank in a reordered list, group members)."""
out = set()
if kind == "policy_hierarchy_rules":
out = {k for k in post[kind] if k in db_objects[kind] and positions.get(k) != db_positions.get(k)}
if kind == "policy_hierarchy_groups":
out = {k for k, v in post[kind].items() if k in db_objects[kind] and db_objects[kind][k] != v}
return out
def _check_api_model(kind: str, item: dict, loc: tuple, errors: list, lines: dict) -> dict:
"""The real API schema on the same values (minus references), so a file can never store what the UI refuses.
Returns the item with the values as the API would store them (stripped fields, cleaned alias entries)."""
values = {k: v for k, v in item.items() if k not in ("key", "targets", "group", "entries")}
if kind == "aliases":
values["content"] = "\n".join(item["entries"])
if kind == "policy_hierarchy_rules":
values["hierarchy_group_id"] = uuid.UUID(int=0)
try:
cleaned = KIND_SPECS[kind].api.model_validate(values)
except ValidationError as exc:
for error in exc.errors(include_url=False):
_err(errors, "bad_value", loc + tuple(error["loc"]), lines)
return item
out = dict(item)
for name in item:
if name in values and name != "content" and hasattr(cleaned, name):
out[name] = getattr(cleaned, name)
if kind == "aliases":
out["entries"] = [e for e in cleaned.content.split("\n") if e]
return out
async def _prune_references(db, items: list[Item], post: dict, loaded: dict, deleted: dict, host_hierarchy: dict) -> None:
by_key = {(i.kind, i.key): i for i in items}
# device groups: post-state objects of the import, plus every other table that points at the group
if deleted["device_groups"]:
users: dict[str, list[str]] = {}
for kind in TARGET_KINDS:
for ident, item in post[kind].items():
for group in item["targets"]["groups"]:
users.setdefault(group, []).append(f"{kind}:{ident}")
ids = {loaded["device_groups"][n].id: n for n in deleted["device_groups"]}
skip = {"device_group_members", "rule_groups", "alias_groups", "port_forward_groups", "one_to_one_groups", "outbound_nat_groups"}
for ref_id, tables in (await _generic_references(db, "device_groups", skip, list(ids))).items():
users.setdefault(ids[ref_id], []).extend(tables)
for name in deleted["device_groups"]:
if users.get(name):
by_key[("device_groups", name)].reasons.append("prune_referenced")
if deleted["policy_hierarchy_groups"]:
users = {}
for kind in HIERARCHY_TARGET_KINDS:
for ident, item in post[kind].items():
if item["targets"]["hierarchy_group"]:
users.setdefault(item["targets"]["hierarchy_group"], []).append(f"{kind}:{ident}")
for ident, item in post["policy_hierarchy_rules"].items():
users.setdefault(item["group"], []).append(f"policy_hierarchy_rules:{ident}")
for name, item in post["policy_hierarchy_groups"].items():
if item["parent"]:
users.setdefault(item["parent"], []).append(f"policy_hierarchy_groups:{name}")
for host_name, group in host_hierarchy.items():
if group:
users.setdefault(group, []).append(f"host:{host_name}")
ids = {loaded["policy_hierarchy_groups"][n].id: n for n in deleted["policy_hierarchy_groups"]}
skip = {"policy_hierarchy_groups", "firewall_aliases", "port_forward_rules", "one_to_one_rules", "outbound_nat_rules",
"policy_hierarchy_rules", "opnsense_hosts"}
for ref_id, tables in (await _generic_references(db, "policy_hierarchy_groups", skip, list(ids))).items():
users.setdefault(ids[ref_id], []).extend(tables)
for name in deleted["policy_hierarchy_groups"]:
if users.get(name):
by_key[("policy_hierarchy_groups", name)].reasons.append("prune_referenced")
if deleted["aliases"]:
used: set[str] = set()
for kind in RULE_KINDS:
for item in post[kind].values():
used.update(str(item[f]) for f in ALIAS_REF_FIELDS if item.get(f) is not None)
for item in post["aliases"].values():
used.update(item["entries"])
for name in deleted["aliases"]:
if name in used and "prune_referenced" not in by_key[("aliases", name)].reasons:
by_key[("aliases", name)].reasons.append("prune_referenced")
def _alias_reference_warnings(post: dict, file_objects: dict, file_locs: dict) -> list[dict]:
aliases = set(post["aliases"])
out = []
for kind in RULE_KINDS:
for ident, item in file_objects.get(kind, {}).items():
for name in ALIAS_REF_FIELDS:
value = item.get(name)
if isinstance(value, str) and _ALIAS_LIKE.match(value) and not _NOT_ALIAS.match(value) and value not in aliases:
out.append({"code": "unknown_alias_reference", "path": _path(file_locs[(kind, ident)] + (name,)),
"params": {"value": display(value)}})
return out
def _variable_warnings(post: dict, deleted: dict) -> list[dict]:
if not deleted["variables"]:
return []
used: set[str] = set()
for kind in RULE_KINDS + ("aliases",):
for item in post[kind].values():
for value in item.values():
if isinstance(value, str):
used |= names_in(value)
elif isinstance(value, list):
for entry in value:
used |= names_in(entry)
out = []
for key in sorted(deleted["variables"]):
host, name = key.split("/", 1)
if name in used:
out.append({"code": "variable_removed_in_use", "path": f"variables.{host}", "params": {"host": display(host), "name": name}})
return out
def _token(plan: Plan, db_canon: dict, db_positions: dict, loaded: dict) -> str:
deployments = {
kind: {ident: sorted((str(d.host_id), d.status) for d in (row.deployments or []))
for ident, row in loaded[kind].items()}
for kind in TARGET_KINDS + ("policy_hierarchy_rules",)
}
material = {
"file": plan.file_objects,
"prune": plan.prune,
"db": db_canon,
"positions": db_positions,
"hosts": sorted((h.name, str(h.id), h.enabled, str(h.policy_hierarchy_group_id)) for h in loaded["hosts"].values()),
"ids": {k: sorted((n, str(o.id)) for n, o in loaded[k].items()) for k in ("device_groups", "policy_hierarchy_groups")},
"deployments": deployments,
"items": [i.public() for i in plan.items],
"warnings": plan.warnings,
}
return hashlib.sha256(json.dumps(material, sort_keys=True, default=str, ensure_ascii=False).encode()).hexdigest()
+242
View File
@@ -0,0 +1,242 @@
"""Schema v1 der Konfigurationsdatei. Strikt: unbekannte Felder, falsche Typen und Steuerzeichen sind Fehler.
Die Grenzen entsprechen den API-Schemas der jeweiligen Objekte (schemas/rule.py usw.); `tests/test_config_code_schema.py`
hält beide gleich. Steuerzeichen sind überall verboten, außer Tab und Zeilenumbruch in Beschreibungen (die UI erlaubt
mehrzeilige Beschreibungen). Namen und Schlüssel (die Identität eines Objekts) dürfen keine umgebenden Leerzeichen haben;
alle anderen Werte säubert der Planer mit demselben API-Schema wie die UI (plan._check_api_model).
"""
import re
from typing import Annotated, Literal
from pydantic import AfterValidator, BaseModel, BeforeValidator, ConfigDict, Field, StringConstraints
from app.models.config_key import CONFIG_KEY_RE
from app.services.variables_service import NAME_RE as VARIABLE_NAME_RE, VALUE_RE as VARIABLE_VALUE_RE
SCHEMA_VERSION = 1
MAX_SCALAR = 8000
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
_CONTROL_TEXT = re.compile(r"[\x00-\x08\x0b-\x1f\x7f]") # tab and newline allowed
def _clean(value: str) -> str:
"""Names and keys: they identify an object, so no control characters and no surrounding whitespace."""
if _CONTROL.search(value):
raise ValueError("control_character")
if value != value.strip():
raise ValueError("surrounding_whitespace")
return value
def _clean_value(value: str) -> str:
if _CONTROL.search(value):
raise ValueError("control_character")
return value
def _clean_text(value: str) -> str:
if _CONTROL_TEXT.search(value):
raise ValueError("control_character")
return value
def _port(value):
# YAML liest `53` als Zahl; Ports sind in OPNsense Text (auch Bereiche wie "8000-8100"). bool bleibt ein Typfehler.
return str(value) if isinstance(value, int) and not isinstance(value, bool) else value
Text = Annotated[str, AfterValidator(_clean)]
Key = Annotated[str, StringConstraints(pattern=CONFIG_KEY_RE.pattern), AfterValidator(_clean)]
Description = Annotated[str, Field(max_length=255), AfterValidator(_clean_text)]
RuleDescription = Annotated[str, Field(min_length=1, max_length=255), AfterValidator(_clean_text)]
GroupDescription = Annotated[str, Field(max_length=2000), AfterValidator(_clean_text)]
Interface = Annotated[str, Field(min_length=1, max_length=64), AfterValidator(_clean_value)]
Protocol = Annotated[str, Field(min_length=1, max_length=16), AfterValidator(_clean_value)]
Net = Annotated[str, Field(min_length=1, max_length=128), AfterValidator(_clean_value)]
PortText = Annotated[str, Field(max_length=32), AfterValidator(_clean_value)]
Port = Annotated[PortText | None, BeforeValidator(_port)]
GroupName = Annotated[str, Field(min_length=1, max_length=120), AfterValidator(_clean)]
HostName = Annotated[str, Field(min_length=1, max_length=120), AfterValidator(_clean)]
AliasName = Annotated[str, Field(min_length=1, max_length=64, pattern=r"^[A-Za-z0-9_]+$")]
AliasEntry = Annotated[str, Field(min_length=1, max_length=MAX_SCALAR), AfterValidator(_clean_value)]
IpProtocol = Literal["inet", "inet6"]
class _Doc(BaseModel):
model_config = ConfigDict(extra="forbid", strict=True)
class Targets(_Doc):
hosts: list[HostName] = Field(default_factory=list)
groups: list[GroupName] = Field(default_factory=list)
class TargetsWithHierarchy(Targets):
hierarchy_group: GroupName | None = None
class DeviceGroupDoc(_Doc):
name: GroupName
description: GroupDescription = ""
hosts: list[HostName] = Field(default_factory=list)
class HierarchyGroupDoc(_Doc):
name: GroupName
parent: GroupName | None = None
description: GroupDescription = ""
hosts: list[HostName] = Field(default_factory=list)
class AliasDoc(_Doc):
name: AliasName
type: Literal["host", "network", "port", "networkgroup", "mac", "asn"] = "host"
description: Description = ""
enabled: bool = True
entries: list[AliasEntry] = Field(default_factory=list)
targets: TargetsWithHierarchy = Field(default_factory=TargetsWithHierarchy)
class RuleDoc(_Doc):
key: Key
description: RuleDescription
interface: Interface
direction: Literal["in", "out"] = "in"
ipprotocol: IpProtocol = "inet"
protocol: Protocol = "any"
action: Literal["pass", "block", "reject"] = "pass"
source_net: Net = "any"
source_port: Port = None
destination_net: Net = "any"
destination_port: Port = None
enabled: bool = True
log: bool = False
targets: Targets = Field(default_factory=Targets)
class PortForwardDoc(_Doc):
key: Key
description: RuleDescription
interface: Interface
ipprotocol: IpProtocol = "inet"
protocol: Protocol = "tcp"
source_net: Net = "any"
source_port: Port = None
destination_net: Net = "any"
destination_port: Port = None
target: Net
local_port: Port = None
natreflection: Literal["", "purenat", "disable"] = ""
enabled: bool = True
targets: TargetsWithHierarchy = Field(default_factory=TargetsWithHierarchy)
class OneToOneDoc(_Doc):
key: Key
description: RuleDescription
interface: Interface
type: Literal["binat", "nat"] = "binat"
external: Net
source_net: Net = "any"
destination_net: Net = "any"
source_not: bool = False
destination_not: bool = False
natreflection: Literal["", "enable", "disable"] = ""
log: bool = False
enabled: bool = True
targets: TargetsWithHierarchy = Field(default_factory=TargetsWithHierarchy)
class OutboundNatDoc(_Doc):
key: Key
description: RuleDescription
interface: Interface
ipprotocol: IpProtocol = "inet"
protocol: Protocol = "any"
source_net: Net = "any"
source_port: Port = None
destination_net: Net = "any"
destination_port: Port = None
source_not: bool = False
destination_not: bool = False
target: Annotated[str, Field(max_length=128), AfterValidator(_clean_value)] = ""
target_port: Port = None
enabled: bool = True
targets: TargetsWithHierarchy = Field(default_factory=TargetsWithHierarchy)
class HierarchyRuleDoc(_Doc):
key: Key
group: GroupName
rulebase: Literal["pre", "post"] = "pre"
description: RuleDescription
interface: Interface
direction: Literal["in", "out"] = "in"
ipprotocol: IpProtocol = "inet"
protocol: Protocol = "any"
action: Literal["pass", "block", "reject"] = "pass"
source_net: Net = "any"
source_port: Port = None
destination_net: Net = "any"
destination_port: Port = None
enabled: bool = True
def _variable_name(value: str) -> str:
if not VARIABLE_NAME_RE.match(value):
raise ValueError("bad_variable_name")
return value
def _variable_value(value: str) -> str:
if not VARIABLE_VALUE_RE.match(value):
raise ValueError("bad_variable_value")
return value
VariableName = Annotated[str, AfterValidator(_variable_name)]
VariableValue = Annotated[str, AfterValidator(_variable_value)]
class ConfigDoc(_Doc):
"""Eine Datei. Ein fehlender Abschnitt lässt die Art unberührt (None); ein leerer Abschnitt ist ein leerer Sollzustand."""
opnexus_config: Literal[1]
device_groups: list[DeviceGroupDoc] | None = None
policy_hierarchy_groups: list[HierarchyGroupDoc] | None = None
aliases: list[AliasDoc] | None = None
rules: list[RuleDoc] | None = None
port_forwards: list[PortForwardDoc] | None = None
one_to_one: list[OneToOneDoc] | None = None
outbound_nat: list[OutboundNatDoc] | None = None
policy_hierarchy_rules: list[HierarchyRuleDoc] | None = None
variables: dict[HostName, dict[VariableName, VariableValue]] | None = None
# Abschnittsfolge der Datei = Reihenfolge, in der Export und Plan die Arten behandeln.
KINDS = (
"device_groups",
"policy_hierarchy_groups",
"aliases",
"rules",
"port_forwards",
"one_to_one",
"outbound_nat",
"policy_hierarchy_rules",
"variables",
)
SCHEMA_FILE = "opnexus-config-v1.schema.json"
def json_schema_text() -> str:
"""JSON Schema of the file for editors and CI; checked in next to this module and compared by a test."""
import json
schema = ConfigDoc.model_json_schema()
schema["$schema"] = "https://json-schema.org/draft/2020-12/schema"
schema["title"] = "OPNexus-Konfiguration (Schema v1)"
return json.dumps(schema, indent=2, ensure_ascii=False, sort_keys=False) + "\n"
+93
View File
@@ -11,6 +11,8 @@ class ErrorCode(str, Enum):
UPDATER_REJECTED = "UPDATER_REJECTED"
UPDATE_NOT_OFFERED = "UPDATE_NOT_OFFERED"
UPDATE_MAJOR_CHANGE = "UPDATE_MAJOR_CHANGE"
UPDATE_CHECK_MANAGED_BY_SERVER = "UPDATE_CHECK_MANAGED_BY_SERVER"
UPDATE_CHECK_URL_INVALID = "UPDATE_CHECK_URL_INVALID"
ADMIN_REQUIRED = "ADMIN_REQUIRED"
BACKUP_EMPTY = "BACKUP_EMPTY"
BACKUP_INVALID_XML = "BACKUP_INVALID_XML"
@@ -60,6 +62,7 @@ class ErrorCode(str, Enum):
GRUPPE_WIRD_NOCH_VON_ALIASEN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_ALIASEN_VERWENDET"
GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET"
GRUPPE_WIRD_NOCH_VON_REGELN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_REGELN_VERWENDET"
GRUPPE_WIRD_NOCH_VON_VPN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_VPN_VERWENDET"
HOSTNAMEN_EXAKT_EINGEBEN = "HOSTNAMEN_EXAKT_EINGEBEN"
HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN = "HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN"
HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0 = "HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0"
@@ -70,6 +73,8 @@ class ErrorCode(str, Enum):
HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN = "HOST_HAT_NOCH_VPN_ODER_ZERTIFIKATS_OBJEKTE_DIESE_ZUERST_ENTFERNEN"
HOST_IST_DEAKTIVIERT = "HOST_IST_DEAKTIVIERT"
HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS = "HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS"
HOST_URL_EXISTS = "HOST_URL_EXISTS"
HOST_CONNECTION_FAILED = "HOST_CONNECTION_FAILED"
HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B = "HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B"
HOST_NICHT_GEFUNDEN = "HOST_NICHT_GEFUNDEN"
HOST_REQUEST_FAILED = "HOST_REQUEST_FAILED"
@@ -85,6 +90,15 @@ class ErrorCode(str, Enum):
MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN = "MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN"
NICHT_AUTHENTIFIZIERT = "NICHT_AUTHENTIFIZIERT"
CROSS_ORIGIN_BLOCKED = "CROSS_ORIGIN_BLOCKED"
CROSS_SITE_REQUEST_BLOCKED = "CROSS_SITE_REQUEST_BLOCKED"
GUI_SESSION_REQUIRED = "GUI_SESSION_REQUIRED"
GUI_GATEWAY_NOT_CONFIGURED = "GUI_GATEWAY_NOT_CONFIGURED"
GUI_REQUIRES_HTTPS = "GUI_REQUIRES_HTTPS"
GUI_ACCESS_DISABLED = "GUI_ACCESS_DISABLED"
GUI_HOST_NOT_ELIGIBLE = "GUI_HOST_NOT_ELIGIBLE"
GUI_NO_FREE_SLOT = "GUI_NO_FREE_SLOT"
GUI_SLOT_COOLING_DOWN = "GUI_SLOT_COOLING_DOWN"
GUI_TOO_MANY_SESSIONS = "GUI_TOO_MANY_SESSIONS"
TLS_PROBE_FAILED = "TLS_PROBE_FAILED"
TLS_PEM_INVALID = "TLS_PEM_INVALID"
TLS_PIN_MISMATCH = "TLS_PIN_MISMATCH"
@@ -114,6 +128,13 @@ class ErrorCode(str, Enum):
NETWORK_FIELD_INVALID = "NETWORK_FIELD_INVALID"
NETWORK_ITEM_NOT_FOUND = "NETWORK_ITEM_NOT_FOUND"
RULE_NOT_ADOPTABLE = "RULE_NOT_ADOPTABLE"
OBJECT_NOT_ADOPTABLE = "OBJECT_NOT_ADOPTABLE"
ADOPT_OBJECT_NOT_FOUND = "ADOPT_OBJECT_NOT_FOUND"
OUTBOUND_MODE_BLOCKS_ADOPT = "OUTBOUND_MODE_BLOCKS_ADOPT"
ADOPT_LINK_TARGET_NOT_FOUND = "ADOPT_LINK_TARGET_NOT_FOUND"
ADOPT_LINK_TARGET_CHANGED = "ADOPT_LINK_TARGET_CHANGED"
ADOPT_LINK_HOST_HAS_OBJECT = "ADOPT_LINK_HOST_HAS_OBJECT"
ADOPT_LINK_BUSY = "ADOPT_LINK_BUSY"
REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0 = "REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0"
RESTORE_VORGANG_NICHT_GEFUNDEN = "RESTORE_VORGANG_NICHT_GEFUNDEN"
ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL = "ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL"
@@ -129,6 +150,10 @@ class ErrorCode(str, Enum):
UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN = "UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN"
UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT = "UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT"
UPSTREAM_REQUEST_FAILED = "UPSTREAM_REQUEST_FAILED"
API_RIGHTS_MISSING = "API_RIGHTS_MISSING"
GATEWAY_NAME_INVALID = "GATEWAY_NAME_INVALID"
GATEWAY_UNKNOWN = "GATEWAY_UNKNOWN"
GATEWAY_MUTE_LIMIT = "GATEWAY_MUTE_LIMIT"
VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN = "VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN"
VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN = "VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN"
VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN = "VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN"
@@ -149,6 +174,7 @@ class ErrorCode(str, Enum):
OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED"
OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY"
OPEN_CLEANUP_BLOCKS_DEPLOY = "OPEN_CLEANUP_BLOCKS_DEPLOY"
UPSTREAM_REJECTED = "UPSTREAM_REJECTED"
INVALID_CHARACTERS = "INVALID_CHARACTERS"
SERVICE_NOT_FOUND = "SERVICE_NOT_FOUND"
@@ -157,7 +183,14 @@ class ErrorCode(str, Enum):
API_TOKEN_LIMIT_REACHED = "API_TOKEN_LIMIT_REACHED"
OFFSITE_TEST_FAILED = "OFFSITE_TEST_FAILED"
OIDC_TEST_FAILED = "OIDC_TEST_FAILED"
VARIABLES_MISSING = "VARIABLES_MISSING"
VARIABLE_INVALID = "VARIABLE_INVALID"
OVERLAY_INVALID = "OVERLAY_INVALID"
NOTIFICATION_CHANNEL_INVALID = "NOTIFICATION_CHANNEL_INVALID"
NOTIFICATION_CHANNEL_NOT_FOUND = "NOTIFICATION_CHANNEL_NOT_FOUND"
NOTIFICATION_CHANNEL_NAME_TAKEN = "NOTIFICATION_CHANNEL_NAME_TAKEN"
NOTIFICATION_CHANNEL_TEST_FAILED = "NOTIFICATION_CHANNEL_TEST_FAILED"
NOTIFICATION_WEBHOOK_URL_INVALID = "NOTIFICATION_WEBHOOK_URL_INVALID"
OVERLAY_SUBNET_LOCKED = "OVERLAY_SUBNET_LOCKED"
OVERLAY_NAME_TAKEN = "OVERLAY_NAME_TAKEN"
OVERLAY_PEER_NOT_FOUND = "OVERLAY_PEER_NOT_FOUND"
@@ -167,6 +200,66 @@ class ErrorCode(str, Enum):
API_TOKEN_NOT_FOUND = "API_TOKEN_NOT_FOUND"
API_TOKEN_SCOPE_FORBIDDEN = "API_TOKEN_SCOPE_FORBIDDEN"
API_TOKEN_SCOPE_TOO_HIGH = "API_TOKEN_SCOPE_TOO_HIGH"
WINDOW_NOT_FOUND = "WINDOW_NOT_FOUND"
WINDOW_INVALID = "WINDOW_INVALID"
WINDOW_NAME_TAKEN = "WINDOW_NAME_TAKEN"
WINDOW_IN_USE = "WINDOW_IN_USE"
WINDOW_EXCEPTION_INVALID = "WINDOW_EXCEPTION_INVALID"
WINDOW_EXCEPTION_EXISTS = "WINDOW_EXCEPTION_EXISTS"
WINDOW_EXCEPTION_NOT_FOUND = "WINDOW_EXCEPTION_NOT_FOUND"
ROLLOUT_PLAN_NOT_FOUND = "ROLLOUT_PLAN_NOT_FOUND"
ROLLOUT_INVALID = "ROLLOUT_INVALID"
ROLLOUT_NAME_TAKEN = "ROLLOUT_NAME_TAKEN"
ROLLOUT_PLAN_INVALID = "ROLLOUT_PLAN_INVALID"
ROLLOUT_PREVIEW_STALE = "ROLLOUT_PREVIEW_STALE"
ROLLOUT_PLAN_BUSY = "ROLLOUT_PLAN_BUSY"
ROLLOUT_GROUP_IN_USE = "ROLLOUT_GROUP_IN_USE"
ROLLOUT_RUN_NOT_FOUND = "ROLLOUT_RUN_NOT_FOUND"
ROLLOUT_ALREADY_ACTIVE = "ROLLOUT_ALREADY_ACTIVE"
ROLLOUT_CONFIRM_NAME = "ROLLOUT_CONFIRM_NAME"
ROLLOUT_ACTION_NOT_ALLOWED = "ROLLOUT_ACTION_NOT_ALLOWED"
ROLLOUT_HOST_NOT_IN_RUN = "ROLLOUT_HOST_NOT_IN_RUN"
ROLLOUT_HOST_RUNNING = "ROLLOUT_HOST_RUNNING"
ROLLOUT_HOST_FAILED = "ROLLOUT_HOST_FAILED"
ANALYSE_ADRESSE_UNGUELTIG = "ANALYSE_ADRESSE_UNGUELTIG"
ANALYSE_FAMILIEN_MISCHUNG = "ANALYSE_FAMILIEN_MISCHUNG"
ANALYSE_INTERFACE_UNBEKANNT = "ANALYSE_INTERFACE_UNBEKANNT"
ANALYSE_PARAMETER_UNGUELTIG = "ANALYSE_PARAMETER_UNGUELTIG"
ANALYSE_FIREWALL_NICHT_LESBAR = "ANALYSE_FIREWALL_NICHT_LESBAR"
# Rule cleanup (1.31, services/rule_cleanup_service.py)
BEREINIGUNG_NICHT_GEFUNDEN = "BEREINIGUNG_NICHT_GEFUNDEN"
BEREINIGUNG_AUSWAHL_UNGUELTIG = "BEREINIGUNG_AUSWAHL_UNGUELTIG"
BEREINIGUNG_VORSCHAU_VERALTET = "BEREINIGUNG_VORSCHAU_VERALTET"
BEREINIGUNG_REGEL_NICHT_AENDERBAR = "BEREINIGUNG_REGEL_NICHT_AENDERBAR"
BEREINIGUNG_OFFENE_OPERATION = "BEREINIGUNG_OFFENE_OPERATION"
BEREINIGUNG_ZU_VIELE_REGELN = "BEREINIGUNG_ZU_VIELE_REGELN"
BEREINIGUNG_BESTAETIGUNG_FEHLT = "BEREINIGUNG_BESTAETIGUNG_FEHLT"
BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN = "BEREINIGUNG_VERWALTUNGSQUELLEN_FEHLEN"
BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG = "BEREINIGUNG_CLUSTER_UNVOLLSTAENDIG"
BEREINIGUNG_FRIST_ABGELAUFEN = "BEREINIGUNG_FRIST_ABGELAUFEN"
BEREINIGUNG_NICHT_BESTAETIGBAR = "BEREINIGUNG_NICHT_BESTAETIGBAR"
BEREINIGUNG_LIVESTAND_ABWEICHEND = "BEREINIGUNG_LIVESTAND_ABWEICHEND"
BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR = "BEREINIGUNG_FIREWALL_NICHT_ERREICHBAR"
BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT = "BEREINIGUNG_WARTEZEIT_NICHT_ERREICHT"
BEREINIGUNG_SNAPSHOT_FEHLT = "BEREINIGUNG_SNAPSHOT_FEHLT"
BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR = "BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR"
BEREINIGUNG_AKTION_NICHT_MOEGLICH = "BEREINIGUNG_AKTION_NICHT_MOEGLICH"
BEREINIGUNG_NICHT_AUSGEFUEHRT = "BEREINIGUNG_NICHT_AUSGEFUEHRT"
BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN = "BEREINIGUNG_ANWENDEN_FEHLGESCHLAGEN"
BEREINIGUNG_AUSGANG_UNKLAR = "BEREINIGUNG_AUSGANG_UNKLAR"
BEREINIGUNG_ABGEBROCHEN_VERWAIST = "BEREINIGUNG_ABGEBROCHEN_VERWAIST"
BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET = "BEREINIGUNG_BESTAETIGUNGSTEXT_VERALTET"
BEREINIGUNG_EINSTELLUNG_UNGUELTIG = "BEREINIGUNG_EINSTELLUNG_UNGUELTIG"
CONFIG_INVALID = "CONFIG_INVALID"
CONFIG_TOO_LARGE = "CONFIG_TOO_LARGE"
CONFIG_UNSUPPORTED_VERSION = "CONFIG_UNSUPPORTED_VERSION"
CONFIG_PLAN_STALE = "CONFIG_PLAN_STALE"
CONFIG_BLOCKED = "CONFIG_BLOCKED"
CONFIG_APPLY_SELFCHECK = "CONFIG_APPLY_SELFCHECK"
CONFIG_BUSY = "CONFIG_BUSY"
ZIEL_HOST_NACH_AUSROLLEN_FEST = "ZIEL_HOST_NACH_AUSROLLEN_FEST"
ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET = "ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET"
VPN_DEPLOY_LAEUFT_BEREITS = "VPN_DEPLOY_LAEUFT_BEREITS"
def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:
+609
View File
@@ -0,0 +1,609 @@
"""The ``gui-gateway`` container: firewall web interfaces in their own browser origin (1.27).
Runs from the backend image (``python -m app.gui_gateway``) behind the proxy, which terminates TLS for the
slot ports and sets ``X-Onx-Slot`` (overwriting any client value). It never touches the database or the
secret stores; it only talks to the backend's internal API (``/internal/gui/*``, shared GUI_GATEWAY_TOKEN)
and, per gateway session, to exactly one firewall.
Security model (see the 1.27 design):
* Every slot port is its own origin. The firewall GUI therefore never runs in the OPNexus origin.
* Cookies never cross: the browser's ``Cookie`` header is not forwarded (it carries the OPNexus session),
and the firewall's ``Set-Cookie`` never reaches the browser — a server-side jar per gateway session
(memory only) holds them. The browser only gets ``__Host-onx-gw-<slot>``.
* Each upstream request is built from scratch: method, path and query from the browser, a small header
allow-list, the jar's cookies; target origin, pinned certificate and overlay proxy from the backend.
Absolute-form targets, CONNECT/TRACE, ``Upgrade`` and service-worker script requests are refused.
* ``Sec-Fetch-Site`` must be same-origin/none (same-site only for the redeem step), fail-closed;
non-GET/HEAD requests also need the slot's own ``Origin``.
* ``Location`` pointing anywhere but the firewall is dropped; responses get CSP, X-Frame-Options DENY,
``Cache-Control: no-store``.
* At most 3 failed GUI logins per firewall in 5 minutes (protects OPNexus' API access from the firewall's
login lockout, which would see all attempts coming from OPNexus).
* Idle and absolute limits; every 20 s the backend re-checks the session (OPNexus session, admin role,
switch, host, and that the firewall's address, pinned certificate and overlay route are still the ones the
session started with).
* At most MAX_SESSIONS sessions at once: expired ones are cleared first, a new session is refused otherwise —
a running session is never evicted by a newer one. Logs carry method, path (no query), status and duration only.
OPNsense behaviour this relies on (opnsense/core, src/www/authgui.inc, master as of 2026-10-21):
* session_auth(): a login is ``isset($_POST['login']) && !empty($_POST['usernamefld']) &&
!empty($_POST['passwordfld'])`` posted to the current URL (login form has no action). Success sends a
``Location:`` header and exits (302); a failure re-renders the login form (200) with "Wrong username or
password." and an auth_log entry.
* check_security_http_referer_enforcement() only runs when HTTP_REFERER is set, and wants the firewall's
own host name/IP — so the gateway sends no Referer (and no Origin) upstream at all.
* check_security_dns_rebind() accepts an IP address or the configured host name as Host — the gateway
sends the Host of the host record's base URL (httpx derives it from the URL).
"""
import asyncio
import hashlib
import logging
import os
import secrets
import ssl
import time
from collections import deque
from dataclasses import dataclass, field
from http.cookies import CookieError, SimpleCookie
from typing import Awaitable, Callable
from urllib.parse import parse_qs, urlsplit
import httpx
logger = logging.getLogger("gui_gateway")
# httpx/httpcore log full request URLs (query strings) at INFO/DEBUG: never in this process.
for _noisy in ("httpx", "httpcore"):
logging.getLogger(_noisy).setLevel(logging.WARNING)
LISTEN_PORT = 8080
COOKIE_PREFIX = "__Host-onx-gw-"
OPNEXUS_SESSION_COOKIE = "__Host-opnexus-session"
BODY_LIMIT = 64 * 1024 * 1024 # config restore uploads
LOGIN_FAILURE_LIMIT = 3
LOGIN_FAILURE_WINDOW = 300.0
UPSTREAM_TIMEOUT = 60.0
MAX_SESSIONS = 64
# Request headers copied from the browser; everything else (Cookie, Authorization, Origin, Referer,
# X-Forwarded-*, hop-by-hop, Sec-*, ...) is dropped.
_FORWARD_REQUEST_HEADERS = {
"accept", "accept-language", "content-type", "user-agent", "x-requested-with", "x-csrftoken", "range",
}
# Response headers copied from the firewall. Set-Cookie goes to the jar, Location is rewritten.
_FORWARD_RESPONSE_HEADERS = {
"content-type", "content-length", "content-encoding", "content-disposition", "content-language",
"content-range", "accept-ranges", "content-security-policy",
}
_GATEWAY_CSP = ("connect-src 'self'; form-action 'self'; frame-ancestors 'none'; base-uri 'self'; "
"object-src 'none'; worker-src 'none'")
_SECURITY_HEADERS = [
(b"content-security-policy", _GATEWAY_CSP.encode()),
(b"x-frame-options", b"DENY"),
(b"x-content-type-options", b"nosniff"),
(b"cache-control", b"no-store"),
(b"pragma", b"no-cache"),
(b"referrer-policy", b"same-origin"),
(b"cross-origin-opener-policy", b"same-origin"),
(b"cross-origin-resource-policy", b"same-origin"),
(b"permissions-policy", b"camera=(), microphone=(), geolocation=(), payment=(), usb=()"),
]
def _sha(value: str) -> str:
return hashlib.sha256(value.encode()).hexdigest()
def _origin_of(url: str) -> str | None:
try:
parts = urlsplit(url)
port = parts.port
except ValueError:
return None
if parts.scheme not in ("http", "https") or not parts.hostname:
return None
default = 443 if parts.scheme == "https" else 80
host = parts.hostname.lower()
if ":" in host:
host = f"[{host}]"
return f"{parts.scheme}://{host}" + (f":{port}" if port and port != default else "")
def slot_origins_from(sites: str) -> dict[int, set[str]]:
"""OPNEXUS_GUI_SITES -> {port: {normalised origins}} (https only, explicit port)."""
slots: dict[int, set[str]] = {}
for raw in sites.split(","):
raw = raw.strip().rstrip("/")
try:
parts = urlsplit(raw)
port = parts.port
except ValueError:
continue
if parts.scheme == "https" and parts.hostname and port:
slots.setdefault(port, set()).add(_origin_of(raw))
return slots
@dataclass
class GatewaySession:
gui_session_id: str
session_hash: str
slot: int
upstream_origin: str
ca_pem: str | None
proxy_url: str | None
idle_timeout: float
absolute_deadline: float
revalidate_every: float
target_fp: str | None = None
last_activity: float = field(default_factory=time.monotonic)
next_check: float = 0.0
jar: dict[str, str] = field(default_factory=dict)
client: httpx.AsyncClient | None = None
class BackendClient:
"""The backend's internal GUI API. Raises httpx.HTTPError when the backend is unreachable."""
def __init__(self, base_url: str, token: str, transport: httpx.AsyncBaseTransport | None = None):
self._client = httpx.AsyncClient(base_url=base_url, headers={"X-Onx-Gateway-Token": token},
timeout=10.0, transport=transport)
async def redeem(self, ticket_hash: str, session_hash: str, slot: int, client_ip: str | None) -> dict:
resp = await self._client.post("/internal/gui/redeem", json={
"ticket_hash": ticket_hash, "session_hash": session_hash, "slot": slot, "client_ip": client_ip})
if resp.status_code == 200:
return resp.json()
if resp.status_code == 403:
return {"ok": False, "reason": (resp.json().get("detail") or {}).get("reason", "denied")}
resp.raise_for_status()
return {"ok": False, "reason": f"http_{resp.status_code}"}
async def revalidate(self, gs: GatewaySession) -> dict:
resp = await self._client.post("/internal/gui/revalidate",
json={"gui_session_id": gs.gui_session_id, "session_hash": gs.session_hash,
"target_fp": gs.target_fp})
resp.raise_for_status()
return resp.json()
async def end(self, gs: GatewaySession, reason: str) -> None:
try:
await self._client.post("/internal/gui/end", json={
"gui_session_id": gs.gui_session_id, "session_hash": gs.session_hash, "reason": reason})
except httpx.HTTPError as exc:
logger.warning("Could not report the end of a GUI session (%s)", type(exc).__name__)
async def aclose(self) -> None:
await self._client.aclose()
def build_upstream_client(gs: GatewaySession) -> httpx.AsyncClient:
"""Same transport rules as opnsense_client._client: pinned certificate (or system CAs), overlay proxy."""
verify: ssl.SSLContext | bool
if gs.ca_pem:
from app.services.tls_pin_service import pinned_context # no database layer behind this import
verify = pinned_context(gs.ca_pem)
else:
verify = True
return httpx.AsyncClient(verify=verify, proxy=gs.proxy_url, timeout=UPSTREAM_TIMEOUT,
follow_redirects=False, trust_env=False)
class LoginLimiter:
"""At most LOGIN_FAILURE_LIMIT failed GUI logins per firewall within LOGIN_FAILURE_WINDOW seconds."""
def __init__(self, clock: Callable[[], float] = time.monotonic):
self._failures: dict[str, deque] = {}
self._clock = clock
def _recent(self, key: str) -> deque:
now = self._clock()
q = self._failures.setdefault(key, deque())
while q and now - q[0] > LOGIN_FAILURE_WINDOW:
q.popleft()
return q
def blocked(self, key: str) -> bool:
return len(self._recent(key)) >= LOGIN_FAILURE_LIMIT
def record_failure(self, key: str) -> None:
self._recent(key).append(self._clock())
def is_login_attempt(method: str, content_type: str, body: bytes) -> bool:
if method != "POST":
return False
if content_type.lower().startswith("multipart/form-data"):
# PHP fills $_POST from multipart bodies too, so authgui.inc accepts a login posted this way.
return all(f'name="{field}"'.encode() in body for field in ("login", "usernamefld", "passwordfld"))
if not content_type.lower().startswith("application/x-www-form-urlencoded"):
return False
try:
form = parse_qs(body.decode("latin-1"), keep_blank_values=True)
except ValueError:
return False
return "login" in form and bool(form.get("usernamefld", [""])[0]) and bool(form.get("passwordfld", [""])[0])
def rewrite_location(location: str, upstream_origin: str) -> str | None:
"""Relative target for the browser, or None when the redirect leaves the firewall."""
location = location.strip()
if not location or "\\" in location or any(ord(c) < 0x20 for c in location):
return None
if location.startswith("//"):
location = "https:" + location
parts = urlsplit(location)
if parts.scheme or parts.netloc:
if _origin_of(location) != _origin_of(upstream_origin):
return None
path = parts.path or "/"
return path + (f"?{parts.query}" if parts.query else "") + (f"#{parts.fragment}" if parts.fragment else "")
return location
def _page(title: str, text_de: str, text_en: str) -> bytes:
return (f"<!doctype html><html lang=\"de\"><head><meta charset=\"utf-8\"><title>{title}</title></head>"
f"<body style=\"font-family:sans-serif;max-width:40rem;margin:3rem auto\"><h1>{title}</h1>"
f"<p>{text_de}</p><p lang=\"en\">{text_en}</p></body></html>").encode()
_SESSION_GONE = _page("OPNexus", "Diese Sitzung ist abgelaufen oder ungültig. Öffne die Oberfläche erneut aus OPNexus.",
"This session has expired or is invalid. Open the interface again from OPNexus.")
_FORBIDDEN = _page("OPNexus", "Diese Anfrage wurde vom GUI-Gateway abgelehnt.", "This request was refused by the GUI gateway.")
_BAD_REDIRECT = _page("OPNexus", "Die Firewall wollte auf eine fremde Adresse umleiten; die Umleitung wurde verworfen.",
"The firewall tried to redirect to a foreign address; the redirect was dropped.")
_LOGIN_LOCKED = _page("OPNexus", "Zu viele fehlgeschlagene Anmeldungen an dieser Firewall. Bitte warte 5 Minuten.",
"Too many failed sign-ins at this firewall. Please wait 5 minutes.")
_UNAVAILABLE = _page("OPNexus", "Die Firewall oder OPNexus ist gerade nicht erreichbar.",
"The firewall or OPNexus is not reachable right now.")
_TOO_MANY = _page("OPNexus", "Es laufen schon zu viele Firewall-Oberflächen gleichzeitig. Versuche es später erneut.",
"Too many firewall interfaces are open at the same time. Try again later.")
_REDEEMED = (b"<!doctype html><html><head><meta charset=\"utf-8\"><meta http-equiv=\"refresh\" content=\"0;url=/\">"
b"<title>OPNexus</title></head><body><a href=\"/\">&rarr;</a></body></html>")
class GuiGateway:
def __init__(self, *, backend: BackendClient, slot_origins: dict[int, set[str]],
upstream_factory: Callable[[GatewaySession], httpx.AsyncClient] = build_upstream_client,
clock: Callable[[], float] = time.monotonic, wall_clock: Callable[[], float] = time.time):
self.backend = backend
self.slot_origins = slot_origins
self.upstream_factory = upstream_factory
self.clock = clock
self.wall_clock = wall_clock
self.sessions: dict[str, GatewaySession] = {} # sha256(gateway cookie) -> session
self.limiter = LoginLimiter(clock)
self._loop_task: asyncio.Task | None = None
# ── ASGI ──────────────────────────────────────────────────────────────
async def __call__(self, scope, receive, send):
if scope["type"] == "lifespan":
await self._lifespan(receive, send)
return
if scope["type"] != "http":
# WebSocket and anything else: not supported (design: no WebSocket).
if scope["type"] == "websocket":
await send({"type": "websocket.close", "code": 1008})
return
started = time.monotonic()
status = await self._handle(scope, receive, send)
# Method, path and status only: never the query string, headers or bodies.
logger.info("%s %s %d %.0fms", scope.get("method", "?"), scope.get("path", "?")[:200], status,
(time.monotonic() - started) * 1000)
async def _lifespan(self, receive, send):
while True:
message = await receive()
if message["type"] == "lifespan.startup":
self._loop_task = asyncio.create_task(self._revalidate_loop())
await send({"type": "lifespan.startup.complete"})
elif message["type"] == "lifespan.shutdown":
if self._loop_task:
self._loop_task.cancel()
for key in list(self.sessions):
await self._drop(key, "shutdown")
await self.backend.aclose()
await send({"type": "lifespan.shutdown.complete"})
return
async def _revalidate_loop(self):
while True:
await asyncio.sleep(10)
try:
await self.sweep()
except Exception: # never let the loop die
logger.exception("GUI session sweep failed")
async def sweep(self) -> None:
"""Idle/absolute limits and the periodic backend re-check for every gateway session."""
for key, gs in list(self.sessions.items()):
reason = self._expired(gs)
if reason:
await self._drop(key, reason)
elif self.clock() >= gs.next_check:
await self._recheck(key, gs)
# ── helpers ───────────────────────────────────────────────────────────
@staticmethod
def _headers(scope) -> dict[str, str]:
result: dict[str, str] = {}
for k, v in scope.get("headers", []):
name = k.decode("latin-1").lower()
value = v.decode("latin-1")
if name in result:
value = f"{result[name]}{'; ' if name == 'cookie' else ', '}{value}"
result[name] = value
return result
@staticmethod
def _cookies(header: str) -> dict[str, str]:
cookies: dict[str, str] = {}
for part in header.split(";"):
name, sep, value = part.strip().partition("=")
if sep and name and name not in cookies:
cookies[name] = value
return cookies
async def _respond(self, send, status: int, body: bytes = b"", headers: list | None = None,
content_type: bytes = b"text/html; charset=utf-8") -> int:
out = [(b"content-type", content_type), (b"content-length", str(len(body)).encode())]
out += list(headers or []) + _SECURITY_HEADERS
await send({"type": "http.response.start", "status": status, "headers": out})
await send({"type": "http.response.body", "body": body})
return status
def _expired(self, gs: GatewaySession) -> str | None:
if self.wall_clock() >= gs.absolute_deadline:
return "absolute"
if self.clock() - gs.last_activity >= gs.idle_timeout:
return "idle"
return None
async def _drop(self, key: str, reason: str, *, report: bool = True) -> None:
gs = self.sessions.pop(key, None)
if gs is None:
return
gs.jar.clear()
if gs.client is not None:
await gs.client.aclose()
if report:
await self.backend.end(gs, reason)
async def _recheck(self, key: str, gs: GatewaySession) -> bool | None:
"""True = still valid, False = ended (dropped), None = backend unreachable (fail closed, keep for retry)."""
try:
result = await self.backend.revalidate(gs)
except httpx.HTTPError as exc:
logger.warning("GUI session re-check failed (%s)", type(exc).__name__)
return None
if not result.get("ok"):
await self._drop(key, str(result.get("reason", "revoked")), report=False) # backend already ended it
return False
gs.absolute_deadline = min(gs.absolute_deadline, float(result.get("absolute_deadline", gs.absolute_deadline)))
gs.idle_timeout = float(result.get("idle_timeout_seconds", gs.idle_timeout))
gs.next_check = self.clock() + gs.revalidate_every
return True
async def _clear_expired(self) -> None:
for key, gs in list(self.sessions.items()):
reason = self._expired(gs)
if reason:
await self._drop(key, reason)
def _has_room(self, browser_hash: str, slot: int) -> bool:
"""Room for one more session; the one this redeem would replace (same browser session and slot) does not count."""
replaced = sum(1 for g in self.sessions.values() if g.slot == slot and g.session_hash == browser_hash)
return len(self.sessions) - replaced < MAX_SESSIONS
# ── request handling ──────────────────────────────────────────────────
async def _handle(self, scope, receive, send) -> int:
method = scope.get("method", "")
path = scope.get("path", "")
raw_path = scope.get("raw_path") or path.encode()
headers = self._headers(scope)
if method in ("CONNECT", "TRACE"):
return await self._respond(send, 501, _FORBIDDEN)
# Origin-form only: "GET http://elsewhere/ HTTP/1.1" (absolute-form) or "//host" never reach upstream.
if not path.startswith("/") or path.startswith("//") or not raw_path.startswith(b"/") \
or raw_path.startswith(b"//") or "\\" in path:
return await self._respond(send, 400, _FORBIDDEN)
if "upgrade" in headers or "upgrade" in headers.get("connection", "").lower():
return await self._respond(send, 400, _FORBIDDEN)
if headers.get("service-worker", "").lower() == "script":
return await self._respond(send, 403, _FORBIDDEN)
try:
slot = int(headers.get("x-onx-slot", ""))
except ValueError:
slot = 0
own_origins = self.slot_origins.get(slot)
if not own_origins:
return await self._respond(send, 400, _FORBIDDEN) # not coming through the proxy's slot block
site = headers.get("sec-fetch-site", "").strip().lower()
if path == "/__onx/redeem":
if method != "GET" or site not in ("same-site", "same-origin", "none"):
return await self._respond(send, 403, _FORBIDDEN)
return await self._redeem(scope, headers, slot, send)
if site not in ("same-origin", "none"):
return await self._respond(send, 403, _FORBIDDEN) # missing header = old/odd client: fail closed
if method not in ("GET", "HEAD"):
origin = headers.get("origin", "")
if _origin_of(origin) not in own_origins:
return await self._respond(send, 403, _FORBIDDEN)
if path.startswith("/__onx/"):
return await self._respond(send, 404, _FORBIDDEN)
cookies = self._cookies(headers.get("cookie", ""))
token = cookies.get(f"{COOKIE_PREFIX}{slot}", "")
key = _sha(token) if token else ""
gs = self.sessions.get(key) if key else None
if gs is None or gs.slot != slot:
return await self._respond(send, 401, _SESSION_GONE)
reason = self._expired(gs)
if reason:
await self._drop(key, reason)
return await self._respond(send, 401, _SESSION_GONE)
if self.clock() >= gs.next_check:
state = await self._recheck(key, gs)
if state is False:
return await self._respond(send, 401, _SESSION_GONE)
if state is None:
return await self._respond(send, 503, _UNAVAILABLE)
gs.last_activity = self.clock()
return await self._proxy(scope, receive, send, headers, gs)
async def _redeem(self, scope, headers, slot: int, send) -> int:
query = parse_qs(scope.get("query_string", b"").decode("latin-1"))
ticket = (query.get("t") or [""])[0]
browser_session = self._cookies(headers.get("cookie", "")).get(OPNEXUS_SESSION_COOKIE, "")
if not ticket or len(ticket) > 128 or not browser_session:
return await self._respond(send, 403, _SESSION_GONE)
client_ip = (headers.get("x-forwarded-for", "").split(",")[0].strip() or None)
# Room first, before the ticket is burned: expired sessions go, running ones are never evicted.
await self._clear_expired()
if not self._has_room(_sha(browser_session), slot):
return await self._respond(send, 503, _TOO_MANY)
try:
result = await self.backend.redeem(_sha(ticket), _sha(browser_session), slot, client_ip and client_ip[:45])
except httpx.HTTPError as exc:
logger.warning("Ticket redeem failed (%s)", type(exc).__name__)
return await self._respond(send, 503, _UNAVAILABLE)
if not result.get("ok"):
return await self._respond(send, 403, _SESSION_GONE)
# One browser session, one gateway session per slot: an older one in this slot is replaced.
for old_key, old in list(self.sessions.items()):
if old.slot == slot and old.session_hash == _sha(browser_session):
await self._drop(old_key, "superseded", report=False)
value = secrets.token_urlsafe(32)
gs = GatewaySession(
gui_session_id=result["gui_session_id"], session_hash=_sha(browser_session), slot=slot,
upstream_origin=result["upstream_origin"], ca_pem=result.get("ca_pem"), proxy_url=result.get("proxy_url"),
idle_timeout=float(result["idle_timeout_seconds"]), absolute_deadline=float(result["absolute_deadline"]),
revalidate_every=float(result.get("revalidate_seconds", 20)), target_fp=result.get("target_fp"),
)
if len(self.sessions) >= MAX_SESSIONS: # filled up while the backend was answering
await self.backend.end(gs, "revoked")
return await self._respond(send, 503, _TOO_MANY)
gs.last_activity = self.clock()
gs.next_check = gs.last_activity + gs.revalidate_every
self.sessions[_sha(value)] = gs
cookie = f"{COOKIE_PREFIX}{slot}={value}; Path=/; Secure; HttpOnly; SameSite=Strict"
# 200 + meta refresh instead of a 303: the follow-up request then starts in this origin
# (Sec-Fetch-Site: same-origin) and the ticket leaves the address bar.
return await self._respond(send, 200, _REDEEMED, headers=[
(b"set-cookie", cookie.encode()), (b"clear-site-data", b'"cache", "storage"'),
])
async def _read_body(self, receive) -> bytes | None:
chunks, size = [], 0
while True:
message = await receive()
if message["type"] == "http.disconnect":
return None
chunk = message.get("body", b"")
size += len(chunk)
if size > BODY_LIMIT:
return None
chunks.append(chunk)
if not message.get("more_body"):
return b"".join(chunks)
def _store_cookies(self, gs: GatewaySession, response: httpx.Response) -> None:
for header in response.headers.get_list("set-cookie"):
jar = SimpleCookie()
try:
jar.load(header)
except CookieError:
continue
for name, morsel in jar.items():
expired = morsel["max-age"] in ("0",) or morsel.value in ("", "deleted")
if not expired and morsel["expires"]:
try:
from email.utils import parsedate_to_datetime
expired = parsedate_to_datetime(morsel["expires"]).timestamp() <= self.wall_clock()
except (TypeError, ValueError):
pass
if expired:
gs.jar.pop(name, None)
else:
gs.jar[name] = morsel.value
async def _proxy(self, scope, receive, send, headers: dict, gs: GatewaySession) -> int:
length = headers.get("content-length", "")
if length.isdigit() and int(length) > BODY_LIMIT:
return await self._respond(send, 413, _FORBIDDEN)
body = await self._read_body(receive)
if body is None:
return await self._respond(send, 413, _FORBIDDEN)
method = scope["method"]
login = is_login_attempt(method, headers.get("content-type", ""), body)
if login and self.limiter.blocked(gs.upstream_origin):
return await self._respond(send, 429, _LOGIN_LOCKED)
out_headers = {k: v for k, v in headers.items() if k in _FORWARD_REQUEST_HEADERS}
out_headers["accept-encoding"] = "identity"
if gs.jar:
out_headers["cookie"] = "; ".join(f"{k}={v}" for k, v in gs.jar.items())
query = scope.get("query_string", b"").decode("latin-1")
# The path exactly as the browser sent it (still percent-encoded): the decoded scope["path"] would
# turn %3F/%23 into query/fragment, %2F into a separator and decode %25xx a second time upstream.
raw_path = (scope.get("raw_path") or scope["path"].encode()).split(b"?", 1)[0].decode("latin-1")
url = gs.upstream_origin + raw_path + (f"?{query}" if query else "")
if gs.client is None:
gs.client = self.upstream_factory(gs)
try:
request = gs.client.build_request(method, url, headers=out_headers, content=body)
except httpx.InvalidURL:
return await self._respond(send, 400, _FORBIDDEN) # e.g. control characters in the path
try:
response = await gs.client.send(request, stream=True)
except (httpx.HTTPError, ssl.SSLError, OSError) as exc:
logger.warning("Upstream request failed (%s)", type(exc).__name__)
return await self._respond(send, 502, _UNAVAILABLE)
try:
self._store_cookies(gs, response)
if login and not (300 <= response.status_code < 400):
self.limiter.record_failure(gs.upstream_origin)
out = []
for name, value in response.headers.multi_items():
lname = name.lower()
if lname in _FORWARD_RESPONSE_HEADERS:
out.append((lname.encode("latin-1"), value.encode("latin-1")))
if "location" in response.headers:
target = rewrite_location(response.headers["location"], gs.upstream_origin)
if target is None:
await response.aclose()
return await self._respond(send, 502, _BAD_REDIRECT)
out.append((b"location", target.encode("latin-1")))
await send({"type": "http.response.start", "status": response.status_code, "headers": out + _SECURITY_HEADERS})
async for chunk in response.aiter_raw():
await send({"type": "http.response.body", "body": chunk, "more_body": True})
await send({"type": "http.response.body", "body": b""})
return response.status_code
finally:
await response.aclose()
def create_app(transport: httpx.AsyncBaseTransport | None = None) -> GuiGateway:
token = os.environ.get("GUI_GATEWAY_TOKEN", "")
if len(token) < 32:
raise SystemExit("GUI_GATEWAY_TOKEN (at least 32 characters) is required")
slots = slot_origins_from(os.environ.get("OPNEXUS_GUI_SITES", ""))
if not slots:
raise SystemExit("OPNEXUS_GUI_SITES must list the https slot origins")
backend = BackendClient(os.environ.get("GUI_BACKEND_URL", "http://backend:8000"), token, transport)
return GuiGateway(backend=backend, slot_origins=slots)
def main() -> None:
import uvicorn
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
# h11: an absolute-form target stays visible in scope["path"] (and is refused); access_log off because
# uvicorn's access log would contain query strings (the redeem ticket).
uvicorn.run(create_app(), host="0.0.0.0", port=LISTEN_PORT, http="h11", ws="none", access_log=False,
server_header=False, date_header=False, proxy_headers=False, lifespan="on", log_config=None)
if __name__ == "__main__":
main()
+69 -3
View File
@@ -18,7 +18,7 @@ from app.security_middleware import OriginCheckMiddleware, SecurityHeadersMiddle
from app.config import settings
from app.database import SessionLocal
from app import migrate
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router, tokens as tokens_router, metrics as metrics_router, mcp as mcp_router, compliance as compliance_router, audit_export as audit_export_router, backups as backups_router, oidc as oidc_router, governance as governance_router, overlay as overlay_router
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router, tokens as tokens_router, metrics as metrics_router, mcp as mcp_router, compliance as compliance_router, audit_export as audit_export_router, backups as backups_router, oidc as oidc_router, governance as governance_router, overlay as overlay_router, variables as variables_router
from app.routers import one_to_one
from app.services.one_to_one_deploy_service import sweep_expired_one_to_one_deployments
from app.routers import outbound_nat
@@ -46,15 +46,25 @@ from app.version import APP_VERSION
from app.routers import dashboard as dashboard_router
from app.routers import cluster as cluster_router
from app.routers import services as services_router
from app.routers import gateways as gateways_router
from app.routers import resource_samples as resource_samples_router
from app.services import resource_monitor_service
from app.routers import scheduled_maintenance as scheduled_maintenance_router
from app.routers import maintenance_windows as maintenance_windows_router
from app.routers import rollouts as rollouts_router
from app.routers import rule_analysis as rule_analysis_router
from app.routers import rule_cleanup as rule_cleanup_router
from app.routers import config_code as config_code_router
from app.routers import gui_access as gui_access_router
from app.services import scheduled_maintenance_service
from app.services import rollout_engine
from app.services import compliance_service
from app.services import audit_export_service
from app.services import overlay_service
from app.services import overlay_service, variables_service
from app.services import backup_schedule_service
from app.services import carp_event_service, carp_maintenance_window_service
from app.services import rule_history_service
from app.services import rule_cleanup_service
# Without this, Python's root logger defaults to WARNING with no handler,
# so every logger.info() call in the 5 background loops below (CARP role
@@ -135,6 +145,13 @@ async def _deploy_revert_sweep_loop():
logger.info("Auto-reverted %d unconfirmed policy-hierarchy deployment(s)", len(reverted))
except Exception:
logger.exception("Policy-hierarchy deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await rule_cleanup_service.sweep_expired_cleanups(db)
if reverted:
logger.info("Regel-Bereinigung: %d Operation(en) automatisch zurückgenommen/abgeschlossen", len(reverted))
except Exception:
logger.exception("Rule-cleanup revert sweep failed")
await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
@@ -228,6 +245,9 @@ async def _resource_poll_loop():
deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
if deleted:
logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
pruned = await rule_history_service.prune(db)
if pruned:
logger.info("Regelverlauf-Retention: %d alte Einträge gelöscht", pruned)
except Exception:
logger.exception("Resource-Poll fehlgeschlagen")
await asyncio.sleep(settings.resource_poll_interval_seconds)
@@ -254,6 +274,12 @@ async def _scheduled_maintenance_poll_loop():
except Exception:
logger.exception("Geplante-Wartung-Poll fehlgeschlagen")
in_progress = 0
# Firmware rollouts (1.25) only ever create schedules; the next tick of the lines above fires them.
try:
async with SessionLocal() as db:
in_progress += await rollout_engine.advance_runs(db)
except Exception:
logger.exception("Rollout-Poll fehlgeschlagen")
# A handover run is stepped quickly (the partner should take over within a minute or two);
# otherwise the relaxed cadence is plenty.
await asyncio.sleep(_RUNNING_MAINTENANCE_POLL_SECONDS if in_progress else _SCHEDULED_MAINTENANCE_POLL_SECONDS)
@@ -310,8 +336,31 @@ async def _backup_poll_loop():
await asyncio.sleep(_BACKUP_POLL_SECONDS)
_RULE_OBSERVE_FIRST_RUN_SECONDS = 150
async def _rule_observe_loop():
"""Ninth background loop: reads the rules and their counters of every enabled firewall (two GETs each),
keeps the usage sampling current and records what changed since the last look (services/rule_history_service.py).
Writes only to the database. The first look at a firewall only stores the starting point."""
await asyncio.sleep(_RULE_OBSERVE_FIRST_RUN_SECONDS)
while True:
try:
async with SessionLocal() as db:
result = await rule_history_service.observe_all(db)
if result["events"] or result["failed"]:
logger.info("Regelverlauf: %s", result)
except Exception:
logger.exception("Regelverlauf fehlgeschlagen")
await asyncio.sleep(max(settings.rule_observe_interval_seconds, settings.rule_observe_min_interval_seconds))
@asynccontextmanager
async def lifespan(app: FastAPI):
conflicts = settings.gui_origin_conflicts()
if conflicts:
# A GUI-gateway slot must never be a trusted OPNexus origin (CORS with credentials).
raise RuntimeError(f"OPNEXUS_GUI_SITES overlaps FRONTEND_ORIGIN: {', '.join(conflicts)} — refusing to start")
await migrate.run()
async with SessionLocal() as db:
await auth_service.bootstrap_admin(db)
@@ -335,6 +384,7 @@ async def lifespan(app: FastAPI):
compliance_task = asyncio.create_task(_compliance_poll_loop())
audit_export_task = asyncio.create_task(_audit_export_poll_loop())
backup_task = asyncio.create_task(_backup_poll_loop())
rule_observe_task = asyncio.create_task(_rule_observe_loop())
try:
yield
finally:
@@ -346,7 +396,8 @@ async def lifespan(app: FastAPI):
compliance_task.cancel()
audit_export_task.cancel()
backup_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task):
rule_observe_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task, rule_observe_task):
try:
await task
except asyncio.CancelledError:
@@ -396,6 +447,12 @@ async def _safe_validation_error(request: Request, exc: RequestValidationError):
)
@app.exception_handler(variables_service.MissingVariables)
async def _missing_variables(request: Request, exc: variables_service.MissingVariables):
"""A deploy that needs a value no target host has defined is refused before anything is written."""
return JSONResponse(status_code=422, content={"detail": {"code": ErrorCode.VARIABLES_MISSING.value, "params": {"details": str(exc)}}})
@app.exception_handler(DBAPIError)
async def _database_text_errors(request: Request, exc: DBAPIError):
"""A NUL byte in any text field cannot be stored by Postgres and used to
@@ -451,6 +508,7 @@ app.include_router(backups_router.router, prefix="/api")
app.include_router(oidc_router.router, prefix="/api")
app.include_router(governance_router.router, prefix="/api")
app.include_router(overlay_router.router, prefix="/api")
app.include_router(variables_router.router, prefix="/api")
app.include_router(logs.router, prefix="/api")
app.include_router(groups.router, prefix="/api")
app.include_router(aliases.router, prefix="/api")
@@ -503,9 +561,17 @@ app.include_router(restore_router.router, prefix="/api")
app.include_router(dashboard_router.router, prefix="/api")
app.include_router(cluster_router.router, prefix="/api")
app.include_router(services_router.router, prefix="/api")
app.include_router(gateways_router.router, prefix="/api")
app.include_router(network_router.router, prefix="/api")
app.include_router(snapshots_router.router, prefix="/api")
app.include_router(network_drift_router.router, prefix="/api")
app.include_router(diagnostics_router.router, prefix="/api")
app.include_router(resource_samples_router.router, prefix="/api")
app.include_router(scheduled_maintenance_router.router, prefix="/api")
app.include_router(maintenance_windows_router.router, prefix="/api")
app.include_router(rollouts_router.router, prefix="/api")
app.include_router(rule_analysis_router.router, prefix="/api")
app.include_router(rule_cleanup_router.router, prefix="/api")
app.include_router(config_code_router.router, prefix="/api")
app.include_router(gui_access_router.router, prefix="/api")
app.include_router(gui_access_router.internal_router)
+20
View File
@@ -93,3 +93,23 @@ __all__ += ["HostResourceSample"]
from app.models.scheduled_maintenance import ScheduledMaintenance
__all__ += ["ScheduledMaintenance"]
from app.models.maintenance_window import MaintenanceWindow, MaintenanceWindowException
__all__ += ["MaintenanceWindow", "MaintenanceWindowException"]
from app.models.rollout import RolloutPlan, RolloutPlanWave, RolloutRun, RolloutRunHost, RolloutRunWave, rollout_wave_groups
__all__ += ["RolloutPlan", "RolloutPlanWave", "RolloutRun", "RolloutRunHost", "RolloutRunWave", "rollout_wave_groups"]
from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
__all__ += ["GuiAccessSettings", "GuiSession", "GuiTicket"]
from app.models.rule_history import LiveRuleState, RuleChangeEvent
__all__ += ["LiveRuleState", "RuleChangeEvent"]
from app.models.rule_cleanup import RuleCleanupOperation, RuleCleanupItem
__all__ += ["RuleCleanupOperation", "RuleCleanupItem"]
# Every mapped class must be registered here: a script that imports only app.models (python -m app.reset_password,
# app.reset_2fa) otherwise fails at the first query because OpnsenseHost refers to classes nobody imported.
from app.models.host_variable import HostVariable
__all__ += ["HostVariable"]
+34
View File
@@ -0,0 +1,34 @@
"""Stabile Schlüssel für „Konfiguration als Code“ (1.32).
Regeln und NAT-Objekte haben außer der UUID keinen natürlichen Schlüssel, die
Beschreibung ist nicht eindeutig. `config_key` ist die Identität in der
YAML-Datei: lesbar, vom Betreiber vergebbar, je Tabelle eindeutig. Bestand und
neue Objekte ohne eigenen Schlüssel bekommen `<präfix>-<12 Hex der UUID>`.
"""
import re
import uuid
from sqlalchemy import String, event
from sqlalchemy.orm import Mapped, mapped_column
CONFIG_KEY_RE = re.compile(r"^[a-z0-9][a-z0-9._-]{0,63}$")
def config_key_column() -> Mapped[str]:
return mapped_column(String(64), unique=True)
def default_config_key(prefix: str, object_id: uuid.UUID) -> str:
return f"{prefix}-{object_id.hex[:12]}"
def register_config_key(model: type, prefix: str) -> None:
"""Fills an empty `config_key` just before INSERT (and the id, which is
otherwise only assigned while the INSERT is built)."""
@event.listens_for(model, "before_insert")
def _fill(_mapper, _connection, target) -> None:
if target.id is None:
target.id = uuid.uuid4()
if not target.config_key:
target.config_key = default_config_key(prefix, target.id)
+12 -1
View File
@@ -1,5 +1,5 @@
"""Separation of duties (singleton row): may operators roll changes out themselves?"""
from sqlalchemy import Boolean, Integer
from sqlalchemy import JSON, Boolean, Integer
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
@@ -11,3 +11,14 @@ class GovernanceSettings(Base):
id: Mapped[int] = mapped_column(Integer, primary_key=True)
# False: an operator prepares changes (edit, preview) but only an admin may deploy them.
operators_may_deploy: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Rule cleanup (1.31). At most this many rules per cleanup operation (default 10, ceiling 25).
cleanup_max_rules: Mapped[int] = mapped_column(Integer, nullable=False, default=10)
# A rule must have been switched off (and confirmed) this many days before it may be deleted (minimum 1).
cleanup_delete_min_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7)
# IPs/CIDRs the admin manages the firewalls from; no rule that could match them may be cleaned up. Empty list =
# the cleanup is locked (fail closed). Only an admin may change it.
cleanup_management_sources: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# Extra TCP ports the firewalls are managed on (besides the API port of base_url, 443, 80 and 22): a rule that
# could match a management source on one of them is protected too. Only an admin may change it.
cleanup_management_ports: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
+65
View File
@@ -0,0 +1,65 @@
"""GUI access to managed firewalls through the separate gui-gateway container (1.27).
Settings are a singleton row (id fixed at 1), off by default. Tickets and gateway
sessions only ever store hashes: the raw ticket travels once through the browser,
the OPNexus session is referenced by the same SHA-256 that `sessions.token_hash` holds.
"""
import uuid
from datetime import datetime
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class GuiAccessSettings(Base):
__tablename__ = "gui_access_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
idle_timeout_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=15, server_default="15")
absolute_timeout_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=60, server_default="60")
slot_count: Mapped[int] = mapped_column(Integer, nullable=False, default=4, server_default="4")
updated_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
class GuiTicket(Base):
"""One-time, 30-second ticket, bound to the OPNexus session, host and slot."""
__tablename__ = "gui_tickets"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
ticket_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True)
session_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False, index=True
)
slot: Mapped[int] = mapped_column(Integer, nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
redeemed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
class GuiSession(Base):
"""A redeemed ticket: one firewall GUI open in one slot of one browser session."""
__tablename__ = "gui_sessions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
session_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False, index=True
)
slot: Mapped[int] = mapped_column(Integer, nullable=False)
client_ip: Mapped[str | None] = mapped_column(String(45), nullable=True)
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_checked_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
end_reason: Mapped[str | None] = mapped_column(String(32), nullable=True)
+4
View File
@@ -44,3 +44,7 @@ class OpnsenseHost(Base):
policy_hierarchy_group: Mapped["PolicyHierarchyGroup | None"] = relationship(
"PolicyHierarchyGroup", back_populates="hosts"
)
# Values for {{NAME}} placeholders (services/variables_service.py); always loaded with the host.
variables: Mapped[list["HostVariable"]] = relationship(
"HostVariable", lazy="selectin", cascade="all, delete-orphan", order_by="HostVariable.name"
)
+18
View File
@@ -0,0 +1,18 @@
"""Per-host values for ``{{NAME}}`` placeholders in centrally authored objects (see services/variables_service.py)."""
import uuid
from sqlalchemy import ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostVariable(Base):
__tablename__ = "host_variables"
__table_args__ = (UniqueConstraint("host_id", "name", name="uq_host_variable"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True)
name: Mapped[str] = mapped_column(String(41), nullable=False)
value: Mapped[str] = mapped_column(String(255), nullable=False)
+65
View File
@@ -0,0 +1,65 @@
"""Recurring maintenance windows (1.25): "every Sunday 02:00-05:00 Europe/Berlin",
"first Tuesday of the month 22:00 for 3 h". A window on its own changes nothing on any
firewall; rollout plans (models/rollout.py) use it to decide WHEN they may start work on a host.
Times are wall-clock times in the window's own IANA time zone, so a window stays at 02:00
local time across daylight-saving changes (see services/window_service.py for the rules).
"""
import uuid
from datetime import date, datetime, time, timezone
from sqlalchemy import JSON, Boolean, Date, DateTime, ForeignKey, Integer, String, Time, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
def _now() -> datetime:
return datetime.now(timezone.utc)
class MaintenanceWindow(Base):
__tablename__ = "maintenance_windows"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(120), unique=True)
timezone: Mapped[str] = mapped_column(String(64))
recurrence: Mapped[str] = mapped_column(String(16)) # weekly | monthly
# weekly: weekdays 0 (Monday) .. 6 (Sunday). JSON columns get a Python default only (Alembic gotcha, see 0007).
weekdays: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# monthly: "nth_weekday" (month_nth 1-4 or -1 = last, month_weekday 0-6) or "day" (month_day 1-28)
month_mode: Mapped[str | None] = mapped_column(String(16), nullable=True)
month_nth: Mapped[int | None] = mapped_column(Integer, nullable=True)
month_weekday: Mapped[int | None] = mapped_column(Integer, nullable=True)
month_day: Mapped[int | None] = mapped_column(Integer, nullable=True)
start_time: Mapped[time] = mapped_column(Time)
duration_minutes: Mapped[int] = mapped_column(Integer)
# After this many minutes into the window no NEW host action starts (running ones finish).
start_cutoff_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=60, server_default="60")
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
created_by: Mapped[str] = mapped_column(String(150))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
exceptions: Mapped[list["MaintenanceWindowException"]] = relationship(
"MaintenanceWindowException", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
order_by="MaintenanceWindowException.occurrence_date",
)
class MaintenanceWindowException(Base):
"""One occurrence skipped or moved. Keyed by the occurrence's LOCAL date."""
__tablename__ = "maintenance_window_exceptions"
__table_args__ = (UniqueConstraint("window_id", "occurrence_date", name="uq_window_exception_date"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
window_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("maintenance_windows.id", ondelete="CASCADE"), index=True
)
occurrence_date: Mapped[date] = mapped_column(Date)
kind: Mapped[str] = mapped_column(String(8)) # skip | move
moved_start: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
reason: Mapped[str] = mapped_column(String(300), default="")
actor: Mapped[str] = mapped_column(String(150))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+56 -1
View File
@@ -6,7 +6,7 @@ OPNsense API secrets stay off the database entirely.
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, Boolean, DateTime, Integer, String, UniqueConstraint
from sqlalchemy import JSON, Boolean, DateTime, Float, Integer, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
@@ -17,6 +17,13 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
DEFAULT_SERVICE_WATCHLIST = (
"unbound", "dnsmasq", "dhcpd", "dhcpd6", "kea-dhcp4", "kea-dhcp6", "ntpd", "openvpn", "wireguard", "strongswan",
)
CHANNEL_TYPES = ("ntfy", "gotify", "telegram", "teams", "matrix")
class NotificationSettings(Base):
"""Single row, id fixed at 1 — see notification_settings_service.get_settings
for the get-or-create helper. The only admin-editable operational config
@@ -52,6 +59,36 @@ class NotificationSettings(Base):
carp_auto_revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
scheduled_maintenance_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
backup_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
# Off by default: switching it on for an existing fleet reports every long-standing finding at once.
compliance_alerts_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# Monitoring alerts (1.24). The gateway alert is on by default (a WAN outage is the one thing every
# operator wants to hear about); resource thresholds and service checks are off, like compliance alerts:
# switching them on for an existing fleet may report long-standing conditions all at once.
gateway_down_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
resource_threshold_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
service_down_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# An API key that cannot read gateway / service state makes those alerts silently blind (1.29): say so.
api_rights_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
# Gateways whose alert is switched off, "<host_id>:<gateway name>" (1.29), e.g. an idle second WAN.
gateway_alert_muted: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# Rollouts (1.25): a run paused (gate, failure, window...) or an automatic start did not happen.
rollout_problems_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
# Gateway: OPNsense' own verdict (down / loss / delay) decides "offline"; these two add an
# OPNexus threshold on the measured values (0 = off).
gateway_loss_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=20, server_default="20")
gateway_delay_threshold_ms: Mapped[int] = mapped_column(Integer, nullable=False, default=500, server_default="500")
# Resources (0 = that threshold is off). Load is per CPU thread.
disk_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=90, server_default="90")
memory_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=95, server_default="95")
swap_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=50, server_default="50")
load_per_cpu_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=2.0, server_default="2.0")
# Debouncing: consecutive 60 s resource samples that must agree before a problem opens or closes.
alert_confirm_samples: Mapped[int] = mapped_column(Integer, nullable=False, default=3, server_default="3")
resource_confirm_samples: Mapped[int] = mapped_column(Integer, nullable=False, default=5, server_default="5")
# OPNsense service names that are reported when they stop (the API cannot tell a crash from a stop on purpose).
# NULL = the default list (an old row never got one).
service_watchlist: Mapped[list | None] = mapped_column(JSON, nullable=True, default=lambda: list(DEFAULT_SERVICE_WATCHLIST))
poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
@@ -83,3 +120,21 @@ class NotificationProblem(Base):
resolved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
last_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
resolved_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
class NotificationChannel(Base):
"""An additional delivery channel (ntfy, Gotify, Telegram, Microsoft Teams, Matrix) next to the SMTP
and generic-webhook settings above. Several channels of one type are allowed. ``config`` holds only
non-secret parameters (server URL, topic, chat id, room id); the credential — token, or the whole
webhook URL for Teams — lives in notification_secret_store under ``channel_<id>``, never here."""
__tablename__ = "notification_channels"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(100), unique=True)
channel_type: Mapped[str] = mapped_column(String(16))
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
config: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
+1 -1
View File
@@ -22,7 +22,7 @@ class OidcSettings(Base):
issuer: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
client_id: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
client_secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
scopes: Mapped[str] = mapped_column(String(255), nullable=False, default="openid profile email groups")
scopes: Mapped[str] = mapped_column(String(255), nullable=False, default="openid profile email")
groups_claim: Mapped[str] = mapped_column(String(64), nullable=False, default="groups")
admin_group: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
operator_group: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
+5
View File
@@ -6,6 +6,7 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.config_key import config_key_column, register_config_key
from app.models.one_to_one_target import one_to_one_targets
from app.models.policy import one_to_one_groups
@@ -14,6 +15,7 @@ class OneToOneRule(Base):
__tablename__ = "one_to_one_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
config_key: Mapped[str] = config_key_column()
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
source_net: Mapped[str] = mapped_column(String(128))
@@ -42,3 +44,6 @@ class OneToOneRule(Base):
deployments: Mapped[list["OneToOneDeployment"]] = relationship(
"OneToOneDeployment", back_populates="one_to_one", cascade="all, delete-orphan"
)
register_config_key(OneToOneRule, "nat1")
+5
View File
@@ -6,6 +6,7 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.config_key import config_key_column, register_config_key
from app.models.outbound_nat_target import outbound_nat_targets
from app.models.policy import outbound_nat_groups
@@ -14,6 +15,7 @@ class OutboundNatRule(Base):
__tablename__ = "outbound_nat_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
config_key: Mapped[str] = config_key_column()
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
source_net: Mapped[str] = mapped_column(String(128))
@@ -44,3 +46,6 @@ class OutboundNatRule(Base):
deployments: Mapped[list["OutboundNatDeployment"]] = relationship(
"OutboundNatDeployment", back_populates="outbound_nat", cascade="all, delete-orphan"
)
register_config_key(OutboundNatRule, "out")
@@ -6,6 +6,7 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.config_key import config_key_column, register_config_key
class PolicyHierarchyRule(Base):
@@ -33,6 +34,7 @@ class PolicyHierarchyRule(Base):
__tablename__ = "policy_hierarchy_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
config_key: Mapped[str] = config_key_column()
hierarchy_group_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="CASCADE")
)
@@ -62,3 +64,6 @@ class PolicyHierarchyRule(Base):
deployments: Mapped[list["PolicyHierarchyRuleDeployment"]] = relationship(
"PolicyHierarchyRuleDeployment", back_populates="rule", cascade="all, delete-orphan"
)
register_config_key(PolicyHierarchyRule, "ph")
+5
View File
@@ -6,6 +6,7 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.config_key import config_key_column, register_config_key
from app.models.port_forward_target import port_forward_targets
from app.models.policy import port_forward_groups
@@ -20,6 +21,7 @@ class PortForwardRule(Base):
__tablename__ = "port_forward_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
config_key: Mapped[str] = config_key_column()
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
ipprotocol: Mapped[str] = mapped_column(String(8), default="inet")
@@ -49,3 +51,6 @@ class PortForwardRule(Base):
deployments: Mapped[list["PortForwardDeployment"]] = relationship(
"PortForwardDeployment", back_populates="port_forward", cascade="all, delete-orphan"
)
register_config_key(PortForwardRule, "pf")
+154
View File
@@ -0,0 +1,154 @@
"""Firmware rollouts in waves (1.25): plans (what, in which order, when) and runs (what happened).
A plan's waves point at device groups ("rings": canary -> pilot -> prod). A run freezes the plan
(``plan_snapshot``) and the resolved hosts (``RolloutRunHost``) when it starts, so later edits of
the plan or of a group never change a run that is under way. The run never fires firmware actions
itself: every host step is an ordinary ScheduledMaintenance row (origin="rollout",
auto_verify=True) linked from ``RolloutRunHost.scheduled_maintenance_id`` — see
services/rollout_service.py.
"""
import uuid
from datetime import datetime, timezone
from sqlalchemy import (JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Table, Text,
UniqueConstraint)
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
def _now() -> datetime:
return datetime.now(timezone.utc)
rollout_wave_groups = Table(
"rollout_wave_groups", Base.metadata,
Column("wave_id", PGUUID(as_uuid=True), ForeignKey("rollout_plan_waves.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class RolloutPlan(Base):
__tablename__ = "rollout_plans"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(120), unique=True)
description: Mapped[str] = mapped_column(Text, default="")
action: Mapped[str] = mapped_column(String(16)) # update | major_update
window_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("maintenance_windows.id", ondelete="SET NULL"), nullable=True, index=True)
# Start a run by itself at every occurrence of the window. Only for "update" and only while the plan
# is activated AND unchanged since (activated_fingerprint).
auto_start: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
gate_checks: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
# Reachability probes run FROM the firewall in the health gate (1.32): [{type: ping|tcp, target, port}].
gate_probes: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
compliance_mode: Mapped[str] = mapped_column(String(8), nullable=False, default="warn", server_default="warn")
activated_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
# Stable id of that account (1.29, no foreign key on purpose: a deleted account stays gone, a new one with the same name is someone else).
activated_by_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
activated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
activated_fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True)
# Why the last automatic start did not happen: {"occurrence", "code", "params", "at"} (notification source).
auto_start_error: Mapped[dict | None] = mapped_column(JSON, nullable=True)
created_by: Mapped[str] = mapped_column(String(150))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
waves: Mapped[list["RolloutPlanWave"]] = relationship(
"RolloutPlanWave", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
order_by="RolloutPlanWave.position")
class RolloutPlanWave(Base):
__tablename__ = "rollout_plan_waves"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
plan_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_plans.id", ondelete="CASCADE"), index=True)
position: Mapped[int] = mapped_column(Integer)
label: Mapped[str] = mapped_column(String(60))
soak_minutes: Mapped[int] = mapped_column(Integer, default=30)
carp_handover: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# Wait for a human "release" before this wave starts (not used for the first wave: starting is the release).
require_approval: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=rollout_wave_groups, lazy="selectin") # noqa: F821
class RolloutRun(Base):
__tablename__ = "rollout_runs"
__table_args__ = (UniqueConstraint("plan_id", "window_occurrence", name="uq_rollout_run_occurrence"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
plan_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("rollout_plans.id", ondelete="SET NULL"), nullable=True, index=True)
plan_name: Mapped[str] = mapped_column(String(120))
plan_snapshot: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
action: Mapped[str] = mapped_column(String(16))
# waiting_window | preflight | wave_running | soaking | gate | awaiting_approval | resuming | paused
# | succeeded | cancelled
status: Mapped[str] = mapped_column(String(20), index=True)
paused_from: Mapped[str | None] = mapped_column(String(20), nullable=True)
current_wave: Mapped[int] = mapped_column(Integer, default=0)
target_version: Mapped[str | None] = mapped_column(String(40), nullable=True)
pause_reason: Mapped[str | None] = mapped_column(String(40), nullable=True)
pause_detail: Mapped[dict | None] = mapped_column(JSON, nullable=True)
started_by: Mapped[str] = mapped_column(String(150))
started_by_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True) # see activated_by_id
trigger: Mapped[str] = mapped_column(String(8)) # manual | window
window_occurrence: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Optimistic lock: every state change bumps it; an engine step computed on an older revision is dropped.
revision: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
progress: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
step_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
waves: Mapped[list["RolloutRunWave"]] = relationship(
"RolloutRunWave", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin", order_by="RolloutRunWave.index")
hosts: Mapped[list["RolloutRunHost"]] = relationship(
"RolloutRunHost", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
order_by=lambda: [RolloutRunHost.wave_index, RolloutRunHost.order])
class RolloutRunWave(Base):
__tablename__ = "rollout_run_waves"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
run_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_runs.id", ondelete="CASCADE"), index=True)
index: Mapped[int] = mapped_column(Integer)
label: Mapped[str] = mapped_column(String(60))
# pending | awaiting_approval | preflight | running | soaking | gate | passed | skipped | cancelled
status: Mapped[str] = mapped_column(String(20), default="pending")
baseline: Mapped[dict | None] = mapped_column(JSON, nullable=True)
gate_result: Mapped[dict | None] = mapped_column(JSON, nullable=True)
soak_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
gate_attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
approved_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
class RolloutRunHost(Base):
__tablename__ = "rollout_run_hosts"
__table_args__ = (UniqueConstraint("run_id", "host_id", name="uq_rollout_run_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
run_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_runs.id", ondelete="CASCADE"), index=True)
wave_index: Mapped[int] = mapped_column(Integer)
host_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="SET NULL"), nullable=True, index=True)
host_name: Mapped[str] = mapped_column(String(120))
cluster_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
order: Mapped[int] = mapped_column(Integer, default=0)
# queued | running | verifying | succeeded | skipped | failed | blocked | cancelled
status: Mapped[str] = mapped_column(String(16), default="queued")
scheduled_maintenance_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("scheduled_maintenance.id", ondelete="SET NULL"), nullable=True, index=True)
version_before: Mapped[str | None] = mapped_column(String(40), nullable=True)
version_after: Mapped[str | None] = mapped_column(String(40), nullable=True)
# {"code": ..., "params": {...}} — translated in the UI (rollouts.hostNote.*)
detail: Mapped[dict | None] = mapped_column(JSON, nullable=True)
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+5
View File
@@ -6,6 +6,7 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.config_key import config_key_column, register_config_key
from app.models.rule_target import rule_targets
from app.models.policy import rule_groups
@@ -18,6 +19,7 @@ class FirewallRule(Base):
__tablename__ = "firewall_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
config_key: Mapped[str] = config_key_column()
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
direction: Mapped[str] = mapped_column(String(8))
@@ -44,3 +46,6 @@ class FirewallRule(Base):
deployments: Mapped[list["RuleDeployment"]] = relationship(
"RuleDeployment", back_populates="rule", cascade="all, delete-orphan"
)
register_config_key(FirewallRule, "r")
+106
View File
@@ -0,0 +1,106 @@
"""Rule cleanup (1.31, B4 part 2): switching rules off or deleting them through preview -> deploy -> confirm.
``RuleCleanupOperation`` is one cleanup run on a host (or a cluster pair, anchored at one host); its id doubles as
the preview token. ``RuleCleanupItem`` is one rule on one node of that run. The columns only carry what the design
(DESIGN-1.31) needs to resume, revert and restore after a crash; the logic lives in services/rule_cleanup_service.py.
"""
import uuid
from datetime import datetime, timezone
from typing import Optional
from sqlalchemy import JSON, DateTime, ForeignKey, Index, Integer, String, Text, text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
# kind: disable | enable (re-enable after a restore) | delete | restore
OPERATION_KINDS = ("disable", "enable", "delete", "restore")
# preview -> running -> awaiting_confirm -> confirmed; the rest are the side exits.
OPERATION_STATUSES = (
"preview", "running", "awaiting_confirm", "confirmed", "partial", "apply_failed", "reverting", "reverted",
"revert_failed", "done", "failed", "cancelled", "expired",
)
ITEM_STATES = (
"queued", "disabling", "disabled", "confirmed", "reverting", "reverted", "deleting", "deleted", "restoring",
"restored", "failed", "unknown", "revert_failed",
# never attempted (the run stopped at an earlier error); not an open state, so the rule is free again
"skipped",
)
# A rule in one of these states is being worked on or its outcome is unresolved: no second item may claim it.
OPEN_ITEM_STATES = ("queued", "disabling", "disabled", "reverting", "deleting", "restoring", "unknown", "revert_failed")
def _now() -> datetime:
return datetime.now(timezone.utc)
class RuleCleanupOperation(Base):
__tablename__ = "rule_cleanup_operations"
__table_args__ = (
Index("ix_rule_cleanup_operations_host", "host_id", "created_at"),
Index("ix_rule_cleanup_operations_status", "status"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
cluster_name: Mapped[Optional[str]] = mapped_column(String(120), nullable=True)
kind: Mapped[str] = mapped_column(String(16), nullable=False)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="preview")
actor: Mapped[str] = mapped_column(String(150), nullable=False)
# The stable account id next to the name (see migration 0026); no foreign key, a deleted account stays gone.
actor_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=_now)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
started_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
# Written at every checkpoint, so another process can tell an abandoned ``running`` operation from a live one.
heartbeat_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
# Auto-revert deadline (disable/enable only); NULL once confirmed or for kinds without a window.
revert_deadline: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
confirmed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
finished_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
# sha256 over everything the preview showed; the deploy recomputes it and refuses on any difference.
preview_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
# unused_days, acknowledged rules (with the text version), protected management sources.
options: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
class RuleCleanupItem(Base):
__tablename__ = "rule_cleanup_items"
__table_args__ = (
Index("ix_rule_cleanup_items_operation", "operation_id"),
# Independent of any lock: one rule on one node can be in only one unfinished item at a time.
Index(
"uq_cleanup_rule_open", "host_id", "rule_uuid", unique=True,
postgresql_where=text("state IN (" + ", ".join(f"'{s}'" for s in OPEN_ITEM_STATES) + ")"),
),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
operation_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rule_cleanup_operations.id", ondelete="CASCADE"), nullable=False)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
seq: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
carp_role: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
state: Mapped[str] = mapped_column(String(16), nullable=False, default="queued")
# What the preview showed.
description: Mapped[str] = mapped_column(Text, nullable=False, default="")
categories: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
confidence: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
row_before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
fp_before: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
# shadowed/duplicate: the covering rule must stay unchanged until the deploy.
cover_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
cover_fp: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
# Flattened getRule, raw answer and firmware; written BEFORE a rule is deleted.
snapshot: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
# Where the rule sat (neighbours in the same prio_group by sort_order), for the restore.
anchor_next_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
anchor_prev_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
prio_group: Mapped[Optional[str]] = mapped_column(String(32), nullable=True)
# delete -> the confirmed disable item it builds on; restore -> the delete item it undoes. Plain id, no foreign key.
source_item_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
restored_uuid: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
changed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
message: Mapped[Optional[str]] = mapped_column(String(1000), nullable=True)
+59
View File
@@ -0,0 +1,59 @@
"""Change history of the rules that are live on a firewall (all rules, also hand-made ones).
``live_rule_states`` is the last observed state per rule (a fingerprint plus the row itself), maintained by the
``rule_observe`` loop; ``rule_change_events`` is what changed between two observations. See
services/rule_history_service.py. Neither table holds anything the firewall itself would not show on its rule page.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import JSON, Boolean, DateTime, ForeignKey, Index, Integer, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class LiveRuleState(Base):
__tablename__ = "live_rule_states"
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True)
# The rule's UUID/label on the firewall; BASELINE_KEY marks "this host has been observed once".
rule_uuid: Mapped[str] = mapped_column(String(64), primary_key=True)
fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
row: Mapped[dict] = mapped_column(JSON, nullable=False)
# False = the rule is gone (kept as a marker so a returning UUID continues the numbering).
present: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Counts the observed changes of this rule; the event of a change gets the new number, which makes the
# observation idempotent when two processes see the same change.
version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
changed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class RuleChangeEvent(Base):
__tablename__ = "rule_change_events"
__table_args__ = (
UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
Index("ix_rule_change_events_rule", "host_id", "rule_uuid", "observed_at"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
seq: Mapped[int] = mapped_column(Integer, nullable=False)
observed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
kind: Mapped[str] = mapped_column(String(16), nullable=False) # added | changed | removed | enabled | disabled
before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
after: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
origin: Mapped[str] = mapped_column(String(16), nullable=False) # opnexus | external
actor: Mapped[Optional[str]] = mapped_column(String(150), nullable=True)
# The audit entry that explains an OPNexus-made change; no foreign key, the audit trail outlives everything.
audit_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
# Start of the time since the previous observation of the host: only audit entries from then on can explain
# this change (a later reconcile must not reach back to an older deployment).
window_start: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
+6 -1
View File
@@ -42,7 +42,12 @@ class ScheduledMaintenance(Base):
host_operation_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
# Opt-in: hand the traffic over to the CARP partner first, run the action, then switch back.
carp_handover: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# Only while status == "running" with a handover: handover_wait | job_wait | leave | leave_wait.
# 1.25: check the result automatically (job_wait -> verify) and close the operation, instead of
# leaving it "submitted" for a human. Every rollout step sets it; off for existing plans.
auto_verify: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# manual | rollout (created by a rollout run, see models/rollout.py RolloutRunHost.scheduled_maintenance_id)
origin: Mapped[str] = mapped_column(String(16), nullable=False, default="manual", server_default="manual")
# Only while status == "running" with a handover or auto_verify: handover_wait | job_wait | verify | leave | leave_wait.
# State lives here, not in memory, so a backend restart in the middle of a run resumes cleanly.
phase: Mapped[str | None] = mapped_column(String(16), nullable=True)
phase_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
@@ -0,0 +1,18 @@
"""Version check configured in the web UI (singleton row, id fixed at 1).
Off by default: OPNexus makes no outgoing request until an admin switches it on. An
UPDATE_CHECK_URL in the server's .env still wins over this row (server-managed)."""
from sqlalchemy import Boolean, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
DEFAULT_FEED_URL = "https://updates.opnexus.dev/latest.json"
class UpdateCheckSettings(Base):
__tablename__ = "update_check_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
feed_url: Mapped[str] = mapped_column(String(500), nullable=False, default=DEFAULT_FEED_URL, server_default=DEFAULT_FEED_URL)
+6
View File
@@ -99,6 +99,12 @@ class Hub:
code, out = self.run("iptables", "-I", *rule)
if code != 0:
return f"iptables: {out}"[:300]
# The hub only needs its own proxy: never route between peers or into the compose network.
fwd = ("FORWARD", "-j", "DROP")
if self.run("iptables", "-C", *fwd)[0] != 0:
code, out = self.run("iptables", "-I", *fwd)
if code != 0:
return f"iptables: {out}"[:300]
return None
def status(self, error: str | None) -> dict:
+79
View File
@@ -0,0 +1,79 @@
"""The three "take over what exists on the firewall" endpoints that aliases, port forwards, 1:1 NAT and
outbound NAT share (services/object_adopt_service.py has the logic and the evidence):
GET <prefix>/unmanaged?host_id=… candidates found live on the host, with reasons for the refused ones
POST <prefix>/adopt take over one object {host_id, uuid}; returns the new central object
POST <prefix>/adopt-all take over everything adoptable {host_id}; returns adopted + skipped
Like rule adoption this only creates central records; nothing is written to the firewall."""
import json
import uuid
from typing import Callable
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.schemas.adopt import AdoptAllRequest, AdoptAllResult, AdoptObjectRequest, UnmanagedObject
from app.services import object_adopt_service
from app.services.adopt_common import LINK_ERRORS, link_api_error
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
from app.services.policy_service import policy_lock
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
def add_adopt_routes(router: APIRouter, *, kind: str, model, load_options: Callable, to_read: Callable, read_model) -> None:
@router.get("/unmanaged", response_model=list[UnmanagedObject])
async def list_unmanaged(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
try:
return await object_adopt_service.list_unmanaged(db, kind, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
@router.post("/adopt", response_model=read_model, status_code=201)
async def adopt(payload: AdoptObjectRequest, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host(db, payload.host_id)
try:
created = await object_adopt_service.adopt(db, kind, host, payload.uuid, link_to=payload.link_to)
except object_adopt_service.NotFoundOnHost as exc:
raise api_error(404, ErrorCode.ADOPT_OBJECT_NOT_FOUND) from exc
except object_adopt_service.NotAdoptable as exc:
raise api_error(409, ErrorCode.OBJECT_NOT_ADOPTABLE, reasons=", ".join(exc.reasons),
reason_details=json.dumps(exc.details, ensure_ascii=False)) from exc
except LINK_ERRORS as exc:
raise link_api_error(exc) from exc
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
await db.commit()
# The new object sits in the session without its relationships loaded; db.get() would return it as-is.
created = (await db.execute(
select(model).where(model.id == created.id).options(*load_options())
.execution_options(populate_existing=True)
)).scalar_one()
await hydrate_hierarchy_targets(db, [created])
return to_read(created, created.targets, created.deployments)
@router.post("/adopt-all", response_model=AdoptAllResult)
async def adopt_all(payload: AdoptAllRequest, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host(db, payload.host_id)
try:
result = await object_adopt_service.adopt_all(db, kind, host, link_existing=payload.link_existing)
except LINK_ERRORS as exc:
raise link_api_error(exc) from exc
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
await db.commit()
return result
+6
View File
@@ -17,6 +17,7 @@ from app.schemas.alias import DeploymentResult, AliasCreate, AliasRead, AliasEdi
from app.services.alias_deploy_service import confirm_alias, deploy_alias, undeploy_alias, revert_alias_deployment
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
from app.routers.adopt_routes import add_adopt_routes
router = APIRouter(prefix="/aliases", tags=["aliases"])
@@ -70,6 +71,9 @@ def _to_read(alias: FirewallAlias, hosts: list[OpnsenseHost], deployments: list[
)
add_adopt_routes(router, kind="alias", model=FirewallAlias, load_options=_load_options, to_read=_to_read, read_model=AliasRead)
@router.get("", response_model=list[AliasRead])
async def list_aliases(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(FirewallAlias).options(*_load_options()))
@@ -201,6 +205,8 @@ async def remove_previous_target(alias_id: uuid.UUID, host_id: uuid.UUID, approv
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in alias.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_alias_item(deployment.host, deployment.opnsense_alias_uuid)
+17 -4
View File
@@ -1,14 +1,15 @@
from datetime import datetime
from fastapi import APIRouter, Depends, Query
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import select, func, or_
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.audit import AuditEvent
from app.services.audit_service import ADMIN_ONLY_DETAILS
router = APIRouter(prefix="/audit", tags=["audit"])
@router.get("")
async def history(kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
async def history(request: Request, kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
status: str | None = None, action: str | None = None, actor: str | None = None,
before: datetime | None = None, since: datetime | None = None,
q: str = Query(default="", max_length=200), offset: int = Query(default=0, ge=0),
@@ -24,5 +25,17 @@ async def history(kind: str | None = None, host_id: str | None = None, entity_id
clauses.append(or_(*(c.ilike('%'+q+'%') for c in [AuditEvent.entity_name, AuditEvent.host_name, AuditEvent.actor, AuditEvent.message])))
total = await db.scalar(select(func.count()).select_from(AuditEvent).where(*clauses))
rows = (await db.scalars(select(AuditEvent).where(*clauses).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).offset(offset).limit(limit))).all()
return {"total": total, "offset": offset, "limit": limit,
"items": [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]}
items = [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]
if not full_view(request):
for item in items:
if isinstance(item.get("details"), dict) and ADMIN_ONLY_DETAILS in item["details"]:
item["details"] = {**{k: v for k, v in item["details"].items() if k != ADMIN_ONLY_DETAILS},
ADMIN_ONLY_DETAILS: "redacted"}
return {"total": total, "offset": offset, "limit": limit, "items": items}
def full_view(request: Request | None) -> bool:
"""An admin account, and with a token only one of scope admin."""
user = getattr(getattr(request, "state", None), "user", None)
token = getattr(getattr(request, "state", None), "api_token", None)
return getattr(user, "role", None) == "admin" and (token is None or token.scope == "admin")
+49 -12
View File
@@ -15,7 +15,31 @@ logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/auth", tags=["auth"])
# Plain name: the session cookie of HTTP-only (lab) deployments. On HTTPS deployments it is NOT accepted any
# more (1.29): it is only still deleted (login, logout) so that a pre-1.27 cookie does not linger.
COOKIE_NAME = "opnexus-session"
# HTTPS deployments: the __Host- prefix makes the browser refuse any cookie of
# this name that is not Secure, has a Domain attribute or a Path other than
# "/". A page on another port of the same host (the GUI gateway slots, any
# other service on that machine) therefore cannot toss a narrower-path cookie
# that shadows the real session, and cannot overwrite it from script (HttpOnly).
SECURE_COOKIE_NAME = "__Host-opnexus-session"
def session_cookie_name() -> str:
return SECURE_COOKIE_NAME if settings.cookie_secure else COOKIE_NAME
def session_token(request: Request) -> str:
"""The session token the browser sent: the __Host- cookie on HTTPS deployments, the plain one otherwise.
Never the other name (1.29): a plain-name cookie can be planted by any plain-HTTP or sibling-subdomain page and
must not be able to carry a session on an HTTPS deployment (the __Host- prefix does not separate PORTS, see
BACKLOG S8)."""
return request.cookies.get(session_cookie_name(), "")
def has_session_cookie(request: Request) -> bool:
return bool(request.cookies.get(SECURE_COOKIE_NAME) or request.cookies.get(COOKIE_NAME))
def client_ip(request: Request) -> str:
@@ -30,20 +54,25 @@ def client_ip(request: Request) -> str:
return request.client.host if request.client else "unknown"
def _set_session_cookie(response: Response, token: str) -> None:
def _set_session_cookie(response: Response, token: str, max_age: int | None = None) -> None:
response.set_cookie(
key=COOKIE_NAME,
key=session_cookie_name(),
value=token,
max_age=settings.session_lifetime,
max_age=max_age or settings.session_lifetime,
httponly=True,
samesite="lax",
secure=settings.cookie_secure,
path="/",
)
if settings.cookie_secure:
# A pre-1.27 plain-name cookie must not linger next to the new one.
response.delete_cookie(key=COOKIE_NAME, path="/", secure=True, httponly=True, samesite="lax")
def _clear_session_cookie(response: Response) -> None:
response.delete_cookie(key=COOKIE_NAME, path="/")
response.delete_cookie(key=COOKIE_NAME, path="/", secure=settings.cookie_secure, httponly=True, samesite="lax")
if settings.cookie_secure:
response.delete_cookie(key=SECURE_COOKIE_NAME, path="/", secure=True, httponly=True, samesite="lax")
def _check_account_budget(user: AdminUser, action: str) -> None:
@@ -71,7 +100,7 @@ async def resolve_user(request: Request, db: AsyncSession) -> AdminUser:
user = await db.get(AdminUser, resolved.id)
if user is not None:
return user
token = request.cookies.get(COOKIE_NAME, "")
token = session_token(request)
user = await auth_service.resolve_session(db, token)
if user is None:
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.NICHT_AUTHENTIFIZIERT)
@@ -110,7 +139,8 @@ async def login(payload: LoginRequest, request: Request, response: Response, db:
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
verified_hash = user.password_hash
if user.totp_enabled:
second_factor_checked = bool(user.totp_enabled) # the state this login is decided on; create_session re-checks it
if second_factor_checked:
# Only revealed after the password was correct, so it cannot be used
# to find out which usernames exist or have 2FA.
if not payload.totp_code:
@@ -124,7 +154,8 @@ async def login(payload: LoginRequest, request: Request, response: Response, db:
auth_service.throttle.reset(ip, payload.username)
failed_since_last_login = auth_service.throttle.pop_failures_since_success(payload.username)
token, expires_at = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip, verified_hash=verified_hash
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip, verified_hash=verified_hash,
verified_totp=second_factor_checked,
)
_set_session_cookie(response, token)
logger.info("Successful login for '%s' from %s", user.username, ip)
@@ -223,13 +254,15 @@ async def totp_enable(
_account_failure(user, "2fa-enable")
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
_account_success(user, "2fa-enable")
# Same row lock as create_session: a sign-in that decided "no second factor needed" is ordered before or after this.
await db.execute(select(AdminUser.id).where(AdminUser.id == user.id).with_for_update())
plain, hashes = totp_service.new_recovery_codes()
totp_service.promote_pending(user.id)
user.totp_enabled = True
user.totp_last_step = step
user.totp_recovery_hashes = hashes
# Other sessions were established without the second factor — end them.
await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
await auth_service.revoke_sessions(db, user.id, keep_token=session_token(request), commit=False)
await db.commit()
logger.info("Two-factor enabled for '%s'", user.username)
return {"enabled": True, "recovery_codes": plain}
@@ -254,7 +287,7 @@ async def totp_disable(
user.totp_enabled = False
user.totp_recovery_hashes = None
user.totp_last_step = 0
await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
await auth_service.revoke_sessions(db, user.id, keep_token=session_token(request), commit=False)
await db.commit()
logger.info("Two-factor disabled for '%s'", user.username)
return {"enabled": False}
@@ -262,14 +295,17 @@ async def totp_disable(
@router.post("/logout")
async def logout(request: Request, response: Response, db: AsyncSession = Depends(get_db)):
token = request.cookies.get(COOKIE_NAME, "")
token = session_token(request)
await auth_service.revoke_session(db, token)
legacy = request.cookies.get(COOKIE_NAME, "")
if settings.cookie_secure and legacy and legacy != token:
await auth_service.revoke_session(db, legacy) # an old plain-name cookie ends with the logout too
_clear_session_cookie(response)
return {"authenticated": False}
@router.get("/me")
async def me(request: Request, db: AsyncSession = Depends(get_db)):
async def me(request: Request, response: Response, db: AsyncSession = Depends(get_db)):
raw = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
if raw is not None:
resolved = await api_token_service.resolve(db, raw, client_ip(request))
@@ -278,7 +314,7 @@ async def me(request: Request, db: AsyncSession = Depends(get_db)):
user, api_token = resolved
return {"authenticated": True, "username": user.username, "role": user.role,
"via": "token", "scope": api_token.scope, "token_name": api_token.name}
token = request.cookies.get(COOKIE_NAME, "")
token = session_token(request)
user = await auth_service.resolve_session(db, token)
if user is None:
return {"authenticated": False}
@@ -305,6 +341,7 @@ async def change_password(
# an attacker may hold) and the replacement session commit as one unit.
user.password_hash = await auth_service.hash_password_async(payload.new_password)
await auth_service.revoke_sessions(db, user.id, commit=False)
await api_token_service.revoke_all(db, user.id, commit=False) # a changed password answers a suspected leak: tokens too
token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=client_ip(request)
)
+14
View File
@@ -37,6 +37,13 @@ acme_accounts_router = APIRouter(prefix="/acme/accounts", tags=["acme"])
acme_certificates_router = APIRouter(prefix="/acme/certificates", tags=["acme"])
def _ensure_host_unchanged_once_deployed(obj, new_host_id: uuid.UUID) -> None:
"""The firewall-local UUID, the stored payload and the success status all belong to the host the object was pushed
to. Re-pointing it would report success for a host that never received it and orphan the object on the old one."""
if new_host_id != obj.host_id and obj.opnsense_uuid:
raise api_error(409, ErrorCode.ZIEL_HOST_NACH_AUSROLLEN_FEST)
async def _get_host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
@@ -82,6 +89,7 @@ async def edit_managed_certificate(cert_id: uuid.UUID, payload: ManagedCertifica
if cert.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
_ensure_host_unchanged_once_deployed(cert, payload.host_id)
for field, value in payload.model_dump(exclude={"expected_updated_at", "private_key"}).items():
setattr(cert, field, value)
if payload.private_key:
@@ -146,6 +154,11 @@ async def edit_acme_account(account_id: uuid.UUID, payload: AcmeAccountEdit, db:
if account.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
_ensure_host_unchanged_once_deployed(account, payload.host_id)
if payload.host_id != account.host_id and await db.scalar(
select(AcmeCertificateRequest.id).where(AcmeCertificateRequest.account_id == account.id).limit(1)
):
raise api_error(409, ErrorCode.ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET)
for field, value in payload.model_dump(exclude={"expected_updated_at", "eab_hmac"}).items():
setattr(account, field, value)
if payload.eab_hmac:
@@ -236,6 +249,7 @@ async def edit_acme_certificate_request(request_id: uuid.UUID, payload: AcmeCert
if req.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
_ensure_host_unchanged_once_deployed(req, payload.host_id)
await _get_account(db, payload.account_id, payload.host_id)
for field, value in payload.model_dump(exclude={"expected_updated_at"}).items():
setattr(req, field, value)
+154
View File
@@ -0,0 +1,154 @@
"""Konfiguration als Code (1.32): YAML-Export, Plan und Übernahme. Schreibt nur in die OPNexus-Datenbank.
Rechte (durchgesetzt in AuthMiddleware/api_token_service, hier nicht noch einmal):
* Export: jede angemeldete Rolle, Token-Scope `read` genügt (nichts, was die GET-Endpunkte nicht schon zeigen).
* Plan: jede Rolle inkl. Nur-Lese und `read`-Token; schreibt nichts (wie eine Vorschau, nicht im Audit).
* Übernehmen: nur Admins (Sitzung oder `admin`-Token): Massenänderung inkl. Prune und Host-Hierarchie-Zuordnung.
"""
import asyncio
import json
from fastapi import APIRouter, Depends, Query, Request
from fastapi.concurrency import run_in_threadpool
from fastapi.responses import Response
from pydantic import BaseModel, ConfigDict, Field, ValidationError
from sqlalchemy.ext.asyncio import AsyncSession
from app.config_code import apply as applier
from app.config_code import plan as planner
from app.config_code import schema
from app.config_code.export import export_yaml
from app.config_code.loader import MAX_BYTES, ConfigError, parse
from app.database import get_db
from app.errors import ErrorCode, api_error
router = APIRouter(prefix="/config", tags=["config"])
# JSON escapes can grow the YAML text (a quote becomes \", a control character \u001b): room for that, but bounded,
# and read before parsing so a huge body never reaches the JSON parser.
MAX_REQUEST_BYTES = 4 * MAX_BYTES
# One plan/apply at a time per process: the pure-Python YAML parser costs seconds for a full-size file, and plan is
# open to read-only accounts and read tokens.
_busy = asyncio.Semaphore(1)
# ... and a short queue: every waiting request holds its body (up to MAX_REQUEST_BYTES) in memory.
MAX_WAITING = 4
_waiting = 0
class _Slot:
"""Admission before the body is read (a waiting request holds its body in memory), then one at a time."""
async def __aenter__(self):
global _waiting
if _waiting >= MAX_WAITING:
raise api_error(429, ErrorCode.CONFIG_BUSY)
_waiting += 1
return self
async def run(self, coro_fn):
async with _busy:
return await coro_fn()
async def __aexit__(self, *exc):
global _waiting
_waiting -= 1
def _encode(text: str) -> bytes:
try:
return text.encode("utf-8")
except UnicodeEncodeError: # a lone surrogate from a JSON \ud800 escape
raise api_error(422, ErrorCode.CONFIG_INVALID, errors=[{"code": "bad_encoding", "path": "", "line": None, "params": {}}]) from None
class ImportRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
yaml: str
prune: bool = False
class ApplyRequest(ImportRequest):
plan_token: str = Field(min_length=64, max_length=64)
def _config_error(exc: ConfigError):
status = 413 if exc.code == "CONFIG_TOO_LARGE" else 422
return api_error(status, ErrorCode(exc.code), errors=exc.errors)
async def _read_body(request: Request, model: type[BaseModel]):
if int(request.headers.get("content-length") or 0) > MAX_REQUEST_BYTES:
raise api_error(413, ErrorCode.CONFIG_TOO_LARGE, errors=[{"code": "too_large", "path": "", "line": None, "params": {"max": MAX_BYTES}}])
chunks, size = [], 0
async for chunk in request.stream():
size += len(chunk)
if size > MAX_REQUEST_BYTES:
raise api_error(413, ErrorCode.CONFIG_TOO_LARGE, errors=[{"code": "too_large", "path": "", "line": None, "params": {"max": MAX_BYTES}}])
chunks.append(chunk)
try:
return model.model_validate(json.loads(b"".join(chunks)))
except (ValueError, ValidationError):
raise api_error(422, ErrorCode.CONFIG_INVALID, errors=[{"code": "bad_request", "path": "", "line": None, "params": {}}]) from None
@router.get("/export")
async def export(kinds: str | None = Query(default=None, max_length=300), db: AsyncSession = Depends(get_db)):
wanted = None
if kinds:
wanted = {k.strip() for k in kinds.split(",") if k.strip()}
if not wanted <= set(schema.KINDS):
raise api_error(422, ErrorCode.CONFIG_INVALID, errors=[{"code": "unknown_kind", "path": "kinds", "line": None, "params": {}}])
text = await export_yaml(db, wanted)
return Response(content=text.encode("utf-8"), media_type="text/yaml; charset=utf-8",
headers={"Content-Disposition": 'attachment; filename="opnexus-config.yaml"', "Cache-Control": "no-store"})
@router.post("/import/plan")
async def plan(request: Request, db: AsyncSession = Depends(get_db)):
async with _Slot() as slot:
body = await _read_body(request, ImportRequest)
raw = _encode(body.yaml)
async def work():
try:
parsed = await run_in_threadpool(parse, raw)
return await planner.build_plan(db, parsed, body.prune)
except ConfigError as exc:
raise _config_error(exc) from None
finally:
await db.rollback() # never leave anything in the session: plan is read-only
result = await slot.run(work)
return result.public()
@router.post("/import/apply")
async def apply(request: Request, db: AsyncSession = Depends(get_db)):
async with _Slot() as slot:
body = await _read_body(request, ApplyRequest)
raw = _encode(body.yaml)
async def work():
try:
parsed = await run_in_threadpool(parse, raw)
result = await applier.apply_config(db, parsed, body.prune, body.plan_token)
except ConfigError as exc:
await db.rollback()
raise _config_error(exc) from None
except applier.PlanStale:
await db.rollback()
raise api_error(409, ErrorCode.CONFIG_PLAN_STALE) from None
except applier.PlanBlocked as exc:
await db.rollback()
raise api_error(409, ErrorCode.CONFIG_BLOCKED, summary=exc.plan.summary()) from None
except applier.SelfCheckFailed as exc:
await db.rollback()
raise api_error(500, ErrorCode.CONFIG_APPLY_SELFCHECK, objects=exc.differences[:20]) from None
except Exception:
await db.rollback()
raise
await db.commit()
return parsed, result
parsed, result = await slot.run(work)
return {"summary": result.summary(), "sha256": parsed.sha256, "kinds": result.touched}
+34
View File
@@ -18,6 +18,8 @@ locks are deliberately NOT included here (no fleet-wide list endpoint
exists for those, only per-host, and it's a rarely-triggered flow) —
a documented v1 gap, not an oversight.
"""
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -46,7 +48,10 @@ from app.models.rule_deployment import RuleDeployment
from app.models.wireguard_tunnel import WireguardTunnel
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
from app.schemas.dashboard import ActivityItem, DashboardHost, DashboardSummary, PendingAction
from app.models.resource_sample import HostResourceSample
from app.services import gateway_health_service, notification_settings_service
from app.services.host_status_service import get_statuses
from app.services.monitor_checks import STALE_AFTER
router = APIRouter(prefix="/dashboard", tags=["dashboard"])
@@ -202,14 +207,43 @@ async def collect_pending_actions(db: AsyncSession) -> list[PendingAction]:
return pending_actions
async def _sample_readings_by_host(db: AsyncSession, hosts, muted: set[str]) -> tuple[dict, dict]:
"""(gateways, missing API rights) from each host's newest resource sample; hosts without a fresh sample are left out."""
gateways: dict = {}
denied: dict = {}
now = datetime.now(timezone.utc)
for host in hosts:
row = (
await db.execute(
select(HostResourceSample.sampled_at, HostResourceSample.raw["gateways"].as_json(), HostResourceSample.raw["denied"].as_json())
.where(HostResourceSample.host_id == host.id)
.order_by(HostResourceSample.sampled_at.desc())
.limit(1)
)
).first()
if row is None or now - row[0] > STALE_AFTER:
continue
if row[1] is not None:
gateways[host.id] = [
g.model_copy(update={"alert_enabled": f"{host.id}:{g.name}" not in muted})
for g in gateway_health_service.from_compact(row[1])
]
if isinstance(row[2], list):
denied[host.id] = [str(x) for x in row[2]]
return gateways, denied
@router.get("", response_model=DashboardSummary)
async def summary(db: AsyncSession = Depends(get_db)):
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
statuses = await get_statuses(hosts)
muted = set((await notification_settings_service.get_settings(db)).gateway_alert_muted or [])
gateways, denied = await _sample_readings_by_host(db, hosts, muted)
host_summaries = [
DashboardHost(
id=host.id, name=host.name, status=statuses.get(host.id, {}).get("status", "offline"),
carp_status=statuses.get(host.id, {}).get("carp_status"), cluster_name=host.cluster_name,
gateways=gateways.get(host.id), api_rights_missing=denied.get(host.id),
)
for host in hosts
]
+99
View File
@@ -0,0 +1,99 @@
import re
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.resource_sample import HostResourceSample
from app.schemas.gateway import ApiRightsState, GatewayAlertState, GatewayAlertUpdate, GatewayHealth
from app.services import audit_service, gateway_health_service, notification_settings_service
from app.services.monitor_checks import STALE_AFTER
from app.services.opnsense_client import OpnsenseAPIError, OpnsenseForbiddenError
router = APIRouter(prefix="/hosts/{host_id}/gateways", tags=["host-gateways"])
# OPNsense gateway names are letters, digits and underscores (WAN_GW, WAN_DHCP6); a little more is allowed.
_GATEWAY_NAME = re.compile(r"[A-Za-z0-9_.-]{1,64}")
MAX_MUTED_GATEWAYS = 200
@router.get("", response_model=list[GatewayHealth])
async def list_gateways(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""Live gateway state (loss, delay, deviation, OPNsense' verdict) of one firewall."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
gateways = await gateway_health_service.get_gateway_health(host)
except OpnsenseForbiddenError as exc:
# The API key may not read the gateway state: say so instead of a generic upstream failure.
raise api_error(403, ErrorCode.API_RIGHTS_MISSING) from exc
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
muted = set((await notification_settings_service.get_settings(db)).gateway_alert_muted or [])
return [g.model_copy(update={"alert_enabled": f"{host.id}:{g.name}" not in muted}) for g in gateways]
@router.get("/api-rights", response_model=ApiRightsState)
async def api_rights(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""Side readings the newest FRESH resource sample could not take because the API key lacks the right
(null = no fresh sample, so nothing is known). Stored data only: no call to the firewall."""
if await db.get(OpnsenseHost, host_id) is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
row = (
await db.execute(
select(HostResourceSample.sampled_at, HostResourceSample.raw["denied"].as_json())
.where(HostResourceSample.host_id == host_id)
.order_by(HostResourceSample.sampled_at.desc())
.limit(1)
)
).first()
if row is None or datetime.now(timezone.utc) - row[0] > STALE_AFTER or not isinstance(row[1], list):
return ApiRightsState(missing=None)
return ApiRightsState(missing=[str(x) for x in row[1]])
@router.put("/{name}/alert", response_model=GatewayAlertState)
async def set_gateway_alert(host_id: uuid.UUID, name: str, payload: GatewayAlertUpdate, db: AsyncSession = Depends(get_db)):
"""Switch the gateway-down alert on or off for ONE gateway (admins only, see api_token_service)."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
if not _GATEWAY_NAME.fullmatch(name):
raise api_error(422, ErrorCode.GATEWAY_NAME_INVALID)
if not payload.enabled:
# Only a gateway the firewall really reported can be muted ("wan_gw" vs "WAN_GW" would silently do nothing).
# Without a usable sample nothing is known, so the request goes through.
known = (
await db.execute(
select(HostResourceSample.raw["gateways"].as_json())
.where(HostResourceSample.host_id == host_id)
.order_by(HostResourceSample.sampled_at.desc())
.limit(1)
)
).scalar()
if isinstance(known, list) and name not in {g.get("name") for g in known if isinstance(g, dict)}:
raise api_error(404, ErrorCode.GATEWAY_UNKNOWN)
row = await notification_settings_service.get_settings(db)
await db.refresh(row, with_for_update=True)
muted = list(row.gateway_alert_muted or [])
key = f"{host.id}:{name}"
if payload.enabled:
muted = [m for m in muted if m != key]
elif key not in muted:
if len(muted) >= MAX_MUTED_GATEWAYS:
raise api_error(422, ErrorCode.GATEWAY_MUTE_LIMIT, limit=str(MAX_MUTED_GATEWAYS))
muted.append(key)
row.gateway_alert_muted = muted # a new list, so the JSON column notices the change
await db.commit()
# The request middleware logs the call itself; this entry says what it did (only the validated name goes in).
await audit_service.persist(
kind="host", action="gateway_alert_on" if payload.enabled else "gateway_alert_off", entity_id=str(host.id),
host_id=str(host.id), host_name=host.name, status="accepted", message=f"Gateway {name}",
)
return GatewayAlertState(name=name, alert_enabled=payload.enabled)
+119 -8
View File
@@ -1,19 +1,102 @@
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it)."""
from fastapi import APIRouter, Depends
from pydantic import BaseModel
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it).
The rule-cleanup settings (1.31) live here too: how many rules one cleanup may touch, how long a rule must stay
switched off before it may be deleted, the management sources (addresses the firewalls are managed from; no rule
that could match them is ever cleaned up) and extra management ports (besides the API port, 443, 80 and 22). Only an admin writes any of it (middleware: ``/api/settings/...`` is
admin-only for operators and tokens below scope admin; checked again here) and every change is audited with the
old and the new values."""
import ipaddress
from typing import Any
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.governance import GovernanceSettings
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.routers.auth import client_ip, current_user
from app.services import audit_service
router = APIRouter(prefix="/settings/governance", tags=["governance"])
MAX_CLEANUP_RULES = 25 # same ceiling as services/rule_cleanup_service.MAX_RULES_CEILING
MAX_DELETE_MIN_DAYS = 365
MAX_MANAGEMENT_SOURCES = 20
MAX_MANAGEMENT_PORTS = 20
class GovernanceRead(BaseModel):
operators_may_deploy: bool
cleanup_max_rules: int = 10
cleanup_delete_min_days: int = 7
# The list itself is shown to admins only; everybody else sees just whether it is set.
cleanup_management_sources: list[str] = Field(default_factory=list)
cleanup_management_sources_configured: bool = False
# Extra management ports (besides API port, 443, 80, 22): also admins only, others see the flag.
cleanup_management_ports: list[int] = Field(default_factory=list)
cleanup_management_ports_configured: bool = False
# The address this request came from, as a suggestion for the management sources (B12; admins only). Only a
# suggestion: the firewalls may see the workstation under another address (NAT, VPN).
your_address: str | None = None
class GovernanceUpdate(BaseModel):
operators_may_deploy: bool | None = None
cleanup_max_rules: int | None = None
cleanup_delete_min_days: int | None = None
cleanup_management_sources: list[str] | None = None
cleanup_management_ports: list[Any] | None = None # validated in clean_management_ports (no bool coercion)
def _bad(field: str, reason: str):
return api_error(422, ErrorCode.BEREINIGUNG_EINSTELLUNG_UNGUELTIG, field=field, reason=reason)
def clean_management_sources(values: list[str]) -> list[str]:
"""IP addresses and CIDR networks only (no host names: they could resolve differently later), canonical form,
without duplicates; not the whole address space, which would protect every rule."""
if len(values) > MAX_MANAGEMENT_SOURCES:
raise _bad("cleanup_management_sources", "too_many")
cleaned: list[str] = []
for raw in values:
text = str(raw).strip()
try:
net = ipaddress.ip_network(text, strict=False)
except ValueError:
raise _bad("cleanup_management_sources", "not_an_address") from None
if net.prefixlen == 0:
raise _bad("cleanup_management_sources", "everything")
entry = str(net.network_address) if net.prefixlen == net.max_prefixlen else str(net)
if entry not in cleaned:
cleaned.append(entry)
return cleaned
def clean_management_ports(values) -> list[int]:
"""TCP ports 1..65535, at most MAX_MANAGEMENT_PORTS, without duplicates, sorted."""
if len(values) > MAX_MANAGEMENT_PORTS:
raise _bad("cleanup_management_ports", "too_many")
ports: set[int] = set()
for raw in values:
text = str(raw).strip()
if isinstance(raw, bool) or not text.isdigit() or not 1 <= int(text) <= 65535:
raise _bad("cleanup_management_ports", "not_a_port")
ports.add(int(text))
return sorted(ports)
def _read(row: GovernanceSettings, user: AdminUser) -> GovernanceRead:
sources = list(row.cleanup_management_sources or [])
ports = list(row.cleanup_management_ports or [])
return GovernanceRead(
operators_may_deploy=row.operators_may_deploy, cleanup_max_rules=row.cleanup_max_rules or 10,
cleanup_delete_min_days=row.cleanup_delete_min_days or 7,
cleanup_management_sources=sources if user.role == "admin" else [],
cleanup_management_sources_configured=bool(sources),
cleanup_management_ports=ports if user.role == "admin" else [],
cleanup_management_ports_configured=bool(ports))
async def get_settings(db: AsyncSession) -> GovernanceSettings:
@@ -25,17 +108,45 @@ async def get_settings(db: AsyncSession) -> GovernanceSettings:
return row
def _suggestion(request: Request, user: AdminUser) -> str | None:
if user.role != "admin":
return None
try:
return str(ipaddress.ip_address(client_ip(request)))
except ValueError:
return None
@router.get("", response_model=GovernanceRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
async def read(request: Request, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Readable by everybody signed in (the UI explains why a deploy was refused)."""
return await get_settings(db)
return _read(await get_settings(db), user).model_copy(update={"your_address": _suggestion(request, user)})
@router.put("", response_model=GovernanceRead)
async def update(payload: GovernanceRead, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
async def update(payload: GovernanceUpdate, request: Request, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
row = await get_settings(db)
changes: dict[str, dict] = {}
if payload.cleanup_max_rules is not None:
if not 1 <= payload.cleanup_max_rules <= MAX_CLEANUP_RULES:
raise _bad("cleanup_max_rules", f"1..{MAX_CLEANUP_RULES}")
if payload.cleanup_delete_min_days is not None:
if not 1 <= payload.cleanup_delete_min_days <= MAX_DELETE_MIN_DAYS:
raise _bad("cleanup_delete_min_days", f"1..{MAX_DELETE_MIN_DAYS}")
sources = clean_management_sources(payload.cleanup_management_sources) if payload.cleanup_management_sources is not None else None
ports = clean_management_ports(payload.cleanup_management_ports) if payload.cleanup_management_ports is not None else None
for field, value in (("cleanup_max_rules", payload.cleanup_max_rules), ("cleanup_delete_min_days", payload.cleanup_delete_min_days),
("cleanup_management_sources", sources), ("cleanup_management_ports", ports)):
if value is not None and value != getattr(row, field):
changes[field] = {"old": getattr(row, field), "new": value}
setattr(row, field, value)
if payload.operators_may_deploy is not None:
row.operators_may_deploy = payload.operators_may_deploy
if changes:
db.add(audit_service.make_event(
kind="settings", action="cleanup_settings", status="accepted", entity_id="governance",
message="Einstellungen der Regel-Bereinigung geändert", details={"changes": changes}))
await db.commit()
return row
return _read(row, user).model_copy(update={"your_address": _suggestion(request, user)})
+9
View File
@@ -99,6 +99,15 @@ async def delete_group(group_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
for assignment in (port_forward_groups, one_to_one_groups, outbound_nat_groups):
if await db.scalar(select(assignment.c.group_id).where(assignment.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET)
# VPN objects target groups too; their link rows are ON DELETE CASCADE, so without this check the group would
# vanish and the VPN would silently lose its targets (B15, found while designing the config import).
from app.models.policy import ipsec_tunnel_groups, openvpn_instance_groups, wireguard_tunnel_groups
for assignment in (ipsec_tunnel_groups, openvpn_instance_groups, wireguard_tunnel_groups):
if await db.scalar(select(assignment.c.group_id).where(assignment.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_VPN_VERWENDET)
from app.models.rollout import rollout_wave_groups
if await db.scalar(select(rollout_wave_groups.c.group_id).where(rollout_wave_groups.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.ROLLOUT_GROUP_IN_USE)
await db.delete(group)
await db.commit()
+190
View File
@@ -0,0 +1,190 @@
"""Firewall GUI access through the gui-gateway container (1.27).
Public API (session cookie, admin only, never API tokens):
* POST /api/hosts/{id}/gui-ticket — one-time ticket + the slot URL the browser opens (noopener).
* GET/PUT /api/settings/gui-access — switch, idle/absolute timeout, number of slots.
Internal API for the gateway only (NOT under /api, so the proxy never forwards it; shared secret
GUI_GATEWAY_TOKEN in the X-Onx-Gateway-Token header, compared in constant time):
* POST /internal/gui/redeem, /internal/gui/revalidate, /internal/gui/end.
The answers never contain firewall credentials: only the firewall origin, its pinned (public) CA
certificate, the overlay proxy address and the timeouts.
"""
import hashlib
import hmac
import uuid
from datetime import datetime, timezone
from typing import Literal
from fastapi import APIRouter, Depends, Header, HTTPException, Request, status
from pydantic import BaseModel, Field, model_validator
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.user import AdminUser
from app.routers.auth import SECURE_COOKIE_NAME, current_user
from app.services import api_token_service, gui_ticket_service as svc
router = APIRouter(tags=["gui-access"])
internal_router = APIRouter(prefix="/internal/gui", include_in_schema=False)
def _require_admin_session(request: Request, user: AdminUser) -> None:
if getattr(request.state, "api_token", None) is not None or api_token_service.bearer_from_header(
request.headers.get("authorization", "")
) is not None:
raise api_error(status.HTTP_403_FORBIDDEN, ErrorCode.GUI_SESSION_REQUIRED)
if user.role != "admin":
raise api_error(status.HTTP_403_FORBIDDEN, ErrorCode.ADMIN_REQUIRED)
class GuiTicketRead(BaseModel):
url: str
slot: int
expires_in: int
@router.post("/hosts/{host_id}/gui-ticket", response_model=GuiTicketRead)
async def create_ticket(host_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user)):
_require_admin_session(request, user)
if not svc.gateway_configured():
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_GATEWAY_NOT_CONFIGURED)
session_cookie = request.cookies.get(SECURE_COOKIE_NAME, "")
if not settings.cookie_secure or not session_cookie:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_REQUIRES_HTTPS)
row = await svc.get_settings(db)
if not row.enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_ACCESS_DISABLED)
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(status.HTTP_404_NOT_FOUND, ErrorCode.HOST_NICHT_GEFUNDEN)
refusal = svc.host_refusal(host)
if refusal:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_HOST_NOT_ELIGIBLE, reason=refusal)
try:
return await svc.issue_ticket(db, user=user, session_token=session_cookie, host=host,
request_host=request.headers.get("host", ""))
except svc.GuiAccessError as exc:
raise api_error(exc.status, exc.code, **exc.params)
class GuiAccessSettingsRead(BaseModel):
enabled: bool
idle_timeout_minutes: int
absolute_timeout_minutes: int
slot_count: int
available: bool
unavailable_reasons: list[str]
slots: list[int]
updated_by: str | None
updated_at: datetime | None
class GuiAccessSettingsUpdate(BaseModel):
enabled: bool
idle_timeout_minutes: int = Field(ge=1, le=240)
absolute_timeout_minutes: int = Field(ge=5, le=720)
slot_count: int = Field(ge=1, le=svc.MAX_SLOTS)
@model_validator(mode="after")
def _idle_within_absolute(self):
if self.idle_timeout_minutes > self.absolute_timeout_minutes:
raise ValueError("idle_timeout_minutes must not exceed absolute_timeout_minutes")
return self
def _settings_read(row) -> dict:
reasons = svc.unavailable_reasons()
return {"enabled": row.enabled, "idle_timeout_minutes": row.idle_timeout_minutes,
"absolute_timeout_minutes": row.absolute_timeout_minutes, "slot_count": row.slot_count,
"available": not reasons, "unavailable_reasons": reasons, "slots": svc.active_slots(row),
"updated_by": row.updated_by, "updated_at": row.updated_at}
@router.get("/settings/gui-access", response_model=GuiAccessSettingsRead)
async def read_settings(request: Request, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin_session(request, user)
row = await svc.get_settings(db)
await db.commit()
return _settings_read(row)
@router.put("/settings/gui-access", response_model=GuiAccessSettingsRead)
async def update_settings(payload: GuiAccessSettingsUpdate, request: Request, db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user)):
_require_admin_session(request, user)
if payload.enabled and svc.unavailable_reasons():
# Never switchable on without HTTPS cookies and a configured gateway.
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_GATEWAY_NOT_CONFIGURED,
reasons=", ".join(svc.unavailable_reasons()))
row = await svc.get_settings(db)
row.enabled = payload.enabled
row.idle_timeout_minutes = payload.idle_timeout_minutes
row.absolute_timeout_minutes = payload.absolute_timeout_minutes
row.slot_count = payload.slot_count
row.updated_by = user.username
row.updated_at = datetime.now(timezone.utc)
if not payload.enabled:
await svc.end_all(db, "disabled")
await db.commit()
return _settings_read(row)
# ── internal: gateway only ────────────────────────────────────────────────
def _gateway_auth(x_onx_gateway_token: str = Header(default="")) -> None:
expected = settings.gui_gateway_token
if len(expected) < svc.MIN_GATEWAY_TOKEN_LENGTH:
raise HTTPException(status_code=404)
# Fixed-length digests: the comparison time does not depend on the presented value.
if not hmac.compare_digest(hashlib.sha256(x_onx_gateway_token.encode()).digest(),
hashlib.sha256(expected.encode()).digest()):
raise HTTPException(status_code=401)
_HEX64 = r"^[0-9a-f]{64}$"
class RedeemRequest(BaseModel):
ticket_hash: str = Field(pattern=_HEX64)
session_hash: str = Field(pattern=_HEX64)
slot: int = Field(ge=1, le=65535)
client_ip: str | None = Field(default=None, max_length=45)
class RevalidateRequest(BaseModel):
gui_session_id: uuid.UUID
session_hash: str = Field(pattern=_HEX64)
# Fingerprint of the target as handed out at redeem time (optional: an older gateway image omits it).
target_fp: str | None = Field(default=None, pattern=_HEX64)
class EndRequest(BaseModel):
gui_session_id: uuid.UUID
session_hash: str = Field(pattern=_HEX64)
reason: Literal["idle", "absolute", "logout", "revoked", "superseded", "shutdown"]
@internal_router.post("/redeem", dependencies=[Depends(_gateway_auth)])
async def internal_redeem(payload: RedeemRequest, db: AsyncSession = Depends(get_db)):
result = await svc.redeem(db, ticket_hash=payload.ticket_hash, session_hash=payload.session_hash,
slot=payload.slot, client_ip=payload.client_ip)
if not result["ok"]:
raise HTTPException(status_code=403, detail={"ok": False, "reason": result["reason"]})
return result
@internal_router.post("/revalidate", dependencies=[Depends(_gateway_auth)])
async def internal_revalidate(payload: RevalidateRequest, db: AsyncSession = Depends(get_db)):
return await svc.revalidate(db, gui_session_id=payload.gui_session_id, session_hash=payload.session_hash,
target_fp=payload.target_fp)
@internal_router.post("/end", dependencies=[Depends(_gateway_auth)])
async def internal_end(payload: EndRequest, db: AsyncSession = Depends(get_db)):
return {"ok": await svc.end(db, gui_session_id=payload.gui_session_id, session_hash=payload.session_hash,
reason=payload.reason)}
+55 -2
View File
@@ -5,7 +5,7 @@ from app.models.port_forward_deployment import PortForwardDeployment
from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.outbound_nat_deployment import OutboundNatDeployment
from fastapi import APIRouter, Depends, HTTPException, Response
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -107,11 +107,37 @@ async def tls_probe(payload: TlsProbeRequest):
raise api_error(502, ErrorCode.TLS_PROBE_FAILED)
def _same_url(a: str, b: str) -> bool:
return a.strip().rstrip("/").lower() == b.strip().rstrip("/").lower()
async def _ensure_url_free(db: AsyncSession, base_url: str, exclude_id: uuid.UUID | None = None) -> None:
"""One firewall, one entry: the same address twice would double every rollout, alert and backup (T4)."""
for other in (await db.scalars(select(OpnsenseHost))).all():
if other.id != exclude_id and _same_url(other.base_url, base_url):
raise api_error(409, ErrorCode.HOST_URL_EXISTS, host=other.name)
def _connection_failure_reason(exc: Exception) -> str:
"""A short machine code for the UI (hosts.connection.<code>): why the first call to the firewall failed."""
text = str(exc)
if isinstance(exc, opnsense_client.OpnsenseForbiddenError):
return "forbidden"
if text.startswith("401 ") or text.startswith("403 "):
return "credentials" if text.startswith("401 ") else "forbidden"
if "TLS-Zertifikat" in text or "CERTIFICATE_VERIFY_FAILED" in text:
return "tls"
if "Verbindung zu" in text:
return "unreachable"
return "other"
@router.post("", response_model=HostRead, status_code=201)
async def create_host(payload: HostCreate, db: AsyncSession = Depends(get_db)):
existing = await db.scalar(select(OpnsenseHost).where(OpnsenseHost.name == payload.name))
if existing:
raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
await _ensure_url_free(db, payload.base_url)
if payload.policy_hierarchy_group_id and not await db.get(PolicyHierarchyGroup, payload.policy_hierarchy_group_id):
raise api_error(400, ErrorCode.POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN)
@@ -130,6 +156,14 @@ async def create_host(payload: HostCreate, db: AsyncSession = Depends(get_db)):
db.add(host)
await db.flush()
opnsense_secret_store.save_secret(host.id, payload.api_secret)
if payload.verify_connection:
# The call that decides "online" in the host list: if it works here, the host shows up online right away.
try:
await opnsense_client.get_vip_status(host)
except opnsense_client.OpnsenseAPIError as exc:
await db.rollback()
opnsense_secret_store.delete_secret(host.id)
raise api_error(422, ErrorCode.HOST_CONNECTION_FAILED, reason=_connection_failure_reason(exc)) from exc
await db.commit()
statuses = await get_statuses([host])
@@ -147,6 +181,11 @@ async def get_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _get_host_or_404(db, host_id)
if (payload.base_url is not None or payload.enabled is not None or payload.clear_cluster_name
or payload.cluster_name is not None):
# An open rule cleanup (1.31) reverts through this address and this cluster pairing: not while it is open.
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
if payload.name is not None:
existing = await db.scalar(
@@ -156,6 +195,7 @@ async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession
raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
host.name = payload.name
if payload.base_url is not None:
await _ensure_url_free(db, payload.base_url, exclude_id=host_id)
host.base_url = payload.base_url
if payload.verify_tls is not None:
host.verify_tls = payload.verify_tls
@@ -183,11 +223,18 @@ async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession
@router.delete("/{host_id}", status_code=204)
async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
async def delete_host(host_id: uuid.UUID, confirm_name: str = Query(default="", max_length=120), db: AsyncSession = Depends(get_db)):
"""Deleting a host drops its stored credentials at once, so the caller must name it (like every other destructive
action in OPNexus), not only a click in the UI: `DELETE /api/hosts/{id}?confirm_name=<name>` (T4)."""
await policy_lock(db)
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
if confirm_name != host.name:
raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
# An open rule cleanup (1.31) holds a revert deadline and snapshots for this host (and its cluster peers).
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
# Both deployment tables FK to opnsense_hosts with ondelete=CASCADE, so
# deleting the host would silently drop OPNexus's only record of what
# it pushed there — the rule/alias itself stays live on the actual
@@ -226,6 +273,12 @@ async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
draft_certs = (await db.scalars(select(ManagedCertificate.id).where(ManagedCertificate.host_id == host_id))).all()
draft_accounts = (await db.scalars(select(AcmeAccount.id).where(AcmeAccount.host_id == host_id))).all()
await db.delete(host)
# Gateway alert switches of this host would otherwise stay in the settings forever (and count against the limit).
from app.services import notification_settings_service
settings_row = await notification_settings_service.get_settings(db)
prefix = f"{host_id}:"
if any(str(m).startswith(prefix) for m in settings_row.gateway_alert_muted or []):
settings_row.gateway_alert_muted = [m for m in settings_row.gateway_alert_muted if not str(m).startswith(prefix)]
await db.commit()
opnsense_secret_store.delete_secret(host_id)
for cert_id in draft_certs:
+4
View File
@@ -16,6 +16,7 @@ from app.services import ipsec_psk_store
from app.services.ipsec_service import effective_hosts, preview, revision, snapshot, workflow, IPSEC_FIELDS
from app.services.ipsec_deploy_service import deploy_tunnel, undeploy_tunnel
from app.services.opnsense_client import OpnsenseAPIError
from app.services.deploy_lease import resource_lease
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/ipsec/tunnels", tags=["ipsec"])
@@ -192,7 +193,10 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
async with resource_lease("ipsec", tunnel.id):
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_child_uuid:
+101 -7
View File
@@ -9,14 +9,14 @@ from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.models.host_operation import HostOperation
from app.services import opnsense_client as client, audit_service, carp_maintenance_window_service
from app.services import opnsense_client as client, audit_service, backup_store, carp_maintenance_window_service, snapshot_service
from app.services.host_maintenance_service import inspect_host, inspect_carp_switch, fingerprint, peers, ACTIVE
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/maintenance", tags=["host-maintenance"])
class Preview(BaseModel):
action: Literal['reboot','update','major_update']
action: Literal['reboot','update','major_update','firmware_rollback']
class Execute(BaseModel):
preview_id: uuid.UUID
@@ -41,10 +41,69 @@ async def host_or_404(db, host_id):
def read_operation(op):
return {"id":op.id,"host_id":op.host_id,"action":op.action,"status":op.status,"created_at":op.created_at,"expires_at":op.expires_at,"actor":op.actor,"message":op.message}
async def snapshot_or_502(db, host):
try: return await inspect_host(db,host)
async def snapshot_or_502(db, host, action=None):
try:
state=await inspect_host(db,host)
if action=='firmware_rollback':
# Read only for this action, so every other preview/status keeps its exact set of firewall calls.
rollback=await snapshot_service.rollback_state(host)
rollback['changes_since']=await _changes_since(db,host,rollback.get('target'))
state['rollback']=rollback
state['actions']['firmware_rollback']=state['actions']['reboot']+(
[] if rollback['supported'] else ['Die Firewall hat keine ZFS-Snapshots (Boot-Environments).'])+(
[] if rollback.get('target') else ['Kein Rückweg-Punkt vorhanden: OPNexus legt ihn vor jedem Firmware-Update an.'])
return state
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
async def _changes_since(db, host, target):
"""OPNexus' own pushes to this host after the rollback point: going back removes them from the firewall."""
if not target:
return []
since=_point_time(target)
if since is None:
return []
out=[]
for kind,model in _DEPLOYMENT_MODELS():
rows=(await db.scalars(select(model).where(model.host_id==host.id, model.deployed_at>since))).all()
out+=[{'kind':kind,'status':r.status,'deployed_at':r.deployed_at.isoformat()} for r in rows]
return sorted(out,key=lambda r:r['deployed_at'])[:50]
def _point_time(target):
try:
return datetime.strptime(target['stamp'],'%Y%m%d-%H%M%S').replace(tzinfo=timezone.utc)
except (KeyError, TypeError, ValueError):
return None
def _DEPLOYMENT_MODELS():
from app.models import (AliasDeployment, OneToOneDeployment, OutboundNatDeployment, PolicyHierarchyRuleDeployment,
PortForwardDeployment, RuleDeployment)
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.models.openvpn_instance_deployment import OpenVpnInstanceDeployment
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
return (('rule',RuleDeployment),('alias',AliasDeployment),('port_forward',PortForwardDeployment),
('one_to_one',OneToOneDeployment),('outbound_nat',OutboundNatDeployment),('policy_hierarchy',PolicyHierarchyRuleDeployment),
('ipsec_tunnel',IpsecTunnelDeployment),('openvpn_instance',OpenVpnInstanceDeployment),('wireguard_tunnel',WireguardTunnelDeployment))
async def _mark_rolled_back(db, host, target):
"""After going back, what OPNexus pushed since the point is no longer on the firewall. A redeploy with an unchanged
payload only re-affirms (writes nothing), so the remembered payload is cleared and the row marked: the next deploy
really writes, and the UI shows the object as not in sync."""
since=_point_time(target)
if since is None:
return 0
count=0
note=f"Firmware-Rückweg auf {target['name']}: Stand auf der Firewall unklar. Live-Stand prüfen und neu ausrollen."
for _kind,model in _DEPLOYMENT_MODELS():
for row in (await db.scalars(select(model).where(model.host_id==host.id, model.deployed_at>since))).all():
row.status='error'; row.message=note[:500]; row.last_payload=None
if hasattr(model,'revert_deadline'): row.revert_deadline=None
count+=1
return count
async def carp_snapshot_or_502(db, host):
try: return await inspect_carp_switch(db,host)
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@@ -62,7 +121,7 @@ async def status(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
async def preview(host_id: uuid.UUID, payload: Preview, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
state=await snapshot_or_502(db,host)
state=await snapshot_or_502(db,host,payload.action)
if state['actions'][payload.action]: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions'][payload.action]))
op=HostOperation(host_id=host.id,host_name=host.name,actor=request.state.user.username,action=payload.action,
expires_at=datetime.now(timezone.utc)+timedelta(minutes=5),fingerprint=fingerprint(state),snapshot=state)
@@ -77,17 +136,52 @@ async def execute(host_id: uuid.UUID, payload: Execute, request: Request, db: As
if not op or op.host_id!=host_id or op.actor!=request.state.user.username: raise api_error(404, ErrorCode.WARTUNGSVORSCHAU_NICHT_GEFUNDEN)
if op.status!='preview' or op.expires_at <= datetime.now(timezone.utc): raise api_error(409, ErrorCode.VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET)
if payload.confirm_name != host.name: raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
state=await snapshot_or_502(db,host)
if op.action not in ('reboot','update','major_update','firmware_rollback'): raise api_error(404, ErrorCode.WARTUNGSVORSCHAU_NICHT_GEFUNDEN)
state=await snapshot_or_502(db,host,op.action)
if state['actions'][op.action]: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions'][op.action]))
if fingerprint(state)!=op.fingerprint: raise api_error(409, ErrorCode.HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
# Consume durably BEFORE the remote request. A timeout is ambiguous, and
# replaying a reboot/update automatically could interrupt recovery.
op.status='starting'; op.started_at=datetime.now(timezone.utc); await db.commit()
await audit_service.persist(kind='host',action=op.action,status='requested',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),details={'snapshot':state})
if op.action in ('update','major_update'):
# Way back first (1.32, B5): a boot environment of the running system. Fail closed: no point, no update.
try:
point=await snapshot_service.create_rollback_point(host,state.get('version'))
except client.OpnsenseAPIError as exc:
op.status='cancelled';op.message='Rückweg-Punkt (Boot-Environment) nicht angelegt: '+str(exc)[:500]+'. Update nicht gestartet.'
await db.commit()
await audit_service.persist(kind='host',action=op.action,status='cancelled',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details={})
return read_operation(op)
op.snapshot={**op.snapshot,'rollback_point':point}
await db.commit()
if op.action=='firmware_rollback':
# Going back discards every change on the firewall since the point: keep a copy of the current config first.
# Fail closed, like the backup before an update.
try:
backup_store.save_snapshot(host.id, await client.download_backup(host))
except client.OpnsenseAPIError as exc:
op.status='cancelled';op.message='Sicherung der aktuellen Konfiguration fehlgeschlagen: '+str(exc)[:500]+'. Rückweg nicht gestartet.'
await db.commit()
await audit_service.persist(kind='host',action=op.action,status='cancelled',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details={})
return read_operation(op)
rollback=state['rollback']
action_fn=lambda h: snapshot_service.activate_and_reboot(h,rollback)
done=(f"Boot-Environment {rollback['target']['name']} aktiviert, Neustart übergeben. Nach dem Start Version prüfen "
f"(erwartet {rollback['target']['version']}) und Wartung abschließen. Alles, was seit dem Rückweg-Punkt auf der "
"Firewall geändert wurde, ist danach nicht mehr vorhanden: zentrale Objekte über die Vorschau neu ausrollen.")
else:
action_fn={'reboot':client.reboot_host,'update':client.update_firmware,'major_update':client.upgrade_firmware}[op.action]
done='An Firewall übergeben. Abschluss und Erreichbarkeit prüfen; anschließend Wartung abschließen.'
if op.action in ('update','major_update') and not (op.snapshot or {}).get('rollback_point'):
done+=' Kein Rückweg-Punkt: die Firewall hat keine ZFS-Snapshots.'
try:
result=await action_fn(host)
op.status='submitted';op.message='An Firewall übergeben. Abschluss und Erreichbarkeit prüfen; anschließend Wartung abschließen.'
op.status='submitted';op.message=done
if op.action=='firmware_rollback':
marked=await _mark_rolled_back(db,host,rollback['target'])
if marked:
op.message+=f' {marked} Deployment(s) als unklar markiert.'
except client.OpnsenseAPIError as exc:
op.status='unknown';op.message='Ausgang unklar: '+str(exc)[:800]+'. Nicht automatisch wiederholen.'
result={}
+244
View File
@@ -0,0 +1,244 @@
"""Recurring maintenance windows and their exceptions (1.25).
Every write here counts as a rollout action (api_token_service.is_rollout): a window decides WHEN
unattended firmware actions may run, so an operator needs the same right as for deploying.
"""
import re
import uuid
from datetime import date, datetime, time, timedelta, timezone
from typing import Literal
from fastapi import APIRouter, Depends, Query, Request
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.maintenance_window import MaintenanceWindow, MaintenanceWindowException
from app.services import audit_service, window_service
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/maintenance-windows", tags=["maintenance-windows"])
_TIME = re.compile(r"^([01]\d|2[0-3]):([0-5]\d)$")
class WindowIn(BaseModel):
name: str = Field(min_length=1, max_length=120)
timezone: str = Field(min_length=1, max_length=64)
recurrence: Literal["weekly", "monthly"]
weekdays: list[int] = Field(default_factory=list, max_length=7)
month_mode: Literal["nth_weekday", "day"] | None = None
month_nth: int | None = None
month_weekday: int | None = None
month_day: int | None = None
start_time: str
duration_minutes: int = Field(ge=15, le=window_service.MAX_DURATION_MINUTES)
start_cutoff_minutes: int = Field(default=60, ge=5, le=window_service.MAX_DURATION_MINUTES)
enabled: bool = True
class ExceptionIn(BaseModel):
occurrence_date: date
kind: Literal["skip", "move"]
moved_start: datetime | None = None
reason: str = Field(default="", max_length=300)
def _invalid(field: str):
return api_error(422, ErrorCode.WINDOW_INVALID, field=field)
def _validated(payload: WindowIn) -> dict:
name = payload.name.strip()
if not name:
raise _invalid("name")
if not window_service.valid_timezone(payload.timezone):
raise _invalid("timezone")
match = _TIME.match(payload.start_time)
if not match:
raise _invalid("start_time")
values = {
"name": name, "timezone": payload.timezone, "recurrence": payload.recurrence,
"weekdays": [], "month_mode": None, "month_nth": None, "month_weekday": None, "month_day": None,
"start_time": time(int(match.group(1)), int(match.group(2))),
"duration_minutes": payload.duration_minutes,
"start_cutoff_minutes": min(payload.start_cutoff_minutes, payload.duration_minutes),
"enabled": payload.enabled,
}
if payload.recurrence == "weekly":
days = sorted(set(payload.weekdays))
if not days or any(d < 0 or d > 6 for d in days):
raise _invalid("weekdays")
values["weekdays"] = days
elif payload.month_mode == "day":
if payload.month_day is None or not 1 <= payload.month_day <= 28:
raise _invalid("month_day")
values.update(month_mode="day", month_day=payload.month_day)
elif payload.month_mode == "nth_weekday":
if payload.month_nth not in (1, 2, 3, 4, -1):
raise _invalid("month_nth")
if payload.month_weekday is None or not 0 <= payload.month_weekday <= 6:
raise _invalid("month_weekday")
values.update(month_mode="nth_weekday", month_nth=payload.month_nth, month_weekday=payload.month_weekday)
else:
raise _invalid("month_mode")
return values
async def _plans_using(db: AsyncSession, window_id) -> list[dict]:
from app.models.rollout import RolloutPlan
rows = (await db.scalars(select(RolloutPlan).where(RolloutPlan.window_id == window_id).order_by(RolloutPlan.name))).all()
return [{"id": str(p.id), "name": p.name, "auto_start": p.auto_start, "active": p.activated_at is not None} for p in rows]
def read_window(window: MaintenanceWindow, *, now: datetime | None = None, count: int = 5) -> dict:
now = now or datetime.now(timezone.utc)
return {
"id": str(window.id), "name": window.name, "timezone": window.timezone, "recurrence": window.recurrence,
"weekdays": window.weekdays or [], "month_mode": window.month_mode, "month_nth": window.month_nth,
"month_weekday": window.month_weekday, "month_day": window.month_day,
"start_time": window.start_time.strftime("%H:%M"), "duration_minutes": window.duration_minutes,
"start_cutoff_minutes": window.start_cutoff_minutes, "enabled": window.enabled,
"created_by": window.created_by, "created_at": window.created_at,
"open_now": window_service.current(window, now) is not None,
"next": [o.as_dict() for o in window_service.occurrences(window, now, count, with_skipped=True)] if window.enabled else [],
"exceptions": [
{"id": str(e.id), "occurrence_date": e.occurrence_date.isoformat(), "kind": e.kind,
"moved_start": e.moved_start, "reason": e.reason, "actor": e.actor, "created_at": e.created_at}
for e in window.exceptions
],
}
async def _window_or_404(db: AsyncSession, window_id: uuid.UUID) -> MaintenanceWindow:
window = await db.get(MaintenanceWindow, window_id)
if window is None:
raise api_error(404, ErrorCode.WINDOW_NOT_FOUND)
return window
async def _name_taken(db: AsyncSession, name: str, own_id=None) -> bool:
other = await db.scalar(select(MaintenanceWindow.id).where(MaintenanceWindow.name == name))
return other is not None and other != own_id
@router.get("")
async def list_windows(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(MaintenanceWindow).order_by(MaintenanceWindow.name))).all()
result = []
for w in rows:
item = read_window(w)
item["used_by"] = await _plans_using(db, w.id)
result.append(item)
return result
@router.post("", status_code=201)
async def create_window(payload: WindowIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
values = _validated(payload)
if await _name_taken(db, values["name"]):
raise api_error(409, ErrorCode.WINDOW_NAME_TAKEN)
window = MaintenanceWindow(**values, created_by=request.state.user.username, created_at=datetime.now(timezone.utc))
window.exceptions = []
db.add(window)
await db.commit()
item = read_window(await _window_or_404(db, window.id))
item["used_by"] = []
return item
@router.get("/{window_id}")
async def get_window(window_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
item = read_window(await _window_or_404(db, window_id))
item["used_by"] = await _plans_using(db, window_id)
return item
@router.put("/{window_id}")
async def update_window(window_id: uuid.UUID, payload: WindowIn, db: AsyncSession = Depends(get_db)):
"""Changing a window changes the fingerprint of every plan that uses it, so an activated
automatic start stops until the plan is activated again (see rollout_service.fingerprint)."""
await policy_lock(db)
window = await _window_or_404(db, window_id)
values = _validated(payload)
if await _name_taken(db, values["name"], window.id):
raise api_error(409, ErrorCode.WINDOW_NAME_TAKEN)
for key, value in values.items():
setattr(window, key, value)
await db.commit()
item = read_window(await _window_or_404(db, window_id))
item["used_by"] = await _plans_using(db, window_id)
return item
@router.delete("/{window_id}", status_code=204)
async def delete_window(window_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
window = await _window_or_404(db, window_id)
# A plan without a window means "no time limit": deleting its window must never turn it into one.
if await _plans_using(db, window_id):
raise api_error(409, ErrorCode.WINDOW_IN_USE)
await db.delete(window)
await db.commit()
@router.get("/{window_id}/occurrences")
async def window_occurrences(window_id: uuid.UUID, count: int = Query(10, ge=1, le=50), db: AsyncSession = Depends(get_db)):
window = await _window_or_404(db, window_id)
return [o.as_dict() for o in window_service.occurrences(window, datetime.now(timezone.utc), count, with_skipped=True)]
@router.post("/{window_id}/exceptions", status_code=201)
async def add_exception(window_id: uuid.UUID, payload: ExceptionIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
window = await _window_or_404(db, window_id)
now = datetime.now(timezone.utc)
if not window_service.is_occurrence_date(window, payload.occurrence_date):
raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="occurrence_date")
# Only future occurrences (or the one open now) can be changed; history stays as it was.
if window_service.local_start(window, payload.occurrence_date) + timedelta(minutes=window.duration_minutes) <= now:
raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="occurrence_date")
moved = None
if payload.kind == "move":
if payload.moved_start is None:
raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="moved_start")
moved = payload.moved_start if payload.moved_start.tzinfo else payload.moved_start.replace(tzinfo=timezone.utc)
if moved <= now:
raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="moved_start")
if any(e.occurrence_date == payload.occurrence_date for e in window.exceptions):
raise api_error(409, ErrorCode.WINDOW_EXCEPTION_EXISTS)
exc = MaintenanceWindowException(
window_id=window.id, occurrence_date=payload.occurrence_date, kind=payload.kind, moved_start=moved,
reason=payload.reason.strip(), actor=request.state.user.username, created_at=now,
)
db.add(exc)
try:
await db.commit()
except IntegrityError:
await db.rollback()
raise api_error(409, ErrorCode.WINDOW_EXCEPTION_EXISTS)
await audit_service.persist(
kind="maintenance_window", action=f"exception_{payload.kind}", status="saved", entity_id=str(window.id),
entity_name=window.name, details={"occurrence_date": payload.occurrence_date.isoformat(),
"moved_start": moved.isoformat() if moved else None, "reason": exc.reason},
)
db.expire_all()
return read_window(await _window_or_404(db, window_id))
@router.delete("/{window_id}/exceptions/{exception_id}")
async def delete_exception(window_id: uuid.UUID, exception_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
window = await _window_or_404(db, window_id)
exc = await db.get(MaintenanceWindowException, exception_id)
if exc is None or exc.window_id != window.id:
raise api_error(404, ErrorCode.WINDOW_EXCEPTION_NOT_FOUND)
await db.delete(exc)
await db.commit()
db.expire_all()
return read_window(await _window_or_404(db, window_id))
+42
View File
@@ -88,6 +88,48 @@ TOOLS = [
"path": "/api/search",
"query": ("q",),
},
{
"name": "rule_match",
"description": "Which firewall rule decides a packet (first packet of a new connection), per firewall, "
"with the verdict (pass/block/reject), the deciding rule and an honest confidence "
"(sure/probable/unknown plus reason codes). Reads only; nothing is changed.",
"inputSchema": {
"type": "object", "required": ["src", "dst"],
"properties": {
"src": {"type": "string", "description": "source IP address"},
"dst": {"type": "string", "description": "destination IP address"},
"proto": {"type": "string", "description": "tcp, udp, icmp, ipv6-icmp or a protocol number", "default": "tcp"},
"dport": {"type": "integer", "minimum": 0, "maximum": 65535},
"sport": {"type": "integer", "minimum": 0, "maximum": 65535, "description": "default: any ephemeral port"},
"icmptype": {"type": "string", "description": "e.g. echoreq or 8"},
"iface": {"type": "string", "description": "ingress interface identifier (e.g. lan); default: guessed per firewall"},
"egress": {"type": "string", "description": "also check the outgoing side: auto or an interface identifier"},
"host_ids": {"type": "string", "description": "comma separated firewall ids (default: all)"},
"group_id": {"type": "string", "description": "device group id instead of host_ids"},
},
},
"path": "/api/rule-analysis/match",
"query": ("src", "dst", "proto", "dport", "sport", "icmptype", "iface", "egress", "host_ids", "group_id"),
# The per-rule trace is for the UI; the verdict, deciding rule and reasons are what matters here.
"drop": ("trace",),
},
{
"name": "rule_cleanup_candidates",
"description": "Cleanup candidates of one firewall: rules that are unused (no hit for N days), shadowed or "
"duplicate (an earlier quick rule already decides everything they match), or disabled for a "
"long time. Each with its confidence (sure/probable/unknown), the last observed hit and "
"whether it could be removed (not for automatic, legacy or OPNexus-managed rules). Reads only.",
"inputSchema": {
"type": "object", "required": ["host_id"],
"properties": {
"host_id": {"type": "string", "description": "Firewall id from list_hosts"},
"unused_days": {"type": "integer", "minimum": 1, "maximum": 3650, "default": 30,
"description": "observation window for unused / disabled rules"},
},
},
"path": "/api/hosts/{host_id}/rule-analysis/cleanup",
"query": ("unused_days",),
},
{
"name": "api_get",
"description": "Read any other OPNexus API endpoint (GET only), e.g. /api/rules, /api/aliases, /api/logs. "
+112 -2
View File
@@ -1,21 +1,26 @@
import uuid
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.models.notification import NotificationProblem
from app.models.notification import NotificationChannel, NotificationProblem
from app.schemas.notification import (
NotificationChannelCreate,
NotificationChannelRead,
NotificationChannelUpdate,
NotificationProblemRead,
NotificationSettingsRead,
NotificationSettingsUpdate,
TestNotificationRequest,
)
from app.services import notification_delivery, notification_settings_service
from app.services import notification_channels, notification_delivery, notification_secret_store, notification_settings_service, url_guard
router = APIRouter(prefix="/notifications", tags=["notifications"])
@@ -43,6 +48,11 @@ async def read_settings(db: AsyncSession = Depends(get_db), user: AdminUser = De
@router.put("/settings", response_model=NotificationSettingsRead)
async def update_settings(payload: NotificationSettingsUpdate, db: AsyncSession = Depends(get_db)):
if payload.webhook_url:
try:
payload.webhook_url = url_guard.validate_url(payload.webhook_url)
except ValueError as exc:
raise api_error(422, ErrorCode.NOTIFICATION_WEBHOOK_URL_INVALID, reason=str(exc)) from None
return await notification_settings_service.update_settings(db, payload)
@@ -79,3 +89,103 @@ async def test_notification(payload: TestNotificationRequest, db: AsyncSession =
except notification_delivery.NotificationDeliveryError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
return {"success": True}
# ── Additional channels (ntfy, Gotify, Telegram, Teams, Matrix) ────────────────
def _channel_read(row: NotificationChannel, user: AdminUser) -> NotificationChannelRead:
read = NotificationChannelRead.model_validate(row)
# An ntfy topic or a chat id is as good as a password for whoever knows it: only admins see the settings.
return read if user.role == "admin" else read.model_copy(update={"config": {}})
async def _get_channel(db: AsyncSession, channel_id: uuid.UUID) -> NotificationChannel:
row = await db.get(NotificationChannel, channel_id)
if row is None:
raise api_error(404, ErrorCode.NOTIFICATION_CHANNEL_NOT_FOUND)
return row
def _invalid(exc: ValueError):
return api_error(422, ErrorCode.NOTIFICATION_CHANNEL_INVALID, reason=str(exc))
@router.get("/channels", response_model=list[NotificationChannelRead])
async def list_channels(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
rows = await db.scalars(select(NotificationChannel).order_by(NotificationChannel.name))
return [_channel_read(r, user) for r in rows.all()]
@router.post("/channels", response_model=NotificationChannelRead, status_code=201)
async def create_channel(payload: NotificationChannelCreate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
try:
config = notification_channels.clean_config(payload.channel_type, payload.config)
secret = notification_channels.clean_secret(payload.channel_type, payload.secret) if payload.secret else None
except ValueError as exc:
raise _invalid(exc) from None
row = NotificationChannel(
name=payload.name.strip(), channel_type=payload.channel_type, enabled=payload.enabled, config=config,
secret_set=secret is not None,
)
row.id = uuid.uuid4()
if secret is None and notification_channels.missing_requirements(row) == "Zugangsdaten fehlen":
raise _invalid(ValueError("Zugangsdaten fehlen"))
db.add(row)
try:
await db.flush()
except IntegrityError:
await db.rollback()
raise api_error(409, ErrorCode.NOTIFICATION_CHANNEL_NAME_TAKEN) from None
if secret is not None:
notification_secret_store.save_secret(notification_channels.secret_name(row.id), secret)
await db.commit()
return _channel_read(row, user)
@router.put("/channels/{channel_id}", response_model=NotificationChannelRead)
async def update_channel(
channel_id: uuid.UUID, payload: NotificationChannelUpdate,
db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user),
):
row = await _get_channel(db, channel_id)
try:
if payload.config is not None:
row.config = notification_channels.clean_config(row.channel_type, payload.config)
new_secret = notification_channels.clean_secret(row.channel_type, payload.secret) if payload.secret else None
except ValueError as exc:
raise _invalid(exc) from None
if payload.name is not None:
row.name = payload.name.strip()
if payload.enabled is not None:
row.enabled = payload.enabled
if payload.clear_secret:
notification_secret_store.delete_secret(notification_channels.secret_name(row.id))
row.secret_set = False
elif new_secret is not None:
notification_secret_store.save_secret(notification_channels.secret_name(row.id), new_secret)
row.secret_set = True
try:
await db.commit()
except IntegrityError:
await db.rollback()
raise api_error(409, ErrorCode.NOTIFICATION_CHANNEL_NAME_TAKEN) from None
return _channel_read(row, user)
@router.delete("/channels/{channel_id}", status_code=204)
async def delete_channel(channel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
row = await _get_channel(db, channel_id)
notification_secret_store.delete_secret(notification_channels.secret_name(row.id))
await db.delete(row)
await db.commit()
@router.post("/channels/{channel_id}/test")
async def test_channel(channel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
row = await _get_channel(db, channel_id)
try:
await notification_channels.send_test(row)
except notification_delivery.NotificationDeliveryError as exc:
# The text names the channel and the HTTP status; it never contains a URL or a token.
raise api_error(502, ErrorCode.NOTIFICATION_CHANNEL_TEST_FAILED, reason=str(exc)) from None
return {"success": True}
+54 -6
View File
@@ -1,5 +1,7 @@
"""OIDC single sign-on: public login endpoints and admin-only settings (see services/oidc_service.py)."""
import logging
import re
import secrets
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, Request
@@ -10,6 +12,7 @@ from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser
from app.routers.auth import _set_session_cookie, client_ip, current_user
from app.config import settings
from app.schemas.oidc import OidcSettingsRead, OidcSettingsUpdate
from app.security_middleware import _origin_allowed
from app.services import auth_service, notification_secret_store, oidc_service
@@ -18,10 +21,43 @@ logger = logging.getLogger(__name__)
router = APIRouter(tags=["oidc"])
CALLBACK_PATH = "/api/auth/oidc/callback"
STATE_COOKIE = "opnexus-oidc-state"
# HTTPS deployments (1.29): __Host- (Secure, no Domain, Path=/) so that no sibling SUBDOMAIN or plain-HTTP page can
# plant a state cookie of its own (login CSRF with the attacker's own authorisation). Known limit: cookies are not
# separated by PORT, so script in a firewall GUI shown on a gateway slot port of the same host could still set it
# (BACKLOG S8, accepted: needs a compromised firewall GUI and a mapped IdP account; the real fix is one hostname
# per slot). The browser only accepts the prefix with Path=/, so the cookie travels with every request for its
# 10 minutes; plain-HTTP labs keep the narrow path.
SECURE_STATE_COOKIE = "__Host-opnexus-oidc-state"
STATE_COOKIE_PATH = "/api/auth/oidc"
def state_cookie_name() -> str:
return SECURE_STATE_COOKIE if settings.cookie_secure else STATE_COOKIE
def _state_cookie_path() -> str:
return "/" if settings.cookie_secure else STATE_COOKIE_PATH
def _clear_state_cookie(response) -> None:
response.delete_cookie(state_cookie_name(), path=_state_cookie_path(), secure=settings.cookie_secure, httponly=True, samesite="lax")
if settings.cookie_secure: # a plain-name cookie of an earlier version
response.delete_cookie(STATE_COOKIE, path=STATE_COOKIE_PATH, secure=True, httponly=True, samesite="lax")
# SSO sessions are short: the provider's group membership is only re-checked at login, so a removed person
# must not keep a 30-day session.
SSO_SESSION_SECONDS = 12 * 3600
def _login_error(code: str) -> RedirectResponse:
return RedirectResponse(f"/login?sso_error={code}", status_code=302)
response = RedirectResponse(f"/login?sso_error={code}", status_code=302)
_clear_state_cookie(response)
return response
def _printable(value: str) -> str:
"""Query text from an unauthenticated caller, made safe for a log line (no CR/LF or other control chars)."""
return re.sub(r"[^\x20-\x7e]", "?", value)[:60]
@router.get("/auth/oidc/config")
@@ -42,11 +78,16 @@ async def start(request: Request, origin: str, db: AsyncSession = Depends(get_db
or not _origin_allowed(f"{parts.scheme}://{parts.netloc}", request.headers.get("host", ""))):
return _login_error("disabled")
try:
url = await oidc_service.start(db, cfg, f"{parts.scheme}://{parts.netloc}{CALLBACK_PATH}")
url, state = await oidc_service.start(db, cfg, f"{parts.scheme}://{parts.netloc}{CALLBACK_PATH}")
except oidc_service.OidcError as exc:
logger.warning("OIDC start fehlgeschlagen: %s", exc)
return _login_error(exc.code)
return RedirectResponse(url, status_code=302)
response = RedirectResponse(url, status_code=302)
# Binds the attempt to THIS browser: a callback URL handed to somebody else (login CSRF) or a captured
# code+state pair is useless without the cookie.
response.set_cookie(state_cookie_name(), state, max_age=600, httponly=True, samesite="lax", secure=settings.cookie_secure,
path=_state_cookie_path())
return response
@router.get("/auth/oidc/callback")
@@ -59,7 +100,11 @@ async def callback(request: Request, code: str = "", state: str = "", error: str
return _login_error("disabled")
try:
if error or not code or not state:
raise oidc_service.OidcError("provider", f"Anbieter meldet {error or 'keinen Code'}")
raise oidc_service.OidcError("provider", "Anbieter meldet " + (_printable(error) if error else "keinen Code"))
# Only the cookie name of this deployment counts (on HTTPS never the plain one); no cookie never matches.
bound = request.cookies.get(state_cookie_name(), "")
if not bound or not secrets.compare_digest(bound.encode(), state.encode()):
raise oidc_service.OidcError("state", "Der Anmeldeversuch gehört nicht zu diesem Browser")
user = await oidc_service.complete(db, cfg, code, state)
except oidc_service.OidcError as exc:
auth_service.throttle.record_failure(ip, "oidc")
@@ -67,9 +112,12 @@ async def callback(request: Request, code: str = "", state: str = "", error: str
return _login_error(exc.code)
auth_service.throttle.reset(ip, "oidc")
await auth_service.finalize_login(db, user, ip)
token, _ = await auth_service.create_session(db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip)
token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip, lifetime_seconds=SSO_SESSION_SECONDS
)
response = RedirectResponse("/dashboard", status_code=302)
_set_session_cookie(response, token)
_set_session_cookie(response, token, max_age=SSO_SESSION_SECONDS)
_clear_state_cookie(response)
logger.info("SSO-Anmeldung von '%s' (%s) aus %s", user.username, user.role, ip)
return response
+7
View File
@@ -20,6 +20,7 @@ from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.one_to_one_deploy_service import confirm_one_to_one, deploy_one_to_one, undeploy_one_to_one, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
from app.routers.adopt_routes import add_adopt_routes
router = APIRouter(prefix="/one-to-one", tags=["one-to-one"])
@@ -49,6 +50,7 @@ def _to_read(rule: OneToOneRule, hosts: list[OpnsenseHost], deployments: list[On
workflow_state=state,
workflow_counts=counts,
id=rule.id,
config_key=rule.config_key,
description=rule.description,
interface=rule.interface,
source_net=rule.source_net,
@@ -79,6 +81,9 @@ def _to_read(rule: OneToOneRule, hosts: list[OpnsenseHost], deployments: list[On
)
add_adopt_routes(router, kind="one_to_one", model=OneToOneRule, load_options=_load_options, to_read=_to_read, read_model=OneToOneRead)
@router.get("", response_model=list[OneToOneRead])
async def list_one_to_one_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OneToOneRule).options(*_load_options()))
@@ -227,6 +232,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_one_to_one_rule(deployment.host, deployment.opnsense_one_to_one_uuid)
+4
View File
@@ -18,6 +18,7 @@ from app.services import openvpn_key_store
from app.services.openvpn_service import effective_hosts, preview, revision, snapshot, workflow, OPENVPN_FIELDS
from app.services.openvpn_deploy_service import deploy_instance, undeploy_instance
from app.services.opnsense_client import OpnsenseAPIError
from app.services.deploy_lease import resource_lease
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/openvpn/instances", tags=["openvpn"])
@@ -194,7 +195,10 @@ async def remove_previous_target(instance_id: uuid.UUID, host_id: uuid.UUID, app
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
async with resource_lease("openvpn", instance.id):
deployment = next(d for d in instance.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_instance_uuid:
+7
View File
@@ -21,6 +21,7 @@ from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.outbound_nat_deploy_service import confirm_outbound_nat, deploy_outbound_nat, undeploy_outbound_nat, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
from app.routers.adopt_routes import add_adopt_routes
router = APIRouter(prefix="/outbound-nat", tags=["outbound-nat"])
@@ -50,6 +51,7 @@ def _to_read(rule: OutboundNatRule, hosts: list[OpnsenseHost], deployments: list
workflow_state=state,
workflow_counts=counts,
id=rule.id,
config_key=rule.config_key,
description=rule.description,
interface=rule.interface,
source_net=rule.source_net,
@@ -82,6 +84,9 @@ def _to_read(rule: OutboundNatRule, hosts: list[OpnsenseHost], deployments: list
)
add_adopt_routes(router, kind="outbound_nat", model=OutboundNatRule, load_options=_load_options, to_read=_to_read, read_model=OutboundNatRead)
@router.get("", response_model=list[OutboundNatRead])
async def list_outbound_nat_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OutboundNatRule).options(*_load_options()))
@@ -234,6 +239,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_outbound_nat_rule(deployment.host, deployment.opnsense_outbound_nat_uuid)
+3
View File
@@ -75,6 +75,7 @@ def _to_read(rule: PolicyHierarchyRule, all_groups: list[PolicyHierarchyGroup])
workflow_state=state,
workflow_counts=counts,
id=rule.id,
config_key=rule.config_key,
hierarchy_group_id=rule.hierarchy_group_id,
rulebase=rule.rulebase,
sequence=rule.sequence,
@@ -330,6 +331,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_policy_hierarchy_uuid)
await opnsense_client.apply(deployment.host)
+7
View File
@@ -20,6 +20,7 @@ from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.port_forward_deploy_service import confirm_port_forward, deploy_port_forward, undeploy_port_forward, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
from app.routers.adopt_routes import add_adopt_routes
router = APIRouter(prefix="/port-forward", tags=["port-forward"])
@@ -49,6 +50,7 @@ def _to_read(rule: PortForwardRule, hosts: list[OpnsenseHost], deployments: list
workflow_state=state,
workflow_counts=counts,
id=rule.id,
config_key=rule.config_key,
description=rule.description,
interface=rule.interface,
ipprotocol=rule.ipprotocol,
@@ -80,6 +82,9 @@ def _to_read(rule: PortForwardRule, hosts: list[OpnsenseHost], deployments: list
)
add_adopt_routes(router, kind="port_forward", model=PortForwardRule, load_options=_load_options, to_read=_to_read, read_model=PortForwardRead)
@router.get("", response_model=list[PortForwardRead])
async def list_port_forward_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(PortForwardRule).options(*_load_options()))
@@ -229,6 +234,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
await opnsense_client.del_port_forward_rule(deployment.host, deployment.opnsense_port_forward_uuid)
+459
View File
@@ -0,0 +1,459 @@
"""Firmware rollouts in waves (1.25): plans and runs.
Roles (enforced in AuthMiddleware, see api_token_service.is_rollout): everybody signed in reads;
operators prepare plans; starting/activating a plan and resuming/approving/skipping/aborting a run
need the deploy right (admin, or operator while operators may deploy). Pausing is deliberately NOT
a rollout action: anybody who may write at all may always brake.
"""
import uuid
from datetime import datetime, timezone
from typing import Literal
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field, model_validator
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.maintenance_window import MaintenanceWindow
from app.models.policy import DeviceGroup
from app.models.rollout import RolloutPlan, RolloutPlanWave, RolloutRun
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.services import audit_service, diagnostics_service, rollout_engine, rollout_health, rollout_service
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/rollouts", tags=["rollouts"])
ACTIVE_RUN = rollout_engine.ACTIVE
class WaveIn(BaseModel):
label: str = Field(min_length=1, max_length=60)
group_ids: list[uuid.UUID] = Field(min_length=1, max_length=20)
soak_minutes: int = Field(default=30, ge=0, le=rollout_service.MAX_SOAK_MINUTES)
carp_handover: bool = False
require_approval: bool = False
class ProbeIn(BaseModel):
"""A reachability probe run FROM the firewall in the health gate (same target rules as the host diagnostics)."""
type: Literal["ping", "tcp"]
target: str = Field(min_length=1, max_length=253, pattern=diagnostics_service.TARGET.pattern)
port: int | None = Field(default=None, ge=1, le=65535)
@model_validator(mode="after")
def _port_only_for_tcp(self):
if (self.type == "tcp") != (self.port is not None):
raise ValueError("port_required_for_tcp_only")
return self
class PlanIn(BaseModel):
name: str = Field(min_length=1, max_length=120)
description: str = Field(default="", max_length=2000)
action: Literal["update", "major_update"] = "update"
window_id: uuid.UUID | None = None
auto_start: bool = False
gate_checks: dict[str, bool] = Field(default_factory=dict)
gate_probes: list[ProbeIn] = Field(default_factory=list, max_length=rollout_health.MAX_PROBES)
compliance_mode: Literal["warn", "fail"] = "warn"
waves: list[WaveIn] = Field(min_length=1, max_length=rollout_service.MAX_WAVES)
class TokenIn(BaseModel):
preview_token: str = Field(min_length=64, max_length=64)
async def _plan_or_404(db: AsyncSession, plan_id: uuid.UUID) -> RolloutPlan:
plan = await rollout_service.load_plan(db, plan_id)
if plan is None:
raise api_error(404, ErrorCode.ROLLOUT_PLAN_NOT_FOUND)
return plan
async def read_plan(db: AsyncSession, plan: RolloutPlan, resolved: dict | None = None) -> dict:
resolved = resolved or await rollout_service.resolve(db, plan)
active = plan.activated_at is not None
last_run = await db.scalar(select(RolloutRun).where(RolloutRun.plan_id == plan.id).order_by(RolloutRun.created_at.desc()).limit(1))
return {
**rollout_service.definition(plan),
"gate_probes": list(plan.gate_probes or []),
"description": plan.description,
"waves": [{"id": str(w.id), "position": w.position, "label": w.label, "soak_minutes": w.soak_minutes,
"carp_handover": w.carp_handover, "require_approval": w.require_approval,
"group_ids": [str(g.id) for g in w.groups], "groups": [g.name for g in w.groups],
"host_count": len(resolved["waves"][i]["hosts"]) if i < len(resolved["waves"]) else 0}
for i, w in enumerate(plan.waves)],
"active": active, "activated_by": plan.activated_by, "activated_at": plan.activated_at,
# Activated but the plan, its groups or its window changed since: the automatic start is held back.
"fingerprint_ok": (not active) or plan.activated_fingerprint == rollout_service.fingerprint(plan, resolved),
"auto_start_error": plan.auto_start_error,
"window": resolved["window"], "errors": resolved["errors"], "warnings": resolved["warnings"],
"host_count": sum(len(w["hosts"]) for w in resolved["waves"]),
"last_run": {"id": str(last_run.id), "status": last_run.status, "created_at": last_run.created_at} if last_run else None,
"created_by": plan.created_by, "created_at": plan.created_at, "updated_at": plan.updated_at,
}
def _unique_probes(probes: list[ProbeIn]) -> list[dict]:
out: list[dict] = []
for probe in probes:
value = {"type": probe.type, "target": probe.target, "port": probe.port}
if value not in out:
out.append(value)
return out
async def _apply(db: AsyncSession, plan: RolloutPlan, payload: PlanIn) -> None:
name = payload.name.strip()
if not name:
raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="name")
other = await db.scalar(select(RolloutPlan.id).where(RolloutPlan.name == name))
if other is not None and other != plan.id:
raise api_error(409, ErrorCode.ROLLOUT_NAME_TAKEN)
if payload.window_id is not None and await db.get(MaintenanceWindow, payload.window_id) is None:
raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="window_id")
if any(k not in rollout_health.CHECKS for k in payload.gate_checks):
raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="gate_checks")
wanted = {g for w in payload.waves for g in w.group_ids}
groups = {g.id: g for g in (await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(wanted)))).all()}
if len(groups) != len(wanted):
raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="group_ids")
plan.name, plan.description, plan.action = name, payload.description.strip(), payload.action
plan.window_id, plan.auto_start = payload.window_id, payload.auto_start
plan.gate_checks = {k: bool(v) for k, v in payload.gate_checks.items()}
plan.gate_probes = _unique_probes(payload.gate_probes)
plan.compliance_mode = payload.compliance_mode
plan.waves = [
RolloutPlanWave(position=i, label=w.label.strip() or f"{i + 1}", soak_minutes=w.soak_minutes, carp_handover=w.carp_handover,
require_approval=w.require_approval, groups=[groups[g] for g in dict.fromkeys(w.group_ids)])
for i, w in enumerate(payload.waves)
]
# Any edit disarms the automatic start: an operator cannot quietly change what an admin activated.
plan.activated_at = plan.activated_by = plan.activated_by_id = plan.activated_fingerprint = None
plan.auto_start_error = None
@router.get("/plans")
async def list_plans(db: AsyncSession = Depends(get_db)):
plans = (await db.scalars(select(RolloutPlan).order_by(RolloutPlan.name))).all()
return [await read_plan(db, p) for p in plans]
@router.post("/plans", status_code=201)
async def create_plan(payload: PlanIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = RolloutPlan(created_by=request.state.user.username, created_at=datetime.now(timezone.utc), waves=[])
await _apply(db, plan, payload)
db.add(plan)
await db.commit()
return await read_plan(db, await _plan_or_404(db, plan.id))
@router.get("/plans/{plan_id}")
async def get_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return await read_plan(db, await _plan_or_404(db, plan_id))
@router.put("/plans/{plan_id}")
async def update_plan(plan_id: uuid.UUID, payload: PlanIn, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = await _plan_or_404(db, plan_id)
was_active = plan.activated_at is not None
await _apply(db, plan, payload)
await db.commit()
if was_active:
await audit_service.persist(kind="rollout", action="deactivate", status="completed", entity_id=str(plan.id),
entity_name=plan.name, message="Plan geändert: automatischer Start deaktiviert")
return await read_plan(db, await _plan_or_404(db, plan_id))
@router.delete("/plans/{plan_id}", status_code=204)
async def delete_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = await _plan_or_404(db, plan_id)
if await db.scalar(select(RolloutRun.id).where(RolloutRun.plan_id == plan.id, RolloutRun.status.in_(ACTIVE_RUN)).limit(1)):
raise api_error(409, ErrorCode.ROLLOUT_PLAN_BUSY)
await db.delete(plan)
await db.commit()
@router.post("/plans/{plan_id}/preview")
async def preview_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
plan = await _plan_or_404(db, plan_id)
resolved = await rollout_service.resolve(db, plan)
return {**resolved, "preview_token": rollout_service.fingerprint(plan, resolved),
"can_start": not resolved["errors"], "plan": await read_plan(db, plan, resolved)}
async def _checked_token(db: AsyncSession, plan: RolloutPlan, token: str) -> tuple[dict, str]:
resolved = await rollout_service.resolve(db, plan)
current = rollout_service.fingerprint(plan, resolved)
if token != current:
raise api_error(409, ErrorCode.ROLLOUT_PREVIEW_STALE)
if resolved["errors"]:
raise api_error(422, ErrorCode.ROLLOUT_PLAN_INVALID, reason=resolved["errors"][0]["code"])
return resolved, current
@router.post("/plans/{plan_id}/activate")
async def activate_plan(plan_id: uuid.UUID, payload: TokenIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = await _plan_or_404(db, plan_id)
if not plan.auto_start:
raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="auto_start")
_resolved, current = await _checked_token(db, plan, payload.preview_token)
plan.activated_by, plan.activated_at, plan.activated_fingerprint = request.state.user.username, datetime.now(timezone.utc), current
plan.activated_by_id = request.state.user.id
plan.auto_start_error = None
await db.commit()
await audit_service.persist(kind="rollout", action="activate", status="completed", entity_id=str(plan.id),
entity_name=plan.name, details={"fingerprint": current})
return await read_plan(db, await _plan_or_404(db, plan_id))
@router.post("/plans/{plan_id}/deactivate")
async def deactivate_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = await _plan_or_404(db, plan_id)
plan.activated_by = plan.activated_by_id = plan.activated_at = plan.activated_fingerprint = None
await db.commit()
await audit_service.persist(kind="rollout", action="deactivate", status="completed", entity_id=str(plan.id), entity_name=plan.name)
return await read_plan(db, await _plan_or_404(db, plan_id))
# ── runs ─────────────────────────────────────────────────────────────────────
class StartIn(TokenIn):
confirm_name: str = Field(min_length=1, max_length=120)
acknowledge_unattended: Literal[True]
class ReasonIn(BaseModel):
reason: str = Field(default="", max_length=500)
class SkipIn(BaseModel):
host_id: uuid.UUID
reason: str = Field(min_length=3, max_length=500)
class AbortIn(BaseModel):
reason: str = Field(min_length=3, max_length=500)
confirm_name: str = Field(min_length=1, max_length=120)
def _read_run_summary(run: RolloutRun) -> dict:
counts: dict[str, int] = {}
for h in run.hosts:
counts[h.status] = counts.get(h.status, 0) + 1
return {
"id": str(run.id), "plan_id": str(run.plan_id) if run.plan_id else None, "plan_name": run.plan_name,
"action": run.action, "status": run.status, "paused_from": run.paused_from, "current_wave": run.current_wave,
"wave_count": len(run.waves), "current_wave_label": next((w.label for w in run.waves if w.index == run.current_wave), None),
"target_version": run.target_version, "pause_reason": run.pause_reason, "pause_detail": run.pause_detail,
"started_by": run.started_by, "trigger": run.trigger, "window_occurrence": run.window_occurrence,
"created_at": run.created_at, "finished_at": run.finished_at, "step_at": run.step_at, "host_counts": counts,
"active": run.status in rollout_engine.ACTIVE,
}
async def read_run(db: AsyncSession, run: RolloutRun) -> dict:
scheds = {}
ids = [h.scheduled_maintenance_id for h in run.hosts if h.scheduled_maintenance_id]
if ids:
scheds = {s.id: s for s in (await db.scalars(select(ScheduledMaintenance).where(ScheduledMaintenance.id.in_(ids)))).all()}
soak_until = next((w.soak_until for w in run.waves if w.index == run.current_wave), None)
return {
**_read_run_summary(run),
"soak_until": soak_until, "progress": {k: v for k, v in (run.progress or {}).items()
if k in ("gate_recheck_at", "cluster_recheck_at", "blocked_since", "resume_gate",
"cluster_gate", "preflight_started", "resume_recheck_at")},
"snapshot": run.plan_snapshot,
"waves": [{"index": w.index, "label": w.label, "status": w.status, "gate_result": w.gate_result,
"gate_attempts": w.gate_attempts, "soak_until": w.soak_until, "approved_by": w.approved_by,
"started_at": w.started_at, "finished_at": w.finished_at,
"spec": run.plan_snapshot["waves"][w.index] if w.index < len(run.plan_snapshot.get("waves", [])) else None}
for w in run.waves],
"hosts": [{"id": str(h.id), "host_id": str(h.host_id) if h.host_id else None, "host_name": h.host_name,
"cluster_name": h.cluster_name, "wave_index": h.wave_index, "order": h.order, "status": h.status,
"version_before": h.version_before, "version_after": h.version_after, "detail": h.detail,
"started_at": h.started_at, "finished_at": h.finished_at,
"schedule": ({"id": str(s.id), "status": s.status, "phase": s.phase, "carp_handover": s.carp_handover,
"host_operation_id": str(s.host_operation_id) if s.host_operation_id else None}
if (s := scheds.get(h.scheduled_maintenance_id)) else None)}
for h in sorted(run.hosts, key=lambda h: (h.wave_index, h.order, h.host_name))],
}
async def _cancel_if_pending(db: AsyncSession, sched_id) -> bool:
"""Cancel a host's schedule only if it has not fired. Locked: the poll loop may be firing it right now
(fire_due holds the row), and a blind write would turn a running action into "cancelled"."""
if sched_id is None:
return False
sched = await db.scalar(select(ScheduledMaintenance).where(ScheduledMaintenance.id == sched_id)
.with_for_update().execution_options(populate_existing=True))
if sched is not None and sched.status == "pending":
sched.status = "cancelled"
return True
return False
async def _run_or_404(db: AsyncSession, run_id: uuid.UUID, *, lock: bool = False) -> RolloutRun:
stmt = select(RolloutRun).where(RolloutRun.id == run_id).execution_options(populate_existing=True)
if lock:
stmt = stmt.with_for_update()
run = await db.scalar(stmt)
if run is None:
raise api_error(404, ErrorCode.ROLLOUT_RUN_NOT_FOUND)
return run
def _not_now(run: RolloutRun):
return api_error(409, ErrorCode.ROLLOUT_ACTION_NOT_ALLOWED, status=run.status)
async def _done(db: AsyncSession, run: RolloutRun, action: str, status: str, request: Request, **details) -> dict:
run.revision += 1
await db.commit()
await audit_service.persist(kind="rollout", action=action, status=status, entity_id=str(run.id), entity_name=run.plan_name,
details={"wave": run.current_wave, **details})
return await read_run(db, await _run_or_404(db, run.id))
@router.post("/plans/{plan_id}/start", status_code=201)
async def start_plan(plan_id: uuid.UUID, payload: StartIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
plan = await _plan_or_404(db, plan_id)
if payload.confirm_name != plan.name:
raise api_error(409, ErrorCode.ROLLOUT_CONFIRM_NAME)
resolved, _token = await _checked_token(db, plan, payload.preview_token)
if (other := await rollout_engine.active_run(db)) is not None:
# One run at a time: two runs could otherwise touch the same cluster at once.
raise api_error(409, ErrorCode.ROLLOUT_ALREADY_ACTIVE, plan=other.plan_name)
run = await rollout_engine.create_run(db, plan, resolved, started_by=request.state.user.username,
started_by_id=request.state.user.id, trigger="manual")
await db.commit()
await audit_service.persist(kind="rollout", action="start", status="started", entity_id=str(run.id), entity_name=plan.name,
details={"trigger": "manual", "hosts": sum(len(w["hosts"]) for w in resolved["waves"]),
"waves": [w["label"] for w in resolved["waves"]]})
return await read_run(db, await _run_or_404(db, run.id))
@router.get("/runs")
async def list_runs(db: AsyncSession = Depends(get_db)):
runs = (await db.scalars(select(RolloutRun).order_by(RolloutRun.created_at.desc()).limit(50))).all()
return [_read_run_summary(r) for r in runs]
@router.get("/status")
async def rollout_status(db: AsyncSession = Depends(get_db)):
"""For /updates and the host pages: the active run and the hosts it still owns."""
run = await rollout_engine.active_run(db)
if run is None:
return {"active": None, "host_ids": [], "current_host_ids": []}
return {
"active": _read_run_summary(run),
"host_ids": [str(h.host_id) for h in run.hosts if h.host_id and h.status in ("queued", "blocked", "running", "verifying", "failed")],
"current_host_ids": [str(h.host_id) for h in run.hosts if h.host_id and h.wave_index == run.current_wave
and h.status in ("queued", "blocked", "running", "verifying", "failed")],
}
@router.get("/runs/{run_id}")
async def get_run(run_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return await read_run(db, await _run_or_404(db, run_id))
@router.post("/runs/{run_id}/pause")
async def pause_run(run_id: uuid.UUID, payload: ReasonIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
run = await _run_or_404(db, run_id, lock=True)
if run.status not in rollout_engine.ACTIVE or run.status == "paused":
raise _not_now(run)
# A brake that lets the next host fire anyway is no brake: a step not fired yet goes back to the queue.
requeued = [h.host_name for h in run.hosts if await rollout_engine.requeue_if_pending(db, h)]
rollout_engine.pause(run, "manual", by=request.state.user.username, text=payload.reason.strip()[:300])
return await _done(db, run, "pause", "paused", request, reason="manual", text=payload.reason.strip()[:300], requeued=requeued)
@router.post("/runs/{run_id}/resume")
async def resume_run(run_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
run = await _run_or_404(db, run_id, lock=True)
if run.status != "paused":
raise _not_now(run)
failed = [h.host_name for h in run.hosts if h.status == "failed"]
if failed:
# A failed host is never retried by itself: check it, then skip it explicitly.
raise api_error(409, ErrorCode.ROLLOUT_HOST_FAILED, hosts=", ".join(failed))
previous = run.started_by
# Whoever resumes takes over the responsibility (and the permission check) from here on.
run.started_by, run.started_by_id = request.state.user.username, request.state.user.id
run.status = "resuming"
run.pause_reason = run.pause_detail = None
run.progress = {**(run.progress or {}), "resume_attempts": 0, "resume_recheck_at": None, "paused_at": None}
return await _done(db, run, "resume", "resuming", request, previous_owner=previous)
@router.post("/runs/{run_id}/approve")
async def approve_wave(run_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
run = await _run_or_404(db, run_id, lock=True)
if run.status != "awaiting_approval":
raise _not_now(run)
wave = rollout_engine.wave_row(run, run.current_wave)
wave.approved_by, wave.status = request.state.user.username, "preflight"
run.started_by, run.started_by_id = request.state.user.username, request.state.user.id
run.status = "preflight"
run.progress = {**(run.progress or {}), "preflight_started": None, "preflight_requested_at": None}
return await _done(db, run, "approve", "approved", request, wave_label=wave.label)
@router.post("/runs/{run_id}/skip-host")
async def skip_host(run_id: uuid.UUID, payload: SkipIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
run = await _run_or_404(db, run_id, lock=True)
if run.status not in rollout_engine.ACTIVE:
raise _not_now(run)
host = next((h for h in run.hosts if h.host_id == payload.host_id), None)
if host is None:
raise api_error(404, ErrorCode.ROLLOUT_HOST_NOT_IN_RUN)
if host.status in ("running", "verifying"):
if not await _cancel_if_pending(db, host.scheduled_maintenance_id):
raise api_error(409, ErrorCode.ROLLOUT_HOST_RUNNING, host=host.host_name)
# not fired yet: nothing happened on the firewall
elif host.status not in ("queued", "blocked", "failed"):
raise api_error(409, ErrorCode.ROLLOUT_HOST_RUNNING, host=host.host_name)
host.status = "skipped"
host.detail = {"code": "skipped_by", "params": {"user": request.state.user.username, "reason": payload.reason.strip()[:300]}}
host.finished_at = datetime.now(timezone.utc)
return await _done(db, run, "skip_host", "skipped", request, host=host.host_name, reason=payload.reason.strip()[:300])
@router.post("/runs/{run_id}/abort")
async def abort_run(run_id: uuid.UUID, payload: AbortIn, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
run = await _run_or_404(db, run_id, lock=True)
if run.status not in rollout_engine.ACTIVE:
raise _not_now(run)
if payload.confirm_name != run.plan_name:
raise api_error(409, ErrorCode.ROLLOUT_CONFIRM_NAME)
now = datetime.now(timezone.utc)
still_running = []
for host in run.hosts:
if host.status in ("running", "verifying") and host.scheduled_maintenance_id:
if await _cancel_if_pending(db, host.scheduled_maintenance_id):
host.status, host.finished_at = "cancelled", now
else:
still_running.append(host.host_name) # an action already on the firewall is finished in a controlled way
elif host.status in ("queued", "blocked"):
host.status, host.finished_at = "cancelled", now
for wave in run.waves:
if wave.status not in ("passed", "skipped"):
wave.status = "cancelled"
run.status, run.finished_at = "cancelled", now
run.pause_reason, run.pause_detail = "aborted", {"by": request.state.user.username, "reason": payload.reason.strip()[:300],
"still_running": still_running}
return await _done(db, run, "abort", "cancelled", request, reason=payload.reason.strip()[:300], still_running=still_running)
+128
View File
@@ -0,0 +1,128 @@
"""Rule analysis: which rule decides a packet (read-only, GET so ``read`` tokens and MCP can use it).
Nothing is stored and nothing is written to a firewall; every request reads the rules, the
loaded pf ruleset and the interfaces (cached for 60 s per firewall, ``refresh=true`` skips
the cache) and evaluates them with services/rule_analysis/engine.py.
"""
import asyncio
import logging
import uuid
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup
from app.models.rule_history import LiveRuleState
from app.services import rule_history_service
from app.services.rule_analysis import service
logger = logging.getLogger(__name__)
router = APIRouter(tags=["rule-analysis"])
_PARALLEL = 5
def _query(src: str, dst: str, proto: str, sport: str | None, dport: str | None, icmptype: str | None,
iface: str | None, direction: str, egress: str | None):
return service.build_query(src=src, dst=dst, proto=proto, sport=sport, dport=dport, icmptype=icmptype,
iface=iface, direction=direction, egress=egress)
async def _run(host: OpnsenseHost, q, refresh: bool) -> dict:
head = {"host_id": str(host.id), "host_name": host.name, "cluster_name": host.cluster_name}
try:
return {**head, **await service.analyze_host(host, q, refresh=refresh), "error": None}
except HTTPException as exc:
return {**head, "error": exc.detail}
except Exception as exc: # noqa: BLE001 - one unreachable firewall must not fail the fleet answer
logger.info("rule analysis failed for %s: %s", host.name, exc)
return {**head, "error": {"code": ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR.value, "params": {}}}
@router.get("/hosts/{host_id}/rule-analysis/match")
async def host_match(
host_id: uuid.UUID, src: str, dst: str, proto: str = "tcp", sport: str | None = None, dport: str | None = None,
icmptype: str | None = None, iface: str | None = None, direction: str = "in", egress: str | None = None,
refresh: bool = False, db: AsyncSession = Depends(get_db),
):
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
q = _query(src, dst, proto, sport, dport, icmptype, iface, direction, egress)
result = await _run(host, q, refresh)
error = result.get("error")
if error:
status = 422 if error.get("code") == ErrorCode.ANALYSE_INTERFACE_UNBEKANNT.value else 502
raise HTTPException(status_code=status, detail=error)
return result
@router.get("/rule-analysis/match")
async def fleet_match(
src: str, dst: str, proto: str = "tcp", sport: str | None = None, dport: str | None = None,
icmptype: str | None = None, iface: str | None = None, direction: str = "in", egress: str | None = None,
host_ids: str | None = Query(None, description="comma separated host ids; default: all enabled hosts"),
group_id: uuid.UUID | None = None, refresh: bool = False, db: AsyncSession = Depends(get_db),
):
q = _query(src, dst, proto, sport, dport, icmptype, iface, direction, egress)
if group_id is not None:
group = await db.scalar(select(DeviceGroup).options(selectinload(DeviceGroup.hosts)).where(DeviceGroup.id == group_id))
if group is None:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
hosts = sorted(group.hosts, key=lambda h: h.name)
elif host_ids:
try:
ids = [uuid.UUID(part.strip()) for part in host_ids.split(",") if part.strip()]
except ValueError:
raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="host_ids", value=host_ids[:64]) from None
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(ids)).order_by(OpnsenseHost.name))).all())
if len(hosts) != len(set(ids)):
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
else:
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all())
gate = asyncio.Semaphore(_PARALLEL)
async def one(host):
async with gate:
return await _run(host, q, refresh)
return list(await asyncio.gather(*(one(h) for h in hosts)))
@router.get("/hosts/{host_id}/rule-analysis/cleanup")
async def host_cleanup(
host_id: uuid.UUID, unused_days: int = Query(30, ge=1, le=3650, description="observation window for 'unused' / 'disabled for a long time'"),
refresh: bool = False, db: AsyncSession = Depends(get_db),
):
"""Cleanup candidates (unused, shadowed, duplicate, disabled for a long time) with confidence and whether
each rule could be offered for removal. Read-only; removal is not part of this endpoint."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return await service.cleanup_candidates(db, host, unused_days=unused_days, refresh=refresh)
except HTTPException:
raise
except Exception as exc: # noqa: BLE001 - an unreachable firewall is a 502, not a crash
logger.info("cleanup candidates failed for %s: %s", host.name, exc)
raise api_error(502, ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR) from exc
@router.get("/hosts/{host_id}/live-rules/{rule_uuid}/history")
async def live_rule_history(host_id: uuid.UUID, rule_uuid: str, limit: int = Query(100, ge=1, le=500),
db: AsyncSession = Depends(get_db)):
"""What happened to one rule on one firewall: observed changes (also hand-made ones), saved revisions of
rules OPNexus manages and the audit entries, newest first. Reads the database only."""
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
baseline = await db.get(LiveRuleState, (host_id, rule_history_service.BASELINE_KEY))
return {
"host_id": str(host.id), "rule_uuid": rule_uuid.lower()[:64],
"watching_since": baseline.first_seen_at if baseline else None,
"events": await rule_history_service.rule_timeline(db, host, rule_uuid, limit),
}
+194
View File
@@ -0,0 +1,194 @@
"""Rule cleanup (1.31, B4 part 2): switch hand-made rules off, delete them later, restore them (DESIGN-1.31 §4).
POST /hosts/{id}/rule-cleanup/preview operator, admin prepare (writes nothing to a firewall)
POST /hosts/{id}/rule-cleanup/{op}/cancel operator, admin the preview's own actor
POST /hosts/{id}/rule-cleanup/{op}/deploy ADMIN, token scope admin
POST /hosts/{id}/rule-cleanup/{op}/confirm operator, admin BROWSER SESSION ONLY, never a token
POST /hosts/{id}/rule-cleanup/{op}/revert admin; operator only with operators_may_deploy
POST /hosts/{id}/rule-cleanup/{op}/retry ADMIN, token scope admin
POST /hosts/{id}/rule-cleanup/items/{item}/restore ADMIN, token scope admin
GET /hosts/{id}/rule-cleanup, /{op} everybody signed in (read tokens too)
Who may do what is enforced twice: AuthMiddleware (services/api_token_service.py: _ADMIN_ONLY_CLEANUP,
_SESSION_ONLY, _ROLLOUT_POST) and here, so a gap in one layer does not open the route. The audit trail is the
service's (one ``rule_cleanup`` entry per step and rule) plus the middleware's request entry (kind ``host``,
action = last path segment; ``/preview`` is not logged by it, the service writes its own entry).
The services raise ``api_error``; nothing is translated here. MCP stays read-only (no tool, ``api_get`` is GET).
"""
import uuid
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.user import AdminUser
from app.routers import governance
from app.routers.auth import resolve_user
from app.schemas.rule_cleanup import CleanupOperationRead, CleanupPreviewRead, ConfirmName, PreviewRequest
from app.services import rule_cleanup_service as service
from app.services.rule_cleanup_service import ACK_TEXT_VERSION
router = APIRouter(prefix="/hosts/{host_id}/rule-cleanup", tags=["rule-cleanup"])
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
def _token(request: Request):
return getattr(request.state, "api_token", None)
def _writer(user: AdminUser, request: Request) -> None:
"""Any write: not for a read-only account, not with a read-scope token."""
if user.role == "read_only":
raise api_error(403, ErrorCode.READ_ONLY_ACCESS)
token = _token(request)
if token is not None and token.scope == "read":
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
def _admin_only(user: AdminUser, request: Request) -> None:
"""Deploy, retry, restore: an admin account, and with a token only one of scope admin."""
_writer(user, request)
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
token = _token(request)
if token is not None and token.scope != "admin":
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
async def _deployer(user: AdminUser, request: Request, db: AsyncSession) -> None:
"""Revert: an admin, or an operator while the governance switch operators_may_deploy is on."""
_writer(user, request)
if user.role == "admin":
return
if user.role == "operator" and (await governance.get_settings(db)).operators_may_deploy:
return
raise api_error(403, ErrorCode.DEPLOY_NEEDS_ADMIN)
def _session_only(user: AdminUser, request: Request) -> None:
"""Confirm: "I still have access" is a human decision in a browser session, never a token's."""
_writer(user, request)
token = _token(request)
if token is not None:
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_FORBIDDEN, scope=token.scope)
def _full_view(request: Request) -> bool:
"""Admins (session or admin-scope token) see the management sources/ports and firewall addresses in a plan."""
from app.routers.audit import full_view
return full_view(request)
async def _names(db: AsyncSession, operations: list[dict], request: Request) -> list[dict]:
"""The firewall names the UI shows next to the ids; for non-admins without the security-relevant plan parts."""
if not _full_view(request):
operations = [service.redact(op) for op in operations]
ids = {uuid.UUID(op["host_id"]) for op in operations} | {uuid.UUID(i["host_id"]) for op in operations for i in op["items"]}
names = {h.id: h.name for h in (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(ids)))).all()} if ids else {}
for op in operations:
op["host_name"] = names.get(uuid.UUID(op["host_id"]))
for item in op["items"]:
item["host_name"] = names.get(uuid.UUID(item["host_id"]))
return operations
async def _one(db: AsyncSession, operation: dict, request: Request) -> dict:
return (await _names(db, [operation], request))[0]
@router.get("", response_model=list[CleanupOperationRead])
async def list_operations(host_id: uuid.UUID, request: Request, limit: int = Query(50, ge=1, le=100),
db: AsyncSession = Depends(get_db)):
"""The newest operations of this host (and its cluster peers) with their items; unexpired previews included."""
host = await _host(db, host_id)
return await _names(db, await service.list_operations(db, host, limit), request)
@router.post("/preview", response_model=CleanupPreviewRead)
async def preview(host_id: uuid.UUID, payload: PreviewRequest, request: Request, db: AsyncSession = Depends(get_db)):
"""Recompute the plan from the live firewall(s) and store a 5-minute preview bound to the caller."""
user = await resolve_user(request, db)
_writer(user, request)
host = await _host(db, host_id)
kind, uuids, acknowledge = payload.resolved()
if kind is None:
raise api_error(422, ErrorCode.BEREINIGUNG_AUSWAHL_UNGUELTIG, field="kind")
result = await service.preview(
db, host, actor=user.username, actor_id=user.id, kind=kind, uuids=uuids, acknowledge=acknowledge,
**({"unused_days": payload.unused_days} if payload.unused_days else {}),
ack_text_version=payload.ack_text_version or ACK_TEXT_VERSION)
result["operation"] = await _one(db, await service.get_operation(db, host, uuid.UUID(result["preview_id"])), request)
return result if _full_view(request) else service.redact(result)
@router.get("/{op_id}", response_model=CleanupOperationRead)
async def get_operation(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
return await _one(db, await service.get_operation(db, host, op_id), request)
@router.post("/{op_id}/cancel", response_model=CleanupOperationRead)
async def cancel(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Drop an unused preview (only its own actor can)."""
user = await resolve_user(request, db)
_writer(user, request)
host = await _host(db, host_id)
return await _one(db, await service.cancel(db, host, op_id, actor=user.username), request)
@router.post("/{op_id}/deploy", response_model=CleanupOperationRead)
async def deploy(host_id: uuid.UUID, op_id: uuid.UUID, payload: ConfirmName, request: Request,
db: AsyncSession = Depends(get_db)):
"""Execute a preview. Admin only; the live state must still give the fingerprint of the preview."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.deploy(db, host, op_id, actor=user.username, confirm_name=payload.confirm_name), request)
@router.post("/{op_id}/confirm", response_model=CleanupOperationRead)
async def confirm(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
""""I still have access": re-reads every node first. Browser session only, never with an API token."""
user = await resolve_user(request, db)
_session_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.confirm(db, host, op_id), request)
@router.post("/{op_id}/revert", response_model=CleanupOperationRead)
async def revert(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Take the change back now (the same code the sweeper runs when the window passes)."""
user = await resolve_user(request, db)
await _deployer(user, request, db)
host = await _host(db, host_id)
return await _one(db, await service.revert_now(db, host, op_id), request)
@router.post("/{op_id}/retry", response_model=CleanupOperationRead)
async def retry(host_id: uuid.UUID, op_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
"""Revert again after revert_failed, or read the firewall again for a delete/restore with an unclear item."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.retry(db, host, op_id), request)
@router.post("/items/{item_id}/restore", response_model=CleanupOperationRead)
async def restore(host_id: uuid.UUID, item_id: uuid.UUID, payload: ConfirmName, request: Request,
db: AsyncSession = Depends(get_db)):
"""Bring a deleted rule back, switched OFF, at its old position (a new UUID). The result is the restore run;
``warnings`` can hold BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR."""
user = await resolve_user(request, db)
_admin_only(user, request)
host = await _host(db, host_id)
return await _one(db, await service.restore_item(
db, host, item_id, actor=user.username, actor_id=user.id, confirm_name=payload.confirm_name), request)
+14 -2
View File
@@ -1,3 +1,4 @@
import json
import uuid
from fastapi import APIRouter, Depends, Body
@@ -18,6 +19,7 @@ from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.rule_deploy_service import confirm_rule, deploy_rule, undeploy_rule, revert_deployment
from app.services import rule_adopt_service
from app.services.adopt_common import LINK_ERRORS, link_api_error
from app.schemas.rule import AdoptRequest, UnmanagedRule
router = APIRouter(prefix="/rules", tags=["rules"])
@@ -50,6 +52,7 @@ def _to_read(
return RuleRead(
workflow_state=state,
workflow_counts=counts, id=rule.id,
config_key=rule.config_key,
description=rule.description,
interface=rule.interface,
direction=rule.direction,
@@ -106,12 +109,19 @@ async def adopt_rule(payload: AdoptRequest, db: AsyncSession = Depends(get_db)):
host = await db.get(OpnsenseHost, payload.host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
# Taking over a hand-made rule that a rule cleanup (1.31) is switching off/deleting would make it "managed"
# under the cleanup's feet: not while one is open on this host or its cluster.
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [host])
try:
rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid)
rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid, link_to=payload.link_to)
except rule_adopt_service.RuleNotFoundOnHost as exc:
raise api_error(404, ErrorCode.REGEL_NICHT_GEFUNDEN) from exc
except rule_adopt_service.NotAdoptable as exc:
raise api_error(409, ErrorCode.RULE_NOT_ADOPTABLE, reasons=", ".join(exc.reasons)) from exc
raise api_error(409, ErrorCode.RULE_NOT_ADOPTABLE, reasons=", ".join(exc.reasons),
reason_details=json.dumps(exc.details, ensure_ascii=False)) from exc
except LINK_ERRORS as exc:
raise link_api_error(exc) from exc
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
await db.commit()
@@ -271,6 +281,8 @@ async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approva
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_rule_uuid)
await opnsense_client.apply(deployment.host)
+8 -1
View File
@@ -24,6 +24,8 @@ class Schedule(BaseModel):
acknowledge_unattended: Literal[True]
# Hand the traffic over to the CARP partner first and switch back afterwards (cluster nodes only).
carp_handover: bool = False
# Check the result automatically afterwards and close the operation (no manual "Wartung abschließen").
auto_verify: bool = False
def _read(sched: ScheduledMaintenance) -> dict:
@@ -32,6 +34,8 @@ def _read(sched: ScheduledMaintenance) -> dict:
"status": sched.status, "actor": sched.actor, "created_at": sched.created_at,
"result": sched.result, "host_operation_id": sched.host_operation_id,
"carp_handover": sched.carp_handover, "phase": sched.phase,
"auto_verify": sched.auto_verify, "origin": sched.origin,
"rollout_run_id": ((sched.progress or {}).get("rollout") or {}).get("run_id"),
}
@@ -80,6 +84,7 @@ async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Reque
sched = ScheduledMaintenance(
host_id=host.id, host_name=host.name, action=payload.action, run_at=run_at,
actor=request.state.user.username, created_at=now, carp_handover=payload.carp_handover,
auto_verify=payload.auto_verify,
)
db.add(sched)
await db.commit()
@@ -90,7 +95,9 @@ async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Reque
async def cancel_scheduled(host_id: uuid.UUID, schedule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _host_or_404(db, host_id)
sched = await db.get(ScheduledMaintenance, schedule_id)
# Locked and re-read: the poll loop may be firing this very row right now (it holds the lock meanwhile).
sched = await db.scalar(select(ScheduledMaintenance).where(ScheduledMaintenance.id == schedule_id)
.with_for_update().execution_options(populate_existing=True))
if not sched or sched.host_id != host_id:
raise api_error(404, ErrorCode.GEPLANTE_WARTUNG_NICHT_GEFUNDEN)
if sched.status != "pending":
+64 -6
View File
@@ -1,19 +1,23 @@
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.services import rollout_engine
from app.services import update_check_service, updater_client
router = APIRouter(prefix="/system", tags=["system"])
@router.get("/version-check")
async def version_check() -> dict:
async def version_check(db: AsyncSession = Depends(get_db)) -> dict:
"""Ist eine neuere OPNexus-Version veröffentlicht? Aus, solange
UPDATE_CHECK_URL nicht gesetzt ist (dann ohne ausgehende Anfrage)."""
return await update_check_service.check()
return await update_check_service.check(db)
def _admin(user: AdminUser) -> AdminUser:
@@ -31,16 +35,63 @@ def _major(version: str | None) -> int | None:
return None
class UpdateCheckRead(BaseModel):
enabled: bool
feed_url: str
managed_by_server: bool
class UpdateCheckUpdate(BaseModel):
enabled: bool | None = None
feed_url: str | None = Field(default=None, max_length=500)
async def _update_check_settings(db: AsyncSession) -> UpdateCheckRead:
row = await update_check_service.get_settings(db)
managed = update_check_service.managed_by_server()
return UpdateCheckRead(
enabled=True if managed else row.enabled,
feed_url=settings.update_check_url.strip() if managed else row.feed_url,
managed_by_server=managed,
)
@router.get("/update-check", response_model=UpdateCheckRead)
async def read_update_check(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Versionsprüfung für „Einstellungen › OPNexus aktualisieren“ (nur Admins)."""
_admin(user)
return await _update_check_settings(db)
@router.put("/update-check", response_model=UpdateCheckRead)
async def update_update_check(payload: UpdateCheckUpdate, db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user)):
_admin(user)
if update_check_service.managed_by_server():
raise api_error(409, ErrorCode.UPDATE_CHECK_MANAGED_BY_SERVER)
row = await update_check_service.get_settings(db)
if payload.feed_url is not None:
url = payload.feed_url.strip()
if not update_check_service.valid_feed_url(url):
raise api_error(422, ErrorCode.UPDATE_CHECK_URL_INVALID)
row.feed_url = url
if payload.enabled is not None:
row.enabled = payload.enabled
await db.commit()
update_check_service.reset_cache()
return await _update_check_settings(db)
class UpdateRequest(BaseModel):
version: str = Field(min_length=5, max_length=20)
@router.get("/updater")
async def updater_status(user: AdminUser = Depends(current_user)) -> dict:
async def updater_status(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)) -> dict:
"""Zustand für den Abschnitt "OPNexus aktualisieren" in den Einstellungen:
Versionshinweis des Release-Feeds plus (falls aktiviert) der Updater."""
_admin(user)
check = await update_check_service.check()
check = await update_check_service.check(db)
result: dict = {
"available": False, "reason": None, "mode": None,
"current": check["current"], "latest": check["latest"], "update_available": check["update_available"],
@@ -49,11 +100,17 @@ async def updater_status(user: AdminUser = Depends(current_user)) -> dict:
"can_update": False, "major_change": False,
"job": None, "phases": [], "log": [], "services": {},
"rollback_available": False, "rollback_loses_data": False, "rollback_version": None,
# A restart in the middle of a firmware rollout is safe (its state is in the database), but the
# UI warns: the run pauses its 15-second steps for as long as OPNexus is down.
"active_rollouts": await rollout_engine.count_active(db),
}
try:
updater = await updater_client.status()
except updater_client.UpdaterUnavailable as exc:
result["reason"] = str(exc)
# Without the updater the install mode is still known from the image registry setting; the UI uses it
# to not offer enabling the updater on an install that builds from source (it could not update anyway).
result["mode"] = "release" if settings.opnexus_image_repo.strip() else "local"
return result
except updater_client.UpdaterRejected:
result["reason"] = "unreachable"
@@ -77,11 +134,12 @@ async def updater_status(user: AdminUser = Depends(current_user)) -> dict:
@router.post("/updater/update", status_code=202)
async def updater_update(payload: UpdateRequest, user: AdminUser = Depends(current_user)) -> dict:
async def updater_update(payload: UpdateRequest, db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user)) -> dict:
"""Aktualisiert OPNexus auf die vom Release-Feed gemeldete neueste stabile
Version. Beliebige Versionen werden bewusst nicht angenommen."""
_admin(user)
check = await update_check_service.check()
check = await update_check_service.check(db)
if not check["update_available"] or payload.version.lstrip("v") != check["latest"]:
raise api_error(409, ErrorCode.UPDATE_NOT_OFFERED)
if _major(check["latest"]) != _major(check["current"]):
+35 -12
View File
@@ -1,7 +1,7 @@
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy import func, select
from sqlalchemy import func, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
@@ -21,10 +21,29 @@ async def _admin_count(db: AsyncSession, exclude_id: uuid.UUID | None = None) ->
return await db.scalar(query)
_ADMIN_LOCK_KEY = 728194032 # next to policy_lock's key; only admin role changes and deletions take it
async def _lock_admins(db: AsyncSession, current: AdminUser) -> None:
"""Serialize every change that can reduce the number of administrators and re-read the caller under the lock.
The "at least one administrator" check counts rows; two requests that each see the other's admin row would both
pass it. Held until commit/rollback (transaction-scoped advisory lock). The caller may have been demoted or
deleted by the request that ran first, so it must still be an admin to proceed."""
await db.execute(text("SELECT pg_advisory_xact_lock(:key)"), {"key": _ADMIN_LOCK_KEY})
fresh = await db.scalar(select(AdminUser).where(AdminUser.id == current.id).execution_options(populate_existing=True))
if fresh is None or fresh.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
@router.get("", response_model=list[UserRead])
async def list_users(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(AdminUser).order_by(AdminUser.username))).all()
return rows
async def list_users(user: AdminUser = Depends(current_user), db: AsyncSession = Depends(get_db)):
"""Admins see everyone. Every other role sees only its own account: the full list (names, roles, last login,
2FA status) is a directory an attacker with any low-privilege account or token could use to pick targets."""
query = select(AdminUser).order_by(AdminUser.username)
if user.role != "admin":
query = query.where(AdminUser.id == user.id)
return (await db.scalars(query)).all()
@router.post("", response_model=UserRead, status_code=201)
@@ -43,8 +62,12 @@ async def create_user(payload: UserCreate, db: AsyncSession = Depends(get_db)):
@router.patch("/{user_id}/role", response_model=UserRead)
async def update_role(user_id: uuid.UUID, payload: UserRoleUpdate, db: AsyncSession = Depends(get_db)):
target = await db.get(AdminUser, user_id)
async def update_role(
user_id: uuid.UUID, payload: UserRoleUpdate, db: AsyncSession = Depends(get_db),
current: AdminUser = Depends(current_user),
):
await _lock_admins(db, current)
target = await db.get(AdminUser, user_id, populate_existing=True)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
if target.auth_source == "oidc":
@@ -96,13 +119,13 @@ async def delete_user(
):
if user_id == current.id:
raise api_error(400, ErrorCode.DER_EIGENE_ACCOUNT_KANN_NICHT_GELOESCHT_WERDEN)
target = await db.get(AdminUser, user_id)
await _lock_admins(db, current)
target = await db.get(AdminUser, user_id, populate_existing=True)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
# No separate "last admin" guard here: only an admin can reach this
# endpoint (read_only is rejected earlier, in AuthMiddleware) and the
# self-delete check above already guarantees target != current — so
# `current` is always a still-existing admin afterward, meaning the
# admin count can never reach zero this way.
# Two admins deleting each other (or demoting/deleting in parallel) would each still see the other: the lock above
# orders them, and the guard below counts what is left without the target.
if target.role == "admin" and await _admin_count(db, exclude_id=target.id) == 0:
raise api_error(409, ErrorCode.MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN)
await db.delete(target)
await db.commit()
+98
View File
@@ -0,0 +1,98 @@
"""Site variables of one host: the values for {{NAME}} placeholders, and which placeholders its objects use."""
import uuid
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.host_variable import HostVariable
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.services import alias_deploy_service, one_to_one_deploy_service, outbound_nat_deploy_service
from app.services import port_forward_deploy_service, rule_deploy_service, variables_service
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
from app.services.policy_service import effective_hosts
router = APIRouter(prefix="/hosts/{host_id}/variables", tags=["variables"])
_KINDS = (
("rule", FirewallRule, rule_deploy_service),
("alias", FirewallAlias, alias_deploy_service),
("port_forward", PortForwardRule, port_forward_deploy_service),
("one_to_one", OneToOneRule, one_to_one_deploy_service),
("outbound_nat", OutboundNatRule, outbound_nat_deploy_service),
)
class VariablesUpdate(BaseModel):
variables: dict[str, str] = Field(default_factory=dict, max_length=200)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
async def _usage(db: AsyncSession, host: OpnsenseHost) -> dict[str, list[dict]]:
"""name -> objects (that are deployed to this host) which use it."""
used: dict[str, list[dict]] = {}
for kind, model, service in _KINDS:
entities = (await db.execute(
select(model).options(selectinload(model.targets), selectinload(model.groups).selectinload(DeviceGroup.hosts))
)).scalars().unique().all()
await hydrate_hierarchy_targets(db, entities)
for entity in entities:
if host.id not in {h.id for h in effective_hosts(entity)}:
continue
label = getattr(entity, "description", None) or getattr(entity, "name", None) or str(entity.id)
for name in variables_service.names_in_payload(service._base_payload(entity)):
used.setdefault(name, []).append({"kind": kind, "id": str(entity.id), "label": label})
return used
async def _read(db: AsyncSession, host: OpnsenseHost) -> dict:
values = variables_service.host_values(host)
used = await _usage(db, host)
names = sorted(set(values) | set(used))
return {"variables": [
{"name": n, "value": values.get(n), "used_by": used.get(n, []), "missing": n in used and n not in values}
for n in names
]}
@router.get("")
async def read(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return await _read(db, await _host(db, host_id))
@router.put("")
async def replace(host_id: uuid.UUID, payload: VariablesUpdate, db: AsyncSession = Depends(get_db)):
"""Replace this host's variables with exactly the given set."""
host = await _host(db, host_id)
for name, value in payload.variables.items():
if not variables_service.NAME_RE.fullmatch(name):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Name „{name}“: Großbuchstaben, Ziffern und _, beginnend mit einem Buchstaben")
if not variables_service.VALUE_RE.fullmatch(value):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Wert von {name}: 1–255 Zeichen aus Buchstaben, Ziffern und . : / , @ + - _ Leerzeichen")
existing = {v.name: v for v in host.variables}
for name in set(existing) - set(payload.variables):
host.variables.remove(existing[name])
for name, value in payload.variables.items():
if name in existing:
existing[name].value = value
else:
host.variables.append(HostVariable(name=name, value=value))
await db.commit()
await db.refresh(host)
return await _read(db, host)
+4
View File
@@ -18,6 +18,7 @@ from app.services import wireguard_key_store
from app.services.wireguard_service import effective_hosts, preview, revision, snapshot, workflow, WIREGUARD_FIELDS
from app.services.wireguard_deploy_service import deploy_tunnel, undeploy_tunnel
from app.services.opnsense_client import OpnsenseAPIError
from app.services.deploy_lease import resource_lease
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/wireguard/tunnels", tags=["wireguard"])
@@ -206,7 +207,10 @@ async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, appro
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
async with resource_lease("wireguard", tunnel.id):
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services.host_maintenance_service import ensure_no_open_cleanup
await ensure_no_open_cleanup(db, [deployment.host]) # a rule cleanup (1.31) owns the firewall: no other writes
from app.services import opnsense_client
try:
if deployment.opnsense_peer_uuid:
+76
View File
@@ -0,0 +1,76 @@
import uuid
from pydantic import BaseModel, Field
class AdoptObjectRequest(BaseModel):
host_id: uuid.UUID
# Link to this existing central object instead of creating a new one (services/adopt_link_service.py).
# Declared before `uuid`: that field's name shadows the module inside the class body.
link_to: uuid.UUID | None = None
uuid: str = Field(min_length=1, max_length=64) # the object's UUID on the firewall
class AdoptAllRequest(BaseModel):
host_id: uuid.UUID
# Link entries that have exactly one exact central twin instead of creating a duplicate.
link_existing: bool = False
class LinkDiff(BaseModel):
field: str # model field ("content", "destination_port", ...) or "variables" (a {{VARIABLE}} has no value on this host)
central: str
live: str
class LinkTarget(BaseModel):
"""An existing central object that a live object could be linked to (exact match) or is similar to."""
id: uuid.UUID
name: str
targets: list[str] # names of the hosts it is deployed to
diff: list[LinkDiff] = Field(default_factory=list) # empty for an exact match
class UnmanagedObject(BaseModel):
"""An alias / NAT rule found on a host that OPNexus does not manage yet (see object_adopt_service.py)."""
uuid: str
title: str
enabled: bool
adoptable: bool # can be adopted on its own right now
# Adoptable in the same "adopt all" run although not on its own (an alias whose referenced aliases come first).
bulk_adoptable: bool
# Codes such as "unsupported_type" or "no_target"; the UI turns them into text.
reasons: list[str]
reason_details: dict[str, str] = Field(default_factory=dict) # code -> extra text (alias names, type, ...)
# Settings OPNexus does not manage that stay untouched on the firewall (log, pool options, tags, ...).
kept: list[str] = Field(default_factory=list)
# Facts for display, depend on the object type.
info: dict[str, str] = Field(default_factory=dict)
# Central objects whose content, resolved for this host, equals the live object exactly (first = preferred).
matches: list[LinkTarget] = Field(default_factory=list)
# Objects with the same name/description but different content: a hint with the differing fields, never linked.
similar: list[LinkTarget] = Field(default_factory=list)
linkable: bool = False # can be linked to `matches[0]` (or another match) on its own
# What "adopt all" would do: "link" | "new" | "skip" (None only on the adopt-all result).
bulk_action: str | None = None
class AdoptedObject(BaseModel):
uuid: str
title: str
id: uuid.UUID
linked: bool = False # linked to an existing central object instead of newly created
class SkippedObject(BaseModel):
uuid: str
title: str
reasons: list[str]
reason_details: dict[str, str] = Field(default_factory=dict)
class AdoptAllResult(BaseModel):
adopted: list[AdoptedObject]
skipped: list[SkippedObject]
+3 -1
View File
@@ -12,7 +12,9 @@ CertAction = Literal["import", "internal", "external", "sign_csr", "import_csr",
# EC curves) — see ManagedCertificate's docstring.
CertKeyType = Literal["2048", "3072", "4096", "prime256v1", "secp384r1", "secp521r1"]
CertDigest = Literal["sha1", "sha224", "sha256", "sha384", "sha512"]
CertType = Literal["usr_cert", "server_cert", "combined_server_client", "v3_ca"]
# No "v3_ca": OPNsense would create a certificate with the CA extension, not a CA that can be used as `caref`
# (found by the 1.31.0 test run, T3). Creating a CA is not offered; the UI never did.
CertType = Literal["usr_cert", "server_cert", "combined_server_client"]
_DN_ACTIONS = {"internal", "external", "reissue"}
_CRT_ACTIONS = {"import", "import_csr", "manual"}
+7
View File
@@ -3,6 +3,8 @@ from datetime import datetime
from pydantic import BaseModel
from app.schemas.gateway import GatewayHealth
class DashboardHost(BaseModel):
id: uuid.UUID
@@ -10,6 +12,11 @@ class DashboardHost(BaseModel):
status: str
carp_status: str | None
cluster_name: str | None
# From the last resource sample (no live call); None when there is no sample from the last 3 minutes.
gateways: list[GatewayHealth] | None = None
# Readings the newest fresh sample could not take because the API key lacks the right (gateways, services, ...);
# None = no fresh sample (or one from before this existed), [] = nothing missing.
api_rights_missing: list[str] | None = None
class PendingAction(BaseModel):
+33
View File
@@ -0,0 +1,33 @@
from typing import Literal
from pydantic import BaseModel
# online: all fine · degraded: OPNsense flags loss and/or latency above the gateway's own limits ·
# offline: down · forced_down: switched off by the admin on purpose · unknown: nothing measured
# (monitoring disabled) or the verdict is still pending.
GatewayState = Literal["online", "degraded", "offline", "forced_down", "unknown"]
class GatewayHealth(BaseModel):
name: str
address: str | None = None
status: str # OPNsense' raw verdict: none | down | force_down | delay | loss | delay+loss | …
state: GatewayState
loss: float | None = None # percent
delay: float | None = None # milliseconds
stddev: float | None = None # milliseconds
monitor: str | None = None # the monitored address
alert_enabled: bool = True # False: the gateway alert is switched off for this gateway (Settings, 1.29)
class GatewayAlertUpdate(BaseModel):
enabled: bool
class GatewayAlertState(BaseModel):
name: str
alert_enabled: bool
class ApiRightsState(BaseModel):
missing: list[str] | None = None # names of the refused readings (gateways, services, swap, cpu); None = unknown
+3
View File
@@ -18,6 +18,9 @@ class HostCreate(BaseModel):
enabled: bool = True
cluster_name: str | None = Field(default=None, max_length=120)
policy_hierarchy_group_id: uuid.UUID | None = None
# Ask the firewall once with these credentials before saving (the UI and the setup assistant do). Off by default so
# scripts can still register a firewall that is not reachable yet (behind NAT, tunnel not up).
verify_connection: bool = False
@field_validator("base_url")
@classmethod
+110 -1
View File
@@ -1,10 +1,12 @@
import re
import uuid
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.config import settings
from app.models.notification import DEFAULT_SERVICE_WATCHLIST
SmtpSecurity = Literal["none", "starttls", "ssl"]
ProblemType = Literal[
@@ -13,7 +15,11 @@ ProblemType = Literal[
"scheduled_maintenance_failed",
"backup_stale",
"backup_offsite_failed",
"compliance_finding",
"gateway_down", "resource_threshold", "service_down", "api_rights_missing",
"rollout_paused", "rollout_start_failed",
]
ChannelType = Literal["ntfy", "gotify", "telegram", "teams", "matrix"]
class NotificationSettingsRead(BaseModel):
@@ -43,10 +49,31 @@ class NotificationSettingsRead(BaseModel):
carp_auto_revert_failed_enabled: bool
scheduled_maintenance_failed_enabled: bool = True
backup_failed_enabled: bool = True
compliance_alerts_enabled: bool = False
gateway_down_enabled: bool = True
resource_threshold_enabled: bool = False
service_down_enabled: bool = False
rollout_problems_enabled: bool = True
api_rights_enabled: bool = True
gateway_alert_muted: list[str] = Field(default_factory=list)
gateway_loss_threshold_pct: int = 20
gateway_delay_threshold_ms: int = 500
disk_threshold_pct: int = 90
memory_threshold_pct: int = 95
swap_threshold_pct: int = 50
load_per_cpu_threshold: float = 2.0
alert_confirm_samples: int = 3
resource_confirm_samples: int = 5
service_watchlist: list[str] = Field(default_factory=lambda: list(DEFAULT_SERVICE_WATCHLIST))
poll_interval_seconds: int
updated_at: datetime
@field_validator("service_watchlist", mode="before")
@classmethod
def _default_watchlist(cls, value):
return list(DEFAULT_SERVICE_WATCHLIST) if value is None else value
class NotificationSettingsUpdate(BaseModel):
"""Partial update — everything optional, only provided fields change.
@@ -81,6 +108,36 @@ class NotificationSettingsUpdate(BaseModel):
carp_auto_revert_failed_enabled: bool | None = None
scheduled_maintenance_failed_enabled: bool | None = None
backup_failed_enabled: bool | None = None
compliance_alerts_enabled: bool | None = None
gateway_down_enabled: bool | None = None
resource_threshold_enabled: bool | None = None
service_down_enabled: bool | None = None
rollout_problems_enabled: bool | None = None
api_rights_enabled: bool | None = None
# 0 switches that single threshold off.
gateway_loss_threshold_pct: int | None = Field(default=None, ge=0, le=100)
gateway_delay_threshold_ms: int | None = Field(default=None, ge=0, le=60000)
disk_threshold_pct: int | None = Field(default=None, ge=0, le=100)
memory_threshold_pct: int | None = Field(default=None, ge=0, le=100)
swap_threshold_pct: int | None = Field(default=None, ge=0, le=100)
load_per_cpu_threshold: float | None = Field(default=None, ge=0, le=1000)
alert_confirm_samples: int | None = Field(default=None, ge=1, le=60)
resource_confirm_samples: int | None = Field(default=None, ge=1, le=60)
service_watchlist: list[str] | None = Field(default=None, max_length=100)
@field_validator("service_watchlist")
@classmethod
def _clean_watchlist(cls, value):
if value is None:
return None
names = []
for raw in value:
name = str(raw).strip()
if not re.fullmatch(r"[A-Za-z0-9._+-]{1,64}", name):
raise ValueError("Dienstnamen bestehen aus Buchstaben, Ziffern und . _ + -")
if name not in names:
names.append(name)
return names
poll_interval_seconds: int | None = Field(
default=None, ge=settings.notification_poll_min_interval_seconds
@@ -103,3 +160,55 @@ class NotificationProblemRead(BaseModel):
class TestNotificationRequest(BaseModel):
channel: Literal["webhook", "email"]
class NotificationChannelRead(BaseModel):
"""Never carries the credential; ``config`` is withheld from non-admins (an ntfy topic is a password)."""
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
channel_type: ChannelType
enabled: bool
config: dict
secret_set: bool
created_at: datetime
updated_at: datetime
def _channel_name(value: str | None) -> str | None:
if value is None:
return None
if not value.strip():
raise ValueError("Name darf nicht leer sein")
return value.strip()
class NotificationChannelCreate(BaseModel):
name: str = Field(min_length=1, max_length=100)
channel_type: ChannelType
enabled: bool = True
config: dict = Field(default_factory=dict)
# Token, or for Teams the webhook address. Write-only.
secret: str | None = Field(default=None, max_length=2100)
@field_validator("name")
@classmethod
def _clean_name(cls, value):
return _channel_name(value)
class NotificationChannelUpdate(BaseModel):
"""Partial: omitted fields stay. ``secret`` replaces the credential, ``clear_secret`` removes it."""
name: str | None = Field(default=None, min_length=1, max_length=100)
enabled: bool | None = None
config: dict | None = None
secret: str | None = Field(default=None, max_length=2100)
clear_secret: bool = False
@field_validator("name")
@classmethod
def _clean_name(cls, value):
return _channel_name(value)
+1
View File
@@ -35,6 +35,7 @@ class OneToOneRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
config_key: str
description: str
interface: str
source_net: str
+1
View File
@@ -37,6 +37,7 @@ class OutboundNatRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
config_key: str
description: str
interface: str
source_net: str
+1
View File
@@ -62,6 +62,7 @@ class PolicyHierarchyRuleRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
config_key: str
hierarchy_group_id: uuid.UUID
rulebase: Literal["pre", "post"]
sequence: int
+1
View File
@@ -36,6 +36,7 @@ class PortForwardRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
config_key: str
description: str
interface: str
ipprotocol: str
+9
View File
@@ -4,6 +4,8 @@ from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.schemas.adopt import LinkTarget
class RuleCreate(BaseModel):
description: str = Field(min_length=1, max_length=255)
@@ -46,6 +48,7 @@ class RuleRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
config_key: str
description: str
interface: str
direction: str
@@ -78,6 +81,8 @@ class DeployApproval(BaseModel):
class AdoptRequest(BaseModel):
host_id: uuid.UUID
# Link to this existing central rule instead of creating a new one (services/adopt_link_service.py).
link_to: uuid.UUID | None = None
rule_uuid: str = Field(min_length=1, max_length=64)
@@ -98,3 +103,7 @@ class UnmanagedRule(BaseModel):
adoptable: bool
# Codes such as "negated_source" or "not_quick"; the UI turns them into text.
reasons: list[str]
# Central rules equal to this one (resolved for the host), same-description rules that differ, link possible.
matches: list[LinkTarget] = Field(default_factory=list)
similar: list[LinkTarget] = Field(default_factory=list)
linkable: bool = False
+117
View File
@@ -0,0 +1,117 @@
"""Request and response shapes of the rule-cleanup API (routers/rule_cleanup.py, DESIGN-1.31 §4).
Responses mirror services/rule_cleanup_service.read_operation. Texts are not part of the API: ``message`` holds an
ErrorCode name (``errors.<CODE>`` in the UI, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR) or the firewall's
own wording; ``warnings`` is a list of ErrorCode names.
"""
from __future__ import annotations
from typing import Any, Literal
from pydantic import BaseModel, Field, model_validator
CleanupKind = Literal["disable", "enable", "delete"]
# Longest list the API accepts at all; the effective limit is the setting cleanup_max_rules (ceiling 25).
MAX_SELECTION = 100
class PreviewItem(BaseModel):
"""One rule of the selection. ``kind`` (alias ``mode``) must be the same for all items of one preview."""
uuid: str = Field(min_length=1, max_length=64)
kind: CleanupKind | None = None
mode: CleanupKind | None = None
# The per-rule acknowledgement for a "probable" rule (the UI's checkbox).
ack: bool = False
class PreviewRequest(BaseModel):
"""``{kind, uuids, acknowledge}`` or ``{items:[{uuid, kind|mode, ack}]}`` (or both: they are merged)."""
kind: CleanupKind | None = None
uuids: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
items: list[PreviewItem] = Field(default_factory=list, max_length=MAX_SELECTION)
acknowledge: list[str] = Field(default_factory=list, max_length=MAX_SELECTION)
unused_days: int | None = Field(default=None, ge=1, le=3650)
ack_text_version: str | None = Field(default=None, max_length=16)
@model_validator(mode="after")
def _single_kind(self):
kinds = {k for k in [self.kind, *[i.kind or i.mode for i in self.items]] if k}
if len(kinds) > 1:
raise ValueError("kind must be the same for every item")
return self
def resolved(self) -> tuple[str | None, list[str], list[str]]:
kind = self.kind or next((i.kind or i.mode for i in self.items if i.kind or i.mode), None)
uuids = [*self.uuids, *[i.uuid for i in self.items]]
acknowledge = [*self.acknowledge, *[i.uuid for i in self.items if i.ack]]
return kind, list(dict.fromkeys(u.strip() for u in uuids)), list(dict.fromkeys(a.strip() for a in acknowledge))
class ConfirmName(BaseModel):
"""Deploy and restore: the exact host name, typed by a human."""
confirm_name: str = Field(max_length=255)
class CleanupItemRead(BaseModel):
id: str
host_id: str
host_name: str | None = None
rule_uuid: str
seq: int
carp_role: str | None = None
# queued, disabling, disabled, confirmed, reverting, reverted, deleting, deleted, restoring, restored, failed,
# unknown, revert_failed, skipped (never attempted: the run stopped earlier)
state: str
description: str | None = None
categories: Any = None
confidence: str | None = None
cover_uuid: str | None = None
prio_group: str | None = None
anchor_next_uuid: str | None = None
anchor_prev_uuid: str | None = None
source_item_id: str | None = None
restored_uuid: str | None = None
changed_at: str | None = None
message: str | None = None
has_snapshot: bool = False
# A deleted rule with a stored snapshot that has not been restored yet (POST items/{id}/restore, admin).
restorable: bool = False
class CleanupOperationRead(BaseModel):
id: str
host_id: str
host_name: str | None = None
cluster_name: str | None = None
kind: str # disable | enable | delete | restore
# preview, running, awaiting_confirm, confirmed, partial, apply_failed, reverting, reverted, revert_failed, done,
# failed, cancelled, expired
status: str
actor: str
created_at: str | None = None
expires_at: str | None = None
started_at: str | None = None
heartbeat_at: str | None = None
revert_deadline: str | None = None
confirmed_at: str | None = None
finished_at: str | None = None
message: str | None = None
options: dict[str, Any] = Field(default_factory=dict)
# Only while status == "preview".
plan: dict[str, Any] | None = None
items: list[CleanupItemRead] = Field(default_factory=list)
can_confirm: bool = False
can_revert: bool = False
can_retry: bool = False
# ErrorCode names, e.g. BEREINIGUNG_POSITION_NICHT_WIEDERHERSTELLBAR (restore only).
warnings: list[str] = Field(default_factory=list)
class CleanupPreviewRead(BaseModel):
preview_id: str
expires_at: str | None = None
kind: str
plan: dict[str, Any]
max_rules: int
window_seconds: int
operation: CleanupOperationRead
+56
View File
@@ -7,9 +7,19 @@ how the proxy setup works for IP *and* name access) nor an explicitly allowed
origin. Requests without either header (curl, scripts) are not browsers, so
they cannot be riding on a victim's cookie and pass through to normal auth.
Sec-Fetch-Site (1.27): a page on another port of the same host (e.g. a
firewall GUI behind the GUI gateway) is *same-site* to OPNexus, so the
browser attaches the session cookie to its requests and SameSite=Lax does
not help. Cookie-carrying /api requests that the browser itself marks as
same-site or cross-site are therefore refused — unless the Origin is an
explicitly configured frontend origin (development setup with the frontend
on its own port). Clients that send no Sec-Fetch-Site header (curl,
scripts, API tokens) are not browsers and stay unaffected.
SecurityHeadersMiddleware: API responses must never be cached by browsers or
proxies and must not be sniffed or framed.
"""
import re
from urllib.parse import urlparse
from starlette.middleware.base import BaseHTTPMiddleware
@@ -18,6 +28,7 @@ from starlette.responses import JSONResponse
from app.config import settings
from app.errors import ErrorCode
from app.routers.auth import has_session_cookie
_SAFE_METHODS = {"GET", "HEAD", "OPTIONS"}
@@ -35,8 +46,53 @@ def _origin_allowed(origin: str, host_header: str) -> bool:
return f"{parsed.scheme}://{parsed.netloc}".lower() in {o.lower() for o in settings.allowed_origins}
# The SSO provider sends the browser back here by a cross-site navigation;
# the endpoint authenticates by its own state cookie and creates the session.
# The exemption is only for exactly that: a top-level GET navigation (a script, image or fetch from another site
# is still blocked, whatever it carries) — and the callback fails without the matching state cookie anyway.
_SEC_FETCH_EXEMPT = {"/api/auth/oidc/callback"}
def _is_top_level_navigation(request: Request) -> bool:
if request.method != "GET":
return False
mode = request.headers.get("sec-fetch-mode", "").strip().lower()
dest = request.headers.get("sec-fetch-dest", "").strip().lower()
return mode in ("", "navigate") and dest in ("", "document")
def _explicit_frontend_origin(origin: str) -> bool:
origin = origin.rstrip("/").lower()
if not origin or origin == "null":
return False
if settings.allowed_origins:
return origin in {o.lower() for o in settings.allowed_origins}
return re.fullmatch(settings.cors_origin_regex, origin) is not None
def sec_fetch_blocked(request: Request) -> bool:
path = request.url.path
if not (path == "/api" or path.startswith("/api/")):
return False
if path in _SEC_FETCH_EXEMPT and _is_top_level_navigation(request):
return False
if not has_session_cookie(request):
return False
site = request.headers.get("sec-fetch-site", "").strip().lower()
if site == "cross-site":
return True
if site == "same-site":
return not _explicit_frontend_origin(request.headers.get("origin", ""))
return False # same-origin, none (typed URL/bookmark) or no header (not a browser)
class OriginCheckMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
if sec_fetch_blocked(request):
return JSONResponse(
status_code=403,
content={"detail": {"code": ErrorCode.CROSS_SITE_REQUEST_BLOCKED.value, "params": {}}},
)
if request.method not in _SAFE_METHODS:
source = request.headers.get("origin") or request.headers.get("referer")
if source and source != "null" and not _origin_allowed(source, request.headers.get("host", "")):
+63
View File
@@ -0,0 +1,63 @@
"""Small pieces shared by every "take over what was made by hand on a firewall" feature
(rule_adopt_service.py for filter rules, object_adopt_service.py for aliases and NAT)."""
class NotAdoptable(Exception):
"""The object exists on the host but cannot be described without distortion.
`reasons` are machine-readable codes (the UI translates them); `details` maps a code to
extra text the translation can show (e.g. the alias names a reference points to)."""
def __init__(self, reasons: list[str], details: dict[str, str] | None = None):
super().__init__(", ".join(reasons))
self.reasons = reasons
self.details = details or {}
class NotFoundOnHost(Exception):
"""The UUID is not (or no longer) present in the host's live configuration."""
def on(value) -> bool:
"""OPNsense booleans arrive as "1"/"0" strings, sometimes as real booleans."""
return value in ("1", 1, True)
def clean(value) -> str:
return "" if value is None else str(value).strip()
class LinkTargetNotFound(Exception):
"""The central object chosen to link to was deleted in the meantime."""
class LinkTargetChanged(Exception):
"""The central object no longer equals what is live on the host. `diff` lists the differing fields."""
def __init__(self, diff: list[dict]):
super().__init__(", ".join(d["field"] for d in diff))
self.diff = diff
class HostAlreadyHasObject(Exception):
"""The central object is already deployed to / managed on this host under another UUID."""
class LinkTargetBusy(Exception):
"""The central object has an unconfirmed deployment or a failed rollback; linking waits until it is settled."""
LINK_ERRORS = (LinkTargetNotFound, LinkTargetChanged, HostAlreadyHasObject, LinkTargetBusy)
def link_api_error(exc: Exception):
"""The HTTP error for a failed link (shared by the rule and the object endpoints)."""
from app.errors import ErrorCode, api_error
if isinstance(exc, LinkTargetNotFound):
return api_error(404, ErrorCode.ADOPT_LINK_TARGET_NOT_FOUND)
if isinstance(exc, LinkTargetChanged):
return api_error(409, ErrorCode.ADOPT_LINK_TARGET_CHANGED, fields=", ".join(d["field"] for d in exc.diff))
if isinstance(exc, HostAlreadyHasObject):
return api_error(409, ErrorCode.ADOPT_LINK_HOST_HAS_OBJECT)
return api_error(409, ErrorCode.ADOPT_LINK_BUSY)
+151
View File
@@ -0,0 +1,151 @@
"""Links a hand-made object that was found on a firewall to a central object that already exists, instead
of creating a second central object for it ("Mit vorhandenem zentralem Objekt verknüpfen").
Typical case: the same rule/alias/NAT entry was made by hand on several firewalls (or on both nodes of an HA
pair). The first adoption creates the central object; every further firewall is LINKED to it: the host
becomes another target of that object and gets a deployment row holding the live UUID (remote_uuid) and
the payload OPNexus would send. Like adopting, linking writes NOTHING to the firewall. The live state is
read fresh from the host at link time (the browser only names the central object), so the preview right
afterwards says "unchanged" and a later rollout updates the one object on every host without a duplicate.
A central object is an EXACT match when every field OPNexus manages is equal after the object's
{{VARIABLE}} placeholders are replaced with this host's values (variables_service.resolve). An object with
the same identity (alias name / rule description) but different content is only shown as a hint with the
differing fields and is NEVER linked automatically.
Callers hold policy_lock (same serialisation as adopting), which also makes a repeated link idempotent.
"""
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Any, Callable
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup
from app.services import variables_service
from app.services.adopt_common import HostAlreadyHasObject, LinkTargetBusy, LinkTargetChanged, LinkTargetNotFound
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
from app.services.policy_service import effective_hosts
LINK_MESSAGE = "Mit vorhandenem Objekt verknüpft"
BUSY_STATUSES = ("pending", "revert_failed") # the central object waits for a confirmation / failed to roll back
@dataclass(frozen=True)
class LinkSpec:
model: type
deployment: type
fk: str # deployment column pointing at the central object
uuid_attr: str # deployment column holding the OPNsense UUID (remote_uuid)
identity: str # attribute naming the object: "name" (alias) or "description"
payload: Callable[[Any, OpnsenseHost], dict]
revision: Callable
@dataclass
class Match:
obj: Any
diff: list[dict] # empty = exact match
busy: bool = False # the central object waits for a confirmation: linking is refused until that is settled
def _norm(value) -> Any:
if value is None:
return ""
return value if isinstance(value, bool) else str(value)
def _show(value) -> str:
return ("true" if value else "false") if isinstance(value, bool) else ("" if value is None else str(value))
def diff_fields(obj, values: dict, host: OpnsenseHost) -> list[dict]:
"""Fields in which the central object (resolved for `host`) differs from the live `values`."""
raw = {key: getattr(obj, key) for key in values}
try:
central = variables_service.resolve(raw, host)
except variables_service.MissingVariables as exc:
# Cannot be compared (and could not be deployed to this host either): report, never match.
return [{"field": "variables", "central": ", ".join(exc.missing[host.name]), "live": ""}]
return [{"field": key, "central": _show(central[key]), "live": _show(values[key])}
for key in values if _norm(central[key]) != _norm(values[key])]
def _options(spec: LinkSpec):
return [
selectinload(spec.model.targets),
selectinload(spec.model.groups).selectinload(DeviceGroup.hosts),
selectinload(spec.model.deployments),
]
async def load_central(db: AsyncSession, spec: LinkSpec) -> list:
objs = list((await db.scalars(select(spec.model).options(*_options(spec)))).unique().all())
await hydrate_hierarchy_targets(db, objs)
return objs
def find(objs: list, spec: LinkSpec, host: OpnsenseHost, values: dict) -> tuple[list[Match], list[Match]]:
"""(exact matches, same-identity objects that differ). Objects that already have a deployment on
this host are not candidates: linking would collide with that deployment."""
exact: list[Match] = []
similar: list[Match] = []
for obj in objs:
if any(d.host_id == host.id for d in obj.deployments):
continue
if _norm(getattr(obj, spec.identity)) != _norm(values[spec.identity]):
continue
d = diff_fields(obj, values, host)
busy = any(dep.status in BUSY_STATUSES for dep in obj.deployments)
(similar if d else exact).append(Match(obj, d, busy))
# prefer the object that already reaches the most hosts (the "real" one), then a stable order
exact.sort(key=lambda m: (-len(effective_hosts(m.obj)), str(m.obj.id)))
return exact, similar
def describe(spec: LinkSpec, match: Match) -> dict:
return {
"id": match.obj.id,
"name": getattr(match.obj, spec.identity),
"targets": [h.name for h in effective_hosts(match.obj)],
"diff": match.diff,
}
async def link(db: AsyncSession, spec: LinkSpec, host: OpnsenseHost, object_id, remote_uuid: str, values: dict):
"""Links the live object `remote_uuid` (whose normalised fields are `values`, read fresh by the caller)
to the central object `object_id`. Returns it; a repeated call for the same pair changes nothing."""
obj = (await db.execute(
select(spec.model).where(spec.model.id == object_id).options(*_options(spec))
.execution_options(populate_existing=True)
)).unique().scalar_one_or_none()
if obj is None:
raise LinkTargetNotFound(str(object_id))
await hydrate_hierarchy_targets(db, [obj])
existing = next((d for d in obj.deployments if d.host_id == host.id), None)
if existing is not None:
if getattr(existing, spec.uuid_attr) == remote_uuid:
return obj # already linked: nothing to do
raise HostAlreadyHasObject(str(object_id))
if any(d.status in BUSY_STATUSES for d in obj.deployments):
raise LinkTargetBusy(str(object_id))
changed = diff_fields(obj, values, host)
if changed:
raise LinkTargetChanged(changed)
if host.id not in {h.id for h in effective_hosts(obj)}:
obj.targets.append(host)
# an association-only change does not touch the row; stale edit forms must still be refused (see edit_rule)
obj.updated_at = datetime.now(timezone.utc)
await db.flush()
spec.revision(db, obj, f"Mit {host.name} verknüpft")
db.add(spec.deployment(**{
spec.fk: obj.id, "host_id": host.id, "status": "success", "message": LINK_MESSAGE,
spec.uuid_attr: remote_uuid, "last_payload": spec.payload(obj, host), "previous_payload": None,
}))
await db.flush()
return obj
+12 -3
View File
@@ -18,7 +18,7 @@ from app.models.host import OpnsenseHost
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -27,7 +27,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(alias: FirewallAlias) -> dict:
def _build_payload(alias: FirewallAlias, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(alias)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(alias: FirewallAlias) -> dict:
return {
"enabled": "1" if alias.enabled else "0",
"name": alias.name,
@@ -38,7 +43,7 @@ def _build_payload(alias: FirewallAlias) -> dict:
async def _deploy_to_host(db: AsyncSession, alias: FirewallAlias, host: OpnsenseHost) -> AliasDeployment:
payload = _build_payload(alias)
payload = _build_payload(alias, host)
existing = await db.scalar(
select(AliasDeployment).where(AliasDeployment.alias_id == alias.id, AliasDeployment.host_id == host.id)
)
@@ -79,6 +84,7 @@ async def deploy_alias(db: AsyncSession, alias: FirewallAlias, hosts: list[Opnse
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, alias, hosts) # before anything is written
deployments = [await _deploy_to_host(db, alias, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
@@ -124,6 +130,9 @@ async def sweep_expired_alias_deployments(db: AsyncSession) -> list[AliasDeploym
async def undeploy_alias(db: AsyncSession, alias: FirewallAlias) -> None:
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, AliasDeployment, AliasDeployment.alias_id == alias.id)
failures = []
result = await db.execute(select(AliasDeployment).where(AliasDeployment.alias_id == alias.id))
for deployment in result.scalars().all():
+10 -4
View File
@@ -9,6 +9,7 @@ import json
from app.models.policy import AliasRevision
from app.services.alias_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
ALIAS_FIELDS = ("name", "type", "content", "description", "enabled")
@@ -28,13 +29,17 @@ def revision(db, alias, note):
def preview(alias):
payload = _build_payload(alias)
deployments = {d.host_id: d for d in alias.deployments}
hosts = effective_hosts(alias)
blocked = any(d.status in ("pending", "revert_failed") for d in alias.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(alias, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_alias_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -61,6 +66,8 @@ def preview(alias):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diesen Alias, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in alias.deployments):
@@ -71,7 +78,7 @@ def preview(alias):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in alias.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -79,12 +86,11 @@ def workflow(alias, deployments):
hosts = effective_hosts(alias)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(alias)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, alias, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+2 -1
View File
@@ -13,6 +13,7 @@ from app.models.alias_deployment import AliasDeployment
from app.models.host import OpnsenseHost
from app.schemas.host import AliasOption
from app.services import opnsense_client
from app.services.deploy_flow import unsettled
from app.services.opnsense_client import OpnsenseAPIError
@@ -87,7 +88,7 @@ async def ensure_aliases_settled(db: AsyncSession, hosts: list[OpnsenseHost], va
.where(
FirewallAlias.name.in_(names),
AliasDeployment.host_id.in_([host.id for host in hosts]),
AliasDeployment.status.in_(("pending", "revert_failed")),
unsettled(AliasDeployment),
)
.order_by(FirewallAlias.name, OpnsenseHost.name)
)).all()
+53 -10
View File
@@ -38,19 +38,35 @@ MAX_TOKENS_PER_USER = 50
_LAST_USED_GRANULARITY = timedelta(minutes=1)
_SAFE_METHODS = ("GET", "HEAD", "OPTIONS")
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)")
# GUI tickets open a firewall's web interface in the browser: an interactive admin session only.
# Rule cleanup (1.31): "I still have access" is a human decision, so confirming a cleanup is never possible with a
# token, whatever its scope (a script must not be able to confirm blindly).
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$|^/api/hosts/[^/]+/rule-cleanup/[^/]+/confirm/?$")
# Reads that expose secrets (config.xml holds password hashes and VPN keys, the
# notification settings the webhook URL/SMTP data): admin scope only.
_ADMIN_SCOPE_READS = re.compile(r"^/api/hosts/[^/]+/backups/[^/]+$|^/api/notifications(/|$)|^/api/settings/|^/api/overlay/peers/[^/]+/instructions$")
# The MCP endpoint is POST (JSON-RPC) but its tools only ever read.
_MCP_PATH = "/api/mcp"
# The MCP endpoint is POST (JSON-RPC) but its tools only ever read; so does the configuration-as-code plan (1.32):
# it compares a posted file with the database and writes nothing.
_READ_ONLY_POSTS = ("/api/mcp", "/api/config/import/plan")
_ADMIN_ONLY_WRITES = re.compile(
r"^/api/(users|settings|system|notifications|overlay)(/|$)"
r"|^/api/logs/settings$"
r"|^/api/hosts(/tls-probe|/[^/]+)?$"
r"|^/api/hosts/[^/]+/gui-ticket$"
# The per-gateway alert switch is a notification setting (same rights as PUT /api/notifications/settings).
r"|^/api/hosts/[^/]+/gateways/[^/]+/alert$"
# Configuration as code (1.32): a bulk change incl. prune and host hierarchy membership (host writes are admin-only).
r"|^/api/config/import/apply$"
)
# Rule cleanup: executing a cleanup, retrying a failed revert/reconcile and restoring a deleted rule are admin-only.
# operators_may_deploy defaults to True, so _ROLLOUT_POST alone cannot express "admin only"; this pattern is checked
# for operator accounts (operator_denies) and for tokens below scope admin (denies). Not part of _ADMIN_ONLY_WRITES:
# that one means "who may access OPNexus / where data goes". The router checks the same rule once more.
_ADMIN_ONLY_CLEANUP = re.compile(r"^/api/hosts/[^/]+/rule-cleanup/([^/]+/(deploy|retry)|items/[^/]+/restore)/?$")
def _now() -> datetime:
return datetime.now(timezone.utc)
@@ -63,13 +79,13 @@ def denies(scope: str, method: str, path: str) -> bool:
"""True if a token with this scope must not make this request."""
if _SESSION_ONLY.match(path):
return True
if path == _MCP_PATH:
if path in _READ_ONLY_POSTS:
return False
if method in _SAFE_METHODS:
return scope != "admin" and bool(_ADMIN_SCOPE_READS.match(path))
if scope == "read":
return True
if scope == "deploy" and _ADMIN_ONLY_WRITES.match(path):
if scope == "deploy" and (_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path)):
return True
return False
@@ -78,18 +94,45 @@ def operator_denies(method: str, path: str) -> bool:
"""True if an `operator` account must not make this request: the writes that
change who may access OPNexus or where it sends data. (Their own theme at
PUT /api/settings stays open; account endpoints are not matched here.)"""
return method not in _SAFE_METHODS and path != "/api/settings" and bool(_ADMIN_ONLY_WRITES.match(path))
return method not in _SAFE_METHODS and path != "/api/settings" and bool(
_ADMIN_ONLY_WRITES.match(path) or _ADMIN_ONLY_CLEANUP.match(path))
# Writes that put a change onto the fleet (as opposed to preparing one): deploy, remove/rollback a deployment,
# maintenance (reboot/update/CARP switch), scheduling it, and applying network changes.
_ROLLOUT = re.compile(
r"/(deploy|deployments/[^/]+/(remove|retry-rollback)|maintenance/(execute|carp-execute|schedule)|network/changes)$"
# maintenance (reboot/update/CARP switch), scheduling it, applying network changes, pushing certificates and
# ACME accounts, re-asserting the hierarchy order, controlling services, and DELETING a centrally managed object
# or snapshot (deleting a rule removes it from every firewall it was deployed to), and creating a snapshot.
_ROLLOUT_POST = re.compile(
r"/(deploy|push|sign|poll|register|reassert-ordering"
r"|deployments/[^/]+/(remove|retry-rollback)"
# Rule cleanup (1.31): switching rules off (/deploy matches above) and taking it back. Confirming and
# previewing stay open, like confirm_rule.
r"|rule-cleanup/[^/]+/(revert|retry)"
r"|maintenance/(execute|carp-execute|schedule)"
r"|network/changes"
r"|snapshots" # creating a boot environment (deleting one is in _ROLLOUT_DELETE)
r"|services/[^/]+/(start|stop|restart)"
# Firmware rollouts (1.25): starting/arming a plan and moving a run forward. Pausing is deliberately absent.
r"|rollouts/plans/[^/]+/(start|activate)"
r"|rollouts/runs/[^/]+/(resume|approve|skip-host|abort))$"
)
_ROLLOUT_DELETE = re.compile(
r"^/api/(rules|aliases|port-forward|one-to-one|outbound-nat|policy-hierarchy/rules|ipsec/tunnels|openvpn/instances"
r"|wireguard/tunnels|certificates/managed|acme/accounts|acme/certificates)/[^/]+$"
r"|^/api/hosts/[^/]+/(snapshots|maintenance/schedule)/[^/]+$"
)
# Recurring maintenance windows decide WHEN unattended firmware actions run (1.25): every write counts.
_ROLLOUT_ANY_WRITE = re.compile(r"^/api/maintenance-windows(/|$)")
def is_rollout(method: str, path: str) -> bool:
return method == "POST" and bool(_ROLLOUT.search(path))
if method in ("POST", "PUT", "PATCH", "DELETE") and _ROLLOUT_ANY_WRITE.match(path):
return True
if method == "POST":
return bool(_ROLLOUT_POST.search(path))
return method == "DELETE" and bool(_ROLLOUT_DELETE.match(path))
def bearer_from_header(header: str) -> str | None:
+6
View File
@@ -13,6 +13,9 @@ from app.database import SessionLocal
from app.models.audit import AuditEvent
context = ContextVar("audit_context", default=None)
# A key in ``details`` whose content only admins (session or admin-scope token) get to read (routers/audit.py):
# firewall addresses and management sources of the rule cleanup.
ADMIN_ONLY_DETAILS = "admin_only"
# Column widths of audit_events. entity_id/host_id/action are taken from the
@@ -88,6 +91,9 @@ def deployment_history(session, flush_context, instances):
if entity is not None and action == "deploy":
from importlib import import_module
builder = import_module(f"app.services.{kind}_deploy_service")._build_payload
try:
details["requested_payload"] = builder(entity, host) if host is not None else builder(entity)
except Exception: # a missing variable must not break the audit write; record the unresolved intent instead
details["requested_payload"] = builder(entity)
session.add(make_event(kind=kind, action=action, status="removed" if deleted else dep.status,
entity_id=str(entity_id), entity_name=(getattr(entity, "name", None) or getattr(entity, "description", None)),
+15 -6
View File
@@ -236,6 +236,8 @@ async def create_session(
user_agent: str | None = None,
ip_address: str | None = None,
verified_hash: str | None = None,
verified_totp: bool | None = None,
lifetime_seconds: int | None = None,
) -> tuple[str, datetime]:
"""Create a session, returning the raw token (to be set as a cookie) and its expiry.
@@ -243,17 +245,24 @@ async def create_session(
against. The user row is re-read under a row lock first: a password
change/reset that committed while the (slow) password check was running
changed the hash, and the login must not outlive that revocation.
``verified_totp`` is whether the caller checked a second factor. Two-factor
enabled in the meantime (it also revoked every session) means this login
was decided without it and must start over.
"""
user_id = user.id
if verified_hash is not None:
locked = await db.scalar(
select(AdminUser.password_hash).where(AdminUser.id == user_id).with_for_update()
)
if locked != verified_hash:
if verified_hash is not None or verified_totp is not None:
locked_hash, locked_totp = (await db.execute(
select(AdminUser.password_hash, AdminUser.totp_enabled).where(AdminUser.id == user_id).with_for_update()
)).one()
if verified_hash is not None and locked_hash != verified_hash:
await db.rollback()
raise api_error(401, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
if locked_totp and not verified_totp:
await db.rollback()
raise api_error(401, ErrorCode.TOTP_REQUIRED)
token = secrets.token_urlsafe(SESSION_TOKEN_BYTES)
expires_at = _now() + timedelta(seconds=settings.session_lifetime)
expires_at = _now() + timedelta(seconds=lifetime_seconds or settings.session_lifetime)
session = Session(
user_id=user_id,
token_hash=_hash_token(token),
+11 -6
View File
@@ -7,6 +7,8 @@ but its value never leaves the server. Output is a unified diff, capped in size.
"""
import difflib
import hashlib
import hmac
import os
import re
from xml.etree.ElementTree import Element, indent, tostring
@@ -14,7 +16,7 @@ import defusedxml.ElementTree as safe_ET
MAX_LINES = 3000
MAX_BYTES = 32 * 1024 * 1024
_SECRET = re.compile(r"password|passwd|psk|secret|key|passphrase|token|prv|hash|community|bindpw|privpass", re.I)
_SECRET = re.compile(r"pass|pwd|psk|secret|key|token|prv|hash|community|bindpw|privpass|otp|seed|cred", re.I)
_HARMLESS_SUFFIX = re.compile(r"(len|length|type|size|bits|file|name)$", re.I)
_BLOB = 120
@@ -23,15 +25,17 @@ class InvalidConfig(ValueError):
pass
def _redact(root: Element) -> None:
def _redact(root: Element, salt: bytes) -> None:
for el in root.iter():
if len(el) == 0 and el.text and el.text.strip():
text = el.text.strip()
if (_SECRET.search(el.tag) and not _HARMLESS_SUFFIX.search(el.tag)) or len(text) > _BLOB:
el.text = "‹sha256:" + hashlib.sha256(text.encode()).hexdigest()[:12] + "›"
# Salted per diff: both sides of one diff share the salt (so a change shows), but the value cannot be
# looked up in a dictionary or compared across hosts/requests.
el.text = "‹" + hmac.new(salt, text.encode(), hashlib.sha256).hexdigest()[:12] + "›"
def normalise(xml_bytes: bytes) -> list[str]:
def normalise(xml_bytes: bytes, salt: bytes) -> list[str]:
if not xml_bytes.strip() or len(xml_bytes) > MAX_BYTES:
raise InvalidConfig("leer oder zu groß")
try:
@@ -42,13 +46,14 @@ def normalise(xml_bytes: bytes) -> list[str]:
raise InvalidConfig("keine OPNsense-Konfiguration")
for volatile in root.findall("revision"):
root.remove(volatile)
_redact(root)
_redact(root, salt)
indent(root, space=" ")
return tostring(root, encoding="unicode").splitlines()
def diff(old: bytes, new: bytes) -> dict:
a, b = normalise(old), normalise(new)
salt = os.urandom(16)
a, b = normalise(old, salt), normalise(new, salt)
lines = list(difflib.unified_diff(a, b, "älter", "neuer", lineterm="", n=2))
added = sum(1 for l in lines if l.startswith("+") and not l.startswith("+++"))
removed = sum(1 for l in lines if l.startswith("-") and not l.startswith("---"))
+47 -35
View File
@@ -12,6 +12,7 @@ in ``filter/rule`` (legacy). Both are normalised to the same shape below.
decided. The score starts at 100 and loses a penalty per failed check, so one
critical finding (an any-any pass rule on the WAN) already turns the host red.
"""
import asyncio
import re
from datetime import datetime, timezone
from xml.etree.ElementTree import Element
@@ -25,12 +26,22 @@ from app.models.host import OpnsenseHost
from app.services import backup_store, rule_hit_service
from app.services import opnsense_client as client
from app.services.opnsense_client import OpnsenseAPIError
from app.services.rule_analysis import coverage
MAX_CONFIG_BYTES = 32 * 1024 * 1024
BACKUP_MAX_AGE_DAYS = 7
PENALTY = {"critical": 30, "high": 15, "medium": 8, "low": 3, "info": 0}
SEVERITY_ORDER = ("critical", "high", "medium", "low", "info")
MAX_EXAMPLES = 5
# German titles for notifications (the UI translates the check ids itself).
TITLES_DE = {
"webgui_https": "Web-Oberfläche nur über HTTPS", "ssh_hardened": "SSH abgesichert", "snmp_community": "SNMP-Community nicht „public“",
"wan_any_any": "Keine Allow-all-Regel am WAN", "internal_any_any": "Keine Allow-all-Regel mit beliebiger Quelle (intern)",
"rules_shadowed": "Keine verdeckten Regeln", "rules_duplicate": "Keine doppelten Regeln", "rules_documented": "Regeln sind beschriftet",
"rules_unused": "Regeln in Benutzung", "ntp_configured": "Zeitserver gesetzt", "remote_logging": "Logs liegen auch woanders",
"ids_enabled": "Einbruchserkennung aktiv", "firmware_current": "Firmware aktuell", "backup_recent": "Aktuelles Backup",
}
SEVERITIES_DE = {"critical": "kritisch", "high": "hoch", "medium": "mittel", "low": "niedrig", "info": "Hinweis"}
def grade(score: int) -> str:
@@ -73,10 +84,15 @@ def _mvc_rule(rule: Element) -> dict:
"sport": _text(rule, "source_port"),
"dst": _text(rule, "destination_net") or "any", "dst_not": _text(rule, "destination_not") == "1",
"dport": _text(rule, "destination_port"),
"floating": False,
"if_not": _text(rule, "interfacenot") == "1",
"floating": False, # decided in extract_rules (needs the list of interfaces)
"quick": _text(rule, "quick") != "0",
"descr": _text(rule, "description") or "",
"order": int(_text(rule, "sequence") or 0),
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
"received_on": _text(rule, "received-on") or "",
"icmptype": ",".join(filter(None, [_text(rule, "icmptype"), _text(rule, "icmp6type")])),
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
}
@@ -91,17 +107,29 @@ def _legacy_rule(rule: Element, index: int) -> dict:
"ipprotocol": _text(rule, "ipprotocol") or "inet",
"protocol": (_text(rule, "protocol") or "any").lower(),
"src": src, "src_not": src_not, "sport": sport, "dst": dst, "dst_not": dst_not, "dport": dport,
"if_not": rule.find("interfacenot") is not None,
"floating": _text(rule, "floating") == "yes",
"quick": _text(rule, "quick") != "0",
# Legacy rules: an interface rule is always evaluated "quick"; only a floating rule has the option,
# and it is off unless <quick> says yes.
"quick": _text(rule, "floating") != "yes" or (_text(rule, "quick") or "").lower() in ("yes", "1"),
"descr": _text(rule, "descr") or "",
"order": index,
"sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
"received_on": _text(rule, "received-on") or "", "icmptype": _text(rule, "icmptype") or "",
"limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
}
def extract_rules(root: Element) -> list[list[dict]]:
"""Rule lists in evaluation order, one per source (MVC, legacy)."""
mvc = sorted((_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")), key=lambda r: r["order"])
mvc = [_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")]
known = {node.tag for node in root.findall("interfaces/*")}
for rule in mvc: # same rule as OPNsense: not exactly one interface (or "not") = floating
rule["floating"] = len(rule["interfaces"]) != 1 or rule["if_not"]
rule["group"] = not rule["floating"] and rule["interfaces"][0] not in known
mvc.sort(key=coverage.config_order)
legacy = [_legacy_rule(r, i) for i, r in enumerate(root.findall("filter/rule"))]
legacy.sort(key=coverage.config_order)
return [group for group in (mvc, legacy) if group]
@@ -123,41 +151,22 @@ def _is_any_any(rule: dict) -> bool:
def _covers(a: dict, b: dict) -> bool:
"""Every packet matched by rule b is already matched by the earlier rule a.
Only exact-or-any comparisons, so it never claims coverage it cannot prove."""
if not (a["enabled"] and b["enabled"]) or a["floating"] or b["floating"] or not a["quick"]:
return False
if not a["interfaces"] or not set(b["interfaces"]) <= set(a["interfaces"]):
return False
if a["direction"] != b["direction"] or a["ipprotocol"] != b["ipprotocol"]:
return False
if a["protocol"] != "any" and a["protocol"] != b["protocol"]:
return False
for side in ("src", "dst"):
if a[side + "_not"] or b[side + "_not"]:
if (a[side], a[side + "_not"]) != (b[side], b[side + "_not"]):
return False
elif a[side] != "any" and a[side] != b[side]:
return False
for port in ("sport", "dport"):
if a[port] and a[port] != b[port]:
return False
return True
def _label(rule: dict) -> str:
return rule["descr"] or f"{rule['action']} {rule['src']} → {rule['dst']}"
def analyse_rules(groups: list[list[dict]]) -> dict:
"""Shadowed and duplicate rules. The coverage logic lives in rule_analysis/coverage.py (shared with the
cleanup candidates); the rules of one list are compared in evaluation order."""
shadowed, duplicates = [], []
for rules in groups:
for j, later in enumerate(rules):
for earlier in rules[:j]:
if _covers(earlier, later):
(duplicates if earlier["action"] == later["action"] and _covers(later, earlier) else shadowed).append(_label(later))
break
ordered = sorted(rules, key=coverage.config_order)
norm = [coverage.from_config_rule(rule) for rule in ordered]
by_identity = {id(n): r for n, r in zip(norm, ordered)}
for overlap in coverage.analyse(norm):
if overlap.by.order == overlap.rule.order:
continue # equal position in config.xml: the real order is unknown, claim nothing
(duplicates if overlap.kind == "duplicate" else shadowed).append(_label(by_identity[id(overlap.rule)]))
return {"shadowed": shadowed, "duplicates": duplicates}
@@ -267,15 +276,18 @@ async def check_host(db: AsyncSession, host: OpnsenseHost) -> ComplianceResult:
await rule_hit_service.sample_host(db, host)
except OpnsenseAPIError:
pass # usage data is a bonus; the check does not depend on it
findings = evaluate(xml, firmware_status=firmware_status, newest_backup_age_s=age,
unused_rules=await rule_hit_service.unused_summary(db, host.id))
# Untrusted XML, O(n^2) rule analysis: keep it off the event loop, and treat ANY parse failure as 'invalid'.
findings = await asyncio.to_thread(
evaluate, xml, firmware_status=firmware_status, newest_backup_age_s=age,
unused_rules=await rule_hit_service.unused_summary(db, host.id),
)
row.findings, row.score, row.error = findings, score(findings), None
except OpnsenseAPIError as exc:
row.findings, row.score, row.error = [], None, "unreachable"
row.error_detail = str(exc)[:300]
except ValueError as exc:
except Exception as exc: # ParseError is a SyntaxError, not a ValueError; one bad config must not stop the other hosts
row.findings, row.score, row.error = [], None, "invalid_config"
row.error_detail = str(exc)[:300]
row.error_detail = f"{type(exc).__name__}: {exc}"[:300]
await db.commit()
return row
+24 -5
View File
@@ -29,6 +29,7 @@ and the sweeper away in the gap (pending/unknown rows are never re-deployed).
from datetime import datetime, timedelta, timezone
from typing import Any, Awaitable, Callable
from sqlalchemy import and_, or_
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
@@ -46,7 +47,21 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
async def _checkpoint(db: AsyncSession) -> None:
def is_unsettled(row) -> bool:
"""A change that may still be live and whose rollback is not yet done: pending, outcome unknown, or an
"error" row whose revert deadline is still armed (written but apply/ordering not confirmed)."""
return row.status in ("pending", "revert_failed") or (row.status == "error" and row.revert_deadline is not None)
def unsettled(model):
"""``is_unsettled`` as a SQL condition for the deployment model."""
return or_(
model.status.in_(("pending", "revert_failed")),
and_(model.status == "error", model.revert_deadline.isnot(None)),
)
async def checkpoint(db: AsyncSession) -> None:
from app.services.policy_service import policy_lock
await db.commit()
@@ -67,12 +82,16 @@ async def push_to_host(
) -> Any:
window = timedelta(seconds=settings.deploy_confirm_window_seconds)
before = {name: getattr(existing, name) for name in _FIELDS} if existing else None
# An armed "error" row (written, apply not confirmed) holds the UNCONFIRMED payload as last_payload: a retry must
# keep the original rollback target instead of promoting that payload to it.
armed = existing is not None and existing.status == "error" and existing.revert_deadline is not None
rollback_target = existing.previous_payload if armed else prior_live_payload
# 1. durable intent
if existing is not None:
row = existing
row.status, row.message = "pending", None
row.previous_payload, row.last_payload = prior_live_payload, payload
row.previous_payload, row.last_payload = rollback_target, payload
row.revert_deadline, row.deployed_at = _now() + window, _now()
else:
row = make(
@@ -83,7 +102,7 @@ async def push_to_host(
# The intent is a technical safety record, not a state change worth a history
# entry of its own (the request itself is already audited as "requested").
row._skip_audit = True
await _checkpoint(db)
await checkpoint(db)
row._skip_audit = False
wrote = False
@@ -93,9 +112,9 @@ async def push_to_host(
# 3. recoverable by UUID from here on
setattr(row, uuid_attr, new_uuid)
row.status, row.message = "pending", None
row.previous_payload = prior_live_payload if existing is not None else None
row.previous_payload = rollback_target if existing is not None else None
row.last_payload, row.revert_deadline, row.deployed_at = payload, _now() + window, _now()
await _checkpoint(db)
await checkpoint(db)
await apply()
except OpnsenseAPIError as exc:
ambiguous = isinstance(exc, OpnsenseAmbiguousError)
+39
View File
@@ -0,0 +1,39 @@
"""An exclusive per-object lock for flows that make several remote calls and commit in between (VPN deploys).
``policy_lock`` is transaction-scoped and is released by every checkpoint commit; this one is a session-level advisory
lock on a connection of its own, so it spans the commits of the request session. If the process dies the connection
closes and PostgreSQL frees the lock. It never waits: a second deploy/removal of the same object is refused at once.
"""
import uuid
import zlib
from contextlib import asynccontextmanager
from sqlalchemy import text
from app.database import engine
from app.errors import ErrorCode, api_error
# Written when the first row is committed, before any remote call: a crash or cancellation leaves this, and a new
# deploy continues from the UUIDs recorded after it.
VPN_INTERRUPTED = (
"Ausrollen unterbrochen oder Ausgang unklar: Die Firewall hat möglicherweise schon Teile der Konfiguration "
"erhalten. Erneutes Ausrollen setzt dort fort."
)
@asynccontextmanager
async def resource_lease(kind: str, resource_id: uuid.UUID):
params = {"a": zlib.crc32(kind.encode()) & 0x7FFFFFFF, "b": resource_id.int & 0x7FFFFFFF}
async with engine.connect() as conn:
if not await conn.scalar(text("SELECT pg_try_advisory_lock(:a, :b)"), params):
raise api_error(409, ErrorCode.VPN_DEPLOY_LAEUFT_BEREITS)
try:
yield
finally:
try:
await conn.execute(text("SELECT pg_advisory_unlock(:a, :b)"), params)
except Exception:
await conn.invalidate() # never hand a connection that may still hold the lock back to the pool
except BaseException: # cancelled while unlocking
await conn.invalidate()
raise
+14 -2
View File
@@ -63,6 +63,17 @@ async def _guarded(host: OpnsenseHost, tool: str, target: str, params: dict, cal
return result
def _with_silent_hops(hops: list[dict]) -> list[dict]:
"""The firewall leaves out hops that never answered (TTL jumps 1 -> 3, T5 of the 1.31.0 test run). Put them back as
explicit rows so a gap reads as "no answer here", not as a missing hop. A route where nothing answered at all stays
an empty list; the UI says so."""
seen = {int(h["ttl"]): h for h in hops if h["ttl"].isdigit()}
if not seen:
return hops
silent = {"address": "", "host": "", "as": "", "probes": "", "silent": True}
return [seen.get(ttl) or {**silent, "ttl": str(ttl)} for ttl in range(1, max(seen) + 1)]
async def traceroute(host: OpnsenseHost, target: str, ipproto: str, protocol: str, source: str | None) -> dict:
target, source = clean_target(target), clean_source(source)
params = {"target": target, "ipproto": ipproto, "protocol": protocol, "source": source}
@@ -73,8 +84,9 @@ async def traceroute(host: OpnsenseHost, target: str, ipproto: str, protocol: st
{"hostname": target, "ipproto": ipproto, "protocol": protocol, "source_address": source})
resp = data.get("response") or {}
hops = [{"ttl": str(r.get("ttl", "")), "address": str(r.get("address", "")), "host": str(r.get("host", "")),
"as": str(r.get("AS", "")), "probes": str(r.get("probes", ""))} for r in resp.get("rows", []) if isinstance(r, dict)]
return {"target": target, "notice": str(resp.get("notice", "")), "hops": hops}
"as": str(r.get("AS", "")), "probes": str(r.get("probes", "")), "silent": False}
for r in resp.get("rows", []) if isinstance(r, dict)]
return {"target": target, "notice": str(resp.get("notice", "")), "hops": _with_silent_hops(hops)}
return await _guarded(host, "traceroute", target, params, call)
@@ -0,0 +1,97 @@
"""Gateway health of a firewall (dpinger numbers, as OPNsense reports them).
``get_gateway_health(host)`` is the one reusable entry point: a live read of
/routes/gateway/status, one :class:`GatewayHealth` per gateway. The notification
engine, the host page and the metrics read the SAME numbers from the periodic
resource sample (``compact`` / ``from_compact``), so a scrape or a page view never
fans out to the firewalls; the 1.25 health gate can call ``get_gateway_health``
directly. Parsing is lenient on purpose: "~" means "nothing measured", never zero.
"""
import re
from app.models.host import OpnsenseHost
from app.schemas.gateway import GatewayHealth, GatewayState
from app.services import opnsense_client as client
_NUMBER = re.compile(r"-?\d+(?:\.\d+)?")
_DEGRADED = {"delay", "loss", "delay+loss"}
def _number(value) -> float | None:
if value is None:
return None
match = _NUMBER.search(str(value))
return float(match.group()) if match else None
def _state(status: str, measured: bool) -> GatewayState:
if status == "down":
return "offline"
if status == "force_down":
return "forced_down"
if status in _DEGRADED:
return "degraded"
if status == "none":
# "none" is also what a gateway without monitoring reports; only numbers make it "online".
return "online" if measured else "unknown"
return "unknown"
def parse_gateway_status(payload) -> list[GatewayHealth]:
items = payload.get("items") if isinstance(payload, dict) else None
result: list[GatewayHealth] = []
for item in items or []:
if not isinstance(item, dict) or not item.get("name"):
continue
status = str(item.get("status") or "").lower()
loss, delay, stddev = _number(item.get("loss")), _number(item.get("delay")), _number(item.get("stddev"))
if loss is not None and loss >= 100:
delay = stddev = None # dpinger prints 0.0 ms when nothing answers: no round trip, not a fast one
measured = loss is not None or delay is not None
address = item.get("address")
monitor = item.get("monitor")
result.append(GatewayHealth(
name=str(item["name"]),
address=None if address in (None, "", "~") else str(address),
status=status or "unknown",
state=_state(status, measured),
loss=loss, delay=delay, stddev=stddev,
monitor=None if monitor in (None, "", "~") else str(monitor),
))
return result
async def get_gateway_health(host: OpnsenseHost) -> list[GatewayHealth]:
"""Live gateway state of ``host``. Raises OpnsenseAPIError if the firewall cannot be asked."""
return parse_gateway_status(await client.get_gateway_status(host))
def compact(gateways: list[GatewayHealth]) -> list[dict]:
"""What the resource sampler stores per sample (small, no addresses)."""
return [
{"name": g.name, "status": g.status, "state": g.state, "loss": g.loss, "delay": g.delay, "stddev": g.stddev}
for g in gateways
]
def from_compact(rows) -> list[GatewayHealth]:
return [
GatewayHealth(
name=r["name"], status=r.get("status") or "unknown", state=r.get("state") or "unknown",
loss=r.get("loss"), delay=r.get("delay"), stddev=r.get("stddev"),
)
for r in rows or [] if isinstance(r, dict) and r.get("name")
]
def is_bad(gateway: GatewayHealth, loss_threshold_pct: float = 0, delay_threshold_ms: float = 0) -> bool:
"""Alarm-worthy: down, or (when the thresholds are on) measured loss / latency beyond them.
A gateway forced down by the admin and one that is not measured are not alarms."""
if gateway.state == "offline":
return True
if loss_threshold_pct and gateway.loss is not None and gateway.loss >= loss_threshold_pct:
return True
if delay_threshold_ms and gateway.delay is not None and gateway.delay >= delay_threshold_ms:
return True
return False
+384
View File
@@ -0,0 +1,384 @@
"""Tickets and gateway sessions for the firewall GUI access (1.27, gui-gateway container).
Flow (see docs in routers/gui_access.py):
* An admin asks for a ticket (POST /api/hosts/{id}/gui-ticket). 32 random bytes, valid 30 s, single use;
only its SHA-256 is stored, bound to the OPNexus session (same hash as sessions.token_hash), the user,
the host and a slot (gateway port = its own browser origin).
* The gateway redeems it through /internal/gui/redeem and gets what it needs to reach the firewall —
base URL, pinned CA, overlay proxy, timeouts. Never the API key or secret.
* Every 20 s the gateway re-checks the gateway session (/internal/gui/revalidate): OPNexus session still
valid, user still admin, switch still on, host still eligible, absolute limit not reached.
A slot stays assigned to one firewall for as long as that browser session has the firewall open; another
firewall gets another slot (another origin), so two firewall GUIs never share an origin at the same time.
After the last gateway session of a slot ended, the slot goes to a DIFFERENT firewall only after SLOT_COOLDOWN
(1.29, S1): a popup of a possibly compromised firewall GUI may still be open and its cookies belong to that
origin. The same firewall may take the slot again at once.
The target the gateway talks to (origin, pinned CA, overlay proxy) is fingerprinted at redeem time; every
re-check compares it with the host's current configuration and ends the session when it changed (S4).
"""
import hashlib
import secrets
import uuid
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from urllib.parse import urlsplit
from sqlalchemy import delete, func, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.errors import ErrorCode
from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
from app.models.host import OpnsenseHost
from app.models.user import AdminUser, Session
from app.services import audit_service, firewall_url, overlay_service
TICKET_TTL = timedelta(seconds=30)
# 20 s: a switch-off, logout or role change must end a gateway session within 30 s of the change, and
# the last re-check can lie up to one interval before it (live test 2026-10-21: 30 s gave 31 s).
REVALIDATE_SECONDS = 20
# A gateway session nobody re-checked for this long is treated as ended (gateway crashed without telling us).
SESSION_STALE_AFTER = timedelta(seconds=REVALIDATE_SECONDS * 3)
MAX_SLOTS = 4
# A slot freed by firewall A goes to firewall B no earlier than this long after A's session ended.
SLOT_COOLDOWN = timedelta(minutes=5)
# Open gateway sessions at once (the gateway container enforces the same number; test_gui_access checks they match).
MAX_GATEWAY_SESSIONS = 64
MIN_GATEWAY_TOKEN_LENGTH = 32
AUDIT_KIND = "gui_access"
@dataclass
class GuiAccessError(Exception):
status: int
code: ErrorCode
params: dict
def hash_value(raw: str) -> str:
return hashlib.sha256(raw.encode()).hexdigest()
def _now() -> datetime:
return datetime.now(timezone.utc)
# ── configuration ─────────────────────────────────────────────────────────
def slot_sites() -> dict[int, list[str]]:
"""Slot port -> configured https origins with that port (OPNEXUS_GUI_SITES)."""
sites: dict[int, list[str]] = {}
for origin in settings.gui_site_origins:
try:
parts = urlsplit(origin)
port = parts.port
except ValueError:
continue
if parts.scheme != "https" or not parts.hostname or not port or parts.path not in ("", "/"):
continue
sites.setdefault(port, []).append(f"https://{parts.netloc}")
return sites
def gateway_configured() -> bool:
return len(settings.gui_gateway_token) >= MIN_GATEWAY_TOKEN_LENGTH and bool(slot_sites())
def unavailable_reasons() -> list[str]:
reasons = []
if not settings.cookie_secure:
reasons.append("cookie_secure")
if len(settings.gui_gateway_token) < MIN_GATEWAY_TOKEN_LENGTH:
reasons.append("gateway_token")
if not slot_sites():
reasons.append("gui_sites")
return reasons
def active_slots(row: GuiAccessSettings) -> list[int]:
return sorted(slot_sites())[: max(1, min(row.slot_count, MAX_SLOTS))]
async def get_settings(db: AsyncSession) -> GuiAccessSettings:
row = await db.get(GuiAccessSettings, 1)
if row is None:
row = GuiAccessSettings(id=1, enabled=False, idle_timeout_minutes=15, absolute_timeout_minutes=60, slot_count=MAX_SLOTS)
db.add(row)
await db.flush()
return row
def host_refusal(host: OpnsenseHost) -> str | None:
"""Why this firewall must not be opened through the gateway (None = eligible)."""
if not host.enabled:
return "disabled"
if urlsplit(host.base_url.strip()).scheme != "https" or firewall_url.check_base_url(host.base_url):
return "https"
if not host.verify_tls:
return "tls"
return None
def firewall_origin(base_url: str) -> str:
parts = urlsplit(base_url.strip())
return f"https://{parts.netloc.rsplit('@', 1)[-1]}"
def target_fingerprint(host: OpnsenseHost) -> str:
"""What a gateway session talks to: origin, pinned CA and overlay route. Changes -> the session must end."""
material = "\n".join([firewall_origin(host.base_url), host.tls_ca_pem or "", overlay_service.proxy_for(host.base_url) or "",
"verify" if host.verify_tls else "noverify"])
return hash_value(material)
# ── audit ─────────────────────────────────────────────────────────────────
async def _audit(actor: str, *, action: str, status: str, host: OpnsenseHost | None = None,
host_id=None, entity_id=None, message: str | None = None, details: dict | None = None) -> None:
token = audit_service.context.set({"actor": actor, "id": str(uuid.uuid4()), "action": action})
try:
await audit_service.persist(
kind=AUDIT_KIND, action=action, status=status, entity_id=str(entity_id) if entity_id else None,
host_id=str(host.id) if host is not None else (str(host_id) if host_id else None),
host_name=host.name if host is not None else None, message=message, details=details or {},
)
finally:
audit_service.context.reset(token)
# ── tickets ───────────────────────────────────────────────────────────────
def _redeem_origin(slot: int, request_host: str) -> str:
candidates = slot_sites().get(slot, [])
wanted = (urlsplit(f"//{request_host}").hostname or "").lower()
for origin in candidates:
if (urlsplit(origin).hostname or "").lower() == wanted:
return origin
return candidates[0]
async def _pick_slot(db: AsyncSession, row: GuiAccessSettings, session_hash: str, user_id, host_id) -> int:
now = _now()
slots = active_slots(row)
absolute = timedelta(minutes=row.absolute_timeout_minutes)
in_use: dict[int, uuid.UUID] = {}
open_sessions = await db.scalars(select(GuiSession).where(
GuiSession.session_hash == session_hash, GuiSession.ended_at.is_(None), GuiSession.started_at > now - absolute))
for gs in open_sessions:
in_use[gs.slot] = gs.host_id
pending = await db.scalars(select(GuiTicket).where(
GuiTicket.session_hash == session_hash, GuiTicket.redeemed_at.is_(None), GuiTicket.expires_at > now))
for ticket in pending:
in_use.setdefault(ticket.slot, ticket.host_id)
for slot in slots:
if in_use.get(slot) == host_id:
return slot # this firewall is already open in this browser session: same origin again
free = [s for s in slots if s not in in_use]
if not free:
raise GuiAccessError(409, ErrorCode.GUI_NO_FREE_SLOT, {"slots": str(len(slots))})
# S1: a slot this user's browser may still have a window of ANOTHER firewall on stays closed to this
# firewall — while a gateway session of the user is open there (any browser session) and for SLOT_COOLDOWN
# after the last one ended. The same firewall may take it again at once.
held = (await db.execute(
select(GuiSession.slot, GuiSession.host_id, GuiSession.ended_at, GuiSession.last_checked_at).where(
GuiSession.user_id == user_id,
((GuiSession.ended_at.is_(None)) & ((GuiSession.started_at > now - absolute) | (GuiSession.last_checked_at > now - SLOT_COOLDOWN)))
| (GuiSession.ended_at > now - SLOT_COOLDOWN)))).all()
cooling: dict[int, datetime] = {} # slot -> when it may go to another firewall
for slot, h_id, ended_at, checked_at in held:
if h_id == host_id or slot not in free:
continue
if ended_at is None and checked_at is not None and now - checked_at > SESSION_STALE_AFTER:
ended_at = checked_at # the gateway stopped re-checking it (crash, kill): it ended when it was last seen
release = (ended_at + SLOT_COOLDOWN) if ended_at is not None else now + SLOT_COOLDOWN
if release <= now:
continue # cooled down long ago (a session that stopped answering a while back)
cooling[slot] = max(cooling.get(slot, release), release)
usable = [s for s in free if s not in cooling]
if not usable:
wait = min(cooling.values()) - now
raise GuiAccessError(409, ErrorCode.GUI_SLOT_COOLING_DOWN,
{"minutes": str(max(1, -(-int(wait.total_seconds()) // 60)))})
free = usable
# Prefer the slot this user had this firewall on before, then the slot the user used least recently
# (an old tab of another firewall on a reused origin is the documented residual risk).
history = (await db.execute(
select(GuiSession.slot, GuiSession.host_id, func.max(GuiSession.started_at))
.where(GuiSession.user_id == user_id).group_by(GuiSession.slot, GuiSession.host_id)
)).all()
for slot, h_id, _ in sorted(history, key=lambda r: r[2], reverse=True):
if h_id == host_id and slot in free:
return slot
last_used = {}
for slot, _, started in history:
last_used[slot] = max(last_used.get(slot, started), started)
epoch = datetime.min.replace(tzinfo=timezone.utc)
return min(free, key=lambda s: (last_used.get(s, epoch), s))
async def issue_ticket(db: AsyncSession, *, user: AdminUser, session_token: str, host: OpnsenseHost,
request_host: str) -> dict:
row = await get_settings(db)
now = _now()
await db.execute(delete(GuiTicket).where(GuiTicket.expires_at < now - timedelta(hours=1)))
session_hash = hash_value(session_token)
slot = await _pick_slot(db, row, session_hash, user.id, host.id)
# Reopening a slot in the same browser session replaces its earlier gateway session, so that one does not count.
open_count = await db.scalar(select(func.count()).select_from(GuiSession).where(
GuiSession.ended_at.is_(None),
GuiSession.started_at > now - timedelta(minutes=row.absolute_timeout_minutes),
~((GuiSession.session_hash == session_hash) & (GuiSession.slot == slot))))
if (open_count or 0) >= MAX_GATEWAY_SESSIONS:
raise GuiAccessError(409, ErrorCode.GUI_TOO_MANY_SESSIONS, {"max": str(MAX_GATEWAY_SESSIONS)})
raw = secrets.token_urlsafe(32)
db.add(GuiTicket(ticket_hash=hash_value(raw), session_hash=session_hash, user_id=user.id, host_id=host.id,
slot=slot, created_at=now, expires_at=now + TICKET_TTL))
await db.commit()
await _audit(user.username, action="ticket", status="accepted", host=host, details={"slot": slot})
return {"url": f"{_redeem_origin(slot, request_host)}/__onx/redeem?t={raw}", "slot": slot,
"expires_in": int(TICKET_TTL.total_seconds())}
# ── gateway side (internal endpoints) ─────────────────────────────────────
async def _live_check(db: AsyncSession, row: GuiAccessSettings, session_hash: str, user_id, host_id,
slot: int) -> tuple[str | None, AdminUser | None, OpnsenseHost | None]:
session = await db.scalar(select(Session).where(Session.token_hash == session_hash))
if session is None or session.expires_at <= _now() or session.user_id != user_id:
return "session", None, None
user = await db.get(AdminUser, user_id)
if user is None or user.role != "admin":
return "role", user, None
if not row.enabled:
return "disabled", user, None
if not settings.cookie_secure or not gateway_configured():
return "config", user, None
host = await db.get(OpnsenseHost, host_id)
if host is None or host_refusal(host):
return "host", user, host
if slot not in active_slots(row):
return "slot", user, host
return None, user, host
def _timing(row: GuiAccessSettings, started_at: datetime) -> dict:
deadline = started_at + timedelta(minutes=row.absolute_timeout_minutes)
return {"idle_timeout_seconds": row.idle_timeout_minutes * 60,
"absolute_deadline": int(deadline.timestamp()),
"revalidate_seconds": REVALIDATE_SECONDS}
async def redeem(db: AsyncSession, *, ticket_hash: str, session_hash: str, slot: int, client_ip: str | None) -> dict:
"""{'ok': True, ...upstream data...} or {'ok': False, 'reason': ...}. Any attempt burns the ticket."""
now = _now()
ticket = await db.scalar(select(GuiTicket).where(GuiTicket.ticket_hash == ticket_hash).with_for_update())
if ticket is None:
await db.rollback()
return {"ok": False, "reason": "unknown"}
replayed = ticket.redeemed_at is not None
ticket.redeemed_at = ticket.redeemed_at or now
row = await get_settings(db)
reason = None
if replayed:
reason = "replayed"
elif ticket.expires_at <= now:
reason = "expired"
elif not secrets.compare_digest(ticket.session_hash, session_hash):
reason = "session"
elif ticket.slot != slot:
reason = "slot"
user = host = None
if reason is None:
reason, user, host = await _live_check(db, row, ticket.session_hash, ticket.user_id, ticket.host_id, slot)
if user is None:
user = await db.get(AdminUser, ticket.user_id)
if host is None:
host = await db.get(OpnsenseHost, ticket.host_id)
actor = user.username if user is not None else "unknown"
if reason is not None:
await db.commit()
await _audit(actor, action="redeem", status="denied", host=host, host_id=ticket.host_id,
message=reason, details={"slot": slot, "client_ip": client_ip, "reason": reason})
return {"ok": False, "reason": reason}
# The same browser session reopening this slot replaces the earlier gateway session there.
superseded = (await db.scalars(select(GuiSession).where(
GuiSession.session_hash == session_hash, GuiSession.slot == slot, GuiSession.ended_at.is_(None)))).all()
if any(old.host_id != host.id for old in superseded):
# Two quick tickets both saw the slot free: the second must not take over another firewall's origin
# without the cooling-down time (S1).
await db.commit()
await _audit(actor, action="redeem", status="denied", host=host, host_id=ticket.host_id,
message="slot", details={"slot": slot, "client_ip": client_ip, "reason": "slot"})
return {"ok": False, "reason": "slot"}
for old in superseded:
old.ended_at, old.end_reason = now, "superseded"
gs = GuiSession(session_hash=session_hash, user_id=user.id, host_id=host.id, slot=slot,
client_ip=(client_ip or None) and client_ip[:45], started_at=now, last_checked_at=now)
db.add(gs)
await db.commit()
await _audit(actor, action="redeem", status="accepted", host=host, entity_id=gs.id,
details={"slot": slot, "client_ip": client_ip})
return {
"ok": True, "gui_session_id": str(gs.id), "host_id": str(host.id), "host_name": host.name,
"upstream_origin": firewall_origin(host.base_url),
"ca_pem": host.tls_ca_pem or None, # public certificate (pin); None = system trust store
"proxy_url": overlay_service.proxy_for(host.base_url),
"target_fp": target_fingerprint(host),
**_timing(row, now),
}
async def _finish(db: AsyncSession, gs: GuiSession, reason: str) -> None:
now = _now()
if gs.ended_at is None:
gs.ended_at, gs.end_reason = now, reason[:32]
await db.commit()
user = await db.get(AdminUser, gs.user_id)
host = await db.get(OpnsenseHost, gs.host_id)
await _audit(user.username if user else "unknown", action="end", status="accepted", host=host,
host_id=gs.host_id, entity_id=gs.id, message=reason,
details={"slot": gs.slot, "client_ip": gs.client_ip, "reason": reason,
"duration_seconds": int((gs.ended_at - gs.started_at).total_seconds())})
async def revalidate(db: AsyncSession, *, gui_session_id: uuid.UUID, session_hash: str,
target_fp: str | None = None) -> dict:
gs = await db.get(GuiSession, gui_session_id)
if gs is None or not secrets.compare_digest(gs.session_hash, session_hash):
return {"ok": False, "reason": "unknown"}
if gs.ended_at is not None:
return {"ok": False, "reason": gs.end_reason or "ended"}
row = await get_settings(db)
if _now() >= gs.started_at + timedelta(minutes=row.absolute_timeout_minutes):
await _finish(db, gs, "absolute")
return {"ok": False, "reason": "absolute"}
reason, _, host = await _live_check(db, row, gs.session_hash, gs.user_id, gs.host_id, gs.slot)
# S4: the gateway presents the fingerprint it got at redeem time. Firewall address, pinned certificate or
# overlay route changed since -> the session (and the upstream connection it holds) ends now.
if reason is None and target_fp is not None and host is not None \
and not secrets.compare_digest(target_fp, target_fingerprint(host)):
reason = "target"
if reason is not None:
await _finish(db, gs, reason)
return {"ok": False, "reason": reason}
gs.last_checked_at = _now()
await db.commit()
return {"ok": True, **_timing(row, gs.started_at)}
async def end(db: AsyncSession, *, gui_session_id: uuid.UUID, session_hash: str, reason: str) -> bool:
gs = await db.get(GuiSession, gui_session_id)
if gs is None or not secrets.compare_digest(gs.session_hash, session_hash):
return False
if gs.ended_at is None:
await _finish(db, gs, reason)
return True
async def end_all(db: AsyncSession, reason: str) -> int:
"""Switch turned off: mark every open gateway session ended (the gateway drops them on its next check)."""
result = await db.execute(update(GuiSession).where(GuiSession.ended_at.is_(None))
.values(ended_at=_now(), end_reason=reason[:32]))
return result.rowcount or 0
@@ -12,6 +12,7 @@ from app.models import OpnsenseHost, RuleDeployment, AliasDeployment, PortForwar
from app.models.host_operation import HostOperation
from app.models.restore_operation import RestoreOperation
from app.services import opnsense_client as client, backup_store, carp_maintenance_window_service
from app.services.deploy_flow import unsettled
from app.services.host_status_service import get_statuses
DEPLOYMENTS = (RuleDeployment, AliasDeployment, PortForwardDeployment, OneToOneDeployment, OutboundNatDeployment)
@@ -34,7 +35,27 @@ async def peers(db, host):
return [host]
async def ensure_no_maintenance(db, hosts):
async def ensure_no_open_cleanup(db, hosts):
"""A rule cleanup (1.31) that is running, waiting for its confirmation or unresolved owns the host and its
cluster peers: nothing else may write to the firewall until it is confirmed, reverted or settled."""
from app.services.rule_cleanup_service import has_open_cleanup
ids = set()
for host in hosts:
ids.update(h.id for h in await peers(db, host))
if ids and await has_open_cleanup(db, ids):
raise api_error(409, ErrorCode.OPEN_CLEANUP_BLOCKS_DEPLOY)
async def ensure_no_open_cleanup_on(db, model, condition):
"""ensure_no_open_cleanup for every host that has a deployment row of ``model`` matching ``condition`` (the
undeploy paths: deleting a central object removes it from all its firewalls)."""
ids = set((await db.scalars(select(model.host_id).where(condition))).all())
hosts = [h for h in [await db.get(OpnsenseHost, i) for i in ids] if h is not None]
await ensure_no_open_cleanup(db, hosts)
async def ensure_no_maintenance(db, hosts, *, cleanup=True):
"""cleanup=False: only the cleanup service itself, which reports an open cleanup with its own error code."""
ids = set()
for host in hosts:
ids.update(h.id for h in await peers(db, host))
@@ -42,13 +63,15 @@ async def ensure_no_maintenance(db, hosts):
raise api_error(409, ErrorCode.OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY)
if ids and await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
raise api_error(409, ErrorCode.OPEN_RESTORE_BLOCKS_DEPLOY)
if cleanup and ids:
await ensure_no_open_cleanup(db, hosts)
async def _shared_change_blockers(db, fleet):
blocked = []
fleet_ids = [h.id for h in fleet]
for model in DEPLOYMENTS:
if await db.scalar(select(model.id).where(model.host_id.in_(fleet_ids), model.status.in_(("pending", "revert_failed"))).limit(1)):
if await db.scalar(select(model.id).where(model.host_id.in_(fleet_ids), unsettled(model)).limit(1)):
blocked.append("Offene Bestätigung oder fehlgeschlagener Rollback im Host/Cluster.")
break
operations = list((await db.scalars(select(HostOperation).where(HostOperation.host_id.in_(fleet_ids), HostOperation.status.in_(ACTIVE)))).all())
@@ -56,6 +79,9 @@ async def _shared_change_blockers(db, fleet):
blocked.append("Eine frühere Wartungsaktion im Host/Cluster muss zuerst geprüft und abgeschlossen werden.")
if await db.scalar(select(RestoreOperation.id).where(RestoreOperation.host_id.in_(fleet_ids), RestoreOperation.status.in_(RESTORE_OPEN)).limit(1)):
blocked.append("Eine Wiederherstellung im Host/Cluster ist offen und muss zuerst bestätigt oder abgebrochen werden.")
from app.services.rule_cleanup_service import has_open_cleanup
if await has_open_cleanup(db, fleet_ids):
blocked.append("Eine Regel-Bereinigung im Host/Cluster ist offen und muss zuerst bestätigt oder zurückgenommen werden.")
return blocked
@@ -32,6 +32,14 @@ def _extract_carp_status(data: dict | None) -> str | None:
return None
def _vip_states(data: dict | None) -> dict[str, str]:
"""Per-VHID CARP state ("MASTER", "BACKUP", "DISABLED", "INIT", ...). The host-level carp_status above
collapses these, so one VHID that is down on a node is invisible there."""
rows = data.get("rows") if isinstance(data, dict) else None
return {str(row.get("vhid")): str(row.get("status", "")).upper()
for row in rows or [] if isinstance(row, dict) and row.get("vhid") not in (None, "")}
async def _status_for(host: OpnsenseHost) -> dict:
if not host.enabled:
return {"status": "disabled", "carp_status": None, "status_detail": None}
@@ -43,6 +51,7 @@ async def _status_for(host: OpnsenseHost) -> dict:
return {
"status": "online",
"carp_status": _extract_carp_status(data),
"carp_vips": _vip_states(data),
"carp_maintenance_mode": _truthy(carp.get("maintenancemode"), default=False),
"carp_allowed": _truthy(carp.get("allow"), default=True),
"carp_demotion": carp.get("demotion"),
+26 -9
View File
@@ -109,6 +109,8 @@ from app.models.ipsec_tunnel import IpsecTunnel
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.services import opnsense_client, ipsec_psk_store
from app.services.opnsense_client import OpnsenseAPIError
from app.services.deploy_flow import checkpoint
from app.services.deploy_lease import VPN_INTERRUPTED, resource_lease
_VERSION_MAP = {"ike": "0", "ikev1": "1", "ikev2": "2"}
@@ -176,7 +178,7 @@ def build_payloads(tunnel: IpsecTunnel) -> dict:
}
async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTunnelDeployment) -> None:
async def _write_config(db: AsyncSession, host: OpnsenseHost, payloads: dict, deployment: IpsecTunnelDeployment) -> None:
"""Runs the 5 config-write calls in dependency order, persisting each
UUID onto `deployment` as soon as it's known so a later retry (after a
partial failure) updates rather than duplicates. Raises
@@ -189,6 +191,7 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTun
deployment.opnsense_connection_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_connection_uuid:
raise OpnsenseAPIError("Firewall hat keine Connection-UUID zurückgegeben; Stand manuell prüfen")
await checkpoint(db) # the UUID is durable before the next remote call
# `payloads` (built by build_payloads()) is also what gets persisted as
# last_payload for no-op-redeploy comparison — the injected `connection`
@@ -209,6 +212,7 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTun
deployment.opnsense_local_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_local_uuid:
raise OpnsenseAPIError("Firewall hat keine Local-UUID zurückgegeben; Stand manuell prüfen")
await checkpoint(db) # the UUID is durable before the next remote call
if deployment.opnsense_remote_uuid:
await opnsense_client.set_ipsec_remote(host, deployment.opnsense_remote_uuid, remote_wire)
@@ -217,6 +221,7 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTun
deployment.opnsense_remote_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_remote_uuid:
raise OpnsenseAPIError("Firewall hat keine Remote-UUID zurückgegeben; Stand manuell prüfen")
await checkpoint(db) # the UUID is durable before the next remote call
if deployment.opnsense_child_uuid:
await opnsense_client.set_ipsec_child(host, deployment.opnsense_child_uuid, child_wire)
@@ -225,6 +230,7 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTun
deployment.opnsense_child_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_child_uuid:
raise OpnsenseAPIError("Firewall hat keine Child-UUID zurückgegeben; Stand manuell prüfen")
await checkpoint(db) # the UUID is durable before the next remote call
if deployment.opnsense_psk_uuid:
await opnsense_client.set_ipsec_psk(host, deployment.opnsense_psk_uuid, payloads["psk"])
@@ -233,6 +239,7 @@ async def _write_config(host: OpnsenseHost, payloads: dict, deployment: IpsecTun
deployment.opnsense_psk_uuid = result.get("uuid") if isinstance(result, dict) else None
if not deployment.opnsense_psk_uuid:
raise OpnsenseAPIError("Firewall hat keine PSK-UUID zurückgegeben; Stand manuell prüfen")
await checkpoint(db) # the UUID is durable before the next remote call
async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseHost) -> IpsecTunnelDeployment:
@@ -241,7 +248,7 @@ async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseH
existing = await db.scalar(
select(IpsecTunnelDeployment).where(
IpsecTunnelDeployment.tunnel_id == tunnel.id, IpsecTunnelDeployment.host_id == host.id
)
).execution_options(populate_existing=True) # fresh: another request may have just changed the row
)
if existing and existing.status == "success" and existing.last_payload == comparable:
# Nothing actually changed on this host — re-affirm without
@@ -252,8 +259,15 @@ async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseH
deployment = existing or IpsecTunnelDeployment(tunnel_id=tunnel.id, host_id=host.id, status="error")
# Durable intent BEFORE the first remote call: from here on a crash or cancellation leaves a row, and every UUID the
# firewall hands out is committed at once (see _write_config), so a retry continues instead of duplicating.
deployment.status, deployment.message, deployment.deployed_at = "error", VPN_INTERRUPTED, _now()
if not existing:
db.add(deployment)
await checkpoint(db)
try:
await _write_config(host, payloads, deployment)
await _write_config(db, host, payloads, deployment)
# Same trap as WireGuard: with the global IPsec switch off the config
# is stored but nothing runs, and reconfigure still answers ok.
await opnsense_client.ensure_ipsec_enabled(host)
@@ -261,8 +275,6 @@ async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseH
deployment.status = "error"
deployment.message = str(exc)
deployment.deployed_at = _now()
if not existing:
db.add(deployment)
return deployment
try:
@@ -277,32 +289,37 @@ async def _deploy_to_host(db: AsyncSession, tunnel: IpsecTunnel, host: OpnsenseH
deployment.message = "Konfiguration wurde geschrieben, Dienst-Reload aber nicht bestätigt — Erreichbarkeit manuell prüfen"
deployment.last_payload = comparable
deployment.deployed_at = _now()
if not existing:
db.add(deployment)
return deployment
deployment.status = "success"
deployment.message = None
deployment.last_payload = comparable
deployment.deployed_at = _now()
if not existing:
db.add(deployment)
return deployment
async def deploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel, hosts: list[OpnsenseHost]) -> list[IpsecTunnelDeployment]:
from app.services.host_maintenance_service import ensure_no_maintenance
await ensure_no_maintenance(db, hosts)
async with resource_lease("ipsec", tunnel.id):
deployments = [await _deploy_to_host(db, tunnel, host) for host in hosts]
await db.commit()
return deployments
async def undeploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel) -> None:
async with resource_lease("ipsec", tunnel.id):
await _undeploy_tunnel(db, tunnel)
async def _undeploy_tunnel(db: AsyncSession, tunnel: IpsecTunnel) -> None:
"""Explicit action, not gated by anything (there is no lockout state
for this entity to gate on) — deletes Child/Local/Remote before their
parent Connection (the ModelRelationField dependency direction), then
the PreSharedKey (independent of the other 4), then reconfigures."""
# Deleting writes to the firewalls: not while a rule cleanup (1.31) owns one of them (or its cluster).
from app.services.host_maintenance_service import ensure_no_open_cleanup_on
await ensure_no_open_cleanup_on(db, IpsecTunnelDeployment, IpsecTunnelDeployment.tunnel_id == tunnel.id)
failures = []
result = await db.execute(select(IpsecTunnelDeployment).where(IpsecTunnelDeployment.tunnel_id == tunnel.id))
for deployment in result.scalars().all():
Loaded 100 of 316 files, more files were not shown because too many files have changed in this diff. Show more