Release v1.30.0

- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
  Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
  trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
  dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
  höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
  verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
  Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
  stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
  Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
  verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
  Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
  automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
  erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
  Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
  15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
  Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
  mehr verdeckte Regeln als bisher, nie weniger belegte.

.env.example

@@ -100,3 +100,26 @@ TRUST_PROXY_HEADERS=false
 # Zum Aktivieren (Handbuch „Firewalls hinter NAT“): COMPOSE_PROFILES=overlay
 # und den UDP-Port aus dem Internet erreichbar machen.
 #OVERLAY_PORT=51820
+
+# ── GUI-Fernzugriff auf verwaltete Firewalls (optional, Standard AUS) ──
+# Öffnet die Weboberfläche einer Firewall über OPNexus, auch hinter NAT
+# (Management-Tunnel), jeweils in einem eigenen Browser-Ursprung (Slot-Port).
+# Dreifach aus, bis alles hier gesetzt ist UND ein Admin den Schalter unter
+# Einstellungen einschaltet. Voraussetzungen: HTTPS über den Proxy und
+# COOKIE_SECURE=true. Siehe Handbuch, „Firewall-Oberfläche über OPNexus“.
+# 1. Profil aktivieren (mit anderen Profilen kommagetrennt, z. B. overlay,gui):
+# COMPOSE_PROFILES=gui
+# 2. Slot-Ursprünge: dieselbe Adresse wie in OPNEXUS_SITES, je Slot ein Port
+#    aus GUI_PORTS. Darf NICHT in FRONTEND_ORIGIN stehen (Start wird sonst verweigert).
+# OPNEXUS_GUI_SITES=https://10.0.0.5:9441, https://10.0.0.5:9442, https://10.0.0.5:9443, https://10.0.0.5:9444
+# 3. Gemeinsames Geheimnis Backend <-> Gateway, mind. 32 Zeichen, z. B.
+#    `openssl rand -hex 32`:
+# GUI_GATEWAY_TOKEN=
+# 4. Wo die Slot-Ports lauschen. Standard nur lokal (127.0.0.1); für den
+#    Zugriff aus dem Netz die Adresse dieses Hosts eintragen (wie bei
+#    PROXY_BIND_ADDRESS). GUI_PORTS ist ein Bereich, Host- und Container-Port
+#    sind gleich. GUI_PORTS zum Einschalten SETZEN: ohne den Eintrag nimmt
+#    Docker zufällige freie Ports (damit ein Update nie an belegten Ports wie
+#    9443 scheitert), und die Slot-Adressen oben wären nicht erreichbar.
+# GUI_BIND_ADDRESS=127.0.0.1
+# GUI_PORTS=9441-9444

CHANGELOG.md

@@ -31,6 +31,252 @@ den Einstellungen.
 Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
 ansteht): Merlin …
 
+## [1.30.0] "Owl" — 2026-10-22
+
+- **Regel-Analyse: Bereinigung (nur Ansicht):** Auf **Regelwerk › Regel-Analyse** gibt es den Tab **Bereinigung** (`?tab=cleanup`). Er listet je Firewall die
+  Regeln, die einen Blick wert sind: **ungenutzt** (OPNexus hat in 7/30/90/180 Tagen keinen Treffer gesehen), **verdeckt** (eine frühere Quick-Regel
+  trifft schon alles, was sie trifft), **doppelt** und **lange deaktiviert**. Jeder Befund trägt eine Sicherheit (**sicher / vermutlich / unbekannt**, Tooltip mit
+  dem Grund): „sicher“ gibt es nur mit vollständigen, frischen Belegen, bei einer Regel mit Zeitplan, ohne geladenen pf-Regelsatz oder bei Aliasen bleibt es
+  höchstens „vermutlich“. Die Tabelle zeigt auch den letzten Treffer und ob die Regel entfernbar wäre (nicht: automatische Regeln, Regeln im Altformat, von OPNexus
+  verwaltete Regeln – die entfernt man über die Regelseite). Es wird nur gelesen; **Entfernen folgt in einer späteren Version**. Die Sicherheitslage verlinkt ihre
+  Befunde „ungenutzt/verdeckt/doppelt“ direkt dorthin; per API `GET /api/hosts/{id}/rule-analysis/cleanup?unused_days=30`, per MCP `rule_cleanup_candidates`.
+- **Änderungsverlauf je Regel:** OPNexus liest alle 15 Minuten die Regeln jeder Firewall und merkt sich, was sich geändert hat – auch bei Regeln, die nicht von OPNexus
+  stammen: angelegt, geändert (mit Vorher/Nachher), aktiviert, deaktiviert, entfernt, jeweils mit der Herkunft „über OPNexus“ (passender Audit-Eintrag) oder „extern“.
+  Auf der Live-Regel-Ansicht (Regelwerk › Live über alle Hosts) öffnet die Zeilenaktion **Verlauf** die Ereignisse zusammen mit den gespeicherten Fassungen und dem Audit der
+  verwalteten Regel. Die erste Beobachtung speichert nur den Ausgangszustand; der Verlauf wird 365 Tage aufbewahrt (`GET /api/hosts/{id}/live-rules/{uuid}/history`).
+  Eine Umsortierung (andere Reihenfolge oder Gruppe) gilt als „verschoben“, nicht als Änderung. Eine leere oder unvollständige Antwort der Firewall (ohne
+  automatische Regeln) wird verworfen; verschwinden mehr als die Hälfte der Regeln auf einmal, zählt erst die zweite gleiche Beobachtung. Die Herkunft „über OPNexus“
+  erklärt nur Audit-Einträge seit der vorherigen Beobachtung, und jeder Eintrag nur eine Änderung.
+  Schemaänderungen 0027 und 0028 (Tabellen und ein Feld für den Beobachtungszeitraum) laufen beim Start.
+- **Regelnutzung genauer:** Die Zähler kommen jetzt aus `ruleStats` (von der Firewall schon je Regel summiert; ältere Versionen weiterhin über den pf-Regelsatz) und werden alle
+  15 Minuten statt alle 6 Stunden gelesen. Regeln im Altformat (noch nicht migriert) werden ebenfalls erfasst. Das Beobachtungsfenster für „ungenutzt“ beginnt neu, wenn eine Regel eingeschaltet oder geändert wird, und nach einem pf-Neuladen mit langer Lücke bei den Zählern.
+- **Verdeckt/doppelt in der Sicherheitslage genauer:** Der Vergleich kennt jetzt die echte Reihenfolge (Floating vor Interface-Gruppen vor Interface-Regeln), Netze und
+  Portbereiche als Mengen (z. B. 10.1.0.0/16 liegt in 10.0.0.0/8), `inet46` und `tcp/udp`, und übergeht Regeln mit Zeitplan, Markierung oder Limits. Es findet dadurch
+  mehr verdeckte Regeln als bisher, nie weniger belegte.
+
+## [1.29.0] "Caracara" — 2026-10-22
+
+- **Fehlende API-Rechte werden sichtbar:** Darf der API-Schlüssel einer Firewall den Gateway- oder Dienststatus nicht lesen (OPNsense antwortet
+  mit 403), fiel das bisher niemandem auf, und der standardmäßig aktive Gateway-Alarm konnte nie auslösen. Jetzt zeigen die Host-Seite
+  (Abschnitt „Gateways“) und die Dashboard-Karte „Fleet-/CARP-Gesundheit“ ein Warn-Badge „API-Rechte fehlen: Gateways, Dienste“; der Tooltip nennt die
+  OPNsense-Berechtigung, die beim Benutzer oder bei der Gruppe fehlt (System: Gateways, Status: Services, Lobby: Dashboard). Zusätzlich gibt es
+  den neuen Alarm **API-Rechte fehlen** (standardmäßig an, abschaltbar unter Einstellungen › Benachrichtigungen, gleiche Entprellung wie die
+  anderen Messungs-Alarme) und die Prometheus-Metrik `opnexus_host_api_rights_missing{host,read}`. Nur eine ausdrückliche Ablehnung zählt;
+  Zeitüberschreitungen oder Serverfehler bleiben „unbekannt“.
+- **Gateway-Alarm pro Gateway abschaltbar:** In der Gateway-Liste der Host-Seite schaltet eine Glocke den Alarm für ein einzelnes Gateway aus
+  (z. B. ruhendes Zweit-WAN oder tote IPv6-Gateways); ein „Alarm aus“-Badge zeigt es an. Ein offener Alarm dieses Gateways löst sich dabei
+  auf. Nur Administratoren, die Änderung steht im Audit-Log. Das Dashboard zeigt bei mehr als acht Gateways „+ N weitere“.
+- **Firewall-Oberfläche (GUI-Zugriff) härter:** Ein frei gewordener Platz wird für eine **andere** Firewall erst 5 Minuten nach dem Ende der letzten
+  Sitzung vergeben (Schutz, falls dort noch ein Tab offen ist, der die Cookies des Platzes teilt); dieselbe Firewall bekommt ihn sofort wieder.
+  Sind alle Plätze belegt oder noch in der Abkühlzeit, nennt die Meldung den Grund. Ändern sich Adresse, festgesetztes Zertifikat oder Tunnelweg
+  einer Firewall, während ihre Oberfläche offen ist, endet die Sitzung spätestens nach 20 Sekunden. Das Limit von 64 gleichzeitigen Sitzungen
+  verdrängt keine laufende Sitzung mehr: eine neue wird abgelehnt (mit Meldung), bis eine endet. **Hinweis:** Das GUI-Gateway-Image muss zur
+  Backend-Version passen (wie bisher beim Update beider Container).
+- **Anmeldung (Cookies) härter:** Das alte Sitzungs-Cookie `opnexus-session` wird bei HTTPS **nicht mehr akzeptiert**. **Alle, die sich seit der
+  Umstellung auf 1.27 nicht neu angemeldet haben, müssen sich einmal neu anmelden**; Skripte mit rohem Sitzungs-Cookie brauchen den Namen
+  `__Host-opnexus-session` (API-Tokens sind nicht betroffen, ohne HTTPS bleibt `opnexus-session` der normale Name). Beim Abmelden wird ein altes
+  Cookie weiterhin gelöscht. Das Cookie, das die SSO-Anmeldung an den Browser bindet, heißt bei HTTPS jetzt `__Host-opnexus-oidc-state`; der
+  Rückweg vom Anbieter wird nur noch als Seitenaufruf (Navigation) von einer fremden Seite zugelassen, und ohne dieses Cookie scheitert er immer.
+- **Webhook, Benachrichtigungskanäle und SSO-Anbieter: Schutz vor DNS-Rebinding:** Bisher prüfte OPNexus die Adresse, auf die ein Name zeigt, und
+  der HTTP-Client löste den Namen danach noch einmal auf; ein Server unter Fremdkontrolle konnte beim zweiten Mal eine interne Adresse liefern. Jetzt
+  verbindet OPNexus genau zu der geprüften Adresse (Host-Name, TLS-Zertifikatsprüfung und Anfrage bleiben beim Namen); zeigt eine Antwort auch nur
+  teilweise auf Loopback, Link-Local oder einen Cloud-Metadaten-Dienst (auch AWS-IPv6 und Alibaba), kommt keine Verbindung zustande und die Zustellung meldet „Zieladresse nicht
+  erlaubt“. Private Netze bleiben erlaubt (eigener ntfy, Gotify, Matrix). Proxy-Umgebungsvariablen werden für diese Aufrufe nicht mehr beachtet.
+- **Benachrichtigungen parallel zugestellt:** Webhook, E-Mail und alle Kanäle werden gleichzeitig bedient, jeder mit eigener Zeitgrenze (20 Sekunden),
+  und ein Prüfdurchlauf sendet insgesamt höchstens 90 Sekunden. Ein hängender Kanal verzögert weder die anderen noch die nächsten Prüfungen; im Log
+  steht dann „timed out“ (ohne Adresse oder Token). Wie bisher wird einmal je Ereignis zugestellt, ohne Wiederholung, **außer** der Durchlauf war so voll, dass
+  für ein Ereignis noch gar kein Versuch stattfand: es wird im nächsten Durchlauf zugestellt. Die Zustellung beginnt jetzt nach allen Prüfungen
+  eines Durchlaufs statt mitten darin.
+- **Rollouts: Fingerabdruck, verantwortliche Person, überlappende Termine:** Der Fingerabdruck eines aktivierten Plans umfasst jetzt auch die
+  ausgelassenen und verschobenen Termine seines Fensters: **wer einen Termin auslässt oder verschiebt, muss einen automatischen Plan danach neu
+  aktivieren** (ohne solche Ausnahmen bleiben bestehende Aktivierungen gültig; **Pläne, deren Fenster schon Ausnahmen haben, nach dem Update
+  einmal neu aktivieren**, sonst meldet der Plan „Plan geändert“ und startet nicht automatisch). Die verantwortliche Person (Aktivierende, Startende, Freigebende)
+  wird zusätzlich als Konto-ID gespeichert (Migration 0026, bestehende Einträge werden über den Namen zugeordnet): ein gelöschtes und unter gleichem
+  Namen neu angelegtes Konto gilt nicht mehr als dieselbe Person, der Lauf pausiert bzw. der automatische Start unterbleibt mit „Person darf nicht mehr
+  ausrollen“. Angezeigt wird weiter der Benutzername. Überlappen sich zwei Termine eines Wartungsfensters, zählt der zuletzt begonnene (bisher der
+  früheste), sodass ein älterer, längst abgeschnittener Termin keinen neuen Start mehr blockiert.
+- **Kleinigkeiten:** Ressourcen-Alarme zeigen bei fehlendem Messwert keine Zahl mehr („None %“ entfällt). Der Dialog „Von Firewall übernehmen“ zeigt ohne aktive
+  Firewall einen Hinweis statt dauerhaft „Lädt…“. Fehlermeldungen beim Übernehmen und Verknüpfen nennen die Gründe in Klartext (Deutsch/Englisch) statt
+  roher Codes wie `name_exists`; dasselbe gilt für „GUI-Zugriff nicht eingerichtet“ und blockierte Netzwerkänderungen. Der Test-Knopf der
+  Benachrichtigungskanäle ist für Nicht-Admins ausgeblendet (der Server verweigert den Test ohnehin). Der Tooltip der aktiven Plätze nennt keine
+  Umgebungsvariable mehr. Eine Gateway-Sitzung, die der Gateway-Container nicht mehr bestätigt (Absturz), zählt für die Abkühlzeit der Plätze als
+  beendet.
+- **Grafana-Dashboard erweitert:** neue Felder für Gateways (erreichbar, Paketverlust, Latenz), Swap, gestoppte Dienste, fehlende API-Rechte und
+  Rollouts nach Status; dazu die Metrik `opnexus_rollout_runs{status}`.
+
+## [1.28.0] "Sparrowhawk" — 2026-10-21
+
+- **Übernehmen: mit vorhandenem zentralem Objekt verknüpfen:** Dieselbe von Hand angelegte Regel, derselbe Alias oder dieselbe NAT-Regel auf mehreren
+  Firewalls (z. B. beide HA-Knoten) wird nicht mehr als zweites Objekt übernommen. Im Dialog „Von Firewall übernehmen“ erscheint bei exakter
+  Übereinstimmung (nach Einsetzen der Host-Variablen, bei Aliasen inkl. Verweisen) die Aktion **Verknüpfen**: die Firewall wird weiteres Ziel des
+  vorhandenen Objekts, die Vorschau zeigt „keine Änderung“, und der nächste Rollout aktualisiert alle Firewalls an einem Objekt. Auf der Firewall wird
+  dabei nichts geändert. Unterscheidet sich ein gleichnamiges Objekt, zeigt der Dialog die abweichenden Felder; verknüpft wird dann nie. „Alle
+  übernehmbaren“ fragt vorher nach („3 verknüpfen, 2 neu übernehmen, 1 nicht übernehmbar“). Aliase, die bisher wegen „Name existiert“ abgelehnt wurden,
+  lassen sich so übernehmen.
+  Wartet das vorhandene Objekt gerade auf eine Bestätigung, wird nicht verknüpft; „Alle übernehmbaren“ lässt den Eintrag dann aus. Verknüpfen
+  macht ein gerade geöffnetes Bearbeiten-Formular des Objekts ungültig, damit die neue Firewall nicht versehentlich wieder aus den Zielen fällt.
+
+- **Tooltips überarbeitet:** Der deaktivierte Button „Oberfläche öffnen“ nennt jetzt den Grund im Klartext samt nächstem Schritt (statt nur einer
+  Variablen wie `GUI_GATEWAY_TOKEN`); dasselbe gilt für ausgeschaltete Funktion, deaktivierte Firewall, Nicht-HTTPS-Adresse und ausgeschaltete
+  Zertifikatsprüfung. Texte durchgehend in Du-Form, kürzere VPN-Einleitungen, verständlichere Hinweise bei gesperrten Buttons (Zertifikate, Benutzer,
+  Dienste). Icon-Buttons (Bearbeiten, Löschen, Schließen, Reihenfolge) zeigen beim Darüberfahren ihren Namen.
+
+## [1.27.2] "Buzzard" — 2026-10-21
+
+- **`opnexus update` prüft den Proxy vor dem Wechsel:** Nach dem Ziehen der Images validiert das Werkzeug die Caddy-Konfiguration des neuen
+  Proxy-Images mit den Umgebungsvariablen *dieser* Installation. Ist sie ungültig, bricht das Update ab, bevor etwas gewechselt wurde.
+  Der Fall aus 1.27.0 (Compose-Datei mit leerem `OPNEXUS_GUI_SITES`) wird dabei automatisch repariert: Das Werkzeug trägt den Platzhalter
+  `OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` ein und sagt es. Installationen, die noch die Compose-Datei aus 1.27.0 haben,
+  müssen also nur einmal `opnexus update` ausführen.
+- **`opnexus update` erkennt einen abstürzenden Proxy:** Ein Dienst ohne Healthcheck (der Proxy) gilt jetzt erst nach 10 Sekunden
+  durchgehender Laufzeit als gesund. Bisher meldete das Update „Fertig“, obwohl der Proxy in einer Neustart-Schleife hing, und das
+  automatische Zurückrollen griff nicht.
+
+## [1.27.1] "Buzzard" — 2026-10-21
+
+- **Fix (1.27.1): Proxy startete mit der neuen Compose-Datei nicht.** Mit der `docker-compose.yml` aus 1.27.0 gab Compose die Variable
+  `OPNEXUS_GUI_SITES` als leeren Text an den Proxy weiter; Caddy nutzt seinen Standardwert aber nur bei *nicht gesetzter* Variable und
+  brach mit „server block without any key“ ab, OPNexus war dann nicht mehr erreichbar. Die Compose-Datei setzt jetzt einen gültigen
+  Platzhalter als Standard. Betroffen waren nur Installationen, die die neue Compose-Datei aus 1.27.0 übernommen hatten (ohne
+  `OPNEXUS_GUI_SITES` in der `.env`); wer nur die Images aktualisiert hatte, war nicht betroffen. Sofort-Abhilfe ohne Update:
+  `OPNEXUS_GUI_SITES=https://localhost:9441` in die `.env` eintragen und `docker compose up -d proxy` ausführen.
+
+## [1.27.0] "Buzzard" — 2026-10-21
+
+Vorgesehen als MINOR-Release **1.27.0** „Firewall-Oberfläche über OPNexus“.
+
+- **Firewall-Oberfläche über OPNexus:** Auf der Host-Seite öffnet **Oberfläche öffnen** die Weboberfläche der Firewall
+  in einem neuen Tab, auch bei Firewalls hinter NAT im Management-Tunnel. Nur für Admins; jede Firewall läuft in einer
+  eigenen Browser-Origin (Platz = Port), Cookies bleiben auf dem Server, das Ticket gilt einmalig 30 Sekunden,
+  Inaktivitäts- und Gesamtlimit sind einstellbar, Ticket, Start und Ende stehen im Audit (Art „GUI-Zugriff“).
+  Standardmäßig **aus**: braucht das Compose-Profil `gui`, `OPNEXUS_GUI_SITES`, `GUI_PORTS`, `GUI_GATEWAY_TOKEN`,
+  `COOKIE_SECURE=true` und den Schalter unter Einstellungen (Handbuch, „Firewall-Oberfläche über OPNexus“).
+  Live getestet auf OPNsense 26.7.
+- **Einschränkungen:** kein WebSocket, keine automatische Anmeldung, höchstens 4 Plätze gleichzeitig. OPNexus-Adresse
+  vorher in die Whitelist des Anmeldeschutzes (sshguard) der Firewall eintragen, sonst können Fehlanmeldungen die
+  API-Zugriffe von OPNexus aussperren. Nach einem Neustart des wg-hub braucht der Tunnel 2 bis 4 Minuten.
+- **Hinweis für Betreiber mit eigenen Skripten:** Bei HTTPS heißt das Sitzungs-Cookie jetzt `__Host-opnexus-session`
+  (bestehende Sitzungen werden ohne Abmelden übernommen), und Cookie-Anfragen an `/api` mit `Sec-Fetch-Site: same-site/cross-site`
+  werden abgelehnt. Zugriffe mit API-Token sind nicht betroffen.
+
+## [1.26.0] "Kite" — 2026-10-21
+
+Geplant als MINOR-Release **1.26.0** (Regel-Analyse). Backend, MCP und Oberfläche.
+
+- **Regel-Analyse „Welche Regel greift?“ (API):** `GET /api/hosts/{id}/rule-analysis/match` und
+  `GET /api/rule-analysis/match` (ganze Flotte, Auswahl oder Gerätegruppe) beantworten für Quelle, Ziel, Protokoll
+  und Port, ob eine Firewall das erste Paket einer neuen Verbindung durchlässt oder blockiert, welche Regel das
+  entscheidet und warum (Prüfprotokoll Regel für Regel). Grundlage ist der tatsächlich geladene pf-Regelsatz
+  inklusive automatischer Regeln, Floating- und Gruppenregeln, `quick`/letzter Treffer, Negation, verschachtelter
+  Aliase (pf-Tabellentest der Firewall), Port-Aliase und Adressfamilien. Jede Antwort trägt eine ehrliche
+  Konfidenz **sicher / vermutlich / unbekannt** mit Begründungscodes; was nicht belegbar ist (z. B. `tagged`,
+  nicht auflösbare Aliase, geratenes Interface, NAT in der Nähe), wird nie als „sicher“ gemeldet. Nur lesend,
+  auch mit `read`-Token nutzbar; es wird nichts gespeichert und nichts an der Firewall geändert.
+- **MCP:** neues Werkzeug `rule_match` für KI-Assistenten (gleiche Abfrage, ohne Prüfprotokoll).
+- **Seite „Regel-Analyse“ (Regelwerk):** Formular für Hosts oder Gerätegruppe, Quell-/Ziel-IP, Protokoll, Ports,
+  Eingangs-Interface („automatisch“ oder Liste) und optional die Ausgangsseite. Die Abfrage steht in der Adresszeile
+  (teilbar, wird beim Öffnen sofort ausgeführt). Je Firewall eine Karte mit Erlaubt/Blockiert/Abgewiesen, Sicherheit
+  des Urteils, entscheidender Regel und Hinweisen; ein „unbekannt“-Ergebnis erscheint bewusst nicht als Urteil.
+  Aufklappbares „Warum?“ zeigt jede geprüfte Regel mit ✓/✗/? je Feld, überstimmte Regeln durchgestrichen. Auch mit
+  Nur-Lesen-Rolle nutzbar.
+
+## [1.25.0] "Goshawk" — 2026-10-21
+
+Geplant als MINOR-Release **1.25.0** (Flotten-Rollout).
+
+- **Rollouts in Wellen (System › Rollouts):** Firmware-Updates für die ganze Flotte in Ringen, z. B. erst
+  `ring-canary`, dann `ring-pilot`, dann `ring-prod` (Wellen aus Gerätegruppen). Jeder Host läuft über die
+  bestehende geplante Wartung (Backup vor dem Update, CARP-Übergabe für den MASTER, Audit), immer nur ein Host
+  gleichzeitig und nur ein Rollout zur selben Zeit; im Cluster zuerst der BACKUP, dann der MASTER. Nach jeder Welle
+  folgen eine Beobachtungszeit und eine **Gesundheitsprüfung** (erreichbar, Zielversion installiert, CARP je VHID,
+  Gateways und Dienste wie vorher, keine offenen Sperren, optional Sicherheits-Score). Rot hält den Rollout an
+  (nach einer Nachprüfung zwei Minuten später) und meldet sich. Anhalten, Fortsetzen (prüft zuerst alles bisher
+  Aktualisierte erneut), Welle freigeben, Host überspringen und Abbrechen auf der Lauf-Seite; die Seite zeigt
+  Wellen, Hosts (Version vorher → nachher, aktueller Schritt) und die Prüfmatrix.
+- **Sicherheitsnetz:** Weicht die angebotene Version in einem späteren Ring von der des ersten Rings ab, hält der
+  Rollout an (OPNsense installiert immer die neueste). Ein Host ohne Update wird übersprungen, ein fehlgeschlagener
+  nie von selbst wiederholt. Angehalten wird auch, wenn die verantwortliche Person das Recht zum Ausrollen verliert,
+  eine fremde Wartung im Cluster länger als 15 Minuten offen ist, der MASTER ohne Übergabe dran wäre oder ein Lauf
+  älter als 14 Tage ist. Höchstens 10 Wellen und 100 Hosts je Plan.
+- **Wiederkehrende Wartungsfenster:** wöchentlich oder monatlich (n-ter Wochentag, letzter, fester Tag) in einer
+  Zeitzone, mit Sommerzeit-Regel, Abschneidezeit („neue Hosts nur in den ersten N Minuten“) und einzelnen Terminen
+  zum Auslassen oder Verschieben. Ein Plan mit Fenster startet neue Hosts nur darin; mit **automatischem Start**
+  (nur stabile Updates) beginnt er zu jedem Termin von selbst, aber nur, solange Plan, Gruppen und Fenster seit
+  der Aktivierung unverändert sind. Jede Änderung deaktiviert ihn.
+- **Geplante Wartung prüft sich selbst:** Neue Option „Ergebnis automatisch prüfen und abschließen“ (im Dialog
+  vorausgewählt, bei Rollouts immer an). Nach dem Auftrag prüft OPNexus, ob die Firmware wieder bereit ist, der
+  erwartete Neustart stattfand und die Version lesbar ist, und schließt die Wartung dann selbst ab. Bisher blieb
+  sie auf „übergeben“ und sperrte Host und Cluster, bis jemand sie von Hand abschloss.
+- **Benachrichtigungen:** „Rollout angehalten“ und „Rollout nicht automatisch gestartet“ (ein Schalter, standardmäßig an).
+- **Rechte:** Rollout starten/aktivieren/fortsetzen/freigeben/überspringen/abbrechen und Wartungsfenster ändern
+  gelten als Ausrollen (Aufgabentrennung); **Anhalten** darf jeder mit Schreibrecht. Update Center: Hosts im
+  laufenden Rollout sind für manuelle Firmware-Aktionen gesperrt. Ein OPNexus-Update während eines Rollouts ist
+  sicher, wird aber angekündigt.
+- **Getestet:** live auf den Test-Firewalls (OPNsense 26.7): automatischer Start im Fenster, Canary-Update
+  26.7.1_1 → 26.7.7, absichtlich gestoppter Dienst → Gate rot → angehalten (zweiter Ring unberührt,
+  Benachrichtigung), Fortsetzen, Pilot-Update 26.7.6 → 26.7.7 auf dem damaligen MASTER mit CARP-Übergabe und einem
+  Neustart des OPNexus-Backends mitten im Auftrag (genau ein Update-Auftrag), Freigabe, Host ohne Update
+  übersprungen, Anhalten/Fortsetzen/Abbrechen von Hand, Termin auslassen. **Nur gemockt:** Major-Upgrade als
+  Rollout, ein Cluster in *einer* Welle, Cluster-Prüfung vor dem zweiten Knoten, abweichende Zielversion,
+  Zeitüberschreitungen, Rechteverlust, Fenster schließt mitten im Lauf; die Sommerzeit-Regeln sind Rechentests.
+
+## [1.24.0] "Merlin" — 2026-10-21
+
+- **Alarme für Gateways, Ressourcen und Dienste:** Neue Ereignisse in Einstellungen › Benachrichtigungen.
+  **Gateway gestört** (standardmäßig **an**) meldet ein Gateway, das OPNsense als „down“ führt oder dessen Paketverlust (ab 20 %) bzw.
+  Latenz (ab 500 ms) über der Schwelle liegt; bewusst abgeschaltete und nicht überwachte Gateways bleiben still. **Ressourcen-Schwelle**
+  (standardmäßig aus) prüft Festplatte (ab 90 %), Arbeitsspeicher (ab 95 %), Swap (ab 50 %) und CPU-Last (ab 2 pro Thread), **Dienst ausgefallen**
+  (standardmäßig aus) eine einstellbare Liste (Unbound, Dnsmasq, DHCP, NTP, OpenVPN, WireGuard, IPsec …) und meldet nur Dienste, die zuvor
+  liefen. Alle drei stützen sich auf die Minutenmessung und lösen erst aus, wenn die letzten 3 (Ressourcen: 5) Messungen übereinstimmen, und
+  enden erst, wenn alle wieder in Ordnung sind: Einzelspitzen und flackernde Verbindungen schicken nichts. Der Gateway-Status (Verlust, Latenz,
+  Abweichung) steht außerdem auf der Host-Seite, als Kachel auf dem Dashboard und als Prometheus-Metriken (`opnexus_host_gateway_*`,
+  `opnexus_host_swap_used_percent`, `opnexus_host_service_running`). Die Messung speichert dafür zusätzlich Swap, CPU-Threads, Gateways
+  und Dienste in der vorhandenen Messreihe (keine neue Tabelle).
+- **Weitere Benachrichtigungskanäle:** ntfy, Gotify, Telegram, Microsoft Teams (Workflows-/Incoming-Webhook) und Matrix, beliebig viele
+  je Typ, mit „Testen“-Knopf pro Kanal. Zugangsdaten (Tokens, die Teams-Adresse) liegen wie das SMTP-Passwort in der Geheimnis-Ablage, nie
+  in der Datenbank oder in Antworten, und tauchen nicht in Fehlermeldungen oder Logs auf. Webhook und neue Kanäle prüfen die Zieladresse:
+  nur http(s), keine Zugangsdaten in der Adresse, keine lokalen/Link-Local-/Metadaten-Adressen; Weiterleitungen werden nicht verfolgt.
+  Betrifft die bestehende Webhook-Adresse: `localhost`, Loopback, 169.254.x.x und Zugangsdaten in der Adresse (`https://user:pass@…`)
+  werden jetzt abgelehnt. Eine solche gespeicherte Adresse wird nicht mehr beliefert (nur ein Log-Eintrag), und die
+  Benachrichtigungs-Einstellungen lassen sich erst wieder speichern, wenn sie korrigiert oder geleert ist (die Meldung nennt den Grund).
+  Neue Einstellungen kommen mit Migration 0020.
+
+- **Bestand übernehmen für Aliase und NAT:** Auf den Seiten Aliase und NAT (Port Forward, 1:1, Outbound) gibt es jetzt **Von Firewall übernehmen**
+  wie bei den Regeln. Der Dialog zeigt, was von Hand in OPNsense angelegt wurde, einzeln zum Übernehmen oder mit „Alle übernehmbaren“ in einem Schritt;
+  verschachtelte Aliase werden in der richtigen Reihenfolge übernommen. Die Übernahme schreibt nichts auf die Firewall: Die Vorschau zeigt direkt
+  danach „keine Änderung“, der nächste Rollout aktualisiert dasselbe Objekt (kein Duplikat). Nicht übernommen werden OPNsense-eigene Aliase,
+  Typen, die OPNexus nicht kennt (z. B. URL, GeoIP), Aliase mit Verweis auf nicht übernommene Aliase sowie NAT-Regeln, die das Modell verfälschen würden
+  (z. B. „Kein Redirect“, verknüpfte Filterregel, mehrere Schnittstellen, „Kein NAT“); die Oberfläche nennt den Grund je Objekt.
+  Outbound-NAT lässt sich nur übernehmen, wenn der Modus der Firewall Hybrid oder Manuell ist. Einstellungen, die OPNexus nicht verwaltet (Protokollierung,
+  Pool-Optionen, Tags, Kategorien), bleiben unverändert und werden angezeigt.
+
+- **Images für ARM:** Alle Release-Images (backend, frontend, proxy, updater) gibt es jetzt für `linux/amd64` und `linux/arm64`
+  (Raspberry Pi 4/5, ARM-Server, Apple-Silicon-Docker).
+
+## [1.23.11] "Harrier" — 2026-10-20
+
+- **Updates per Klick prüfen jetzt ebenfalls die Signatur:** Der Updater-Container enthält `cosign` (feste Version, Prüfsumme beim Bau
+  geprüft) und verifiziert die neuen Images vor dem Wechsel, wenn `cosign.pub` im Projektverzeichnis liegt. Das Image wird dadurch größer
+  (rund 266 statt 123 MB).
+
+## [1.23.10] "Harrier" — 2026-10-20
+
+- **Signierte Release-Images:** Die Images (backend, frontend, proxy, updater) werden mit cosign signiert, beginnend mit 1.23.9. Der öffentliche
+  Schlüssel liegt als `cosign.pub` im Repository und im Release-Bundle. `opnexus update` prüft die Signaturen vor dem Wechsel, wenn `cosign`
+  installiert ist. Anleitung: Handbuch, „Images prüfen“.
+
+## [1.23.9] "Harrier" — 2026-10-20
+
+- Update-Karte: Der Hinweis für Installationen ohne Release-Images nennt jetzt die tatsächliche Ursache („OPNEXUS_IMAGE_REPO ist nicht gesetzt“)
+  statt „baut aus dem Quellcode“.
+- `cosign.pub`: öffentlicher Schlüssel für die Signaturen der Release-Images (das Release-Skript kann Images jetzt signieren).
+
+## [1.23.8] "Harrier" — 2026-10-20
+
+- **Einstellungen › OPNexus aktualisieren:** Auf einer Installation, die aus dem Quellcode gebaut wird (kein `OPNEXUS_IMAGE_REPO`), erscheint nicht mehr
+  die Anleitung zum Aktivieren des Updaters – Updates per Klick gibt es dort ohnehin nicht –, sondern nur der Hinweis auf `git pull` und
+  `docker compose up -d --build`. Das Backend erkennt den Modus jetzt selbst (neue Umgebungsvariable im Compose-File).
+
 ## [1.23.7] "Harrier" — 2026-10-20
 
 - Hinweistext der Versionsprüfung verweist auf den Schalter „oben“ statt „unten“.

README.de.md

@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
 
 ![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
 
-> **Stand:** `1.23.7` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
+> **Stand:** `1.30.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
 
 ## Funktionen
 
@@ -16,14 +16,17 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
 - **Sichere Änderungen** — jede Änderung läuft *Prüfen → Ausrollen → Bestätigen*; bleibt die Bestätigung im Zeitfenster aus, rollt OPNexus zurück.
 - **Hierarchische Policies** — Pre-/Post-Rulebases mit Vererbung über Gruppenbäume.
 - **VPN** — IPsec-Site-to-Site, OpenVPN und WireGuard, einmal definiert und ausgerollt.
-- **Netzwerk** — Schnittstellen, VLANs, virtuelle IPs, Gateways, Routen, LAGG/Bridge/GRE, DNS und DHCP je Firewall, dazu Tunables, Cron, Syslog, Snapshots und Diagnose (Ping, Traceroute, Port-Prüfung, DNS), ein Abgleich der Cluster-Knoten, mit Diff-Vorschau, Backup vor jeder Änderung und Schutz davor, die eigene Verbindung zu kappen; von Hand angelegte Regeln lassen sich übernehmen.
+- **Netzwerk** — Schnittstellen, VLANs, virtuelle IPs, Gateways, Routen, LAGG/Bridge/GRE, DNS und DHCP je Firewall, dazu Tunables, Cron, Syslog, Snapshots und Diagnose (Ping, Traceroute, Port-Prüfung, DNS), ein Abgleich der Cluster-Knoten, mit Diff-Vorschau, Backup vor jeder Änderung und Schutz davor, die eigene Verbindung zu kappen; von Hand angelegte Regeln, Aliase und NAT-Regeln lassen sich übernehmen.
 - **Zertifikate** — Trust-Store für alle Firewalls und Ausstellung per ACME / Let's Encrypt.
+- **Alarme** — Gateway-/WAN-Ausfall (down, Paketverlust, Latenz), Schwellen für Platte, Speicher, Swap und CPU sowie ausgefallene Dienste; über mehrere Messungen entprellt, damit Einzelspitzen und flackernde Verbindungen schweigen. Zustellung per E-Mail, Webhook, ntfy, Gotify, Telegram, Microsoft Teams oder Matrix, jedes Ereignis einzeln schaltbar.
+- **Firewall-Oberfläche über OPNexus** — die Weboberfläche einer Firewall in einem eigenen Browser-Tab über OPNexus öffnen, auch hinter NAT; nur für Admins, Einmal-Ticket, eigene Origin, Zeitlimits und Audit (standardmäßig aus).
 - **Logs & Audit** — zentrale Log-Suche mit Korrelation, Dashboards und lückenloser Deploy-Verlauf.
-- **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus.
+- **Update-Center und Rollouts** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus; Rollouts in Wellen (Canary → Pilot → Prod) mit wiederkehrenden Wartungsfenstern, Gesundheitsprüfung zwischen den Wellen und automatischem Halt bei jeder Auffälligkeit.
 - **Einrichtungsassistent** — geführter Erststart nach der Installation: eigenes Passwort, Zwei-Faktor und die erste Firewall verbinden.
 - **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
 - **Automatisierung** — persönliche API-Tokens (Lesen / Ausrollen / Admin), Prometheus-Metriken, ein nur lesender MCP-Endpunkt für KI-Assistenten und ein OpenAPI-Schema.
 - **Sicherheitslage** — ein Wert von 0 bis 100 je Firewall aus Härtungs- und Regel-Hygiene-Prüfungen (offene WAN-Regeln, schwaches SSH/SNMP, verdeckte Regeln, alte Backups …), nur lesend.
+- **Regel-Analyse** — „Welche Regel trifft?“ für Quelle, Ziel, Protokoll und Port: die entscheidende Regel je Firewall (oder Flotte/Gruppe) mit ehrlicher Sicherheit (sicher / vermutlich / unbekannt) und Prüfprotokoll Regel für Regel, nur lesend. Eine Bereinigungsliste nennt ungenutzte, verdeckte, doppelte und lange deaktivierte Regeln (jeweils mit Sicherheit), und jede Regel hat einen Änderungsverlauf – auch für Änderungen außerhalb von OPNexus.
 - **Backups & Wiederherstellung** — geplante Konfigurationssicherungen mit Aufbewahrung, verschlüsselte Offsite-Kopie auf S3-kompatiblen Speicher, Änderungsansicht zwischen Sicherungen und eine vollständige verschlüsselte Sicherung von OPNexus selbst (`opnexus backup --full`).
 - **Single Sign-on** — Anmeldung über deinen Identitätsanbieter (OIDC); die Rolle kommt bei jeder Anmeldung aus dessen Gruppen.
 - **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Operator / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
@@ -58,11 +61,11 @@ Nach der ersten Anmeldung öffnet sich von selbst ein kurzer **Einrichtungsassis
 
 Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelieferter Caddy-Proxy) siehe das [Handbuch](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls).
 
-**Fertige Images** (amd64) liegen in der Registry dieses Gitea. In der `.env`:
+**Fertige Images** (amd64 und arm64) liegen in der Registry dieses Gitea. In der `.env`:
 
 ```bash
 OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
-OPNEXUS_VERSION=1.23.7
+OPNEXUS_VERSION=1.30.0
 ```
 
 ```bash

README.md

@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
 
 ![Central firewall rules](docs/screenshots/readme-rules-en.webp)
 
-> **Status:** `1.23.7` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
+> **Status:** `1.30.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
 
 ## What it does
 
@@ -16,14 +16,17 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
 - **Safe changes** — every change goes *preview → deploy → confirm*; without your confirmation inside the time window OPNexus rolls it back.
 - **Hierarchical policies** — pre/post rulebases with inheritance across group trees.
 - **VPN** — IPsec site-to-site, OpenVPN and WireGuard, defined once and rolled out.
-- **Network** — interfaces, VLANs, virtual IPs, gateways, routes, LAGG/bridge/GRE, DNS and DHCP per firewall, plus tunables, cron, syslog, snapshots and diagnostics (ping, traceroute, port check, DNS), a comparison of cluster nodes, with a diff preview, a backup before every change and a guard against cutting your own connection; rules created by hand on a firewall can be adopted.
+- **Network** — interfaces, VLANs, virtual IPs, gateways, routes, LAGG/bridge/GRE, DNS and DHCP per firewall, plus tunables, cron, syslog, snapshots and diagnostics (ping, traceroute, port check, DNS), a comparison of cluster nodes, with a diff preview, a backup before every change and a guard against cutting your own connection; rules, aliases and NAT rules created by hand on a firewall can be adopted.
 - **Certificates** — trust store for all firewalls and ACME / Let's Encrypt issuance.
+- **Alerts** — gateway/WAN outage (down, packet loss, latency), disk/memory/swap/CPU thresholds and stopped services, debounced over several measurements so spikes and flapping links stay quiet; delivered by email, webhook, ntfy, Gotify, Telegram, Microsoft Teams or Matrix, all switchable per event.
+- **Firewall interface through OPNexus** — open a firewall's own web interface in a separate browser tab through OPNexus, even behind NAT; admins only, one-time ticket, isolated origin, time limits and audit (off by default).
 - **Logs & audit** — central log search with correlation, dashboards and a complete deploy audit trail.
-- **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode.
+- **Update center & rollouts** — firmware updates for the fleet, aware of HA order and maintenance mode; rollouts in waves (canary → pilot → prod) with recurring maintenance windows, a health gate between the waves and an automatic stop on anything unusual.
 - **Setup assistant** — a guided first run after installation: your own password, two-factor, and connecting the first firewall.
 - **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
 - **Automation** — personal API tokens (read / deploy / admin), Prometheus metrics, a read-only MCP endpoint for AI assistants and an OpenAPI schema.
 - **Security posture** — a 0–100 score per firewall from hardening and rule-hygiene checks (open WAN rules, weak SSH/SNMP, shadowed rules, stale backups …), read-only.
+- **Rule analysis** — "Which rule matches?" for a source, destination, protocol and port: the deciding rule per firewall (or fleet/group) with an honest confidence (sure / probable / unknown) and a rule-by-rule trace, read-only. A cleanup list names unused, shadowed, duplicate and long-disabled rules (each with its confidence) and every rule has a change history, including changes made outside OPNexus.
 - **Backups & recovery** — scheduled config backups with retention, an encrypted offsite copy to S3-compatible storage, a redacted change view between backups, and a full encrypted backup/restore of OPNexus itself (`opnexus backup --full`).
 - **Single sign-on** — sign in with your identity provider (OIDC); roles come from its groups at every login.
 - **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / operator / read-only), keyboard operation, dark mode, German and English UI.
@@ -60,11 +63,11 @@ After the first sign-in a short **setup assistant** opens on its own: set your o
 
 For access from other machines and for production use with TLS (bundled Caddy proxy), see the [handbook](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls) (German).
 
-**Prebuilt images** (amd64) are published to this Gitea's registry. In `.env`:
+**Prebuilt images** (amd64 and arm64) are published to this Gitea's registry. In `.env`:
 
 ```bash
 OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
-OPNEXUS_VERSION=1.23.7
+OPNEXUS_VERSION=1.30.0
 ```
 
 ```bash

backend/app/auth_middleware.py

@@ -7,7 +7,7 @@ from starlette.responses import JSONResponse
 
 from app.database import SessionLocal
 from app.errors import ErrorCode
-from app.routers.auth import COOKIE_NAME
+from app.routers.auth import session_token
 from app.routers import governance
 from app.routers.auth import client_ip
 from app.services import api_token_service, auth_service
@@ -56,9 +56,9 @@ _READ_ONLY_ALLOWED_WRITES = {
 _SECURITY_KINDS = {
     "ipsec": "ipsec_tunnel", "openvpn": "openvpn_instance", "wireguard": "wireguard_tunnel",
     "certificates": "certificate", "acme": "acme", "notifications": "notification", "auth": "auth",
-    "tokens": "api_token",
+    "tokens": "api_token", "rollouts": "rollout",
 }
-_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings"}
+_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings", "plans", "runs"}
 
 
 def _is_uuid(value: str) -> bool:
@@ -94,10 +94,14 @@ class AuthMiddleware(BaseHTTPMiddleware):
     async def dispatch(self, request: Request, call_next):
         if request.url.path in _PUBLIC_PATHS:
             return await call_next(request)
+        if request.url.path.startswith("/internal/gui/"):
+            # GUI gateway only: never routed by the proxy, authenticated by the
+            # shared gateway token inside routers/gui_access.py.
+            return await call_next(request)
 
         api_token = None
         bearer = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
-        token = request.cookies.get(COOKIE_NAME, "")
+        token = session_token(request)
         if bearer is not None:
             # An API token wins over a cookie; a bad one never falls back to it.
             async with SessionLocal() as db:
@@ -117,7 +121,7 @@ class AuthMiddleware(BaseHTTPMiddleware):
                 )
             if user is not None:
                 parts = request.url.path.strip("/").split("/")
-                kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system", "backups": "backup"}
+                kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system", "backups": "backup", "maintenance-windows": "maintenance_window"}
                 writing = request.method in ("POST", "PUT", "PATCH", "DELETE")
                 if len(parts) > 1 and parts[1] in _SECURITY_KINDS:
                     event = {**_security_event(parts, request.method), "details": {"method": request.method, "path": request.url.path}}

backend/app/config.py

@@ -6,6 +6,17 @@ class Settings(BaseSettings):
     def allowed_origins(self) -> list[str]:
         return [o.strip().rstrip("/") for o in self.frontend_origin.split(",") if o.strip()]
 
+    @property
+    def gui_site_origins(self) -> list[str]:
+        return [o.strip().rstrip("/") for o in self.opnexus_gui_sites.split(",") if o.strip()]
+
+    def gui_origin_conflicts(self) -> list[str]:
+        """Slot origins of the GUI gateway that are also listed in FRONTEND_ORIGIN.
+        Such an origin would get CORS with credentials and pass the Sec-Fetch-Site
+        exception — a firewall GUI could then read OPNexus' API. Startup is refused."""
+        allowed = {o.lower() for o in self.allowed_origins}
+        return [o for o in self.gui_site_origins if o.lower() in allowed]
+
     database_url: str = (
         "postgresql+asyncpg://opnexus:opnexus@postgres:5432/opnexus"
     )
@@ -27,6 +38,9 @@ class Settings(BaseSettings):
     # keine Instanz- oder Nutzerdaten.
     update_check_url: str = ""
     update_check_interval_hours: int = 12
+    # Registry der Release-Images (OPNEXUS_IMAGE_REPO). Leer = OPNexus wird lokal aus dem
+    # Quellcode gebaut; dann gibt es keine Updates per Klick (nur der Hinweis in den Einstellungen).
+    opnexus_image_repo: str = ""
 
     # ── Updates aus dem Webinterface (app/services/updater_client.py) ─────
     # Optionaler Updater-Container (Compose-Profil "updater"). Das Backend hat
@@ -89,6 +103,14 @@ class Settings(BaseSettings):
     # per-IP lockout entirely.
     trust_proxy_headers: bool = False
 
+    # ── GUI gateway (1.27, optional container "gui-gateway") ─────────
+    # Slot origins the gateway is served on, comma-separated, e.g.
+    # "https://192.0.2.10:9441,https://192.0.2.10:9442". Empty = feature unavailable.
+    opnexus_gui_sites: str = ""
+    # Shared secret between backend and gateway for /internal/gui/* (at least
+    # 32 characters). Empty = the internal endpoints answer 404.
+    gui_gateway_token: str = ""
+
     # ── Deploy lockout protection ────────────────────────────────────
     deploy_confirm_window_seconds: int = 180
 
@@ -138,6 +160,13 @@ class Settings(BaseSettings):
     # wireguard_key_store.py.
     wireguard_key_secret_dir: str = "/data/wireguard-key-secrets"
 
+    # ── Rule observation (usage counters + change history of the live rules) ─
+    # Two read-only GETs per enabled firewall per cycle (searchRule, ruleStats). The interval is an operator
+    # setting (env RULE_OBSERVE_INTERVAL_SECONDS); it can not go below the floor, which keeps a typo from
+    # polling every firewall in a tight loop.
+    rule_observe_interval_seconds: int = 900
+    rule_observe_min_interval_seconds: int = 60
+
     # ── Resource monitoring ──────────────────────────────────────────────
     # Fixed interval, no admin-editable settings row/UI for this — v1
     # deliberately kept minimal (see resource_monitor_service.py).

backend/app/errors.py

@@ -87,6 +87,15 @@ class ErrorCode(str, Enum):
     MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN = "MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN"
     NICHT_AUTHENTIFIZIERT = "NICHT_AUTHENTIFIZIERT"
     CROSS_ORIGIN_BLOCKED = "CROSS_ORIGIN_BLOCKED"
+    CROSS_SITE_REQUEST_BLOCKED = "CROSS_SITE_REQUEST_BLOCKED"
+    GUI_SESSION_REQUIRED = "GUI_SESSION_REQUIRED"
+    GUI_GATEWAY_NOT_CONFIGURED = "GUI_GATEWAY_NOT_CONFIGURED"
+    GUI_REQUIRES_HTTPS = "GUI_REQUIRES_HTTPS"
+    GUI_ACCESS_DISABLED = "GUI_ACCESS_DISABLED"
+    GUI_HOST_NOT_ELIGIBLE = "GUI_HOST_NOT_ELIGIBLE"
+    GUI_NO_FREE_SLOT = "GUI_NO_FREE_SLOT"
+    GUI_SLOT_COOLING_DOWN = "GUI_SLOT_COOLING_DOWN"
+    GUI_TOO_MANY_SESSIONS = "GUI_TOO_MANY_SESSIONS"
     TLS_PROBE_FAILED = "TLS_PROBE_FAILED"
     TLS_PEM_INVALID = "TLS_PEM_INVALID"
     TLS_PIN_MISMATCH = "TLS_PIN_MISMATCH"
@@ -116,6 +125,13 @@ class ErrorCode(str, Enum):
     NETWORK_FIELD_INVALID = "NETWORK_FIELD_INVALID"
     NETWORK_ITEM_NOT_FOUND = "NETWORK_ITEM_NOT_FOUND"
     RULE_NOT_ADOPTABLE = "RULE_NOT_ADOPTABLE"
+    OBJECT_NOT_ADOPTABLE = "OBJECT_NOT_ADOPTABLE"
+    ADOPT_OBJECT_NOT_FOUND = "ADOPT_OBJECT_NOT_FOUND"
+    OUTBOUND_MODE_BLOCKS_ADOPT = "OUTBOUND_MODE_BLOCKS_ADOPT"
+    ADOPT_LINK_TARGET_NOT_FOUND = "ADOPT_LINK_TARGET_NOT_FOUND"
+    ADOPT_LINK_TARGET_CHANGED = "ADOPT_LINK_TARGET_CHANGED"
+    ADOPT_LINK_HOST_HAS_OBJECT = "ADOPT_LINK_HOST_HAS_OBJECT"
+    ADOPT_LINK_BUSY = "ADOPT_LINK_BUSY"
     REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0 = "REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0"
     RESTORE_VORGANG_NICHT_GEFUNDEN = "RESTORE_VORGANG_NICHT_GEFUNDEN"
     ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL = "ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL"
@@ -131,6 +147,10 @@ class ErrorCode(str, Enum):
     UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN = "UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN"
     UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT = "UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT"
     UPSTREAM_REQUEST_FAILED = "UPSTREAM_REQUEST_FAILED"
+    API_RIGHTS_MISSING = "API_RIGHTS_MISSING"
+    GATEWAY_NAME_INVALID = "GATEWAY_NAME_INVALID"
+    GATEWAY_UNKNOWN = "GATEWAY_UNKNOWN"
+    GATEWAY_MUTE_LIMIT = "GATEWAY_MUTE_LIMIT"
     VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN = "VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN"
     VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN = "VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN"
     VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN = "VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN"
@@ -162,6 +182,11 @@ class ErrorCode(str, Enum):
     VARIABLES_MISSING = "VARIABLES_MISSING"
     VARIABLE_INVALID = "VARIABLE_INVALID"
     OVERLAY_INVALID = "OVERLAY_INVALID"
+    NOTIFICATION_CHANNEL_INVALID = "NOTIFICATION_CHANNEL_INVALID"
+    NOTIFICATION_CHANNEL_NOT_FOUND = "NOTIFICATION_CHANNEL_NOT_FOUND"
+    NOTIFICATION_CHANNEL_NAME_TAKEN = "NOTIFICATION_CHANNEL_NAME_TAKEN"
+    NOTIFICATION_CHANNEL_TEST_FAILED = "NOTIFICATION_CHANNEL_TEST_FAILED"
+    NOTIFICATION_WEBHOOK_URL_INVALID = "NOTIFICATION_WEBHOOK_URL_INVALID"
     OVERLAY_SUBNET_LOCKED = "OVERLAY_SUBNET_LOCKED"
     OVERLAY_NAME_TAKEN = "OVERLAY_NAME_TAKEN"
     OVERLAY_PEER_NOT_FOUND = "OVERLAY_PEER_NOT_FOUND"
@@ -171,6 +196,32 @@ class ErrorCode(str, Enum):
     API_TOKEN_NOT_FOUND = "API_TOKEN_NOT_FOUND"
     API_TOKEN_SCOPE_FORBIDDEN = "API_TOKEN_SCOPE_FORBIDDEN"
     API_TOKEN_SCOPE_TOO_HIGH = "API_TOKEN_SCOPE_TOO_HIGH"
+    WINDOW_NOT_FOUND = "WINDOW_NOT_FOUND"
+    WINDOW_INVALID = "WINDOW_INVALID"
+    WINDOW_NAME_TAKEN = "WINDOW_NAME_TAKEN"
+    WINDOW_IN_USE = "WINDOW_IN_USE"
+    WINDOW_EXCEPTION_INVALID = "WINDOW_EXCEPTION_INVALID"
+    WINDOW_EXCEPTION_EXISTS = "WINDOW_EXCEPTION_EXISTS"
+    WINDOW_EXCEPTION_NOT_FOUND = "WINDOW_EXCEPTION_NOT_FOUND"
+    ROLLOUT_PLAN_NOT_FOUND = "ROLLOUT_PLAN_NOT_FOUND"
+    ROLLOUT_INVALID = "ROLLOUT_INVALID"
+    ROLLOUT_NAME_TAKEN = "ROLLOUT_NAME_TAKEN"
+    ROLLOUT_PLAN_INVALID = "ROLLOUT_PLAN_INVALID"
+    ROLLOUT_PREVIEW_STALE = "ROLLOUT_PREVIEW_STALE"
+    ROLLOUT_PLAN_BUSY = "ROLLOUT_PLAN_BUSY"
+    ROLLOUT_GROUP_IN_USE = "ROLLOUT_GROUP_IN_USE"
+    ROLLOUT_RUN_NOT_FOUND = "ROLLOUT_RUN_NOT_FOUND"
+    ROLLOUT_ALREADY_ACTIVE = "ROLLOUT_ALREADY_ACTIVE"
+    ROLLOUT_CONFIRM_NAME = "ROLLOUT_CONFIRM_NAME"
+    ROLLOUT_ACTION_NOT_ALLOWED = "ROLLOUT_ACTION_NOT_ALLOWED"
+    ROLLOUT_HOST_NOT_IN_RUN = "ROLLOUT_HOST_NOT_IN_RUN"
+    ROLLOUT_HOST_RUNNING = "ROLLOUT_HOST_RUNNING"
+    ROLLOUT_HOST_FAILED = "ROLLOUT_HOST_FAILED"
+    ANALYSE_ADRESSE_UNGUELTIG = "ANALYSE_ADRESSE_UNGUELTIG"
+    ANALYSE_FAMILIEN_MISCHUNG = "ANALYSE_FAMILIEN_MISCHUNG"
+    ANALYSE_INTERFACE_UNBEKANNT = "ANALYSE_INTERFACE_UNBEKANNT"
+    ANALYSE_PARAMETER_UNGUELTIG = "ANALYSE_PARAMETER_UNGUELTIG"
+    ANALYSE_FIREWALL_NICHT_LESBAR = "ANALYSE_FIREWALL_NICHT_LESBAR"
 
 
 def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:

backend/app/gui_gateway.py

@@ -0,0 +1,609 @@
+"""The ``gui-gateway`` container: firewall web interfaces in their own browser origin (1.27).
+
+Runs from the backend image (``python -m app.gui_gateway``) behind the proxy, which terminates TLS for the
+slot ports and sets ``X-Onx-Slot`` (overwriting any client value). It never touches the database or the
+secret stores; it only talks to the backend's internal API (``/internal/gui/*``, shared GUI_GATEWAY_TOKEN)
+and, per gateway session, to exactly one firewall.
+
+Security model (see the 1.27 design):
+* Every slot port is its own origin. The firewall GUI therefore never runs in the OPNexus origin.
+* Cookies never cross: the browser's ``Cookie`` header is not forwarded (it carries the OPNexus session),
+  and the firewall's ``Set-Cookie`` never reaches the browser — a server-side jar per gateway session
+  (memory only) holds them. The browser only gets ``__Host-onx-gw-<slot>``.
+* Each upstream request is built from scratch: method, path and query from the browser, a small header
+  allow-list, the jar's cookies; target origin, pinned certificate and overlay proxy from the backend.
+  Absolute-form targets, CONNECT/TRACE, ``Upgrade`` and service-worker script requests are refused.
+* ``Sec-Fetch-Site`` must be same-origin/none (same-site only for the redeem step), fail-closed;
+  non-GET/HEAD requests also need the slot's own ``Origin``.
+* ``Location`` pointing anywhere but the firewall is dropped; responses get CSP, X-Frame-Options DENY,
+  ``Cache-Control: no-store``.
+* At most 3 failed GUI logins per firewall in 5 minutes (protects OPNexus' API access from the firewall's
+  login lockout, which would see all attempts coming from OPNexus).
+* Idle and absolute limits; every 20 s the backend re-checks the session (OPNexus session, admin role,
+  switch, host, and that the firewall's address, pinned certificate and overlay route are still the ones the
+  session started with).
+* At most MAX_SESSIONS sessions at once: expired ones are cleared first, a new session is refused otherwise —
+  a running session is never evicted by a newer one. Logs carry method, path (no query), status and duration only.
+
+OPNsense behaviour this relies on (opnsense/core, src/www/authgui.inc, master as of 2026-10-21):
+* session_auth(): a login is ``isset($_POST['login']) && !empty($_POST['usernamefld']) &&
+  !empty($_POST['passwordfld'])`` posted to the current URL (login form has no action). Success sends a
+  ``Location:`` header and exits (302); a failure re-renders the login form (200) with "Wrong username or
+  password." and an auth_log entry.
+* check_security_http_referer_enforcement() only runs when HTTP_REFERER is set, and wants the firewall's
+  own host name/IP — so the gateway sends no Referer (and no Origin) upstream at all.
+* check_security_dns_rebind() accepts an IP address or the configured host name as Host — the gateway
+  sends the Host of the host record's base URL (httpx derives it from the URL).
+"""
+import asyncio
+import hashlib
+import logging
+import os
+import secrets
+import ssl
+import time
+from collections import deque
+from dataclasses import dataclass, field
+from http.cookies import CookieError, SimpleCookie
+from typing import Awaitable, Callable
+from urllib.parse import parse_qs, urlsplit
+
+import httpx
+
+logger = logging.getLogger("gui_gateway")
+# httpx/httpcore log full request URLs (query strings) at INFO/DEBUG: never in this process.
+for _noisy in ("httpx", "httpcore"):
+    logging.getLogger(_noisy).setLevel(logging.WARNING)
+
+LISTEN_PORT = 8080
+COOKIE_PREFIX = "__Host-onx-gw-"
+OPNEXUS_SESSION_COOKIE = "__Host-opnexus-session"
+BODY_LIMIT = 64 * 1024 * 1024  # config restore uploads
+LOGIN_FAILURE_LIMIT = 3
+LOGIN_FAILURE_WINDOW = 300.0
+UPSTREAM_TIMEOUT = 60.0
+MAX_SESSIONS = 64
+
+# Request headers copied from the browser; everything else (Cookie, Authorization, Origin, Referer,
+# X-Forwarded-*, hop-by-hop, Sec-*, ...) is dropped.
+_FORWARD_REQUEST_HEADERS = {
+    "accept", "accept-language", "content-type", "user-agent", "x-requested-with", "x-csrftoken", "range",
+}
+# Response headers copied from the firewall. Set-Cookie goes to the jar, Location is rewritten.
+_FORWARD_RESPONSE_HEADERS = {
+    "content-type", "content-length", "content-encoding", "content-disposition", "content-language",
+    "content-range", "accept-ranges", "content-security-policy",
+}
+_GATEWAY_CSP = ("connect-src 'self'; form-action 'self'; frame-ancestors 'none'; base-uri 'self'; "
+                "object-src 'none'; worker-src 'none'")
+_SECURITY_HEADERS = [
+    (b"content-security-policy", _GATEWAY_CSP.encode()),
+    (b"x-frame-options", b"DENY"),
+    (b"x-content-type-options", b"nosniff"),
+    (b"cache-control", b"no-store"),
+    (b"pragma", b"no-cache"),
+    (b"referrer-policy", b"same-origin"),
+    (b"cross-origin-opener-policy", b"same-origin"),
+    (b"cross-origin-resource-policy", b"same-origin"),
+    (b"permissions-policy", b"camera=(), microphone=(), geolocation=(), payment=(), usb=()"),
+]
+
+
+def _sha(value: str) -> str:
+    return hashlib.sha256(value.encode()).hexdigest()
+
+
+def _origin_of(url: str) -> str | None:
+    try:
+        parts = urlsplit(url)
+        port = parts.port
+    except ValueError:
+        return None
+    if parts.scheme not in ("http", "https") or not parts.hostname:
+        return None
+    default = 443 if parts.scheme == "https" else 80
+    host = parts.hostname.lower()
+    if ":" in host:
+        host = f"[{host}]"
+    return f"{parts.scheme}://{host}" + (f":{port}" if port and port != default else "")
+
+
+def slot_origins_from(sites: str) -> dict[int, set[str]]:
+    """OPNEXUS_GUI_SITES -> {port: {normalised origins}} (https only, explicit port)."""
+    slots: dict[int, set[str]] = {}
+    for raw in sites.split(","):
+        raw = raw.strip().rstrip("/")
+        try:
+            parts = urlsplit(raw)
+            port = parts.port
+        except ValueError:
+            continue
+        if parts.scheme == "https" and parts.hostname and port:
+            slots.setdefault(port, set()).add(_origin_of(raw))
+    return slots
+
+
+@dataclass
+class GatewaySession:
+    gui_session_id: str
+    session_hash: str
+    slot: int
+    upstream_origin: str
+    ca_pem: str | None
+    proxy_url: str | None
+    idle_timeout: float
+    absolute_deadline: float
+    revalidate_every: float
+    target_fp: str | None = None
+    last_activity: float = field(default_factory=time.monotonic)
+    next_check: float = 0.0
+    jar: dict[str, str] = field(default_factory=dict)
+    client: httpx.AsyncClient | None = None
+
+
+class BackendClient:
+    """The backend's internal GUI API. Raises httpx.HTTPError when the backend is unreachable."""
+
+    def __init__(self, base_url: str, token: str, transport: httpx.AsyncBaseTransport | None = None):
+        self._client = httpx.AsyncClient(base_url=base_url, headers={"X-Onx-Gateway-Token": token},
+                                         timeout=10.0, transport=transport)
+
+    async def redeem(self, ticket_hash: str, session_hash: str, slot: int, client_ip: str | None) -> dict:
+        resp = await self._client.post("/internal/gui/redeem", json={
+            "ticket_hash": ticket_hash, "session_hash": session_hash, "slot": slot, "client_ip": client_ip})
+        if resp.status_code == 200:
+            return resp.json()
+        if resp.status_code == 403:
+            return {"ok": False, "reason": (resp.json().get("detail") or {}).get("reason", "denied")}
+        resp.raise_for_status()
+        return {"ok": False, "reason": f"http_{resp.status_code}"}
+
+    async def revalidate(self, gs: GatewaySession) -> dict:
+        resp = await self._client.post("/internal/gui/revalidate",
+                                       json={"gui_session_id": gs.gui_session_id, "session_hash": gs.session_hash,
+                                                    "target_fp": gs.target_fp})
+        resp.raise_for_status()
+        return resp.json()
+
+    async def end(self, gs: GatewaySession, reason: str) -> None:
+        try:
+            await self._client.post("/internal/gui/end", json={
+                "gui_session_id": gs.gui_session_id, "session_hash": gs.session_hash, "reason": reason})
+        except httpx.HTTPError as exc:
+            logger.warning("Could not report the end of a GUI session (%s)", type(exc).__name__)
+
+    async def aclose(self) -> None:
+        await self._client.aclose()
+
+
+def build_upstream_client(gs: GatewaySession) -> httpx.AsyncClient:
+    """Same transport rules as opnsense_client._client: pinned certificate (or system CAs), overlay proxy."""
+    verify: ssl.SSLContext | bool
+    if gs.ca_pem:
+        from app.services.tls_pin_service import pinned_context  # no database layer behind this import
+        verify = pinned_context(gs.ca_pem)
+    else:
+        verify = True
+    return httpx.AsyncClient(verify=verify, proxy=gs.proxy_url, timeout=UPSTREAM_TIMEOUT,
+                             follow_redirects=False, trust_env=False)
+
+
+class LoginLimiter:
+    """At most LOGIN_FAILURE_LIMIT failed GUI logins per firewall within LOGIN_FAILURE_WINDOW seconds."""
+
+    def __init__(self, clock: Callable[[], float] = time.monotonic):
+        self._failures: dict[str, deque] = {}
+        self._clock = clock
+
+    def _recent(self, key: str) -> deque:
+        now = self._clock()
+        q = self._failures.setdefault(key, deque())
+        while q and now - q[0] > LOGIN_FAILURE_WINDOW:
+            q.popleft()
+        return q
+
+    def blocked(self, key: str) -> bool:
+        return len(self._recent(key)) >= LOGIN_FAILURE_LIMIT
+
+    def record_failure(self, key: str) -> None:
+        self._recent(key).append(self._clock())
+
+
+def is_login_attempt(method: str, content_type: str, body: bytes) -> bool:
+    if method != "POST":
+        return False
+    if content_type.lower().startswith("multipart/form-data"):
+        # PHP fills $_POST from multipart bodies too, so authgui.inc accepts a login posted this way.
+        return all(f'name="{field}"'.encode() in body for field in ("login", "usernamefld", "passwordfld"))
+    if not content_type.lower().startswith("application/x-www-form-urlencoded"):
+        return False
+    try:
+        form = parse_qs(body.decode("latin-1"), keep_blank_values=True)
+    except ValueError:
+        return False
+    return "login" in form and bool(form.get("usernamefld", [""])[0]) and bool(form.get("passwordfld", [""])[0])
+
+
+def rewrite_location(location: str, upstream_origin: str) -> str | None:
+    """Relative target for the browser, or None when the redirect leaves the firewall."""
+    location = location.strip()
+    if not location or "\\" in location or any(ord(c) < 0x20 for c in location):
+        return None
+    if location.startswith("//"):
+        location = "https:" + location
+    parts = urlsplit(location)
+    if parts.scheme or parts.netloc:
+        if _origin_of(location) != _origin_of(upstream_origin):
+            return None
+        path = parts.path or "/"
+        return path + (f"?{parts.query}" if parts.query else "") + (f"#{parts.fragment}" if parts.fragment else "")
+    return location
+
+
+def _page(title: str, text_de: str, text_en: str) -> bytes:
+    return (f"<!doctype html><html lang=\"de\"><head><meta charset=\"utf-8\"><title>{title}</title></head>"
+            f"<body style=\"font-family:sans-serif;max-width:40rem;margin:3rem auto\"><h1>{title}</h1>"
+            f"<p>{text_de}</p><p lang=\"en\">{text_en}</p></body></html>").encode()
+
+
+_SESSION_GONE = _page("OPNexus", "Diese Sitzung ist abgelaufen oder ungültig. Öffne die Oberfläche erneut aus OPNexus.",
+                      "This session has expired or is invalid. Open the interface again from OPNexus.")
+_FORBIDDEN = _page("OPNexus", "Diese Anfrage wurde vom GUI-Gateway abgelehnt.", "This request was refused by the GUI gateway.")
+_BAD_REDIRECT = _page("OPNexus", "Die Firewall wollte auf eine fremde Adresse umleiten; die Umleitung wurde verworfen.",
+                      "The firewall tried to redirect to a foreign address; the redirect was dropped.")
+_LOGIN_LOCKED = _page("OPNexus", "Zu viele fehlgeschlagene Anmeldungen an dieser Firewall. Bitte warte 5 Minuten.",
+                      "Too many failed sign-ins at this firewall. Please wait 5 minutes.")
+_UNAVAILABLE = _page("OPNexus", "Die Firewall oder OPNexus ist gerade nicht erreichbar.",
+                     "The firewall or OPNexus is not reachable right now.")
+_TOO_MANY = _page("OPNexus", "Es laufen schon zu viele Firewall-Oberflächen gleichzeitig. Versuche es später erneut.",
+                  "Too many firewall interfaces are open at the same time. Try again later.")
+_REDEEMED = (b"<!doctype html><html><head><meta charset=\"utf-8\"><meta http-equiv=\"refresh\" content=\"0;url=/\">"
+             b"<title>OPNexus</title></head><body><a href=\"/\">&rarr;</a></body></html>")
+
+
+class GuiGateway:
+    def __init__(self, *, backend: BackendClient, slot_origins: dict[int, set[str]],
+                 upstream_factory: Callable[[GatewaySession], httpx.AsyncClient] = build_upstream_client,
+                 clock: Callable[[], float] = time.monotonic, wall_clock: Callable[[], float] = time.time):
+        self.backend = backend
+        self.slot_origins = slot_origins
+        self.upstream_factory = upstream_factory
+        self.clock = clock
+        self.wall_clock = wall_clock
+        self.sessions: dict[str, GatewaySession] = {}  # sha256(gateway cookie) -> session
+        self.limiter = LoginLimiter(clock)
+        self._loop_task: asyncio.Task | None = None
+
+    # ── ASGI ──────────────────────────────────────────────────────────────
+    async def __call__(self, scope, receive, send):
+        if scope["type"] == "lifespan":
+            await self._lifespan(receive, send)
+            return
+        if scope["type"] != "http":
+            # WebSocket and anything else: not supported (design: no WebSocket).
+            if scope["type"] == "websocket":
+                await send({"type": "websocket.close", "code": 1008})
+            return
+        started = time.monotonic()
+        status = await self._handle(scope, receive, send)
+        # Method, path and status only: never the query string, headers or bodies.
+        logger.info("%s %s %d %.0fms", scope.get("method", "?"), scope.get("path", "?")[:200], status,
+                    (time.monotonic() - started) * 1000)
+
+    async def _lifespan(self, receive, send):
+        while True:
+            message = await receive()
+            if message["type"] == "lifespan.startup":
+                self._loop_task = asyncio.create_task(self._revalidate_loop())
+                await send({"type": "lifespan.startup.complete"})
+            elif message["type"] == "lifespan.shutdown":
+                if self._loop_task:
+                    self._loop_task.cancel()
+                for key in list(self.sessions):
+                    await self._drop(key, "shutdown")
+                await self.backend.aclose()
+                await send({"type": "lifespan.shutdown.complete"})
+                return
+
+    async def _revalidate_loop(self):
+        while True:
+            await asyncio.sleep(10)
+            try:
+                await self.sweep()
+            except Exception:  # never let the loop die
+                logger.exception("GUI session sweep failed")
+
+    async def sweep(self) -> None:
+        """Idle/absolute limits and the periodic backend re-check for every gateway session."""
+        for key, gs in list(self.sessions.items()):
+            reason = self._expired(gs)
+            if reason:
+                await self._drop(key, reason)
+            elif self.clock() >= gs.next_check:
+                await self._recheck(key, gs)
+
+    # ── helpers ───────────────────────────────────────────────────────────
+    @staticmethod
+    def _headers(scope) -> dict[str, str]:
+        result: dict[str, str] = {}
+        for k, v in scope.get("headers", []):
+            name = k.decode("latin-1").lower()
+            value = v.decode("latin-1")
+            if name in result:
+                value = f"{result[name]}{'; ' if name == 'cookie' else ', '}{value}"
+            result[name] = value
+        return result
+
+    @staticmethod
+    def _cookies(header: str) -> dict[str, str]:
+        cookies: dict[str, str] = {}
+        for part in header.split(";"):
+            name, sep, value = part.strip().partition("=")
+            if sep and name and name not in cookies:
+                cookies[name] = value
+        return cookies
+
+    async def _respond(self, send, status: int, body: bytes = b"", headers: list | None = None,
+                       content_type: bytes = b"text/html; charset=utf-8") -> int:
+        out = [(b"content-type", content_type), (b"content-length", str(len(body)).encode())]
+        out += list(headers or []) + _SECURITY_HEADERS
+        await send({"type": "http.response.start", "status": status, "headers": out})
+        await send({"type": "http.response.body", "body": body})
+        return status
+
+    def _expired(self, gs: GatewaySession) -> str | None:
+        if self.wall_clock() >= gs.absolute_deadline:
+            return "absolute"
+        if self.clock() - gs.last_activity >= gs.idle_timeout:
+            return "idle"
+        return None
+
+    async def _drop(self, key: str, reason: str, *, report: bool = True) -> None:
+        gs = self.sessions.pop(key, None)
+        if gs is None:
+            return
+        gs.jar.clear()
+        if gs.client is not None:
+            await gs.client.aclose()
+        if report:
+            await self.backend.end(gs, reason)
+
+    async def _recheck(self, key: str, gs: GatewaySession) -> bool | None:
+        """True = still valid, False = ended (dropped), None = backend unreachable (fail closed, keep for retry)."""
+        try:
+            result = await self.backend.revalidate(gs)
+        except httpx.HTTPError as exc:
+            logger.warning("GUI session re-check failed (%s)", type(exc).__name__)
+            return None
+        if not result.get("ok"):
+            await self._drop(key, str(result.get("reason", "revoked")), report=False)  # backend already ended it
+            return False
+        gs.absolute_deadline = min(gs.absolute_deadline, float(result.get("absolute_deadline", gs.absolute_deadline)))
+        gs.idle_timeout = float(result.get("idle_timeout_seconds", gs.idle_timeout))
+        gs.next_check = self.clock() + gs.revalidate_every
+        return True
+
+    async def _clear_expired(self) -> None:
+        for key, gs in list(self.sessions.items()):
+            reason = self._expired(gs)
+            if reason:
+                await self._drop(key, reason)
+
+    def _has_room(self, browser_hash: str, slot: int) -> bool:
+        """Room for one more session; the one this redeem would replace (same browser session and slot) does not count."""
+        replaced = sum(1 for g in self.sessions.values() if g.slot == slot and g.session_hash == browser_hash)
+        return len(self.sessions) - replaced < MAX_SESSIONS
+
+    # ── request handling ──────────────────────────────────────────────────
+    async def _handle(self, scope, receive, send) -> int:
+        method = scope.get("method", "")
+        path = scope.get("path", "")
+        raw_path = scope.get("raw_path") or path.encode()
+        headers = self._headers(scope)
+
+        if method in ("CONNECT", "TRACE"):
+            return await self._respond(send, 501, _FORBIDDEN)
+        # Origin-form only: "GET http://elsewhere/ HTTP/1.1" (absolute-form) or "//host" never reach upstream.
+        if not path.startswith("/") or path.startswith("//") or not raw_path.startswith(b"/") \
+                or raw_path.startswith(b"//") or "\\" in path:
+            return await self._respond(send, 400, _FORBIDDEN)
+        if "upgrade" in headers or "upgrade" in headers.get("connection", "").lower():
+            return await self._respond(send, 400, _FORBIDDEN)
+        if headers.get("service-worker", "").lower() == "script":
+            return await self._respond(send, 403, _FORBIDDEN)
+
+        try:
+            slot = int(headers.get("x-onx-slot", ""))
+        except ValueError:
+            slot = 0
+        own_origins = self.slot_origins.get(slot)
+        if not own_origins:
+            return await self._respond(send, 400, _FORBIDDEN)  # not coming through the proxy's slot block
+
+        site = headers.get("sec-fetch-site", "").strip().lower()
+        if path == "/__onx/redeem":
+            if method != "GET" or site not in ("same-site", "same-origin", "none"):
+                return await self._respond(send, 403, _FORBIDDEN)
+            return await self._redeem(scope, headers, slot, send)
+        if site not in ("same-origin", "none"):
+            return await self._respond(send, 403, _FORBIDDEN)  # missing header = old/odd client: fail closed
+        if method not in ("GET", "HEAD"):
+            origin = headers.get("origin", "")
+            if _origin_of(origin) not in own_origins:
+                return await self._respond(send, 403, _FORBIDDEN)
+        if path.startswith("/__onx/"):
+            return await self._respond(send, 404, _FORBIDDEN)
+
+        cookies = self._cookies(headers.get("cookie", ""))
+        token = cookies.get(f"{COOKIE_PREFIX}{slot}", "")
+        key = _sha(token) if token else ""
+        gs = self.sessions.get(key) if key else None
+        if gs is None or gs.slot != slot:
+            return await self._respond(send, 401, _SESSION_GONE)
+        reason = self._expired(gs)
+        if reason:
+            await self._drop(key, reason)
+            return await self._respond(send, 401, _SESSION_GONE)
+        if self.clock() >= gs.next_check:
+            state = await self._recheck(key, gs)
+            if state is False:
+                return await self._respond(send, 401, _SESSION_GONE)
+            if state is None:
+                return await self._respond(send, 503, _UNAVAILABLE)
+        gs.last_activity = self.clock()
+        return await self._proxy(scope, receive, send, headers, gs)
+
+    async def _redeem(self, scope, headers, slot: int, send) -> int:
+        query = parse_qs(scope.get("query_string", b"").decode("latin-1"))
+        ticket = (query.get("t") or [""])[0]
+        browser_session = self._cookies(headers.get("cookie", "")).get(OPNEXUS_SESSION_COOKIE, "")
+        if not ticket or len(ticket) > 128 or not browser_session:
+            return await self._respond(send, 403, _SESSION_GONE)
+        client_ip = (headers.get("x-forwarded-for", "").split(",")[0].strip() or None)
+        # Room first, before the ticket is burned: expired sessions go, running ones are never evicted.
+        await self._clear_expired()
+        if not self._has_room(_sha(browser_session), slot):
+            return await self._respond(send, 503, _TOO_MANY)
+        try:
+            result = await self.backend.redeem(_sha(ticket), _sha(browser_session), slot, client_ip and client_ip[:45])
+        except httpx.HTTPError as exc:
+            logger.warning("Ticket redeem failed (%s)", type(exc).__name__)
+            return await self._respond(send, 503, _UNAVAILABLE)
+        if not result.get("ok"):
+            return await self._respond(send, 403, _SESSION_GONE)
+        # One browser session, one gateway session per slot: an older one in this slot is replaced.
+        for old_key, old in list(self.sessions.items()):
+            if old.slot == slot and old.session_hash == _sha(browser_session):
+                await self._drop(old_key, "superseded", report=False)
+        value = secrets.token_urlsafe(32)
+        gs = GatewaySession(
+            gui_session_id=result["gui_session_id"], session_hash=_sha(browser_session), slot=slot,
+            upstream_origin=result["upstream_origin"], ca_pem=result.get("ca_pem"), proxy_url=result.get("proxy_url"),
+            idle_timeout=float(result["idle_timeout_seconds"]), absolute_deadline=float(result["absolute_deadline"]),
+            revalidate_every=float(result.get("revalidate_seconds", 20)), target_fp=result.get("target_fp"),
+        )
+        if len(self.sessions) >= MAX_SESSIONS:  # filled up while the backend was answering
+            await self.backend.end(gs, "revoked")
+            return await self._respond(send, 503, _TOO_MANY)
+        gs.last_activity = self.clock()
+        gs.next_check = gs.last_activity + gs.revalidate_every
+        self.sessions[_sha(value)] = gs
+        cookie = f"{COOKIE_PREFIX}{slot}={value}; Path=/; Secure; HttpOnly; SameSite=Strict"
+        # 200 + meta refresh instead of a 303: the follow-up request then starts in this origin
+        # (Sec-Fetch-Site: same-origin) and the ticket leaves the address bar.
+        return await self._respond(send, 200, _REDEEMED, headers=[
+            (b"set-cookie", cookie.encode()), (b"clear-site-data", b'"cache", "storage"'),
+        ])
+
+    async def _read_body(self, receive) -> bytes | None:
+        chunks, size = [], 0
+        while True:
+            message = await receive()
+            if message["type"] == "http.disconnect":
+                return None
+            chunk = message.get("body", b"")
+            size += len(chunk)
+            if size > BODY_LIMIT:
+                return None
+            chunks.append(chunk)
+            if not message.get("more_body"):
+                return b"".join(chunks)
+
+    def _store_cookies(self, gs: GatewaySession, response: httpx.Response) -> None:
+        for header in response.headers.get_list("set-cookie"):
+            jar = SimpleCookie()
+            try:
+                jar.load(header)
+            except CookieError:
+                continue
+            for name, morsel in jar.items():
+                expired = morsel["max-age"] in ("0",) or morsel.value in ("", "deleted")
+                if not expired and morsel["expires"]:
+                    try:
+                        from email.utils import parsedate_to_datetime
+                        expired = parsedate_to_datetime(morsel["expires"]).timestamp() <= self.wall_clock()
+                    except (TypeError, ValueError):
+                        pass
+                if expired:
+                    gs.jar.pop(name, None)
+                else:
+                    gs.jar[name] = morsel.value
+
+    async def _proxy(self, scope, receive, send, headers: dict, gs: GatewaySession) -> int:
+        length = headers.get("content-length", "")
+        if length.isdigit() and int(length) > BODY_LIMIT:
+            return await self._respond(send, 413, _FORBIDDEN)
+        body = await self._read_body(receive)
+        if body is None:
+            return await self._respond(send, 413, _FORBIDDEN)
+        method = scope["method"]
+        login = is_login_attempt(method, headers.get("content-type", ""), body)
+        if login and self.limiter.blocked(gs.upstream_origin):
+            return await self._respond(send, 429, _LOGIN_LOCKED)
+
+        out_headers = {k: v for k, v in headers.items() if k in _FORWARD_REQUEST_HEADERS}
+        out_headers["accept-encoding"] = "identity"
+        if gs.jar:
+            out_headers["cookie"] = "; ".join(f"{k}={v}" for k, v in gs.jar.items())
+        query = scope.get("query_string", b"").decode("latin-1")
+        # The path exactly as the browser sent it (still percent-encoded): the decoded scope["path"] would
+        # turn %3F/%23 into query/fragment, %2F into a separator and decode %25xx a second time upstream.
+        raw_path = (scope.get("raw_path") or scope["path"].encode()).split(b"?", 1)[0].decode("latin-1")
+        url = gs.upstream_origin + raw_path + (f"?{query}" if query else "")
+        if gs.client is None:
+            gs.client = self.upstream_factory(gs)
+        try:
+            request = gs.client.build_request(method, url, headers=out_headers, content=body)
+        except httpx.InvalidURL:
+            return await self._respond(send, 400, _FORBIDDEN)  # e.g. control characters in the path
+        try:
+            response = await gs.client.send(request, stream=True)
+        except (httpx.HTTPError, ssl.SSLError, OSError) as exc:
+            logger.warning("Upstream request failed (%s)", type(exc).__name__)
+            return await self._respond(send, 502, _UNAVAILABLE)
+        try:
+            self._store_cookies(gs, response)
+            if login and not (300 <= response.status_code < 400):
+                self.limiter.record_failure(gs.upstream_origin)
+            out = []
+            for name, value in response.headers.multi_items():
+                lname = name.lower()
+                if lname in _FORWARD_RESPONSE_HEADERS:
+                    out.append((lname.encode("latin-1"), value.encode("latin-1")))
+            if "location" in response.headers:
+                target = rewrite_location(response.headers["location"], gs.upstream_origin)
+                if target is None:
+                    await response.aclose()
+                    return await self._respond(send, 502, _BAD_REDIRECT)
+                out.append((b"location", target.encode("latin-1")))
+            await send({"type": "http.response.start", "status": response.status_code, "headers": out + _SECURITY_HEADERS})
+            async for chunk in response.aiter_raw():
+                await send({"type": "http.response.body", "body": chunk, "more_body": True})
+            await send({"type": "http.response.body", "body": b""})
+            return response.status_code
+        finally:
+            await response.aclose()
+
+
+def create_app(transport: httpx.AsyncBaseTransport | None = None) -> GuiGateway:
+    token = os.environ.get("GUI_GATEWAY_TOKEN", "")
+    if len(token) < 32:
+        raise SystemExit("GUI_GATEWAY_TOKEN (at least 32 characters) is required")
+    slots = slot_origins_from(os.environ.get("OPNEXUS_GUI_SITES", ""))
+    if not slots:
+        raise SystemExit("OPNEXUS_GUI_SITES must list the https slot origins")
+    backend = BackendClient(os.environ.get("GUI_BACKEND_URL", "http://backend:8000"), token, transport)
+    return GuiGateway(backend=backend, slot_origins=slots)
+
+
+def main() -> None:
+    import uvicorn
+    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
+    # h11: an absolute-form target stays visible in scope["path"] (and is refused); access_log off because
+    # uvicorn's access log would contain query strings (the redeem ticket).
+    uvicorn.run(create_app(), host="0.0.0.0", port=LISTEN_PORT, http="h11", ws="none", access_log=False,
+                server_header=False, date_header=False, proxy_headers=False, lifespan="on", log_config=None)
+
+
+if __name__ == "__main__":
+    main()

backend/app/main.py

@@ -46,15 +46,22 @@ from app.version import APP_VERSION
 from app.routers import dashboard as dashboard_router
 from app.routers import cluster as cluster_router
 from app.routers import services as services_router
+from app.routers import gateways as gateways_router
 from app.routers import resource_samples as resource_samples_router
 from app.services import resource_monitor_service
 from app.routers import scheduled_maintenance as scheduled_maintenance_router
+from app.routers import maintenance_windows as maintenance_windows_router
+from app.routers import rollouts as rollouts_router
+from app.routers import rule_analysis as rule_analysis_router
+from app.routers import gui_access as gui_access_router
 from app.services import scheduled_maintenance_service
+from app.services import rollout_engine
 from app.services import compliance_service
 from app.services import audit_export_service
 from app.services import overlay_service, variables_service
 from app.services import backup_schedule_service
 from app.services import carp_event_service, carp_maintenance_window_service
+from app.services import rule_history_service
 
 # Without this, Python's root logger defaults to WARNING with no handler,
 # so every logger.info() call in the 5 background loops below (CARP role
@@ -228,6 +235,9 @@ async def _resource_poll_loop():
                     deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
                     if deleted:
                         logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
+                    pruned = await rule_history_service.prune(db)
+                    if pruned:
+                        logger.info("Regelverlauf-Retention: %d alte Einträge gelöscht", pruned)
         except Exception:
             logger.exception("Resource-Poll fehlgeschlagen")
         await asyncio.sleep(settings.resource_poll_interval_seconds)
@@ -254,6 +264,12 @@ async def _scheduled_maintenance_poll_loop():
         except Exception:
             logger.exception("Geplante-Wartung-Poll fehlgeschlagen")
             in_progress = 0
+        # Firmware rollouts (1.25) only ever create schedules; the next tick of the lines above fires them.
+        try:
+            async with SessionLocal() as db:
+                in_progress += await rollout_engine.advance_runs(db)
+        except Exception:
+            logger.exception("Rollout-Poll fehlgeschlagen")
         # A handover run is stepped quickly (the partner should take over within a minute or two);
         # otherwise the relaxed cadence is plenty.
         await asyncio.sleep(_RUNNING_MAINTENANCE_POLL_SECONDS if in_progress else _SCHEDULED_MAINTENANCE_POLL_SECONDS)
@@ -310,8 +326,31 @@ async def _backup_poll_loop():
         await asyncio.sleep(_BACKUP_POLL_SECONDS)
 
 
+_RULE_OBSERVE_FIRST_RUN_SECONDS = 150
+
+
+async def _rule_observe_loop():
+    """Ninth background loop: reads the rules and their counters of every enabled firewall (two GETs each),
+    keeps the usage sampling current and records what changed since the last look (services/rule_history_service.py).
+    Writes only to the database. The first look at a firewall only stores the starting point."""
+    await asyncio.sleep(_RULE_OBSERVE_FIRST_RUN_SECONDS)
+    while True:
+        try:
+            async with SessionLocal() as db:
+                result = await rule_history_service.observe_all(db)
+            if result["events"] or result["failed"]:
+                logger.info("Regelverlauf: %s", result)
+        except Exception:
+            logger.exception("Regelverlauf fehlgeschlagen")
+        await asyncio.sleep(max(settings.rule_observe_interval_seconds, settings.rule_observe_min_interval_seconds))
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
+    conflicts = settings.gui_origin_conflicts()
+    if conflicts:
+        # A GUI-gateway slot must never be a trusted OPNexus origin (CORS with credentials).
+        raise RuntimeError(f"OPNEXUS_GUI_SITES overlaps FRONTEND_ORIGIN: {', '.join(conflicts)} — refusing to start")
     await migrate.run()
     async with SessionLocal() as db:
         await auth_service.bootstrap_admin(db)
@@ -335,6 +374,7 @@ async def lifespan(app: FastAPI):
     compliance_task = asyncio.create_task(_compliance_poll_loop())
     audit_export_task = asyncio.create_task(_audit_export_poll_loop())
     backup_task = asyncio.create_task(_backup_poll_loop())
+    rule_observe_task = asyncio.create_task(_rule_observe_loop())
     try:
         yield
     finally:
@@ -346,7 +386,8 @@ async def lifespan(app: FastAPI):
         compliance_task.cancel()
         audit_export_task.cancel()
         backup_task.cancel()
-        for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task):
+        rule_observe_task.cancel()
+        for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task, rule_observe_task):
             try:
                 await task
             except asyncio.CancelledError:
@@ -510,9 +551,15 @@ app.include_router(restore_router.router, prefix="/api")
 app.include_router(dashboard_router.router, prefix="/api")
 app.include_router(cluster_router.router, prefix="/api")
 app.include_router(services_router.router, prefix="/api")
+app.include_router(gateways_router.router, prefix="/api")
 app.include_router(network_router.router, prefix="/api")
 app.include_router(snapshots_router.router, prefix="/api")
 app.include_router(network_drift_router.router, prefix="/api")
 app.include_router(diagnostics_router.router, prefix="/api")
 app.include_router(resource_samples_router.router, prefix="/api")
 app.include_router(scheduled_maintenance_router.router, prefix="/api")
+app.include_router(maintenance_windows_router.router, prefix="/api")
+app.include_router(rollouts_router.router, prefix="/api")
+app.include_router(rule_analysis_router.router, prefix="/api")
+app.include_router(gui_access_router.router, prefix="/api")
+app.include_router(gui_access_router.internal_router)

backend/app/models/__init__.py

@@ -93,3 +93,15 @@ __all__ += ["HostResourceSample"]
 
 from app.models.scheduled_maintenance import ScheduledMaintenance
 __all__ += ["ScheduledMaintenance"]
+
+from app.models.maintenance_window import MaintenanceWindow, MaintenanceWindowException
+__all__ += ["MaintenanceWindow", "MaintenanceWindowException"]
+
+from app.models.rollout import RolloutPlan, RolloutPlanWave, RolloutRun, RolloutRunHost, RolloutRunWave, rollout_wave_groups
+__all__ += ["RolloutPlan", "RolloutPlanWave", "RolloutRun", "RolloutRunHost", "RolloutRunWave", "rollout_wave_groups"]
+
+from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
+__all__ += ["GuiAccessSettings", "GuiSession", "GuiTicket"]
+
+from app.models.rule_history import LiveRuleState, RuleChangeEvent
+__all__ += ["LiveRuleState", "RuleChangeEvent"]

backend/app/models/gui_access.py

@@ -0,0 +1,65 @@
+"""GUI access to managed firewalls through the separate gui-gateway container (1.27).
+
+Settings are a singleton row (id fixed at 1), off by default. Tickets and gateway
+sessions only ever store hashes: the raw ticket travels once through the browser,
+the OPNexus session is referenced by the same SHA-256 that `sessions.token_hash` holds.
+"""
+import uuid
+from datetime import datetime
+
+from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String
+from sqlalchemy.dialects.postgresql import UUID as PGUUID
+from sqlalchemy.orm import Mapped, mapped_column
+
+from app.database import Base
+
+
+class GuiAccessSettings(Base):
+    __tablename__ = "gui_access_settings"
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    idle_timeout_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=15, server_default="15")
+    absolute_timeout_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=60, server_default="60")
+    slot_count: Mapped[int] = mapped_column(Integer, nullable=False, default=4, server_default="4")
+    updated_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
+    updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+
+
+class GuiTicket(Base):
+    """One-time, 30-second ticket, bound to the OPNexus session, host and slot."""
+    __tablename__ = "gui_tickets"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    ticket_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True)
+    session_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
+    user_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    host_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    slot: Mapped[int] = mapped_column(Integer, nullable=False)
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    redeemed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+
+
+class GuiSession(Base):
+    """A redeemed ticket: one firewall GUI open in one slot of one browser session."""
+    __tablename__ = "gui_sessions"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    session_hash: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
+    user_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    host_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    slot: Mapped[int] = mapped_column(Integer, nullable=False)
+    client_ip: Mapped[str | None] = mapped_column(String(45), nullable=True)
+    started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    last_checked_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    end_reason: Mapped[str | None] = mapped_column(String(32), nullable=True)

backend/app/models/maintenance_window.py

@@ -0,0 +1,65 @@
+"""Recurring maintenance windows (1.25): "every Sunday 02:00-05:00 Europe/Berlin",
+"first Tuesday of the month 22:00 for 3 h". A window on its own changes nothing on any
+firewall; rollout plans (models/rollout.py) use it to decide WHEN they may start work on a host.
+
+Times are wall-clock times in the window's own IANA time zone, so a window stays at 02:00
+local time across daylight-saving changes (see services/window_service.py for the rules).
+"""
+import uuid
+from datetime import date, datetime, time, timezone
+
+from sqlalchemy import JSON, Boolean, Date, DateTime, ForeignKey, Integer, String, Time, UniqueConstraint
+from sqlalchemy.dialects.postgresql import UUID as PGUUID
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from app.database import Base
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+class MaintenanceWindow(Base):
+    __tablename__ = "maintenance_windows"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    name: Mapped[str] = mapped_column(String(120), unique=True)
+    timezone: Mapped[str] = mapped_column(String(64))
+    recurrence: Mapped[str] = mapped_column(String(16))  # weekly | monthly
+    # weekly: weekdays 0 (Monday) .. 6 (Sunday). JSON columns get a Python default only (Alembic gotcha, see 0007).
+    weekdays: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
+    # monthly: "nth_weekday" (month_nth 1-4 or -1 = last, month_weekday 0-6) or "day" (month_day 1-28)
+    month_mode: Mapped[str | None] = mapped_column(String(16), nullable=True)
+    month_nth: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    month_weekday: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    month_day: Mapped[int | None] = mapped_column(Integer, nullable=True)
+    start_time: Mapped[time] = mapped_column(Time)
+    duration_minutes: Mapped[int] = mapped_column(Integer)
+    # After this many minutes into the window no NEW host action starts (running ones finish).
+    start_cutoff_minutes: Mapped[int] = mapped_column(Integer, nullable=False, default=60, server_default="60")
+    enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
+    created_by: Mapped[str] = mapped_column(String(150))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+
+    exceptions: Mapped[list["MaintenanceWindowException"]] = relationship(
+        "MaintenanceWindowException", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
+        order_by="MaintenanceWindowException.occurrence_date",
+    )
+
+
+class MaintenanceWindowException(Base):
+    """One occurrence skipped or moved. Keyed by the occurrence's LOCAL date."""
+
+    __tablename__ = "maintenance_window_exceptions"
+    __table_args__ = (UniqueConstraint("window_id", "occurrence_date", name="uq_window_exception_date"),)
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    window_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("maintenance_windows.id", ondelete="CASCADE"), index=True
+    )
+    occurrence_date: Mapped[date] = mapped_column(Date)
+    kind: Mapped[str] = mapped_column(String(8))  # skip | move
+    moved_start: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    reason: Mapped[str] = mapped_column(String(300), default="")
+    actor: Mapped[str] = mapped_column(String(150))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)

backend/app/models/notification.py

@@ -6,7 +6,7 @@ OPNsense API secrets stay off the database entirely.
 import uuid
 from datetime import datetime, timezone
 
-from sqlalchemy import JSON, Boolean, DateTime, Integer, String, UniqueConstraint
+from sqlalchemy import JSON, Boolean, DateTime, Float, Integer, String, UniqueConstraint
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 
@@ -17,6 +17,13 @@ def _now() -> datetime:
     return datetime.now(timezone.utc)
 
 
+DEFAULT_SERVICE_WATCHLIST = (
+    "unbound", "dnsmasq", "dhcpd", "dhcpd6", "kea-dhcp4", "kea-dhcp6", "ntpd", "openvpn", "wireguard", "strongswan",
+)
+
+CHANNEL_TYPES = ("ntfy", "gotify", "telegram", "teams", "matrix")
+
+
 class NotificationSettings(Base):
     """Single row, id fixed at 1 — see notification_settings_service.get_settings
     for the get-or-create helper. The only admin-editable operational config
@@ -55,6 +62,34 @@ class NotificationSettings(Base):
     # Off by default: switching it on for an existing fleet reports every long-standing finding at once.
     compliance_alerts_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
 
+    # Monitoring alerts (1.24). The gateway alert is on by default (a WAN outage is the one thing every
+    # operator wants to hear about); resource thresholds and service checks are off, like compliance alerts:
+    # switching them on for an existing fleet may report long-standing conditions all at once.
+    gateway_down_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
+    resource_threshold_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    service_down_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    # An API key that cannot read gateway / service state makes those alerts silently blind (1.29): say so.
+    api_rights_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
+    # Gateways whose alert is switched off, "<host_id>:<gateway name>" (1.29), e.g. an idle second WAN.
+    gateway_alert_muted: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
+    # Rollouts (1.25): a run paused (gate, failure, window...) or an automatic start did not happen.
+    rollout_problems_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
+    # Gateway: OPNsense' own verdict (down / loss / delay) decides "offline"; these two add an
+    # OPNexus threshold on the measured values (0 = off).
+    gateway_loss_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=20, server_default="20")
+    gateway_delay_threshold_ms: Mapped[int] = mapped_column(Integer, nullable=False, default=500, server_default="500")
+    # Resources (0 = that threshold is off). Load is per CPU thread.
+    disk_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=90, server_default="90")
+    memory_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=95, server_default="95")
+    swap_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=50, server_default="50")
+    load_per_cpu_threshold: Mapped[float] = mapped_column(Float, nullable=False, default=2.0, server_default="2.0")
+    # Debouncing: consecutive 60 s resource samples that must agree before a problem opens or closes.
+    alert_confirm_samples: Mapped[int] = mapped_column(Integer, nullable=False, default=3, server_default="3")
+    resource_confirm_samples: Mapped[int] = mapped_column(Integer, nullable=False, default=5, server_default="5")
+    # OPNsense service names that are reported when they stop (the API cannot tell a crash from a stop on purpose).
+    # NULL = the default list (an old row never got one).
+    service_watchlist: Mapped[list | None] = mapped_column(JSON, nullable=True, default=lambda: list(DEFAULT_SERVICE_WATCHLIST))
+
     poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300)
     updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
 
@@ -85,3 +120,21 @@ class NotificationProblem(Base):
     resolved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
     last_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
     resolved_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+
+
+class NotificationChannel(Base):
+    """An additional delivery channel (ntfy, Gotify, Telegram, Microsoft Teams, Matrix) next to the SMTP
+    and generic-webhook settings above. Several channels of one type are allowed. ``config`` holds only
+    non-secret parameters (server URL, topic, chat id, room id); the credential — token, or the whole
+    webhook URL for Teams — lives in notification_secret_store under ``channel_<id>``, never here."""
+
+    __tablename__ = "notification_channels"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    name: Mapped[str] = mapped_column(String(100), unique=True)
+    channel_type: Mapped[str] = mapped_column(String(16))
+    enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
+    config: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
+    secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)

backend/app/models/rollout.py

@@ -0,0 +1,152 @@
+"""Firmware rollouts in waves (1.25): plans (what, in which order, when) and runs (what happened).
+
+A plan's waves point at device groups ("rings": canary -> pilot -> prod). A run freezes the plan
+(``plan_snapshot``) and the resolved hosts (``RolloutRunHost``) when it starts, so later edits of
+the plan or of a group never change a run that is under way. The run never fires firmware actions
+itself: every host step is an ordinary ScheduledMaintenance row (origin="rollout",
+auto_verify=True) linked from ``RolloutRunHost.scheduled_maintenance_id`` — see
+services/rollout_service.py.
+"""
+import uuid
+from datetime import datetime, timezone
+
+from sqlalchemy import (JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Table, Text,
+                        UniqueConstraint)
+from sqlalchemy.dialects.postgresql import UUID as PGUUID
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from app.database import Base
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+rollout_wave_groups = Table(
+    "rollout_wave_groups", Base.metadata,
+    Column("wave_id", PGUUID(as_uuid=True), ForeignKey("rollout_plan_waves.id", ondelete="CASCADE"), primary_key=True),
+    Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
+)
+
+
+class RolloutPlan(Base):
+    __tablename__ = "rollout_plans"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    name: Mapped[str] = mapped_column(String(120), unique=True)
+    description: Mapped[str] = mapped_column(Text, default="")
+    action: Mapped[str] = mapped_column(String(16))  # update | major_update
+    window_id: Mapped[uuid.UUID | None] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("maintenance_windows.id", ondelete="SET NULL"), nullable=True, index=True)
+    # Start a run by itself at every occurrence of the window. Only for "update" and only while the plan
+    # is activated AND unchanged since (activated_fingerprint).
+    auto_start: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    gate_checks: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
+    compliance_mode: Mapped[str] = mapped_column(String(8), nullable=False, default="warn", server_default="warn")
+    activated_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
+    # Stable id of that account (1.29, no foreign key on purpose: a deleted account stays gone, a new one with the same name is someone else).
+    activated_by_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
+    activated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    activated_fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True)
+    # Why the last automatic start did not happen: {"occurrence", "code", "params", "at"} (notification source).
+    auto_start_error: Mapped[dict | None] = mapped_column(JSON, nullable=True)
+    created_by: Mapped[str] = mapped_column(String(150))
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+    updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
+
+    waves: Mapped[list["RolloutPlanWave"]] = relationship(
+        "RolloutPlanWave", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
+        order_by="RolloutPlanWave.position")
+
+
+class RolloutPlanWave(Base):
+    __tablename__ = "rollout_plan_waves"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    plan_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_plans.id", ondelete="CASCADE"), index=True)
+    position: Mapped[int] = mapped_column(Integer)
+    label: Mapped[str] = mapped_column(String(60))
+    soak_minutes: Mapped[int] = mapped_column(Integer, default=30)
+    carp_handover: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    # Wait for a human "release" before this wave starts (not used for the first wave: starting is the release).
+    require_approval: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+
+    groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=rollout_wave_groups, lazy="selectin")  # noqa: F821
+
+
+class RolloutRun(Base):
+    __tablename__ = "rollout_runs"
+    __table_args__ = (UniqueConstraint("plan_id", "window_occurrence", name="uq_rollout_run_occurrence"),)
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    plan_id: Mapped[uuid.UUID | None] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("rollout_plans.id", ondelete="SET NULL"), nullable=True, index=True)
+    plan_name: Mapped[str] = mapped_column(String(120))
+    plan_snapshot: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
+    action: Mapped[str] = mapped_column(String(16))
+    # waiting_window | preflight | wave_running | soaking | gate | awaiting_approval | resuming | paused
+    # | succeeded | cancelled
+    status: Mapped[str] = mapped_column(String(20), index=True)
+    paused_from: Mapped[str | None] = mapped_column(String(20), nullable=True)
+    current_wave: Mapped[int] = mapped_column(Integer, default=0)
+    target_version: Mapped[str | None] = mapped_column(String(40), nullable=True)
+    pause_reason: Mapped[str | None] = mapped_column(String(40), nullable=True)
+    pause_detail: Mapped[dict | None] = mapped_column(JSON, nullable=True)
+    started_by: Mapped[str] = mapped_column(String(150))
+    started_by_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)  # see activated_by_id
+    trigger: Mapped[str] = mapped_column(String(8))  # manual | window
+    window_occurrence: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    # Optimistic lock: every state change bumps it; an engine step computed on an older revision is dropped.
+    revision: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
+    progress: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
+    step_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+    finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+
+    waves: Mapped[list["RolloutRunWave"]] = relationship(
+        "RolloutRunWave", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin", order_by="RolloutRunWave.index")
+    hosts: Mapped[list["RolloutRunHost"]] = relationship(
+        "RolloutRunHost", cascade="all, delete-orphan", passive_deletes=True, lazy="selectin",
+        order_by=lambda: [RolloutRunHost.wave_index, RolloutRunHost.order])
+
+
+class RolloutRunWave(Base):
+    __tablename__ = "rollout_run_waves"
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    run_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_runs.id", ondelete="CASCADE"), index=True)
+    index: Mapped[int] = mapped_column(Integer)
+    label: Mapped[str] = mapped_column(String(60))
+    # pending | awaiting_approval | preflight | running | soaking | gate | passed | skipped | cancelled
+    status: Mapped[str] = mapped_column(String(20), default="pending")
+    baseline: Mapped[dict | None] = mapped_column(JSON, nullable=True)
+    gate_result: Mapped[dict | None] = mapped_column(JSON, nullable=True)
+    soak_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    gate_attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=0, server_default="0")
+    approved_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
+    started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+
+
+class RolloutRunHost(Base):
+    __tablename__ = "rollout_run_hosts"
+    __table_args__ = (UniqueConstraint("run_id", "host_id", name="uq_rollout_run_host"),)
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    run_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("rollout_runs.id", ondelete="CASCADE"), index=True)
+    wave_index: Mapped[int] = mapped_column(Integer)
+    host_id: Mapped[uuid.UUID | None] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="SET NULL"), nullable=True, index=True)
+    host_name: Mapped[str] = mapped_column(String(120))
+    cluster_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
+    order: Mapped[int] = mapped_column(Integer, default=0)
+    # queued | running | verifying | succeeded | skipped | failed | blocked | cancelled
+    status: Mapped[str] = mapped_column(String(16), default="queued")
+    scheduled_maintenance_id: Mapped[uuid.UUID | None] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("scheduled_maintenance.id", ondelete="SET NULL"), nullable=True, index=True)
+    version_before: Mapped[str | None] = mapped_column(String(40), nullable=True)
+    version_after: Mapped[str | None] = mapped_column(String(40), nullable=True)
+    # {"code": ..., "params": {...}} — translated in the UI (rollouts.hostNote.*)
+    detail: Mapped[dict | None] = mapped_column(JSON, nullable=True)
+    started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
+    finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)

backend/app/models/rule_history.py

@@ -0,0 +1,59 @@
+"""Change history of the rules that are live on a firewall (all rules, also hand-made ones).
+
+``live_rule_states`` is the last observed state per rule (a fingerprint plus the row itself), maintained by the
+``rule_observe`` loop; ``rule_change_events`` is what changed between two observations. See
+services/rule_history_service.py. Neither table holds anything the firewall itself would not show on its rule page.
+"""
+import uuid
+from datetime import datetime
+from typing import Optional
+
+from sqlalchemy import JSON, Boolean, DateTime, ForeignKey, Index, Integer, String, UniqueConstraint
+from sqlalchemy.dialects.postgresql import UUID as PGUUID
+from sqlalchemy.orm import Mapped, mapped_column
+
+from app.database import Base
+
+
+class LiveRuleState(Base):
+    __tablename__ = "live_rule_states"
+
+    host_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True)
+    # The rule's UUID/label on the firewall; BASELINE_KEY marks "this host has been observed once".
+    rule_uuid: Mapped[str] = mapped_column(String(64), primary_key=True)
+    fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
+    row: Mapped[dict] = mapped_column(JSON, nullable=False)
+    # False = the rule is gone (kept as a marker so a returning UUID continues the numbering).
+    present: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
+    # Counts the observed changes of this rule; the event of a change gets the new number, which makes the
+    # observation idempotent when two processes see the same change.
+    version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
+    first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    changed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+    last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+
+
+class RuleChangeEvent(Base):
+    __tablename__ = "rule_change_events"
+    __table_args__ = (
+        UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
+        Index("ix_rule_change_events_rule", "host_id", "rule_uuid", "observed_at"),
+    )
+
+    id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    host_id: Mapped[uuid.UUID] = mapped_column(
+        PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False)
+    rule_uuid: Mapped[str] = mapped_column(String(64), nullable=False)
+    seq: Mapped[int] = mapped_column(Integer, nullable=False)
+    observed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, index=True)
+    kind: Mapped[str] = mapped_column(String(16), nullable=False)       # added | changed | removed | enabled | disabled
+    before: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
+    after: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
+    origin: Mapped[str] = mapped_column(String(16), nullable=False)     # opnexus | external
+    actor: Mapped[Optional[str]] = mapped_column(String(150), nullable=True)
+    # The audit entry that explains an OPNexus-made change; no foreign key, the audit trail outlives everything.
+    audit_id: Mapped[Optional[uuid.UUID]] = mapped_column(PGUUID(as_uuid=True), nullable=True)
+    # Start of the time since the previous observation of the host: only audit entries from then on can explain
+    # this change (a later reconcile must not reach back to an older deployment).
+    window_start: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)

backend/app/models/scheduled_maintenance.py

@@ -42,7 +42,12 @@ class ScheduledMaintenance(Base):
     host_operation_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
     # Opt-in: hand the traffic over to the CARP partner first, run the action, then switch back.
     carp_handover: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
-    # Only while status == "running" with a handover: handover_wait | job_wait | leave | leave_wait.
+    # 1.25: check the result automatically (job_wait -> verify) and close the operation, instead of
+    # leaving it "submitted" for a human. Every rollout step sets it; off for existing plans.
+    auto_verify: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
+    # manual | rollout (created by a rollout run, see models/rollout.py RolloutRunHost.scheduled_maintenance_id)
+    origin: Mapped[str] = mapped_column(String(16), nullable=False, default="manual", server_default="manual")
+    # Only while status == "running" with a handover or auto_verify: handover_wait | job_wait | verify | leave | leave_wait.
     # State lives here, not in memory, so a backend restart in the middle of a run resumes cleanly.
     phase: Mapped[str | None] = mapped_column(String(16), nullable=True)
     phase_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)

backend/app/routers/adopt_routes.py

@@ -0,0 +1,79 @@
+"""The three "take over what exists on the firewall" endpoints that aliases, port forwards, 1:1 NAT and
+outbound NAT share (services/object_adopt_service.py has the logic and the evidence):
+
+  GET  <prefix>/unmanaged?host_id=…   candidates found live on the host, with reasons for the refused ones
+  POST <prefix>/adopt                 take over one object {host_id, uuid}; returns the new central object
+  POST <prefix>/adopt-all             take over everything adoptable {host_id}; returns adopted + skipped
+
+Like rule adoption this only creates central records; nothing is written to the firewall."""
+import json
+import uuid
+from typing import Callable
+
+from fastapi import APIRouter, Depends
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.host import OpnsenseHost
+from app.schemas.adopt import AdoptAllRequest, AdoptAllResult, AdoptObjectRequest, UnmanagedObject
+from app.services import object_adopt_service
+from app.services.adopt_common import LINK_ERRORS, link_api_error
+from app.services.opnsense_client import OpnsenseAPIError
+from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
+from app.services.policy_service import policy_lock
+
+
+async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
+    host = await db.get(OpnsenseHost, host_id)
+    if not host:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    return host
+
+
+def add_adopt_routes(router: APIRouter, *, kind: str, model, load_options: Callable, to_read: Callable, read_model) -> None:
+    @router.get("/unmanaged", response_model=list[UnmanagedObject])
+    async def list_unmanaged(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+        host = await _host(db, host_id)
+        try:
+            return await object_adopt_service.list_unmanaged(db, kind, host)
+        except OpnsenseAPIError as exc:
+            raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
+
+    @router.post("/adopt", response_model=read_model, status_code=201)
+    async def adopt(payload: AdoptObjectRequest, db: AsyncSession = Depends(get_db)):
+        await policy_lock(db)
+        host = await _host(db, payload.host_id)
+        try:
+            created = await object_adopt_service.adopt(db, kind, host, payload.uuid, link_to=payload.link_to)
+        except object_adopt_service.NotFoundOnHost as exc:
+            raise api_error(404, ErrorCode.ADOPT_OBJECT_NOT_FOUND) from exc
+        except object_adopt_service.NotAdoptable as exc:
+            raise api_error(409, ErrorCode.OBJECT_NOT_ADOPTABLE, reasons=", ".join(exc.reasons),
+                            reason_details=json.dumps(exc.details, ensure_ascii=False)) from exc
+        except LINK_ERRORS as exc:
+            raise link_api_error(exc) from exc
+        except OpnsenseAPIError as exc:
+            raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
+        await db.commit()
+        # The new object sits in the session without its relationships loaded; db.get() would return it as-is.
+        created = (await db.execute(
+            select(model).where(model.id == created.id).options(*load_options())
+            .execution_options(populate_existing=True)
+        )).scalar_one()
+        await hydrate_hierarchy_targets(db, [created])
+        return to_read(created, created.targets, created.deployments)
+
+    @router.post("/adopt-all", response_model=AdoptAllResult)
+    async def adopt_all(payload: AdoptAllRequest, db: AsyncSession = Depends(get_db)):
+        await policy_lock(db)
+        host = await _host(db, payload.host_id)
+        try:
+            result = await object_adopt_service.adopt_all(db, kind, host, link_existing=payload.link_existing)
+        except LINK_ERRORS as exc:
+            raise link_api_error(exc) from exc
+        except OpnsenseAPIError as exc:
+            raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
+        await db.commit()
+        return result

backend/app/routers/aliases.py

@@ -17,6 +17,7 @@ from app.schemas.alias import DeploymentResult, AliasCreate, AliasRead, AliasEdi
 from app.services.alias_deploy_service import confirm_alias, deploy_alias, undeploy_alias, revert_alias_deployment
 from app.services.opnsense_client import OpnsenseAPIError
 from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
+from app.routers.adopt_routes import add_adopt_routes
 
 router = APIRouter(prefix="/aliases", tags=["aliases"])
 
@@ -70,6 +71,9 @@ def _to_read(alias: FirewallAlias, hosts: list[OpnsenseHost], deployments: list[
     )
 
 
+add_adopt_routes(router, kind="alias", model=FirewallAlias, load_options=_load_options, to_read=_to_read, read_model=AliasRead)
+
+
 @router.get("", response_model=list[AliasRead])
 async def list_aliases(db: AsyncSession = Depends(get_db)):
     result = await db.execute(select(FirewallAlias).options(*_load_options()))

backend/app/routers/auth.py

@@ -15,7 +15,31 @@ logger = logging.getLogger(__name__)
 
 router = APIRouter(prefix="/api/auth", tags=["auth"])
 
+# Plain name: the session cookie of HTTP-only (lab) deployments. On HTTPS deployments it is NOT accepted any
+# more (1.29): it is only still deleted (login, logout) so that a pre-1.27 cookie does not linger.
 COOKIE_NAME = "opnexus-session"
+# HTTPS deployments: the __Host- prefix makes the browser refuse any cookie of
+# this name that is not Secure, has a Domain attribute or a Path other than
+# "/". A page on another port of the same host (the GUI gateway slots, any
+# other service on that machine) therefore cannot toss a narrower-path cookie
+# that shadows the real session, and cannot overwrite it from script (HttpOnly).
+SECURE_COOKIE_NAME = "__Host-opnexus-session"
+
+
+def session_cookie_name() -> str:
+    return SECURE_COOKIE_NAME if settings.cookie_secure else COOKIE_NAME
+
+
+def session_token(request: Request) -> str:
+    """The session token the browser sent: the __Host- cookie on HTTPS deployments, the plain one otherwise.
+    Never the other name (1.29): a plain-name cookie can be planted by any plain-HTTP or sibling-subdomain page and
+    must not be able to carry a session on an HTTPS deployment (the __Host- prefix does not separate PORTS, see
+    BACKLOG S8)."""
+    return request.cookies.get(session_cookie_name(), "")
+
+
+def has_session_cookie(request: Request) -> bool:
+    return bool(request.cookies.get(SECURE_COOKIE_NAME) or request.cookies.get(COOKIE_NAME))
 
 
 def client_ip(request: Request) -> str:
@@ -32,7 +56,7 @@ def client_ip(request: Request) -> str:
 
 def _set_session_cookie(response: Response, token: str, max_age: int | None = None) -> None:
     response.set_cookie(
-        key=COOKIE_NAME,
+        key=session_cookie_name(),
         value=token,
         max_age=max_age or settings.session_lifetime,
         httponly=True,
@@ -40,10 +64,15 @@ def _set_session_cookie(response: Response, token: str, max_age: int | None = No
         secure=settings.cookie_secure,
         path="/",
     )
+    if settings.cookie_secure:
+        # A pre-1.27 plain-name cookie must not linger next to the new one.
+        response.delete_cookie(key=COOKIE_NAME, path="/", secure=True, httponly=True, samesite="lax")
 
 
 def _clear_session_cookie(response: Response) -> None:
-    response.delete_cookie(key=COOKIE_NAME, path="/")
+    response.delete_cookie(key=COOKIE_NAME, path="/", secure=settings.cookie_secure, httponly=True, samesite="lax")
+    if settings.cookie_secure:
+        response.delete_cookie(key=SECURE_COOKIE_NAME, path="/", secure=True, httponly=True, samesite="lax")
 
 
 def _check_account_budget(user: AdminUser, action: str) -> None:
@@ -71,7 +100,7 @@ async def resolve_user(request: Request, db: AsyncSession) -> AdminUser:
         user = await db.get(AdminUser, resolved.id)
         if user is not None:
             return user
-    token = request.cookies.get(COOKIE_NAME, "")
+    token = session_token(request)
     user = await auth_service.resolve_session(db, token)
     if user is None:
         raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.NICHT_AUTHENTIFIZIERT)
@@ -229,7 +258,7 @@ async def totp_enable(
     user.totp_last_step = step
     user.totp_recovery_hashes = hashes
     # Other sessions were established without the second factor — end them.
-    await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
+    await auth_service.revoke_sessions(db, user.id, keep_token=session_token(request), commit=False)
     await db.commit()
     logger.info("Two-factor enabled for '%s'", user.username)
     return {"enabled": True, "recovery_codes": plain}
@@ -254,7 +283,7 @@ async def totp_disable(
     user.totp_enabled = False
     user.totp_recovery_hashes = None
     user.totp_last_step = 0
-    await auth_service.revoke_sessions(db, user.id, keep_token=request.cookies.get(COOKIE_NAME, ""), commit=False)
+    await auth_service.revoke_sessions(db, user.id, keep_token=session_token(request), commit=False)
     await db.commit()
     logger.info("Two-factor disabled for '%s'", user.username)
     return {"enabled": False}
@@ -262,14 +291,17 @@ async def totp_disable(
 
 @router.post("/logout")
 async def logout(request: Request, response: Response, db: AsyncSession = Depends(get_db)):
-    token = request.cookies.get(COOKIE_NAME, "")
+    token = session_token(request)
     await auth_service.revoke_session(db, token)
+    legacy = request.cookies.get(COOKIE_NAME, "")
+    if settings.cookie_secure and legacy and legacy != token:
+        await auth_service.revoke_session(db, legacy)  # an old plain-name cookie ends with the logout too
     _clear_session_cookie(response)
     return {"authenticated": False}
 
 
 @router.get("/me")
-async def me(request: Request, db: AsyncSession = Depends(get_db)):
+async def me(request: Request, response: Response, db: AsyncSession = Depends(get_db)):
     raw = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
     if raw is not None:
         resolved = await api_token_service.resolve(db, raw, client_ip(request))
@@ -278,7 +310,7 @@ async def me(request: Request, db: AsyncSession = Depends(get_db)):
         user, api_token = resolved
         return {"authenticated": True, "username": user.username, "role": user.role,
                 "via": "token", "scope": api_token.scope, "token_name": api_token.name}
-    token = request.cookies.get(COOKIE_NAME, "")
+    token = session_token(request)
     user = await auth_service.resolve_session(db, token)
     if user is None:
         return {"authenticated": False}

backend/app/routers/dashboard.py

@@ -18,6 +18,8 @@ locks are deliberately NOT included here (no fleet-wide list endpoint
 exists for those, only per-host, and it's a rarely-triggered flow) —
 a documented v1 gap, not an oversight.
 """
+from datetime import datetime, timezone
+
 from fastapi import APIRouter, Depends
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -46,7 +48,10 @@ from app.models.rule_deployment import RuleDeployment
 from app.models.wireguard_tunnel import WireguardTunnel
 from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
 from app.schemas.dashboard import ActivityItem, DashboardHost, DashboardSummary, PendingAction
+from app.models.resource_sample import HostResourceSample
+from app.services import gateway_health_service, notification_settings_service
 from app.services.host_status_service import get_statuses
+from app.services.monitor_checks import STALE_AFTER
 
 router = APIRouter(prefix="/dashboard", tags=["dashboard"])
 
@@ -202,14 +207,43 @@ async def collect_pending_actions(db: AsyncSession) -> list[PendingAction]:
     return pending_actions
 
 
+async def _sample_readings_by_host(db: AsyncSession, hosts, muted: set[str]) -> tuple[dict, dict]:
+    """(gateways, missing API rights) from each host's newest resource sample; hosts without a fresh sample are left out."""
+    gateways: dict = {}
+    denied: dict = {}
+    now = datetime.now(timezone.utc)
+    for host in hosts:
+        row = (
+            await db.execute(
+                select(HostResourceSample.sampled_at, HostResourceSample.raw["gateways"].as_json(), HostResourceSample.raw["denied"].as_json())
+                .where(HostResourceSample.host_id == host.id)
+                .order_by(HostResourceSample.sampled_at.desc())
+                .limit(1)
+            )
+        ).first()
+        if row is None or now - row[0] > STALE_AFTER:
+            continue
+        if row[1] is not None:
+            gateways[host.id] = [
+                g.model_copy(update={"alert_enabled": f"{host.id}:{g.name}" not in muted})
+                for g in gateway_health_service.from_compact(row[1])
+            ]
+        if isinstance(row[2], list):
+            denied[host.id] = [str(x) for x in row[2]]
+    return gateways, denied
+
+
 @router.get("", response_model=DashboardSummary)
 async def summary(db: AsyncSession = Depends(get_db)):
     hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
     statuses = await get_statuses(hosts)
+    muted = set((await notification_settings_service.get_settings(db)).gateway_alert_muted or [])
+    gateways, denied = await _sample_readings_by_host(db, hosts, muted)
     host_summaries = [
         DashboardHost(
             id=host.id, name=host.name, status=statuses.get(host.id, {}).get("status", "offline"),
             carp_status=statuses.get(host.id, {}).get("carp_status"), cluster_name=host.cluster_name,
+            gateways=gateways.get(host.id), api_rights_missing=denied.get(host.id),
         )
         for host in hosts
     ]

backend/app/routers/gateways.py

@@ -0,0 +1,99 @@
+import re
+import uuid
+from datetime import datetime, timezone
+
+from fastapi import APIRouter, Depends
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.host import OpnsenseHost
+from app.models.resource_sample import HostResourceSample
+from app.schemas.gateway import ApiRightsState, GatewayAlertState, GatewayAlertUpdate, GatewayHealth
+from app.services import audit_service, gateway_health_service, notification_settings_service
+from app.services.monitor_checks import STALE_AFTER
+from app.services.opnsense_client import OpnsenseAPIError, OpnsenseForbiddenError
+
+router = APIRouter(prefix="/hosts/{host_id}/gateways", tags=["host-gateways"])
+
+# OPNsense gateway names are letters, digits and underscores (WAN_GW, WAN_DHCP6); a little more is allowed.
+_GATEWAY_NAME = re.compile(r"[A-Za-z0-9_.-]{1,64}")
+MAX_MUTED_GATEWAYS = 200
+
+
+@router.get("", response_model=list[GatewayHealth])
+async def list_gateways(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    """Live gateway state (loss, delay, deviation, OPNsense' verdict) of one firewall."""
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    try:
+        gateways = await gateway_health_service.get_gateway_health(host)
+    except OpnsenseForbiddenError as exc:
+        # The API key may not read the gateway state: say so instead of a generic upstream failure.
+        raise api_error(403, ErrorCode.API_RIGHTS_MISSING) from exc
+    except OpnsenseAPIError as exc:
+        raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
+    muted = set((await notification_settings_service.get_settings(db)).gateway_alert_muted or [])
+    return [g.model_copy(update={"alert_enabled": f"{host.id}:{g.name}" not in muted}) for g in gateways]
+
+
+@router.get("/api-rights", response_model=ApiRightsState)
+async def api_rights(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    """Side readings the newest FRESH resource sample could not take because the API key lacks the right
+    (null = no fresh sample, so nothing is known). Stored data only: no call to the firewall."""
+    if await db.get(OpnsenseHost, host_id) is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    row = (
+        await db.execute(
+            select(HostResourceSample.sampled_at, HostResourceSample.raw["denied"].as_json())
+            .where(HostResourceSample.host_id == host_id)
+            .order_by(HostResourceSample.sampled_at.desc())
+            .limit(1)
+        )
+    ).first()
+    if row is None or datetime.now(timezone.utc) - row[0] > STALE_AFTER or not isinstance(row[1], list):
+        return ApiRightsState(missing=None)
+    return ApiRightsState(missing=[str(x) for x in row[1]])
+
+
+@router.put("/{name}/alert", response_model=GatewayAlertState)
+async def set_gateway_alert(host_id: uuid.UUID, name: str, payload: GatewayAlertUpdate, db: AsyncSession = Depends(get_db)):
+    """Switch the gateway-down alert on or off for ONE gateway (admins only, see api_token_service)."""
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    if not _GATEWAY_NAME.fullmatch(name):
+        raise api_error(422, ErrorCode.GATEWAY_NAME_INVALID)
+    if not payload.enabled:
+        # Only a gateway the firewall really reported can be muted ("wan_gw" vs "WAN_GW" would silently do nothing).
+        # Without a usable sample nothing is known, so the request goes through.
+        known = (
+            await db.execute(
+                select(HostResourceSample.raw["gateways"].as_json())
+                .where(HostResourceSample.host_id == host_id)
+                .order_by(HostResourceSample.sampled_at.desc())
+                .limit(1)
+            )
+        ).scalar()
+        if isinstance(known, list) and name not in {g.get("name") for g in known if isinstance(g, dict)}:
+            raise api_error(404, ErrorCode.GATEWAY_UNKNOWN)
+    row = await notification_settings_service.get_settings(db)
+    await db.refresh(row, with_for_update=True)
+    muted = list(row.gateway_alert_muted or [])
+    key = f"{host.id}:{name}"
+    if payload.enabled:
+        muted = [m for m in muted if m != key]
+    elif key not in muted:
+        if len(muted) >= MAX_MUTED_GATEWAYS:
+            raise api_error(422, ErrorCode.GATEWAY_MUTE_LIMIT, limit=str(MAX_MUTED_GATEWAYS))
+        muted.append(key)
+    row.gateway_alert_muted = muted  # a new list, so the JSON column notices the change
+    await db.commit()
+    # The request middleware logs the call itself; this entry says what it did (only the validated name goes in).
+    await audit_service.persist(
+        kind="host", action="gateway_alert_on" if payload.enabled else "gateway_alert_off", entity_id=str(host.id),
+        host_id=str(host.id), host_name=host.name, status="accepted", message=f"Gateway {name}",
+    )
+    return GatewayAlertState(name=name, alert_enabled=payload.enabled)

backend/app/routers/groups.py

@@ -99,6 +99,9 @@ async def delete_group(group_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
     for assignment in (port_forward_groups, one_to_one_groups, outbound_nat_groups):
         if await db.scalar(select(assignment.c.group_id).where(assignment.c.group_id == group_id).limit(1)):
             raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET)
+    from app.models.rollout import rollout_wave_groups
+    if await db.scalar(select(rollout_wave_groups.c.group_id).where(rollout_wave_groups.c.group_id == group_id).limit(1)):
+        raise api_error(409, ErrorCode.ROLLOUT_GROUP_IN_USE)
     await db.delete(group)
     await db.commit()
 

backend/app/routers/gui_access.py

@@ -0,0 +1,190 @@
+"""Firewall GUI access through the gui-gateway container (1.27).
+
+Public API (session cookie, admin only, never API tokens):
+* POST /api/hosts/{id}/gui-ticket — one-time ticket + the slot URL the browser opens (noopener).
+* GET/PUT /api/settings/gui-access — switch, idle/absolute timeout, number of slots.
+
+Internal API for the gateway only (NOT under /api, so the proxy never forwards it; shared secret
+GUI_GATEWAY_TOKEN in the X-Onx-Gateway-Token header, compared in constant time):
+* POST /internal/gui/redeem, /internal/gui/revalidate, /internal/gui/end.
+The answers never contain firewall credentials: only the firewall origin, its pinned (public) CA
+certificate, the overlay proxy address and the timeouts.
+"""
+import hashlib
+import hmac
+import uuid
+from datetime import datetime, timezone
+from typing import Literal
+
+from fastapi import APIRouter, Depends, Header, HTTPException, Request, status
+from pydantic import BaseModel, Field, model_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.config import settings
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.host import OpnsenseHost
+from app.models.user import AdminUser
+from app.routers.auth import SECURE_COOKIE_NAME, current_user
+from app.services import api_token_service, gui_ticket_service as svc
+
+router = APIRouter(tags=["gui-access"])
+internal_router = APIRouter(prefix="/internal/gui", include_in_schema=False)
+
+
+def _require_admin_session(request: Request, user: AdminUser) -> None:
+    if getattr(request.state, "api_token", None) is not None or api_token_service.bearer_from_header(
+        request.headers.get("authorization", "")
+    ) is not None:
+        raise api_error(status.HTTP_403_FORBIDDEN, ErrorCode.GUI_SESSION_REQUIRED)
+    if user.role != "admin":
+        raise api_error(status.HTTP_403_FORBIDDEN, ErrorCode.ADMIN_REQUIRED)
+
+
+class GuiTicketRead(BaseModel):
+    url: str
+    slot: int
+    expires_in: int
+
+
+@router.post("/hosts/{host_id}/gui-ticket", response_model=GuiTicketRead)
+async def create_ticket(host_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db),
+                        user: AdminUser = Depends(current_user)):
+    _require_admin_session(request, user)
+    if not svc.gateway_configured():
+        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_GATEWAY_NOT_CONFIGURED)
+    session_cookie = request.cookies.get(SECURE_COOKIE_NAME, "")
+    if not settings.cookie_secure or not session_cookie:
+        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_REQUIRES_HTTPS)
+    row = await svc.get_settings(db)
+    if not row.enabled:
+        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_ACCESS_DISABLED)
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(status.HTTP_404_NOT_FOUND, ErrorCode.HOST_NICHT_GEFUNDEN)
+    refusal = svc.host_refusal(host)
+    if refusal:
+        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_HOST_NOT_ELIGIBLE, reason=refusal)
+    try:
+        return await svc.issue_ticket(db, user=user, session_token=session_cookie, host=host,
+                                      request_host=request.headers.get("host", ""))
+    except svc.GuiAccessError as exc:
+        raise api_error(exc.status, exc.code, **exc.params)
+
+
+class GuiAccessSettingsRead(BaseModel):
+    enabled: bool
+    idle_timeout_minutes: int
+    absolute_timeout_minutes: int
+    slot_count: int
+    available: bool
+    unavailable_reasons: list[str]
+    slots: list[int]
+    updated_by: str | None
+    updated_at: datetime | None
+
+
+class GuiAccessSettingsUpdate(BaseModel):
+    enabled: bool
+    idle_timeout_minutes: int = Field(ge=1, le=240)
+    absolute_timeout_minutes: int = Field(ge=5, le=720)
+    slot_count: int = Field(ge=1, le=svc.MAX_SLOTS)
+
+    @model_validator(mode="after")
+    def _idle_within_absolute(self):
+        if self.idle_timeout_minutes > self.absolute_timeout_minutes:
+            raise ValueError("idle_timeout_minutes must not exceed absolute_timeout_minutes")
+        return self
+
+
+def _settings_read(row) -> dict:
+    reasons = svc.unavailable_reasons()
+    return {"enabled": row.enabled, "idle_timeout_minutes": row.idle_timeout_minutes,
+            "absolute_timeout_minutes": row.absolute_timeout_minutes, "slot_count": row.slot_count,
+            "available": not reasons, "unavailable_reasons": reasons, "slots": svc.active_slots(row),
+            "updated_by": row.updated_by, "updated_at": row.updated_at}
+
+
+@router.get("/settings/gui-access", response_model=GuiAccessSettingsRead)
+async def read_settings(request: Request, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
+    _require_admin_session(request, user)
+    row = await svc.get_settings(db)
+    await db.commit()
+    return _settings_read(row)
+
+
+@router.put("/settings/gui-access", response_model=GuiAccessSettingsRead)
+async def update_settings(payload: GuiAccessSettingsUpdate, request: Request, db: AsyncSession = Depends(get_db),
+                          user: AdminUser = Depends(current_user)):
+    _require_admin_session(request, user)
+    if payload.enabled and svc.unavailable_reasons():
+        # Never switchable on without HTTPS cookies and a configured gateway.
+        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_GATEWAY_NOT_CONFIGURED,
+                        reasons=", ".join(svc.unavailable_reasons()))
+    row = await svc.get_settings(db)
+    row.enabled = payload.enabled
+    row.idle_timeout_minutes = payload.idle_timeout_minutes
+    row.absolute_timeout_minutes = payload.absolute_timeout_minutes
+    row.slot_count = payload.slot_count
+    row.updated_by = user.username
+    row.updated_at = datetime.now(timezone.utc)
+    if not payload.enabled:
+        await svc.end_all(db, "disabled")
+    await db.commit()
+    return _settings_read(row)
+
+
+# ── internal: gateway only ────────────────────────────────────────────────
+
+def _gateway_auth(x_onx_gateway_token: str = Header(default="")) -> None:
+    expected = settings.gui_gateway_token
+    if len(expected) < svc.MIN_GATEWAY_TOKEN_LENGTH:
+        raise HTTPException(status_code=404)
+    # Fixed-length digests: the comparison time does not depend on the presented value.
+    if not hmac.compare_digest(hashlib.sha256(x_onx_gateway_token.encode()).digest(),
+                               hashlib.sha256(expected.encode()).digest()):
+        raise HTTPException(status_code=401)
+
+
+_HEX64 = r"^[0-9a-f]{64}$"
+
+
+class RedeemRequest(BaseModel):
+    ticket_hash: str = Field(pattern=_HEX64)
+    session_hash: str = Field(pattern=_HEX64)
+    slot: int = Field(ge=1, le=65535)
+    client_ip: str | None = Field(default=None, max_length=45)
+
+
+class RevalidateRequest(BaseModel):
+    gui_session_id: uuid.UUID
+    session_hash: str = Field(pattern=_HEX64)
+    # Fingerprint of the target as handed out at redeem time (optional: an older gateway image omits it).
+    target_fp: str | None = Field(default=None, pattern=_HEX64)
+
+
+class EndRequest(BaseModel):
+    gui_session_id: uuid.UUID
+    session_hash: str = Field(pattern=_HEX64)
+    reason: Literal["idle", "absolute", "logout", "revoked", "superseded", "shutdown"]
+
+
+@internal_router.post("/redeem", dependencies=[Depends(_gateway_auth)])
+async def internal_redeem(payload: RedeemRequest, db: AsyncSession = Depends(get_db)):
+    result = await svc.redeem(db, ticket_hash=payload.ticket_hash, session_hash=payload.session_hash,
+                              slot=payload.slot, client_ip=payload.client_ip)
+    if not result["ok"]:
+        raise HTTPException(status_code=403, detail={"ok": False, "reason": result["reason"]})
+    return result
+
+
+@internal_router.post("/revalidate", dependencies=[Depends(_gateway_auth)])
+async def internal_revalidate(payload: RevalidateRequest, db: AsyncSession = Depends(get_db)):
+    return await svc.revalidate(db, gui_session_id=payload.gui_session_id, session_hash=payload.session_hash,
+                                 target_fp=payload.target_fp)
+
+
+@internal_router.post("/end", dependencies=[Depends(_gateway_auth)])
+async def internal_end(payload: EndRequest, db: AsyncSession = Depends(get_db)):
+    return {"ok": await svc.end(db, gui_session_id=payload.gui_session_id, session_hash=payload.session_hash,
+                                reason=payload.reason)}

backend/app/routers/hosts.py

@@ -226,6 +226,12 @@ async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
     draft_certs = (await db.scalars(select(ManagedCertificate.id).where(ManagedCertificate.host_id == host_id))).all()
     draft_accounts = (await db.scalars(select(AcmeAccount.id).where(AcmeAccount.host_id == host_id))).all()
     await db.delete(host)
+    # Gateway alert switches of this host would otherwise stay in the settings forever (and count against the limit).
+    from app.services import notification_settings_service
+    settings_row = await notification_settings_service.get_settings(db)
+    prefix = f"{host_id}:"
+    if any(str(m).startswith(prefix) for m in settings_row.gateway_alert_muted or []):
+        settings_row.gateway_alert_muted = [m for m in settings_row.gateway_alert_muted if not str(m).startswith(prefix)]
     await db.commit()
     opnsense_secret_store.delete_secret(host_id)
     for cert_id in draft_certs:

backend/app/routers/maintenance_windows.py

@@ -0,0 +1,244 @@
+"""Recurring maintenance windows and their exceptions (1.25).
+
+Every write here counts as a rollout action (api_token_service.is_rollout): a window decides WHEN
+unattended firmware actions may run, so an operator needs the same right as for deploying.
+"""
+import re
+import uuid
+from datetime import date, datetime, time, timedelta, timezone
+from typing import Literal
+
+from fastapi import APIRouter, Depends, Query, Request
+from pydantic import BaseModel, Field
+from sqlalchemy import select
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.maintenance_window import MaintenanceWindow, MaintenanceWindowException
+from app.services import audit_service, window_service
+from app.services.policy_service import policy_lock
+
+router = APIRouter(prefix="/maintenance-windows", tags=["maintenance-windows"])
+
+_TIME = re.compile(r"^([01]\d|2[0-3]):([0-5]\d)$")
+
+
+class WindowIn(BaseModel):
+    name: str = Field(min_length=1, max_length=120)
+    timezone: str = Field(min_length=1, max_length=64)
+    recurrence: Literal["weekly", "monthly"]
+    weekdays: list[int] = Field(default_factory=list, max_length=7)
+    month_mode: Literal["nth_weekday", "day"] | None = None
+    month_nth: int | None = None
+    month_weekday: int | None = None
+    month_day: int | None = None
+    start_time: str
+    duration_minutes: int = Field(ge=15, le=window_service.MAX_DURATION_MINUTES)
+    start_cutoff_minutes: int = Field(default=60, ge=5, le=window_service.MAX_DURATION_MINUTES)
+    enabled: bool = True
+
+
+class ExceptionIn(BaseModel):
+    occurrence_date: date
+    kind: Literal["skip", "move"]
+    moved_start: datetime | None = None
+    reason: str = Field(default="", max_length=300)
+
+
+def _invalid(field: str):
+    return api_error(422, ErrorCode.WINDOW_INVALID, field=field)
+
+
+def _validated(payload: WindowIn) -> dict:
+    name = payload.name.strip()
+    if not name:
+        raise _invalid("name")
+    if not window_service.valid_timezone(payload.timezone):
+        raise _invalid("timezone")
+    match = _TIME.match(payload.start_time)
+    if not match:
+        raise _invalid("start_time")
+    values = {
+        "name": name, "timezone": payload.timezone, "recurrence": payload.recurrence,
+        "weekdays": [], "month_mode": None, "month_nth": None, "month_weekday": None, "month_day": None,
+        "start_time": time(int(match.group(1)), int(match.group(2))),
+        "duration_minutes": payload.duration_minutes,
+        "start_cutoff_minutes": min(payload.start_cutoff_minutes, payload.duration_minutes),
+        "enabled": payload.enabled,
+    }
+    if payload.recurrence == "weekly":
+        days = sorted(set(payload.weekdays))
+        if not days or any(d < 0 or d > 6 for d in days):
+            raise _invalid("weekdays")
+        values["weekdays"] = days
+    elif payload.month_mode == "day":
+        if payload.month_day is None or not 1 <= payload.month_day <= 28:
+            raise _invalid("month_day")
+        values.update(month_mode="day", month_day=payload.month_day)
+    elif payload.month_mode == "nth_weekday":
+        if payload.month_nth not in (1, 2, 3, 4, -1):
+            raise _invalid("month_nth")
+        if payload.month_weekday is None or not 0 <= payload.month_weekday <= 6:
+            raise _invalid("month_weekday")
+        values.update(month_mode="nth_weekday", month_nth=payload.month_nth, month_weekday=payload.month_weekday)
+    else:
+        raise _invalid("month_mode")
+    return values
+
+
+async def _plans_using(db: AsyncSession, window_id) -> list[dict]:
+    from app.models.rollout import RolloutPlan
+    rows = (await db.scalars(select(RolloutPlan).where(RolloutPlan.window_id == window_id).order_by(RolloutPlan.name))).all()
+    return [{"id": str(p.id), "name": p.name, "auto_start": p.auto_start, "active": p.activated_at is not None} for p in rows]
+
+
+def read_window(window: MaintenanceWindow, *, now: datetime | None = None, count: int = 5) -> dict:
+    now = now or datetime.now(timezone.utc)
+    return {
+        "id": str(window.id), "name": window.name, "timezone": window.timezone, "recurrence": window.recurrence,
+        "weekdays": window.weekdays or [], "month_mode": window.month_mode, "month_nth": window.month_nth,
+        "month_weekday": window.month_weekday, "month_day": window.month_day,
+        "start_time": window.start_time.strftime("%H:%M"), "duration_minutes": window.duration_minutes,
+        "start_cutoff_minutes": window.start_cutoff_minutes, "enabled": window.enabled,
+        "created_by": window.created_by, "created_at": window.created_at,
+        "open_now": window_service.current(window, now) is not None,
+        "next": [o.as_dict() for o in window_service.occurrences(window, now, count, with_skipped=True)] if window.enabled else [],
+        "exceptions": [
+            {"id": str(e.id), "occurrence_date": e.occurrence_date.isoformat(), "kind": e.kind,
+             "moved_start": e.moved_start, "reason": e.reason, "actor": e.actor, "created_at": e.created_at}
+            for e in window.exceptions
+        ],
+    }
+
+
+async def _window_or_404(db: AsyncSession, window_id: uuid.UUID) -> MaintenanceWindow:
+    window = await db.get(MaintenanceWindow, window_id)
+    if window is None:
+        raise api_error(404, ErrorCode.WINDOW_NOT_FOUND)
+    return window
+
+
+async def _name_taken(db: AsyncSession, name: str, own_id=None) -> bool:
+    other = await db.scalar(select(MaintenanceWindow.id).where(MaintenanceWindow.name == name))
+    return other is not None and other != own_id
+
+
+@router.get("")
+async def list_windows(db: AsyncSession = Depends(get_db)):
+    rows = (await db.scalars(select(MaintenanceWindow).order_by(MaintenanceWindow.name))).all()
+    result = []
+    for w in rows:
+        item = read_window(w)
+        item["used_by"] = await _plans_using(db, w.id)
+        result.append(item)
+    return result
+
+
+@router.post("", status_code=201)
+async def create_window(payload: WindowIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    values = _validated(payload)
+    if await _name_taken(db, values["name"]):
+        raise api_error(409, ErrorCode.WINDOW_NAME_TAKEN)
+    window = MaintenanceWindow(**values, created_by=request.state.user.username, created_at=datetime.now(timezone.utc))
+    window.exceptions = []
+    db.add(window)
+    await db.commit()
+    item = read_window(await _window_or_404(db, window.id))
+    item["used_by"] = []
+    return item
+
+
+@router.get("/{window_id}")
+async def get_window(window_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    item = read_window(await _window_or_404(db, window_id))
+    item["used_by"] = await _plans_using(db, window_id)
+    return item
+
+
+@router.put("/{window_id}")
+async def update_window(window_id: uuid.UUID, payload: WindowIn, db: AsyncSession = Depends(get_db)):
+    """Changing a window changes the fingerprint of every plan that uses it, so an activated
+    automatic start stops until the plan is activated again (see rollout_service.fingerprint)."""
+    await policy_lock(db)
+    window = await _window_or_404(db, window_id)
+    values = _validated(payload)
+    if await _name_taken(db, values["name"], window.id):
+        raise api_error(409, ErrorCode.WINDOW_NAME_TAKEN)
+    for key, value in values.items():
+        setattr(window, key, value)
+    await db.commit()
+    item = read_window(await _window_or_404(db, window_id))
+    item["used_by"] = await _plans_using(db, window_id)
+    return item
+
+
+@router.delete("/{window_id}", status_code=204)
+async def delete_window(window_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    window = await _window_or_404(db, window_id)
+    # A plan without a window means "no time limit": deleting its window must never turn it into one.
+    if await _plans_using(db, window_id):
+        raise api_error(409, ErrorCode.WINDOW_IN_USE)
+    await db.delete(window)
+    await db.commit()
+
+
+@router.get("/{window_id}/occurrences")
+async def window_occurrences(window_id: uuid.UUID, count: int = Query(10, ge=1, le=50), db: AsyncSession = Depends(get_db)):
+    window = await _window_or_404(db, window_id)
+    return [o.as_dict() for o in window_service.occurrences(window, datetime.now(timezone.utc), count, with_skipped=True)]
+
+
+@router.post("/{window_id}/exceptions", status_code=201)
+async def add_exception(window_id: uuid.UUID, payload: ExceptionIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    window = await _window_or_404(db, window_id)
+    now = datetime.now(timezone.utc)
+    if not window_service.is_occurrence_date(window, payload.occurrence_date):
+        raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="occurrence_date")
+    # Only future occurrences (or the one open now) can be changed; history stays as it was.
+    if window_service.local_start(window, payload.occurrence_date) + timedelta(minutes=window.duration_minutes) <= now:
+        raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="occurrence_date")
+    moved = None
+    if payload.kind == "move":
+        if payload.moved_start is None:
+            raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="moved_start")
+        moved = payload.moved_start if payload.moved_start.tzinfo else payload.moved_start.replace(tzinfo=timezone.utc)
+        if moved <= now:
+            raise api_error(422, ErrorCode.WINDOW_EXCEPTION_INVALID, field="moved_start")
+    if any(e.occurrence_date == payload.occurrence_date for e in window.exceptions):
+        raise api_error(409, ErrorCode.WINDOW_EXCEPTION_EXISTS)
+    exc = MaintenanceWindowException(
+        window_id=window.id, occurrence_date=payload.occurrence_date, kind=payload.kind, moved_start=moved,
+        reason=payload.reason.strip(), actor=request.state.user.username, created_at=now,
+    )
+    db.add(exc)
+    try:
+        await db.commit()
+    except IntegrityError:
+        await db.rollback()
+        raise api_error(409, ErrorCode.WINDOW_EXCEPTION_EXISTS)
+    await audit_service.persist(
+        kind="maintenance_window", action=f"exception_{payload.kind}", status="saved", entity_id=str(window.id),
+        entity_name=window.name, details={"occurrence_date": payload.occurrence_date.isoformat(),
+                                          "moved_start": moved.isoformat() if moved else None, "reason": exc.reason},
+    )
+    db.expire_all()
+    return read_window(await _window_or_404(db, window_id))
+
+
+@router.delete("/{window_id}/exceptions/{exception_id}")
+async def delete_exception(window_id: uuid.UUID, exception_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    window = await _window_or_404(db, window_id)
+    exc = await db.get(MaintenanceWindowException, exception_id)
+    if exc is None or exc.window_id != window.id:
+        raise api_error(404, ErrorCode.WINDOW_EXCEPTION_NOT_FOUND)
+    await db.delete(exc)
+    await db.commit()
+    db.expire_all()
+    return read_window(await _window_or_404(db, window_id))
+

backend/app/routers/mcp.py

@@ -88,6 +88,48 @@ TOOLS = [
         "path": "/api/search",
         "query": ("q",),
     },
+    {
+        "name": "rule_match",
+        "description": "Which firewall rule decides a packet (first packet of a new connection), per firewall, "
+                       "with the verdict (pass/block/reject), the deciding rule and an honest confidence "
+                       "(sure/probable/unknown plus reason codes). Reads only; nothing is changed.",
+        "inputSchema": {
+            "type": "object", "required": ["src", "dst"],
+            "properties": {
+                "src": {"type": "string", "description": "source IP address"},
+                "dst": {"type": "string", "description": "destination IP address"},
+                "proto": {"type": "string", "description": "tcp, udp, icmp, ipv6-icmp or a protocol number", "default": "tcp"},
+                "dport": {"type": "integer", "minimum": 0, "maximum": 65535},
+                "sport": {"type": "integer", "minimum": 0, "maximum": 65535, "description": "default: any ephemeral port"},
+                "icmptype": {"type": "string", "description": "e.g. echoreq or 8"},
+                "iface": {"type": "string", "description": "ingress interface identifier (e.g. lan); default: guessed per firewall"},
+                "egress": {"type": "string", "description": "also check the outgoing side: auto or an interface identifier"},
+                "host_ids": {"type": "string", "description": "comma separated firewall ids (default: all)"},
+                "group_id": {"type": "string", "description": "device group id instead of host_ids"},
+            },
+        },
+        "path": "/api/rule-analysis/match",
+        "query": ("src", "dst", "proto", "dport", "sport", "icmptype", "iface", "egress", "host_ids", "group_id"),
+        # The per-rule trace is for the UI; the verdict, deciding rule and reasons are what matters here.
+        "drop": ("trace",),
+    },
+    {
+        "name": "rule_cleanup_candidates",
+        "description": "Cleanup candidates of one firewall: rules that are unused (no hit for N days), shadowed or "
+                       "duplicate (an earlier quick rule already decides everything they match), or disabled for a "
+                       "long time. Each with its confidence (sure/probable/unknown), the last observed hit and "
+                       "whether it could be removed (not for automatic, legacy or OPNexus-managed rules). Reads only.",
+        "inputSchema": {
+            "type": "object", "required": ["host_id"],
+            "properties": {
+                "host_id": {"type": "string", "description": "Firewall id from list_hosts"},
+                "unused_days": {"type": "integer", "minimum": 1, "maximum": 3650, "default": 30,
+                                "description": "observation window for unused / disabled rules"},
+            },
+        },
+        "path": "/api/hosts/{host_id}/rule-analysis/cleanup",
+        "query": ("unused_days",),
+    },
     {
         "name": "api_get",
         "description": "Read any other OPNexus API endpoint (GET only), e.g. /api/rules, /api/aliases, /api/logs. "

backend/app/routers/notifications.py

@@ -1,21 +1,26 @@
+import uuid
 from urllib.parse import urlsplit
 
 from fastapi import APIRouter, Depends, Query
 from sqlalchemy import select
+from sqlalchemy.exc import IntegrityError
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.errors import ErrorCode, api_error
 from app.database import get_db
 from app.models.user import AdminUser
 from app.routers.auth import current_user
-from app.models.notification import NotificationProblem
+from app.models.notification import NotificationChannel, NotificationProblem
 from app.schemas.notification import (
+    NotificationChannelCreate,
+    NotificationChannelRead,
+    NotificationChannelUpdate,
     NotificationProblemRead,
     NotificationSettingsRead,
     NotificationSettingsUpdate,
     TestNotificationRequest,
 )
-from app.services import notification_delivery, notification_settings_service
+from app.services import notification_channels, notification_delivery, notification_secret_store, notification_settings_service, url_guard
 
 router = APIRouter(prefix="/notifications", tags=["notifications"])
 
@@ -43,6 +48,11 @@ async def read_settings(db: AsyncSession = Depends(get_db), user: AdminUser = De
 
 @router.put("/settings", response_model=NotificationSettingsRead)
 async def update_settings(payload: NotificationSettingsUpdate, db: AsyncSession = Depends(get_db)):
+    if payload.webhook_url:
+        try:
+            payload.webhook_url = url_guard.validate_url(payload.webhook_url)
+        except ValueError as exc:
+            raise api_error(422, ErrorCode.NOTIFICATION_WEBHOOK_URL_INVALID, reason=str(exc)) from None
     return await notification_settings_service.update_settings(db, payload)
 
 
@@ -79,3 +89,103 @@ async def test_notification(payload: TestNotificationRequest, db: AsyncSession =
     except notification_delivery.NotificationDeliveryError as exc:
         raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
     return {"success": True}
+
+
+# ── Additional channels (ntfy, Gotify, Telegram, Teams, Matrix) ────────────────
+
+def _channel_read(row: NotificationChannel, user: AdminUser) -> NotificationChannelRead:
+    read = NotificationChannelRead.model_validate(row)
+    # An ntfy topic or a chat id is as good as a password for whoever knows it: only admins see the settings.
+    return read if user.role == "admin" else read.model_copy(update={"config": {}})
+
+
+async def _get_channel(db: AsyncSession, channel_id: uuid.UUID) -> NotificationChannel:
+    row = await db.get(NotificationChannel, channel_id)
+    if row is None:
+        raise api_error(404, ErrorCode.NOTIFICATION_CHANNEL_NOT_FOUND)
+    return row
+
+
+def _invalid(exc: ValueError):
+    return api_error(422, ErrorCode.NOTIFICATION_CHANNEL_INVALID, reason=str(exc))
+
+
+@router.get("/channels", response_model=list[NotificationChannelRead])
+async def list_channels(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
+    rows = await db.scalars(select(NotificationChannel).order_by(NotificationChannel.name))
+    return [_channel_read(r, user) for r in rows.all()]
+
+
+@router.post("/channels", response_model=NotificationChannelRead, status_code=201)
+async def create_channel(payload: NotificationChannelCreate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
+    try:
+        config = notification_channels.clean_config(payload.channel_type, payload.config)
+        secret = notification_channels.clean_secret(payload.channel_type, payload.secret) if payload.secret else None
+    except ValueError as exc:
+        raise _invalid(exc) from None
+    row = NotificationChannel(
+        name=payload.name.strip(), channel_type=payload.channel_type, enabled=payload.enabled, config=config,
+        secret_set=secret is not None,
+    )
+    row.id = uuid.uuid4()
+    if secret is None and notification_channels.missing_requirements(row) == "Zugangsdaten fehlen":
+        raise _invalid(ValueError("Zugangsdaten fehlen"))
+    db.add(row)
+    try:
+        await db.flush()
+    except IntegrityError:
+        await db.rollback()
+        raise api_error(409, ErrorCode.NOTIFICATION_CHANNEL_NAME_TAKEN) from None
+    if secret is not None:
+        notification_secret_store.save_secret(notification_channels.secret_name(row.id), secret)
+    await db.commit()
+    return _channel_read(row, user)
+
+
+@router.put("/channels/{channel_id}", response_model=NotificationChannelRead)
+async def update_channel(
+    channel_id: uuid.UUID, payload: NotificationChannelUpdate,
+    db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user),
+):
+    row = await _get_channel(db, channel_id)
+    try:
+        if payload.config is not None:
+            row.config = notification_channels.clean_config(row.channel_type, payload.config)
+        new_secret = notification_channels.clean_secret(row.channel_type, payload.secret) if payload.secret else None
+    except ValueError as exc:
+        raise _invalid(exc) from None
+    if payload.name is not None:
+        row.name = payload.name.strip()
+    if payload.enabled is not None:
+        row.enabled = payload.enabled
+    if payload.clear_secret:
+        notification_secret_store.delete_secret(notification_channels.secret_name(row.id))
+        row.secret_set = False
+    elif new_secret is not None:
+        notification_secret_store.save_secret(notification_channels.secret_name(row.id), new_secret)
+        row.secret_set = True
+    try:
+        await db.commit()
+    except IntegrityError:
+        await db.rollback()
+        raise api_error(409, ErrorCode.NOTIFICATION_CHANNEL_NAME_TAKEN) from None
+    return _channel_read(row, user)
+
+
+@router.delete("/channels/{channel_id}", status_code=204)
+async def delete_channel(channel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    row = await _get_channel(db, channel_id)
+    notification_secret_store.delete_secret(notification_channels.secret_name(row.id))
+    await db.delete(row)
+    await db.commit()
+
+
+@router.post("/channels/{channel_id}/test")
+async def test_channel(channel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    row = await _get_channel(db, channel_id)
+    try:
+        await notification_channels.send_test(row)
+    except notification_delivery.NotificationDeliveryError as exc:
+        # The text names the channel and the HTTP status; it never contains a URL or a token.
+        raise api_error(502, ErrorCode.NOTIFICATION_CHANNEL_TEST_FAILED, reason=str(exc)) from None
+    return {"success": True}

backend/app/routers/oidc.py

@@ -22,6 +22,28 @@ logger = logging.getLogger(__name__)
 router = APIRouter(tags=["oidc"])
 CALLBACK_PATH = "/api/auth/oidc/callback"
 STATE_COOKIE = "opnexus-oidc-state"
+# HTTPS deployments (1.29): __Host- (Secure, no Domain, Path=/) so that no sibling SUBDOMAIN or plain-HTTP page can
+# plant a state cookie of its own (login CSRF with the attacker's own authorisation). Known limit: cookies are not
+# separated by PORT, so script in a firewall GUI shown on a gateway slot port of the same host could still set it
+# (BACKLOG S8, accepted: needs a compromised firewall GUI and a mapped IdP account; the real fix is one hostname
+# per slot). The browser only accepts the prefix with Path=/, so the cookie travels with every request for its
+# 10 minutes; plain-HTTP labs keep the narrow path.
+SECURE_STATE_COOKIE = "__Host-opnexus-oidc-state"
+STATE_COOKIE_PATH = "/api/auth/oidc"
+
+
+def state_cookie_name() -> str:
+    return SECURE_STATE_COOKIE if settings.cookie_secure else STATE_COOKIE
+
+
+def _state_cookie_path() -> str:
+    return "/" if settings.cookie_secure else STATE_COOKIE_PATH
+
+
+def _clear_state_cookie(response) -> None:
+    response.delete_cookie(state_cookie_name(), path=_state_cookie_path(), secure=settings.cookie_secure, httponly=True, samesite="lax")
+    if settings.cookie_secure:  # a plain-name cookie of an earlier version
+        response.delete_cookie(STATE_COOKIE, path=STATE_COOKIE_PATH, secure=True, httponly=True, samesite="lax")
 # SSO sessions are short: the provider's group membership is only re-checked at login, so a removed person
 # must not keep a 30-day session.
 SSO_SESSION_SECONDS = 12 * 3600
@@ -29,7 +51,7 @@ SSO_SESSION_SECONDS = 12 * 3600
 
 def _login_error(code: str) -> RedirectResponse:
     response = RedirectResponse(f"/login?sso_error={code}", status_code=302)
-    response.delete_cookie(STATE_COOKIE, path="/api/auth/oidc")
+    _clear_state_cookie(response)
     return response
 
 
@@ -63,7 +85,8 @@ async def start(request: Request, origin: str, db: AsyncSession = Depends(get_db
     response = RedirectResponse(url, status_code=302)
     # Binds the attempt to THIS browser: a callback URL handed to somebody else (login CSRF) or a captured
     # code+state pair is useless without the cookie.
-    response.set_cookie(STATE_COOKIE, state, max_age=600, httponly=True, samesite="lax", secure=settings.cookie_secure, path="/api/auth/oidc")
+    response.set_cookie(state_cookie_name(), state, max_age=600, httponly=True, samesite="lax", secure=settings.cookie_secure,
+                        path=_state_cookie_path())
     return response
 
 
@@ -78,7 +101,9 @@ async def callback(request: Request, code: str = "", state: str = "", error: str
     try:
         if error or not code or not state:
             raise oidc_service.OidcError("provider", "Anbieter meldet " + (_printable(error) if error else "keinen Code"))
-        if not secrets.compare_digest(request.cookies.get(STATE_COOKIE, ""), state):
+        # Only the cookie name of this deployment counts (on HTTPS never the plain one); no cookie never matches.
+        bound = request.cookies.get(state_cookie_name(), "")
+        if not bound or not secrets.compare_digest(bound.encode(), state.encode()):
             raise oidc_service.OidcError("state", "Der Anmeldeversuch gehört nicht zu diesem Browser")
         user = await oidc_service.complete(db, cfg, code, state)
     except oidc_service.OidcError as exc:
@@ -92,7 +117,7 @@ async def callback(request: Request, code: str = "", state: str = "", error: str
     )
     response = RedirectResponse("/dashboard", status_code=302)
     _set_session_cookie(response, token, max_age=SSO_SESSION_SECONDS)
-    response.delete_cookie(STATE_COOKIE, path="/api/auth/oidc")
+    _clear_state_cookie(response)
     logger.info("SSO-Anmeldung von '%s' (%s) aus %s", user.username, user.role, ip)
     return response
 

backend/app/routers/one_to_one.py

@@ -20,6 +20,7 @@ from app.services.interface_service import check_interface_consistency
 from app.services.opnsense_client import OpnsenseAPIError
 from app.services.one_to_one_deploy_service import confirm_one_to_one, deploy_one_to_one, undeploy_one_to_one, revert_deployment
 from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
+from app.routers.adopt_routes import add_adopt_routes
 
 router = APIRouter(prefix="/one-to-one", tags=["one-to-one"])
 
@@ -79,6 +80,9 @@ def _to_read(rule: OneToOneRule, hosts: list[OpnsenseHost], deployments: list[On
     )
 
 
+add_adopt_routes(router, kind="one_to_one", model=OneToOneRule, load_options=_load_options, to_read=_to_read, read_model=OneToOneRead)
+
+
 @router.get("", response_model=list[OneToOneRead])
 async def list_one_to_one_rules(db: AsyncSession = Depends(get_db)):
     result = await db.execute(select(OneToOneRule).options(*_load_options()))

backend/app/routers/outbound_nat.py

@@ -21,6 +21,7 @@ from app.services.interface_service import check_interface_consistency
 from app.services.opnsense_client import OpnsenseAPIError
 from app.services.outbound_nat_deploy_service import confirm_outbound_nat, deploy_outbound_nat, undeploy_outbound_nat, revert_deployment
 from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
+from app.routers.adopt_routes import add_adopt_routes
 
 router = APIRouter(prefix="/outbound-nat", tags=["outbound-nat"])
 
@@ -82,6 +83,9 @@ def _to_read(rule: OutboundNatRule, hosts: list[OpnsenseHost], deployments: list
     )
 
 
+add_adopt_routes(router, kind="outbound_nat", model=OutboundNatRule, load_options=_load_options, to_read=_to_read, read_model=OutboundNatRead)
+
+
 @router.get("", response_model=list[OutboundNatRead])
 async def list_outbound_nat_rules(db: AsyncSession = Depends(get_db)):
     result = await db.execute(select(OutboundNatRule).options(*_load_options()))

backend/app/routers/port_forward.py

@@ -20,6 +20,7 @@ from app.services.interface_service import check_interface_consistency
 from app.services.opnsense_client import OpnsenseAPIError
 from app.services.port_forward_deploy_service import confirm_port_forward, deploy_port_forward, undeploy_port_forward, revert_deployment
 from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
+from app.routers.adopt_routes import add_adopt_routes
 
 router = APIRouter(prefix="/port-forward", tags=["port-forward"])
 
@@ -80,6 +81,9 @@ def _to_read(rule: PortForwardRule, hosts: list[OpnsenseHost], deployments: list
     )
 
 
+add_adopt_routes(router, kind="port_forward", model=PortForwardRule, load_options=_load_options, to_read=_to_read, read_model=PortForwardRead)
+
+
 @router.get("", response_model=list[PortForwardRead])
 async def list_port_forward_rules(db: AsyncSession = Depends(get_db)):
     result = await db.execute(select(PortForwardRule).options(*_load_options()))

backend/app/routers/rollouts.py

@@ -0,0 +1,434 @@
+"""Firmware rollouts in waves (1.25): plans and runs.
+
+Roles (enforced in AuthMiddleware, see api_token_service.is_rollout): everybody signed in reads;
+operators prepare plans; starting/activating a plan and resuming/approving/skipping/aborting a run
+need the deploy right (admin, or operator while operators may deploy). Pausing is deliberately NOT
+a rollout action: anybody who may write at all may always brake.
+"""
+import uuid
+from datetime import datetime, timezone
+from typing import Literal
+
+from fastapi import APIRouter, Depends, Request
+from pydantic import BaseModel, Field
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.maintenance_window import MaintenanceWindow
+from app.models.policy import DeviceGroup
+from app.models.rollout import RolloutPlan, RolloutPlanWave, RolloutRun
+from app.models.scheduled_maintenance import ScheduledMaintenance
+from app.services import audit_service, rollout_engine, rollout_health, rollout_service
+from app.services.policy_service import policy_lock
+
+router = APIRouter(prefix="/rollouts", tags=["rollouts"])
+
+ACTIVE_RUN = rollout_engine.ACTIVE
+
+
+class WaveIn(BaseModel):
+    label: str = Field(min_length=1, max_length=60)
+    group_ids: list[uuid.UUID] = Field(min_length=1, max_length=20)
+    soak_minutes: int = Field(default=30, ge=0, le=rollout_service.MAX_SOAK_MINUTES)
+    carp_handover: bool = False
+    require_approval: bool = False
+
+
+class PlanIn(BaseModel):
+    name: str = Field(min_length=1, max_length=120)
+    description: str = Field(default="", max_length=2000)
+    action: Literal["update", "major_update"] = "update"
+    window_id: uuid.UUID | None = None
+    auto_start: bool = False
+    gate_checks: dict[str, bool] = Field(default_factory=dict)
+    compliance_mode: Literal["warn", "fail"] = "warn"
+    waves: list[WaveIn] = Field(min_length=1, max_length=rollout_service.MAX_WAVES)
+
+
+class TokenIn(BaseModel):
+    preview_token: str = Field(min_length=64, max_length=64)
+
+
+async def _plan_or_404(db: AsyncSession, plan_id: uuid.UUID) -> RolloutPlan:
+    plan = await rollout_service.load_plan(db, plan_id)
+    if plan is None:
+        raise api_error(404, ErrorCode.ROLLOUT_PLAN_NOT_FOUND)
+    return plan
+
+
+async def read_plan(db: AsyncSession, plan: RolloutPlan, resolved: dict | None = None) -> dict:
+    resolved = resolved or await rollout_service.resolve(db, plan)
+    active = plan.activated_at is not None
+    last_run = await db.scalar(select(RolloutRun).where(RolloutRun.plan_id == plan.id).order_by(RolloutRun.created_at.desc()).limit(1))
+    return {
+        **rollout_service.definition(plan),
+        "description": plan.description,
+        "waves": [{"id": str(w.id), "position": w.position, "label": w.label, "soak_minutes": w.soak_minutes,
+                   "carp_handover": w.carp_handover, "require_approval": w.require_approval,
+                   "group_ids": [str(g.id) for g in w.groups], "groups": [g.name for g in w.groups],
+                   "host_count": len(resolved["waves"][i]["hosts"]) if i < len(resolved["waves"]) else 0}
+                  for i, w in enumerate(plan.waves)],
+        "active": active, "activated_by": plan.activated_by, "activated_at": plan.activated_at,
+        # Activated but the plan, its groups or its window changed since: the automatic start is held back.
+        "fingerprint_ok": (not active) or plan.activated_fingerprint == rollout_service.fingerprint(plan, resolved),
+        "auto_start_error": plan.auto_start_error,
+        "window": resolved["window"], "errors": resolved["errors"], "warnings": resolved["warnings"],
+        "host_count": sum(len(w["hosts"]) for w in resolved["waves"]),
+        "last_run": {"id": str(last_run.id), "status": last_run.status, "created_at": last_run.created_at} if last_run else None,
+        "created_by": plan.created_by, "created_at": plan.created_at, "updated_at": plan.updated_at,
+    }
+
+
+async def _apply(db: AsyncSession, plan: RolloutPlan, payload: PlanIn) -> None:
+    name = payload.name.strip()
+    if not name:
+        raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="name")
+    other = await db.scalar(select(RolloutPlan.id).where(RolloutPlan.name == name))
+    if other is not None and other != plan.id:
+        raise api_error(409, ErrorCode.ROLLOUT_NAME_TAKEN)
+    if payload.window_id is not None and await db.get(MaintenanceWindow, payload.window_id) is None:
+        raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="window_id")
+    if any(k not in rollout_health.CHECKS for k in payload.gate_checks):
+        raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="gate_checks")
+    wanted = {g for w in payload.waves for g in w.group_ids}
+    groups = {g.id: g for g in (await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(wanted)))).all()}
+    if len(groups) != len(wanted):
+        raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="group_ids")
+    plan.name, plan.description, plan.action = name, payload.description.strip(), payload.action
+    plan.window_id, plan.auto_start = payload.window_id, payload.auto_start
+    plan.gate_checks = {k: bool(v) for k, v in payload.gate_checks.items()}
+    plan.compliance_mode = payload.compliance_mode
+    plan.waves = [
+        RolloutPlanWave(position=i, label=w.label.strip() or f"{i + 1}", soak_minutes=w.soak_minutes, carp_handover=w.carp_handover,
+                        require_approval=w.require_approval, groups=[groups[g] for g in dict.fromkeys(w.group_ids)])
+        for i, w in enumerate(payload.waves)
+    ]
+    # Any edit disarms the automatic start: an operator cannot quietly change what an admin activated.
+    plan.activated_at = plan.activated_by = plan.activated_by_id = plan.activated_fingerprint = None
+    plan.auto_start_error = None
+
+
+@router.get("/plans")
+async def list_plans(db: AsyncSession = Depends(get_db)):
+    plans = (await db.scalars(select(RolloutPlan).order_by(RolloutPlan.name))).all()
+    return [await read_plan(db, p) for p in plans]
+
+
+@router.post("/plans", status_code=201)
+async def create_plan(payload: PlanIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = RolloutPlan(created_by=request.state.user.username, created_at=datetime.now(timezone.utc), waves=[])
+    await _apply(db, plan, payload)
+    db.add(plan)
+    await db.commit()
+    return await read_plan(db, await _plan_or_404(db, plan.id))
+
+
+@router.get("/plans/{plan_id}")
+async def get_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    return await read_plan(db, await _plan_or_404(db, plan_id))
+
+
+@router.put("/plans/{plan_id}")
+async def update_plan(plan_id: uuid.UUID, payload: PlanIn, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = await _plan_or_404(db, plan_id)
+    was_active = plan.activated_at is not None
+    await _apply(db, plan, payload)
+    await db.commit()
+    if was_active:
+        await audit_service.persist(kind="rollout", action="deactivate", status="completed", entity_id=str(plan.id),
+                                    entity_name=plan.name, message="Plan geändert: automatischer Start deaktiviert")
+    return await read_plan(db, await _plan_or_404(db, plan_id))
+
+
+@router.delete("/plans/{plan_id}", status_code=204)
+async def delete_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = await _plan_or_404(db, plan_id)
+    if await db.scalar(select(RolloutRun.id).where(RolloutRun.plan_id == plan.id, RolloutRun.status.in_(ACTIVE_RUN)).limit(1)):
+        raise api_error(409, ErrorCode.ROLLOUT_PLAN_BUSY)
+    await db.delete(plan)
+    await db.commit()
+
+
+@router.post("/plans/{plan_id}/preview")
+async def preview_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    plan = await _plan_or_404(db, plan_id)
+    resolved = await rollout_service.resolve(db, plan)
+    return {**resolved, "preview_token": rollout_service.fingerprint(plan, resolved),
+            "can_start": not resolved["errors"], "plan": await read_plan(db, plan, resolved)}
+
+
+async def _checked_token(db: AsyncSession, plan: RolloutPlan, token: str) -> tuple[dict, str]:
+    resolved = await rollout_service.resolve(db, plan)
+    current = rollout_service.fingerprint(plan, resolved)
+    if token != current:
+        raise api_error(409, ErrorCode.ROLLOUT_PREVIEW_STALE)
+    if resolved["errors"]:
+        raise api_error(422, ErrorCode.ROLLOUT_PLAN_INVALID, reason=resolved["errors"][0]["code"])
+    return resolved, current
+
+
+@router.post("/plans/{plan_id}/activate")
+async def activate_plan(plan_id: uuid.UUID, payload: TokenIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = await _plan_or_404(db, plan_id)
+    if not plan.auto_start:
+        raise api_error(422, ErrorCode.ROLLOUT_INVALID, field="auto_start")
+    _resolved, current = await _checked_token(db, plan, payload.preview_token)
+    plan.activated_by, plan.activated_at, plan.activated_fingerprint = request.state.user.username, datetime.now(timezone.utc), current
+    plan.activated_by_id = request.state.user.id
+    plan.auto_start_error = None
+    await db.commit()
+    await audit_service.persist(kind="rollout", action="activate", status="completed", entity_id=str(plan.id),
+                                entity_name=plan.name, details={"fingerprint": current})
+    return await read_plan(db, await _plan_or_404(db, plan_id))
+
+
+@router.post("/plans/{plan_id}/deactivate")
+async def deactivate_plan(plan_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = await _plan_or_404(db, plan_id)
+    plan.activated_by = plan.activated_by_id = plan.activated_at = plan.activated_fingerprint = None
+    await db.commit()
+    await audit_service.persist(kind="rollout", action="deactivate", status="completed", entity_id=str(plan.id), entity_name=plan.name)
+    return await read_plan(db, await _plan_or_404(db, plan_id))
+
+
+# ── runs ─────────────────────────────────────────────────────────────────────
+
+class StartIn(TokenIn):
+    confirm_name: str = Field(min_length=1, max_length=120)
+    acknowledge_unattended: Literal[True]
+
+
+class ReasonIn(BaseModel):
+    reason: str = Field(default="", max_length=500)
+
+
+class SkipIn(BaseModel):
+    host_id: uuid.UUID
+    reason: str = Field(min_length=3, max_length=500)
+
+
+class AbortIn(BaseModel):
+    reason: str = Field(min_length=3, max_length=500)
+    confirm_name: str = Field(min_length=1, max_length=120)
+
+
+def _read_run_summary(run: RolloutRun) -> dict:
+    counts: dict[str, int] = {}
+    for h in run.hosts:
+        counts[h.status] = counts.get(h.status, 0) + 1
+    return {
+        "id": str(run.id), "plan_id": str(run.plan_id) if run.plan_id else None, "plan_name": run.plan_name,
+        "action": run.action, "status": run.status, "paused_from": run.paused_from, "current_wave": run.current_wave,
+        "wave_count": len(run.waves), "current_wave_label": next((w.label for w in run.waves if w.index == run.current_wave), None),
+        "target_version": run.target_version, "pause_reason": run.pause_reason, "pause_detail": run.pause_detail,
+        "started_by": run.started_by, "trigger": run.trigger, "window_occurrence": run.window_occurrence,
+        "created_at": run.created_at, "finished_at": run.finished_at, "step_at": run.step_at, "host_counts": counts,
+        "active": run.status in rollout_engine.ACTIVE,
+    }
+
+
+async def read_run(db: AsyncSession, run: RolloutRun) -> dict:
+    scheds = {}
+    ids = [h.scheduled_maintenance_id for h in run.hosts if h.scheduled_maintenance_id]
+    if ids:
+        scheds = {s.id: s for s in (await db.scalars(select(ScheduledMaintenance).where(ScheduledMaintenance.id.in_(ids)))).all()}
+    soak_until = next((w.soak_until for w in run.waves if w.index == run.current_wave), None)
+    return {
+        **_read_run_summary(run),
+        "soak_until": soak_until, "progress": {k: v for k, v in (run.progress or {}).items()
+                                               if k in ("gate_recheck_at", "cluster_recheck_at", "blocked_since", "resume_gate",
+                                                        "cluster_gate", "preflight_started", "resume_recheck_at")},
+        "snapshot": run.plan_snapshot,
+        "waves": [{"index": w.index, "label": w.label, "status": w.status, "gate_result": w.gate_result,
+                   "gate_attempts": w.gate_attempts, "soak_until": w.soak_until, "approved_by": w.approved_by,
+                   "started_at": w.started_at, "finished_at": w.finished_at,
+                   "spec": run.plan_snapshot["waves"][w.index] if w.index < len(run.plan_snapshot.get("waves", [])) else None}
+                  for w in run.waves],
+        "hosts": [{"id": str(h.id), "host_id": str(h.host_id) if h.host_id else None, "host_name": h.host_name,
+                   "cluster_name": h.cluster_name, "wave_index": h.wave_index, "order": h.order, "status": h.status,
+                   "version_before": h.version_before, "version_after": h.version_after, "detail": h.detail,
+                   "started_at": h.started_at, "finished_at": h.finished_at,
+                   "schedule": ({"id": str(s.id), "status": s.status, "phase": s.phase, "carp_handover": s.carp_handover,
+                                 "host_operation_id": str(s.host_operation_id) if s.host_operation_id else None}
+                                if (s := scheds.get(h.scheduled_maintenance_id)) else None)}
+                  for h in sorted(run.hosts, key=lambda h: (h.wave_index, h.order, h.host_name))],
+    }
+
+
+async def _cancel_if_pending(db: AsyncSession, sched_id) -> bool:
+    """Cancel a host's schedule only if it has not fired. Locked: the poll loop may be firing it right now
+    (fire_due holds the row), and a blind write would turn a running action into "cancelled"."""
+    if sched_id is None:
+        return False
+    sched = await db.scalar(select(ScheduledMaintenance).where(ScheduledMaintenance.id == sched_id)
+                            .with_for_update().execution_options(populate_existing=True))
+    if sched is not None and sched.status == "pending":
+        sched.status = "cancelled"
+        return True
+    return False
+
+
+async def _run_or_404(db: AsyncSession, run_id: uuid.UUID, *, lock: bool = False) -> RolloutRun:
+    stmt = select(RolloutRun).where(RolloutRun.id == run_id).execution_options(populate_existing=True)
+    if lock:
+        stmt = stmt.with_for_update()
+    run = await db.scalar(stmt)
+    if run is None:
+        raise api_error(404, ErrorCode.ROLLOUT_RUN_NOT_FOUND)
+    return run
+
+
+def _not_now(run: RolloutRun):
+    return api_error(409, ErrorCode.ROLLOUT_ACTION_NOT_ALLOWED, status=run.status)
+
+
+async def _done(db: AsyncSession, run: RolloutRun, action: str, status: str, request: Request, **details) -> dict:
+    run.revision += 1
+    await db.commit()
+    await audit_service.persist(kind="rollout", action=action, status=status, entity_id=str(run.id), entity_name=run.plan_name,
+                                details={"wave": run.current_wave, **details})
+    return await read_run(db, await _run_or_404(db, run.id))
+
+
+@router.post("/plans/{plan_id}/start", status_code=201)
+async def start_plan(plan_id: uuid.UUID, payload: StartIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    plan = await _plan_or_404(db, plan_id)
+    if payload.confirm_name != plan.name:
+        raise api_error(409, ErrorCode.ROLLOUT_CONFIRM_NAME)
+    resolved, _token = await _checked_token(db, plan, payload.preview_token)
+    if (other := await rollout_engine.active_run(db)) is not None:
+        # One run at a time: two runs could otherwise touch the same cluster at once.
+        raise api_error(409, ErrorCode.ROLLOUT_ALREADY_ACTIVE, plan=other.plan_name)
+    run = await rollout_engine.create_run(db, plan, resolved, started_by=request.state.user.username,
+                                          started_by_id=request.state.user.id, trigger="manual")
+    await db.commit()
+    await audit_service.persist(kind="rollout", action="start", status="started", entity_id=str(run.id), entity_name=plan.name,
+                                details={"trigger": "manual", "hosts": sum(len(w["hosts"]) for w in resolved["waves"]),
+                                         "waves": [w["label"] for w in resolved["waves"]]})
+    return await read_run(db, await _run_or_404(db, run.id))
+
+
+@router.get("/runs")
+async def list_runs(db: AsyncSession = Depends(get_db)):
+    runs = (await db.scalars(select(RolloutRun).order_by(RolloutRun.created_at.desc()).limit(50))).all()
+    return [_read_run_summary(r) for r in runs]
+
+
+@router.get("/status")
+async def rollout_status(db: AsyncSession = Depends(get_db)):
+    """For /updates and the host pages: the active run and the hosts it still owns."""
+    run = await rollout_engine.active_run(db)
+    if run is None:
+        return {"active": None, "host_ids": [], "current_host_ids": []}
+    return {
+        "active": _read_run_summary(run),
+        "host_ids": [str(h.host_id) for h in run.hosts if h.host_id and h.status in ("queued", "blocked", "running", "verifying", "failed")],
+        "current_host_ids": [str(h.host_id) for h in run.hosts if h.host_id and h.wave_index == run.current_wave
+                             and h.status in ("queued", "blocked", "running", "verifying", "failed")],
+    }
+
+
+@router.get("/runs/{run_id}")
+async def get_run(run_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
+    return await read_run(db, await _run_or_404(db, run_id))
+
+
+@router.post("/runs/{run_id}/pause")
+async def pause_run(run_id: uuid.UUID, payload: ReasonIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    run = await _run_or_404(db, run_id, lock=True)
+    if run.status not in rollout_engine.ACTIVE or run.status == "paused":
+        raise _not_now(run)
+    # A brake that lets the next host fire anyway is no brake: a step not fired yet goes back to the queue.
+    requeued = [h.host_name for h in run.hosts if await rollout_engine.requeue_if_pending(db, h)]
+    rollout_engine.pause(run, "manual", by=request.state.user.username, text=payload.reason.strip()[:300])
+    return await _done(db, run, "pause", "paused", request, reason="manual", text=payload.reason.strip()[:300], requeued=requeued)
+
+
+@router.post("/runs/{run_id}/resume")
+async def resume_run(run_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    run = await _run_or_404(db, run_id, lock=True)
+    if run.status != "paused":
+        raise _not_now(run)
+    failed = [h.host_name for h in run.hosts if h.status == "failed"]
+    if failed:
+        # A failed host is never retried by itself: check it, then skip it explicitly.
+        raise api_error(409, ErrorCode.ROLLOUT_HOST_FAILED, hosts=", ".join(failed))
+    previous = run.started_by
+    # Whoever resumes takes over the responsibility (and the permission check) from here on.
+    run.started_by, run.started_by_id = request.state.user.username, request.state.user.id
+    run.status = "resuming"
+    run.pause_reason = run.pause_detail = None
+    run.progress = {**(run.progress or {}), "resume_attempts": 0, "resume_recheck_at": None, "paused_at": None}
+    return await _done(db, run, "resume", "resuming", request, previous_owner=previous)
+
+
+@router.post("/runs/{run_id}/approve")
+async def approve_wave(run_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    run = await _run_or_404(db, run_id, lock=True)
+    if run.status != "awaiting_approval":
+        raise _not_now(run)
+    wave = rollout_engine.wave_row(run, run.current_wave)
+    wave.approved_by, wave.status = request.state.user.username, "preflight"
+    run.started_by, run.started_by_id = request.state.user.username, request.state.user.id
+    run.status = "preflight"
+    run.progress = {**(run.progress or {}), "preflight_started": None, "preflight_requested_at": None}
+    return await _done(db, run, "approve", "approved", request, wave_label=wave.label)
+
+
+@router.post("/runs/{run_id}/skip-host")
+async def skip_host(run_id: uuid.UUID, payload: SkipIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    run = await _run_or_404(db, run_id, lock=True)
+    if run.status not in rollout_engine.ACTIVE:
+        raise _not_now(run)
+    host = next((h for h in run.hosts if h.host_id == payload.host_id), None)
+    if host is None:
+        raise api_error(404, ErrorCode.ROLLOUT_HOST_NOT_IN_RUN)
+    if host.status in ("running", "verifying"):
+        if not await _cancel_if_pending(db, host.scheduled_maintenance_id):
+            raise api_error(409, ErrorCode.ROLLOUT_HOST_RUNNING, host=host.host_name)
+        # not fired yet: nothing happened on the firewall
+    elif host.status not in ("queued", "blocked", "failed"):
+        raise api_error(409, ErrorCode.ROLLOUT_HOST_RUNNING, host=host.host_name)
+    host.status = "skipped"
+    host.detail = {"code": "skipped_by", "params": {"user": request.state.user.username, "reason": payload.reason.strip()[:300]}}
+    host.finished_at = datetime.now(timezone.utc)
+    return await _done(db, run, "skip_host", "skipped", request, host=host.host_name, reason=payload.reason.strip()[:300])
+
+
+@router.post("/runs/{run_id}/abort")
+async def abort_run(run_id: uuid.UUID, payload: AbortIn, request: Request, db: AsyncSession = Depends(get_db)):
+    await policy_lock(db)
+    run = await _run_or_404(db, run_id, lock=True)
+    if run.status not in rollout_engine.ACTIVE:
+        raise _not_now(run)
+    if payload.confirm_name != run.plan_name:
+        raise api_error(409, ErrorCode.ROLLOUT_CONFIRM_NAME)
+    now = datetime.now(timezone.utc)
+    still_running = []
+    for host in run.hosts:
+        if host.status in ("running", "verifying") and host.scheduled_maintenance_id:
+            if await _cancel_if_pending(db, host.scheduled_maintenance_id):
+                host.status, host.finished_at = "cancelled", now
+            else:
+                still_running.append(host.host_name)  # an action already on the firewall is finished in a controlled way
+        elif host.status in ("queued", "blocked"):
+            host.status, host.finished_at = "cancelled", now
+    for wave in run.waves:
+        if wave.status not in ("passed", "skipped"):
+            wave.status = "cancelled"
+    run.status, run.finished_at = "cancelled", now
+    run.pause_reason, run.pause_detail = "aborted", {"by": request.state.user.username, "reason": payload.reason.strip()[:300],
+                                                     "still_running": still_running}
+    return await _done(db, run, "abort", "cancelled", request, reason=payload.reason.strip()[:300], still_running=still_running)

backend/app/routers/rule_analysis.py

@@ -0,0 +1,128 @@
+"""Rule analysis: which rule decides a packet (read-only, GET so ``read`` tokens and MCP can use it).
+
+Nothing is stored and nothing is written to a firewall; every request reads the rules, the
+loaded pf ruleset and the interfaces (cached for 60 s per firewall, ``refresh=true`` skips
+the cache) and evaluates them with services/rule_analysis/engine.py.
+"""
+import asyncio
+import logging
+import uuid
+
+from fastapi import APIRouter, Depends, HTTPException, Query
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.orm import selectinload
+
+from app.database import get_db
+from app.errors import ErrorCode, api_error
+from app.models.host import OpnsenseHost
+from app.models.policy import DeviceGroup
+from app.models.rule_history import LiveRuleState
+from app.services import rule_history_service
+from app.services.rule_analysis import service
+
+logger = logging.getLogger(__name__)
+router = APIRouter(tags=["rule-analysis"])
+_PARALLEL = 5
+
+
+def _query(src: str, dst: str, proto: str, sport: str | None, dport: str | None, icmptype: str | None,
+           iface: str | None, direction: str, egress: str | None):
+    return service.build_query(src=src, dst=dst, proto=proto, sport=sport, dport=dport, icmptype=icmptype,
+                               iface=iface, direction=direction, egress=egress)
+
+
+async def _run(host: OpnsenseHost, q, refresh: bool) -> dict:
+    head = {"host_id": str(host.id), "host_name": host.name, "cluster_name": host.cluster_name}
+    try:
+        return {**head, **await service.analyze_host(host, q, refresh=refresh), "error": None}
+    except HTTPException as exc:
+        return {**head, "error": exc.detail}
+    except Exception as exc:  # noqa: BLE001 - one unreachable firewall must not fail the fleet answer
+        logger.info("rule analysis failed for %s: %s", host.name, exc)
+        return {**head, "error": {"code": ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR.value, "params": {}}}
+
+
+@router.get("/hosts/{host_id}/rule-analysis/match")
+async def host_match(
+    host_id: uuid.UUID, src: str, dst: str, proto: str = "tcp", sport: str | None = None, dport: str | None = None,
+    icmptype: str | None = None, iface: str | None = None, direction: str = "in", egress: str | None = None,
+    refresh: bool = False, db: AsyncSession = Depends(get_db),
+):
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    q = _query(src, dst, proto, sport, dport, icmptype, iface, direction, egress)
+    result = await _run(host, q, refresh)
+    error = result.get("error")
+    if error:
+        status = 422 if error.get("code") == ErrorCode.ANALYSE_INTERFACE_UNBEKANNT.value else 502
+        raise HTTPException(status_code=status, detail=error)
+    return result
+
+
+@router.get("/rule-analysis/match")
+async def fleet_match(
+    src: str, dst: str, proto: str = "tcp", sport: str | None = None, dport: str | None = None,
+    icmptype: str | None = None, iface: str | None = None, direction: str = "in", egress: str | None = None,
+    host_ids: str | None = Query(None, description="comma separated host ids; default: all enabled hosts"),
+    group_id: uuid.UUID | None = None, refresh: bool = False, db: AsyncSession = Depends(get_db),
+):
+    q = _query(src, dst, proto, sport, dport, icmptype, iface, direction, egress)
+    if group_id is not None:
+        group = await db.scalar(select(DeviceGroup).options(selectinload(DeviceGroup.hosts)).where(DeviceGroup.id == group_id))
+        if group is None:
+            raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
+        hosts = sorted(group.hosts, key=lambda h: h.name)
+    elif host_ids:
+        try:
+            ids = [uuid.UUID(part.strip()) for part in host_ids.split(",") if part.strip()]
+        except ValueError:
+            raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="host_ids", value=host_ids[:64]) from None
+        hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(ids)).order_by(OpnsenseHost.name))).all())
+        if len(hosts) != len(set(ids)):
+            raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    else:
+        hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all())
+    gate = asyncio.Semaphore(_PARALLEL)
+
+    async def one(host):
+        async with gate:
+            return await _run(host, q, refresh)
+
+    return list(await asyncio.gather(*(one(h) for h in hosts)))
+
+
+@router.get("/hosts/{host_id}/rule-analysis/cleanup")
+async def host_cleanup(
+    host_id: uuid.UUID, unused_days: int = Query(30, ge=1, le=3650, description="observation window for 'unused' / 'disabled for a long time'"),
+    refresh: bool = False, db: AsyncSession = Depends(get_db),
+):
+    """Cleanup candidates (unused, shadowed, duplicate, disabled for a long time) with confidence and whether
+    each rule could be offered for removal. Read-only; removal is not part of this endpoint."""
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    try:
+        return await service.cleanup_candidates(db, host, unused_days=unused_days, refresh=refresh)
+    except HTTPException:
+        raise
+    except Exception as exc:  # noqa: BLE001 - an unreachable firewall is a 502, not a crash
+        logger.info("cleanup candidates failed for %s: %s", host.name, exc)
+        raise api_error(502, ErrorCode.ANALYSE_FIREWALL_NICHT_LESBAR) from exc
+
+
+@router.get("/hosts/{host_id}/live-rules/{rule_uuid}/history")
+async def live_rule_history(host_id: uuid.UUID, rule_uuid: str, limit: int = Query(100, ge=1, le=500),
+                            db: AsyncSession = Depends(get_db)):
+    """What happened to one rule on one firewall: observed changes (also hand-made ones), saved revisions of
+    rules OPNexus manages and the audit entries, newest first. Reads the database only."""
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None:
+        raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
+    baseline = await db.get(LiveRuleState, (host_id, rule_history_service.BASELINE_KEY))
+    return {
+        "host_id": str(host.id), "rule_uuid": rule_uuid.lower()[:64],
+        "watching_since": baseline.first_seen_at if baseline else None,
+        "events": await rule_history_service.rule_timeline(db, host, rule_uuid, limit),
+    }

backend/app/routers/rules.py

@@ -1,3 +1,4 @@
+import json
 import uuid
 
 from fastapi import APIRouter, Depends, Body
@@ -18,6 +19,7 @@ from app.services.interface_service import check_interface_consistency
 from app.services.opnsense_client import OpnsenseAPIError
 from app.services.rule_deploy_service import confirm_rule, deploy_rule, undeploy_rule, revert_deployment
 from app.services import rule_adopt_service
+from app.services.adopt_common import LINK_ERRORS, link_api_error
 from app.schemas.rule import AdoptRequest, UnmanagedRule
 
 router = APIRouter(prefix="/rules", tags=["rules"])
@@ -107,11 +109,14 @@ async def adopt_rule(payload: AdoptRequest, db: AsyncSession = Depends(get_db)):
     if not host:
         raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
     try:
-        rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid)
+        rule = await rule_adopt_service.adopt(db, host, payload.rule_uuid, link_to=payload.link_to)
     except rule_adopt_service.RuleNotFoundOnHost as exc:
         raise api_error(404, ErrorCode.REGEL_NICHT_GEFUNDEN) from exc
     except rule_adopt_service.NotAdoptable as exc:
-        raise api_error(409, ErrorCode.RULE_NOT_ADOPTABLE, reasons=", ".join(exc.reasons)) from exc
+        raise api_error(409, ErrorCode.RULE_NOT_ADOPTABLE, reasons=", ".join(exc.reasons),
+                        reason_details=json.dumps(exc.details, ensure_ascii=False)) from exc
+    except LINK_ERRORS as exc:
+        raise link_api_error(exc) from exc
     except OpnsenseAPIError as exc:
         raise api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name) from exc
     await db.commit()

backend/app/routers/scheduled_maintenance.py

@@ -24,6 +24,8 @@ class Schedule(BaseModel):
     acknowledge_unattended: Literal[True]
     # Hand the traffic over to the CARP partner first and switch back afterwards (cluster nodes only).
     carp_handover: bool = False
+    # Check the result automatically afterwards and close the operation (no manual "Wartung abschließen").
+    auto_verify: bool = False
 
 
 def _read(sched: ScheduledMaintenance) -> dict:
@@ -32,6 +34,8 @@ def _read(sched: ScheduledMaintenance) -> dict:
         "status": sched.status, "actor": sched.actor, "created_at": sched.created_at,
         "result": sched.result, "host_operation_id": sched.host_operation_id,
         "carp_handover": sched.carp_handover, "phase": sched.phase,
+        "auto_verify": sched.auto_verify, "origin": sched.origin,
+        "rollout_run_id": ((sched.progress or {}).get("rollout") or {}).get("run_id"),
     }
 
 
@@ -80,6 +84,7 @@ async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Reque
     sched = ScheduledMaintenance(
         host_id=host.id, host_name=host.name, action=payload.action, run_at=run_at,
         actor=request.state.user.username, created_at=now, carp_handover=payload.carp_handover,
+        auto_verify=payload.auto_verify,
     )
     db.add(sched)
     await db.commit()
@@ -90,7 +95,9 @@ async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Reque
 async def cancel_scheduled(host_id: uuid.UUID, schedule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
     await policy_lock(db)
     await _host_or_404(db, host_id)
-    sched = await db.get(ScheduledMaintenance, schedule_id)
+    # Locked and re-read: the poll loop may be firing this very row right now (it holds the lock meanwhile).
+    sched = await db.scalar(select(ScheduledMaintenance).where(ScheduledMaintenance.id == schedule_id)
+                            .with_for_update().execution_options(populate_existing=True))
     if not sched or sched.host_id != host_id:
         raise api_error(404, ErrorCode.GEPLANTE_WARTUNG_NICHT_GEFUNDEN)
     if sched.status != "pending":

backend/app/routers/system.py

@@ -7,6 +7,7 @@ from app.database import get_db
 from app.errors import ErrorCode, api_error
 from app.models.user import AdminUser
 from app.routers.auth import current_user
+from app.services import rollout_engine
 from app.services import update_check_service, updater_client
 
 router = APIRouter(prefix="/system", tags=["system"])
@@ -99,11 +100,17 @@ async def updater_status(db: AsyncSession = Depends(get_db), user: AdminUser = D
         "can_update": False, "major_change": False,
         "job": None, "phases": [], "log": [], "services": {},
         "rollback_available": False, "rollback_loses_data": False, "rollback_version": None,
+        # A restart in the middle of a firmware rollout is safe (its state is in the database), but the
+        # UI warns: the run pauses its 15-second steps for as long as OPNexus is down.
+        "active_rollouts": await rollout_engine.count_active(db),
     }
     try:
         updater = await updater_client.status()
     except updater_client.UpdaterUnavailable as exc:
         result["reason"] = str(exc)
+        # Without the updater the install mode is still known from the image registry setting; the UI uses it
+        # to not offer enabling the updater on an install that builds from source (it could not update anyway).
+        result["mode"] = "release" if settings.opnexus_image_repo.strip() else "local"
         return result
     except updater_client.UpdaterRejected:
         result["reason"] = "unreachable"

backend/app/schemas/adopt.py

@@ -0,0 +1,76 @@
+import uuid
+
+from pydantic import BaseModel, Field
+
+
+class AdoptObjectRequest(BaseModel):
+    host_id: uuid.UUID
+    # Link to this existing central object instead of creating a new one (services/adopt_link_service.py).
+    # Declared before `uuid`: that field's name shadows the module inside the class body.
+    link_to: uuid.UUID | None = None
+    uuid: str = Field(min_length=1, max_length=64)  # the object's UUID on the firewall
+
+
+class AdoptAllRequest(BaseModel):
+    host_id: uuid.UUID
+    # Link entries that have exactly one exact central twin instead of creating a duplicate.
+    link_existing: bool = False
+
+
+class LinkDiff(BaseModel):
+    field: str  # model field ("content", "destination_port", ...) or "variables" (a {{VARIABLE}} has no value on this host)
+    central: str
+    live: str
+
+
+class LinkTarget(BaseModel):
+    """An existing central object that a live object could be linked to (exact match) or is similar to."""
+
+    id: uuid.UUID
+    name: str
+    targets: list[str]  # names of the hosts it is deployed to
+    diff: list[LinkDiff] = Field(default_factory=list)  # empty for an exact match
+
+
+class UnmanagedObject(BaseModel):
+    """An alias / NAT rule found on a host that OPNexus does not manage yet (see object_adopt_service.py)."""
+
+    uuid: str
+    title: str
+    enabled: bool
+    adoptable: bool  # can be adopted on its own right now
+    # Adoptable in the same "adopt all" run although not on its own (an alias whose referenced aliases come first).
+    bulk_adoptable: bool
+    # Codes such as "unsupported_type" or "no_target"; the UI turns them into text.
+    reasons: list[str]
+    reason_details: dict[str, str] = Field(default_factory=dict)  # code -> extra text (alias names, type, ...)
+    # Settings OPNexus does not manage that stay untouched on the firewall (log, pool options, tags, ...).
+    kept: list[str] = Field(default_factory=list)
+    # Facts for display, depend on the object type.
+    info: dict[str, str] = Field(default_factory=dict)
+    # Central objects whose content, resolved for this host, equals the live object exactly (first = preferred).
+    matches: list[LinkTarget] = Field(default_factory=list)
+    # Objects with the same name/description but different content: a hint with the differing fields, never linked.
+    similar: list[LinkTarget] = Field(default_factory=list)
+    linkable: bool = False  # can be linked to `matches[0]` (or another match) on its own
+    # What "adopt all" would do: "link" | "new" | "skip" (None only on the adopt-all result).
+    bulk_action: str | None = None
+
+
+class AdoptedObject(BaseModel):
+    uuid: str
+    title: str
+    id: uuid.UUID
+    linked: bool = False  # linked to an existing central object instead of newly created
+
+
+class SkippedObject(BaseModel):
+    uuid: str
+    title: str
+    reasons: list[str]
+    reason_details: dict[str, str] = Field(default_factory=dict)
+
+
+class AdoptAllResult(BaseModel):
+    adopted: list[AdoptedObject]
+    skipped: list[SkippedObject]

backend/app/schemas/dashboard.py

@@ -3,6 +3,8 @@ from datetime import datetime
 
 from pydantic import BaseModel
 
+from app.schemas.gateway import GatewayHealth
+
 
 class DashboardHost(BaseModel):
     id: uuid.UUID
@@ -10,6 +12,11 @@ class DashboardHost(BaseModel):
     status: str
     carp_status: str | None
     cluster_name: str | None
+    # From the last resource sample (no live call); None when there is no sample from the last 3 minutes.
+    gateways: list[GatewayHealth] | None = None
+    # Readings the newest fresh sample could not take because the API key lacks the right (gateways, services, ...);
+    # None = no fresh sample (or one from before this existed), [] = nothing missing.
+    api_rights_missing: list[str] | None = None
 
 
 class PendingAction(BaseModel):

backend/app/schemas/gateway.py

@@ -0,0 +1,33 @@
+from typing import Literal
+
+from pydantic import BaseModel
+
+# online: all fine · degraded: OPNsense flags loss and/or latency above the gateway's own limits ·
+# offline: down · forced_down: switched off by the admin on purpose · unknown: nothing measured
+# (monitoring disabled) or the verdict is still pending.
+GatewayState = Literal["online", "degraded", "offline", "forced_down", "unknown"]
+
+
+class GatewayHealth(BaseModel):
+    name: str
+    address: str | None = None
+    status: str                  # OPNsense' raw verdict: none | down | force_down | delay | loss | delay+loss | …
+    state: GatewayState
+    loss: float | None = None    # percent
+    delay: float | None = None   # milliseconds
+    stddev: float | None = None  # milliseconds
+    monitor: str | None = None   # the monitored address
+    alert_enabled: bool = True   # False: the gateway alert is switched off for this gateway (Settings, 1.29)
+
+
+class GatewayAlertUpdate(BaseModel):
+    enabled: bool
+
+
+class GatewayAlertState(BaseModel):
+    name: str
+    alert_enabled: bool
+
+
+class ApiRightsState(BaseModel):
+    missing: list[str] | None = None  # names of the refused readings (gateways, services, swap, cpu); None = unknown

backend/app/schemas/notification.py

@@ -1,10 +1,12 @@
+import re
 import uuid
 from datetime import datetime
 from typing import Literal
 
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import BaseModel, ConfigDict, Field, field_validator
 
 from app.config import settings
+from app.models.notification import DEFAULT_SERVICE_WATCHLIST
 
 SmtpSecurity = Literal["none", "starttls", "ssl"]
 ProblemType = Literal[
@@ -14,7 +16,10 @@ ProblemType = Literal[
     "backup_stale",
     "backup_offsite_failed",
     "compliance_finding",
+    "gateway_down", "resource_threshold", "service_down", "api_rights_missing",
+    "rollout_paused", "rollout_start_failed",
 ]
+ChannelType = Literal["ntfy", "gotify", "telegram", "teams", "matrix"]
 
 
 class NotificationSettingsRead(BaseModel):
@@ -45,10 +50,30 @@ class NotificationSettingsRead(BaseModel):
     scheduled_maintenance_failed_enabled: bool = True
     backup_failed_enabled: bool = True
     compliance_alerts_enabled: bool = False
+    gateway_down_enabled: bool = True
+    resource_threshold_enabled: bool = False
+    service_down_enabled: bool = False
+    rollout_problems_enabled: bool = True
+    api_rights_enabled: bool = True
+    gateway_alert_muted: list[str] = Field(default_factory=list)
+    gateway_loss_threshold_pct: int = 20
+    gateway_delay_threshold_ms: int = 500
+    disk_threshold_pct: int = 90
+    memory_threshold_pct: int = 95
+    swap_threshold_pct: int = 50
+    load_per_cpu_threshold: float = 2.0
+    alert_confirm_samples: int = 3
+    resource_confirm_samples: int = 5
+    service_watchlist: list[str] = Field(default_factory=lambda: list(DEFAULT_SERVICE_WATCHLIST))
 
     poll_interval_seconds: int
     updated_at: datetime
 
+    @field_validator("service_watchlist", mode="before")
+    @classmethod
+    def _default_watchlist(cls, value):
+        return list(DEFAULT_SERVICE_WATCHLIST) if value is None else value
+
 
 class NotificationSettingsUpdate(BaseModel):
     """Partial update — everything optional, only provided fields change.
@@ -84,6 +109,35 @@ class NotificationSettingsUpdate(BaseModel):
     scheduled_maintenance_failed_enabled: bool | None = None
     backup_failed_enabled: bool | None = None
     compliance_alerts_enabled: bool | None = None
+    gateway_down_enabled: bool | None = None
+    resource_threshold_enabled: bool | None = None
+    service_down_enabled: bool | None = None
+    rollout_problems_enabled: bool | None = None
+    api_rights_enabled: bool | None = None
+    # 0 switches that single threshold off.
+    gateway_loss_threshold_pct: int | None = Field(default=None, ge=0, le=100)
+    gateway_delay_threshold_ms: int | None = Field(default=None, ge=0, le=60000)
+    disk_threshold_pct: int | None = Field(default=None, ge=0, le=100)
+    memory_threshold_pct: int | None = Field(default=None, ge=0, le=100)
+    swap_threshold_pct: int | None = Field(default=None, ge=0, le=100)
+    load_per_cpu_threshold: float | None = Field(default=None, ge=0, le=1000)
+    alert_confirm_samples: int | None = Field(default=None, ge=1, le=60)
+    resource_confirm_samples: int | None = Field(default=None, ge=1, le=60)
+    service_watchlist: list[str] | None = Field(default=None, max_length=100)
+
+    @field_validator("service_watchlist")
+    @classmethod
+    def _clean_watchlist(cls, value):
+        if value is None:
+            return None
+        names = []
+        for raw in value:
+            name = str(raw).strip()
+            if not re.fullmatch(r"[A-Za-z0-9._+-]{1,64}", name):
+                raise ValueError("Dienstnamen bestehen aus Buchstaben, Ziffern und . _ + -")
+            if name not in names:
+                names.append(name)
+        return names
 
     poll_interval_seconds: int | None = Field(
         default=None, ge=settings.notification_poll_min_interval_seconds
@@ -106,3 +160,55 @@ class NotificationProblemRead(BaseModel):
 
 class TestNotificationRequest(BaseModel):
     channel: Literal["webhook", "email"]
+
+
+class NotificationChannelRead(BaseModel):
+    """Never carries the credential; ``config`` is withheld from non-admins (an ntfy topic is a password)."""
+
+    model_config = ConfigDict(from_attributes=True)
+
+    id: uuid.UUID
+    name: str
+    channel_type: ChannelType
+    enabled: bool
+    config: dict
+    secret_set: bool
+    created_at: datetime
+    updated_at: datetime
+
+
+def _channel_name(value: str | None) -> str | None:
+    if value is None:
+        return None
+    if not value.strip():
+        raise ValueError("Name darf nicht leer sein")
+    return value.strip()
+
+
+class NotificationChannelCreate(BaseModel):
+    name: str = Field(min_length=1, max_length=100)
+    channel_type: ChannelType
+    enabled: bool = True
+    config: dict = Field(default_factory=dict)
+    # Token, or for Teams the webhook address. Write-only.
+    secret: str | None = Field(default=None, max_length=2100)
+
+    @field_validator("name")
+    @classmethod
+    def _clean_name(cls, value):
+        return _channel_name(value)
+
+
+class NotificationChannelUpdate(BaseModel):
+    """Partial: omitted fields stay. ``secret`` replaces the credential, ``clear_secret`` removes it."""
+
+    name: str | None = Field(default=None, min_length=1, max_length=100)
+    enabled: bool | None = None
+    config: dict | None = None
+    secret: str | None = Field(default=None, max_length=2100)
+    clear_secret: bool = False
+
+    @field_validator("name")
+    @classmethod
+    def _clean_name(cls, value):
+        return _channel_name(value)

backend/app/schemas/rule.py

@@ -4,6 +4,8 @@ from typing import Literal
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 
+from app.schemas.adopt import LinkTarget
+
 
 class RuleCreate(BaseModel):
     description: str = Field(min_length=1, max_length=255)
@@ -78,6 +80,8 @@ class DeployApproval(BaseModel):
 
 class AdoptRequest(BaseModel):
     host_id: uuid.UUID
+    # Link to this existing central rule instead of creating a new one (services/adopt_link_service.py).
+    link_to: uuid.UUID | None = None
     rule_uuid: str = Field(min_length=1, max_length=64)
 
 
@@ -98,3 +102,7 @@ class UnmanagedRule(BaseModel):
     adoptable: bool
     # Codes such as "negated_source" or "not_quick"; the UI turns them into text.
     reasons: list[str]
+    # Central rules equal to this one (resolved for the host), same-description rules that differ, link possible.
+    matches: list[LinkTarget] = Field(default_factory=list)
+    similar: list[LinkTarget] = Field(default_factory=list)
+    linkable: bool = False

backend/app/security_middleware.py

@@ -7,9 +7,19 @@ how the proxy setup works for IP *and* name access) nor an explicitly allowed
 origin. Requests without either header (curl, scripts) are not browsers, so
 they cannot be riding on a victim's cookie and pass through to normal auth.
 
+Sec-Fetch-Site (1.27): a page on another port of the same host (e.g. a
+firewall GUI behind the GUI gateway) is *same-site* to OPNexus, so the
+browser attaches the session cookie to its requests and SameSite=Lax does
+not help. Cookie-carrying /api requests that the browser itself marks as
+same-site or cross-site are therefore refused — unless the Origin is an
+explicitly configured frontend origin (development setup with the frontend
+on its own port). Clients that send no Sec-Fetch-Site header (curl,
+scripts, API tokens) are not browsers and stay unaffected.
+
 SecurityHeadersMiddleware: API responses must never be cached by browsers or
 proxies and must not be sniffed or framed.
 """
+import re
 from urllib.parse import urlparse
 
 from starlette.middleware.base import BaseHTTPMiddleware
@@ -18,6 +28,7 @@ from starlette.responses import JSONResponse
 
 from app.config import settings
 from app.errors import ErrorCode
+from app.routers.auth import has_session_cookie
 
 _SAFE_METHODS = {"GET", "HEAD", "OPTIONS"}
 
@@ -35,8 +46,53 @@ def _origin_allowed(origin: str, host_header: str) -> bool:
     return f"{parsed.scheme}://{parsed.netloc}".lower() in {o.lower() for o in settings.allowed_origins}
 
 
+# The SSO provider sends the browser back here by a cross-site navigation;
+# the endpoint authenticates by its own state cookie and creates the session.
+# The exemption is only for exactly that: a top-level GET navigation (a script, image or fetch from another site
+# is still blocked, whatever it carries) — and the callback fails without the matching state cookie anyway.
+_SEC_FETCH_EXEMPT = {"/api/auth/oidc/callback"}
+
+
+def _is_top_level_navigation(request: Request) -> bool:
+    if request.method != "GET":
+        return False
+    mode = request.headers.get("sec-fetch-mode", "").strip().lower()
+    dest = request.headers.get("sec-fetch-dest", "").strip().lower()
+    return mode in ("", "navigate") and dest in ("", "document")
+
+
+def _explicit_frontend_origin(origin: str) -> bool:
+    origin = origin.rstrip("/").lower()
+    if not origin or origin == "null":
+        return False
+    if settings.allowed_origins:
+        return origin in {o.lower() for o in settings.allowed_origins}
+    return re.fullmatch(settings.cors_origin_regex, origin) is not None
+
+
+def sec_fetch_blocked(request: Request) -> bool:
+    path = request.url.path
+    if not (path == "/api" or path.startswith("/api/")):
+        return False
+    if path in _SEC_FETCH_EXEMPT and _is_top_level_navigation(request):
+        return False
+    if not has_session_cookie(request):
+        return False
+    site = request.headers.get("sec-fetch-site", "").strip().lower()
+    if site == "cross-site":
+        return True
+    if site == "same-site":
+        return not _explicit_frontend_origin(request.headers.get("origin", ""))
+    return False  # same-origin, none (typed URL/bookmark) or no header (not a browser)
+
+
 class OriginCheckMiddleware(BaseHTTPMiddleware):
     async def dispatch(self, request: Request, call_next):
+        if sec_fetch_blocked(request):
+            return JSONResponse(
+                status_code=403,
+                content={"detail": {"code": ErrorCode.CROSS_SITE_REQUEST_BLOCKED.value, "params": {}}},
+            )
         if request.method not in _SAFE_METHODS:
             source = request.headers.get("origin") or request.headers.get("referer")
             if source and source != "null" and not _origin_allowed(source, request.headers.get("host", "")):

backend/app/services/adopt_common.py

@@ -0,0 +1,63 @@
+"""Small pieces shared by every "take over what was made by hand on a firewall" feature
+(rule_adopt_service.py for filter rules, object_adopt_service.py for aliases and NAT)."""
+
+
+class NotAdoptable(Exception):
+    """The object exists on the host but cannot be described without distortion.
+
+    `reasons` are machine-readable codes (the UI translates them); `details` maps a code to
+    extra text the translation can show (e.g. the alias names a reference points to)."""
+
+    def __init__(self, reasons: list[str], details: dict[str, str] | None = None):
+        super().__init__(", ".join(reasons))
+        self.reasons = reasons
+        self.details = details or {}
+
+
+class NotFoundOnHost(Exception):
+    """The UUID is not (or no longer) present in the host's live configuration."""
+
+
+def on(value) -> bool:
+    """OPNsense booleans arrive as "1"/"0" strings, sometimes as real booleans."""
+    return value in ("1", 1, True)
+
+
+def clean(value) -> str:
+    return "" if value is None else str(value).strip()
+
+
+class LinkTargetNotFound(Exception):
+    """The central object chosen to link to was deleted in the meantime."""
+
+
+class LinkTargetChanged(Exception):
+    """The central object no longer equals what is live on the host. `diff` lists the differing fields."""
+
+    def __init__(self, diff: list[dict]):
+        super().__init__(", ".join(d["field"] for d in diff))
+        self.diff = diff
+
+
+class HostAlreadyHasObject(Exception):
+    """The central object is already deployed to / managed on this host under another UUID."""
+
+
+class LinkTargetBusy(Exception):
+    """The central object has an unconfirmed deployment or a failed rollback; linking waits until it is settled."""
+
+
+LINK_ERRORS = (LinkTargetNotFound, LinkTargetChanged, HostAlreadyHasObject, LinkTargetBusy)
+
+
+def link_api_error(exc: Exception):
+    """The HTTP error for a failed link (shared by the rule and the object endpoints)."""
+    from app.errors import ErrorCode, api_error
+
+    if isinstance(exc, LinkTargetNotFound):
+        return api_error(404, ErrorCode.ADOPT_LINK_TARGET_NOT_FOUND)
+    if isinstance(exc, LinkTargetChanged):
+        return api_error(409, ErrorCode.ADOPT_LINK_TARGET_CHANGED, fields=", ".join(d["field"] for d in exc.diff))
+    if isinstance(exc, HostAlreadyHasObject):
+        return api_error(409, ErrorCode.ADOPT_LINK_HOST_HAS_OBJECT)
+    return api_error(409, ErrorCode.ADOPT_LINK_BUSY)

backend/app/services/adopt_link_service.py

@@ -0,0 +1,151 @@
+"""Links a hand-made object that was found on a firewall to a central object that already exists, instead
+of creating a second central object for it ("Mit vorhandenem zentralem Objekt verknüpfen").
+
+Typical case: the same rule/alias/NAT entry was made by hand on several firewalls (or on both nodes of an HA
+pair). The first adoption creates the central object; every further firewall is LINKED to it: the host
+becomes another target of that object and gets a deployment row holding the live UUID (remote_uuid) and
+the payload OPNexus would send. Like adopting, linking writes NOTHING to the firewall. The live state is
+read fresh from the host at link time (the browser only names the central object), so the preview right
+afterwards says "unchanged" and a later rollout updates the one object on every host without a duplicate.
+
+A central object is an EXACT match when every field OPNexus manages is equal after the object's
+{{VARIABLE}} placeholders are replaced with this host's values (variables_service.resolve). An object with
+the same identity (alias name / rule description) but different content is only shown as a hint with the
+differing fields and is NEVER linked automatically.
+
+Callers hold policy_lock (same serialisation as adopting), which also makes a repeated link idempotent.
+"""
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from typing import Any, Callable
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.orm import selectinload
+
+from app.models.host import OpnsenseHost
+from app.models.policy import DeviceGroup
+from app.services import variables_service
+from app.services.adopt_common import HostAlreadyHasObject, LinkTargetBusy, LinkTargetChanged, LinkTargetNotFound
+from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
+from app.services.policy_service import effective_hosts
+
+LINK_MESSAGE = "Mit vorhandenem Objekt verknüpft"
+BUSY_STATUSES = ("pending", "revert_failed")  # the central object waits for a confirmation / failed to roll back
+
+
+@dataclass(frozen=True)
+class LinkSpec:
+    model: type
+    deployment: type
+    fk: str  # deployment column pointing at the central object
+    uuid_attr: str  # deployment column holding the OPNsense UUID (remote_uuid)
+    identity: str  # attribute naming the object: "name" (alias) or "description"
+    payload: Callable[[Any, OpnsenseHost], dict]
+    revision: Callable
+
+
+@dataclass
+class Match:
+    obj: Any
+    diff: list[dict]  # empty = exact match
+    busy: bool = False  # the central object waits for a confirmation: linking is refused until that is settled
+
+
+def _norm(value) -> Any:
+    if value is None:
+        return ""
+    return value if isinstance(value, bool) else str(value)
+
+
+def _show(value) -> str:
+    return ("true" if value else "false") if isinstance(value, bool) else ("" if value is None else str(value))
+
+
+def diff_fields(obj, values: dict, host: OpnsenseHost) -> list[dict]:
+    """Fields in which the central object (resolved for `host`) differs from the live `values`."""
+    raw = {key: getattr(obj, key) for key in values}
+    try:
+        central = variables_service.resolve(raw, host)
+    except variables_service.MissingVariables as exc:
+        # Cannot be compared (and could not be deployed to this host either): report, never match.
+        return [{"field": "variables", "central": ", ".join(exc.missing[host.name]), "live": ""}]
+    return [{"field": key, "central": _show(central[key]), "live": _show(values[key])}
+            for key in values if _norm(central[key]) != _norm(values[key])]
+
+
+def _options(spec: LinkSpec):
+    return [
+        selectinload(spec.model.targets),
+        selectinload(spec.model.groups).selectinload(DeviceGroup.hosts),
+        selectinload(spec.model.deployments),
+    ]
+
+
+async def load_central(db: AsyncSession, spec: LinkSpec) -> list:
+    objs = list((await db.scalars(select(spec.model).options(*_options(spec)))).unique().all())
+    await hydrate_hierarchy_targets(db, objs)
+    return objs
+
+
+def find(objs: list, spec: LinkSpec, host: OpnsenseHost, values: dict) -> tuple[list[Match], list[Match]]:
+    """(exact matches, same-identity objects that differ). Objects that already have a deployment on
+    this host are not candidates: linking would collide with that deployment."""
+    exact: list[Match] = []
+    similar: list[Match] = []
+    for obj in objs:
+        if any(d.host_id == host.id for d in obj.deployments):
+            continue
+        if _norm(getattr(obj, spec.identity)) != _norm(values[spec.identity]):
+            continue
+        d = diff_fields(obj, values, host)
+        busy = any(dep.status in BUSY_STATUSES for dep in obj.deployments)
+        (similar if d else exact).append(Match(obj, d, busy))
+    # prefer the object that already reaches the most hosts (the "real" one), then a stable order
+    exact.sort(key=lambda m: (-len(effective_hosts(m.obj)), str(m.obj.id)))
+    return exact, similar
+
+
+def describe(spec: LinkSpec, match: Match) -> dict:
+    return {
+        "id": match.obj.id,
+        "name": getattr(match.obj, spec.identity),
+        "targets": [h.name for h in effective_hosts(match.obj)],
+        "diff": match.diff,
+    }
+
+
+async def link(db: AsyncSession, spec: LinkSpec, host: OpnsenseHost, object_id, remote_uuid: str, values: dict):
+    """Links the live object `remote_uuid` (whose normalised fields are `values`, read fresh by the caller)
+    to the central object `object_id`. Returns it; a repeated call for the same pair changes nothing."""
+    obj = (await db.execute(
+        select(spec.model).where(spec.model.id == object_id).options(*_options(spec))
+        .execution_options(populate_existing=True)
+    )).unique().scalar_one_or_none()
+    if obj is None:
+        raise LinkTargetNotFound(str(object_id))
+    await hydrate_hierarchy_targets(db, [obj])
+
+    existing = next((d for d in obj.deployments if d.host_id == host.id), None)
+    if existing is not None:
+        if getattr(existing, spec.uuid_attr) == remote_uuid:
+            return obj  # already linked: nothing to do
+        raise HostAlreadyHasObject(str(object_id))
+    if any(d.status in BUSY_STATUSES for d in obj.deployments):
+        raise LinkTargetBusy(str(object_id))
+    changed = diff_fields(obj, values, host)
+    if changed:
+        raise LinkTargetChanged(changed)
+
+    if host.id not in {h.id for h in effective_hosts(obj)}:
+        obj.targets.append(host)
+    # an association-only change does not touch the row; stale edit forms must still be refused (see edit_rule)
+    obj.updated_at = datetime.now(timezone.utc)
+    await db.flush()
+    spec.revision(db, obj, f"Mit {host.name} verknüpft")
+    db.add(spec.deployment(**{
+        spec.fk: obj.id, "host_id": host.id, "status": "success", "message": LINK_MESSAGE,
+        spec.uuid_attr: remote_uuid, "last_payload": spec.payload(obj, host), "previous_payload": None,
+    }))
+    await db.flush()
+    return obj

backend/app/services/api_token_service.py

@@ -38,7 +38,8 @@ MAX_TOKENS_PER_USER = 50
 _LAST_USED_GRANULARITY = timedelta(minutes=1)
 
 _SAFE_METHODS = ("GET", "HEAD", "OPTIONS")
-_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)")
+# GUI tickets open a firewall's web interface in the browser: an interactive admin session only.
+_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)|^/api/hosts/[^/]+/gui-ticket$")
 # Reads that expose secrets (config.xml holds password hashes and VPN keys, the
 # notification settings the webhook URL/SMTP data): admin scope only.
 _ADMIN_SCOPE_READS = re.compile(r"^/api/hosts/[^/]+/backups/[^/]+$|^/api/notifications(/|$)|^/api/settings/|^/api/overlay/peers/[^/]+/instructions$")
@@ -48,6 +49,9 @@ _ADMIN_ONLY_WRITES = re.compile(
     r"^/api/(users|settings|system|notifications|overlay)(/|$)"
     r"|^/api/logs/settings$"
     r"|^/api/hosts(/tls-probe|/[^/]+)?$"
+    r"|^/api/hosts/[^/]+/gui-ticket$"
+    # The per-gateway alert switch is a notification setting (same rights as PUT /api/notifications/settings).
+    r"|^/api/hosts/[^/]+/gateways/[^/]+/alert$"
 )
 
 
@@ -90,7 +94,10 @@ _ROLLOUT_POST = re.compile(
     r"|deployments/[^/]+/(remove|retry-rollback)"
     r"|maintenance/(execute|carp-execute|schedule)"
     r"|network/changes"
-    r"|services/[^/]+/(start|stop|restart))$"
+    r"|services/[^/]+/(start|stop|restart)"
+    # Firmware rollouts (1.25): starting/arming a plan and moving a run forward. Pausing is deliberately absent.
+    r"|rollouts/plans/[^/]+/(start|activate)"
+    r"|rollouts/runs/[^/]+/(resume|approve|skip-host|abort))$"
 )
 _ROLLOUT_DELETE = re.compile(
     r"^/api/(rules|aliases|port-forward|one-to-one|outbound-nat|policy-hierarchy/rules|ipsec/tunnels|openvpn/instances"
@@ -99,7 +106,13 @@ _ROLLOUT_DELETE = re.compile(
 )
 
 
+# Recurring maintenance windows decide WHEN unattended firmware actions run (1.25): every write counts.
+_ROLLOUT_ANY_WRITE = re.compile(r"^/api/maintenance-windows(/|$)")
+
+
 def is_rollout(method: str, path: str) -> bool:
+    if method in ("POST", "PUT", "PATCH", "DELETE") and _ROLLOUT_ANY_WRITE.match(path):
+        return True
     if method == "POST":
         return bool(_ROLLOUT_POST.search(path))
     return method == "DELETE" and bool(_ROLLOUT_DELETE.match(path))

backend/app/services/compliance_service.py

@@ -26,6 +26,7 @@ from app.models.host import OpnsenseHost
 from app.services import backup_store, rule_hit_service
 from app.services import opnsense_client as client
 from app.services.opnsense_client import OpnsenseAPIError
+from app.services.rule_analysis import coverage
 
 MAX_CONFIG_BYTES = 32 * 1024 * 1024
 BACKUP_MAX_AGE_DAYS = 7
@@ -83,10 +84,15 @@ def _mvc_rule(rule: Element) -> dict:
         "sport": _text(rule, "source_port"),
         "dst": _text(rule, "destination_net") or "any", "dst_not": _text(rule, "destination_not") == "1",
         "dport": _text(rule, "destination_port"),
-        "floating": False,
+        "if_not": _text(rule, "interfacenot") == "1",
+        "floating": False,  # decided in extract_rules (needs the list of interfaces)
         "quick": _text(rule, "quick") != "0",
         "descr": _text(rule, "description") or "",
         "order": int(_text(rule, "sequence") or 0),
+        "sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
+        "received_on": _text(rule, "received-on") or "",
+        "icmptype": ",".join(filter(None, [_text(rule, "icmptype"), _text(rule, "icmp6type")])),
+        "limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
     }
 
 
@@ -101,17 +107,29 @@ def _legacy_rule(rule: Element, index: int) -> dict:
         "ipprotocol": _text(rule, "ipprotocol") or "inet",
         "protocol": (_text(rule, "protocol") or "any").lower(),
         "src": src, "src_not": src_not, "sport": sport, "dst": dst, "dst_not": dst_not, "dport": dport,
+        "if_not": rule.find("interfacenot") is not None,
         "floating": _text(rule, "floating") == "yes",
-        "quick": _text(rule, "quick") != "0",
+        # Legacy rules: an interface rule is always evaluated "quick"; only a floating rule has the option,
+        # and it is off unless <quick> says yes.
+        "quick": _text(rule, "floating") != "yes" or (_text(rule, "quick") or "").lower() in ("yes", "1"),
         "descr": _text(rule, "descr") or "",
         "order": index,
+        "sched": _text(rule, "sched") or "", "tagged": _text(rule, "tagged") or "",
+        "received_on": _text(rule, "received-on") or "", "icmptype": _text(rule, "icmptype") or "",
+        "limited": any(_text(rule, tag) for tag in ("max-src-conn-rate", "max-pkt-rate-number", "overload", "tcpflags1")),
     }
 
 
 def extract_rules(root: Element) -> list[list[dict]]:
     """Rule lists in evaluation order, one per source (MVC, legacy)."""
-    mvc = sorted((_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")), key=lambda r: r["order"])
+    mvc = [_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")]
+    known = {node.tag for node in root.findall("interfaces/*")}
+    for rule in mvc:  # same rule as OPNsense: not exactly one interface (or "not") = floating
+        rule["floating"] = len(rule["interfaces"]) != 1 or rule["if_not"]
+        rule["group"] = not rule["floating"] and rule["interfaces"][0] not in known
+    mvc.sort(key=coverage.config_order)
     legacy = [_legacy_rule(r, i) for i, r in enumerate(root.findall("filter/rule"))]
+    legacy.sort(key=coverage.config_order)
     return [group for group in (mvc, legacy) if group]
 
 
@@ -133,41 +151,22 @@ def _is_any_any(rule: dict) -> bool:
 
 
 
-def _covers(a: dict, b: dict) -> bool:
-    """Every packet matched by rule b is already matched by the earlier rule a.
-    Only exact-or-any comparisons, so it never claims coverage it cannot prove."""
-    if not (a["enabled"] and b["enabled"]) or a["floating"] or b["floating"] or not a["quick"]:
-        return False
-    if not a["interfaces"] or not set(b["interfaces"]) <= set(a["interfaces"]):
-        return False
-    if a["direction"] != b["direction"] or a["ipprotocol"] != b["ipprotocol"]:
-        return False
-    if a["protocol"] != "any" and a["protocol"] != b["protocol"]:
-        return False
-    for side in ("src", "dst"):
-        if a[side + "_not"] or b[side + "_not"]:
-            if (a[side], a[side + "_not"]) != (b[side], b[side + "_not"]):
-                return False
-        elif a[side] != "any" and a[side] != b[side]:
-            return False
-    for port in ("sport", "dport"):
-        if a[port] and a[port] != b[port]:
-            return False
-    return True
-
-
 def _label(rule: dict) -> str:
     return rule["descr"] or f"{rule['action']} {rule['src']} → {rule['dst']}"
 
 
 def analyse_rules(groups: list[list[dict]]) -> dict:
+    """Shadowed and duplicate rules. The coverage logic lives in rule_analysis/coverage.py (shared with the
+    cleanup candidates); the rules of one list are compared in evaluation order."""
     shadowed, duplicates = [], []
     for rules in groups:
-        for j, later in enumerate(rules):
-            for earlier in rules[:j]:
-                if _covers(earlier, later):
-                    (duplicates if earlier["action"] == later["action"] and _covers(later, earlier) else shadowed).append(_label(later))
-                    break
+        ordered = sorted(rules, key=coverage.config_order)
+        norm = [coverage.from_config_rule(rule) for rule in ordered]
+        by_identity = {id(n): r for n, r in zip(norm, ordered)}
+        for overlap in coverage.analyse(norm):
+            if overlap.by.order == overlap.rule.order:
+                continue  # equal position in config.xml: the real order is unknown, claim nothing
+            (duplicates if overlap.kind == "duplicate" else shadowed).append(_label(by_identity[id(overlap.rule)]))
     return {"shadowed": shadowed, "duplicates": duplicates}
 
 

backend/app/services/gateway_health_service.py

@@ -0,0 +1,97 @@
+"""Gateway health of a firewall (dpinger numbers, as OPNsense reports them).
+
+``get_gateway_health(host)`` is the one reusable entry point: a live read of
+/routes/gateway/status, one :class:`GatewayHealth` per gateway. The notification
+engine, the host page and the metrics read the SAME numbers from the periodic
+resource sample (``compact`` / ``from_compact``), so a scrape or a page view never
+fans out to the firewalls; the 1.25 health gate can call ``get_gateway_health``
+directly. Parsing is lenient on purpose: "~" means "nothing measured", never zero.
+"""
+import re
+
+from app.models.host import OpnsenseHost
+from app.schemas.gateway import GatewayHealth, GatewayState
+from app.services import opnsense_client as client
+
+_NUMBER = re.compile(r"-?\d+(?:\.\d+)?")
+_DEGRADED = {"delay", "loss", "delay+loss"}
+
+
+def _number(value) -> float | None:
+    if value is None:
+        return None
+    match = _NUMBER.search(str(value))
+    return float(match.group()) if match else None
+
+
+def _state(status: str, measured: bool) -> GatewayState:
+    if status == "down":
+        return "offline"
+    if status == "force_down":
+        return "forced_down"
+    if status in _DEGRADED:
+        return "degraded"
+    if status == "none":
+        # "none" is also what a gateway without monitoring reports; only numbers make it "online".
+        return "online" if measured else "unknown"
+    return "unknown"
+
+
+def parse_gateway_status(payload) -> list[GatewayHealth]:
+    items = payload.get("items") if isinstance(payload, dict) else None
+    result: list[GatewayHealth] = []
+    for item in items or []:
+        if not isinstance(item, dict) or not item.get("name"):
+            continue
+        status = str(item.get("status") or "").lower()
+        loss, delay, stddev = _number(item.get("loss")), _number(item.get("delay")), _number(item.get("stddev"))
+        if loss is not None and loss >= 100:
+            delay = stddev = None  # dpinger prints 0.0 ms when nothing answers: no round trip, not a fast one
+        measured = loss is not None or delay is not None
+        address = item.get("address")
+        monitor = item.get("monitor")
+        result.append(GatewayHealth(
+            name=str(item["name"]),
+            address=None if address in (None, "", "~") else str(address),
+            status=status or "unknown",
+            state=_state(status, measured),
+            loss=loss, delay=delay, stddev=stddev,
+            monitor=None if monitor in (None, "", "~") else str(monitor),
+        ))
+    return result
+
+
+async def get_gateway_health(host: OpnsenseHost) -> list[GatewayHealth]:
+    """Live gateway state of ``host``. Raises OpnsenseAPIError if the firewall cannot be asked."""
+    return parse_gateway_status(await client.get_gateway_status(host))
+
+
+def compact(gateways: list[GatewayHealth]) -> list[dict]:
+    """What the resource sampler stores per sample (small, no addresses)."""
+    return [
+        {"name": g.name, "status": g.status, "state": g.state, "loss": g.loss, "delay": g.delay, "stddev": g.stddev}
+        for g in gateways
+    ]
+
+
+def from_compact(rows) -> list[GatewayHealth]:
+    return [
+        GatewayHealth(
+            name=r["name"], status=r.get("status") or "unknown", state=r.get("state") or "unknown",
+            loss=r.get("loss"), delay=r.get("delay"), stddev=r.get("stddev"),
+        )
+        for r in rows or [] if isinstance(r, dict) and r.get("name")
+    ]
+
+
+def is_bad(gateway: GatewayHealth, loss_threshold_pct: float = 0, delay_threshold_ms: float = 0) -> bool:
+    """Alarm-worthy: down, or (when the thresholds are on) measured loss / latency beyond them.
+    A gateway forced down by the admin and one that is not measured are not alarms."""
+    if gateway.state == "offline":
+        return True
+    if loss_threshold_pct and gateway.loss is not None and gateway.loss >= loss_threshold_pct:
+        return True
+    if delay_threshold_ms and gateway.delay is not None and gateway.delay >= delay_threshold_ms:
+        return True
+    return False
+

backend/app/services/gui_ticket_service.py

@@ -0,0 +1,384 @@
+"""Tickets and gateway sessions for the firewall GUI access (1.27, gui-gateway container).
+
+Flow (see docs in routers/gui_access.py):
+* An admin asks for a ticket (POST /api/hosts/{id}/gui-ticket). 32 random bytes, valid 30 s, single use;
+  only its SHA-256 is stored, bound to the OPNexus session (same hash as sessions.token_hash), the user,
+  the host and a slot (gateway port = its own browser origin).
+* The gateway redeems it through /internal/gui/redeem and gets what it needs to reach the firewall —
+  base URL, pinned CA, overlay proxy, timeouts. Never the API key or secret.
+* Every 20 s the gateway re-checks the gateway session (/internal/gui/revalidate): OPNexus session still
+  valid, user still admin, switch still on, host still eligible, absolute limit not reached.
+
+A slot stays assigned to one firewall for as long as that browser session has the firewall open; another
+firewall gets another slot (another origin), so two firewall GUIs never share an origin at the same time.
+After the last gateway session of a slot ended, the slot goes to a DIFFERENT firewall only after SLOT_COOLDOWN
+(1.29, S1): a popup of a possibly compromised firewall GUI may still be open and its cookies belong to that
+origin. The same firewall may take the slot again at once.
+
+The target the gateway talks to (origin, pinned CA, overlay proxy) is fingerprinted at redeem time; every
+re-check compares it with the host's current configuration and ends the session when it changed (S4).
+"""
+import hashlib
+import secrets
+import uuid
+from dataclasses import dataclass
+from datetime import datetime, timedelta, timezone
+from urllib.parse import urlsplit
+
+from sqlalchemy import delete, func, select, update
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.config import settings
+from app.errors import ErrorCode
+from app.models.gui_access import GuiAccessSettings, GuiSession, GuiTicket
+from app.models.host import OpnsenseHost
+from app.models.user import AdminUser, Session
+from app.services import audit_service, firewall_url, overlay_service
+
+TICKET_TTL = timedelta(seconds=30)
+# 20 s: a switch-off, logout or role change must end a gateway session within 30 s of the change, and
+# the last re-check can lie up to one interval before it (live test 2026-10-21: 30 s gave 31 s).
+REVALIDATE_SECONDS = 20
+# A gateway session nobody re-checked for this long is treated as ended (gateway crashed without telling us).
+SESSION_STALE_AFTER = timedelta(seconds=REVALIDATE_SECONDS * 3)
+MAX_SLOTS = 4
+# A slot freed by firewall A goes to firewall B no earlier than this long after A's session ended.
+SLOT_COOLDOWN = timedelta(minutes=5)
+# Open gateway sessions at once (the gateway container enforces the same number; test_gui_access checks they match).
+MAX_GATEWAY_SESSIONS = 64
+MIN_GATEWAY_TOKEN_LENGTH = 32
+AUDIT_KIND = "gui_access"
+
+
+@dataclass
+class GuiAccessError(Exception):
+    status: int
+    code: ErrorCode
+    params: dict
+
+
+def hash_value(raw: str) -> str:
+    return hashlib.sha256(raw.encode()).hexdigest()
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+# ── configuration ─────────────────────────────────────────────────────────
+
+def slot_sites() -> dict[int, list[str]]:
+    """Slot port -> configured https origins with that port (OPNEXUS_GUI_SITES)."""
+    sites: dict[int, list[str]] = {}
+    for origin in settings.gui_site_origins:
+        try:
+            parts = urlsplit(origin)
+            port = parts.port
+        except ValueError:
+            continue
+        if parts.scheme != "https" or not parts.hostname or not port or parts.path not in ("", "/"):
+            continue
+        sites.setdefault(port, []).append(f"https://{parts.netloc}")
+    return sites
+
+
+def gateway_configured() -> bool:
+    return len(settings.gui_gateway_token) >= MIN_GATEWAY_TOKEN_LENGTH and bool(slot_sites())
+
+
+def unavailable_reasons() -> list[str]:
+    reasons = []
+    if not settings.cookie_secure:
+        reasons.append("cookie_secure")
+    if len(settings.gui_gateway_token) < MIN_GATEWAY_TOKEN_LENGTH:
+        reasons.append("gateway_token")
+    if not slot_sites():
+        reasons.append("gui_sites")
+    return reasons
+
+
+def active_slots(row: GuiAccessSettings) -> list[int]:
+    return sorted(slot_sites())[: max(1, min(row.slot_count, MAX_SLOTS))]
+
+
+async def get_settings(db: AsyncSession) -> GuiAccessSettings:
+    row = await db.get(GuiAccessSettings, 1)
+    if row is None:
+        row = GuiAccessSettings(id=1, enabled=False, idle_timeout_minutes=15, absolute_timeout_minutes=60, slot_count=MAX_SLOTS)
+        db.add(row)
+        await db.flush()
+    return row
+
+
+def host_refusal(host: OpnsenseHost) -> str | None:
+    """Why this firewall must not be opened through the gateway (None = eligible)."""
+    if not host.enabled:
+        return "disabled"
+    if urlsplit(host.base_url.strip()).scheme != "https" or firewall_url.check_base_url(host.base_url):
+        return "https"
+    if not host.verify_tls:
+        return "tls"
+    return None
+
+
+def firewall_origin(base_url: str) -> str:
+    parts = urlsplit(base_url.strip())
+    return f"https://{parts.netloc.rsplit('@', 1)[-1]}"
+
+
+def target_fingerprint(host: OpnsenseHost) -> str:
+    """What a gateway session talks to: origin, pinned CA and overlay route. Changes -> the session must end."""
+    material = "\n".join([firewall_origin(host.base_url), host.tls_ca_pem or "", overlay_service.proxy_for(host.base_url) or "",
+                          "verify" if host.verify_tls else "noverify"])
+    return hash_value(material)
+
+
+# ── audit ─────────────────────────────────────────────────────────────────
+
+async def _audit(actor: str, *, action: str, status: str, host: OpnsenseHost | None = None,
+                 host_id=None, entity_id=None, message: str | None = None, details: dict | None = None) -> None:
+    token = audit_service.context.set({"actor": actor, "id": str(uuid.uuid4()), "action": action})
+    try:
+        await audit_service.persist(
+            kind=AUDIT_KIND, action=action, status=status, entity_id=str(entity_id) if entity_id else None,
+            host_id=str(host.id) if host is not None else (str(host_id) if host_id else None),
+            host_name=host.name if host is not None else None, message=message, details=details or {},
+        )
+    finally:
+        audit_service.context.reset(token)
+
+
+# ── tickets ───────────────────────────────────────────────────────────────
+
+def _redeem_origin(slot: int, request_host: str) -> str:
+    candidates = slot_sites().get(slot, [])
+    wanted = (urlsplit(f"//{request_host}").hostname or "").lower()
+    for origin in candidates:
+        if (urlsplit(origin).hostname or "").lower() == wanted:
+            return origin
+    return candidates[0]
+
+
+async def _pick_slot(db: AsyncSession, row: GuiAccessSettings, session_hash: str, user_id, host_id) -> int:
+    now = _now()
+    slots = active_slots(row)
+    absolute = timedelta(minutes=row.absolute_timeout_minutes)
+    in_use: dict[int, uuid.UUID] = {}
+    open_sessions = await db.scalars(select(GuiSession).where(
+        GuiSession.session_hash == session_hash, GuiSession.ended_at.is_(None), GuiSession.started_at > now - absolute))
+    for gs in open_sessions:
+        in_use[gs.slot] = gs.host_id
+    pending = await db.scalars(select(GuiTicket).where(
+        GuiTicket.session_hash == session_hash, GuiTicket.redeemed_at.is_(None), GuiTicket.expires_at > now))
+    for ticket in pending:
+        in_use.setdefault(ticket.slot, ticket.host_id)
+    for slot in slots:
+        if in_use.get(slot) == host_id:
+            return slot  # this firewall is already open in this browser session: same origin again
+    free = [s for s in slots if s not in in_use]
+    if not free:
+        raise GuiAccessError(409, ErrorCode.GUI_NO_FREE_SLOT, {"slots": str(len(slots))})
+    # S1: a slot this user's browser may still have a window of ANOTHER firewall on stays closed to this
+    # firewall — while a gateway session of the user is open there (any browser session) and for SLOT_COOLDOWN
+    # after the last one ended. The same firewall may take it again at once.
+    held = (await db.execute(
+        select(GuiSession.slot, GuiSession.host_id, GuiSession.ended_at, GuiSession.last_checked_at).where(
+            GuiSession.user_id == user_id,
+            ((GuiSession.ended_at.is_(None)) & ((GuiSession.started_at > now - absolute) | (GuiSession.last_checked_at > now - SLOT_COOLDOWN)))
+            | (GuiSession.ended_at > now - SLOT_COOLDOWN)))).all()
+    cooling: dict[int, datetime] = {}  # slot -> when it may go to another firewall
+    for slot, h_id, ended_at, checked_at in held:
+        if h_id == host_id or slot not in free:
+            continue
+        if ended_at is None and checked_at is not None and now - checked_at > SESSION_STALE_AFTER:
+            ended_at = checked_at  # the gateway stopped re-checking it (crash, kill): it ended when it was last seen
+        release = (ended_at + SLOT_COOLDOWN) if ended_at is not None else now + SLOT_COOLDOWN
+        if release <= now:
+            continue  # cooled down long ago (a session that stopped answering a while back)
+        cooling[slot] = max(cooling.get(slot, release), release)
+    usable = [s for s in free if s not in cooling]
+    if not usable:
+        wait = min(cooling.values()) - now
+        raise GuiAccessError(409, ErrorCode.GUI_SLOT_COOLING_DOWN,
+                             {"minutes": str(max(1, -(-int(wait.total_seconds()) // 60)))})
+    free = usable
+    # Prefer the slot this user had this firewall on before, then the slot the user used least recently
+    # (an old tab of another firewall on a reused origin is the documented residual risk).
+    history = (await db.execute(
+        select(GuiSession.slot, GuiSession.host_id, func.max(GuiSession.started_at))
+        .where(GuiSession.user_id == user_id).group_by(GuiSession.slot, GuiSession.host_id)
+    )).all()
+    for slot, h_id, _ in sorted(history, key=lambda r: r[2], reverse=True):
+        if h_id == host_id and slot in free:
+            return slot
+    last_used = {}
+    for slot, _, started in history:
+        last_used[slot] = max(last_used.get(slot, started), started)
+    epoch = datetime.min.replace(tzinfo=timezone.utc)
+    return min(free, key=lambda s: (last_used.get(s, epoch), s))
+
+
+async def issue_ticket(db: AsyncSession, *, user: AdminUser, session_token: str, host: OpnsenseHost,
+                       request_host: str) -> dict:
+    row = await get_settings(db)
+    now = _now()
+    await db.execute(delete(GuiTicket).where(GuiTicket.expires_at < now - timedelta(hours=1)))
+    session_hash = hash_value(session_token)
+    slot = await _pick_slot(db, row, session_hash, user.id, host.id)
+    # Reopening a slot in the same browser session replaces its earlier gateway session, so that one does not count.
+    open_count = await db.scalar(select(func.count()).select_from(GuiSession).where(
+        GuiSession.ended_at.is_(None),
+        GuiSession.started_at > now - timedelta(minutes=row.absolute_timeout_minutes),
+        ~((GuiSession.session_hash == session_hash) & (GuiSession.slot == slot))))
+    if (open_count or 0) >= MAX_GATEWAY_SESSIONS:
+        raise GuiAccessError(409, ErrorCode.GUI_TOO_MANY_SESSIONS, {"max": str(MAX_GATEWAY_SESSIONS)})
+    raw = secrets.token_urlsafe(32)
+    db.add(GuiTicket(ticket_hash=hash_value(raw), session_hash=session_hash, user_id=user.id, host_id=host.id,
+                     slot=slot, created_at=now, expires_at=now + TICKET_TTL))
+    await db.commit()
+    await _audit(user.username, action="ticket", status="accepted", host=host, details={"slot": slot})
+    return {"url": f"{_redeem_origin(slot, request_host)}/__onx/redeem?t={raw}", "slot": slot,
+            "expires_in": int(TICKET_TTL.total_seconds())}
+
+
+# ── gateway side (internal endpoints) ─────────────────────────────────────
+
+async def _live_check(db: AsyncSession, row: GuiAccessSettings, session_hash: str, user_id, host_id,
+                      slot: int) -> tuple[str | None, AdminUser | None, OpnsenseHost | None]:
+    session = await db.scalar(select(Session).where(Session.token_hash == session_hash))
+    if session is None or session.expires_at <= _now() or session.user_id != user_id:
+        return "session", None, None
+    user = await db.get(AdminUser, user_id)
+    if user is None or user.role != "admin":
+        return "role", user, None
+    if not row.enabled:
+        return "disabled", user, None
+    if not settings.cookie_secure or not gateway_configured():
+        return "config", user, None
+    host = await db.get(OpnsenseHost, host_id)
+    if host is None or host_refusal(host):
+        return "host", user, host
+    if slot not in active_slots(row):
+        return "slot", user, host
+    return None, user, host
+
+
+def _timing(row: GuiAccessSettings, started_at: datetime) -> dict:
+    deadline = started_at + timedelta(minutes=row.absolute_timeout_minutes)
+    return {"idle_timeout_seconds": row.idle_timeout_minutes * 60,
+            "absolute_deadline": int(deadline.timestamp()),
+            "revalidate_seconds": REVALIDATE_SECONDS}
+
+
+async def redeem(db: AsyncSession, *, ticket_hash: str, session_hash: str, slot: int, client_ip: str | None) -> dict:
+    """{'ok': True, ...upstream data...} or {'ok': False, 'reason': ...}. Any attempt burns the ticket."""
+    now = _now()
+    ticket = await db.scalar(select(GuiTicket).where(GuiTicket.ticket_hash == ticket_hash).with_for_update())
+    if ticket is None:
+        await db.rollback()
+        return {"ok": False, "reason": "unknown"}
+    replayed = ticket.redeemed_at is not None
+    ticket.redeemed_at = ticket.redeemed_at or now
+    row = await get_settings(db)
+    reason = None
+    if replayed:
+        reason = "replayed"
+    elif ticket.expires_at <= now:
+        reason = "expired"
+    elif not secrets.compare_digest(ticket.session_hash, session_hash):
+        reason = "session"
+    elif ticket.slot != slot:
+        reason = "slot"
+    user = host = None
+    if reason is None:
+        reason, user, host = await _live_check(db, row, ticket.session_hash, ticket.user_id, ticket.host_id, slot)
+    if user is None:
+        user = await db.get(AdminUser, ticket.user_id)
+    if host is None:
+        host = await db.get(OpnsenseHost, ticket.host_id)
+    actor = user.username if user is not None else "unknown"
+    if reason is not None:
+        await db.commit()
+        await _audit(actor, action="redeem", status="denied", host=host, host_id=ticket.host_id,
+                     message=reason, details={"slot": slot, "client_ip": client_ip, "reason": reason})
+        return {"ok": False, "reason": reason}
+    # The same browser session reopening this slot replaces the earlier gateway session there.
+    superseded = (await db.scalars(select(GuiSession).where(
+        GuiSession.session_hash == session_hash, GuiSession.slot == slot, GuiSession.ended_at.is_(None)))).all()
+    if any(old.host_id != host.id for old in superseded):
+        # Two quick tickets both saw the slot free: the second must not take over another firewall's origin
+        # without the cooling-down time (S1).
+        await db.commit()
+        await _audit(actor, action="redeem", status="denied", host=host, host_id=ticket.host_id,
+                     message="slot", details={"slot": slot, "client_ip": client_ip, "reason": "slot"})
+        return {"ok": False, "reason": "slot"}
+    for old in superseded:
+        old.ended_at, old.end_reason = now, "superseded"
+    gs = GuiSession(session_hash=session_hash, user_id=user.id, host_id=host.id, slot=slot,
+                    client_ip=(client_ip or None) and client_ip[:45], started_at=now, last_checked_at=now)
+    db.add(gs)
+    await db.commit()
+    await _audit(actor, action="redeem", status="accepted", host=host, entity_id=gs.id,
+                 details={"slot": slot, "client_ip": client_ip})
+    return {
+        "ok": True, "gui_session_id": str(gs.id), "host_id": str(host.id), "host_name": host.name,
+        "upstream_origin": firewall_origin(host.base_url),
+        "ca_pem": host.tls_ca_pem or None,  # public certificate (pin); None = system trust store
+        "proxy_url": overlay_service.proxy_for(host.base_url),
+        "target_fp": target_fingerprint(host),
+        **_timing(row, now),
+    }
+
+
+async def _finish(db: AsyncSession, gs: GuiSession, reason: str) -> None:
+    now = _now()
+    if gs.ended_at is None:
+        gs.ended_at, gs.end_reason = now, reason[:32]
+    await db.commit()
+    user = await db.get(AdminUser, gs.user_id)
+    host = await db.get(OpnsenseHost, gs.host_id)
+    await _audit(user.username if user else "unknown", action="end", status="accepted", host=host,
+                 host_id=gs.host_id, entity_id=gs.id, message=reason,
+                 details={"slot": gs.slot, "client_ip": gs.client_ip, "reason": reason,
+                          "duration_seconds": int((gs.ended_at - gs.started_at).total_seconds())})
+
+
+async def revalidate(db: AsyncSession, *, gui_session_id: uuid.UUID, session_hash: str,
+                     target_fp: str | None = None) -> dict:
+    gs = await db.get(GuiSession, gui_session_id)
+    if gs is None or not secrets.compare_digest(gs.session_hash, session_hash):
+        return {"ok": False, "reason": "unknown"}
+    if gs.ended_at is not None:
+        return {"ok": False, "reason": gs.end_reason or "ended"}
+    row = await get_settings(db)
+    if _now() >= gs.started_at + timedelta(minutes=row.absolute_timeout_minutes):
+        await _finish(db, gs, "absolute")
+        return {"ok": False, "reason": "absolute"}
+    reason, _, host = await _live_check(db, row, gs.session_hash, gs.user_id, gs.host_id, gs.slot)
+    # S4: the gateway presents the fingerprint it got at redeem time. Firewall address, pinned certificate or
+    # overlay route changed since -> the session (and the upstream connection it holds) ends now.
+    if reason is None and target_fp is not None and host is not None \
+            and not secrets.compare_digest(target_fp, target_fingerprint(host)):
+        reason = "target"
+    if reason is not None:
+        await _finish(db, gs, reason)
+        return {"ok": False, "reason": reason}
+    gs.last_checked_at = _now()
+    await db.commit()
+    return {"ok": True, **_timing(row, gs.started_at)}
+
+
+async def end(db: AsyncSession, *, gui_session_id: uuid.UUID, session_hash: str, reason: str) -> bool:
+    gs = await db.get(GuiSession, gui_session_id)
+    if gs is None or not secrets.compare_digest(gs.session_hash, session_hash):
+        return False
+    if gs.ended_at is None:
+        await _finish(db, gs, reason)
+    return True
+
+
+async def end_all(db: AsyncSession, reason: str) -> int:
+    """Switch turned off: mark every open gateway session ended (the gateway drops them on its next check)."""
+    result = await db.execute(update(GuiSession).where(GuiSession.ended_at.is_(None))
+                              .values(ended_at=_now(), end_reason=reason[:32]))
+    return result.rowcount or 0

backend/app/services/metrics_service.py

@@ -5,7 +5,7 @@ fan out live calls to every firewall (an offline host would stall it past the
 scraper's timeout and a 15 s scrape interval would hammer the fleet). The
 inputs are what OPNexus' own pollers already store:
 
-* host reachability and resources — the 60 s resource sampler (a host that
+* host reachability, resources, gateway health and service state — the 60 s resource sampler (a host that
   produced no sample for ``STALE_AFTER`` is reported down),
 * CARP role — the latest observed transition (carp_events),
 * open problems — the notification engine's problem table,
@@ -19,11 +19,12 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.models.carp_event import CarpEvent
 from app.models.compliance_result import ComplianceResult
 from app.models.host import OpnsenseHost
-from app.models.notification import NotificationProblem
+from app.models.notification import DEFAULT_SERVICE_WATCHLIST, NotificationProblem
 from app.models.resource_sample import HostResourceSample
+from app.models.rollout import RolloutRun
 from app.models.scheduled_maintenance import ScheduledMaintenance
 from app.routers.dashboard import collect_pending_actions
-from app.services import compliance_service
+from app.services import compliance_service, gateway_health_service, notification_settings_service
 from app.version import APP_VERSION
 
 CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
@@ -99,6 +100,15 @@ async def render(db: AsyncSession) -> str:
     out.family("opnexus_host_memory_used_percent", "Memory in use, percent (last sample).")
     out.family("opnexus_host_load1", "1-minute load average (last sample).")
     out.family("opnexus_host_disk_used_percent", "Fullest disk, percent (last sample).")
+    out.family("opnexus_host_swap_used_percent", "Swap in use, percent (last sample; absent without swap).")
+    out.family("opnexus_host_gateway_up", "1 if the gateway is not down (online, degraded or not monitored), 0 if OPNsense reports it down.")
+    out.family("opnexus_host_gateway_loss_percent", "Packet loss to the gateway monitor address, percent (last sample).")
+    out.family("opnexus_host_gateway_delay_seconds", "Round-trip time to the gateway monitor address (last sample).")
+    out.family("opnexus_host_gateway_stddev_seconds", "Round-trip time deviation to the gateway monitor address (last sample).")
+    out.family("opnexus_host_service_running", "1 if a watched service (Settings > Notifications) runs, 0 if it is stopped (last sample).")
+    out.family("opnexus_host_api_rights_missing", "1 if the firewall refused a side reading (read=gateways, services, swap, cpu) because the API key lacks the right (last sample); absent otherwise.")
+    configured = (await notification_settings_service.get_settings(db)).service_watchlist
+    watchlist = set(DEFAULT_SERVICE_WATCHLIST if configured is None else configured)
     out.family("opnexus_host_carp_master", "1 if the host's last observed CARP role is MASTER, 0 if BACKUP; absent if unknown.")
     for host in hosts:
         labels = {"host": host.name, "cluster": host.cluster_name or ""}
@@ -115,6 +125,24 @@ async def render(db: AsyncSession) -> str:
             ):
                 if value is not None:
                     out.sample(name, value, **labels)
+        if sample is not None:
+            raw = sample.raw or {}
+            if raw.get("swap_used_pct") is not None:
+                out.sample("opnexus_host_swap_used_percent", raw["swap_used_pct"], **labels)
+            for gw in gateway_health_service.from_compact(raw.get("gateways")):
+                gw_labels = {**labels, "gateway": gw.name}
+                out.sample("opnexus_host_gateway_up", 0 if gw.state == "offline" else 1, **gw_labels)
+                for name, value, factor in (
+                    ("opnexus_host_gateway_loss_percent", gw.loss, 1), ("opnexus_host_gateway_delay_seconds", gw.delay, 0.001),
+                    ("opnexus_host_gateway_stddev_seconds", gw.stddev, 0.001),
+                ):
+                    if value is not None:
+                        out.sample(name, round(value * factor, 6), **gw_labels)
+            for read in sorted(str(r) for r in raw.get("denied") or []):
+                out.sample("opnexus_host_api_rights_missing", 1, **labels, read=read)
+            for key, running in sorted((raw.get("services") or {}).items()):
+                if key.split("/")[0] in watchlist:
+                    out.sample("opnexus_host_service_running", running, **labels, service=key.split("/")[0], instance=key.split("/", 1)[1] if "/" in key else None)
         if carp_state.get(host.id) in ("MASTER", "BACKUP"):
             out.sample("opnexus_host_carp_master", 1 if carp_state[host.id] == "MASTER" else 0, **labels)
 
@@ -152,4 +180,8 @@ async def render(db: AsyncSession) -> str:
     pending = await db.scalar(select(func.count()).select_from(ScheduledMaintenance).where(ScheduledMaintenance.status == "pending"))
     out.sample("opnexus_scheduled_maintenance_pending", pending or 0)
 
+    out.family("opnexus_rollout_runs", "Fleet rollout runs by status (waiting_window, wave_running, paused, succeeded, ...).")
+    for status, count in sorted((await db.execute(select(RolloutRun.status, func.count()).group_by(RolloutRun.status))).all()):
+        out.sample("opnexus_rollout_runs", count, status=status)
+
     return out.text()

backend/app/services/monitor_checks.py

@@ -0,0 +1,291 @@
+"""Monitoring alerts (1.24): gateway down, resource thresholds, stopped services.
+
+Same contract as notification_checks.py (each check returns {dedup_key: context} of what is
+bad right now), with one addition: they never judge a single reading. The inputs are the
+60-second samples the resource sampler stores (resource_monitor_service, HostResourceSample),
+and a problem needs the last N samples to agree before it opens AND before it closes again:
+
+* all N bad            -> problem,
+* all N good           -> no problem,
+* anything in between  -> keep the current state (open stays open, closed stays closed).
+
+So a single spike never alerts, a flapping link does not open and close the alert every few
+minutes, and a sample that could not be read ("unknown") neither raises nor clears anything.
+A host without a fresh sample is not judged at all (check_host_offline reports it); its open
+problems are kept.
+"""
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.host import OpnsenseHost
+from app.models.notification import DEFAULT_SERVICE_WATCHLIST, NotificationProblem
+from app.models.resource_sample import HostResourceSample
+from app.services import gateway_health_service, notification_settings_service
+from app.services.notification_dedup import Problems
+
+# The sampler runs every 60 s; three missed rounds mean the host is not reachable (same limit as metrics_service).
+STALE_AFTER = timedelta(seconds=180)
+# How far back a service must have been seen running before its absence is reported (samples, ~2 h at 60 s).
+SERVICE_EVIDENCE_SAMPLES = 120
+_DISK_TYPES = {"ufs", "zfs"}
+
+
+def verdict(flags: list[bool | None], confirm: int, is_open: bool) -> bool:
+    """flags: newest first; True = bad, False = good, None = unknown."""
+    window = flags[:confirm]
+    if len(window) >= confirm:
+        if all(f is True for f in window):
+            return True
+        if all(f is False for f in window):
+            return False
+    return is_open
+
+
+async def _enabled_hosts(db: AsyncSession) -> list[OpnsenseHost]:
+    return list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)))).all())
+
+
+async def _open_keys(db: AsyncSession, problem_type: str) -> set[str]:
+    rows = await db.scalars(
+        select(NotificationProblem.dedup_key).where(
+            NotificationProblem.problem_type == problem_type, NotificationProblem.status == "open"
+        )
+    )
+    return set(rows.all())
+
+
+async def _recent(db: AsyncSession, host_id, limit: int, *raw_keys: str) -> list[dict]:
+    """The newest ``limit`` samples of a host, newest first, with just the JSON keys asked for."""
+    columns = [HostResourceSample.sampled_at, HostResourceSample.mem_used_pct, HostResourceSample.load_avg_1m]
+    columns += [HostResourceSample.raw[key].as_json() for key in raw_keys]
+    rows = (
+        await db.execute(
+            select(*columns).where(HostResourceSample.host_id == host_id).order_by(HostResourceSample.sampled_at.desc()).limit(limit)
+        )
+    ).all()
+    return [
+        {"sampled_at": r[0], "mem_used_pct": r[1], "load_avg_1m": r[2], **{k: r[3 + i] for i, k in enumerate(raw_keys)}}
+        for r in rows
+    ]
+
+
+def _fresh(samples: list[dict]) -> bool:
+    return bool(samples) and datetime.now(timezone.utc) - samples[0]["sampled_at"] <= STALE_AFTER
+
+
+def _keys_of_host(open_keys: set[str], host: OpnsenseHost) -> set[str]:
+    prefix = f"{host.id}:"
+    return {k[len(prefix):] for k in open_keys if k.startswith(prefix)}
+
+
+# ── Gateways ────────────────────────────────────────────────────────────────
+
+def _gateway_reason(g) -> str:
+    if g.state == "offline":
+        return "Gateway offline"
+    parts = []
+    if g.loss is not None:
+        parts.append(f"Paketverlust {g.loss:g} %")
+    if g.delay is not None:
+        parts.append(f"Latenz {g.delay:g} ms")
+    return ", ".join(parts) or g.status
+
+
+async def check_gateway_down(db: AsyncSession) -> dict[str, dict]:
+    cfg = await notification_settings_service.get_settings(db)
+    confirm = max(1, cfg.alert_confirm_samples)
+    loss_limit, delay_limit = cfg.gateway_loss_threshold_pct, cfg.gateway_delay_threshold_ms
+    muted = set(cfg.gateway_alert_muted or [])
+    open_keys = await _open_keys(db, "gateway_down")
+    problems = Problems()
+    for host in await _enabled_hosts(db):
+        samples = await _recent(db, host.id, confirm, "gateways")
+        if not _fresh(samples):
+            problems.preserve_prefixes.add(f"{host.id}:")
+            continue
+        parsed = [
+            None if s["gateways"] is None else {g.name: g for g in gateway_health_service.from_compact(s["gateways"])}
+            for s in samples
+        ]
+        names = {name for p in parsed if p for name in p} | _keys_of_host(open_keys, host)
+        for name in sorted(names):
+            if f"{host.id}:{name}" in muted:
+                continue  # alert switched off for this gateway: not judged, an open problem of it resolves
+            flags: list[bool | None] = []
+            for p in parsed:
+                if p is None:
+                    flags.append(None)
+                else:
+                    g = p.get(name)  # a gateway that no longer exists is "fine": its problem resolves
+                    flags.append(False if g is None else gateway_health_service.is_bad(g, loss_limit, delay_limit))
+            key = f"{host.id}:{name}"
+            if not verdict(flags, confirm, key in open_keys):
+                continue
+            latest = next((p[name] for p in parsed if p and name in p), None)
+            problems[key] = {
+                "host_id": str(host.id), "host_name": host.name, "gateway": name,
+                "grund": _gateway_reason(latest) if latest else "keine aktuellen Messwerte",
+                "status": latest.status if latest else None,
+                "paketverlust_prozent": latest.loss if latest else None,
+                "latenz_ms": latest.delay if latest else None,
+            }
+    return problems
+
+
+# ── API rights ──────────────────────────────────────────────────────────────
+
+# What the sampler reads on the side (resource_monitor_service) -> (ACL entry of the OPNsense user/group that
+# grants it, endpoint; the sample's "ok" list proves a reading went through). ACL names: opnsense/core
+# src/opnsense/mvc/app/models/OPNsense/Core/ACL/ACL.xml (page-system-gateways: api/routes/gateway/status;
+# page-status-services: api/core/service/*; page-system-login-logout "Lobby: Dashboard": api/diagnostics/cpu_usage/*
+# and api/diagnostics/system/system_swap).
+API_READS: dict[str, tuple[str, str]] = {
+    "gateways": ("System: Gateways", "/api/routes/gateway/status"),
+    "services": ("Status: Services", "/api/core/service/search"),
+    "swap": ("Lobby: Dashboard", "/api/diagnostics/system/systemSwap"),
+    "cpu": ("Lobby: Dashboard", "/api/diagnostics/cpu_usage/getCPUType"),
+}
+
+
+async def check_api_rights_missing(db: AsyncSession) -> dict[str, dict]:
+    """The API key of a firewall may be refused (403) on the side readings the alerts rely on (gateway state,
+    service state): those alerts can then never fire, and nobody would notice. Only a refusal counts; a timeout
+    or an error stays "unknown"."""
+    cfg = await notification_settings_service.get_settings(db)
+    confirm = max(1, cfg.alert_confirm_samples)
+    open_keys = await _open_keys(db, "api_rights_missing")
+    problems = Problems()
+    for host in await _enabled_hosts(db):
+        samples = await _recent(db, host.id, confirm, "denied", "ok")
+        if not _fresh(samples):
+            problems.preserve_prefixes.add(f"{host.id}:")
+            continue
+        was_open = _keys_of_host(open_keys, host)
+        names = {n for s in samples for n in (s["denied"] or []) if n in API_READS} | (was_open & set(API_READS))
+        for name in sorted(names):
+            acl, path = API_READS[name]
+            flags: list[bool | None] = []
+            for s in samples:
+                if s["denied"] is None:
+                    flags.append(None)  # a sample from before this check existed
+                elif name in s["denied"]:
+                    flags.append(True)
+                else:
+                    # Only a reading that went through proves the right is back; a timeout or a sample from
+                    # before "ok" existed stays unknown (the problem neither opens nor closes on it).
+                    flags.append(False if s["ok"] is not None and name in s["ok"] else None)
+            if verdict(flags, confirm, name in was_open):
+                problems[f"{host.id}:{name}"] = {
+                    "host_id": str(host.id), "host_name": host.name, "lesezugriff": name, "endpunkt": path,
+                    "fehlendes_recht": acl,
+                    "hinweis": f"Dem API-Benutzer fehlt das Recht, {path} zu lesen (OPNsense-Berechtigung \"{acl}\"). "
+                               "Alarme, die darauf beruhen, können nicht auslösen.",
+                }
+    return problems
+
+
+# ── Resources ───────────────────────────────────────────────────────────────
+
+def disk_groups(devices) -> dict[str, tuple[str, float]]:
+    """{group: (label, used %)} — one entry per ZFS pool / UFS device, the fullest dataset of a pool wins."""
+    groups: dict[str, tuple[str, float]] = {}
+    for dev in devices or []:
+        if not isinstance(dev, dict) or dev.get("type") not in _DISK_TYPES:
+            continue
+        try:
+            pct = float(dev["used_pct"])
+        except (KeyError, TypeError, ValueError):
+            continue
+        device = str(dev.get("device") or dev.get("mountpoint") or "?")
+        group = device.split("/")[0] if dev["type"] == "zfs" else device
+        label = group if dev["type"] == "zfs" else str(dev.get("mountpoint") or device)
+        if group not in groups or pct > groups[group][1]:
+            groups[group] = (label, pct)
+    return groups
+
+
+async def check_resource_threshold(db: AsyncSession) -> dict[str, dict]:
+    cfg = await notification_settings_service.get_settings(db)
+    confirm = max(1, cfg.resource_confirm_samples)
+    open_keys = await _open_keys(db, "resource_threshold")
+    problems = Problems()
+
+    def over(value, limit) -> bool | None:
+        if not limit or value is None:
+            return None if value is None else False
+        return value >= limit
+
+    for host in await _enabled_hosts(db):
+        samples = await _recent(db, host.id, confirm, "disk", "swap_used_pct", "cpu_threads")
+        if not _fresh(samples):
+            problems.preserve_prefixes.add(f"{host.id}:")
+            continue
+        base = {"host_id": str(host.id), "host_name": host.name}
+        was_open = _keys_of_host(open_keys, host)
+
+        # Disk: one problem per pool/device.
+        disk = [None if s["disk"] is None else disk_groups(s["disk"]) for s in samples]
+        group_names = {g for d in disk if d for g in d} | {k[5:] for k in was_open if k.startswith("disk:")}
+        for group in sorted(group_names):
+            flags = [None if d is None else (over(d[group][1], cfg.disk_threshold_pct) if group in d else False) for d in disk]
+            if verdict(flags, confirm, f"disk:{group}" in was_open):
+                label, pct = next((d[group] for d in disk if d and group in d), (group, None))
+                problems[f"{host.id}:disk:{group}"] = {
+                    **base, "ressource": f"Festplatte {label}", "wert_prozent": pct, "schwelle_prozent": cfg.disk_threshold_pct,
+                }
+
+        def simple(key: str, label: str, flags: list, value, threshold, unit: str) -> None:
+            if verdict(flags, confirm, key in was_open):
+                # An open alert stays open while the reading is unavailable: then there is no number to show (never "None %").
+                problems[f"{host.id}:{key}"] = {**base, "ressource": label, "wert": None if value is None else f"{value}{unit}",
+                                                "schwelle": f"{threshold}{unit}"}
+
+        latest = samples[0]
+        simple("memory", "Arbeitsspeicher", [over(s["mem_used_pct"], cfg.memory_threshold_pct) for s in samples],
+               latest["mem_used_pct"], cfg.memory_threshold_pct, " %")
+        simple("swap", "Swap", [over(s["swap_used_pct"], cfg.swap_threshold_pct) for s in samples],
+               latest["swap_used_pct"], cfg.swap_threshold_pct, " %")
+
+        def per_cpu(s):
+            if s["load_avg_1m"] is None or not s["cpu_threads"]:
+                return None  # thread count unknown (older firmware): the load cannot be judged
+            return s["load_avg_1m"] / s["cpu_threads"]
+
+        simple("cpu", "CPU-Last (pro Thread)", [over(per_cpu(s), cfg.load_per_cpu_threshold) for s in samples],
+               None if per_cpu(latest) is None else round(per_cpu(latest), 2), cfg.load_per_cpu_threshold, "")
+    return problems
+
+
+# ── Services ────────────────────────────────────────────────────────────────
+
+async def check_service_down(db: AsyncSession) -> dict[str, dict]:
+    cfg = await notification_settings_service.get_settings(db)
+    confirm = max(1, cfg.alert_confirm_samples)
+    watch = set(cfg.service_watchlist if cfg.service_watchlist is not None else DEFAULT_SERVICE_WATCHLIST)
+    open_keys = await _open_keys(db, "service_down")
+    problems = Problems()
+    for host in await _enabled_hosts(db):
+        samples = await _recent(db, host.id, confirm + SERVICE_EVIDENCE_SAMPLES, "services")
+        if not _fresh(samples):
+            problems.preserve_prefixes.add(f"{host.id}:")
+            continue
+        listings = [s["services"] for s in samples]
+        was_open = _keys_of_host(open_keys, host)
+        candidates = {k for k in (listings[0] or {}) if k.split("/")[0] in watch} | was_open
+        for svc in sorted(candidates):
+            if svc.split("/")[0] not in watch:
+                continue  # taken off the watchlist: the problem resolves
+            flags = [None if sv is None else (sv[svc] == 0 if svc in sv else False) for sv in listings]
+            bad = verdict(flags, confirm, svc in was_open)
+            # Only a service that was seen running before counts: one that never ran (enabled but idle,
+            # e.g. DHCPv6 without a prefix) or was stopped on purpose long ago is not a failure.
+            if bad and svc not in was_open and not any(sv and sv.get(svc) == 1 for sv in listings[confirm:]):
+                bad = False
+            if bad:
+                problems[f"{host.id}:{svc}"] = {
+                    "host_id": str(host.id), "host_name": host.name, "dienst": svc.split("/")[0],
+                    "hinweis": "Dienst läuft nicht mehr; absichtlich gestoppt? Dann aus der Überwachungsliste nehmen.",
+                }
+    return problems

backend/app/services/notification_channels.py

@@ -0,0 +1,250 @@
+"""Extra notification channels: ntfy, Gotify, Telegram, Microsoft Teams, Matrix.
+
+They sit beside SMTP and the generic webhook (notification_delivery.py) and reuse its message
+text. Each channel is a row in ``notification_channels`` (see models/notification.py); the
+credential never touches the database: it is stored through notification_secret_store as
+``channel_<id>`` (mode 0600 file) and read only at send time. Where the credential is part of the
+URL (a Telegram bot token, the whole Teams webhook URL) the URL is built per request, and the
+HTTP request log is muted while it is in flight (``url_guard.muted``), so neither the logs nor an
+error message ever contain it.
+
+API formats, checked against the vendors' documentation:
+
+* ntfy      — JSON publish: POST <server>/ with {"topic", "title", "message", "priority", "tags"}
+              (docs.ntfy.sh/publish, "Publish as JSON"); optional ``Authorization: Bearer <token>``.
+* Gotify    — POST <server>/message, header ``X-Gotify-Key: <app token>`` (keeps the token out of
+              the URL), JSON {"title", "message", "priority"} (gotify.net/docs/pushmsg).
+* Telegram  — POST <api>/bot<token>/sendMessage, JSON {"chat_id", "text"}; text limit 4096
+              characters; answers {"ok": true, ...} (core.telegram.org/bots/api#sendmessage).
+* Teams     — POST to the (Workflows / incoming) webhook URL with a message envelope carrying an
+              Adaptive Card (learn.microsoft.com ... add-incoming-webhook); the URL is the secret.
+* Matrix    — PUT <homeserver>/_matrix/client/v3/rooms/<roomId>/send/m.room.message/<txnId> with
+              ``Authorization: Bearer <access token>``, JSON {"msgtype": "m.text", "body"}; a fresh
+              transaction id per message (Matrix client-server API, "Sending events to a room").
+"""
+import re
+import uuid
+from urllib.parse import quote
+
+import httpx
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.config import settings
+from app.models.notification import CHANNEL_TYPES, NotificationChannel, NotificationProblem
+from app.services import notification_delivery, notification_secret_store, url_guard
+from app.services.notification_delivery import NotificationDeliveryError
+
+TELEGRAM_LIMIT = 4096
+TEAMS_LIMIT = 20000
+_NTFY_TOPIC = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
+_MATRIX_ROOM = re.compile(r"^![^\s:]+:[^\s/]+$")
+
+# type -> (required config keys, secret required?)
+_SPEC = {
+    "ntfy": ({"topic"}, False),
+    "gotify": ({"server_url"}, True),
+    "telegram": ({"chat_id"}, True),
+    "teams": (set(), True),
+    "matrix": ({"homeserver_url", "room_id"}, True),
+}
+assert set(_SPEC) == set(CHANNEL_TYPES)
+
+
+def secret_name(channel_id) -> str:
+    return f"channel_{channel_id}"
+
+
+def clean_config(channel_type: str, config: dict) -> dict:
+    """Validates and normalises the non-secret settings of a channel. Raises ValueError (German reason)."""
+    if channel_type not in _SPEC:
+        raise ValueError("Unbekannter Kanaltyp")
+    config = dict(config or {})
+    out: dict = {}
+
+    def url(key: str, default: str | None = None) -> str | None:
+        raw = str(config.get(key) or "").strip() or default
+        if raw is None:
+            return None
+        return url_guard.validate_url(raw).rstrip("/")
+
+    def integer(key: str, low: int, high: int, default: int) -> int:
+        value = config.get(key)
+        if value in (None, ""):
+            return default
+        try:
+            number = int(value)
+        except (TypeError, ValueError):
+            raise ValueError(f"{key}: ganze Zahl erwartet") from None
+        if not low <= number <= high:
+            raise ValueError(f"{key}: Wert zwischen {low} und {high} erwartet")
+        return number
+
+    if channel_type == "ntfy":
+        out["server_url"] = url("server_url", "https://ntfy.sh")
+        topic = str(config.get("topic") or "").strip()
+        if not _NTFY_TOPIC.match(topic):
+            raise ValueError("Thema: 1–64 Zeichen aus Buchstaben, Ziffern, _ und -")
+        out["topic"] = topic
+        out["priority"] = integer("priority", 1, 5, 4)
+    elif channel_type == "gotify":
+        out["server_url"] = url("server_url")
+        if not out["server_url"]:
+            raise ValueError("Serveradresse fehlt")
+        out["priority"] = integer("priority", 0, 10, 5)
+    elif channel_type == "telegram":
+        chat_id = str(config.get("chat_id") or "").strip()
+        if not re.match(r"^(-?\d{1,20}|@[A-Za-z0-9_]{4,64})$", chat_id):
+            raise ValueError("Chat-ID: Zahl (z. B. -1001234567890) oder @Kanalname")
+        out["chat_id"] = chat_id
+        out["api_base"] = url("api_base", "https://api.telegram.org")
+    elif channel_type == "matrix":
+        out["homeserver_url"] = url("homeserver_url")
+        if not out["homeserver_url"]:
+            raise ValueError("Homeserver-Adresse fehlt")
+        room = str(config.get("room_id") or "").strip()
+        if not _MATRIX_ROOM.match(room):
+            raise ValueError("Raum-ID: Form !abc123:example.org (nicht der Alias #raum:…)")
+        out["room_id"] = room
+    return out
+
+
+def clean_secret(channel_type: str, secret: str) -> str:
+    value = (secret or "").strip()
+    if not value:
+        raise ValueError("Zugangsdaten fehlen")
+    if channel_type == "teams":
+        value = url_guard.validate_url(value, max_length=2000)
+        if not value.startswith("https://"):
+            raise ValueError("Die Teams-Webhook-Adresse muss mit https:// beginnen")
+    elif any(c.isspace() for c in value) or len(value) > 500:
+        raise ValueError("Token: keine Leerzeichen, höchstens 500 Zeichen")
+    return value
+
+
+def missing_requirements(channel: NotificationChannel) -> str | None:
+    required, needs_secret = _SPEC[channel.channel_type]
+    if any(not channel.config.get(k) for k in required):
+        return "Konfiguration unvollständig"
+    if needs_secret and not channel.secret_set:
+        return "Zugangsdaten fehlen"
+    return None
+
+
+# ── Message formatting (one function per channel, pure — unit-testable) ───────────
+
+def _parts(problem: NotificationProblem, transition: str) -> tuple[str, str]:
+    """(title, details) — the title repeated at the top of render_message's body is dropped."""
+    subject, body = notification_delivery.render_message(problem, transition)
+    details = body.split("\n", 2)[2] if body.count("\n") >= 2 else ""
+    return subject, details.strip()
+
+
+def _clip(text: str, limit: int) -> str:
+    return text if len(text) <= limit else text[: limit - 1] + "…"
+
+
+def format_ntfy(config: dict, problem, transition: str) -> dict:
+    title, details = _parts(problem, transition)
+    resolved = transition == "resolved"
+    return {
+        "topic": config["topic"], "title": title, "message": details or title,
+        "priority": 3 if resolved else int(config.get("priority", 4)),
+        "tags": ["white_check_mark"] if resolved else ["warning"],
+    }
+
+
+def format_gotify(config: dict, problem, transition: str) -> dict:
+    title, details = _parts(problem, transition)
+    return {"title": title, "message": details or title, "priority": 1 if transition == "resolved" else int(config.get("priority", 5))}
+
+
+def format_telegram(config: dict, problem, transition: str) -> dict:
+    title, details = _parts(problem, transition)
+    return {
+        "chat_id": config["chat_id"], "text": _clip(f"{title}\n\n{details}".strip(), TELEGRAM_LIMIT),
+        "disable_web_page_preview": True,
+    }
+
+
+def format_teams(config: dict, problem, transition: str) -> dict:
+    title, details = _parts(problem, transition)
+    body: list[dict] = [{"type": "TextBlock", "text": title, "weight": "Bolder", "size": "Medium", "wrap": True}]
+    if details:
+        body.append({"type": "TextBlock", "text": _clip(details, TEAMS_LIMIT), "wrap": True})
+    return {
+        "type": "message",
+        "attachments": [{
+            "contentType": "application/vnd.microsoft.card.adaptive", "contentUrl": None,
+            "content": {
+                "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
+                "type": "AdaptiveCard", "version": "1.4", "body": body,
+            },
+        }],
+    }
+
+
+def format_matrix(config: dict, problem, transition: str) -> dict:
+    title, details = _parts(problem, transition)
+    return {"msgtype": "m.text", "body": f"{title}\n\n{details}".strip()}
+
+
+# ── Sending ───────────────────────────────────────────────────────────────────────
+
+def _request_for(channel: NotificationChannel, secret: str, problem, transition: str) -> tuple[str, str, dict, dict, str | None]:
+    """(method, url, headers, json body, expected "ok" check key)."""
+    cfg, kind = channel.config, channel.channel_type
+    if kind == "ntfy":
+        headers = {"Authorization": f"Bearer {secret}"} if secret else {}
+        return "POST", cfg["server_url"].rstrip("/") + "/", headers, format_ntfy(cfg, problem, transition), None
+    if kind == "gotify":
+        return "POST", cfg["server_url"].rstrip("/") + "/message", {"X-Gotify-Key": secret}, format_gotify(cfg, problem, transition), None
+    if kind == "telegram":
+        return "POST", f"{cfg['api_base'].rstrip('/')}/bot{secret}/sendMessage", {}, format_telegram(cfg, problem, transition), "ok"
+    if kind == "teams":
+        return "POST", secret, {}, format_teams(cfg, problem, transition), None
+    room = quote(cfg["room_id"], safe="")
+    url = f"{cfg['homeserver_url'].rstrip('/')}/_matrix/client/v3/rooms/{room}/send/m.room.message/{uuid.uuid4().hex}"
+    return "PUT", url, {"Authorization": f"Bearer {secret}"}, format_matrix(cfg, problem, transition), None
+
+
+async def send(channel: NotificationChannel, problem: NotificationProblem, transition: str) -> None:
+    problem_missing = missing_requirements(channel)
+    if problem_missing:
+        raise NotificationDeliveryError(f"Kanal „{channel.name}“: {problem_missing}")
+    secret = notification_secret_store.read_secret(secret_name(channel.id)) or ""
+    if channel.secret_set and not secret:
+        raise NotificationDeliveryError(f"Kanal „{channel.name}“: Zugangsdaten nicht lesbar – bitte neu setzen")
+    method, url, headers, payload, ok_key = _request_for(channel, secret, problem, transition)
+    label = f"Kanal „{channel.name}“ ({channel.channel_type})"
+    with url_guard.muted():
+        try:
+            url_guard.validate_url(url, max_length=2100)
+            await url_guard.ensure_resolved_safe(url)
+        except ValueError as exc:
+            raise NotificationDeliveryError(f"{label}: {exc}") from None
+        try:
+            async with url_guard.guarded_client(timeout=settings.opnsense_timeout, follow_redirects=False) as client:
+                resp = await client.request(method, url, json=payload, headers=headers)
+                resp.raise_for_status()
+                if ok_key and not (resp.json() or {}).get(ok_key):
+                    raise NotificationDeliveryError(f"{label}: Zustellung abgelehnt")
+        except httpx.HTTPError as exc:
+            # str(exc) names the full URL, which for Telegram/Teams is the credential itself.
+            reason = url_guard.failure_reason(exc)
+            raise NotificationDeliveryError(f"{label}: Zustellung fehlgeschlagen ({reason})") from None
+        except ValueError:
+            raise NotificationDeliveryError(f"{label}: Antwort nicht lesbar") from None
+
+
+async def send_test(channel: NotificationChannel) -> None:
+    fake = NotificationProblem(
+        problem_type="test", dedup_key="test",
+        context={"message": "Dies ist eine Testbenachrichtigung von OPNexus."},
+    )
+    await send(channel, fake, "opened")
+
+
+async def enabled_channels(db: AsyncSession) -> list[NotificationChannel]:
+    rows = await db.scalars(select(NotificationChannel).where(NotificationChannel.enabled.is_(True)).order_by(NotificationChannel.name))
+    return list(rows.all())

backend/app/services/notification_checks.py

@@ -363,3 +363,49 @@ async def check_compliance(db: AsyncSession) -> dict[str, dict]:
                     "beispiele": ", ".join(params.get("examples", [])) or None,
                 }
     return problems
+
+
+# ── Rollouts (1.25) ─────────────────────────────────────────────────────────
+
+def _short(detail) -> str | None:
+    if not detail:
+        return None
+    parts = []
+    for key, value in detail.items():
+        if isinstance(value, list):
+            value = ", ".join(str(v) for v in value[:8])
+        parts.append(f"{key}: {value}")
+    return "; ".join(parts)[:500]
+
+
+async def check_rollout_paused(db: AsyncSession) -> dict[str, dict]:
+    """Open while a rollout run stands still waiting for a human (gate, failed host, window gone...)."""
+    from app.models.rollout import RolloutRun
+    problems: dict[str, dict] = {}
+    for run in (await db.scalars(select(RolloutRun).where(RolloutRun.status == "paused"))).all():
+        wave = next((w.label for w in run.waves if w.index == run.current_wave), None)
+        problems[str(run.id)] = {
+            "run_id": str(run.id), "plan": run.plan_name, "grund": run.pause_reason, "welle": wave,
+            "zielversion": run.target_version, "details": _short(run.pause_detail),
+        }
+    return problems
+
+
+ROLLOUT_START_FAILURE_WINDOW = timedelta(days=3)
+
+
+async def check_rollout_start_failed(db: AsyncSession) -> dict[str, dict]:
+    """An automatic start at a window occurrence did not happen (event: closed without a 'resolved' notice)."""
+    from app.models.rollout import RolloutPlan
+    since = datetime.now(timezone.utc) - ROLLOUT_START_FAILURE_WINDOW
+    problems: dict[str, dict] = {}
+    for plan in (await db.scalars(select(RolloutPlan).where(RolloutPlan.auto_start_error.is_not(None)))).all():
+        error = plan.auto_start_error or {}
+        at = datetime.fromisoformat(error["at"]) if error.get("at") else None
+        if at is None or at < since:
+            continue
+        problems[f"{plan.id}:{error.get('occurrence')}"] = {
+            "plan_id": str(plan.id), "plan": plan.name, "grund": error.get("code"), "termin": error.get("occurrence"),
+            "details": _short(error.get("params")),
+        }
+    return problems

backend/app/services/notification_delivery.py

@@ -16,7 +16,7 @@ import httpx
 
 from app.config import settings
 from app.models.notification import NotificationProblem, NotificationSettings
-from app.services import notification_secret_store
+from app.services import notification_secret_store, url_guard
 
 
 class NotificationDeliveryError(Exception):
@@ -37,6 +37,13 @@ _LABELS = {
     "backup_stale": "Kein aktuelles Backup",
     "backup_offsite_failed": "Offsite-Backup fehlgeschlagen",
     "compliance_finding": "Sicherheitsbefund",
+    "gateway_down": "Gateway gestört",
+    "resource_threshold": "Ressourcen-Schwelle überschritten",
+    "service_down": "Dienst ausgefallen",
+    "api_rights_missing": "API-Rechte fehlen",
+    "rollout_paused": "Rollout angehalten",
+    "rollout_start_failed": "Rollout nicht automatisch gestartet",
+    "test": "Testbenachrichtigung",
 }
 
 
@@ -84,13 +91,20 @@ async def send_webhook(settings_row: NotificationSettings, problem: Notification
         headers["X-OPNexus-Signature"] = f"sha256={signature}"
 
     try:
-        async with httpx.AsyncClient(timeout=settings.opnsense_timeout) as client:
-            resp = await client.post(settings_row.webhook_url, content=raw, headers=headers)
-            resp.raise_for_status()
+        url_guard.validate_url(settings_row.webhook_url)
+        await url_guard.ensure_resolved_safe(settings_row.webhook_url)
+    except ValueError as exc:
+        raise NotificationDeliveryError(f"Webhook-Adresse nicht erlaubt: {exc}") from None
+    try:
+        # follow_redirects stays off: a redirect must not move the request to an address the check above never saw.
+        async with url_guard.guarded_client(timeout=settings.opnsense_timeout, follow_redirects=False) as client:
+            with url_guard.muted():
+                resp = await client.post(settings_row.webhook_url, content=raw, headers=headers)
+                resp.raise_for_status()
     except httpx.HTTPError as exc:
         # str(exc) names the full URL, which often is the credential itself
         # (token in path/query) — keep it out of messages and logs.
-        reason = f"HTTP {exc.response.status_code}" if isinstance(exc, httpx.HTTPStatusError) else type(exc).__name__
+        reason = url_guard.failure_reason(exc)
         raise NotificationDeliveryError(f"Webhook-Zustellung fehlgeschlagen ({reason})") from None
 
 

backend/app/services/notification_service.py

@@ -3,13 +3,15 @@ against previously open problems, and delivers on transitions. The single
 testable entry point — called directly by main.py's background loop AND
 directly by tests, same pattern as rule_deploy_service.sweep_expired_deployments.
 """
+import asyncio
 import logging
 from datetime import datetime, timezone
 
+from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.models.notification import NotificationProblem, NotificationSettings
-from app.services import notification_checks, notification_dedup, notification_delivery
+from app.models.notification import NotificationChannel, NotificationProblem, NotificationSettings
+from app.services import monitor_checks, notification_channels, notification_checks, notification_dedup, notification_delivery
 from app.services import notification_settings_service
 
 logger = logging.getLogger(__name__)
@@ -34,41 +36,108 @@ _CHECKS = [
     ("backup_stale", "backup_failed_enabled", notification_checks.check_backup_stale),
     ("backup_offsite_failed", "backup_failed_enabled", notification_checks.check_backup_offsite),
     ("compliance_finding", "compliance_alerts_enabled", notification_checks.check_compliance),
+    ("gateway_down", "gateway_down_enabled", monitor_checks.check_gateway_down),
+    ("resource_threshold", "resource_threshold_enabled", monitor_checks.check_resource_threshold),
+    ("service_down", "service_down_enabled", monitor_checks.check_service_down),
+    ("api_rights_missing", "api_rights_enabled", monitor_checks.check_api_rights_missing),
+    ("rollout_paused", "rollout_problems_enabled", notification_checks.check_rollout_paused),
+    ("rollout_start_failed", "rollout_problems_enabled", notification_checks.check_rollout_start_failed),
 ]
 
 # Event-like problems: when they drop out of the window nothing was "fixed", so no
 # "behoben" message goes out for them.
-_NO_RESOLVED_NOTICE = {"scheduled_maintenance_failed"}
+_NO_RESOLVED_NOTICE = {"scheduled_maintenance_failed", "rollout_start_failed"}
 
 
-async def _deliver(settings_row: NotificationSettings, problem: NotificationProblem, transition: str) -> None:
+# Delivery limits (1.29): a hanging channel must neither delay the others nor hold up the check loop.
+DELIVERY_TIMEOUT_SECONDS = 20.0    # one target (webhook, e-mail, one channel) for one message
+DELIVERY_CYCLE_BUDGET_SECONDS = 90.0  # everything one check cycle sends, in total
+DELIVERY_PARALLEL = 10             # targets served at the same time within a cycle
+
+
+async def _attempt(label: str, make_call, gate: asyncio.Semaphore | None = None, started: list | None = None) -> None:
+    """One delivery target: its own timeout, its own failure — never an exception for the caller. Texts logged
+    here come from NotificationDeliveryError (names the channel and the HTTP status, never the URL or token)."""
+    try:
+        if gate is not None:
+            await gate.acquire()
+        if started is not None:
+            started.append(label)  # the target was really tried (a cancelled wait for the gate does not count)
+        try:
+            await asyncio.wait_for(make_call(), DELIVERY_TIMEOUT_SECONDS)
+        finally:
+            if gate is not None:
+                gate.release()
+    except notification_delivery.NotificationDeliveryError:
+        logger.exception("%s delivery failed", label)
+    except asyncio.TimeoutError:
+        logger.warning("%s delivery timed out after %.0f s", label, DELIVERY_TIMEOUT_SECONDS)
+    except asyncio.CancelledError:
+        raise
+    except Exception as exc:  # a broken target must not take the others down
+        logger.warning("%s delivery failed unexpectedly (%s)", label, type(exc).__name__)
+
+
+async def _deliver(
+    settings_row: NotificationSettings, problem: NotificationProblem, transition: str, channels: list[NotificationChannel] = (),
+    gate: asyncio.Semaphore | None = None,
+) -> None:
     """Best-effort, not an audit trail: attempted once per transition and
     never retried, even on failure — avoids retry-storming a broken webhook
-    URL or SMTP relay every poll cycle."""
+    URL or SMTP relay every poll cycle. Webhook, e-mail and every channel are
+    served in parallel, each with its own timeout (the order has no meaning)."""
     if transition == "resolved" and problem.problem_type in _NO_RESOLVED_NOTICE:
         problem.resolved_notified_at = _now()
         return
+    where = f"{problem.problem_type}/{problem.dedup_key}"
+    calls = []
+    started: list = []
     if settings_row.webhook_enabled:
-        try:
-            await notification_delivery.send_webhook(settings_row, problem, transition)
-        except notification_delivery.NotificationDeliveryError:
-            logger.exception("Webhook delivery failed for %s/%s", problem.problem_type, problem.dedup_key)
+        calls.append(_attempt(f"Webhook for {where}", lambda: notification_delivery.send_webhook(settings_row, problem, transition), gate, started))
     if settings_row.email_enabled:
-        try:
+        async def email():
             subject, body = notification_delivery.render_message(problem, transition)
             await notification_delivery.send_email(settings_row, subject, body)
-        except notification_delivery.NotificationDeliveryError:
-            logger.exception("Email delivery failed for %s/%s", problem.problem_type, problem.dedup_key)
+        calls.append(_attempt(f"Email for {where}", email, gate, started))
+    for channel in channels:
+        calls.append(_attempt(f"Channel {channel.name} ({channel.channel_type}) for {where}",
+                              lambda channel=channel: notification_channels.send(channel, problem, transition), gate, started))
+    try:
+        await asyncio.gather(*calls, return_exceptions=True)
+    finally:
+        # "Attempted" is what is recorded (also when a target failed or timed out). A message the cycle budget
+        # cancelled before ANY target was tried stays unrecorded: an opened problem is sent again next cycle.
+        if started or not calls:
+            if transition == "resolved":
+                problem.resolved_notified_at = _now()
+            else:
+                problem.last_notified_at = _now()
 
-    if transition == "resolved":
-        problem.resolved_notified_at = _now()
-    else:
-        problem.last_notified_at = _now()
+
+async def _deliver_all(settings_row: NotificationSettings, pending: list[tuple[NotificationProblem, str]],
+                       channels: list[NotificationChannel]) -> None:
+    """Everything this cycle found, in parallel, within DELIVERY_CYCLE_BUDGET_SECONDS in total."""
+    if not pending:
+        return
+    gate = asyncio.Semaphore(DELIVERY_PARALLEL)
+    tasks = [asyncio.ensure_future(_deliver(settings_row, problem, transition, channels, gate)) for problem, transition in pending]
+    done, late = await asyncio.wait(tasks, timeout=DELIVERY_CYCLE_BUDGET_SECONDS)
+    for task in late:
+        task.cancel()
+    if late:
+        logger.warning("Notification delivery stopped after %.0f s: %d of %d messages were not completed",
+                       DELIVERY_CYCLE_BUDGET_SECONDS, len(late), len(tasks))
+        await asyncio.gather(*late, return_exceptions=True)  # let the cancelled ones record "attempted"
+    for task in done:
+        if not task.cancelled() and task.exception() is not None:
+            logger.warning("Notification delivery failed unexpectedly (%s)", type(task.exception()).__name__)
 
 
 async def run_notification_checks(db: AsyncSession) -> dict:
     settings_row = await notification_settings_service.get_settings(db)
     result = {"opened": 0, "resolved": 0}
+    channels = await notification_channels.enabled_channels(db)
+    pending: list[tuple[NotificationProblem, str]] = []
 
     for problem_type, enabled_flag, check_fn in _CHECKS:
         if not getattr(settings_row, enabled_flag):
@@ -84,12 +153,21 @@ async def run_notification_checks(db: AsyncSession) -> dict:
             continue
 
         opened, resolved = await notification_dedup.diff_and_update(db, problem_type, current)
-        for problem in opened:
-            await _deliver(settings_row, problem, "opened")
-        for problem in resolved:
-            await _deliver(settings_row, problem, "resolved")
+        pending += [(problem, "opened") for problem in opened] + [(problem, "resolved") for problem in resolved]
         result["opened"] += len(opened)
         result["resolved"] += len(resolved)
 
+    # Open problems that no target was ever tried for (an earlier cycle ran out of time) are sent now.
+    enabled_types = [t for t, flag, _fn in _CHECKS if getattr(settings_row, flag)]
+    already = {id(problem) for problem, _transition in pending}
+    undelivered = (await db.scalars(
+        select(NotificationProblem).where(
+            NotificationProblem.status == "open", NotificationProblem.last_notified_at.is_(None),
+            NotificationProblem.problem_type.in_(enabled_types),
+        )
+    )).all() if enabled_types else []
+    pending += [(problem, "opened") for problem in undelivered if id(problem) not in already]
+    # Delivered once all checks ran (and in parallel): a slow target no longer holds up the checks after it.
+    await _deliver_all(settings_row, pending, channels)
     await db.commit()
     return result

backend/app/services/notification_settings_service.py

@@ -26,6 +26,10 @@ async def update_settings(db: AsyncSession, payload: NotificationSettingsUpdate)
         "host_offline_enabled", "cert_expiring_enabled", "rule_drift_enabled",
         "revert_failed_enabled", "maintenance_unknown_enabled", "carp_health_enabled", "nat_drift_enabled",
         "network_drift_enabled", "carp_auto_revert_failed_enabled", "scheduled_maintenance_failed_enabled", "backup_failed_enabled", "compliance_alerts_enabled",
+        "gateway_down_enabled", "resource_threshold_enabled", "service_down_enabled", "rollout_problems_enabled", "api_rights_enabled",
+        "gateway_loss_threshold_pct", "gateway_delay_threshold_ms", "disk_threshold_pct", "memory_threshold_pct",
+        "swap_threshold_pct", "load_per_cpu_threshold", "alert_confirm_samples", "resource_confirm_samples",
+        "service_watchlist",
         "poll_interval_seconds",
     ]
     for field in simple_fields:

backend/app/services/object_adopt_service.py

@@ -0,0 +1,627 @@
+"""Takes over aliases, port forwards, 1:1 NAT and outbound NAT rules that were created by hand on a
+firewall, so they can be edited, versioned and deployed from OPNexus like any other object. This is
+the generalisation of rule_adopt_service.py (filter rules keep their own module: their model has
+more special cases and the code is relied upon).
+
+Adoption itself writes NOTHING to the firewall. It records the object as it is right now (read fresh
+from the host at adoption time, never from what the browser sent) together with its OPNsense UUID
+(`remote_uuid` = the deployment's opnsense_*_uuid), and stores the payload OPNexus would send for it as
+the deployment's last_payload. So the preview directly afterwards says "unchanged", a later deploy
+updates the same object via setItem/setRule (no duplicate), and the normal preview → push →
+confirm/auto-revert flow applies from the first change on.
+
+An object that already exists centrally for another firewall is LINKED instead of adopted a second time
+(adopt_link_service.py: exact match after resolving the host's variables, never automatic for a mere
+resemblance).
+
+Only objects OPNexus can describe without distortion are adopted. Anything the model cannot express
+is refused with machine-readable reasons (the UI translates them) instead of being imported in a way
+that would silently change its meaning on the next edit. Fields OPNexus does not manage (log flag of a
+port forward, pool options, tags, categories, ...) are never sent, and OPNsense's set*/setItem only
+overwrites the posted fields (ApiMutableModelControllerBase::setBase -> setNodes), so they stay as
+they are; the candidate list shows them as `kept`.
+
+Evidence (OPNsense core master, src/opnsense/mvc/app/…, read 2026-10-20, and live against OPNsense
+26.7 on test-fw-3 on the same day with hand-made objects of every case below):
+
+* Aliases — controllers/OPNsense/Firewall/Api/AliasController.php::searchItemAction -> searchBase
+  ("aliases.alias"), model models/OPNsense/Firewall/Alias.xml. Row keys used: uuid, name, type
+  ("host", "network", "port", "networkgroup", "mac", "asn", plus the unsupported "url", "urltable",
+  "urljson", "geoip", "dynipv6host", "authgroup", and the system types "internal"/"external"),
+  enabled "1"/"0", content (entries separated by "\n", live-verified; for "authgroup" the controller
+  rewrites it to group names), description, proto (only meaningful for asn/geoip). The built-in rows
+  are bogons, bogonsv6, sshlockout, virusprot (type external) and one __<if>_network per interface
+  (type internal); they are OPNsense's own and never adoptable. A "networkgroup" lists alias NAMES;
+  host/network/port aliases may also contain alias names (nesting).
+* Port forward — controllers/OPNsense/Firewall/Api/DNatController.php::searchRuleAction merges the
+  configured rules (getFlatNodes: dotted keys "source.network", "source.port", "source.not",
+  "destination.network", "destination.port", "destination.not", plus descr, disabled, interface,
+  ipprotocol, protocol, target, local-port, natreflection, nordr, pass, log, poolopts, tag, tagged,
+  nosync, categories, sequence) with synthetic automatic rows (uuid "automatic_*"/"lockout_*",
+  is_automatic true) that setRule/delRule cannot target. models/OPNsense/Firewall/DNat.xml: interface is
+  Multiple=Y (comma separated), ipprotocol may be blank/inet/inet6/inet46, protocol blank = any,
+  `nordr` ("no redirect") requires an empty target/local-port (live: "Target must be empty when No
+  RDR is set"), `pass` ("" manual / "pass" / "rule") links an associated filter rule, natreflection is
+  ""/purenat/disable. OPNexus sends pass "" and not "0", so a rule using one of those would change
+  meaning on its first edit and is refused.
+* 1:1 NAT — OneToOneController::searchRuleAction (flat keys), model Filter.xml onetoone.rule:
+  enabled, log, interface, type (binat/nat), source_net/source_not, destination_net/destination_not,
+  external, natreflection ""/enable/disable, description. All of these are managed by OPNexus.
+* Outbound NAT — SourceNatController::searchRuleAction returns the configured rules ONLY when the
+  global mode (filter.settings.nat.snat_mode) is hybrid or advanced; in automatic/disabled mode the
+  manual rules do not show up at all, so adopting is refused up front (outbound_nat_mode.py) instead
+  of presenting an empty list. Model Filter.xml snatrules.rule: `nonat` rules ("do not NAT") are not
+  expressible and refused; staticnatport, poolopts, log, tag, tagged, nosync, endpoint-independent
+  are left alone.
+"""
+import re
+from dataclasses import dataclass, field
+from typing import Any, Awaitable, Callable, get_args
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.errors import ErrorCode, api_error
+from app.models.alias import FirewallAlias
+from app.models.alias_deployment import AliasDeployment
+from app.models.host import OpnsenseHost
+from app.models.one_to_one import OneToOneRule
+from app.models.one_to_one_deployment import OneToOneDeployment
+from app.models.outbound_nat import OutboundNatRule
+from app.models.outbound_nat_deployment import OutboundNatDeployment
+from app.models.port_forward import PortForwardRule
+from app.models.port_forward_deployment import PortForwardDeployment
+from app.schemas.alias import AliasType
+from app.services import alias_deploy_service, alias_policy_service
+from app.services import one_to_one_deploy_service, one_to_one_policy_service
+from app.services import opnsense_client, outbound_nat_deploy_service, outbound_nat_policy_service
+from app.services import port_forward_deploy_service, port_forward_policy_service
+from app.services import adopt_link_service
+from app.services.adopt_common import NotAdoptable, NotFoundOnHost, clean, on
+from app.services.nat_live_service import split_rows
+from app.services.outbound_nat_mode import host_mode
+
+ADOPT_NOTE = "Von Firewall übernommen"
+ADOPT_MESSAGE = "Von der Firewall übernommen"
+
+ALIAS_NAME = re.compile(r"^[A-Za-z0-9_]+$")  # schemas/alias.py AliasCreate.name
+ALIAS_TYPES = frozenset(get_args(AliasType))
+ALIAS_CONTENT_MAX = 8000  # schemas/alias.py AliasCreate.content
+# OPNsense's own aliases that are not user objects (Firewall/Alias.php: bogons & co. are `external`
+# rows created by the system, __<if>_network are `internal`).
+SYSTEM_ALIAS_NAMES = frozenset({"bogons", "bogonsv6", "sshlockout", "virusprot"})
+LINK_TOLERATED = frozenset({"name_exists", "references_unmanaged_alias"})
+
+
+@dataclass
+class Entry:
+    row: dict
+    reasons: list[str] = field(default_factory=list)
+    details: dict[str, str] = field(default_factory=dict)
+    bulk: bool = False  # not adoptable alone yet, but adoptable in the same run (its references come first)
+    matches: list = field(default_factory=list)  # central objects equal to the live object (link candidates)
+    similar: list = field(default_factory=list)  # same name/description but different content (hint only)
+
+    @property
+    def linkable(self) -> bool:
+        """An exact central twin exists and nothing but "the name is taken" / "its references are not
+        managed here yet" speaks against it (both are moot when the twin is linked, its content is equal)."""
+        return any(not m.busy for m in self.matches) and set(self.reasons) <= LINK_TOLERATED
+
+    @property
+    def busy_twin(self) -> bool:
+        """The one exact twin waits for a confirmation: \"adopt all\" leaves this entry alone (neither link nor duplicate)."""
+        return len(self.matches) == 1 and self.matches[0].busy
+
+    @property
+    def uuid(self) -> str:
+        return clean(self.row.get("uuid"))
+
+
+@dataclass(frozen=True)
+class Kind:
+    key: str
+    model: type
+    deployment: type
+    fk: str  # deployment column pointing at the central object
+    uuid_attr: str  # deployment column holding the OPNsense UUID (= remote_uuid)
+    fetch: Callable[[OpnsenseHost], Awaitable[list[dict]]]
+    check: Callable[[dict, dict], tuple[list[str], dict[str, str]]]
+    title: Callable[[dict], str]
+    info: Callable[[dict], dict[str, str]]
+    build: Callable[[dict], dict]
+    payload: Callable[[Any], dict]
+    revision: Callable
+    kept: Callable[[dict], list[str]] = lambda row: []
+    context: Callable[[AsyncSession, list[dict]], Awaitable[dict]] | None = None
+    refs: Callable[[dict], set[str]] | None = None
+    name: Callable[[dict], str] | None = None
+    identity: str = "description"  # attribute that names the object (the alias name for aliases)
+
+    @property
+    def link_spec(self) -> adopt_link_service.LinkSpec:
+        return adopt_link_service.LinkSpec(
+            model=self.model, deployment=self.deployment, fk=self.fk, uuid_attr=self.uuid_attr,
+            identity=self.identity, payload=self.payload, revision=self.revision,
+        )
+
+
+# ── shared helpers ───────────────────────────────────────────────────────
+
+def _single_interface(row: dict) -> bool:
+    interface = clean(row.get("interface"))
+    return bool(interface) and "," not in interface
+
+
+def _too_long(kind_model: type, values: dict) -> list[str]:
+    """Columns are bounded; a live value that does not fit would fail (or be cut) on the first edit."""
+    columns = kind_model.__table__.columns
+    return [name for name, value in values.items()
+            if name in columns and getattr(columns[name].type, "length", None)
+            and isinstance(value, str) and len(value) > columns[name].type.length]
+
+
+def _port(value) -> str | None:
+    return clean(value) or None
+
+
+def _kept(row: dict, keys: tuple[str, ...]) -> list[str]:
+    return [key for key in keys if clean(row.get(key)) not in ("", "0")]
+
+
+def _fetch_nat(search) -> Callable[[OpnsenseHost], Awaitable[list[dict]]]:
+    async def fetch(host: OpnsenseHost) -> list[dict]:
+        data = await search(host)
+        # split_rows drops automatic_*/lockout_* and is_automatic rows: they are OPNsense's own
+        return [r for r in split_rows(data)["rules"] if clean(r.get("uuid"))]
+    return fetch
+
+
+# ── aliases ──────────────────────────────────────────────────────────────
+
+def _alias_content(row: dict) -> str:
+    # same normalisation as AliasCreate.clean_content
+    return "\n".join(line.strip() for line in str(row.get("content") or "").splitlines() if line.strip())
+
+
+def _alias_is_system(row: dict) -> bool:
+    name = clean(row.get("name"))
+    return clean(row.get("type")) == "internal" or name.startswith("__") or name in SYSTEM_ALIAS_NAMES
+
+
+async def _fetch_aliases(host: OpnsenseHost) -> list[dict]:
+    data = await opnsense_client.search_aliases(host)
+    rows = data.get("rows", []) if isinstance(data, dict) else []
+    return [r for r in rows if isinstance(r, dict) and clean(r.get("uuid")) and clean(r.get("name"))]
+
+
+async def _alias_context(db: AsyncSession, rows: list[dict]) -> dict:
+    return {"central_names": set((await db.scalars(select(FirewallAlias.name))).all())}
+
+
+def _alias_refs(row: dict) -> set[str]:
+    return {line.lstrip("!").strip() for line in _alias_content(row).splitlines()} - {""}
+
+
+def _alias_check(row: dict, ctx: dict) -> tuple[list[str], dict[str, str]]:
+    if _alias_is_system(row):
+        return ["system"], {}
+    reasons: list[str] = []
+    details: dict[str, str] = {}
+    name, type_ = clean(row.get("name")), clean(row.get("type"))
+    if type_ not in ALIAS_TYPES:
+        reasons.append("unsupported_type")
+        details["unsupported_type"] = clean(row.get("%type")) or type_
+    if not ALIAS_NAME.fullmatch(name) or len(name) > 64:
+        reasons.append("name_invalid")
+    if name in ctx["central_names"]:
+        reasons.append("name_exists")
+    if len(_alias_content(row)) > ALIAS_CONTENT_MAX:
+        reasons.append("content_too_large")
+    if type_ == "asn" and clean(row.get("proto")):
+        reasons.append("proto_set")  # OPNexus has no field for it: a deploy elsewhere would drop the restriction
+    if _too_long(FirewallAlias, {"description": clean(row.get("description"))}):
+        reasons.append("value_too_long")
+    return reasons, details
+
+
+def _alias_info(row: dict) -> dict[str, str]:
+    lines = _alias_content(row).splitlines()
+    return {
+        "type": clean(row.get("type")),
+        "type_label": clean(row.get("%type")) or clean(row.get("type")),
+        "description": clean(row.get("description")),
+        "content": "\n".join(lines[:6]) + ("\n…" if len(lines) > 6 else ""),
+        "entries": str(len(lines)),
+    }
+
+
+def _alias_build(row: dict) -> dict:
+    return {
+        "name": clean(row.get("name")),
+        "type": clean(row.get("type")),
+        "content": _alias_content(row),
+        "description": clean(row.get("description")),
+        "enabled": on(row.get("enabled")),
+    }
+
+
+# ── port forward ─────────────────────────────────────────────────────────
+
+def _pf_check(row: dict, ctx: dict) -> tuple[list[str], dict[str, str]]:
+    reasons = []
+    if not _single_interface(row):
+        reasons.append("interface_not_single")
+    if clean(row.get("ipprotocol")) not in ("inet", "inet6"):
+        reasons.append("ipprotocol_unsupported")  # blank / inet46 have no counterpart in OPNexus
+    if not clean(row.get("protocol")):
+        reasons.append("protocol_any")
+    if on(row.get("nordr")):
+        reasons.append("no_redirect")
+    elif not clean(row.get("target")):
+        reasons.append("no_target")
+    if clean(row.get("pass")):
+        reasons.append("filter_rule_linked")  # OPNexus sends pass "" (manual) and would cut the link
+    if on(row.get("source.not")):
+        reasons.append("negated_source")
+    if on(row.get("destination.not")):
+        reasons.append("negated_destination")
+    if clean(row.get("natreflection")) not in ("", "purenat", "disable"):
+        reasons.append("natreflection_unsupported")
+    if not clean(row.get("descr")):
+        reasons.append("no_description")
+    if _too_long(PortForwardRule, _pf_build(row)):
+        reasons.append("value_too_long")
+    return reasons, {}
+
+
+def _pf_build(row: dict) -> dict:
+    return {
+        "description": clean(row.get("descr")),
+        "interface": clean(row.get("interface")),
+        "ipprotocol": clean(row.get("ipprotocol")),
+        "protocol": clean(row.get("protocol")),
+        "source_net": clean(row.get("source.network")) or "any",
+        "source_port": _port(row.get("source.port")),
+        "destination_net": clean(row.get("destination.network")) or "any",
+        "destination_port": _port(row.get("destination.port")),
+        "target": clean(row.get("target")),
+        "local_port": _port(row.get("local-port")),
+        "natreflection": clean(row.get("natreflection")),
+        "enabled": not on(row.get("disabled")),
+    }
+
+
+def _pf_info(row: dict) -> dict[str, str]:
+    return {
+        "interface_label": clean(row.get("%interface")) or clean(row.get("interface")),
+        "protocol": clean(row.get("%protocol")) or clean(row.get("protocol")) or "any",
+        "source": clean(row.get("source.network")) or "any",
+        "source_port": clean(row.get("source.port")),
+        "destination": clean(row.get("destination.network")) or "any",
+        "destination_port": clean(row.get("destination.port")),
+        "target": clean(row.get("target")),
+        "target_port": clean(row.get("local-port")),
+    }
+
+
+# ── 1:1 NAT ──────────────────────────────────────────────────────────────
+
+def _o2o_check(row: dict, ctx: dict) -> tuple[list[str], dict[str, str]]:
+    reasons = []
+    if not _single_interface(row):
+        reasons.append("interface_not_single")
+    if clean(row.get("type")) not in ("binat", "nat"):
+        reasons.append("type_unsupported")
+    if clean(row.get("natreflection")) not in ("", "enable", "disable"):
+        reasons.append("natreflection_unsupported")
+    if not clean(row.get("source_net")) or not clean(row.get("external")):
+        reasons.append("incomplete")
+    if not clean(row.get("description")):
+        reasons.append("no_description")
+    if _too_long(OneToOneRule, _o2o_build(row)):
+        reasons.append("value_too_long")
+    return reasons, {}
+
+
+def _o2o_build(row: dict) -> dict:
+    return {
+        "description": clean(row.get("description")),
+        "interface": clean(row.get("interface")),
+        "source_net": clean(row.get("source_net")),
+        "destination_net": clean(row.get("destination_net")) or "any",
+        "source_not": on(row.get("source_not")),
+        "destination_not": on(row.get("destination_not")),
+        "enabled": on(row.get("enabled")),
+        "type": clean(row.get("type")),
+        "external": clean(row.get("external")),
+        "natreflection": clean(row.get("natreflection")),
+        "log": on(row.get("log")),
+    }
+
+
+def _o2o_info(row: dict) -> dict[str, str]:
+    return {
+        "interface_label": clean(row.get("%interface")) or clean(row.get("interface")),
+        "type": clean(row.get("type")),
+        "source": ("! " if on(row.get("source_not")) else "") + clean(row.get("source_net")),
+        "destination": ("! " if on(row.get("destination_not")) else "") + (clean(row.get("destination_net")) or "any"),
+        "external": clean(row.get("external")),
+    }
+
+
+# ── outbound NAT ─────────────────────────────────────────────────────────
+
+async def _fetch_outbound(host: OpnsenseHost) -> list[dict]:
+    # In automatic/disabled mode OPNsense does not list manual rules at all (see module docstring):
+    # say so instead of showing an empty list.
+    state = await host_mode(host)
+    if not state["can_deploy"]:
+        raise api_error(409, ErrorCode.OUTBOUND_MODE_BLOCKS_ADOPT, host=host.name, mode=state["mode"])
+    return await _fetch_nat(opnsense_client.search_outbound_nat_rules)(host)
+
+
+def _snat_check(row: dict, ctx: dict) -> tuple[list[str], dict[str, str]]:
+    reasons = []
+    if not _single_interface(row):
+        reasons.append("interface_not_single")
+    if on(row.get("nonat")):
+        reasons.append("no_nat_rule")
+    if clean(row.get("ipprotocol")) not in ("inet", "inet6"):
+        reasons.append("ipprotocol_unsupported")
+    if not clean(row.get("protocol")):
+        reasons.append("protocol_missing")
+    if not clean(row.get("description")):
+        reasons.append("no_description")
+    if _too_long(OutboundNatRule, _snat_build(row)):
+        reasons.append("value_too_long")
+    return reasons, {}
+
+
+def _snat_build(row: dict) -> dict:
+    return {
+        "description": clean(row.get("description")),
+        "interface": clean(row.get("interface")),
+        "source_net": clean(row.get("source_net")) or "any",
+        "destination_net": clean(row.get("destination_net")) or "any",
+        "source_not": on(row.get("source_not")),
+        "destination_not": on(row.get("destination_not")),
+        "enabled": on(row.get("enabled")),
+        "ipprotocol": clean(row.get("ipprotocol")),
+        "protocol": clean(row.get("protocol")),
+        "source_port": _port(row.get("source_port")),
+        "destination_port": _port(row.get("destination_port")),
+        "target": clean(row.get("target")),
+        "target_port": _port(row.get("target_port")),
+    }
+
+
+def _snat_info(row: dict) -> dict[str, str]:
+    return {
+        "interface_label": clean(row.get("%interface")) or clean(row.get("interface")),
+        "protocol": clean(row.get("protocol")) or "any",
+        "source": ("! " if on(row.get("source_not")) else "") + (clean(row.get("source_net")) or "any"),
+        "source_port": clean(row.get("source_port")),
+        "destination": ("! " if on(row.get("destination_not")) else "") + (clean(row.get("destination_net")) or "any"),
+        "destination_port": clean(row.get("destination_port")),
+        "target": clean(row.get("target")),
+        "target_port": clean(row.get("target_port")),
+    }
+
+
+KINDS: dict[str, Kind] = {
+    "alias": Kind(
+        key="alias", model=FirewallAlias, deployment=AliasDeployment, fk="alias_id",
+        uuid_attr="opnsense_alias_uuid", fetch=_fetch_aliases, check=_alias_check,
+        title=lambda row: clean(row.get("name")), info=_alias_info, build=_alias_build,
+        payload=alias_deploy_service._build_payload, revision=alias_policy_service.revision,
+        context=_alias_context, refs=_alias_refs, name=lambda row: clean(row.get("name")), identity="name",
+    ),
+    "port_forward": Kind(
+        key="port_forward", model=PortForwardRule, deployment=PortForwardDeployment, fk="port_forward_id",
+        uuid_attr="opnsense_port_forward_uuid", fetch=_fetch_nat(opnsense_client.search_port_forward_rules),
+        check=_pf_check, title=lambda row: clean(row.get("descr")), info=_pf_info, build=_pf_build,
+        payload=port_forward_deploy_service._build_payload, revision=port_forward_policy_service.revision,
+        kept=lambda row: _kept(row, ("log", "poolopts", "tag", "tagged", "nosync", "categories")),
+    ),
+    "one_to_one": Kind(
+        key="one_to_one", model=OneToOneRule, deployment=OneToOneDeployment, fk="one_to_one_id",
+        uuid_attr="opnsense_one_to_one_uuid", fetch=_fetch_nat(opnsense_client.search_one_to_one_rules),
+        check=_o2o_check, title=lambda row: clean(row.get("description")), info=_o2o_info, build=_o2o_build,
+        payload=one_to_one_deploy_service._build_payload, revision=one_to_one_policy_service.revision,
+        kept=lambda row: _kept(row, ("categories",)),
+    ),
+    "outbound_nat": Kind(
+        key="outbound_nat", model=OutboundNatRule, deployment=OutboundNatDeployment, fk="outbound_nat_id",
+        uuid_attr="opnsense_outbound_nat_uuid", fetch=_fetch_outbound,
+        check=_snat_check, title=lambda row: clean(row.get("description")), info=_snat_info, build=_snat_build,
+        payload=outbound_nat_deploy_service._build_payload, revision=outbound_nat_policy_service.revision,
+        kept=lambda row: _kept(row, ("log", "staticnatport", "poolopts", "poolopts_sourcehashkey", "tag", "tagged",
+                                      "nosync", "endpoint-independent", "categories")),
+    ),
+}
+
+
+# ── engine ───────────────────────────────────────────────────────────────
+
+async def _managed_uuids(db: AsyncSession, kind: Kind, host: OpnsenseHost) -> set[str]:
+    column = getattr(kind.deployment, kind.uuid_attr)
+    rows = await db.scalars(select(column).where(kind.deployment.host_id == host.id, column.is_not(None)))
+    return set(rows.all())
+
+
+def _resolve_references(kind: Kind, entries: list[Entry], rows: list[dict], managed: set[str], ctx: dict) -> list[Entry]:
+    """Aliases may refer to other aliases by name. A reference to a live alias OPNexus does not manage
+    (and that is not OPNsense's own) would be dangling in the central model, so the referring alias
+    waits until its target is managed. Returns the entries in an order in which they can be adopted
+    in ONE run (dependencies first); entries whose references can never be satisfied (target is a
+    url alias, a user-defined external alias, ...) keep the reason `references_unmanaged_alias`."""
+    live = {kind.name(r): r for r in rows}
+    available = set(ctx["central_names"])
+    available |= {n for n, r in live.items() if _alias_is_system(r) or clean(r.get("uuid")) in managed}
+
+    candidates = [e for e in entries if not e.reasons]
+    for entry in candidates:
+        missing = {ref for ref in kind.refs(entry.row) if ref in live and ref not in available}
+        if missing:
+            entry.reasons.append("references_unmanaged_alias")
+            entry.details["references_unmanaged_alias"] = ", ".join(sorted(missing))
+
+    order: list[Entry] = []
+    ready = set(available)
+    progress = True
+    while progress:
+        progress = False
+        for entry in candidates:
+            if entry in order:
+                continue
+            if all(ref not in live or ref in ready for ref in kind.refs(entry.row)):
+                order.append(entry)
+                ready.add(kind.name(entry.row))
+                progress = True
+    for entry in order:
+        entry.bulk = "references_unmanaged_alias" in entry.reasons
+    return order
+
+
+async def _plan(db: AsyncSession, kind: Kind, host: OpnsenseHost) -> tuple[list[Entry], list[Entry]]:
+    """(all unmanaged entries with their reasons, the ones adoptable in one run in a safe order)."""
+    rows = await kind.fetch(host)
+    managed = await _managed_uuids(db, kind, host)
+    ctx = await kind.context(db, rows) if kind.context else {}
+    central = await adopt_link_service.load_central(db, kind.link_spec)
+    entries = []
+    for row in rows:
+        if clean(row.get("uuid")) in managed:
+            continue
+        reasons, details = kind.check(row, ctx)
+        entry = Entry(row, list(reasons), dict(details))
+        if reasons != ["system"]:
+            entry.matches, entry.similar = adopt_link_service.find(central, kind.link_spec, host, kind.build(row))
+        entries.append(entry)
+    if kind.refs:
+        order = _resolve_references(kind, entries, rows, managed, ctx)
+    else:
+        order = [e for e in entries if not e.reasons]
+    return entries, order
+
+
+def _bulk_actions(entries: list[Entry], order: list[Entry], link_existing: bool) -> dict[str, tuple[str, Any]]:
+    """What "adopt all" does with each entry: ("link", central object) when exactly one central twin exists
+    (and no other entry of this run took it), ("new", None), or nothing. One function for the dialog's
+    summary and for the real run, so they cannot disagree. Several twins are never guessed between."""
+    actions: dict[str, tuple[str, Any]] = {}
+    taken: set = set()
+    if link_existing:
+        for entry in entries:
+            if entry.linkable and len(entry.matches) == 1 and entry.matches[0].obj.id not in taken:
+                actions[entry.uuid] = ("link", entry.matches[0].obj)
+                taken.add(entry.matches[0].obj.id)
+    for entry in order:
+        if link_existing and entry.busy_twin:
+            continue
+        actions.setdefault(entry.uuid, ("new", None))
+    return actions
+
+
+def _describe(kind: Kind, entry: Entry, action: str | None = None) -> dict:
+    spec = kind.link_spec
+    return {
+        "matches": [adopt_link_service.describe(spec, m) for m in entry.matches],
+        "similar": [adopt_link_service.describe(spec, m) for m in entry.similar],
+        "linkable": entry.linkable,
+        "bulk_action": action,
+        "uuid": entry.uuid,
+        "title": kind.title(entry.row),
+        "enabled": on(entry.row.get("enabled")) if "disabled" not in entry.row else not on(entry.row.get("disabled")),
+        "adoptable": not entry.reasons,
+        "bulk_adoptable": not entry.reasons or entry.bulk,
+        "reasons": entry.reasons,
+        "reason_details": entry.details,
+        "kept": kind.kept(entry.row),
+        "info": kind.info(entry.row),
+    }
+
+
+async def list_unmanaged(db: AsyncSession, kind_key: str, host: OpnsenseHost) -> list[dict]:
+    kind = KINDS[kind_key]
+    entries, order = await _plan(db, kind, host)
+    actions = _bulk_actions(entries, order, True)
+    return [_describe(kind, e, actions.get(e.uuid, ("skip", None))[0]) for e in entries]
+
+
+def _create(db: AsyncSession, kind: Kind, host: OpnsenseHost, entry: Entry):
+    obj = kind.model(**kind.build(entry.row), targets=[host], groups=[])
+    db.add(obj)
+    return obj
+
+
+async def _record(db: AsyncSession, kind: Kind, host: OpnsenseHost, obj, remote_uuid: str) -> None:
+    await db.flush()
+    kind.revision(db, obj, ADOPT_NOTE)
+    db.add(kind.deployment(**{
+        kind.fk: obj.id, "host_id": host.id, "status": "success", "message": ADOPT_MESSAGE,
+        kind.uuid_attr: remote_uuid, "last_payload": kind.payload(obj), "previous_payload": None,
+    }))
+    await db.flush()
+
+
+async def adopt(db: AsyncSession, kind_key: str, host: OpnsenseHost, remote_uuid: str, link_to=None):
+    """Takes over one object as a NEW central object, or - with `link_to` - links it to that existing
+    central object (adopt_link_service). Either way the live state is read fresh here."""
+    kind = KINDS[kind_key]
+    if link_to is not None:
+        existing = await db.scalar(select(kind.deployment).where(
+            kind.deployment.host_id == host.id, getattr(kind.deployment, kind.uuid_attr) == remote_uuid))
+        if existing is not None:
+            if getattr(existing, kind.fk) == link_to:
+                return await db.get(kind.model, link_to)  # second click: already linked, nothing to do
+            raise NotAdoptable(["already_managed"])
+    entries, _ = await _plan(db, kind, host)
+    entry = next((e for e in entries if e.uuid == remote_uuid), None)
+    if entry is None:
+        # either gone from the host or already managed
+        if remote_uuid in await _managed_uuids(db, kind, host):
+            raise NotAdoptable(["already_managed"])
+        raise NotFoundOnHost(remote_uuid)
+    if link_to is not None:
+        if not set(entry.reasons) <= LINK_TOLERATED:
+            raise NotAdoptable(entry.reasons, entry.details)
+        obj = await adopt_link_service.link(db, kind.link_spec, host, link_to, remote_uuid, kind.build(entry.row))
+        return obj
+    if entry.reasons:
+        raise NotAdoptable(entry.reasons, entry.details)
+    obj = _create(db, kind, host, entry)
+    await _record(db, kind, host, obj, remote_uuid)
+    return obj
+
+
+async def adopt_all(db: AsyncSession, kind_key: str, host: OpnsenseHost, link_existing: bool = False) -> dict:
+    """Adopts everything adoptable in one transaction (the caller commits), nested aliases in
+    dependency order. With `link_existing`, entries with exactly one exact central twin are linked to it
+    instead (see _bulk_actions). System objects are not reported; everything else that stays behind is,
+    with reasons."""
+    kind = KINDS[kind_key]
+    entries, order = await _plan(db, kind, host)
+    actions = _bulk_actions(entries, order, link_existing)
+    adopted = []
+    done: set[str] = set()
+    for entry in entries:  # linked first: their names are central already, so nested aliases can refer to them
+        action, target = actions.get(entry.uuid, ("skip", None))
+        if action == "link":
+            obj = await adopt_link_service.link(db, kind.link_spec, host, target.id, entry.uuid, kind.build(entry.row))
+            adopted.append({"uuid": entry.uuid, "title": kind.title(entry.row), "id": obj.id, "linked": True})
+            done.add(entry.uuid)
+    for entry in order:
+        if actions.get(entry.uuid, ("skip", None))[0] != "new":
+            continue
+        obj = _create(db, kind, host, entry)
+        await _record(db, kind, host, obj, entry.uuid)
+        adopted.append({"uuid": entry.uuid, "title": kind.title(entry.row), "id": obj.id, "linked": False})
+        done.add(entry.uuid)
+    skipped = [
+        {"uuid": e.uuid, "title": kind.title(e.row),
+         "reasons": e.reasons + (["twin_busy"] if link_existing and e.busy_twin else []), "reason_details": e.details}
+        for e in entries if e.uuid not in done and e.reasons != ["system"]
+    ]
+    return {"adopted": adopted, "skipped": skipped}

backend/app/services/oidc_service.py

@@ -25,7 +25,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.oidc import OidcLoginState, OidcSettings
 from app.models.user import AdminUser
-from app.services import notification_secret_store
+from app.services import notification_secret_store, url_guard
 
 logger = logging.getLogger(__name__)
 
@@ -77,7 +77,7 @@ async def _get_json(url: str, **kw) -> dict:
     if not url.startswith("https://"):
         raise OidcError("provider", "Nur https:// wird akzeptiert")
     try:
-        async with httpx.AsyncClient(timeout=10, follow_redirects=False) as client:
+        async with url_guard.guarded_client(timeout=10, follow_redirects=False) as client:
             response = await client.get(url, **kw)
             response.raise_for_status()
             data = response.json()
@@ -168,7 +168,7 @@ async def _exchange(cfg: OidcSettings, doc: dict, code: str, state: OidcLoginSta
         else:
             data["client_secret"] = secret
     try:
-        async with httpx.AsyncClient(timeout=15, follow_redirects=False) as client:
+        async with url_guard.guarded_client(timeout=15, follow_redirects=False) as client:
             response = await client.post(doc["token_endpoint"], data=data, auth=auth)
         if response.status_code >= 400:
             raise OidcError("token", f"Token-Endpunkt antwortet mit HTTP {response.status_code}")

backend/app/services/opnsense_client.py

@@ -33,6 +33,12 @@ class OpnsenseAmbiguousError(OpnsenseAPIError):
     outcome of a write is unknown and must be reconciled, not retried blindly."""
 
 
+class OpnsenseForbiddenError(OpnsenseAPIError):
+    """The firewall answered 403: the API key lacks the privilege for this endpoint. A refusal, not an unknown
+    outcome; callers that only read can tell "no rights" from "unreachable". 401 (wrong key or secret) stays a
+    plain OpnsenseAPIError."""
+
+
 def _client(host: OpnsenseHost) -> httpx.AsyncClient:
     # Before the secret is even read: an http:// host must never receive credentials.
     reason = firewall_url.check_base_url(host.base_url)
@@ -71,7 +77,8 @@ async def _do_request(host: OpnsenseHost, method: str, path: str, **kwargs) -> h
         except httpx.HTTPStatusError as exc:
             # The body may reflect submitted secrets: keep only a scrubbed, bounded excerpt.
             secret = opnsense_secret_store.read_secret(host.id)
-            error = OpnsenseAmbiguousError if exc.response.status_code >= 500 else OpnsenseAPIError
+            status = exc.response.status_code
+            error = OpnsenseAmbiguousError if status >= 500 else OpnsenseForbiddenError if status == 403 else OpnsenseAPIError
             raise error(
                 f"{exc.response.status_code} {path}: {redact_text(exc.response.text, host.api_key, secret)}"
             ) from exc
@@ -145,6 +152,34 @@ async def get_interface_statistics(host: OpnsenseHost) -> dict:
     return await _request(host, "GET", "/diagnostics/interface/getInterfaceStatistics")
 
 
+# Swap and CPU count, for the resource-threshold alert (1.24). Verified against core source
+# (Diagnostics/Api/SystemController::systemSwapAction -> configd "system show swapinfo") and live
+# (read-only GET, test-fw-3, OPNsense 26.7):
+#   systemSwap  -> {"swap": [{"device": "/dev/da0p3", "total": "8388608", "used": "0"}]}   (KiB, strings;
+#                  an empty list when the firewall has no swap)
+#   cpu_usage/getCPUType -> ["QEMU Virtual CPU version 2.5+ (6 cores, 6 threads)"]
+async def get_system_swap(host: OpnsenseHost) -> dict:
+    return await _request(host, "GET", "/diagnostics/system/systemSwap")
+
+
+async def get_cpu_type(host: OpnsenseHost) -> Any:
+    return await _request(host, "GET", "/diagnostics/cpu_usage/getCPUType")
+
+
+# Gateway health (read-only). Routes\Api\GatewayController::statusAction runs configd "interface gateways status"
+# -> scripts/routes/gateway_status.php, which merges the configured gateways with dpinger's live numbers
+# (src/etc/inc/plugins.inc.d/dpinger.inc, dpinger_status()). Live-verified on test-fw-3 (26.7):
+#   {"items": [{"name": "WAN_GW", "address": "203.0.113.1", "status": "none", "loss": "~", "delay": "~",
+#               "stddev": "~", "monitor": "~", "status_translated": "Online"}, ...], "status": "ok"}
+# "status" is the machine-readable verdict: none (fine), down, force_down (switched off by the admin), delay,
+# loss, delay+loss (thresholds of the gateway itself), anything else = pending. loss/delay/stddev are "12.3 %" /
+# "4.5 ms" strings, or "~" while nothing is measured (e.g. monitoring disabled on that gateway). "status" is
+# "failed" with no items when the firewall has no gateways. searchGateway carries the same numbers but with a
+# translated status text ("Online"), so it is not used here.
+async def get_gateway_status(host: OpnsenseHost) -> dict:
+    return await _request(host, "GET", "/routes/gateway/status")
+
+
 async def toggle_carp_maintenance(host: OpnsenseHost) -> dict:
     """Toggle persistent CARP maintenance mode.
 
@@ -391,6 +426,47 @@ async def get_pf_rule_statistics(host: OpnsenseHost) -> dict:
     return await _request(host, "GET", "/diagnostics/firewall/pfStatistics/rules")
 
 
+# ── Rule analysis (read-only) ─────────────────────────────────────────────
+# Checked against opnsense/core master (2026-10-21) and live on OPNsense 26.7:
+# - Firewall/Api/FilterUtilController::ruleStatsAction runs configd `!filter rule stats` and
+#   returns {"status": "ok", "stats": {"<label>": {pf_rules, evaluations, packets, bytes, states}}};
+#   the label is the rule UUID (MVC), an md5 (automatic) or a 32-hex label (unmigrated legacy).
+# - Firewall/Api/AliasUtilController::findReferencesAction requires POST with "ip" (only hex,
+#   ":", ".", "/", "_" accepted, else {"status": "Not an IP address!"}), runs configd
+#   `filter find_table_references` (a pf table test over all aliases) and returns
+#   {"status": "ok", "matches": ["<alias>", ...]}. It changes nothing; POST is only the
+#   controller's calling convention.
+# - Firewall/Api/FilterController::getInterfaceListAction: {"floating"|"groups"|"interfaces"|"any":
+#   {"label", "items": [{"value", "label", "type"}]}} (lo0 omitted).
+# - Firewall/Api/GroupController::searchItemAction -> searchBase("ifgroupentry"): rows with
+#   ifname, members (comma separated identifiers), sequence; the built-in groups (enc0, openvpn,
+#   wireguard) appear with their name as uuid and empty members.
+async def get_rule_stats(host: OpnsenseHost) -> dict:
+    """Per-label counters, already summed over every pf line of a rule. Used for the rule usage sampling
+    (rule_hit_service). Live check 2026-10-22 on all three test firewalls (26.7, one with unmigrated legacy
+    rules): HTTP 200, ``stats`` has exactly one key per label of ``pfStatistics/rules`` (30/30/25 labels,
+    including the 32-hex label of the legacy user rule on the third), and each ``packets`` equals the sum over
+    that label's pf lines (differences only by traffic between the two reads)."""
+    return await _request(host, "GET", "/firewall/filter_util/ruleStats")
+
+
+async def find_alias_references(host: OpnsenseHost, ip: str) -> list[str] | None:
+    """Names of the aliases (pf tables) that contain ``ip`` right now; None when the firewall
+    could not answer (callers then treat table membership as unknown)."""
+    data = await _request(host, "POST", "/firewall/alias_util/findReferences", json={"ip": ip})
+    if not isinstance(data, dict) or data.get("status") != "ok" or not isinstance(data.get("matches"), list):
+        return None
+    return [str(name) for name in data["matches"]]
+
+
+async def get_filter_interface_list(host: OpnsenseHost) -> dict:
+    return await _request(host, "GET", "/firewall/filter/get_interface_list")
+
+
+async def search_interface_groups(host: OpnsenseHost) -> dict:
+    return await _request(host, "GET", "/firewall/group/searchItem", params={"rowCount": -1})
+
+
 async def get_firmware_status(host: OpnsenseHost) -> dict:
     # GET reads cached results. POST would trigger a synchronous probe.
     return await _request(host, "GET", "/core/firmware/status")

backend/app/services/resource_monitor_service.py

@@ -5,7 +5,9 @@ poll_all_hosts. v1 deliberately has no admin-editable poll interval (see
 config.py) — just a fixed constant, to keep this feature's first cut
 small.
 """
+import asyncio
 import logging
+import re
 from datetime import datetime, timedelta, timezone
 
 from sqlalchemy import delete, select
@@ -13,8 +15,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.host import OpnsenseHost
 from app.models.resource_sample import HostResourceSample
+from app.services import gateway_health_service
 from app.services import opnsense_client as client
-from app.services.opnsense_client import OpnsenseAPIError
+from app.services.opnsense_client import OpnsenseAPIError, OpnsenseForbiddenError
 
 logger = logging.getLogger(__name__)
 
@@ -55,11 +58,70 @@ def _disk_used_pct(disk: dict) -> float | None:
     return None
 
 
+def _swap_used_pct(swap) -> float | None:
+    """systemSwap: {"swap": [{"device", "total", "used"}]} in KiB; None when there is no swap at all."""
+    try:
+        rows = swap.get("swap") or []
+        total = sum(int(r["total"]) for r in rows)
+        used = sum(int(r["used"]) for r in rows)
+    except (AttributeError, KeyError, TypeError, ValueError):
+        return None
+    return round(used / total * 100, 1) if total > 0 else None
+
+
+_CPU_THREADS = re.compile(r"(\d+)\s+threads?", re.IGNORECASE)
+_CPU_CORES = re.compile(r"(\d+)\s+cores?", re.IGNORECASE)
+
+
+def _cpu_threads(cpu_type) -> int | None:
+    """getCPUType: ["<model> (6 cores, 6 threads)"]."""
+    text = " ".join(str(x) for x in cpu_type) if isinstance(cpu_type, list) else str(cpu_type or "")
+    match = _CPU_THREADS.search(text) or _CPU_CORES.search(text)
+    return int(match.group(1)) if match else None
+
+
+def _services_compact(listing) -> dict[str, int]:
+    """{"unbound": 1, "openvpn/<uuid>": 0}: 1 = running. Only services an admin can stop (not 'locked')."""
+    result: dict[str, int] = {}
+    for row in (listing.get("rows") if isinstance(listing, dict) else None) or []:
+        if not isinstance(row, dict) or not row.get("name") or str(row.get("locked")) in ("1", "True", "true"):
+            continue
+        name, id_ = str(row["name"]), str(row.get("id") or row["name"])
+        result[name if id_ == name else f"{name}/{id_}"] = 1 if str(row.get("running")) in ("1", "True", "true") else 0
+    return result
+
+
+async def _optional(operation, name: str, denied: list[str], ok: list[str]):
+    """The extra readings must never cost the base sample (older firmware, an endpoint that is missing).
+    A refusal by the firewall (403: the API key lacks the privilege) is recorded in ``denied`` so the
+    missing right becomes visible; a reading that went through is recorded in ``ok`` (the proof that a right is
+    back, whatever the answer contains); any other failure stays "unknown" and is no finding about rights."""
+    try:
+        result = await operation
+        ok.append(name)
+        return result
+    except OpnsenseForbiddenError:
+        denied.append(name)
+        logger.debug("Optionale Messung %s: API-Recht fehlt", name)
+        return None
+    except Exception as exc:
+        logger.debug("Optionale Messung fehlgeschlagen: %s", type(exc).__name__)
+        return None
+
+
 async def sample_host(db: AsyncSession, host: OpnsenseHost) -> None:
     resources = await client.get_system_resources(host)
     system_time = await client.get_system_time(host)
     disk = await client.get_system_disk(host)
     interfaces = await client.get_interface_statistics(host)
+    denied: list[str] = []
+    ok: list[str] = []
+    swap, cpu_type, gateways, services = await asyncio.gather(
+        _optional(client.get_system_swap(host), "swap", denied, ok),
+        _optional(client.get_cpu_type(host), "cpu", denied, ok),
+        _optional(gateway_health_service.get_gateway_health(host), "gateways", denied, ok),
+        _optional(client.search_services(host), "services", denied, ok),
+    )
     db.add(
         HostResourceSample(
             host_id=host.id,
@@ -67,7 +129,17 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost) -> None:
             mem_used_pct=_mem_used_pct(resources),
             load_avg_1m=_load_avg_1m(system_time),
             disk_used_pct=_disk_used_pct(disk),
-            raw={"memory": resources.get("memory"), "disk": disk.get("devices"), "interfaces": interfaces.get("statistics")},
+            raw={
+                "memory": resources.get("memory"), "disk": disk.get("devices"), "interfaces": interfaces.get("statistics"),
+                # None = not readable this round (the alert checks treat that as "unknown", not as "fine").
+                "swap_used_pct": _swap_used_pct(swap) if swap is not None else None,
+                "cpu_threads": _cpu_threads(cpu_type) if cpu_type is not None else None,
+                "gateways": gateway_health_service.compact(gateways) if gateways is not None else None,
+                "services": _services_compact(services) if services is not None else None,
+                # Readings the firewall refused (403) because the API key lacks the right; always a list.
+                "denied": sorted(denied),
+                "ok": sorted(ok),
+            },
         )
     )
 

backend/app/services/rollout_engine.py

@@ -0,0 +1,811 @@
+"""The rollout run engine (1.25): moves runs forward, one small step per poll tick.
+
+Ground rules (an unattended engine that triggers firmware updates is the riskiest thing in OPNexus):
+
+* It never calls a firmware action itself. Each host step is an ordinary ScheduledMaintenance
+  (run_at=now, origin="rollout", auto_verify=True) — locks, backup before the update, CARP handover,
+  audit and restart safety all come from that existing path.
+* Never twice: a host gets a schedule only while its ``scheduled_maintenance_id`` is empty, written
+  in the same transaction as the schedule itself, under the run's row lock.
+* Never in parallel: one host at a time per run, and only one active run at all; nodes of a cluster
+  go BACKUP first, the MASTER only with the wave's CARP handover, and before every node of an
+  already touched cluster the cluster is checked again.
+* When in doubt, stop: anything unexpected pauses the run (``pause_reason``) and notifies; nothing
+  is retried on its own. A paused run only continues when a human resumes it, and resuming first
+  checks the health of everything done so far.
+* Every step re-checks that the person responsible for the run may still roll out.
+* Hard limits: waves/hosts (rollout_service), soak time, preflight/host timeouts, run age.
+
+Concurrency: a tick (1) does DB-only bookkeeping under the run's row lock, (2) decides with the
+lock released (remote reads may take seconds), and (3) applies the decision only if the run's
+``revision`` is still the one it decided on — a pause/abort in between, or a second backend, wins.
+"""
+import logging
+import uuid
+from datetime import datetime, timedelta, timezone
+from typing import Awaitable, Callable
+
+from fastapi import HTTPException
+from sqlalchemy import func, select
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import SessionLocal
+from app.models.host import OpnsenseHost
+from app.models.maintenance_window import MaintenanceWindow
+from app.models.rollout import RolloutPlan, RolloutRun, RolloutRunHost, RolloutRunWave
+from app.models.scheduled_maintenance import ScheduledMaintenance
+from app.models.user import AdminUser
+from app.services import audit_service, rollout_health, rollout_service, window_service
+from app.services import opnsense_client as client
+from app.services.host_maintenance_service import ensure_no_maintenance, peers
+from app.services.host_status_service import get_statuses
+from app.services.policy_service import policy_lock
+
+logger = logging.getLogger(__name__)
+
+ACTIVE = ("waiting_window", "preflight", "wave_running", "soaking", "gate", "awaiting_approval", "resuming", "paused")
+TERMINAL = ("succeeded", "cancelled")
+# Statuses in which the poll loop should tick quickly.
+BUSY = ("preflight", "wave_running", "gate", "resuming")
+HOST_OPEN = ("running", "verifying")
+
+PREFLIGHT_CHECK_DELAY = timedelta(seconds=20)  # after asking for an update check, before reading the result
+PREFLIGHT_TIMEOUT = timedelta(minutes=10)
+GATE_RECHECK = timedelta(minutes=2)  # a failed gate is looked at once more (slow start after a reboot)
+BLOCKED_PATIENCE = timedelta(minutes=15)  # someone else's maintenance in the cluster
+HOST_START_TIMEOUT = timedelta(minutes=10)  # the schedule never fired
+HOST_MAX_DURATION = timedelta(hours=4)  # backstop above the schedule's own timeouts
+MAX_RUN_AGE = timedelta(days=14)
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _iso(value: datetime | None) -> str | None:
+    return value.isoformat() if value else None
+
+
+def _ts(value: str | None) -> datetime | None:
+    return datetime.fromisoformat(value) if value else None
+
+
+def _note(code: str, **params) -> dict:
+    return {"code": code, "params": params}
+
+
+# ── small helpers on a loaded run ─────────────────────────────────────────────
+
+def wave_row(run: RolloutRun, index: int) -> RolloutRunWave:
+    return next(w for w in run.waves if w.index == index)
+
+
+def wave_spec(run: RolloutRun, index: int) -> dict:
+    return run.plan_snapshot["waves"][index]
+
+
+def hosts_of(run: RolloutRun, index: int) -> list[RolloutRunHost]:
+    return sorted((h for h in run.hosts if h.wave_index == index), key=lambda h: (h.order, h.host_name))
+
+
+def _named(run: RolloutRun, failed: list[str]) -> list[str]:
+    """"<host-id>:<check>" -> "<host name>:<check>" for pause details people read."""
+    names = {str(h.host_id): h.host_name for h in run.hosts if h.host_id}
+    return [f"{names.get(item.split(':', 1)[0], item.split(':', 1)[0])}:{item.split(':', 1)[1]}" for item in failed]
+
+
+def _progress(run: RolloutRun, **changes) -> None:
+    run.progress = {**(run.progress or {}), **changes}
+
+
+async def _audit(run: RolloutRun, action: str, status: str, message: str | None = None, **details) -> None:
+    token = audit_service.context.set({"actor": f"{run.started_by} (Rollout)"[:150], "id": str(run.id)})
+    try:
+        await audit_service.persist(kind="rollout", action=action, status=status, entity_id=str(run.id),
+                                    entity_name=run.plan_name, message=message,
+                                    details={"wave": run.current_wave, **details})
+    finally:
+        audit_service.context.reset(token)
+
+
+def _requeue(host: RolloutRunHost, code: str) -> None:
+    """Back to the queue: only for a host whose schedule provably never fired (cancelled under its row lock,
+    or refused by fire_due as overdue). Clearing the schedule id lets a resumed run plan it once more."""
+    host.status, host.scheduled_maintenance_id, host.started_at, host.detail = "queued", None, None, _note(code)
+
+
+async def requeue_if_pending(db: AsyncSession, host: RolloutRunHost) -> bool:
+    """Cancel a host's schedule that has not fired yet and put the host back in the queue. Locked like the
+    router's abort: fire_due holds the row while firing, and a running action is never touched."""
+    if host.status not in HOST_OPEN or host.scheduled_maintenance_id is None:
+        return False
+    sched = await db.scalar(select(ScheduledMaintenance).where(ScheduledMaintenance.id == host.scheduled_maintenance_id)
+                            .with_for_update().execution_options(populate_existing=True))
+    if sched is None or sched.status != "pending":
+        return False
+    sched.status = "cancelled"
+    sched.result = {"error": "Rollout angehalten, nicht ausgeführt", "not_fired": True}
+    _requeue(host, "not_fired")
+    return True
+
+
+def pause(run: RolloutRun, reason: str, **detail) -> None:
+    if run.status != "paused":
+        run.paused_from = run.status
+    run.status = "paused"
+    run.pause_reason = reason
+    run.pause_detail = detail
+    _progress(run, paused_at=_iso(_now()))
+
+
+async def may_roll_out(db: AsyncSession, username: str, user_id=None) -> bool:
+    """Does this account (still) have the right to roll out? Same rule as AuthMiddleware's is_rollout.
+    With the stable account id (1.29) the id decides: a deleted account that was re-created under the same name is
+    someone else. Rows from before the id existed (and still without one) are matched by name as before."""
+    if user_id is not None:
+        user = await db.get(AdminUser, user_id)
+    else:
+        user = await db.scalar(select(AdminUser).where(AdminUser.username == username))
+    if user is None:
+        return False
+    if user.role == "admin":
+        return True
+    if user.role == "operator":
+        from app.routers import governance
+        return (await governance.get_settings(db)).operators_may_deploy
+    return False
+
+
+async def _window(db: AsyncSession, run: RolloutRun) -> tuple[bool, MaintenanceWindow | None]:
+    """(window configured, window). Configured but gone means: stop, never 'no time limit'."""
+    window_id = run.plan_snapshot.get("window_id")
+    if not window_id:
+        return False, None
+    return True, await db.get(MaintenanceWindow, window_id)
+
+
+async def _may_start_now(db: AsyncSession, run: RolloutRun) -> str | None:
+    """None if new host work may start now, else the reason ('window_closed' / 'window_missing')."""
+    configured, window = await _window(db, run)
+    if not configured:
+        return None
+    if window is None:
+        return "window_missing"
+    return None if window_service.startable(window, _now()) else "window_closed"
+
+
+async def _lock(db: AsyncSession, run_id, revision: int | None = None) -> RolloutRun | None:
+    stmt = select(RolloutRun).where(RolloutRun.id == run_id)
+    if revision is not None:
+        stmt = stmt.where(RolloutRun.revision == revision)
+    return await db.scalar(stmt.with_for_update().execution_options(populate_existing=True))
+
+
+# ── creating a run ────────────────────────────────────────────────────────────
+
+async def active_run(db: AsyncSession) -> RolloutRun | None:
+    return await db.scalar(select(RolloutRun).where(RolloutRun.status.in_(ACTIVE)).order_by(RolloutRun.created_at).limit(1))
+
+
+async def create_run(db: AsyncSession, plan: RolloutPlan, resolved: dict, *, started_by: str, trigger: str,
+                     occurrence: datetime | None = None, started_by_id=None) -> RolloutRun:
+    """Freeze the plan and its resolved hosts. The caller holds policy_lock and has checked the token."""
+    now = _now()
+    snapshot = {
+        "plan_id": str(plan.id), "name": plan.name, "action": plan.action,
+        "window_id": str(plan.window_id) if plan.window_id else None,
+        "gate_checks": rollout_service.effective_checks(plan.gate_checks), "compliance_mode": plan.compliance_mode,
+        "waves": [{k: w[k] for k in ("label", "soak_minutes", "carp_handover", "require_approval", "groups")} for w in resolved["waves"]],
+    }
+    run = RolloutRun(plan_id=plan.id, plan_name=plan.name, plan_snapshot=snapshot, action=plan.action, status="preflight",
+                     current_wave=0, started_by=started_by, started_by_id=started_by_id, trigger=trigger, window_occurrence=occurrence,
+                     progress={}, created_at=now, step_at=now, revision=0)
+    run.waves = [RolloutRunWave(index=w["index"], label=w["label"], status="preflight" if w["index"] == 0 else "pending")
+                 for w in resolved["waves"]]
+    run.hosts = [RolloutRunHost(wave_index=w["index"], host_id=uuid.UUID(h["id"]), host_name=h["name"], cluster_name=h["cluster_name"],
+                                order=i, status="queued")
+                 for w in resolved["waves"] for i, h in enumerate(w["hosts"])]
+    db.add(run)
+    await db.flush()
+    return run
+
+
+# ── the tick ──────────────────────────────────────────────────────────────────
+
+async def advance_runs(db: AsyncSession) -> int:
+    """One step for every run. Returns how many runs want a quick next tick."""
+    try:
+        await auto_start(db)
+    except Exception:
+        logger.exception("Rollout: automatischer Start fehlgeschlagen")
+        await db.rollback()
+    ids = list((await db.scalars(select(RolloutRun.id).where(
+        (RolloutRun.status.in_(ACTIVE)) | RolloutRun.hosts.any(RolloutRunHost.status.in_(HOST_OPEN))))).all())
+    await db.commit()
+    busy = 0
+    for run_id in ids:
+        try:
+            await tick(run_id)
+        except Exception as exc:
+            logger.exception("Rollout %s: Schritt fehlgeschlagen", run_id)
+            async with SessionLocal() as s:
+                run = await _lock(s, run_id)
+                if run is not None and run.status in ACTIVE and run.status != "paused":
+                    pause(run, "internal_error", error=type(exc).__name__)
+                    run.revision += 1
+                    await s.commit()
+                    await _audit(run, "pause", "paused", "Interner Fehler: Rollout angehalten", reason="internal_error")
+        async with SessionLocal() as s:
+            status = await s.scalar(select(RolloutRun.status).where(RolloutRun.id == run_id))
+        busy += status in BUSY
+    return busy
+
+
+async def tick(run_id) -> None:
+    async with SessionLocal() as db:
+        # (1) bookkeeping under the lock: what did the schedules do?
+        run = await _lock(db, run_id)
+        if run is None:
+            return
+        changed = await _sync_hosts(db, run)
+        if (run.status in ACTIVE and run.status != "paused" and _now() - run.created_at > MAX_RUN_AGE
+                and not any(h.status in HOST_OPEN for h in run.hosts)):
+            pause(run, "run_too_old", days=MAX_RUN_AGE.days)
+            changed = True
+        if changed:
+            run.revision += 1
+        await db.commit()
+        if changed and run.status == "paused" and run.pause_reason in ("host_failed", "host_timeout", "host_not_started", "run_too_old"):
+            await _audit(run, "pause", "paused", reason=run.pause_reason, detail=run.pause_detail)
+        if run.status not in ACTIVE or run.status in ("paused", "awaiting_approval"):
+            return
+        revision = run.revision
+        # (2) decide, without holding the lock
+        decision = await _HANDLERS[run.status](db, run)
+        await db.commit()
+        if decision is None:
+            return
+        # (3) apply only on the revision it was decided on
+        run = await _lock(db, run_id, revision)
+        if run is None:
+            await db.commit()
+            return
+        events = await decision(db, run)
+        run.revision += 1
+        run.step_at = _now()
+        await db.commit()
+        for event in events or []:
+            await _audit(run, *event[:2], **(event[2] if len(event) > 2 else {}))
+
+
+Decision = Callable[[AsyncSession, RolloutRun], Awaitable[list | None]]
+
+
+async def _sync_hosts(db: AsyncSession, run: RolloutRun) -> bool:
+    """Carry the outcome of each host's schedule over to the run. Returns True if anything changed."""
+    changed = False
+    now = _now()
+    for host in run.hosts:
+        if host.status not in HOST_OPEN:
+            continue
+        sched = await db.get(ScheduledMaintenance, host.scheduled_maintenance_id, populate_existing=True) \
+            if host.scheduled_maintenance_id else None
+        problem = None
+        if sched is None:
+            host.status, host.detail, host.finished_at = "failed", _note("schedule_missing"), now
+            problem = ("host_failed", {"host": host.host_name, "error": "schedule_missing"})
+        elif sched.status == "pending":
+            if host.started_at and now - host.started_at > HOST_START_TIMEOUT:
+                # Never leave it pending behind a paused run: it would fire whenever the poll loop comes back.
+                if await requeue_if_pending(db, host):
+                    problem = ("host_not_started", {"host": host.host_name})
+                    changed = True
+        elif (sched.result or {}).get("not_fired"):
+            _requeue(host, "not_fired")  # fire_due refused it as overdue: nothing happened on the firewall
+            problem = ("host_not_started", {"host": host.host_name})
+            changed = True
+        elif sched.status == "running":
+            want = "verifying" if sched.phase == "verify" else "running"
+            if host.status != want:
+                host.status = want
+                changed = True
+            if host.started_at and now - host.started_at > HOST_MAX_DURATION:
+                problem = ("host_timeout", {"host": host.host_name})
+        elif sched.status == "succeeded":
+            result = sched.result or {}
+            host.status, host.finished_at = "succeeded", now
+            host.version_after = result.get("version_after")
+            host.detail = _note("verified", version=result.get("version_after"), rebooted=bool(result.get("rebooted")))
+            changed = True
+        else:  # failed / cancelled
+            error = (sched.result or {}).get("error") or sched.status
+            host.status, host.finished_at = "failed", now
+            host.detail = _note("host_failed", error=error[:300])
+            problem = ("host_failed", {"host": host.host_name, "error": error[:300]})
+        if problem and run.status in ACTIVE and run.status != "paused":
+            pause(run, problem[0], **problem[1])
+            changed = True
+        elif problem and host.status == "failed":
+            changed = True
+    return changed
+
+
+# ── handlers: one per run status; each returns an apply-function or None ──────
+
+async def _actor_problem(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    if await may_roll_out(db, run.started_by, run.started_by_id):
+        return None
+
+    async def apply(db, run):
+        pause(run, "actor_invalid", user=run.started_by)
+        return [("pause", "paused", {"reason": "actor_invalid"})]
+    return apply
+
+
+def _to_waiting(reason: str) -> Decision:
+    async def apply(db, run):
+        if reason == "window_missing":
+            pause(run, "window_missing")
+            return [("pause", "paused", {"reason": reason})]
+        run.paused_from = run.status
+        run.status = "waiting_window"
+        return [("waiting_window", "waiting")]
+    return apply
+
+
+async def _handle_waiting_window(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    reason = await _may_start_now(db, run)
+    if reason == "window_missing":
+        return _to_waiting(reason)
+    if reason:
+        return None
+
+    async def apply(db, run):
+        _progress(run, resume_recheck_at=None, resume_attempts=0)
+        run.status = "resuming"  # the gate of everything done so far is checked again first
+        return [("window_open", "resuming")]
+    return apply
+
+
+async def _handle_resuming(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    recheck = _ts((run.progress or {}).get("resume_recheck_at"))
+    if recheck and _now() < recheck:
+        return None
+    done = [h for h in run.hosts if h.status == "succeeded" and h.host_id]
+    gate = {"hosts": {}, "verdict": "ok"}
+    if done:
+        hosts = [h for h in [await db.get(OpnsenseHost, d.host_id) for d in done] if h is not None]
+        baselines = {}
+        for w in run.waves:
+            baselines.update((w.baseline or {}))
+        gate = await rollout_health.evaluate(db, hosts, baselines, target_version=run.target_version,
+                                             checks=run.plan_snapshot.get("gate_checks"),
+                                             compliance_mode=run.plan_snapshot.get("compliance_mode", "warn"))
+
+    async def apply(db, run):
+        if gate["verdict"] == "fail":
+            attempts = (run.progress or {}).get("resume_attempts", 0) + 1
+            _progress(run, resume_gate=gate, resume_attempts=attempts)
+            if attempts < 2:
+                _progress(run, resume_recheck_at=_iso(_now() + GATE_RECHECK))
+                return [("resume_gate", "recheck", {"failed": rollout_health.failed_checks(gate)})]
+            run.paused_from = run.paused_from or "preflight"
+            _progress(run, resume_recheck_at=None)
+            run.status = "paused"
+            run.pause_reason, run.pause_detail = "gate_failed", {"failed": _named(run, rollout_health.failed_checks(gate)), "on_resume": True}
+            return [("resume_gate", "failed", {"failed": rollout_health.failed_checks(gate)})]
+        _progress(run, resume_gate=gate, resume_recheck_at=None, resume_attempts=0)
+        target = run.paused_from if run.paused_from in ("preflight", "wave_running", "soaking", "gate") else "preflight"
+        if target == "gate":
+            wave_row(run, run.current_wave).gate_attempts = 0
+            _progress(run, gate_recheck_at=None)
+        if target == "preflight":
+            _progress(run, preflight_started=None, preflight_requested_at=None)
+        run.status, run.paused_from, run.pause_reason, run.pause_detail = target, None, None, None
+        return [("resume_gate", "passed", {"continue_with": target})]
+    return apply
+
+
+def target_of(status: dict, action: str) -> str | None:
+    """The version a firmware action will install, from core/firmware/status (read after a fresh check).
+
+    Live finding (test-fw-3, 26.7.1_1, 2026-10-21): ``product.product_latest`` said 26.7.2 while the
+    ``opnsense`` core package was offered as 26.7.7 — and the package is what gets installed. So the
+    core package's new version comes first; product_latest is only a fallback."""
+    if action == "major_update":
+        return status.get("upgrade_major_version") or None
+    packages = status.get("all_packages") if isinstance(status.get("all_packages"), dict) else {}
+    core = packages.get("opnsense") if isinstance(packages.get("opnsense"), dict) else None
+    if core and core.get("new") and core.get("new") != "N/A":
+        return str(core["new"])
+    for row in status.get("upgrade_packages") or []:
+        if isinstance(row, dict) and row.get("name") == "opnsense" and row.get("new_version"):
+            return str(row["new_version"])
+    product = status.get("product") if isinstance(status.get("product"), dict) else {}
+    return product.get("product_latest") or status.get("product_latest") or None
+
+
+async def _handle_preflight(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    if problem := await _actor_problem(db, run):
+        return problem
+    reason = await _may_start_now(db, run)
+    if reason:
+        return _to_waiting(reason)
+    index = run.current_wave
+    prog = run.progress or {}
+    started = _ts(prog.get("preflight_started")) or _now()
+    requested = _ts(prog.get("preflight_requested_at"))
+    candidates = [h for h in hosts_of(run, index) if h.status in ("queued", "blocked")]
+    now = _now()
+
+    # Which hosts are usable at all?
+    usable: dict = {}
+    skips: dict = {}
+    for rh in candidates:
+        host = await db.get(OpnsenseHost, rh.host_id) if rh.host_id else None
+        if host is None:
+            skips[rh.id] = _note("host_deleted")
+        elif not host.enabled:
+            skips[rh.id] = _note("host_disabled")
+        else:
+            usable[rh.id] = host
+
+    if requested is None:
+        # Step 1: ask every usable host to look for updates (a firmware *check*, changes nothing installed).
+        asked = []
+        for rh_id, host in usable.items():
+            try:
+                await ensure_no_maintenance(db, [host])
+                if (await client.get_firmware_running(host)).get("status") == "ready":
+                    await client.check_firmware_updates(host)
+                    asked.append(str(rh_id))
+            except (client.OpnsenseAPIError, HTTPException):
+                pass  # judged in step 2 (and blocked after the timeout)
+
+        async def apply(db, run):
+            _progress(run, preflight_started=_iso(started), preflight_requested_at=_iso(_now()), preflight_asked=asked)
+            w = wave_row(run, index)
+            w.status = "preflight"
+            w.started_at = w.started_at or _now()
+            return [("preflight", "started", {"hosts": len(usable)})]
+        return apply
+    if now - requested < PREFLIGHT_CHECK_DELAY:
+        return None
+
+    # Step 2: read what each host offers.
+    offers: dict = {}
+    waiting: dict = {}
+    for rh_id, host in usable.items():
+        try:
+            await ensure_no_maintenance(db, [host])
+        except HTTPException:
+            waiting[rh_id] = _note("locked")
+            continue
+        try:
+            running = (await client.get_firmware_running(host)).get("status")
+            if running != "ready":
+                waiting[rh_id] = _note("firmware_busy")
+                continue
+            status = await client.get_firmware_status(host)
+        except client.OpnsenseAPIError:
+            waiting[rh_id] = _note("offline")
+            continue
+        fw = status.get("status")
+        wanted = "upgrade" if run.action == "major_update" else "update"
+        if fw == wanted:
+            offers[rh_id] = target_of(status, run.action)
+        elif fw in ("none", "update", "upgrade"):
+            skips[rh_id] = _note("no_update_offered", status=fw)
+        else:
+            waiting[rh_id] = _note("firmware_status", status=str(fw)[:40])
+
+    if waiting:
+        if now - started < PREFLIGHT_TIMEOUT:
+            return None
+
+        async def apply_blocked(db, run):
+            names = []
+            for rh in hosts_of(run, index):
+                if rh.id in waiting:
+                    rh.status, rh.detail = "blocked", waiting[rh.id]
+                    names.append(rh.host_name)
+            pause(run, "preflight_blocked", hosts=", ".join(names))
+            _progress(run, preflight_started=None, preflight_requested_at=None)
+            return [("pause", "paused", {"reason": "preflight_blocked", "hosts": names})]
+        return apply_blocked
+
+    targets = {t for t in offers.values()}
+    expected = run.target_version
+    mismatch = None
+    if None in targets:
+        mismatch = {"offered": "?"}
+    elif expected is None and len(targets) > 1:
+        mismatch = {"offered": ", ".join(sorted(targets))}
+    elif expected is not None and targets - {expected}:
+        mismatch = {"expected": expected, "offered": ", ".join(sorted(targets - {expected}))}
+    if mismatch:
+        async def apply_mismatch(db, run):
+            # OPNsense always installs the newest offered version; a later ring must not get one the earlier ring never ran.
+            pause(run, "target_version_changed", **mismatch)
+            _progress(run, preflight_started=None, preflight_requested_at=None)
+            return [("pause", "paused", {"reason": "target_version_changed", **mismatch})]
+        return apply_mismatch
+
+    # Baselines and order (BACKUP before MASTER inside a cluster).
+    baselines = {}
+    versions = {}
+    for rh_id in offers:
+        host = usable[rh_id]
+        baselines[str(host.id)] = await rollout_health.baseline(db, host)
+        versions[rh_id] = baselines[str(host.id)].get("version")
+    roles = await get_statuses([usable[i] for i in offers])
+    rank = {"BACKUP": 0, None: 1, "MASTER": 2}
+
+    def order_key(rh_id):
+        host = usable[rh_id]
+        role = roles.get(host.id, {}).get("carp_status") if host.cluster_name else None
+        return (1 if host.cluster_name else 0, host.cluster_name or "", rank.get(role, 1), host.name)
+    ordered = sorted(offers, key=order_key)
+    target = next(iter(targets)) if targets else expected
+
+    async def apply_ready(db, run):
+        for rh in hosts_of(run, index):
+            if rh.id in skips:
+                rh.status, rh.detail = "skipped", skips[rh.id]
+            elif rh.id in offers:
+                rh.status, rh.detail = "queued", None
+                rh.order = ordered.index(rh.id)
+                rh.version_before = versions.get(rh.id)
+        w = wave_row(run, index)
+        w.baseline = baselines
+        if run.target_version is None and target:
+            run.target_version = target
+        _progress(run, preflight_started=None, preflight_requested_at=None, blocked_since=None, cluster_recheck_at=None)
+        if not offers:
+            w.status, w.finished_at = "skipped", _now()
+            events = [("wave", "skipped", {"wave": index})]
+            events += _next_wave(run)
+            return events
+        w.status = "running"
+        run.status = "wave_running"
+        return [("wave", "started", {"wave": index, "target_version": run.target_version,
+                                     "hosts": [usable[i].name for i in ordered], "skipped": len(skips)})]
+    return apply_ready
+
+
+def _next_wave(run: RolloutRun) -> list:
+    index = run.current_wave + 1
+    if index >= len(run.waves):
+        run.status, run.finished_at = "succeeded", _now()
+        return [("finish", "succeeded")]
+    run.current_wave = index
+    w = wave_row(run, index)
+    if wave_spec(run, index).get("require_approval"):
+        w.status = run.status = "awaiting_approval"
+        return [("wave", "awaiting_approval", {"wave": index})]
+    w.status, run.status = "preflight", "preflight"
+    return []
+
+
+async def _handle_wave_running(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    if any(h.status in HOST_OPEN for h in run.hosts):
+        return None  # strictly one host at a time
+    index = run.current_wave
+    queued = [h for h in hosts_of(run, index) if h.status == "queued"]
+    if not queued:
+        soak = int(wave_spec(run, index).get("soak_minutes") or 0)
+
+        async def apply_soak(db, run):
+            w = wave_row(run, index)
+            w.status, w.soak_until = "soaking", _now() + timedelta(minutes=soak)
+            run.status = "soaking"
+            return [("wave", "soaking", {"wave": index, "minutes": soak})]
+        return apply_soak
+    if problem := await _actor_problem(db, run):
+        return problem
+    reason = await _may_start_now(db, run)
+    if reason:
+        return _to_waiting(reason)
+    nxt = queued[0]
+    host = await db.get(OpnsenseHost, nxt.host_id) if nxt.host_id else None
+    if host is None or not host.enabled:
+        async def apply_skip(db, run):
+            rh = next(h for h in run.hosts if h.id == nxt.id)
+            rh.status, rh.detail = "skipped", _note("host_deleted" if host is None else "host_disabled")
+            return [("host", "skipped", {"host": nxt.host_name})]
+        return apply_skip
+    spec = wave_spec(run, index)
+    prog = run.progress or {}
+
+    if host.cluster_name:
+        fleet = await peers(db, host)
+        statuses = await get_statuses(fleet)
+        done_ids = {h.host_id for h in run.hosts if h.status == "succeeded"}
+        done_peers = [p for p in fleet if p.id in done_ids and p.id != host.id]
+        baselines = {}
+        for w in run.waves:
+            baselines.update(w.baseline or {})
+        # The cluster as a whole must be healthy before a node is touched; nodes done earlier in this run
+        # must pass their checks again (BACKUP updated -> now it may carry the traffic for the MASTER).
+        gate = await rollout_health.evaluate(
+            db, done_peers + [host], baselines, target_version=run.target_version,
+            checks={**(run.plan_snapshot.get("gate_checks") or {}), "carp": True, "compliance": False})
+        # The node itself is not updated yet: only its reachability, the cluster and the locks count for it.
+        failed = [f"{hid}:{c}" for hid, row in gate["hosts"].items() for c, r in row.items()
+                  if r["result"] == "fail" and not (hid == str(host.id) and c in ("version", "gateways", "services"))]
+        if failed:
+            recheck = _ts(prog.get("cluster_recheck_at"))
+            if recheck is None:
+                async def apply_recheck(db, run):
+                    _progress(run, cluster_recheck_at=_iso(_now() + GATE_RECHECK), cluster_gate=gate)
+                    return [("cluster_gate", "recheck", {"host": host.name, "failed": failed})]
+                return apply_recheck
+            if _now() < recheck:
+                return None
+
+            async def apply_cluster_fail(db, run):
+                pause(run, "cluster_gate_failed", cluster=host.cluster_name, host=host.name, failed=_named(run, failed))
+                _progress(run, cluster_recheck_at=None, cluster_gate=gate)
+                return [("cluster_gate", "failed", {"host": host.name, "failed": failed})]
+            return apply_cluster_fail
+        if statuses.get(host.id, {}).get("carp_status") == "MASTER" and not spec.get("carp_handover"):
+            async def apply_master(db, run):
+                pause(run, "master_needs_handover", host=host.name, cluster=host.cluster_name)
+                return [("pause", "paused", {"reason": "master_needs_handover", "host": host.name})]
+            return apply_master
+
+    try:
+        await ensure_no_maintenance(db, [host])
+    except HTTPException:
+        since = _ts(prog.get("blocked_since"))
+        if since is None:
+            async def apply_blocked_start(db, run):
+                _progress(run, blocked_since=_iso(_now()))
+                return [("host", "waiting_for_lock", {"host": host.name})]
+            return apply_blocked_start
+        if _now() - since < BLOCKED_PATIENCE:
+            return None
+
+        async def apply_blocked(db, run):
+            pause(run, "blocked_by_maintenance", host=host.name)
+            _progress(run, blocked_since=None)
+            return [("pause", "paused", {"reason": "blocked_by_maintenance", "host": host.name})]
+        return apply_blocked
+
+    async def apply_start(db, run):
+        await policy_lock(db)
+        rh = next(h for h in run.hosts if h.id == nxt.id)
+        # Idempotency guard: a host that already has a schedule never gets a second one.
+        if rh.status != "queued" or rh.scheduled_maintenance_id is not None:
+            return []
+        now = _now()
+        sched = ScheduledMaintenance(
+            host_id=host.id, host_name=host.name, action=run.action, run_at=now, status="pending",
+            actor=run.started_by, created_at=now, carp_handover=bool(spec.get("carp_handover") and host.cluster_name),
+            auto_verify=True, origin="rollout",
+            progress={"rollout": {"run_id": str(run.id), "wave": index, "plan": run.plan_name}},
+        )
+        db.add(sched)
+        await db.flush()
+        rh.scheduled_maintenance_id, rh.status, rh.started_at = sched.id, "running", now
+        _progress(run, blocked_since=None, cluster_recheck_at=None)
+        return [("host", "started", {"host": host.name, "schedule_id": str(sched.id), "handover": sched.carp_handover})]
+    return apply_start
+
+
+async def _handle_soaking(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    w = wave_row(run, run.current_wave)
+    if w.soak_until and _now() < w.soak_until:
+        return None
+
+    async def apply(db, run):
+        w = wave_row(run, run.current_wave)
+        w.status, w.gate_attempts = "gate", 0
+        run.status = "gate"
+        _progress(run, gate_recheck_at=None)
+        return []
+    return apply
+
+
+async def _handle_gate(db: AsyncSession, run: RolloutRun) -> Decision | None:
+    recheck = _ts((run.progress or {}).get("gate_recheck_at"))
+    if recheck and _now() < recheck:
+        return None
+    index = run.current_wave
+    w = wave_row(run, index)
+    done = [h for h in hosts_of(run, index) if h.status == "succeeded" and h.host_id]
+    hosts = [h for h in [await db.get(OpnsenseHost, d.host_id) for d in done] if h is not None]
+    gate = await rollout_health.evaluate(db, hosts, w.baseline or {}, target_version=run.target_version,
+                                         checks=run.plan_snapshot.get("gate_checks"),
+                                         compliance_mode=run.plan_snapshot.get("compliance_mode", "warn"))
+
+    async def apply(db, run):
+        w = wave_row(run, index)
+        w.gate_result = gate
+        w.gate_attempts = (w.gate_attempts or 0) + 1
+        failed = rollout_health.failed_checks(gate)
+        events = [("gate", gate["verdict"], {"wave": index, "attempt": w.gate_attempts, "failed": failed, "result": gate})]
+        if gate["verdict"] == "fail":
+            if w.gate_attempts < 2:
+                _progress(run, gate_recheck_at=_iso(_now() + GATE_RECHECK))
+                return events
+            _progress(run, gate_recheck_at=None)
+            pause(run, "gate_failed", wave=w.label, failed=_named(run, failed))
+            return events + [("pause", "paused", {"reason": "gate_failed"})]
+        w.status, w.finished_at = "passed", _now()
+        _progress(run, gate_recheck_at=None)
+        return events + _next_wave(run)
+    return apply
+
+
+_HANDLERS = {
+    "waiting_window": _handle_waiting_window,
+    "resuming": _handle_resuming,
+    "preflight": _handle_preflight,
+    "wave_running": _handle_wave_running,
+    "soaking": _handle_soaking,
+    "gate": _handle_gate,
+}
+
+
+# ── automatic start at a window occurrence ────────────────────────────────────
+
+async def auto_start(db: AsyncSession) -> int:
+    started = 0
+    plans = (await db.scalars(select(RolloutPlan).where(
+        RolloutPlan.auto_start.is_(True), RolloutPlan.activated_at.is_not(None), RolloutPlan.window_id.is_not(None)))).all()
+    for plan in plans:
+        window = await db.get(MaintenanceWindow, plan.window_id)
+        occ = window_service.startable(window, _now()) if window else None
+        if occ is None:
+            continue
+        if await db.scalar(select(RolloutRun.id).where(RolloutRun.plan_id == plan.id, RolloutRun.window_occurrence == occ.start)):
+            continue
+        if (plan.auto_start_error or {}).get("occurrence") == _iso(occ.start):
+            continue  # already reported for this occurrence
+        await policy_lock(db)
+        plan = await rollout_service.load_plan(db, plan.id)
+        if await db.scalar(select(RolloutRun.id).where(RolloutRun.plan_id == plan.id, RolloutRun.window_occurrence == occ.start)):
+            continue  # another backend started it while this one waited for the lock (not a refusal)
+        resolved = await rollout_service.resolve(db, plan)
+        problem = None
+        if plan.action != "update":
+            problem = _note("auto_start_update_only")
+        elif plan.activated_fingerprint != rollout_service.fingerprint(plan, resolved):
+            problem = _note("plan_changed")  # someone changed the plan, its groups or its window since activation
+        elif not await may_roll_out(db, plan.activated_by or "", plan.activated_by_id):
+            problem = _note("actor_invalid", user=plan.activated_by)
+        elif resolved["errors"]:
+            problem = _note("plan_invalid", reason=resolved["errors"][0]["code"])
+        elif (other := await active_run(db)) is not None:
+            problem = _note("other_run_active", plan=other.plan_name)
+        if problem:
+            plan.auto_start_error = {"occurrence": _iso(occ.start), "at": _iso(_now()), **problem}
+            await db.commit()
+            token = audit_service.context.set({"actor": "system:rollout", "id": str(plan.id)})
+            try:
+                await audit_service.persist(kind="rollout", action="auto_start", status="refused", entity_id=str(plan.id),
+                                            entity_name=plan.name, details=plan.auto_start_error)
+            finally:
+                audit_service.context.reset(token)
+            continue
+        try:
+            run = await create_run(db, plan, resolved, started_by=plan.activated_by, started_by_id=plan.activated_by_id,
+                               trigger="window", occurrence=occ.start)
+            plan.auto_start_error = None
+            await db.commit()
+        except IntegrityError:
+            await db.rollback()  # another backend started it for this occurrence
+            continue
+        started += 1
+        await _audit(run, "start", "started", trigger="window", occurrence=_iso(occ.start),
+                     hosts=sum(len(w["hosts"]) for w in resolved["waves"]))
+    await db.commit()
+    return started
+
+
+# ── human actions (called by the router under policy_lock) ────────────────────
+
+async def count_active(db: AsyncSession) -> int:
+    return await db.scalar(select(func.count()).select_from(RolloutRun).where(RolloutRun.status.in_(ACTIVE))) or 0

backend/app/services/rollout_health.py

@@ -0,0 +1,218 @@
+"""Health gate of a firmware rollout (1.25): is a host, after its update, as healthy as before?
+
+``baseline(db, host)`` is taken right before a wave starts, ``evaluate(...)`` after the soak time
+(and before each node of an already touched cluster, and on resume). Every check of every host
+yields ``{"result": ok|warn|fail|skip, "code": <machine code>, "params": {...}}``; the UI translates
+the code (rollouts.gate.code.*), so no German prose travels through here.
+
+What the checks look at (all read-only on the firewall, except that the compliance check downloads
+the configuration like the 6-hourly security check does):
+
+* online     — enabled, the API answers, firmware job idle (``running == ready``).
+* version    — the installed version is the rollout's target version.
+* carp       — per cluster: every node online, exactly one MASTER per VHID, no node in CARP
+               maintenance mode, no CARP auto-revert timer running.
+* gateways   — gateways that were up in the baseline are up again; a gateway that was already bad
+               before only warns (1.24 ``get_gateway_health``).
+* services   — services that ran in the baseline run again.
+* locks      — no open maintenance operation, restore or unconfirmed deployment in the host/cluster.
+* compliance — security score not lower than before: warn (default) or fail. A weak signal: an
+               update may legitimately change defaults.
+
+A reading that cannot be taken is never "ok": unreadable now but readable in the baseline fails,
+unreadable in both is "skip" (nothing to compare; e.g. an API key without that permission).
+"""
+import asyncio
+from collections import defaultdict
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.carp_maintenance_window import CarpMaintenanceWindow
+from app.models.compliance_result import ComplianceResult
+from app.models.host import OpnsenseHost
+from app.services import compliance_service, gateway_health_service, opnsense_client as client
+from app.services.host_maintenance_service import _shared_change_blockers, peers
+from app.services.host_status_service import get_statuses
+from app.services.resource_monitor_service import _services_compact
+
+CHECKS = ("online", "version", "carp", "gateways", "services", "locks", "compliance")
+DEFAULT_CHECKS = {"online": True, "version": True, "carp": True, "gateways": True, "services": True, "locks": True,
+                  "compliance": False}
+ALWAYS_ON = ("online", "version", "locks")  # a rollout without these would be blind
+_RANK = {"skip": 0, "ok": 0, "warn": 1, "fail": 2}
+
+
+def _r(result: str, code: str, **params) -> dict:
+    return {"result": result, "code": code, "params": params}
+
+
+async def _optional(coro):
+    try:
+        return await coro
+    except client.OpnsenseAPIError:
+        return None
+
+
+async def _gateways(host) -> dict | None:
+    gws = await _optional(gateway_health_service.get_gateway_health(host))
+    return None if gws is None else {g.name: g.state for g in gws}
+
+
+async def _services(host) -> dict | None:
+    listing = await _optional(client.search_services(host))
+    return None if listing is None else _services_compact(listing)
+
+
+async def baseline(db: AsyncSession, host: OpnsenseHost) -> dict:
+    info, gateways, services = await asyncio.gather(
+        _optional(client.get_firmware_info(host)), _gateways(host), _services(host))
+    score = await db.scalar(select(ComplianceResult.score).where(ComplianceResult.host_id == host.id))
+    return {
+        "version": (info or {}).get("product_version"),
+        "gateways": gateways,
+        "services": services,
+        "compliance_score": score,
+    }
+
+
+def _check_gateways(before: dict | None, now: dict | None) -> dict:
+    if now is None:
+        return _r("skip", "unreadable_both") if before is None else _r("fail", "unreadable")
+    if before is None:
+        bad = sorted(n for n, s in now.items() if s == "offline")
+        return _r("warn", "gateway_down", names=", ".join(bad)) if bad else _r("ok", "no_baseline")
+    down = sorted(n for n, s in before.items() if s in ("online", "unknown") and now.get(n) == "offline")
+    gone = sorted(n for n in before if n not in now)
+    degraded = sorted(n for n, s in now.items() if s == "degraded" and before.get(n) != "degraded")
+    still_bad = sorted(n for n, s in before.items() if s in ("offline", "degraded") and now.get(n) in ("offline", "degraded"))
+    # "unknown" (not monitored) going to "offline" is a real change too: monitoring just started and reports down.
+    if down:
+        return _r("fail", "gateway_down", names=", ".join(down))
+    if gone:
+        return _r("warn", "gateway_missing", names=", ".join(gone))
+    if degraded:
+        return _r("warn", "gateway_degraded", names=", ".join(degraded))
+    if still_bad:
+        return _r("warn", "gateway_bad_before", names=", ".join(still_bad))
+    return _r("ok", "ok")
+
+
+def _check_services(before: dict | None, now: dict | None) -> dict:
+    if now is None:
+        return _r("skip", "unreadable_both") if before is None else _r("fail", "unreadable")
+    if before is None:
+        return _r("ok", "no_baseline")
+    stopped = sorted(name.split("/")[0] for name, running in before.items() if running == 1 and now.get(name) != 1)
+    if stopped:
+        return _r("fail", "service_stopped", names=", ".join(dict.fromkeys(stopped)))
+    return _r("ok", "ok")
+
+
+async def _check_carp(db: AsyncSession, fleet: list[OpnsenseHost], statuses: dict) -> dict:
+    offline = sorted(h.name for h in fleet if h.enabled and statuses.get(h.id, {}).get("status") != "online")
+    if offline:
+        return _r("fail", "node_offline", names=", ".join(offline))
+    online = [h for h in fleet if statuses.get(h.id, {}).get("status") == "online"]
+    demoted = sorted(h.name for h in online if statuses[h.id].get("carp_maintenance_mode"))
+    if demoted:
+        return _r("fail", "maintenance_mode", names=", ".join(demoted))
+    timers = (await db.scalars(select(CarpMaintenanceWindow.host_name).where(
+        CarpMaintenanceWindow.host_id.in_([h.id for h in fleet]), CarpMaintenanceWindow.status == "active"))).all()
+    if timers:
+        return _r("fail", "auto_revert_active", names=", ".join(sorted(timers)))
+    masters: dict[str, int] = defaultdict(int)
+    broken: set[str] = set()
+    for h in online:
+        for vhid, state in (statuses[h.id].get("carp_vips") or {}).items():
+            masters[vhid] += state == "MASTER"
+            if state not in ("MASTER", "BACKUP"):
+                broken.add(vhid)
+    wrong = sorted(set(v for v, n in masters.items() if n != 1) | broken, key=lambda v: (len(v), v))
+    if wrong:
+        return _r("fail", "vhid_roles", vhids=", ".join(wrong))
+    if not masters and not any(statuses[h.id].get("carp_status") == "MASTER" for h in online):
+        return _r("fail", "no_master")
+    return _r("ok", "ok")
+
+
+async def evaluate(db: AsyncSession, hosts: list[OpnsenseHost], baselines: dict, *, target_version: str | None,
+                   checks: dict | None = None, compliance_mode: str = "warn") -> dict:
+    """{"hosts": {host_id: {check: result}}, "verdict": ok|warn|fail}. ``baselines`` is keyed by str(host.id)."""
+    enabled = {**DEFAULT_CHECKS, **(checks or {})}
+    for name in ALWAYS_ON:
+        enabled[name] = True
+    results: dict[str, dict] = {}
+    fleets: dict[str, list[OpnsenseHost]] = {}
+    for host in hosts:
+        fleets[str(host.id)] = await peers(db, host) if host.cluster_name else [host]
+    every = {h.id: h for fleet in fleets.values() for h in fleet}
+    statuses = await get_statuses(list(every.values()))
+    carp_cache: dict[str, dict] = {}
+    for host in hosts:
+        hid = str(host.id)
+        before = baselines.get(hid) or {}
+        row: dict[str, dict] = {}
+        st = statuses.get(host.id, {})
+        running = await _optional(client.get_firmware_running(host)) if st.get("status") == "online" else None
+        if not host.enabled:
+            row["online"] = _r("fail", "disabled")
+        elif st.get("status") != "online":
+            row["online"] = _r("fail", "offline")
+        elif (running or {}).get("status") != "ready":
+            row["online"] = _r("fail", "firmware_busy")
+        else:
+            row["online"] = _r("ok", "ok")
+        reachable = row["online"]["result"] == "ok"
+
+        if enabled["version"]:
+            info = await _optional(client.get_firmware_info(host)) if reachable else None
+            version = (info or {}).get("product_version")
+            if not target_version:
+                row["version"] = _r("skip", "no_target")
+            elif version is None:
+                row["version"] = _r("fail", "unreadable")
+            elif version != target_version:
+                row["version"] = _r("fail", "version_mismatch", expected=target_version, actual=version)
+            else:
+                row["version"] = _r("ok", "ok", actual=version)
+
+        if enabled["carp"]:
+            if not host.cluster_name:
+                row["carp"] = _r("skip", "no_cluster")
+            else:
+                if host.cluster_name not in carp_cache:
+                    carp_cache[host.cluster_name] = await _check_carp(db, fleets[hid], statuses)
+                row["carp"] = carp_cache[host.cluster_name]
+
+        if enabled["gateways"]:
+            row["gateways"] = _check_gateways(before.get("gateways"), await _gateways(host) if reachable else None)
+        if enabled["services"]:
+            row["services"] = _check_services(before.get("services"), await _services(host) if reachable else None)
+
+        blockers = await _shared_change_blockers(db, fleets[hid])
+        row["locks"] = _r("fail", "open_locks", count=len(blockers)) if blockers else _r("ok", "ok")
+
+        if enabled["compliance"]:
+            base_score = before.get("compliance_score")
+            if base_score is None:
+                row["compliance"] = _r("skip", "no_baseline")
+            elif not reachable:
+                row["compliance"] = _r("fail", "unreadable")
+            else:
+                fresh = await compliance_service.check_host(db, host)
+                if fresh.score is None:
+                    row["compliance"] = _r("warn", "unreadable")
+                elif fresh.score < base_score:
+                    row["compliance"] = _r("fail" if compliance_mode == "fail" else "warn", "score_lower",
+                                           before=base_score, after=fresh.score)
+                else:
+                    row["compliance"] = _r("ok", "ok", before=base_score, after=fresh.score)
+        results[hid] = row
+    worst = max((_RANK[c["result"]] for row in results.values() for c in row.values()), default=0)
+    return {"hosts": results, "verdict": ("ok", "warn", "fail")[worst]}
+
+
+def failed_checks(gate: dict) -> list[str]:
+    """Flat "host-id:check" list of everything that failed (for notifications and the audit trail)."""
+    return [f"{hid}:{name}" for hid, row in (gate.get("hosts") or {}).items() for name, c in row.items() if c["result"] == "fail"]

backend/app/services/rollout_service.py

@@ -0,0 +1,138 @@
+"""Rollout plans: resolving waves to hosts, validation, preview token / activation fingerprint.
+
+The run engine (rollout_engine.py) builds on ``resolve``: what a preview shows is exactly what a run
+freezes when it starts.
+"""
+import hashlib
+import json
+from datetime import datetime, timezone
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.orm import selectinload
+
+from app.models.host import OpnsenseHost
+from app.models.maintenance_window import MaintenanceWindow
+from app.models.policy import DeviceGroup
+from app.models.rollout import RolloutPlan
+from app.services import rollout_health, window_service
+
+# Hard limits: an unattended engine that fires firmware updates must stay small and explainable.
+MAX_WAVES = 10
+MAX_HOSTS = 100
+MAX_SOAK_MINUTES = 24 * 60
+
+
+def _issue(code: str, **params) -> dict:
+    return {"code": code, "params": params}
+
+
+async def resolve(db: AsyncSession, plan: RolloutPlan) -> dict:
+    """Waves with their hosts (a host in several waves belongs to the earliest), errors that block a
+    start/activation, and warnings."""
+    errors: list[dict] = []
+    warnings: list[dict] = []
+    seen: dict = {}
+    duplicates: set[str] = set()
+    waves = []
+    group_ids = {g.id for w in plan.waves for g in w.groups}
+    groups = {g.id: g for g in (await db.scalars(
+        select(DeviceGroup).where(DeviceGroup.id.in_(group_ids)).options(selectinload(DeviceGroup.hosts)))).all()} if group_ids else {}
+    for index, wave in enumerate(plan.waves):
+        hosts: list[OpnsenseHost] = []
+        for group in sorted(wave.groups, key=lambda g: g.name):
+            for host in sorted(groups[group.id].hosts, key=lambda h: h.name):
+                if host.id in seen:
+                    if seen[host.id] != index:
+                        duplicates.add(host.name)
+                    continue
+                seen[host.id] = index
+                hosts.append(host)
+        if not hosts:
+            warnings.append(_issue("empty_wave", wave=wave.label))
+        clusters = sorted({h.cluster_name for h in hosts if h.cluster_name})
+        if clusters and not wave.carp_handover:
+            warnings.append(_issue("master_needs_handover", wave=wave.label, clusters=", ".join(clusters)))
+        waves.append({
+            "index": index, "label": wave.label, "soak_minutes": wave.soak_minutes, "carp_handover": wave.carp_handover,
+            "require_approval": wave.require_approval, "groups": sorted(g.name for g in wave.groups),
+            "hosts": [{"id": str(h.id), "name": h.name, "cluster_name": h.cluster_name, "enabled": h.enabled} for h in hosts],
+        })
+    if duplicates:
+        warnings.append(_issue("duplicate_hosts", names=", ".join(sorted(duplicates))))
+    all_hosts = [h for w in waves for h in w["hosts"]]
+    disabled = sorted(h["name"] for h in all_hosts if not h["enabled"])
+    if disabled:
+        warnings.append(_issue("disabled_hosts", names=", ".join(disabled)))
+    if not plan.waves:
+        errors.append(_issue("no_waves"))
+    elif not all_hosts:
+        errors.append(_issue("no_hosts"))
+    if len(plan.waves) > MAX_WAVES:
+        errors.append(_issue("too_many_waves", limit=MAX_WAVES))
+    if len(all_hosts) > MAX_HOSTS:
+        errors.append(_issue("too_many_hosts", limit=MAX_HOSTS))
+
+    # Clusters: where are their nodes?
+    cluster_names = sorted({h["cluster_name"] for h in all_hosts if h["cluster_name"]})
+    for cluster in cluster_names:
+        members = (await db.scalars(select(OpnsenseHost.id).where(OpnsenseHost.cluster_name == cluster))).all()
+        in_plan = {h["id"]: w["index"] for w in waves for h in w["hosts"] if h["cluster_name"] == cluster}
+        partial = len(in_plan) < len(members)
+        split = len(set(in_plan.values())) > 1
+        if plan.action == "major_update" and (partial or split):
+            # Never two major releases side by side in one cluster longer than one wave.
+            errors.append(_issue("cluster_split", cluster=cluster))
+        elif partial:
+            warnings.append(_issue("cluster_partial", cluster=cluster))
+        elif split:
+            warnings.append(_issue("mixed_versions", cluster=cluster, wave=waves[max(in_plan.values())]["label"]))
+
+    window = await db.get(MaintenanceWindow, plan.window_id) if plan.window_id else None
+    if plan.auto_start and plan.action != "update":
+        errors.append(_issue("auto_start_update_only"))
+    if plan.auto_start and window is None:
+        errors.append(_issue("auto_start_needs_window"))
+    if window is None:
+        warnings.append(_issue("no_window"))
+    elif not window.enabled:
+        warnings.append(_issue("window_disabled"))
+    now = datetime.now(timezone.utc)
+    return {
+        "waves": waves, "errors": errors, "warnings": warnings,
+        "window": {"id": str(window.id), "name": window.name, "definition": window_service.definition(window),
+                   "next": [o.as_dict() for o in window_service.occurrences(window, now, 3)] if window.enabled else []}
+        if window else None,
+    }
+
+
+def definition(plan: RolloutPlan) -> dict:
+    return {
+        "id": str(plan.id), "name": plan.name, "action": plan.action, "window_id": str(plan.window_id) if plan.window_id else None,
+        "auto_start": plan.auto_start, "gate_checks": effective_checks(plan.gate_checks), "compliance_mode": plan.compliance_mode,
+        "waves": [{"position": w.position, "label": w.label, "soak_minutes": w.soak_minutes, "carp_handover": w.carp_handover,
+                   "require_approval": w.require_approval, "group_ids": sorted(str(g.id) for g in w.groups)} for w in plan.waves],
+    }
+
+
+def effective_checks(checks: dict | None) -> dict:
+    merged = {**rollout_health.DEFAULT_CHECKS, **{k: bool(v) for k, v in (checks or {}).items() if k in rollout_health.CHECKS}}
+    for name in rollout_health.ALWAYS_ON:
+        merged[name] = True
+    return merged
+
+
+def fingerprint(plan: RolloutPlan, resolved: dict) -> str:
+    """Covers the plan, every resolved host (id, name, cluster, enabled) and the window definition: a change
+    to any of them invalidates a preview and stops an activated automatic start."""
+    state = {
+        "plan": definition(plan),
+        "hosts": [[w["index"], h["id"], h["name"], h["cluster_name"], h["enabled"]] for w in resolved["waves"] for h in w["hosts"]],
+        "window": (resolved.get("window") or {}).get("definition"),
+    }
+    return hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
+
+
+async def load_plan(db: AsyncSession, plan_id) -> RolloutPlan | None:
+    return await db.scalar(
+        select(RolloutPlan).where(RolloutPlan.id == plan_id).execution_options(populate_existing=True))

backend/app/services/rule_adopt_service.py

@@ -19,27 +19,18 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.models.host import OpnsenseHost
 from app.models.rule import FirewallRule
 from app.models.rule_deployment import RuleDeployment
-from app.services import opnsense_client
+from app.services import adopt_link_service, opnsense_client
+from app.services.adopt_common import NotAdoptable, NotFoundOnHost, clean, on  # noqa: F401  (NotAdoptable is re-exported for routers/rules.py)
 from app.services.policy_service import revision
 from app.services.rule_deploy_service import _build_payload
 
 
-class RuleNotFoundOnHost(Exception):
+class RuleNotFoundOnHost(NotFoundOnHost):
     pass
 
 
-class NotAdoptable(Exception):
-    def __init__(self, reasons: list[str]):
-        super().__init__(", ".join(reasons))
-        self.reasons = reasons
-
-
-def _on(value) -> bool:
-    return value in ("1", 1, True)
-
-
-def _clean(value) -> str:
-    return "" if value is None else str(value).strip()
+_on = on
+_clean = clean
 
 
 def reasons_not_adoptable(row: dict, managed_uuids: set[str]) -> list[str]:
@@ -73,6 +64,30 @@ def reasons_not_adoptable(row: dict, managed_uuids: set[str]) -> list[str]:
     return reasons
 
 
+def _values(row: dict) -> dict:
+    """The fields OPNexus manages, normalised exactly as adopting stores them (also what a link compares)."""
+    return {
+        "description": _clean(row.get("description")),
+        "interface": _clean(row.get("interface")),
+        "direction": _clean(row.get("direction")),
+        "ipprotocol": _clean(row.get("ipprotocol")),
+        "protocol": _clean(row.get("protocol")) or "any",
+        "action": _clean(row.get("action")),
+        "source_net": _clean(row.get("source_net")) or "any",
+        "source_port": _clean(row.get("source_port")) or None,
+        "destination_net": _clean(row.get("destination_net")) or "any",
+        "destination_port": _clean(row.get("destination_port")) or None,
+        "enabled": _on(row.get("enabled")),
+        "log": _on(row.get("log")),
+    }
+
+
+LINK_SPEC = adopt_link_service.LinkSpec(
+    model=FirewallRule, deployment=RuleDeployment, fk="rule_id", uuid_attr="opnsense_rule_uuid",
+    identity="description", payload=_build_payload, revision=revision,
+)
+
+
 async def _managed_uuids(db: AsyncSession, host: OpnsenseHost) -> set[str]:
     rows = await db.scalars(
         select(RuleDeployment.opnsense_rule_uuid).where(
@@ -90,6 +105,7 @@ async def _live_rows(host: OpnsenseHost) -> list[dict]:
 
 async def list_unmanaged(db: AsyncSession, host: OpnsenseHost) -> list[dict]:
     managed = await _managed_uuids(db, host)
+    central = await adopt_link_service.load_central(db, LINK_SPEC)
     result = []
     for row in await _live_rows(host):
         if row.get("is_automatic"):
@@ -97,7 +113,11 @@ async def list_unmanaged(db: AsyncSession, host: OpnsenseHost) -> list[dict]:
         reasons = reasons_not_adoptable(row, managed)
         if "already_managed" in reasons:
             continue
+        matches, similar = adopt_link_service.find(central, LINK_SPEC, host, _values(row))
         result.append({
+            "matches": [adopt_link_service.describe(LINK_SPEC, m) for m in matches],
+            "similar": [adopt_link_service.describe(LINK_SPEC, m) for m in similar],
+            "linkable": any(not m.busy for m in matches) and not reasons,
             "uuid": _clean(row.get("uuid")),
             "description": _clean(row.get("description")),
             "interface": _clean(row.get("interface")),
@@ -117,7 +137,15 @@ async def list_unmanaged(db: AsyncSession, host: OpnsenseHost) -> list[dict]:
     return result
 
 
-async def adopt(db: AsyncSession, host: OpnsenseHost, rule_uuid: str) -> FirewallRule:
+async def adopt(db: AsyncSession, host: OpnsenseHost, rule_uuid: str, link_to=None) -> FirewallRule:
+    """Takes the rule over as a new central rule, or - with `link_to` - links it to that existing rule."""
+    if link_to is not None:
+        existing = await db.scalar(select(RuleDeployment).where(
+            RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid == rule_uuid))
+        if existing is not None:
+            if existing.rule_id == link_to:
+                return await db.get(FirewallRule, link_to)  # second click: already linked
+            raise NotAdoptable(["already_managed"])
     managed = await _managed_uuids(db, host)
     row = next((r for r in await _live_rows(host) if _clean(r.get("uuid")) == rule_uuid), None)
     if row is None:
@@ -125,23 +153,10 @@ async def adopt(db: AsyncSession, host: OpnsenseHost, rule_uuid: str) -> Firewal
     reasons = reasons_not_adoptable(row, managed)
     if reasons:
         raise NotAdoptable(reasons)
+    if link_to is not None:
+        return await adopt_link_service.link(db, LINK_SPEC, host, link_to, rule_uuid, _values(row))
 
-    rule = FirewallRule(
-        description=_clean(row.get("description")),
-        interface=_clean(row.get("interface")),
-        direction=_clean(row.get("direction")),
-        ipprotocol=_clean(row.get("ipprotocol")),
-        protocol=_clean(row.get("protocol")) or "any",
-        action=_clean(row.get("action")),
-        source_net=_clean(row.get("source_net")) or "any",
-        source_port=_clean(row.get("source_port")) or None,
-        destination_net=_clean(row.get("destination_net")) or "any",
-        destination_port=_clean(row.get("destination_port")) or None,
-        enabled=_on(row.get("enabled")),
-        log=_on(row.get("log")),
-        targets=[host],
-        groups=[],
-    )
+    rule = FirewallRule(**_values(row), targets=[host], groups=[])
     db.add(rule)
     await db.flush()
     revision(db, rule, "Von Firewall übernommen")

backend/app/services/rule_analysis/__init__.py

@@ -0,0 +1 @@
+"""Rule analysis (1.26): which firewall rule decides a packet, with honest confidence."""

backend/app/services/rule_analysis/cleanup.py

@@ -0,0 +1,113 @@
+"""Cleanup candidates of one firewall: unused, shadowed, duplicate and long-disabled rules (read-only).
+
+Nothing here changes a firewall. The list says *which rules are worth a look and how sure that is*; removing
+them is a separate step (not part of this module).
+
+Categories
+* ``unused``       OPNexus has watched the rule's pf counters for at least N days and saw no hit
+                   (services/rule_hit_service.py; N is a parameter, default 30).
+* ``shadowed``     an earlier quick rule already decides everything the rule matches (rule_analysis/coverage.py).
+* ``duplicate``    like shadowed, with the same action and the rule also covering the earlier one.
+* ``disabled_long`` disabled, and OPNexus has seen it unchanged for N days or more (rule history).
+
+Confidence (never higher than what is proven; what is not proven is never "sure")
+* ``sure``      the evidence is complete: unused = the rule is loaded in pf, has no schedule, the counters were
+                sampled recently and over the whole window; shadowed/duplicate = literal comparison, order from
+                ``sort_order``, the covering rule is loaded; disabled_long = observed unchanged for N days.
+* ``probable``  relies on a guess or on static alias content, or pf's loaded ruleset is not available, or the rule
+                has a schedule (a hit might happen at another time).
+* ``unknown``   there is a hint but no support: the rule is not loaded in pf (no counters can exist), or the
+                counters are stale.
+
+``removable`` says whether the rule could be offered for removal at all; the reason is a code:
+``automatic`` (generated by OPNsense), ``legacy`` (not yet migrated, migrate it in OPNsense first),
+``managed`` (OPNexus manages the rule: remove it through the normal flow of the rules page).
+"""
+from __future__ import annotations
+
+from datetime import datetime, timedelta
+from typing import Iterable
+
+from app.services import rule_hit_service
+from app.services.rule_analysis import coverage
+from app.services.rule_analysis.context import Context
+from app.services.rule_analysis.model import SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO, NormRule
+
+CATEGORIES = ("unused", "shadowed", "duplicate", "disabled_long")
+DEFAULT_UNUSED_DAYS = 30
+MIN_DAYS, MAX_DAYS = 1, 3650
+
+
+def removal_block(rule: NormRule, managed: dict | None) -> str | None:
+    """Why the rule could not be offered for removal (None = it could)."""
+    if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_NAT_AUTO):
+        return "automatic"
+    if rule.legacy:
+        return "legacy"
+    if managed is not None:
+        return "managed"
+    return None
+
+
+def unused_confidence(rule: NormRule, hit, now: datetime) -> str:
+    """Confidence of an 'unused' verdict (the caller already knows the window passed without a hit)."""
+    if rule.loaded is False:
+        return "unknown"   # pf did not load it (schedule, missing interface, gateway down): no counter can exist
+    if hit.last_sampled_at is None or now - hit.last_sampled_at > rule_hit_service.SAMPLE_FRESH:
+        return "unknown"   # stale counters say nothing about the present
+    if rule.loaded is None or rule.sched:
+        return "probable"  # pf's ruleset not available / the rule only works at certain times
+    return "sure"
+
+
+def _name(rule: NormRule) -> str:
+    return rule.description or rule.label or rule.uuid or (f"@{rule.pf_numbers[0]}" if rule.pf_numbers else "")
+
+
+def build(rules: list[NormRule], ctx: Context, *, hits: dict, states: dict, managed: dict[str, dict],
+          now: datetime, unused_days: int = DEFAULT_UNUSED_DAYS, order_guessed: bool = False) -> list[dict]:
+    """Candidate rows, in evaluation order. ``hits``: RuleHit per rule label, ``states``: LiveRuleState per label,
+    ``managed``: {label: {"kind", "id", "name"}} of the rules OPNexus deployed to this firewall."""
+    unused_days = max(MIN_DAYS, min(MAX_DAYS, int(unused_days)))
+    overlaps = {id(o.rule): o for o in coverage.analyse(rules, ctx, order_guessed=order_guessed)}
+    rows: list[dict] = []
+    for rule in rules:
+        if rule.automatic or rule.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
+            continue
+        categories: list[dict] = []
+        key = rule.uuid.lower()
+        hit = hits.get(key)
+        if hit is not None and rule_hit_service.classify(hit, now, unused_days) == "unused":
+            categories.append({"kind": "unused", "confidence": unused_confidence(rule, hit, now), "days": unused_days})
+        overlap = overlaps.get(id(rule))
+        if overlap is not None:
+            categories.append({
+                "kind": overlap.kind, "confidence": overlap.confidence,
+                "by": {"uuid": overlap.by.uuid, "description": _name(overlap.by), "section": overlap.by.section,
+                       "automatic": overlap.by.automatic},
+            })
+        state = states.get(key)
+        if (not rule.enabled and state is not None and state.present
+                and now - state.changed_at >= timedelta(days=unused_days)):
+            categories.append({"kind": "disabled_long", "confidence": "sure", "days": unused_days,
+                               "since": state.changed_at})
+        if not categories:
+            continue
+        block = removal_block(rule, managed.get(key))
+        rows.append({
+            **rule.summary(), "enabled": rule.enabled, "categories": categories,
+            "hit_state": rule_hit_service.classify(hit, now, unused_days) if hit is not None else None,
+            "last_hit_at": hit.last_hit_at if hit is not None else None,
+            "watched_since": hit.first_seen_at if hit is not None else None,
+            "removable": block is None, "not_removable_reason": block,
+            "managed_by": managed.get(key),
+        })
+    return rows
+
+
+def summarize(rows: Iterable[dict]) -> dict:
+    counts = {kind: 0 for kind in CATEGORIES}
+    for row in rows:
+        for category in row["categories"]:
+            counts[category["kind"]] += 1
+    return counts

backend/app/services/rule_analysis/context.py

@@ -0,0 +1,159 @@
+"""Everything about one firewall the analysis needs, fetched read-only and cached briefly.
+
+Sources (see the evidence block in opnsense_client.py): ``firewall/filter/searchRule`` (all
+rules incl. automatic ones, with ``sort_order``), ``diagnostics/firewall/pfStatistics/rules``
+(what pf has loaded), ``interfaces/overview/interfacesInfo`` (identifier, device, addresses
+incl. CARP VIPs, gateways), ``firewall/group/searchItem`` (interface groups),
+``firewall/alias/searchItem`` (alias definitions), ``firewall/d_nat/searchRule`` (port
+forwards, only to warn about NAT) and, per queried IP, ``firewall/alias_util/findReferences``.
+Optional sources may fail; the analysis then lowers its confidence instead of guessing.
+"""
+from __future__ import annotations
+
+import asyncio
+import ipaddress
+import logging
+import time
+from dataclasses import dataclass, field
+
+from app.models.host import OpnsenseHost
+from app.services import opnsense_client
+from app.services.rule_analysis import pf_ruleset
+from app.services.rule_analysis.model import IPAddr, IPNet
+
+logger = logging.getLogger(__name__)
+
+CACHE_SECONDS = 60
+# Built-in interface groups whose members are the matching dynamic devices.
+BUILTIN_GROUP_DEVICES = {"wireguard": ("wg",), "openvpn": ("ovpns", "ovpnc"), "enc0": ("enc0",)}
+
+
+@dataclass
+class IfInfo:
+    identifier: str
+    device: str
+    description: str
+    nets: list[IPNet] = field(default_factory=list)
+    addrs: list[IPAddr] = field(default_factory=list)      # primary addresses
+    vips: list[IPAddr] = field(default_factory=list)       # CARP / alias VIPs on this interface
+    has_gateway: bool = False
+
+
+@dataclass
+class Context:
+    rows: list[dict]
+    pf: pf_ruleset.PfRuleset | None
+    interfaces: dict[str, IfInfo]
+    groups: dict[str, set[str]] | None            # name -> member identifiers; None = unknown
+    aliases: dict[str, dict]
+    port_forwards: list[dict] | None
+    refs: dict[str, list[str] | None] = field(default_factory=dict)
+    notes: list[str] = field(default_factory=list)
+    loaded_at: float = 0.0
+
+    @property
+    def self_addrs(self) -> list[IPAddr]:
+        addrs = [a for i in self.interfaces.values() for a in (*i.addrs, *i.vips)]
+        return addrs + [ipaddress.ip_address("127.0.0.1"), ipaddress.ip_address("::1")]
+
+    def by_device(self, device: str) -> IfInfo | None:
+        return next((i for i in self.interfaces.values() if i.device == device), None)
+
+    def by_label(self, label: str) -> IfInfo | None:
+        lowered = label.strip().lower()
+        return self.interfaces.get(lowered) or next(
+            (i for i in self.interfaces.values() if i.description.lower() == lowered), None)
+
+
+def _interfaces(info: dict) -> dict[str, IfInfo]:
+    result: dict[str, IfInfo] = {}
+    for row in (info or {}).get("rows", []) or []:
+        identifier = str(row.get("identifier") or "").strip()
+        if not identifier:
+            continue
+        item = IfInfo(identifier=identifier, device=str(row.get("device") or ""),
+                      description=str(row.get("description") or identifier),
+                      has_gateway=bool(row.get("gateways")) or "default" in (row.get("routes") or []))
+        for entry in [*(row.get("ipv4") or []), *(row.get("ipv6") or [])]:
+            try:
+                iface = ipaddress.ip_interface(str(entry.get("ipaddr")).split("%")[0])
+            except ValueError:
+                continue
+            (item.vips if entry.get("vhid") or entry.get("vip") else item.addrs).append(iface.ip)
+            if iface.network not in item.nets:
+                item.nets.append(iface.network)
+        result[identifier] = item
+    return result
+
+
+def _groups(data: dict | None, interfaces: dict[str, IfInfo]) -> dict[str, set[str]] | None:
+    if not isinstance(data, dict) or not isinstance(data.get("rows"), list):
+        return None
+    groups: dict[str, set[str]] = {}
+    for row in data["rows"]:
+        name = str(row.get("ifname") or "")
+        members = {m.strip() for m in str(row.get("members") or "").split(",") if m.strip()}
+        prefixes = BUILTIN_GROUP_DEVICES.get(name)
+        if prefixes:
+            members |= {i.identifier for i in interfaces.values() if i.device.startswith(prefixes)}
+        if name:
+            groups[name] = members
+    return groups
+
+
+async def _optional(coro, note: str, notes: list[str]):
+    try:
+        return await coro
+    except Exception as exc:  # noqa: BLE001 - optional source: degrade, never fail the analysis
+        logger.info("rule analysis: optional source failed (%s): %s", note, exc)
+        notes.append(note)
+        return None
+
+
+_CACHE: dict[str, Context] = {}
+
+
+async def load_context(host: OpnsenseHost, *, use_cache: bool = True) -> Context:
+    key = str(host.id)
+    cached = _CACHE.get(key)
+    if use_cache and cached and time.monotonic() - cached.loaded_at < CACHE_SECONDS:
+        return cached
+    notes: list[str] = []
+    rules, info = await asyncio.gather(
+        opnsense_client.search_rules(host, all_rows=True), opnsense_client.get_interfaces_info(host))
+    stats, groups, aliases, forwards = await asyncio.gather(
+        _optional(opnsense_client.get_pf_rule_statistics(host), "PF_RULESET_UNAVAILABLE", notes),
+        _optional(opnsense_client.search_interface_groups(host), "GROUPS_UNAVAILABLE", notes),
+        _optional(opnsense_client._request(host, "GET", "/firewall/alias/searchItem", params={"rowCount": -1}),
+                  "ALIASES_UNAVAILABLE", notes),
+        _optional(opnsense_client.search_port_forward_rules(host), "NAT_UNAVAILABLE", notes),
+    )
+    interfaces = _interfaces(info)
+    pf = pf_ruleset.parse(stats)
+    if stats is not None and pf is None:
+        notes.append("PF_RULESET_UNAVAILABLE")
+    ctx = Context(
+        rows=list((rules or {}).get("rows") or []), pf=pf, interfaces=interfaces,
+        groups=_groups(groups, interfaces),
+        aliases={str(a.get("name")): a for a in ((aliases or {}).get("rows") or []) if a.get("name")},
+        port_forwards=None if forwards is None else list(forwards.get("rows") or []),
+        notes=notes, loaded_at=time.monotonic(),
+    )
+    _CACHE[key] = ctx
+    return ctx
+
+
+async def ensure_refs(host: OpnsenseHost, ctx: Context, ips: list[str]) -> None:
+    """Ask pf which tables contain each IP (cached in the context)."""
+    for ip in ips:
+        if ip in ctx.refs:
+            continue
+        try:
+            ctx.refs[ip] = await opnsense_client.find_alias_references(host, ip)
+        except Exception as exc:  # noqa: BLE001 - table membership becomes "unknown"
+            logger.info("rule analysis: findReferences failed: %s", exc)
+            ctx.refs[ip] = None
+
+
+def clear_cache() -> None:
+    _CACHE.clear()

backend/app/services/rule_analysis/coverage.py

@@ -0,0 +1,290 @@
+"""Does an earlier rule make a later one pointless? (shadowed / duplicate)
+
+Pure functions on ``NormRule``; no I/O. Used by the security check (``compliance_service``, rules taken
+from config.xml) and by the cleanup candidates (rules taken from ``searchRule``).
+
+pf evaluates all rules in order, the last matching rule wins unless a matching rule is ``quick``. So a later
+rule b can only be made dead by an earlier rule a when
+
+* a is ``quick`` (a non-quick match is overridden by later rules and decides nothing by itself),
+* a is in force (enabled, not known to be unloaded) and has no condition we cannot compare
+  (schedule, tags, ``received-on``, TCP flags, rate limits), and
+* every packet b matches is also matched by a: set coverage per field (interfaces, direction, address
+  family, protocols, source/destination, ports, ICMP types).
+
+The order is ``sort_order`` (floating 200000 before groups 300000+ before interfaces 400000, then the
+sequence); nothing is re-implemented here, ``NormRule.order`` already holds it.
+
+Every comparison answers ``"sure"`` (literal values or identical names), ``"probable"`` (the result relies
+on the static content of aliases or on a guessed order) or ``None`` (not provable, so no claim is made).
+Whatever is not proven is never "sure": the weakest link decides.
+"""
+from __future__ import annotations
+
+import ipaddress
+import re
+from dataclasses import dataclass
+
+from app.services.rule_analysis.context import Context
+from app.services.rule_analysis.model import (
+    SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID, SECTION_NAT_AUTO,
+    Endpoint, NormRule,
+)
+from app.services.rule_analysis.normalize import _af, _protos
+from app.services.rule_analysis.resolve import ADDRESS_TYPES, MAX_ALIAS_DEPTH, port_ranges
+
+SURE, PROBABLE = "sure", "probable"
+_RANK = {None: 0, PROBABLE: 1, SURE: 2}
+
+
+def weakest(*values: str | None) -> str | None:
+    """The lowest confidence of the arguments (None beats everything: one unproven part spoils the claim)."""
+    return min(values, key=lambda v: _RANK[v]) if values else SURE
+
+
+def _empty_context() -> Context:
+    return Context(rows=[], pf=None, interfaces={}, groups=None, aliases={}, port_forwards=None)
+
+
+# ── Addresses ─────────────────────────────────────────────────────────────
+
+Range = tuple[int, int, int]  # (ip version, low, high)
+
+
+def _literal(token: str) -> Range | None:
+    """CIDR, single address or ``low-high`` range as an integer interval; None for names."""
+    token = token.strip()
+    try:
+        if "-" in token and not token.startswith("-"):
+            low, _, high = token.partition("-")
+            lo, hi = ipaddress.ip_address(low.strip()), ipaddress.ip_address(high.strip())
+            return (lo.version, int(lo), int(hi)) if lo.version == hi.version and lo <= hi else None
+        net = ipaddress.ip_network(token, strict=False)
+    except ValueError:
+        return None
+    return (net.version, int(net.network_address), int(net.broadcast_address))
+
+
+def _alias_ranges(name: str, ctx: Context, depth: int = 0, seen: tuple = ()) -> list[Range] | None:
+    """Static content of a host/network alias as intervals; None when anything about it is not static
+    (tables, URLs, FQDNs, exclusions, unknown names, cycles)."""
+    alias = ctx.aliases.get(name)
+    if alias is None or name in seen or depth > MAX_ALIAS_DEPTH:
+        return None
+    if str(alias.get("type") or "").lower() not in ADDRESS_TYPES:
+        return None
+    if str(alias.get("enabled", "1")).strip().lower() in ("0", "false", "no", "off"):
+        return None  # a disabled alias is an empty table in pf: its configured content proves nothing
+    result: list[Range] = []
+    for entry in (e.strip() for e in re.split(r"[\n,]", str(alias.get("content") or "")) if e.strip()):
+        if entry.startswith("!"):
+            return None
+        literal = _literal(entry)
+        if literal is not None:
+            result.append(literal)
+            continue
+        nested = _alias_ranges(entry, ctx, depth + 1, (*seen, name)) if entry in ctx.aliases else None
+        if nested is None:
+            return None
+        result += nested
+    return result or None
+
+
+def _ranges_of(token: str, ctx: Context) -> tuple[list[Range] | None, bool]:
+    """(intervals, came_from_alias)"""
+    literal = _literal(token)
+    if literal is not None:
+        return [literal], False
+    return _alias_ranges(token, ctx), True
+
+
+def _covered(inner: list[Range], outer: list[Range]) -> bool:
+    return all(any(o[0] == i[0] and o[1] <= i[1] and i[2] <= o[2] for o in outer) for i in inner)
+
+
+def _item_covers(x: str, y: str, ctx: Context) -> str | None:
+    """Does the address item x contain everything the item y contains?"""
+    x, y = x.strip(), y.strip()
+    if x == y:
+        return SURE
+    outer, outer_alias = _ranges_of(x, ctx)
+    inner, inner_alias = _ranges_of(y, ctx)
+    if outer is None or inner is None or not _covered(inner, outer):
+        return None
+    return PROBABLE if (outer_alias or inner_alias) else SURE
+
+
+def endpoint_covers(a: Endpoint, b: Endpoint, ctx: Context) -> str | None:
+    if a.is_any and not a.negated:
+        return SURE
+    if b.is_any and not b.negated:
+        return None  # a is narrower than "any"
+    if a.negated != b.negated:
+        return None
+    if not a.negated:  # every item of b lies inside some item of a
+        parts = []
+        for y in b.items:
+            best = None
+            for x in a.items:
+                got = _item_covers(x, y, ctx)
+                if _RANK[got] > _RANK[best]:
+                    best = got
+            parts.append(best)
+        return weakest(*parts)
+    # "!A" covers "!B" when A lies inside B (outside of B is a subset of outside of A)
+    parts = []
+    for x in a.items:
+        best = None
+        for y in b.items:
+            got = _item_covers(y, x, ctx)
+            if _RANK[got] > _RANK[best]:
+                best = got
+        parts.append(best)
+    return weakest(*parts)
+
+
+# ── Ports, interfaces, the rest ───────────────────────────────────────────
+
+def _norm_ports(spec: str) -> str:
+    return re.sub(r"\s+", "", spec or "").lower()
+
+
+def ports_cover(a: str, b: str, ctx: Context) -> str | None:
+    if _norm_ports(a) in ("", "any"):
+        return SURE
+    if _norm_ports(b) in ("", "any"):
+        return None
+    if _norm_ports(a) == _norm_ports(b):
+        return SURE
+    outer, _ = port_ranges(a, ctx)
+    inner, _ = port_ranges(b, ctx)
+    if outer is None or inner is None:
+        return None
+    if not all(any(lo <= il and ih <= hi for lo, hi in outer) for il, ih in inner):
+        return None
+    alias_involved = any(part.strip() in ctx.aliases for spec in (a, b) for part in re.split(r"[\n,]", spec))
+    return PROBABLE if alias_involved else SURE
+
+
+def _expand(names: set[str], ctx: Context) -> set[str]:
+    result: set[str] = set()
+    for name in names:
+        result |= (ctx.groups[name] | {name}) if ctx.groups and name in ctx.groups else {name}
+    return result
+
+
+def interfaces_cover(a: NormRule, b: NormRule, ctx: Context) -> str | None:
+    if not a.interfaces and not a.if_not:
+        return SURE  # every interface
+    if a.if_not or b.if_not:
+        return SURE if (a.interfaces, a.if_not) == (b.interfaces, b.if_not) else None
+    if not b.interfaces:
+        return None  # b applies everywhere, a only to some interfaces
+    return SURE if _expand(b.interfaces, ctx) <= _expand(a.interfaces, ctx) else None
+
+
+def _conditions_ok(a: NormRule) -> bool:
+    """a must not depend on something we cannot compare: schedule, tags, received-on, TCP flags, limits."""
+    return not (a.sched or a.tagged or a.received_on or a.extra_notes)
+
+
+def covers_fields(a: NormRule, b: NormRule, ctx: Context | None = None) -> str | None:
+    """Every packet rule b matches is already matched by rule a (order and enabled state not considered)."""
+    ctx = ctx or _empty_context()
+    if a.quick is not True or not _conditions_ok(a):
+        return None
+    if a.direction != "any" and a.direction != b.direction:
+        return None
+    if not b.af <= a.af:
+        return None
+    if a.protos is not None and (b.protos is None or not b.protos <= a.protos):
+        return None
+    if a.icmptypes is not None and (b.icmptypes is None or not b.icmptypes <= a.icmptypes):
+        return None
+    parts = [
+        interfaces_cover(a, b, ctx),
+        endpoint_covers(a.src, b.src, ctx), endpoint_covers(a.dst, b.dst, ctx),
+        ports_cover(a.sport, b.sport, ctx), ports_cover(a.dport, b.dport, ctx),
+    ]
+    return weakest(*parts)
+
+
+# ── Order-aware analysis ──────────────────────────────────────────────────
+
+def _in_force(rule: NormRule) -> bool:
+    return rule.enabled and rule.loaded is not False and rule.section not in (SECTION_INVALID, SECTION_NAT_AUTO)
+
+
+# normalize() notes after which the evaluation order is not trustworthy: findings stay at "probable" at most.
+ORDER_DOUBT = frozenset({"ORDER_GUESSED", "ORDER_MISMATCH"})
+
+
+def order_doubtful(notes) -> bool:
+    return bool(ORDER_DOUBT & set(notes))
+
+
+@dataclass
+class Overlap:
+    rule: NormRule        # the later rule that is made pointless
+    by: NormRule          # the earlier rule that already decides everything the later one matches
+    kind: str             # "shadowed" | "duplicate"
+    confidence: str       # "sure" | "probable"
+
+
+def analyse(rules: list[NormRule], ctx: Context | None = None, *, order_guessed: bool = False) -> list[Overlap]:
+    """``rules`` in evaluation order. Each later rule is reported at most once, with the first earlier rule that
+    covers it. Duplicate = same action and the later rule also covers the earlier one."""
+    ctx = ctx or _empty_context()
+    result: list[Overlap] = []
+    for index, later in enumerate(rules):
+        if not later.enabled or later.section in (SECTION_ANTISPOOF, SECTION_INVALID, SECTION_NAT_AUTO):
+            continue
+        if later.action is None:
+            continue
+        for earlier in rules[:index]:
+            if not _in_force(earlier) or earlier.action is None:
+                continue
+            confidence = covers_fields(earlier, later, ctx)
+            if confidence is None:
+                continue
+            # (Equal sort_order without pf line numbers always comes with loaded=None, i.e. "probable" here.)
+            if order_guessed or not (earlier.order_known and later.order_known) or earlier.loaded is None:
+                confidence = weakest(confidence, PROBABLE)
+            duplicate = earlier.action == later.action and covers_fields(later, earlier, ctx) is not None
+            if duplicate:
+                confidence = weakest(confidence, covers_fields(later, earlier, ctx))
+            result.append(Overlap(later, earlier, "duplicate" if duplicate else "shadowed", confidence or PROBABLE))
+            break
+    return result
+
+
+# ── Rules read from a config.xml (security check) ─────────────────────────
+
+def config_order(rule: dict) -> tuple[int, int]:
+    """sort_order of a rule from config.xml in OPNsense's terms: floating 200000, interface group 300000,
+    single interface 400000, each by sequence."""
+    return (200000 if rule["floating"] else 300000 if rule.get("group") else 400000, rule["order"])
+
+
+def _config_endpoint(raw: str, negated: bool) -> Endpoint:
+    raw = (raw or "").strip()
+    items = [] if raw.lower() in ("", "any") else [i.strip() for i in raw.split(",") if i.strip()]
+    return Endpoint(raw=raw, negated=negated, items=items)
+
+
+def from_config_rule(rule: dict) -> NormRule:
+    """Adapter for the rule dicts of compliance_service (MVC and legacy shapes already merged). Nothing is known
+    about the loaded pf ruleset there, so ``loaded`` stays None (coverage then caps at "probable")."""
+    order = config_order(rule)
+    section = SECTION_FLOATING if rule["floating"] else SECTION_GROUP if rule.get("group") else SECTION_INTERFACE
+    return NormRule(
+        uuid="", label="", description=rule["descr"], order=order, order_known=True, section=section,
+        enabled=rule["enabled"], action=rule["action"] if rule["action"] in ("pass", "block", "reject") else None,
+        quick=bool(rule["quick"]), interfaces=set(rule["interfaces"]), if_not=bool(rule.get("if_not")),
+        direction=rule["direction"] or "in", af=_af(rule["ipprotocol"]), protos=_protos(rule["protocol"]),
+        src=_config_endpoint(rule["src"], rule["src_not"]), dst=_config_endpoint(rule["dst"], rule["dst_not"]),
+        sport=rule["sport"] or "", dport=rule["dport"] or "",
+        icmptypes={t.strip() for t in (rule.get("icmptype") or "").split(",") if t.strip()} or None,
+        tagged=rule.get("tagged") or "", received_on=rule.get("received_on") or "", sched=rule.get("sched") or "",
+        extra_notes=["RATE_LIMIT"] if rule.get("limited") else [],
+    )

backend/app/services/rule_analysis/engine.py

@@ -0,0 +1,317 @@
+"""Which rule decides a packet, following pf semantics.
+
+pf walks the loaded ruleset top to bottom; the *last* matching rule wins unless a matching
+rule is ``quick``, which ends the walk. OPNsense orders rules by ``sort_order`` (system,
+floating, groups, interfaces). The analysis looks at the first packet of a new connection.
+
+Confidence:
+- ``rule_confidence`` is "sure" only when no rule that could still change the outcome
+  matched with an unknown result;
+- ``confidence`` (the verdict) is "sure" when every such unknown rule has the same action as
+  the deciding rule (then the verdict holds either way), "unknown" otherwise;
+- notes about guesses (interface guessed from the address, NAT nearby, pf ruleset missing,
+  TCP flags, rate limits) lower a "sure" to "probable".
+"""
+from __future__ import annotations
+
+import ipaddress
+from dataclasses import dataclass, field
+
+from app.services.rule_analysis import resolve
+from app.services.rule_analysis.context import Context
+from app.services.rule_analysis.model import IPAddr, NormRule, Outcome, t_and, tri_str
+from app.services.rule_analysis.normalize import norm_proto, normalize
+
+SURE, PROBABLE, UNKNOWN = "sure", "probable", "unknown"
+_RANK = {SURE: 2, PROBABLE: 1, UNKNOWN: 0}
+DOWNGRADING_NOTES = {
+    "IFACE_GUESSED", "EGRESS_GUESSED", "NAT_NEARBY", "TCP_FLAGS", "RATE_LIMIT", "ORDER_GUESSED",
+    "ORDER_MISMATCH", "PF_RULESET_UNAVAILABLE", "NAT_UNAVAILABLE", "QUICK_UNKNOWN",
+}
+_PORT_PROTOS = {"6", "17"}
+_ICMP_PROTOS = {"1", "58"}
+ICMP_NAMES = {
+    4: {"0": "echorep", "3": "unreach", "4": "squench", "5": "redir", "8": "echoreq", "9": "routeradv",
+        "10": "routersol", "11": "timex", "12": "paramprob", "13": "timereq", "14": "timerep"},
+    6: {"1": "unreach", "2": "toobig", "3": "timex", "4": "paramprob", "128": "echoreq", "129": "echorep",
+        "133": "routersol", "134": "routeradv", "135": "neighbrsol", "136": "neighbradv", "137": "redir"},
+}
+
+
+@dataclass
+class Query:
+    src: IPAddr
+    dst: IPAddr
+    proto: str                 # normalized ("6", "17", "1", "58", other number or name)
+    proto_label: str
+    sport: int | None = None
+    dport: int | None = None
+    icmptype: str | None = None
+    iface: str | None = None   # None = guess from the source address
+    direction: str = "in"
+    egress: str | None = "none"  # "none" = only the first leg, None = guess, else identifier
+
+    @property
+    def af(self) -> int:
+        return self.src.version
+
+
+@dataclass
+class LegResult:
+    direction: str
+    iface: str | None
+    iface_source: str
+    action: str | None
+    decisive: NormRule | None
+    confidence: str
+    rule_confidence: str
+    notes: list[str] = field(default_factory=list)
+    unknowns: list[str] = field(default_factory=list)
+    steps: list[dict] = field(default_factory=list)
+
+
+def _min_conf(*values: str) -> str:
+    return min(values, key=lambda v: _RANK[v])
+
+
+def _max_conf(*values: str) -> str:
+    return max(values, key=lambda v: _RANK[v])
+
+
+def _icmp_name(q: Query) -> str | None:
+    if q.icmptype is None:
+        return None
+    return ICMP_NAMES[q.af].get(q.icmptype, q.icmptype)
+
+
+def rule_fields(r: NormRule, q: Query, direction: str, iface: str, ctx: Context,
+                tags: set[str], first_leg_iface: str | None) -> dict[str, Outcome]:
+    fields: dict[str, Outcome] = {
+        "interface": resolve.iface_match(r.interfaces, r.if_not, iface, ctx),
+        "direction": Outcome(r.direction in ("any", direction)),
+        "family": Outcome(q.af in r.af),
+        "protocol": Outcome(True if r.protos is None else q.proto in r.protos),
+        "source": resolve.endpoint_match(r.src, q.src, ctx),
+        "destination": resolve.endpoint_match(r.dst, q.dst, ctx),
+    }
+    if q.proto in _PORT_PROTOS:
+        fields["source_port"] = resolve.port_match(r.sport, q.sport, ctx, ephemeral_default=True)
+        fields["destination_port"] = resolve.port_match(r.dport, q.dport, ctx)
+    elif r.sport or r.dport:
+        fields["ports"] = Outcome(False)  # pf only generates port rules for tcp/udp
+    if r.icmptypes is not None and q.proto in _ICMP_PROTOS:
+        name = _icmp_name(q)
+        fields["icmp_type"] = Outcome(None, ["ICMP_TYPE_NOT_GIVEN"]) if name is None else \
+            Outcome(name in r.icmptypes or q.icmptype in r.icmptypes)
+    if r.tagged:
+        fields["tagged"] = Outcome(True) if r.tagged in tags else Outcome(None, ["TAGGED"])
+    if r.received_on:
+        if direction == "in":
+            fields["received_on"] = Outcome(r.received_on == iface)
+        elif first_leg_iface:
+            fields["received_on"] = Outcome(r.received_on == first_leg_iface)
+        else:
+            fields["received_on"] = Outcome(None, ["RECEIVED_ON"])
+    if r.action is None:
+        fields["action"] = Outcome(None, ["ACTION_UNKNOWN"])
+    return fields
+
+
+def _step(r: NormRule, fields: dict[str, Outcome], result, effect: str) -> dict:
+    return {
+        "rule": r.summary(), "result": tri_str(result), "effect": effect,
+        "fields": {k: tri_str(v.value) for k, v in fields.items()},
+        "reasons": sorted({reason for v in fields.values() if v.value is None for reason in v.reasons}),
+    }
+
+
+def eval_leg(rules: list[NormRule], q: Query, ctx: Context, direction: str, iface: str,
+             first_leg_iface: str | None = None) -> LegResult:
+    steps: list[dict] = []
+    candidate: NormRule | None = None
+    possible: list[tuple[NormRule, bool]] = []   # (rule, came after the current candidate)
+    tags: set[str] = set()
+    final: NormRule | None = None
+    for r in rules:
+        if not r.enabled:
+            steps.append({"rule": r.summary(), "result": "false", "effect": "disabled", "fields": {}, "reasons": []})
+            continue
+        if r.loaded is False:
+            steps.append({"rule": r.summary(), "result": "false", "effect": "not_loaded", "fields": {}, "reasons": []})
+            continue
+        fields = rule_fields(r, q, direction, iface, ctx, tags, first_leg_iface)
+        result = t_and(*(f.value for f in fields.values()))
+        if result is False:
+            steps.append(_step(r, fields, result, "no_match"))
+            continue
+        if result is None or r.quick is None:
+            possible.append((r, True))
+            steps.append(_step(r, fields, result, "possible"))
+            continue
+        if r.quick:
+            final = r
+            for s in steps:
+                if s["effect"] == "candidate":
+                    s["effect"] = "overridden"
+            steps.append(_step(r, fields, result, "final"))
+            break
+        if candidate is not None:
+            for s in steps:
+                if s["effect"] == "candidate":
+                    s["effect"] = "overridden"
+        candidate = r
+        possible = [(p, False) for p, _ in possible]
+        if r.tag:
+            tags.add(r.tag)
+        steps.append(_step(r, fields, result, "candidate"))
+    decisive = final or candidate
+    if final is None:
+        for s in steps:
+            if s["effect"] == "candidate":
+                s["effect"] = "decisive"
+    else:
+        steps[-1]["effect"] = "decisive"
+    action = decisive.action if decisive else "pass"  # pf without any matching rule passes
+    # Unknown matches that could still change the outcome: quick (or quick-unknown) ones always,
+    # non-quick ones only after the deciding candidate (an earlier one is overridden anyway).
+    relevant = [p for p, after in possible if p.quick is not False or (final is None and after)]
+    rule_conf = SURE if not relevant else PROBABLE
+    conf = SURE if all(p.action == action for p in relevant) else UNKNOWN
+    unknowns = sorted({reason for s in steps if s["effect"] == "possible" for reason in s["reasons"]})
+    if any(p.quick is None for p, _ in possible):
+        unknowns = sorted({*unknowns, "QUICK_UNKNOWN"})
+    notes: list[str] = []
+    if decisive is None:
+        notes.append("PF_DEFAULT_PASS")
+    else:
+        notes += decisive.extra_notes
+        if decisive.gateway and action == "pass":
+            notes.append("POLICY_ROUTE")
+        if decisive.statetype.lower() == "none" or "no state" in getattr(decisive.pf_line, "text", ""):
+            notes.append("STATELESS")
+        if decisive.sched:
+            notes.append("SCHEDULE_NOW")
+    if any(s["effect"] == "not_loaded" and s["rule"]["loaded"] is False for s in steps):
+        notes.append("RULES_NOT_LOADED")
+    return LegResult(direction=direction, iface=iface, iface_source="given", action=action, decisive=decisive,
+                     confidence=conf, rule_confidence=rule_conf, notes=notes, unknowns=unknowns, steps=steps)
+
+
+def guess_iface(ip: IPAddr, ctx: Context) -> str | None:
+    """The interface whose connected network holds ``ip`` (longest prefix), else the one with
+    the default gateway."""
+    best, best_len = None, -1
+    for info in ctx.interfaces.values():
+        if info.identifier == "lo0":
+            continue
+        for net in info.nets:
+            if net.version == ip.version and ip in net and net.prefixlen > best_len and not net.is_link_local:
+                best, best_len = info.identifier, net.prefixlen
+    if best:
+        return best
+    gateways = [i.identifier for i in ctx.interfaces.values() if i.has_gateway]
+    return gateways[0] if len(gateways) == 1 else None
+
+
+def _nat_nearby(q: Query, iface: str, ctx: Context) -> bool:
+    for row in ctx.port_forwards or []:
+        if str(row.get("disabled") or "0") == "1" or str(row.get("enabled", "1")) == "0":
+            continue
+        if str(row.get("nordr") or "0") == "1":  # "no rdr" exclusions (anti-lockout) redirect nothing
+            continue
+        if q.dport is not None and resolve.port_match(str(row.get("destination.port") or ""), q.dport, ctx).value is False:
+            continue
+        ifaces = {i.strip().lower() for i in str(row.get("interface") or "").split(",") if i.strip()}
+        if ifaces and iface not in ifaces:
+            continue
+        proto = str(row.get("protocol") or "").lower()
+        if proto and proto not in ("any", "tcp/udp") and norm_proto(proto) != q.proto:
+            continue
+        return True
+    return False
+
+
+def evaluate(ctx: Context, q: Query) -> dict:
+    rules, order_notes = normalize(ctx)
+    context_notes = [*ctx.notes, *order_notes]
+    legs: list[LegResult] = []
+    iface, iface_source = q.iface, "given"
+    if iface is None:
+        iface, iface_source = guess_iface(q.src if q.direction == "in" else q.dst, ctx), "guessed"
+    if iface is None:
+        return _result(q, legs, context_notes + ["IFACE_UNRESOLVED"], unresolved=True)
+    first = eval_leg(rules, q, ctx, q.direction, iface)
+    first.iface_source = iface_source
+    if iface_source == "guessed":
+        first.notes.append("IFACE_GUESSED")
+    if q.direction == "in" and (ctx.port_forwards is None or _nat_nearby(q, iface, ctx)):
+        first.notes.append("NAT_NEARBY" if ctx.port_forwards is not None else "NAT_UNAVAILABLE")
+    legs.append(first)
+    if q.direction == "in" and q.egress != "none":
+        egress, source = q.egress, "given"
+        if egress is None:
+            egress, source = guess_iface(q.dst, ctx), "guessed"
+        if egress is None:
+            context_notes.append("EGRESS_UNRESOLVED")
+        else:
+            out = eval_leg(rules, q, ctx, "out", egress, first_leg_iface=iface)
+            out.iface_source = source
+            if source == "guessed":
+                out.notes.append("EGRESS_GUESSED")
+            legs.append(out)
+    return _result(q, legs, context_notes)
+
+
+def _leg_conf(leg: LegResult, context_notes: list[str]) -> tuple[str, str]:
+    downgrade = any(n in DOWNGRADING_NOTES for n in (*leg.notes, *context_notes))
+    conf, rule_conf = leg.confidence, leg.rule_confidence
+    if downgrade:
+        conf = _min_conf(conf, PROBABLE)
+        rule_conf = _min_conf(rule_conf, PROBABLE)
+    return conf, rule_conf
+
+
+def _result(q: Query, legs: list[LegResult], context_notes: list[str], unresolved: bool = False) -> dict:
+    leg_dicts, confs = [], []
+    for leg in legs:
+        conf, rule_conf = _leg_conf(leg, context_notes)
+        confs.append((leg, conf, rule_conf))
+        leg_dicts.append({
+            "direction": leg.direction, "interface": leg.iface, "interface_source": leg.iface_source,
+            "action": leg.action, "confidence": conf, "rule_confidence": rule_conf,
+            "decisive_rule": leg.decisive.summary() if leg.decisive else None,
+            "notes": sorted(set(leg.notes)), "unknowns": leg.unknowns,
+        })
+    if unresolved or not legs:
+        action, conf, rule_conf, decisive = None, UNKNOWN, UNKNOWN, None
+    else:
+        blocking = [c for c in confs if c[0].action != "pass"]
+        if blocking:
+            leg, _, rule_conf = blocking[0]
+            action, decisive = leg.action, leg.decisive
+            # Blocked if any blocking leg is right: the best of their confidences counts.
+            conf = _max_conf(*(c[1] for c in blocking))
+            if conf == SURE and any(c[0].action != action for c in blocking):
+                conf = PROBABLE
+        else:
+            leg, _, rule_conf = confs[0]
+            action, decisive = "pass", leg.decisive
+            conf = _min_conf(*(c[1] for c in confs))
+    return {
+        "query": {
+            "src": str(q.src), "dst": str(q.dst), "proto": q.proto_label, "sport": q.sport, "dport": q.dport,
+            "icmptype": q.icmptype, "direction": q.direction,
+            "assumptions": ["FIRST_PACKET"] + (["SRC_PORT_EPHEMERAL"] if q.sport is None and q.proto in _PORT_PROTOS else []),
+        },
+        "action": action, "confidence": conf, "rule_confidence": rule_conf,
+        "decisive_rule": decisive.summary() if decisive else None,
+        "notes": sorted({*context_notes, *(n for leg in legs for n in leg.notes)}),
+        "unknowns": sorted({u for leg in legs for u in leg.unknowns}),
+        "legs": leg_dicts,
+        "trace": [{"leg": i, "direction": leg.direction, "interface": leg.iface, "steps": leg.steps}
+                  for i, leg in enumerate(legs)],
+    }
+
+
+def parse_ip(value: str) -> IPAddr:
+    return ipaddress.ip_address(value.strip())

backend/app/services/rule_analysis/model.py

@@ -0,0 +1,122 @@
+"""Shared types of the rule analysis: three-valued logic and the normalized rule.
+
+``Tri`` is Kleene logic with ``True``/``False``/``None`` (unknown). An unknown never turns
+into a definite answer by negation, and an AND is only False when one side is definitely
+False. This is what keeps the engine from claiming "sure" for something it cannot prove.
+"""
+from __future__ import annotations
+
+import ipaddress
+from dataclasses import dataclass, field
+from typing import Optional
+
+Tri = Optional[bool]
+
+IPAddr = ipaddress.IPv4Address | ipaddress.IPv6Address
+IPNet = ipaddress.IPv4Network | ipaddress.IPv6Network
+
+
+def t_and(*values: Tri) -> Tri:
+    if any(v is False for v in values):
+        return False
+    if any(v is None for v in values):
+        return None
+    return True
+
+
+def t_or(*values: Tri) -> Tri:
+    if any(v is True for v in values):
+        return True
+    if any(v is None for v in values):
+        return None
+    return False
+
+
+def t_not(value: Tri) -> Tri:
+    return None if value is None else not value
+
+
+def tri_str(value: Tri) -> str:
+    return "unknown" if value is None else ("true" if value else "false")
+
+
+class Outcome:
+    """A Tri plus the reason codes that made it unknown (or noteworthy)."""
+    __slots__ = ("value", "reasons")
+
+    def __init__(self, value: Tri, reasons: tuple[str, ...] | list[str] = ()):
+        self.value = value
+        self.reasons = list(reasons)
+
+    def __repr__(self) -> str:  # pragma: no cover - debugging aid
+        return f"Outcome({self.value!r}, {self.reasons!r})"
+
+
+def o_or(outcomes: list[Outcome]) -> Outcome:
+    value = t_or(*(o.value for o in outcomes)) if outcomes else False
+    reasons = [] if value is True else [r for o in outcomes if o.value is None for r in o.reasons]
+    return Outcome(value, reasons)
+
+
+SECTION_SYSTEM, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE = "system", "floating", "group", "interface"
+SECTION_NAT_AUTO, SECTION_INVALID, SECTION_ANTISPOOF, SECTION_OTHER = "nat_auto", "invalid", "antispoof", "other"
+
+
+@dataclass
+class Endpoint:
+    raw: str
+    negated: bool = False
+    items: list[str] = field(default_factory=list)  # empty = any
+
+    @property
+    def is_any(self) -> bool:
+        return not self.items
+
+
+@dataclass
+class NormRule:
+    uuid: str
+    label: str
+    description: str
+    order: tuple[int, int]
+    order_known: bool
+    section: str
+    enabled: bool
+    action: str | None             # pass | block | reject | None (unknown)
+    quick: bool | None             # None = unknown (automatic rule without pf ruleset)
+    interfaces: set[str]           # identifiers or group names; empty = every interface
+    if_not: bool
+    direction: str                 # in | out | any
+    af: set[int]                   # {4}, {6} or {4, 6}
+    protos: set[str] | None        # None = any; normalized names/numbers as strings
+    src: Endpoint
+    dst: Endpoint
+    sport: str
+    dport: str
+    icmptypes: set[str] | None = None   # None = no restriction
+    tag: str = ""
+    tagged: str = ""
+    received_on: str = ""
+    gateway: str = ""
+    statetype: str = ""
+    sched: str = ""
+    automatic: bool = False
+    legacy: bool = False
+    loaded: bool | None = None     # None = pf ruleset not available
+    pf_numbers: list[int] = field(default_factory=list)
+    extra_notes: list[str] = field(default_factory=list)   # e.g. TCP_FLAGS, RATE_LIMIT from pf text
+    pf_line: object = None         # set for unlabeled pf lines (antispoof)
+
+    def summary(self) -> dict:
+        return {
+            "uuid": self.uuid, "label": self.label, "description": self.description,
+            "section": self.section, "sort_order": f"{self.order[0]:06d}.{self.order[1]:07d}" if self.order_known else None,
+            "pf_numbers": self.pf_numbers, "action": self.action, "quick": self.quick,
+            "interfaces": sorted(self.interfaces), "interface_not": self.if_not, "direction": self.direction,
+            "automatic": self.automatic, "legacy": self.legacy, "loaded": self.loaded,
+            "editable": not self.automatic and not self.legacy and self.section != SECTION_ANTISPOOF,
+            "source": ("!" if self.src.negated else "") + (self.src.raw or "any"),
+            "destination": ("!" if self.dst.negated else "") + (self.dst.raw or "any"),
+            "source_port": self.sport or None, "destination_port": self.dport or None,
+            "gateway": self.gateway or None, "schedule": self.sched or None,
+        }

backend/app/services/rule_analysis/normalize.py

@@ -0,0 +1,221 @@
+"""Turn ``searchRule`` rows (+ the loaded pf ruleset) into ``NormRule`` objects.
+
+Format pitfalls seen live on OPNsense 26.7:
+- automatic rows carry the interface *description* (``"LAN"``), MVC rows the identifier;
+- automatic rows often have ``quick``/``action`` empty; the pf lines with the same label
+  know the truth (``pass in quick on vtnet1_vlan10 ...``);
+- empty source/destination means any; address lists are comma separated;
+- unmigrated legacy rules have a 32-hex label instead of a UUID.
+
+Order is ``sort_order`` (``prio.0seq`` from FilterRuleField::getPriority: 1/5 system,
+10000 force gateway, 200000 floating, 300000+n groups, 400000 interface, 500000 automatic
+NAT, 600000 = interface missing, never loaded).
+"""
+from __future__ import annotations
+
+import re
+
+from app.services.rule_analysis.context import Context
+from app.services.rule_analysis.model import (
+    SECTION_ANTISPOOF, SECTION_FLOATING, SECTION_GROUP, SECTION_INTERFACE, SECTION_INVALID,
+    SECTION_NAT_AUTO, SECTION_OTHER, SECTION_SYSTEM, Endpoint, NormRule,
+)
+from app.services.rule_analysis.pf_ruleset import PfLine
+
+PROTO_NUMBERS = {
+    "icmp": "1", "igmp": "2", "ggp": "3", "ipencap": "4", "tcp": "6", "egp": "8", "udp": "17",
+    "ipv6": "41", "gre": "47", "esp": "50", "ah": "51", "ipv6-icmp": "58", "icmp6": "58",
+    "icmpv6": "58", "ospf": "89", "pim": "103", "vrrp": "112", "carp": "112", "sctp": "132", "pfsync": "240",
+}
+
+
+def norm_proto(value: str) -> str:
+    value = str(value).strip().lower()
+    return PROTO_NUMBERS.get(value, value)
+
+
+def _truthy(value) -> bool:
+    return value is True or str(value).strip().lower() in ("1", "true", "yes", "on")
+
+
+def _section(prio: int) -> str:
+    if prio < 10001:
+        return SECTION_SYSTEM
+    if prio == 200000:
+        return SECTION_FLOATING
+    if 300000 <= prio < 400000:
+        return SECTION_GROUP
+    if prio == 400000:
+        return SECTION_INTERFACE
+    if prio == 500000:
+        return SECTION_NAT_AUTO
+    if prio >= 600000:
+        return SECTION_INVALID
+    return SECTION_OTHER
+
+
+def _order(row: dict) -> tuple[tuple[int, int], bool]:
+    raw = str(row.get("sort_order") or "")
+    if "." in raw:
+        head, _, tail = raw.partition(".")
+        if head.isdigit() and tail.isdigit():
+            return (int(head), int(tail)), True
+    # Older releases without sort_order: best effort from prio_group/sequence (confidence drops).
+    prio = str(row.get("prio_group") or row.get("#priority") or "")
+    seq = str(row.get("sequence") or "")
+    return (int(prio) if prio.isdigit() else 400000, int(seq) if seq.isdigit() else 0), False
+
+
+def _endpoint(row: dict, side: str) -> Endpoint:
+    raw = str(row.get(f"{side}_net") or "").strip()
+    items = [] if raw.lower() in ("", "any") else [i.strip() for i in raw.split(",") if i.strip()]
+    return Endpoint(raw=raw, negated=_truthy(row.get(f"{side}_not")), items=items)
+
+
+def _af(value: str) -> set[int]:
+    value = str(value or "").strip().lower()
+    return {4} if value == "inet" else ({6} if value == "inet6" else {4, 6})
+
+
+def _protos(value: str) -> set[str] | None:
+    value = str(value or "").strip().lower()
+    if value in ("", "any"):
+        return None
+    return {norm_proto(p) for p in value.split("/") if p}
+
+
+def _pf_interfaces(lines: list[PfLine], ctx: Context) -> tuple[set[str], bool] | None:
+    devices = {(line.on_device, line.on_negated) for line in lines}
+    if any(device is None for device, _ in devices):
+        return set(), False
+    if len({neg for _, neg in devices}) != 1:
+        return None
+    names: set[str] = set()
+    for device, _ in devices:
+        info = ctx.by_device(device)
+        if info is not None:
+            names.add(info.identifier)
+        elif ctx.groups is not None and device in ctx.groups:
+            names.add(device)
+        else:
+            return None
+    return names, next(iter(devices))[1]
+
+
+def normalize_row(row: dict, ctx: Context) -> NormRule:
+    order, order_known = _order(row)
+    label = str(row.get("uuid") or "")
+    automatic = _truthy(row.get("is_automatic"))
+    lines = ctx.pf.by_label.get(label, []) if ctx.pf else []
+    rule = NormRule(
+        uuid=label, label=label, description=str(row.get("description") or ""),
+        order=order, order_known=order_known, section=_section(order[0]),
+        enabled=_truthy(row.get("enabled", "1")),
+        action=(str(row.get("action") or "").lower() or None),
+        quick=None, interfaces=set(), if_not=_truthy(row.get("interfacenot")),
+        direction=(str(row.get("direction") or "in").lower() or "in"),
+        af=_af(row.get("ipprotocol")), protos=_protos(row.get("protocol")),
+        src=_endpoint(row, "source"), dst=_endpoint(row, "destination"),
+        sport=str(row.get("source_port") or "").strip(), dport=str(row.get("destination_port") or "").strip(),
+        tag=str(row.get("tag") or ""), tagged=str(row.get("tagged") or ""),
+        received_on=str(row.get("received-on") or row.get("received_on") or ""),
+        gateway=str(row.get("gateway") or ""), statetype=str(row.get("statetype") or ""),
+        sched=str(row.get("sched") or ""), automatic=automatic, legacy=_truthy(row.get("legacy")),
+        pf_numbers=[line.number for line in lines],
+    )
+    if rule.action not in ("pass", "block", "reject"):
+        rule.action = None
+    if row.get("quick") not in (None, ""):
+        rule.quick = _truthy(row.get("quick"))
+    raw_if = str(row.get("interface") or "").strip()
+    names = {n.strip() for n in raw_if.split(",") if n.strip()}
+    # Automatic and unmigrated legacy rows carry descriptions ("LAN"): map them to identifiers.
+    # Identifiers and group names stay as they are; the pf "on" clause decides when available.
+    mapped = set()
+    for name in names:
+        info = None if name in ctx.interfaces or name in (ctx.groups or {}) else ctx.by_label(name)
+        mapped.add(info.identifier if info else name)
+    names = mapped
+    rule.interfaces = names
+    if ctx.pf is not None:
+        rule.loaded = bool(lines) and rule.enabled
+        if lines:
+            actions = {line.action for line in lines}
+            if len(actions) == 1 and next(iter(actions)) in ("pass", "block", "reject"):
+                pf_action = next(iter(actions))
+                if rule.action is None or automatic:
+                    rule.action = pf_action
+            quick = {line.quick for line in lines}
+            rule.quick = next(iter(quick)) if len(quick) == 1 else None
+            if automatic or rule.legacy:
+                on = _pf_interfaces(lines, ctx)
+                if on is not None:
+                    rule.interfaces, rule.if_not = on
+                directions = {line.direction for line in lines}
+                if len(directions) == 1:
+                    rule.direction = next(iter(directions))
+            families = {line.af for line in lines}
+            if None not in families:
+                rule.af &= families
+            types = {t for line in lines for t in line.icmptypes}
+            if types:
+                rule.icmptypes = types
+            if any(line.flags not in (None, "S/SA", "any") for line in lines):
+                rule.extra_notes.append("TCP_FLAGS")
+            if any(word in line.text for line in lines for word in ("max-src-", "overload", "max-pkt-rate")):
+                rule.extra_notes.append("RATE_LIMIT")
+    elif rule.section == SECTION_INVALID:
+        rule.loaded = False
+    if rule.icmptypes is None and str(row.get("icmptype") or "").strip():
+        rule.icmptypes = {t.strip() for t in str(row["icmptype"]).split(",") if t.strip()}
+    return rule
+
+
+# Words in an unlabeled pf line that restrict it in ways antispoof_rule() does not model (protocol, ports, flags,
+# tags, user/group, received-on, ICMP types, OS): such a line must never be used as a covering rule.
+_UNMODELLED = re.compile(r"\b(proto|port|flags|tagged|received-on|user|group|icmp6?-type|os|tag|label)\b")
+
+
+def antispoof_rule(line: PfLine, order: tuple[int, int]) -> NormRule:
+    """An unlabeled pf line OPNsense generated itself (antispoof / own-address block)."""
+    return NormRule(
+        uuid=f"pf@{line.number}", label="", description=line.text, order=order, order_known=True,
+        section=SECTION_ANTISPOOF, enabled=True, action=line.action if line.action != "other" else None,
+        quick=line.quick, interfaces=set(), if_not=False, direction=line.direction,
+        af={line.af} if line.af else {4, 6}, protos=None,
+        src=Endpoint(raw="" if line.src == "any" else line.src, items=[] if line.src == "any" else [line.src]),
+        dst=Endpoint(raw="" if line.dst == "any" else line.dst, items=[] if line.dst == "any" else [line.dst]),
+        sport="", dport="", automatic=True, loaded=True, pf_numbers=[line.number], pf_line=line,
+        extra_notes=["UNMODELLED_CONDITION"] if _UNMODELLED.search(line.text) else [],
+    )
+
+
+def normalize(ctx: Context) -> tuple[list[NormRule], list[str]]:
+    """All rules in evaluation order plus context notes (order problems)."""
+    notes: list[str] = []
+    rules = [normalize_row(row, ctx) for row in ctx.rows]
+    if any(not r.order_known for r in rules):
+        notes.append("ORDER_GUESSED")
+    # Equal sort_order (e.g. two rules with the same sequence): what pf actually loaded decides, the uuid only as
+    # a last resort (then coverage.analyse() lowers its confidence, it cannot know the real order).
+    rules.sort(key=lambda r: (r.order, min(r.pf_numbers) if r.pf_numbers else 10**9, r.uuid))
+    if ctx.pf is not None:
+        loaded = [r for r in rules if r.loaded and r.pf_numbers]
+        firsts = [min(r.pf_numbers) for r in loaded]
+        if firsts != sorted(firsts):
+            notes.append("ORDER_MISMATCH")
+        # Unlabeled pf lines: placed before the first configured rule that pf evaluates later.
+        extra: list[tuple[tuple, NormRule]] = []
+        for line in ctx.pf.unlabeled:
+            after = next((r for r in loaded if min(r.pf_numbers) > line.number), None)
+            key = after.order if after else (999999, 9999999)
+            spoof = antispoof_rule(line, key)
+            if line.on_device:
+                info = ctx.by_device(line.on_device)
+                spoof.interfaces = {info.identifier if info else line.on_device}
+                spoof.if_not = line.on_negated
+            extra.append(((key, 0, line.number), spoof))
+        keyed = [((r.order, 1, 0), r) for r in rules] + extra
+        keyed.sort(key=lambda item: item[0])
+        rules = [r for _, r in keyed]
+    return rules, notes

backend/app/services/rule_analysis/pf_ruleset.py

@@ -0,0 +1,93 @@
+"""The ruleset pf has actually loaded (``diagnostics/firewall/pfStatistics/rules``).
+
+Each key looks like ``@76 pass in quick on vtnet1_vlan10 inet from (...) to any flags S/SA
+keep state label "<uuid>"``. One configured rule can expand to several pf lines (address
+families, ICMP types, port lists); they share one label. Lines without a label are generated
+by OPNsense itself (antispoof: ``block drop in log on ! vtnet1_vlan10 inet from 192.0.2.0/24
+to any``). Only what the analysis needs is parsed: position, action, quick, direction, the
+``on`` interface, address family, ICMP types and a few pf options that make the answer
+uncertain.
+"""
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field
+
+_HEAD = re.compile(r"^@(\d+)\s+(.*)$")
+_LABEL = re.compile(r'label "([^"]+)"')
+_ON = re.compile(r"\bon (!\s*)?([A-Za-z0-9_.]+)")
+_ICMP = re.compile(r"\bicmp6?-type (\S+)")
+_FROM = re.compile(r"\bfrom (\S+) to (\S+)")
+_FLAGS = re.compile(r"\bflags (\S+)")
+
+
+@dataclass
+class PfLine:
+    number: int
+    text: str
+    action: str           # pass | block | reject | match | scrub | other
+    quick: bool
+    direction: str        # in | out | any
+    on_negated: bool
+    on_device: str | None
+    af: int | None        # 4, 6 or None (both)
+    label: str | None
+    icmptypes: list[str] = field(default_factory=list)
+    src: str = "any"
+    dst: str = "any"
+    flags: str | None = None
+
+
+def parse_line(number: int, text: str) -> PfLine:
+    words = text.split()
+    first = words[0] if words else ""
+    action = first if first in ("pass", "match", "scrub") else "other"
+    if first == "block":
+        action = "reject" if len(words) > 1 and words[1].startswith("return") else "block"
+    direction = "in" if " in " in f" {text} " else ("out" if " out " in f" {text} " else "any")
+    on = _ON.search(text)
+    af = 4 if re.search(r"\binet\b", text) else (6 if re.search(r"\binet6\b", text) else None)
+    label = _LABEL.search(text)
+    endpoints = _FROM.search(text)
+    flags = _FLAGS.search(text)
+    return PfLine(
+        number=number, text=text, action=action, quick=" quick " in f" {text} ",
+        direction=direction, on_negated=bool(on and on.group(1)), on_device=on.group(2) if on else None,
+        af=af, label=label.group(1) if label else None, icmptypes=_ICMP.findall(text),
+        src=endpoints.group(1) if endpoints else ("any" if " all" in f" {text}" else "?"),
+        dst=endpoints.group(2) if endpoints else ("any" if " all" in f" {text}" else "?"),
+        flags=flags.group(1) if flags else None,
+    )
+
+
+@dataclass
+class PfRuleset:
+    lines: list[PfLine]
+    by_label: dict[str, list[PfLine]]
+
+    @property
+    def unlabeled(self) -> list[PfLine]:
+        return [line for line in self.lines if line.label is None and line.action in ("pass", "block", "reject")]
+
+
+def parse(stats: dict | None) -> PfRuleset | None:
+    """None when the answer does not look like a pf ruleset at all."""
+    rules = (stats or {}).get("rules") if isinstance(stats, dict) else None
+    filter_rules = rules.get("filter rules") if isinstance(rules, dict) else None
+    if not isinstance(filter_rules, dict):
+        return None
+    lines: list[PfLine] = []
+    for key in filter_rules:
+        head = _HEAD.match(str(key).strip())
+        if not head:
+            continue
+        line = parse_line(int(head.group(1)), head.group(2))
+        if line.action == "scrub":
+            continue
+        lines.append(line)
+    lines.sort(key=lambda line: line.number)
+    by_label: dict[str, list[PfLine]] = {}
+    for line in lines:
+        if line.label:
+            by_label.setdefault(line.label, []).append(line)
+    return PfRuleset(lines=lines, by_label=by_label)

backend/app/services/rule_analysis/resolve.py

@@ -0,0 +1,211 @@
+"""Does a rule field match the queried packet? Addresses, interfaces, aliases, ports.
+
+Every function answers with an ``Outcome`` (True/False/None + reason codes). Whatever cannot
+be proven from the firewall's own data is ``None`` (unknown), never a guess.
+"""
+from __future__ import annotations
+
+import ipaddress
+import re
+
+from app.services.rule_analysis.context import Context
+from app.services.rule_analysis.model import IPAddr, Outcome, o_or, t_not
+
+MAX_ALIAS_DEPTH = 10
+TABLE_ONLY_TYPES = {"url", "urltable", "geoip", "asn", "mac", "dynipv6host", "authgroup", "external", "internal"}
+ADDRESS_TYPES = {"host", "network", "networkgroup"}
+
+SERVICES = {
+    "ftp-data": 20, "ftp": 21, "ssh": 22, "telnet": 23, "smtp": 25, "domain": 53, "dns": 53, "bootps": 67,
+    "bootpc": 68, "tftp": 69, "http": 80, "www": 80, "kerberos": 88, "pop3": 110, "ntp": 123, "imap": 143,
+    "snmp": 161, "snmptrap": 162, "ldap": 389, "https": 443, "smb": 445, "microsoft-ds": 445, "isakmp": 500,
+    "syslog": 514, "submission": 587, "ldaps": 636, "imaps": 993, "pop3s": 995, "openvpn": 1194,
+    "ms-sql-s": 1433, "radius": 1812, "radius-acct": 1813, "nfs": 2049, "mysql": 3306, "rdp": 3389,
+    "ipsec-nat-t": 4500, "sip": 5060, "postgresql": 5432, "dhcpv6-client": 546, "dhcpv6-server": 547,
+    "http-alt": 8080, "wireguard": 51820,
+}
+EPHEMERAL = (1024, 65535)
+
+
+def _ip_in(token: str, ip: IPAddr) -> bool | None:
+    """Literal IP / CIDR / range; None when the token is not a literal."""
+    token = token.strip()
+    if "-" in token and not token.startswith("-"):
+        low, _, high = token.partition("-")
+        try:
+            low_ip, high_ip = ipaddress.ip_address(low.strip()), ipaddress.ip_address(high.strip())
+        except ValueError:
+            return None
+        return low_ip.version == ip.version and low_ip <= ip <= high_ip
+    try:
+        net = ipaddress.ip_network(token, strict=False)
+    except ValueError:
+        return None
+    return net.version == ip.version and ip in net
+
+
+def _interface_ref(token: str, ip: IPAddr, ctx: Context) -> Outcome | None:
+    name = token.strip()
+    lower = name.lower()
+    m = re.fullmatch(r"__(.+)_network", lower)
+    if m and m.group(1) in ctx.interfaces:
+        lower = m.group(1)
+    info = ctx.interfaces.get(lower)
+    if info is not None:  # "lan" = the interface's networks (pf: (dev:network), incl. VIP subnets)
+        return Outcome(any(ip in net for net in info.nets))
+    if lower.endswith("ip") and lower[:-2] in ctx.interfaces:  # "lanip" = the interface address
+        info = ctx.interfaces[lower[:-2]]
+        if ip in info.addrs:
+            return Outcome(True)
+        if ip in info.vips:
+            return Outcome(None, ["VIP_AS_INTERFACE_ADDRESS"])
+        return Outcome(False)
+    return None
+
+
+def addr_item(token: str, ip: IPAddr, ctx: Context, *, depth: int = 0, seen: tuple = ()) -> Outcome:
+    token = token.strip()
+    if token.lower() in ("any", ""):
+        return Outcome(True)
+    if token.lower() in ("(self)", "self", "this firewall"):
+        return Outcome(ip in ctx.self_addrs)
+    literal = _ip_in(token, ip)
+    if literal is not None:
+        return Outcome(literal)
+    ref = _interface_ref(token, ip, ctx)
+    if ref is not None:
+        return ref
+    return alias_contains(token, ip, ctx, depth=depth, seen=seen)
+
+
+def alias_contains(name: str, ip: IPAddr, ctx: Context, *, depth: int = 0, seen: tuple = ()) -> Outcome:
+    if name in seen:
+        return Outcome(None, ["ALIAS_CYCLE"])
+    if depth > MAX_ALIAS_DEPTH:
+        return Outcome(None, ["ALIAS_TOO_DEEP"])
+    alias = ctx.aliases.get(name)
+    refs = ctx.refs.get(str(ip))
+    if alias is None:
+        # pf knows tables the alias list may not show (e.g. internal ones): trust a positive hit.
+        if refs is not None and name in refs:
+            return Outcome(True)
+        return Outcome(None, ["ALIAS_MISSING"])
+    kind = str(alias.get("type") or "").lower()
+    if depth == 0 and refs is not None:
+        # pf's own table test: authoritative for every alias type (nesting, "!" entries, FQDNs).
+        return Outcome(name in refs)
+    if kind in ADDRESS_TYPES:
+        return _alias_content(alias, ip, ctx, depth, (*seen, name))
+    if refs is not None:
+        return Outcome(name in refs)
+    return Outcome(None, ["ALIAS_TABLE_UNAVAILABLE"])
+
+
+def _alias_content(alias: dict, ip: IPAddr, ctx: Context, depth: int, seen: tuple) -> Outcome:
+    entries = [e.strip() for e in re.split(r"[\n,]", str(alias.get("content") or "")) if e.strip()]
+    if any(e.startswith("!") for e in entries):
+        return Outcome(None, ["ALIAS_EXCLUSION"])
+    results = []
+    for entry in entries:
+        literal = _ip_in(entry, ip)
+        if literal is not None:
+            results.append(Outcome(literal))
+        elif entry in ctx.aliases or entry in seen:
+            results.append(alias_contains(entry, ip, ctx, depth=depth + 1, seen=seen))
+        elif "." in entry or ":" in entry:
+            results.append(Outcome(None, ["FQDN_UNRESOLVED"]))
+        else:
+            results.append(Outcome(None, ["ALIAS_MISSING"]))
+    return o_or(results)
+
+
+def endpoint_match(endpoint, ip: IPAddr, ctx: Context) -> Outcome:
+    if endpoint.is_any:
+        return Outcome(True)
+    result = o_or([addr_item(item, ip, ctx) for item in endpoint.items])
+    if endpoint.negated:
+        return Outcome(t_not(result.value), result.reasons)
+    return result
+
+
+# ── Ports ─────────────────────────────────────────────────────────────────
+
+def port_ranges(spec: str, ctx: Context, *, depth: int = 0, seen: tuple = ()) -> tuple[list[tuple[int, int]] | None, list[str]]:
+    """The port ranges a spec stands for; (None, reasons) when that cannot be known."""
+    ranges: list[tuple[int, int]] = []
+    reasons: list[str] = []
+    for part in [p.strip() for p in re.split(r"[\n,]", spec) if p.strip()]:
+        m = re.fullmatch(r"(\w[\w-]*?)\s*[-:]\s*(\w[\w-]*)", part) if not part.isdigit() and part not in ctx.aliases else None
+        if part.isdigit():
+            ranges.append((int(part), int(part)))
+        elif m and _port_number(m.group(1)) is not None and _port_number(m.group(2)) is not None:
+            ranges.append((_port_number(m.group(1)), _port_number(m.group(2))))
+        elif part.lower() in SERVICES:
+            ranges.append((SERVICES[part.lower()],) * 2)
+        elif part in ctx.aliases:
+            if part in seen or depth >= MAX_ALIAS_DEPTH:
+                return None, ["ALIAS_CYCLE" if part in seen else "ALIAS_TOO_DEEP"]
+            alias = ctx.aliases[part]
+            if str(alias.get("type") or "").lower() != "port":
+                return None, ["PORT_ALIAS_UNRESOLVED"]
+            sub, why = port_ranges(str(alias.get("content") or ""), ctx, depth=depth + 1, seen=(*seen, part))
+            if sub is None:
+                return None, why
+            ranges += sub
+        else:
+            return None, ["PORT_ALIAS_UNRESOLVED" if part.isidentifier() else "PORT_UNPARSEABLE"]
+    return ranges, reasons
+
+
+def _port_number(token: str) -> int | None:
+    if token.isdigit():
+        return int(token)
+    return SERVICES.get(token.lower())
+
+
+def port_match(spec: str, port: int | None, ctx: Context, *, ephemeral_default: bool = False) -> Outcome:
+    spec = (spec or "").strip()
+    if spec.lower() in ("", "any"):
+        return Outcome(True)
+    ranges, reasons = port_ranges(spec, ctx)
+    if ranges is None:
+        return Outcome(None, reasons)
+    if port is not None:
+        return Outcome(any(low <= port <= high for low, high in ranges))
+    if not ephemeral_default:
+        return Outcome(None, ["PORT_NOT_GIVEN"])
+    # No source port given: the query stands for an ephemeral client port (1024-65535).
+    low_e, high_e = EPHEMERAL
+    overlap = [(max(low, low_e), min(high, high_e)) for low, high in ranges if high >= low_e and low <= high_e]
+    if not overlap:
+        return Outcome(False)
+    covered, cursor = True, low_e
+    for low, high in sorted(overlap):
+        if low > cursor:
+            covered = False
+            break
+        cursor = max(cursor, high + 1)
+    if covered and cursor > high_e:
+        return Outcome(True)
+    return Outcome(None, ["SRC_PORT_NOT_GIVEN"])
+
+
+# ── Interfaces ────────────────────────────────────────────────────────────
+
+def iface_match(interfaces: set[str], if_not: bool, iface: str, ctx: Context) -> Outcome:
+    if not interfaces:
+        return Outcome(True)
+    results = []
+    for name in interfaces:
+        if name == iface:
+            results.append(Outcome(True))
+        elif name in ctx.interfaces:
+            results.append(Outcome(False))
+        elif ctx.groups is not None and name in ctx.groups:
+            results.append(Outcome(iface in ctx.groups[name]))
+        else:
+            results.append(Outcome(None, ["IFACE_UNRESOLVED" if ctx.groups is not None else "GROUPS_UNAVAILABLE"]))
+    result = o_or(results)
+    if if_not:
+        return Outcome(t_not(result.value), result.reasons)
+    return result

backend/app/services/rule_analysis/service.py

@@ -0,0 +1,132 @@
+"""Entry point used by the API: validate a query, load the firewall's context, evaluate."""
+from __future__ import annotations
+
+import ipaddress
+import logging
+from datetime import datetime, timezone
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.errors import ErrorCode, api_error
+from app.models.host import OpnsenseHost
+from app.models.policy_hierarchy_rule import PolicyHierarchyRule
+from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
+from app.models.rule import FirewallRule
+from app.models.rule_deployment import RuleDeployment
+from app.models.rule_hit import RuleHit
+from app.models.rule_history import LiveRuleState
+from app.services import rule_hit_service
+from app.services.rule_history_service import BASELINE_KEY
+from app.services.rule_analysis import cleanup, coverage
+from app.services.rule_analysis import context as ctx_mod
+from app.services.rule_analysis.engine import Query, evaluate
+from app.services.rule_analysis.normalize import normalize
+from app.services.rule_analysis.normalize import PROTO_NUMBERS, norm_proto
+
+logger = logging.getLogger(__name__)
+
+
+def _ip(value: str | None, field: str):
+    try:
+        return ipaddress.ip_address(str(value or "").strip())
+    except ValueError:
+        raise api_error(422, ErrorCode.ANALYSE_ADRESSE_UNGUELTIG, field=field, value=str(value or "")[:64]) from None
+
+
+def _port(value: str | int | None, field: str) -> int | None:
+    if value in (None, ""):
+        return None
+    text = str(value).strip()
+    if not text.isdigit() or int(text) > 65535:
+        raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field=field, value=text[:32])
+    return int(text)
+
+
+def build_query(*, src: str, dst: str, proto: str = "tcp", sport=None, dport=None, icmptype: str | None = None,
+                iface: str | None = None, direction: str = "in", egress: str | None = None) -> Query:
+    source, destination = _ip(src, "src"), _ip(dst, "dst")
+    if source.version != destination.version:
+        raise api_error(422, ErrorCode.ANALYSE_FAMILIEN_MISCHUNG)
+    label = str(proto or "tcp").strip().lower()
+    if label == "icmp" and source.version == 6:
+        label = "ipv6-icmp"
+    number = norm_proto(label)
+    if not (number.isdigit() and int(number) <= 255) and label not in PROTO_NUMBERS:
+        raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="proto", value=label[:32])
+    q = Query(src=source, dst=destination, proto=number, proto_label=label,
+              sport=_port(sport, "sport"), dport=_port(dport, "dport"))
+    if (q.sport is not None or q.dport is not None) and number not in ("6", "17"):
+        raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="dport", value=str(dport or sport))
+    if icmptype not in (None, ""):
+        if number not in ("1", "58"):
+            raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="icmptype", value=str(icmptype)[:32])
+        q.icmptype = str(icmptype).strip().lower()
+    if direction not in ("in", "out"):
+        raise api_error(422, ErrorCode.ANALYSE_PARAMETER_UNGUELTIG, field="direction", value=str(direction)[:8])
+    q.direction = direction
+    q.iface = None if str(iface or "").strip().lower() in ("", "auto") else str(iface).strip()
+    egress_value = str(egress or "").strip().lower()
+    q.egress = "none" if egress_value in ("", "none") else (None if egress_value == "auto" else str(egress).strip())
+    return q
+
+
+def check_interfaces(q: Query, ctx: ctx_mod.Context) -> None:
+    for name in (q.iface, None if q.egress in (None, "none") else q.egress):
+        if name is not None and name not in ctx.interfaces:
+            raise api_error(422, ErrorCode.ANALYSE_INTERFACE_UNBEKANNT, interface=name[:64],
+                            known=", ".join(sorted(i for i in ctx.interfaces if i != "lo0")))
+
+
+async def analyze_host(host: OpnsenseHost, q: Query, *, refresh: bool = False) -> dict:
+    """Raises api_error for a bad interface; OpnsenseAPIError/httpx errors when the firewall
+    cannot be read (callers map that)."""
+    ctx = await ctx_mod.load_context(host, use_cache=not refresh)
+    check_interfaces(q, ctx)
+    await ctx_mod.ensure_refs(host, ctx, [str(q.src), str(q.dst)])
+    result = evaluate(ctx, q)
+    if any(ctx.refs.get(ip) is None for ip in (str(q.src), str(q.dst))):
+        result["notes"] = sorted({*result["notes"], "ALIAS_REFERENCES_UNAVAILABLE"})
+    return result
+
+
+async def _managed_rules(db: AsyncSession, host: OpnsenseHost) -> dict[str, dict]:
+    """{label on the firewall: what manages it} for every rule OPNexus deployed to this host."""
+    managed: dict[str, dict] = {}
+    for dep_uuid, rule_id, description in (await db.execute(
+            select(RuleDeployment.opnsense_rule_uuid, FirewallRule.id, FirewallRule.description)
+            .join(FirewallRule, FirewallRule.id == RuleDeployment.rule_id)
+            .where(RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid.is_not(None)))).all():
+        managed[dep_uuid.lower()] = {"kind": "rule", "id": str(rule_id), "name": description}
+    for dep_uuid, rule_id, description in (await db.execute(
+            select(PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid, PolicyHierarchyRule.id, PolicyHierarchyRule.description)
+            .join(PolicyHierarchyRule, PolicyHierarchyRule.id == PolicyHierarchyRuleDeployment.policy_hierarchy_id)
+            .where(PolicyHierarchyRuleDeployment.host_id == host.id,
+                   PolicyHierarchyRuleDeployment.opnsense_policy_hierarchy_uuid.is_not(None)))).all():
+        managed[dep_uuid.lower()] = {"kind": "policy_hierarchy", "id": str(rule_id), "name": description}
+    return managed
+
+
+async def cleanup_candidates(db: AsyncSession, host: OpnsenseHost, *, unused_days: int = cleanup.DEFAULT_UNUSED_DAYS,
+                             refresh: bool = False, now: datetime | None = None) -> dict:
+    """Read-only. Raises when the firewall cannot be read (callers map that)."""
+    now = now or datetime.now(timezone.utc)
+    ctx = await ctx_mod.load_context(host, use_cache=not refresh)
+    rules, notes = normalize(ctx)
+    hits = {r.rule_uuid.lower(): r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host.id))).all()}
+    states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
+    rows = cleanup.build(rules, ctx, hits=hits, states=states, managed=await _managed_rules(db, host), now=now,
+                         unused_days=unused_days, order_guessed=coverage.order_doubtful(notes))
+    all_notes = sorted({*notes, *ctx.notes})
+    return {
+        "host_id": str(host.id), "host_name": host.name, "cluster_name": host.cluster_name,
+        "unused_days": max(cleanup.MIN_DAYS, min(cleanup.MAX_DAYS, int(unused_days))), "generated_at": now,
+        "rules_total": sum(1 for r in rules if not r.automatic and r.section not in ("antispoof", "nat_auto", "invalid")),
+        "candidates": rows, "counts": cleanup.summarize(rows), "notes": all_notes,
+        "usage": {
+            "tracked": len(hits),
+            "collecting": sum(1 for h in hits.values() if rule_hit_service.classify(h, now, unused_days) == "collecting"),
+            "watching_since": min((h.first_seen_at for h in hits.values()), default=None),
+        },
+        "history": {"watching_since": baseline.first_seen_at if (baseline := states.get(BASELINE_KEY)) else None},
+    }

backend/app/services/rule_history_service.py

@@ -0,0 +1,325 @@
+"""Change history of the rules that are live on a firewall, also those OPNexus did not create.
+
+Every ``rule_observe`` cycle reads ``searchRule`` (and the counters, for the usage sampling) of each enabled
+firewall and compares the user rules with the last observed state (``live_rule_states``). A difference becomes
+one ``rule_change_events`` row: added, changed, removed, enabled or disabled, with the rule as it was and as it
+is. The first observation of a firewall only records the starting point (no flood of "added" events).
+
+Origin: a change is ``opnexus`` when the audit trail has a deployment event for that rule UUID on that host in
+the time since the previous observation (the actor is taken from it), else ``external`` (changed in the
+firewall's own GUI, by a script, ...). The audit entry is written a moment after the change reaches the firewall,
+so an event classified ``external`` is re-checked in the next cycles (``reconcile_origins``).
+
+Two processes may run the loop at once (overlapping web updates, a replica): every change is claimed with an
+optimistic ``version`` on the state row and the event carries that version in a unique key, so a change is
+recorded exactly once.
+
+Only the rows' own fields are kept; derived display fields (``%…``, ``alias_meta_*``, colours, priorities) are
+dropped so that a changed alias or colour does not look like a rule change.
+"""
+import hashlib
+import json
+import logging
+import uuid as uuid_lib
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy import delete, select, update
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.audit import AuditEvent
+from app.models.host import OpnsenseHost
+from app.models.policy import RuleRevision
+from app.models.rule_deployment import RuleDeployment
+from app.models.rule_history import LiveRuleState, RuleChangeEvent
+from app.services import opnsense_client as client
+from app.services import rule_hit_service
+from app.services.opnsense_client import OpnsenseAPIError
+
+logger = logging.getLogger(__name__)
+
+BASELINE_KEY = "__baseline__"
+RETENTION_DAYS = 365
+# Audit kinds whose events carry the UUID of a rule on a firewall in details["remote_uuid"].
+AUDIT_KINDS = ("rule", "policy_hierarchy")
+# Audit statuses that mean the firewall's rule was (or was meant to be) touched.
+AUDIT_STATUSES_NO_CHANGE = ("requested", "interrupted", "error")
+# The audit entry is written right after the firewall changed; allow for the observer running just before it.
+AUDIT_SLACK = timedelta(minutes=2)
+RECHECK_WINDOW = timedelta(minutes=45)
+_DROP_PREFIXES = ("%", "alias_meta", "#")
+_DROP_KEYS = {"category_colors", "categories_colors", "icons", "ref"}
+
+
+def clean_row(row: dict) -> dict:
+    """The part of a ``searchRule`` row that describes the rule itself."""
+    return {k: v for k, v in row.items() if not k.startswith(_DROP_PREFIXES) and k not in _DROP_KEYS}
+
+
+def fingerprint(row: dict) -> str:
+    return hashlib.sha256(json.dumps(row, sort_keys=True, separators=(",", ":"), default=str).encode()).hexdigest()
+
+
+def _enabled(row: dict) -> bool:
+    return str(row.get("enabled", "1")).strip().lower() in ("1", "true", "yes", "on")
+
+
+# Fields that only say where the rule sits in the order. Another rule being inserted or a group being re-sorted
+# changes them for many rules at once; that is a "moved", not an edit of the rule.
+ORDER_FIELDS = frozenset({"sort_order", "prio_group", "sequence"})
+# Kinds after which what OPNexus learned about the rule's traffic no longer describes it.
+RESTARTS_USAGE = frozenset({"added", "changed", "enabled", "disabled"})
+
+
+def change_kind(before: dict | None, after: dict | None) -> str:
+    if before is None:
+        return "added"
+    if after is None:
+        return "removed"
+    differing = {k for k in {*before, *after} if before.get(k) != after.get(k)}
+    if differing == {"enabled"}:
+        return "enabled" if _enabled(after) else "disabled"
+    if differing and differing <= ORDER_FIELDS:
+        return "moved"
+    return "changed"
+
+
+def diff_fields(before: dict | None, after: dict | None, limit: int = 12) -> list[dict]:
+    """Short field-by-field difference: [{field, before, after}] (a missing side is None)."""
+    before, after = before or {}, after or {}
+    changes = [{"field": key, "before": before.get(key), "after": after.get(key)}
+               for key in sorted({*before, *after}) if before.get(key) != after.get(key)]
+    return changes[:limit]
+
+
+def _tracked(search: dict) -> dict[str, dict] | None:
+    """{rule label: cleaned row} of the user rules; None when the answer is not a rule list at all (an error
+    page must never look like 'every rule was removed')."""
+    rows = search.get("rows") if isinstance(search, dict) else None
+    if not isinstance(rows, list):
+        return None
+    result: dict[str, dict] = {}
+    for row in rows:
+        if not isinstance(row, dict) or not row.get("uuid") or row.get("is_automatic") in (True, 1, "1"):
+            continue
+        result[str(row["uuid"]).lower()[:64]] = clean_row(row)
+    return result
+
+
+async def _find_audit(db: AsyncSession, host_id, rule_uuid: str, start: datetime, end: datetime) -> AuditEvent | None:
+    """The newest deployment audit entry of this rule after ``start`` that no other event has claimed yet. The
+    audit entry is written after the firewall changed, so it can only lie after the previous observation."""
+    claimed = select(RuleChangeEvent.audit_id).where(RuleChangeEvent.audit_id.is_not(None))
+    return await db.scalar(
+        select(AuditEvent).where(
+            AuditEvent.host_id == str(host_id), AuditEvent.kind.in_(AUDIT_KINDS),
+            AuditEvent.created_at >= start, AuditEvent.created_at <= end + AUDIT_SLACK,
+            AuditEvent.id.notin_(claimed),
+            AuditEvent.status.notin_(AUDIT_STATUSES_NO_CHANGE),
+            AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
+        ).order_by(AuditEvent.created_at.desc()).limit(1))
+
+
+async def observe_rows(db: AsyncSession, host: OpnsenseHost, search: dict, now: datetime | None = None) -> dict:
+    """Compare a ``searchRule`` answer with the stored state of the host. Returns {"baseline": bool, "events": n}."""
+    now = now or datetime.now(timezone.utc)
+    current = _tracked(search)
+    if current is None or not rule_hit_service.looks_complete(search):
+        # Not a full rule list (no automatic rules in it): never read an empty or partial answer as "all removed".
+        return {"baseline": False, "events": 0, "skipped": True, "touched": []}
+    states = {s.rule_uuid: s for s in (await db.scalars(select(LiveRuleState).where(LiveRuleState.host_id == host.id))).all()}
+    baseline = states.get(BASELINE_KEY)
+    since = baseline.last_seen_at if baseline else now
+    events = 0
+    touched: list[str] = []
+    if baseline is not None:
+        known = [u for u, st in states.items() if u != BASELINE_KEY and st.present]
+        missing = sorted(u for u in known if u not in current)
+        if known and len(missing) * 2 > len(known):
+            # More than half of the known rules vanished at once: possible, but far more often a broken answer.
+            # Only the same picture on two observations in a row counts.
+            marker = hashlib.sha256("\n".join(missing).encode()).hexdigest()
+            if (baseline.row or {}).get("pending") != marker:
+                await db.execute(update(LiveRuleState).where(
+                    LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY)
+                    .values(row={"pending": marker}).execution_options(synchronize_session=False))
+                await db.commit()
+                return {"baseline": False, "events": 0, "skipped": True, "touched": [], "unconfirmed": len(missing)}
+
+    async def claim(rule_uuid: str, state: LiveRuleState, new_row: dict | None) -> int | None:
+        """Move the state to its new version; None when another process already did."""
+        values = {"version": LiveRuleState.version + 1, "changed_at": now, "last_seen_at": now,
+                  "present": new_row is not None}
+        if new_row is not None:
+            values.update(fingerprint=fingerprint(new_row), row=new_row)
+        else:
+            values.update(fingerprint="")
+        return await db.scalar(update(LiveRuleState).where(
+            LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == rule_uuid, LiveRuleState.version == state.version,
+        ).values(**values).returning(LiveRuleState.version).execution_options(synchronize_session=False))
+
+    async def record(rule_uuid: str, seq: int, before: dict | None, after: dict | None) -> None:
+        nonlocal events
+        kind = change_kind(before, after)
+        if kind in RESTARTS_USAGE:
+            touched.append(rule_uuid)
+        audit = await _find_audit(db, host.id, rule_uuid, since, now)
+        stmt = pg_insert(RuleChangeEvent).values(
+            id=uuid_lib.uuid4(), host_id=host.id, rule_uuid=rule_uuid, seq=seq, observed_at=now,
+            kind=kind, before=before, after=after,
+            origin="opnexus" if audit else "external", actor=audit.actor if audit else None,
+            audit_id=audit.id if audit else None, window_start=since,
+        ).on_conflict_do_nothing(constraint="uq_rule_change_event")
+        events += 1 if (await db.execute(stmt)).rowcount else 0
+
+    for rule_uuid, row in current.items():
+        state = states.get(rule_uuid)
+        if state is None:
+            stmt = pg_insert(LiveRuleState).values(
+                host_id=host.id, rule_uuid=rule_uuid, fingerprint=fingerprint(row), row=row, present=True, version=1,
+                first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing()
+            inserted = (await db.execute(stmt)).rowcount
+            if inserted and baseline is not None:
+                await record(rule_uuid, 1, None, row)
+        elif not state.present:
+            seq = await claim(rule_uuid, state, row)
+            if seq is not None:
+                await record(rule_uuid, seq, None, row)
+        elif state.fingerprint != fingerprint(row):
+            seq = await claim(rule_uuid, state, row)
+            if seq is not None:
+                await record(rule_uuid, seq, state.row, row)
+    if baseline is not None:
+        for rule_uuid, state in states.items():
+            if rule_uuid == BASELINE_KEY or rule_uuid in current or not state.present:
+                continue
+            seq = await claim(rule_uuid, state, None)
+            if seq is not None:
+                await record(rule_uuid, seq, state.row, None)
+    unchanged = [u for u in current if u in states and states[u].present]
+    if unchanged:
+        await db.execute(update(LiveRuleState).where(
+            LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid.in_(unchanged)).values(last_seen_at=now)
+            .execution_options(synchronize_session=False))
+    if baseline is None:
+        await db.execute(pg_insert(LiveRuleState).values(
+            host_id=host.id, rule_uuid=BASELINE_KEY, fingerprint="", row={}, present=False, version=1,
+            first_seen_at=now, changed_at=now, last_seen_at=now).on_conflict_do_nothing())
+    else:
+        await db.execute(update(LiveRuleState).where(
+            LiveRuleState.host_id == host.id, LiveRuleState.rule_uuid == BASELINE_KEY).values(last_seen_at=now, row={})
+            .execution_options(synchronize_session=False))
+    await db.commit()
+    return {"baseline": baseline is None, "events": events, "touched": touched}
+
+
+async def reconcile_origins(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
+    """Recent 'external' events may only look so because the audit entry was written after the observation."""
+    now = now or datetime.now(timezone.utc)
+    recent = (await db.scalars(select(RuleChangeEvent).where(
+        RuleChangeEvent.host_id == host.id, RuleChangeEvent.origin == "external",
+        RuleChangeEvent.observed_at >= now - RECHECK_WINDOW).order_by(RuleChangeEvent.observed_at, RuleChangeEvent.seq))).all()
+    fixed = 0
+    for event in recent:  # oldest first: an audit entry belongs to the earliest change it can explain
+        start = event.window_start or (event.observed_at - RECHECK_WINDOW)
+        audit = await _find_audit(db, host.id, event.rule_uuid, start, event.observed_at)
+        if audit is not None:
+            event.origin, event.actor, event.audit_id = "opnexus", audit.actor, audit.id
+            # (autoflush makes the claim visible to the next event's query)
+            fixed += 1
+    if fixed:
+        await db.commit()
+    return fixed
+
+
+async def observe_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> dict:
+    """Two GETs: the rules and their counters. One read serves the usage sampling and the history."""
+    # First settle the origin of earlier events (their audit entries may have arrived late), so that an entry is
+    # not taken by a newer change of the same rule that happens to fall in its window.
+    await reconcile_origins(db, host, now)
+    search = await client.search_rules(host, all_rows=True)
+    try:
+        counters = await rule_hit_service.read_counters(host)
+    except OpnsenseAPIError as exc:
+        counters = None
+        logger.info("Regelzähler von %s nicht lesbar: %s", host.name, exc)
+    result = await observe_rows(db, host, search, now)
+    if not result.get("skipped"):  # a rejected or unconfirmed answer must not touch the usage data either
+        if counters is not None:
+            await rule_hit_service.apply_sample(db, host, counters, search, now)
+        await rule_hit_service.restart_window(db, host.id, result["touched"], now)
+    return result
+
+
+async def observe_all(db: AsyncSession, now: datetime | None = None) -> dict:
+    hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all()
+    summary = {"hosts": 0, "failed": 0, "events": 0}
+    for host in hosts:
+        try:
+            result = await observe_host(db, host, now)
+            summary["hosts"] += 1
+            summary["events"] += result["events"]
+        except OpnsenseAPIError as exc:
+            summary["failed"] += 1
+            logger.warning("Regelverlauf von %s nicht lesbar: %s", host.name, exc)
+        except Exception:  # noqa: BLE001 - one host must never stop the cycle for the others
+            summary["failed"] += 1
+            await db.rollback()
+            logger.exception("Regelverlauf von %s fehlgeschlagen", host.name)
+    return summary
+
+
+async def prune(db: AsyncSession, retention_days: int = RETENTION_DAYS) -> int:
+    cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
+    events = await db.execute(delete(RuleChangeEvent).where(RuleChangeEvent.observed_at < cutoff))
+    gone = await db.execute(delete(LiveRuleState).where(
+        LiveRuleState.present.is_(False), LiveRuleState.rule_uuid != BASELINE_KEY, LiveRuleState.changed_at < cutoff))
+    await db.commit()
+    return (events.rowcount or 0) + (gone.rowcount or 0)
+
+
+# ── Reading the history of one rule ───────────────────────────────────────
+
+def _iso(value: datetime | None) -> str | None:
+    return value.isoformat() if value else None
+
+
+async def rule_timeline(db: AsyncSession, host: OpnsenseHost, rule_uuid: str, limit: int = 200) -> list[dict]:
+    """Observed changes, saved revisions (OPNexus-managed rules) and audit entries of one rule, newest first."""
+    rule_uuid = rule_uuid.lower()[:64]
+    items: list[dict] = []
+    for event in (await db.scalars(select(RuleChangeEvent).where(
+            RuleChangeEvent.host_id == host.id, RuleChangeEvent.rule_uuid == rule_uuid)
+            .order_by(RuleChangeEvent.observed_at.desc()).limit(limit))).all():
+        items.append({
+            "source": "observed", "at": event.observed_at, "kind": event.kind, "origin": event.origin, "actor": event.actor,
+            "changes": diff_fields(event.before, event.after), "audit_id": event.audit_id,
+        })
+    audits = (await db.scalars(select(AuditEvent).where(
+        AuditEvent.host_id == str(host.id), AuditEvent.kind.in_(AUDIT_KINDS),
+        AuditEvent.details["remote_uuid"].as_string() == rule_uuid,
+        AuditEvent.status.notin_(("requested",))).order_by(AuditEvent.created_at.desc()).limit(limit))).all()
+    explained = {item["audit_id"] for item in items if item["audit_id"]}  # already shown as the observed change
+    for audit in audits:
+        if audit.id in explained:
+            continue
+        items.append({
+            "source": "audit", "at": audit.created_at, "kind": audit.action, "status": audit.status, "actor": audit.actor,
+            "origin": "opnexus", "message": audit.message, "audit_id": audit.id, "changes": [],
+        })
+    deployment = await db.scalar(select(RuleDeployment).where(
+        RuleDeployment.host_id == host.id, RuleDeployment.opnsense_rule_uuid == rule_uuid))
+    if deployment is not None:
+        revisions = (await db.scalars(select(RuleRevision).where(RuleRevision.rule_id == deployment.rule_id)
+                                      .order_by(RuleRevision.created_at.desc()).limit(limit))).all()
+        ordered = sorted(revisions, key=lambda r: r.created_at)
+        previous = None
+        for revision in ordered:
+            items.append({
+                "source": "revision", "at": revision.created_at, "kind": "revision", "origin": "opnexus", "actor": None,
+                "note": revision.note, "changes": diff_fields(previous, revision.snapshot) if previous is not None else [],
+                "audit_id": None, "managed_rule_id": deployment.rule_id,
+            })
+            previous = revision.snapshot
+    items.sort(key=lambda item: item["at"], reverse=True)
+    return items[:limit]

backend/app/services/rule_hit_service.py

@@ -1,7 +1,10 @@
 """Rule usage from pf's counters: which rules never match anything?
 
-``sample_host`` reads the per-rule packet counters (``diagnostics/firewall/pfStatistics/rules``) and the
-configured rules (``searchRule``) and updates one row per rule: the time of the last observed hit.
+``sample_host`` reads the per-rule packet counters and the configured rules (``searchRule``) and updates one
+row per rule: the time of the last observed hit. The counters come from ``firewall/filter_util/ruleStats``
+(already summed per label by the firewall; verified live on OPNsense 26.7 on all three test firewalls, see
+the evidence block in opnsense_client.py, where its numbers equal the sum over ``pfStatistics/rules``). Older
+releases without that endpoint fall back to parsing ``diagnostics/firewall/pfStatistics/rules``.
 
 * A counter that grew since the last sample is a hit "now" (at most one sampling interval late).
 * A counter that is *lower* than before means pf reloaded its ruleset (an apply or a reboot), which
@@ -10,14 +13,16 @@ configured rules (``searchRule``) and updates one row per rule: the time of the
 * A rule is called unused only after OPNexus has watched it for ``UNUSED_AFTER_DAYS`` without a hit.
   A new rule, or a brand-new installation, is "still collecting data" instead of a false alarm.
 
-Only rules with a UUID label are tied to a configured rule (rules created in the rule editor or API; OPNsense's
-built-in automatic rules carry hash labels and are not user rules).
+A counter belongs to a configured rule by its label: the UUID of a rule created in the rule editor or API, or the
+32-hex label of a not yet migrated legacy rule (the ``searchRule`` row carries the same string as ``uuid``).
+OPNsense's built-in automatic rules also have hash labels but are flagged ``is_automatic`` there, so they are
+never tracked.
 """
 import logging
 import re
 from datetime import datetime, timedelta, timezone
 
-from sqlalchemy import delete, select
+from sqlalchemy import delete, select, update
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.host import OpnsenseHost
@@ -28,7 +33,12 @@ from app.services.opnsense_client import OpnsenseAPIError
 logger = logging.getLogger(__name__)
 
 UNUSED_AFTER_DAYS = 30
-_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})"')
+_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}|[0-9a-fA-F]{32})"')
+# A sample counts as fresh for the "unused" confidence when it is at most this old.
+SAMPLE_FRESH = timedelta(hours=24)
+# A pf reload zeroes the counters; whatever was hit between the last sample and the reload is lost. After a gap
+# this long the loss is too big to call the rule unused on that evidence: the observation window starts over.
+SAMPLE_GAP = timedelta(hours=2)
 
 
 def parse_counters(stats: dict) -> dict[str, dict]:
@@ -45,18 +55,65 @@ def parse_counters(stats: dict) -> dict[str, dict]:
     return result
 
 
+def parse_rule_stats(data: dict) -> dict[str, dict] | None:
+    """{label: {packets, bytes, evaluations}} from ``filter_util/ruleStats``; None when the answer is not
+    that shape (older release, error page), so the caller can fall back to pfStatistics."""
+    stats = (data or {}).get("stats") if isinstance(data, dict) else None
+    if not isinstance(stats, dict) or (data or {}).get("status") != "ok":
+        return None
+    result: dict[str, dict] = {}
+    for label, values in stats.items():
+        if isinstance(values, dict):
+            result[str(label).lower()] = {key: int(values.get(key) or 0) for key in ("packets", "bytes", "evaluations")}
+    return result
+
+
+async def read_counters(host: OpnsenseHost) -> dict[str, dict]:
+    """Counters per rule label: ruleStats first, the pf ruleset text as fallback."""
+    try:
+        parsed = parse_rule_stats(await client.get_rule_stats(host))
+    except OpnsenseAPIError:
+        parsed = None
+    if parsed is not None:
+        return parsed
+    return parse_counters(await client.get_pf_rule_statistics(host))
+
+
+def looks_complete(search) -> bool:
+    """A usable ``searchRule`` answer: a list that contains OPNsense's own automatic rules. A real firewall always
+    has some, so an empty or partial answer (a firewall just restarting a service, a proxy error page) is rejected
+    instead of being read as "every rule is gone"."""
+    rows = search.get("rows") if isinstance(search, dict) else None
+    if not isinstance(rows, list) or not rows:
+        return False
+    dicts = [r for r in rows if isinstance(r, dict)]
+    if not any("is_automatic" in r for r in dicts):
+        return bool(dicts)  # firmware that does not flag automatic rules at all: a non-empty list is all we can ask for
+    return any(r.get("is_automatic") in (True, 1, "1") for r in dicts)
+
+
 def configured_rules(search: dict) -> dict[str, dict]:
     rows = (search or {}).get("rows") or []
     return {
         str(r["uuid"]).lower(): {"description": r.get("description") or "", "enabled": r.get("enabled") in ("1", 1, True)}
-        for r in rows if r.get("uuid") and not r.get("is_automatic")
+        for r in rows if r.get("uuid") and r.get("is_automatic") not in (True, 1, "1")
     }
 
 
 async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
+    """Read the counters and the rules from the firewall and update the hit rows."""
+    counters = await read_counters(host)
+    return await apply_sample(db, host, counters, await client.search_rules(host, all_rows=True), now)
+
+
+async def apply_sample(db: AsyncSession, host: OpnsenseHost, counters: dict[str, dict], search: dict,
+                       now: datetime | None = None) -> int:
+    """Update the hit rows from counters (per label) and a ``searchRule`` answer that are already read
+    (the observe loop shares one read between usage and change history)."""
     now = now or datetime.now(timezone.utc)
-    counters = parse_counters(await client.get_pf_rule_statistics(host))
-    rules = configured_rules(await client.search_rules(host, all_rows=True))
+    if not looks_complete(search):
+        return 0  # not a full rule list: an error or partial answer must not look like "every rule was deleted"
+    rules = configured_rules(search)
     rows = {r.rule_uuid: r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host.id))).all()}
     for uuid_, meta in rules.items():
         counter = counters.get(uuid_, {"packets": 0, "bytes": 0})
@@ -68,6 +125,10 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
                 row.last_hit_at = now  # already matching when first seen
         elif counter["packets"] > row.last_packets or (counter["packets"] < row.last_packets and counter["packets"] > 0):
             row.last_hit_at = now
+        elif counter["packets"] < row.last_packets and now - row.last_sampled_at > SAMPLE_GAP:
+            row.first_seen_at = now  # reloaded after a long gap: the hits in between are unknown
+        if meta["enabled"] and not row.enabled:
+            row.first_seen_at = now  # a rule that was off has not been "waiting for traffic" while it was off
         row.description = meta["description"][:255]
         row.enabled = meta["enabled"]
         row.last_packets, row.last_bytes, row.last_sampled_at = counter["packets"], counter["bytes"], now
@@ -78,6 +139,18 @@ async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None
     return len(rules)
 
 
+async def restart_window(db: AsyncSession, host_id, rule_uuids, now: datetime | None = None) -> int:
+    """The rules were edited, enabled or created: what OPNexus saw before does not describe them any more, so the
+    observation window for "unused" starts over (an earlier last hit is kept)."""
+    keys = [u.lower() for u in rule_uuids]
+    if not keys:
+        return 0
+    result = await db.execute(update(RuleHit).where(RuleHit.host_id == host_id, RuleHit.rule_uuid.in_(keys))
+                              .values(first_seen_at=now or datetime.now(timezone.utc)).execution_options(synchronize_session=False))
+    await db.commit()
+    return result.rowcount or 0
+
+
 async def sample_all(db: AsyncSession) -> int:
     hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)))).all()
     sampled = 0
@@ -92,14 +165,14 @@ async def sample_all(db: AsyncSession) -> int:
     return sampled
 
 
-def classify(row: RuleHit, now: datetime) -> str:
-    """'used' | 'unused' | 'collecting' | 'disabled'"""
+def classify(row: RuleHit, now: datetime, days: int = UNUSED_AFTER_DAYS) -> str:
+    """'used' | 'unused' | 'collecting' | 'disabled'; ``days`` is the observation window."""
     if not row.enabled:
         return "disabled"
-    if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=UNUSED_AFTER_DAYS):
+    if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=days):
         return "used"
     watched = now - row.first_seen_at
-    if watched < timedelta(days=UNUSED_AFTER_DAYS):
+    if watched < timedelta(days=days):
         return "used" if row.last_hit_at is not None else "collecting"
     return "unused"
 

backend/app/services/scheduled_maintenance_service.py

@@ -15,18 +15,31 @@ the UI) but is NOT wired into the NotificationSettings webhook/email
 system — that system is built around persistent NotificationProblem rows
 with their own admin-configurable toggles, and adding a new problem kind
 for this is a bigger change than this feature's own scope. Flagged here,
-not silently dropped, in case it's wanted later.
+not silently dropped, in case it's wanted later. (Later: the failure is reported by
+notification_checks.check_scheduled_maintenance_failed since 1.17.)
+
+auto_verify (1.25, set for every rollout step): instead of leaving the operation "submitted"
+for a human, the run waits for the job (phase job_wait) and then checks the result itself
+(phase verify: firmware idle for a while, a reboot really happened when one was due, version
+readable). Only then the operation is closed as "acknowledged"; anything unclear ends as
+"unknown" (stays locked, notification). Without auto_verify nothing changes.
+
+Concurrency: due rows and running rows are each taken with SELECT ... FOR UPDATE SKIP LOCKED,
+one row per transaction, so a second backend (briefly possible during a web update) can never
+fire or step the same row at the same time.
 """
 import logging
 from datetime import datetime, timedelta, timezone
 
 from fastapi import HTTPException
-from sqlalchemy import select
+from sqlalchemy import select, true
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.host import OpnsenseHost
 from app.models.host_operation import HostOperation
 from app.models.scheduled_maintenance import ScheduledMaintenance
+from contextlib import contextmanager
+
 from app.services import audit_service, backup_store, opnsense_client as client
 from app.services.host_maintenance_service import ACTIVE, ensure_no_maintenance, inspect_carp_switch, inspect_host, peers
 from app.services.host_status_service import get_statuses
@@ -51,40 +64,93 @@ HANDOVER_TIMEOUT = timedelta(minutes=3)
 LEAVE_TIMEOUT = timedelta(minutes=5)
 JOB_GRACE = timedelta(minutes=5)  # no sign of a job after this long: treat a package update as already done
 JOB_TIMEOUT = {"reboot": timedelta(minutes=20), "update": timedelta(minutes=60), "major_update": timedelta(minutes=150)}
+# auto_verify: the firmware must report "ready" for this long in a row (a reboot at the end of an update
+# can follow a short "ready" moment), and the whole check gives up after VERIFY_TIMEOUT.
+VERIFY_SETTLE = timedelta(seconds=60)
+VERIFY_TIMEOUT = timedelta(minutes=20)
+# A rollout step is created with run_at=now and fires on the next poll. Found much later (OPNexus was down,
+# e.g. during an update), it no longer belongs to the decision that created it: the window may have closed
+# and the run may have been paused meanwhile. It is then not fired; the rollout engine re-queues the host.
+ROLLOUT_FIRE_GRACE = timedelta(minutes=5)
 
 
 def _now() -> datetime:
     return datetime.now(timezone.utc)
 
 
+@contextmanager
+def _as_actor(sched: ScheduledMaintenance):
+    """Audit events of an unattended run name the person who planned it (not the rollback worker)."""
+    token = audit_service.context.set({"actor": f"{sched.actor} (geplant)"[:150], "id": str(sched.id)})
+    try:
+        yield
+    finally:
+        audit_service.context.reset(token)
+
+
+def _rollout_details(sched: ScheduledMaintenance) -> dict:
+    rollout = (sched.progress or {}).get("rollout")
+    return {"rollout": rollout} if rollout else {}
+
+
 async def fire_due(db: AsyncSession) -> dict:
-    due = list(
-        (
-            await db.scalars(
-                select(ScheduledMaintenance).where(
-                    ScheduledMaintenance.status == "pending", ScheduledMaintenance.run_at <= _now()
-                )
-            )
-        ).all()
-    )
-    fired = 0
-    blocked = 0
-    for sched in due:
-        try:
-            await _fire_one(db, sched)
-        except Exception:
-            logger.exception("Geplante Wartung %s fehlgeschlagen", sched.id)
-            sched.status = "failed"
-            sched.result = {"error": "Unerwarteter Fehler beim Ausführen"}
+    due = fired = blocked = 0
+    seen: set = set()
+    while True:
+        # One row per transaction, locked: a second backend skips it instead of firing it again.
+        sched = await db.scalar(
+            select(ScheduledMaintenance)
+            .where(ScheduledMaintenance.status == "pending", ScheduledMaintenance.run_at <= _now(),
+                   ScheduledMaintenance.id.not_in(seen) if seen else true())
+            .order_by(ScheduledMaintenance.run_at)
+            .limit(1)
+            .with_for_update(skip_locked=True)
+            .execution_options(populate_existing=True)
+        )
+        if sched is None:
             await db.commit()
+            break
+        seen.add(sched.id)
+        due += 1
+        with _as_actor(sched):
+            try:
+                await _fire_one(db, sched)
+            except Exception:
+                logger.exception("Geplante Wartung %s fehlgeschlagen", sched.id)
+                await db.rollback()
+                sched = await db.get(ScheduledMaintenance, sched.id, populate_existing=True)
+                if sched.status in ("pending", "running") and sched.phase is None:
+                    sched.status = "failed"
+                    sched.result = {"error": "Unerwarteter Fehler beim Ausführen"}
+                await db.commit()
+        await db.commit()  # never keep the row lock past this row
         if sched.status == "failed":
             blocked += 1
         else:
             fired += 1
-    return {"due": len(due), "fired": fired, "blocked_or_failed": blocked}
+    return {"due": due, "fired": fired, "blocked_or_failed": blocked}
+
+
+def _reboot_expected(action: str, state: dict) -> bool:
+    return action in ("reboot", "major_update") or bool(state.get("reboot_required"))
+
+
+async def _boot_time(host: OpnsenseHost) -> str | None:
+    """systemTime's "boottime" (Diagnostics SystemController::systemTimeAction, read-only, already used by the
+    resource sampler). A different value afterwards proves a reboot even if no poll caught the host offline."""
+    try:
+        return (await client.get_system_time(host)).get("boottime") or None
+    except client.OpnsenseAPIError:
+        return None
 
 
 async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
+    if sched.origin == "rollout" and _now() - sched.run_at > ROLLOUT_FIRE_GRACE:
+        sched.status = "failed"
+        sched.result = {"error": "Rollout-Schritt nicht rechtzeitig gestartet, nichts ausgeführt", "not_fired": True}
+        await db.commit()
+        logger.warning("Geplante Wartung %s: Rollout-Schritt überfällig, nicht ausgeführt", sched.id)
+        return
     host = await db.get(OpnsenseHost, sched.host_id)
     if not host or not host.enabled:
         sched.status = "failed"
@@ -137,6 +203,7 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
         logger.warning("Geplante Wartung %s blockiert: %s", sched.id, sched.result["error"])
         return
 
+    boot_before = await _boot_time(host) if sched.auto_verify else None
     op = HostOperation(
         host_id=host.id, host_name=host.name, actor=sched.actor, action=sched.action,
         expires_at=_now(), fingerprint="", snapshot=state, status="starting", started_at=_now(),
@@ -147,14 +214,23 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
     await audit_service.persist(
         kind="host", action=sched.action, status="requested",
         host_id=str(host.id), host_name=host.name, entity_id=str(op.id),
-        details={"scheduled": True, "schedule_id": str(sched.id), "snapshot": state},
+        details={"scheduled": True, "schedule_id": str(sched.id), "snapshot": state, **_rollout_details(sched)},
     )
     try:
         result = await _ACTION_FN[sched.action](host)
         op.status = "submitted"
-        op.message = "Geplant übergeben. Abschluss und Erreichbarkeit prüfen; anschließend Wartung abschließen."
-        sched.status = "succeeded"
-        sched.result = {"response": result}
+        if sched.auto_verify:
+            # Not done yet: the poll loop waits for the job and checks the result (job_wait -> verify).
+            op.message = "Geplant übergeben. Abschluss wird automatisch geprüft."
+            sched.phase, sched.phase_since = "job_wait", _now()
+            _progress(sched, handover=False, job_started_at=_now().isoformat(), seen_down=False, seen_running=False,
+                      version_before=state.get("version"), boot_before=boot_before,
+                      reboot_expected=_reboot_expected(sched.action, state))
+            sched.result = {"response": result}
+        else:
+            op.message = "Geplant übergeben. Abschluss und Erreichbarkeit prüfen; anschließend Wartung abschließen."
+            sched.status = "succeeded"
+            sched.result = {"response": result}
     except client.OpnsenseAPIError as exc:
         op.status = "unknown"
         op.message = "Ausgang unklar: " + str(exc)[:800] + ". Nicht automatisch wiederholen."
@@ -166,7 +242,7 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
     await audit_service.persist(
         kind="host", action=sched.action, status=op.status,
         host_id=str(host.id), host_name=host.name, entity_id=str(op.id),
-        message=op.message, details={"scheduled": True, "schedule_id": str(sched.id), "response": result},
+        message=op.message, details={"scheduled": True, "schedule_id": str(sched.id), "response": result, **_rollout_details(sched)},
     )
 
 
@@ -182,21 +258,24 @@ async def _audit(host, sched, op, action, status, message=None, **details):
     await audit_service.persist(
         kind="host", action=action, status=status, host_id=str(host.id), host_name=host.name,
         entity_id=str(op.id) if op else None, message=message,
-        details={"scheduled": True, "schedule_id": str(sched.id), "handover": True, **details},
+        details={"scheduled": True, "schedule_id": str(sched.id), "handover": bool((sched.progress or {}).get("handover")),
+                 **_rollout_details(sched), **details},
     )
 
 
 async def _end(db, sched, host, op, *, ok, message, op_status=None, **result):
+    handover = bool((sched.progress or {}).get("handover"))
     sched.status = "succeeded" if ok else "failed"
     sched.phase = None
-    sched.result = {"handover": True, **result} if ok else {"error": message[:900], **result}
+    sched.result = ({"handover": handover, "auto_verified": bool(sched.auto_verify), **result} if ok
+                    else {"error": message[:900], **result})
     if op is not None:
         op.status = op_status or ("acknowledged" if ok else "unknown")
         op.message = message[:1000]
     await db.commit()
     await _audit(host, sched, op, sched.action, "completed" if ok else (op_status or "unknown"), message)
     if not ok:
-        logger.warning("Geplante Wartung %s (CARP-Übergabe) beendet: %s", sched.id, message)
+        logger.warning("Geplante Wartung %s beendet: %s", sched.id, message)
 
 
 async def _toggle(host, expected: str) -> None:
@@ -230,6 +309,7 @@ async def _start_handover(db: AsyncSession, sched: ScheduledMaintenance, host: O
         await fail(" ".join(blocked))
         return
 
+    boot_before = await _boot_time(host) if sched.auto_verify else None
     now = _now()
     op = HostOperation(
         host_id=host.id, host_name=host.name, actor=sched.actor, action=sched.action,
@@ -237,6 +317,7 @@ async def _start_handover(db: AsyncSession, sched: ScheduledMaintenance, host: O
     )
     db.add(op)
     sched.status = "running"
+    _progress(sched, handover=True, boot_before=boot_before, reboot_expected=_reboot_expected(sched.action, state))
     await db.commit()
     sched.host_operation_id = op.id
     await _audit(host, sched, op, "carp_enter", "requested", snapshot=state)
@@ -255,32 +336,56 @@ async def _start_handover(db: AsyncSession, sched: ScheduledMaintenance, host: O
 
 
 async def advance_running(db: AsyncSession) -> int:
-    """One step for every handover run in progress. Returns how many are still running."""
-    rows = list((await db.scalars(
-        select(ScheduledMaintenance).where(ScheduledMaintenance.status == "running", ScheduledMaintenance.phase.is_not(None))
+    """One step for every run in progress (handover and/or auto_verify). Returns how many are still running."""
+    ids = list((await db.scalars(
+        select(ScheduledMaintenance.id).where(ScheduledMaintenance.status == "running", ScheduledMaintenance.phase.is_not(None))
     )).all())
-    for sched in rows:
-        host = await db.get(OpnsenseHost, sched.host_id)
-        op = await db.get(HostOperation, sched.host_operation_id) if sched.host_operation_id else None
-        try:
-            if host is None:
-                raise RuntimeError("Host nicht mehr vorhanden")
-            if op is not None and op.status not in ACTIVE:
-                # A human closed or cancelled the operation: stop here and touch nothing more.
-                sched.status, sched.phase = "failed", None
-                sched.result = {"error": f"Vorgang von Hand beendet, Ablauf gestoppt. CARP-Rolle von {host.name} prüfen."}
-                await db.commit()
-                continue
-            await _STEPS[sched.phase](db, sched, host, op)
-        except Exception as exc:  # a stuck run must not loop forever or take the poll loop down
-            logger.exception("Geplante Wartung %s: Schritt %s fehlgeschlagen", sched.id, sched.phase)
-            if host is not None:
-                await _end(db, sched, host, op, ok=False,
-                           message=f"Unerwarteter Fehler während der CARP-Übergabe ({type(exc).__name__}). "
-                                   f"{host.name} kann im CARP-Wartungsmodus verblieben sein — Rollen prüfen.")
-    return sum(1 for s in rows if s.status == "running")
+    await db.commit()
+    still_running = 0
+    for sched_id in ids:
+        # Locked for the whole step: a second backend skips the row instead of toggling or submitting twice.
+        sched = await db.scalar(
+            select(ScheduledMaintenance)
+            .where(ScheduledMaintenance.id == sched_id, ScheduledMaintenance.status == "running",
+                   ScheduledMaintenance.phase.is_not(None))
+            .with_for_update(skip_locked=True)
+            .execution_options(populate_existing=True)
+        )
+        if sched is None:
+            await db.commit()
+            still_running += 1 if await db.scalar(select(ScheduledMaintenance.status).where(ScheduledMaintenance.id == sched_id)) == "running" else 0
+            await db.commit()
+            continue
+        with _as_actor(sched):
+            await _advance_one(db, sched)
+        await db.commit()
+        if sched.status == "running":
+            still_running += 1
+    return still_running
 
 
+async def _advance_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
+    host = await db.get(OpnsenseHost, sched.host_id)
+    op = await db.get(HostOperation, sched.host_operation_id) if sched.host_operation_id else None
+    try:
+        if host is None:
+            raise RuntimeError("Host nicht mehr vorhanden")
+        if op is not None and op.status not in ACTIVE:
+            # A human closed or cancelled the operation: stop here and touch nothing more.
+            hint = f" CARP-Rolle von {host.name} prüfen." if (sched.progress or {}).get("handover") else ""
+            sched.status, sched.phase = "failed", None
+            sched.result = {"error": f"Vorgang von Hand beendet, Ablauf gestoppt.{hint}"}
+            await db.commit()
+            return
+        await _STEPS[sched.phase](db, sched, host, op)
+    except Exception as exc:  # a stuck run must not loop forever or take the poll loop down
+        logger.exception("Geplante Wartung %s: Schritt %s fehlgeschlagen", sched.id, sched.phase)
+        if host is not None:
+            hint = (f"{host.name} kann im CARP-Wartungsmodus verblieben sein — Rollen prüfen."
+                    if (sched.progress or {}).get("handover") else f"Zustand von {host.name} prüfen.")
+            await _end(db, sched, host, op, ok=False,
+                       message=f"Unerwarteter Fehler während der geplanten Wartung ({type(exc).__name__}). {hint}")
+
 def _since(sched: ScheduledMaintenance) -> timedelta:
     return _now() - (sched.phase_since or _now())
 
@@ -352,17 +457,75 @@ async def _step_job_wait(db, sched, host, op):
         else:
             done = bool(prog.get("seen_down") or prog.get("seen_running")) or elapsed >= JOB_GRACE
         if done:
-            sched.phase, sched.phase_since = "leave", _now()
+            sched.phase = "verify" if sched.auto_verify else "leave"
+            sched.phase_since = _now()
             await db.commit()
             return
     if elapsed >= JOB_TIMEOUT[sched.action]:
-        await _end(db, sched, host, op, ok=False,
-                   message=f"Auftrag „{sched.action}“ auf {host.name} nicht rechtzeitig abgeschlossen. {host.name} bleibt im "
-                           f"CARP-Wartungsmodus, der Partner trägt den Verkehr. Bitte prüfen und von Hand zurückschalten.")
+        if prog.get("handover"):
+            text = (f"Auftrag „{sched.action}“ auf {host.name} nicht rechtzeitig abgeschlossen. {host.name} bleibt im "
+                    f"CARP-Wartungsmodus, der Partner trägt den Verkehr. Bitte prüfen und von Hand zurückschalten.")
+        else:
+            text = f"Auftrag „{sched.action}“ auf {host.name} nicht rechtzeitig abgeschlossen. Bitte {host.name} prüfen."
+        await _end(db, sched, host, op, ok=False, message=text)
         return
     await db.commit()
 
 
+async def _step_verify(db, sched, host, op):
+    """auto_verify: is the job really over and the host healthy? Read-only, never repeats the action."""
+    prog = dict(sched.progress or {})
+    handover = bool(prog.get("handover"))
+
+    async def wait_or_give_up(why: str):
+        if _since(sched) >= VERIFY_TIMEOUT:
+            tail = (f" {host.name} bleibt im CARP-Wartungsmodus, der Partner trägt den Verkehr." if handover else "")
+            await _end(db, sched, host, op, ok=False,
+                       message=f"Automatische Prüfung nach „{sched.action}“ nicht bestanden: {why}.{tail} "
+                               f"Bitte {host.name} prüfen und die Wartung von Hand abschließen.")
+            return
+        _progress(sched, waiting_for=why)
+        await db.commit()
+
+    try:
+        running = (await client.get_firmware_running(host)).get("status")
+    except client.OpnsenseAPIError:
+        running = None
+        _progress(sched, seen_down=True)
+    if running != "ready":
+        _progress(sched, ready_since=None)
+        await wait_or_give_up("Firmware-Auftrag läuft noch oder Host nicht erreichbar")
+        return
+    if not prog.get("ready_since"):
+        _progress(sched, ready_since=_now().isoformat())
+        await db.commit()
+        return
+    if _now() - datetime.fromisoformat(prog["ready_since"]) < VERIFY_SETTLE:
+        await db.commit()
+        return
+    boot_now = await _boot_time(host)
+    rebooted = bool(prog.get("seen_down")) or bool(prog.get("boot_before") and boot_now and boot_now != prog["boot_before"])
+    if prog.get("reboot_expected") and not rebooted:
+        await wait_or_give_up("der erwartete Neustart wurde nicht beobachtet")
+        return
+    try:
+        version = (await client.get_firmware_info(host)).get("product_version")
+    except client.OpnsenseAPIError:
+        version = None
+    online = (await get_statuses([host])).get(host.id, {}).get("status") == "online"
+    if not version or not online:
+        await wait_or_give_up("Host nicht vollständig erreichbar (Version/Status nicht lesbar)")
+        return
+    _progress(sched, version_after=version, boot_after=boot_now, rebooted=rebooted, waiting_for=None)
+    if handover:
+        sched.phase, sched.phase_since = "leave", _now()  # only now take the traffic back
+        await db.commit()
+        return
+    await _end(db, sched, host, op, ok=True,
+               message=f"Automatisch geprüft: {host.name} erreichbar, Firmware-Auftrag beendet, Version {version}.",
+               version_before=prog.get("version_before"), version_after=version, rebooted=rebooted)
+
+
 async def _step_leave(db, sched, host, op):
     me, _partner = await _roles(db, host)
     if me.get("status") != "online":
@@ -397,6 +560,8 @@ async def _step_leave_wait(db, sched, host, op):
         except client.OpnsenseAPIError:
             pass
         extra["version_before"] = (sched.progress or {}).get("version_before")
+        if sched.auto_verify:
+            extra["rebooted"] = bool((sched.progress or {}).get("rebooted"))  # what verify saw before leaving
         text = "Automatisch abgeschlossen: Übergabe, Auftrag und Zurückschalten bestätigt."
         if extra.get("reboot_required"):
             text += " Ein Neustart ist noch erforderlich."
@@ -410,6 +575,7 @@ async def _step_leave_wait(db, sched, host, op):
 _STEPS = {
     "handover_wait": _step_handover_wait,
     "job_wait": _step_job_wait,
+    "verify": _step_verify,
     "leave": _step_leave,
     "leave_wait": _step_leave_wait,
 }

backend/app/services/tls_pin_service.py

@@ -18,7 +18,7 @@ from cryptography import x509
 from cryptography.hazmat.primitives import hashes, serialization
 from cryptography.x509.oid import ExtensionOID, NameOID
 
-from app.services import overlay_service, proxy_connect
+from app.services import proxy_connect
 
 MAX_PEM_BYTES = 32 * 1024
 
@@ -94,6 +94,8 @@ def verify_argument(verify_tls: bool, tls_ca_pem: str | None):
 
 async def _open_tls(base_url: str, parsed, ctx: ssl.SSLContext, timeout: float):
     """TLS connection to the firewall: directly, or through the overlay hub's proxy for overlay addresses."""
+    # Imported here so that pinned_context() stays usable without the database layer (gui-gateway container).
+    from app.services import overlay_service
     proxy = overlay_service.proxy_for(base_url)
     port = parsed.port or 443
     if proxy is None:

backend/app/services/url_guard.py

@@ -0,0 +1,199 @@
+"""Destination check for URLs OPNexus is told to call (notification channels, webhook).
+
+Admins legitimately point these at servers on their own network (a self-hosted ntfy, Gotify or
+Matrix), so private ranges are allowed. What is refused is what cannot be a notification target:
+anything but http(s), credentials inside the URL, and addresses that only ever mean "this machine"
+or "the cloud metadata service" (loopback, link-local such as 169.254.169.254, unspecified,
+multicast). Redirects are never followed by the callers, so a public host cannot bounce a request
+to such an address. The hostname is resolved at send time and every address is checked (a name that
+does not resolve is let through; the request then fails on its own).
+
+DNS rebinding (1.29): a check followed by a second, independent resolution inside the HTTP client could be
+answered differently ("first public, then 127.0.0.1"). ``guarded_client`` closes that window: its transport
+resolves the name itself when it opens the connection, refuses the connection if ANY answer is forbidden and
+connects to exactly the address it checked. Host header, SNI and certificate verification keep using the name.
+The up-front ``ensure_resolved_safe`` stays as the early, friendly error.
+
+How: httpcore (the layer under httpx) accepts a pluggable network backend. httpx does not expose it, so
+``GuardedTransport`` replaces the backend of the pool its parent class built (``_pool._network_backend``, httpx
+0.28 / httpcore 1.0). If those internals change, constructing the transport raises instead of silently
+unguarding; tests/test_url_guard.py opens a real socket against it, so an upgrade that breaks the hook fails the
+suite. The smaller alternative (rewriting the request URL to the checked IP with an ``sni_hostname`` extension)
+was rejected: it changes the URL every caller, log line and mock sees.
+"""
+import asyncio
+import contextlib
+import contextvars
+import ipaddress
+import logging
+import socket
+from urllib.parse import urlsplit
+
+import httpcore
+import httpx
+
+# httpx logs every request line ("HTTP Request: POST https://api.telegram.org/bot<token>/sendMessage"). For
+# targets whose URL is the credential that line must not reach the logs: callers wrap the request in muted().
+_muted = contextvars.ContextVar("url_guard_muted", default=False)
+
+
+class _MuteFilter(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        return not _muted.get()
+
+
+for _name in ("httpx", "httpcore"):
+    logging.getLogger(_name).addFilter(_MuteFilter())
+
+
+@contextlib.contextmanager
+def muted():
+    token = _muted.set(True)
+    try:
+        yield
+    finally:
+        _muted.reset(token)
+
+
+# Cloud metadata services outside the link-local range (AWS IPv6 IMDS, Alibaba Cloud).
+_METADATA_ADDRESSES = {ipaddress.ip_address("fd00:ec2::254"), ipaddress.ip_address("100.100.100.200")}
+
+
+def _forbidden(ip: ipaddress._BaseAddress) -> bool:
+    if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
+        ip = ip.ipv4_mapped
+    return ip.is_loopback or ip.is_link_local or ip.is_unspecified or ip.is_multicast or ip in _METADATA_ADDRESSES
+
+
+def validate_url(url: str, *, max_length: int = 500) -> str:
+    """Returns the cleaned URL or raises ValueError with a short reason (shown to the admin)."""
+    value = (url or "").strip()
+    if not value or len(value) > max_length or any(c.isspace() for c in value):
+        raise ValueError("Die Adresse ist leer, zu lang oder enthält Leerzeichen")
+    try:
+        parts = urlsplit(value)
+        host = parts.hostname
+        parts.port  # noqa: B018 - raises ValueError for an invalid port
+    except ValueError:
+        raise ValueError("Die Adresse ist ungültig") from None
+    if parts.scheme not in ("http", "https") or not host:
+        raise ValueError("Die Adresse muss mit http:// oder https:// beginnen")
+    if parts.username or parts.password:
+        raise ValueError("Zugangsdaten gehören nicht in die Adresse")
+    try:
+        literal = ipaddress.ip_address(host)
+    except ValueError:
+        literal = None
+    if literal is not None and _forbidden(literal):
+        raise ValueError("Diese Adresse (lokal, Link-Local oder Metadaten-Dienst) ist als Ziel nicht erlaubt")
+    if literal is None and host.lower() in ("localhost", "localhost.localdomain"):
+        raise ValueError("Diese Adresse (lokal, Link-Local oder Metadaten-Dienst) ist als Ziel nicht erlaubt")
+    return value
+
+
+async def resolve_host(host: str, port: int | None = None) -> list[str]:
+    """Every address the name currently resolves to (a literal answers itself). Raises OSError."""
+    host = host.strip("[]").split("%")[0]
+    try:
+        ipaddress.ip_address(host)
+        return [host]
+    except ValueError:
+        pass
+    infos = await asyncio.get_running_loop().getaddrinfo(host, port, type=socket.SOCK_STREAM)
+    seen: list[str] = []
+    for info in infos:
+        address = info[4][0].split("%")[0]
+        if address not in seen:
+            seen.append(address)
+    return seen
+
+
+def _any_forbidden(addresses: list[str]) -> bool:
+    for address in addresses:
+        try:
+            if _forbidden(ipaddress.ip_address(address)):
+                return True
+        except ValueError:
+            continue
+    return False
+
+
+async def ensure_resolved_safe(url: str) -> None:
+    """Send-time check against DNS answers pointing at forbidden addresses. Raises ValueError."""
+    host = urlsplit(url).hostname
+    if not host:
+        raise ValueError("Die Adresse ist ungültig")
+    try:
+        ipaddress.ip_address(host)
+        return  # a literal was already checked by validate_url
+    except ValueError:
+        pass
+    try:
+        addresses = await resolve_host(host)
+    except OSError:
+        return
+    if _any_forbidden(addresses):
+        raise ValueError("Der Name zeigt auf eine nicht erlaubte Adresse (lokal, Link-Local oder Metadaten-Dienst)")
+
+
+# ── connection-time guard (DNS rebinding) ─────────────────────────────────
+
+BLOCKED_MESSAGE = "Zieladresse nicht erlaubt"
+
+
+class BlockedAddress(httpcore.ConnectError):
+    """The name resolved to a forbidden address when the connection was opened (httpx reports ConnectError)."""
+
+
+class _GuardedBackend(httpcore.AsyncNetworkBackend):
+    def __init__(self, inner: httpcore.AsyncNetworkBackend):
+        self._inner = inner
+
+    async def connect_tcp(self, host, port, timeout=None, local_address=None, socket_options=None):
+        try:
+            addresses = await asyncio.wait_for(resolve_host(host, port), timeout)
+        except asyncio.TimeoutError:
+            raise httpcore.ConnectTimeout("name resolution timed out") from None
+        except OSError as exc:
+            raise httpcore.ConnectError(f"name resolution failed ({type(exc).__name__})") from None
+        if not addresses or _any_forbidden(addresses):
+            raise BlockedAddress(BLOCKED_MESSAGE)
+        last: Exception | None = None
+        for address in addresses:  # the checked addresses only, in resolver order
+            try:
+                return await self._inner.connect_tcp(address, port, timeout=timeout, local_address=local_address,
+                                                     socket_options=socket_options)
+            except (httpcore.ConnectError, httpcore.ConnectTimeout) as exc:
+                last = exc
+        raise last  # type: ignore[misc]
+
+    async def connect_unix_socket(self, path, timeout=None, socket_options=None):
+        raise httpcore.ConnectError("unix sockets are not allowed")
+
+    async def sleep(self, seconds):
+        await self._inner.sleep(seconds)
+
+
+class GuardedTransport(httpx.AsyncHTTPTransport):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        pool = getattr(self, "_pool", None)
+        backend = getattr(pool, "_network_backend", None)
+        if not isinstance(backend, httpcore.AsyncNetworkBackend):
+            raise RuntimeError("httpx/httpcore internals changed: the connection guard cannot be installed")
+        pool._network_backend = _GuardedBackend(backend)
+
+
+def guarded_client(**kwargs) -> httpx.AsyncClient:
+    """An httpx client whose connections only ever go to addresses that passed the check. Environment proxies are
+    ignored (a proxy would resolve the name itself, where the check cannot see it)."""
+    return httpx.AsyncClient(transport=GuardedTransport(), trust_env=False, **kwargs)
+
+
+def failure_reason(exc: httpx.HTTPError) -> str:
+    """Short, URL-free reason for a failed request (the URL often is the credential)."""
+    if isinstance(exc, httpx.HTTPStatusError):
+        return f"HTTP {exc.response.status_code}"
+    if isinstance(exc.__cause__, BlockedAddress):
+        return BLOCKED_MESSAGE
+    return type(exc).__name__

backend/app/services/window_service.py

@@ -0,0 +1,156 @@
+"""Occurrences of recurring maintenance windows.
+
+Rules (deliberately simple and written down, because a firmware rollout depends on them):
+
+* A window is a wall-clock start time in its own IANA time zone plus a duration. "Sunday 02:00
+  Europe/Berlin" stays 02:00 local time in summer and in winter.
+* Daylight saving: a local start time that does not exist on that day (spring forward, e.g. 02:30 on
+  the last Sunday of March in Europe/Berlin) moves FORWARD by the gap (03:30). A local time that
+  exists twice (fall back) means the FIRST of the two (fold=0, still summer time). Both follow from
+  Python's zoneinfo with fold=0 and are covered by tests.
+* The duration is real elapsed time (a 3 h window over the fall-back night still lasts 3 h).
+* Exceptions are keyed by the occurrence's local date: "skip" drops it, "move" replaces its start
+  with an absolute instant (same duration and cut-off).
+* New host actions may only START within the first ``start_cutoff_minutes`` of an occurrence
+  (``startable``); actions already running finish even after the window has closed.
+"""
+from dataclasses import dataclass
+from datetime import date, datetime, timedelta, timezone
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError, available_timezones
+
+from app.models.maintenance_window import MaintenanceWindow
+
+# How far ahead occurrences are searched. A monthly "5th ..." does not exist; the longest real gap
+# between two occurrences is a month, so two years is plenty and bounds the work per call.
+_SEARCH_DAYS = 2 * 366
+MAX_DURATION_MINUTES = 24 * 60
+
+
+@dataclass(frozen=True)
+class Occurrence:
+    local_date: date
+    start: datetime  # UTC
+    end: datetime  # UTC
+    cutoff: datetime  # UTC: no new host action after this instant
+    moved: bool = False
+    skipped: bool = False
+
+    def as_dict(self) -> dict:
+        return {"local_date": self.local_date.isoformat(), "start": self.start.isoformat(), "end": self.end.isoformat(),
+                "cutoff": self.cutoff.isoformat(), "moved": self.moved, "skipped": self.skipped}
+
+
+def valid_timezone(name: str) -> bool:
+    if not name or name not in available_timezones():
+        return False
+    try:
+        ZoneInfo(name)
+    except (ZoneInfoNotFoundError, ValueError):
+        return False
+    return True
+
+
+def _matches(window: MaintenanceWindow, day: date) -> bool:
+    if window.recurrence == "weekly":
+        return day.weekday() in (window.weekdays or [])
+    if window.recurrence == "monthly":
+        if window.month_mode == "day":
+            return day.day == window.month_day
+        if window.month_mode == "nth_weekday":
+            if day.weekday() != window.month_weekday:
+                return False
+            if window.month_nth == -1:
+                return (day + timedelta(days=7)).month != day.month
+            return (day.day - 1) // 7 + 1 == window.month_nth
+    return False
+
+
+def local_start(window: MaintenanceWindow, day: date) -> datetime:
+    """The UTC instant of the window's start on local date ``day`` (DST rules see module docstring)."""
+    tz = ZoneInfo(window.timezone)
+    wall = datetime.combine(day, window.start_time, tzinfo=tz).replace(fold=0)
+    return wall.astimezone(timezone.utc)
+
+
+def _occurrence(window: MaintenanceWindow, day: date, start: datetime, *, moved: bool = False, skipped: bool = False) -> Occurrence:
+    duration = timedelta(minutes=window.duration_minutes)
+    cutoff = timedelta(minutes=min(window.start_cutoff_minutes, window.duration_minutes))
+    return Occurrence(local_date=day, start=start, end=start + duration, cutoff=start + cutoff, moved=moved, skipped=skipped)
+
+
+def occurrences(window: MaintenanceWindow, after: datetime, n: int = 10, *, with_skipped: bool = False) -> list[Occurrence]:
+    """The next ``n`` occurrences whose END lies after ``after`` (so one that is open right now is first).
+    ``with_skipped`` lists skipped ones too (flagged, for the UI); the engine never sees them."""
+    if n <= 0:
+        return []
+    after = after.astimezone(timezone.utc)
+    tz = ZoneInfo(window.timezone)
+    exceptions = {e.occurrence_date: e for e in (window.exceptions or [])}
+    duration = timedelta(minutes=window.duration_minutes)
+    # Start one day early: an occurrence that began yesterday (local) may still be open.
+    day = (after - duration).astimezone(tz).date() - timedelta(days=1)
+    found: list[Occurrence] = []
+    # Moved occurrences can land anywhere, so they are collected separately and merged at the end.
+    for exc in exceptions.values():
+        if exc.kind == "move" and exc.moved_start is not None and _matches(window, exc.occurrence_date):
+            occ = _occurrence(window, exc.occurrence_date, exc.moved_start.astimezone(timezone.utc), moved=True)
+            if occ.end > after:
+                found.append(occ)
+    regular = 0
+    for _ in range(_SEARCH_DAYS):
+        if regular >= n:
+            break
+        if _matches(window, day):
+            exc = exceptions.get(day)
+            if exc is None or (with_skipped and exc.kind == "skip"):
+                occ = _occurrence(window, day, local_start(window, day), skipped=exc is not None)
+                if occ.end > after:
+                    found.append(occ)
+                    regular += 1
+        day += timedelta(days=1)
+    found.sort(key=lambda o: o.start)
+    return found[:n]
+
+
+def current(window: MaintenanceWindow, now: datetime) -> Occurrence | None:
+    """The occurrence open at ``now`` (start <= now < end), if any.
+
+    Occurrences can overlap (a moved one reaching into the next regular one, a 24 h window on a 25 h day). Then the
+    one that started LAST counts: its start cut-off lies latest, so "may a new host action start now" is not
+    decided by an older occurrence whose cut-off has long passed (1.29; before, the earliest one was taken)."""
+    if not window.enabled:
+        return None
+    open_now = [occ for occ in occurrences(window, now, 5) if occ.start <= now < occ.end]
+    return max(open_now, key=lambda occ: occ.start) if open_now else None
+
+
+def startable(window: MaintenanceWindow | None, now: datetime) -> Occurrence | None:
+    """The occurrence in which a NEW host action may start right now (start <= now < cutoff)."""
+    if window is None or not window.enabled:
+        return None
+    occ = current(window, now)
+    if occ is not None and occ.start <= now < occ.cutoff:
+        return occ
+    return None
+
+
+def is_occurrence_date(window: MaintenanceWindow, day: date) -> bool:
+    return _matches(window, day)
+
+
+def definition(window: MaintenanceWindow) -> dict:
+    """What a rollout plan's fingerprint covers: changing any of this re-requires activation. Skipped and moved
+    occurrences count too (1.29; keyed by the occurrence's local date, the reason text and the actor do not matter).
+    Without any exception the result is the same as before 1.29, so existing activations stay valid."""
+    result = {
+        "id": str(window.id), "timezone": window.timezone, "recurrence": window.recurrence,
+        "weekdays": sorted(window.weekdays or []), "month_mode": window.month_mode, "month_nth": window.month_nth,
+        "month_weekday": window.month_weekday, "month_day": window.month_day,
+        "start_time": window.start_time.strftime("%H:%M"), "duration_minutes": window.duration_minutes,
+        "start_cutoff_minutes": window.start_cutoff_minutes, "enabled": window.enabled,
+    }
+    if window.exceptions:
+        result["exceptions"] = sorted(
+            [e.occurrence_date.isoformat(), e.kind, e.moved_start.astimezone(timezone.utc).isoformat() if e.moved_start else None]
+            for e in window.exceptions)
+    return result

backend/app/version.py

@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
 Versioning scheme: see CHANGELOG.md ("Versionierung").
 """
 
-APP_VERSION = "1.23.7"
-APP_CODENAME = "Harrier"
+APP_VERSION = "1.30.0"
+APP_CODENAME = "Owl"

backend/migrations/versions/0020_monitoring_alerts_channels.py

@@ -0,0 +1,55 @@
+"""Alarme für Gateways, Ressourcen und Dienste; zusätzliche Benachrichtigungskanäle.
+
+Revision ID: 0020
+Revises: 0019
+Create Date: 2026-10-21
+"""
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision = "0020"
+down_revision = "0019"
+branch_labels = None
+depends_on = None
+
+_COLUMNS = [
+    ("gateway_down_enabled", "boolean NOT NULL DEFAULT true"),
+    ("resource_threshold_enabled", "boolean NOT NULL DEFAULT false"),
+    ("service_down_enabled", "boolean NOT NULL DEFAULT false"),
+    ("gateway_loss_threshold_pct", "integer NOT NULL DEFAULT 20"),
+    ("gateway_delay_threshold_ms", "integer NOT NULL DEFAULT 500"),
+    ("disk_threshold_pct", "integer NOT NULL DEFAULT 90"),
+    ("memory_threshold_pct", "integer NOT NULL DEFAULT 95"),
+    ("swap_threshold_pct", "integer NOT NULL DEFAULT 50"),
+    ("load_per_cpu_threshold", "double precision NOT NULL DEFAULT 2.0"),
+    ("alert_confirm_samples", "integer NOT NULL DEFAULT 3"),
+    ("resource_confirm_samples", "integer NOT NULL DEFAULT 5"),
+    ("service_watchlist", "json"),
+]
+
+
+def upgrade() -> None:
+    for name, ddl in _COLUMNS:
+        op.execute(f"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS {name} {ddl}")
+    if sa.inspect(op.get_bind()).has_table("notification_channels"):
+        return
+    op.create_table(
+        "notification_channels",
+        sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("name", sa.String(length=100), nullable=False),
+        sa.Column("channel_type", sa.String(length=16), nullable=False),
+        sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
+        sa.Column("config", sa.JSON(), nullable=False),
+        sa.Column("secret_set", sa.Boolean(), server_default="false", nullable=False),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("name"),
+    )
+
+
+def downgrade() -> None:
+    op.execute("DROP TABLE IF EXISTS notification_channels")
+    for name, _ddl in reversed(_COLUMNS):
+        op.execute(f"ALTER TABLE notification_settings DROP COLUMN IF EXISTS {name}")

backend/migrations/versions/0021_maintenance_windows.py

@@ -0,0 +1,60 @@
+"""Wiederkehrende Wartungsfenster mit Ausnahmen (1.25).
+
+Revision ID: 0021
+Revises: 0020
+Create Date: 2026-10-21
+"""
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision = "0021"
+down_revision = "0020"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    # A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
+    if sa.inspect(op.get_bind()).has_table("maintenance_windows"):
+        return
+    op.create_table(
+        "maintenance_windows",
+        sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("name", sa.String(length=120), nullable=False),
+        sa.Column("timezone", sa.String(length=64), nullable=False),
+        sa.Column("recurrence", sa.String(length=16), nullable=False),
+        sa.Column("weekdays", sa.JSON(), nullable=False),
+        sa.Column("month_mode", sa.String(length=16), nullable=True),
+        sa.Column("month_nth", sa.Integer(), nullable=True),
+        sa.Column("month_weekday", sa.Integer(), nullable=True),
+        sa.Column("month_day", sa.Integer(), nullable=True),
+        sa.Column("start_time", sa.Time(), nullable=False),
+        sa.Column("duration_minutes", sa.Integer(), nullable=False),
+        sa.Column("start_cutoff_minutes", sa.Integer(), server_default="60", nullable=False),
+        sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
+        sa.Column("created_by", sa.String(length=150), nullable=False),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("name"),
+    )
+    op.create_table(
+        "maintenance_window_exceptions",
+        sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("window_id", postgresql.UUID(as_uuid=True), nullable=False),
+        sa.Column("occurrence_date", sa.Date(), nullable=False),
+        sa.Column("kind", sa.String(length=8), nullable=False),
+        sa.Column("moved_start", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("reason", sa.String(length=300), nullable=False),
+        sa.Column("actor", sa.String(length=150), nullable=False),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
+        sa.ForeignKeyConstraint(["window_id"], ["maintenance_windows.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("window_id", "occurrence_date", name="uq_window_exception_date"),
+    )
+    op.create_index("ix_maintenance_window_exceptions_window_id", "maintenance_window_exceptions", ["window_id"])
+
+
+def downgrade() -> None:
+    op.execute("DROP TABLE IF EXISTS maintenance_window_exceptions")
+    op.execute("DROP TABLE IF EXISTS maintenance_windows")

backend/migrations/versions/0022_scheduled_auto_verify.py

@@ -0,0 +1,22 @@
+"""Geplante Wartung: automatische Prüfung nach dem Auftrag (auto_verify) und Herkunft (manuell/Rollout).
+
+Revision ID: 0022
+Revises: 0021
+Create Date: 2026-10-21
+"""
+from alembic import op
+
+revision = "0022"
+down_revision = "0021"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS auto_verify boolean NOT NULL DEFAULT false")
+    op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS origin varchar(16) NOT NULL DEFAULT 'manual'")
+
+
+def downgrade() -> None:
+    op.execute("ALTER TABLE scheduled_maintenance DROP COLUMN IF EXISTS origin")
+    op.execute("ALTER TABLE scheduled_maintenance DROP COLUMN IF EXISTS auto_verify")

backend/migrations/versions/0023_rollouts.py

@@ -0,0 +1,143 @@
+"""Firmware-Rollouts in Wellen: Pläne, Läufe, Hosts je Lauf; Benachrichtigung bei Rollout-Problemen (1.25).
+
+Revision ID: 0023
+Revises: 0022
+Create Date: 2026-10-21
+"""
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision = "0023"
+down_revision = "0022"
+branch_labels = None
+depends_on = None
+
+UUID = postgresql.UUID(as_uuid=True)
+TS = sa.DateTime(timezone=True)
+
+
+def upgrade() -> None:
+    op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS rollout_problems_enabled boolean NOT NULL DEFAULT true")
+    # A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
+    if sa.inspect(op.get_bind()).has_table("rollout_plans"):
+        return
+    op.create_table(
+        "rollout_plans",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("name", sa.String(120), nullable=False),
+        sa.Column("description", sa.Text(), nullable=False),
+        sa.Column("action", sa.String(16), nullable=False),
+        sa.Column("window_id", UUID, nullable=True),
+        sa.Column("auto_start", sa.Boolean(), server_default="false", nullable=False),
+        sa.Column("gate_checks", sa.JSON(), nullable=False),
+        sa.Column("compliance_mode", sa.String(8), server_default="warn", nullable=False),
+        sa.Column("activated_by", sa.String(150), nullable=True),
+        sa.Column("activated_at", TS, nullable=True),
+        sa.Column("activated_fingerprint", sa.String(64), nullable=True),
+        sa.Column("auto_start_error", sa.JSON(), nullable=True),
+        sa.Column("created_by", sa.String(150), nullable=False),
+        sa.Column("created_at", TS, nullable=False),
+        sa.Column("updated_at", TS, nullable=False),
+        sa.ForeignKeyConstraint(["window_id"], ["maintenance_windows.id"], ondelete="SET NULL"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("name"),
+    )
+    op.create_index("ix_rollout_plans_window_id", "rollout_plans", ["window_id"])
+    op.create_table(
+        "rollout_plan_waves",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("plan_id", UUID, nullable=False),
+        sa.Column("position", sa.Integer(), nullable=False),
+        sa.Column("label", sa.String(60), nullable=False),
+        sa.Column("soak_minutes", sa.Integer(), nullable=False),
+        sa.Column("carp_handover", sa.Boolean(), server_default="false", nullable=False),
+        sa.Column("require_approval", sa.Boolean(), server_default="false", nullable=False),
+        sa.ForeignKeyConstraint(["plan_id"], ["rollout_plans.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index("ix_rollout_plan_waves_plan_id", "rollout_plan_waves", ["plan_id"])
+    op.create_table(
+        "rollout_wave_groups",
+        sa.Column("wave_id", UUID, nullable=False),
+        sa.Column("group_id", UUID, nullable=False),
+        sa.ForeignKeyConstraint(["wave_id"], ["rollout_plan_waves.id"], ondelete="CASCADE"),
+        sa.ForeignKeyConstraint(["group_id"], ["device_groups.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("wave_id", "group_id"),
+    )
+    op.create_table(
+        "rollout_runs",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("plan_id", UUID, nullable=True),
+        sa.Column("plan_name", sa.String(120), nullable=False),
+        sa.Column("plan_snapshot", sa.JSON(), nullable=False),
+        sa.Column("action", sa.String(16), nullable=False),
+        sa.Column("status", sa.String(20), nullable=False),
+        sa.Column("paused_from", sa.String(20), nullable=True),
+        sa.Column("current_wave", sa.Integer(), nullable=False),
+        sa.Column("target_version", sa.String(40), nullable=True),
+        sa.Column("pause_reason", sa.String(40), nullable=True),
+        sa.Column("pause_detail", sa.JSON(), nullable=True),
+        sa.Column("started_by", sa.String(150), nullable=False),
+        sa.Column("trigger", sa.String(8), nullable=False),
+        sa.Column("window_occurrence", TS, nullable=True),
+        sa.Column("revision", sa.Integer(), server_default="0", nullable=False),
+        sa.Column("progress", sa.JSON(), nullable=False),
+        sa.Column("step_at", TS, nullable=True),
+        sa.Column("created_at", TS, nullable=False),
+        sa.Column("finished_at", TS, nullable=True),
+        sa.ForeignKeyConstraint(["plan_id"], ["rollout_plans.id"], ondelete="SET NULL"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("plan_id", "window_occurrence", name="uq_rollout_run_occurrence"),
+    )
+    op.create_index("ix_rollout_runs_plan_id", "rollout_runs", ["plan_id"])
+    op.create_index("ix_rollout_runs_status", "rollout_runs", ["status"])
+    op.create_table(
+        "rollout_run_waves",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("run_id", UUID, nullable=False),
+        sa.Column("index", sa.Integer(), nullable=False),
+        sa.Column("label", sa.String(60), nullable=False),
+        sa.Column("status", sa.String(20), nullable=False),
+        sa.Column("baseline", sa.JSON(), nullable=True),
+        sa.Column("gate_result", sa.JSON(), nullable=True),
+        sa.Column("soak_until", TS, nullable=True),
+        sa.Column("gate_attempts", sa.Integer(), server_default="0", nullable=False),
+        sa.Column("approved_by", sa.String(150), nullable=True),
+        sa.Column("started_at", TS, nullable=True),
+        sa.Column("finished_at", TS, nullable=True),
+        sa.ForeignKeyConstraint(["run_id"], ["rollout_runs.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index("ix_rollout_run_waves_run_id", "rollout_run_waves", ["run_id"])
+    op.create_table(
+        "rollout_run_hosts",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("run_id", UUID, nullable=False),
+        sa.Column("wave_index", sa.Integer(), nullable=False),
+        sa.Column("host_id", UUID, nullable=True),
+        sa.Column("host_name", sa.String(120), nullable=False),
+        sa.Column("cluster_name", sa.String(120), nullable=True),
+        sa.Column("order", sa.Integer(), nullable=False),
+        sa.Column("status", sa.String(16), nullable=False),
+        sa.Column("scheduled_maintenance_id", UUID, nullable=True),
+        sa.Column("version_before", sa.String(40), nullable=True),
+        sa.Column("version_after", sa.String(40), nullable=True),
+        sa.Column("detail", sa.JSON(), nullable=True),
+        sa.Column("started_at", TS, nullable=True),
+        sa.Column("finished_at", TS, nullable=True),
+        sa.ForeignKeyConstraint(["run_id"], ["rollout_runs.id"], ondelete="CASCADE"),
+        sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="SET NULL"),
+        sa.ForeignKeyConstraint(["scheduled_maintenance_id"], ["scheduled_maintenance.id"], ondelete="SET NULL"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("run_id", "host_id", name="uq_rollout_run_host"),
+    )
+    op.create_index("ix_rollout_run_hosts_run_id", "rollout_run_hosts", ["run_id"])
+    op.create_index("ix_rollout_run_hosts_host_id", "rollout_run_hosts", ["host_id"])
+    op.create_index("ix_rollout_run_hosts_scheduled_maintenance_id", "rollout_run_hosts", ["scheduled_maintenance_id"])
+
+
+def downgrade() -> None:
+    for table in ("rollout_run_hosts", "rollout_run_waves", "rollout_runs", "rollout_wave_groups", "rollout_plan_waves", "rollout_plans"):
+        op.execute(f"DROP TABLE IF EXISTS {table}")
+    op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS rollout_problems_enabled")

backend/migrations/versions/0024_gui_access.py

@@ -0,0 +1,78 @@
+"""GUI-Zugriff über das gui-gateway (1.27): Einstellungen, Einmal-Tickets, Gateway-Sitzungen.
+
+Revision ID: 0024
+Revises: 0023
+Create Date: 2026-10-21
+"""
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision = "0024"
+down_revision = "0023"
+branch_labels = None
+depends_on = None
+
+UUID = postgresql.UUID(as_uuid=True)
+TS = sa.DateTime(timezone=True)
+
+
+def upgrade() -> None:
+    # A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
+    if sa.inspect(op.get_bind()).has_table("gui_access_settings"):
+        return
+    op.create_table(
+        "gui_access_settings",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("enabled", sa.Boolean(), server_default="false", nullable=False),
+        sa.Column("idle_timeout_minutes", sa.Integer(), server_default="15", nullable=False),
+        sa.Column("absolute_timeout_minutes", sa.Integer(), server_default="60", nullable=False),
+        sa.Column("slot_count", sa.Integer(), server_default="4", nullable=False),
+        sa.Column("updated_by", sa.String(150), nullable=True),
+        sa.Column("updated_at", TS, nullable=True),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_table(
+        "gui_tickets",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("ticket_hash", sa.String(64), nullable=False),
+        sa.Column("session_hash", sa.String(64), nullable=False),
+        sa.Column("user_id", UUID, nullable=False),
+        sa.Column("host_id", UUID, nullable=False),
+        sa.Column("slot", sa.Integer(), nullable=False),
+        sa.Column("created_at", TS, nullable=False),
+        sa.Column("expires_at", TS, nullable=False),
+        sa.Column("redeemed_at", TS, nullable=True),
+        sa.ForeignKeyConstraint(["user_id"], ["admin_users.id"], ondelete="CASCADE"),
+        sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("ticket_hash"),
+    )
+    op.create_index("ix_gui_tickets_session_hash", "gui_tickets", ["session_hash"])
+    op.create_index("ix_gui_tickets_user_id", "gui_tickets", ["user_id"])
+    op.create_index("ix_gui_tickets_host_id", "gui_tickets", ["host_id"])
+    op.create_table(
+        "gui_sessions",
+        sa.Column("id", UUID, nullable=False),
+        sa.Column("session_hash", sa.String(64), nullable=False),
+        sa.Column("user_id", UUID, nullable=False),
+        sa.Column("host_id", UUID, nullable=False),
+        sa.Column("slot", sa.Integer(), nullable=False),
+        sa.Column("client_ip", sa.String(45), nullable=True),
+        sa.Column("started_at", TS, nullable=False),
+        sa.Column("last_checked_at", TS, nullable=False),
+        sa.Column("ended_at", TS, nullable=True),
+        sa.Column("end_reason", sa.String(32), nullable=True),
+        sa.ForeignKeyConstraint(["user_id"], ["admin_users.id"], ondelete="CASCADE"),
+        sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index("ix_gui_sessions_session_hash", "gui_sessions", ["session_hash"])
+    op.create_index("ix_gui_sessions_user_id", "gui_sessions", ["user_id"])
+    op.create_index("ix_gui_sessions_host_id", "gui_sessions", ["host_id"])
+
+
+def downgrade() -> None:
+    op.drop_table("gui_sessions")
+    op.drop_table("gui_tickets")
+    op.drop_table("gui_access_settings")

backend/migrations/versions/0025_api_rights_gateway_mute.py

@@ -0,0 +1,31 @@
+"""Alarm bei fehlenden API-Rechten und pro Gateway abschaltbarer Gateway-Alarm.
+
+Revision ID: 0025
+Revises: 0024
+Create Date: 2026-10-21
+"""
+from alembic import op
+
+revision = "0025"
+down_revision = "0024"
+branch_labels = None
+depends_on = None
+
+_COLUMNS = [
+    ("api_rights_enabled", "boolean NOT NULL DEFAULT true"),
+    ("gateway_alert_muted", "json NOT NULL DEFAULT '[]'::json"),
+]
+
+
+def upgrade() -> None:
+    # IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have the columns.
+    for name, ddl in _COLUMNS:
+        op.execute(f"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS {name} {ddl}")
+    # The list default only fills existing rows; the model supplies it for new ones (and Alembic's schema
+    # comparison cannot compare a JSON server default).
+    op.execute("ALTER TABLE notification_settings ALTER COLUMN gateway_alert_muted DROP DEFAULT")
+
+
+def downgrade() -> None:
+    for name, _ddl in reversed(_COLUMNS):
+        op.execute(f"ALTER TABLE notification_settings DROP COLUMN IF EXISTS {name}")

backend/migrations/versions/0026_rollout_actor_ids.py

@@ -0,0 +1,33 @@
+"""Rollouts: die verantwortliche Person zusätzlich als stabile Benutzer-ID (1.29).
+
+Bisher stand nur der Benutzername im Plan (``activated_by``) und im Lauf (``started_by``). Wird ein Konto gelöscht
+und ein neues mit gleichem Namen angelegt, hätte der automatische Start unter der Rechten der neuen Person
+weitergelaufen. Die ID ändert sich nie. Vorhandene Zeilen werden über den Benutzernamen zugeordnet (soweit das Konto
+noch existiert); ohne Treffer bleibt die ID leer und der Name gilt wie bisher.
+
+Revision ID: 0026
+Revises: 0025
+Create Date: 2026-10-22
+"""
+from alembic import op
+
+revision = "0026"
+down_revision = "0025"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    # IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have the columns.
+    op.execute("ALTER TABLE rollout_plans ADD COLUMN IF NOT EXISTS activated_by_id uuid")
+    op.execute("ALTER TABLE rollout_runs ADD COLUMN IF NOT EXISTS started_by_id uuid")
+    # No foreign key on purpose: a deleted account must stay recognisably gone (SET NULL would fall back to the name).
+    op.execute("UPDATE rollout_plans p SET activated_by_id = u.id FROM admin_users u "
+               "WHERE p.activated_by_id IS NULL AND p.activated_by IS NOT NULL AND u.username = p.activated_by")
+    op.execute("UPDATE rollout_runs r SET started_by_id = u.id FROM admin_users u "
+               "WHERE r.started_by_id IS NULL AND u.username = r.started_by")
+
+
+def downgrade() -> None:
+    op.execute("ALTER TABLE rollout_runs DROP COLUMN IF EXISTS started_by_id")
+    op.execute("ALTER TABLE rollout_plans DROP COLUMN IF EXISTS activated_by_id")

backend/migrations/versions/0027_rule_history.py

@@ -0,0 +1,62 @@
+"""Änderungsverlauf der Regeln auf den Firewalls (1.30): letzter beobachteter Stand und Änderungsereignisse.
+
+Revision ID: 0027
+Revises: 0026
+Create Date: 2026-10-22
+"""
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects import postgresql
+
+revision = "0027"
+down_revision = "0026"
+branch_labels = None
+depends_on = None
+
+UUID = postgresql.UUID(as_uuid=True)
+TS = sa.DateTime(timezone=True)
+
+
+def upgrade() -> None:
+    # A pre-Alembic install is bridged with create_all() first (app/migrate.py), which already made the tables.
+    inspector = sa.inspect(op.get_bind())
+    if not inspector.has_table("live_rule_states"):
+        op.create_table(
+            "live_rule_states",
+            sa.Column("host_id", UUID, nullable=False),
+            sa.Column("rule_uuid", sa.String(64), nullable=False),
+            sa.Column("fingerprint", sa.String(64), nullable=False),
+            sa.Column("row", sa.JSON(), nullable=False),
+            sa.Column("present", sa.Boolean(), nullable=False),
+            sa.Column("version", sa.Integer(), nullable=False),
+            sa.Column("first_seen_at", TS, nullable=False),
+            sa.Column("changed_at", TS, nullable=False),
+            sa.Column("last_seen_at", TS, nullable=False),
+            sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
+            sa.PrimaryKeyConstraint("host_id", "rule_uuid"),
+        )
+    if not inspector.has_table("rule_change_events"):
+        op.create_table(
+            "rule_change_events",
+            sa.Column("id", UUID, nullable=False),
+            sa.Column("host_id", UUID, nullable=False),
+            sa.Column("rule_uuid", sa.String(64), nullable=False),
+            sa.Column("seq", sa.Integer(), nullable=False),
+            sa.Column("observed_at", TS, nullable=False),
+            sa.Column("kind", sa.String(16), nullable=False),
+            sa.Column("before", sa.JSON(), nullable=True),
+            sa.Column("after", sa.JSON(), nullable=True),
+            sa.Column("origin", sa.String(16), nullable=False),
+            sa.Column("actor", sa.String(150), nullable=True),
+            sa.Column("audit_id", UUID, nullable=True),
+            sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
+            sa.PrimaryKeyConstraint("id"),
+            sa.UniqueConstraint("host_id", "rule_uuid", "seq", name="uq_rule_change_event"),
+        )
+        op.create_index("ix_rule_change_events_observed_at", "rule_change_events", ["observed_at"])
+        op.create_index("ix_rule_change_events_rule", "rule_change_events", ["host_id", "rule_uuid", "observed_at"])
+
+
+def downgrade() -> None:
+    op.execute("DROP TABLE IF EXISTS rule_change_events")
+    op.execute("DROP TABLE IF EXISTS live_rule_states")

backend/migrations/versions/0028_rule_event_window.py

@@ -0,0 +1,21 @@
+"""Änderungsverlauf: Fensteranfang je Ereignis für die Herkunftsbestimmung (1.30).
+
+Revision ID: 0028
+Revises: 0027
+Create Date: 2026-10-22
+"""
+from alembic import op
+
+revision = "0028"
+down_revision = "0027"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    # IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have the column.
+    op.execute("ALTER TABLE rule_change_events ADD COLUMN IF NOT EXISTS window_start timestamptz")
+
+
+def downgrade() -> None:
+    op.execute("ALTER TABLE rule_change_events DROP COLUMN IF EXISTS window_start")

backend/requirements.txt

@@ -13,3 +13,4 @@ respx==0.23.1
 cryptography==50.0.2
 alembic==1.20.0
 PyJWT==2.10.1
+tzdata==2025.2

backend/tests/fixtures/rule_stats_live.json

@@ -0,0 +1,116 @@
+{
+ "_comment": "Anonymized from live OPNsense 26.7 test firewalls (2026-10-22): firewall/filter_util/ruleStats (3 automatic labels, 3 MVC rules, 1 unmigrated legacy rule with a 32-hex label; counters as read) and the matching searchRule row of the legacy rule. No addresses involved.",
+ "ruleStats": {
+  "status": "ok",
+  "stats": {
+   "02f4bab031b57d1e30553ce08e0ec131": {
+    "pf_rules": 2,
+    "evaluations": 17734,
+    "packets": 250,
+    "bytes": 28670,
+    "states": 0
+   },
+   "7355b5da99f40893221b6c4140de1006": {
+    "pf_rules": 6,
+    "evaluations": 860,
+    "packets": 0,
+    "bytes": 0,
+    "states": 0
+   },
+   "80f90f0d4c5aba0c28d1c539e1e35766": {
+    "pf_rules": 12,
+    "evaluations": 1219,
+    "packets": 0,
+    "bytes": 0,
+    "states": 0
+   },
+   "fccf1b9d-2461-43d6-938f-2c3c39b3ddee": {
+    "pf_rules": 1,
+    "evaluations": 171,
+    "packets": 458,
+    "bytes": 35302,
+    "states": 0
+   },
+   "920a850f-68c3-42f6-a5a4-5d06984e417b": {
+    "pf_rules": 2,
+    "evaluations": 8,
+    "packets": 4,
+    "bytes": 756,
+    "states": 0
+   },
+   "2ec1c6db-61e1-4476-b3f6-0599a28990a2": {
+    "pf_rules": 1,
+    "evaluations": 9625,
+    "packets": 234816,
+    "bytes": 99582131,
+    "states": 40
+   },
+   "db610dbc7c6467261cf08e8a72ade096": {
+    "pf_rules": 1,
+    "evaluations": 4712,
+    "packets": 110719,
+    "bytes": 55995167,
+    "states": 24
+   }
+  }
+ },
+ "legacy_row": {
+  "@attributes": {
+   "uuid": "a00d53d2-8107-4d87-a71d-53ac85c1e9da"
+  },
+  "interface": "WAN",
+  "ipprotocol": "inet",
+  "statetype": "keep state",
+  "direction": "in",
+  "quick": "1",
+  "updated": {
+   "username": "[email protected]",
+   "time": "1791536720.85",
+   "description": "/firewall_rules_edit.php made changes"
+  },
+  "created": {
+   "username": "[email protected]",
+   "time": "1791536720.85",
+   "description": "/firewall_rules_edit.php made changes"
+  },
+  "seq": 0,
+  "#priority": 400000,
+  "enabled": "1",
+  "action": "pass",
+  "replyto": "",
+  "description": "ALLOW ANY",
+  "source_net": "any",
+  "source_port": "",
+  "destination_net": "any",
+  "destination_port": "",
+  "ref": "ui/firewall/migration",
+  "uuid": "db610dbc7c6467261cf08e8a72ade096",
+  "source_not": "0",
+  "destination_not": "0",
+  "%direction": "In",
+  "%action": "Pass",
+  "%ipprotocol": "IPv4",
+  "protocol": "",
+  "%protocol": "",
+  "sort_order": "400000.1000023",
+  "legacy": true,
+  "is_automatic": false,
+  "category_colors": [],
+  "alias_meta_source_net": [
+   {
+    "value": "any",
+    "%value": "any",
+    "isAlias": false,
+    "description": ""
+   }
+  ],
+  "alias_meta_destination_net": [
+   {
+    "value": "any",
+    "%value": "any",
+    "isAlias": false,
+    "description": ""
+   }
+  ]
+ }
+}

backend/tests/mutation_check.py

@@ -0,0 +1,39 @@
+"""Mutation check helper (run manually, not collected by pytest).
+
+    python tests/mutation_check.py tests/mutations_gui.json
+
+Each mutation: {"file": ..., "old": ..., "new": ..., "tests": "pytest selection"}.
+The file is patched, the selected tests must FAIL, then the file is restored.
+"""
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+
+def main(spec_path: str) -> int:
+    mutations = json.loads(Path(spec_path).read_text())
+    survivors = []
+    for i, m in enumerate(mutations, 1):
+        path = Path(m["file"])
+        original = path.read_text()
+        if original.count(m["old"]) != 1:
+            print(f"[{i}] SKIP (anchor not unique/found): {m['file']}: {m['old'][:60]!r}")
+            survivors.append(m)
+            continue
+        try:
+            path.write_text(original.replace(m["old"], m["new"]))
+            result = subprocess.run([sys.executable, "-m", "pytest", "-q", "-x", "-p", "no:cacheprovider", *m["tests"].split()],
+                                    capture_output=True, text=True)
+        finally:
+            path.write_text(original)
+        killed = result.returncode != 0
+        print(f"[{i}] {'KILLED ' if killed else 'SURVIVED'} {m.get('name', m['old'][:50])}")
+        if not killed:
+            survivors.append(m)
+    print(f"{len(mutations) - len(survivors)}/{len(mutations)} mutations killed")
+    return 1 if survivors else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1]))

backend/tests/mutations_gateway.json

@@ -0,0 +1,226 @@
+[
+ {
+  "name": "GW browser Cookie forwarded",
+  "file": "app/gui_gateway.py",
+  "old": "    out_headers = {k: v for k, v in headers.items() if k in _FORWARD_REQUEST_HEADERS}",
+  "new": "    out_headers = {k: v for k, v in headers.items() if k in _FORWARD_REQUEST_HEADERS or k == \"cookie\"}",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW Referer/Authorization forwarded",
+  "file": "app/gui_gateway.py",
+  "old": "\"accept\", \"accept-language\", \"content-type\", \"user-agent\", \"x-requested-with\", \"x-csrftoken\", \"range\",",
+  "new": "\"accept\", \"accept-language\", \"content-type\", \"user-agent\", \"x-requested-with\", \"x-csrftoken\", \"range\", \"referer\", \"authorization\",",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW Set-Cookie passed to browser",
+  "file": "app/gui_gateway.py",
+  "old": "\"content-range\", \"accept-ranges\", \"content-security-policy\",",
+  "new": "\"content-range\", \"accept-ranges\", \"content-security-policy\", \"set-cookie\",",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW jar never forgets deleted cookies",
+  "file": "app/gui_gateway.py",
+  "old": "                if expired:\n                    gs.jar.pop(name, None)",
+  "new": "                if False:\n                    gs.jar.pop(name, None)",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW foreign Location kept",
+  "file": "app/gui_gateway.py",
+  "old": "        if _origin_of(location) != _origin_of(upstream_origin):\n            return None",
+  "new": "        pass",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW backslash Location kept",
+  "file": "app/gui_gateway.py",
+  "old": "    if not location or \"\\\\\" in location",
+  "new": "    if not location",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW Sec-Fetch-Site lax",
+  "file": "app/gui_gateway.py",
+  "old": "        if site not in (\"same-origin\", \"none\"):",
+  "new": "        if site in (\"cross-site\",):",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW redeem Sec-Fetch-Site unchecked",
+  "file": "app/gui_gateway.py",
+  "old": "            if method != \"GET\" or site not in (\"same-site\", \"same-origin\", \"none\"):",
+  "new": "            if method != \"GET\":",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW redeem without OPNexus cookie",
+  "file": "app/gui_gateway.py",
+  "old": "        if not ticket or len(ticket) > 128 or not browser_session:",
+  "new": "        if not ticket or len(ticket) > 128:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW Origin unchecked for writes",
+  "file": "app/gui_gateway.py",
+  "old": "            if _origin_of(origin) not in own_origins:",
+  "new": "            if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW absolute-form accepted",
+  "file": "app/gui_gateway.py",
+  "old": "        if not path.startswith(\"/\") or path.startswith(\"//\") or not raw_path.startswith(b\"/\") \\",
+  "new": "        if False \\",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW CONNECT accepted",
+  "file": "app/gui_gateway.py",
+  "old": "        if method in (\"CONNECT\", \"TRACE\"):",
+  "new": "        if method in (\"TRACE\",):",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW Upgrade accepted",
+  "file": "app/gui_gateway.py",
+  "old": "        if \"upgrade\" in headers or \"upgrade\" in headers.get(\"connection\", \"\").lower():",
+  "new": "        if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW service worker accepted",
+  "file": "app/gui_gateway.py",
+  "old": "        if headers.get(\"service-worker\", \"\").lower() == \"script\":",
+  "new": "        if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW slot header optional",
+  "file": "app/gui_gateway.py",
+  "old": "        if not own_origins:\n            return await self._respond(send, 400, _FORBIDDEN)",
+  "new": "        if False:\n            return await self._respond(send, 400, _FORBIDDEN)",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW foreign slot cookie accepted",
+  "file": "app/gui_gateway.py",
+  "old": "        if gs is None or gs.slot != slot:",
+  "new": "        if gs is None:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW login rate limit off",
+  "file": "app/gui_gateway.py",
+  "old": "        if login and self.limiter.blocked(gs.upstream_origin):",
+  "new": "        if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW login failures not counted",
+  "file": "app/gui_gateway.py",
+  "old": "            if login and not (300 <= response.status_code < 400):",
+  "new": "            if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW no periodic re-check",
+  "file": "app/gui_gateway.py",
+  "old": "        if self.clock() >= gs.next_check:\n            state = await self._recheck(key, gs)",
+  "new": "        if False:\n            state = await self._recheck(key, gs)",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW fail-open when backend down",
+  "file": "app/gui_gateway.py",
+  "old": "            if state is None:\n                return await self._respond(send, 503, _UNAVAILABLE)",
+  "new": "",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW idle timeout off",
+  "file": "app/gui_gateway.py",
+  "old": "        if self.clock() - gs.last_activity >= gs.idle_timeout:",
+  "new": "        if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW absolute timeout off",
+  "file": "app/gui_gateway.py",
+  "old": "        if self.wall_clock() >= gs.absolute_deadline:",
+  "new": "        if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW no CSP",
+  "file": "app/gui_gateway.py",
+  "old": "    (b\"content-security-policy\", _GATEWAY_CSP.encode()),",
+  "new": "",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW no X-Frame-Options",
+  "file": "app/gui_gateway.py",
+  "old": "    (b\"x-frame-options\", b\"DENY\"),",
+  "new": "",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW no no-store",
+  "file": "app/gui_gateway.py",
+  "old": "    (b\"cache-control\", b\"no-store\"),",
+  "new": "",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW streaming body limit off",
+  "file": "app/gui_gateway.py",
+  "old": "            if size > BODY_LIMIT:\n                return None",
+  "new": "",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW query logged",
+  "file": "app/gui_gateway.py",
+  "old": "scope.get(\"path\", \"?\")[:200], status,",
+  "new": "scope.get(\"path\", \"?\")[:200] + \"?\" + scope.get(\"query_string\", b\"\").decode(), status,",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW httpx URL logging on",
+  "file": "app/gui_gateway.py",
+  "old": "    logging.getLogger(_noisy).setLevel(logging.WARNING)",
+  "new": "    pass",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW weak gateway cookie",
+  "file": "app/gui_gateway.py",
+  "old": "={value}; Path=/; Secure; HttpOnly; SameSite=Strict\"",
+  "new": "={value}; Path=/\"",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW no Clear-Site-Data",
+  "file": "app/gui_gateway.py",
+  "old": "            (b\"set-cookie\", cookie.encode()), (b\"clear-site-data\", b'\"cache\", \"storage\"'),",
+  "new": "            (b\"set-cookie\", cookie.encode()),",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW pin not used",
+  "file": "app/gui_gateway.py",
+  "old": "        verify = pinned_context(gs.ca_pem)",
+  "new": "        verify = False",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "GW access log on",
+  "file": "app/gui_gateway.py",
+  "old": "http=\"h11\", ws=\"none\", access_log=False,",
+  "new": "http=\"h11\", ws=\"none\", access_log=True,",
+  "tests": "tests/test_gui_gateway.py"
+ }
+]

backend/tests/mutations_gui.json

@@ -0,0 +1,170 @@
+[
+ {
+  "name": "AP1 sec-fetch: cross-site not blocked",
+  "file": "app/security_middleware.py",
+  "old": "    if site == \"cross-site\":\n        return True",
+  "new": "    if site == \"cross-site\":\n        return False",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP1 sec-fetch: same-site always allowed",
+  "file": "app/security_middleware.py",
+  "old": "        return not _explicit_frontend_origin(request.headers.get(\"origin\", \"\"))",
+  "new": "        return False",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP1 sec-fetch: check not wired",
+  "file": "app/security_middleware.py",
+  "old": "        if sec_fetch_blocked(request):",
+  "new": "        if False:",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP1 cookie: plain name falls back even with __Host-",
+  "file": "app/routers/auth.py",
+  "old": "        return request.cookies.get(SECURE_COOKIE_NAME) or request.cookies.get(COOKIE_NAME, \"\")",
+  "new": "        return request.cookies.get(COOKIE_NAME) or request.cookies.get(SECURE_COOKIE_NAME, \"\")",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP1 cookie: no __Host- prefix",
+  "file": "app/routers/auth.py",
+  "old": "    return SECURE_COOKIE_NAME if settings.cookie_secure else COOKIE_NAME",
+  "new": "    return COOKIE_NAME",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP1 startup: conflict check removed",
+  "file": "app/main.py",
+  "old": "    if conflicts:\n",
+  "new": "    if False:\n",
+  "tests": "tests/test_session_hardening.py"
+ },
+ {
+  "name": "AP2 ticket expiry ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    elif ticket.expires_at <= now:",
+  "new": "    elif False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 replay allowed",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if replayed:\n        reason = \"replayed\"",
+  "new": "    if False:\n        reason = \"replayed\"",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 session binding ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    elif not secrets.compare_digest(ticket.session_hash, session_hash):",
+  "new": "    elif False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 slot binding ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    elif ticket.slot != slot:",
+  "new": "    elif False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 revalidate: role not rechecked",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if user is None or user.role != \"admin\":",
+  "new": "    if user is None:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 revalidate: session not rechecked",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if session is None or session.expires_at <= _now() or session.user_id != user_id:",
+  "new": "    if False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 revalidate: switch not rechecked",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if not row.enabled:\n        return \"disabled\", user, None",
+  "new": "",
+  "tests": "tests/test_gui_access.py::test_revalidate_fails_when_switched_off"
+ },
+ {
+  "name": "AP2 absolute limit ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if _now() >= gs.started_at + timedelta(minutes=row.absolute_timeout_minutes):",
+  "new": "    if False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 host tls requirement dropped",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if not host.verify_tls:\n        return \"tls\"",
+  "new": "",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 slot limit ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    if not free:\n        raise",
+  "new": "    if False:\n        raise",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 gateway token not compared",
+  "file": "app/routers/gui_access.py",
+  "old": "    if not hmac.compare_digest(",
+  "new": "    if False and not hmac.compare_digest(",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 switch not checked for tickets",
+  "file": "app/routers/gui_access.py",
+  "old": "    if not row.enabled:\n        raise api_error(status.HTTP_409_CONFLICT, ErrorCode.GUI_ACCESS_DISABLED)",
+  "new": "",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 legacy cookie accepted for tickets",
+  "file": "app/routers/gui_access.py",
+  "old": "    session_cookie = request.cookies.get(SECURE_COOKIE_NAME, \"\")",
+  "new": "    session_cookie = request.cookies.get(SECURE_COOKIE_NAME) or request.cookies.get(\"opnexus-session\", \"\")",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 enabling without https allowed",
+  "file": "app/routers/gui_access.py",
+  "old": "    if payload.enabled and svc.unavailable_reasons():",
+  "new": "    if False:",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP2 redeem leaks api key",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "        \"upstream_origin\": firewall_origin(host.base_url),",
+  "new": "        \"upstream_origin\": firewall_origin(host.base_url), \"k\": host.api_key,",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "AP7 revalidation interval back to 30 s (switch-off took 31 s live)",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "REVALIDATE_SECONDS = 20",
+  "new": "REVALIDATE_SECONDS = 30",
+  "tests": "tests/test_gui_access.py"
+ },
+ {
+  "name": "Review: multipart login not counted",
+  "file": "app/gui_gateway.py",
+  "old": "    if content_type.lower().startswith(\"multipart/form-data\"):",
+  "new": "    if False:",
+  "tests": "tests/test_gui_gateway.py"
+ },
+ {
+  "name": "Review: decoded path forwarded upstream",
+  "file": "app/gui_gateway.py",
+  "old": "url = gs.upstream_origin + raw_path +",
+  "new": "url = gs.upstream_origin + scope[\"path\"] +",
+  "tests": "tests/test_gui_gateway.py"
+ }
+]

backend/tests/mutations_security_1_29.json

@@ -0,0 +1,303 @@
+[
+ {
+  "name": "S1 cooldown ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "    usable = [s for s in free if s not in cooling]",
+  "new": "    usable = list(free)",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S1 cooldown window zero",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "            | (GuiSession.ended_at > now - SLOT_COOLDOWN)))).all()",
+  "new": "            | (GuiSession.ended_at > now)))).all()",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S1 same firewall blocked too",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "        if h_id == host_id or slot not in free:",
+  "new": "        if slot not in free:",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S1 open sessions of other browser ignored",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "((GuiSession.ended_at.is_(None)) & (GuiSession.started_at > now - absolute))\n            | ",
+  "new": "(GuiSession.ended_at.is_(None) & (GuiSession.started_at > now))\n            | ",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S1 cooldown not per user",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "            GuiSession.user_id == user_id,\n            ((GuiSession",
+  "new": "            ((GuiSession",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 fingerprint not compared",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "and not secrets.compare_digest(target_fp, target_fingerprint(host)):",
+  "new": "and False:",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 fingerprint ignores CA",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "host.tls_ca_pem or \"\", overlay",
+  "new": "\"\", overlay",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 fingerprint ignores overlay",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "overlay_service.proxy_for(host.base_url) or \"\",\n",
+  "new": "\"\",\n",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 fingerprint ignores origin",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "firewall_origin(host.base_url), host.tls_ca_pem",
+  "new": "\"x\", host.tls_ca_pem",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 gateway forgets fingerprint",
+  "file": "app/gui_gateway.py",
+  "old": "\"target_fp\": gs.target_fp})",
+  "new": "\"target_fp\": None})",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 gateway evicts again",
+  "file": "app/gui_gateway.py",
+  "old": "        return len(self.sessions) - replaced < MAX_SESSIONS",
+  "new": "        return True",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 backend cap off",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "if (open_count or 0) >= MAX_GATEWAY_SESSIONS:",
+  "new": "if (open_count or 0) > 10**9:",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S4 cap counts own replaced session",
+  "file": "app/services/gui_ticket_service.py",
+  "old": "        ~((GuiSession.session_hash == session_hash) & (GuiSession.slot == slot))))",
+  "new": "        GuiSession.slot > 0))",
+  "tests": "tests/test_gui_access.py tests/test_gui_gateway.py"
+ },
+ {
+  "name": "S3 plain cookie accepted again on HTTPS",
+  "file": "app/routers/auth.py",
+  "old": "    return request.cookies.get(session_cookie_name(), \"\")",
+  "new": "    return request.cookies.get(session_cookie_name()) or request.cookies.get(COOKIE_NAME, \"\")",
+  "tests": "tests/test_session_hardening.py tests/test_gui_access.py"
+ },
+ {
+  "name": "S3 logout leaves legacy session",
+  "file": "app/routers/auth.py",
+  "old": "    if settings.cookie_secure and legacy and legacy != token:",
+  "new": "    if False:",
+  "tests": "tests/test_session_hardening.py tests/test_gui_access.py"
+ },
+ {
+  "name": "S3 logout does not delete plain cookie",
+  "file": "app/routers/auth.py",
+  "old": "    response.delete_cookie(key=COOKIE_NAME, path=\"/\", secure=settings.cookie_secure, httponly=True, samesite=\"lax\")\n    if settings.cookie_secure:\n        response.delete_cookie(key=SECURE",
+  "new": "    if settings.cookie_secure:\n        response.delete_cookie(key=SECURE",
+  "tests": "tests/test_session_hardening.py tests/test_gui_access.py"
+ },
+ {
+  "name": "S2 plain state cookie name on HTTPS",
+  "file": "app/routers/oidc.py",
+  "old": "    return SECURE_STATE_COOKIE if settings.cookie_secure else STATE_COOKIE",
+  "new": "    return STATE_COOKIE",
+  "tests": "tests/test_oidc.py"
+ },
+ {
+  "name": "S2 state cookie path not /",
+  "file": "app/routers/oidc.py",
+  "old": "    return \"/\" if settings.cookie_secure else STATE_COOKIE_PATH",
+  "new": "    return STATE_COOKIE_PATH",
+  "tests": "tests/test_oidc.py"
+ },
+ {
+  "name": "S2 callback without cookie accepted",
+  "file": "app/routers/oidc.py",
+  "old": "        if not bound or not secrets.compare_digest(bound.encode(), state.encode()):",
+  "new": "        if False:",
+  "tests": "tests/test_oidc.py"
+ },
+ {
+  "name": "S2 sec-fetch exemption for any request",
+  "file": "app/security_middleware.py",
+  "old": "    if path in _SEC_FETCH_EXEMPT and _is_top_level_navigation(request):",
+  "new": "    if path in _SEC_FETCH_EXEMPT:",
+  "tests": "tests/test_oidc.py"
+ },
+ {
+  "name": "S5 no guard in transport",
+  "file": "app/services/url_guard.py",
+  "old": "        pool._network_backend = _GuardedBackend(backend)",
+  "new": "        pass",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 forbidden check skipped at connect",
+  "file": "app/services/url_guard.py",
+  "old": "        if not addresses or _any_forbidden(addresses):",
+  "new": "        if not addresses:",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 only first answer checked",
+  "file": "app/services/url_guard.py",
+  "old": "def _any_forbidden(addresses: list[str]) -> bool:\n    for address in addresses:",
+  "new": "def _any_forbidden(addresses: list[str]) -> bool:\n    for address in addresses[:1]:",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 connects by name again",
+  "file": "app/services/url_guard.py",
+  "old": "                return await self._inner.connect_tcp(address, port,",
+  "new": "                return await self._inner.connect_tcp(host, port,",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 webhook without guarded client",
+  "file": "app/services/notification_delivery.py",
+  "old": "async with url_guard.guarded_client(timeout=settings.opnsense_timeout, follow_redirects=False) as client:\n            with url_guard.muted():",
+  "new": "async with httpx.AsyncClient(timeout=settings.opnsense_timeout, follow_redirects=False) as client:\n            with url_guard.muted():",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 channel without guarded client",
+  "file": "app/services/notification_channels.py",
+  "old": "async with url_guard.guarded_client(",
+  "new": "async with httpx.AsyncClient(",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S5 oidc without guarded client",
+  "file": "app/services/oidc_service.py",
+  "old": "async with url_guard.guarded_client(timeout=10, follow_redirects=False) as client:",
+  "new": "async with httpx.AsyncClient(timeout=10, follow_redirects=False) as client:",
+  "tests": "tests/test_url_guard.py"
+ },
+ {
+  "name": "S7 sequential delivery",
+  "file": "app/services/notification_service.py",
+  "old": "        await asyncio.gather(*calls, return_exceptions=True)",
+  "new": "        for call in calls:\n            await call",
+  "tests": "tests/test_notification_parallel.py"
+ },
+ {
+  "name": "S7 no per-target timeout",
+  "file": "app/services/notification_service.py",
+  "old": "            await asyncio.wait_for(make_call(), DELIVERY_TIMEOUT_SECONDS)",
+  "new": "            await make_call()",
+  "tests": "tests/test_notification_parallel.py"
+ },
+ {
+  "name": "S7 no cycle budget",
+  "file": "app/services/notification_service.py",
+  "old": "    done, late = await asyncio.wait(tasks, timeout=DELIVERY_CYCLE_BUDGET_SECONDS)",
+  "new": "    done, late = await asyncio.wait(tasks)",
+  "tests": "tests/test_notification_parallel.py"
+ },
+ {
+  "name": "S7 no parallel limit",
+  "file": "app/services/notification_service.py",
+  "old": "    gate = asyncio.Semaphore(DELIVERY_PARALLEL)",
+  "new": "    gate = asyncio.Semaphore(10**6)",
+  "tests": "tests/test_notification_parallel.py"
+ },
+ {
+  "name": "S7 attempt not recorded when cancelled",
+  "file": "app/services/notification_service.py",
+  "old": "    finally:\n        # \"Attempted\" is what is recorded",
+  "new": "    except ZeroDivisionError:\n        pass\n    else:\n        # \"Attempted\" is what is recorded",
+  "tests": "tests/test_notification_parallel.py"
+ },
+ {
+  "name": "S6 exceptions not in definition",
+  "file": "app/services/window_service.py",
+  "old": "    if window.exceptions:\n        result[",
+  "new": "    if False:\n        result[",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 moved start not in definition",
+  "file": "app/services/window_service.py",
+  "old": "e.moved_start.astimezone(timezone.utc).isoformat() if e.moved_start else None]",
+  "new": "None]",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 current takes earliest again",
+  "file": "app/services/window_service.py",
+  "old": "return max(open_now, key=lambda occ: occ.start) if open_now else None",
+  "new": "return min(open_now, key=lambda occ: occ.start) if open_now else None",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 may_roll_out ignores id",
+  "file": "app/services/rollout_engine.py",
+  "old": "    if user_id is not None:\n        user = await db.get(AdminUser, user_id)",
+  "new": "    if False:\n        user = await db.get(AdminUser, user_id)",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 auto start check ignores id",
+  "file": "app/services/rollout_engine.py",
+  "old": "may_roll_out(db, plan.activated_by or \"\", plan.activated_by_id)",
+  "new": "may_roll_out(db, plan.activated_by or \"\")",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 run does not get id",
+  "file": "app/services/rollout_engine.py",
+  "old": "started_by=plan.activated_by, started_by_id=plan.activated_by_id,",
+  "new": "started_by=plan.activated_by,",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 activation does not store id",
+  "file": "app/routers/rollouts.py",
+  "old": "    plan.activated_by_id = request.state.user.id\n",
+  "new": "",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "S6 migration no backfill of plans",
+  "file": "migrations/versions/0026_rollout_actor_ids.py",
+  "old": "    op.execute(\"UPDATE rollout_plans p SET",
+  "new": "    op.execute(\"SELECT 1 -- UPDATE rollout_plans p SET",
+  "tests": "tests/test_rollout_engine.py tests/test_maintenance_windows.py tests/test_migrations.py"
+ },
+ {
+  "name": "UI None % shown again",
+  "file": "app/services/monitor_checks.py",
+  "old": "\"wert\": None if value is None else f\"{value}{unit}\",",
+  "new": "\"wert\": f\"{value}{unit}\",",
+  "tests": "tests/test_monitoring_alerts.py"
+ },
+ {
+  "name": "UI adopt reason_details dropped",
+  "file": "app/routers/adopt_routes.py",
+  "old": "reason_details=json.dumps(exc.details, ensure_ascii=False)) from exc",
+  "new": "reason_details=\"{}\") from exc",
+  "tests": "tests/test_object_adopt.py"
+ },
+ {
+  "name": "UI channel test open to operators",
+  "file": "app/services/api_token_service.py",
+  "old": "r\"^/api/(users|settings|system|notifications|overlay)(/|$)\"",
+  "new": "r\"^/api/(users|settings|system|overlay)(/|$)\"",
+  "tests": "tests/test_operator_role.py"
+ }
+]

backend/tests/test_adopt_link.py

@@ -0,0 +1,486 @@
+"""Linking hand-made objects on a second firewall to the central object that already exists
+(services/adopt_link_service.py), instead of creating a duplicate."""
+import json
+
+import httpx
+import pytest
+from sqlalchemy import func, select
+
+from app.database import SessionLocal
+from app.models.alias import FirewallAlias
+from app.models.audit import AuditEvent
+from app.models.port_forward import PortForwardRule
+from tests.helpers import deploy_approved
+from tests.test_api_tokens import _login
+from tests.test_object_adopt import AUTOMATIC_NAT, NAT_KINDS, SYSTEM_ALIASES, alias_row, o2o_row, pf_row, snat_row
+from tests.test_object_adopt import _mode
+from tests.test_rule_adopt import _row as rule_row
+
+A = "https://fw-link-a.test"
+B = "https://fw-link-b.test"
+
+
+async def _host(client, respx_mock, base, name):
+    respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
+    r = await client.post("/api/hosts", json={"name": name, "base_url": base, "api_key": "k", "api_secret": "s"})
+    assert r.status_code == 201, r.text
+    return r.json()["id"]
+
+
+def _search(respx_mock, base, path, rows):
+    return respx_mock.get(f"{base}/api/firewall/{path}").mock(return_value=httpx.Response(200, json={"rows": rows, "total": len(rows)}))
+
+
+def _live_writes(respx_mock):
+    return [c for c in respx_mock.calls if c.request.method != "GET" and "/firewall/" in str(c.request.url)]
+
+
+@pytest.fixture
+async def two(client, respx_mock):
+    a = await _host(client, respx_mock, A, "fw-link-a")
+    b = await _host(client, respx_mock, B, "fw-link-b")
+    _mode(respx_mock)  # outbound NAT mode is read per base url below
+    for base in (A, B):
+        respx_mock.get(f"{base}/api/firewall/source_nat/get").mock(return_value=httpx.Response(
+            200, json={"filter": {"general": {"snat_mode": {m: {"selected": 1 if m == "hybrid" else 0} for m in
+                                                              ("automatic", "hybrid", "advanced", "disabled")}}}}))
+    return a, b
+
+
+async def _put_vars(client, host_id, values):
+    r = await client.put(f"/api/hosts/{host_id}/variables", json={"variables": values})
+    assert r.status_code == 200, r.text
+
+
+# ── aliases ─────────────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_alias_is_linked_not_duplicated_and_rolls_out_to_both_hosts(client, respx_mock, two):
+    a, b = two
+    live_a = alias_row("web_servers", "host", "10.0.0.1\n10.0.0.2", uuid="uuid-a")
+    live_b = alias_row("web_servers", "host", "10.0.0.1\n10.0.0.2", uuid="uuid-b")
+    _search(respx_mock, A, "alias/searchItem", SYSTEM_ALIASES + [live_a])
+    _search(respx_mock, B, "alias/searchItem", SYSTEM_ALIASES + [live_b])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "uuid-a"})).json()
+
+    # host B: the candidate carries the exact twin (name_exists no longer decides alone)
+    cand = (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()
+    row = next(r for r in cand if r["uuid"] == "uuid-b")
+    assert row["linkable"] is True and row["reasons"] == ["name_exists"]
+    assert [(m["id"], m["name"], m["targets"], m["diff"]) for m in row["matches"]] == [(first["id"], "web_servers", ["fw-link-a"], [])]
+    assert row["similar"] == [] and row["bulk_action"] == "link"
+
+    # adopting it as an own object is still refused (the name is taken)
+    assert (await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "uuid-b"})).status_code == 409
+
+    resp = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "uuid-b", "link_to": first["id"]})
+    assert resp.status_code == 201, resp.text
+    linked = resp.json()
+    assert linked["id"] == first["id"] and sorted(linked["target_host_ids"]) == sorted([a, b])
+    assert {d["host_id"]: d["status"] for d in linked["deployments"]} == {a: "success", b: "success"}
+    assert not _live_writes(respx_mock)  # nothing is written to a firewall
+    async with SessionLocal() as db:
+        assert await db.scalar(select(func.count()).select_from(FirewallAlias)) == 1
+        events = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "alias"))).all()
+    assert any(e.message == "Mit vorhandenem Objekt verknüpft" and e.details["remote_uuid"] == "uuid-b" for e in events)
+    revisions = (await client.get(f"/api/aliases/{first['id']}/revisions")).json()
+    assert any("fw-link-b verknüpft" in r["note"] for r in revisions)
+
+    # preview: unchanged on both hosts; it is no longer offered on B
+    preview = (await client.get(f"/api/aliases/{first['id']}/preview")).json()
+    assert sorted(h["action"] for h in preview["hosts"]) == ["unchanged", "unchanged"]
+    assert [r for r in (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json() if r["reasons"] != ["system"]] == []
+
+    # one edit rolls out to both remote objects, never as a new one
+    edit = {"name": "web_servers", "type": "host", "content": "10.0.0.1\n10.0.0.3", "description": "web_servers desc", "enabled": True,
+            "target_host_ids": [a, b], "target_group_ids": [], "expected_updated_at": linked["updated_at"]}
+    assert (await client.put(f"/api/aliases/{first['id']}", json=edit)).status_code == 200
+    add = respx_mock.post(url__regex=r".*/alias/addItem").mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": "dup"}))
+    set_a = respx_mock.post(f"{A}/api/firewall/alias/setItem/uuid-a").mock(return_value=httpx.Response(200, json={"result": "saved"}))
+    set_b = respx_mock.post(f"{B}/api/firewall/alias/setItem/uuid-b").mock(return_value=httpx.Response(200, json={"result": "saved"}))
+    respx_mock.post(url__regex=r".*/alias/reconfigure").mock(return_value=httpx.Response(200, json={"status": "ok"}))
+    assert (await deploy_approved(client, f"/api/aliases/{first['id']}/deploy")).status_code == 200
+    assert set_a.called and set_b.called and not add.called
+
+
+@pytest.mark.asyncio
+async def test_link_is_idempotent(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("dup_me", uuid="uuid-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("dup_me", uuid="uuid-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "uuid-a"})).json()
+    body = {"host_id": b, "uuid": "uuid-b", "link_to": first["id"]}
+    assert (await client.post("/api/aliases/adopt", json=body)).status_code == 201
+    again = await client.post("/api/aliases/adopt", json=body)
+    assert again.status_code == 201, again.text
+    assert len(again.json()["deployments"]) == 2 and len(again.json()["target_host_ids"]) == 2
+    # linking the same live object to ANOTHER central object is refused
+    other = (await client.post("/api/aliases", json={"name": "other", "type": "host", "content": "1.1.1.1"})).json()
+    assert (await client.post("/api/aliases/adopt", json={**body, "link_to": other["id"]})).status_code == 409
+
+
+@pytest.mark.asyncio
+async def test_different_content_shows_a_diff_and_is_never_linked(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("ntp", "host", "10.0.0.1", uuid="uuid-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("ntp", "host", "10.0.0.1\n10.0.0.9", uuid="uuid-b", description="other text")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "uuid-a"})).json()
+    row = (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()[0]
+    assert row["linkable"] is False and row["matches"] == [] and row["bulk_action"] == "skip"
+    sim = row["similar"][0]
+    assert sim["id"] == first["id"]
+    assert {d["field"]: (d["central"], d["live"]) for d in sim["diff"]} == {
+        "content": ("10.0.0.1", "10.0.0.1\n10.0.0.9"), "description": ("ntp desc", "other text")}
+    # forcing the link by id is refused too: the live state is compared again on the server
+    resp = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "uuid-b", "link_to": first["id"]})
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "ADOPT_LINK_TARGET_CHANGED"
+    assert "content" in resp.json()["detail"]["params"]["fields"]
+    # nothing was linked
+    alias = next(x for x in (await client.get("/api/aliases")).json() if x["id"] == first["id"])
+    assert alias["target_host_ids"] == [a] and len(alias["deployments"]) == 1
+
+
+@pytest.mark.asyncio
+async def test_browser_state_is_not_trusted_live_state_decides(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("srv", "host", "10.0.0.1", uuid="uuid-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("srv", "host", "10.0.0.1", uuid="uuid-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "uuid-a"})).json()
+    assert (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()[0]["linkable"] is True
+    # between listing and clicking somebody changes the alias on host B by hand
+    _search(respx_mock, B, "alias/searchItem", [alias_row("srv", "host", "10.0.0.1\n10.0.0.77", uuid="uuid-b")])
+    resp = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "uuid-b", "link_to": first["id"]})
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "ADOPT_LINK_TARGET_CHANGED"
+
+
+@pytest.mark.asyncio
+async def test_variables_are_resolved_per_host(client, respx_mock, two):
+    a, b = two
+    await _put_vars(client, a, {"LAN_NET": "192.168.1.0/24"})
+    await _put_vars(client, b, {"LAN_NET": "192.168.2.0/24"})
+    central = (await client.post("/api/aliases", json={"name": "lan_net", "type": "network", "content": "{{LAN_NET}}", "description": "lan_net desc",
+                                                       "target_host_ids": [a]})).json()
+    _search(respx_mock, B, "alias/searchItem", [alias_row("lan_net", "network", "192.168.2.0/24", uuid="uuid-b")])
+    row = (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()[0]
+    assert row["linkable"] is True and row["matches"][0]["id"] == central["id"]
+    resp = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "uuid-b", "link_to": central["id"]})
+    assert resp.status_code == 201, resp.text
+    dep_b = next(d for d in resp.json()["deployments"] if d["host_id"] == b)
+    assert dep_b["status"] == "success"
+    async with SessionLocal() as db:
+        from app.models.alias_deployment import AliasDeployment
+        deps = {str(d.host_id): d for d in (await db.scalars(select(AliasDeployment))).all()}
+    assert deps[b].opnsense_alias_uuid == "uuid-b" and deps[b].last_payload["content"] == "192.168.2.0/24"
+    preview = (await client.get(f"/api/aliases/{central['id']}/preview")).json()
+    assert {h["host_name"]: h["action"] for h in preview["hosts"]}["fw-link-b"] == "unchanged"  # resolved payload == live
+
+    # a host with a DIFFERENT value for the variable does not match; the diff shows the resolved value
+    _search(respx_mock, B, "alias/searchItem", [alias_row("lan_net", "network", "192.168.9.0/24", uuid="uuid-x")])
+    # (host B already has the object, so ask for it through a third host)
+    c = await _host(client, respx_mock, "https://fw-link-c.test", "fw-link-c")
+    await _put_vars(client, c, {"LAN_NET": "192.168.3.0/24"})
+    _search(respx_mock, "https://fw-link-c.test", "alias/searchItem", [alias_row("lan_net", "network", "192.168.9.0/24", uuid="uuid-c")])
+    row = (await client.get(f"/api/aliases/unmanaged?host_id={c}")).json()[0]
+    assert row["matches"] == [] and row["similar"][0]["diff"] == [{"field": "content", "central": "192.168.3.0/24", "live": "192.168.9.0/24"}]
+
+
+@pytest.mark.asyncio
+async def test_missing_variable_on_the_host_never_matches(client, respx_mock, two):
+    a, b = two
+    await _put_vars(client, a, {"LAN_NET": "192.168.1.0/24"})
+    await client.post("/api/aliases", json={"name": "lan_net", "type": "network", "content": "{{LAN_NET}}", "description": "lan_net desc", "target_host_ids": [a]})
+    _search(respx_mock, B, "alias/searchItem", [alias_row("lan_net", "network", "192.168.2.0/24", uuid="uuid-b")])
+    row = (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()[0]
+    assert row["matches"] == [] and row["similar"][0]["diff"][0]["field"] == "variables"
+
+
+@pytest.mark.asyncio
+async def test_nested_alias_references_are_part_of_the_comparison(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [
+        alias_row("inner", "network", "10.1.0.0/16", uuid="in-a"),
+        alias_row("outer", "networkgroup", "inner\n10.2.0.0/16", uuid="out-a")])
+    result = (await client.post("/api/aliases/adopt-all", json={"host_id": a})).json()
+    assert len(result["adopted"]) == 2
+    _search(respx_mock, B, "alias/searchItem", [
+        alias_row("inner", "network", "10.1.0.0/16", uuid="in-b"),
+        alias_row("outer", "networkgroup", "inner\n10.2.0.0/16", uuid="out-b")])
+    rows = {r["title"]: r for r in (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()}
+    assert rows["inner"]["linkable"] and rows["outer"]["linkable"]
+    # a different reference is a diff, not a match
+    _search(respx_mock, B, "alias/searchItem", [
+        alias_row("inner", "network", "10.1.0.0/16", uuid="in-b"),
+        alias_row("outer", "networkgroup", "other_alias\n10.2.0.0/16", uuid="out-b"),
+        alias_row("other_alias", "network", "10.5.0.0/16", uuid="oth-b")])
+    rows = {r["title"]: r for r in (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json()}
+    assert rows["outer"]["matches"] == [] and rows["outer"]["similar"][0]["diff"][0]["field"] == "content"
+
+    # "adopt all" with link: both linked, nothing created
+    _search(respx_mock, B, "alias/searchItem", [
+        alias_row("inner", "network", "10.1.0.0/16", uuid="in-b"),
+        alias_row("outer", "networkgroup", "inner\n10.2.0.0/16", uuid="out-b")])
+    done = (await client.post("/api/aliases/adopt-all", json={"host_id": b, "link_existing": True})).json()
+    assert sorted((x["title"], x["linked"]) for x in done["adopted"]) == [("inner", True), ("outer", True)] and done["skipped"] == []
+    async with SessionLocal() as db:
+        assert await db.scalar(select(func.count()).select_from(FirewallAlias)) == 2
+
+
+@pytest.mark.asyncio
+async def test_adopt_all_without_link_flag_keeps_old_behaviour(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("x1", uuid="u-a")])
+    await client.post("/api/aliases/adopt-all", json={"host_id": a})
+    _search(respx_mock, B, "alias/searchItem", [alias_row("x1", uuid="u-b")])
+    done = (await client.post("/api/aliases/adopt-all", json={"host_id": b})).json()
+    assert done["adopted"] == [] and done["skipped"][0]["reasons"] == ["name_exists"]
+
+
+# ── NAT + rules, every kind ─────────────────────────────────────────────
+
+KINDS = {
+    "port-forward": ("d_nat/searchRule", pf_row, {"descr": "Web"}, {"destination.port": "9999"}, "local_port"),
+    "one-to-one": ("one_to_one/searchRule", o2o_row, {"description": "Mail 1:1"}, {"external": "203.0.113.77"}, "external"),
+    "outbound-nat": ("source_nat/searchRule", snat_row, {"description": "LAN out"}, {"source_net": "192.168.9.0/24"}, "source_net"),
+}
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("prefix", list(KINDS))
+async def test_nat_kinds_link_diff_and_bulk(client, respx_mock, two, prefix):
+    a, b = two
+    path, factory, _, change, _ = KINDS[prefix]
+    _search(respx_mock, A, path, [factory(uuid="a-1"), *AUTOMATIC_NAT])
+    first = (await client.post(f"/api/{prefix}/adopt", json={"host_id": a, "uuid": "a-1"})).json()
+
+    # different on B -> hint + diff, no link possible
+    _search(respx_mock, B, path, [factory(uuid="b-1", **change)])
+    row = (await client.get(f"/api/{prefix}/unmanaged?host_id={b}")).json()[0]
+    assert row["adoptable"] is True and row["linkable"] is False and row["matches"] == []
+    assert row["similar"][0]["id"] == first["id"] and row["similar"][0]["diff"]
+    resp = await client.post(f"/api/{prefix}/adopt", json={"host_id": b, "uuid": "b-1", "link_to": first["id"]})
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "ADOPT_LINK_TARGET_CHANGED"
+    adopt_all = (await client.post(f"/api/{prefix}/adopt-all", json={"host_id": b, "link_existing": True})).json()
+    assert adopt_all["adopted"][0]["linked"] is False  # a different object is adopted as its own, not linked
+
+    # fresh start for the exact case on a third host
+    c = await _host(client, respx_mock, "https://fw-link-c.test", "fw-link-c")
+    respx_mock.get("https://fw-link-c.test/api/firewall/source_nat/get").mock(return_value=httpx.Response(
+        200, json={"filter": {"general": {"snat_mode": {m: {"selected": 1 if m == "hybrid" else 0} for m in
+                                                          ("automatic", "hybrid", "advanced", "disabled")}}}}))
+    _search(respx_mock, "https://fw-link-c.test", path, [factory(uuid="c-1")])
+    row = (await client.get(f"/api/{prefix}/unmanaged?host_id={c}")).json()[0]
+    ids = [m["id"] for m in row["matches"]]
+    assert ids == [first["id"]] and row["bulk_action"] == "link" and row["linkable"]
+    resp = await client.post(f"/api/{prefix}/adopt", json={"host_id": c, "uuid": "c-1", "link_to": first["id"]})
+    assert resp.status_code == 201, resp.text
+    assert resp.json()["id"] == first["id"] and c in resp.json()["target_host_ids"]
+    preview = (await client.get(f"/api/{prefix}/{first['id']}/preview")).json()
+    assert {h["host_name"]: h["action"] for h in preview["hosts"]}["fw-link-c"] == "unchanged"
+    assert not _live_writes(respx_mock)
+    listed = (await client.get(f"/api/{prefix}")).json()
+    assert len(listed) == 2  # first (A + C) and the different one from B
+
+
+@pytest.mark.asyncio
+async def test_port_forward_link_updates_remote_object_without_duplicate(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "d_nat/searchRule", [pf_row(uuid="pf-a")])
+    _search(respx_mock, B, "d_nat/searchRule", [pf_row(uuid="pf-b")])
+    first = (await client.post("/api/port-forward/adopt", json={"host_id": a, "uuid": "pf-a"})).json()
+    linked = (await client.post("/api/port-forward/adopt", json={"host_id": b, "uuid": "pf-b", "link_to": first["id"]})).json()
+    edit = {k: linked[k] for k in ("description", "interface", "ipprotocol", "protocol", "source_net", "source_port",
+                                    "destination_net", "destination_port", "target", "local_port", "natreflection", "enabled")}
+    for base in (A, B):  # alias lookups and the interface consistency check of a multi-host rollout
+        _search(respx_mock, base, "alias/searchItem", [])
+    for base in (A, B):  # the interface consistency check of a multi-host rollout
+        respx_mock.get(f"{base}/api/firewall/filter/getRule").mock(return_value=httpx.Response(
+            200, json={"rule": {"interface": {"wan": {"value": "WAN", "selected": 1}}}}))
+    edit.update(target="192.168.1.99", target_host_ids=[a, b], target_group_ids=[], expected_updated_at=linked["updated_at"])
+    assert (await client.put(f"/api/port-forward/{first['id']}", json=edit)).status_code == 200
+    add = respx_mock.post(url__regex=r".*/d_nat/addRule").mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": "dup"}))
+    sa = respx_mock.post(f"{A}/api/firewall/d_nat/setRule/pf-a").mock(return_value=httpx.Response(200, json={"result": "saved"}))
+    sb = respx_mock.post(f"{B}/api/firewall/d_nat/setRule/pf-b").mock(return_value=httpx.Response(200, json={"result": "saved"}))
+    respx_mock.post(url__regex=r".*/d_nat/apply").mock(return_value=httpx.Response(200, json={"status": "ok"}))
+    assert (await deploy_approved(client, f"/api/port-forward/{first['id']}/deploy")).status_code == 200
+    assert sa.called and sb.called and not add.called
+    assert json.loads(sb.calls.last.request.content)["rule"]["target"] == "192.168.1.99"
+
+
+@pytest.mark.asyncio
+async def test_rule_link_diff_and_errors(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "filter/searchRule", [rule_row("r-a")])
+    _search(respx_mock, B, "filter/searchRule", [rule_row("r-b")])
+    first = (await client.post("/api/rules/adopt", json={"host_id": a, "rule_uuid": "r-a"})).json()
+    row = (await client.get(f"/api/rules/unmanaged?host_id={b}")).json()[0]
+    assert row["linkable"] is True and [m["id"] for m in row["matches"]] == [first["id"]]
+    resp = await client.post("/api/rules/adopt", json={"host_id": b, "rule_uuid": "r-b", "link_to": first["id"]})
+    assert resp.status_code == 201, resp.text
+    assert resp.json()["id"] == first["id"] and sorted(resp.json()["target_host_ids"]) == sorted([a, b])
+    again = await client.post("/api/rules/adopt", json={"host_id": b, "rule_uuid": "r-b", "link_to": first["id"]})
+    assert again.status_code == 201 and len(again.json()["deployments"]) == 2
+    preview = (await client.get(f"/api/rules/{first['id']}/preview")).json()
+    assert sorted(h["action"] for h in preview["hosts"]) == ["unchanged", "unchanged"]
+    assert not _live_writes(respx_mock)
+
+    # different port -> diff only
+    c = await _host(client, respx_mock, "https://fw-link-c.test", "fw-link-c")
+    _search(respx_mock, "https://fw-link-c.test", "filter/searchRule", [rule_row("r-c", destination_port="2222")])
+    row = (await client.get(f"/api/rules/unmanaged?host_id={c}")).json()[0]
+    assert row["linkable"] is False and row["similar"][0]["diff"] == [{"field": "destination_port", "central": "22", "live": "2222"}]
+    refused = await client.post("/api/rules/adopt", json={"host_id": c, "rule_uuid": "r-c", "link_to": first["id"]})
+    assert refused.status_code == 409 and refused.json()["detail"]["code"] == "ADOPT_LINK_TARGET_CHANGED"
+
+
+# ── error cases ─────────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_deleted_target_and_host_already_has_object_and_busy(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("gone", uuid="u-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("gone", uuid="u-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})).json()
+    # busy: an unconfirmed deployment on the central object
+    respx_mock.post(f"{A}/api/firewall/alias/setItem/u-a").mock(return_value=httpx.Response(200, json={"result": "saved"}))
+    respx_mock.post(url__regex=r".*/alias/reconfigure").mock(return_value=httpx.Response(200, json={"status": "ok"}))
+    edit = {"name": "gone", "type": "host", "content": "10.9.9.1\n10.9.9.2", "description": "gone desc", "enabled": False,
+            "target_host_ids": [a], "target_group_ids": [], "expected_updated_at": first["updated_at"]}
+    assert (await client.put(f"/api/aliases/{first['id']}", json=edit)).status_code == 200
+    assert (await deploy_approved(client, f"/api/aliases/{first['id']}/deploy")).status_code == 200  # pending now
+    busy = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "u-b", "link_to": first["id"]})
+    assert busy.status_code == 409  # content differs now AND it is pending; either way nothing is linked
+    await client.post(f"/api/aliases/{first['id']}/confirm")
+
+    # deleted in the meantime
+    respx_mock.post(url__regex=r".*/alias/delItem/.*").mock(return_value=httpx.Response(200, json={"result": "deleted"}))
+    assert (await client.delete(f"/api/aliases/{first['id']}")).status_code == 204
+    gone = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "u-b", "link_to": first["id"]})
+    assert gone.status_code == 404 and gone.json()["detail"]["code"] == "ADOPT_LINK_TARGET_NOT_FOUND"
+
+
+@pytest.mark.asyncio
+async def test_pending_deployment_blocks_the_link(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("busy1", uuid="u-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("busy1", uuid="u-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})).json()
+    async with SessionLocal() as db:
+        from app.models.alias_deployment import AliasDeployment
+        dep = (await db.scalars(select(AliasDeployment))).one()
+        dep.status = "pending"
+        await db.commit()
+    resp = await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "u-b", "link_to": first["id"]})
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "ADOPT_LINK_BUSY"
+
+
+@pytest.mark.asyncio
+async def test_host_that_already_has_the_object_is_not_a_candidate_and_cannot_link(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("twice", uuid="u-a")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})).json()
+    # a second, differently-UUID'd copy on host A itself: A already has the object
+    _search(respx_mock, A, "alias/searchItem", [alias_row("twice", uuid="u-a"), alias_row("twice", uuid="u-a2")])
+    rows = (await client.get(f"/api/aliases/unmanaged?host_id={a}")).json()
+    assert [r["uuid"] for r in rows] == ["u-a2"] and rows[0]["matches"] == [] and rows[0]["linkable"] is False
+    resp = await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a2", "link_to": first["id"]})
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "ADOPT_LINK_HOST_HAS_OBJECT"
+
+
+@pytest.mark.asyncio
+async def test_bulk_never_guesses_between_two_twins_and_links_each_object_once(client, respx_mock, two):
+    a, b = two
+    # two identical central objects exist from earlier separate adoptions
+    for _ in range(2):
+        await client.post("/api/port-forward", json={
+            "description": "Web", "interface": "wan", "ipprotocol": "inet", "protocol": "tcp", "source_net": "any",
+            "destination_net": "wanip", "destination_port": "8443", "target": "192.168.1.50", "local_port": "443",
+            "natreflection": "", "enabled": True, "target_host_ids": [a], "target_group_ids": []})
+    _search(respx_mock, B, "d_nat/searchRule", [pf_row(uuid="pf-b")])
+    row = (await client.get(f"/api/port-forward/unmanaged?host_id={b}")).json()[0]
+    assert len(row["matches"]) == 2 and row["bulk_action"] == "new"  # ask the user, do not guess
+    done = (await client.post("/api/port-forward/adopt-all", json={"host_id": b, "link_existing": True})).json()
+    assert [x["linked"] for x in done["adopted"]] == [False]
+
+
+@pytest.mark.asyncio
+async def test_bulk_links_each_central_object_only_once(client, respx_mock, two):
+    a, b = two
+    await client.post("/api/port-forward", json={
+        "description": "Web", "interface": "wan", "ipprotocol": "inet", "protocol": "tcp", "source_net": "any",
+        "destination_net": "wanip", "destination_port": "8443", "target": "192.168.1.50", "local_port": "443",
+        "natreflection": "", "enabled": True, "target_host_ids": [a], "target_group_ids": []})
+    # two identical live rows on B, ONE twin: the first links, the second becomes its own object
+    _search(respx_mock, B, "d_nat/searchRule", [pf_row(uuid="b-1"), pf_row(uuid="b-2")])
+    rows = (await client.get(f"/api/port-forward/unmanaged?host_id={b}")).json()
+    assert sorted(r["bulk_action"] for r in rows) == ["link", "new"]  # the dialog's summary equals what happens
+    done = (await client.post("/api/port-forward/adopt-all", json={"host_id": b, "link_existing": True})).json()
+    assert sorted(x["linked"] for x in done["adopted"]) == [False, True]
+    assert len((await client.get("/api/port-forward")).json()) == 2
+
+
+@pytest.mark.asyncio
+async def test_roles_for_linking(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("roles1", uuid="u-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("roles1", uuid="u-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})).json()
+    for role, expected in (("read_only", 403), ("operator", 201)):
+        user = await client.post("/api/users", json={"username": f"lnk-{role}", "password": "adopt-test-password-1", "role": role})
+        assert user.status_code == 201, user.text
+        session = await _login(f"lnk-{role}", "adopt-test-password-1")
+        try:
+            resp = await session.post("/api/aliases/adopt", json={"host_id": b, "uuid": "u-b", "link_to": first["id"]})
+            assert resp.status_code == expected, (role, resp.text)
+        finally:
+            await session.aclose()
+
+
+@pytest.mark.asyncio
+async def test_bulk_leaves_a_twin_alone_that_waits_for_a_confirmation(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("busy2", uuid="u-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("busy2", uuid="u-b")])
+    await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})
+    async with SessionLocal() as db:
+        from app.models.alias_deployment import AliasDeployment
+        (await db.scalars(select(AliasDeployment))).one().status = "pending"
+        await db.commit()
+    row = next(r for r in (await client.get(f"/api/aliases/unmanaged?host_id={b}")).json() if r["uuid"] == "u-b")
+    assert row["linkable"] is False and row["bulk_action"] == "skip"
+    resp = await client.post("/api/aliases/adopt-all", json={"host_id": b, "link_existing": True})
+    assert resp.status_code == 200, resp.text
+    assert resp.json()["adopted"] == []
+    assert "twin_busy" in resp.json()["skipped"][0]["reasons"]
+    async with SessionLocal() as db:
+        assert await db.scalar(select(func.count()).select_from(FirewallAlias)) == 1
+
+
+@pytest.mark.asyncio
+async def test_linking_invalidates_an_open_edit_form(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "alias/searchItem", [alias_row("stale1", uuid="u-a")])
+    _search(respx_mock, B, "alias/searchItem", [alias_row("stale1", uuid="u-b")])
+    first = (await client.post("/api/aliases/adopt", json={"host_id": a, "uuid": "u-a"})).json()
+    linked = (await client.post("/api/aliases/adopt", json={"host_id": b, "uuid": "u-b", "link_to": first["id"]})).json()
+    assert linked["updated_at"] != first["updated_at"]
+    stale = {"name": "stale1", "type": "host", "content": linked["content"], "description": linked["description"],
+             "enabled": linked["enabled"], "target_host_ids": [a], "target_group_ids": [],
+             "expected_updated_at": first["updated_at"]}
+    assert (await client.put(f"/api/aliases/{first['id']}", json=stale)).status_code == 409
+
+
+@pytest.mark.asyncio
+async def test_bulk_neither_links_nor_duplicates_a_busy_nat_twin(client, respx_mock, two):
+    a, b = two
+    _search(respx_mock, A, "d_nat/searchRule", [pf_row(uuid="pf-a")])
+    _search(respx_mock, B, "d_nat/searchRule", [pf_row(uuid="pf-b")])
+    await client.post("/api/port-forward/adopt", json={"host_id": a, "uuid": "pf-a"})
+    async with SessionLocal() as db:
+        from app.models.port_forward_deployment import PortForwardDeployment
+        (await db.scalars(select(PortForwardDeployment))).one().status = "pending"
+        await db.commit()
+    resp = await client.post("/api/port-forward/adopt-all", json={"host_id": b, "link_existing": True})
+    assert resp.status_code == 200, resp.text
+    assert resp.json()["adopted"] == [] and "twin_busy" in resp.json()["skipped"][0]["reasons"]
+    async with SessionLocal() as db:
+        assert await db.scalar(select(func.count()).select_from(PortForwardRule)) == 1

backend/tests/test_api_rights_gateway_mute.py

@@ -0,0 +1,417 @@
+"""1.29: missing API rights become visible (OpnsenseForbiddenError, `denied` in the sample, api_rights_missing alert,
+dashboard, metrics) and the gateway alert can be switched off per gateway."""
+import uuid
+
+import httpx
+import pytest
+from sqlalchemy import select
+
+from app.database import SessionLocal
+from app.models.audit import AuditEvent
+from app.models.host import OpnsenseHost
+from app.models.notification import NotificationProblem
+from app.models.resource_sample import HostResourceSample
+from app.services import monitor_checks, notification_delivery, opnsense_client, resource_monitor_service
+from app.services.notification_service import run_notification_checks
+from tests.test_monitoring_alerts import (
+    STATUS_PAYLOAD, _add_samples, _gw, _gateway_problems, _only, _open_count, _set,
+)
+from tests.test_notifications import _create_host
+from tests.test_resource_monitor import _mock_resource_endpoints, _mock_vip
+
+
+async def _host_row(host_id: str) -> OpnsenseHost:
+    async with SessionLocal() as db:
+        return await db.get(OpnsenseHost, uuid.UUID(host_id))
+
+
+async def _clear_samples():
+    async with SessionLocal() as db:
+        await db.execute(HostResourceSample.__table__.delete())
+        await db.commit()
+
+
+async def _rights_problems():
+    async with SessionLocal() as db:
+        return await monitor_checks.check_api_rights_missing(db)
+
+
+# ── Client: 403 is told apart from everything else (a 401 is a wrong key, not a missing right) ──────────────────────
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("status", [403])
+async def test_403_raises_forbidden_error_with_unchanged_message(client, respx_mock, status):
+    _mock_vip(respx_mock)
+    host = await _host_row((await _create_host(client, "fw1", "https://fw1.test"))["id"])
+    respx_mock.get("https://fw1.test/api/routes/gateway/status").respond(status, text="denied")
+    with pytest.raises(opnsense_client.OpnsenseForbiddenError) as exc:
+        await opnsense_client.get_gateway_status(host)
+    assert isinstance(exc.value, opnsense_client.OpnsenseAPIError)  # existing `except OpnsenseAPIError` stays valid
+    assert not isinstance(exc.value, opnsense_client.OpnsenseAmbiguousError)
+    assert str(exc.value).startswith(f"{status} /routes/gateway/status")
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("status,ambiguous", [(500, True), (502, True), (404, False), (400, False), (401, False)])
+async def test_other_errors_are_not_forbidden(client, respx_mock, status, ambiguous):
+    _mock_vip(respx_mock)
+    host = await _host_row((await _create_host(client, "fw1", "https://fw1.test"))["id"])
+    respx_mock.get("https://fw1.test/api/routes/gateway/status").respond(status)
+    with pytest.raises(opnsense_client.OpnsenseAPIError) as exc:
+        await opnsense_client.get_gateway_status(host)
+    assert not isinstance(exc.value, opnsense_client.OpnsenseForbiddenError)
+    assert isinstance(exc.value, opnsense_client.OpnsenseAmbiguousError) is ambiguous
+
+
+@pytest.mark.asyncio
+async def test_gateway_endpoint_reports_missing_rights_not_a_generic_502(client, respx_mock):
+    _mock_vip(respx_mock)
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    route = respx_mock.get("https://fw1.test/api/routes/gateway/status")
+    route.respond(403)
+    resp = await client.get(f"/api/hosts/{host['id']}/gateways")
+    assert resp.status_code == 403 and resp.json()["detail"]["code"] == "API_RIGHTS_MISSING"
+    route.respond(500)
+    assert (await client.get(f"/api/hosts/{host['id']}/gateways")).status_code == 502
+
+
+# ── Sampler records what was refused ─────────────────────────────────────────
+
+def _mock_optional_reads(respx_mock, *, gateways=200, services=200, swap=200, cpu=200):
+    base = "https://fw1.test/api"
+    respx_mock.get(f"{base}/diagnostics/system/systemSwap").respond(swap, json={"swap": []})
+    respx_mock.get(f"{base}/diagnostics/cpu_usage/getCPUType").respond(cpu, json=["X (4 cores, 4 threads)"])
+    respx_mock.get(f"{base}/routes/gateway/status").respond(gateways, json=STATUS_PAYLOAD)
+    respx_mock.get(f"{base}/core/service/search").respond(services, json={"rows": []})
+
+
+async def _sample_raw() -> dict:
+    async with SessionLocal() as db:
+        await resource_monitor_service.poll_all_hosts(db)
+        return (await db.scalars(select(HostResourceSample).order_by(HostResourceSample.sampled_at.desc()))).first().raw
+
+
+@pytest.mark.asyncio
+async def test_sample_records_only_refused_reads_in_denied(client, respx_mock):
+    _mock_vip(respx_mock)
+    await _create_host(client, "fw1", "https://fw1.test")
+    _mock_resource_endpoints(respx_mock)
+    _mock_optional_reads(respx_mock, gateways=403, services=403, swap=500, cpu=401)
+    raw = await _sample_raw()
+    # forbidden -> denied (sorted); a server error and a missing endpoint are "unknown", no finding about rights
+    assert raw["denied"] == ["gateways", "services"]
+    assert raw["gateways"] is None and raw["services"] is None and raw["swap_used_pct"] is None
+    assert raw["ok"] == []  # nothing went through
+
+
+@pytest.mark.asyncio
+async def test_sample_denied_is_an_empty_list_when_nothing_is_refused(client, respx_mock):
+    _mock_vip(respx_mock)
+    await _create_host(client, "fw1", "https://fw1.test")
+    _mock_resource_endpoints(respx_mock)
+    _mock_optional_reads(respx_mock)
+    raw = await _sample_raw()
+    assert raw["denied"] == [] and raw["ok"] == ["cpu", "gateways", "services", "swap"]
+    # unreachable optional endpoints (connect error / timeout) are not refusals either
+    _mock_optional_reads(respx_mock)
+    respx_mock.get("https://fw1.test/api/routes/gateway/status").mock(side_effect=httpx.ReadTimeout("slow"))
+    assert (await _sample_raw())["denied"] == []
+
+
+@pytest.mark.asyncio
+async def test_base_read_refusal_still_fails_the_sample(client, respx_mock):
+    _mock_vip(respx_mock)
+    await _create_host(client, "fw1", "https://fw1.test")
+    _mock_resource_endpoints(respx_mock)
+    respx_mock.get("https://fw1.test/api/diagnostics/system/systemResources").respond(403)
+    async with SessionLocal() as db:
+        result = await resource_monitor_service.poll_all_hosts(db)
+    assert result["hosts_failed"] == 1 and result["sampled"] == 0
+
+
+# ── The alert ────────────────────────────────────────────────────────────────
+
+DENIED_GW = {"denied": ["gateways"], "ok": ["cpu", "services", "swap"], "gateways": None, "services": {"unbound": 1}, "cpu_threads": 4}
+FINE = {"denied": [], "ok": ["cpu", "gateways", "services", "swap"], "gateways": [_gw()], "services": {"unbound": 1}, "cpu_threads": 4}
+
+
+@pytest.mark.asyncio
+async def test_api_rights_alert_needs_consecutive_samples_and_names_the_missing_right(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    await _add_samples(hid, [DENIED_GW, DENIED_GW, FINE])
+    assert dict(await _rights_problems()) == {}  # two in a row are not enough (debounce)
+    await _clear_samples()
+    await _add_samples(hid, [DENIED_GW] * 3)
+    problems = await _rights_problems()
+    assert list(problems) == [f"{hid}:gateways"]
+    ctx = problems[f"{hid}:gateways"]
+    assert ctx["host_name"] == "fw1" and ctx["fehlendes_recht"] == "System: Gateways"
+    assert ctx["endpunkt"] == "/api/routes/gateway/status" and "fehlt das Recht" in ctx["hinweis"]
+
+
+@pytest.mark.asyncio
+async def test_api_rights_alert_opens_once_resolves_and_is_delivered(client, respx_mock):
+    _mock_vip(respx_mock)
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    await _only("api_rights_enabled")
+    await _set(webhook_enabled=True, webhook_url="https://hooks.test/n")
+    hook = respx_mock.post("https://hooks.test/n").mock(return_value=httpx.Response(200))
+    await _add_samples(hid, [DENIED_GW] * 3)
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["opened"] == 1
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["opened"] == 0
+        assert await _open_count(db, "api_rights_missing") == 1
+    assert hook.call_count == 1
+    assert b"API-Rechte fehlen" in hook.calls.last.request.content and b"System: Gateways" in hook.calls.last.request.content
+    await _clear_samples()
+    await _add_samples(hid, [FINE] * 3)
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["resolved"] == 1
+    assert hook.call_count == 2 and b"behoben" in hook.calls.last.request.content
+
+
+@pytest.mark.asyncio
+async def test_api_rights_alert_stays_open_through_unknown_and_flapping_samples(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    key = f"{hid}:gateways"
+    await _only("api_rights_enabled")
+    await _add_samples(hid, [DENIED_GW] * 3)
+    async with SessionLocal() as db:
+        await run_notification_checks(db)
+    # the gateway read now times out (gateways None, not denied): no proof the right is back
+    await _clear_samples()
+    await _add_samples(hid, [{"denied": [], "gateways": None}] * 3)
+    assert list(await _rights_problems()) == [key]
+    # samples from before the denied list existed are unknown too
+    await _clear_samples()
+    await _add_samples(hid, [{"gateways": [_gw()]}] * 3)
+    assert list(await _rights_problems()) == [key]
+    # one good sample between refusals does not close it
+    await _clear_samples()
+    await _add_samples(hid, [DENIED_GW, FINE, DENIED_GW])
+    assert list(await _rights_problems()) == [key]
+
+
+@pytest.mark.asyncio
+async def test_api_rights_alert_ignores_stale_probe_and_keeps_open_problem(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    await _only("api_rights_enabled")
+    await _add_samples(hid, [DENIED_GW] * 3)
+    async with SessionLocal() as db:
+        await run_notification_checks(db)
+    await _clear_samples()
+    await _add_samples(hid, [FINE] * 3, age_minutes=30)  # the host stopped answering
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["resolved"] == 0
+        assert await _open_count(db) == 1
+    # a stale refusal does not open anything either
+    await _clear_samples()
+    async with SessionLocal() as db:
+        await db.execute(NotificationProblem.__table__.delete())
+        await db.commit()
+    await _add_samples(hid, [DENIED_GW] * 3, age_minutes=30)
+    assert dict(await _rights_problems()) == {}
+
+
+@pytest.mark.asyncio
+async def test_api_rights_alert_is_on_by_default_and_can_be_switched_off(client):
+    data = (await client.get("/api/notifications/settings")).json()
+    assert data["api_rights_enabled"] is True
+    resp = await client.put("/api/notifications/settings", json={"api_rights_enabled": False})
+    assert resp.status_code == 200 and resp.json()["api_rights_enabled"] is False
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    await _add_samples(uuid.UUID(host["id"]), [DENIED_GW] * 3)
+    await _only()  # everything off, including api_rights_enabled
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["opened"] == 0
+
+
+def test_alert_label_exists():
+    assert notification_delivery._LABELS["api_rights_missing"] == "API-Rechte fehlen"
+
+
+# ── Dashboard and metrics ────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_dashboard_and_metrics_show_missing_rights(client):
+    host = await _create_host(client, "fw1", "https://fw1.test", cluster_name=None)
+    hid = uuid.UUID(host["id"])
+    await _add_samples(hid, [{"mem": 10, "load": 0.1, **DENIED_GW, "denied": ["gateways", "services"]}])
+    summary = (await client.get("/api/dashboard")).json()["hosts"][0]
+    assert summary["api_rights_missing"] == ["gateways", "services"] and summary["gateways"] is None
+    text = (await client.get("/api/metrics")).text
+    assert 'opnexus_host_api_rights_missing{host="fw1",cluster="",read="gateways"} 1' in text
+    assert 'opnexus_host_api_rights_missing{host="fw1",cluster="",read="services"} 1' in text
+
+    await _clear_samples()
+    await _add_samples(hid, [{"mem": 10, "load": 0.1, **FINE}])
+    assert (await client.get("/api/dashboard")).json()["hosts"][0]["api_rights_missing"] == []
+    assert 'api_rights_missing{' not in (await client.get("/api/metrics")).text.replace("# HELP", "").replace("# TYPE", "")
+
+    await _clear_samples()
+    await _add_samples(hid, [DENIED_GW], age_minutes=20)
+    assert (await client.get("/api/dashboard")).json()["hosts"][0]["api_rights_missing"] is None
+
+
+# ── Mute a single gateway ────────────────────────────────────────────────────
+
+BAD = {"gateways": [_gw("offline", 100.0, None), _gw("offline", 100.0, None, name="LTE_GW")]}
+
+
+@pytest.mark.asyncio
+async def test_muted_gateway_raises_no_alert_but_the_others_do(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = host["id"]
+    await _add_samples(uuid.UUID(hid), [BAD] * 3)
+    assert sorted(await _gateway_problems()) == sorted([f"{hid}:WAN_GW", f"{hid}:LTE_GW"])
+    resp = await client.put(f"/api/hosts/{hid}/gateways/LTE_GW/alert", json={"enabled": False})
+    assert resp.status_code == 200 and resp.json() == {"name": "LTE_GW", "alert_enabled": False}
+    assert list(await _gateway_problems()) == [f"{hid}:WAN_GW"]
+    # a mute is per host: the same name on another host is still watched
+    other = await _create_host(client, "fw2", "https://fw2.test")
+    await _add_samples(uuid.UUID(other["id"]), [BAD] * 3)
+    assert f"{other['id']}:LTE_GW" in await _gateway_problems()
+    # switching it on again brings the alert back
+    await client.put(f"/api/hosts/{hid}/gateways/LTE_GW/alert", json={"enabled": True})
+    assert f"{hid}:LTE_GW" in await _gateway_problems()
+
+
+@pytest.mark.asyncio
+async def test_muting_resolves_an_open_gateway_problem(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = host["id"]
+    await _only("gateway_down_enabled")
+    await _add_samples(uuid.UUID(hid), [BAD] * 3)
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["opened"] == 2
+    await client.put(f"/api/hosts/{hid}/gateways/LTE_GW/alert", json={"enabled": False})
+    async with SessionLocal() as db:
+        assert (await run_notification_checks(db))["resolved"] == 1
+        open_keys = (await db.scalars(select(NotificationProblem.dedup_key).where(NotificationProblem.status == "open"))).all()
+    assert open_keys == [f"{hid}:WAN_GW"]
+
+
+@pytest.mark.asyncio
+async def test_mute_endpoint_is_idempotent_validates_and_audits(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = host["id"]
+    for _ in range(2):
+        assert (await client.put(f"/api/hosts/{hid}/gateways/WAN_DHCP6/alert", json={"enabled": False})).status_code == 200
+    stored = (await client.get("/api/notifications/settings")).json()["gateway_alert_muted"]
+    assert stored == [f"{hid}:WAN_DHCP6"]  # no duplicate
+
+    assert (await client.put(f"/api/hosts/{hid}/gateways/WAN_DHCP6/alert", json={})).status_code == 422
+    assert (await client.put(f"/api/hosts/{hid}/gateways/WAN_DHCP6/alert", json={"enabled": "maybe"})).status_code == 422
+    for bad in ["x" * 65, "a%0Ab", "a%20b", "%E2%80%AE", "a%3Ab", "a%2Fb"]:
+        resp = await client.put(f"/api/hosts/{hid}/gateways/{bad}/alert", json={"enabled": False})
+        assert resp.status_code in (404, 422), (bad, resp.status_code)
+        assert resp.status_code != 200
+    assert (await client.get("/api/notifications/settings")).json()["gateway_alert_muted"] == stored
+    missing = await client.put(f"/api/hosts/{uuid.uuid4()}/gateways/WAN_GW/alert", json={"enabled": False})
+    assert missing.status_code == 404
+
+    async with SessionLocal() as db:
+        events = (await db.scalars(select(AuditEvent).where(AuditEvent.action.like("gateway_alert_%")))).all()
+    assert [e.action for e in events] == ["gateway_alert_off", "gateway_alert_off"]
+    assert events[0].host_id == hid and events[0].message == "Gateway WAN_DHCP6" and events[0].actor
+
+
+@pytest.mark.asyncio
+async def test_mute_list_is_capped(client, monkeypatch):
+    from app.routers import gateways
+    monkeypatch.setattr(gateways, "MAX_MUTED_GATEWAYS", 3)
+    hid = (await _create_host(client, "fw1", "https://fw1.test"))["id"]
+    for i in range(3):
+        assert (await client.put(f"/api/hosts/{hid}/gateways/GW{i}/alert", json={"enabled": False})).status_code == 200
+    resp = await client.put(f"/api/hosts/{hid}/gateways/GW3/alert", json={"enabled": False})
+    assert resp.status_code == 422 and resp.json()["detail"]["code"] == "GATEWAY_MUTE_LIMIT"
+    # an already muted one and un-muting still work at the limit
+    assert (await client.put(f"/api/hosts/{hid}/gateways/GW0/alert", json={"enabled": False})).status_code == 200
+    assert (await client.put(f"/api/hosts/{hid}/gateways/GW0/alert", json={"enabled": True})).status_code == 200
+
+
+@pytest.mark.asyncio
+async def test_only_admins_may_switch_a_gateway_alert(client):
+    from tests.test_api_tokens import _login
+    from tests.test_operator_role import PW, _operator
+    hid = (await _create_host(client, "fw1", "https://fw1.test"))["id"]
+    ro = await client.post("/api/users", json={"username": "reader", "password": PW, "role": "read_only"})
+    assert ro.status_code == 201
+    readers = [await _operator(client, "gw-op"), await _login("reader", PW)]
+    try:
+        for who in readers:
+            resp = await who.put(f"/api/hosts/{hid}/gateways/WAN_GW/alert", json={"enabled": False})
+            assert resp.status_code == 403, resp.text
+    finally:
+        for who in readers:
+            await who.aclose()
+    assert (await client.get("/api/notifications/settings")).json()["gateway_alert_muted"] == []
+
+
+@pytest.mark.asyncio
+async def test_gateway_list_and_dashboard_carry_the_alert_flag(client, respx_mock):
+    _mock_vip(respx_mock)
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = host["id"]
+    respx_mock.get("https://fw1.test/api/routes/gateway/status").respond(200, json=STATUS_PAYLOAD)
+    await client.put(f"/api/hosts/{hid}/gateways/LTE_GW/alert", json={"enabled": False})
+    rows = {g["name"]: g for g in (await client.get(f"/api/hosts/{hid}/gateways")).json()}
+    assert rows["LTE_GW"]["alert_enabled"] is False and rows["WAN_GW"]["alert_enabled"] is True
+    await _add_samples(uuid.UUID(hid), [{"gateways": [_gw(), _gw(name="LTE_GW")]}])
+    dash = {g["name"]: g for g in (await client.get("/api/dashboard")).json()["hosts"][0]["gateways"]}
+    assert dash["LTE_GW"]["alert_enabled"] is False and dash["WAN_GW"]["alert_enabled"] is True
+
+
+@pytest.mark.asyncio
+async def test_api_rights_endpoint_reads_the_newest_fresh_sample(client):
+    hid = (await _create_host(client, "fw1", "https://fw1.test"))["id"]
+    url = f"/api/hosts/{hid}/gateways/api-rights"
+    assert (await client.get(url)).json() == {"missing": None}  # no sample at all: unknown, not "fine"
+    await _add_samples(uuid.UUID(hid), [DENIED_GW, FINE])
+    assert (await client.get(url)).json() == {"missing": ["gateways"]}
+    await _clear_samples()
+    await _add_samples(uuid.UUID(hid), [FINE, DENIED_GW])
+    assert (await client.get(url)).json() == {"missing": []}
+    await _clear_samples()
+    await _add_samples(uuid.UUID(hid), [DENIED_GW], age_minutes=20)
+    assert (await client.get(url)).json() == {"missing": None}  # stale
+    assert (await client.get(f"/api/hosts/{uuid.uuid4()}/gateways/api-rights")).status_code == 404
+
+
+@pytest.mark.asyncio
+async def test_right_back_closes_the_alert_even_when_the_answer_is_unparsable(client):
+    """cpu_threads stays None when the CPU string has no "threads"/"cores": only the `ok` list proves the read went through."""
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    await _only("api_rights_enabled")
+    await _add_samples(hid, [{"denied": ["cpu"], "ok": ["gateways", "services", "swap"], "cpu_threads": None}] * 3)
+    assert list(await _rights_problems()) == [f"{hid}:cpu"]
+    await _clear_samples()
+    await _add_samples(hid, [{"denied": [], "ok": ["cpu", "gateways", "services", "swap"], "cpu_threads": None}] * 3)
+    assert list(await _rights_problems()) == []
+
+
+@pytest.mark.asyncio
+async def test_mute_refuses_an_unknown_gateway_name_but_unmute_always_works(client):
+    host = await _create_host(client, "fw1", "https://fw1.test")
+    hid = uuid.UUID(host["id"])
+    await _add_samples(hid, [FINE])
+    assert (await client.put(f"/api/hosts/{hid}/gateways/wan_gw/alert", json={"enabled": False})).status_code == 404
+    assert (await client.get("/api/notifications/settings")).json()["gateway_alert_muted"] == []
+    assert (await client.put(f"/api/hosts/{hid}/gateways/{_gw()['name']}/alert", json={"enabled": False})).status_code == 200
+    assert (await client.put(f"/api/hosts/{hid}/gateways/wan_gw/alert", json={"enabled": True})).status_code == 200
+
+
+@pytest.mark.asyncio
+async def test_deleting_a_host_removes_its_gateway_mutes(client):
+    keep = await _create_host(client, "fw-keep", "https://fwkeep.test")
+    gone = await _create_host(client, "fw-gone", "https://fwgone.test")
+    for h in (keep, gone):
+        assert (await client.put(f"/api/hosts/{h['id']}/gateways/WAN_GW/alert", json={"enabled": False})).status_code == 200
+    assert (await client.delete(f"/api/hosts/{gone['id']}")).status_code == 204
+    assert (await client.get("/api/notifications/settings")).json()["gateway_alert_muted"] == [f"{keep['id']}:WAN_GW"]

backend/tests/test_compliance.py

@@ -12,12 +12,12 @@ from tests.test_rules import _create_host
 
 
 def _mvc_rule(*, interface="lan", action="pass", src="any", dst="any", protocol="any", descr="r", seq=1,
-              enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0"):
+              enabled="1", dport="", direction="in", ipprotocol="inet", quick="1", src_not="0", extra=""):
     return (f"<rule><enabled>{enabled}</enabled><action>{action}</action><quick>{quick}</quick><interface>{interface}</interface>"
             f"<direction>{direction}</direction><ipprotocol>{ipprotocol}</ipprotocol><protocol>{protocol}</protocol>"
             f"<source_net>{src}</source_net><source_not>{src_not}</source_not><destination_net>{dst}</destination_net>"
             f"<destination_not>0</destination_not><destination_port>{dport}</destination_port>"
-            f"<description>{descr}</description><sequence>{seq}</sequence></rule>")
+            f"<description>{descr}</description><sequence>{seq}</sequence>{extra}</rule>")
 
 
 def _config(*, rules="", legacy="", protocol="https", ssh="", snmpd="", timeservers="0.pool.ntp.org", syslog="",
@@ -105,6 +105,52 @@ def test_rules_are_ordered_by_sequence_not_file_order():
     assert _run(_config(rules=rules))["rules_shadowed"]["params"]["examples"] == ["narrow"]
 
 
+def test_coverage_uses_networks_and_the_floating_section_like_the_firewall_does():
+    """1.30: a CIDR inside a CIDR is shadowed, a floating quick rule is evaluated before every interface rule,
+    and an interface rule never shadows a floating one that comes first."""
+    shadow = _run(_config(rules=(
+        _mvc_rule(src="10.0.0.0/8", action="block", descr="wide", seq=1)
+        + _mvc_rule(src="10.1.0.0/16", descr="inside", seq=2))))
+    assert shadow["rules_shadowed"]["params"]["examples"] == ["inside"]
+    floating = _run(_config(rules=(
+        _mvc_rule(interface="lan", src="10.1.0.0/16", descr="interface rule", seq=1)
+        + _mvc_rule(interface="lan,wan", src="10.0.0.0/8", action="block", descr="floating block", seq=9))))
+    assert floating["rules_shadowed"]["params"]["examples"] == ["interface rule"]
+    reverse = _run(_config(rules=(
+        _mvc_rule(interface="lan", src="10.0.0.0/8", action="block", descr="interface rule", seq=1)
+        + _mvc_rule(interface="lan,wan", src="10.1.0.0/16", descr="floating narrow", seq=9))))
+    assert reverse["rules_shadowed"]["passed"] is True
+
+
+def test_a_legacy_floating_rule_is_only_quick_when_it_says_so():
+    """mutation: defaulting legacy rules to quick makes a plain floating rule 'cover' everything after it."""
+    def legacy_rule(descr, *, floating, quick=None, src="any"):
+        return (f"<rule><type>block</type><interface>lan,wan</interface><ipprotocol>inet</ipprotocol>"
+                f"{'<floating>yes</floating>' if floating else ''}{f'<quick>{quick}</quick>' if quick else ''}"
+                f"<descr>{descr}</descr><source><any>1</any></source><destination><any>1</any></destination></rule>"
+                if src == "any" else
+                f"<rule><type>pass</type><interface>lan</interface><ipprotocol>inet</ipprotocol><descr>{descr}</descr>"
+                f"<source><address>{src}</address></source><destination><any/></destination></rule>")
+    later = legacy_rule("later", floating=False, src="10.0.0.0/8")
+    assert _run(_config(legacy=legacy_rule("not quick", floating=True) + later))["rules_shadowed"]["passed"] is True
+    quick = _run(_config(legacy=legacy_rule("quick", floating=True, quick="yes") + later))
+    assert quick["rules_shadowed"]["params"]["examples"] == ["later"]
+    # a non-floating legacy rule is always quick, whatever <quick> says
+    plain = _run(_config(legacy=legacy_rule("interface rule", floating=False) .replace("lan,wan", "lan") + later))
+    assert plain["rules_shadowed"]["params"]["examples"] == ["later"]
+
+
+def test_rules_at_the_same_position_in_the_config_are_not_compared():
+    rules = _mvc_rule(src="any", action="block", descr="first", seq=5) + _mvc_rule(src="10.0.0.0/8", descr="second", seq=5)
+    assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True
+
+
+def test_a_scheduled_or_tagged_rule_does_not_shadow_anything():
+    for extra in ("<sched>office</sched>", "<tagged>mark</tagged>", "<received-on>wan</received-on>"):
+        rules = _mvc_rule(src="any", action="block", descr="timed", seq=1, extra=extra) + _mvc_rule(src="10.0.0.0/8", descr="later", seq=2)
+        assert _run(_config(rules=rules))["rules_shadowed"]["passed"] is True, extra
+
+
 def test_ssh_levels():
     on = "<ssh><enabled>enabled</enabled><permitrootlogin>{}</permitrootlogin><passwordauth>{}</passwordauth></ssh>"
     worst = _run(_config(ssh=on.format(1, 1)))["ssh_hardened"]
@@ -159,6 +205,8 @@ def _mock_host(respx_mock, xml, firmware_status="none"):
     respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
     respx_mock.get(url__regex=r".*/core/backup/download/this").mock(return_value=httpx.Response(200, content=xml))
     respx_mock.get(url__regex=r".*/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": firmware_status}))
+    # An older release without ruleStats: the usage sampling falls back to the pf ruleset text.
+    respx_mock.get(url__regex=r".*/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={"message": "not found"}))
     respx_mock.get(url__regex=r".*/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
     respx_mock.get(url__regex=r".*/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
 
@@ -246,7 +294,10 @@ def test_pf_counters_are_tied_to_rule_uuids_and_summed_over_expanded_lines():
     stats["rules"]["filter rules"][f'@7 pass in quick on vtnet0 inet6 all label "{U1}"'] = {"evaluations": 2, "packets": 3, "bytes": 30}
     got = rh.parse_counters(stats)
     assert got[U1] == {"packets": 8, "bytes": 530, "evaluations": 12}
-    assert "2f2c37025cd02b5cd29fd330f8e48954" not in got  # md5 labels are automatic rules, not user rules
+    # An md5 label is either an automatic rule or an unmigrated legacy rule: only searchRule (is_automatic) tells them apart.
+    assert got["2f2c37025cd02b5cd29fd330f8e48954"]["packets"] == 9
+    assert "2f2c37025cd02b5cd29fd330f8e48954" not in rh.configured_rules(
+        {"rows": [{"uuid": "2f2c37025cd02b5cd29fd330f8e48954", "is_automatic": True}]})
     assert rh.parse_counters({}) == {} and rh.parse_counters({"rules": None}) == {}
 
 
@@ -259,7 +310,8 @@ async def _sample(host_id, stats, search, at):
     async with SessionLocal() as db:
         host = await db.get(OpnsenseHost, __import__("uuid").UUID(host_id))
         from unittest.mock import AsyncMock, patch
-        with patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
+        with patch.object(rh.client, "get_rule_stats", AsyncMock(side_effect=rh.OpnsenseAPIError("404"))), \
+             patch.object(rh.client, "get_pf_rule_statistics", AsyncMock(return_value=stats)), \
              patch.object(rh.client, "search_rules", AsyncMock(return_value=search)):
             return await rh.sample_host(db, host, now=at)
 
@@ -373,6 +425,7 @@ async def test_one_truncated_config_does_not_stop_the_others(client, respx_mock)
     for base, body in (("bad", b"<opnsense><system><webgui>"), ("good", _config())):
         respx_mock.get(f"https://{base}.test/api/core/backup/download/this").mock(return_value=httpx.Response(200, content=body))
         respx_mock.get(f"https://{base}.test/api/core/firmware/status").mock(return_value=httpx.Response(200, json={"status": "none"}))
+        respx_mock.get(f"https://{base}.test/api/firewall/filter_util/ruleStats").mock(return_value=httpx.Response(404, json={}))
         respx_mock.get(f"https://{base}.test/api/diagnostics/firewall/pfStatistics/rules").mock(return_value=httpx.Response(200, json={"rules": {"filter rules": {}}}))
         respx_mock.get(f"https://{base}.test/api/firewall/filter/searchRule").mock(return_value=httpx.Response(200, json={"rows": []}))
     resp = await client.post("/api/compliance/check")

backend/tests/test_gui_access.py

@@ -0,0 +1,493 @@
+"""GUI access (1.27): settings, one-time tickets and the gateway's internal redeem/revalidate/end API."""
+import hashlib
+import uuid
+from datetime import datetime, timedelta, timezone
+
+import pytest
+import pytest_asyncio
+from httpx import ASGITransport, AsyncClient
+from sqlalchemy import delete, select, update
+
+from app.config import settings
+from app.database import SessionLocal
+from app.main import app
+from app.models.audit import AuditEvent
+from app.models.gui_access import GuiSession, GuiTicket
+from app.models.host import OpnsenseHost
+from app.routers.auth import SECURE_COOKIE_NAME
+
+GW_TOKEN = "gateway-token-" + "x" * 40
+SITES = "https://opnexus.example:9441,https://opnexus.example:9442,https://opnexus.example:9443"
+PW = "gui-user-password-1"
+
+
+def sha(v: str) -> str:
+    return hashlib.sha256(v.encode()).hexdigest()
+
+
+@pytest.fixture
+def gui_env(monkeypatch):
+    monkeypatch.setattr(settings, "cookie_secure", True)
+    monkeypatch.setattr(settings, "gui_gateway_token", GW_TOKEN)
+    monkeypatch.setattr(settings, "opnexus_gui_sites", SITES)
+
+
+async def _https_login(username: str, password: str) -> AsyncClient:
+    ac = AsyncClient(transport=ASGITransport(app=app), base_url="https://opnexus.example")
+    resp = await ac.post("/api/auth/login", json={"username": username, "password": password})
+    assert resp.status_code == 200, resp.text
+    assert ac.cookies.get(SECURE_COOKIE_NAME)
+    return ac
+
+
+@pytest_asyncio.fixture
+async def admin(gui_env):
+    ac = await _https_login(settings.admin_username, settings.admin_password)
+    yield ac
+    await ac.aclose()
+
+
+async def _user(admin: AsyncClient, name: str, role: str) -> AsyncClient:
+    resp = await admin.post("/api/users", json={"username": name, "password": PW, "role": role})
+    if resp.status_code == 409 or (resp.status_code == 400 and "BENUTZERNAME" in resp.text):
+        users = (await admin.get("/api/users")).json()
+        uid = next(u["id"] for u in users if u["username"] == name)
+        await admin.patch(f"/api/users/{uid}/role", json={"role": role})
+    else:
+        assert resp.status_code == 201, resp.text
+    return await _https_login(name, PW)
+
+
+async def _host(name="fw-gui", base_url="https://fw-gui.test", verify_tls=True, enabled=True) -> uuid.UUID:
+    async with SessionLocal() as db:
+        host = OpnsenseHost(name=name, base_url=base_url, api_key="apikey-must-not-leak", verify_tls=verify_tls,
+                            enabled=enabled, tls_ca_pem="-----BEGIN CERTIFICATE-----\nPIN\n-----END CERTIFICATE-----\n")
+        db.add(host)
+        await db.commit()
+        return host.id
+
+
+async def _enable(admin: AsyncClient, **over) -> dict:
+    body = {"enabled": True, "idle_timeout_minutes": 15, "absolute_timeout_minutes": 60, "slot_count": 4, **over}
+    resp = await admin.put("/api/settings/gui-access", json=body)
+    assert resp.status_code == 200, resp.text
+    return resp.json()
+
+
+def _ticket_from(url: str) -> str:
+    return url.split("?t=", 1)[1]
+
+
+async def _internal(path: str, body: dict, token: str = GW_TOKEN):
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://backend:8000") as ac:
+        return await ac.post(f"/internal/gui/{path}", json=body, headers={"X-Onx-Gateway-Token": token})
+
+
+async def _redeem(admin: AsyncClient, url: str, slot: int | None = None, session_hash: str | None = None):
+    return await _internal("redeem", {
+        "ticket_hash": sha(_ticket_from(url)), "slot": slot or int(url.split(":")[2].split("/")[0]),
+        "session_hash": session_hash or sha(admin.cookies.get(SECURE_COOKIE_NAME)), "client_ip": "192.0.2.10",
+    })
+
+
+# ── settings ──────────────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_settings_default_off_and_switchable(admin):
+    data = (await admin.get("/api/settings/gui-access")).json()
+    assert data["enabled"] is False and data["available"] is True and data["slots"] == [9441, 9442, 9443]
+    assert (await _enable(admin, slot_count=2))["slots"] == [9441, 9442]
+    bad = await admin.put("/api/settings/gui-access", json={"enabled": True, "idle_timeout_minutes": 90,
+                                                              "absolute_timeout_minutes": 60, "slot_count": 2})
+    assert bad.status_code == 422
+
+
+@pytest.mark.asyncio
+async def test_cannot_enable_without_https_cookies(admin, monkeypatch):
+    monkeypatch.setattr(settings, "cookie_secure", False)
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as plain:
+        await plain.post("/api/auth/login", json={"username": settings.admin_username, "password": settings.admin_password})
+        resp = await plain.put("/api/settings/gui-access", json={"enabled": True, "idle_timeout_minutes": 15,
+                                                                  "absolute_timeout_minutes": 60, "slot_count": 4})
+        assert resp.status_code == 409 and resp.json()["detail"]["code"] == "GUI_GATEWAY_NOT_CONFIGURED"
+        assert "cookie_secure" in (await plain.get("/api/settings/gui-access")).json()["unavailable_reasons"]
+
+
+@pytest.mark.asyncio
+async def test_settings_admin_only(admin):
+    op = await _user(admin, "gui-op-settings", "operator")
+    assert (await op.get("/api/settings/gui-access")).status_code == 403
+    assert (await op.put("/api/settings/gui-access", json={"enabled": False, "idle_timeout_minutes": 15,
+                                                             "absolute_timeout_minutes": 60, "slot_count": 4})).status_code == 403
+
+
+# ── tickets ───────────────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_ticket_issued_and_only_hash_stored(admin):
+    host_id = await _host()
+    await _enable(admin)
+    resp = await admin.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 200, resp.text
+    data = resp.json()
+    assert data["url"].startswith("https://opnexus.example:9441/__onx/redeem?t=") and data["expires_in"] == 30
+    raw = _ticket_from(data["url"])
+    async with SessionLocal() as db:
+        ticket = (await db.scalars(select(GuiTicket))).one()
+        assert ticket.ticket_hash == sha(raw) and raw not in repr(ticket.__dict__)
+        assert ticket.session_hash == sha(admin.cookies.get(SECURE_COOKIE_NAME))
+        assert (ticket.expires_at - ticket.created_at) == timedelta(seconds=30)
+        events = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "gui_access"))).all()
+        assert [e.action for e in events] == ["ticket"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("role,code", [("operator", "ADMIN_REQUIRED"), ("read_only", "READ_ONLY_ACCESS")])
+async def test_ticket_refused_for_non_admins(admin, role, code):
+    host_id = await _host()
+    await _enable(admin)
+    other = await _user(admin, f"gui-{role}", role)
+    resp = await other.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 403 and resp.json()["detail"]["code"] == code
+
+
+@pytest.mark.asyncio
+async def test_ticket_refused_for_api_tokens(admin, anon_client):
+    host_id = await _host()
+    await _enable(admin)
+    token = (await admin.post("/api/tokens", json={"name": "gui", "scope": "admin", "current_password": settings.admin_password})).json()["token"]
+    resp = await anon_client.post(f"/api/hosts/{host_id}/gui-ticket", headers={"Authorization": f"Bearer {token}"})
+    assert resp.status_code == 403
+
+
+@pytest.mark.asyncio
+async def test_ticket_refused_when_switch_off_or_not_configured(admin, monkeypatch):
+    host_id = await _host()
+    resp = await admin.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "GUI_ACCESS_DISABLED"
+    await _enable(admin)
+    monkeypatch.setattr(settings, "gui_gateway_token", "short")
+    resp = await admin.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] == "GUI_GATEWAY_NOT_CONFIGURED"
+
+
+@pytest.mark.asyncio
+async def test_ticket_refused_without_secure_cookie(admin, monkeypatch):
+    host_id = await _host()
+    await _enable(admin)
+    monkeypatch.setattr(settings, "cookie_secure", False)
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as plain:
+        await plain.post("/api/auth/login", json={"username": settings.admin_username, "password": settings.admin_password})
+        resp = await plain.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 409 and resp.json()["detail"]["code"] in ("GUI_REQUIRES_HTTPS", "GUI_GATEWAY_NOT_CONFIGURED")
+    monkeypatch.setattr(settings, "cookie_secure", True)
+    # A legacy plain-name cookie is not a session on an HTTPS deployment any more (1.29): not authenticated at all.
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="https://opnexus.example") as legacy:
+        legacy.cookies.set("opnexus-session", admin.cookies.get(SECURE_COOKIE_NAME))
+        resp = await legacy.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 401
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("kwargs,reason", [
+    ({"verify_tls": False}, "tls"), ({"base_url": "http://fw-plain.test"}, "https"), ({"enabled": False}, "disabled"),
+])
+async def test_ticket_refused_for_ineligible_host(admin, kwargs, reason):
+    host_id = await _host(**kwargs)
+    await _enable(admin)
+    resp = await admin.post(f"/api/hosts/{host_id}/gui-ticket")
+    assert resp.status_code == 409 and resp.json()["detail"] == {"code": "GUI_HOST_NOT_ELIGIBLE", "params": {"reason": reason}}
+
+
+@pytest.mark.asyncio
+async def test_slots_fixed_per_firewall_and_limited(admin):
+    a, b, c = await _host("fw-a"), await _host("fw-b", "https://fw-b.test"), await _host("fw-c", "https://fw-c.test")
+    await _enable(admin, slot_count=2)
+    slot_a = (await admin.post(f"/api/hosts/{a}/gui-ticket")).json()["slot"]
+    slot_b = (await admin.post(f"/api/hosts/{b}/gui-ticket")).json()["slot"]
+    assert slot_a != slot_b and {slot_a, slot_b} == {9441, 9442}
+    assert (await admin.post(f"/api/hosts/{a}/gui-ticket")).json()["slot"] == slot_a
+    full = await admin.post(f"/api/hosts/{c}/gui-ticket")
+    assert full.status_code == 409 and full.json()["detail"]["code"] == "GUI_NO_FREE_SLOT"
+
+
+# ── internal API ──────────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_internal_api_needs_gateway_token(admin, monkeypatch):
+    body = {"gui_session_id": str(uuid.uuid4()), "session_hash": "0" * 64}
+    assert (await _internal("revalidate", body, token="wrong")).status_code == 401
+    assert (await _internal("revalidate", body, token="")).status_code == 401
+    monkeypatch.setattr(settings, "gui_gateway_token", "")
+    assert (await _internal("revalidate", body, token="")).status_code == 404
+    # Not part of the public API surface.
+    assert "/internal/gui/redeem" not in (await admin.get("/api/openapi.json")).text
+
+
+@pytest.mark.asyncio
+async def test_redeem_returns_no_credentials_and_is_single_use(admin):
+    host_id = await _host()
+    await _enable(admin)
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    resp = await _redeem(admin, url)
+    assert resp.status_code == 200, resp.text
+    data = resp.json()
+    assert data["upstream_origin"] == "https://fw-gui.test" and "PIN" in data["ca_pem"]
+    assert data["idle_timeout_seconds"] == 900 and data["revalidate_seconds"] == 20
+    assert "apikey-must-not-leak" not in resp.text and "secret" not in resp.text.lower() and "api_key" not in resp.text
+    again = await _redeem(admin, url)
+    assert again.status_code == 403 and again.json()["detail"]["reason"] == "replayed"
+    async with SessionLocal() as db:
+        actions = [e.action for e in (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "gui_access")
+                                                        .order_by(AuditEvent.created_at))).all()]
+    assert actions[:2] == ["ticket", "redeem"] and actions.count("redeem") == 2
+
+
+@pytest.mark.asyncio
+async def test_redeem_bound_to_session_slot_and_time(admin):
+    host_id = await _host()
+    await _enable(admin)
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    wrong = await _redeem(admin, url, session_hash=sha("another-browser-session"))
+    assert wrong.status_code == 403 and wrong.json()["detail"]["reason"] == "session"
+    assert (await _redeem(admin, url)).json()["detail"]["reason"] == "replayed"  # burned by the failed attempt
+
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    assert (await _redeem(admin, url, slot=9443)).json()["detail"]["reason"] == "slot"
+
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    async with SessionLocal() as db:
+        await db.execute(update(GuiTicket).where(GuiTicket.ticket_hash == sha(_ticket_from(url)))
+                         .values(expires_at=datetime.now(timezone.utc) - timedelta(seconds=1)))
+        await db.commit()
+    assert (await _redeem(admin, url)).json()["detail"]["reason"] == "expired"
+    unknown = await _internal("redeem", {"ticket_hash": sha("nope"), "session_hash": sha("x"), "slot": 9441})
+    assert unknown.status_code == 403 and unknown.json()["detail"]["reason"] == "unknown"
+
+
+async def _open(admin: AsyncClient, host_id) -> dict:
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    data = (await _redeem(admin, url)).json()
+    return {"gui_session_id": data["gui_session_id"], "session_hash": sha(admin.cookies.get(SECURE_COOKIE_NAME))}
+
+
+@pytest.mark.asyncio
+async def test_revalidate_ok_then_logout_ends_it(admin):
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open(admin, host_id)
+    ok = (await _internal("revalidate", ref)).json()
+    assert ok["ok"] is True and ok["revalidate_seconds"] == 20
+    await admin.post("/api/auth/logout")
+    assert (await _internal("revalidate", ref)).json() == {"ok": False, "reason": "session"}
+    assert (await _internal("revalidate", ref)).json()["ok"] is False
+    async with SessionLocal() as db:
+        ends = (await db.scalars(select(AuditEvent).where(AuditEvent.kind == "gui_access", AuditEvent.action == "end"))).all()
+        assert len(ends) == 1 and ends[0].details["reason"] == "session" and "duration_seconds" in ends[0].details
+
+
+@pytest.mark.asyncio
+async def test_revalidate_fails_after_demotion(admin):
+    host_id = await _host()
+    await _enable(admin)
+    second = await _user(admin, "gui-second-admin", "admin")
+    ref = await _open(second, host_id)
+    users = (await admin.get("/api/users")).json()
+    uid = next(u["id"] for u in users if u["username"] == "gui-second-admin")
+    assert (await admin.patch(f"/api/users/{uid}/role", json={"role": "operator"})).status_code == 200
+    assert (await _internal("revalidate", ref)).json()["ok"] is False
+
+
+@pytest.mark.asyncio
+async def test_revalidate_fails_when_switched_off(admin):
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open(admin, host_id)
+    await _enable(admin, enabled=False)
+    assert (await _internal("revalidate", ref)).json()["ok"] is False
+    # The live re-check alone (switch flipped without the end-all sweep) also ends it.
+    await _enable(admin)
+    ref = await _open(admin, host_id)
+    from app.models.gui_access import GuiAccessSettings
+    async with SessionLocal() as db:
+        await db.execute(update(GuiAccessSettings).values(enabled=False))
+        await db.commit()
+    assert (await _internal("revalidate", ref)).json() == {"ok": False, "reason": "disabled"}
+
+
+@pytest.mark.asyncio
+async def test_revalidate_enforces_absolute_limit_and_end(admin):
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open(admin, host_id)
+    async with SessionLocal() as db:
+        await db.execute(update(GuiSession).values(started_at=datetime.now(timezone.utc) - timedelta(minutes=61)))
+        await db.commit()
+    assert (await _internal("revalidate", ref)).json() == {"ok": False, "reason": "absolute"}
+    ref2 = await _open(admin, host_id)
+    assert (await _internal("end", {**ref2, "reason": "idle"})).json() == {"ok": True}
+    assert (await _internal("revalidate", ref2)).json() == {"ok": False, "reason": "idle"}
+    assert (await _internal("end", {**ref2, "session_hash": sha("x"), "reason": "idle"})).json() == {"ok": False}
+
+
+# ── S1: slot cooldown ─────────────────────────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_slot_cooldown_for_another_firewall_but_not_for_the_same(admin):
+    a, b = await _host("fw-cool-a"), await _host("fw-cool-b", "https://fw-cool-b.test")
+    await _enable(admin, slot_count=1)
+    ref = await _open(admin, a)
+    # Session of firewall A still open (any browser session of the user): B must not get the slot.
+    other_browser = await _https_login(settings.admin_username, settings.admin_password)
+    blocked = await other_browser.post(f"/api/hosts/{b}/gui-ticket")
+    assert blocked.status_code == 409 and blocked.json()["detail"]["code"] == "GUI_SLOT_COOLING_DOWN"
+    # Just ended: still cooling down for B, free again at once for A.
+    assert (await _internal("end", {**ref, "reason": "logout"})).json() == {"ok": True}
+    blocked = await admin.post(f"/api/hosts/{b}/gui-ticket")
+    assert blocked.status_code == 409 and blocked.json()["detail"] == {
+        "code": "GUI_SLOT_COOLING_DOWN", "params": {"minutes": "5"}}
+    assert (await admin.post(f"/api/hosts/{a}/gui-ticket")).status_code == 200
+    # After the cooldown B takes the slot (the pending ticket for A is gone).
+    async with SessionLocal() as db:
+        await db.execute(delete(GuiTicket))
+        await db.execute(update(GuiSession).values(ended_at=datetime.now(timezone.utc) - timedelta(minutes=4)))
+        await db.commit()
+    still = await admin.post(f"/api/hosts/{b}/gui-ticket")
+    assert still.status_code == 409 and still.json()["detail"] == {
+        "code": "GUI_SLOT_COOLING_DOWN", "params": {"minutes": "1"}}
+    async with SessionLocal() as db:
+        await db.execute(update(GuiSession).values(ended_at=datetime.now(timezone.utc) - timedelta(minutes=5, seconds=5)))
+        await db.commit()
+    assert (await admin.post(f"/api/hosts/{b}/gui-ticket")).status_code == 200
+    await other_browser.aclose()
+
+
+@pytest.mark.asyncio
+async def test_cooling_slot_skipped_when_another_slot_is_free(admin):
+    a, b = await _host("fw-cool-a"), await _host("fw-cool-b", "https://fw-cool-b.test")
+    await _enable(admin, slot_count=2)
+    ref = await _open(admin, a)
+    slot_a = (await admin.post(f"/api/hosts/{a}/gui-ticket")).json()["slot"]
+    await _internal("end", {**ref, "reason": "logout"})
+    slot_b = (await admin.post(f"/api/hosts/{b}/gui-ticket")).json()["slot"]
+    assert slot_b != slot_a  # A's slot cools down although it is the least recently used
+
+
+@pytest.mark.asyncio
+async def test_cooldown_is_per_user(admin):
+    a, b = await _host("fw-cool-a"), await _host("fw-cool-b", "https://fw-cool-b.test")
+    await _enable(admin, slot_count=1)
+    ref = await _open(admin, a)
+    await _internal("end", {**ref, "reason": "logout"})
+    second = await _user(admin, "gui-cool-admin-2", "admin")  # another person, another browser
+    assert (await second.post(f"/api/hosts/{b}/gui-ticket")).status_code == 200
+
+
+# ── S4: target change, session cap ────────────────────────────────────────
+
+async def _open_with_fp(admin: AsyncClient, host_id) -> dict:
+    url = (await admin.post(f"/api/hosts/{host_id}/gui-ticket")).json()["url"]
+    data = (await _redeem(admin, url)).json()
+    assert len(data["target_fp"]) == 64
+    return {"gui_session_id": data["gui_session_id"], "session_hash": sha(admin.cookies.get(SECURE_COOKIE_NAME)),
+            "target_fp": data["target_fp"]}
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("field,value", [
+    ("base_url", "https://fw-moved.test"),
+    ("tls_ca_pem", "-----BEGIN CERTIFICATE-----\nOTHER\n-----END CERTIFICATE-----\n"),
+])
+async def test_session_ends_when_target_changes(admin, field, value):
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open_with_fp(admin, host_id)
+    assert (await _internal("revalidate", ref)).json()["ok"] is True
+    async with SessionLocal() as db:
+        await db.execute(update(OpnsenseHost).where(OpnsenseHost.id == host_id).values(**{field: value}))
+        await db.commit()
+    assert (await _internal("revalidate", ref)).json() == {"ok": False, "reason": "target"}
+    async with SessionLocal() as db:
+        assert (await db.scalars(select(GuiSession.end_reason))).one() == "target"
+
+
+@pytest.mark.asyncio
+async def test_session_ends_when_overlay_route_changes(admin, monkeypatch):
+    from app.services import overlay_service
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open_with_fp(admin, host_id)
+    monkeypatch.setattr(overlay_service, "proxy_for", lambda base_url: "http://wg-hub:8888")
+    assert (await _internal("revalidate", ref)).json() == {"ok": False, "reason": "target"}
+
+
+@pytest.mark.asyncio
+async def test_revalidate_without_fingerprint_still_works(admin):
+    host_id = await _host()
+    await _enable(admin)
+    ref = await _open_with_fp(admin, host_id)
+    ref.pop("target_fp")  # older gateway image
+    assert (await _internal("revalidate", ref)).json()["ok"] is True
+
+
+@pytest.mark.asyncio
+async def test_session_cap_rejects_new_ticket_and_keeps_running_ones(admin, monkeypatch):
+    from app.services import gui_ticket_service
+    monkeypatch.setattr(gui_ticket_service, "MAX_GATEWAY_SESSIONS", 2)
+    a, b, c = await _host("fw-a"), await _host("fw-b", "https://fw-b.test"), await _host("fw-c", "https://fw-c.test")
+    await _enable(admin)
+    ref_a, ref_b = await _open(admin, a), await _open(admin, b)
+    full = await admin.post(f"/api/hosts/{c}/gui-ticket")
+    assert full.status_code == 409 and full.json()["detail"]["code"] == "GUI_TOO_MANY_SESSIONS"
+    assert full.json()["detail"]["params"] == {"max": "2"}
+    # The same firewall again replaces its own session and does not count twice.
+    assert (await admin.post(f"/api/hosts/{a}/gui-ticket")).status_code == 200
+    assert (await _internal("revalidate", ref_a)).json()["ok"] and (await _internal("revalidate", ref_b)).json()["ok"]
+    # Ended sessions free room again.
+    await _internal("end", {**ref_b, "reason": "logout"})
+    assert (await admin.post(f"/api/hosts/{c}/gui-ticket")).status_code == 200
+
+
+def test_session_cap_equal_in_gateway_and_backend():
+    from app import gui_gateway
+    from app.services import gui_ticket_service
+    assert gui_gateway.MAX_SESSIONS == gui_ticket_service.MAX_GATEWAY_SESSIONS
+
+
+# ── Review findings 1.29 (S1 follow-up) ───────────────────────────────────
+
+@pytest.mark.asyncio
+async def test_redeem_refuses_to_take_over_a_slot_that_holds_another_firewall(admin):
+    """Two quick tickets both saw the slot free: the second redeem must not supersede A's session for B."""
+    a, b = await _host("fw-race-a"), await _host("fw-race-b", "https://fw-race-b.test")
+    await _enable(admin, slot_count=1)
+    ref = await _open(admin, a)
+    async with SessionLocal() as db:
+        gs = await db.get(GuiSession, uuid.UUID(ref["gui_session_id"]))
+        slot, user_id = gs.slot, gs.user_id
+        now = datetime.now(timezone.utc)
+        db.add(GuiTicket(ticket_hash=sha("race-ticket"), session_hash=ref["session_hash"], user_id=user_id, host_id=b,
+                         slot=slot, created_at=now, expires_at=now + timedelta(seconds=30)))
+        await db.commit()
+    denied = await _internal("redeem", {"ticket_hash": sha("race-ticket"), "slot": slot, "session_hash": ref["session_hash"],
+                                        "client_ip": "192.0.2.10"})
+    assert denied.status_code != 200 and "slot" in str(denied.json())  # the same refusal as a slot conflict at redeem
+    assert (await _internal("revalidate", ref)).json()["ok"] is True  # A's session was left alone
+
+
+@pytest.mark.asyncio
+async def test_session_nobody_re_checked_counts_as_ended_for_the_cooldown(admin):
+    a, b = await _host("fw-zombie-a"), await _host("fw-zombie-b", "https://fw-zombie-b.test")
+    await _enable(admin, slot_count=1)
+    await _open(admin, a)
+    other_browser = await _https_login(settings.admin_username, settings.admin_password)  # another browser of the same user
+    async with SessionLocal() as db:  # the gateway died: no /end, no re-check for 2 minutes
+        await db.execute(update(GuiSession).values(last_checked_at=datetime.now(timezone.utc) - timedelta(minutes=2)))
+        await db.commit()
+    blocked = await other_browser.post(f"/api/hosts/{b}/gui-ticket")
+    assert blocked.status_code == 409 and blocked.json()["detail"] == {"code": "GUI_SLOT_COOLING_DOWN", "params": {"minutes": "3"}}
+    async with SessionLocal() as db:  # silent for longer than the cooldown: free again, also past the absolute limit
+        await db.execute(update(GuiSession).values(last_checked_at=datetime.now(timezone.utc) - timedelta(minutes=6)))
+        await db.commit()
+    assert (await other_browser.post(f"/api/hosts/{b}/gui-ticket")).status_code == 200
+    await other_browser.aclose()