Compare commits

...
13 Commits
Author SHA1 Message Date
Julian Boldt 5e5d51b57f Release v1.23.5
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fix (Ursache des Vorfalls vom 20.10. in 1.23.4):** OPNsense merkt ein CARP-VIP zum Löschen vor, wenn eine Änderung kein `interface` mitschickt,
  und der anschließende „Anwenden“-Schritt entfernt es dann live, ohne es wieder anzulegen. OPNexus schickt bei jeder VIP-Änderung jetzt die
  bisherige Schnittstelle mit (wie schon die Adresse). Auf dem Test-Cluster mit echten Firewalls nachgestellt und bestätigt.
- **Einstellungen › OPNexus aktualisieren:** Die Anleitung zum Aktivieren nennt jetzt alles Nötige in einem Block (auch `UPDATE_CHECK_URL`) und weist
  darauf hin, dass Updates per Klick Release-Images brauchen; wer aus dem Quellcode baut, aktualisiert mit `git pull` und `docker compose up -d --build`.
2026-10-20 14:22:12 +02:00
Julian Boldt 1aba618ad5 Release v1.23.4
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Fix (Produktiv-Vorfall 2026-10-20):** Das Ändern der Beschreibung eines CARP-VIP wendete das VIP neu an und ließ es auf beiden Firewalls
  im Zustand INIT hängen – OPNexus zeigte trotzdem „alles in Ordnung“. Nur die Beschreibung zu ändern, wendet jetzt nichts mehr an. Nach jeder
  anderen CARP-VIP-Änderung prüft OPNexus den Live-Zustand (MASTER/BACKUP), wendet bei Bedarf einmal erneut an und meldet sonst „Angewendet,
  Prüfung weicht ab“.
- **CARP-Überwachung je VHID:** Neue Warnungen `vip_down` (ein VIP ist auf einem Node nicht aktiv) und `role_mismatch` (ein Node ist nur für
  einen Teil der VIPs MASTER). Bisher zählte der Host schon als MASTER, sobald ein einziges VIP MASTER war.
2026-10-20 13:56:25 +02:00
Julian Boldt ba954c05bf Release v1.23.3
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **SSO:** Liefert der Anbieter `preferred_username`/`email` nicht im ID-Token (Authelia), fragt OPNexus sie beim Userinfo-Endpunkt ab; der neue
  Benutzer heißt dann `alice` statt wie die kryptische Benutzerkennung des Anbieters. Geprüft gegen Keycloak 26 und Authelia 4.39; S3-Offsite gegen
  Garage, MinIO, RustFS und Versity S3 Gateway.
2026-10-20 12:56:39 +02:00
Julian Boldt 63d014b14c Release v1.23.2
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **SSO-Fix (gefunden beim Test gegen ein echtes Keycloak):** Die Standard-Scopes enthalten nicht mehr `groups` – Keycloak und Entra ID
  lehnten die Anmeldung damit mit `invalid_scope` ab. Bestehende Einstellungen mit dem alten Standardwert werden umgestellt (Migration
  `0018`); Anbieter wie Okta tragen `groups` bei Bedarf selbst ein (Hinweis in den Einstellungen).
2026-10-20 09:17:31 +02:00
Julian Boldt 4838dad588 Release v1.23.1
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Sicherheit (Ergebnis eines unabhängigen Reviews der Versionen 1.18–1.23):**
  - Aufgabentrennung: Mit „nur Admins rollen aus“ ist jetzt auch das Löschen verwalteter Objekte (entfernt sie von den Firewalls), Zertifikat-/ACME-Push, Dienste starten/stoppen und „Reihenfolge erneut durchsetzen“ den Admins vorbehalten, nicht nur `/deploy`.
  - Backup-Änderungsansicht: weitere Geheimnis-Felder (`otp_seed`, `pass`, `pwd`, …) werden ausgeblendet, die Platzhalter sind pro Abruf gesalzen (kein Wörterbuchabgleich, kein Vergleich über Abrufe/Firewalls).
  - SSO: der Anmeldeversuch ist per Cookie an den Browser gebunden (kein Login-CSRF, abgefangene Code/State-Paare sind nutzlos); SSO-Sitzungen dauern 12 Stunden statt 30 Tage; Gruppen-Strings werden nur an Komma/Zeilenumbruch getrennt (Gruppen mit Leerzeichen); Fehlertext des Anbieters wird für das Log bereinigt.
  - Das Ändern des eigenen Passworts widerruft jetzt auch die eigenen API-Tokens.
  - Management-Tunnel: der Hub routet nicht mehr zwischen Firewalls oder ins Compose-Netz (`FORWARD` verworfen, `ip_forward=0`); loopback-/link-local-Netze sind als Tunnel-Netz nicht erlaubt.
  - Sicherheitslage: eine defekte `config.xml` stoppt die Prüfung der übrigen Firewalls nicht mehr; die Auswertung läuft außerhalb der Ereignisschleife.
  - Behoben: S3-Offsite mit Port 443 im Endpunkt (Signatur passte nicht); Variablenwerte mit angehängtem Zeilenumbruch werden abgelehnt.
- Fertiges **Grafana-Dashboard** (`docs/grafana-opnexus.json`) für die Prometheus-Metriken von OPNexus.
2026-10-20 02:24:15 +02:00
Julian Boldt c4c8c4ff3a Release v1.23.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Sicherheitslage im Alltag**: Dashboard-Karte mit den Firewalls mit dem niedrigsten Wert; Benachrichtigung bei kritischen und hohen
  Befunden (je Firewall und Prüfung, mit „behoben“-Meldung; standardmäßig aus; Migration `0017`); druckbarer **Sicherheitsbericht**
  (Knopf „Bericht“, Drucken/PDF über den Browser).
2026-10-20 02:02:04 +02:00
Julian Boldt 295555cfd6 Release v1.22.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
  (Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
  Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
  Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
2026-10-20 01:42:30 +02:00
Julian Boldt f5dbe8a41a Release v1.21.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Management-Tunnel** (System › Management-Tunnel): Firewalls hinter NAT/CGNAT verwalten. Ein optionaler WireGuard-Hub auf dem
  OPNexus-Server (Compose-Profil `overlay`, Container `wg-hub` aus dem Backend-Image mit `NET_ADMIN`) nimmt die Firewalls entgegen, die sich
  nach außen einwählen; OPNexus erreicht deren API über einen kleinen CONNECT-Proxy des Hubs, der nur ins Tunnel-Netz verbindet. Je Firewall
  eine Tunnel-Adresse und ein eigener Pre-Shared Key, Anleitung mit kopierbaren Werten, Live-Status (Handshake, Datenmenge). Der Hub nimmt
  keine neuen Verbindungen aus dem Tunnel an. Ende-zu-Ende gegen eine echte OPNsense 26.7 geprüft (Handshake, API, Zertifikat festsetzen).
  Neue Tabellen `overlay_settings`, `overlay_peers` (Migration `0015`); das Backend-Image enthält jetzt `wireguard-tools`, `iproute2`, `iptables`.
2026-10-20 01:21:45 +02:00
Julian Boldt 552c88e425 Release v1.20.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Single Sign-on (OIDC)**: Anmeldung über Authentik, Keycloak, Entra ID, Google u. a. (Authorization Code mit PKCE). Die Rolle
  kommt bei jeder Anmeldung aus den Gruppen des Anbieters, ohne zugeordnete Gruppe kommt niemand hinein; SSO-Konten werden nie mit
  lokalen Konten verknüpft; lokale Anmeldung bleibt als Notzugang. ID-Token werden streng geprüft (Signatur nur asymmetrisch, Aussteller,
  Zielgruppe, Ablauf, Nonce, einmaliger State), gegen einen nachgebauten Anbieter getestet (inkl. Fälschungsversuche). Neue Tabellen
  `oidc_settings`, `oidc_login_states`, Spalten `admin_users.auth_source/oidc_subject` (Migration `0013`). Neue Abhängigkeit: PyJWT.
- **Behoben:** `opnexus backup --full` und `opnexus restore` (Vollsicherung, in 1.19.0 beschrieben) fehlten im 1.19.0-Bundle und -Updater-Image,
  weil die Änderung am Skript nicht mit eingecheckt war; sie sind erst ab dieser Version enthalten.
- **Aufgabentrennung**: Schalter, mit dem nur Admins ausrollen dürfen, Operatoren aber vorbereiten (Migration `0014`).
2026-10-20 00:50:45 +02:00
Julian Boldt b2c69f51a0 Release v1.19.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Backups, Governance, Regelnutzung. Auf einem Test-Stack mit drei Test-Firewalls geprüft; die Vollsicherung Ende-zu-Ende
(sichern, Schaden anrichten, zurückspielen), siehe Handbuch „Konfigurationssicherungen“, „Sichern und Wiederherstellen“.

- **Rolle „Operator“**: rollt aus und betreibt (Regeln, NAT, VPN, Wartung, Sicherungen), verwaltet aber keine
  Benutzer, Einstellungen oder Hosts und lädt keine Roh-Backups herunter. Abgelehnte Versuche stehen im Audit.
  Tokens eines Operators gehen bis zum Bereich „Ausrollen“.
- **Audit-Export** (Einstellungen): Syslog (RFC 5424, UDP/TCP/TLS) und/oder signierter Webhook, mindestens
  einmal zugestellt, ohne Objektinhalte. Neue Tabelle `audit_export_settings` (Migration `0010`).
- **Geplante Backups** mit Aufbewahrung, **verschlüsseltem Offsite-Ziel** (S3-kompatibel, AES-256-GCM,
  Passphrase bleibt auf dem Server), Benachrichtigungen bei Problemen und einer **Änderungsansicht** zwischen
  zwei Sicherungen (Geheimnisse nur als Hash). Neue Tabellen `backup_settings`, `backup_uploads`, Spalte
  `notification_settings.backup_failed_enabled` (Migration `0011`). Die S3-Signatur ist gegen die offiziellen
  AWS-Beispiele geprüft; ein Lauf gegen einen echten Anbieter steht noch aus.
- **Vollsicherung** `opnexus backup --full` und `opnexus restore`: Datenbank und alle Geheimnis-Volumes in einer
  verschlüsselten Datei; Ende-zu-Ende auf dem Test-Stack geprüft (Sichern, Schaden anrichten, Zurückspielen).
- **Regelnutzung**: wann hat jede Regel zuletzt etwas getroffen (aus den pf-Zählern, überlebt Neuladen und
  Neustart); Befund „Regeln in Benutzung“ in der Sicherheitslage nach 30 Tagen ohne Treffer. Neue Tabelle
  `rule_hits` (Migration `0012`).
- Änderungen an Einstellungen (`/api/settings/…`) und das Auslösen von Sicherungen stehen jetzt im Audit.
2026-10-20 00:26:57 +02:00
Julian Boldt 218a1f14ac Release v1.18.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Automatisierung und Sicherheitslage. Ende-zu-Ende gegen einen Test-Stack und drei Test-Firewalls geprüft,
siehe Handbuch „Sicherheitslage“ und „Automatisierung“.

- **API-Tokens** (System › API-Tokens): persönliche Tokens für Skripte, CI, Prometheus und KI-Assistenten,
  Bereiche *Lesen*, *Ausrollen*, *Admin*. Nur als Hash gespeichert, einmal angezeigt, widerrufbar, mit
  Ablaufdatum; Anlegen verlangt das Passwort. Ein Token überschreitet nie die Rechte seines Besitzers.
  Tokens verwalten und das eigene Konto geht nur in einer normalen Anmeldung. Geheimnisse (Roh-Backups,
  Benachrichtigungseinstellungen) liest nur der Bereich *Admin*. Das Audit nennt das Token. Neue Tabelle
  `api_tokens` (Migration `0008`).
- **Prometheus-Metriken** unter `GET /api/metrics` (Host erreichbar, Last, Speicher, Platte, CARP-Rolle,
  offene Aktionen und Probleme, Sicherheitswert), nur aus der Datenbank, ohne Anfragen an die Firewalls.
- **MCP-Endpunkt** `POST /api/mcp` (nur lesend) für KI-Assistenten: Flottenübersicht, Hosts, Auslastung, CARP-Ereignisse,
  Audit, Suche und ein Leser für jeden GET-Endpunkt, mit den Rechten des Tokens.
- **OpenAPI-Schema** jetzt unter `/api/openapi.json` (bisher unter `/openapi.json`, vom Proxy nicht
  erreichbar); die Swagger-/ReDoc-Seiten entfallen (ihre CDN-Skripte verbietet die CSP).
- **Sicherheitslage** (Monitoring › Sicherheitslage): Härtungs- und Regel-Hygiene-Prüfungen je Firewall mit Wert
  0–100: Allow-all am WAN, HTTPS, SSH, SNMP, verdeckte/doppelte/unbeschriftete Regeln, Zeitserver, externe
  Logs, Firmware, Backup-Alter. Liest die Konfiguration nur lesend, speichert sie nicht, alle 6 Stunden und
  auf Knopfdruck. Neue Tabelle `compliance_results` (Migration `0009`).
- Behoben: Änderungen am Benutzer in einer Anfrage (z. B. 2FA aktivieren) gingen verloren, wenn die
  Middleware den Benutzer aus einer anderen Datenbank-Sitzung lieferte; er wird jetzt in der Sitzung der
  Anfrage neu geladen.
2026-10-19 23:54:02 +02:00
Julian Boldt 79783d6d62 Release v1.17.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
Geplante Wartung: Fehler zeigen sich sofort statt erst um 3 Uhr nachts, und der Master lässt sich
unbeaufsichtigt aktualisieren. Auf einem Test-Cluster live geprüft (Neustart und ein echtes Update
mit Umschaltung), siehe Handbuch „Geplante Wartung“.

- **Rollenprüfung beim Einplanen:** Ein Host, der in einem Cluster gerade CARP-MASTER ist,
  lässt sich nicht mehr einplanen (Neustart/Update/Major-Upgrade). Bisher wurde der Plan
  angenommen und scheiterte erst zur Uhrzeit. Der Planungsdialog zeigt den Grund an und
  sperrt den Button. Ist die Rolle nicht feststellbar (Host gerade nicht erreichbar), wird
  weiter angenommen; die volle Prüfung läuft wie bisher zum Zündzeitpunkt.
- **Benachrichtigung „Geplante Wartung nicht ausgeführt“:** Eine fehlgeschlagene geplante
  Wartung meldet sich per Webhook/E-Mail (einzeln schaltbar in den Einstellungen, standardmäßig
  an). Offen für drei Tage nach dem geplanten Zeitpunkt, dann ohne „behoben“-Meldung geschlossen.
  Neue Spalte `notification_settings.scheduled_maintenance_failed_enabled` (Migration `0006`).
- **Automatische CARP-Umschaltung (Option pro Plan, standardmäßig aus):** Ein geplanter Neustart,
  Update oder Major-Upgrade auf dem MASTER schaltet vorher auf den Partner um, prüft die
  Übernahme, führt die Aktion aus, wartet auf die Rückkehr des Hosts und schaltet zurück. Fehler
  vor dem Start der Aktion stellen den Normalbetrieb wieder her; danach bleibt der Host bewusst im
  Wartungsmodus (Partner trägt den Verkehr) und es kommt eine Benachrichtigung. Der Ablauf steht in
  der Datenbank und überlebt einen Neustart des Backends. Neue Spalten an `scheduled_maintenance`
  (Migration `0007`).
- **Backup vor geplantem Update:** Ein geplantes Update oder Major-Upgrade zieht unmittelbar vor
  dem Start selbst ein Konfigurationsbackup (nur lesend auf der Firewall). Bisher scheiterte jeder
  Plan, der mehr als 24 Stunden in der Zukunft lag, an der Backup-Aktualität. Schlägt das Backup
  fehl, wird nicht aktualisiert.
2026-10-19 16:45:07 +02:00
Julian Boldt c15417869b Handbook rewritten as a user manual
docs/HANDBUCH.md is now organised for operators (install, TLS, configuration, users, firewalls, rules, network, VPN, certificates, HA, updates, monitoring, operation, security, troubleshooting, limits). The unused September screenshots are removed; README and configuration comments point at the new section names.
2026-10-10 14:36:58 +02:00
166 changed files with 17153 additions and 3686 deletions

No files matched your search

+11 -6
View File
@@ -36,7 +36,7 @@ PROXY_BIND_ADDRESS=0.0.0.0
# Host interface the backend/frontend ports are published on. Defaults
# to 127.0.0.1 (this host only) if unset. Set to 0.0.0.0 for access from
# other machines without a reverse proxy (see README's "Produktivbetrieb"
# other machines without a reverse proxy (see the handbook, "Zugriff aus dem Netz und TLS"
# for the safer TLS-proxy alternative), or to one specific interface IP.
BIND_ADDRESS=
# Host-Ports von Backend und Frontend (nur nötig, wenn 8100/3100 schon belegt
@@ -44,15 +44,15 @@ BIND_ADDRESS=
# BACKEND_PORT=8100
# FRONTEND_PORT=3100
# Set to true once this is running behind TLS (see README's
# "Produktivbetrieb" section) — otherwise the session cookie is sent
# Set to true once this is running behind TLS (see the handbook,
# "Zugriff aus dem Netz und TLS") — otherwise the session cookie is sent
# unencrypted and browsers won't even attach a Secure cookie without
# HTTPS in the first place.
COOKIE_SECURE=false
# Set to true ONLY when running behind a reverse proxy you trust to
# always set/overwrite X-Forwarded-For itself (every example in README's
# "Produktivbetrieb" does). Used to attribute failed logins to the real
# always set/overwrite X-Forwarded-For itself (every example in the handbook's
# "Zugriff aus dem Netz und TLS" does). Used to attribute failed logins to the real
# client IP for the login lockout instead of the proxy's own IP. Leave
# false (default) if the app is reachable directly with no proxy in
# front — otherwise any client could spoof this header to dodge the
@@ -80,7 +80,7 @@ TRUST_PROXY_HEADERS=false
# Startet den Updater-Container, mit dem Admins unter Einstellungen › „OPNexus
# aktualisieren“ ein Update auslösen können. Er ist der einzige Container mit
# Docker-Socket-Zugriff (= root-gleiche Rechte auf diesem Host) und standardmäßig
# AUS. Benötigt außerdem UPDATE_CHECK_URL (oben). Siehe Handbuch.
# AUS. Benötigt außerdem UPDATE_CHECK_URL (oben). Siehe Handbuch, "Updates per Klick".
# COMPOSE_PROFILES=updater
# ── Transportsicherheit (nur ändern, wenn nötig) ──────────────────────
@@ -93,3 +93,8 @@ TRUST_PROXY_HEADERS=false
# SMTP_CA_FILE=/data/smtp-ca.pem
# SMTP-Anmeldung ohne Verschlüsselung (Sicherheit "none") bleibt gesperrt.
# ALLOW_INSECURE_SMTP_AUTH=false
# Management-Tunnel (WireGuard-Hub für Firewalls hinter NAT), optional.
# Zum Aktivieren (Handbuch „Firewalls hinter NAT“): COMPOSE_PROFILES=overlay
# und den UDP-Port aus dem Internet erreichbar machen.
#OVERLAY_PORT=51820
+148
View File
@@ -31,6 +31,154 @@ den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin …
## [1.23.5] "Harrier" — 2026-10-20
- **Fix (Ursache des Vorfalls vom 20.10. in 1.23.4):** OPNsense merkt ein CARP-VIP zum Löschen vor, wenn eine Änderung kein `interface` mitschickt,
und der anschließende „Anwenden“-Schritt entfernt es dann live, ohne es wieder anzulegen. OPNexus schickt bei jeder VIP-Änderung jetzt die
bisherige Schnittstelle mit (wie schon die Adresse). Auf dem Test-Cluster mit echten Firewalls nachgestellt und bestätigt.
- **Einstellungen › OPNexus aktualisieren:** Die Anleitung zum Aktivieren nennt jetzt alles Nötige in einem Block (auch `UPDATE_CHECK_URL`) und weist
darauf hin, dass Updates per Klick Release-Images brauchen; wer aus dem Quellcode baut, aktualisiert mit `git pull` und `docker compose up -d --build`.
## [1.23.4] "Harrier" — 2026-10-20
- **Fix (Produktiv-Vorfall 2026-10-20):** Das Ändern der Beschreibung eines CARP-VIP wendete das VIP neu an und ließ es auf beiden Firewalls
im Zustand INIT hängen – OPNexus zeigte trotzdem „alles in Ordnung“. Nur die Beschreibung zu ändern, wendet jetzt nichts mehr an. Nach jeder
anderen CARP-VIP-Änderung prüft OPNexus den Live-Zustand (MASTER/BACKUP), wendet bei Bedarf einmal erneut an und meldet sonst „Angewendet,
Prüfung weicht ab“.
- **CARP-Überwachung je VHID:** Neue Warnungen `vip_down` (ein VIP ist auf einem Node nicht aktiv) und `role_mismatch` (ein Node ist nur für
einen Teil der VIPs MASTER). Bisher zählte der Host schon als MASTER, sobald ein einziges VIP MASTER war.
## [1.23.3] "Harrier" — 2026-10-20
- **SSO:** Liefert der Anbieter `preferred_username`/`email` nicht im ID-Token (Authelia), fragt OPNexus sie beim Userinfo-Endpunkt ab; der neue
Benutzer heißt dann `alice` statt wie die kryptische Benutzerkennung des Anbieters. Geprüft gegen Keycloak 26 und Authelia 4.39; S3-Offsite gegen
Garage, MinIO, RustFS und Versity S3 Gateway.
## [1.23.2] "Harrier" — 2026-10-20
- **SSO-Fix (gefunden beim Test gegen ein echtes Keycloak):** Die Standard-Scopes enthalten nicht mehr `groups` – Keycloak und Entra ID
lehnten die Anmeldung damit mit `invalid_scope` ab. Bestehende Einstellungen mit dem alten Standardwert werden umgestellt (Migration
`0018`); Anbieter wie Okta tragen `groups` bei Bedarf selbst ein (Hinweis in den Einstellungen).
## [1.23.1] "Harrier" — 2026-10-20
- **Sicherheit (Ergebnis eines unabhängigen Reviews der Versionen 1.18–1.23):**
- Aufgabentrennung: Mit „nur Admins rollen aus“ ist jetzt auch das Löschen verwalteter Objekte (entfernt sie von den Firewalls), Zertifikat-/ACME-Push, Dienste starten/stoppen und „Reihenfolge erneut durchsetzen“ den Admins vorbehalten, nicht nur `/deploy`.
- Backup-Änderungsansicht: weitere Geheimnis-Felder (`otp_seed`, `pass`, `pwd`, …) werden ausgeblendet, die Platzhalter sind pro Abruf gesalzen (kein Wörterbuchabgleich, kein Vergleich über Abrufe/Firewalls).
- SSO: der Anmeldeversuch ist per Cookie an den Browser gebunden (kein Login-CSRF, abgefangene Code/State-Paare sind nutzlos); SSO-Sitzungen dauern 12 Stunden statt 30 Tage; Gruppen-Strings werden nur an Komma/Zeilenumbruch getrennt (Gruppen mit Leerzeichen); Fehlertext des Anbieters wird für das Log bereinigt.
- Das Ändern des eigenen Passworts widerruft jetzt auch die eigenen API-Tokens.
- Management-Tunnel: der Hub routet nicht mehr zwischen Firewalls oder ins Compose-Netz (`FORWARD` verworfen, `ip_forward=0`); loopback-/link-local-Netze sind als Tunnel-Netz nicht erlaubt.
- Sicherheitslage: eine defekte `config.xml` stoppt die Prüfung der übrigen Firewalls nicht mehr; die Auswertung läuft außerhalb der Ereignisschleife.
- Behoben: S3-Offsite mit Port 443 im Endpunkt (Signatur passte nicht); Variablenwerte mit angehängtem Zeilenumbruch werden abgelehnt.
- Fertiges **Grafana-Dashboard** (`docs/grafana-opnexus.json`) für die Prometheus-Metriken von OPNexus.
## [1.23.0] "Harrier" — 2026-10-20
- **Sicherheitslage im Alltag**: Dashboard-Karte mit den Firewalls mit dem niedrigsten Wert; Benachrichtigung bei kritischen und hohen
Befunden (je Firewall und Prüfung, mit „behoben“-Meldung; standardmäßig aus; Migration `0017`); druckbarer **Sicherheitsbericht**
(Knopf „Bericht“, Drucken/PDF über den Browser).
## [1.22.0] "Peregrine" — 2026-10-20
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
(Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
## [1.21.0] "Osprey" — 2026-10-20
- **Management-Tunnel** (System › Management-Tunnel): Firewalls hinter NAT/CGNAT verwalten. Ein optionaler WireGuard-Hub auf dem
OPNexus-Server (Compose-Profil `overlay`, Container `wg-hub` aus dem Backend-Image mit `NET_ADMIN`) nimmt die Firewalls entgegen, die sich
nach außen einwählen; OPNexus erreicht deren API über einen kleinen CONNECT-Proxy des Hubs, der nur ins Tunnel-Netz verbindet. Je Firewall
eine Tunnel-Adresse und ein eigener Pre-Shared Key, Anleitung mit kopierbaren Werten, Live-Status (Handshake, Datenmenge). Der Hub nimmt
keine neuen Verbindungen aus dem Tunnel an. Ende-zu-Ende gegen eine echte OPNsense 26.7 geprüft (Handshake, API, Zertifikat festsetzen).
Neue Tabellen `overlay_settings`, `overlay_peers` (Migration `0015`); das Backend-Image enthält jetzt `wireguard-tools`, `iproute2`, `iptables`.
## [1.20.0] "Lilli" — 2026-10-20
- **Single Sign-on (OIDC)**: Anmeldung über Authentik, Keycloak, Entra ID, Google u. a. (Authorization Code mit PKCE). Die Rolle
kommt bei jeder Anmeldung aus den Gruppen des Anbieters, ohne zugeordnete Gruppe kommt niemand hinein; SSO-Konten werden nie mit
lokalen Konten verknüpft; lokale Anmeldung bleibt als Notzugang. ID-Token werden streng geprüft (Signatur nur asymmetrisch, Aussteller,
Zielgruppe, Ablauf, Nonce, einmaliger State), gegen einen nachgebauten Anbieter getestet (inkl. Fälschungsversuche). Neue Tabellen
`oidc_settings`, `oidc_login_states`, Spalten `admin_users.auth_source/oidc_subject` (Migration `0013`). Neue Abhängigkeit: PyJWT.
- **Behoben:** `opnexus backup --full` und `opnexus restore` (Vollsicherung, in 1.19.0 beschrieben) fehlten im 1.19.0-Bundle und -Updater-Image,
weil die Änderung am Skript nicht mit eingecheckt war; sie sind erst ab dieser Version enthalten.
- **Aufgabentrennung**: Schalter, mit dem nur Admins ausrollen dürfen, Operatoren aber vorbereiten (Migration `0014`).
## [1.19.0] "Lanner" — 2026-10-20
Backups, Governance, Regelnutzung. Auf einem Test-Stack mit drei Test-Firewalls geprüft; die Vollsicherung Ende-zu-Ende
(sichern, Schaden anrichten, zurückspielen), siehe Handbuch „Konfigurationssicherungen“, „Sichern und Wiederherstellen“.
- **Rolle „Operator“**: rollt aus und betreibt (Regeln, NAT, VPN, Wartung, Sicherungen), verwaltet aber keine
Benutzer, Einstellungen oder Hosts und lädt keine Roh-Backups herunter. Abgelehnte Versuche stehen im Audit.
Tokens eines Operators gehen bis zum Bereich „Ausrollen“.
- **Audit-Export** (Einstellungen): Syslog (RFC 5424, UDP/TCP/TLS) und/oder signierter Webhook, mindestens
einmal zugestellt, ohne Objektinhalte. Neue Tabelle `audit_export_settings` (Migration `0010`).
- **Geplante Backups** mit Aufbewahrung, **verschlüsseltem Offsite-Ziel** (S3-kompatibel, AES-256-GCM,
Passphrase bleibt auf dem Server), Benachrichtigungen bei Problemen und einer **Änderungsansicht** zwischen
zwei Sicherungen (Geheimnisse nur als Hash). Neue Tabellen `backup_settings`, `backup_uploads`, Spalte
`notification_settings.backup_failed_enabled` (Migration `0011`). Die S3-Signatur ist gegen die offiziellen
AWS-Beispiele geprüft; ein Lauf gegen einen echten Anbieter steht noch aus.
- **Vollsicherung** `opnexus backup --full` und `opnexus restore`: Datenbank und alle Geheimnis-Volumes in einer
verschlüsselten Datei; Ende-zu-Ende auf dem Test-Stack geprüft (Sichern, Schaden anrichten, Zurückspielen).
- **Regelnutzung**: wann hat jede Regel zuletzt etwas getroffen (aus den pf-Zählern, überlebt Neuladen und
Neustart); Befund „Regeln in Benutzung“ in der Sicherheitslage nach 30 Tagen ohne Treffer. Neue Tabelle
`rule_hits` (Migration `0012`).
- Änderungen an Einstellungen (`/api/settings/…`) und das Auslösen von Sicherungen stehen jetzt im Audit.
## [1.18.0] "Kestrel" — 2026-10-19
Automatisierung und Sicherheitslage. Ende-zu-Ende gegen einen Test-Stack und drei Test-Firewalls geprüft,
siehe Handbuch „Sicherheitslage“ und „Automatisierung“.
- **API-Tokens** (System › API-Tokens): persönliche Tokens für Skripte, CI, Prometheus und KI-Assistenten,
Bereiche *Lesen*, *Ausrollen*, *Admin*. Nur als Hash gespeichert, einmal angezeigt, widerrufbar, mit
Ablaufdatum; Anlegen verlangt das Passwort. Ein Token überschreitet nie die Rechte seines Besitzers.
Tokens verwalten und das eigene Konto geht nur in einer normalen Anmeldung. Geheimnisse (Roh-Backups,
Benachrichtigungseinstellungen) liest nur der Bereich *Admin*. Das Audit nennt das Token. Neue Tabelle
`api_tokens` (Migration `0008`).
- **Prometheus-Metriken** unter `GET /api/metrics` (Host erreichbar, Last, Speicher, Platte, CARP-Rolle,
offene Aktionen und Probleme, Sicherheitswert), nur aus der Datenbank, ohne Anfragen an die Firewalls.
- **MCP-Endpunkt** `POST /api/mcp` (nur lesend) für KI-Assistenten: Flottenübersicht, Hosts, Auslastung, CARP-Ereignisse,
Audit, Suche und ein Leser für jeden GET-Endpunkt, mit den Rechten des Tokens.
- **OpenAPI-Schema** jetzt unter `/api/openapi.json` (bisher unter `/openapi.json`, vom Proxy nicht
erreichbar); die Swagger-/ReDoc-Seiten entfallen (ihre CDN-Skripte verbietet die CSP).
- **Sicherheitslage** (Monitoring › Sicherheitslage): Härtungs- und Regel-Hygiene-Prüfungen je Firewall mit Wert
0–100: Allow-all am WAN, HTTPS, SSH, SNMP, verdeckte/doppelte/unbeschriftete Regeln, Zeitserver, externe
Logs, Firmware, Backup-Alter. Liest die Konfiguration nur lesend, speichert sie nicht, alle 6 Stunden und
auf Knopfdruck. Neue Tabelle `compliance_results` (Migration `0009`).
- Behoben: Änderungen am Benutzer in einer Anfrage (z. B. 2FA aktivieren) gingen verloren, wenn die
Middleware den Benutzer aus einer anderen Datenbank-Sitzung lieferte; er wird jetzt in der Sitzung der
Anfrage neu geladen.
## [1.17.0] "Imperial" — 2026-10-19
Geplante Wartung: Fehler zeigen sich sofort statt erst um 3 Uhr nachts, und der Master lässt sich
unbeaufsichtigt aktualisieren. Auf einem Test-Cluster live geprüft (Neustart und ein echtes Update
mit Umschaltung), siehe Handbuch „Geplante Wartung“.
- **Rollenprüfung beim Einplanen:** Ein Host, der in einem Cluster gerade CARP-MASTER ist,
lässt sich nicht mehr einplanen (Neustart/Update/Major-Upgrade). Bisher wurde der Plan
angenommen und scheiterte erst zur Uhrzeit. Der Planungsdialog zeigt den Grund an und
sperrt den Button. Ist die Rolle nicht feststellbar (Host gerade nicht erreichbar), wird
weiter angenommen; die volle Prüfung läuft wie bisher zum Zündzeitpunkt.
- **Benachrichtigung „Geplante Wartung nicht ausgeführt“:** Eine fehlgeschlagene geplante
Wartung meldet sich per Webhook/E-Mail (einzeln schaltbar in den Einstellungen, standardmäßig
an). Offen für drei Tage nach dem geplanten Zeitpunkt, dann ohne „behoben“-Meldung geschlossen.
Neue Spalte `notification_settings.scheduled_maintenance_failed_enabled` (Migration `0006`).
- **Automatische CARP-Umschaltung (Option pro Plan, standardmäßig aus):** Ein geplanter Neustart,
Update oder Major-Upgrade auf dem MASTER schaltet vorher auf den Partner um, prüft die
Übernahme, führt die Aktion aus, wartet auf die Rückkehr des Hosts und schaltet zurück. Fehler
vor dem Start der Aktion stellen den Normalbetrieb wieder her; danach bleibt der Host bewusst im
Wartungsmodus (Partner trägt den Verkehr) und es kommt eine Benachrichtigung. Der Ablauf steht in
der Datenbank und überlebt einen Neustart des Backends. Neue Spalten an `scheduled_maintenance`
(Migration `0007`).
- **Backup vor geplantem Update:** Ein geplantes Update oder Major-Upgrade zieht unmittelbar vor
dem Start selbst ein Konfigurationsbackup (nur lesend auf der Firewall). Bisher scheiterte jeder
Plan, der mehr als 24 Stunden in der Zukunft lag, an der Backup-Aktualität. Schlägt das Backup
fehl, wird nicht aktualisiert.
## [1.16.0] "Hobby" — 2026-10-10
Erste Schritte nach der Installation. Ende-zu-Ende gegen einen isolierten Stack getestet, siehe
+8 -4
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
> **Stand:** `1.16.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
> **Stand:** `1.23.5` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen
@@ -22,7 +22,11 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
- **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus.
- **Einrichtungsassistent** — geführter Erststart nach der Installation: eigenes Passwort, Zwei-Faktor und die erste Firewall verbinden.
- **Selbst-Update** — OPNexus aktualisiert sich auf Wunsch aus der eigenen Oberfläche, mit Fortschritt und automatischem Rollback (optional).
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
- **Automatisierung** — persönliche API-Tokens (Lesen / Ausrollen / Admin), Prometheus-Metriken, ein nur lesender MCP-Endpunkt für KI-Assistenten und ein OpenAPI-Schema.
- **Sicherheitslage** — ein Wert von 0 bis 100 je Firewall aus Härtungs- und Regel-Hygiene-Prüfungen (offene WAN-Regeln, schwaches SSH/SNMP, verdeckte Regeln, alte Backups …), nur lesend.
- **Backups & Wiederherstellung** — geplante Konfigurationssicherungen mit Aufbewahrung, verschlüsselte Offsite-Kopie auf S3-kompatiblen Speicher, Änderungsansicht zwischen Sicherungen und eine vollständige verschlüsselte Sicherung von OPNexus selbst (`opnexus backup --full`).
- **Single Sign-on** — Anmeldung über deinen Identitätsanbieter (OIDC); die Rolle kommt bei jeder Anmeldung aus dessen Gruppen.
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Operator / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
Alle Screenshots zeigen fiktive Beispieldaten.
@@ -52,13 +56,13 @@ scripts/opnexus password
Nach der ersten Anmeldung öffnet sich von selbst ein kurzer **Einrichtungsassistent**: eigenes Passwort setzen, optional Zwei-Faktor-Anmeldung einschalten, die erste Firewall verbinden (samt Anleitung für den API-Key in OPNsense). Er lässt sich überspringen und jederzeit unter System › Einrichtung wieder öffnen.
Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelieferter Caddy-Proxy) siehe das [Handbuch](docs/HANDBUCH.md#zugriff-von-außerhalb-dieser-vm--produktivbetrieb-mit-tls).
Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelieferter Caddy-Proxy) siehe das [Handbuch](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls).
**Fertige Images** (amd64) liegen in der Registry dieses Gitea. In der `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.16.0
OPNEXUS_VERSION=1.23.5
```
```bash
+8 -4
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](docs/screenshots/readme-rules-en.webp)
> **Status:** `1.16.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
> **Status:** `1.23.5` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does
@@ -22,7 +22,11 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
- **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode.
- **Setup assistant** — a guided first run after installation: your own password, two-factor, and connecting the first firewall.
- **Self-update** — OPNexus can update itself from its web interface, with progress and automatic rollback (opt-in).
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / read-only), keyboard operation, dark mode, German and English UI.
- **Automation** — personal API tokens (read / deploy / admin), Prometheus metrics, a read-only MCP endpoint for AI assistants and an OpenAPI schema.
- **Security posture** — a 0–100 score per firewall from hardening and rule-hygiene checks (open WAN rules, weak SSH/SNMP, shadowed rules, stale backups …), read-only.
- **Backups & recovery** — scheduled config backups with retention, an encrypted offsite copy to S3-compatible storage, a redacted change view between backups, and a full encrypted backup/restore of OPNexus itself (`opnexus backup --full`).
- **Single sign-on** — sign in with your identity provider (OIDC); roles come from its groups at every login.
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / operator / read-only), keyboard operation, dark mode, German and English UI.
![Cluster view](docs/screenshots/readme-cluster-en.webp)
@@ -54,13 +58,13 @@ scripts/opnexus password
After the first sign-in a short **setup assistant** opens on its own: set your own password, optionally enable two-factor sign-in, connect your first firewall (including how to create the API key in OPNsense). You can skip it and reopen it any time under System › Setup.
For access from other machines and for production use with TLS (bundled Caddy proxy), see the [handbook](docs/HANDBUCH.md#zugriff-von-außerhalb-dieser-vm--produktivbetrieb-mit-tls) (German).
For access from other machines and for production use with TLS (bundled Caddy proxy), see the [handbook](docs/HANDBUCH.md#zugriff-aus-dem-netz-und-tls) (German).
**Prebuilt images** (amd64) are published to this Gitea's registry. In `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.16.0
OPNEXUS_VERSION=1.23.5
```
```bash
+4
View File
@@ -8,6 +8,10 @@ WORKDIR /app
COPY scripts/install_pg_client.sh /tmp/install_pg_client.sh
RUN /tmp/install_pg_client.sh 16 && rm /tmp/install_pg_client.sh
# Management-Overlay (nur der wg-hub-Container nutzt das, siehe app/overlay_hub.py).
RUN apt-get update && apt-get install -y --no-install-recommends wireguard-tools iproute2 iptables \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
+54 -5
View File
@@ -8,13 +8,19 @@ from starlette.responses import JSONResponse
from app.database import SessionLocal
from app.errors import ErrorCode
from app.routers.auth import COOKIE_NAME
from app.services import auth_service
from app.routers import governance
from app.routers.auth import client_ip
from app.services import api_token_service, auth_service
_PUBLIC_PATHS = {
"/api/auth/login",
"/api/auth/logout",
"/api/auth/me",
"/api/health",
# SSO: the browser arrives here without a session; every step is checked inside.
"/api/auth/oidc/config",
"/api/auth/oidc/start",
"/api/auth/oidc/callback",
}
# /docs, /redoc and /openapi.json used to be public — fully documenting a
@@ -35,6 +41,8 @@ _PUBLIC_PATHS = {
_READ_ONLY_ALLOWED_WRITES = {
"/api/auth/change-password",
"/api/settings",
# Read-only JSON-RPC tools (see routers/mcp.py).
"/api/mcp",
# Self-scoped: a user managing their own second factor.
"/api/auth/2fa/setup",
"/api/auth/2fa/enable",
@@ -48,6 +56,7 @@ _READ_ONLY_ALLOWED_WRITES = {
_SECURITY_KINDS = {
"ipsec": "ipsec_tunnel", "openvpn": "openvpn_instance", "wireguard": "wireguard_tunnel",
"certificates": "certificate", "acme": "acme", "notifications": "notification", "auth": "auth",
"tokens": "api_token",
}
_COLLECTIONS = {"tunnels", "instances", "managed", "accounts", "certificates", "settings"}
@@ -86,19 +95,37 @@ class AuthMiddleware(BaseHTTPMiddleware):
if request.url.path in _PUBLIC_PATHS:
return await call_next(request)
api_token = None
bearer = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
token = request.cookies.get(COOKIE_NAME, "")
if token:
if bearer is not None:
# An API token wins over a cookie; a bad one never falls back to it.
async with SessionLocal() as db:
resolved = await api_token_service.resolve(db, bearer, client_ip(request))
user, api_token = resolved if resolved is not None else (None, None)
token = bearer
elif token:
async with SessionLocal() as db:
user = await auth_service.resolve_session(db, token)
if token:
if user is not None and api_token is not None and api_token_service.denies(
api_token.scope, request.method, request.url.path
):
return JSONResponse(
status_code=403,
content={"detail": {"code": ErrorCode.API_TOKEN_SCOPE_FORBIDDEN.value, "params": {"scope": api_token.scope}}},
)
if user is not None:
parts = request.url.path.strip("/").split("/")
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system"}
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user", "system": "system", "backups": "backup"}
writing = request.method in ("POST", "PUT", "PATCH", "DELETE")
if len(parts) > 1 and parts[1] in _SECURITY_KINDS:
event = {**_security_event(parts, request.method), "details": {"method": request.method, "path": request.url.path}}
kind = event["kind"]
else:
kind = kinds.get(parts[1]) if len(parts) > 1 else None
if len(parts) > 2 and parts[1] == "settings":
kind = "settings" # where OPNexus stores/sends things (own prefs at /api/settings stay unaudited)
action = ("delete" if request.method == "DELETE" else parts[-1] if len(parts) > 3 else "edit" if request.method in ("PUT", "PATCH") else "create")
event = {"kind": kind, "action": action, "entity_id": parts[2] if len(parts) > 2 else None,
"host_id": parts[2] if kind == "host" and len(parts) > 2 else None,
@@ -109,10 +136,12 @@ class AuthMiddleware(BaseHTTPMiddleware):
user.role == "read_only"
and request.method not in ("GET", "HEAD", "OPTIONS")
and request.url.path not in _READ_ONLY_ALLOWED_WRITES
# Own API tokens (the router limits a read_only user to read scope).
and not request.url.path.startswith("/api/tokens")
):
if tracked:
# Refused attempts at security-relevant writes stay attributable.
ctx = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": event["action"]})
ctx = audit_service.context.set({"actor": user.username if api_token is None else f"{user.username} (API: {api_token.name})"[:150], "id": str(uuid.uuid4()), "action": event["action"]})
try:
await audit_service.persist(**event, status="denied", message="Schreibzugriff für Nur-Lese-Konto verweigert")
finally:
@@ -121,8 +150,28 @@ class AuthMiddleware(BaseHTTPMiddleware):
status_code=403,
content={"detail": {"code": ErrorCode.READ_ONLY_ACCESS.value, "params": {}}},
)
if user.role == "operator" and api_token_service.operator_denies(request.method, request.url.path):
if tracked:
ctx = audit_service.context.set({"actor": user.username if api_token is None else f"{user.username} (API: {api_token.name})"[:150], "id": str(uuid.uuid4()), "action": event["action"]})
try:
await audit_service.persist(**event, status="denied", message="Nur für Administratoren")
finally:
audit_service.context.reset(ctx)
return JSONResponse(status_code=403, content={"detail": {"code": ErrorCode.ADMIN_REQUIRED.value, "params": {}}})
if user.role == "operator" and api_token_service.is_rollout(request.method, request.url.path):
async with SessionLocal() as gdb:
allowed = (await governance.get_settings(gdb)).operators_may_deploy
if not allowed:
ctx = audit_service.context.set({"actor": user.username if api_token is None else f"{user.username} (API: {api_token.name})"[:150], "id": str(uuid.uuid4()), "action": event["action"]})
try:
await audit_service.persist(**event, status="denied", message="Ausrollen nur durch Administratoren (Aufgabentrennung)")
finally:
audit_service.context.reset(ctx)
return JSONResponse(status_code=403, content={"detail": {"code": ErrorCode.DEPLOY_NEEDS_ADMIN.value, "params": {}}})
request.state.user = user
token = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": event["action"]})
request.state.api_token = api_token
actor = user.username if api_token is None else f"{user.username} (API: {api_token.name})"[:150]
token = audit_service.context.set({"actor": actor, "id": str(uuid.uuid4()), "action": event["action"]})
try:
if tracked:
await audit_service.persist(**event, status="requested")
+31
View File
@@ -0,0 +1,31 @@
"""Decrypt an offsite backup (``.opxbak``) back into a config.xml.
docker compose exec -T backend python -m app.backup_decrypt < file.opxbak > config.xml
The passphrase comes from the BACKUP_PASSPHRASE environment variable or, if that is
not set, from the passphrase stored in OPNexus. For a disaster-recovery machine
without OPNexus run it with the passphrase you kept elsewhere:
docker run --rm -i -e BACKUP_PASSPHRASE=... <backend image> python -m app.backup_decrypt < file.opxbak > config.xml
"""
import os
import sys
from app.services import backup_crypto, notification_secret_store
def main() -> int:
passphrase = os.environ.get("BACKUP_PASSPHRASE") or notification_secret_store.read_secret("backup_passphrase")
if not passphrase:
print("Keine Passphrase: BACKUP_PASSPHRASE setzen.", file=sys.stderr)
return 2
try:
sys.stdout.buffer.write(backup_crypto.decrypt(sys.stdin.buffer.read(), passphrase))
except backup_crypto.DecryptError as exc:
print(str(exc), file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+5
View File
@@ -53,6 +53,11 @@ class Settings(BaseSettings):
# reachable both by IP and by name: "https://10.0.0.5,https://opnexus.lan".
frontend_origin: str = ""
opnsense_secret_dir: str = "/data/opnsense-secrets"
# Management overlay (WireGuard hub in the optional `wg-hub` container, see services/overlay_service.py).
overlay_dir: str = "/data/overlay"
overlay_proxy_url: str = "http://wg-hub:8888"
# Lab only: allow an http:// endpoint for offsite backups (credentials in the clear).
allow_insecure_offsite: bool = False
opnsense_backup_dir: str = "/data/opnsense-backups"
opnsense_timeout: float = 10.0
# Firewall API credentials are sent as HTTP Basic auth. Plain http://
+17
View File
@@ -49,6 +49,8 @@ class ErrorCode(str, Enum):
FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998 = "FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998"
FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR = "FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR"
GEPLANTE_WARTUNG_NICHT_GEFUNDEN = "GEPLANTE_WARTUNG_NICHT_GEFUNDEN"
GEPLANTE_WARTUNG_NUR_AUF_DEM_CARP_BACKUP_MOEGLICH = "GEPLANTE_WARTUNG_NUR_AUF_DEM_CARP_BACKUP_MOEGLICH"
CARP_UEBERGABE_BRAUCHT_EINEN_CLUSTER = "CARP_UEBERGABE_BRAUCHT_EINEN_CLUSTER"
GROUP_HAS_ASSIGNED_ITEMS = "GROUP_HAS_ASSIGNED_ITEMS"
GRUPPENMITGLIEDER_ODER_REGELN_WURDEN_GEAENDERT_AUSWIRKUNGEN_ERNEUT_PRUEFEN = "GRUPPENMITGLIEDER_ODER_REGELN_WURDEN_GEAENDERT_AUSWIRKUNGEN_ERNEUT_PRUEFEN"
GRUPPENNAME_BEREITS_VERGEBEN = "GRUPPENNAME_BEREITS_VERGEBEN"
@@ -152,6 +154,21 @@ class ErrorCode(str, Enum):
SERVICE_NOT_FOUND = "SERVICE_NOT_FOUND"
SERVICE_LOCKED = "SERVICE_LOCKED"
READ_ONLY_ACCESS = "READ_ONLY_ACCESS"
API_TOKEN_LIMIT_REACHED = "API_TOKEN_LIMIT_REACHED"
OFFSITE_TEST_FAILED = "OFFSITE_TEST_FAILED"
OIDC_TEST_FAILED = "OIDC_TEST_FAILED"
VARIABLES_MISSING = "VARIABLES_MISSING"
VARIABLE_INVALID = "VARIABLE_INVALID"
OVERLAY_INVALID = "OVERLAY_INVALID"
OVERLAY_SUBNET_LOCKED = "OVERLAY_SUBNET_LOCKED"
OVERLAY_NAME_TAKEN = "OVERLAY_NAME_TAKEN"
OVERLAY_PEER_NOT_FOUND = "OVERLAY_PEER_NOT_FOUND"
DEPLOY_NEEDS_ADMIN = "DEPLOY_NEEDS_ADMIN"
OIDC_MANAGED_USER = "OIDC_MANAGED_USER"
BACKUP_UNLESBAR = "BACKUP_UNLESBAR"
API_TOKEN_NOT_FOUND = "API_TOKEN_NOT_FOUND"
API_TOKEN_SCOPE_FORBIDDEN = "API_TOKEN_SCOPE_FORBIDDEN"
API_TOKEN_SCOPE_TOO_HIGH = "API_TOKEN_SCOPE_TOO_HIGH"
def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:
+208
View File
@@ -0,0 +1,208 @@
"""Full OPNexus backup: the database and every secret store in one encrypted file.
The nightly database dump is not enough to rebuild an OPNexus server: the firewalls' API secrets, TOTP
secrets, VPN/certificate keys and the stored config snapshots live in separate volumes. This packs all of it
(``database.dump`` + the data directories + ``manifest.json``) into a tar.gz and encrypts it with a passphrase
(AES-256-GCM, services/backup_crypto.py). Without the passphrase the file is useless, so keep it elsewhere.
Normally driven by the host CLI:
./opnexus backup --full [FILE] # writes an encrypted .opxfull file
./opnexus restore FILE # onto a fresh or existing installation
or directly (passphrase in FULL_BACKUP_PASSPHRASE):
python -m app.full_backup create > file.opxfull
python -m app.full_backup info < file.opxfull # manifest only
python -m app.full_backup restore < file.opxfull
"""
import io
import json
import os
import subprocess
import sys
import tarfile
import tempfile
from datetime import datetime, timezone
from pathlib import Path
from sqlalchemy.engine import make_url
from app.config import settings
from app.services import backup_crypto
from app.version import APP_VERSION
FORMAT = 1
MIN_PASSPHRASE = 12
# manifest key -> settings attribute holding the directory
_DATA_DIRS = {
"opnsense_secrets": "opnsense_secret_dir",
"opnsense_backups": "opnsense_backup_dir",
"ipsec_psk_secrets": "ipsec_psk_secret_dir",
"openvpn_key_secrets": "openvpn_key_secret_dir",
"wireguard_key_secrets": "wireguard_key_secret_dir",
"cert_key_secrets": "cert_key_secret_dir",
"acme_secrets": "acme_secret_dir",
"totp_secrets": "totp_secret_dir",
"notification_secrets": "notification_secret_dir",
"overlay": "overlay_dir",
}
class FullBackupError(Exception):
pass
def _dir(name: str) -> Path:
return Path(getattr(settings, _DATA_DIRS[name]))
def _db_args() -> tuple[list[str], dict]:
url = make_url(settings.database_url)
return (
["-h", url.host or "localhost", "-p", str(url.port or 5432), "-U", url.username or "", "-d", url.database or ""],
{**os.environ, "PGPASSWORD": url.password or ""},
)
def _dump_database(target: Path) -> None:
args, env = _db_args()
skip = [f"--exclude-table-data={t}" for t in ("firewall_log_entries", "host_resource_samples")]
subprocess.run(["pg_dump", *args, "-F", "c", *skip, "-f", str(target)], env=env, check=True, capture_output=True, timeout=1800)
def _restore_database(source: Path) -> None:
args, env = _db_args()
result = subprocess.run(
["pg_restore", *args, "--clean", "--if-exists", "--no-owner", "--exit-on-error", str(source)],
env=env, capture_output=True, text=True, timeout=1800,
)
if result.returncode != 0:
raise FullBackupError(f"pg_restore fehlgeschlagen: {result.stderr.strip()[:400]}")
def _head_revision() -> str:
from alembic.script import ScriptDirectory
from app import migrate
return ScriptDirectory.from_config(migrate._config()).get_current_head()
def create(passphrase: str, *, dump=_dump_database, revision=None) -> bytes:
if len(passphrase) < MIN_PASSPHRASE:
raise FullBackupError(f"Die Passphrase braucht mindestens {MIN_PASSPHRASE} Zeichen")
manifest = {
"format": FORMAT, "app_version": APP_VERSION, "db_revision": revision or _head_revision(),
"created_at": datetime.now(timezone.utc).isoformat(), "data": sorted(_DATA_DIRS),
}
plain = io.BytesIO()
with tempfile.TemporaryDirectory() as tmp, tarfile.open(fileobj=plain, mode="w:gz") as tar:
raw = json.dumps(manifest, indent=1).encode()
info = tarfile.TarInfo("manifest.json")
info.size = len(raw)
tar.addfile(info, io.BytesIO(raw))
dump_path = Path(tmp) / "database.dump"
dump(dump_path)
tar.add(dump_path, arcname="database.dump")
for name in _DATA_DIRS:
path = _dir(name)
if path.is_dir():
tar.add(path, arcname=f"data/{name}")
return backup_crypto.encrypt(plain.getvalue(), passphrase)
def _open(blob: bytes, passphrase: str) -> tarfile.TarFile:
try:
plain = backup_crypto.decrypt(blob, passphrase)
except backup_crypto.DecryptError as exc:
raise FullBackupError(str(exc)) from None
try:
return tarfile.open(fileobj=io.BytesIO(plain), mode="r:gz")
except tarfile.TarError:
raise FullBackupError("Das ist keine vollständige OPNexus-Sicherung (Archiv unlesbar)") from None
def _manifest(tar: tarfile.TarFile) -> dict:
try:
member = tar.extractfile("manifest.json")
manifest = json.loads(member.read()) if member else None
except (KeyError, ValueError):
manifest = None
if not isinstance(manifest, dict) or manifest.get("format") != FORMAT:
raise FullBackupError("Das ist keine vollständige OPNexus-Sicherung (Manifest fehlt oder ist neuer)")
return manifest
def info(blob: bytes, passphrase: str) -> dict:
with _open(blob, passphrase) as tar:
return _manifest(tar)
def restore(blob: bytes, passphrase: str, *, restore_db=_restore_database, head=None) -> dict:
"""Put database and data directories back. The caller stops the backend first."""
with _open(blob, passphrase) as tar:
manifest = _manifest(tar)
if not _older_or_equal(manifest.get("db_revision"), head):
raise FullBackupError(
f"Die Sicherung stammt aus einer neueren OPNexus-Version ({manifest.get('app_version')}). "
"Erst auf diese Version aktualisieren, dann zurückspielen."
)
members = tar.getmembers()
for member in members: # reject anything that could escape: absolute paths, .., links, devices
if member.name.startswith(("/", "\\")) or ".." in Path(member.name).parts or not (member.isfile() or member.isdir()):
raise FullBackupError(f"Unzulässiger Eintrag im Archiv: {member.name!r}")
known = {f"data/{name}" for name in _DATA_DIRS}
with tempfile.TemporaryDirectory() as tmp:
dump = tar.extractfile("database.dump")
if dump is None:
raise FullBackupError("Die Sicherung enthält keine Datenbank")
dump_path = Path(tmp) / "database.dump"
dump_path.write_bytes(dump.read())
restore_db(dump_path)
restored = []
for member in members:
parts = Path(member.name).parts
if len(parts) < 2 or parts[0] != "data" or f"data/{parts[1]}" not in known:
continue
target_root = _dir(parts[1])
target = target_root.joinpath(*parts[2:])
if member.isdir():
target.mkdir(parents=True, exist_ok=True, mode=0o700)
continue
target.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
source = tar.extractfile(member)
fd = os.open(target, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(fd, "wb") as out:
out.write(source.read())
os.utime(target, (member.mtime, member.mtime)) # snapshot ages come from file times
restored.append(parts[1])
return {**manifest, "files_restored": len(restored)}
def _older_or_equal(revision: str | None, head: str | None) -> bool:
"""Revisions are zero-padded numbers ('0009'): a dump from an older schema is fine, the app migrates it."""
try:
return int(revision) <= int(head or _head_revision())
except (TypeError, ValueError):
return False
def _main(argv: list[str]) -> int:
passphrase = os.environ.get("FULL_BACKUP_PASSPHRASE", "")
if len(argv) != 2 or argv[1] not in ("create", "info", "restore"):
print(__doc__, file=sys.stderr)
return 2
try:
if argv[1] == "create":
sys.stdout.buffer.write(create(passphrase))
elif argv[1] == "info":
print(json.dumps(info(sys.stdin.buffer.read(), passphrase), indent=1))
else:
print(json.dumps(restore(sys.stdin.buffer.read(), passphrase), indent=1))
except (FullBackupError, subprocess.SubprocessError, OSError) as exc:
print(f"Fehler: {exc}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(_main(sys.argv))
+120 -4
View File
@@ -5,6 +5,7 @@ from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.openapi.utils import get_openapi
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from sqlalchemy.exc import DBAPIError
@@ -17,7 +18,7 @@ from app.security_middleware import OriginCheckMiddleware, SecurityHeadersMiddle
from app.config import settings
from app.database import SessionLocal
from app import migrate
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router, tokens as tokens_router, metrics as metrics_router, mcp as mcp_router, compliance as compliance_router, audit_export as audit_export_router, backups as backups_router, oidc as oidc_router, governance as governance_router, overlay as overlay_router, variables as variables_router
from app.routers import one_to_one
from app.services.one_to_one_deploy_service import sweep_expired_one_to_one_deployments
from app.routers import outbound_nat
@@ -49,6 +50,10 @@ from app.routers import resource_samples as resource_samples_router
from app.services import resource_monitor_service
from app.routers import scheduled_maintenance as scheduled_maintenance_router
from app.services import scheduled_maintenance_service
from app.services import compliance_service
from app.services import audit_export_service
from app.services import overlay_service, variables_service
from app.services import backup_schedule_service
from app.services import carp_event_service, carp_maintenance_window_service
# Without this, Python's root logger defaults to WARNING with no handler,
@@ -229,6 +234,7 @@ async def _resource_poll_loop():
_SCHEDULED_MAINTENANCE_POLL_SECONDS = 60
_RUNNING_MAINTENANCE_POLL_SECONDS = 15
async def _scheduled_maintenance_poll_loop():
@@ -244,9 +250,64 @@ async def _scheduled_maintenance_poll_loop():
"Geplante Wartung: %d fällig, %d ausgeführt, %d blockiert/fehlgeschlagen",
result["due"], result["fired"], result["blocked_or_failed"],
)
in_progress = await scheduled_maintenance_service.advance_running(db)
except Exception:
logger.exception("Geplante-Wartung-Poll fehlgeschlagen")
await asyncio.sleep(_SCHEDULED_MAINTENANCE_POLL_SECONDS)
in_progress = 0
# A handover run is stepped quickly (the partner should take over within a minute or two);
# otherwise the relaxed cadence is plenty.
await asyncio.sleep(_RUNNING_MAINTENANCE_POLL_SECONDS if in_progress else _SCHEDULED_MAINTENANCE_POLL_SECONDS)
_COMPLIANCE_FIRST_RUN_SECONDS = 120
_COMPLIANCE_INTERVAL_SECONDS = 6 * 3600
async def _compliance_poll_loop():
"""Sixth background loop: refreshes every enabled host's security-posture result.
Each check is one read-only config download per firewall, so every six hours
is plenty; the first run waits a little so a restart is not a thundering herd."""
await asyncio.sleep(_COMPLIANCE_FIRST_RUN_SECONDS)
while True:
try:
async with SessionLocal() as db:
await compliance_service.check_all(db)
except Exception:
logger.exception("Sicherheitslage-Prüfung fehlgeschlagen")
await asyncio.sleep(_COMPLIANCE_INTERVAL_SECONDS)
_AUDIT_EXPORT_POLL_SECONDS = 10
async def _audit_export_poll_loop():
"""Seventh background loop: forwards new audit events (syslog / webhook) when configured.
Cheap when off (one settings read); keeps draining while batches come back full."""
while True:
shipped = 0
try:
async with SessionLocal() as db:
shipped = await audit_export_service.run_once(db)
except Exception:
logger.exception("Audit-Export fehlgeschlagen")
await asyncio.sleep(0 if shipped >= audit_export_service.BATCH else _AUDIT_EXPORT_POLL_SECONDS)
_BACKUP_POLL_SECONDS = 15 * 60
async def _backup_poll_loop():
"""Eighth background loop: takes due scheduled config backups, prunes, copies offsite."""
await asyncio.sleep(90)
while True:
try:
async with SessionLocal() as db:
result = await backup_schedule_service.run_due(db)
if result["taken"] or result["failed"] or result["uploaded"]:
logger.info("Geplante Backups: %s", result)
except Exception:
logger.exception("Geplante Backups fehlgeschlagen")
await asyncio.sleep(_BACKUP_POLL_SECONDS)
@asynccontextmanager
@@ -254,6 +315,10 @@ async def lifespan(app: FastAPI):
await migrate.run()
async with SessionLocal() as db:
await auth_service.bootstrap_admin(db)
try:
await overlay_service.sync(db) # hub config + proxy rule follow the database after every start
except Exception:
logger.exception("Overlay-Konfiguration konnte nicht geschrieben werden")
if any(o.startswith("https://") for o in settings.allowed_origins) and not settings.cookie_secure:
logger.warning(
"FRONTEND_ORIGIN is https:// but COOKIE_SECURE is false — the session cookie "
@@ -267,6 +332,9 @@ async def lifespan(app: FastAPI):
log_ingest_task = asyncio.create_task(_log_ingest_poll_loop())
resource_task = asyncio.create_task(_resource_poll_loop())
scheduled_maintenance_task = asyncio.create_task(_scheduled_maintenance_poll_loop())
compliance_task = asyncio.create_task(_compliance_poll_loop())
audit_export_task = asyncio.create_task(_audit_export_poll_loop())
backup_task = asyncio.create_task(_backup_poll_loop())
try:
yield
finally:
@@ -275,14 +343,46 @@ async def lifespan(app: FastAPI):
log_ingest_task.cancel()
resource_task.cancel()
scheduled_maintenance_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task):
compliance_task.cancel()
audit_export_task.cancel()
backup_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task):
try:
await task
except asyncio.CancelledError:
pass
app = FastAPI(title="OPNexus", version=APP_VERSION, lifespan=lifespan)
API_DESCRIPTION = (
"REST API of OPNexus, the central management for OPNsense firewall fleets.\n\n"
"**Authentication:** send a personal API token as `Authorization: Bearer opx_...` "
"(create one under *System > API tokens*). Scopes: `read` (GET only), `deploy` "
"(+ roll out rules, NAT, VPN, maintenance), `admin` (+ users, settings, hosts). "
"A token never exceeds its owner's role. Token and account management is session-only.\n\n"
"Errors are `{\"detail\": {\"code\": \"...\", \"params\": {...}}}` with a stable `code`."
)
# The interactive Swagger/ReDoc pages load scripts from a CDN, which the proxy's
# Content-Security-Policy forbids; the schema itself is served (behind auth) for
# any client that renders it (Postman, openapi-generator, an AI assistant).
app = FastAPI(
title="OPNexus", version=APP_VERSION, lifespan=lifespan, description=API_DESCRIPTION,
docs_url=None, redoc_url=None, openapi_url="/api/openapi.json",
)
def _custom_openapi():
if app.openapi_schema is None:
schema = get_openapi(title=app.title, version=app.version, description=app.description, routes=app.routes)
schema.setdefault("components", {}).setdefault("securitySchemes", {})["bearerAuth"] = {
"type": "http", "scheme": "bearer", "description": "Personal API token (opx_...)",
}
schema["security"] = [{"bearerAuth": []}]
app.openapi_schema = schema
return app.openapi_schema
app.openapi = _custom_openapi
@app.exception_handler(RequestValidationError)
async def _safe_validation_error(request: Request, exc: RequestValidationError):
@@ -296,6 +396,12 @@ async def _safe_validation_error(request: Request, exc: RequestValidationError):
)
@app.exception_handler(variables_service.MissingVariables)
async def _missing_variables(request: Request, exc: variables_service.MissingVariables):
"""A deploy that needs a value no target host has defined is refused before anything is written."""
return JSONResponse(status_code=422, content={"detail": {"code": ErrorCode.VARIABLES_MISSING.value, "params": {"details": str(exc)}}})
@app.exception_handler(DBAPIError)
async def _database_text_errors(request: Request, exc: DBAPIError):
"""A NUL byte in any text field cannot be stored by Postgres and used to
@@ -342,6 +448,16 @@ app.include_router(hosts.router, prefix="/api")
app.include_router(rules.router, prefix="/api")
app.include_router(search_router.router, prefix="/api")
app.include_router(users_router.router, prefix="/api")
app.include_router(tokens_router.router, prefix="/api")
app.include_router(metrics_router.router, prefix="/api")
app.include_router(mcp_router.router, prefix="/api")
app.include_router(compliance_router.router, prefix="/api")
app.include_router(audit_export_router.router, prefix="/api")
app.include_router(backups_router.router, prefix="/api")
app.include_router(oidc_router.router, prefix="/api")
app.include_router(governance_router.router, prefix="/api")
app.include_router(overlay_router.router, prefix="/api")
app.include_router(variables_router.router, prefix="/api")
app.include_router(logs.router, prefix="/api")
app.include_router(groups.router, prefix="/api")
app.include_router(aliases.router, prefix="/api")
+38
View File
@@ -0,0 +1,38 @@
"""Personal API tokens for scripts, CI jobs, Prometheus and MCP clients.
Only the SHA-256 of the token is stored (the token is 256 random bits, so a
fast hash is enough and lets a request be resolved with one indexed lookup);
the raw value is shown exactly once, at creation. A token belongs to a user and
never exceeds that user's role: see services/api_token_service.py for the
scopes and AuthMiddleware for where they are enforced.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import DateTime, ForeignKey, String, func
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class ApiToken(Base):
__tablename__ = "api_tokens"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
)
name: Mapped[str] = mapped_column(String(80), nullable=False)
# "read" | "deploy" | "admin"
scope: Mapped[str] = mapped_column(String(10), nullable=False)
token_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True)
# First characters of the token, so a list can tell tokens apart. Not secret.
prefix: Mapped[str] = mapped_column(String(16), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
expires_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_used_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_used_ip: Mapped[Optional[str]] = mapped_column(String(45), nullable=True)
user: Mapped["AdminUser"] = relationship("AdminUser")
+33
View File
@@ -0,0 +1,33 @@
"""Forwarding of the audit trail to a syslog server and/or a webhook (singleton row).
``cursor_*`` is the position of the last shipped event, so delivery resumes where
it stopped (at-least-once). Enabling a channel starts at "now": the existing history
is not replayed. The webhook signing secret lives on disk, like every other secret
(services/notification_secret_store.py).
"""
from datetime import datetime
from typing import Optional
from sqlalchemy import Boolean, DateTime, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class AuditExportSettings(Base):
__tablename__ = "audit_export_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
syslog_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
syslog_host: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
syslog_port: Mapped[int] = mapped_column(Integer, nullable=False, default=514)
# "udp" | "tcp" | "tls"
syslog_protocol: Mapped[str] = mapped_column(String(3), nullable=False, default="udp")
webhook_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
webhook_url: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
webhook_signing_secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
cursor_created_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
cursor_id: Mapped[Optional[str]] = mapped_column(String(36), nullable=True)
last_success_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_error: Mapped[Optional[str]] = mapped_column(String(300), nullable=True)
last_error_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
+48
View File
@@ -0,0 +1,48 @@
"""Scheduled config backups and their offsite copy (singleton row + upload ledger).
The S3 secret key and the encryption passphrase live on disk (notification_secret_store,
names ``backup_s3_secret`` / ``backup_passphrase``), never here.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class BackupSettings(Base):
__tablename__ = "backup_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
scheduled_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
interval_hours: Mapped[int] = mapped_column(Integer, nullable=False, default=24)
keep_count: Mapped[int] = mapped_column(Integer, nullable=False, default=30)
offsite_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
s3_endpoint: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
s3_region: Mapped[str] = mapped_column(String(64), nullable=False, default="us-east-1")
s3_bucket: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
s3_prefix: Mapped[str] = mapped_column(String(255), nullable=False, default="opnexus")
s3_access_key: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
s3_path_style: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
s3_secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
passphrase_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
last_run_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_offsite_ok_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_offsite_error: Mapped[Optional[str]] = mapped_column(String(300), nullable=True)
class BackupUpload(Base):
"""Which local snapshot already exists offsite (so nothing is uploaded twice)."""
__tablename__ = "backup_uploads"
__table_args__ = (UniqueConstraint("host_id", "snapshot_id", name="uq_backup_upload"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True)
snapshot_id: Mapped[str] = mapped_column(String(64), nullable=False)
remote_key: Mapped[str] = mapped_column(String(400), nullable=False)
uploaded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
+31
View File
@@ -0,0 +1,31 @@
"""Latest security-posture result per host (see services/compliance_service.py).
One row per host, replaced on every check. The findings never contain
configuration values beyond rule descriptions and the protocol names they
report on; secrets from config.xml are never read into them.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class ComplianceResult(Base):
__tablename__ = "compliance_results"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), nullable=False, unique=True, index=True
)
checked_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# None when the check could not run (see error).
score: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
findings: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# None | "unreachable" | "invalid_config"
error: Mapped[Optional[str]] = mapped_column(String(20), nullable=True)
error_detail: Mapped[Optional[str]] = mapped_column(String(300), nullable=True)
+13
View File
@@ -0,0 +1,13 @@
"""Separation of duties (singleton row): may operators roll changes out themselves?"""
from sqlalchemy import Boolean, Integer
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class GovernanceSettings(Base):
__tablename__ = "governance_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
# False: an operator prepares changes (edit, preview) but only an admin may deploy them.
operators_may_deploy: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
+4
View File
@@ -44,3 +44,7 @@ class OpnsenseHost(Base):
policy_hierarchy_group: Mapped["PolicyHierarchyGroup | None"] = relationship(
"PolicyHierarchyGroup", back_populates="hosts"
)
# Values for {{NAME}} placeholders (services/variables_service.py); always loaded with the host.
variables: Mapped[list["HostVariable"]] = relationship(
"HostVariable", lazy="selectin", cascade="all, delete-orphan", order_by="HostVariable.name"
)
+18
View File
@@ -0,0 +1,18 @@
"""Per-host values for ``{{NAME}}`` placeholders in centrally authored objects (see services/variables_service.py)."""
import uuid
from sqlalchemy import ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostVariable(Base):
__tablename__ = "host_variables"
__table_args__ = (UniqueConstraint("host_id", "name", name="uq_host_variable"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True)
name: Mapped[str] = mapped_column(String(41), nullable=False)
value: Mapped[str] = mapped_column(String(255), nullable=False)
+5 -1
View File
@@ -50,6 +50,10 @@ class NotificationSettings(Base):
nat_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
network_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
carp_auto_revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
scheduled_maintenance_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
backup_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
# Off by default: switching it on for an existing fleet reports every long-standing finding at once.
compliance_alerts_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
@@ -71,7 +75,7 @@ class NotificationProblem(Base):
__table_args__ = (UniqueConstraint("problem_type", "dedup_key", name="uq_notification_problem_key"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
# host_offline | cert_expiry | rule_drift | revert_failed | maintenance_unknown
# host_offline | cert_expiry | rule_drift | revert_failed | maintenance_unknown | ... (see ProblemType)
problem_type: Mapped[str] = mapped_column(String(32))
dedup_key: Mapped[str] = mapped_column(String(300))
status: Mapped[str] = mapped_column(String(16), default="open") # open | resolved
+42
View File
@@ -0,0 +1,42 @@
"""Single sign-on via OpenID Connect: settings (singleton) and in-flight login attempts.
The client secret lives on disk (notification_secret_store, name ``oidc_client_secret``), not here.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import Boolean, DateTime, Integer, String, func
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class OidcSettings(Base):
__tablename__ = "oidc_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
display_name: Mapped[str] = mapped_column(String(60), nullable=False, default="SSO")
issuer: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
client_id: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
client_secret_set: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
scopes: Mapped[str] = mapped_column(String(255), nullable=False, default="openid profile email")
groups_claim: Mapped[str] = mapped_column(String(64), nullable=False, default="groups")
admin_group: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
operator_group: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
read_only_group: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
class OidcLoginState(Base):
"""One row per started login; consumed (deleted) by the callback, expires after minutes."""
__tablename__ = "oidc_login_states"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
state_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True)
nonce: Mapped[str] = mapped_column(String(64), nullable=False)
code_verifier: Mapped[str] = mapped_column(String(128), nullable=False)
redirect_uri: Mapped[str] = mapped_column(String(500), nullable=False)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
+39
View File
@@ -0,0 +1,39 @@
"""Management overlay: a WireGuard hub so OPNexus can reach firewalls behind NAT/CGNAT.
The firewall dials OUT to the hub; OPNexus then talks to its API at the firewall's overlay address. Keys live
on disk (services/overlay_service.py), never in the database.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, func
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class OverlaySettings(Base):
__tablename__ = "overlay_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# host or host:port that firewalls dial (public name/IP of this server and the published UDP port)
endpoint: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
subnet: Mapped[str] = mapped_column(String(18), nullable=False, default="10.99.0.0/24")
listen_port: Mapped[int] = mapped_column(Integer, nullable=False, default=51820)
class OverlayPeer(Base):
__tablename__ = "overlay_peers"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(80), nullable=False, unique=True)
ip: Mapped[str] = mapped_column(String(15), nullable=False, unique=True)
# the firewall's WireGuard public key, entered once the firewall side exists
public_key: Mapped[Optional[str]] = mapped_column(String(44), nullable=True, unique=True)
host_id: Mapped[Optional[uuid.UUID]] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="SET NULL"), nullable=True
)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
+32
View File
@@ -0,0 +1,32 @@
"""When did a firewall rule last match something? (per host and rule UUID)
pf's own counters restart whenever the ruleset is reloaded (every apply, every reboot), so
"packets = 0" alone proves nothing. OPNexus samples them periodically and keeps the time of the
last observed hit, which survives those resets. See services/rule_hit_service.py.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import BigInteger, Boolean, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class RuleHit(Base):
__tablename__ = "rule_hits"
__table_args__ = (UniqueConstraint("host_id", "rule_uuid", name="uq_rule_hit"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True)
rule_uuid: Mapped[str] = mapped_column(String(36), nullable=False)
description: Mapped[str] = mapped_column(String(255), nullable=False, default="")
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_sampled_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# None = never seen a hit since OPNexus started watching.
last_hit_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_packets: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
last_bytes: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0)
+8 -1
View File
@@ -13,7 +13,7 @@ table only tracks the *schedule* itself, not the resulting operation.
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Index, JSON, String
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, JSON, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
@@ -40,3 +40,10 @@ class ScheduledMaintenance(Base):
# Set to the resulting HostOperation's id once fired, so the UI can
# link from the schedule to the operation it produced.
host_operation_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
# Opt-in: hand the traffic over to the CARP partner first, run the action, then switch back.
carp_handover: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
# Only while status == "running" with a handover: handover_wait | job_wait | leave | leave_wait.
# State lives here, not in memory, so a backend restart in the middle of a run resumes cleanly.
phase: Mapped[str | None] = mapped_column(String(16), nullable=True)
phase_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
progress: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
+4
View File
@@ -32,6 +32,10 @@ class AdminUser(Base):
totp_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
totp_last_step: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0, server_default="0")
totp_recovery_hashes: Mapped[Optional[list]] = mapped_column(JSON, nullable=True)
# "local" (password) or "oidc" (single sign-on: role follows the identity provider's groups at every
# login, the password hash is an unusable placeholder). oidc_subject is "<issuer>|<sub>".
auth_source: Mapped[str] = mapped_column(String(10), nullable=False, default="local", server_default="local")
oidc_subject: Mapped[Optional[str]] = mapped_column(String(600), nullable=True, unique=True, index=True)
sessions: Mapped[list["Session"]] = relationship(
"Session", back_populates="user", cascade="all, delete-orphan"
+197
View File
@@ -0,0 +1,197 @@
"""The ``wg-hub`` container: WireGuard interface + a CONNECT proxy for the management overlay.
Runs from the backend image (``python -m app.overlay_hub``) with CAP_NET_ADMIN and the WireGuard UDP port
published. Everything it needs comes from ``<overlay_dir>/hub.json`` (written by the backend, see
services/overlay_service.py); it reports back through ``status.json``. It never talks to the database.
* WireGuard: wg0 with the hub address, peers synced with ``wg syncconf`` whenever hub.json changes. Inbound
NEW connections from the tunnel are dropped (iptables), so a compromised firewall cannot reach anything on this
container; OPNexus' own requests to the firewalls and their replies are unaffected.
* Proxy: an HTTP CONNECT proxy (port 8888, compose network only). It is not an open proxy: it only connects to
IPv4 addresses inside the overlay subnet from hub.json.
"""
import asyncio
import ipaddress
import json
import logging
import os
import subprocess
import sys
import time
from pathlib import Path
from app.config import settings
logger = logging.getLogger("overlay_hub")
INTERFACE = "wg0"
PROXY_PORT = 8888
APPLY_EVERY = 5
def run(*cmd: str, stdin: str | None = None) -> tuple[int, str]:
try:
result = subprocess.run(list(cmd), input=stdin, capture_output=True, text=True, timeout=20)
except (OSError, subprocess.SubprocessError) as exc:
return 127, str(exc)
return result.returncode, (result.stdout + result.stderr).strip()
def wg_conf(config: dict) -> str:
lines = ["[Interface]", f"PrivateKey = {config['private_key']}", f"ListenPort = {int(config['listen_port'])}"]
for peer in config.get("peers", []):
lines += ["", "[Peer]", f"PublicKey = {peer['public_key']}", f"PresharedKey = {peer['psk']}", f"AllowedIPs = {peer['ip']}/32"]
return "\n".join(lines) + "\n"
def parse_dump(text: str) -> list[dict]:
"""`wg show <if> dump`: first line is the interface, then one tab-separated line per peer."""
peers = []
for line in text.splitlines()[1:]:
f = line.split("\t")
if len(f) >= 7:
peers.append({"public_key": f[0], "endpoint": None if f[2] == "(none)" else f[2], "handshake_at": int(f[4] or 0),
"rx": int(f[5] or 0), "tx": int(f[6] or 0)})
return peers
class Hub:
def __init__(self, directory: Path, runner=run):
self.dir, self.run = directory, runner
self.subnet: ipaddress.IPv4Network | None = None
self._applied_mtime = 0.0
def read_config(self) -> dict | None:
try:
config = json.loads((self.dir / "hub.json").read_text())
return config if isinstance(config, dict) else None
except (OSError, ValueError):
return None
def apply(self, config: dict) -> str | None:
"""Bring the interface in line with the config. Returns an error text or None."""
self.subnet = ipaddress.ip_network(config["subnet"]) if config.get("enabled") and config.get("subnet") else None
exists = self.run("ip", "link", "show", INTERFACE)[0] == 0
if not config.get("enabled"):
if exists:
self.run("ip", "link", "del", INTERFACE)
return None
steps = []
if not exists:
steps.append(("ip", "link", "add", INTERFACE, "type", "wireguard"))
steps += [("ip", "address", "replace", config["address"], "dev", INTERFACE), ("ip", "link", "set", INTERFACE, "up")]
for step in steps:
code, out = self.run(*step)
if code != 0:
return f"{' '.join(step[:3])}: {out}"[:300]
conf = self.dir / ".wg0.conf"
fd = os.open(conf, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(fd, "w") as handle:
handle.write(wg_conf(config))
try:
code, out = self.run("wg", "syncconf", INTERFACE, str(conf))
finally:
conf.unlink(missing_ok=True)
if code != 0:
return f"wg syncconf: {out}"[:300]
# Nothing may start a connection INTO this container from the tunnel; replies to ours still pass.
rule = ("INPUT", "-i", INTERFACE, "-m", "conntrack", "--ctstate", "NEW", "-j", "DROP")
if self.run("iptables", "-C", *rule)[0] != 0:
code, out = self.run("iptables", "-I", *rule)
if code != 0:
return f"iptables: {out}"[:300]
# The hub only needs its own proxy: never route between peers or into the compose network.
fwd = ("FORWARD", "-j", "DROP")
if self.run("iptables", "-C", *fwd)[0] != 0:
code, out = self.run("iptables", "-I", *fwd)
if code != 0:
return f"iptables: {out}"[:300]
return None
def status(self, error: str | None) -> dict:
code, out = self.run("wg", "show", INTERFACE, "dump")
return {"updated_at": time.time(), "up": code == 0, "peers": parse_dump(out) if code == 0 else [], "error": error}
def write_status(self, error: str | None) -> None:
tmp = self.dir / "status.json.tmp"
tmp.write_text(json.dumps(self.status(error)))
os.replace(tmp, self.dir / "status.json")
def tick(self, last_error: str | None) -> str | None:
"""One round: re-apply if hub.json changed, then report. Returns the current error."""
config = self.read_config()
if config is not None:
mtime = (self.dir / "hub.json").stat().st_mtime
if mtime != self._applied_mtime or last_error:
last_error = self.apply(config)
self._applied_mtime = mtime
self.write_status(last_error)
return last_error
# ── CONNECT proxy ────────────────────────────────────────────────────
def allowed(self, host: str) -> bool:
try:
return self.subnet is not None and ipaddress.ip_address(host) in self.subnet
except ValueError:
return False
async def handle_client(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
try:
head = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), 10)
method, target, _ = head.split(b"\r\n", 1)[0].decode("latin-1").split(" ", 2)
host, _, port_text = target.rpartition(":")
if method != "CONNECT" or not port_text.isdigit() or not 0 < int(port_text) < 65536 or not self.allowed(host.strip("[]")):
writer.write(b"HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\n\r\n")
await writer.drain()
return
try:
up_reader, up_writer = await asyncio.wait_for(asyncio.open_connection(host, int(port_text)), 8)
except (OSError, asyncio.TimeoutError):
writer.write(b"HTTP/1.1 502 Bad Gateway\r\nContent-Length: 0\r\n\r\n")
await writer.drain()
return
writer.write(b"HTTP/1.1 200 Connection established\r\n\r\n")
await writer.drain()
await asyncio.gather(self._pipe(reader, up_writer), self._pipe(up_reader, writer))
except (asyncio.IncompleteReadError, asyncio.LimitOverrunError, asyncio.TimeoutError, ValueError, OSError):
pass
finally:
writer.close()
@staticmethod
async def _pipe(src: asyncio.StreamReader, dst: asyncio.StreamWriter) -> None:
try:
while data := await src.read(65536):
dst.write(data)
await dst.drain()
except (OSError, asyncio.CancelledError):
pass
finally:
dst.close()
async def main() -> None:
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s")
directory = Path(settings.overlay_dir)
directory.mkdir(parents=True, exist_ok=True)
hub = Hub(directory)
server = await asyncio.start_server(hub.handle_client, "0.0.0.0", PROXY_PORT)
logger.info("Overlay-Hub gestartet (Proxy auf :%d)", PROXY_PORT)
error = None
async with server:
while True:
try:
new_error = await asyncio.to_thread(hub.tick, error)
except Exception:
logger.exception("Overlay-Hub-Runde fehlgeschlagen")
new_error = "interner Fehler"
if new_error and new_error != error:
logger.warning("Overlay: %s", new_error)
error = new_error
await asyncio.sleep(APPLY_EVERY)
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
sys.exit(0)
+3 -2
View File
@@ -3,7 +3,7 @@
For a fresh install whose generated first-start password scrolled out of the
log, and for an admin who forgot theirs. Needs shell access to the server,
which is the trust boundary for this tool anyway (same as app.reset_2fa).
All sessions of the account are ended. Normally called through the host
All sessions and API tokens of the account are ended. Normally called through the host
CLI:
./opnexus password [username]
@@ -24,7 +24,7 @@ from sqlalchemy import select
from app.config import settings
from app.database import SessionLocal
from app.models.user import AdminUser
from app.services import auth_service
from app.services import api_token_service, auth_service
async def main(username: str | None) -> int:
@@ -48,6 +48,7 @@ async def main(username: str | None) -> int:
password = secrets.token_urlsafe(18)
user.password_hash = await auth_service.hash_password_async(password)
await auth_service.revoke_sessions(db, user.id, commit=False)
await api_token_service.revoke_all(db, user.id, commit=False)
await db.commit()
name = user.username
has_totp = user.totp_enabled
+59
View File
@@ -0,0 +1,59 @@
"""Audit export settings (admin only): where the audit trail is forwarded to."""
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.audit_export import AuditExportRead, AuditExportUpdate
from app.services import audit_export_service as svc
from app.services import notification_secret_store
router = APIRouter(prefix="/settings/audit-export", tags=["audit-export"])
def _require_admin(user: AdminUser) -> None:
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
@router.get("", response_model=AuditExportRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
return await svc.get_settings(db)
@router.put("", response_model=AuditExportRead)
async def update(payload: AuditExportUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
row = await svc.get_settings(db)
was_active = svc.active(row)
for field in ("syslog_enabled", "syslog_host", "syslog_port", "syslog_protocol", "webhook_enabled", "webhook_url"):
value = getattr(payload, field)
if value is not None:
setattr(row, field, value.strip() if isinstance(value, str) else value)
if payload.clear_webhook_signing_secret:
notification_secret_store.delete_secret(svc.SECRET_NAME)
row.webhook_signing_secret_set = False
elif payload.webhook_signing_secret is not None:
notification_secret_store.save_secret(svc.SECRET_NAME, payload.webhook_signing_secret)
row.webhook_signing_secret_set = True
if not svc.active(row):
row.cursor_created_at = row.cursor_id = None # next enabling starts from "now" again
elif not was_active:
row.cursor_created_at, row.cursor_id = datetime.now(timezone.utc), None
await db.commit()
return row
@router.post("/test")
async def test(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
try:
await svc.send_test(await svc.get_settings(db))
except svc.ExportError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
return {"success": True}
+20 -3
View File
@@ -9,7 +9,7 @@ from app.errors import ErrorCode, api_error
from app.config import settings
from app.database import get_db
from app.models.user import AdminUser
from app.services import auth_service, totp_service
from app.services import api_token_service, auth_service, totp_service
logger = logging.getLogger(__name__)
@@ -30,11 +30,11 @@ def client_ip(request: Request) -> str:
return request.client.host if request.client else "unknown"
def _set_session_cookie(response: Response, token: str) -> None:
def _set_session_cookie(response: Response, token: str, max_age: int | None = None) -> None:
response.set_cookie(
key=COOKIE_NAME,
value=token,
max_age=settings.session_lifetime,
max_age=max_age or settings.session_lifetime,
httponly=True,
samesite="lax",
secure=settings.cookie_secure,
@@ -63,6 +63,14 @@ def _account_success(user: AdminUser, action: str) -> None:
async def resolve_user(request: Request, db: AsyncSession) -> AdminUser:
# AuthMiddleware already resolved the caller (session cookie or API token), but
# in its own, closed DB session: re-load into this request's session so that
# changes made to the user (2FA, password...) are actually persisted.
resolved = getattr(request.state, "user", None)
if resolved is not None:
user = await db.get(AdminUser, resolved.id)
if user is not None:
return user
token = request.cookies.get(COOKIE_NAME, "")
user = await auth_service.resolve_session(db, token)
if user is None:
@@ -262,6 +270,14 @@ async def logout(request: Request, response: Response, db: AsyncSession = Depend
@router.get("/me")
async def me(request: Request, db: AsyncSession = Depends(get_db)):
raw = api_token_service.bearer_from_header(request.headers.get("authorization", ""))
if raw is not None:
resolved = await api_token_service.resolve(db, raw, client_ip(request))
if resolved is None:
return {"authenticated": False}
user, api_token = resolved
return {"authenticated": True, "username": user.username, "role": user.role,
"via": "token", "scope": api_token.scope, "token_name": api_token.name}
token = request.cookies.get(COOKIE_NAME, "")
user = await auth_service.resolve_session(db, token)
if user is None:
@@ -289,6 +305,7 @@ async def change_password(
# an attacker may hold) and the replacement session commit as one unit.
user.password_hash = await auth_service.hash_password_async(payload.new_password)
await auth_service.revoke_sessions(db, user.id, commit=False)
await api_token_service.revoke_all(db, user.id, commit=False) # a changed password answers a suspected leak: tokens too
token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=client_ip(request)
)
+104
View File
@@ -0,0 +1,104 @@
"""Scheduled backups: status for everybody, settings and offsite test for admins, "run now" for operators."""
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Query
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.backup_settings import BackupSettingsRead, BackupSettingsUpdate, BackupStatusRead
from app.services import backup_diff_service, backup_store
from app.services import backup_schedule_service as svc
from app.services import notification_secret_store
from app.services.s3_client import S3Error
router = APIRouter(tags=["backups"])
def _require_admin(user: AdminUser) -> None:
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
@router.get("/backups/status", response_model=BackupStatusRead)
async def status(db: AsyncSession = Depends(get_db)):
return await svc.get_settings(db)
@router.post("/backups/run")
async def run_now(db: AsyncSession = Depends(get_db)):
"""Back up every enabled host now (and copy offsite if configured)."""
return await svc.run_due(db, force=True)
@router.get("/settings/backups", response_model=BackupSettingsRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
return await svc.get_settings(db)
@router.put("/settings/backups", response_model=BackupSettingsRead)
async def update(payload: BackupSettingsUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
row = await svc.get_settings(db)
for field in ("scheduled_enabled", "interval_hours", "keep_count", "offsite_enabled", "s3_region", "s3_path_style"):
value = getattr(payload, field)
if value is not None:
setattr(row, field, value)
for field in ("s3_endpoint", "s3_bucket", "s3_access_key", "s3_prefix"):
value = getattr(payload, field)
if value is not None:
setattr(row, field, value.strip().rstrip("/") if field == "s3_endpoint" else value.strip())
if payload.clear_s3_secret:
notification_secret_store.delete_secret(svc.S3_SECRET)
row.s3_secret_set = False
elif payload.s3_secret_key is not None:
notification_secret_store.save_secret(svc.S3_SECRET, payload.s3_secret_key)
row.s3_secret_set = True
if payload.passphrase is not None:
notification_secret_store.save_secret(svc.PASSPHRASE, payload.passphrase)
row.passphrase_set = True
await db.commit()
return row
@router.post("/settings/backups/test-offsite")
async def test_offsite(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Write, read back and delete a probe object in the bucket."""
_require_admin(user)
cfg = await svc.get_settings(db)
try:
key = "/".join(p for p in (cfg.s3_prefix.strip("/"), f".opnexus-test-{int(datetime.now(timezone.utc).timestamp())}") if p)
await svc.s3_for(cfg).test(key)
except S3Error as exc:
raise api_error(502, ErrorCode.OFFSITE_TEST_FAILED, reason=str(exc)) from exc
return {"success": True}
@router.get("/hosts/{host_id}/backup-diff")
async def backup_diff(
host_id: uuid.UUID,
a: str | None = Query(default=None, description="Older backup id (default: the one before b)"),
b: str | None = Query(default=None, description="Newer backup id (default: the newest)"),
db: AsyncSession = Depends(get_db),
):
"""What changed between two stored backups of this host. Secrets are replaced by hashes."""
if await db.get(OpnsenseHost, host_id) is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
snapshots = [s["id"] for s in backup_store.list_snapshots(host_id)] # newest first
new_id = b or (snapshots[0] if snapshots else None)
old_id = a or (snapshots[snapshots.index(new_id) + 1] if new_id in snapshots and snapshots.index(new_id) + 1 < len(snapshots) else None)
if not new_id or not old_id:
raise api_error(404, ErrorCode.BACKUP_NICHT_GEFUNDEN)
old, new = backup_store.read_snapshot(host_id, old_id), backup_store.read_snapshot(host_id, new_id)
if old is None or new is None:
raise api_error(404, ErrorCode.BACKUP_NICHT_GEFUNDEN)
try:
result = backup_diff_service.diff(old, new)
except backup_diff_service.InvalidConfig as exc:
raise api_error(422, ErrorCode.BACKUP_UNLESBAR, reason=str(exc)) from exc
return {"a": old_id, "b": new_id, **result}
+88
View File
@@ -0,0 +1,88 @@
"""Security posture ("Sicherheitslage"): stored results and on-demand checks.
Reads (GET) only return what the last check stored, so they are cheap and safe
for dashboards, Prometheus and MCP. A check (POST) downloads the host's config
read-only, evaluates it and stores the result; see services/compliance_service.py.
"""
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.compliance_result import ComplianceResult
from app.models.host import OpnsenseHost
from app.models.rule_hit import RuleHit
from app.services import compliance_service, rule_hit_service
router = APIRouter(tags=["compliance"])
def _read(host: OpnsenseHost, row: ComplianceResult | None, *, detail: bool) -> dict:
result = {
"host_id": host.id, "host_name": host.name, "cluster_name": host.cluster_name, "enabled": host.enabled,
"checked_at": row.checked_at if row else None,
"score": row.score if row else None,
"grade": compliance_service.grade(row.score) if row and row.score is not None else None,
"error": row.error if row else None,
"counts": compliance_service.counts(row.findings) if row else None,
}
if detail:
result["findings"] = row.findings if row else []
return result
@router.get("/compliance")
async def fleet_compliance(db: AsyncSession = Depends(get_db)):
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
rows = {r.host_id: r for r in (await db.scalars(select(ComplianceResult))).all()}
return [_read(host, rows.get(host.id), detail=False) for host in hosts]
@router.post("/compliance/check")
async def check_fleet(db: AsyncSession = Depends(get_db)):
await compliance_service.check_all(db)
return await fleet_compliance(db)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.get("/hosts/{host_id}/compliance")
async def host_compliance(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
row = await db.scalar(select(ComplianceResult).where(ComplianceResult.host_id == host_id))
return _read(host, row, detail=True)
@router.post("/hosts/{host_id}/compliance/check")
async def check_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _host(db, host_id)
row = await compliance_service.check_host(db, host)
return _read(host, row, detail=True)
@router.get("/hosts/{host_id}/rule-hits")
async def rule_hits(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""When each rule last matched something, as observed by OPNexus (see services/rule_hit_service.py)."""
await _host(db, host_id)
now = datetime.now(timezone.utc)
rows = (await db.scalars(select(RuleHit).where(RuleHit.host_id == host_id))).all()
return sorted(
(
{
"rule_uuid": r.rule_uuid, "description": r.description, "enabled": r.enabled,
"state": rule_hit_service.classify(r, now), "packets": r.last_packets, "bytes": r.last_bytes,
"last_hit_at": r.last_hit_at, "first_seen_at": r.first_seen_at, "last_sampled_at": r.last_sampled_at,
}
for r in rows
),
key=lambda item: (item["state"] != "unused", item["last_hit_at"] or datetime.min.replace(tzinfo=timezone.utc), item["description"]),
)
+18 -12
View File
@@ -184,18 +184,8 @@ async def _carp_auto_revert_actions(db: AsyncSession) -> list[PendingAction]:
return actions
@router.get("", response_model=DashboardSummary)
async def summary(db: AsyncSession = Depends(get_db)):
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
statuses = await get_statuses(hosts)
host_summaries = [
DashboardHost(
id=host.id, name=host.name, status=statuses.get(host.id, {}).get("status", "offline"),
carp_status=statuses.get(host.id, {}).get("carp_status"), cluster_name=host.cluster_name,
)
for host in hosts
]
async def collect_pending_actions(db: AsyncSession) -> list[PendingAction]:
"""Everything that needs a human, newest first (shared with the metrics endpoint)."""
pending_actions: list[PendingAction] = []
pending_actions += await _rule_actions(db)
pending_actions += await _nat_actions(db, PortForwardDeployment, PortForwardRule, "port_forward_id", "port_forward", "port-forward")
@@ -209,6 +199,22 @@ async def summary(db: AsyncSession = Depends(get_db)):
pending_actions += await _stuck_host_operations(db)
pending_actions += await _carp_auto_revert_actions(db)
pending_actions.sort(key=lambda item: item.created_at, reverse=True)
return pending_actions
@router.get("", response_model=DashboardSummary)
async def summary(db: AsyncSession = Depends(get_db)):
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
statuses = await get_statuses(hosts)
host_summaries = [
DashboardHost(
id=host.id, name=host.name, status=statuses.get(host.id, {}).get("status", "offline"),
carp_status=statuses.get(host.id, {}).get("carp_status"), cluster_name=host.cluster_name,
)
for host in hosts
]
pending_actions = await collect_pending_actions(db)
activity_rows = (
await db.scalars(select(AuditEvent).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).limit(15))
+41
View File
@@ -0,0 +1,41 @@
"""Separation of duties: with ``operators_may_deploy`` off, only admins roll changes out (AuthMiddleware enforces it)."""
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.governance import GovernanceSettings
from app.models.user import AdminUser
from app.routers.auth import current_user
router = APIRouter(prefix="/settings/governance", tags=["governance"])
class GovernanceRead(BaseModel):
operators_may_deploy: bool
async def get_settings(db: AsyncSession) -> GovernanceSettings:
row = await db.get(GovernanceSettings, 1)
if row is None:
row = GovernanceSettings(id=1, operators_may_deploy=True)
db.add(row)
await db.flush()
return row
@router.get("", response_model=GovernanceRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Readable by everybody signed in (the UI explains why a deploy was refused)."""
return await get_settings(db)
@router.put("", response_model=GovernanceRead)
async def update(payload: GovernanceRead, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
row = await get_settings(db)
row.operators_may_deploy = payload.operators_may_deploy
await db.commit()
return row
+202
View File
@@ -0,0 +1,202 @@
"""Read-only MCP server (Model Context Protocol, streamable HTTP transport).
Lets an AI assistant (Claude Code, Claude Desktop, ...) look at the fleet:
claude mcp add --transport http opnexus https://<opnexus>/api/mcp \
--header "Authorization: Bearer opx_..."
Stateless JSON-RPC over POST (the spec allows plain ``application/json``
replies; there is no server-initiated stream, GET answers 405). Every tool is a
GET against OPNexus' own API, sent through the app itself with the caller's own
credentials, so exactly the caller's permissions apply: a ``read`` token sees
what a ``read`` token sees, secret-bearing endpoints stay closed. Nothing here
can change state.
"""
import json
from urllib.parse import urlencode
import httpx
from fastapi import APIRouter, Request, Response
from fastapi.responses import JSONResponse
from app.version import APP_VERSION
router = APIRouter(tags=["mcp"])
SUPPORTED_VERSIONS = ("2025-06-18", "2025-03-26", "2024-11-05")
MAX_RESULT_CHARS = 60_000
# Not reachable through the generic reader (credentials management, the MCP endpoint itself).
_BLOCKED_PREFIXES = ("/api/tokens", "/api/auth", "/api/mcp")
TOOLS = [
{
"name": "fleet_overview",
"description": "Fleet status at a glance: every firewall with online state and CARP role, items that need "
"attention (failed or unconfirmed deployments, stuck maintenance) and the latest activity.",
"inputSchema": {"type": "object", "properties": {}},
"path": "/api/dashboard",
},
{
"name": "list_hosts",
"description": "All managed firewalls with name, URL, cluster and whether monitoring is enabled.",
"inputSchema": {"type": "object", "properties": {}},
"path": "/api/hosts",
},
{
"name": "host_resources",
"description": "Recent CPU load, memory and disk usage samples of one firewall (a sample per minute).",
"inputSchema": {
"type": "object", "required": ["host_id"],
"properties": {
"host_id": {"type": "string", "description": "Firewall id from list_hosts"},
"since_hours": {"type": "integer", "minimum": 1, "maximum": 720, "default": 1},
},
},
"path": "/api/hosts/{host_id}/resource-samples",
"query": ("since_hours",),
# The raw per-sample payload (every interface counter) would drown the useful numbers.
"drop": ("raw",),
},
{
"name": "carp_events",
"description": "History of CARP role changes (MASTER/BACKUP) across the clusters.",
"inputSchema": {"type": "object", "properties": {}},
"path": "/api/cluster/carp-events",
},
{
"name": "recent_audit",
"description": "The audit trail: who changed or deployed what, newest first.",
"inputSchema": {
"type": "object",
"properties": {
"limit": {"type": "integer", "minimum": 1, "maximum": 200, "default": 25},
"kind": {"type": "string", "description": "e.g. rule, alias, host, user"},
"status": {"type": "string", "description": "e.g. accepted, rejected, denied"},
"q": {"type": "string", "description": "free-text filter"},
},
},
"path": "/api/audit",
"query": ("limit", "kind", "status", "q"),
},
{
"name": "search",
"description": "Search rules, aliases, NAT, VPN, certificates, groups and hosts by text.",
"inputSchema": {
"type": "object", "required": ["q"],
"properties": {"q": {"type": "string", "description": "at least 2 characters"}},
},
"path": "/api/search",
"query": ("q",),
},
{
"name": "api_get",
"description": "Read any other OPNexus API endpoint (GET only), e.g. /api/rules, /api/aliases, /api/logs. "
"The schema of all endpoints is at /api/openapi.json.",
"inputSchema": {
"type": "object", "required": ["path"],
"properties": {
"path": {"type": "string", "description": "Starts with /api/"},
"query": {"type": "object", "description": "Query parameters", "additionalProperties": {"type": "string"}},
},
},
},
]
def _tool_list() -> list[dict]:
return [{k: v for k, v in tool.items() if k in ("name", "description", "inputSchema")} for tool in TOOLS]
def _text(text: str, is_error: bool = False) -> dict:
if len(text) > MAX_RESULT_CHARS:
text = text[:MAX_RESULT_CHARS] + f"\n… truncated ({len(text)} characters in total); narrow the request."
return {"content": [{"type": "text", "text": text}], "isError": is_error}
async def _get(request: Request, path: str, query: dict, drop: tuple = ()) -> dict:
if not path.startswith("/api/") or ".." in path or "?" in path or "#" in path or path.startswith(_BLOCKED_PREFIXES):
return _text("This path cannot be read through MCP.", True)
headers = {k: v for k in ("authorization", "cookie") if (v := request.headers.get(k))}
url = path + (("?" + urlencode(query)) if query else "")
transport = httpx.ASGITransport(app=request.app)
async with httpx.AsyncClient(transport=transport, base_url="http://opnexus", headers=headers, timeout=30) as client:
response = await client.get(url)
body = response.text
try:
data = response.json()
if drop and isinstance(data, list):
data = [{k: v for k, v in item.items() if k not in drop} if isinstance(item, dict) else item for item in data]
body = json.dumps(data, ensure_ascii=False, indent=1)
except ValueError:
pass
return _text(body, response.status_code >= 400)
async def _call_tool(request: Request, name: str, arguments: dict) -> dict:
tool = next((t for t in TOOLS if t["name"] == name), None)
if tool is None:
return _text(f"Unknown tool: {name}", True)
if name == "api_get":
query = arguments.get("query") or {}
if not isinstance(arguments.get("path"), str) or not isinstance(query, dict):
return _text("path (string) is required.", True)
return await _get(request, arguments["path"], {str(k): str(v) for k, v in query.items()})
path = tool["path"]
if "{host_id}" in path:
host_id = str(arguments.get("host_id", ""))
if not host_id.replace("-", "").isalnum():
return _text("host_id is required (see list_hosts).", True)
path = path.format(host_id=host_id)
query = {key: arguments[key] for key in tool.get("query", ()) if arguments.get(key) not in (None, "")}
return await _get(request, path, query, tool.get("drop", ()))
def _error(request_id, code: int, message: str) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "error": {"code": code, "message": message}}
async def _handle(request: Request, message: dict) -> dict | None:
"""The JSON-RPC response for one message, None for a notification."""
if not isinstance(message, dict) or message.get("jsonrpc") != "2.0" or "method" not in message:
return _error(message.get("id") if isinstance(message, dict) else None, -32600, "Invalid request")
method, request_id = message["method"], message.get("id")
params = message.get("params") or {}
if "id" not in message: # notification (e.g. notifications/initialized): nothing to answer
return None
if method == "initialize":
requested = params.get("protocolVersion")
return {"jsonrpc": "2.0", "id": request_id, "result": {
"protocolVersion": requested if requested in SUPPORTED_VERSIONS else SUPPORTED_VERSIONS[0],
"capabilities": {"tools": {"listChanged": False}},
"serverInfo": {"name": "opnexus", "version": APP_VERSION},
"instructions": "Read-only access to an OPNsense fleet managed by OPNexus.",
}}
if method == "ping":
return {"jsonrpc": "2.0", "id": request_id, "result": {}}
if method == "tools/list":
return {"jsonrpc": "2.0", "id": request_id, "result": {"tools": _tool_list()}}
if method == "tools/call":
arguments = params.get("arguments") or {}
if not isinstance(params.get("name"), str) or not isinstance(arguments, dict):
return _error(request_id, -32602, "name (string) and arguments (object) are required")
return {"jsonrpc": "2.0", "id": request_id, "result": await _call_tool(request, params["name"], arguments)}
return _error(request_id, -32601, f"Method not found: {method}")
@router.post("/mcp")
async def mcp_endpoint(request: Request):
try:
payload = await request.json()
except ValueError:
return JSONResponse(_error(None, -32700, "Parse error"), status_code=400)
if isinstance(payload, list):
replies = [reply for message in payload if (reply := await _handle(request, message)) is not None]
return JSONResponse(replies) if replies else Response(status_code=202)
reply = await _handle(request, payload)
return JSONResponse(reply) if reply is not None else Response(status_code=202)
@router.get("/mcp")
async def mcp_no_stream():
# No server-initiated messages: tell clients not to expect an SSE stream.
return Response(status_code=405, headers={"Allow": "POST"})
+18
View File
@@ -0,0 +1,18 @@
"""GET /api/metrics — Prometheus text exposition (see services/metrics_service.py).
Authenticated like everything else; the intended client is a Prometheus job
with ``authorization: {credentials: <read token>}``.
"""
from fastapi import APIRouter, Depends
from fastapi.responses import PlainTextResponse
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.services import metrics_service
router = APIRouter(tags=["metrics"])
@router.get("/metrics", response_class=PlainTextResponse)
async def metrics(db: AsyncSession = Depends(get_db)):
return PlainTextResponse(await metrics_service.render(db), media_type=metrics_service.CONTENT_TYPE)
+143
View File
@@ -0,0 +1,143 @@
"""OIDC single sign-on: public login endpoints and admin-only settings (see services/oidc_service.py)."""
import logging
import re
import secrets
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, Request
from fastapi.responses import RedirectResponse
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser
from app.routers.auth import _set_session_cookie, client_ip, current_user
from app.config import settings
from app.schemas.oidc import OidcSettingsRead, OidcSettingsUpdate
from app.security_middleware import _origin_allowed
from app.services import auth_service, notification_secret_store, oidc_service
logger = logging.getLogger(__name__)
router = APIRouter(tags=["oidc"])
CALLBACK_PATH = "/api/auth/oidc/callback"
STATE_COOKIE = "opnexus-oidc-state"
# SSO sessions are short: the provider's group membership is only re-checked at login, so a removed person
# must not keep a 30-day session.
SSO_SESSION_SECONDS = 12 * 3600
def _login_error(code: str) -> RedirectResponse:
response = RedirectResponse(f"/login?sso_error={code}", status_code=302)
response.delete_cookie(STATE_COOKIE, path="/api/auth/oidc")
return response
def _printable(value: str) -> str:
"""Query text from an unauthenticated caller, made safe for a log line (no CR/LF or other control chars)."""
return re.sub(r"[^\x20-\x7e]", "?", value)[:60]
@router.get("/auth/oidc/config")
async def public_config(db: AsyncSession = Depends(get_db)):
"""For the login page: is there an SSO button, and what does it say. Nothing else is revealed."""
cfg = await oidc_service.get_settings(db)
ready = oidc_service.configured(cfg)
return {"enabled": ready, "name": cfg.display_name if ready else None}
@router.get("/auth/oidc/start")
async def start(request: Request, origin: str, db: AsyncSession = Depends(get_db)):
cfg = await oidc_service.get_settings(db)
parts = urlsplit(origin)
# The redirect URI is built from the browser's own origin (so any hostname the operator uses works), but only
# an origin this server is actually reachable at is accepted: no path, no credentials, same host as the request.
if (not oidc_service.configured(cfg) or parts.path not in ("", "/") or parts.query or parts.fragment or parts.username
or not _origin_allowed(f"{parts.scheme}://{parts.netloc}", request.headers.get("host", ""))):
return _login_error("disabled")
try:
url, state = await oidc_service.start(db, cfg, f"{parts.scheme}://{parts.netloc}{CALLBACK_PATH}")
except oidc_service.OidcError as exc:
logger.warning("OIDC start fehlgeschlagen: %s", exc)
return _login_error(exc.code)
response = RedirectResponse(url, status_code=302)
# Binds the attempt to THIS browser: a callback URL handed to somebody else (login CSRF) or a captured
# code+state pair is useless without the cookie.
response.set_cookie(STATE_COOKIE, state, max_age=600, httponly=True, samesite="lax", secure=settings.cookie_secure, path="/api/auth/oidc")
return response
@router.get("/auth/oidc/callback")
async def callback(request: Request, code: str = "", state: str = "", error: str = "", db: AsyncSession = Depends(get_db)):
ip = client_ip(request)
if auth_service.throttle.retry_after(ip, "oidc") > 0:
return _login_error("throttled")
cfg = await oidc_service.get_settings(db)
if not oidc_service.configured(cfg):
return _login_error("disabled")
try:
if error or not code or not state:
raise oidc_service.OidcError("provider", "Anbieter meldet " + (_printable(error) if error else "keinen Code"))
if not secrets.compare_digest(request.cookies.get(STATE_COOKIE, ""), state):
raise oidc_service.OidcError("state", "Der Anmeldeversuch gehört nicht zu diesem Browser")
user = await oidc_service.complete(db, cfg, code, state)
except oidc_service.OidcError as exc:
auth_service.throttle.record_failure(ip, "oidc")
logger.warning("OIDC-Anmeldung abgelehnt (%s): %s", exc.code, exc)
return _login_error(exc.code)
auth_service.throttle.reset(ip, "oidc")
await auth_service.finalize_login(db, user, ip)
token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip, lifetime_seconds=SSO_SESSION_SECONDS
)
response = RedirectResponse("/dashboard", status_code=302)
_set_session_cookie(response, token, max_age=SSO_SESSION_SECONDS)
response.delete_cookie(STATE_COOKIE, path="/api/auth/oidc")
logger.info("SSO-Anmeldung von '%s' (%s) aus %s", user.username, user.role, ip)
return response
def _require_admin(user: AdminUser) -> None:
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
@router.get("/settings/oidc", response_model=OidcSettingsRead)
async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
return await oidc_service.get_settings(db)
@router.put("/settings/oidc", response_model=OidcSettingsRead)
async def update(payload: OidcSettingsUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
cfg = await oidc_service.get_settings(db)
for field in ("enabled", "display_name", "issuer", "client_id", "scopes", "groups_claim", "admin_group", "operator_group", "read_only_group"):
value = getattr(payload, field)
if value is not None:
setattr(cfg, field, value.strip() if isinstance(value, str) else value)
if payload.clear_client_secret:
notification_secret_store.delete_secret(oidc_service.SECRET_NAME)
cfg.client_secret_set = False
elif payload.client_secret is not None:
notification_secret_store.save_secret(oidc_service.SECRET_NAME, payload.client_secret)
cfg.client_secret_set = True
await db.commit()
oidc_service.clear_caches()
return cfg
@router.post("/settings/oidc/test")
async def test(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Fetch the provider's discovery document and signing keys: proves the issuer URL, not the login."""
_require_admin(user)
cfg = await oidc_service.get_settings(db)
if not cfg.issuer:
raise api_error(422, ErrorCode.OIDC_TEST_FAILED, reason="Kein Aussteller eingetragen")
try:
oidc_service.clear_caches()
doc = await oidc_service.discovery(cfg.issuer)
await oidc_service._jwk_for(doc, None)
except oidc_service.OidcError as exc:
raise api_error(502, ErrorCode.OIDC_TEST_FAILED, reason=str(exc)) from exc
return {"success": True, "authorization_endpoint": doc["authorization_endpoint"]}
+142
View File
@@ -0,0 +1,142 @@
"""Management overlay API (see services/overlay_service.py). Writes and the pre-shared key are admin-only."""
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.host import OpnsenseHost
from app.models.overlay import OverlayPeer
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.overlay import OverlaySettingsRead, OverlaySettingsUpdate, PeerCreate, PeerRead, PeerUpdate
from app.services import overlay_service as svc
router = APIRouter(prefix="/overlay", tags=["overlay"])
def _require_admin(user: AdminUser) -> None:
if user.role != "admin":
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
async def _settings_read(db: AsyncSession) -> dict:
cfg = await svc.get_settings(db)
net = svc.parse_subnet(cfg.subnet)
return {
"enabled": cfg.enabled, "endpoint": cfg.endpoint, "subnet": cfg.subnet, "listen_port": cfg.listen_port,
"hub_public_key": svc.hub_public_key() if cfg.enabled else None, "hub_address": str(svc.hub_address(net)),
"hub_state": svc.hub_state(svc.read_status()),
}
async def _peers(db: AsyncSession) -> list[dict]:
status = svc.read_status()
rows = (await db.execute(select(OverlayPeer, OpnsenseHost.name).outerjoin(OpnsenseHost, OverlayPeer.host_id == OpnsenseHost.id).order_by(OverlayPeer.ip))).all()
return [
{"id": p.id, "name": p.name, "ip": p.ip, "public_key": p.public_key, "host_id": p.host_id, "host_name": host_name,
**svc.peer_status(status, p.public_key)}
for p, host_name in rows
]
@router.get("")
async def overview(db: AsyncSession = Depends(get_db)):
return {"settings": await _settings_read(db), "peers": await _peers(db)}
@router.put("/settings", response_model=OverlaySettingsRead)
async def update_settings(payload: OverlaySettingsUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
cfg = await svc.get_settings(db)
if payload.subnet is not None and payload.subnet.strip() != cfg.subnet:
try:
svc.parse_subnet(payload.subnet)
except svc.OverlayError as exc:
raise api_error(422, ErrorCode.OVERLAY_INVALID, reason=str(exc)) from exc
if (await db.scalar(select(OverlayPeer.id).limit(1))) is not None:
raise api_error(409, ErrorCode.OVERLAY_SUBNET_LOCKED) # peers already hold addresses in the old network
cfg.subnet = payload.subnet.strip()
for field in ("enabled", "listen_port"):
value = getattr(payload, field)
if value is not None:
setattr(cfg, field, value)
if payload.endpoint is not None:
cfg.endpoint = payload.endpoint.strip() or None
if cfg.enabled and not cfg.endpoint:
raise api_error(422, ErrorCode.OVERLAY_INVALID, reason="Ohne Endpunkt wissen die Firewalls nicht, wohin sie sich verbinden sollen")
await db.commit()
await svc.sync(db)
return await _settings_read(db)
@router.post("/peers", status_code=201)
async def create_peer(payload: PeerCreate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
cfg = await svc.get_settings(db)
taken = set((await db.scalars(select(OverlayPeer.ip))).all())
try:
ip = svc.next_free_ip(svc.parse_subnet(cfg.subnet), taken)
except svc.OverlayError as exc:
raise api_error(409, ErrorCode.OVERLAY_INVALID, reason=str(exc)) from exc
peer = OverlayPeer(name=payload.name.strip(), ip=ip)
db.add(peer)
try:
await db.commit()
except IntegrityError:
await db.rollback()
raise api_error(409, ErrorCode.OVERLAY_NAME_TAKEN) from None
await svc.sync(db)
return {"id": peer.id, "name": peer.name, "ip": peer.ip, "instructions": svc.instructions(cfg, peer)}
@router.get("/peers/{peer_id}/instructions")
async def peer_instructions(peer_id: uuid.UUID, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
peer = await db.get(OverlayPeer, peer_id)
if peer is None:
raise api_error(404, ErrorCode.OVERLAY_PEER_NOT_FOUND)
return svc.instructions(await svc.get_settings(db), peer)
@router.patch("/peers/{peer_id}")
async def update_peer(peer_id: uuid.UUID, payload: PeerUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
peer = await db.get(OverlayPeer, peer_id)
if peer is None:
raise api_error(404, ErrorCode.OVERLAY_PEER_NOT_FOUND)
if payload.public_key is not None:
key = payload.public_key.strip()
if key and not svc.valid_public_key(key):
raise api_error(422, ErrorCode.OVERLAY_INVALID, reason="Das ist kein WireGuard-Schlüssel (44 Zeichen, Base64)")
if key and key == svc.hub_public_key():
raise api_error(422, ErrorCode.OVERLAY_INVALID, reason="Das ist der Schlüssel des Hubs, nicht der der Firewall")
peer.public_key = key or None
if payload.clear_host:
peer.host_id = None
elif payload.host_id is not None:
if await db.get(OpnsenseHost, payload.host_id) is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
peer.host_id = payload.host_id
try:
await db.commit()
except IntegrityError:
await db.rollback()
raise api_error(409, ErrorCode.OVERLAY_INVALID, reason="Dieser Schlüssel gehört schon zu einem anderen Eintrag") from None
await svc.sync(db)
return {"ok": True}
@router.delete("/peers/{peer_id}", status_code=204)
async def delete_peer(peer_id: uuid.UUID, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
peer = await db.get(OverlayPeer, peer_id)
if peer is None:
raise api_error(404, ErrorCode.OVERLAY_PEER_NOT_FOUND)
await db.delete(peer)
await db.commit()
svc.delete_peer_secret(peer_id)
await svc.sync(db)
+17 -1
View File
@@ -11,6 +11,8 @@ from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.services.host_maintenance_service import peers
from app.services.host_status_service import get_statuses
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/maintenance/schedule", tags=["host-maintenance"])
@@ -20,6 +22,8 @@ class Schedule(BaseModel):
action: Literal["reboot", "update", "major_update"]
run_at: datetime
acknowledge_unattended: Literal[True]
# Hand the traffic over to the CARP partner first and switch back afterwards (cluster nodes only).
carp_handover: bool = False
def _read(sched: ScheduledMaintenance) -> dict:
@@ -27,6 +31,7 @@ def _read(sched: ScheduledMaintenance) -> dict:
"id": sched.id, "host_id": sched.host_id, "action": sched.action, "run_at": sched.run_at,
"status": sched.status, "actor": sched.actor, "created_at": sched.created_at,
"result": sched.result, "host_operation_id": sched.host_operation_id,
"carp_handover": sched.carp_handover, "phase": sched.phase,
}
@@ -61,9 +66,20 @@ async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Reque
run_at = payload.run_at if payload.run_at.tzinfo else payload.run_at.replace(tzinfo=timezone.utc)
if run_at <= now:
raise api_error(422, ErrorCode.ZEITPUNKT_MUSS_IN_DER_ZUKUNFT_LIEGEN)
if payload.carp_handover:
if not host.cluster_name or len(await peers(db, host)) < 2:
raise api_error(409, ErrorCode.CARP_UEBERGABE_BRAUCHT_EINEN_CLUSTER)
elif host.cluster_name:
# Without a handover, fire time refuses every cluster node that is not the CARP BACKUP, so a
# plan for the current MASTER can only ever end as "failed" at 3 a.m. Say so now.
# Only a *known* MASTER is rejected here: an unreachable node or an unclear role
# is still judged by the full check when the plan fires.
statuses = await get_statuses(await peers(db, host))
if statuses.get(host.id, {}).get("carp_status") == "MASTER":
raise api_error(409, ErrorCode.GEPLANTE_WARTUNG_NUR_AUF_DEM_CARP_BACKUP_MOEGLICH)
sched = ScheduledMaintenance(
host_id=host.id, host_name=host.name, action=payload.action, run_at=run_at,
actor=request.state.user.username, created_at=now,
actor=request.state.user.username, created_at=now, carp_handover=payload.carp_handover,
)
db.add(sched)
await db.commit()
+68
View File
@@ -0,0 +1,68 @@
"""Manage personal API tokens (see services/api_token_service.py for scopes).
Session-only: an API token cannot reach these endpoints (AuthMiddleware), so a
leaked token can never mint a replacement or list/revoke its siblings. A
read_only user may manage their own tokens, but only with read scope (an operator up to deploy).
"""
import uuid
from fastapi import APIRouter, Depends, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.api_token import ApiToken
from app.models.user import AdminUser
from app.routers.auth import _account_failure, _account_success, _check_account_budget, current_user
from app.schemas.api_token import ApiTokenCreate, ApiTokenCreated, ApiTokenRead, TokenScope
from app.services import api_token_service, auth_service
router = APIRouter(prefix="/tokens", tags=["tokens"])
def _read(token: ApiToken, owner: str) -> dict:
return {
"id": token.id, "name": token.name, "scope": token.scope, "prefix": token.prefix, "owner": owner,
"created_at": token.created_at, "expires_at": token.expires_at,
"last_used_at": token.last_used_at, "last_used_ip": token.last_used_ip,
}
@router.get("", response_model=list[ApiTokenRead])
async def list_tokens(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
"""Own tokens; an admin sees everybody's (to be able to revoke them)."""
query = select(ApiToken, AdminUser.username).join(AdminUser, ApiToken.user_id == AdminUser.id)
if user.role != "admin":
query = query.where(ApiToken.user_id == user.id)
rows = (await db.execute(query.order_by(ApiToken.created_at.desc()))).all()
return [_read(token, owner) for token, owner in rows]
@router.post("", response_model=ApiTokenCreated, status_code=status.HTTP_201_CREATED)
async def create_token(
payload: ApiTokenCreate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)
):
allowed = {"admin": (TokenScope.read, TokenScope.deploy, TokenScope.admin), "operator": (TokenScope.read, TokenScope.deploy)}
if payload.scope not in allowed.get(user.role, (TokenScope.read,)):
raise api_error(403, ErrorCode.API_TOKEN_SCOPE_TOO_HIGH)
_check_account_budget(user, "create-token")
if not await auth_service.verify_password_async(user.password_hash, payload.current_password):
_account_failure(user, "create-token")
raise api_error(401, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
_account_success(user, "create-token")
token, raw = await api_token_service.create(
db, user, name=payload.name.strip(), scope=payload.scope.value, expires_in_days=payload.expires_in_days
)
return {**_read(token, user.username), "token": raw}
@router.delete("/{token_id}", status_code=204)
async def revoke_token(
token_id: uuid.UUID, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)
):
token = await db.get(ApiToken, token_id)
if token is None or (token.user_id != user.id and user.role != "admin"):
raise api_error(404, ErrorCode.API_TOKEN_NOT_FOUND)
await db.delete(token)
await db.commit()
+6 -1
View File
@@ -9,7 +9,7 @@ from app.database import get_db
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.user import UserCreate, UserPasswordReset, UserRead, UserRoleUpdate
from app.services import auth_service, totp_service
from app.services import api_token_service, auth_service, totp_service
router = APIRouter(prefix="/users", tags=["users"])
@@ -47,6 +47,8 @@ async def update_role(user_id: uuid.UUID, payload: UserRoleUpdate, db: AsyncSess
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
if target.auth_source == "oidc":
raise api_error(409, ErrorCode.OIDC_MANAGED_USER) # the identity provider's groups decide, at every login
if target.role == "admin" and payload.role.value != "admin" and await _admin_count(db, exclude_id=target.id) == 0:
raise api_error(409, ErrorCode.MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN)
target.role = payload.role.value
@@ -75,10 +77,13 @@ async def reset_password(user_id: uuid.UUID, payload: UserPasswordReset, db: Asy
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
if target.auth_source == "oidc":
raise api_error(409, ErrorCode.OIDC_MANAGED_USER)
target.password_hash = await auth_service.hash_password_async(payload.new_password)
# A reset is the answer to a suspected compromise: every session of the
# account ends with the old password, atomically with the new hash.
await auth_service.revoke_sessions(db, target.id, commit=False)
await api_token_service.revoke_all(db, target.id, commit=False)
await db.commit()
return target
+98
View File
@@ -0,0 +1,98 @@
"""Site variables of one host: the values for {{NAME}} placeholders, and which placeholders its objects use."""
import uuid
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.host_variable import HostVariable
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.services import alias_deploy_service, one_to_one_deploy_service, outbound_nat_deploy_service
from app.services import port_forward_deploy_service, rule_deploy_service, variables_service
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
from app.services.policy_service import effective_hosts
router = APIRouter(prefix="/hosts/{host_id}/variables", tags=["variables"])
_KINDS = (
("rule", FirewallRule, rule_deploy_service),
("alias", FirewallAlias, alias_deploy_service),
("port_forward", PortForwardRule, port_forward_deploy_service),
("one_to_one", OneToOneRule, one_to_one_deploy_service),
("outbound_nat", OutboundNatRule, outbound_nat_deploy_service),
)
class VariablesUpdate(BaseModel):
variables: dict[str, str] = Field(default_factory=dict, max_length=200)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
async def _usage(db: AsyncSession, host: OpnsenseHost) -> dict[str, list[dict]]:
"""name -> objects (that are deployed to this host) which use it."""
used: dict[str, list[dict]] = {}
for kind, model, service in _KINDS:
entities = (await db.execute(
select(model).options(selectinload(model.targets), selectinload(model.groups).selectinload(DeviceGroup.hosts))
)).scalars().unique().all()
await hydrate_hierarchy_targets(db, entities)
for entity in entities:
if host.id not in {h.id for h in effective_hosts(entity)}:
continue
label = getattr(entity, "description", None) or getattr(entity, "name", None) or str(entity.id)
for name in variables_service.names_in_payload(service._base_payload(entity)):
used.setdefault(name, []).append({"kind": kind, "id": str(entity.id), "label": label})
return used
async def _read(db: AsyncSession, host: OpnsenseHost) -> dict:
values = variables_service.host_values(host)
used = await _usage(db, host)
names = sorted(set(values) | set(used))
return {"variables": [
{"name": n, "value": values.get(n), "used_by": used.get(n, []), "missing": n in used and n not in values}
for n in names
]}
@router.get("")
async def read(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return await _read(db, await _host(db, host_id))
@router.put("")
async def replace(host_id: uuid.UUID, payload: VariablesUpdate, db: AsyncSession = Depends(get_db)):
"""Replace this host's variables with exactly the given set."""
host = await _host(db, host_id)
for name, value in payload.variables.items():
if not variables_service.NAME_RE.fullmatch(name):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Name „{name}“: Großbuchstaben, Ziffern und _, beginnend mit einem Buchstaben")
if not variables_service.VALUE_RE.fullmatch(value):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Wert von {name}: 1–255 Zeichen aus Buchstaben, Ziffern und . : / , @ + - _ Leerzeichen")
existing = {v.name: v for v in host.variables}
for name in set(existing) - set(payload.variables):
host.variables.remove(existing[name])
for name, value in payload.variables.items():
if name in existing:
existing[name].value = value
else:
host.variables.append(HostVariable(name=name, value=value))
await db.commit()
await db.refresh(host)
return await _read(db, host)
+40
View File
@@ -0,0 +1,40 @@
import uuid
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, ConfigDict, Field
class TokenScope(str, Enum):
read = "read"
deploy = "deploy"
admin = "admin"
class ApiTokenCreate(BaseModel):
name: str = Field(min_length=1, max_length=80)
# Least privilege by default.
scope: TokenScope = TokenScope.read
# None = never expires. The UI defaults to 90 days.
expires_in_days: int | None = Field(default=None, ge=1, le=3650)
# Creating a long-lived credential needs the account password again.
current_password: str = Field(min_length=1, max_length=1024)
class ApiTokenRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
scope: TokenScope
prefix: str
owner: str
created_at: datetime
expires_at: datetime | None
last_used_at: datetime | None
last_used_ip: str | None
class ApiTokenCreated(ApiTokenRead):
# The only time the raw token is ever returned.
token: str
+32
View File
@@ -0,0 +1,32 @@
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
class AuditExportRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
syslog_enabled: bool
syslog_host: str | None
syslog_port: int
syslog_protocol: Literal["udp", "tcp", "tls"]
webhook_enabled: bool
webhook_url: str | None
webhook_signing_secret_set: bool
last_success_at: datetime | None
last_error: str | None
last_error_at: datetime | None
class AuditExportUpdate(BaseModel):
"""Partial update; the signing secret is write-only (set it or clear it)."""
syslog_enabled: bool | None = None
syslog_host: str | None = Field(default=None, max_length=255)
syslog_port: int | None = Field(default=None, ge=1, le=65535)
syslog_protocol: Literal["udp", "tcp", "tls"] | None = None
webhook_enabled: bool | None = None
webhook_url: str | None = Field(default=None, max_length=500)
webhook_signing_secret: str | None = Field(default=None, min_length=1)
clear_webhook_signing_secret: bool = False
+46
View File
@@ -0,0 +1,46 @@
from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field
class BackupStatusRead(BaseModel):
"""What every signed-in user may see: no endpoints, no keys."""
model_config = ConfigDict(from_attributes=True)
scheduled_enabled: bool
interval_hours: int
keep_count: int
offsite_enabled: bool
last_run_at: datetime | None
last_offsite_ok_at: datetime | None
last_offsite_error: str | None
class BackupSettingsRead(BackupStatusRead):
s3_endpoint: str | None
s3_region: str
s3_bucket: str | None
s3_prefix: str
s3_access_key: str | None
s3_path_style: bool
s3_secret_set: bool
passphrase_set: bool
class BackupSettingsUpdate(BaseModel):
"""Partial update; the S3 secret key and the passphrase are write-only."""
scheduled_enabled: bool | None = None
interval_hours: int | None = Field(default=None, ge=1, le=24 * 30)
keep_count: int | None = Field(default=None, ge=3, le=1000)
offsite_enabled: bool | None = None
s3_endpoint: str | None = Field(default=None, max_length=255)
s3_region: str | None = Field(default=None, min_length=1, max_length=64)
s3_bucket: str | None = Field(default=None, max_length=255)
s3_prefix: str | None = Field(default=None, max_length=255)
s3_access_key: str | None = Field(default=None, max_length=255)
s3_path_style: bool | None = None
s3_secret_key: str | None = Field(default=None, min_length=1, max_length=512)
clear_s3_secret: bool = False
passphrase: str | None = Field(default=None, min_length=12, max_length=512)
+10
View File
@@ -10,6 +10,10 @@ SmtpSecurity = Literal["none", "starttls", "ssl"]
ProblemType = Literal[
"host_offline", "cert_expiry", "rule_drift", "revert_failed", "maintenance_unknown",
"carp_split_brain", "nat_drift", "carp_auto_revert_failed", "network_drift",
"scheduled_maintenance_failed",
"backup_stale",
"backup_offsite_failed",
"compliance_finding",
]
@@ -38,6 +42,9 @@ class NotificationSettingsRead(BaseModel):
nat_drift_enabled: bool
network_drift_enabled: bool = True
carp_auto_revert_failed_enabled: bool
scheduled_maintenance_failed_enabled: bool = True
backup_failed_enabled: bool = True
compliance_alerts_enabled: bool = False
poll_interval_seconds: int
updated_at: datetime
@@ -74,6 +81,9 @@ class NotificationSettingsUpdate(BaseModel):
nat_drift_enabled: bool | None = None
network_drift_enabled: bool | None = None
carp_auto_revert_failed_enabled: bool | None = None
scheduled_maintenance_failed_enabled: bool | None = None
backup_failed_enabled: bool | None = None
compliance_alerts_enabled: bool | None = None
poll_interval_seconds: int | None = Field(
default=None, ge=settings.notification_poll_min_interval_seconds
+39
View File
@@ -0,0 +1,39 @@
from pydantic import BaseModel, ConfigDict, Field, field_validator
class OidcSettingsRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
enabled: bool
display_name: str
issuer: str | None
client_id: str | None
client_secret_set: bool
scopes: str
groups_claim: str
admin_group: str | None
operator_group: str | None
read_only_group: str | None
class OidcSettingsUpdate(BaseModel):
"""Partial update; the client secret is write-only (set it or clear it)."""
enabled: bool | None = None
display_name: str | None = Field(default=None, min_length=1, max_length=60)
issuer: str | None = Field(default=None, max_length=500)
client_id: str | None = Field(default=None, max_length=255)
client_secret: str | None = Field(default=None, min_length=1, max_length=512)
clear_client_secret: bool = False
scopes: str | None = Field(default=None, min_length=1, max_length=255)
groups_claim: str | None = Field(default=None, min_length=1, max_length=64)
admin_group: str | None = Field(default=None, max_length=255)
operator_group: str | None = Field(default=None, max_length=255)
read_only_group: str | None = Field(default=None, max_length=255)
@field_validator("issuer")
@classmethod
def https_only(cls, value):
if value and not value.startswith("https://"):
raise ValueError("issuer must be an https:// URL")
return value.strip().rstrip("/") if value else value
+45
View File
@@ -0,0 +1,45 @@
from datetime import datetime
import uuid
from pydantic import BaseModel, Field
class OverlaySettingsRead(BaseModel):
enabled: bool
endpoint: str | None
subnet: str
listen_port: int
hub_public_key: str | None
hub_address: str
hub_state: str # running | stale | absent
class OverlaySettingsUpdate(BaseModel):
enabled: bool | None = None
endpoint: str | None = Field(default=None, max_length=255)
subnet: str | None = Field(default=None, max_length=18)
listen_port: int | None = Field(default=None, ge=1, le=65535)
class PeerCreate(BaseModel):
name: str = Field(min_length=1, max_length=80)
class PeerUpdate(BaseModel):
public_key: str | None = Field(default=None, max_length=44)
host_id: uuid.UUID | None = None
clear_host: bool = False
class PeerRead(BaseModel):
id: uuid.UUID
name: str
ip: str
public_key: str | None
host_id: uuid.UUID | None
host_name: str | None
connected: bool
last_handshake_at: datetime | None
rx_bytes: int
tx_bytes: int
remote_endpoint: str | None
+4
View File
@@ -7,6 +7,9 @@ from pydantic import BaseModel, ConfigDict, Field
class UserRole(str, Enum):
admin = "admin"
# Rolls out and operates (rules, NAT, VPN, maintenance, tokens of its own) but cannot
# change who may access OPNexus or where it sends data (users, settings, hosts).
operator = "operator"
read_only = "read_only"
@@ -36,3 +39,4 @@ class UserRead(BaseModel):
created_at: datetime
last_login_at: datetime | None
totp_enabled: bool = False
auth_source: str = "local"
+9 -3
View File
@@ -18,7 +18,7 @@ from app.models.host import OpnsenseHost
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -27,7 +27,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(alias: FirewallAlias) -> dict:
def _build_payload(alias: FirewallAlias, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(alias)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(alias: FirewallAlias) -> dict:
return {
"enabled": "1" if alias.enabled else "0",
"name": alias.name,
@@ -38,7 +43,7 @@ def _build_payload(alias: FirewallAlias) -> dict:
async def _deploy_to_host(db: AsyncSession, alias: FirewallAlias, host: OpnsenseHost) -> AliasDeployment:
payload = _build_payload(alias)
payload = _build_payload(alias, host)
existing = await db.scalar(
select(AliasDeployment).where(AliasDeployment.alias_id == alias.id, AliasDeployment.host_id == host.id)
)
@@ -79,6 +84,7 @@ async def deploy_alias(db: AsyncSession, alias: FirewallAlias, hosts: list[Opnse
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, alias, hosts) # before anything is written
deployments = [await _deploy_to_host(db, alias, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
+10 -4
View File
@@ -9,6 +9,7 @@ import json
from app.models.policy import AliasRevision
from app.services.alias_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
ALIAS_FIELDS = ("name", "type", "content", "description", "enabled")
@@ -28,13 +29,17 @@ def revision(db, alias, note):
def preview(alias):
payload = _build_payload(alias)
deployments = {d.host_id: d for d in alias.deployments}
hosts = effective_hosts(alias)
blocked = any(d.status in ("pending", "revert_failed") for d in alias.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(alias, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_alias_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -61,6 +66,8 @@ def preview(alias):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diesen Alias, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in alias.deployments):
@@ -71,7 +78,7 @@ def preview(alias):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in alias.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -79,12 +86,11 @@ def workflow(alias, deployments):
hosts = effective_hosts(alias)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(alias)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, alias, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+157
View File
@@ -0,0 +1,157 @@
"""Personal API tokens: creation, lookup and scope rules.
Scopes (a token can never do more than its owner's role allows; a read_only
user's token is read-only whatever its scope says):
* ``read`` — GET/HEAD/OPTIONS only, except endpoints that return secrets (raw
config backups, notification settings): those need ``admin``. For Prometheus,
dashboards, MCP clients.
* ``deploy`` — additionally every write that rolls out or operates on the fleet
(rules, aliases, NAT, VPN, maintenance, backups...). NOT the writes that
change who can access OPNexus or where it sends data: users, settings,
notifications, host registration/removal, log settings, OPNexus self-update.
* ``admin`` — everything the owner may do, except what is session-only.
Session-only for every scope: managing tokens themselves and the account
endpoints (/api/auth/*), so a leaked token can neither mint more tokens nor
change a password or second factor.
The enforcement lives in AuthMiddleware (one place, like the read_only role).
"""
import hashlib
import re
import secrets
from datetime import datetime, timedelta, timezone
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.models.api_token import ApiToken
from app.models.user import AdminUser
TOKEN_PREFIX = "opx_"
SCOPES = ("read", "deploy", "admin")
MAX_TOKENS_PER_USER = 50
# last_used_at is only rewritten when older than this, so a scraper hitting
# /api/metrics every few seconds does not turn into a write per request.
_LAST_USED_GRANULARITY = timedelta(minutes=1)
_SAFE_METHODS = ("GET", "HEAD", "OPTIONS")
_SESSION_ONLY = re.compile(r"^/api/(tokens|auth)(/|$)")
# Reads that expose secrets (config.xml holds password hashes and VPN keys, the
# notification settings the webhook URL/SMTP data): admin scope only.
_ADMIN_SCOPE_READS = re.compile(r"^/api/hosts/[^/]+/backups/[^/]+$|^/api/notifications(/|$)|^/api/settings/|^/api/overlay/peers/[^/]+/instructions$")
# The MCP endpoint is POST (JSON-RPC) but its tools only ever read.
_MCP_PATH = "/api/mcp"
_ADMIN_ONLY_WRITES = re.compile(
r"^/api/(users|settings|system|notifications|overlay)(/|$)"
r"|^/api/logs/settings$"
r"|^/api/hosts(/tls-probe|/[^/]+)?$"
)
def _now() -> datetime:
return datetime.now(timezone.utc)
def hash_token(raw: str) -> str:
return hashlib.sha256(raw.encode()).hexdigest()
def denies(scope: str, method: str, path: str) -> bool:
"""True if a token with this scope must not make this request."""
if _SESSION_ONLY.match(path):
return True
if path == _MCP_PATH:
return False
if method in _SAFE_METHODS:
return scope != "admin" and bool(_ADMIN_SCOPE_READS.match(path))
if scope == "read":
return True
if scope == "deploy" and _ADMIN_ONLY_WRITES.match(path):
return True
return False
def operator_denies(method: str, path: str) -> bool:
"""True if an `operator` account must not make this request: the writes that
change who may access OPNexus or where it sends data. (Their own theme at
PUT /api/settings stays open; account endpoints are not matched here.)"""
return method not in _SAFE_METHODS and path != "/api/settings" and bool(_ADMIN_ONLY_WRITES.match(path))
# Writes that put a change onto the fleet (as opposed to preparing one): deploy, remove/rollback a deployment,
# maintenance (reboot/update/CARP switch), scheduling it, applying network changes, pushing certificates and
# ACME accounts, re-asserting the hierarchy order, controlling services, and DELETING a centrally managed object
# or snapshot (deleting a rule removes it from every firewall it was deployed to).
_ROLLOUT_POST = re.compile(
r"/(deploy|push|sign|poll|register|reassert-ordering"
r"|deployments/[^/]+/(remove|retry-rollback)"
r"|maintenance/(execute|carp-execute|schedule)"
r"|network/changes"
r"|services/[^/]+/(start|stop|restart))$"
)
_ROLLOUT_DELETE = re.compile(
r"^/api/(rules|aliases|port-forward|one-to-one|outbound-nat|policy-hierarchy/rules|ipsec/tunnels|openvpn/instances"
r"|wireguard/tunnels|certificates/managed|acme/accounts|acme/certificates)/[^/]+$"
r"|^/api/hosts/[^/]+/(snapshots|maintenance/schedule)/[^/]+$"
)
def is_rollout(method: str, path: str) -> bool:
if method == "POST":
return bool(_ROLLOUT_POST.search(path))
return method == "DELETE" and bool(_ROLLOUT_DELETE.match(path))
def bearer_from_header(header: str) -> str | None:
"""The token of an ``Authorization: Bearer opx_...`` header, else None."""
scheme, _, value = header.partition(" ")
value = value.strip()
if scheme.lower() == "bearer" and value.startswith(TOKEN_PREFIX):
return value
return None
async def create(
db: AsyncSession, user: AdminUser, *, name: str, scope: str, expires_in_days: int | None
) -> tuple[ApiToken, str]:
count = len((await db.scalars(select(ApiToken.id).where(ApiToken.user_id == user.id))).all())
if count >= MAX_TOKENS_PER_USER:
raise api_error(409, ErrorCode.API_TOKEN_LIMIT_REACHED, limit=MAX_TOKENS_PER_USER)
raw = TOKEN_PREFIX + secrets.token_urlsafe(32)
token = ApiToken(
user_id=user.id,
name=name,
scope=scope,
token_hash=hash_token(raw),
prefix=raw[: len(TOKEN_PREFIX) + 6],
expires_at=_now() + timedelta(days=expires_in_days) if expires_in_days else None,
)
db.add(token)
await db.commit()
await db.refresh(token)
return token, raw
async def resolve(db: AsyncSession, raw: str, ip: str | None) -> tuple[AdminUser, ApiToken] | None:
"""The owner and token for a raw bearer token, or None if unknown/expired."""
token = await db.scalar(select(ApiToken).where(ApiToken.token_hash == hash_token(raw)))
if token is None or (token.expires_at is not None and token.expires_at <= _now()):
return None
user = await db.get(AdminUser, token.user_id)
if user is None:
return None
if token.last_used_at is None or _now() - token.last_used_at > _LAST_USED_GRANULARITY:
token.last_used_at = _now()
token.last_used_ip = ip
await db.commit()
return user, token
async def revoke_all(db: AsyncSession, user_id, *, commit: bool = True) -> None:
"""Delete every token of an account (password reset: answer to a suspected compromise)."""
await db.execute(delete(ApiToken).where(ApiToken.user_id == user_id))
if commit:
await db.commit()
@@ -0,0 +1,177 @@
"""Ships new audit events to syslog (RFC 5424, UDP / TCP / TLS) and/or a webhook.
A poll loop (main.py) calls ``run_once``: events after the stored cursor, oldest
first, in batches. A channel failing leaves the cursor where it is, so the same
events are tried again next round (at-least-once; every event carries its id so a
receiver can de-duplicate). If both channels are on, both must succeed to advance.
What is sent: who, what kind/action/status, which entity and host, the message and
a correlation id. NOT ``details`` — those hold before/after payloads of firewall
objects, which have no business in a log aggregator.
"""
import asyncio
import hashlib
import hmac
import json
import logging
import socket
import ssl
import uuid
from datetime import datetime, timedelta, timezone
import httpx
from sqlalchemy import and_, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.models.audit import AuditEvent
from app.models.audit_export import AuditExportSettings
from app.services import notification_secret_store
from app.version import APP_VERSION
logger = logging.getLogger(__name__)
SINGLETON_ID = 1
BATCH = 200
SECRET_NAME = "audit_webhook_signing_secret"
_FACILITY_LOCAL0 = 16
_BAD_STATUSES = {"denied", "rejected", "error", "revert_failed", "failed"}
class ExportError(Exception):
pass
async def get_settings(db: AsyncSession) -> AuditExportSettings:
row = await db.get(AuditExportSettings, SINGLETON_ID)
if row is None:
row = AuditExportSettings(
id=SINGLETON_ID, syslog_enabled=False, syslog_port=514, syslog_protocol="udp",
webhook_enabled=False, webhook_signing_secret_set=False,
)
db.add(row)
await db.flush()
return row
def event_dict(event: AuditEvent) -> dict:
return {
"id": str(event.id), "time": event.created_at.isoformat(), "correlation_id": event.correlation_id,
"actor": event.actor, "kind": event.kind, "action": event.action, "status": event.status,
"entity_id": event.entity_id, "entity_name": event.entity_name,
"host_id": event.host_id, "host_name": event.host_name, "message": event.message,
}
def syslog_line(event: dict, hostname: str = "opnexus") -> bytes:
"""RFC 5424: <PRI>1 TIMESTAMP HOSTNAME APP-NAME PROCID MSGID - MSG (JSON)."""
severity = 4 if event["status"] in _BAD_STATUSES else 6
pri = _FACILITY_LOCAL0 * 8 + severity
return f"<{pri}>1 {event['time']} {hostname} opnexus - AUDIT - {json.dumps(event, ensure_ascii=False)}".encode()
def _frame_tcp(lines: list[bytes]) -> bytes:
"""RFC 6587 octet counting: '<len> <message>' per message."""
return b"".join(str(len(line)).encode() + b" " + line for line in lines)
def _send_syslog_sync(row: AuditExportSettings, lines: list[bytes]) -> None:
host, port = row.syslog_host, row.syslog_port
if row.syslog_protocol == "udp":
with socket.socket(socket.AF_INET6 if ":" in (host or "") else socket.AF_INET, socket.SOCK_DGRAM) as sock:
for line in lines:
sock.sendto(line[:8000], (host, port))
return
with socket.create_connection((host, port), timeout=settings.opnsense_timeout) as raw:
if row.syslog_protocol == "tls":
context = ssl.create_default_context(cafile=settings.smtp_ca_file or None)
with context.wrap_socket(raw, server_hostname=host) as tls:
tls.sendall(_frame_tcp(lines))
else:
raw.sendall(_frame_tcp(lines))
async def send_syslog(row: AuditExportSettings, events: list[dict]) -> None:
if not row.syslog_host:
raise ExportError("Kein Syslog-Ziel konfiguriert")
try:
await asyncio.to_thread(_send_syslog_sync, row, [syslog_line(e) for e in events])
except (OSError, ssl.SSLError) as exc:
raise ExportError(f"Syslog-Zustellung fehlgeschlagen ({type(exc).__name__})") from None
async def send_webhook(row: AuditExportSettings, events: list[dict]) -> None:
if not row.webhook_url:
raise ExportError("Keine Webhook-URL konfiguriert")
raw = json.dumps({"source": "opnexus", "version": APP_VERSION, "events": events}, ensure_ascii=False).encode()
headers = {"Content-Type": "application/json"}
if row.webhook_signing_secret_set:
secret = notification_secret_store.read_secret(SECRET_NAME)
if not secret:
raise ExportError("Signaturschlüssel ist konfiguriert, aber nicht lesbar – bitte neu setzen")
headers["X-OPNexus-Signature"] = "sha256=" + hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest()
try:
async with httpx.AsyncClient(timeout=settings.opnsense_timeout) as client:
(await client.post(row.webhook_url, content=raw, headers=headers)).raise_for_status()
except httpx.HTTPError as exc:
# The URL often is the credential: never put it in a message.
reason = f"HTTP {exc.response.status_code}" if isinstance(exc, httpx.HTTPStatusError) else type(exc).__name__
raise ExportError(f"Webhook-Zustellung fehlgeschlagen ({reason})") from None
def active(row: AuditExportSettings) -> bool:
return bool((row.syslog_enabled and row.syslog_host) or (row.webhook_enabled and row.webhook_url))
async def _ship(row: AuditExportSettings, events: list[dict]) -> None:
if row.syslog_enabled and row.syslog_host:
await send_syslog(row, events)
if row.webhook_enabled and row.webhook_url:
await send_webhook(row, events)
async def run_once(db: AsyncSession, lag_seconds: float = 3.0) -> int:
"""Ship one batch; returns how many events went out (0 when idle or failing).
Only events at least ``lag_seconds`` old are shipped: an event is stamped when it
is created, a moment before its transaction commits, and the cursor must not
jump past one that becomes visible a little later."""
row = await get_settings(db)
if not active(row):
return 0
if row.cursor_created_at is None:
# First round after enabling: start from now, do not replay history.
row.cursor_created_at = datetime.now(timezone.utc)
row.cursor_id = None
await db.commit()
return 0
after = AuditEvent.created_at > row.cursor_created_at
if row.cursor_id:
after = or_(after, and_(AuditEvent.created_at == row.cursor_created_at, AuditEvent.id > uuid.UUID(row.cursor_id)))
ripe = AuditEvent.created_at <= datetime.now(timezone.utc) - timedelta(seconds=lag_seconds)
# Ordering by (created_at, id) must match the cursor comparison above.
events = (await db.scalars(select(AuditEvent).where(after, ripe).order_by(AuditEvent.created_at, AuditEvent.id).limit(BATCH))).all()
if not events:
return 0
try:
await _ship(row, [event_dict(e) for e in events])
except ExportError as exc:
row.last_error, row.last_error_at = str(exc)[:300], datetime.now(timezone.utc)
await db.commit()
logger.warning("Audit-Export: %s", exc)
return 0
row.cursor_created_at, row.cursor_id = events[-1].created_at, str(events[-1].id)
row.last_success_at, row.last_error = datetime.now(timezone.utc), None
await db.commit()
return len(events)
async def send_test(row: AuditExportSettings) -> None:
event = {
"id": "00000000-0000-0000-0000-000000000000", "time": datetime.now(timezone.utc).isoformat(),
"correlation_id": "test", "actor": "opnexus", "kind": "test", "action": "test", "status": "accepted",
"entity_id": None, "entity_name": None, "host_id": None, "host_name": None,
"message": "Dies ist ein Test des Audit-Exports von OPNexus.",
}
if not (row.syslog_enabled or row.webhook_enabled):
raise ExportError("Kein Kanal eingeschaltet")
await _ship(row, [event])
+3
View File
@@ -88,6 +88,9 @@ def deployment_history(session, flush_context, instances):
if entity is not None and action == "deploy":
from importlib import import_module
builder = import_module(f"app.services.{kind}_deploy_service")._build_payload
try:
details["requested_payload"] = builder(entity, host) if host is not None else builder(entity)
except Exception: # a missing variable must not break the audit write; record the unresolved intent instead
details["requested_payload"] = builder(entity)
session.add(make_event(kind=kind, action=action, status="removed" if deleted else dep.status,
entity_id=str(entity_id), entity_name=(getattr(entity, "name", None) or getattr(entity, "description", None)),
+3 -2
View File
@@ -1,7 +1,7 @@
"""Authentication core: password hashing, server-side sessions, brute-force
protection and the initial admin bootstrap.
Two roles ("admin", "read_only"), no orgs, no API keys — see
Two roles ("admin", "read_only"), no orgs; API tokens live in api_token_service — see
AdminUser/AuthMiddleware for how a role is enforced. Originally mirrored
InfraPanel's own app/services/auth_service.py (single-admin-only); the role
column and its enforcement are OPNexus-specific additions on top of that.
@@ -236,6 +236,7 @@ async def create_session(
user_agent: str | None = None,
ip_address: str | None = None,
verified_hash: str | None = None,
lifetime_seconds: int | None = None,
) -> tuple[str, datetime]:
"""Create a session, returning the raw token (to be set as a cookie) and its expiry.
@@ -253,7 +254,7 @@ async def create_session(
await db.rollback()
raise api_error(401, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
token = secrets.token_urlsafe(SESSION_TOKEN_BYTES)
expires_at = _now() + timedelta(seconds=settings.session_lifetime)
expires_at = _now() + timedelta(seconds=lifetime_seconds or settings.session_lifetime)
session = Session(
user_id=user_id,
token_hash=_hash_token(token),
+45
View File
@@ -0,0 +1,45 @@
"""Client-side encryption of config backups before they leave the server.
File format (``.opxbak``): ``OPXBAK1`` | salt (16) | nonce (12) | AES-256-GCM ciphertext+tag.
The key is scrypt(passphrase, salt). The header is authenticated as associated data, so a
file cannot be re-labelled. The passphrase never leaves OPNexus' secret store; without it
an offsite copy is unreadable, which is the point — and also why it must be kept somewhere
else too (see the handbook). Decrypt with ``python -m app.backup_decrypt``.
"""
import os
from cryptography.exceptions import InvalidTag
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
MAGIC = b"OPXBAK1"
_SALT, _NONCE = 16, 12
# scrypt cost: ~64 MiB, well above interactive defaults; a backup is encrypted rarely.
_N, _R, _P = 2**16, 8, 1
class DecryptError(Exception):
pass
def _key(passphrase: str, salt: bytes) -> bytes:
return Scrypt(salt=salt, length=32, n=_N, r=_R, p=_P).derive(passphrase.encode())
def encrypt(plaintext: bytes, passphrase: str) -> bytes:
salt, nonce = os.urandom(_SALT), os.urandom(_NONCE)
header = MAGIC + salt + nonce
return header + AESGCM(_key(passphrase, salt)).encrypt(nonce, plaintext, header)
def decrypt(blob: bytes, passphrase: str) -> bytes:
header_len = len(MAGIC) + _SALT + _NONCE
if len(blob) < header_len + 16 or not blob.startswith(MAGIC):
raise DecryptError("Keine OPNexus-Backup-Datei (Kopfzeile fehlt).")
header, body = blob[:header_len], blob[header_len:]
salt = header[len(MAGIC): len(MAGIC) + _SALT]
nonce = header[len(MAGIC) + _SALT:]
try:
return AESGCM(_key(passphrase, salt)).decrypt(nonce, body, header)
except InvalidTag:
raise DecryptError("Falsche Passphrase oder beschädigte Datei.") from None
@@ -0,0 +1,60 @@
"""Readable, redacted diff between two stored config backups of the same host.
Both files are parsed (defusedxml), the volatile ``revision`` block is dropped, and every leaf that
looks like a secret (password, key, token, PSK, community...) or is a long blob (certificates,
private keys) is replaced by a short hash. A changed secret therefore still shows up as a change,
but its value never leaves the server. Output is a unified diff, capped in size.
"""
import difflib
import hashlib
import hmac
import os
import re
from xml.etree.ElementTree import Element, indent, tostring
import defusedxml.ElementTree as safe_ET
MAX_LINES = 3000
MAX_BYTES = 32 * 1024 * 1024
_SECRET = re.compile(r"pass|pwd|psk|secret|key|token|prv|hash|community|bindpw|privpass|otp|seed|cred", re.I)
_HARMLESS_SUFFIX = re.compile(r"(len|length|type|size|bits|file|name)$", re.I)
_BLOB = 120
class InvalidConfig(ValueError):
pass
def _redact(root: Element, salt: bytes) -> None:
for el in root.iter():
if len(el) == 0 and el.text and el.text.strip():
text = el.text.strip()
if (_SECRET.search(el.tag) and not _HARMLESS_SUFFIX.search(el.tag)) or len(text) > _BLOB:
# Salted per diff: both sides of one diff share the salt (so a change shows), but the value cannot be
# looked up in a dictionary or compared across hosts/requests.
el.text = "‹" + hmac.new(salt, text.encode(), hashlib.sha256).hexdigest()[:12] + "›"
def normalise(xml_bytes: bytes, salt: bytes) -> list[str]:
if not xml_bytes.strip() or len(xml_bytes) > MAX_BYTES:
raise InvalidConfig("leer oder zu groß")
try:
root = safe_ET.fromstring(xml_bytes)
except Exception as exc: # defusedxml raises several parse/forbidden-entity types
raise InvalidConfig(type(exc).__name__) from None
if root.tag != "opnsense":
raise InvalidConfig("keine OPNsense-Konfiguration")
for volatile in root.findall("revision"):
root.remove(volatile)
_redact(root, salt)
indent(root, space=" ")
return tostring(root, encoding="unicode").splitlines()
def diff(old: bytes, new: bytes) -> dict:
salt = os.urandom(16)
a, b = normalise(old, salt), normalise(new, salt)
lines = list(difflib.unified_diff(a, b, "älter", "neuer", lineterm="", n=2))
added = sum(1 for l in lines if l.startswith("+") and not l.startswith("+++"))
removed = sum(1 for l in lines if l.startswith("-") and not l.startswith("---"))
return {"identical": not lines, "added": added, "removed": removed, "truncated": len(lines) > MAX_LINES, "lines": lines[:MAX_LINES]}
@@ -0,0 +1,136 @@
"""Scheduled config backups: take, keep the newest N, copy encrypted offsite.
Called by a poll loop (main.py). Per enabled host a new snapshot is taken when the newest one
is older than the interval; the oldest beyond ``keep_count`` are deleted. When an offsite target
is set, every local snapshot that is not offsite yet is encrypted (backup_crypto) and uploaded
(s3_client); the ledger ``backup_uploads`` remembers what is already there. Nothing is ever
deleted offsite: use the bucket's lifecycle rules for that.
"""
import logging
import re
from datetime import datetime, timedelta, timezone
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.backup_settings import BackupSettings, BackupUpload
from app.models.host import OpnsenseHost
from app.services import backup_crypto, backup_store, notification_secret_store
from app.services import opnsense_client as client
from app.services.opnsense_client import OpnsenseAPIError
from app.services.s3_client import S3Client, S3Error
logger = logging.getLogger(__name__)
SINGLETON_ID = 1
S3_SECRET, PASSPHRASE = "backup_s3_secret", "backup_passphrase"
MAX_UPLOADS_PER_RUN = 20
async def get_settings(db: AsyncSession) -> BackupSettings:
row = await db.get(BackupSettings, SINGLETON_ID)
if row is None:
row = BackupSettings(
id=SINGLETON_ID, scheduled_enabled=True, interval_hours=24, keep_count=30, offsite_enabled=False,
s3_region="us-east-1", s3_prefix="opnexus", s3_path_style=True, s3_secret_set=False, passphrase_set=False,
)
db.add(row)
await db.flush()
return row
def offsite_ready(cfg: BackupSettings) -> bool:
return bool(cfg.offsite_enabled and cfg.s3_endpoint and cfg.s3_bucket and cfg.s3_access_key and cfg.s3_secret_set and cfg.passphrase_set)
def s3_for(cfg: BackupSettings) -> S3Client:
secret = notification_secret_store.read_secret(S3_SECRET)
if not (cfg.s3_endpoint and cfg.s3_bucket and cfg.s3_access_key and secret):
raise S3Error("Offsite-Ziel unvollständig (Endpunkt, Bucket, Zugangsdaten)")
return S3Client(endpoint=cfg.s3_endpoint, region=cfg.s3_region, bucket=cfg.s3_bucket, access_key=cfg.s3_access_key,
secret_key=secret, path_style=cfg.s3_path_style)
def remote_key(cfg: BackupSettings, host: OpnsenseHost, snapshot_id: str) -> str:
safe = re.sub(r"[^A-Za-z0-9._-]+", "-", host.name).strip("-") or "host"
prefix = cfg.s3_prefix.strip("/")
return "/".join(p for p in (prefix, f"{safe}-{str(host.id)[:8]}", snapshot_id.removesuffix(".xml") + ".opxbak") if p)
async def _take(host: OpnsenseHost) -> str:
content = await client.download_backup(host)
if b"<opnsense" not in content[:4096]:
raise OpnsenseAPIError("Antwort ist keine OPNsense-Konfiguration")
return backup_store.save_snapshot(host.id, content)
async def _prune(db: AsyncSession, host: OpnsenseHost, keep: int) -> int:
old = backup_store.list_snapshots(host.id)[keep:]
for snapshot in old:
backup_store.delete_snapshot(host.id, snapshot["id"])
if old:
rows = (await db.scalars(select(BackupUpload).where(BackupUpload.host_id == host.id, BackupUpload.snapshot_id.in_([s["id"] for s in old])))).all()
for row in rows:
await db.delete(row)
return len(old)
async def upload_pending(db: AsyncSession, cfg: BackupSettings, hosts: list[OpnsenseHost]) -> int:
"""Encrypt and upload local snapshots that are not offsite yet (newest first, bounded per run)."""
passphrase = notification_secret_store.read_secret(PASSPHRASE)
if not passphrase:
raise S3Error("Passphrase fehlt oder ist nicht lesbar – bitte neu setzen")
s3 = s3_for(cfg)
done, budget = 0, MAX_UPLOADS_PER_RUN
for host in hosts:
have = set((await db.scalars(select(BackupUpload.snapshot_id).where(BackupUpload.host_id == host.id))).all())
for snapshot in backup_store.list_snapshots(host.id):
if snapshot["id"] in have:
continue
if budget == 0:
return done
content = backup_store.read_snapshot(host.id, snapshot["id"])
if content is None:
continue
key = remote_key(cfg, host, snapshot["id"])
await s3.put(key, backup_crypto.encrypt(content, passphrase))
db.add(BackupUpload(host_id=host.id, snapshot_id=snapshot["id"], remote_key=key, uploaded_at=datetime.now(timezone.utc)))
await db.commit()
budget -= 1
done += 1
return done
async def run_due(db: AsyncSession, *, force: bool = False) -> dict:
cfg = await get_settings(db)
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all())
taken = failed = pruned = 0
if cfg.scheduled_enabled or force:
now = datetime.now(timezone.utc)
for host in hosts:
snapshots = backup_store.list_snapshots(host.id)
if not force and snapshots and now - snapshots[0]["created_at"] < timedelta(hours=cfg.interval_hours):
pruned += await _prune(db, host, cfg.keep_count)
continue
try:
await _take(host)
taken += 1
except OpnsenseAPIError as exc:
failed += 1
logger.warning("Geplantes Backup von %s fehlgeschlagen: %s", host.name, exc)
pruned += await _prune(db, host, cfg.keep_count)
cfg.last_run_at = now
uploaded = 0
if cfg.offsite_enabled:
try:
if not offsite_ready(cfg):
raise S3Error("Offsite-Ziel unvollständig (Endpunkt, Bucket, Zugangsdaten, Passphrase)")
uploaded = await upload_pending(db, cfg, hosts)
cfg.last_offsite_error = None
if uploaded or cfg.last_offsite_ok_at is None:
cfg.last_offsite_ok_at = datetime.now(timezone.utc)
except S3Error as exc:
cfg.last_offsite_error = str(exc)[:300]
logger.warning("Offsite-Backup: %s", exc)
await db.commit()
return {"hosts": len(hosts), "taken": taken, "failed": failed, "pruned": pruned, "uploaded": uploaded}
+10 -1
View File
@@ -42,7 +42,8 @@ def list_snapshots(host_id: uuid.UUID) -> list[dict]:
"created_at": datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc),
"size": stat.st_size,
})
return sorted(items, key=lambda i: i["created_at"], reverse=True)
# Names are sortable timestamps: they break ties between files written within one mtime tick.
return sorted(items, key=lambda i: (i["created_at"], i["id"]), reverse=True)
def read_snapshot(host_id: uuid.UUID, backup_id: str) -> bytes | None:
@@ -53,3 +54,11 @@ def read_snapshot(host_id: uuid.UUID, backup_id: str) -> bytes | None:
if backup_id != path.name or not path.is_file():
return None
return path.read_bytes()
def delete_snapshot(host_id: uuid.UUID, backup_id: str) -> bool:
path = _host_dir(host_id) / backup_id
if backup_id != path.name or not path.is_file():
return False
path.unlink()
return True
+300
View File
@@ -0,0 +1,300 @@
"""Security posture of a firewall: hardening checks and rule hygiene.
The checks read the firewall's own config.xml (one read-only download, parsed
in memory with defusedxml, never stored) plus two facts OPNexus already knows:
whether firmware updates are pending and how old the newest local backup is.
The XML layout was verified against real OPNsense 26.7 configs: rules live in
``OPNsense/Firewall/Filter/rules/rule`` (MVC) or, on installs not yet migrated,
in ``filter/rule`` (legacy). Both are normalised to the same shape below.
``evaluate`` is pure (bytes in, findings out). A finding is
``{id, severity, passed, params}``; ``passed`` is None when a check could not be
decided. The score starts at 100 and loses a penalty per failed check, so one
critical finding (an any-any pass rule on the WAN) already turns the host red.
"""
import asyncio
import re
from datetime import datetime, timezone
from xml.etree.ElementTree import Element
import defusedxml.ElementTree as safe_ET
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.compliance_result import ComplianceResult
from app.models.host import OpnsenseHost
from app.services import backup_store, rule_hit_service
from app.services import opnsense_client as client
from app.services.opnsense_client import OpnsenseAPIError
MAX_CONFIG_BYTES = 32 * 1024 * 1024
BACKUP_MAX_AGE_DAYS = 7
PENALTY = {"critical": 30, "high": 15, "medium": 8, "low": 3, "info": 0}
SEVERITY_ORDER = ("critical", "high", "medium", "low", "info")
MAX_EXAMPLES = 5
# German titles for notifications (the UI translates the check ids itself).
TITLES_DE = {
"webgui_https": "Web-Oberfläche nur über HTTPS", "ssh_hardened": "SSH abgesichert", "snmp_community": "SNMP-Community nicht „public“",
"wan_any_any": "Keine Allow-all-Regel am WAN", "internal_any_any": "Keine Allow-all-Regel mit beliebiger Quelle (intern)",
"rules_shadowed": "Keine verdeckten Regeln", "rules_duplicate": "Keine doppelten Regeln", "rules_documented": "Regeln sind beschriftet",
"rules_unused": "Regeln in Benutzung", "ntp_configured": "Zeitserver gesetzt", "remote_logging": "Logs liegen auch woanders",
"ids_enabled": "Einbruchserkennung aktiv", "firmware_current": "Firmware aktuell", "backup_recent": "Aktuelles Backup",
}
SEVERITIES_DE = {"critical": "kritisch", "high": "hoch", "medium": "mittel", "low": "niedrig", "info": "Hinweis"}
def grade(score: int) -> str:
return "good" if score >= 90 else "fair" if score >= 70 else "poor"
def _text(el: Element | None, path: str) -> str | None:
node = el.find(path) if el is not None else None
if node is None or node.text is None or not node.text.strip():
return None
return node.text.strip()
# ── Rule normalisation ───────────────────────────────────────────────────
def _endpoint(node: Element | None) -> tuple[str, bool, str | None]:
"""(network, negated, port) of a legacy <source>/<destination> element."""
if node is None:
return "any", False, None
negated = node.find("not") is not None
port = _text(node, "port")
if node.find("any") is not None:
return "any", negated, port
for tag in ("network", "address"):
value = _text(node, tag)
if value:
return value, negated, port
return "any", negated, port
def _mvc_rule(rule: Element) -> dict:
return {
"enabled": _text(rule, "enabled") != "0",
"action": (_text(rule, "action") or "pass").lower(),
"interfaces": [i for i in (_text(rule, "interface") or "").split(",") if i],
"direction": _text(rule, "direction") or "in",
"ipprotocol": _text(rule, "ipprotocol") or "inet",
"protocol": (_text(rule, "protocol") or "any").lower(),
"src": _text(rule, "source_net") or "any", "src_not": _text(rule, "source_not") == "1",
"sport": _text(rule, "source_port"),
"dst": _text(rule, "destination_net") or "any", "dst_not": _text(rule, "destination_not") == "1",
"dport": _text(rule, "destination_port"),
"floating": False,
"quick": _text(rule, "quick") != "0",
"descr": _text(rule, "description") or "",
"order": int(_text(rule, "sequence") or 0),
}
def _legacy_rule(rule: Element, index: int) -> dict:
src, src_not, sport = _endpoint(rule.find("source"))
dst, dst_not, dport = _endpoint(rule.find("destination"))
return {
"enabled": rule.find("disabled") is None,
"action": (_text(rule, "type") or "pass").lower(),
"interfaces": [i for i in (_text(rule, "interface") or "").split(",") if i],
"direction": _text(rule, "direction") or "in",
"ipprotocol": _text(rule, "ipprotocol") or "inet",
"protocol": (_text(rule, "protocol") or "any").lower(),
"src": src, "src_not": src_not, "sport": sport, "dst": dst, "dst_not": dst_not, "dport": dport,
"floating": _text(rule, "floating") == "yes",
"quick": _text(rule, "quick") != "0",
"descr": _text(rule, "descr") or "",
"order": index,
}
def extract_rules(root: Element) -> list[list[dict]]:
"""Rule lists in evaluation order, one per source (MVC, legacy)."""
mvc = sorted((_mvc_rule(r) for r in root.findall("OPNsense/Firewall/Filter/rules/rule")), key=lambda r: r["order"])
legacy = [_legacy_rule(r, i) for i, r in enumerate(root.findall("filter/rule"))]
return [group for group in (mvc, legacy) if group]
def wan_interfaces(root: Element) -> set[str]:
"""Interfaces facing the outside: the one named wan*, or any with an upstream gateway."""
result = set()
for node in root.findall("interfaces/*"):
if node.tag.lower().startswith("wan") or _text(node, "gateway"):
result.add(node.tag)
return result
def _is_any_any(rule: dict) -> bool:
return (
rule["enabled"] and rule["action"] == "pass" and rule["src"] == "any" and not rule["src_not"]
and rule["dst"] == "any" and not rule["dst_not"] and rule["protocol"] == "any"
and not rule["sport"] and not rule["dport"]
)
def _covers(a: dict, b: dict) -> bool:
"""Every packet matched by rule b is already matched by the earlier rule a.
Only exact-or-any comparisons, so it never claims coverage it cannot prove."""
if not (a["enabled"] and b["enabled"]) or a["floating"] or b["floating"] or not a["quick"]:
return False
if not a["interfaces"] or not set(b["interfaces"]) <= set(a["interfaces"]):
return False
if a["direction"] != b["direction"] or a["ipprotocol"] != b["ipprotocol"]:
return False
if a["protocol"] != "any" and a["protocol"] != b["protocol"]:
return False
for side in ("src", "dst"):
if a[side + "_not"] or b[side + "_not"]:
if (a[side], a[side + "_not"]) != (b[side], b[side + "_not"]):
return False
elif a[side] != "any" and a[side] != b[side]:
return False
for port in ("sport", "dport"):
if a[port] and a[port] != b[port]:
return False
return True
def _label(rule: dict) -> str:
return rule["descr"] or f"{rule['action']} {rule['src']} → {rule['dst']}"
def analyse_rules(groups: list[list[dict]]) -> dict:
shadowed, duplicates = [], []
for rules in groups:
for j, later in enumerate(rules):
for earlier in rules[:j]:
if _covers(earlier, later):
(duplicates if earlier["action"] == later["action"] and _covers(later, earlier) else shadowed).append(_label(later))
break
return {"shadowed": shadowed, "duplicates": duplicates}
# ── Checks ───────────────────────────────────────────────────────────────
def _finding(check_id: str, severity: str, passed: bool | None, **params) -> dict:
return {"id": check_id, "severity": severity, "passed": passed, "params": params}
def _listing(items: list[str]) -> dict:
return {"count": len(items), "examples": items[:MAX_EXAMPLES]}
def evaluate(xml_bytes: bytes, *, firmware_status: str | None, newest_backup_age_s: float | None,
unused_rules: dict | None = None) -> list[dict]:
if len(xml_bytes) > MAX_CONFIG_BYTES or not xml_bytes.strip():
raise ValueError("config.xml missing or too large")
root = safe_ET.fromstring(xml_bytes)
if root.tag != "opnsense":
raise ValueError("not an OPNsense config.xml")
findings: list[dict] = []
# Management access
protocol = _text(root, "system/webgui/protocol")
findings.append(_finding("webgui_https", "high", protocol == "https", protocol=protocol or "http"))
ssh = root.find("system/ssh")
ssh_enabled = ssh is not None and _text(ssh, "enabled") in ("enabled", "1")
root_login = ssh_enabled and _text(ssh, "permitrootlogin") == "1"
password_login = ssh_enabled and _text(ssh, "passwordauth") == "1"
if root_login and password_login:
findings.append(_finding("ssh_hardened", "high", False, reason="root_password"))
elif password_login:
findings.append(_finding("ssh_hardened", "medium", False, reason="password"))
else:
findings.append(_finding("ssh_hardened", "high", True))
community = _text(root, "snmpd/rocommunity")
if root.find("snmpd") is not None:
findings.append(_finding("snmp_community", "high", (community or "").lower() not in ("", "public")))
# Rules
wans = wan_interfaces(root)
groups = extract_rules(root)
all_rules = [rule for group in groups for rule in group]
wan_open = [_label(r) for r in all_rules if _is_any_any(r) and set(r["interfaces"]) & wans]
findings.append(_finding("wan_any_any", "critical", not wan_open, **_listing(wan_open)))
inner_open = [_label(r) for r in all_rules if _is_any_any(r) and r["interfaces"] and not set(r["interfaces"]) & wans]
findings.append(_finding("internal_any_any", "medium", not inner_open, **_listing(inner_open)))
analysis = analyse_rules(groups)
findings.append(_finding("rules_shadowed", "medium", not analysis["shadowed"], **_listing(analysis["shadowed"])))
findings.append(_finding("rules_duplicate", "low", not analysis["duplicates"], **_listing(analysis["duplicates"])))
undocumented = [f"{r['action']} {r['src']} → {r['dst']}" for r in all_rules if r["enabled"] and not r["descr"]]
findings.append(_finding("rules_documented", "low", not undocumented, **_listing(undocumented)))
# Usage (from the pf counters OPNexus has been sampling; None = no data yet)
if unused_rules is not None:
names = unused_rules["unused"]
findings.append(_finding("rules_unused", "low", None if (not names and unused_rules["collecting"]) else not names, **_listing(names)))
# Operations
findings.append(_finding("ntp_configured", "low", bool(_text(root, "system/timeservers"))))
syslog_targets = [d for d in root.findall("OPNsense/Syslog/destinations/destination") if _text(d, "enabled") != "0"]
findings.append(_finding("remote_logging", "low", bool(syslog_targets) or bool(_text(root, "syslog/remoteserver"))))
findings.append(_finding("ids_enabled", "info", _text(root, "OPNsense/IDS/general/enabled") == "1"))
# From what OPNexus already knows
findings.append(_finding("firmware_current", "medium", None if firmware_status is None else firmware_status != "update"))
if newest_backup_age_s is None:
findings.append(_finding("backup_recent", "medium", False, age_days=None))
else:
age_days = int(newest_backup_age_s // 86400)
findings.append(_finding("backup_recent", "medium", age_days < BACKUP_MAX_AGE_DAYS, age_days=age_days))
return findings
def score(findings: list[dict]) -> int:
return max(0, 100 - sum(PENALTY[f["severity"]] for f in findings if f["passed"] is False))
def counts(findings: list[dict]) -> dict:
result = {severity: 0 for severity in SEVERITY_ORDER}
for finding in findings:
if finding["passed"] is False:
result[finding["severity"]] += 1
return result
# ── Running it for a host ────────────────────────────────────────────────
async def check_host(db: AsyncSession, host: OpnsenseHost) -> ComplianceResult:
"""Download the config, evaluate, store (one row per host, replaced each time)."""
row = await db.scalar(select(ComplianceResult).where(ComplianceResult.host_id == host.id))
if row is None:
row = ComplianceResult(host_id=host.id)
db.add(row)
row.checked_at = datetime.now(timezone.utc)
try:
xml = await client.download_backup(host)
try:
firmware_status = (await client.get_firmware_status(host)).get("status")
except OpnsenseAPIError:
firmware_status = None
snapshots = backup_store.list_snapshots(host.id)
age = (datetime.now(timezone.utc) - snapshots[0]["created_at"]).total_seconds() if snapshots else None
try:
await rule_hit_service.sample_host(db, host)
except OpnsenseAPIError:
pass # usage data is a bonus; the check does not depend on it
# Untrusted XML, O(n^2) rule analysis: keep it off the event loop, and treat ANY parse failure as 'invalid'.
findings = await asyncio.to_thread(
evaluate, xml, firmware_status=firmware_status, newest_backup_age_s=age,
unused_rules=await rule_hit_service.unused_summary(db, host.id),
)
row.findings, row.score, row.error = findings, score(findings), None
except OpnsenseAPIError as exc:
row.findings, row.score, row.error = [], None, "unreachable"
row.error_detail = str(exc)[:300]
except Exception as exc: # ParseError is a SyntaxError, not a ValueError; one bad config must not stop the other hosts
row.findings, row.score, row.error = [], None, "invalid_config"
row.error_detail = f"{type(exc).__name__}: {exc}"[:300]
await db.commit()
return row
async def check_all(db: AsyncSession) -> int:
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)).order_by(OpnsenseHost.name))).all()
for host in hosts:
await check_host(db, host)
return len(hosts)
@@ -59,7 +59,9 @@ async def _shared_change_blockers(db, fleet):
return blocked
async def inspect_host(db, host):
async def inspect_host(db, host, *, skip_role_checks=False):
"""skip_role_checks: for the planned CARP handover, where the node is still MASTER when
it is inspected and becomes BACKUP only afterwards (the role is verified separately)."""
fleet = await peers(db, host)
statuses, info, firmware, running = await asyncio.gather(
get_statuses(fleet), client.get_firmware_info(host), client.get_firmware_status(host), client.get_firmware_running(host))
@@ -68,7 +70,7 @@ async def inspect_host(db, host):
if statuses.get(host.id, {}).get("status") != "online": blocked.append("Host ist nicht erreichbar.")
# Fail closed: only the verified ready value permits a disruptive action.
if running.get("status") != "ready": blocked.append("Firmware-Auftrag läuft oder Status ist unbekannt.")
if host.cluster_name:
if host.cluster_name and not skip_role_checks:
others = [h for h in fleet if h.id != host.id]
if statuses.get(host.id, {}).get("carp_status") != "BACKUP":
blocked.append("Cluster-Wartung nur auf dem CARP-BACKUP. Rollenwechsel separat in OPNsense durchführen.")
@@ -32,6 +32,14 @@ def _extract_carp_status(data: dict | None) -> str | None:
return None
def _vip_states(data: dict | None) -> dict[str, str]:
"""Per-VHID CARP state ("MASTER", "BACKUP", "DISABLED", "INIT", ...). The host-level carp_status above
collapses these, so one VHID that is down on a node is invisible there."""
rows = data.get("rows") if isinstance(data, dict) else None
return {str(row.get("vhid")): str(row.get("status", "")).upper()
for row in rows or [] if isinstance(row, dict) and row.get("vhid") not in (None, "")}
async def _status_for(host: OpnsenseHost) -> dict:
if not host.enabled:
return {"status": "disabled", "carp_status": None, "status_detail": None}
@@ -43,6 +51,7 @@ async def _status_for(host: OpnsenseHost) -> dict:
return {
"status": "online",
"carp_status": _extract_carp_status(data),
"carp_vips": _vip_states(data),
"carp_maintenance_mode": _truthy(carp.get("maintenancemode"), default=False),
"carp_allowed": _truthy(carp.get("allow"), default=True),
"carp_demotion": carp.get("demotion"),
+155
View File
@@ -0,0 +1,155 @@
"""Prometheus exposition of the fleet state (GET /api/metrics).
Deliberately database-only, like the dashboard summary: a scrape must never
fan out live calls to every firewall (an offline host would stall it past the
scraper's timeout and a 15 s scrape interval would hammer the fleet). The
inputs are what OPNexus' own pollers already store:
* host reachability and resources — the 60 s resource sampler (a host that
produced no sample for ``STALE_AFTER`` is reported down),
* CARP role — the latest observed transition (carp_events),
* open problems — the notification engine's problem table,
* pending actions — the same list the dashboard shows.
"""
from datetime import datetime, timedelta, timezone
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.carp_event import CarpEvent
from app.models.compliance_result import ComplianceResult
from app.models.host import OpnsenseHost
from app.models.notification import NotificationProblem
from app.models.resource_sample import HostResourceSample
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.routers.dashboard import collect_pending_actions
from app.services import compliance_service
from app.version import APP_VERSION
CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
# The sampler runs every 60 s; three missed rounds mean the host is not reachable.
STALE_AFTER = timedelta(seconds=180)
def _escape(value: str) -> str:
return value.replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")
class _Exposition:
def __init__(self) -> None:
self.lines: list[str] = []
def family(self, name: str, help_text: str, kind: str = "gauge") -> None:
self.lines.append(f"# HELP {name} {help_text}")
self.lines.append(f"# TYPE {name} {kind}")
def sample(self, name: str, value: float, **labels: str) -> None:
rendered = ",".join(f'{key}="{_escape(str(val))}"' for key, val in labels.items() if val is not None)
number = repr(float(value)) if not float(value).is_integer() else str(int(value))
self.lines.append(f"{name}{{{rendered}}} {number}" if rendered else f"{name} {number}")
def text(self) -> str:
return "\n".join(self.lines) + "\n"
async def render(db: AsyncSession) -> str:
now = datetime.now(timezone.utc)
out = _Exposition()
out.family("opnexus_info", "OPNexus version (always 1).")
out.sample("opnexus_info", 1, version=APP_VERSION)
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
latest_sample = (
select(HostResourceSample.host_id, func.max(HostResourceSample.sampled_at).label("at"))
.group_by(HostResourceSample.host_id)
.subquery()
)
samples = {
sample.host_id: sample
for sample in (
await db.scalars(
select(HostResourceSample).join(
latest_sample,
(HostResourceSample.host_id == latest_sample.c.host_id)
& (HostResourceSample.sampled_at == latest_sample.c.at),
)
)
).all()
}
latest_event = (
select(CarpEvent.host_id, func.max(CarpEvent.detected_at).label("at")).group_by(CarpEvent.host_id).subquery()
)
carp_state = {
event.host_id: event.to_status
for event in (
await db.scalars(
select(CarpEvent).join(
latest_event, (CarpEvent.host_id == latest_event.c.host_id) & (CarpEvent.detected_at == latest_event.c.at)
)
)
).all()
}
out.family("opnexus_hosts", "Number of managed firewalls.")
out.sample("opnexus_hosts", len(hosts))
out.family("opnexus_host_enabled", "1 if monitoring of the host is enabled.")
out.family("opnexus_host_up", "1 if the host answered the resource sampler within the last 3 minutes.")
out.family("opnexus_host_last_sample_timestamp_seconds", "Unix time of the last successful resource sample.")
out.family("opnexus_host_memory_used_percent", "Memory in use, percent (last sample).")
out.family("opnexus_host_load1", "1-minute load average (last sample).")
out.family("opnexus_host_disk_used_percent", "Fullest disk, percent (last sample).")
out.family("opnexus_host_carp_master", "1 if the host's last observed CARP role is MASTER, 0 if BACKUP; absent if unknown.")
for host in hosts:
labels = {"host": host.name, "cluster": host.cluster_name or ""}
sample = samples.get(host.id)
up = bool(host.enabled and sample is not None and now - sample.sampled_at <= STALE_AFTER)
out.sample("opnexus_host_enabled", 1 if host.enabled else 0, **labels)
out.sample("opnexus_host_up", 1 if up else 0, **labels)
if sample is not None:
out.sample("opnexus_host_last_sample_timestamp_seconds", sample.sampled_at.timestamp(), **labels)
for name, value in (
("opnexus_host_memory_used_percent", sample.mem_used_pct),
("opnexus_host_load1", sample.load_avg_1m),
("opnexus_host_disk_used_percent", sample.disk_used_pct),
):
if value is not None:
out.sample(name, value, **labels)
if carp_state.get(host.id) in ("MASTER", "BACKUP"):
out.sample("opnexus_host_carp_master", 1 if carp_state[host.id] == "MASTER" else 0, **labels)
out.family("opnexus_host_compliance_score", "Security posture score 0-100 of the last check (absent if never checked).")
out.family("opnexus_host_compliance_findings", "Failed security checks of the last check, by severity.")
results = {r.host_id: r for r in (await db.scalars(select(ComplianceResult))).all()}
for host in hosts:
row = results.get(host.id)
if row is None or row.score is None:
continue
labels = {"host": host.name, "cluster": host.cluster_name or ""}
out.sample("opnexus_host_compliance_score", row.score, **labels)
for severity, count in compliance_service.counts(row.findings).items():
out.sample("opnexus_host_compliance_findings", count, severity=severity, **labels)
out.family("opnexus_pending_actions", "Deployments and operations that need attention, by kind and status.")
counts: dict[tuple[str, str], int] = {}
for action in await collect_pending_actions(db):
counts[(action.kind, action.status)] = counts.get((action.kind, action.status), 0) + 1
for (kind, status), count in sorted(counts.items()):
out.sample("opnexus_pending_actions", count, kind=kind, status=status)
out.family("opnexus_problems_open", "Open problems known to the notification engine, by type.")
rows = (
await db.execute(
select(NotificationProblem.problem_type, func.count())
.where(NotificationProblem.status == "open")
.group_by(NotificationProblem.problem_type)
)
).all()
for problem_type, count in sorted(rows):
out.sample("opnexus_problems_open", count, type=problem_type)
out.family("opnexus_scheduled_maintenance_pending", "Scheduled maintenance jobs that have not run yet.")
pending = await db.scalar(select(func.count()).select_from(ScheduledMaintenance).where(ScheduledMaintenance.status == "pending"))
out.sample("opnexus_scheduled_maintenance_pending", pending or 0)
return out.text()
@@ -22,6 +22,7 @@ taken before the change is the way back.
Secrets (the CARP password) are never returned, hashed into the token only, and
never written to the audit log.
"""
import asyncio
import hashlib
import ipaddress
import json
@@ -362,6 +363,20 @@ async def _verify(host: OpnsenseHost, kind: str, action: str, uuid: str | None,
return False, ["unreadable"]
async def _carp_vip_active(host: OpnsenseHost, vhid: str, attempts: int = 5) -> bool:
"""True once the firewall reports the VHID as MASTER or BACKUP, i.e. the VIP really is on the interface and
taking part in CARP (not DISABLED/INIT). The config can look right while the live VIP is gone."""
for attempt in range(attempts):
if attempt:
await asyncio.sleep(2)
data = await opnsense_client.get_vip_status(host)
rows = data.get("rows") if isinstance(data, dict) else None
states = {str(r.get("vhid")): str(r.get("status", "")).upper() for r in rows or [] if isinstance(r, dict)}
if states.get(vhid) in ("MASTER", "BACKUP"):
return True
return False
async def execute(db: AsyncSession, host: OpnsenseHost, kind: str, action: str, uuid: str | None, fields: dict,
token: str, acknowledged: bool) -> dict:
await ensure_no_maintenance(db, [host])
@@ -384,6 +399,12 @@ async def execute(db: AsyncSession, host: OpnsenseHost, kind: str, action: str,
# VipSettingsController::getVipOverlay blanks address and prefix whenever "network" is missing from
# the request, so every VIP update must repeat it (found live: a partial set was rejected / could wipe it).
send.setdefault("network", plan["_before"]["network"])
# VipSettingsController::setItemAction queues "delete the old address" whenever the posted interface differs
# from the stored one, and a partial update that leaves "interface" out counts as different. The following
# reconfigure then removes the live VIP and, working from a stale address list, never puts it back (found
# live 2026-10-20: CARP VIP stuck on DISABLED after a description edit). So repeat the interface as well.
if plan["_before"].get("interface"):
send.setdefault("interface", plan["_before"]["interface"])
# What the object must look like afterwards: for an edit that is everything it had, plus the change.
expected = {**plan["_before"], **send} if action == "set" else dict(send)
entity = fields.get("name") or fields.get("descr") or plan["_before"].get("name") or plan["_before"].get("descr") or kind
@@ -407,14 +428,27 @@ async def execute(db: AsyncSession, host: OpnsenseHost, kind: str, action: str,
await audit_service.persist(**audit, status="rejected", message=str(exc)[:500], details=details)
raise api_error(502, ErrorCode.UPSTREAM_REJECTED, reason=str(exc)[:300]) from exc
new_uuid = uuid or (result.get("uuid") if isinstance(result, dict) else None)
# A description is not wired into anything, so there is nothing to re-apply for it.
cosmetic = kind == "vip" and action == "set" and {c["field"] for c in plan["changes"]} <= {"descr"}
carp_live = kind == "vip" and action != "del" and expected.get("mode") == "carp" and not cosmetic
try:
if not cosmetic:
await opnsense_client.reconfigure_network(host, kind)
live_ok = True
if carp_live:
live_ok = await _carp_vip_active(host, str(expected.get("vhid")))
if not live_ok: # one more apply usually brings it back up
await opnsense_client.reconfigure_network(host, kind)
live_ok = await _carp_vip_active(host, str(expected.get("vhid")))
except OpnsenseAPIError as exc:
# Written but not (provably) applied: say so instead of reporting success.
await audit_service.persist(**audit, status="apply_uncertain", message=str(exc)[:500], details=details)
raise api_error(502, ErrorCode.NETWORK_APPLY_UNCERTAIN, host=host.name, backup_id=backup_id) from exc
verified, mismatches = await _verify(host, kind, action, new_uuid, expected)
if not live_ok:
verified = False
mismatches = [*mismatches, "carp_vip_inactive"]
await audit_service.persist(**audit, status="applied" if verified else "unverified",
message=None if verified else ", ".join(mismatches)[:500], details=details)
return {"status": "applied", "uuid": new_uuid, "verified": verified, "mismatches": mismatches, "backup_id": backup_id}
+102 -1
View File
@@ -3,18 +3,22 @@ whatever's currently bad. Pure detection — no delivery, no state tracking
here (see notification_dedup.py for the open/resolved diffing that turns
these snapshots into actual notifications)."""
from collections import defaultdict
from datetime import datetime, timedelta, timezone
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.alias_deployment import AliasDeployment
from app.models.backup_settings import BackupSettings
from app.models.compliance_result import ComplianceResult
from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.models.port_forward_deployment import PortForwardDeployment
from app.models.rule_deployment import RuleDeployment
from app.services import nat_drift_service, network_drift_service, rule_drift_service
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.services import backup_store, nat_drift_service, network_drift_service, rule_drift_service
from app.services.notification_dedup import Problems
from app.services.host_status_service import get_statuses
from app.services.live_certificates_service import get_live_certificates
@@ -232,6 +236,28 @@ async def check_carp_health(db: AsyncSession) -> dict[str, dict]:
"condition": "no_master",
"host_names": sorted(h.name for h in online),
}
problems.update(_carp_vip_problems(cluster_name, online, statuses, split_brain=len(masters) == 2))
return problems
def _carp_vip_problems(cluster_name: str, online: list[OpnsenseHost], statuses: dict, split_brain: bool) -> dict[str, dict]:
"""Per-VHID checks. The host-level role is "MASTER" as soon as one VIP is, so a VIP that is down on a
node (a node without the VIP never advertises it, the peer takes it over) looks healthy in the totals."""
problems: dict[str, dict] = {}
for host in online:
vips = statuses.get(host.id, {}).get("carp_vips") or {}
down = sorted(v for v, state in vips.items() if state not in ("MASTER", "BACKUP"))
if down:
problems[f"{cluster_name}:vip_down:{host.name}"] = {
"cluster_name": cluster_name, "condition": "vip_down", "host_names": [host.name], "vhids": down,
}
# One node MASTER on some VHIDs and BACKUP on others: a partial failover. Already covered by the
# split-brain alert when both nodes hold a MASTER.
if not split_brain and {"MASTER", "BACKUP"} <= set(vips.values()):
problems[f"{cluster_name}:role_mismatch:{host.name}"] = {
"cluster_name": cluster_name, "condition": "role_mismatch", "host_names": [host.name],
"vhids": sorted(v for v, state in vips.items() if state == "MASTER"),
}
return problems
@@ -262,3 +288,78 @@ async def check_carp_auto_revert_failed(db: AsyncSession) -> dict[str, dict]:
"message": window.message,
}
return problems
# A failed scheduled run never turns "fixed" by itself, so it is reported as an event:
# open for this long after its planned time, then closed without a "resolved" message
# (see notification_service._NO_RESOLVED_NOTICE).
SCHEDULED_FAILURE_WINDOW = timedelta(days=3)
async def check_scheduled_maintenance_failed(db: AsyncSession) -> dict[str, dict]:
since = datetime.now(timezone.utc) - SCHEDULED_FAILURE_WINDOW
result = await db.execute(
select(ScheduledMaintenance).where(
ScheduledMaintenance.status == "failed", ScheduledMaintenance.run_at >= since
)
)
problems: dict[str, dict] = {}
for sched in result.scalars().all():
problems[str(sched.id)] = {
"schedule_id": str(sched.id),
"host_id": str(sched.host_id),
"host_name": sched.host_name,
"action": sched.action,
"run_at": sched.run_at.isoformat(),
"message": (sched.result or {}).get("error"),
}
return problems
async def check_backup_stale(db: AsyncSession) -> dict[str, dict]:
"""Scheduled backups are on but a host has no recent one: three missed intervals
(at least 6 hours) cover a reboot or a short outage without crying wolf."""
cfg = await db.get(BackupSettings, 1)
if cfg is not None and not cfg.scheduled_enabled:
return {}
interval = cfg.interval_hours if cfg is not None else 24
limit = timedelta(hours=max(3 * interval, 6))
now = datetime.now(timezone.utc)
problems: dict[str, dict] = {}
for host in await _enabled_hosts(db):
snapshots = backup_store.list_snapshots(host.id)
newest = snapshots[0]["created_at"] if snapshots else None
if (newest is None and now - host.created_at > limit) or (newest is not None and now - newest > limit):
problems[str(host.id)] = {
"host_id": str(host.id), "host_name": host.name,
"last_backup": newest.isoformat() if newest else "noch nie",
}
return problems
async def check_backup_offsite(db: AsyncSession) -> dict[str, dict]:
cfg = await db.get(BackupSettings, 1)
if cfg is None or not cfg.offsite_enabled or not cfg.last_offsite_error:
return {}
return {"offsite": {"message": cfg.last_offsite_error}}
async def check_compliance(db: AsyncSession) -> dict[str, dict]:
"""Critical and high findings of the last security check, one problem per (host, check); resolved when it passes."""
from app.services import compliance_service as cs
hosts = {h.id: h for h in await _enabled_hosts(db)}
problems: dict[str, dict] = {}
for row in (await db.scalars(select(ComplianceResult))).all():
host = hosts.get(row.host_id)
if host is None or row.error:
continue
for finding in row.findings:
if finding["passed"] is False and finding["severity"] in ("critical", "high"):
params = finding.get("params") or {}
problems[f"{host.id}:{finding['id']}"] = {
"host_id": str(host.id), "host_name": host.name, "check": finding["id"],
"befund": cs.TITLES_DE.get(finding["id"], finding["id"]),
"schweregrad": cs.SEVERITIES_DE[finding["severity"]],
"beispiele": ", ".join(params.get("examples", [])) or None,
}
return problems
@@ -33,6 +33,10 @@ _LABELS = {
"nat_drift": "NAT-Drift im Cluster",
"network_drift": "Netzwerk-Abweichung im Cluster",
"carp_auto_revert_failed": "CARP-Auto-Revert fehlgeschlagen",
"scheduled_maintenance_failed": "Geplante Wartung nicht ausgeführt",
"backup_stale": "Kein aktuelles Backup",
"backup_offsite_failed": "Offsite-Backup fehlgeschlagen",
"compliance_finding": "Sicherheitsbefund",
}
@@ -30,13 +30,24 @@ _CHECKS = [
("nat_drift", "nat_drift_enabled", notification_checks.check_nat_drift),
("network_drift", "network_drift_enabled", notification_checks.check_network_drift),
("carp_auto_revert_failed", "carp_auto_revert_failed_enabled", notification_checks.check_carp_auto_revert_failed),
("scheduled_maintenance_failed", "scheduled_maintenance_failed_enabled", notification_checks.check_scheduled_maintenance_failed),
("backup_stale", "backup_failed_enabled", notification_checks.check_backup_stale),
("backup_offsite_failed", "backup_failed_enabled", notification_checks.check_backup_offsite),
("compliance_finding", "compliance_alerts_enabled", notification_checks.check_compliance),
]
# Event-like problems: when they drop out of the window nothing was "fixed", so no
# "behoben" message goes out for them.
_NO_RESOLVED_NOTICE = {"scheduled_maintenance_failed"}
async def _deliver(settings_row: NotificationSettings, problem: NotificationProblem, transition: str) -> None:
"""Best-effort, not an audit trail: attempted once per transition and
never retried, even on failure — avoids retry-storming a broken webhook
URL or SMTP relay every poll cycle."""
if transition == "resolved" and problem.problem_type in _NO_RESOLVED_NOTICE:
problem.resolved_notified_at = _now()
return
if settings_row.webhook_enabled:
try:
await notification_delivery.send_webhook(settings_row, problem, transition)
@@ -25,7 +25,7 @@ async def update_settings(db: AsyncSession, payload: NotificationSettingsUpdate)
"smtp_from_address", "smtp_to_addresses",
"host_offline_enabled", "cert_expiring_enabled", "rule_drift_enabled",
"revert_failed_enabled", "maintenance_unknown_enabled", "carp_health_enabled", "nat_drift_enabled",
"network_drift_enabled", "carp_auto_revert_failed_enabled",
"network_drift_enabled", "carp_auto_revert_failed_enabled", "scheduled_maintenance_failed_enabled", "backup_failed_enabled", "compliance_alerts_enabled",
"poll_interval_seconds",
]
for field in simple_fields:
+288
View File
@@ -0,0 +1,288 @@
"""OpenID Connect login (authorization code flow with PKCE) and role mapping.
What is checked on the ID token, in this order of suspicion: the signature (against the provider's JWKS,
asymmetric algorithms only: ``none`` and HMAC are never accepted), ``iss`` equal to the configured issuer,
``aud`` containing our client id (and ``azp`` equal to it if there are several audiences), ``exp``/``nbf``,
and the ``nonce`` we sent. The role comes from the groups claim (ID token, else userinfo) at EVERY login, so
removing someone from the group takes the access away at their next sign-in. Local accounts are never linked
to or taken over by an identity: an OIDC user is keyed by ``<issuer>|<sub>`` and a name clash with a local
account gets a suffix. Without a mapped group nobody is let in.
"""
import base64
import hashlib
import logging
import re
import secrets
import time
from datetime import datetime, timedelta, timezone
from urllib.parse import urlencode
import httpx
import jwt
from jwt import PyJWK
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.oidc import OidcLoginState, OidcSettings
from app.models.user import AdminUser
from app.services import notification_secret_store
logger = logging.getLogger(__name__)
SINGLETON_ID = 1
SECRET_NAME = "oidc_client_secret"
STATE_TTL = timedelta(minutes=10)
ALGORITHMS = ["RS256", "RS384", "RS512", "PS256", "PS384", "PS512", "ES256", "ES384", "ES512"]
UNUSABLE_PASSWORD = "!oidc"
_ROLE_ORDER = ("admin", "operator", "read_only")
class OidcError(Exception):
"""Carries a short, user-safe code (shown on the login page), never provider internals."""
def __init__(self, code: str, detail: str = ""):
super().__init__(detail or code)
self.code = code
async def get_settings(db: AsyncSession) -> OidcSettings:
row = await db.get(OidcSettings, SINGLETON_ID)
if row is None:
row = OidcSettings(
id=SINGLETON_ID, enabled=False, display_name="SSO", client_secret_set=False,
scopes="openid profile email", groups_claim="groups",
)
db.add(row)
await db.flush()
return row
def configured(cfg: OidcSettings) -> bool:
return bool(cfg.enabled and cfg.issuer and cfg.client_id and (cfg.admin_group or cfg.operator_group or cfg.read_only_group))
# ── provider metadata (cached) ───────────────────────────────────────────
_discovery_cache: dict[str, tuple[float, dict]] = {}
_jwks_cache: dict[str, tuple[float, dict]] = {}
_DISCOVERY_TTL, _JWKS_TTL, _JWKS_MIN_REFETCH = 600, 3600, 60
def clear_caches() -> None:
_discovery_cache.clear()
_jwks_cache.clear()
async def _get_json(url: str, **kw) -> dict:
if not url.startswith("https://"):
raise OidcError("provider", "Nur https:// wird akzeptiert")
try:
async with httpx.AsyncClient(timeout=10, follow_redirects=False) as client:
response = await client.get(url, **kw)
response.raise_for_status()
data = response.json()
except (httpx.HTTPError, ValueError):
raise OidcError("provider", "Der Identitätsanbieter ist nicht erreichbar oder antwortet unverständlich") from None
if not isinstance(data, dict):
raise OidcError("provider", "Unerwartete Antwort des Identitätsanbieters")
return data
async def discovery(issuer: str) -> dict:
cached = _discovery_cache.get(issuer)
if cached and time.monotonic() - cached[0] < _DISCOVERY_TTL:
return cached[1]
doc = await _get_json(issuer.rstrip("/") + "/.well-known/openid-configuration")
if doc.get("issuer") != issuer and doc.get("issuer", "").rstrip("/") != issuer.rstrip("/"):
raise OidcError("provider", "Der Aussteller in der Discovery-Antwort passt nicht zur eingetragenen URL")
for key in ("authorization_endpoint", "token_endpoint", "jwks_uri"):
if not str(doc.get(key, "")).startswith("https://"):
raise OidcError("provider", f"Discovery enthält kein gültiges {key}")
_discovery_cache[issuer] = (time.monotonic(), doc)
return doc
async def _jwk_for(doc: dict, kid: str | None) -> PyJWK:
uri = doc["jwks_uri"]
for attempt in (0, 1):
cached = _jwks_cache.get(uri)
stale = cached is None or time.monotonic() - cached[0] > _JWKS_TTL
if stale or attempt == 1:
if attempt == 1 and cached and time.monotonic() - cached[0] < _JWKS_MIN_REFETCH:
break # unknown kid, but we just fetched: do not let a forged kid hammer the provider
_jwks_cache[uri] = (time.monotonic(), await _get_json(uri))
for key in _jwks_cache[uri][1].get("keys", []):
if (kid is None or key.get("kid") == kid) and key.get("use", "sig") == "sig":
try:
return PyJWK(key)
except Exception: # unsupported key type: try the next one
continue
raise OidcError("token", "Kein passender Signaturschlüssel")
# ── the flow ─────────────────────────────────────────────────────────────
def _hash(value: str) -> str:
return hashlib.sha256(value.encode()).hexdigest()
def _challenge(verifier: str) -> str:
return base64.urlsafe_b64encode(hashlib.sha256(verifier.encode()).digest()).rstrip(b"=").decode()
async def start(db: AsyncSession, cfg: OidcSettings, redirect_uri: str) -> tuple[str, str]:
"""Remember a fresh login attempt and return the provider URL to send the browser to."""
doc = await discovery(cfg.issuer)
state, nonce, verifier = secrets.token_urlsafe(32), secrets.token_urlsafe(24), secrets.token_urlsafe(48)
await db.execute(delete(OidcLoginState).where(OidcLoginState.created_at < datetime.now(timezone.utc) - STATE_TTL))
db.add(OidcLoginState(state_hash=_hash(state), nonce=nonce, code_verifier=verifier, redirect_uri=redirect_uri))
await db.commit()
query = urlencode({
"response_type": "code", "client_id": cfg.client_id, "redirect_uri": redirect_uri, "scope": cfg.scopes,
"state": state, "nonce": nonce, "code_challenge": _challenge(verifier), "code_challenge_method": "S256",
})
return f"{doc['authorization_endpoint']}{'&' if '?' in doc['authorization_endpoint'] else '?'}{query}", state
async def _consume_state(db: AsyncSession, state: str) -> OidcLoginState:
row = await db.scalar(select(OidcLoginState).where(OidcLoginState.state_hash == _hash(state)).with_for_update())
if row is None:
raise OidcError("state", "Unbekannter oder bereits benutzter Anmeldeversuch")
created = row.created_at
await db.delete(row) # single use, whatever happens next
await db.commit()
if datetime.now(timezone.utc) - created > STATE_TTL:
raise OidcError("state", "Der Anmeldeversuch ist abgelaufen")
return row
async def _exchange(cfg: OidcSettings, doc: dict, code: str, state: OidcLoginState) -> dict:
data = {"grant_type": "authorization_code", "code": code, "redirect_uri": state.redirect_uri,
"client_id": cfg.client_id, "code_verifier": state.code_verifier}
secret = notification_secret_store.read_secret(SECRET_NAME) if cfg.client_secret_set else None
auth = None
if secret:
methods = doc.get("token_endpoint_auth_methods_supported") or ["client_secret_basic"]
if "client_secret_basic" in methods:
auth = httpx.BasicAuth(cfg.client_id, secret)
else:
data["client_secret"] = secret
try:
async with httpx.AsyncClient(timeout=15, follow_redirects=False) as client:
response = await client.post(doc["token_endpoint"], data=data, auth=auth)
if response.status_code >= 400:
raise OidcError("token", f"Token-Endpunkt antwortet mit HTTP {response.status_code}")
tokens = response.json()
except httpx.HTTPError:
raise OidcError("provider", "Token-Endpunkt nicht erreichbar") from None
except ValueError:
raise OidcError("token", "Unverständliche Token-Antwort") from None
if not isinstance(tokens, dict) or not tokens.get("id_token"):
raise OidcError("token", "Die Antwort enthält kein ID-Token")
return tokens
async def _verify_id_token(cfg: OidcSettings, doc: dict, id_token: str, nonce: str) -> dict:
try:
header = jwt.get_unverified_header(id_token)
except jwt.PyJWTError:
raise OidcError("token", "ID-Token unlesbar") from None
if header.get("alg") not in ALGORITHMS:
raise OidcError("token", f"Nicht erlaubter Signaturalgorithmus {header.get('alg')!r}")
key = await _jwk_for(doc, header.get("kid"))
try:
claims = jwt.decode(
id_token, key.key, algorithms=ALGORITHMS, audience=cfg.client_id, issuer=cfg.issuer,
options={"require": ["exp", "iss", "aud", "sub"]}, leeway=60,
)
except jwt.PyJWTError as exc:
raise OidcError("token", f"ID-Token ungültig ({type(exc).__name__})") from None
audience = claims.get("aud")
if isinstance(audience, list) and len(audience) > 1 and claims.get("azp") != cfg.client_id:
raise OidcError("token", "azp passt nicht zur Client-ID")
if not secrets.compare_digest(str(claims.get("nonce", "")), nonce):
raise OidcError("token", "Nonce stimmt nicht")
return claims
def _as_groups(value) -> list[str]:
if isinstance(value, str):
# Group names may contain spaces ("Network Admins"): a string claim is split on commas and line breaks only.
return [g.strip() for g in re.split(r"[,\n]+", value) if g.strip()]
if isinstance(value, list):
return [str(g) for g in value]
return []
def role_for(cfg: OidcSettings, groups: list[str]) -> str | None:
"""The most privileged role whose group the user is in; None = not allowed in."""
wanted = {"admin": cfg.admin_group, "operator": cfg.operator_group, "read_only": cfg.read_only_group}
have = {g.lower() for g in groups}
for role in _ROLE_ORDER:
group = wanted[role]
if group and group.strip().lower() in have:
return role
return None
async def _userinfo(doc: dict, claims: dict, access_token: str | None) -> dict:
"""The provider's userinfo answer for THIS identity ({} if there is none or it belongs to somebody else)."""
if access_token and str(doc.get("userinfo_endpoint", "")).startswith("https://"):
info = await _get_json(doc["userinfo_endpoint"], headers={"Authorization": f"Bearer {access_token}"})
if info.get("sub") == claims.get("sub"): # a userinfo answer for someone else is worthless
return info
return {}
async def _groups(cfg: OidcSettings, doc: dict, claims: dict, access_token: str | None) -> list[str]:
if cfg.groups_claim in claims:
return _as_groups(claims[cfg.groups_claim])
return _as_groups((await _userinfo(doc, claims, access_token)).get(cfg.groups_claim))
async def _with_display_name(doc: dict, claims: dict, access_token: str | None) -> dict:
"""Some providers (Authelia) keep preferred_username/email out of the ID token: ask userinfo, else the user would be named
after the opaque ``sub``. Best effort: a failing userinfo call must not block a login that already has its role."""
if claims.get("preferred_username") or claims.get("email"):
return claims
try:
info = await _userinfo(doc, claims, access_token)
except OidcError:
return claims
return {**claims, **{k: info[k] for k in ("preferred_username", "email") if info.get(k)}}
def _username(claims: dict) -> str:
raw = claims.get("preferred_username") or claims.get("email") or claims["sub"]
return re.sub(r"[\x00-\x1f]", "", str(raw)).strip()[:140] or "sso-user"
async def provision(db: AsyncSession, cfg: OidcSettings, claims: dict, role: str) -> AdminUser:
"""The account for this identity: created on first login, role synced on every login."""
subject = f"{cfg.issuer}|{claims['sub']}"[:600]
user = await db.scalar(select(AdminUser).where(AdminUser.oidc_subject == subject))
if user is None:
base, name, n = _username(claims), None, 0
while True:
name = base if n == 0 else f"{base}~sso{n if n > 1 else ''}"
if await db.scalar(select(AdminUser.id).where(AdminUser.username == name)) is None:
break
n += 1
user = AdminUser(username=name, password_hash=UNUSABLE_PASSWORD, role=role, auth_source="oidc", oidc_subject=subject)
db.add(user)
else:
user.role = role
await db.commit()
return user
async def complete(db: AsyncSession, cfg: OidcSettings, code: str, state_value: str) -> AdminUser:
"""Callback: state -> tokens -> verified identity -> role -> account. Raises OidcError."""
state = await _consume_state(db, state_value)
doc = await discovery(cfg.issuer)
tokens = await _exchange(cfg, doc, code, state)
claims = await _verify_id_token(cfg, doc, tokens["id_token"], state.nonce)
role = role_for(cfg, await _groups(cfg, doc, claims, tokens.get("access_token")))
if role is None:
raise OidcError("no_role", "Kein zugeordnete Gruppe")
return await provision(db, cfg, await _with_display_name(doc, claims, tokens.get("access_token")), role)
@@ -9,7 +9,7 @@ from app.models.host import OpnsenseHost
from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -18,7 +18,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: OneToOneRule) -> dict:
def _build_payload(rule: OneToOneRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: OneToOneRule) -> dict:
return {
"description": rule.description,
"interface": rule.interface,
@@ -35,7 +40,7 @@ def _build_payload(rule: OneToOneRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: OneToOneRule, host: OpnsenseHost) -> OneToOneDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(OneToOneDeployment).where(
OneToOneDeployment.one_to_one_id == rule.id, OneToOneDeployment.host_id == host.id
@@ -78,6 +83,7 @@ async def deploy_one_to_one(db: AsyncSession, rule: OneToOneRule, hosts: list[Op
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
@@ -8,6 +8,7 @@ import json
from app.models.policy import OneToOneRevision
from app.services.one_to_one_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
ONE_TO_ONE_FIELDS = ('description', 'interface', 'source_net', 'destination_net', 'source_not', 'destination_not', 'enabled', 'type', 'external', 'natreflection', 'log')
@@ -27,13 +28,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_one_to_one_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -60,6 +65,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -70,7 +77,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -78,12 +85,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+23 -1
View File
@@ -16,7 +16,7 @@ import httpx
from app.config import settings
from app.models.host import OpnsenseHost
from app.services import firewall_url, opnsense_secret_store, tls_pin_service
from app.services import firewall_url, opnsense_secret_store, overlay_service, tls_pin_service
from app.services.redaction import redact_data, redact_text
logger = logging.getLogger(__name__)
@@ -46,10 +46,23 @@ def _client(host: OpnsenseHost) -> httpx.AsyncClient:
auth=(host.api_key, secret),
verify=tls_pin_service.verify_argument(host.verify_tls, host.tls_ca_pem),
timeout=settings.opnsense_timeout,
# Firewalls reached through the management overlay (WireGuard hub) go via the hub's CONNECT proxy.
proxy=overlay_service.proxy_for(host.base_url),
)
def _refuse_unresolved_placeholders(method: str, path: str, kwargs: dict) -> None:
"""Last line of defence for site variables: a payload that still contains a {{NAME}} placeholder was never
resolved for this host and must not reach a firewall."""
if method.upper() == "GET" or "json" not in kwargs:
return
from app.services.variables_service import PLACEHOLDER
if PLACEHOLDER.search(json.dumps(kwargs["json"], default=str)):
raise OpnsenseAPIError(f"{path}: Die Anfrage enthält eine nicht aufgelöste Variable ({{{{NAME}}}}) und wird nicht gesendet")
async def _do_request(host: OpnsenseHost, method: str, path: str, **kwargs) -> httpx.Response:
_refuse_unresolved_placeholders(method, path, kwargs)
async with _client(host) as client:
try:
resp = await client.request(method, path, **kwargs)
@@ -369,6 +382,15 @@ async def get_source_nat_settings(host: OpnsenseHost) -> dict:
# Core controllers, checked against OPNsense core source:
# https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/controllers/OPNsense/Core/Api/FirmwareController.php
# https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/controllers/OPNsense/Core/Api/SystemController.php
async def get_pf_rule_statistics(host: OpnsenseHost) -> dict:
"""`diagnostics/firewall/pfStatistics/rules` (Diagnostics Api FirewallController):
{"rules": {"filter rules": {"@N <pf rule text> label \"<uuid>\"": {evaluations, packets,
bytes, states, ...}}}}. Verified live on OPNsense 26.7: the pf label of a rule created in
the rule editor/API is its UUID, so counters can be tied to rules; automatic rules carry
an md5 label instead."""
return await _request(host, "GET", "/diagnostics/firewall/pfStatistics/rules")
async def get_firmware_status(host: OpnsenseHost) -> dict:
# GET reads cached results. POST would trigger a synchronous probe.
return await _request(host, "GET", "/core/firmware/status")
@@ -10,7 +10,7 @@ from app.models.outbound_nat import OutboundNatRule
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.services.outbound_nat_mode import require_manual_mode
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -19,7 +19,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: OutboundNatRule) -> dict:
def _build_payload(rule: OutboundNatRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: OutboundNatRule) -> dict:
return {
"description": rule.description,
"interface": rule.interface,
@@ -39,7 +44,7 @@ def _build_payload(rule: OutboundNatRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: OutboundNatRule, host: OpnsenseHost) -> OutboundNatDeployment:
await require_manual_mode(host)
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(OutboundNatDeployment).where(
OutboundNatDeployment.outbound_nat_id == rule.id, OutboundNatDeployment.host_id == host.id
@@ -82,6 +87,7 @@ async def deploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule, hosts: li
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = []
try:
for host in hosts:
@@ -8,6 +8,7 @@ import json
from app.models.policy import OutboundNatRevision
from app.services.outbound_nat_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
OUTBOUND_NAT_FIELDS = ('description', 'interface', 'source_net', 'destination_net', 'source_not', 'destination_not', 'enabled', 'ipprotocol', 'protocol', 'source_port', 'destination_port', 'target', 'target_port')
@@ -27,13 +28,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_outbound_nat_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -60,6 +65,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -70,7 +77,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -78,12 +85,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+208
View File
@@ -0,0 +1,208 @@
"""Management overlay: keys, addressing, the hub's config file and the live status.
How it fits together
--------------------
* The ``wg-hub`` container (app/overlay_hub.py, same image as the backend, NET_ADMIN, UDP port published)
owns the WireGuard interface. It reads ``<overlay_dir>/hub.json`` — written here whenever settings or peers
change — and writes ``<overlay_dir>/status.json`` (handshakes, traffic) every few seconds.
* The backend itself needs no extra privileges: requests to a firewall whose address lies in the overlay subnet
go through the hub's small CONNECT proxy (``proxy_for``), which only lets traffic into that subnet.
* The firewall dials out to the hub. Its private key never leaves it; OPNexus only learns its public key. A
per-peer pre-shared key is generated here and shown to an admin for entering on the firewall.
"""
import base64
import ipaddress
import json
import os
import re
import secrets
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlsplit
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.models.overlay import OverlayPeer, OverlaySettings
from app.services import secure_files
SINGLETON_ID = 1
KEEPALIVE = 25
CONNECTED_WITHIN = 180 # seconds since the last handshake
_B64_KEY = re.compile(r"^[A-Za-z0-9+/]{43}=$")
_active_network: ipaddress.IPv4Network | None = None
class OverlayError(Exception):
"""A user-facing problem; ``code`` is stable, ``detail`` is for the log."""
def __init__(self, code: str, detail: str = ""):
super().__init__(detail or code)
self.code = code
def _dir() -> Path:
return Path(settings.overlay_dir)
# ── keys ─────────────────────────────────────────────────────────────────
def valid_public_key(value: str) -> bool:
if not _B64_KEY.match(value or ""):
return False
try:
return len(base64.b64decode(value, validate=True)) == 32
except ValueError:
return False
def _raw_b64(data: bytes) -> str:
return base64.b64encode(data).decode()
def generate_private_key() -> str:
key = X25519PrivateKey.generate()
return _raw_b64(key.private_bytes(serialization.Encoding.Raw, serialization.PrivateFormat.Raw, serialization.NoEncryption()))
def public_from_private(private_b64: str) -> str:
key = X25519PrivateKey.from_private_bytes(base64.b64decode(private_b64))
return _raw_b64(key.public_key().public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw))
def hub_private_key() -> str:
"""The hub's private key, created on first use."""
path = _dir() / "hub.key"
if not path.exists():
secure_files.write_private(path, generate_private_key())
return path.read_text().strip()
def hub_public_key() -> str:
return public_from_private(hub_private_key())
def peer_psk(peer_id) -> str:
path = _dir() / f"psk-{peer_id}"
if not path.exists():
secure_files.write_private(path, _raw_b64(secrets.token_bytes(32)))
return path.read_text().strip()
def delete_peer_secret(peer_id) -> None:
(_dir() / f"psk-{peer_id}").unlink(missing_ok=True)
# ── addressing ───────────────────────────────────────────────────────────
def parse_subnet(value: str) -> ipaddress.IPv4Network:
try:
net = ipaddress.ip_network(value.strip(), strict=True)
except ValueError:
raise OverlayError("subnet", "kein gültiges Netz") from None
if (not isinstance(net, ipaddress.IPv4Network) or not net.is_private or not (16 <= net.prefixlen <= 29)
or net.is_loopback or net.is_link_local or net.is_multicast or net.is_unspecified):
raise OverlayError("subnet", "ein privates IPv4-Netz zwischen /16 und /29 ist nötig")
return net
def hub_address(net: ipaddress.IPv4Network) -> ipaddress.IPv4Address:
return net.network_address + 1
def next_free_ip(net: ipaddress.IPv4Network, taken: set[str]) -> str:
hub = hub_address(net)
for candidate in net.hosts():
if candidate != hub and str(candidate) not in taken:
return str(candidate)
raise OverlayError("full", "Im Overlay-Netz ist keine Adresse mehr frei")
def proxy_for(base_url: str) -> str | None:
"""The hub's proxy if this firewall address is inside the overlay subnet, else None (direct)."""
if _active_network is None:
return None
try:
host = urlsplit(base_url).hostname
return settings.overlay_proxy_url if host and ipaddress.ip_address(host) in _active_network else None
except ValueError:
return None
# ── settings, hub config ─────────────────────────────────────────────────
async def get_settings(db: AsyncSession) -> OverlaySettings:
row = await db.get(OverlaySettings, SINGLETON_ID)
if row is None:
row = OverlaySettings(id=SINGLETON_ID, enabled=False, subnet="10.99.0.0/24", listen_port=51820)
db.add(row)
await db.flush()
return row
def endpoint_with_port(cfg: OverlaySettings) -> str | None:
if not cfg.endpoint:
return None
return cfg.endpoint if re.search(r":\d+$", cfg.endpoint) else f"{cfg.endpoint}:{cfg.listen_port}"
async def sync(db: AsyncSession) -> None:
"""Make hub.json match the database (call after every change and at startup) and refresh the proxy rule."""
global _active_network
cfg = await get_settings(db)
net = parse_subnet(cfg.subnet)
_active_network = net if cfg.enabled else None
peers = (await db.scalars(select(OverlayPeer).order_by(OverlayPeer.ip))).all()
config = {
"enabled": bool(cfg.enabled), "address": f"{hub_address(net)}/{net.prefixlen}", "subnet": str(net),
"listen_port": cfg.listen_port,
"private_key": hub_private_key() if cfg.enabled else None,
"peers": [{"public_key": p.public_key, "psk": peer_psk(p.id), "ip": p.ip} for p in peers if p.public_key] if cfg.enabled else [],
}
_dir().mkdir(parents=True, exist_ok=True, mode=0o700)
tmp = _dir() / "hub.json.tmp"
secure_files.write_private(tmp, json.dumps(config))
os.replace(tmp, _dir() / "hub.json")
# ── live status (written by the hub) ─────────────────────────────────────
def read_status() -> dict | None:
try:
data = json.loads((_dir() / "status.json").read_text())
except (OSError, ValueError):
return None
return data if isinstance(data, dict) else None
def hub_state(status: dict | None, now: float | None = None) -> str:
"""'running' | 'stale' (the hub stopped reporting) | 'absent' (never reported: container not running)"""
if not status:
return "absent"
now = now or datetime.now(timezone.utc).timestamp()
return "running" if now - float(status.get("updated_at", 0)) < 30 else "stale"
def peer_status(status: dict | None, public_key: str | None, now: float | None = None) -> dict:
now = now or datetime.now(timezone.utc).timestamp()
entry = next((p for p in (status or {}).get("peers", []) if p.get("public_key") == public_key), None) if public_key else None
handshake = int(entry.get("handshake_at", 0)) if entry else 0
return {
"connected": bool(handshake and now - handshake < CONNECTED_WITHIN and hub_state(status, now) == "running"),
"last_handshake_at": datetime.fromtimestamp(handshake, timezone.utc) if handshake else None,
"rx_bytes": int(entry.get("rx", 0)) if entry else 0, "tx_bytes": int(entry.get("tx", 0)) if entry else 0,
"remote_endpoint": entry.get("endpoint") if entry else None,
}
def instructions(cfg: OverlaySettings, peer: OverlayPeer) -> dict:
net = parse_subnet(cfg.subnet)
return {
"hub_public_key": hub_public_key(), "endpoint": endpoint_with_port(cfg),
"tunnel_address": f"{peer.ip}/32", "allowed_ips": f"{hub_address(net)}/32", "psk": peer_psk(peer.id),
"keepalive": KEEPALIVE, "hub_ip": str(hub_address(net)), "host_url": f"https://{peer.ip}",
}
+10 -4
View File
@@ -6,6 +6,7 @@ from sqlalchemy import text
from app.models.policy import RuleRevision
from app.services.rule_deploy_service import _build_payload
from app.services import variables_service
RULE_FIELDS = ("description", "interface", "direction", "ipprotocol", "protocol", "action", "source_net", "source_port", "destination_net", "destination_port", "enabled", "log")
@@ -45,13 +46,17 @@ def ensure_editable(rule):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_rule_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -78,6 +83,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten ihre Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -89,7 +96,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -98,12 +105,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
@@ -21,7 +21,7 @@ from app.models.host import OpnsenseHost
from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -30,7 +30,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: PortForwardRule) -> dict:
def _build_payload(rule: PortForwardRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: PortForwardRule) -> dict:
return {
"disabled": "0" if rule.enabled else "1",
"interface": rule.interface,
@@ -47,7 +52,7 @@ def _build_payload(rule: PortForwardRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: PortForwardRule, host: OpnsenseHost) -> PortForwardDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(PortForwardDeployment).where(
PortForwardDeployment.port_forward_id == rule.id, PortForwardDeployment.host_id == host.id
@@ -90,6 +95,7 @@ async def deploy_port_forward(db: AsyncSession, rule: PortForwardRule, hosts: li
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
@@ -8,6 +8,7 @@ import json
from app.models.policy import PortForwardRevision
from app.services.port_forward_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
PORT_FORWARD_FIELDS = (
@@ -30,13 +31,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_port_forward_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -63,6 +68,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -73,7 +80,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -81,12 +88,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+20
View File
@@ -0,0 +1,20 @@
"""HTTP CONNECT through the overlay hub's proxy, for the few places that open a raw TLS connection."""
import asyncio
from urllib.parse import urlsplit
async def open_tunnel(proxy_url: str, host: str, port: int, timeout: float = 8.0):
"""(reader, writer) connected to host:port through the proxy; TLS can be started on top with ``writer.start_tls``."""
proxy = urlsplit(proxy_url)
reader, writer = await asyncio.wait_for(asyncio.open_connection(proxy.hostname, proxy.port or 8888), timeout)
try:
writer.write(f"CONNECT {host}:{port} HTTP/1.1\r\nHost: {host}:{port}\r\n\r\n".encode())
await writer.drain()
head = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), timeout)
status = int(head.split(b" ", 2)[1])
if status != 200:
raise OSError(f"Der Overlay-Proxy lehnt die Verbindung ab (HTTP {status})")
except Exception:
writer.close()
raise
return reader, writer
+9 -3
View File
@@ -27,7 +27,7 @@ from app.models.host import OpnsenseHost
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -36,7 +36,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: FirewallRule) -> dict:
def _build_payload(rule: FirewallRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: FirewallRule) -> dict:
return {
"enabled": "1" if rule.enabled else "0",
"action": rule.action,
@@ -55,7 +60,7 @@ def _build_payload(rule: FirewallRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: FirewallRule, host: OpnsenseHost) -> RuleDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(RuleDeployment).where(RuleDeployment.rule_id == rule.id, RuleDeployment.host_id == host.id)
)
@@ -98,6 +103,7 @@ async def deploy_rule(db: AsyncSession, rule: FirewallRule, hosts: list[Opnsense
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
+116
View File
@@ -0,0 +1,116 @@
"""Rule usage from pf's counters: which rules never match anything?
``sample_host`` reads the per-rule packet counters (``diagnostics/firewall/pfStatistics/rules``) and the
configured rules (``searchRule``) and updates one row per rule: the time of the last observed hit.
* A counter that grew since the last sample is a hit "now" (at most one sampling interval late).
* A counter that is *lower* than before means pf reloaded its ruleset (an apply or a reboot), which
zeroes everything. Whatever it shows above zero was hit since the reload, so that also counts as a hit now;
zero tells nothing, the old ``last_hit_at`` stays.
* A rule is called unused only after OPNexus has watched it for ``UNUSED_AFTER_DAYS`` without a hit.
A new rule, or a brand-new installation, is "still collecting data" instead of a false alarm.
Only rules with a UUID label are tied to a configured rule (rules created in the rule editor or API; OPNsense's
built-in automatic rules carry hash labels and are not user rules).
"""
import logging
import re
from datetime import datetime, timedelta, timezone
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.host import OpnsenseHost
from app.models.rule_hit import RuleHit
from app.services import opnsense_client as client
from app.services.opnsense_client import OpnsenseAPIError
logger = logging.getLogger(__name__)
UNUSED_AFTER_DAYS = 30
_LABEL = re.compile(r'label "([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})"')
def parse_counters(stats: dict) -> dict[str, dict]:
"""{rule uuid: {packets, bytes, evaluations}} summed over every pf line a rule expands to."""
lines = ((stats or {}).get("rules") or {}).get("filter rules") or {}
result: dict[str, dict] = {}
for text, values in lines.items():
match = _LABEL.search(text)
if not match or not isinstance(values, dict):
continue
entry = result.setdefault(match.group(1).lower(), {"packets": 0, "bytes": 0, "evaluations": 0})
for key in entry:
entry[key] += int(values.get(key) or 0)
return result
def configured_rules(search: dict) -> dict[str, dict]:
rows = (search or {}).get("rows") or []
return {
str(r["uuid"]).lower(): {"description": r.get("description") or "", "enabled": r.get("enabled") in ("1", 1, True)}
for r in rows if r.get("uuid") and not r.get("is_automatic")
}
async def sample_host(db: AsyncSession, host: OpnsenseHost, now: datetime | None = None) -> int:
now = now or datetime.now(timezone.utc)
counters = parse_counters(await client.get_pf_rule_statistics(host))
rules = configured_rules(await client.search_rules(host, all_rows=True))
rows = {r.rule_uuid: r for r in (await db.scalars(select(RuleHit).where(RuleHit.host_id == host.id))).all()}
for uuid_, meta in rules.items():
counter = counters.get(uuid_, {"packets": 0, "bytes": 0})
row = rows.get(uuid_)
if row is None:
row = RuleHit(host_id=host.id, rule_uuid=uuid_, first_seen_at=now, last_packets=0, last_bytes=0, last_sampled_at=now)
db.add(row)
if counter["packets"] > 0:
row.last_hit_at = now # already matching when first seen
elif counter["packets"] > row.last_packets or (counter["packets"] < row.last_packets and counter["packets"] > 0):
row.last_hit_at = now
row.description = meta["description"][:255]
row.enabled = meta["enabled"]
row.last_packets, row.last_bytes, row.last_sampled_at = counter["packets"], counter["bytes"], now
for uuid_, row in rows.items(): # rule deleted on the firewall
if uuid_ not in rules:
await db.delete(row)
await db.commit()
return len(rules)
async def sample_all(db: AsyncSession) -> int:
hosts = (await db.scalars(select(OpnsenseHost).where(OpnsenseHost.enabled.is_(True)))).all()
sampled = 0
for host in hosts:
try:
await sample_host(db, host)
sampled += 1
except OpnsenseAPIError as exc:
logger.warning("Regelnutzung von %s nicht lesbar: %s", host.name, exc)
except Exception:
logger.exception("Regelnutzung von %s fehlgeschlagen", host.name)
return sampled
def classify(row: RuleHit, now: datetime) -> str:
"""'used' | 'unused' | 'collecting' | 'disabled'"""
if not row.enabled:
return "disabled"
if row.last_hit_at is not None and now - row.last_hit_at < timedelta(days=UNUSED_AFTER_DAYS):
return "used"
watched = now - row.first_seen_at
if watched < timedelta(days=UNUSED_AFTER_DAYS):
return "used" if row.last_hit_at is not None else "collecting"
return "unused"
async def unused_summary(db: AsyncSession, host_id) -> dict | None:
"""For the security check: None while nothing is known yet, else {"ready": bool, "rules": [names]}."""
rows = (await db.scalars(select(RuleHit).where(RuleHit.host_id == host_id))).all()
if not rows:
return None
now = datetime.now(timezone.utc)
states = [(r, classify(r, now)) for r in rows]
unused = [r.description or r.rule_uuid for r, s in states if s == "unused"]
watching = [s for _, s in states if s == "collecting"]
return {"unused": unused, "collecting": len(watching)}
+114
View File
@@ -0,0 +1,114 @@
"""Minimal S3-compatible object client (AWS Signature V4) for offsite backups.
Just what is needed — PUT, GET, DELETE and a connection test — so no AWS SDK is pulled
into the image. Works with AWS S3, MinIO, Backblaze B2, Wasabi, Cloudflare R2, Hetzner
Object Storage... Path-style addressing (``https://endpoint/bucket/key``) is the default
because it works everywhere; virtual-hosted style is available for providers that need it.
"""
import hashlib
import hmac
from datetime import datetime, timezone
from urllib.parse import quote, urlsplit
import httpx
from app.config import settings
class S3Error(Exception):
pass
def _sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _hmac(key: bytes, message: str) -> bytes:
return hmac.new(key, message.encode(), hashlib.sha256).digest()
def _quote(path: str) -> str:
return quote(path, safe="/-_.~")
def sign(*, method: str, host: str, path: str, query: str, payload_hash: str, access_key: str, secret_key: str,
region: str, now: datetime, extra_headers: dict[str, str] | None = None) -> dict[str, str]:
"""SigV4 headers for one request (the algorithm of the AWS docs, service ``s3``)."""
amz_date = now.strftime("%Y%m%dT%H%M%SZ")
day = now.strftime("%Y%m%d")
headers = {"host": host, "x-amz-content-sha256": payload_hash, "x-amz-date": amz_date}
headers.update({k.lower(): v.strip() for k, v in (extra_headers or {}).items()})
names = sorted(headers)
canonical_headers = "".join(f"{n}:{headers[n]}\n" for n in names)
signed = ";".join(names)
canonical_request = "\n".join([method, _quote(path), query, canonical_headers, signed, payload_hash])
scope = f"{day}/{region}/s3/aws4_request"
to_sign = "\n".join(["AWS4-HMAC-SHA256", amz_date, scope, _sha256(canonical_request.encode())])
key = _hmac(_hmac(_hmac(_hmac(("AWS4" + secret_key).encode(), day), region), "s3"), "aws4_request")
signature = hmac.new(key, to_sign.encode(), hashlib.sha256).hexdigest()
headers["authorization"] = f"AWS4-HMAC-SHA256 Credential={access_key}/{scope}, SignedHeaders={signed}, Signature={signature}"
return headers
class S3Client:
def __init__(self, *, endpoint: str, region: str, bucket: str, access_key: str, secret_key: str, path_style: bool = True):
parts = urlsplit(endpoint)
if parts.scheme not in ("http", "https") or not parts.hostname:
raise S3Error("Ungültiger Endpunkt")
if parts.scheme == "http" and not settings.allow_insecure_offsite:
# Credentials and (encrypted, but still) backups must not travel in the clear.
raise S3Error("Der Endpunkt muss https:// sein")
# httpx leaves a scheme's default port out of the Host header, so the signed host must not carry it either.
port = parts.port
default = {"https": 443, "http": 80}[parts.scheme]
host = parts.hostname if ":" not in parts.hostname else f"[{parts.hostname}]"
self.scheme = parts.scheme
self.netloc = host if port in (None, default) else f"{host}:{port}"
self.region, self.bucket, self.path_style = region or "us-east-1", bucket, path_style
self.access_key, self.secret_key = access_key, secret_key
def _target(self, key: str) -> tuple[str, str, str]:
"""(host header, URL path, URL)"""
if self.path_style:
host, path = self.netloc, f"/{self.bucket}/{key}"
else:
host, path = f"{self.bucket}.{self.netloc}", f"/{key}"
return host, path, f"{self.scheme}://{host}{_quote(path)}"
async def _request(self, method: str, key: str, body: bytes = b"") -> httpx.Response:
host, path, url = self._target(key)
headers = sign(
method=method, host=host, path=path, query="", payload_hash=_sha256(body), access_key=self.access_key,
secret_key=self.secret_key, region=self.region, now=datetime.now(timezone.utc),
)
try:
async with httpx.AsyncClient(timeout=60) as client:
return await client.request(method, url, content=body if method == "PUT" else None, headers=headers)
except httpx.HTTPError as exc:
raise S3Error(f"Verbindung fehlgeschlagen ({type(exc).__name__})") from None
@staticmethod
def _check(response: httpx.Response, what: str) -> None:
if response.status_code >= 300:
raise S3Error(f"{what}: HTTP {response.status_code}")
async def put(self, key: str, data: bytes) -> None:
self._check(await self._request("PUT", key, data), "Hochladen")
async def get(self, key: str) -> bytes:
response = await self._request("GET", key)
self._check(response, "Herunterladen")
return response.content
async def delete(self, key: str) -> None:
response = await self._request("DELETE", key)
if response.status_code not in (200, 204, 404):
self._check(response, "Löschen")
async def test(self, key: str) -> None:
"""Write, read back and delete a tiny object: proves credentials, bucket and permissions."""
probe = b"opnexus connection test"
await self.put(key, probe)
if await self.get(key) != probe:
raise S3Error("Der Speicher lieferte etwas anderes zurück")
await self.delete(key)
@@ -6,6 +6,9 @@ action dispatch table) rather than duplicating that logic differently,
since this is the exact same remote action, just fired by a timer instead
of a click.
Updates take a fresh config backup right before firing (a plan for the day after tomorrow
would otherwise always trip over the 24h freshness rule).
v1 scope decision: a blocked or failed scheduled fire is logged and
recorded on the ScheduledMaintenance row's own `result` field (visible in
the UI) but is NOT wired into the NotificationSettings webhook/email
@@ -15,7 +18,7 @@ for this is a bigger change than this feature's own scope. Flagged here,
not silently dropped, in case it's wanted later.
"""
import logging
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from fastapi import HTTPException
from sqlalchemy import select
@@ -24,12 +27,30 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.services import audit_service, opnsense_client as client
from app.services.host_maintenance_service import ensure_no_maintenance, inspect_host
from app.services import audit_service, backup_store, opnsense_client as client
from app.services.host_maintenance_service import ACTIVE, ensure_no_maintenance, inspect_carp_switch, inspect_host, peers
from app.services.host_status_service import get_statuses
logger = logging.getLogger(__name__)
_ACTION_FN = {"reboot": client.reboot_host, "update": client.update_firmware, "major_update": client.upgrade_firmware}
# Actions that need a config backup no older than 24h (see host_maintenance_service.inspect_host).
# Unattended, nobody can take it by hand beforehand, so it is taken right before firing.
_BACKUP_BEFORE = {"update", "major_update"}
# ── CARP handover run (opt-in per plan) ─────────────────────────────────────
# MASTER -> maintenance mode -> partner must become MASTER -> action -> wait until the node is
# back -> leave maintenance mode. One step per poll tick, state in the schedule row.
# Rules that keep it safe:
# * the CARP toggle is a toggle: it is only ever sent against freshly read state and never
# replayed after an unclear answer (see routers/maintenance.py carp_execute);
# * until the action was submitted, any problem restores normal operation (nothing happened yet);
# * after that, any problem leaves the node demoted — the partner keeps carrying traffic, which is
# the safe side — and the run ends as failed (=> notification) with an "unknown" operation.
HANDOVER_TIMEOUT = timedelta(minutes=3)
LEAVE_TIMEOUT = timedelta(minutes=5)
JOB_GRACE = timedelta(minutes=5) # no sign of a job after this long: treat a package update as already done
JOB_TIMEOUT = {"reboot": timedelta(minutes=20), "update": timedelta(minutes=60), "major_update": timedelta(minutes=150)}
def _now() -> datetime:
@@ -79,6 +100,16 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
await db.commit()
logger.warning("Geplante Wartung %s blockiert: %s", sched.id, exc.detail)
return
if sched.carp_handover and host.cluster_name:
try:
statuses = await get_statuses(await peers(db, host))
except client.OpnsenseAPIError as exc:
statuses = {}
logger.warning("Geplante Wartung %s: Rolle nicht lesbar: %s", sched.id, exc)
if statuses.get(host.id, {}).get("carp_status") == "MASTER":
await _start_handover(db, sched, host)
return
# Already BACKUP (or role unclear): no handover needed, the normal path below decides.
try:
state = await inspect_host(db, host)
except client.OpnsenseAPIError as exc:
@@ -87,6 +118,18 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
await db.commit()
logger.warning("Geplante Wartung %s: Host nicht erreichbar", sched.id)
return
if sched.action in _BACKUP_BEFORE and not state["blocked"]:
# Read-only on the firewall. Fail closed: no backup, no update. Taken only after the
# other blockers passed, so a plan that is going to fail anyway does not pile up backups.
try:
backup_store.save_snapshot(host.id, await client.download_backup(host))
state = await inspect_host(db, host)
except client.OpnsenseAPIError as exc:
sched.status = "failed"
sched.result = {"error": f"Automatisches Backup vor dem Update fehlgeschlagen: {str(exc)[:300]}"}
await db.commit()
logger.warning("Geplante Wartung %s: Backup vor dem Update fehlgeschlagen", sched.id)
return
if state["actions"][sched.action]:
sched.status = "failed"
sched.result = {"error": " ".join(state["actions"][sched.action])}
@@ -125,3 +168,248 @@ async def _fire_one(db: AsyncSession, sched: ScheduledMaintenance) -> None:
host_id=str(host.id), host_name=host.name, entity_id=str(op.id),
message=op.message, details={"scheduled": True, "schedule_id": str(sched.id), "response": result},
)
# ── handover run ─────────────────────────────────────────────────────────────
def _progress(sched: ScheduledMaintenance, **changes) -> dict:
"""JSON columns only notice reassignment, so always hand back a new dict."""
sched.progress = {**(sched.progress or {}), **changes}
return sched.progress
async def _audit(host, sched, op, action, status, message=None, **details):
await audit_service.persist(
kind="host", action=action, status=status, host_id=str(host.id), host_name=host.name,
entity_id=str(op.id) if op else None, message=message,
details={"scheduled": True, "schedule_id": str(sched.id), "handover": True, **details},
)
async def _end(db, sched, host, op, *, ok, message, op_status=None, **result):
sched.status = "succeeded" if ok else "failed"
sched.phase = None
sched.result = {"handover": True, **result} if ok else {"error": message[:900], **result}
if op is not None:
op.status = op_status or ("acknowledged" if ok else "unknown")
op.message = message[:1000]
await db.commit()
await _audit(host, sched, op, sched.action, "completed" if ok else (op_status or "unknown"), message)
if not ok:
logger.warning("Geplante Wartung %s (CARP-Übergabe) beendet: %s", sched.id, message)
async def _toggle(host, expected: str) -> None:
"""One CARP maintenance toggle; raises OpnsenseAPIError for an unclear or wrong-direction answer."""
result = await client.toggle_carp_maintenance(host)
if result.get("action") != expected:
raise client.OpnsenseAPIError("CARP-Antwort passt nicht zur angeforderten Richtung")
async def _start_handover(db: AsyncSession, sched: ScheduledMaintenance, host: OpnsenseHost) -> None:
"""The node is MASTER now. All preconditions are checked while nothing has been changed yet."""
def fail(message: str):
sched.status = "failed"
sched.result = {"error": message[:900]}
return db.commit()
try:
carp = await inspect_carp_switch(db, host)
state = await inspect_host(db, host, skip_role_checks=True)
blocked = list(dict.fromkeys(carp["blocked"] + state["blocked"]))
if carp["action"] != "carp_enter": # MASTER but already in maintenance mode: unclear, do not toggle blindly
blocked.append(f"{host.name} steht bereits im CARP-Wartungsmodus. Rollen prüfen.")
if not blocked and sched.action in _BACKUP_BEFORE:
backup_store.save_snapshot(host.id, await client.download_backup(host))
state = await inspect_host(db, host, skip_role_checks=True)
blocked = list(dict.fromkeys(blocked + state["actions"][sched.action]))
except client.OpnsenseAPIError as exc:
await fail(f"Vorbereitung der CARP-Übergabe fehlgeschlagen, nichts wurde verändert: {str(exc)[:300]}")
return
if blocked:
await fail(" ".join(blocked))
return
now = _now()
op = HostOperation(
host_id=host.id, host_name=host.name, actor=sched.actor, action=sched.action,
expires_at=now, fingerprint="", snapshot=state, status="starting", started_at=now,
)
db.add(op)
sched.status = "running"
await db.commit()
sched.host_operation_id = op.id
await _audit(host, sched, op, "carp_enter", "requested", snapshot=state)
try:
await _toggle(host, "enter_maintenance")
except client.OpnsenseAPIError as exc:
await _end(db, sched, host, op, ok=False,
message=f"CARP-Umschalten mit unklarem Ergebnis: {str(exc)[:300]}. Nicht automatisch wiederholt — "
f"Rollen von {host.name} und Partner prüfen.")
return
sched.phase, sched.phase_since = "handover_wait", _now()
_progress(sched, version_before=state.get("version"))
op.message = "CARP-Übergabe angefordert, warte auf Bestätigung."
await db.commit()
await _audit(host, sched, op, "carp_enter", "submitted")
async def advance_running(db: AsyncSession) -> int:
"""One step for every handover run in progress. Returns how many are still running."""
rows = list((await db.scalars(
select(ScheduledMaintenance).where(ScheduledMaintenance.status == "running", ScheduledMaintenance.phase.is_not(None))
)).all())
for sched in rows:
host = await db.get(OpnsenseHost, sched.host_id)
op = await db.get(HostOperation, sched.host_operation_id) if sched.host_operation_id else None
try:
if host is None:
raise RuntimeError("Host nicht mehr vorhanden")
if op is not None and op.status not in ACTIVE:
# A human closed or cancelled the operation: stop here and touch nothing more.
sched.status, sched.phase = "failed", None
sched.result = {"error": f"Vorgang von Hand beendet, Ablauf gestoppt. CARP-Rolle von {host.name} prüfen."}
await db.commit()
continue
await _STEPS[sched.phase](db, sched, host, op)
except Exception as exc: # a stuck run must not loop forever or take the poll loop down
logger.exception("Geplante Wartung %s: Schritt %s fehlgeschlagen", sched.id, sched.phase)
if host is not None:
await _end(db, sched, host, op, ok=False,
message=f"Unerwarteter Fehler während der CARP-Übergabe ({type(exc).__name__}). "
f"{host.name} kann im CARP-Wartungsmodus verblieben sein — Rollen prüfen.")
return sum(1 for s in rows if s.status == "running")
def _since(sched: ScheduledMaintenance) -> timedelta:
return _now() - (sched.phase_since or _now())
async def _roles(db, host) -> tuple[dict, bool]:
"""(own status, a partner is online and MASTER)."""
fleet = await peers(db, host)
statuses = await get_statuses(fleet)
partner_master = any(
statuses.get(h.id, {}).get("status") == "online" and statuses.get(h.id, {}).get("carp_status") == "MASTER"
for h in fleet if h.id != host.id
)
return statuses.get(host.id, {}), partner_master
async def _step_handover_wait(db, sched, host, op):
me, partner_master = await _roles(db, host)
if me.get("status") == "online" and me.get("carp_maintenance_mode") and me.get("carp_status") == "BACKUP" and partner_master:
try:
await _ACTION_FN[sched.action](host)
except client.OpnsenseAPIError as exc:
await _end(db, sched, host, op, ok=False,
message=f"Auftrag „{sched.action}“ mit unklarem Ergebnis: {str(exc)[:300]}. Nicht wiederholt. "
f"{host.name} bleibt im CARP-Wartungsmodus, der Partner trägt den Verkehr.")
return
sched.phase, sched.phase_since = "job_wait", _now()
_progress(sched, job_started_at=_now().isoformat(), seen_down=False, seen_running=False)
op.status, op.message = "submitted", "Übergabe bestätigt, Auftrag gestartet."
await db.commit()
await _audit(host, sched, op, sched.action, "submitted")
return
if _since(sched) < HANDOVER_TIMEOUT:
return
# The partner never took over. Nothing has run yet, so put the node back to normal.
if me.get("status") == "online" and me.get("carp_maintenance_mode"):
try:
await _toggle(host, "leave_maintenance")
except client.OpnsenseAPIError as exc:
await _end(db, sched, host, op, ok=False,
message=f"CARP-Übergabe nicht bestätigt, und der Normalbetrieb konnte nicht wiederhergestellt werden "
f"({str(exc)[:200]}). {host.name} prüfen.")
return
await _end(db, sched, host, op, ok=False, op_status="cancelled",
message="CARP-Übergabe nicht bestätigt (der Partner wurde nicht MASTER). Normalbetrieb wiederhergestellt, nichts aktualisiert.")
elif me.get("status") == "online":
await _end(db, sched, host, op, ok=False, op_status="cancelled",
message="CARP-Übergabe hat nicht stattgefunden (Wartungsmodus nicht aktiv). Nichts aktualisiert.")
else:
await _end(db, sched, host, op, ok=False,
message=f"CARP-Übergabe nicht bestätigt und {host.name} nicht erreichbar. Rollen prüfen.")
async def _step_job_wait(db, sched, host, op):
prog = dict(sched.progress or {})
started = datetime.fromisoformat(prog["job_started_at"])
elapsed = _now() - started
try:
running = (await client.get_firmware_running(host)).get("status")
except client.OpnsenseAPIError:
_progress(sched, seen_down=True) # rebooting: exactly what a reboot or base update looks like
running = None
else:
if running != "ready":
_progress(sched, seen_running=True)
prog = sched.progress
if running == "ready":
if sched.action == "reboot":
done = bool(prog.get("seen_down"))
else:
done = bool(prog.get("seen_down") or prog.get("seen_running")) or elapsed >= JOB_GRACE
if done:
sched.phase, sched.phase_since = "leave", _now()
await db.commit()
return
if elapsed >= JOB_TIMEOUT[sched.action]:
await _end(db, sched, host, op, ok=False,
message=f"Auftrag „{sched.action}“ auf {host.name} nicht rechtzeitig abgeschlossen. {host.name} bleibt im "
f"CARP-Wartungsmodus, der Partner trägt den Verkehr. Bitte prüfen und von Hand zurückschalten.")
return
await db.commit()
async def _step_leave(db, sched, host, op):
me, _partner = await _roles(db, host)
if me.get("status") != "online":
if _since(sched) >= LEAVE_TIMEOUT:
await _end(db, sched, host, op, ok=False,
message=f"{host.name} ist nach dem Auftrag nicht wieder erreichbar. Der Partner trägt den Verkehr.")
return
if me.get("carp_maintenance_mode"):
try:
await _toggle(host, "leave_maintenance")
except client.OpnsenseAPIError as exc:
await _end(db, sched, host, op, ok=False,
message=f"Zurückschalten mit unklarem Ergebnis: {str(exc)[:300]}. Nicht automatisch wiederholt — "
f"{host.name} prüfen, ggf. „CARP-Normalbetrieb herstellen“.")
return
await _audit(host, sched, op, "carp_leave", "submitted")
sched.phase, sched.phase_since = "leave_wait", _now()
await db.commit()
async def _step_leave_wait(db, sched, host, op):
fleet = await peers(db, host)
statuses = await get_statuses(fleet)
me = statuses.get(host.id, {})
online = [h for h in fleet if statuses.get(h.id, {}).get("status") == "online"]
masters = [h for h in online if statuses.get(h.id, {}).get("carp_status") == "MASTER"]
if me.get("status") == "online" and not me.get("carp_maintenance_mode") and len(masters) == 1 and len(online) == len(fleet):
extra = {}
try:
extra["version_after"] = (await client.get_firmware_info(host)).get("product_version")
extra["reboot_required"] = str((await client.get_firmware_status(host)).get("status_reboot", "0")) == "1"
except client.OpnsenseAPIError:
pass
extra["version_before"] = (sched.progress or {}).get("version_before")
text = "Automatisch abgeschlossen: Übergabe, Auftrag und Zurückschalten bestätigt."
if extra.get("reboot_required"):
text += " Ein Neustart ist noch erforderlich."
await _end(db, sched, host, op, ok=True, message=text, **extra)
return
if _since(sched) >= LEAVE_TIMEOUT:
await _end(db, sched, host, op, ok=False,
message=f"Normalbetrieb nach dem Zurückschalten nicht bestätigt. Rollen von {host.name} und Partner prüfen.")
_STEPS = {
"handover_wait": _step_handover_wait,
"job_wait": _step_job_wait,
"leave": _step_leave,
"leave_wait": _step_leave_wait,
}
+21 -8
View File
@@ -18,6 +18,8 @@ from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.x509.oid import ExtensionOID, NameOID
from app.services import overlay_service, proxy_connect
MAX_PEM_BYTES = 32 * 1024
@@ -90,6 +92,23 @@ def verify_argument(verify_tls: bool, tls_ca_pem: str | None):
return True
async def _open_tls(base_url: str, parsed, ctx: ssl.SSLContext, timeout: float):
"""TLS connection to the firewall: directly, or through the overlay hub's proxy for overlay addresses."""
proxy = overlay_service.proxy_for(base_url)
port = parsed.port or 443
if proxy is None:
return await asyncio.wait_for(
asyncio.open_connection(parsed.hostname, port, ssl=ctx, server_hostname=parsed.hostname), timeout
)
reader, writer = await proxy_connect.open_tunnel(proxy, parsed.hostname, port, timeout)
try:
await asyncio.wait_for(writer.start_tls(ctx, server_hostname=parsed.hostname), timeout)
except Exception:
writer.close()
raise
return reader, writer
async def probe(base_url: str, timeout: float = 8.0) -> dict:
"""Fetch the certificate a firewall presents, *without* trusting it. The
caller must show the fingerprint to a human to compare with the firewall."""
@@ -100,10 +119,7 @@ async def probe(base_url: str, timeout: float = 8.0) -> dict:
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
try:
_, writer = await asyncio.wait_for(
asyncio.open_connection(parsed.hostname, parsed.port or 443, ssl=ctx, server_hostname=parsed.hostname),
timeout,
)
_, writer = await _open_tls(base_url, parsed, ctx, timeout)
except (OSError, ssl.SSLError, asyncio.TimeoutError) as exc:
raise TlsPinError(f"Verbindung fehlgeschlagen: {exc}") from exc
try:
@@ -122,10 +138,7 @@ async def verify_matches(base_url: str, pem: str, timeout: float = 8.0) -> None:
presents anything else."""
parsed = urlparse(base_url)
try:
_, writer = await asyncio.wait_for(
asyncio.open_connection(parsed.hostname, parsed.port or 443, ssl=pinned_context(pem), server_hostname=parsed.hostname),
timeout,
)
_, writer = await _open_tls(base_url, parsed, pinned_context(pem), timeout)
writer.close()
except ssl.SSLError as exc:
raise TlsPinError("Das Zertifikat der Firewall passt nicht zum festgesetzten Zertifikat") from exc
+78
View File
@@ -0,0 +1,78 @@
"""Site variables: ``{{NAME}}`` placeholders in centrally authored objects, filled per host.
A rule, alias or NAT entry can say ``{{LAN_NET}}`` instead of a literal network; every host that object is
deployed to supplies its own value (Host page > Variables). Resolution happens when the payload for one host is
built, so preview, deploy, the change counters and the audit all see the host's real values.
Fail closed: a placeholder without a value for a target host is never pushed. Preview marks that host
``missing_variable``, the deploy is refused before anything is written, and as a last line of defence
``opnsense_client`` refuses to send any payload that still contains a placeholder.
"""
import re
from typing import Callable
from app.models.host import OpnsenseHost
NAME_RE = re.compile(r"^[A-Z][A-Z0-9_]{0,39}$")
VALUE_RE = re.compile(r"^[\w .:/,@+\-]{1,255}$") # networks, addresses, hostnames, names, lists: no braces, quotes or newlines
PLACEHOLDER = re.compile(r"\{\{\s*([A-Z][A-Z0-9_]{0,39})\s*\}\}")
class MissingVariables(Exception):
"""One or more hosts lack values for placeholders the object uses. ``missing`` maps host name -> names."""
def __init__(self, missing: dict[str, list[str]]):
self.missing = missing
super().__init__("; ".join(f"{host}: {', '.join(names)}" for host, names in missing.items()))
def names_in(value) -> set[str]:
return set(PLACEHOLDER.findall(value)) if isinstance(value, str) else set()
def names_in_payload(payload: dict) -> set[str]:
found: set[str] = set()
for value in payload.values():
found |= names_in(value)
return found
def host_values(host: OpnsenseHost) -> dict[str, str]:
return {v.name: v.value for v in host.variables}
def resolve(payload: dict, host: OpnsenseHost) -> dict:
"""The payload with this host's values filled in; raises MissingVariables if one is not defined."""
values = host_values(host)
missing = sorted(n for n in names_in_payload(payload) if n not in values)
if missing:
raise MissingVariables({host.name: missing})
return {k: PLACEHOLDER.sub(lambda m: values[m.group(1)], v) if isinstance(v, str) else v for k, v in payload.items()}
def ensure_resolvable(builder: Callable, entity, hosts: list[OpnsenseHost]) -> None:
"""Raise MissingVariables (listing every affected host) before anything is written."""
missing: dict[str, list[str]] = {}
for host in hosts:
try:
builder(entity, host)
except MissingVariables as exc:
missing.update(exc.missing)
if missing:
raise MissingVariables(missing)
def payload_or_none(builder: Callable, entity, host: OpnsenseHost):
try:
return builder(entity, host)
except MissingVariables:
return None
def preview_row_missing(host: OpnsenseHost, names: list[str], dep) -> dict:
return {
"host_id": str(host.id), "host_name": host.name, "action": "missing_variable",
"status": dep.status if dep else None,
"message": "Variable fehlt: " + ", ".join(names), "can_remove": False,
"can_retry_rollback": False, "changes": [],
}
+2 -2
View File
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
Versioning scheme: see CHANGELOG.md ("Versionierung").
"""
APP_VERSION = "1.16.0"
APP_CODENAME = "Hobby"
APP_VERSION = "1.23.5"
APP_CODENAME = "Harrier"
@@ -0,0 +1,23 @@
"""Benachrichtigung, wenn eine geplante Wartung nicht ausgeführt werden konnte.
Revision ID: 0006
Revises: 0005
Create Date: 2026-10-10
"""
from alembic import op
revision = "0006"
down_revision = "0005"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute(
"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS "
"scheduled_maintenance_failed_enabled boolean NOT NULL DEFAULT true"
)
def downgrade() -> None:
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS scheduled_maintenance_failed_enabled")
@@ -0,0 +1,26 @@
"""Geplante Wartung mit automatischer CARP-Übergabe: Option, Phase und Fortschritt.
Revision ID: 0007
Revises: 0006
Create Date: 2026-10-19
"""
from alembic import op
revision = "0007"
down_revision = "0006"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS carp_handover boolean NOT NULL DEFAULT false")
op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS phase varchar(16)")
op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS phase_since timestamp with time zone")
# The default only fills existing rows; like every JSON column here the model has no server default.
op.execute("ALTER TABLE scheduled_maintenance ADD COLUMN IF NOT EXISTS progress json NOT NULL DEFAULT '{}'::json")
op.execute("ALTER TABLE scheduled_maintenance ALTER COLUMN progress DROP DEFAULT")
def downgrade() -> None:
for column in ("progress", "phase_since", "phase", "carp_handover"):
op.execute(f"ALTER TABLE scheduled_maintenance DROP COLUMN IF EXISTS {column}")
@@ -0,0 +1,42 @@
"""API-Tokens für Skripte, CI, Prometheus und MCP.
Revision ID: 0008
Revises: 0007
Create Date: 2026-10-19
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0008"
down_revision = "0007"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Eine Altinstallation (vor Alembic) wird mit create_all() nach den aktuellen
# Modellen gebaut und hat die Tabelle dann schon.
if sa.inspect(op.get_bind()).has_table("api_tokens"):
return
op.create_table(
"api_tokens",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("user_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("name", sa.String(length=80), nullable=False),
sa.Column("scope", sa.String(length=10), nullable=False),
sa.Column("token_hash", sa.String(length=64), nullable=False),
sa.Column("prefix", sa.String(length=16), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_used_ip", sa.String(length=45), nullable=True),
sa.ForeignKeyConstraint(["user_id"], ["admin_users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_api_tokens_user_id", "api_tokens", ["user_id"])
op.create_index("ix_api_tokens_token_hash", "api_tokens", ["token_hash"], unique=True)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS api_tokens")
@@ -0,0 +1,37 @@
"""Sicherheitslage: letztes Prüfergebnis je Host.
Revision ID: 0009
Revises: 0008
Create Date: 2026-10-19
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0009"
down_revision = "0008"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Eine Altinstallation wird mit create_all() nach den aktuellen Modellen gebaut.
if sa.inspect(op.get_bind()).has_table("compliance_results"):
return
op.create_table(
"compliance_results",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("checked_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("score", sa.Integer(), nullable=True),
sa.Column("findings", sa.JSON(), nullable=False),
sa.Column("error", sa.String(length=20), nullable=True),
sa.Column("error_detail", sa.String(length=300), nullable=True),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_compliance_results_host_id", "compliance_results", ["host_id"], unique=True)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS compliance_results")
@@ -0,0 +1,39 @@
"""Audit-Export: Syslog und Webhook.
Revision ID: 0010
Revises: 0009
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
revision = "0010"
down_revision = "0009"
branch_labels = None
depends_on = None
def upgrade() -> None:
if sa.inspect(op.get_bind()).has_table("audit_export_settings"):
return
op.create_table(
"audit_export_settings",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("syslog_enabled", sa.Boolean(), nullable=False),
sa.Column("syslog_host", sa.String(length=255), nullable=True),
sa.Column("syslog_port", sa.Integer(), nullable=False),
sa.Column("syslog_protocol", sa.String(length=3), nullable=False),
sa.Column("webhook_enabled", sa.Boolean(), nullable=False),
sa.Column("webhook_url", sa.String(length=500), nullable=True),
sa.Column("webhook_signing_secret_set", sa.Boolean(), nullable=False),
sa.Column("cursor_created_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("cursor_id", sa.String(length=36), nullable=True),
sa.Column("last_success_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error", sa.String(length=300), nullable=True),
sa.Column("last_error_at", sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS audit_export_settings")
@@ -0,0 +1,59 @@
"""Geplante Backups mit Aufbewahrung und verschlüsseltem Offsite-Ziel; Benachrichtigung bei Backup-Problemen.
Revision ID: 0011
Revises: 0010
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0011"
down_revision = "0010"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS backup_failed_enabled boolean NOT NULL DEFAULT true")
if not inspector.has_table("backup_settings"):
op.create_table(
"backup_settings",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("scheduled_enabled", sa.Boolean(), nullable=False),
sa.Column("interval_hours", sa.Integer(), nullable=False),
sa.Column("keep_count", sa.Integer(), nullable=False),
sa.Column("offsite_enabled", sa.Boolean(), nullable=False),
sa.Column("s3_endpoint", sa.String(length=255), nullable=True),
sa.Column("s3_region", sa.String(length=64), nullable=False),
sa.Column("s3_bucket", sa.String(length=255), nullable=True),
sa.Column("s3_prefix", sa.String(length=255), nullable=False),
sa.Column("s3_access_key", sa.String(length=255), nullable=True),
sa.Column("s3_path_style", sa.Boolean(), nullable=False),
sa.Column("s3_secret_set", sa.Boolean(), nullable=False),
sa.Column("passphrase_set", sa.Boolean(), nullable=False),
sa.Column("last_run_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_offsite_ok_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_offsite_error", sa.String(length=300), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if not inspector.has_table("backup_uploads"):
op.create_table(
"backup_uploads",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("snapshot_id", sa.String(length=64), nullable=False),
sa.Column("remote_key", sa.String(length=400), nullable=False),
sa.Column("uploaded_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("host_id", "snapshot_id", name="uq_backup_upload"),
)
op.create_index("ix_backup_uploads_host_id", "backup_uploads", ["host_id"])
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS backup_uploads")
op.execute("DROP TABLE IF EXISTS backup_settings")
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS backup_failed_enabled")
@@ -0,0 +1,40 @@
"""Regelnutzung: wann hat eine Regel zuletzt etwas getroffen.
Revision ID: 0012
Revises: 0011
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0012"
down_revision = "0011"
branch_labels = None
depends_on = None
def upgrade() -> None:
if sa.inspect(op.get_bind()).has_table("rule_hits"):
return
op.create_table(
"rule_hits",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("rule_uuid", sa.String(length=36), nullable=False),
sa.Column("description", sa.String(length=255), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("first_seen_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_sampled_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_hit_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_packets", sa.BigInteger(), nullable=False),
sa.Column("last_bytes", sa.BigInteger(), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("host_id", "rule_uuid", name="uq_rule_hit"),
)
op.create_index("ix_rule_hits_host_id", "rule_hits", ["host_id"])
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS rule_hits")
+57
View File
@@ -0,0 +1,57 @@
"""Single Sign-on (OIDC): Einstellungen, Anmeldeversuche, Herkunft der Konten.
Revision ID: 0013
Revises: 0012
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0013"
down_revision = "0012"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
op.execute("ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS auth_source varchar(10) NOT NULL DEFAULT 'local'")
op.execute("ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS oidc_subject varchar(600)")
op.execute("CREATE UNIQUE INDEX IF NOT EXISTS ix_admin_users_oidc_subject ON admin_users (oidc_subject)")
if not inspector.has_table("oidc_settings"):
op.create_table(
"oidc_settings",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("display_name", sa.String(length=60), nullable=False),
sa.Column("issuer", sa.String(length=500), nullable=True),
sa.Column("client_id", sa.String(length=255), nullable=True),
sa.Column("client_secret_set", sa.Boolean(), nullable=False),
sa.Column("scopes", sa.String(length=255), nullable=False),
sa.Column("groups_claim", sa.String(length=64), nullable=False),
sa.Column("admin_group", sa.String(length=255), nullable=True),
sa.Column("operator_group", sa.String(length=255), nullable=True),
sa.Column("read_only_group", sa.String(length=255), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
if not inspector.has_table("oidc_login_states"):
op.create_table(
"oidc_login_states",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("state_hash", sa.String(length=64), nullable=False),
sa.Column("nonce", sa.String(length=64), nullable=False),
sa.Column("code_verifier", sa.String(length=128), nullable=False),
sa.Column("redirect_uri", sa.String(length=500), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_oidc_login_states_state_hash", "oidc_login_states", ["state_hash"], unique=True)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS oidc_login_states")
op.execute("DROP TABLE IF EXISTS oidc_settings")
op.execute("DROP INDEX IF EXISTS ix_admin_users_oidc_subject")
op.execute("ALTER TABLE admin_users DROP COLUMN IF EXISTS oidc_subject")
op.execute("ALTER TABLE admin_users DROP COLUMN IF EXISTS auth_source")
@@ -0,0 +1,28 @@
"""Aufgabentrennung: dürfen Operatoren selbst ausrollen?
Revision ID: 0014
Revises: 0013
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
revision = "0014"
down_revision = "0013"
branch_labels = None
depends_on = None
def upgrade() -> None:
if sa.inspect(op.get_bind()).has_table("governance_settings"):
return
op.create_table(
"governance_settings",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("operators_may_deploy", sa.Boolean(), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS governance_settings")
@@ -0,0 +1,48 @@
"""Management-Overlay (WireGuard-Hub) für Firewalls hinter NAT.
Revision ID: 0015
Revises: 0014
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0015"
down_revision = "0014"
branch_labels = None
depends_on = None
def upgrade() -> None:
inspector = sa.inspect(op.get_bind())
if not inspector.has_table("overlay_settings"):
op.create_table(
"overlay_settings",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("endpoint", sa.String(length=255), nullable=True),
sa.Column("subnet", sa.String(length=18), nullable=False),
sa.Column("listen_port", sa.Integer(), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
if not inspector.has_table("overlay_peers"):
op.create_table(
"overlay_peers",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("name", sa.String(length=80), nullable=False),
sa.Column("ip", sa.String(length=15), nullable=False),
sa.Column("public_key", sa.String(length=44), nullable=True),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("name"),
sa.UniqueConstraint("ip"),
sa.UniqueConstraint("public_key"),
)
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS overlay_peers")
op.execute("DROP TABLE IF EXISTS overlay_settings")
@@ -0,0 +1,34 @@
"""Standort-Variablen: Werte für {{NAME}}-Platzhalter je Host.
Revision ID: 0016
Revises: 0015
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0016"
down_revision = "0015"
branch_labels = None
depends_on = None
def upgrade() -> None:
if sa.inspect(op.get_bind()).has_table("host_variables"):
return
op.create_table(
"host_variables",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("name", sa.String(length=41), nullable=False),
sa.Column("value", sa.String(length=255), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("host_id", "name", name="uq_host_variable"),
)
op.create_index("ix_host_variables_host_id", "host_variables", ["host_id"])
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS host_variables")
@@ -0,0 +1,20 @@
"""Benachrichtigung bei kritischen/hohen Sicherheitsbefunden (standardmäßig aus).
Revision ID: 0017
Revises: 0016
Create Date: 2026-10-20
"""
from alembic import op
revision = "0017"
down_revision = "0016"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS compliance_alerts_enabled boolean NOT NULL DEFAULT false")
def downgrade() -> None:
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS compliance_alerts_enabled")
@@ -0,0 +1,21 @@
"""OIDC: Standard-Scopes ohne "groups" (Keycloak und Entra ID lehnen den Scope als ungültig ab).
Revision ID: 0018
Revises: 0017
Create Date: 2026-10-20
"""
from alembic import op
revision = "0018"
down_revision = "0017"
branch_labels = None
depends_on = None
def upgrade() -> None:
# Only rows still carrying the old default; a value somebody typed on purpose stays.
op.execute("UPDATE oidc_settings SET scopes = 'openid profile email' WHERE scopes = 'openid profile email groups'")
def downgrade() -> None:
pass
+1
View File
@@ -12,3 +12,4 @@ pytest-asyncio==1.4.0
respx==0.23.1
cryptography==50.0.2
alembic==1.20.0
PyJWT==2.10.1
+1
View File
@@ -20,6 +20,7 @@ os.environ["WIREGUARD_KEY_SECRET_DIR"] = "/tmp/opnexus-test-wireguard-secrets"
os.environ["CERT_KEY_SECRET_DIR"] = "/tmp/opnexus-test-cert-key-secrets"
os.environ["ACME_SECRET_DIR"] = "/tmp/opnexus-test-acme-secrets"
os.environ["TOTP_SECRET_DIR"] = "/tmp/opnexus-test-totp-secrets"
os.environ["OVERLAY_DIR"] = "/tmp/opnexus-test-overlay"
os.environ.setdefault("ADMIN_USERNAME", "admin")
# Force-set rather than setdefault: docker-compose's backend service sets
# ADMIN_PASSWORD="" (present-but-empty) by default, which setdefault would
+296
View File
@@ -0,0 +1,296 @@
"""API tokens: creation, scopes, session-only boundaries, revocation, audit."""
from datetime import datetime, timedelta, timezone
import pytest
from httpx import ASGITransport, AsyncClient
from sqlalchemy import select, update
from app.config import settings
from app.database import SessionLocal
from app.main import app
from app.models.api_token import ApiToken
from app.services import api_token_service
PASSWORD = settings.admin_password
async def _make(client, scope="read", name="ci", **extra):
resp = await client.post("/api/tokens", json={"name": name, "scope": scope, "current_password": PASSWORD, **extra})
assert resp.status_code == 201, resp.text
return resp.json()
def _bearer(token: str) -> AsyncClient:
return AsyncClient(
transport=ASGITransport(app=app), base_url="http://test", headers={"Authorization": f"Bearer {token}"}
)
async def _login(username: str, password: str) -> AsyncClient:
ac = AsyncClient(transport=ASGITransport(app=app), base_url="http://test")
resp = await ac.post("/api/auth/login", json={"username": username, "password": password})
assert resp.status_code == 200, resp.text
return ac
@pytest.mark.parametrize(
"scope,method,path,denied",
[
("read", "GET", "/api/hosts", False),
("read", "POST", "/api/rules", True),
# Secret-bearing reads need the admin scope; MCP (read-only tools) is open to every scope.
("read", "GET", "/api/hosts/abc/backups/config-1.xml", True),
("deploy", "GET", "/api/hosts/abc/backups/config-1.xml", True),
("admin", "GET", "/api/hosts/abc/backups/config-1.xml", False),
("read", "GET", "/api/hosts/abc/backups", False),
("read", "GET", "/api/notifications/settings", True),
("admin", "GET", "/api/notifications/settings", False),
("read", "POST", "/api/mcp", False),
("deploy", "POST", "/api/rules", False),
("deploy", "DELETE", "/api/aliases/x", False),
("deploy", "POST", "/api/hosts", True),
("deploy", "POST", "/api/hosts/tls-probe", True),
("deploy", "PATCH", "/api/hosts/abc", True),
("deploy", "DELETE", "/api/hosts/abc", True),
("deploy", "POST", "/api/hosts/abc/maintenance/execute", False),
("deploy", "POST", "/api/users", True),
("deploy", "PUT", "/api/settings", True),
("deploy", "PUT", "/api/notifications/settings", True),
("deploy", "PUT", "/api/logs/settings", True),
("deploy", "POST", "/api/system/updater/update", True),
("admin", "POST", "/api/users", False),
("admin", "POST", "/api/hosts", False),
# Session-only for every scope, reads included.
("admin", "GET", "/api/tokens", True),
("admin", "POST", "/api/tokens", True),
("read", "GET", "/api/tokens", True),
("admin", "POST", "/api/auth/change-password", True),
("admin", "POST", "/api/auth/2fa/disable", True),
],
)
def test_scope_matrix(scope, method, path, denied):
assert api_token_service.denies(scope, method, path) is denied
@pytest.mark.asyncio
async def test_create_needs_password_and_shows_token_once(client):
bad = await client.post("/api/tokens", json={"name": "x", "current_password": "wrong"})
assert bad.status_code == 401
assert bad.json()["detail"]["code"] == "AKTUELLES_PASSWORT_IST_FALSCH"
created = await _make(client, expires_in_days=30)
assert created["token"].startswith("opx_")
assert created["scope"] == "read"
assert created["prefix"] == created["token"][:10]
assert created["expires_at"] is not None
listed = (await client.get("/api/tokens")).json()
assert [t["id"] for t in listed] == [created["id"]]
assert "token" not in listed[0] and created["token"] not in str(listed)
@pytest.mark.asyncio
async def test_only_the_hash_is_stored(client):
created = await _make(client)
async with SessionLocal() as db:
row = await db.scalar(select(ApiToken))
assert row.token_hash == api_token_service.hash_token(created["token"])
assert created["token"] not in (row.token_hash, row.prefix)
@pytest.mark.asyncio
async def test_bearer_authenticates_without_a_session(client, anon_client):
created = await _make(client)
async with _bearer(created["token"]) as api:
assert (await api.get("/api/hosts")).status_code == 200
me = (await api.get("/api/auth/me")).json()
assert me["authenticated"] and me["via"] == "token" and me["scope"] == "read"
assert (await anon_client.get("/api/hosts")).status_code == 401
@pytest.mark.asyncio
async def test_read_scope_cannot_write(client):
created = await _make(client, "read")
async with _bearer(created["token"]) as api:
resp = await api.post("/api/groups", json={"name": "g"})
assert resp.status_code == 403
assert resp.json()["detail"]["code"] == "API_TOKEN_SCOPE_FORBIDDEN"
@pytest.mark.asyncio
async def test_deploy_scope_writes_fleet_objects_but_not_access_control(client):
created = await _make(client, "deploy")
async with _bearer(created["token"]) as api:
assert (await api.post("/api/groups", json={"name": "from-ci"})).status_code == 201
for method, path, body in [
("POST", "/api/users", {"username": "evil", "password": "evil-password-123", "role": "admin"}),
("POST", "/api/hosts", {}),
]:
resp = await api.request(method, path, json=body)
assert resp.status_code == 403, (path, resp.text)
assert resp.json()["detail"]["code"] == "API_TOKEN_SCOPE_FORBIDDEN"
@pytest.mark.asyncio
async def test_tokens_cannot_manage_tokens_or_accounts(client):
created = await _make(client, "admin")
async with _bearer(created["token"]) as api:
for method, path in [("GET", "/api/tokens"), ("POST", "/api/tokens"), ("DELETE", f"/api/tokens/{created['id']}"),
("POST", "/api/auth/change-password")]:
resp = await api.request(method, path, json={"name": "x", "current_password": PASSWORD})
assert resp.status_code == 403, (path, resp.text)
assert resp.json()["detail"]["code"] == "API_TOKEN_SCOPE_FORBIDDEN"
@pytest.mark.asyncio
async def test_admin_scope_acts_as_admin_and_is_audited_with_token_name(client):
created = await _make(client, "admin", name="terraform")
async with _bearer(created["token"]) as api:
resp = await api.post("/api/users", json={"username": "viaapi", "password": "via-api-password-1", "role": "read_only"})
assert resp.status_code == 201, resp.text
items = (await client.get("/api/audit", params={"kind": "user"})).json()["items"]
assert any(e["actor"] == "admin (API: terraform)" for e in items), items
@pytest.mark.asyncio
async def test_bad_or_expired_token_is_401_and_never_falls_back_to_the_cookie(client):
# `client` carries a valid session cookie; a broken bearer header must still fail.
resp = await client.get("/api/hosts", headers={"Authorization": "Bearer opx_doesnotexist"})
assert resp.status_code == 401
created = await _make(client)
async with SessionLocal() as db:
await db.execute(update(ApiToken).values(expires_at=datetime.now(timezone.utc) - timedelta(seconds=1)))
await db.commit()
async with _bearer(created["token"]) as api:
assert (await api.get("/api/hosts")).status_code == 401
@pytest.mark.asyncio
async def test_non_token_bearer_values_are_ignored(client):
# Only opx_ values are API tokens; anything else leaves normal cookie auth alone.
resp = await client.get("/api/hosts", headers={"Authorization": "Bearer something-else"})
assert resp.status_code == 200
@pytest.mark.asyncio
async def test_revoked_token_stops_working(client):
created = await _make(client)
async with _bearer(created["token"]) as api:
assert (await api.get("/api/hosts")).status_code == 200
assert (await client.delete(f"/api/tokens/{created['id']}")).status_code == 204
assert (await api.get("/api/hosts")).status_code == 401
assert (await client.delete(f"/api/tokens/{created['id']}")).status_code == 404
@pytest.mark.asyncio
async def test_last_used_is_recorded(client):
created = await _make(client)
async with _bearer(created["token"]) as api:
await api.get("/api/hosts")
listed = (await client.get("/api/tokens")).json()[0]
assert listed["last_used_at"] is not None
@pytest.mark.asyncio
async def test_read_only_user_gets_only_read_tokens_and_sees_only_their_own(client):
resp = await client.post("/api/users", json={"username": "tokviewer", "password": "viewer-password-123", "role": "read_only"})
assert resp.status_code == 201, resp.text
admin_token = await _make(client, "admin", name="admins")
viewer = await _login("tokviewer", "viewer-password-123")
try:
denied = await viewer.post("/api/tokens", json={"name": "x", "scope": "deploy", "current_password": "viewer-password-123"})
assert denied.status_code == 403
assert denied.json()["detail"]["code"] == "API_TOKEN_SCOPE_TOO_HIGH"
ok = await viewer.post("/api/tokens", json={"name": "grafana", "scope": "read", "current_password": "viewer-password-123"})
assert ok.status_code == 201, ok.text
mine = (await viewer.get("/api/tokens")).json()
assert [t["name"] for t in mine] == ["grafana"]
# Not someone else's token.
assert (await viewer.delete(f"/api/tokens/{admin_token['id']}")).status_code == 404
finally:
await viewer.aclose()
# The admin sees both and may revoke the viewer's.
all_tokens = {t["name"]: t for t in (await client.get("/api/tokens")).json()}
assert set(all_tokens) == {"admins", "grafana"} and all_tokens["grafana"]["owner"] == "tokviewer"
assert (await client.delete(f"/api/tokens/{all_tokens['grafana']['id']}")).status_code == 204
@pytest.mark.asyncio
async def test_demoted_owner_makes_the_token_read_only(client):
users = await client.post("/api/users", json={"username": "demotee", "password": "demotee-password-1", "role": "admin"})
uid = users.json()["id"]
owner = await _login("demotee", "demotee-password-1")
try:
created = (await owner.post("/api/tokens", json={"name": "t", "scope": "admin", "current_password": "demotee-password-1"})).json()
finally:
await owner.aclose()
assert (await client.patch(f"/api/users/{uid}/role", json={"role": "read_only"})).status_code == 200
async with _bearer(created["token"]) as api:
resp = await api.post("/api/groups", json={"name": "g"})
assert resp.status_code == 403
assert resp.json()["detail"]["code"] == "READ_ONLY_ACCESS"
@pytest.mark.asyncio
async def test_password_reset_and_user_delete_revoke_tokens(client):
resp = await client.post("/api/users", json={"username": "resetme", "password": "resetme-password-1", "role": "admin"})
uid = resp.json()["id"]
owner = await _login("resetme", "resetme-password-1")
try:
t1 = (await owner.post("/api/tokens", json={"name": "a", "current_password": "resetme-password-1"})).json()
finally:
await owner.aclose()
async with _bearer(t1["token"]) as api:
assert (await api.get("/api/hosts")).status_code == 200
assert (await client.post(f"/api/users/{uid}/reset-password", json={"new_password": "brand-new-password-1"})).status_code == 200
assert (await api.get("/api/hosts")).status_code == 401
assert (await client.delete(f"/api/users/{uid}")).status_code == 204
async with SessionLocal() as db:
assert (await db.scalars(select(ApiToken))).all() == []
@pytest.mark.asyncio
async def test_token_count_is_limited(client, monkeypatch):
monkeypatch.setattr(api_token_service, "MAX_TOKENS_PER_USER", 2)
await _make(client, name="1")
await _make(client, name="2")
resp = await client.post("/api/tokens", json={"name": "3", "current_password": PASSWORD})
assert resp.status_code == 409
assert resp.json()["detail"]["code"] == "API_TOKEN_LIMIT_REACHED"
@pytest.mark.asyncio
async def test_wrong_password_attempts_are_throttled(client):
codes = []
for _ in range(15):
codes.append((await client.post("/api/tokens", json={"name": "x", "current_password": "nope"})).status_code)
assert 429 in codes
@pytest.mark.asyncio
async def test_openapi_schema_is_served_behind_auth_with_bearer_scheme(client, anon_client):
assert (await anon_client.get("/api/openapi.json")).status_code == 401
schema = (await client.get("/api/openapi.json")).json()
assert schema["components"]["securitySchemes"]["bearerAuth"]["scheme"] == "bearer"
assert "/api/tokens" in schema["paths"] and "/api/hosts" in schema["paths"]
created = await _make(client)
async with _bearer(created["token"]) as api:
assert (await api.get("/api/openapi.json")).status_code == 200
# No interactive docs page (its CDN scripts would be blocked by the CSP anyway).
assert (await client.get("/docs")).status_code != 200
@pytest.mark.asyncio
async def test_changing_your_own_password_revokes_your_api_tokens(client):
assert (await client.post("/api/users", json={"username": "pwtok", "password": "first-password-12", "role": "operator"})).status_code == 201
user = await _login("pwtok", "first-password-12")
try:
token = (await user.post("/api/tokens", json={"name": "ci", "scope": "read", "current_password": "first-password-12"})).json()["token"]
async with _bearer(token) as api:
assert (await api.get("/api/hosts")).status_code == 200
resp = await user.post("/api/auth/change-password", json={"current_password": "first-password-12", "new_password": "second-password-34"})
assert resp.status_code == 200, resp.text
assert (await api.get("/api/hosts")).status_code == 401
finally:
await user.aclose()
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff. Show more