OPNexus/OPNexus
1
0 Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases 38 Wiki Activity

Release v1.10.1-rc2
Release / verify (push) Canceled after 0s
Details
CI / checks (push) Canceled after 0s
Details
CI / frontend (push) Canceled after 0s
Details
CI / backend (push) Canceled after 0s
Details
Release / test (push) Canceled after 0s
Details
Release / publish (backend) (push) Canceled after 0s
Details
Release / publish (frontend) (push) Canceled after 0s
Details
Release / publish (proxy) (push) Canceled after 0s
Details

Browse Source 
Siehe CHANGELOG.md
This commit is contained in:
Julian Boldt committed 2026-10-07 09:34:26 +02:00
commit e26b26eca6
355 files changed
+58160
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.env.example
+76
View File
@@ -0,0 +1,76 @@
POSTGRES_PASSWORD=change-me
# Datenbank- und Benutzername (Standard: opnexus). Gelten nur beim ersten Start
# eines leeren Volumes. Eine bestehende Installation trägt hier ihre bisherigen
# Namen ein, sonst findet das Backend die Datenbank nicht (Handbuch, "Datenbank-Namen").
# POSTGRES_DB=opnexus
# POSTGRES_USER=opnexus
# Optional — if left unset, a random admin password is generated on first
# startup and printed once to `docker compose logs backend`.
ADMIN_USERNAME=admin
ADMIN_PASSWORD=
# Required for any deployment reachable under a hostname other than
# localhost/127.0.0.1 — the exact origin (scheme://host[:port], no
# trailing slash) the frontend is actually served from, e.g.
# "https://opnexus.example.com" or "http://192.168.1.50:3100" (include
# the port unless a reverse proxy fronts this on the default 80/443).
# Without this, the backend only accepts credentialed requests from
# localhost on the default frontend port.
# Several origins (IP and name) may be listed comma-separated:
# FRONTEND_ORIGIN=https://10.0.0.5,https://opnexus.lan
# Behind the bundled TLS proxy requests are same-origin, so this only matters
# for cross-origin API clients.
FRONTEND_ORIGIN=
# Addresses OPNexus is reached by through the TLS proxy (Caddy), comma-
# separated: the server's IP and/or names. No DNS needed - the proxy issues
# certificates from its own CA (import /opnexus-ca.crt once in your browser).
# OPNEXUS_SITES=10.0.0.5, opnexus.lan
OPNEXUS_SITES=localhost
# The primary address from the list above (normally the IP): used when a
# client connects by bare IP and sends no server name.
OPNEXUS_DEFAULT_SNI=localhost
# Interface the proxy listens on (0.0.0.0 = whole network).
PROXY_BIND_ADDRESS=0.0.0.0
# Host interface the backend/frontend ports are published on. Defaults
# to 127.0.0.1 (this host only) if unset. Set to 0.0.0.0 for access from
# other machines without a reverse proxy (see README's "Produktivbetrieb"
# for the safer TLS-proxy alternative), or to one specific interface IP.
BIND_ADDRESS=
# Host-Ports von Backend und Frontend (nur nötig, wenn 8100/3100 schon belegt
# sind, z. B. bei einer zweiten Instanz auf demselben Rechner; Standard 8100/3100).
# BACKEND_PORT=8100
# FRONTEND_PORT=3100
# Set to true once this is running behind TLS (see README's
# "Produktivbetrieb" section) — otherwise the session cookie is sent
# unencrypted and browsers won't even attach a Secure cookie without
# HTTPS in the first place.
COOKIE_SECURE=false
# Set to true ONLY when running behind a reverse proxy you trust to
# always set/overwrite X-Forwarded-For itself (every example in README's
# "Produktivbetrieb" does). Used to attribute failed logins to the real
# client IP for the login lockout instead of the proxy's own IP. Leave
# false (default) if the app is reachable directly with no proxy in
# front — otherwise any client could spoof this header to dodge the
# per-IP lockout entirely.
TRUST_PROXY_HEADERS=false
# ── Release-Images (statt lokalem Build) ──────────────────────────────
# Registry + Namespace, unter dem die OPNexus-Images liegen; daraus werden
# <repo>/opnexus-backend, <repo>/opnexus-frontend, <repo>/opnexus-proxy. Leer lassen = lokal bauen
# mit `docker compose up -d --build`.
# OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
# Version fest pinnen (z. B. 1.9.0), statt automatisch "latest" zu folgen.
# OPNEXUS_VERSION=1.9.0
# ── Hinweis auf neue Versionen (optional) ─────────────────────────────
# Release-Feed (GitHub/Forgejo "releases/latest"). Leer = aus: dann macht
# OPNexus keine ausgehende Anfrage. Gesendet wird nur ein GET mit
# User-Agent "OPNexus/<Version>". Admins sehen bei einer neueren Version einen
# Hinweis; aktualisiert wird nie automatisch, sondern mit `opnexus update`.
# UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json
# UPDATE_CHECK_INTERVAL_HOURS=12
.github/workflows/ci.yml
+61
View File
@@ -0,0 +1,61 @@
# Tests bei jedem Push/PR. Wird auch vom Release-Workflow wiederverwendet,
# damit ein Release-Tag nie Images baut, deren Tests nicht grün waren.
#
# Die Jobs laufen bewusst in Containern (`container:`): dann ist der
# Postgres-Service überall unter dem Namen "postgres" erreichbar — auf
# GitHub genauso wie auf einem Forgejo/Gitea-Runner im Docker-Modus.
name: CI
on:
push:
branches: [master]
pull_request:
workflow_call:
jobs:
backend:
runs-on: ubuntu-latest
# Volles Image (nicht -slim): enthält git für actions/checkout.
container: python:3.12
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: opnexus
POSTGRES_PASSWORD: ci
# conftest.py erwartet eine DB, deren Name auf _test endet.
POSTGRES_DB: opnexus_test
options: >-
--health-cmd "pg_isready -U opnexus"
--health-interval 5s --health-timeout 5s --health-retries 10
env:
DATABASE_URL: postgresql+asyncpg://opnexus:ci@postgres:5432/opnexus_test
steps:
- uses: actions/checkout@v4
# pg_dump/pg_restore 16 wie im Backend-Image (Dumps nur gleiche Hauptversion).
- run: backend/scripts/install_pg_client.sh 16
- run: pip install -r backend/requirements.txt
- run: python -m pytest -q
working-directory: backend
frontend:
runs-on: ubuntu-latest
container: node:20
steps:
- uses: actions/checkout@v4
- run: npm ci --no-audit --no-fund
working-directory: frontend
- run: npm run lint
working-directory: frontend
# next build prüft auch die Typen.
- run: npm run build
working-directory: frontend
# Schnelle Konsistenzprüfungen, die ohne Datenbank und Build auskommen.
checks:
runs-on: ubuntu-latest
container: node:20
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check_i18n.py
- run: bash -n scripts/opnexus && bash -n scripts/check_version.sh && bash -n scripts/make_release_bundle.sh
.github/workflows/release.yml
+84
View File
@@ -0,0 +1,84 @@
# Baut und veröffentlicht die Release-Images, wenn ein Tag vX.Y.Z gepusht wird.
#
# Bewusst registry-neutral: dieselbe Datei läuft auf GitHub (ghcr.io) und auf
# Forgejo/Gitea mit Actions (Forgejo liest .github/workflows, wenn es kein
# .forgejo/workflows gibt). Wohin gepusht wird, steht NICHT hier, sondern in
# den Repository-Einstellungen:
# Variablen: REGISTRY (Default ghcr.io, z. B. git.opnexus.dev)
# IMAGE_NAMESPACE (Default: Repo-Besitzer, kleingeschrieben)
# Secrets: REGISTRY_USERNAME (Default: der auslösende Benutzer)
# REGISTRY_TOKEN (Default: GITHUB_TOKEN — das reicht nur für
# ghcr.io; auf Forgejo ein Zugriffstoken mit
# Berechtigung "write:package" eintragen)
# Wer auf mehrere Registries veröffentlichen will, pusht den Tag einfach zu
# mehreren Remotes — jeder Remote baut mit seiner eigenen Konfiguration.
# Ergebnis: <REGISTRY>/<IMAGE_NAMESPACE>/opnexus-{backend,frontend,proxy}:<version>
# (flache Namen, weil verschachtelte Pfade je Registry unterschiedlich gehen).
#
# Der Publish-Job braucht einen Runner mit Docker-Zugriff, der QEMU-
# Emulation für arm64 erlaubt (privilegiert).
name: Release
on:
push:
tags: ["v[0-9]+.[0-9]+.[0-9]+*"]
jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: scripts/check_version.sh "${GITHUB_REF_NAME%%-*}"
test:
uses: ./.github/workflows/ci.yml
publish:
needs: [verify, test]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
fail-fast: true
matrix:
service: [backend, frontend, proxy]
steps:
- uses: actions/checkout@v4
- name: Ziel bestimmen
id: target
env:
REGISTRY: ${{ vars.REGISTRY || 'ghcr.io' }}
NAMESPACE: ${{ vars.IMAGE_NAMESPACE || github.repository_owner }}
run: |
# Registry-Pfade müssen kleingeschrieben sein.
echo "image=$(echo "$REGISTRY/$NAMESPACE/opnexus-${{ matrix.service }}" | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ${{ vars.REGISTRY || 'ghcr.io' }}
username: ${{ secrets.REGISTRY_USERNAME || github.actor }}
password: ${{ secrets.REGISTRY_TOKEN || secrets.GITHUB_TOKEN }}
- id: meta
uses: docker/metadata-action@v5
with:
images: ${{ steps.target.outputs.image }}
# v1.9.0 -> 1.9.0, 1.9, 1 (+ latest); Vorabversionen bekommen nur
# den vollen Tag und nie "latest".
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
- uses: docker/build-push-action@v6
with:
context: ./${{ matrix.service }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
.gitignore
+11
View File
@@ -0,0 +1,11 @@
.env
frontend/node_modules/
frontend/.next/
frontend/tsconfig.tsbuildinfo
frontend/next-env.d.ts
backend/__pycache__/
backend/**/__pycache__/
backend/.pytest_cache/
.opnexus-update-state
dist/
.public-repo/
CHANGELOG.md
+369
View File
@@ -0,0 +1,369 @@
# Changelog
Alle nennenswerten Änderungen an OPNexus, mit Version und Datum. Für die
vollständige Feature-Beschreibung (Screenshots, Verifikationsnachweise,
v1-Scope-Grenzen) siehe die datierten Abschnitte in [Handbuch](docs/HANDBUCH.md)
— dieses Dokument ist die kurze Versions-/Namens-Übersicht, nicht der
Ersatz dafür.
## Versionierung
OPNexus folgt [Semantic Versioning](https://semver.org/lang/de/)
(`MAJOR.MINOR.PATCH`):
- **MAJOR** — Breaking Changes (z.B. inkompatible API-/Schema-Änderungen,
entfernte Features).
- **MINOR** — neue Features, rückwärtskompatibel (der Regelfall bei einem
Projekt, das noch aktiv wächst).
- **PATCH** — Bugfixes und kleine Korrekturen ohne neues Feature.
Zusätzlich bekommt **jede MINOR- und MAJOR-Version einen Codenamen** —
Raubvögel, alphabetisch aufsteigend über die Releases hinweg (ähnliches
Prinzip wie Ubuntus alliterierende Tier-Codenamen, nur ohne den doppelten
Anfangsbuchstaben-Zwang). Ein PATCH-Release behält den Codenamen seiner
MINOR-Version. Die aktuelle Version steht in `frontend/lib/version.ts`
(`APP_VERSION`/`APP_CODENAME`) und `backend/app/version.py` — beide von
Hand gepflegt, keine Build-Pipeline/kein Git-Hash-Tagging, bewusst analog
zur bisherigen Konvention (siehe Handbuch, "Login-Infos" 2026-09-09). Sichtbar
in der App auf der Login-Seite, im Footer jeder Seite, auf `/about` und in
den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin …
## [1.10.1] "Lanner" — 2026-10-06
PATCH-Release (Bugfixes, Dokumentation, Lizenz/Repo-Umstellung) nach dem ersten Live-Test
gegen frische OPNsense-26.7-Firewalls; löst die Release Candidates
`1.10.0-rc1`/`-rc2` ab.
- **Fehlerbehebung aus dem ersten Live-Test mit OPNsense 26.7:**
- Beschädigte HTTP-Antworten einer Firewall (bekannter kTLS-Fehler in
OPNsense 26.x) werden als solche gemeldet, mit dem Workaround
`kern.ipc.tls.enable=0` im Log (vorher nur „Verbindung fehlgeschlagen“).
- Die Log-Ingest-Warnung nennt den echten Grund statt pauschal „nicht
erreichbar?“ – die Firewall war in diesem Fall erreichbar.
- `BACKEND_PORT`/`FRONTEND_PORT` in der `.env`, damit eine zweite (Test-)Instanz
auf demselben Rechner ohne Änderung am Compose-File läuft.
- Neues Handbuch-Kapitel „Fehlerbehebung“ (kTLS, zweite Instanz,
`OUTBOUND_MODE_BLOCKS_PUSH`, Erkennungstakt der CARP-Historie) und zwei
Regressionstests.
- **Datenbank-Namen:** Standard für Datenbank und Benutzer ist jetzt `opnexus`
(`POSTGRES_DB`/`POSTGRES_USER` in der `.env`, auch von Backup-Dienst,
`opnexus`-Kommando und CI gelesen). Ältere Installationen tragen ihre
bisherigen Namen in der `.env` ein (Handbuch, „Datenbank-Namen“). Alte
Projektnamen sind aus Quelltext, Handbuch und Changelog entfernt.
- **Lizenzwechsel zur PolyForm Shield License 1.0.0** (siehe Handbuch, „Lizenz“):
Download, Quellcode-Einsicht und eigener Einsatz bleiben kostenlos, der
Weiterverkauf und konkurrierende Produkte durch Dritte sind ausgeschlossen.
Bereits unter AGPL-3.0 veröffentlichte Stände bleiben unter der AGPL.
- Website: „Open Source“-Aussagen entfernt, Texte zur Lizenz angepasst.
- **Öffentliches Repo** `git.opnexus.dev/opnexus/opnexus`: pro Release ein sauberer
Snapshot-Commit (`scripts/publish_public.sh`), Images in der Registry derselben
Gitea (`scripts/build_release_images.sh`).
- **Neue kurze `README.md`** (Englisch) und `README.de.md`; die ausführliche
Dokumentation ist jetzt das **Handbuch** `docs/HANDBUCH.md`. Dazu `SECURITY.md`.
- Update-Feed liegt unter `website/latest.json` (ausgeliefert als
`https://updates.opnexus.dev/latest.json`); Website-Auslieferung über
Cloudflare Pages (`website/_headers`).
## [1.10.0] "Lanner" — 2026-10-05
Update-Fähigkeit, Teil 2: Release-Images (siehe Handbuch „Release-Images & CI“):
- **CI-Workflows** (`.github/workflows/`): Tests bei Push/PR; bei einem Tag
`vX.Y.Z` baut der Release-Workflow Multi-Arch-Images (amd64/arm64) für
Backend, Frontend und Proxy und pusht sie. Die Registry ist reine
Repository-Konfiguration (GitHub oder Forgejo).
- **`docker-compose.yml`:** jeder Dienst hat `build:` und `image:`
(`OPNEXUS_IMAGE_REPO`/`OPNEXUS_VERSION`, Image-Namen `opnexus-backend`/`-frontend`/`-proxy`); Update aus Release-Images ist
`docker compose pull && docker compose up -d --no-build`.
- **Proxy-Image** mit eingebackener Caddyfile statt Bind-Mount.
- `scripts/check_version.sh` prüft Tag, Versionsdateien und CHANGELOG.
Update-Fähigkeit, Teil 3: Update-Werkzeug (siehe Handbuch „Update und Rollback mit
`opnexus`“):
- **`scripts/opnexus`** mit `status`, `update`, `rollback`, `backup`: Pull vor
dem Eingriff, Warten auf „gesund“, automatisches Zurückrollen, Rollback
schließt die Datenbank-Wiederherstellung aus dem Pre-Upgrade-Backup ein.
- **Behoben — Pre-Upgrade-Backup war nicht zurückspielbar:** das Backend-Image
hatte `pg_dump` 17 (Debian 13) gegen einen Postgres-16-Server, solche Dumps
lassen sich dort nicht einspielen. Das Image installiert jetzt den Client 16
(`backend/scripts/install_pg_client.sh`); der Start bricht vor der Migration ab,
wenn Client- und Server-Hauptversion nicht übereinstimmen. Ein neuer Test
spielt das Backup tatsächlich zurück.
- `PROXY_HTTPS_PORT` und `PROXY_HTTP_PORT` machen die Proxy-Ports konfigurierbar.
- **Compose getrennt:** `docker-compose.yml` nur mit Images (= Release-Compose),
`docker-compose.override.yml` mit den Build-Kontexten für den Repo-Checkout.
- **`scripts/make_release_bundle.sh`** erzeugt ein Release-Bundle (Compose mit
eingebackener Registry/Version, `.env.example`, `opnexus`, Anleitung).
- ESLint ignoriert die generierte `next-env.d.ts`.
Update-Fähigkeit, Teil 4: Hinweis auf neue Versionen:
- **„Neue Version verfügbar“-Banner** für Administratoren. Quelle ist ein
Release-Feed im Format von GitHub/Forgejo `releases/latest`
(`UPDATE_CHECK_URL`); ohne URL ist die Funktion aus und OPNexus macht keine
ausgehende Anfrage. Neuer Endpunkt `GET /api/system/version-check`
(Ergebnis 12 Stunden gecacht, Fehler werden leise abgefangen und nicht
sofort wiederholt, Vorabversionen werden nie angeboten, nur http(s)-Links
werden weitergegeben). Pro Version wegklickbar. Aktualisiert wird nie
automatisch, sondern mit `opnexus update`.
## [1.9.0] "Kite" — 2026-10-04
Update-Fähigkeit, Teil 1: Schema-Migrationen (siehe Handbuch „Schema-Migrationen &
Upgrade“):
- **Alembic ersetzt `schema_patches.py`.** Das Schema wird versioniert
(`backend/migrations/`); `app/migrate.py` migriert beim Start unter einem
Advisory-Lock und in einer Transaktion. Frische Installationen bauen das
Schema aus der Baseline `0001` auf, Installationen bis 1.8.x werden einmalig
überbrückt und gestempelt (`0002` gleicht dabei fehlende Server-Defaults und
einen fehlenden Index an).
- **Automatisches Backup vor jeder Migration:** `pg_dump` nach
`/backups/pre-upgrade-…dump`; schlägt es fehl, wird nicht migriert.
- Eine Datenbank, die von einer neueren Version migriert wurde, wird nicht mehr
stillschweigend weiterbenutzt — der Start bricht mit klarer Meldung ab.
- `app/schema_patches.py` ist jetzt `app/legacy_schema_patches.py`
(eingefroren, nur noch die Brücke).
- Neue Tests (`tests/test_migrations.py`): Frisch-Aufbau gleicht den Modellen,
Altinstallation behält ihre Daten, Backup-Pflicht, parallele Starts.
- Server-Defaults, die bisher nur auf gepatchten Installationen existierten,
stehen jetzt auch in den Modellen.
## [1.8.1] "Harris" — 2026-10-04
- **Laufende Aktionen sind sichtbar:** Während ein Push, Speichern oder eine
andere Aktion läuft, zeigt der Button jetzt einen drehenden Spinner vor dem
Text („Push läuft…“) statt nur den Text zu wechseln (neue Prop `loading` an
`Button`, gesetzt in allen Review-Dialogen, Formularen und Bestätigungen).
Lade-Hinweise („Wird geladen…“) nutzen die neue Komponente
`components/ui/Loading.tsx` mit Spinner. Bei „Bewegung reduzieren“ im
Betriebssystem bleibt der Spinner stehen.
## [1.8.0] "Harris" — 2026-10-04
Barrierefreiheit (siehe Handbuch „Barrierefreiheit“):
- **Kontrast:** Die Grautöne für Nebentext (`slate-400`/`slate-500`) erfüllen
im Hellmodus jetzt WCAG AA (4,5:1); Links in `sky-700` statt `sky-600`;
im Dunkelmodus war der aktive Menüpunkt hell auf hell (Kontrast 1,99) und
ist behoben.
- **Tastatur:** sichtbarer Fokusring für alle Elemente, „Zum Inhalt springen“-
Link, `aria-current` im Menü, Bereichsnavigation als eigene Landmarke,
scrollbare Tabellenbereiche per Tab erreichbar.
- **Screenreader:** sprechende Seitentitel pro Seite („Regeln · OPNexus“),
genau eine `h1` je Seite (Login und Über-Seite hatten keine),
korrekte Überschriftenreihenfolge in Regeln, Aliasen und Logs, Icons nur
dekorativ, leere Tabellenköpfe tragen „Aktionen“, Host-Status wird
jetzt übersetzt statt fest deutsch.
- Neues Prüfskript `scripts/verify_a11y.mjs` (axe-core, alle Seiten, Hell/
Dunkel/Mobil): 0 Verstöße.
## [1.7.0] "Goshawk" — 2026-10-04
Sichere Übertragung (siehe Handbuch „Sichere Übertragung“):
- **HTTPS-Proxy (Caddy)** mit interner CA: Zertifikate auch für reine IPs,
Zugriff per IP und Namen ohne DNS, CA-Download unter `/opnexus-ca.crt`.
Backend/Frontend nur noch auf `127.0.0.1`; API same-origin.
- **Zertifikats-Pinning pro Firewall** (Hosts → Bearbeiten, Fingerprint-
Bestätigung); `verify_tls=false` ist nicht mehr nötig. Neue Spalte
`opnsense_hosts.tls_ca_pem`, Endpunkt `POST /api/hosts/tls-probe`.
- **Zwei-Faktor-Anmeldung (TOTP)** optional pro Benutzer mit
Wiederherstellungscodes, Replay-Schutz, Admin-Reset und Notfall-CLI
`python -m app.reset_2fa`. Neue Spalten `admin_users.totp_*`.
- **Origin-Prüfung** gegen CSRF, **Sicherheits-Header** (HSTS, CSP, …),
`FRONTEND_ORIGIN` akzeptiert mehrere Ursprünge, API-Antworten `no-store`.
- Login-Screen fragt bei aktivem 2FA den Code ab; Hosts zeigen ein Schild für
den TLS-Status.
- Abhängigkeiten: PostCSS-Lücken per `overrides` behoben (`npm audit`: 0),
`pip-audit`: keine bekannten Lücken; neue Abhängigkeiten `cryptography`
(Backend) und `qrcode` (Frontend).
- Tests laufen unabhängig von den Transport-Einstellungen in `.env`.
- Fehler behoben: Kehrte ein bereits gelöstes Problem zurück (z. B. „Rollback
fehlgeschlagen“), scheiterte die Benachrichtigungsprüfung an der
Unique-Constraint, und der Alarm ging verloren. Die Zeile wird jetzt
wiederverwendet (`notification_dedup.py`).
## [1.6.0] "Peregrine" — 2026-10-04
Bedienbarkeit: Hinweise statt Textblöcke.
- Neue Tooltip-Komponente (`components/ui/Tooltip.tsx`): erscheint bei Hover,
Tastaturfokus und Tippen, schließt mit Esc, wird nie von Karten oder
Dialogen abgeschnitten und funktioniert auch auf deaktivierten Buttons.
- Update Center: Die großen Hinweis- und Sperrgrund-Kästen sind entfallen.
Warum „Update vorbereiten“ gesperrt ist, steht jetzt im Tooltip des
Buttons (mit Schloss-Symbol); Intro, Backup-Regel, Neustart- und
CARP-Hinweise sind Tooltips bzw. kompakte Badges. Erfolgsmeldungen
erscheinen als Toast. Die Bestätigungsdialoge (Hostname + Häkchen) bleiben
unverändert, sind aber kürzer, fokussieren das Eingabefeld und sagen am
deaktivierten Button, was noch fehlt.
- Dauerhafte Hinweisabsätze auf Audit, Einstellungen, Host-Wartung,
Live-NAT, Review-Verläufen und in Formularen (Hierarchie, Private Key)
sind zu Info-Tooltips geworden (`FieldGroup` hat dafür ein `hint`).
- Zeilenaktionen vereinheitlicht (Regeln, NAT, Aliase, VPN, Hosts, Benutzer):
eine Hauptaktion („Prüfen & Push“), Bearbeiten als Outline-Button, Löschen
dezent in Rot, Verlauf/Duplizieren als Icon mit Tooltip. Deploy-Ergebnisse
stehen kompakt in einer Zeile.
- Seitenintros auf allen Seiten sind Info-Tooltips am Seitentitel; der
Push-Hinweis (Bestätigungsfrist, automatischer Rollback) ist eine kurze
Zeile mit Tooltip.
- Host-Detail: Live-Regeln und Live-Aliase sind einklappbar (mit Anzahl) und
scrollen intern; die Seite ist von ~10.900 px auf ~2.300 px geschrumpft.
- Restore-Wizard: ein Schritt statt zwei, Warnung bleibt sichtbar, Übergabe
nur nach Eingabe des Hostnamens.
- Dashboard-Aktivität zeigt lesbare Aktionsnamen („Update-Suche“ statt
`check-updates`); große Zahlen in den Logs sind mit Tausendertrennung.
- Log-Erfassung: neues Größenlimit in GB (Einstellungen → Log-Erfassung,
Standard 10 GB, 0 = unbegrenzt). Ist es erreicht, werden die ältesten
Einträge gelöscht; die Einstellungen zeigen den belegten Speicher mit
Balken. Neue Spalte `log_settings.max_size_gb` (per `schema_patches.py`).
Beschriftungen klargestellt: „gespeichert“ (ganze Tabelle) in den
Einstellungen, „Treffern“ (aktueller Filter) auf der Logs-Seite.
- UI-Korrekturen: Karten in Regel-/Alias-/NAT-Listen haben wieder Abstand
(`space-y` wirkte nicht über `fieldset.contents`), `Input`/`Select`
behalten ihre Standardstyles auch mit eigener `className`, alle Buttons
sind gleich hoch, Einstellungen mit klaren Feldabständen.
- Dark Mode jetzt in neutralem Anthrazit statt Navy-Blau (Skala zentral in
`globals.css`); Info-/Neutral-Badges, Toasts und der Sekundär-Button sind
entsättigt, der Dialog-Hintergrund ist ein dunkler Schleier.
- Login-Screen: animierter Netzwerk-Hintergrund (Knoten, Verbindungen,
wandernde Datenpakete – blau = erlaubt, gelegentlich orange = blockiert),
gestaffelte Eingangsanimation, Schütteln bei falschen Zugangsdaten,
Ausblenden nach erfolgreichem Login. Respektiert `prefers-reduced-motion`
(dann statisch) und pausiert bei verstecktem Tab.
- Dialoge halten den Tastaturfokus (Tab-Falle) und geben ihn beim Schließen
zurück; Login-Felder haben Autocomplete-Attribute.
## [1.5.0] "Osprey" — 2026-09-19
Mehrsprachigkeit (Deutsch/Englisch) vollständig — siehe Handbuch,
"Mehrsprachigkeit (Deutsch/Englisch)":
- Alle Seiten und Komponenten der Oberfläche sind auf Katalogschlüssel
umgestellt (vorher nur Login, Navigation, Dashboard, Hosts und ein Teil
der Regeln); Datums-/Zeitformate folgen der gewählten Sprache.
- API-Fehler bleiben sprachneutral (`detail: {code, params}`): die letzten
rohen `HTTPException`s in `services/` sowie die Middleware-Antworten
(401 „nicht angemeldet“, 403 „Nur-Lese“) liefern jetzt ebenfalls Codes.
- Deutsche Fließtexte in Erfolgsantworten (Deploy-Vorschau-Warnungen,
Wartungs-/Deploy-Meldungen, Revisionsnotizen, Validierungsfehler) werden im
Client über `frontend/lib/i18n/serverMessages.ts` ins Englische übersetzt.
- Neu: `scripts/check_i18n.py` prüft Katalog-Parität, Platzhalter,
`ErrorCode`-Abdeckung und referenzierte Schlüssel.
- `tsconfig.json`: `target` auf ES2018 (nur Typprüfung, `noEmit`) wegen
benannter Regex-Gruppen.
378 Backend-Tests grün, `tsc --noEmit`/`next build` sauber.
## [1.4.0] "Merlin" — 2026-09-17
Lizenz und dritte Härtungsrunde:
- **AGPL-3.0** als Projektlizenz festgelegt — komplett kostenlos nutzbar,
Netzwerk-Klausel verhindert unveränderten/leicht angepassten Weiterverkauf
als proprietären Hosted-Dienst (siehe Handbuch, "Lizenz").
- `/docs`, `/redoc`, `/openapi.json` erfordern jetzt eine Anmeldung statt
öffentlich erreichbar zu sein.
- Login-Sperrschutz kann jetzt optional (`TRUST_PROXY_HEADERS`) die echte
Client-IP aus `X-Forwarded-For` lesen, für den Betrieb hinter einem
Reverse-Proxy — standardmäßig aus, nur mit explizitem Vertrauen in den
Proxy zu aktivieren.
378 Backend-Tests grün.
## [1.3.0] "Kestrel" — 2026-09-17
Zweite Härtungsrunde, ausgelöst durch das Live-Rollout von 1.2.0 selbst
(siehe Handbuch, "Zweite Härtungsrunde"):
- Keine rohen OPNsense-Fehlertexte mehr in 502-Antworten (Info-Leckage
an `read_only`-Nutzer).
- Healthchecks für Backend/Frontend ergänzt; dabei einen echten Next.js-
Bindungs-Bug gefunden und behoben (`HOSTNAME=0.0.0.0`).
- Nächtliches Backup der eigenen OPNexus-Datenbank (neuer
`postgres-backup`-Service), plus Fix für zu kleines Postgres-`/dev/shm`.
- Log-Erfassungs-Einstellungen (Intervall/Aufbewahrung) erstmals über
UI/API sicht- und editierbar; Default-Aufbewahrung 30→7 Tage.
- Realer Disk-Vorfall behoben: `firewall_log_entries` war bei den zwei
echten Firewalls in 9 Tagen auf ~9,7GB angewachsen, hat zusammen mit
Rebuilds zweimal kurz die Disk gefüllt — bereinigt, keine Datenverluste.
375 Backend-Tests grün, `tsc --noEmit`/`next build` sauber.
## [1.2.0] "Harrier" — 2026-09-17
Sicherheits- und Stabilitäts-Härtung vor dem geplanten öffentlichen
Release, aus einem vollständigen Audit (siehe Handbuch, "Sicherheits- und
Stabilitäts-Härtung vor dem öffentlichen Release"):
- CORS-Default auf `localhost`/`127.0.0.1` verengt, neue `FRONTEND_ORIGIN`-
Variable für alle anderen Deployments.
- `POSTGRES_PASSWORD` ohne unsicheren Fallback — muss explizit gesetzt sein.
- Ports standardmäßig nur auf `127.0.0.1`, steuerbar über neue
`BIND_ADDRESS`-Variable statt manuellem Editieren von
`docker-compose.yml`.
- TLS-/Reverse-Proxy-Betrieb dokumentiert (Caddy-Beispiel).
- Backend-Logging aktiviert (war zuvor komplett stumm) + Log-Rotation für
alle Container.
- Neuer automatischer Schema-Patch-Mechanismus (`app/schema_patches.py`)
ersetzt die bisherigen manuellen `ALTER TABLE`-Schritte bei jedem Start.
- Reproduzierbare Builds: Frontend-Lockfile committet (`npm ci`),
Backend-Abhängigkeiten exakt gepinnt.
Live auf der Produktivinstanz ausgerollt und verifiziert, 371 Backend-Tests
grün.
## [1.1.0] "Goshawk" — 2026-09-17
Vier funktionale Lücken gegenüber DynFi Manager geschlossen (dem einzigen
echten Software-Wettbewerber in der Nische zentrales OPNsense-Fleet-
Management) — Multi-Tenancy/MSP-Betrieb bewusst ausgeklammert, Zielgruppe
bleiben Privatleute/kleine Unternehmen mit eigener Firewall:
- **Direct-View-Link**: Ein-Klick-Button öffnet die native OPNsense-GUI
eines Hosts in einem neuen Tab.
- **Generische Dienst-Steuerung**: Beliebigen von OPNsense gemeldeten
Dienst direkt aus OPNexus starten/stoppen/neustarten.
- **Resource-Monitoring**: Kompakte 24h-Zeitverlaufs-Charts für CPU-Last,
Speicher- und Plattenbelegung je Host, eigene minimale
Inline-SVG-Sparkline-Komponente, live gegen die reale Flotte im
Hintergrund alle 60s abgetastet.
- **Geplante Wartung**: Neustart/Update/Major-Upgrade auf einen
zukünftigen Zeitpunkt statt sofort legen (einmalig, kein Recurring).
14 neue Backend-Tests (371 gesamt grün). Siehe Handbuch, "Direct-View-Link,
Dienst-Steuerung, Resource-Monitoring, geplante Wartung".
## [1.0.0] "Falcon" — 2026-09-17
Erstes Release unter dem neuen Versionsschema — markiert den aktuellen,
bereits sehr ausgereiften Stand des Projekts (siehe Handbuch für die lange
Vorgeschichte des Projekts). Fasst
zwei zuvor unversionierte Arbeits-Batches zusammen, die mit diesem Release
erstmals committet wurden:
- **Fleet-Dashboard, neue UI-Bausteine, Konsistenz-Pass & Toast-Feedback**
(`/dashboard` als neue Startseite; `Badge`/`Tabs`/`Table`/`EmptyState`/
`Toast`-Primitives; CSS-Variablen-basierter Dark Mode für die
Slate-Palette; Erfolgs-Toasts für praktisch alle schreibenden Aktionen
inkl. aller Deploy-Push-Overlays). Siehe Handbuch, "Fleet-Dashboard, neue
UI-Bausteine und Konsistenz-Pass".
- **CARP/HA vertieft**: Ereignis-Historie pro Cluster, Split-Brain-/
Kein-Master-Alarm, NAT-Drift-Erkennung (Port Forward/1:1/Outbound),
optionaler Auto-Revert-Timer für den CARP-Wartungsmodus. Siehe Handbuch,
"CARP/HA vertiefen".
- **Versionierung & Update-System** (dieses Release selbst): Semantic
Versioning mit Raubvogel-Codenamen, `CHANGELOG.md`, Versionsanzeige an
vier Stellen in der App (Login, Footer, `/about`, Einstellungen),
Backend-`FastAPI`-App trägt dieselbe Version. Kleiner Nebenfund dabei
behoben: Login leitete nach erfolgreicher Anmeldung noch auf `/hosts`
weiter statt auf die seit dem Dashboard-Release neue Startseite
`/dashboard`.
357 Backend-Tests grün, `tsc --noEmit`/`next build` sauber. Details, offene
Punkte und Verifikationsstand: siehe docs/HANDBUCH.md.
LICENSE
+91
View File
@@ -0,0 +1,91 @@
Required Notice: Copyright 2026 Julian Boldt (https://opnexus.dev)
# PolyForm Shield License 1.0.0
<https://polyformproject.org/licenses/shield/1.0.0>
## Acceptance
In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
## Copyright License
The licensor grants you a copyright license for the software to do everything you might do with the software that would otherwise infringe the licensor's copyright in it for any permitted purpose. However, you may only distribute the software according to [Distribution License](#distribution-license) and make changes or new works based on the software according to [Changes and New Works License](#changes-and-new-works-license).
## Distribution License
The licensor grants you an additional copyright license to distribute copies of the software. Your license to distribute covers distributing the software with changes and new works permitted by [Changes and New Works License](#changes-and-new-works-license).
## Notices
You must ensure that anyone who gets a copy of any part of the software from you also gets a copy of these terms or the URL for them above, as well as copies of any plain-text lines beginning with `Required Notice:` that the licensor provided with the software. For example:
> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
## Changes and New Works License
The licensor grants you an additional copyright license to make changes and new works based on the software for any permitted purpose.
## Patent License
The licensor grants you a patent license for the software that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the software.
## Noncompete
Any purpose is a permitted purpose, except for providing any product that competes with the software or any product the licensor or any of its affiliates provides using the software.
## Competition
Goods and services compete even when they provide functionality through different kinds of interfaces or for different technical platforms. Applications can compete with services, libraries with plugins, frameworks with development tools, and so on, even if they're written in different programming languages or for different computer architectures. Goods and services compete even when provided free of charge. If you market a product as a practical substitute for the software or another product, it definitely competes.
## New Products
If you are using the software to provide a product that does not compete, but the licensor or any of its affiliates brings your product into competition by providing a new version of the software or another product using the software, you may continue using versions of the software available under these terms beforehand to provide your competing product, but not any later versions.
## Discontinued Products
You may begin using the software to compete with a product or service that the licensor or any of its affiliates has stopped providing, unless the licensor includes a plain-text line beginning with `Licensor Line of Business:` with the software that mentions that line of business. For example:
> Licensor Line of Business: YoyodyneCMS Content Management System (http://example.com/cms)
## Sales of Business
If the licensor or any of its affiliates sells a line of business developing the software or using the software to provide a product, the buyer can also enforce [Noncompete](#noncompete) for that product.
## Fair Use
You may have "fair use" rights for the software under the law. These terms do not limit them.
## No Other Rights
These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply any other licenses.
## Patent Defense
If you make any written claim that the software infringes or contributes to infringement of any patent, your patent license for the software granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
## Violations
The first time you are notified in writing that you have violated any of these terms, or done anything with the software not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 32 days of receiving notice. Otherwise, all your licenses end immediately.
## No Liability
***As far as the law allows, the software comes as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the software, under any kind of legal claim.***
## Definitions
The **licensor** is the individual or entity offering these terms, and the **software** is the software the licensor makes available under these terms.
A **product** can be a good or service, or a combination of them.
**You** refers to the individual or entity agreeing to these terms.
**Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all its affiliates.
**Affiliates** means the other organizations than an organization has control over, is under the control of, or is under common control with.
**Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise. Control can be direct or indirect.
**Your licenses** are all the licenses granted to you for the software under these terms.
**Use** means anything you do with the software requiring one of your licenses.
README.de.md
+72
View File
@@ -0,0 +1,72 @@
# OPNexus
**Alle OPNsense-Firewalls. Eine Konsole.**
Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln, Aliase, NAT, VPN, Zertifikate, Logs und Updates — per OPNsense-API ausgerollt, mit Vorschau, Bestätigung und automatischem Rollback.
[Website](https://opnexus.dev/) · [Changelog](CHANGELOG.md) · [Handbuch](docs/HANDBUCH.md) · [English](README.md)
![Zentrale Firewall-Regeln](website/assets/screenshots/de/rules-light-desktop.webp)
> **Stand:** `1.10.1-rc2` ist der aktuelle öffentliche Release-Candidate (er löst `1.10.0-rc1`/`-rc2` ab). Entwickelt wurde gegen echte OPNsense-Instanzen, eine frische Installation aus den veröffentlichten Images ist aber noch nicht im Feld erprobt — bitte Ecken und Kanten melden.
## Funktionen
- **Flotte & Hochverfügbarkeit** — Live-Status aller Hosts, CARP-Rollen nebeneinander, Ereignis-Historie, Alarm bei Split-Brain oder fehlendem MASTER, Wartungsmodus mit optionalem Auto-Revert-Timer.
- **Regeln, Aliase & NAT** — zentral gepflegt, gezielt auf Hosts oder Gerätegruppen ausgerollt; Port-Forward, 1:1- und Outbound-NAT mit Drift-Erkennung zwischen Cluster-Knoten.
- **Sichere Änderungen** — jede Änderung läuft *Prüfen → Ausrollen → Bestätigen*; bleibt die Bestätigung im Zeitfenster aus, rollt OPNexus zurück.
- **Hierarchische Policies** — Pre-/Post-Rulebases mit Vererbung über Gruppenbäume.
- **VPN** — IPsec-Site-to-Site, OpenVPN und WireGuard, einmal definiert und ausgerollt.
- **Zertifikate** — Trust-Store für alle Firewalls und Ausstellung per ACME / Let's Encrypt.
- **Logs & Audit** — zentrale Log-Suche mit Korrelation, Dashboards und lückenloser Deploy-Verlauf.
- **Update-Center** — Firmware-Updates für die Flotte, mit Blick auf HA-Reihenfolge und Wartungsmodus.
- **Zugang** — HTTPS mit gepinnten Firewall-Zertifikaten, optionale Zwei-Faktor-Anmeldung, Rollen (Admin / Nur-Lesen), Tastaturbedienung, Dark Mode, Oberfläche auf Deutsch und Englisch.
Alle Screenshots zeigen fiktive Beispieldaten.
## Schnellstart
Voraussetzung: Docker mit Compose-Plugin.
```bash
git clone https://git.opnexus.dev/opnexus/opnexus.git
cd opnexus
cp .env.example .env # POSTGRES_PASSWORD (und ADMIN_PASSWORD) setzen
docker compose up -d --build
```
Die Oberfläche läuft dann unter `http://localhost:3100`. Ohne `ADMIN_PASSWORD` wird beim ersten Start ein zufälliges Admin-Passwort erzeugt und **einmalig** im Backend-Log ausgegeben:
```bash
docker compose logs backend | grep -A5 "generated password"
```
Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelieferter Caddy-Proxy) siehe das [Handbuch](docs/HANDBUCH.md#zugriff-von-außerhalb-dieser-vm--produktivbetrieb-mit-tls).
**Fertige Images** (amd64) liegen in der Registry dieses Gitea. In der `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.10.1-rc2
```
```bash
docker compose pull && docker compose up -d --no-build
```
Aktualisiert wird danach mit `scripts/opnexus update` / `rollback` (erst ziehen, vor Migrationen ein Datenbank-Backup, automatisches Zurückrollen, wenn die Dienste nicht gesund werden).
## Datenschutz
OPNexus wird selbst betrieben und hat keine Telemetrie. Es verbindet sich nur mit deinen Firewalls und mit Benachrichtigungszielen, die du selbst einrichtest. Der optionale Hinweis auf neue Versionen (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) ist **aus**, solange du ihn nicht setzt; eingeschaltet geht höchstens alle 12 Stunden ein einfaches `GET` mit dem User-Agent `OPNexus/<Version>` hinaus — ohne Instanz- oder Nutzerdaten.
## Lizenz
OPNexus ist **quelloffen einsehbar (source-available)** unter der [PolyForm Shield License 1.0.0](LICENSE) — es ist *keine* Open-Source-Software im Sinne der OSI. Du darfst den Code herunterladen, lesen, verändern und nutzen, auch in einem Unternehmen für die eigenen Firewalls. Nicht erlaubt ist, damit ein konkurrierendes Produkt anzubieten (etwa Weiterverkauf oder Betrieb als gehosteter Dienst). Maßgeblich ist der Lizenztext.
## Sicherheit
Sicherheitslücken bitte vertraulich an **[email protected]** melden, siehe [SECURITY.md](SECURITY.md).
---
OPNexus ist ein unabhängiges Projekt und steht in keiner Verbindung zu Deciso B.V. oder dem OPNsense®-Projekt. OPNsense® ist eine Marke von Deciso B.V.
README.md
+74
View File
@@ -0,0 +1,74 @@
# OPNexus
**All your OPNsense firewalls. One console.**
Central management for multiple [OPNsense](https://opnsense.org) firewalls: rules, aliases, NAT, VPN, certificates, logs and updates — rolled out through the OPNsense API with preview, confirmation and automatic rollback.
[Website](https://opnexus.dev) · [Changelog](CHANGELOG.md) · [Handbook (German)](docs/HANDBUCH.md) · [Deutsch](README.de.md)
![Central firewall rules](website/assets/screenshots/en/rules-light-desktop.webp)
> **Status:** `1.10.1-rc2` is the current public release candidate (it replaces `1.10.0-rc1`/`-rc2`). It has been developed against real OPNsense instances, but a fresh install from the published images has not been field-tested yet — please expect rough edges and report them.
## What it does
- **Fleet & high availability** — live status of all hosts, CARP roles side by side, event history, alerts for split-brain or a missing MASTER, maintenance mode with an optional auto-revert timer.
- **Rules, aliases & NAT** — maintained centrally, rolled out to single hosts or device groups; port forward, 1:1 and outbound NAT with drift detection between cluster nodes.
- **Safe changes** — every change goes *preview → deploy → confirm*; without your confirmation inside the time window OPNexus rolls it back.
- **Hierarchical policies** — pre/post rulebases with inheritance across group trees.
- **VPN** — IPsec site-to-site, OpenVPN and WireGuard, defined once and rolled out.
- **Certificates** — trust store for all firewalls and ACME / Let's Encrypt issuance.
- **Logs & audit** — central log search with correlation, dashboards and a complete deploy audit trail.
- **Update center** — firmware updates for the fleet, aware of HA order and maintenance mode.
- **Access** — HTTPS with pinned firewall certificates, optional two-factor sign-in, roles (admin / read-only), keyboard operation, dark mode, German and English UI.
![Cluster view](website/assets/screenshots/en/cluster-light-desktop.webp)
All screenshots show fictitious sample data.
## Quick start
Requires Docker with the Compose plugin.
```bash
git clone https://git.opnexus.dev/opnexus/opnexus.git
cd opnexus
cp .env.example .env # set POSTGRES_PASSWORD (and ADMIN_PASSWORD)
docker compose up -d --build
```
The UI is then available at `http://localhost:3100`. Without `ADMIN_PASSWORD` a random admin password is generated on first start and printed **once** in the backend log:
```bash
docker compose logs backend | grep -A5 "generated password"
```
For access from other machines and for production use with TLS (bundled Caddy proxy), see the [handbook](docs/HANDBUCH.md#zugriff-von-außerhalb-dieser-vm--produktivbetrieb-mit-tls) (German).
**Prebuilt images** (amd64) are published to this Gitea's registry. In `.env`:
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.10.1-rc2
```
```bash
docker compose pull && docker compose up -d --no-build
```
`scripts/opnexus update` / `rollback` then update the installation (pull first, database backup before migrations, automatic rollback if the services do not become healthy).
## Privacy
OPNexus is self-hosted and has no telemetry. It only connects to your firewalls and to notification targets you configure yourself. The optional new-version notice (`UPDATE_CHECK_URL=https://updates.opnexus.dev/latest.json`) is **off** unless you set it; when on, it makes one plain `GET` at most every 12 hours with the user agent `OPNexus/<version>` — no instance or user data.
## License
OPNexus is **source-available** under the [PolyForm Shield License 1.0.0](LICENSE) — it is *not* open source in the OSI sense. You may download, read, modify and use it, including inside a company for its own firewalls. You may not use it to provide a product that competes with OPNexus (for example reselling it or offering it as a hosted service). The license text is authoritative.
## Security
Please report vulnerabilities privately to **[email protected]**, see [SECURITY.md](SECURITY.md).
---
OPNexus is an independent project and is not affiliated with Deciso B.V. or the OPNsense® project. OPNsense® is a trademark of Deciso B.V.
SECURITY.md
+15
View File
@@ -0,0 +1,15 @@
# Security policy / Sicherheitshinweise
**Please do not open a public issue for security problems.** Report them privately to **[email protected]** (German or English) with the affected version, steps to reproduce and the impact you see. Do not include real credentials or firewall data.
I will acknowledge your report as soon as I can, keep you informed about the fix and credit you in the changelog if you wish. This is a one-person project, so please allow some days for a first reply.
**Supported versions:** the latest release only.
---
**Bitte keine öffentlichen Issues für Sicherheitsprobleme.** Meldungen vertraulich an **[email protected]** (Deutsch oder Englisch), mit betroffener Version, Schritten zur Reproduktion und der Auswirkung. Bitte keine echten Zugangsdaten oder Firewall-Daten mitschicken.
Ich bestätige den Eingang so bald wie möglich, halte dich über die Behebung auf dem Laufenden und nenne dich auf Wunsch im Changelog. Es ist ein Ein-Personen-Projekt — für eine erste Antwort können ein paar Tage vergehen.
**Unterstützte Versionen:** nur das jeweils neueste Release.
backend/Dockerfile
+16
View File
@@ -0,0 +1,16 @@
FROM python:3.12-slim
WORKDIR /app
# pg_dump/pg_restore für das automatische Sicherheits-Backup vor Schema-
# Migrationen (app/migrate.py) — bewusst in der Hauptversion des Servers
# (siehe das Skript für das Warum).
COPY scripts/install_pg_client.sh /tmp/install_pg_client.sh
RUN /tmp/install_pg_client.sh 16 && rm /tmp/install_pg_client.sh
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
backend/alembic.ini
+41
View File
@@ -0,0 +1,41 @@
# Alembic-Konfiguration. Die DB-URL steht bewusst NICHT hier — migrations/env.py
# liest sie aus app.config.settings (DATABASE_URL), genau wie die App selbst.
[alembic]
script_location = %(here)s/migrations
file_template = %%(rev)s_%%(slug)s
prepend_sys_path = .
path_separator = os
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
backend/app/__init__.py
View File
Whitespace-only changes.
backend/app/auth_middleware.py
+89
View File
@@ -0,0 +1,89 @@
import uuid
from app.services import audit_service
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import JSONResponse
from app.database import SessionLocal
from app.errors import ErrorCode
from app.routers.auth import COOKIE_NAME
from app.services import auth_service
_PUBLIC_PATHS = {
"/api/auth/login",
"/api/auth/logout",
"/api/auth/me",
"/api/health",
}
# /docs, /redoc and /openapi.json used to be public — fully documenting a
# firewall-fleet-management API surface to anyone unauthenticated is
# unnecessary recon for an attacker probing a public instance. Kept out
# of _PUBLIC_PATHS so a logged-in browser session (cookie sent
# automatically, same-origin) still sees them fine, anonymous requests
# now get the normal 401 like any other endpoint.
# A read_only user can call any GET, plus these specific writes — both are
# purely self-scoped (only ever touch the caller's own row, never app
# config or another account) so blocking them would serve no security
# purpose while breaking normal self-service: change-password so a
# read_only account can always get out of a leaked/weak password, and
# PUT /settings because that endpoint only ever writes the caller's own
# UserPreferences.theme (see routers/settings.py) — a read_only user must
# still be able to pick light/dark/system for themselves.
_READ_ONLY_ALLOWED_WRITES = {
"/api/auth/change-password",
"/api/settings",
# Self-scoped: a user managing their own second factor.
"/api/auth/2fa/setup",
"/api/auth/2fa/enable",
"/api/auth/2fa/disable",
}
class AuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
if request.url.path in _PUBLIC_PATHS:
return await call_next(request)
token = request.cookies.get(COOKIE_NAME, "")
if token:
async with SessionLocal() as db:
user = await auth_service.resolve_session(db, token)
if user is not None:
if (
user.role == "read_only"
and request.method not in ("GET", "HEAD", "OPTIONS")
and request.url.path not in _READ_ONLY_ALLOWED_WRITES
):
return JSONResponse(
status_code=403,
content={"detail": {"code": ErrorCode.READ_ONLY_ACCESS.value, "params": {}}},
)
request.state.user = user
parts = request.url.path.strip("/").split("/")
kinds = {"rules": "rule", "aliases": "alias", "port-forward": "port_forward", "one-to-one": "one_to_one", "outbound-nat": "outbound_nat", "hosts": "host", "users": "user"}
kind = kinds.get(parts[1]) if len(parts) > 1 else None
tracked = kind and request.method in ("POST", "PUT", "PATCH", "DELETE") and not request.url.path.endswith(("/preview", "/carp-preview", "/revisions"))
action = ("delete" if request.method == "DELETE" else parts[-1] if len(parts) > 3 else "edit" if request.method in ("PUT", "PATCH") else "create")
token = audit_service.context.set({"actor": user.username, "id": str(uuid.uuid4()), "action": action})
event = {"kind": kind, "action": action, "entity_id": parts[2] if len(parts) > 2 else None,
"host_id": parts[2] if kind == "host" and len(parts) > 2 else None,
"details": {"method": request.method, "path": request.url.path}}
try:
if tracked:
await audit_service.persist(**event, status="requested")
response = await call_next(request)
if tracked:
await audit_service.persist(**event, status="accepted" if response.status_code < 400 else "rejected",
message=f"HTTP {response.status_code}")
return response
except Exception:
if tracked:
await audit_service.persist(**event, status="interrupted", message="Anfrage nicht abgeschlossen; Zustand vor Wiederholung prüfen.")
raise
finally:
audit_service.context.reset(token)
return JSONResponse(status_code=401, content={"detail": {"code": ErrorCode.NICHT_AUTHENTIFIZIERT.value, "params": {}}})
backend/app/config.py
+135
View File
@@ -0,0 +1,135 @@
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
@property
def allowed_origins(self) -> list[str]:
return [o.strip().rstrip("/") for o in self.frontend_origin.split(",") if o.strip()]
database_url: str = (
"postgresql+asyncpg://opnexus:opnexus@postgres:5432/opnexus"
)
# ── Schema migrations (app/migrate.py) ────────────────────────────
# Vor jeder Migration an einer bestehenden Datenbank wird per pg_dump
# ein Sicherheits-Backup abgelegt (dasselbe Volume wie der
# postgres-backup-Service). Schlägt es fehl, wird NICHT migriert.
migration_backup_dir: str = "/backups"
# Nur für Entwicklung/Notfall: Backup überspringen.
skip_migration_backup: bool = False
migration_backup_keep: int = 5
# ── Update check (app/services/update_check_service.py) ───────────
# Release-Feed im Format von GitHub/Forgejo `releases/latest`, z. B.
# https://api.github.com/repos/<owner>/<repo>/releases/latest
# https://git.example.com/api/v1/repos/<owner>/<repo>/releases/latest
# Leer = die Funktion ist aus: OPNexus macht dann KEINE ausgehende
# Anfrage. Gesendet wird nur ein GET mit User-Agent "OPNexus/<Version>",
# keine Instanz- oder Nutzerdaten.
update_check_url: str = ""
update_check_interval_hours: int = 12
# Cookies now carry the session, so this can no longer be a bare "*"
# (the Fetch spec forbids credentialed requests against a wildcard
# response) — see main.py's allow_origin_regex. Default only matches
# localhost/127.0.0.1 on the compose-default frontend port, i.e. local
# dev out of the box. A real deployment reachable under any other
# hostname MUST set FRONTEND_ORIGIN instead (see below) — a bare
# port-only regex would accept a credentialed cross-site request from
# *any* host serving on that port.
cors_origin_regex: str = r"^https?://(localhost|127\.0\.0\.1):3100$"
# Exact origin (scheme://host[:port], no trailing slash) of the
# deployed frontend, e.g. "https://opnexus.example.com". When set,
# this replaces cors_origin_regex entirely with an exact match — set
# it for any non-localhost deployment.
# Several origins may be given comma-separated, e.g. when OPNexus is
# reachable both by IP and by name: "https://10.0.0.5,https://opnexus.lan".
frontend_origin: str = ""
opnsense_secret_dir: str = "/data/opnsense-secrets"
opnsense_backup_dir: str = "/data/opnsense-backups"
opnsense_timeout: float = 10.0
# ── Auth ──────────────────────────────────────────────────────────
admin_username: str = "admin"
# Empty → bootstrap_admin() generates one and logs it once.
admin_password: str = ""
session_lifetime: int = 60 * 60 * 24 * 30 # 30 days, seconds
# No TLS termination in front of this tool by default (see README) —
# a Secure cookie would just never be sent. Set to true once HTTPS is
# in place.
cookie_secure: bool = False
login_max_attempts: int = 5
login_ip_max_attempts: int = 20
login_window_seconds: int = 300
login_lockout_seconds: int = 300
# Off by default: request.client.host is the actual TCP peer, safe to
# trust with no proxy in front. Behind a reverse proxy (see README's
# "Produktivbetrieb"), that peer is the proxy itself, collapsing every
# real client onto one IP for the login-throttle bucket — set this to
# true ONLY when the proxy is trusted to always set/overwrite
# X-Forwarded-For itself (every proxy setup in the README's example
# does). Never enable this if the app is reachable directly (no
# proxy) — any client could then spoof the header to dodge the
# per-IP lockout entirely.
trust_proxy_headers: bool = False
# ── Deploy lockout protection ────────────────────────────────────
deploy_confirm_window_seconds: int = 180
# ── Notifications ─────────────────────────────────────────────────
# Infra-only knobs (deployment concerns); delivery config itself
# (webhook URL, SMTP settings, per-event toggles, poll interval) is
# admin-editable via NotificationSettings — see app/models/notification.py.
notification_secret_dir: str = "/data/notification-secrets"
totp_secret_dir: str = "/data/totp-secrets"
# Floor against an admin setting an absurdly low poll interval, which
# would hammer both real firewalls with live status/cert/rule-drift
# calls every cycle.
notification_poll_min_interval_seconds: int = 60
notification_smtp_timeout_seconds: float = 10.0
# ── Log ingest ────────────────────────────────────────────────────
# Floor against an admin setting an absurdly low poll interval — same
# rationale as notification_poll_min_interval_seconds, but for the
# digest-based firewall-log ingest loop (see log_ingest_service.py).
# A single GET per enabled host per cycle, so a lower floor than
# notifications' is acceptable — this is the same call the log page
# already makes on-demand today, just automated.
log_poll_min_interval_seconds: int = 10
# ── IPsec VPN management ────────────────────────────────────────────
# Pre-shared-key secrets, one file per tunnel id — same convention as
# opnsense_secret_dir above, see ipsec_psk_store.py.
ipsec_psk_secret_dir: str = "/data/ipsec-psk-secrets"
# ── OpenVPN instance management ─────────────────────────────────────
# Static-key (tls-auth/tls-crypt hardening) key material, one file per
# instance id — same convention as ipsec_psk_secret_dir above, see
# openvpn_key_store.py.
openvpn_key_secret_dir: str = "/data/openvpn-key-secrets"
# ── WireGuard tunnel management ─────────────────────────────────────
# Server private key + optional per-peer pre-shared key, both stored
# one-file-per-tunnel-id under this directory — same convention as
# ipsec_psk_secret_dir/openvpn_key_secret_dir above, see
# wireguard_key_store.py.
wireguard_key_secret_dir: str = "/data/wireguard-key-secrets"
# ── Resource monitoring ──────────────────────────────────────────────
# Fixed interval, no admin-editable settings row/UI for this — v1
# deliberately kept minimal (see resource_monitor_service.py).
resource_poll_interval_seconds: int = 60
resource_sample_retention_days: int = 30
# ── Trust-Store + ACME certificate management ────────────────────────
# Private-key material for action="import" ManagedCertificate rows,
# one file per entity id — same convention as ipsec_psk_secret_dir
# above, see cert_key_store.py.
cert_key_secret_dir: str = "/data/cert-key-secrets"
# External-Account-Binding HMAC for AcmeAccount rows, one file per
# entity id — see acme_secret_store.py.
acme_secret_dir: str = "/data/acme-secrets"
model_config = SettingsConfigDict(case_sensitive=False)
settings = Settings()
backend/app/database.py
+17
View File
@@ -0,0 +1,17 @@
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.orm import DeclarativeBase
from app.config import settings
class Base(DeclarativeBase):
pass
engine = create_async_engine(settings.database_url, pool_pre_ping=True)
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)
async def get_db() -> AsyncSession:
async with SessionLocal() as session:
yield session
backend/app/errors.py
+140
View File
@@ -0,0 +1,140 @@
"""Stable, language-neutral errors returned by the HTTP API."""
from enum import Enum
from typing import Any
from fastapi import HTTPException
class ErrorCode(str, Enum):
BACKUP_EMPTY = "BACKUP_EMPTY"
BACKUP_INVALID_XML = "BACKUP_INVALID_XML"
BACKUP_TOO_LARGE = "BACKUP_TOO_LARGE"
BACKUP_WRONG_ROOT = "BACKUP_WRONG_ROOT"
ACCOUNT_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_EEEA04A9 = "ACCOUNT_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_EEEA04A9"
ACCOUNT_NICHT_GEFUNDEN = "ACCOUNT_NICHT_GEFUNDEN"
ACCOUNT_WIRD_NOCH_VON_MINDESTENS_EINER_ZERTIFIKATSANFRAGE_REFERENZIERT = "ACCOUNT_WIRD_NOCH_VON_MINDESTENS_EINER_ZERTIFIKATSANFRAGE_REFERENZIERT"
ACME_ACCOUNT_GEHOERT_ZU_EINEM_ANDEREN_HOST = "ACME_ACCOUNT_GEHOERT_ZU_EINEM_ANDEREN_HOST"
ACME_ACCOUNT_NICHT_GEFUNDEN = "ACME_ACCOUNT_NICHT_GEFUNDEN"
ACTION_NOT_ALLOWED = "ACTION_NOT_ALLOWED"
AKTUELLES_PASSWORT_IST_FALSCH = "AKTUELLES_PASSWORT_IST_FALSCH"
ALIAS_CONTENT_DIFFERS = "ALIAS_CONTENT_DIFFERS"
ALIAS_HAT_KEINE_ZIEL_HOSTS = "ALIAS_HAT_KEINE_ZIEL_HOSTS"
ALIAS_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS = "ALIAS_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS"
ALIAS_NICHT_GEFUNDEN = "ALIAS_NICHT_GEFUNDEN"
ALIAS_NOCH_NICHT_BESTAETIGT = "ALIAS_NOCH_NICHT_BESTAETIGT"
ALIAS_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_5A2C1888 = "ALIAS_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_5A2C1888"
BACKUP_NICHT_GEFUNDEN = "BACKUP_NICHT_GEFUNDEN"
BENUTZERNAME_BEREITS_VERGEBEN = "BENUTZERNAME_BEREITS_VERGEBEN"
BENUTZER_NICHT_GEFUNDEN = "BENUTZER_NICHT_GEFUNDEN"
CARP_VORSCHAU_NICHT_GEFUNDEN = "CARP_VORSCHAU_NICHT_GEFUNDEN"
CARP_ZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN = "CARP_ZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN"
DAS_NEUE_PASSWORT_MUSS_SICH_VOM_AKTUELLEN_UNTERSCHEIDEN = "DAS_NEUE_PASSWORT_MUSS_SICH_VOM_AKTUELLEN_UNTERSCHEIDEN"
DEPLOYMENT_NICHT_GEFUNDEN = "DEPLOYMENT_NICHT_GEFUNDEN"
DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN = "DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN"
DER_EIGENE_ACCOUNT_KANN_NICHT_GELOESCHT_WERDEN = "DER_EIGENE_ACCOUNT_KANN_NICHT_GELOESCHT_WERDEN"
EINE_REGEL_DARF_IN_DER_REIHENFOLGE_NUR_EINMAL_F05DF47C = "EINE_REGEL_DARF_IN_DER_REIHENFOLGE_NUR_EINMAL_F05DF47C"
EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87 = "EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87"
EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN = "EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN"
ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C = "ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C"
ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN = "ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN"
ERROR_1_1_NAT_REGEL_NICHT_GEFUNDEN = "ERROR_1_1_NAT_REGEL_NICHT_GEFUNDEN"
FIRMWARE_AUFTRAG_LAEUFT_BEREITS_ODER_ZUSTAND_IST_UNBEKANNT = "FIRMWARE_AUFTRAG_LAEUFT_BEREITS_ODER_ZUSTAND_IST_UNBEKANNT"
FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_PRE_SHARED_4AE1FC63 = "FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_PRE_SHARED_4AE1FC63"
FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998 = "FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998"
FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR = "FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR"
GEPLANTE_WARTUNG_NICHT_GEFUNDEN = "GEPLANTE_WARTUNG_NICHT_GEFUNDEN"
GROUP_HAS_ASSIGNED_ITEMS = "GROUP_HAS_ASSIGNED_ITEMS"
GRUPPENMITGLIEDER_ODER_REGELN_WURDEN_GEAENDERT_AUSWIRKUNGEN_ERNEUT_PRUEFEN = "GRUPPENMITGLIEDER_ODER_REGELN_WURDEN_GEAENDERT_AUSWIRKUNGEN_ERNEUT_PRUEFEN"
GRUPPENNAME_BEREITS_VERGEBEN = "GRUPPENNAME_BEREITS_VERGEBEN"
GRUPPE_HAT_NOCH_UNTERGRUPPEN_DIESE_ZUERST_VERSCHIEBEN_ODER_9FFFD853 = "GRUPPE_HAT_NOCH_UNTERGRUPPEN_DIESE_ZUERST_VERSCHIEBEN_ODER_9FFFD853"
GRUPPE_HAT_NOCH_ZUGEORDNETE_REGELN_DIESE_ZUERST_VERSCHIEBEN_2CCBC48B = "GRUPPE_HAT_NOCH_ZUGEORDNETE_REGELN_DIESE_ZUERST_VERSCHIEBEN_2CCBC48B"
GRUPPE_NICHT_GEFUNDEN = "GRUPPE_NICHT_GEFUNDEN"
GRUPPE_WIRD_NOCH_VON_ALIASEN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_ALIASEN_VERWENDET"
GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET"
GRUPPE_WIRD_NOCH_VON_REGELN_VERWENDET = "GRUPPE_WIRD_NOCH_VON_REGELN_VERWENDET"
HOSTNAMEN_EXAKT_EINGEBEN = "HOSTNAMEN_EXAKT_EINGEBEN"
HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN = "HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN"
HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0 = "HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0"
HOST_HAT_NOCH_AKTIVE_ALIAS_DEPLOYMENTS_DIESE_ZUERST_F4A3ED46 = "HOST_HAT_NOCH_AKTIVE_ALIAS_DEPLOYMENTS_DIESE_ZUERST_F4A3ED46"
HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845 = "HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845"
HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721 = "HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721"
HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85 = "HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85"
HOST_IST_DEAKTIVIERT = "HOST_IST_DEAKTIVIERT"
HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS = "HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS"
HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B = "HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B"
HOST_NICHT_GEFUNDEN = "HOST_NICHT_GEFUNDEN"
HOST_REQUEST_FAILED = "HOST_REQUEST_FAILED"
INSTANZ_HAT_KEINE_ZIEL_HOSTS = "INSTANZ_HAT_KEINE_ZIEL_HOSTS"
INSTANZ_NICHT_GEFUNDEN = "INSTANZ_NICHT_GEFUNDEN"
INSTANZ_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_DB923B59 = "INSTANZ_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_DB923B59"
INTERFACE_MEANING_DIFFERS = "INTERFACE_MEANING_DIFFERS"
KEINE_OFFENE_WARTUNGSAKTION = "KEINE_OFFENE_WARTUNGSAKTION"
KEIN_OFFENER_RESTORE_VORSCHAU_VORGANG = "KEIN_OFFENER_RESTORE_VORSCHAU_VORGANG"
KEIN_OFFENER_UEBERGEBENER_RESTORE_VORGANG = "KEIN_OFFENER_UEBERGEBENER_RESTORE_VORGANG"
MAINTENANCE_WAIT_REQUIRED = "MAINTENANCE_WAIT_REQUIRED"
MINDESTENS_EINE_REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_932925BD = "MINDESTENS_EINE_REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_932925BD"
MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN = "MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN"
NICHT_AUTHENTIFIZIERT = "NICHT_AUTHENTIFIZIERT"
CROSS_ORIGIN_BLOCKED = "CROSS_ORIGIN_BLOCKED"
TLS_PROBE_FAILED = "TLS_PROBE_FAILED"
TLS_PEM_INVALID = "TLS_PEM_INVALID"
TLS_PIN_MISMATCH = "TLS_PIN_MISMATCH"
TOTP_REQUIRED = "TOTP_REQUIRED"
TOTP_INVALID = "TOTP_INVALID"
TOTP_ALREADY_ENABLED = "TOTP_ALREADY_ENABLED"
TOTP_NOT_ENABLED = "TOTP_NOT_ENABLED"
TOTP_SETUP_MISSING = "TOTP_SETUP_MISSING"
NORMALBETRIEB_NOCH_NICHT_BESTAETIGT_WARTUNGSMODUS_MUSS_AUS_UND_8AF9290B = "NORMALBETRIEB_NOCH_NICHT_BESTAETIGT_WARTUNGSMODUS_MUSS_AUS_UND_8AF9290B"
NUR_AUSSTEHENDE_GEPLANTE_WARTUNGEN_KOENNEN_ABGEBROCHEN_WERDEN = "NUR_AUSSTEHENDE_GEPLANTE_WARTUNGEN_KOENNEN_ABGEBROCHEN_WERDEN"
NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN = "NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN"
NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB = "NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB"
OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN = "OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN"
OFFENES_DEPLOYMENT_ZUERST_KLAEREN = "OFFENES_DEPLOYMENT_ZUERST_KLAEREN"
ORDERING_FAILED = "ORDERING_FAILED"
OUTBOUND_NAT_REGEL_NICHT_GEFUNDEN = "OUTBOUND_NAT_REGEL_NICHT_GEFUNDEN"
PEER_PRE_SHARED_KEY_IST_AKTIVIERT_ABER_ES_FCFAC129 = "PEER_PRE_SHARED_KEY_IST_AKTIVIERT_ABER_ES_FCFAC129"
POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN = "POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN"
PORT_FORWARD_REGEL_NICHT_GEFUNDEN = "PORT_FORWARD_REGEL_NICHT_GEFUNDEN"
REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203 = "REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203"
REGEL_HAT_KEINE_ZIEL_HOSTS = "REGEL_HAT_KEINE_ZIEL_HOSTS"
REGEL_NICHT_GEFUNDEN = "REGEL_NICHT_GEFUNDEN"
REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0 = "REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0"
RESTORE_VORGANG_NICHT_GEFUNDEN = "RESTORE_VORGANG_NICHT_GEFUNDEN"
ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL = "ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL"
ROLLENWECHSEL_NOCH_NICHT_BESTAETIGT_DIESER_HOST_MUSS_IM_4676309B = "ROLLENWECHSEL_NOCH_NICHT_BESTAETIGT_DIESER_HOST_MUSS_IM_4676309B"
RULEBASE_NICHT_GEFUNDEN = "RULEBASE_NICHT_GEFUNDEN"
SICHERUNG_NICHT_GEFUNDEN = "SICHERUNG_NICHT_GEFUNDEN"
SICHERUNG_WURDE_SEIT_DER_VORSCHAU_VERAENDERT_ODER_IST_2732C4DD = "SICHERUNG_WURDE_SEIT_DER_VORSCHAU_VERAENDERT_ODER_IST_2732C4DD"
TLS_HARDENING_IST_AKTIVIERT_ABER_ES_WURDE_NOCH_F53C81DD = "TLS_HARDENING_IST_AKTIVIERT_ABER_ES_WURDE_NOCH_F53C81DD"
TOO_MANY_LOGIN_ATTEMPTS = "TOO_MANY_LOGIN_ATTEMPTS"
TUNNEL_HAT_KEINE_ZIEL_HOSTS = "TUNNEL_HAT_KEINE_ZIEL_HOSTS"
TUNNEL_NICHT_GEFUNDEN = "TUNNEL_NICHT_GEFUNDEN"
TUNNEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_50830F70 = "TUNNEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_50830F70"
UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN = "UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN"
UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT = "UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT"
UPSTREAM_REQUEST_FAILED = "UPSTREAM_REQUEST_FAILED"
VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN = "VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN"
VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN = "VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN"
VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN = "VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN"
VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET = "VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET"
WARTUNGSAKTION_NICHT_GEFUNDEN = "WARTUNGSAKTION_NICHT_GEFUNDEN"
WARTUNGSVORSCHAU_NICHT_GEFUNDEN = "WARTUNGSVORSCHAU_NICHT_GEFUNDEN"
ZEITPUNKT_MUSS_IN_DER_ZUKUNFT_LIEGEN = "ZEITPUNKT_MUSS_IN_DER_ZUKUNFT_LIEGEN"
ZERTIFIKATSANFRAGE_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_F684C449 = "ZERTIFIKATSANFRAGE_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_F684C449"
ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN = "ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN"
ZERTIFIKATSANFRAGE_WURDE_NOCH_NICHT_AUF_DIE_FIREWALL_GESCHRIEBEN = "ZERTIFIKATSANFRAGE_WURDE_NOCH_NICHT_AUF_DIE_FIREWALL_GESCHRIEBEN"
ZERTIFIKAT_NICHT_GEFUNDEN = "ZERTIFIKAT_NICHT_GEFUNDEN"
ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN = "ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN"
ZIEL_HOST_NICHT_GEFUNDEN = "ZIEL_HOST_NICHT_GEFUNDEN"
OUTBOUND_MODE_UNREADABLE = "OUTBOUND_MODE_UNREADABLE"
OUTBOUND_MODE_BLOCKS_PUSH = "OUTBOUND_MODE_BLOCKS_PUSH"
GROUP_CANNOT_BE_OWN_PARENT = "GROUP_CANNOT_BE_OWN_PARENT"
GROUP_HIERARCHY_CYCLE = "GROUP_HIERARCHY_CYCLE"
OPEN_DEPLOYMENT_DRAFT_UNCHANGED = "OPEN_DEPLOYMENT_DRAFT_UNCHANGED"
OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY = "OPEN_HOST_MAINTENANCE_BLOCKS_DEPLOY"
OPEN_RESTORE_BLOCKS_DEPLOY = "OPEN_RESTORE_BLOCKS_DEPLOY"
READ_ONLY_ACCESS = "READ_ONLY_ACCESS"
def api_error(status_code: int, code: ErrorCode, **params: Any) -> HTTPException:
return HTTPException(status_code=status_code, detail={"code": code.value, "params": params})
backend/app/legacy_schema_patches.py
+69
View File
@@ -0,0 +1,69 @@
"""EINGEFROREN — nur noch die Brücke für Installationen aus der Zeit vor Alembic.
Bis 1.8.x hat OPNexus Schema-Änderungen an bestehenden Tabellen über diese
idempotenten `ALTER TABLE ... ADD COLUMN IF NOT EXISTS`-Statements nach
`create_all()` nachgezogen. Ab 1.9.0 verwaltet Alembic das Schema (siehe
migrations/ und app/migrate.py); diese Datei wird nur noch ein einziges Mal
ausgeführt: beim ersten Start einer Datenbank, die Tabellen, aber noch keine
`alembic_version` hat. Danach wird sie auf Revision 0001 gestempelt.
REGELN:
- NIE wieder einen Eintrag hinzufügen. Neue Schema-Änderungen sind eine
Alembic-Migration: `alembic revision --autogenerate -m "..."`.
- Keinen Eintrag ändern oder entfernen: die Brücke muss jede Datenbank von
1.2.0 bis 1.8.x auf das 1.8.1-Schema bringen.
- Die Brücke ruft `create_all()` mit den AKTUELLEN Modellen auf, um fehlende
Tabellen anzulegen. Das ist nur korrekt, solange diese Tabellen dem
Baseline-Schema entsprechen. Installationen älter als 1.8.1 müssen deshalb
erst auf 1.9.x aktualisiert werden, bevor sie ein späteres Release
überspringen (siehe README, "Upgrade").
"""
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncConnection
_STATEMENTS: tuple[str, ...] = (
# 2026-09-08 — cluster identity for CARP-aware fleet grouping.
"ALTER TABLE opnsense_hosts ADD COLUMN IF NOT EXISTS cluster_name VARCHAR(120)",
# 2026-09-08 — deploy lockout / auto-revert protection for rule pushes.
"ALTER TABLE rule_deployments ADD COLUMN IF NOT EXISTS revert_deadline TIMESTAMPTZ",
"ALTER TABLE rule_deployments ADD COLUMN IF NOT EXISTS previous_payload JSON",
"ALTER TABLE rule_deployments ADD COLUMN IF NOT EXISTS last_payload JSON",
# 2026-09-08 — hierarchical policy grouping, one column per rule-shaped
# table (host + the four alias/NAT tables that can belong to a group).
"ALTER TABLE opnsense_hosts ADD COLUMN IF NOT EXISTS policy_hierarchy_group_id UUID REFERENCES policy_hierarchy_groups(id) ON DELETE SET NULL",
"ALTER TABLE firewall_aliases ADD COLUMN IF NOT EXISTS policy_hierarchy_group_id UUID REFERENCES policy_hierarchy_groups(id) ON DELETE SET NULL",
"ALTER TABLE port_forward_rules ADD COLUMN IF NOT EXISTS policy_hierarchy_group_id UUID REFERENCES policy_hierarchy_groups(id) ON DELETE SET NULL",
"ALTER TABLE one_to_one_rules ADD COLUMN IF NOT EXISTS policy_hierarchy_group_id UUID REFERENCES policy_hierarchy_groups(id) ON DELETE SET NULL",
"ALTER TABLE outbound_nat_rules ADD COLUMN IF NOT EXISTS policy_hierarchy_group_id UUID REFERENCES policy_hierarchy_groups(id) ON DELETE SET NULL",
# 2026-09-09 — per-session login security info shown post-login.
"ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS last_login_ip VARCHAR(45)",
# 2026-09-10 — role-based access control (admin / read_only). Default
# keeps every pre-existing account (until then, necessarily a single
# admin) as "admin" with no manual backfill needed.
"ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS role VARCHAR(20) NOT NULL DEFAULT 'admin'",
# 2026-09-15 — CARP health / NAT drift / CARP auto-revert alerting
# toggles on the single existing NotificationSettings row.
"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS carp_health_enabled BOOLEAN NOT NULL DEFAULT true",
"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS nat_drift_enabled BOOLEAN NOT NULL DEFAULT true",
"ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS carp_auto_revert_failed_enabled BOOLEAN NOT NULL DEFAULT true",
# 2026-09-17 — per-user UI language (German remains the default).
"ALTER TABLE user_preferences ADD COLUMN IF NOT EXISTS language VARCHAR(5) NOT NULL DEFAULT 'de'",
# 2026-10-04 — size cap for stored firewall logs (GB, 0 = unlimited).
# 2026-10-04 — optional TOTP second factor.
"ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS totp_enabled BOOLEAN NOT NULL DEFAULT false",
"ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS totp_last_step BIGINT NOT NULL DEFAULT 0",
"ALTER TABLE admin_users ADD COLUMN IF NOT EXISTS totp_recovery_hashes JSON",
# 2026-10-04 — pinned TLS certificate per firewall.
"ALTER TABLE opnsense_hosts ADD COLUMN IF NOT EXISTS tls_ca_pem TEXT",
"ALTER TABLE log_settings ADD COLUMN IF NOT EXISTS max_size_gb DOUBLE PRECISION NOT NULL DEFAULT 10",
)
async def apply(conn: AsyncConnection) -> None:
"""Run every statement, in order, inside the caller's connection/
transaction. Postgres DDL is transactional, so a failure partway
through rolls back cleanly rather than leaving the schema half-patched.
"""
for statement in _STATEMENTS:
await conn.execute(text(statement))
backend/app/main.py
+357
View File
@@ -0,0 +1,357 @@
import asyncio
import logging
import re
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app import models # noqa: F401 - registers models with Base.metadata
from app.auth_middleware import AuthMiddleware
from app.security_middleware import OriginCheckMiddleware, SecurityHeadersMiddleware
from app.config import settings
from app.database import SessionLocal
from app import migrate
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router
from app.routers import one_to_one
from app.services.one_to_one_deploy_service import sweep_expired_one_to_one_deployments
from app.routers import outbound_nat
from app.services.outbound_nat_deploy_service import sweep_expired_outbound_nat_deployments
from app.services import auth_service, audit_service
from app.routers import audit, maintenance, settings as settings_router, notifications as notifications_router
from app.routers import system as system_router
from app.services.rule_deploy_service import sweep_expired_deployments
from app.services.alias_deploy_service import sweep_expired_alias_deployments
from app.services.port_forward_deploy_service import sweep_expired_port_forward_deployments
from app.services import notification_service, notification_settings_service
from app.routers import policy_hierarchy, policy_hierarchy_groups
from app.services.policy_hierarchy_deploy_service import sweep_expired_hierarchy_deployments
from app.services import log_ingest_service, log_settings_service
from app.routers import ipsec as ipsec_router
from app.routers import openvpn as openvpn_router
from app.routers import wireguard as wireguard_router
from app.routers import certificates as certificates_router
from app.routers import restore as restore_router
from app.version import APP_VERSION
from app.routers import dashboard as dashboard_router
from app.routers import cluster as cluster_router
from app.routers import services as services_router
from app.routers import resource_samples as resource_samples_router
from app.services import resource_monitor_service
from app.routers import scheduled_maintenance as scheduled_maintenance_router
from app.services import scheduled_maintenance_service
from app.services import carp_event_service, carp_maintenance_window_service
# Without this, Python's root logger defaults to WARNING with no handler,
# so every logger.info() call in the 5 background loops below (CARP role
# changes, log-ingest counts, sweep results, ...) is silently dropped —
# `docker compose logs backend` then shows only uvicorn's own access log,
# even after days of the loops actually running. Only warning/exception
# calls survived before this (e.g. the one-time generated-admin-password
# banner), which made debugging "is X even running" needlessly hard.
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
_SWEEP_INTERVAL_SECONDS = 10
# Prune retention-expired log rows once every N poll cycles rather than
# every tick — cleanup has no reason to run at poll cadence (default
# 30s), and a DELETE scanning by timestamp on every cycle would be wasted
# work once the table is non-trivially sized.
_LOG_PRUNE_EVERY_N_CYCLES = 20
# Same rationale as _LOG_PRUNE_EVERY_N_CYCLES, but the resource-sampling
# loop's own cadence (settings.resource_poll_interval_seconds, default 60s).
_RESOURCE_PRUNE_EVERY_N_CYCLES = 20
async def _deploy_revert_sweep_loop():
"""Background safety net for the deploy lockout protection: reverts any
RuleDeployment whose confirm window passed without a human confirming
(see rule_deploy_service.sweep_expired_deployments)."""
while True:
try:
async with SessionLocal() as db:
reverted = await sweep_expired_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed rule deployment(s)", len(reverted))
except Exception:
logger.exception("Deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await sweep_expired_alias_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed alias deployment(s)", len(reverted))
except Exception:
logger.exception("Alias deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await sweep_expired_port_forward_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed port-forward deployment(s)", len(reverted))
except Exception:
logger.exception("Port-forward deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await sweep_expired_outbound_nat_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed outbound-nat deployment(s)", len(reverted))
except Exception:
logger.exception("Outbound NAT deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await sweep_expired_one_to_one_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed one-to-one deployment(s)", len(reverted))
except Exception:
logger.exception("1:1 NAT deploy-revert sweep failed")
try:
async with SessionLocal() as db:
reverted = await carp_maintenance_window_service.sweep_expired(db)
if reverted:
logger.info("CARP-Auto-Revert: %d Zeitfenster verarbeitet", len(reverted))
except Exception:
logger.exception("CARP-Auto-Revert-Sweep fehlgeschlagen")
try:
async with SessionLocal() as db:
reverted = await sweep_expired_hierarchy_deployments(db)
if reverted:
logger.info("Auto-reverted %d unconfirmed policy-hierarchy deployment(s)", len(reverted))
except Exception:
logger.exception("Policy-hierarchy deploy-revert sweep failed")
await asyncio.sleep(_SWEEP_INTERVAL_SECONDS)
async def _notification_poll_loop():
"""Separate, slower loop from the deploy-revert sweep above: this one
makes live HTTP calls to every real firewall per cycle (host status,
certificates, live rules for drift), so it must not run at the sweep's
10s cadence — that cadence is a security control for the deploy-confirm
window, not applicable here. Interval is admin-configurable via
NotificationSettings.poll_interval_seconds, floored by
settings.notification_poll_min_interval_seconds."""
while True:
interval = settings.notification_poll_min_interval_seconds
try:
async with SessionLocal() as db:
await notification_service.run_notification_checks(db)
row = await notification_settings_service.get_settings(db)
interval = max(row.poll_interval_seconds, settings.notification_poll_min_interval_seconds)
except Exception:
logger.exception("Notification poll failed")
try:
async with SessionLocal() as db:
events = await carp_event_service.detect_and_record(db)
for event in events:
logger.info(
"CARP-Rollenwechsel auf %s: %s -> %s", event.host_name, event.from_status, event.to_status,
)
except Exception:
logger.exception("CARP-Ereignis-Erkennung fehlgeschlagen")
await asyncio.sleep(interval)
async def _log_ingest_poll_loop():
"""Third background loop, mirroring _notification_poll_loop's shape:
a live GET call (diagnostics/firewall/log) to every enabled host per
cycle, so it runs on its own admin-configurable cadence rather than
the 10s deploy-revert sweep's. Interval comes from
LogSettings.poll_interval_seconds, floored by
settings.log_poll_min_interval_seconds. Retention pruning runs only
every _LOG_PRUNE_EVERY_N_CYCLES cycles (see constant above)."""
cycle = 0
while True:
interval = settings.log_poll_min_interval_seconds
cycle += 1
try:
async with SessionLocal() as db:
row = await log_settings_service.get_settings(db)
interval = max(row.poll_interval_seconds, settings.log_poll_min_interval_seconds)
enabled = row.enabled
retention_days = row.retention_days
max_size_gb = row.max_size_gb
await db.commit()
if enabled:
result = await log_ingest_service.poll_all_hosts(db)
if result["inserted"]:
logger.info(
"Log-Ingest: %d neue Einträge über %d Host(s) (%d fehlgeschlagen)",
result["inserted"], result["hosts_polled"], result["hosts_failed"],
)
if cycle % _LOG_PRUNE_EVERY_N_CYCLES == 0:
deleted = await log_ingest_service.prune_old_entries(db, retention_days)
if deleted:
logger.info("Log-Retention: %d alte Einträge gelöscht", deleted)
trimmed = await log_ingest_service.enforce_size_limit(db, max_size_gb)
if trimmed:
logger.info("Log-Größenlimit (%.1f GB): %d älteste Einträge gelöscht", max_size_gb, trimmed)
except Exception:
logger.exception("Log-Ingest-Poll fehlgeschlagen")
await asyncio.sleep(interval)
async def _resource_poll_loop():
"""Fourth background loop, same shape as _log_ingest_poll_loop but with
a fixed interval (settings.resource_poll_interval_seconds) — v1
deliberately has no admin-editable settings row for this, see
resource_monitor_service.py's module docstring."""
cycle = 0
while True:
cycle += 1
try:
async with SessionLocal() as db:
result = await resource_monitor_service.poll_all_hosts(db)
if result["hosts_failed"]:
logger.warning(
"Resource-Sampling: %d von %d Host(s) fehlgeschlagen",
result["hosts_failed"], result["hosts_polled"],
)
if cycle % _RESOURCE_PRUNE_EVERY_N_CYCLES == 0:
deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
if deleted:
logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
except Exception:
logger.exception("Resource-Poll fehlgeschlagen")
await asyncio.sleep(settings.resource_poll_interval_seconds)
_SCHEDULED_MAINTENANCE_POLL_SECONDS = 60
async def _scheduled_maintenance_poll_loop():
"""Fifth background loop: fires due one-shot ScheduledMaintenance rows.
A fixed 60s cadence is plenty — these are human-picked timestamps, not
a sub-minute-precision cron."""
while True:
try:
async with SessionLocal() as db:
result = await scheduled_maintenance_service.fire_due(db)
if result["due"]:
logger.info(
"Geplante Wartung: %d fällig, %d ausgeführt, %d blockiert/fehlgeschlagen",
result["due"], result["fired"], result["blocked_or_failed"],
)
except Exception:
logger.exception("Geplante-Wartung-Poll fehlgeschlagen")
await asyncio.sleep(_SCHEDULED_MAINTENANCE_POLL_SECONDS)
@asynccontextmanager
async def lifespan(app: FastAPI):
await migrate.run()
async with SessionLocal() as db:
await auth_service.bootstrap_admin(db)
sweep_task = asyncio.create_task(_deploy_revert_sweep_loop())
notification_task = asyncio.create_task(_notification_poll_loop())
log_ingest_task = asyncio.create_task(_log_ingest_poll_loop())
resource_task = asyncio.create_task(_resource_poll_loop())
scheduled_maintenance_task = asyncio.create_task(_scheduled_maintenance_poll_loop())
try:
yield
finally:
sweep_task.cancel()
notification_task.cancel()
log_ingest_task.cancel()
resource_task.cancel()
scheduled_maintenance_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task):
try:
await task
except asyncio.CancelledError:
pass
app = FastAPI(title="OPNexus", version=APP_VERSION, lifespan=lifespan)
app.add_middleware(AuthMiddleware)
# Inside CORS so preflights are answered first; outside Auth so a forged
# cross-site write is rejected before any session lookup happens.
app.add_middleware(OriginCheckMiddleware)
app.add_middleware(SecurityHeadersMiddleware)
# Added last so it wraps outermost (Starlette runs the last-added
# middleware first) — CORS must see and answer OPTIONS preflight requests
# before AuthMiddleware ever gets a chance to 401 them.
# An explicit FRONTEND_ORIGIN (required for any non-localhost deployment,
# see config.py) is matched exactly, never as a regex fragment — avoids
# turning operator-supplied input into an accidental regex hole.
_cors_origin_regex = (
"^(" + "|".join(re.escape(o) for o in settings.allowed_origins) + ")$"
if settings.allowed_origins else settings.cors_origin_regex
)
app.add_middleware(
CORSMiddleware,
allow_origin_regex=_cors_origin_regex,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
# Browsers hide all but a few standard response headers cross-origin
# by default — the backup-download endpoint's filename (carried in
# Content-Disposition) needs this to reach the frontend's fetch().
expose_headers=["Content-Disposition"],
)
app.include_router(auth.router)
app.include_router(hosts.router, prefix="/api")
app.include_router(rules.router, prefix="/api")
app.include_router(search_router.router, prefix="/api")
app.include_router(users_router.router, prefix="/api")
app.include_router(logs.router, prefix="/api")
app.include_router(groups.router, prefix="/api")
app.include_router(aliases.router, prefix="/api")
app.include_router(port_forward.router, prefix="/api")
@app.get("/api/health")
async def health():
return {"status": "ok"}
app.include_router(one_to_one.router, prefix="/api")
app.include_router(outbound_nat.router, prefix="/api")
app.include_router(audit.router, prefix="/api")
app.include_router(maintenance.router, prefix="/api")
app.include_router(settings_router.router, prefix="/api")
app.include_router(system_router.router, prefix="/api")
app.include_router(notifications_router.router, prefix="/api")
app.include_router(policy_hierarchy_groups.router, prefix="/api")
app.include_router(policy_hierarchy.router, prefix="/api")
# No background sweep loop for this router — see
# ipsec_deploy_service.py's module docstring: there is no pending/
# auto-revert state here to sweep.
app.include_router(ipsec_router.router, prefix="/api")
# Same as above — no sweep loop, see openvpn_deploy_service.py's module
# docstring (identical no-pending/auto-revert rationale as IPsec).
app.include_router(openvpn_router.router, prefix="/api")
# Same as above — no sweep loop, see wireguard_deploy_service.py's module
# docstring (identical no-pending/auto-revert rationale as IPsec/OpenVPN).
app.include_router(wireguard_router.router, prefix="/api")
# Trust-Store + ACME certificate management — single-host entities (no
# multi-host targeting, see the models' own docstrings), no sweep loop
# either: Trust-Store has no restart/apply step to auto-revert at all, and
# ACME's register/sign are one-shot external-network triggers, not
# config-write-then-reconfigure flows with a "config live, reload
# unconfirmed" middle state to sweep.
app.include_router(certificates_router.managed_router, prefix="/api")
app.include_router(certificates_router.acme_accounts_router, prefix="/api")
app.include_router(certificates_router.acme_certificates_router, prefix="/api")
app.include_router(restore_router.router, prefix="/api")
app.include_router(dashboard_router.router, prefix="/api")
app.include_router(cluster_router.router, prefix="/api")
app.include_router(services_router.router, prefix="/api")
app.include_router(resource_samples_router.router, prefix="/api")
app.include_router(scheduled_maintenance_router.router, prefix="/api")
backend/app/migrate.py
+225
View File
@@ -0,0 +1,225 @@
"""Schema-Migrationen beim Start (Alembic, in-process).
Ablauf von `run()` — bei jedem Backend-Start:
1. Session-weiten Postgres-Advisory-Lock nehmen. Starten zwei Instanzen
gleichzeitig, migriert nur eine, die andere wartet und sieht dann "schon
aktuell".
2. Zustand der Datenbank bestimmen:
- **frisch** (keine Tabellen): `alembic upgrade head` baut alles auf.
- **Altinstallation** (Tabellen, aber keine `alembic_version`; alles bis
1.8.x): Sicherheits-Backup, dann `create_all()` + die eingefrorenen
Legacy-Patches, auf Revision 0001 stempeln, `upgrade head`.
- **Alembic-Installation**: nur wenn die Revision hinter `head` liegt,
Sicherheits-Backup und `upgrade head`.
3. Lock freigeben.
Das Backup vor einer Migration ist Pflicht: schlägt `pg_dump` fehl, wird
nicht migriert und der Start bricht mit klarer Meldung ab. Wer das bewusst
nicht will, setzt SKIP_MIGRATION_BACKUP=true.
Eine Datenbank, die schon von einer NEUEREN Version migriert wurde (unbekannte
Revision), wird nicht angefasst — der Start bricht ab, statt mit einem Schema
zu laufen, das der Code nicht kennt. Rollback = das Pre-Upgrade-Backup
einspielen oder die neuere Version wieder starten.
"""
import asyncio
import importlib
import logging
import os
import pkgutil
import re
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from alembic import command
from alembic.config import Config
from alembic.script import ScriptDirectory
from sqlalchemy import text
from sqlalchemy.engine import make_url
from sqlalchemy.ext.asyncio import AsyncConnection
import app.models
from app import legacy_schema_patches
from app.config import settings
from app.database import Base, engine
logger = logging.getLogger(__name__)
BACKEND_DIR = Path(__file__).resolve().parent.parent
BASELINE_REVISION = "0001"
# "OPNX" — beliebige, aber feste Zahl für den Advisory-Lock.
_LOCK_KEY = 0x4F504E58
# Hochvolumige, verzichtbare Betriebsdaten: nur das Schema sichern, sonst
# wird das Backup selbst zum Platzproblem (siehe docker-compose.yml,
# postgres-backup).
_BACKUP_SKIP_DATA = ("firewall_log_entries", "host_resource_samples")
_BACKUP_PREFIX = "pre-upgrade-"
_BACKUP_TIMEOUT_SECONDS = 1800
def load_models() -> None:
"""Jedes Modell-Modul importieren, sonst fehlen Tabellen in Base.metadata
(app/models/__init__.py importiert nicht alle)."""
for module in pkgutil.iter_modules(app.models.__path__):
importlib.import_module(f"app.models.{module.name}")
def _config() -> Config:
return Config(str(BACKEND_DIR / "alembic.ini"))
def _run_alembic(connection, action: str, revision: str) -> None:
"""Alembic-Kommando auf einer bereits offenen (sync-)Verbindung — env.py
nimmt sie aus config.attributes, damit alles in EINER Transaktion läuft."""
cfg = _config()
cfg.attributes["connection"] = connection
getattr(command, action)(cfg, revision)
async def _database_state(conn: AsyncConnection) -> tuple[bool, str | None]:
"""(hat_app_tabellen, aktuelle_alembic_revision oder None)."""
has_tables = (
await conn.execute(text("SELECT to_regclass('admin_users') IS NOT NULL"))
).scalar_one()
has_version = (
await conn.execute(text("SELECT to_regclass('alembic_version') IS NOT NULL"))
).scalar_one()
current = None
if has_version:
current = (await conn.execute(text("SELECT version_num FROM alembic_version"))).scalar()
return bool(has_tables), current
def _dump_to(target: Path) -> None:
url = make_url(settings.database_url)
partial = target.with_suffix(".partial")
cmd = [
"pg_dump",
"-h", url.host or "localhost",
"-p", str(url.port or 5432),
"-U", url.username or "",
"-d", url.database or "",
"-F", "c",
*(f"--exclude-table-data={table}" for table in _BACKUP_SKIP_DATA),
"-f", str(partial),
]
try:
subprocess.run(
cmd,
env={**os.environ, "PGPASSWORD": url.password or ""},
check=True,
capture_output=True,
text=True,
timeout=_BACKUP_TIMEOUT_SECONDS,
)
os.replace(partial, target)
finally:
partial.unlink(missing_ok=True)
def _pg_dump_major() -> int:
out = subprocess.run(["pg_dump", "--version"], check=True, capture_output=True, text=True).stdout
match = re.search(r"PostgreSQL\) (\d+)", out)
if not match:
raise RuntimeError(f"pg_dump-Version nicht erkennbar: {out.strip()!r}")
return int(match.group(1))
async def _ensure_pg_client_matches_server() -> None:
"""Ein pg_dump anderer Hauptversion als der Server schreibt Backups, die sich
nicht zurückspielen lassen (z. B. Client 17 → Server 16: `SET
transaction_timeout`). Lieber vor der Migration abbrechen als ein
Backup anlegen, auf das man sich nicht verlassen kann."""
async with engine.connect() as conn:
server_major = int((await conn.execute(text("SHOW server_version_num"))).scalar_one()) // 10000
client_major = await asyncio.to_thread(_pg_dump_major)
if client_major != server_major:
raise RuntimeError(
f"pg_dump im Backend-Image ist Version {client_major}, der Postgres-Server Version {server_major}. "
"Ein solches Backup ließe sich nicht zurückspielen. backend/scripts/install_pg_client.sh und "
"das postgres-Image in docker-compose.yml müssen dieselbe Hauptversion haben. Die Datenbank "
"wurde NICHT migriert (SKIP_MIGRATION_BACKUP=true überspringt Backup und diese Prüfung bewusst)."
)
def _prune_backups(directory: Path) -> None:
dumps = sorted(directory.glob(f"{_BACKUP_PREFIX}*.dump"), key=lambda p: p.stat().st_mtime)
for old in dumps[: max(0, len(dumps) - settings.migration_backup_keep)]:
old.unlink(missing_ok=True)
async def _backup_before_migration(from_revision: str, to_revision: str) -> None:
if settings.skip_migration_backup:
logger.warning("Pre-Upgrade-Backup übersprungen (SKIP_MIGRATION_BACKUP=true)")
return
directory = Path(settings.migration_backup_dir)
stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S")
target = directory / f"{_BACKUP_PREFIX}{from_revision}-to-{to_revision}-{stamp}.dump"
logger.info("Schema-Migration %s → %s: sichere Datenbank nach %s", from_revision, to_revision, target)
await _ensure_pg_client_matches_server()
try:
directory.mkdir(parents=True, exist_ok=True)
await asyncio.to_thread(_dump_to, target)
_prune_backups(directory)
except Exception as exc:
detail = exc.stderr.strip() if isinstance(exc, subprocess.CalledProcessError) and exc.stderr else repr(exc)
raise RuntimeError(
f"Pre-Upgrade-Backup nach {target} fehlgeschlagen ({detail}). Die Datenbank wurde "
"NICHT migriert. Backup-Ziel prüfen oder mit SKIP_MIGRATION_BACKUP=true bewusst überspringen."
) from exc
async def _migrate() -> None:
script = ScriptDirectory.from_config(_config())
head = script.get_current_head()
async with engine.connect() as conn:
has_tables, current = await _database_state(conn)
if current is not None:
try:
script.get_revision(current)
except Exception as exc:
raise RuntimeError(
f"Die Datenbank hat Schema-Revision {current}, die diese OPNexus-Version nicht kennt — "
"sie wurde von einer neueren Version migriert. Die neuere Version wieder starten oder "
"das Pre-Upgrade-Backup einspielen."
) from exc
if current is None and not has_tables:
logger.info("Leere Datenbank: lege Schema an (Revision %s)", head)
async with engine.begin() as conn:
await conn.run_sync(_run_alembic, "upgrade", "head")
return
if current is None:
await _backup_before_migration("legacy", head)
logger.info("Datenbank aus der Zeit vor Alembic erkannt: Legacy-Brücke, dann Migration auf %s", head)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
await legacy_schema_patches.apply(conn)
await conn.run_sync(_run_alembic, "stamp", BASELINE_REVISION)
await conn.run_sync(_run_alembic, "upgrade", "head")
return
if current == head:
logger.info("Datenbankschema ist aktuell (Revision %s)", head)
return
await _backup_before_migration(current, head)
logger.info("Migriere Datenbankschema %s → %s", current, head)
async with engine.begin() as conn:
await conn.run_sync(_run_alembic, "upgrade", "head")
async def run() -> None:
load_models()
async with engine.connect() as lock_conn:
await lock_conn.execute(text("SELECT pg_advisory_lock(:key)"), {"key": _LOCK_KEY})
try:
await _migrate()
finally:
await lock_conn.execute(text("SELECT pg_advisory_unlock(:key)"), {"key": _LOCK_KEY})
await lock_conn.commit()
backend/app/models/__init__.py
+95
View File
@@ -0,0 +1,95 @@
from app.models.host import OpnsenseHost
from app.models.rule_target import rule_targets
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.models.alias_target import alias_targets
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.models.port_forward_target import port_forward_targets
from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment
from app.models.user import AdminUser, Session
__all__ = [
"OpnsenseHost", "FirewallRule", "RuleDeployment", "rule_targets",
"FirewallAlias", "AliasDeployment", "alias_targets",
"PortForwardRule", "PortForwardDeployment", "port_forward_targets",
"AdminUser", "Session",
]
from app.models.policy import DeviceGroup, RuleRevision, AliasRevision, PortForwardRevision
from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.one_to_one_target import one_to_one_targets
from app.models.policy import OneToOneRevision
__all__ += ["OneToOneRule", "OneToOneDeployment", "one_to_one_targets", "OneToOneRevision"]
from app.models.outbound_nat import OutboundNatRule
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.models.outbound_nat_target import outbound_nat_targets
from app.models.policy import OutboundNatRevision
__all__ += ["OutboundNatRule", "OutboundNatDeployment", "outbound_nat_targets", "OutboundNatRevision"]
from app.models.audit import AuditEvent
__all__ += ["AuditEvent"]
from app.models.host_operation import HostOperation
__all__ += ["HostOperation"]
from app.models.preferences import UserPreferences
__all__ += ["UserPreferences"]
from app.models.notification import NotificationSettings, NotificationProblem
__all__ += ["NotificationSettings", "NotificationProblem"]
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.models.policy import PolicyHierarchyRuleRevision
__all__ += ["PolicyHierarchyGroup", "PolicyHierarchyRule", "PolicyHierarchyRuleDeployment", "PolicyHierarchyRuleRevision"]
from app.models.log_entry import FirewallLogEntry
from app.models.log_cursor import HostLogCursor
from app.models.log_settings import LogSettings
__all__ += ["FirewallLogEntry", "HostLogCursor", "LogSettings"]
from app.models.ipsec_tunnel_target import ipsec_tunnel_targets
from app.models.ipsec_tunnel import IpsecTunnel
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.models.policy import ipsec_tunnel_groups, IpsecTunnelRevision
from app.models.openvpn_instance_target import openvpn_instance_targets
from app.models.openvpn_instance import OpenVpnInstance
from app.models.openvpn_instance_deployment import OpenVpnInstanceDeployment
from app.models.policy import openvpn_instance_groups, OpenVpnInstanceRevision
__all__ += ["IpsecTunnel", "IpsecTunnelDeployment", "ipsec_tunnel_targets", "ipsec_tunnel_groups", "IpsecTunnelRevision"]
__all__ += ["OpenVpnInstance", "OpenVpnInstanceDeployment", "openvpn_instance_targets", "openvpn_instance_groups", "OpenVpnInstanceRevision"]
from app.models.wireguard_tunnel_target import wireguard_tunnel_targets
from app.models.wireguard_tunnel import WireguardTunnel
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
from app.models.policy import wireguard_tunnel_groups, WireguardTunnelRevision
__all__ += ["WireguardTunnel", "WireguardTunnelDeployment", "wireguard_tunnel_targets", "wireguard_tunnel_groups", "WireguardTunnelRevision"]
# Trust-Store + ACME certificate management — deliberately per-host (plain
# host_id FK, no DeviceGroup/effective_hosts targeting) — see
# ManagedCertificate/AcmeAccount/AcmeCertificateRequest's own docstrings
# for why.
from app.models.managed_certificate import ManagedCertificate
from app.models.acme_account import AcmeAccount
from app.models.acme_certificate_request import AcmeCertificateRequest
__all__ += ["ManagedCertificate", "AcmeAccount", "AcmeCertificateRequest"]
from app.models.restore_operation import RestoreOperation
__all__ += ["RestoreOperation"]
from app.models.carp_event import CarpEvent
from app.models.carp_maintenance_window import CarpMaintenanceWindow
__all__ += ["CarpEvent", "CarpMaintenanceWindow"]
from app.models.resource_sample import HostResourceSample
__all__ += ["HostResourceSample"]
from app.models.scheduled_maintenance import ScheduledMaintenance
__all__ += ["ScheduledMaintenance"]
backend/app/models/acme_account.py
+76
View File
@@ -0,0 +1,76 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class AcmeAccount(Base):
"""A centrally-authored ACME account (`os-acme-client` plugin,
`AcmeClient.xml` -> `accounts.account`, `AccountsController.php`) —
per-host like ManagedCertificate, same reasoning: the account's
OPNsense-assigned uuid (referenced by AcmeCertificateRequest.account at
deploy time — see acme_service.py's UUID-injection, mirroring
ipsec_deploy_service.py's `connection` UUID pattern) is host-local.
Field names verified directly against AcmeClient.xml (fetched
2026-09-09): `ca` OptionValues are letsencrypt/letsencrypt_test/zerossl/
buypass/buypass_test/google/google_test/sslcom/custom (Default
"letsencrypt"). `custom_ca` only applies when ca="custom".
`eab_hmac` is sensitive External-Account-Binding material — like
ManagedCertificate's private key, it NEVER touches Postgres as
plaintext, only disk (see acme_secret_store.py, mirroring
ipsec_psk_store.py). `eab_hmac_set` tracks whether a value has been
provided. `eab_kid` is a plain (non-secret) key identifier, stored
directly.
Local add/set (create/edit) is a purely local config write, same
"no restart, no lockout risk" status model as ManagedCertificate
(status: draft/success/error). The SEPARATE `register` action
(AccountsController::registerAction(), `POST
/acmeclient/accounts/register/{uuid}`) is a REAL, blocking network
call to the configured external CA — this is what needs the two-step
UI warning gate (see AcmeActionConfirm.tsx / README), not the local
add/set. register_status/register_message/register_at record the last
attempt's outcome for display; they are NOT re-derived from OPNsense
(there is no "is this account registered" flag on the model itself to
poll — see acme_service.py)."""
__tablename__ = "acme_accounts"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(255))
description: Mapped[str] = mapped_column(String(255), default="")
email: Mapped[str] = mapped_column(String(255), default="")
ca: Mapped[str] = mapped_column(String(32), default="letsencrypt")
custom_ca: Mapped[str] = mapped_column(String(255), default="")
eab_kid: Mapped[str] = mapped_column(String(255), default="")
eab_hmac_set: Mapped[bool] = mapped_column(default=False)
enabled: Mapped[bool] = mapped_column(default=True)
opnsense_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
status: Mapped[str] = mapped_column(String(16), default="draft")
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
# Excludes eab_hmac's actual value (see acme_secret_store.py) — a
# boolean flag stands in for it so no-op-push detection still notices
# when the secret changes without ever persisting it.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
deployed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
register_status: Mapped[str | None] = mapped_column(String(16), nullable=True)
register_message: Mapped[str | None] = mapped_column(String(2000), nullable=True)
register_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), onupdate=lambda: datetime.now(timezone.utc)
)
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/acme_certificate_request.py
+93
View File
@@ -0,0 +1,93 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class AcmeCertificateRequest(Base):
"""A centrally-authored ACME certificate request (`AcmeClient.xml` ->
`certificates.certificate`, `CertificatesController.php`) — per-host,
same reasoning as AcmeAccount/ManagedCertificate.
Field names verified directly against AcmeClient.xml (fetched
2026-09-09): `name` is the CN/FQDN (Required). `altNames` is a
CSVListField (comma-separated on the wire) — modeled here as a plain
`alt_names` string, NOT a sub-array, matching OPNsense's own
convention. `account` and `validationMethod` are BOTH Required
ModelRelationFields on the wire — `account` is filled in by THIS app
at deploy time (injecting the linked AcmeAccount's own per-host
OPNsense uuid — see acme_service.py's build_payload(), the same
UUID-injection pattern ipsec_deploy_service.py documents for its own
`connection` field, including the SAME no-op-redeploy pitfall: the
injected uuid must never be baked into the dict compared/stored as
`last_payload`). `validationMethod` (this app's `validation_ref`) is
plain, OPNexus-unvalidated: the plan is explicit that DNS-provider
validation config (`validations.validation`) is entirely out of v1
scope — the operator configures it themselves in OPNsense's own GUI
and types the resulting UUID in here; OPNexus never validates the
UUID's content, only that a target host actually has one selected
(same trust model IpsecTunnel's local_address/remote_address use).
`keyLength` OptionValues are PREFIXED (`key_2048`/`key_3072`/
`key_4096`/`key_ec256`/`key_ec384`), NOT bare numbers — a real
field-shape trap the plan's prose summary ("2048/3072/4096/ec256/
ec384") would have produced a silently-rejected payload for if not
re-verified against the model XML directly.
`renew_interval` (IntegerField, Default 60): DAYS SINCE LAST
ISSUANCE, NOT days-before-expiry (verified in the plan against
AcmeClient's own cron semantics) — this is the exact semantic trap
the plan calls out; see the UI label in AcmeCertificateForm.tsx and
the README for the same warning surfaced to the operator.
Local add/set is a purely local config write (status:
draft/success/error, no restart/lockout risk — same model as
ManagedCertificate/AcmeAccount). The separate `sign` action
(`POST /acmeclient/certificates/sign/{uuid}`) is asynchronous
(configd forks and returns immediately) AND reaches the real external
CA — sign_status/sign_triggered_at record only that a trigger was
sent, never a claimed outcome; remote_status_code/
remote_status_last_update/remote_last_update are populated by polling
`GET /acmeclient/certificates/get/{uuid}` afterwards (see
acme_service.py's poll_certificate_status())."""
__tablename__ = "acme_certificate_requests"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(255))
alt_names: Mapped[str] = mapped_column(String(2000), default="")
account_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("acme_accounts.id"))
validation_ref: Mapped[str] = mapped_column(String(64))
key_length: Mapped[str] = mapped_column(String(16), default="key_4096")
ocsp: Mapped[bool] = mapped_column(default=False)
auto_renewal: Mapped[bool] = mapped_column(default=True)
renew_interval: Mapped[int] = mapped_column(Integer, default=60)
enabled: Mapped[bool] = mapped_column(default=True)
opnsense_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
status: Mapped[str] = mapped_column(String(16), default="draft")
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
deployed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
sign_status: Mapped[str | None] = mapped_column(String(16), nullable=True)
sign_message: Mapped[str | None] = mapped_column(String(2000), nullable=True)
sign_triggered_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
remote_status_code: Mapped[int | None] = mapped_column(Integer, nullable=True)
remote_status_last_update: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
remote_last_update: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), onupdate=lambda: datetime.now(timezone.utc)
)
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
account: Mapped["AcmeAccount"] = relationship("AcmeAccount")
backend/app/models/alias.py
+51
View File
@@ -0,0 +1,51 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, String, Text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.alias_target import alias_targets
from app.models.policy import alias_groups
class FirewallAlias(Base):
"""A centrally-authored alias (object), pushed to hosts the same way
FirewallRule is — see services/alias_deploy_service.py. Editing an
alias's content changes what every rule referencing it enforces on
every host it reaches, so it goes through the same
draft/preview/push/confirm/auto-rollback safety net as rules, not an
immediate write.
"""
__tablename__ = "firewall_aliases"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(64))
# OPNsense alias types this app manages centrally — the common,
# user-editable ones. "internal" (auto-generated __if_network) and
# "external" (built-ins like bogons/sshlockout) aren't created here.
type: Mapped[str] = mapped_column(String(32))
# Newline-separated entries, matching OPNsense's own wire format for
# this field (verified live via GET .../alias/searchItem).
content: Mapped[str] = mapped_column(Text, default="")
description: Mapped[str] = mapped_column(String(255), default="")
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
policy_hierarchy_group_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="SET NULL"), nullable=True
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=alias_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=alias_groups)
deployments: Mapped[list["AliasDeployment"]] = relationship(
"AliasDeployment", back_populates="alias", cascade="all, delete-orphan"
)
backend/app/models/alias_deployment.py
+38
View File
@@ -0,0 +1,38 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class AliasDeployment(Base):
"""Last known deploy result of one alias on one host — mirrors
RuleDeployment exactly (see that model for the status meanings);
opnsense_alias_uuid is the UUID OPNsense assigned this alias on THAT
host, needed to target setItem/delItem there later."""
__tablename__ = "alias_deployments"
__table_args__ = (UniqueConstraint("alias_id", "host_id", name="uq_alias_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
alias_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("firewall_aliases.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_alias_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
alias: Mapped["FirewallAlias"] = relationship("FirewallAlias", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/alias_target.py
+12
View File
@@ -0,0 +1,12 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
alias_targets = Table(
"alias_targets",
Base.metadata,
Column("alias_id", PGUUID(as_uuid=True), ForeignKey("firewall_aliases.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/audit.py
+32
View File
@@ -0,0 +1,32 @@
"""Append-only history, deliberately no foreign keys: deletion keeps evidence."""
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, String, JSON, Text, event, DDL
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class AuditEvent(Base):
__tablename__ = "audit_events"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), index=True)
correlation_id: Mapped[str] = mapped_column(String(36), index=True)
actor: Mapped[str] = mapped_column(String(150), index=True)
kind: Mapped[str] = mapped_column(String(32), index=True)
action: Mapped[str] = mapped_column(String(40), index=True)
status: Mapped[str] = mapped_column(String(32), index=True)
entity_id: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
entity_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
host_id: Mapped[str | None] = mapped_column(String(36), nullable=True, index=True)
host_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
message: Mapped[str | None] = mapped_column(Text, nullable=True)
details: Mapped[dict] = mapped_column(JSON, default=dict)
event.listen(AuditEvent.__table__, "after_create", DDL("""
CREATE OR REPLACE FUNCTION opnexus_audit_immutable() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN RAISE EXCEPTION 'Audit history is append-only'; END $$;
""").execute_if(dialect="postgresql"))
event.listen(AuditEvent.__table__, "after_create", DDL("""
CREATE TRIGGER audit_events_immutable BEFORE UPDATE OR DELETE ON audit_events
FOR EACH ROW EXECUTE FUNCTION opnexus_audit_immutable();
""").execute_if(dialect="postgresql"))
backend/app/models/carp_event.py
+31
View File
@@ -0,0 +1,31 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class CarpEvent(Base):
"""One row per observed CARP-role transition on a host, e.g. BACKUP -> MASTER.
Populated by carp_event_service.detect_and_record, called from main.py's
existing _notification_poll_loop. Deliberately has no "current state"
column on OpnsenseHost to diff against — the previous state is read back
from this table's own latest row per host, so no manual ALTER is needed
on an existing table for this feature.
"""
__tablename__ = "carp_events"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), index=True)
host_name: Mapped[str] = mapped_column(String(120))
cluster_name: Mapped[str | None] = mapped_column(String(120), nullable=True, index=True)
from_status: Mapped[str | None] = mapped_column(String(20), nullable=True)
to_status: Mapped[str | None] = mapped_column(String(20), nullable=True)
detected_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), index=True
)
backend/app/models/carp_maintenance_window.py
+33
View File
@@ -0,0 +1,33 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class CarpMaintenanceWindow(Base):
"""Tracks an optional auto-revert timer on top of the existing CARP
maintenance-mode toggle (opnsense_client.toggle_carp_maintenance).
One open ("active") row per host at most. The sweep in
carp_maintenance_sweep_service checks for rows past auto_revert_at and
calls the same toggle used by a manual "CARP-Normalbetrieb herstellen"
click. status="confirmed_permanent" means the user chose no auto-revert
(or explicitly cancelled the timer) — the row is kept for history rather
than deleted.
"""
__tablename__ = "carp_maintenance_windows"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), index=True)
host_name: Mapped[str] = mapped_column(String(120))
entered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
auto_revert_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# active | confirmed_permanent | reverted | revert_failed
status: Mapped[str] = mapped_column(String(20), default="active", index=True)
message: Mapped[str | None] = mapped_column(String(1000), nullable=True)
resolved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
backend/app/models/host.py
+46
View File
@@ -0,0 +1,46 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, String, Text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class OpnsenseHost(Base):
"""A managed OPNsense firewall. The API secret never lives here — see
services/opnsense_secret_store.py, which keeps it on disk only."""
__tablename__ = "opnsense_hosts"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(120), unique=True)
base_url: Mapped[str] = mapped_column(String(255))
api_key: Mapped[str] = mapped_column(String(255))
verify_tls: Mapped[bool] = mapped_column(Boolean, default=True)
# Pinned certificate (or CA) of the firewall, PEM. When set together with
# verify_tls, only this certificate is trusted — see tls_pin_service.
tls_ca_pem: Mapped[str | None] = mapped_column(Text, nullable=True)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
# Free-text grouping label — hosts sharing the same value are shown
# together on the cluster view (e.g. an HA pair). Not a separate
# entity/table on purpose: a host belongs to at most one such group,
# and there's no cluster-level data to manage beyond the name itself.
cluster_name: Mapped[str | None] = mapped_column(String(120), nullable=True, index=True)
# A host belongs to at most one policy-hierarchy group (see
# models/policy_hierarchy_group.py for why this is nullable+singular
# rather than the DeviceGroup multi-membership pattern). NOT created by
# Base.metadata.create_all() on an existing table — see README for the
# required manual ALTER TABLE step, following the cluster_name /
# deploy-lockout precedent.
policy_hierarchy_group_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="SET NULL"), nullable=True
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
policy_hierarchy_group: Mapped["PolicyHierarchyGroup | None"] = relationship(
"PolicyHierarchyGroup", back_populates="hosts"
)
backend/app/models/host_operation.py
+21
View File
@@ -0,0 +1,21 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, String, JSON
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostOperation(Base):
__tablename__ = "host_operations"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), index=True)
host_name: Mapped[str] = mapped_column(String(120))
actor: Mapped[str] = mapped_column(String(150))
action: Mapped[str] = mapped_column(String(20))
status: Mapped[str] = mapped_column(String(20), default="preview")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
fingerprint: Mapped[str] = mapped_column(String(64))
snapshot: Mapped[dict] = mapped_column(JSON)
message: Mapped[str | None] = mapped_column(String(1000), nullable=True)
backend/app/models/ipsec_tunnel.py
+79
View File
@@ -0,0 +1,79 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.ipsec_tunnel_target import ipsec_tunnel_targets
from app.models.policy import ipsec_tunnel_groups
class IpsecTunnel(Base):
"""A centrally-authored site-to-site IPsec tunnel (PSK auth only — v1
scope, see README/plan). Hides 5 linked OPNsense objects (Swanctl
Connection, Local, Remote, Child, and IPsec.preSharedKeys.preSharedKey)
behind one composite form, mirroring how an admin actually thinks about
"a VPN tunnel to site X" rather than five separate model array items —
see services/ipsec_deploy_service.py for exactly how each field maps.
Targeting mirrors FirewallRule exactly (direct host targets + DeviceGroup
membership, resolved via policy_service.effective_hosts) — the classic
use case is pushing the identical tunnel config to both HA cluster
members (CARP VIP as local endpoint).
The pre-shared-key SECRET itself is never stored here — like
OpnsenseHost's OPNsense API secret (see opnsense_secret_store.py), it
lives only on disk, one file per tunnel id (see ipsec_psk_store.py).
pre_shared_key_set just tracks whether one has been provided yet.
Deploy status model is deliberately simpler than FirewallRule's — see
services/ipsec_deploy_service.py's module docstring for why (no
pending/confirm/auto-revert window here, on purpose)."""
__tablename__ = "ipsec_tunnels"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
# Connection.local_addrs / remote_addrs (IKEAddressField) — v1 keeps
# this to a single IP/hostname per side (no comma-separated list in the
# form), matching the "simple 1:1 network-to-network tunnel" v1 scope.
local_address: Mapped[str] = mapped_column(String(255))
remote_address: Mapped[str] = mapped_column(String(255))
# Connection.version: "ike" (0, IKEv1+IKEv2), "ikev1" (1), "ikev2" (2).
ike_version: Mapped[str] = mapped_column(String(8), default="ikev2")
# Connection.proposals (IPsecProposalField, phase 1). "default" is
# OPNsense's own model default (Swanctl.xml <Default>default</Default>)
# — the conservative choice documented in ipsec_deploy_service.py.
proposal: Mapped[str] = mapped_column(String(128), default="default")
# locals.local.id (this side's IKE identity) — also becomes
# preSharedKeys.preSharedKey.ident. Required: PSK lookup keys off it.
local_identity: Mapped[str] = mapped_column(String(255))
# remotes.remote.id — also becomes preSharedKeys.preSharedKey.remote_ident.
# Optional in the OPNsense model (unlike local id); default "".
remote_identity: Mapped[str] = mapped_column(String(255), default="")
pre_shared_key_set: Mapped[bool] = mapped_column(Boolean, default=False)
# children.child.local_ts / remote_ts (NetworkField, AsList=Y) —
# comma-separated CIDRs; v1 form still just takes a single CIDR each,
# matching the "no pools / wildcard traffic selectors" v1 scope.
local_networks: Mapped[str] = mapped_column(String(255))
remote_networks: Mapped[str] = mapped_column(String(255))
# children.child.esp_proposals (IPsecProposalField, phase 2). "default"
# for the same reason as `proposal` above.
esp_proposal: Mapped[str] = mapped_column(String(128), default="default")
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=ipsec_tunnel_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=ipsec_tunnel_groups)
deployments: Mapped[list["IpsecTunnelDeployment"]] = relationship(
"IpsecTunnelDeployment", back_populates="tunnel", cascade="all, delete-orphan"
)
backend/app/models/ipsec_tunnel_deployment.py
+60
View File
@@ -0,0 +1,60 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class IpsecTunnelDeployment(Base):
"""Last known deploy result of one tunnel on one host. The five
opnsense_*_uuid columns are the per-host UUIDs OPNsense assigned to
this tunnel's Connection/Local/Remote/Child/PreSharedKey objects —
required to target setX/delX there later, exactly like
RuleDeployment.opnsense_rule_uuid.
status: "success" (all 5 objects written AND the follow-up reconfigure
call was confirmed) or "error" (something failed — see `message`).
Deliberately NO "pending"/"reverted"/"revert_failed" here — see
services/ipsec_deploy_service.py's module docstring for why this
entity has no auto-revert safety net (an IPsec reconfigure can cut off
the very management path OPNexus would need to auto-revert it).
A host with no IpsecTunnelDeployment row at all is this tunnel's
"draft" state on that host (never pushed there) — surfaced by
ipsec_service.preview()'s action="create" for that row, not by a
persisted status value. This mirrors how RuleDeployment rows also
don't exist until a rule's first deploy attempt.
"""
__tablename__ = "ipsec_tunnel_deployments"
__table_args__ = (UniqueConstraint("tunnel_id", "host_id", name="uq_ipsec_tunnel_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
tunnel_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("ipsec_tunnels.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_connection_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_local_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_remote_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_child_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_psk_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
# The OPNsense-shaped payloads (keyed "connection"/"local"/"remote"/
# "child"/"psk") last successfully written to this host — compared
# against a fresh deploy's payloads to detect a true no-op re-deploy.
# Unlike RuleDeployment there is deliberately no previous_payload:
# nothing here is ever restored automatically.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
tunnel: Mapped["IpsecTunnel"] = relationship("IpsecTunnel", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/ipsec_tunnel_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
ipsec_tunnel_targets = Table(
"ipsec_tunnel_targets",
Base.metadata,
Column("tunnel_id", PGUUID(as_uuid=True), ForeignKey("ipsec_tunnels.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/log_cursor.py
+26
View File
@@ -0,0 +1,26 @@
"""Per-host log-ingest cursor. A separate table rather than a column on
`opnsense_hosts` — deliberately, so this brand-new feature needs no manual
`ALTER TABLE` on the existing table for already-running installs (see
host.py's `policy_hierarchy_group_id` comment for the alternative that DID
need one). `Base.metadata.create_all()` alone is enough since this whole
table is new.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostLogCursor(Base):
__tablename__ = "host_log_cursor"
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True
)
# None until the first successful poll of this host.
last_digest: Mapped[str | None] = mapped_column(String(64), nullable=True)
last_polled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
backend/app/models/log_entry.py
+52
View File
@@ -0,0 +1,52 @@
"""Persisted firewall log entries, ingested incrementally by
log_ingest_service.py via OPNsense's digest-based cursor (`diagnostics/
firewall/log`'s `digest` param — verified against core's
scripts/filter/read_log.py, see opnsense_client.get_log).
Counter-example to models/audit.py on purpose: audit_events is made
immutable (DB trigger blocks UPDATE/DELETE) because it's an audit trail.
This table needs the OPPOSITE property — retention pruning
(log_ingest_service.prune_old_entries) must be able to delete old rows,
so no such trigger is added here.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Index, JSON, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class FirewallLogEntry(Base):
__tablename__ = "firewall_log_entries"
__table_args__ = (
# "unique je Host" per the design — a digest is only guaranteed
# unique within one host's own log stream, and this constraint is
# also what the ingest service's ON CONFLICT DO NOTHING dedupe
# keys off of.
UniqueConstraint("host_id", "digest", name="uq_firewall_log_entry_host_digest"),
Index("ix_firewall_log_entries_host_timestamp", "host_id", "timestamp"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
timestamp: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
action: Mapped[str | None] = mapped_column(String(32), nullable=True, index=True)
interface: Mapped[str | None] = mapped_column(String(64), nullable=True, index=True)
direction: Mapped[str | None] = mapped_column(String(16), nullable=True)
protocol: Mapped[str | None] = mapped_column(String(32), nullable=True)
source_net: Mapped[str | None] = mapped_column(String(255), nullable=True)
source_port: Mapped[str | None] = mapped_column(String(16), nullable=True)
destination_net: Mapped[str | None] = mapped_column(String(255), nullable=True)
destination_port: Mapped[str | None] = mapped_column(String(16), nullable=True)
label: Mapped[str | None] = mapped_column(String(500), nullable=True)
# Full original OPNsense record, for future-proofing / debugging —
# the mapped columns above only cover today's known-useful fields.
raw: Mapped[dict] = mapped_column(JSON, default=dict)
# OPNsense's own `__digest__` for this exact line — the cursor value
# log_ingest_service.py hands back as `digest` on the next poll.
digest: Mapped[str] = mapped_column(String(64))
backend/app/models/log_settings.py
+36
View File
@@ -0,0 +1,36 @@
"""Log-ingest configuration, singleton row (id fixed at 1) — exactly the
NotificationSettings pattern (see models/notification.py /
services/notification_settings_service.py)."""
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, Float, Integer
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
def _now() -> datetime:
return datetime.now(timezone.utc)
class LogSettings(Base):
__tablename__ = "log_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=30)
# Lowered from 30 to 7 (2026-09-17): two real, moderately busy
# firewalls filled firewall_log_entries to ~8.7M rows / ~9.7GB in just
# 9 days at the old default, with no settings UI to notice or fix it
# (see log_settings_service's update_settings() docstring) — a public
# self-hoster with busier traffic could fill a small disk in weeks.
# Still just a default; admin-editable per-install via the new
# GET/PUT /api/logs/settings.
retention_days: Mapped[int] = mapped_column(Integer, default=7)
# Upper bound for the log table in GB (0 = unlimited). Retention by age
# alone cannot protect the disk on a busy firewall — the oldest entries
# are deleted once the estimated live data exceeds this.
max_size_gb: Mapped[float] = mapped_column(Float, nullable=False, default=10.0, server_default="10")
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
backend/app/models/managed_certificate.py
+103
View File
@@ -0,0 +1,103 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String, Text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class ManagedCertificate(Base):
"""A centrally-authored Trust-Store certificate (OPNsense core,
`Trust/Api/CertController.php` — no plugin involved), deliberately
modeled per-host (a plain `host_id` FK), NOT with the
DeviceGroup/`effective_hosts` multi-target pattern `policy_service.py`
provides for rules/IPsec/OpenVPN/WireGuard. Reason (per the plan): a
certificate's `caref` and its own OPNsense-assigned `refid`/uuid are
host-local — an internally-signed cert or CSR on host A has no
meaningful counterpart UUID on host B, so a "push identical payload to
N hosts" model would be actively misleading here, unlike a firewall
rule or a PSK-based IPsec tunnel where the same payload legitimately
IS the same intent on every target.
Field names verified directly against Trust/Cert.xml (fetched
2026-09-09): the model's DescriptionField wire key is `descr`, NOT
`description` (this app's own `description` column is translated to
`descr` in managed_certificate_service.py's payload builder). The
plaintext-in fields are `crt_payload`/`csr_payload`/`prv_payload` —
CertController::setBaseHook() base64-encodes these into `crt`/`csr`/
`prv` server-side, so this app always sends plain PEM/CSR text, never
base64 itself. `key_type` OptionValues are RSA sizes ("512".."8192")
or EC curve names ("prime256v1"/"secp384r1"/"secp521r1") — this app
exposes a curated subset (2048/3072/4096 RSA + the three EC curves).
`cert_type` (Required, Default "usr_cert") is a verified field beyond
what the plan enumerated — exposed here with a "server_cert" default
since TLS server certs are this project's realistic use case, whereas
OPNsense's own UI default targets client certs.
No deploy/apply step exists for Trust-Store writes at all (confirmed
in the plan, `opnsense/core#9399`) — a push IS the add/set API call,
immediately live, no follow-up reconfigure/restart. `status` is
therefore only "draft" (never pushed) / "success" / "error" — even
simpler than IPsec/OpenVPN/WireGuard's already-reduced model, and this
entity needs NO warning gate in the UI (no lockout-shaped risk exists
here — see README).
The private key (`action="import"` only, user-supplied) is NEVER
stored in Postgres — disk-only, one file per entity id, mode 0600 (see
cert_key_store.py, mirroring ipsec_psk_store.py exactly).
`private_key_set` just tracks whether a file exists.
Deleting an in-use certificate is blocked by OPNsense itself
(`checkAndThrowValueInUse()` inside `CertController::delAction()`) —
this app does not pre-check that condition, it surfaces OPNsense's own
error message verbatim (see managed_certificate_service.py)."""
__tablename__ = "managed_certificates"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
# Cert.action OptionValues, verified against Cert.xml: import/internal/
# external/sign_csr/import_csr/reissue/manual.
action: Mapped[str] = mapped_column(String(16))
description: Mapped[str] = mapped_column(String(255)) # -> wire "descr"
# DN fields, only meaningful for action in {internal, external, reissue}
# (CertStore::createCert()/reIssueCert()'s $node->dn() argument).
commonname: Mapped[str] = mapped_column(String(255), default="")
country: Mapped[str] = mapped_column(String(8), default="DE")
state: Mapped[str] = mapped_column(String(255), default="")
city: Mapped[str] = mapped_column(String(255), default="")
organization: Mapped[str] = mapped_column(String(255), default="")
email: Mapped[str] = mapped_column(String(255), default="")
key_type: Mapped[str] = mapped_column(String(16), default="2048")
digest: Mapped[str] = mapped_column(String(16), default="sha256")
cert_type: Mapped[str] = mapped_column(String(32), default="server_cert")
lifetime: Mapped[int] = mapped_column(Integer, default=397)
# Optional, manually-entered internal-CA UUID already existing on this
# host (host-local — see class docstring for why this can't be a
# cross-host reference).
caref: Mapped[str | None] = mapped_column(String(64), nullable=True)
csr_payload: Mapped[str | None] = mapped_column(Text, nullable=True)
crt_payload: Mapped[str | None] = mapped_column(Text, nullable=True)
private_key_set: Mapped[bool] = mapped_column(default=False)
opnsense_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
status: Mapped[str] = mapped_column(String(16), default="draft")
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
# The OPNsense-shaped payload last successfully written — compared
# against a fresh push's payload to detect a true no-op re-push. The
# private key is deliberately never part of this (see cert_key_store.py
# module docstring for why this deviates from IPsec/OpenVPN/WireGuard's
# precedent of embedding secret material in last_payload).
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
deployed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), onupdate=lambda: datetime.now(timezone.utc)
)
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/notification.py
+82
View File
@@ -0,0 +1,82 @@
"""Delivery config (singleton row) and open/resolved problem tracking for
the notification feature. Secrets (SMTP password, webhook signing secret)
never live here — see services/notification_secret_store.py, mirroring how
OPNsense API secrets stay off the database entirely.
"""
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, Boolean, DateTime, Integer, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
def _now() -> datetime:
return datetime.now(timezone.utc)
class NotificationSettings(Base):
"""Single row, id fixed at 1 — see notification_settings_service.get_settings
for the get-or-create helper. The only admin-editable operational config
in the codebase today; everywhere else config is env-var only, but this
feature is meant for future non-technical users without compose access."""
__tablename__ = "notification_settings"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
webhook_enabled: Mapped[bool] = mapped_column(Boolean, default=False)
webhook_url: Mapped[str | None] = mapped_column(String(500), nullable=True)
webhook_signing_secret_set: Mapped[bool] = mapped_column(Boolean, default=False)
email_enabled: Mapped[bool] = mapped_column(Boolean, default=False)
smtp_host: Mapped[str | None] = mapped_column(String(255), nullable=True)
smtp_port: Mapped[int] = mapped_column(Integer, default=587)
# "none" | "starttls" | "ssl"
smtp_security: Mapped[str] = mapped_column(String(10), default="starttls")
smtp_username: Mapped[str | None] = mapped_column(String(255), nullable=True)
smtp_password_set: Mapped[bool] = mapped_column(Boolean, default=False)
smtp_from_address: Mapped[str | None] = mapped_column(String(255), nullable=True)
smtp_to_addresses: Mapped[list] = mapped_column(JSON, default=list)
host_offline_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
cert_expiring_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
rule_drift_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
maintenance_unknown_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
carp_health_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
nat_drift_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
carp_auto_revert_failed_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
poll_interval_seconds: Mapped[int] = mapped_column(Integer, default=300)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
class NotificationProblem(Base):
"""Open/resolved state per detected problem, keyed by (problem_type,
dedup_key) so a still-open problem is never re-notified every poll —
only transitions (newly open, newly resolved) trigger delivery.
Delivery is best-effort, not an audit trail: last_notified_at/
resolved_notified_at are set once per transition and never retried.
audit_events remains the real immutable record for deploy actions;
this table is a convenience signal layered on top, not a new source
of truth.
"""
__tablename__ = "notification_problems"
__table_args__ = (UniqueConstraint("problem_type", "dedup_key", name="uq_notification_problem_key"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
# host_offline | cert_expiry | rule_drift | revert_failed | maintenance_unknown
problem_type: Mapped[str] = mapped_column(String(32))
dedup_key: Mapped[str] = mapped_column(String(300))
status: Mapped[str] = mapped_column(String(16), default="open") # open | resolved
context: Mapped[dict] = mapped_column(JSON, default=dict)
first_detected_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
last_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
resolved_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
last_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
resolved_notified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
backend/app/models/one_to_one.py
+44
View File
@@ -0,0 +1,44 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.one_to_one_target import one_to_one_targets
from app.models.policy import one_to_one_groups
class OneToOneRule(Base):
__tablename__ = "one_to_one_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
source_net: Mapped[str] = mapped_column(String(128))
destination_net: Mapped[str] = mapped_column(String(128))
source_not: Mapped[bool] = mapped_column(Boolean, default=False)
destination_not: Mapped[bool] = mapped_column(Boolean, default=False)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
type: Mapped[str] = mapped_column(String(8), default="binat")
external: Mapped[str] = mapped_column(String(128))
natreflection: Mapped[str] = mapped_column(String(16), default="")
log: Mapped[bool] = mapped_column(Boolean, default=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
policy_hierarchy_group_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="SET NULL"), nullable=True
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=one_to_one_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=one_to_one_groups)
deployments: Mapped[list["OneToOneDeployment"]] = relationship(
"OneToOneDeployment", back_populates="one_to_one", cascade="all, delete-orphan"
)
backend/app/models/one_to_one_deployment.py
+37
View File
@@ -0,0 +1,37 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class OneToOneDeployment(Base):
"""Last known deploy result of one 1:1 NAT rule on one host —
mirrors RuleDeployment/AliasDeployment exactly; see rule_deployment.py
for the status-value docstring, unchanged here."""
__tablename__ = "one_to_one_deployments"
__table_args__ = (UniqueConstraint("one_to_one_id", "host_id", name="uq_one_to_one_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
one_to_one_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("one_to_one_rules.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_one_to_one_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
one_to_one: Mapped["OneToOneRule"] = relationship("OneToOneRule", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/one_to_one_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
one_to_one_targets = Table(
"one_to_one_targets",
Base.metadata,
Column("one_to_one_id", PGUUID(as_uuid=True), ForeignKey("one_to_one_rules.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/openvpn_instance.py
+128
View File
@@ -0,0 +1,128 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, Boolean, DateTime, String, text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.openvpn_instance_target import openvpn_instance_targets
from app.models.policy import openvpn_instance_groups
class OpenVpnInstance(Base):
"""A centrally-authored OpenVPN instance (server OR client role — v1
scope, see services/openvpn_deploy_service.py's module docstring for
the full field verification and the scope decision this class embodies).
Unlike IpsecTunnel (5 linked OPNsense objects), OPNsense's modern
"Instance" object (OpenVPN.xml -> Instances.Instance, 24.1+) is a
single flat model item — EXCEPT for the optional tls-auth/tls-crypt
control-channel hardening key, which is its OWN separate OPNsense
object (StaticKeys.StaticKey) referenced from the Instance via the
`tls_key` ModelRelationField. So this entity is closer to a 2-object
composite (StaticKey optionally created first, Instance referencing
its UUID) than a perfectly flat single-row entity like FirewallRule —
see OpenVpnInstanceDeployment's docstring for the resulting 2-UUID
shape, and openvpn_deploy_service.py for exactly why.
IMPORTANT SCOPE FINDING (verified directly against OpenVPN.xml, see
openvpn_deploy_service.py's module docstring): the modern Instance
model has NO "shared-key-only" role (unlike legacy pre-24.1
openvpn-server/-client sections, which had a `mode` of
"p2p_shared_key" needing no PKI at all) — `role` here is only
"server"/"client", and BOTH still fundamentally require a working
TLS handshake (a `cert` + `ca`, both ModelRelationFields into
OPNsense's own certificate store) to actually pass traffic. The
`tls_key`/StaticKey mechanism this class supports is only an
OPTIONAL hardening layer on TOP of that TLS handshake (OpenVPN's
--tls-auth/--tls-crypt), not a substitute for certificates — so,
unlike IPsec's PSK, it can't be this entity's ONLY form of "auth
material". cert_ref/ca_ref reference an already-existing certificate/CA
on the TARGET host — but a cert's UUID is host-local, so a single
shared string can't be correct for more than one target host at once
(confirmed as a real, not hypothetical, gap: instances already support
genuine multi-host targeting below). cert_refs/ca_refs are therefore
**per-host maps** (`{host_id_str: uuid}`, JSON columns), populated via
the frontend's per-target-host picker (backed by this project's
existing read-only `GET /api/hosts/{id}/certificates` plus the sibling
`GET /api/hosts/{id}/cas`) rather than a single hand-typed value. A
target host missing from either map fails closed at deploy time — see
openvpn_deploy_service.py's build_payloads().
Targeting mirrors IpsecTunnel/FirewallRule exactly (direct host
targets + DeviceGroup membership, resolved via
policy_service.effective_hosts) — realistic v1 use cases: an
identical server-role instance pushed to both HA members sharing a
CARP VIP, or a client-role instance pointed at one specific remote
peer (targeted at just that one host).
The static key's raw material is never stored here — like
IpsecTunnel's PSK, it lives only on disk, one file per instance id
(see openvpn_key_store.py). tls_key_material_set just tracks whether
one has been provided yet. tls_key_enabled/tls_key_mode/material are
all optional — a valid instance can exist with tls hardening off.
Deploy status model is deliberately as simple as IpsecTunnel's — see
services/openvpn_deploy_service.py's module docstring for why (no
pending/confirm/auto-revert window here either, same rationale: an
OpenVPN `reconfigure` can equally sever the management path OPNexus
would need to auto-revert it)."""
__tablename__ = "openvpn_instances"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
# Instances.Instance.role — OptionValues are literally only
# "client"/"server" (verified against OpenVPN.xml), no p2p/shared-key
# role exists in the modern model.
role: Mapped[str] = mapped_column(String(8), default="server")
# Instances.Instance.proto — model OptionValues also include
# udp4/udp6/tcp4/tcp6; v1 only exposes the dual-stack "udp"/"tcp"
# defaults, same "don't over-engineer" simplification IPsec applied
# to its own proposal/esp_proposal fields.
proto: Mapped[str] = mapped_column(String(8), default="udp")
# Instances.Instance.port (PortField) — kept as a plain string, sent
# verbatim, same as every other OPNsense-validated field in this app.
port: Mapped[str] = mapped_column(String(8), default="1194")
# Instances.Instance.local (NetworkField, no netmask) — server-role
# bind address; blank means "any" (OPNsense's own model default).
local: Mapped[str] = mapped_column(String(255), default="")
# Instances.Instance.remote (RemoteHostField) — client-role remote
# peer ("host" or "host:port"); required for role=client, sent
# verbatim with no comma-list building (v1 keeps to one peer).
remote: Mapped[str] = mapped_column(String(255), default="")
# Instances.Instance.server (NetworkField, NetMaskRequired=Y) —
# server-role tunnel network in CIDR notation; required for role=server.
server_network: Mapped[str] = mapped_column(String(255), default="")
# Instances.Instance.cert / .ca (CertificateField -> ModelRelationField
# into Trust\Cert) — per-host maps ({host_id_str: uuid}), see class
# docstring for why a single shared value can't work across hosts.
cert_refs: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict, server_default=text("'{}'::json"))
ca_refs: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict, server_default=text("'{}'::json"))
# Whether to also create/attach a StaticKeys.StaticKey for
# tls-auth/tls-crypt hardening (Instances.Instance.tls_key). Optional —
# see class docstring.
tls_key_enabled: Mapped[bool] = mapped_column(Boolean, default=False)
# StaticKeys.StaticKey.mode — OptionValues are auth/crypt/crypt-v2;
# v1 exposes auth/crypt only (crypt-v2 needs the "server" vs "client"
# key-half distinction genKeyAction's tls-crypt-v2-server type hints
# at, deliberately deferred as extra complexity not needed for the v1
# HA-pair / single-peer use cases).
tls_key_mode: Mapped[str] = mapped_column(String(16), default="crypt")
tls_key_material_set: Mapped[bool] = mapped_column(Boolean, default=False)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=openvpn_instance_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=openvpn_instance_groups)
deployments: Mapped[list["OpenVpnInstanceDeployment"]] = relationship(
"OpenVpnInstanceDeployment", back_populates="instance", cascade="all, delete-orphan"
)
backend/app/models/openvpn_instance_deployment.py
+61
View File
@@ -0,0 +1,61 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class OpenVpnInstanceDeployment(Base):
"""Last known deploy result of one OpenVPN instance on one host.
Two opnsense_*_uuid columns, not one — see OpenVpnInstance's docstring
for why this entity isn't perfectly flat: `opnsense_statickey_uuid` is
only populated when tls_key_enabled (the StaticKeys.StaticKey object,
written FIRST so its UUID can be injected into the Instance's tls_key
field), `opnsense_instance_uuid` is the Instances.Instance object
itself, required to target setAction/delAction there later — exactly
like IpsecTunnelDeployment.opnsense_connection_uuid etc.
status: "success" (the instance — and, if enabled, its StaticKey —
were written AND the follow-up reconfigure call was confirmed) or
"error". Deliberately NO "pending"/"reverted"/"revert_failed" here —
see services/openvpn_deploy_service.py's module docstring for why this
entity has no auto-revert safety net (same rationale as
IpsecTunnelDeployment: an OpenVPN reconfigure can cut off the very
management path OPNexus would need to auto-revert it).
A host with no OpenVpnInstanceDeployment row at all is this instance's
"draft" state on that host (never pushed there) — surfaced by
openvpn_service.preview()'s action="create" for that row, not by a
persisted status value.
"""
__tablename__ = "openvpn_instance_deployments"
__table_args__ = (UniqueConstraint("instance_id", "host_id", name="uq_openvpn_instance_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
instance_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("openvpn_instances.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_statickey_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_instance_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
# The OPNsense-shaped payloads (keyed "statickey"/"instance") last
# successfully written to this host — compared against a fresh
# deploy's payloads to detect a true no-op re-deploy. Like
# IpsecTunnelDeployment there is deliberately no previous_payload:
# nothing here is ever restored automatically.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
instance: Mapped["OpenVpnInstance"] = relationship("OpenVpnInstance", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/openvpn_instance_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
openvpn_instance_targets = Table(
"openvpn_instance_targets",
Base.metadata,
Column("instance_id", PGUUID(as_uuid=True), ForeignKey("openvpn_instances.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/outbound_nat.py
+46
View File
@@ -0,0 +1,46 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.outbound_nat_target import outbound_nat_targets
from app.models.policy import outbound_nat_groups
class OutboundNatRule(Base):
__tablename__ = "outbound_nat_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
source_net: Mapped[str] = mapped_column(String(128))
destination_net: Mapped[str] = mapped_column(String(128))
source_not: Mapped[bool] = mapped_column(Boolean, default=False)
destination_not: Mapped[bool] = mapped_column(Boolean, default=False)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
ipprotocol: Mapped[str] = mapped_column(String(8), default="inet")
protocol: Mapped[str] = mapped_column(String(16))
source_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
target: Mapped[str] = mapped_column(String(128))
target_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
policy_hierarchy_group_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="SET NULL"), nullable=True
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=outbound_nat_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=outbound_nat_groups)
deployments: Mapped[list["OutboundNatDeployment"]] = relationship(
"OutboundNatDeployment", back_populates="outbound_nat", cascade="all, delete-orphan"
)
backend/app/models/outbound_nat_deployment.py
+37
View File
@@ -0,0 +1,37 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class OutboundNatDeployment(Base):
"""Last known deploy result of one Outbound NAT rule on one host —
mirrors RuleDeployment/AliasDeployment exactly; see rule_deployment.py
for the status-value docstring, unchanged here."""
__tablename__ = "outbound_nat_deployments"
__table_args__ = (UniqueConstraint("outbound_nat_id", "host_id", name="uq_outbound_nat_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
outbound_nat_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("outbound_nat_rules.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_outbound_nat_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
outbound_nat: Mapped["OutboundNatRule"] = relationship("OutboundNatRule", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/outbound_nat_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
outbound_nat_targets = Table(
"outbound_nat_targets",
Base.metadata,
Column("outbound_nat_id", PGUUID(as_uuid=True), ForeignKey("outbound_nat_rules.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/policy.py
+167
View File
@@ -0,0 +1,167 @@
"""Additive tables: existing rule/host schemas and data remain valid."""
import uuid
from datetime import datetime, timezone
from sqlalchemy import Column, DateTime, ForeignKey, JSON, String, Table, Text
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
group_members = Table(
"device_group_members", Base.metadata,
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
rule_groups = Table(
"rule_groups", Base.metadata,
Column("rule_id", PGUUID(as_uuid=True), ForeignKey("firewall_rules.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
alias_groups = Table(
"alias_groups", Base.metadata,
Column("alias_id", PGUUID(as_uuid=True), ForeignKey("firewall_aliases.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
port_forward_groups = Table(
"port_forward_groups", Base.metadata,
Column("port_forward_id", PGUUID(as_uuid=True), ForeignKey("port_forward_rules.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class DeviceGroup(Base):
__tablename__ = "device_groups"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(120), unique=True)
description: Mapped[str] = mapped_column(Text, default="")
hosts: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=group_members)
class RuleRevision(Base):
__tablename__ = "rule_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
rule_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("firewall_rules.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
snapshot: Mapped[dict] = mapped_column(JSON)
class AliasRevision(Base):
__tablename__ = "alias_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
alias_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("firewall_aliases.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
snapshot: Mapped[dict] = mapped_column(JSON)
class PortForwardRevision(Base):
__tablename__ = "port_forward_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
port_forward_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("port_forward_rules.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
snapshot: Mapped[dict] = mapped_column(JSON)
one_to_one_groups = Table(
"one_to_one_groups", Base.metadata,
Column("one_to_one_id", PGUUID(as_uuid=True), ForeignKey("one_to_one_rules.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class OneToOneRevision(Base):
__tablename__ = "one_to_one_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
one_to_one_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("one_to_one_rules.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
snapshot: Mapped[dict] = mapped_column(JSON)
outbound_nat_groups = Table(
"outbound_nat_groups", Base.metadata,
Column("outbound_nat_id", PGUUID(as_uuid=True), ForeignKey("outbound_nat_rules.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class OutboundNatRevision(Base):
__tablename__ = "outbound_nat_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
outbound_nat_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("outbound_nat_rules.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
snapshot: Mapped[dict] = mapped_column(JSON)
ipsec_tunnel_groups = Table(
"ipsec_tunnel_groups", Base.metadata,
Column("tunnel_id", PGUUID(as_uuid=True), ForeignKey("ipsec_tunnels.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class IpsecTunnelRevision(Base):
__tablename__ = "ipsec_tunnel_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
tunnel_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("ipsec_tunnels.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
# Snapshot never includes the PSK secret itself (not stored in the DB
# at all, see models/ipsec_tunnel.py) — only pre_shared_key_set, so a
# restored revision reflects whether a key was configured, not what it
# was.
snapshot: Mapped[dict] = mapped_column(JSON)
openvpn_instance_groups = Table(
"openvpn_instance_groups", Base.metadata,
Column("instance_id", PGUUID(as_uuid=True), ForeignKey("openvpn_instances.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class OpenVpnInstanceRevision(Base):
__tablename__ = "openvpn_instance_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
instance_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("openvpn_instances.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
# Snapshot never includes the static-key material itself (not stored in
# the DB at all, see models/openvpn_instance.py) — only
# tls_key_material_set, mirroring IpsecTunnelRevision's PSK handling.
snapshot: Mapped[dict] = mapped_column(JSON)
wireguard_tunnel_groups = Table(
"wireguard_tunnel_groups", Base.metadata,
Column("tunnel_id", PGUUID(as_uuid=True), ForeignKey("wireguard_tunnels.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", PGUUID(as_uuid=True), ForeignKey("device_groups.id", ondelete="CASCADE"), primary_key=True),
)
class WireguardTunnelRevision(Base):
__tablename__ = "wireguard_tunnel_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
tunnel_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("wireguard_tunnels.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
# Snapshot never includes the server private key or peer PSK material
# itself (not stored in the DB at all, see models/wireguard_tunnel.py)
# — only server_private_key_set/peer_psk_set, mirroring
# IpsecTunnelRevision/OpenVpnInstanceRevision's secret handling.
snapshot: Mapped[dict] = mapped_column(JSON)
class PolicyHierarchyRuleRevision(Base):
__tablename__ = "policy_hierarchy_rule_revisions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
policy_hierarchy_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_rules.id", ondelete="CASCADE"), index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
note: Mapped[str] = mapped_column(String(255))
# Snapshot includes hierarchy_group_id/rulebase/sequence (unlike the
# other *Revision classes above) since, unlike a plain rule's
# target_host_ids/target_group_ids, those three fields directly change
# deploy ordering/targeting for this entity and must be restorable too.
snapshot: Mapped[dict] = mapped_column(JSON)
backend/app/models/policy_hierarchy_group.py
+45
View File
@@ -0,0 +1,45 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class PolicyHierarchyGroup(Base):
"""A node in the pre-/post-rulebase inheritance tree — deliberately
separate from DeviceGroup (see models/policy.py), which allows a host
to belong to more than one group at once. That multi-membership model
is fine for plain rule-target grouping, but would make "which ancestor
chain applies to this host" ambiguous for ordered inheritance, which is
the entire point of this feature. A host belongs to at most one
PolicyHierarchyGroup (see OpnsenseHost.policy_hierarchy_group_id).
parent_id uses ondelete="RESTRICT": a group with children can't be
deleted out from under them (see policy_hierarchy_group_service /
routers/policy_hierarchy_groups.py, which also blocks deletion while
rules are still attached, mirroring DeviceGroup's own delete guard)."""
__tablename__ = "policy_hierarchy_groups"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name: Mapped[str] = mapped_column(String(120), unique=True)
description: Mapped[str] = mapped_column(String(2000), default="")
parent_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="RESTRICT"), nullable=True
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
parent: Mapped["PolicyHierarchyGroup | None"] = relationship(
"PolicyHierarchyGroup", remote_side=[id], back_populates="children"
)
children: Mapped[list["PolicyHierarchyGroup"]] = relationship(
"PolicyHierarchyGroup", back_populates="parent"
)
hosts: Mapped[list["OpnsenseHost"]] = relationship(
"OpnsenseHost", back_populates="policy_hierarchy_group"
)
backend/app/models/policy_hierarchy_rule.py
+64
View File
@@ -0,0 +1,64 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class PolicyHierarchyRule(Base):
"""A rule authored on a PolicyHierarchyGroup rather than a specific
host/DeviceGroup — see services/policy_hierarchy_service.py for how it
gets pushed to every host in the group's subtree (effective_hosts) in
the correct cross-group order (ordered_hierarchy_rules).
Deliberately a separate table/entity from FirewallRule (not an
extension of it) — the existing rule-deploy path is security-critical
and this keeps zero regression risk on it. Same field set as
FirewallRule plus hierarchy_group_id/rulebase/sequence.
rulebase: "pre" or "post" — splits the rules into a pre- and a
post-rulebase. Pre-rules are evaluated root-most-group-first (the top of the
org hierarchy wins first, like a pre-rule should); post-rules
leaf-most-group-first (the most specific group gets the last, most
specific word before any host-local NAT/default-deny). See
ordered_hierarchy_rules() for the actual implementation of this.
sequence: manual ordering within the same (hierarchy_group_id,
rulebase) pair — persisted atomically by the drag-and-drop endpoint;
up/down buttons remain as an accessible keyboard fallback."""
__tablename__ = "policy_hierarchy_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
hierarchy_group_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="CASCADE")
)
rulebase: Mapped[str] = mapped_column(String(8))
sequence: Mapped[int] = mapped_column(Integer, default=0)
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
direction: Mapped[str] = mapped_column(String(8))
ipprotocol: Mapped[str] = mapped_column(String(8), default="inet")
protocol: Mapped[str] = mapped_column(String(16), default="any")
action: Mapped[str] = mapped_column(String(16), default="pass")
source_net: Mapped[str] = mapped_column(String(128), default="any")
source_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
destination_net: Mapped[str] = mapped_column(String(128), default="any")
destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
group: Mapped["PolicyHierarchyGroup"] = relationship("PolicyHierarchyGroup")
deployments: Mapped[list["PolicyHierarchyRuleDeployment"]] = relationship(
"PolicyHierarchyRuleDeployment", back_populates="rule", cascade="all, delete-orphan"
)
backend/app/models/policy_hierarchy_rule_deployment.py
+48
View File
@@ -0,0 +1,48 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class PolicyHierarchyRuleDeployment(Base):
"""Copy of models/rule_deployment.py for PolicyHierarchyRule — see that
file's docstring for the status lifecycle and lockout mechanism, which
this mirrors exactly (policy_hierarchy_deploy_service.py is a 1:1 port
of rule_deploy_service.py). Columns are named policy_hierarchy_id /
opnsense_policy_hierarchy_uuid (not rule_id/opnsense_rule_uuid) so
audit_service.tracked_rollback("policy_hierarchy") — which derives
attribute names from its `kind` argument — resolves them correctly.
"""
__tablename__ = "policy_hierarchy_rule_deployments"
__table_args__ = (UniqueConstraint("policy_hierarchy_id", "host_id", name="uq_policy_hierarchy_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
policy_hierarchy_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_rules.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_policy_hierarchy_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
# Set while status == "pending"; sweep_expired_hierarchy_deployments
# reverts once this passes without a confirm call.
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# The OPNsense-shaped payload we last successfully applied on this host,
# BEFORE the current pending change (None if this deploy created the
# rule fresh on this host).
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
# The payload actually active on this host right now.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
rule: Mapped["PolicyHierarchyRule"] = relationship("PolicyHierarchyRule", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/port_forward.py
+51
View File
@@ -0,0 +1,51 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.port_forward_target import port_forward_targets
from app.models.policy import port_forward_groups
class PortForwardRule(Base):
"""A centrally-authored Port Forward (destination NAT) rule. OPNsense's
wire field for this is "disabled", not "enabled" (see
port_forward_deploy_service._build_payload) — this model stores the
same enabled=True/False polarity as FirewallRule/FirewallAlias for
consistency within OPNexus; only the outgoing payload flips it."""
__tablename__ = "port_forward_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
ipprotocol: Mapped[str] = mapped_column(String(8), default="inet")
protocol: Mapped[str] = mapped_column(String(16), default="tcp")
source_net: Mapped[str] = mapped_column(String(128), default="any")
source_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
destination_net: Mapped[str] = mapped_column(String(128), default="any")
destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
target: Mapped[str] = mapped_column(String(128))
local_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
natreflection: Mapped[str] = mapped_column(String(16), default="")
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
policy_hierarchy_group_id: Mapped[uuid.UUID | None] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("policy_hierarchy_groups.id", ondelete="SET NULL"), nullable=True
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=port_forward_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=port_forward_groups)
deployments: Mapped[list["PortForwardDeployment"]] = relationship(
"PortForwardDeployment", back_populates="port_forward", cascade="all, delete-orphan"
)
backend/app/models/port_forward_deployment.py
+37
View File
@@ -0,0 +1,37 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class PortForwardDeployment(Base):
"""Last known deploy result of one Port Forward rule on one host —
mirrors RuleDeployment/AliasDeployment exactly; see rule_deployment.py
for the status-value docstring, unchanged here."""
__tablename__ = "port_forward_deployments"
__table_args__ = (UniqueConstraint("port_forward_id", "host_id", name="uq_port_forward_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
port_forward_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("port_forward_rules.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_port_forward_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
port_forward: Mapped["PortForwardRule"] = relationship("PortForwardRule", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/port_forward_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
port_forward_targets = Table(
"port_forward_targets",
Base.metadata,
Column("port_forward_id", PGUUID(as_uuid=True), ForeignKey("port_forward_rules.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/preferences.py
+11
View File
@@ -0,0 +1,11 @@
import uuid
from sqlalchemy import ForeignKey, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class UserPreferences(Base):
__tablename__ = "user_preferences"
user_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey('admin_users.id',ondelete='CASCADE'), primary_key=True)
theme: Mapped[str] = mapped_column(String(10), default='system')
language: Mapped[str] = mapped_column(String(5), nullable=False, default='de', server_default='de')
backend/app/models/resource_sample.py
+39
View File
@@ -0,0 +1,39 @@
"""Periodic host resource samples (CPU load, memory, disk, interface
counters) — OPNexus's own time series, polled independently of whatever
retention OPNsense's own on-device RRD graphs keep. A brand-new table,
same "no manual ALTER needed" situation as host_log_cursor/
firewall_log_entries (see those modules' docstrings) since nothing here
touches an existing table.
Like firewall_log_entries and unlike audit_events, this table needs
retention pruning (resource_monitor_service.prune_old_entries) to be able
to delete old rows, so no immutability trigger is added here either.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, Float, ForeignKey, Index, JSON
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostResourceSample(Base):
__tablename__ = "host_resource_samples"
__table_args__ = (
Index("ix_host_resource_samples_host_sampled_at", "host_id", "sampled_at"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
sampled_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
mem_used_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
load_avg_1m: Mapped[float | None] = mapped_column(Float, nullable=True)
disk_used_pct: Mapped[float | None] = mapped_column(Float, nullable=True)
# Catch-all: full memory/disk-device/interface-statistics payloads, for
# anything the three columns above don't cover (per-interface
# throughput counters in particular — see resource_monitor_service.py).
raw: Mapped[dict] = mapped_column(JSON, default=dict)
backend/app/models/restore_operation.py
+33
View File
@@ -0,0 +1,33 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import DateTime, String, JSON
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class RestoreOperation(Base):
"""Stage 1 restore: preview a locally-stored config.xml backup and
record a human-performed handoff to the OPNsense GUI. No row here ever
triggers an OPNsense write — see restore_service.py's module docstring.
Shaped like HostOperation (no FK to opnsense_hosts, denormalized name).
"""
__tablename__ = "restore_operations"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), index=True)
host_name: Mapped[str] = mapped_column(String(120))
actor: Mapped[str] = mapped_column(String(150))
backup_id: Mapped[str] = mapped_column(String(255))
status: Mapped[str] = mapped_column(String(20), default="preview")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
fingerprint: Mapped[str] = mapped_column(String(64))
manifest: Mapped[dict] = mapped_column(JSON)
warnings: Mapped[list] = mapped_column(JSON, default=list)
confirm_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
handed_off_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
confirmed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
cancelled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
message: Mapped[str | None] = mapped_column(String(1000), nullable=True)
backend/app/models/rule.py
+45
View File
@@ -0,0 +1,45 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.rule_target import rule_targets
from app.models.policy import rule_groups
class FirewallRule(Base):
"""A centrally-authored rule. Pushing it to a host happens via
services/rule_deploy_service.py, which records one RuleDeployment row
per (rule, host)."""
__tablename__ = "firewall_rules"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
description: Mapped[str] = mapped_column(String(255))
interface: Mapped[str] = mapped_column(String(64))
direction: Mapped[str] = mapped_column(String(8))
ipprotocol: Mapped[str] = mapped_column(String(8), default="inet")
protocol: Mapped[str] = mapped_column(String(16), default="any")
action: Mapped[str] = mapped_column(String(16), default="pass")
source_net: Mapped[str] = mapped_column(String(128), default="any")
source_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
destination_net: Mapped[str] = mapped_column(String(128), default="any")
destination_port: Mapped[str | None] = mapped_column(String(32), nullable=True)
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=rule_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=rule_groups)
deployments: Mapped[list["RuleDeployment"]] = relationship(
"RuleDeployment", back_populates="rule", cascade="all, delete-orphan"
)
backend/app/models/rule_deployment.py
+57
View File
@@ -0,0 +1,57 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class RuleDeployment(Base):
"""Last known deploy result of one rule on one host. opnsense_rule_uuid
is the UUID OPNsense assigned on THAT host (each host gets its own copy
of the rule) — required to target setRule/delRule there later.
status: "pending" (applied, awaiting confirmation — see
rule_deploy_service.sweep_expired_deployments), "success" (confirmed,
or a no-op re-deploy of an unchanged payload), "error" (the deploy
call itself failed), "reverted" (auto-rolled-back after the
confirmation window expired unconfirmed), "revert_failed" (the
rollback call itself failed — needs manual attention).
"""
__tablename__ = "rule_deployments"
__table_args__ = (UniqueConstraint("rule_id", "host_id", name="uq_rule_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
rule_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("firewall_rules.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_rule_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
# Set while status == "pending"; sweep_expired_deployments reverts once
# this passes without a confirm call.
revert_deadline: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# The OPNsense-shaped payload we last successfully applied on this host,
# BEFORE the current pending change (None if this deploy created the
# rule fresh on this host — the revert path for that case is delRule,
# not setRule). This is our own record of "what was there", not a value
# fetched back from OPNsense — its getRule response shape is more
# complex/uncertain than we want to depend on for a safety mechanism.
previous_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
# The payload actually active on this host right now (mirrors
# previous_payload's shape) — compared against a new deploy's payload
# to detect a true no-op re-deploy, which skips the pending window
# entirely since nothing risky happened.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
rule: Mapped["FirewallRule"] = relationship("FirewallRule", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/rule_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
rule_targets = Table(
"rule_targets",
Base.metadata,
Column("rule_id", PGUUID(as_uuid=True), ForeignKey("firewall_rules.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/models/scheduled_maintenance.py
+42
View File
@@ -0,0 +1,42 @@
"""One-shot scheduled host-maintenance actions (reboot/update/major_update
at a future timestamp instead of immediately). A brand-new table, same
"no manual ALTER needed" situation as host_resource_samples.
Deliberately v1-minimal: one fixed `run_at`, no recurrence/cron
expression — see resource_monitor_service.py's sibling note in
config.py for the same "keep v1 small" rationale. Firing reuses the
existing HostOperation/ensure_no_maintenance lock machinery
(routers/scheduled_maintenance.py, main.py's
_scheduled_maintenance_poll_loop) rather than duplicating it — this
table only tracks the *schedule* itself, not the resulting operation.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Index, JSON, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class ScheduledMaintenance(Base):
__tablename__ = "scheduled_maintenance"
__table_args__ = (
Index("ix_scheduled_maintenance_status_run_at", "status", "run_at"),
)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True
)
host_name: Mapped[str] = mapped_column(String(120))
action: Mapped[str] = mapped_column(String(32)) # reboot | update | major_update
run_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
status: Mapped[str] = mapped_column(String(16), default="pending") # pending|running|succeeded|failed|cancelled
actor: Mapped[str] = mapped_column(String(120))
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
result: Mapped[dict] = mapped_column(JSON, default=dict)
# Set to the resulting HostOperation's id once fired, so the UI can
# link from the schedule to the operation it produced.
host_operation_id: Mapped[uuid.UUID | None] = mapped_column(PGUUID(as_uuid=True), nullable=True)
backend/app/models/user.py
+57
View File
@@ -0,0 +1,57 @@
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import JSON, BigInteger, Boolean, DateTime, ForeignKey, String, func
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class AdminUser(Base):
"""A user account. Two roles, no orgs: "admin" (full read/write, incl.
managing other users) and "read_only" (every GET works, every write is
rejected by AuthMiddleware before it reaches a router — see there for
the exact allowlist). See auth_service.bootstrap_admin for how the
first account is created."""
__tablename__ = "admin_users"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
username: Mapped[str] = mapped_column(String(150), nullable=False, unique=True, index=True)
# Argon2id PHC string — never the raw password.
password_hash: Mapped[str] = mapped_column(String(255), nullable=False)
role: Mapped[str] = mapped_column(String(20), nullable=False, default="admin", server_default="admin")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
last_login_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
last_login_ip: Mapped[Optional[str]] = mapped_column(String(45), nullable=True)
# Optional TOTP second factor. The secret itself is on disk
# (services/totp_service), not here; only SHA-256 hashes of the one-time
# recovery codes and the last accepted time-step (replay guard) are.
totp_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
totp_last_step: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0, server_default="0")
totp_recovery_hashes: Mapped[Optional[list]] = mapped_column(JSON, nullable=True)
sessions: Mapped[list["Session"]] = relationship(
"Session", back_populates="user", cascade="all, delete-orphan"
)
class Session(Base):
__tablename__ = "sessions"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
user_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("admin_users.id", ondelete="CASCADE"), nullable=False, index=True
)
# SHA-256 hex of the random session token. The raw token only ever lives
# in the client's httpOnly cookie, so a DB leak yields no usable sessions.
token_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True, index=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_used_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
user_agent: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
ip_address: Mapped[Optional[str]] = mapped_column(String(45), nullable=True)
user: Mapped["AdminUser"] = relationship("AdminUser", back_populates="sessions")
backend/app/models/wireguard_tunnel.py
+141
View File
@@ -0,0 +1,141 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, String
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
from app.models.wireguard_tunnel_target import wireguard_tunnel_targets
from app.models.policy import wireguard_tunnel_groups
class WireguardTunnel(Base):
"""A centrally-authored WireGuard interconnect (v1 scope, see
services/wireguard_deploy_service.py's module docstring for the full
field verification and the scope decision this class embodies).
ENTITY-SHAPE DECISION (see wireguard_deploy_service.py for the full
reasoning): WireGuard's real OPNsense object model is one `Server`
(the local wg-quick interface: keypair, listen port, own tunnel
address) that owns a comma-list `peers` field pointing at zero or more
`Client` (peer) objects — a genuine one-to-many, NOT a 1:1 pairing.
Nonetheless, this class models exactly ONE Server + exactly ONE Client
per WireguardTunnel row (a 2-object composite, closest in shape to
OpenVpnInstance's Instance+optional-StaticKey composite, NOT IPsec's
5-object one) rather than either (a) a single flat entity, or (b) two
fully independent top-level CRUD entities (WireguardServer/
WireguardPeer) that the operator would have to keep in sync by hand.
Reasoning: the realistic v1 fleet use case named in the plan — one
interconnect between two managed firewalls, or a managed firewall and
one external peer — is exactly a 1:1 Server<->Peer pairing, the same
way IpsecTunnel models "a site-to-site VPN connection" as one form an
admin actually thinks in, instead of exposing Swanctl's Connection/
Local/Remote/Child as four separate CRUD screens. A multi-peer WG hub
(one Server, many Clients) is a real, deliberately deferred v1 gap —
see the module docstring for why "always exactly one peer" was chosen
over building that out tonight.
The peer<->server linkage itself is NOT a plain ModelRelationField
written by this app directly (unlike IPsec's Local/Remote/Child, which
carry their own `connection` UUID field this app sets) — it is a
denormalized, bidirectionally-maintained relationship where the
SERVER's `peers` field is the only persisted list, and OPNsense's own
`ClientController::setClientAction`/`delClientAction` (verified
directly from source, see module docstring) update that list
server-side as a side effect of writing/deleting the Client, whenever
this app's Client payload includes a `servers` field naming the
Server's UUID. This app therefore never reads or rewrites the
Server's `peers` field itself — seebuild_payloads()/deploy service for
exactly how the `servers` field is injected at deploy time.
Targeting mirrors IpsecTunnel/OpenVpnInstance exactly (direct host
targets + DeviceGroup membership, resolved via
policy_service.effective_hosts).
Two secrets live on disk only, never in Postgres as their own column
(see wireguard_key_store.py, mirroring ipsec_psk_store.py/
openvpn_key_store.py): the Server's WireGuard private key (required —
OPNsense's own model requires `privkey`, and does NOT derive a public
key from it anywhere in the model/field-type source, confirmed by
reading ServerField.php — so both server_public_key AND the private
key must be operator-supplied, see module docstring for why this app
doesn't call `keyPairAction` to generate one), and the peer's optional
WireGuard pre-shared key (an extra symmetric hardening layer WireGuard
itself supports per-peer, unrelated to IPsec's PSK despite the shared
name — OPNsense's `Client.psk` field). `server_private_key_set` /
`peer_psk_set` just track whether material has been provided, same
pattern as IpsecTunnel.pre_shared_key_set /
OpenVpnInstance.tls_key_material_set.
Deploy status model is deliberately as simple as IpsecTunnel's/
OpenVpnInstance's — see services/wireguard_deploy_service.py's module
docstring for why (no pending/confirm/auto-revert window here either,
same rationale: a WireGuard `reconfigure` can equally sever the
management path OPNexus would need to auto-revert it)."""
__tablename__ = "wireguard_tunnels"
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
# ── Server (local wg-quick interface) — Wireguard\Server.xml
# servers.server fields, see module docstring for the exact mapping.
# `name` doubles as this entity's display label (WireGuard's model has
# no separate free-text `description` field the way Swanctl/OpenVPN
# do — masked identically to OPNsense's own Mask,
# `/^([0-9a-zA-Z._\-]){1,64}$/u`, enforced again in schemas/wireguard.py).
server_name: Mapped[str] = mapped_column(String(64))
server_public_key: Mapped[str] = mapped_column(String(64))
server_private_key_set: Mapped[bool] = mapped_column(Boolean, default=False)
# Server.port (PortField) — kept as a plain string, sent verbatim,
# same convention as every other OPNsense-validated field in this app.
listen_port: Mapped[str] = mapped_column(String(8), default="51820")
# Server.tunneladdress (NetworkField, NetMaskRequired=Y, AsList=Y) —
# this interface's own tunnel address in CIDR notation. v1 keeps this
# to a single CIDR (no comma-list UI), same simplification precedent
# as IpsecTunnel.local_networks/OpenVpnInstance.server_network.
server_tunnel_address: Mapped[str] = mapped_column(String(255))
# Server.dns / .mtu — both optional in the OPNsense model.
dns: Mapped[str] = mapped_column(String(255), default="")
mtu: Mapped[str] = mapped_column(String(8), default="")
# ── Peer (Wireguard\Client.xml clients.client fields) — v1 pins
# exactly one peer per tunnel, see class docstring.
peer_name: Mapped[str] = mapped_column(String(64))
peer_public_key: Mapped[str] = mapped_column(String(64))
peer_psk_enabled: Mapped[bool] = mapped_column(Boolean, default=False)
peer_psk_set: Mapped[bool] = mapped_column(Boolean, default=False)
# Client.tunneladdress (NetworkField, NetMaskRequired=Y, Required=Y,
# AsList=Y) — this peer's Allowed-IPs entry in the server's config.
# Single CIDR only in v1, same precedent as above.
peer_allowed_ips: Mapped[str] = mapped_column(String(255))
# Client.serveraddress/.serverport — the Endpoint (host/port) this
# OPNsense box should dial to reach the peer. Optional: a peer that
# only ever connects inward (e.g. a road-warrior-style client with no
# fixed address) legitimately has neither set — see module docstring
# for why these are believed (not independently confirmed beyond
# field placement/naming in Client.xml) to be the real, persisted
# Endpoint fields, as opposed to the sibling `endpoint` field, which
# is explicitly marked `volatile="true"` in the model XML (i.e.
# runtime/status-only, never written to config).
peer_endpoint_address: Mapped[str] = mapped_column(String(255), default="")
peer_endpoint_port: Mapped[str] = mapped_column(String(8), default="")
# Client.keepalive (IntegerField, 1-65535) — optional persistent
# keepalive interval in seconds.
peer_keepalive: Mapped[str] = mapped_column(String(8), default="")
enabled: Mapped[bool] = mapped_column(Boolean, default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
default=lambda: datetime.now(timezone.utc),
onupdate=lambda: datetime.now(timezone.utc),
)
targets: Mapped[list["OpnsenseHost"]] = relationship("OpnsenseHost", secondary=wireguard_tunnel_targets)
groups: Mapped[list["DeviceGroup"]] = relationship("DeviceGroup", secondary=wireguard_tunnel_groups)
deployments: Mapped[list["WireguardTunnelDeployment"]] = relationship(
"WireguardTunnelDeployment", back_populates="tunnel", cascade="all, delete-orphan"
)
backend/app/models/wireguard_tunnel_deployment.py
+61
View File
@@ -0,0 +1,61 @@
import uuid
from datetime import datetime, timezone
from sqlalchemy import JSON, DateTime, ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.database import Base
class WireguardTunnelDeployment(Base):
"""Last known deploy result of one WireguardTunnel (one Server + one
Peer, see WireguardTunnel's docstring) on one host.
Two opnsense_*_uuid columns, mirroring OpenVpnInstanceDeployment's
shape: `opnsense_server_uuid` (Wireguard\\Server) is written FIRST
(the Peer's payload needs to reference it via the `servers` field —
see wireguard_deploy_service.py for why getting this order wrong would
silently produce an unlinked peer), `opnsense_peer_uuid`
(Wireguard\\Client) required to target setClient/delClient there later.
status: "success" (both objects written AND the follow-up reconfigure
call was confirmed) or "error". Deliberately NO "pending"/"reverted"/
"revert_failed" here — see services/wireguard_deploy_service.py's
module docstring for why this entity has no auto-revert safety net
(same rationale as IpsecTunnelDeployment/OpenVpnInstanceDeployment: a
WireGuard reconfigure can cut off the very management path OPNexus
would need to auto-revert it).
A host with no WireguardTunnelDeployment row at all is this tunnel's
"draft" state on that host (never pushed there) — surfaced by
wireguard_service.preview()'s action="create" for that row, not by a
persisted status value.
"""
__tablename__ = "wireguard_tunnel_deployments"
__table_args__ = (UniqueConstraint("tunnel_id", "host_id", name="uq_wireguard_tunnel_host"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
tunnel_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("wireguard_tunnels.id", ondelete="CASCADE")
)
host_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE")
)
status: Mapped[str] = mapped_column(String(16))
message: Mapped[str | None] = mapped_column(String(500), nullable=True)
opnsense_server_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
opnsense_peer_uuid: Mapped[str | None] = mapped_column(String(64), nullable=True)
deployed_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)
)
# The OPNsense-shaped payloads (keyed "server"/"peer") last
# successfully written to this host — compared against a fresh
# deploy's payloads to detect a true no-op re-deploy. Like the other
# two VPN features, deliberately no previous_payload: nothing here is
# ever restored automatically.
last_payload: Mapped[dict | None] = mapped_column(JSON, nullable=True)
tunnel: Mapped["WireguardTunnel"] = relationship("WireguardTunnel", back_populates="deployments")
host: Mapped["OpnsenseHost"] = relationship("OpnsenseHost")
backend/app/models/wireguard_tunnel_target.py
+11
View File
@@ -0,0 +1,11 @@
from sqlalchemy import Column, ForeignKey, Table
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from app.database import Base
wireguard_tunnel_targets = Table(
"wireguard_tunnel_targets",
Base.metadata,
Column("tunnel_id", PGUUID(as_uuid=True), ForeignKey("wireguard_tunnels.id", ondelete="CASCADE"), primary_key=True),
Column("host_id", PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), primary_key=True),
)
backend/app/reset_2fa.py
+38
View File
@@ -0,0 +1,38 @@
"""Emergency recovery: switch off two-factor sign-in for one account.
For the case where the only admin lost both their authenticator and their
recovery codes (no other admin can use the UI reset). Needs shell access to the
server, which is the trust boundary for this tool anyway:
docker compose exec backend python -m app.reset_2fa <username>
"""
import asyncio
import sys
from sqlalchemy import select
from app.database import SessionLocal
from app.models.user import AdminUser
from app.services import totp_service
async def main(username: str) -> int:
async with SessionLocal() as db:
user = (await db.execute(select(AdminUser).where(AdminUser.username == username))).scalar_one_or_none()
if user is None:
print(f"No such user: {username}")
return 1
totp_service.delete_secrets(user.id)
user.totp_enabled = False
user.totp_recovery_hashes = None
user.totp_last_step = 0
await db.commit()
print(f"Two-factor sign-in disabled for {username}.")
return 0
if __name__ == "__main__":
if len(sys.argv) != 2:
print("usage: python -m app.reset_2fa <username>")
sys.exit(2)
sys.exit(asyncio.run(main(sys.argv[1])))
backend/app/routers/__init__.py
View File
Whitespace-only changes.
backend/app/routers/aliases.py
+232
View File
@@ -0,0 +1,232 @@
import uuid
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, AliasRevision
from app.services.policy_service import effective_hosts, policy_lock, ensure_editable
from app.services.alias_policy_service import ALIAS_FIELDS, snapshot, revision, preview, workflow
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.schemas.alias import DeploymentResult, AliasCreate, AliasRead, AliasEdit, DeployApproval
from app.services.alias_deploy_service import confirm_alias, deploy_alias, undeploy_alias, revert_alias_deployment
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
router = APIRouter(prefix="/aliases", tags=["aliases"])
def _load_options():
return [
selectinload(FirewallAlias.targets),
selectinload(FirewallAlias.groups).selectinload(DeviceGroup.hosts),
selectinload(FirewallAlias.deployments).selectinload(AliasDeployment.host),
]
async def _load_alias(db: AsyncSession, alias_id: uuid.UUID) -> FirewallAlias:
await policy_lock(db)
alias = await db.get(FirewallAlias, alias_id, options=_load_options())
if not alias:
raise api_error(404, ErrorCode.ALIAS_NICHT_GEFUNDEN)
await hydrate_hierarchy_targets(db, [alias])
return alias
def _to_read(alias: FirewallAlias, hosts: list[OpnsenseHost], deployments: list[AliasDeployment]) -> AliasRead:
hosts_by_id = {host.id: host for host in effective_hosts(alias)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(alias, deployments)
return AliasRead(
workflow_state=state,
workflow_counts=counts,
id=alias.id,
name=alias.name,
type=alias.type,
content=alias.content,
description=alias.description,
enabled=alias.enabled,
created_at=alias.created_at,
updated_at=alias.updated_at,
target_host_ids=[host.id for host in alias.targets],
target_group_ids=[g.id for g in alias.groups],
policy_hierarchy_group_id=alias.policy_hierarchy_group_id,
effective_host_ids=[h.id for h in effective_hosts(alias)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in deployments
],
)
@router.get("", response_model=list[AliasRead])
async def list_aliases(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(FirewallAlias).options(*_load_options()))
aliases = result.scalars().unique().all()
await hydrate_hierarchy_targets(db, aliases)
return [_to_read(alias, alias.targets, alias.deployments) for alias in aliases]
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.post("", response_model=AliasRead, status_code=201)
async def create_alias(payload: AliasCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
existing = await db.scalar(select(FirewallAlias).where(FirewallAlias.name == payload.name))
if existing:
raise api_error(409, ErrorCode.ALIAS_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
alias = FirewallAlias(
name=payload.name,
type=payload.type,
content=payload.content,
description=payload.description,
enabled=payload.enabled,
targets=hosts,
groups=groups,
policy_hierarchy_group_id=payload.policy_hierarchy_group_id,
)
db.add(alias)
await db.flush()
revision(db, alias, "Alias angelegt")
await db.commit()
await hydrate_hierarchy_targets(db, [alias])
return _to_read(alias, hosts, [])
@router.post("/{alias_id}/deploy", response_model=AliasRead)
async def deploy(alias_id: uuid.UUID, approval: DeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
plan = preview(alias)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in alias.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(alias) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.ALIAS_HAT_KEINE_ZIEL_HOSTS)
await deploy_alias(db, alias, hosts)
await db.refresh(alias, attribute_names=["deployments"])
for d in alias.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(alias, hosts, alias.deployments)
@router.post("/{alias_id}/confirm", response_model=AliasRead)
async def confirm(alias_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
await confirm_alias(db, alias)
return _to_read(alias, alias.targets, alias.deployments)
@router.delete("/{alias_id}", status_code=204)
async def delete_alias(alias_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
if any(d.status in ("pending", "revert_failed") for d in alias.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_alias(db, alias)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(alias, attribute_names=["deployments"])
await db.delete(alias)
await db.commit()
@router.put("/{alias_id}", response_model=AliasRead)
async def edit_alias(alias_id: uuid.UUID, payload: AliasEdit, db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
ensure_editable(alias)
if alias.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.ALIAS_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_5A2C1888)
duplicate = await db.scalar(select(FirewallAlias).where(FirewallAlias.name == payload.name, FirewallAlias.id != alias_id))
if duplicate:
raise api_error(409, ErrorCode.ALIAS_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
if not await db.scalar(select(AliasRevision.id).where(AliasRevision.alias_id == alias.id).limit(1)):
revision(db, alias, "Vor erster Bearbeitung")
for field in ALIAS_FIELDS:
setattr(alias, field, getattr(payload, field))
alias.targets, alias.groups = hosts, groups
alias.policy_hierarchy_group_id = payload.policy_hierarchy_group_id
from datetime import datetime, timezone
alias.updated_at = datetime.now(timezone.utc)
revision(db, alias, "Entwurf gespeichert")
await db.commit()
await hydrate_hierarchy_targets(db, [alias])
return _to_read(alias, hosts, alias.deployments)
@router.get("/{alias_id}/preview")
async def preview_alias(alias_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_alias(db, alias_id))
@router.get("/{alias_id}/revisions")
async def list_revisions(alias_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_alias(db, alias_id)
rows = (await db.scalars(select(AliasRevision).where(AliasRevision.alias_id == alias_id).order_by(AliasRevision.created_at.desc(), AliasRevision.id))).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{alias_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(alias_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
plan = preview(alias)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in alias.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_alias_item(deployment.host, deployment.opnsense_alias_uuid)
await opnsense_client.reconfigure_aliases(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/{alias_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(alias_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
alias = await _load_alias(db, alias_id)
plan = preview(alias)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in alias.deployments if d.host_id == host_id)
reverted = await revert_alias_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
backend/app/routers/audit.py
+28
View File
@@ -0,0 +1,28 @@
from datetime import datetime
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select, func, or_
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.audit import AuditEvent
router = APIRouter(prefix="/audit", tags=["audit"])
@router.get("")
async def history(kind: str | None = None, host_id: str | None = None, entity_id: str | None = None,
status: str | None = None, action: str | None = None, actor: str | None = None,
before: datetime | None = None, since: datetime | None = None,
q: str = Query(default="", max_length=200), offset: int = Query(default=0, ge=0),
limit: int = Query(default=50, ge=1, le=200), db: AsyncSession = Depends(get_db)):
clauses = []
for column, value in [(AuditEvent.kind,kind), (AuditEvent.host_id,host_id), (AuditEvent.entity_id,entity_id),
(AuditEvent.status,status), (AuditEvent.action,action), (AuditEvent.actor,actor)]:
if value:
clauses.append(column == value)
if before: clauses.append(AuditEvent.created_at <= before)
if since: clauses.append(AuditEvent.created_at >= since)
if q:
clauses.append(or_(*(c.ilike('%'+q+'%') for c in [AuditEvent.entity_name, AuditEvent.host_name, AuditEvent.actor, AuditEvent.message])))
total = await db.scalar(select(func.count()).select_from(AuditEvent).where(*clauses))
rows = (await db.scalars(select(AuditEvent).where(*clauses).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).offset(offset).limit(limit))).all()
return {"total": total, "offset": offset, "limit": limit,
"items": [{c.name: getattr(row, c.name) for c in AuditEvent.__table__.columns} for row in rows]}
backend/app/routers/auth.py
+241
View File
@@ -0,0 +1,241 @@
import logging
from fastapi import APIRouter, Depends, Request, Response, status
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.config import settings
from app.database import get_db
from app.models.user import AdminUser
from app.services import auth_service, totp_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/auth", tags=["auth"])
COOKIE_NAME = "opnexus-session"
def client_ip(request: Request) -> str:
if settings.trust_proxy_headers:
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
# First entry is the original client; a trusted proxy
# overwrites (not appends blindly to) this header, so this
# is safe only under the condition documented on the config
# flag itself.
return forwarded.split(",")[0].strip()
return request.client.host if request.client else "unknown"
def _set_session_cookie(response: Response, token: str) -> None:
response.set_cookie(
key=COOKIE_NAME,
value=token,
max_age=settings.session_lifetime,
httponly=True,
samesite="lax",
secure=settings.cookie_secure,
path="/",
)
def _clear_session_cookie(response: Response) -> None:
response.delete_cookie(key=COOKIE_NAME, path="/")
async def resolve_user(request: Request, db: AsyncSession) -> AdminUser:
token = request.cookies.get(COOKIE_NAME, "")
user = await auth_service.resolve_session(db, token)
if user is None:
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.NICHT_AUTHENTIFIZIERT)
return user
async def current_user(request: Request, db: AsyncSession = Depends(get_db)) -> AdminUser:
return await resolve_user(request, db)
class LoginRequest(BaseModel):
username: str = Field(min_length=1, max_length=150)
password: str = Field(min_length=1, max_length=1024)
# Authenticator code or one-time recovery code; only needed (and only
# requested) once the password was right and the account has 2FA enabled.
totp_code: str | None = Field(default=None, max_length=32)
class ChangePasswordRequest(BaseModel):
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=10, max_length=1024)
@router.post("/login")
async def login(payload: LoginRequest, request: Request, response: Response, db: AsyncSession = Depends(get_db)):
ip = client_ip(request)
wait = auth_service.throttle.retry_after(ip, payload.username)
if wait > 0:
raise api_error(status.HTTP_429_TOO_MANY_REQUESTS, ErrorCode.TOO_MANY_LOGIN_ATTEMPTS, wait=wait)
user = await auth_service.authenticate(db, payload.username, payload.password, ip, finalize=False)
if user is None:
auth_service.throttle.record_failure(ip, payload.username)
logger.info("Failed login for '%s' from %s", payload.username, ip)
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.UNGUELTIGER_BENUTZERNAME_ODER_PASSWORT)
if user.totp_enabled:
# Only revealed after the password was correct, so it cannot be used
# to find out which usernames exist or have 2FA.
if not payload.totp_code:
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_REQUIRED)
if not await _accept_second_factor(db, user, payload.totp_code):
auth_service.throttle.record_failure(ip, payload.username)
logger.info("Failed second factor for '%s' from %s", payload.username, ip)
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
await auth_service.finalize_login(db, user, ip)
auth_service.throttle.reset(ip, payload.username)
failed_since_last_login = auth_service.throttle.pop_failures_since_success(payload.username)
token, expires_at = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=ip
)
_set_session_cookie(response, token)
logger.info("Successful login for '%s' from %s", user.username, ip)
return {
"authenticated": True,
"username": user.username,
"role": user.role,
"previous_login_at": getattr(user, "previous_login_at", None),
"previous_login_ip": getattr(user, "previous_login_ip", None),
"failed_attempts_since_last_login": failed_since_last_login,
"session_expires_at": expires_at,
}
async def _accept_second_factor(db: AsyncSession, user: AdminUser, code: str) -> bool:
"""Authenticator code (replay-protected) or, failing that, a one-time recovery code."""
secret = totp_service.read_secret(user.id)
if secret:
step = totp_service.verify_code(secret, code, user.totp_last_step)
if step is not None:
user.totp_last_step = step
await db.commit()
return True
remaining = totp_service.consume_recovery(user.totp_recovery_hashes, code)
if remaining is not None:
user.totp_recovery_hashes = remaining
await db.commit()
logger.warning("Recovery code used by '%s' (%d left)", user.username, len(remaining))
return True
return False
class TotpSetupRequest(BaseModel):
password: str = Field(min_length=1, max_length=1024)
class TotpEnableRequest(BaseModel):
code: str = Field(min_length=1, max_length=32)
class TotpDisableRequest(BaseModel):
password: str = Field(min_length=1, max_length=1024)
code: str = Field(min_length=1, max_length=32)
@router.get("/2fa")
async def totp_status(user: AdminUser = Depends(current_user)):
return {"enabled": user.totp_enabled, "recovery_codes_left": len(user.totp_recovery_hashes or [])}
@router.post("/2fa/setup")
async def totp_setup(payload: TotpSetupRequest, user: AdminUser = Depends(current_user)):
"""Step 1: needs the current password (a stolen session alone must not be able
to add an attacker's authenticator). Returns the secret to scan; nothing is
active until /2fa/enable proves the authenticator works."""
if user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED)
if not auth_service.verify_password(user.password_hash, payload.password):
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
secret = totp_service.generate_secret()
totp_service.save_secret(user.id, secret, pending=True)
return {"secret": secret, "otpauth_uri": totp_service.otpauth_uri(user.username, secret)}
@router.post("/2fa/enable")
async def totp_enable(payload: TotpEnableRequest, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
if user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_ALREADY_ENABLED)
pending = totp_service.read_secret(user.id, pending=True)
if not pending:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_SETUP_MISSING)
step = totp_service.verify_code(pending, payload.code)
if step is None:
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
plain, hashes = totp_service.new_recovery_codes()
totp_service.promote_pending(user.id)
user.totp_enabled = True
user.totp_last_step = step
user.totp_recovery_hashes = hashes
await db.commit()
logger.info("Two-factor enabled for '%s'", user.username)
return {"enabled": True, "recovery_codes": plain}
@router.post("/2fa/disable")
async def totp_disable(payload: TotpDisableRequest, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
if not user.totp_enabled:
raise api_error(status.HTTP_409_CONFLICT, ErrorCode.TOTP_NOT_ENABLED)
if not auth_service.verify_password(user.password_hash, payload.password):
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
if not await _accept_second_factor(db, user, payload.code):
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.TOTP_INVALID)
totp_service.delete_secrets(user.id)
user.totp_enabled = False
user.totp_recovery_hashes = None
user.totp_last_step = 0
await db.commit()
logger.info("Two-factor disabled for '%s'", user.username)
return {"enabled": False}
@router.post("/logout")
async def logout(request: Request, response: Response, db: AsyncSession = Depends(get_db)):
token = request.cookies.get(COOKIE_NAME, "")
await auth_service.revoke_session(db, token)
_clear_session_cookie(response)
return {"authenticated": False}
@router.get("/me")
async def me(request: Request, db: AsyncSession = Depends(get_db)):
token = request.cookies.get(COOKIE_NAME, "")
user = await auth_service.resolve_session(db, token)
if user is None:
return {"authenticated": False}
return {"authenticated": True, "username": user.username, "role": user.role}
@router.post("/change-password")
async def change_password(
payload: ChangePasswordRequest,
response: Response,
request: Request,
db: AsyncSession = Depends(get_db),
user: AdminUser = Depends(current_user),
):
if not auth_service.verify_password(user.password_hash, payload.current_password):
raise api_error(status.HTTP_401_UNAUTHORIZED, ErrorCode.AKTUELLES_PASSWORT_IST_FALSCH)
if payload.new_password == payload.current_password:
raise api_error(status.HTTP_400_BAD_REQUEST, ErrorCode.DAS_NEUE_PASSWORT_MUSS_SICH_VOM_AKTUELLEN_UNTERSCHEIDEN)
user.password_hash = auth_service.hash_password(payload.new_password)
await db.commit()
await auth_service.revoke_session(db, request.cookies.get(COOKIE_NAME, ""))
token, _ = await auth_service.create_session(
db, user, user_agent=request.headers.get("User-Agent"), ip_address=client_ip(request)
)
_set_session_cookie(response, token)
return {"ok": True}
backend/app/routers/certificates.py
+301
View File
@@ -0,0 +1,301 @@
"""Trust-Store (`ManagedCertificate`) + ACME (`AcmeAccount`,
`AcmeCertificateRequest`) management routers. Deliberately simpler shape
than ipsec.py/openvpn.py/wireguard.py: single-host (no target
resolution), no preview/revision/workflow machinery (no multi-host
no-op/drift concept applies to a single fixed host), and — for
Trust-Store only — no deploy warning gate (see managed_certificate_
service.py's module docstring for why). ACME's `register`/`sign` DO get
a distinct warning in the frontend (AcmeActionConfirm.tsx) even though
the backend endpoints here are plain POSTs — the two-step confirmation is
a UI-level gate, the same architectural choice ipsec.py/openvpn.py/
wireguard.py make for their own "push" action.
"""
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.acme_account import AcmeAccount
from app.models.acme_certificate_request import AcmeCertificateRequest
from app.models.host import OpnsenseHost
from app.models.managed_certificate import ManagedCertificate
from app.schemas.certificates import (
AcmeAccountCreate, AcmeAccountEdit, AcmeAccountRead,
AcmeCertificateRequestCreate, AcmeCertificateRequestEdit, AcmeCertificateRequestRead,
ManagedCertificateCreate, ManagedCertificateEdit, ManagedCertificateRead,
)
from app.services import acme_secret_store, acme_service, cert_key_store, managed_certificate_service
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_service import policy_lock
managed_router = APIRouter(prefix="/certificates/managed", tags=["certificates"])
acme_accounts_router = APIRouter(prefix="/acme/accounts", tags=["acme"])
acme_certificates_router = APIRouter(prefix="/acme/certificates", tags=["acme"])
async def _get_host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(400, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
async def _get_enabled_host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await _get_host(db, host_id)
if not host.enabled:
raise api_error(400, ErrorCode.HOST_IST_DEAKTIVIERT)
return host
# ── Trust-Store certificates ──────────────────────────────────────────
@managed_router.get("", response_model=list[ManagedCertificateRead])
async def list_managed_certificates(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(ManagedCertificate).order_by(ManagedCertificate.created_at.desc()))).all()
return rows
@managed_router.post("", response_model=ManagedCertificateRead, status_code=201)
async def create_managed_certificate(payload: ManagedCertificateCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _get_host(db, payload.host_id)
data = payload.model_dump(exclude={"private_key"})
cert = ManagedCertificate(**data)
db.add(cert)
await db.flush()
if payload.private_key:
cert_key_store.save_secret(cert.id, payload.private_key)
cert.private_key_set = True
await db.commit()
return cert
@managed_router.put("/{cert_id}", response_model=ManagedCertificateRead)
async def edit_managed_certificate(cert_id: uuid.UUID, payload: ManagedCertificateEdit, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
cert = await db.get(ManagedCertificate, cert_id)
if not cert:
raise api_error(404, ErrorCode.ZERTIFIKAT_NICHT_GEFUNDEN)
if cert.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
for field, value in payload.model_dump(exclude={"expected_updated_at", "private_key"}).items():
setattr(cert, field, value)
if payload.private_key:
cert_key_store.save_secret(cert.id, payload.private_key)
cert.private_key_set = True
await db.commit()
return cert
@managed_router.post("/{cert_id}/push", response_model=ManagedCertificateRead)
async def push_managed_certificate(cert_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
cert = await db.get(ManagedCertificate, cert_id)
if not cert:
raise api_error(404, ErrorCode.ZERTIFIKAT_NICHT_GEFUNDEN)
host = await _get_enabled_host(db, cert.host_id)
await ensure_no_maintenance(db, [host])
return await managed_certificate_service.push_certificate(db, cert, host)
@managed_router.delete("/{cert_id}", status_code=204)
async def delete_managed_certificate(cert_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
cert = await db.get(ManagedCertificate, cert_id)
if not cert:
raise api_error(404, ErrorCode.ZERTIFIKAT_NICHT_GEFUNDEN)
host = await db.get(OpnsenseHost, cert.host_id)
try:
await managed_certificate_service.delete_certificate(db, cert, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
# ── ACME accounts ──────────────────────────────────────────────────────
@acme_accounts_router.get("", response_model=list[AcmeAccountRead])
async def list_acme_accounts(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(AcmeAccount).order_by(AcmeAccount.created_at.desc()))).all()
return rows
@acme_accounts_router.post("", response_model=AcmeAccountRead, status_code=201)
async def create_acme_account(payload: AcmeAccountCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _get_host(db, payload.host_id)
data = payload.model_dump(exclude={"eab_hmac"})
account = AcmeAccount(**data)
db.add(account)
await db.flush()
if payload.eab_hmac:
acme_secret_store.save_secret(account.id, payload.eab_hmac)
account.eab_hmac_set = True
await db.commit()
return account
@acme_accounts_router.put("/{account_id}", response_model=AcmeAccountRead)
async def edit_acme_account(account_id: uuid.UUID, payload: AcmeAccountEdit, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
account = await db.get(AcmeAccount, account_id)
if not account:
raise api_error(404, ErrorCode.ACCOUNT_NICHT_GEFUNDEN)
if account.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
for field, value in payload.model_dump(exclude={"expected_updated_at", "eab_hmac"}).items():
setattr(account, field, value)
if payload.eab_hmac:
acme_secret_store.save_secret(account.id, payload.eab_hmac)
account.eab_hmac_set = True
await db.commit()
return account
@acme_accounts_router.post("/{account_id}/push", response_model=AcmeAccountRead)
async def push_acme_account(account_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
account = await db.get(AcmeAccount, account_id)
if not account:
raise api_error(404, ErrorCode.ACCOUNT_NICHT_GEFUNDEN)
host = await _get_enabled_host(db, account.host_id)
await ensure_no_maintenance(db, [host])
return await acme_service.push_account(db, account, host)
@acme_accounts_router.post("/{account_id}/register", response_model=AcmeAccountRead)
async def register_acme_account(account_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""REAL, blocking network call to the configured external CA — see
acme_service.register_account()'s docstring. The frontend gates this
behind an explicit two-step warning (AcmeActionConfirm.tsx); this
endpoint itself fires immediately on request, same as ipsec.py's
deploy endpoint firing immediately once the frontend has armed it."""
account = await db.get(AcmeAccount, account_id)
if not account:
raise api_error(404, ErrorCode.ACCOUNT_NICHT_GEFUNDEN)
if not account.opnsense_uuid:
raise api_error(400, ErrorCode.ACCOUNT_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_EEEA04A9)
host = await _get_enabled_host(db, account.host_id)
try:
return await acme_service.register_account(db, account, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@acme_accounts_router.delete("/{account_id}", status_code=204)
async def delete_acme_account(account_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
account = await db.get(AcmeAccount, account_id)
if not account:
raise api_error(404, ErrorCode.ACCOUNT_NICHT_GEFUNDEN)
if await db.scalar(select(AcmeCertificateRequest.id).where(AcmeCertificateRequest.account_id == account_id).limit(1)):
raise api_error(409, ErrorCode.ACCOUNT_WIRD_NOCH_VON_MINDESTENS_EINER_ZERTIFIKATSANFRAGE_REFERENZIERT)
host = await db.get(OpnsenseHost, account.host_id)
try:
await acme_service.delete_account(db, account, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
# ── ACME certificate requests ─────────────────────────────────────────
@acme_certificates_router.get("", response_model=list[AcmeCertificateRequestRead])
async def list_acme_certificate_requests(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(AcmeCertificateRequest).order_by(AcmeCertificateRequest.created_at.desc()))).all()
return rows
async def _get_account(db: AsyncSession, account_id: uuid.UUID, host_id: uuid.UUID) -> AcmeAccount:
account = await db.get(AcmeAccount, account_id)
if not account:
raise api_error(400, ErrorCode.ACME_ACCOUNT_NICHT_GEFUNDEN)
if account.host_id != host_id:
raise api_error(400, ErrorCode.ACME_ACCOUNT_GEHOERT_ZU_EINEM_ANDEREN_HOST)
return account
@acme_certificates_router.post("", response_model=AcmeCertificateRequestRead, status_code=201)
async def create_acme_certificate_request(payload: AcmeCertificateRequestCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _get_host(db, payload.host_id)
await _get_account(db, payload.account_id, payload.host_id)
req = AcmeCertificateRequest(**payload.model_dump())
db.add(req)
await db.commit()
return req
@acme_certificates_router.put("/{request_id}", response_model=AcmeCertificateRequestRead)
async def edit_acme_certificate_request(request_id: uuid.UUID, payload: AcmeCertificateRequestEdit, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
req = await db.get(AcmeCertificateRequest, request_id)
if not req:
raise api_error(404, ErrorCode.ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN)
if req.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.EINTRAG_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_B5A5CE87)
await _get_host(db, payload.host_id)
await _get_account(db, payload.account_id, payload.host_id)
for field, value in payload.model_dump(exclude={"expected_updated_at"}).items():
setattr(req, field, value)
await db.commit()
return req
@acme_certificates_router.post("/{request_id}/push", response_model=AcmeCertificateRequestRead)
async def push_acme_certificate_request(request_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
req = await db.get(AcmeCertificateRequest, request_id)
if not req:
raise api_error(404, ErrorCode.ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN)
host = await _get_enabled_host(db, req.host_id)
account = await db.get(AcmeAccount, req.account_id)
if not account:
raise api_error(400, ErrorCode.VERKNUEPFTER_ACME_ACCOUNT_NICHT_GEFUNDEN)
await ensure_no_maintenance(db, [host])
return await acme_service.push_certificate_request(db, req, host, account)
@acme_certificates_router.post("/{request_id}/sign", response_model=AcmeCertificateRequestRead)
async def sign_acme_certificate_request(request_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""REAL, asynchronous network call to the configured external CA — see
acme_service.sign_certificate_request()'s docstring. Frontend-gated
the same way register_acme_account() above is."""
req = await db.get(AcmeCertificateRequest, request_id)
if not req:
raise api_error(404, ErrorCode.ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN)
if not req.opnsense_uuid:
raise api_error(400, ErrorCode.ZERTIFIKATSANFRAGE_MUSS_ZUERST_AUF_DIE_FIREWALL_GESCHRIEBEN_WERDEN_F684C449)
host = await _get_enabled_host(db, req.host_id)
try:
return await acme_service.sign_certificate_request(db, req, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@acme_certificates_router.post("/{request_id}/poll", response_model=AcmeCertificateRequestRead)
async def poll_acme_certificate_request(request_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
req = await db.get(AcmeCertificateRequest, request_id)
if not req:
raise api_error(404, ErrorCode.ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN)
if not req.opnsense_uuid:
raise api_error(400, ErrorCode.ZERTIFIKATSANFRAGE_WURDE_NOCH_NICHT_AUF_DIE_FIREWALL_GESCHRIEBEN)
host = await _get_host(db, req.host_id)
try:
return await acme_service.poll_certificate_status(db, req, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@acme_certificates_router.delete("/{request_id}", status_code=204)
async def delete_acme_certificate_request(request_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
req = await db.get(AcmeCertificateRequest, request_id)
if not req:
raise api_error(404, ErrorCode.ZERTIFIKATSANFRAGE_NICHT_GEFUNDEN)
host = await db.get(OpnsenseHost, req.host_id)
try:
await acme_service.delete_certificate_request(db, req, host)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
backend/app/routers/cluster.py
+22
View File
@@ -0,0 +1,22 @@
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.carp_event import CarpEvent
from app.schemas.carp_event import CarpEventRead
router = APIRouter(prefix="/cluster", tags=["cluster"])
@router.get("/carp-events", response_model=list[CarpEventRead])
async def list_carp_events(
cluster_name: str | None = None,
limit: int = Query(default=50, ge=1, le=500),
db: AsyncSession = Depends(get_db),
):
clauses = [] if cluster_name is None else [CarpEvent.cluster_name == cluster_name]
result = await db.execute(
select(CarpEvent).where(*clauses).order_by(CarpEvent.detected_at.desc()).limit(limit)
)
return list(result.scalars().all())
backend/app/routers/dashboard.py
+224
View File
@@ -0,0 +1,224 @@
"""Fleet-wide "what needs my attention" summary for the frontend dashboard
(app/dashboard/page.tsx). Deliberately DB-only — no live OPNsense calls —
so this endpoint stays fast and doesn't fail just because one host is
offline; the dashboard fetches live-only data (certificate expiry, firmware
update availability) itself per host via the existing
/hosts/{id}/certificates and /hosts/{id}/maintenance endpoints, exactly the
way the Certificates and Update Center pages already do, so a slow/offline
host degrades only that one panel instead of the whole page.
"Pending actions" aggregates the same three attention-worthy deployment
statuses ("pending" awaiting confirmation, "error" the deploy call itself
failed, "revert_failed" the rollback call failed) across all deployable
entity kinds (rules, aliases, the three NAT types, policy-hierarchy rules,
IPsec/OpenVPN/WireGuard — the VPN kinds only ever reach "error") plus any host_operations still awaiting manual confirmation
(status in "starting"/"submitted"/"unknown" — mirrors
HostMaintenance.tsx's own definition of "needs a click"). Restore-operation
locks are deliberately NOT included here (no fleet-wide list endpoint
exists for those, only per-host, and it's a rarely-triggered flow) —
a documented v1 gap, not an oversight.
"""
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.models.audit import AuditEvent
from app.models.carp_maintenance_window import CarpMaintenanceWindow
from app.models.host import OpnsenseHost
from app.models.host_operation import HostOperation
from app.models.ipsec_tunnel import IpsecTunnel
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.outbound_nat import OutboundNatRule
from app.models.openvpn_instance import OpenVpnInstance
from app.models.openvpn_instance_deployment import OpenVpnInstanceDeployment
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.models.wireguard_tunnel import WireguardTunnel
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
from app.schemas.dashboard import ActivityItem, DashboardHost, DashboardSummary, PendingAction
from app.services.host_status_service import get_statuses
router = APIRouter(prefix="/dashboard", tags=["dashboard"])
ATTENTION_STATUSES = ("pending", "error", "revert_failed")
STUCK_OPERATION_STATUSES = ("starting", "submitted", "unknown")
OPERATION_LABELS = {
"reboot": "Neustart",
"update": "Firmware-Update",
"major_update": "Major-Upgrade",
"carp_enter": "CARP-Übergabe",
"carp_leave": "CARP-Normalbetrieb",
}
async def _rule_actions(db: AsyncSession) -> list[PendingAction]:
rows = (
await db.execute(
select(RuleDeployment, FirewallRule.description, OpnsenseHost.name)
.join(FirewallRule, RuleDeployment.rule_id == FirewallRule.id)
.join(OpnsenseHost, RuleDeployment.host_id == OpnsenseHost.id)
.where(RuleDeployment.status.in_(ATTENTION_STATUSES))
)
).all()
return [
PendingAction(
kind="rule", status=dep.status, entity_name=description, host_name=host_name,
message=dep.message, created_at=dep.deployed_at, link=f"/rules?rule={dep.rule_id}",
)
for dep, description, host_name in rows
]
async def _nat_actions(
db: AsyncSession, deployment_model, rule_model, fk_attr: str, kind: str, nat_type: str
) -> list[PendingAction]:
fk_column = getattr(deployment_model, fk_attr)
rows = (
await db.execute(
select(deployment_model, rule_model.description, OpnsenseHost.name)
.join(rule_model, fk_column == rule_model.id)
.join(OpnsenseHost, deployment_model.host_id == OpnsenseHost.id)
.where(deployment_model.status.in_(ATTENTION_STATUSES))
)
).all()
return [
PendingAction(
kind=kind, status=dep.status, entity_name=description, host_name=host_name,
message=dep.message, created_at=dep.deployed_at,
link=f"/nat?type={nat_type}&rule={getattr(dep, fk_attr)}",
)
for dep, description, host_name in rows
]
async def _simple_actions(
db: AsyncSession, deployment_model, parent_model, fk_attr: str, name_column, kind: str, link_template: str
) -> list[PendingAction]:
"""Same shape as _nat_actions for kinds whose entity label is not
`description` and whose page takes a different query parameter;
`link_template` is formatted with the parent id."""
rows = (
await db.execute(
select(deployment_model, name_column, OpnsenseHost.name)
.join(parent_model, getattr(deployment_model, fk_attr) == parent_model.id)
.join(OpnsenseHost, deployment_model.host_id == OpnsenseHost.id)
.where(deployment_model.status.in_(ATTENTION_STATUSES))
)
).all()
return [
PendingAction(
kind=kind, status=dep.status, entity_name=name, host_name=host_name,
message=dep.message, created_at=dep.deployed_at, link=link_template.format(getattr(dep, fk_attr)),
)
for dep, name, host_name in rows
]
async def _policy_hierarchy_actions(db: AsyncSession) -> list[PendingAction]:
rows = (
await db.execute(
select(PolicyHierarchyRuleDeployment, PolicyHierarchyRule.description,
PolicyHierarchyRule.hierarchy_group_id, OpnsenseHost.name)
.join(PolicyHierarchyRule, PolicyHierarchyRuleDeployment.policy_hierarchy_id == PolicyHierarchyRule.id)
.join(OpnsenseHost, PolicyHierarchyRuleDeployment.host_id == OpnsenseHost.id)
.where(PolicyHierarchyRuleDeployment.status.in_(ATTENTION_STATUSES))
)
).all()
return [
PendingAction(
kind="policy_hierarchy", status=dep.status, entity_name=description, host_name=host_name,
message=dep.message, created_at=dep.deployed_at, link=f"/policy-hierarchy?group={group_id}",
)
for dep, description, group_id, host_name in rows
]
async def _stuck_host_operations(db: AsyncSession) -> list[PendingAction]:
rows = (
await db.scalars(
select(HostOperation)
.where(HostOperation.status.in_(STUCK_OPERATION_STATUSES))
.order_by(HostOperation.created_at.desc())
)
).all()
return [
PendingAction(
kind="host_operation", status=op.status,
entity_name=OPERATION_LABELS.get(op.action, op.action), host_name=op.host_name,
message=op.message, created_at=op.created_at, link=f"/hosts/{op.host_id}",
)
for op in rows
]
async def _carp_auto_revert_actions(db: AsyncSession) -> list[PendingAction]:
rows = (
await db.scalars(
select(CarpMaintenanceWindow)
.where(CarpMaintenanceWindow.status.in_(("active", "revert_failed")))
.order_by(CarpMaintenanceWindow.entered_at.desc())
)
).all()
actions = []
for window in rows:
if window.status == "active":
message = f"Automatischer Rückschalt-Zeitpunkt: {window.auto_revert_at.strftime('%d.%m. %H:%M') if window.auto_revert_at else 'unbekannt'}"
else:
message = window.message
actions.append(PendingAction(
kind="carp_auto_revert", status=window.status, entity_name="CARP-Wartungsmodus",
host_name=window.host_name, message=message, created_at=window.entered_at,
link=f"/hosts/{window.host_id}",
))
return actions
@router.get("", response_model=DashboardSummary)
async def summary(db: AsyncSession = Depends(get_db)):
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
statuses = await get_statuses(hosts)
host_summaries = [
DashboardHost(
id=host.id, name=host.name, status=statuses.get(host.id, {}).get("status", "offline"),
carp_status=statuses.get(host.id, {}).get("carp_status"), cluster_name=host.cluster_name,
)
for host in hosts
]
pending_actions: list[PendingAction] = []
pending_actions += await _rule_actions(db)
pending_actions += await _nat_actions(db, PortForwardDeployment, PortForwardRule, "port_forward_id", "port_forward", "port-forward")
pending_actions += await _nat_actions(db, OneToOneDeployment, OneToOneRule, "one_to_one_id", "one_to_one", "one-to-one")
pending_actions += await _nat_actions(db, OutboundNatDeployment, OutboundNatRule, "outbound_nat_id", "outbound_nat", "outbound-nat")
pending_actions += await _policy_hierarchy_actions(db)
pending_actions += await _simple_actions(db, AliasDeployment, FirewallAlias, "alias_id", FirewallAlias.name, "alias", "/aliases?alias={}")
pending_actions += await _simple_actions(db, IpsecTunnelDeployment, IpsecTunnel, "tunnel_id", IpsecTunnel.description, "ipsec", "/ipsec?tunnel={}")
pending_actions += await _simple_actions(db, OpenVpnInstanceDeployment, OpenVpnInstance, "instance_id", OpenVpnInstance.description, "openvpn", "/openvpn?instance={}")
pending_actions += await _simple_actions(db, WireguardTunnelDeployment, WireguardTunnel, "tunnel_id", WireguardTunnel.server_name, "wireguard", "/wireguard?tunnel={}")
pending_actions += await _stuck_host_operations(db)
pending_actions += await _carp_auto_revert_actions(db)
pending_actions.sort(key=lambda item: item.created_at, reverse=True)
activity_rows = (
await db.scalars(select(AuditEvent).order_by(AuditEvent.created_at.desc(), AuditEvent.id.desc()).limit(15))
).all()
recent_activity = [
ActivityItem(
id=row.id, created_at=row.created_at, actor=row.actor, kind=row.kind, action=row.action,
status=row.status, entity_name=row.entity_name, host_name=row.host_name, message=row.message,
)
for row in activity_rows
]
return DashboardSummary(hosts=host_summaries, pending_actions=pending_actions[:30], recent_activity=recent_activity)
backend/app/routers/groups.py
+166
View File
@@ -0,0 +1,166 @@
import uuid
from app.errors import ErrorCode, api_error
from app.models.port_forward import PortForwardRule
from app.models.policy import port_forward_groups
from app.models.one_to_one import OneToOneRule
from app.models.policy import one_to_one_groups
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import outbound_nat_groups
import hashlib
import json
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, rule_groups, alias_groups
from app.models.rule import FirewallRule
from app.models.alias import FirewallAlias
from app.services.policy_service import policy_lock, effective_hosts
router = APIRouter(prefix="/groups", tags=["device-groups"])
class GroupWrite(BaseModel):
name: str = Field(min_length=1, max_length=120)
description: str = Field(default="", max_length=2000)
host_ids: list[uuid.UUID] = Field(default_factory=list)
@field_validator("name")
@classmethod
def clean_name(cls, value):
if not value.strip():
raise ValueError("Name darf nicht leer sein")
return value.strip()
class GroupUpdate(GroupWrite):
preview_token: str | None = Field(default=None, min_length=64, max_length=64)
def read(group):
return {"id": group.id, "name": group.name, "description": group.description,
"host_ids": [h.id for h in group.hosts]}
@router.get("")
async def list_groups(db: AsyncSession = Depends(get_db)):
groups = (await db.scalars(select(DeviceGroup).options(selectinload(DeviceGroup.hosts)).order_by(DeviceGroup.name))).all()
return [read(g) for g in groups]
async def save(db, payload, group=None):
duplicate = await db.scalar(select(DeviceGroup).where(DeviceGroup.name == payload.name))
if duplicate and (group is None or duplicate.id != group.id):
raise api_error(409, ErrorCode.GRUPPENNAME_BEREITS_VERGEBEN)
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.host_ids)))).all())
if len(hosts) != len(set(payload.host_ids)):
raise api_error(400, ErrorCode.ZIEL_HOST_NICHT_GEFUNDEN)
if group is None:
group = DeviceGroup()
db.add(group)
group.name, group.description, group.hosts = payload.name, payload.description, hosts
await db.commit()
return read(group)
@router.post("", status_code=201)
async def create_group(payload: GroupWrite, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
return await save(db, payload)
@router.put("/{group_id}")
async def update_group(group_id: uuid.UUID, payload: GroupUpdate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
group = await db.get(DeviceGroup, group_id, options=[selectinload(DeviceGroup.hosts)])
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
impact = await group_impact(db, group, payload)
if payload.preview_token and payload.preview_token != impact["token"]:
raise api_error(409, ErrorCode.GRUPPENMITGLIEDER_ODER_REGELN_WURDEN_GEAENDERT_AUSWIRKUNGEN_ERNEUT_PRUEFEN)
return await save(db, payload, group)
@router.delete("/{group_id}", status_code=204)
async def delete_group(group_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
group = await db.get(DeviceGroup, group_id)
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
if await db.scalar(select(rule_groups.c.rule_id).where(rule_groups.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_REGELN_VERWENDET)
if await db.scalar(select(alias_groups.c.alias_id).where(alias_groups.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_ALIASEN_VERWENDET)
for assignment in (port_forward_groups, one_to_one_groups, outbound_nat_groups):
if await db.scalar(select(assignment.c.group_id).where(assignment.c.group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_WIRD_NOCH_VON_NAT_REGELN_VERWENDET)
await db.delete(group)
await db.commit()
def _impact_rows(entities, group, proposed, opnsense_uuid_attr):
impacts = []
for entity in entities:
before = {h.id for h in effective_hosts(entity)}
after = {h.id for h in entity.targets}
for target_group in entity.groups:
after.update(h.id for h in (proposed if target_group.id == group.id else target_group.hosts))
removed = before - after
deployed = {d.host_id for d in entity.deployments if getattr(d, opnsense_uuid_attr)}
impacts.append({"added_host_ids": sorted(map(str, after - before)),
"removed_host_ids": sorted(map(str, removed)),
"retained_host_ids": sorted(map(str, removed & deployed))})
return impacts
async def group_impact(db, group, payload):
proposed = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.host_ids)))).all())
if len(proposed) != len(set(payload.host_ids)):
raise api_error(400, ErrorCode.ZIEL_HOST_NICHT_GEFUNDEN)
rules = (await db.scalars(select(FirewallRule).where(FirewallRule.groups.any(DeviceGroup.id == group.id)).options(
selectinload(FirewallRule.targets),
selectinload(FirewallRule.groups).selectinload(DeviceGroup.hosts),
selectinload(FirewallRule.deployments),
).order_by(FirewallRule.id))).all()
aliases = (await db.scalars(select(FirewallAlias).where(FirewallAlias.groups.any(DeviceGroup.id == group.id)).options(
selectinload(FirewallAlias.targets),
selectinload(FirewallAlias.groups).selectinload(DeviceGroup.hosts),
selectinload(FirewallAlias.deployments),
).order_by(FirewallAlias.id))).all()
rule_impacts = [{"rule_id": str(r.id), "description": r.description, **row}
for r, row in zip(rules, _impact_rows(rules, group, proposed, "opnsense_rule_uuid"))]
alias_impacts = [{"alias_id": str(a.id), "description": a.name, **row}
for a, row in zip(aliases, _impact_rows(aliases, group, proposed, "opnsense_alias_uuid"))]
nat_impacts = []
for model, kind in ((PortForwardRule, "port_forward"), (OneToOneRule, "one_to_one"), (OutboundNatRule, "outbound_nat")):
entities = (await db.scalars(select(model).where(model.groups.any(DeviceGroup.id == group.id)).options(
selectinload(model.targets), selectinload(model.groups).selectinload(DeviceGroup.hosts),
selectinload(model.deployments),
).order_by(model.id))).all()
nat_impacts.extend({"id": str(e.id), "kind": kind, "description": e.description, **row}
for e, row in zip(entities, _impact_rows(entities, group, proposed, f"opnsense_{kind}_uuid")))
old_ids = {h.id for h in group.hosts}
new_ids = {h.id for h in proposed}
result = {"added_members": [{"id": str(h.id), "name": h.name} for h in sorted(proposed, key=lambda h: str(h.id)) if h.id not in old_ids],
"removed_members": [{"id": str(h.id), "name": h.name} for h in sorted(group.hosts, key=lambda h: str(h.id)) if h.id not in new_ids],
"rules": rule_impacts,
"aliases": alias_impacts, "nat": nat_impacts}
state = {"current": {"name": group.name, "description": group.description, "host_ids": sorted(map(str, old_ids))},
"proposed": {"name": payload.name, "description": payload.description, "host_ids": sorted(map(str, new_ids))},
"impact": result}
result["token"] = hashlib.sha256(json.dumps(state, sort_keys=True).encode()).hexdigest()
return result
@router.post("/{group_id}/preview")
async def preview_group(group_id: uuid.UUID, payload: GroupWrite, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
group = await db.get(DeviceGroup, group_id, options=[selectinload(DeviceGroup.hosts)])
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
return await group_impact(db, group, payload)
backend/app/routers/hosts.py
+294
View File
@@ -0,0 +1,294 @@
import logging
import uuid
from app.errors import ErrorCode, api_error
from app.models.port_forward_deployment import PortForwardDeployment
from app.models.one_to_one_deployment import OneToOneDeployment
from app.models.outbound_nat_deployment import OutboundNatDeployment
from fastapi import APIRouter, Depends, HTTPException, Response
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.services.policy_service import policy_lock
from app.models.host import OpnsenseHost
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.models.rule_deployment import RuleDeployment
from app.models.alias_deployment import AliasDeployment
from app.schemas.host import AliasOption, FirmwareInfo, HostBackup, HostCa, HostCertificate, HostCreate, HostRead, HostUpdate, TlsProbeRequest, TlsProbeResult, InterfaceOption, LiveRule
from app.services import backup_store, opnsense_client, opnsense_secret_store, tls_pin_service
from app.services.alias_service import get_aliases
from app.services.firmware_service import get_firmware_info
from app.services.host_status_service import get_statuses
from app.services.interface_service import get_interfaces, get_protocols
from app.services.live_certificates_service import get_live_cas, get_live_certificates
from app.services.live_rules_service import get_live_rules
from app.services.opnsense_client import OpnsenseAPIError
router = APIRouter(prefix="/hosts", tags=["hosts"])
logger = logging.getLogger(__name__)
def _upstream_error(host: OpnsenseHost, exc: OpnsenseAPIError) -> HTTPException:
"""OpnsenseAPIError's message embeds the raw upstream response body
(see opnsense_client._do_request) — useful in a log, not something to
hand an authenticated-but-otherwise-unprivileged (read_only) caller.
Full detail goes to the backend log (visible via `docker compose logs
backend` since the logging fix), client gets a generic 502."""
logger.warning("OPNsense-Anfrage an Host '%s' fehlgeschlagen: %s", host.name, exc)
return api_error(502, ErrorCode.HOST_REQUEST_FAILED, host=host.name)
def _to_read(host: OpnsenseHost, info: dict) -> HostRead:
return HostRead(
id=host.id,
name=host.name,
base_url=host.base_url,
verify_tls=host.verify_tls,
tls_pinned=bool(host.tls_ca_pem),
tls_fingerprint=tls_pin_service.fingerprint_of(host.tls_ca_pem),
enabled=host.enabled,
cluster_name=host.cluster_name,
policy_hierarchy_group_id=host.policy_hierarchy_group_id,
created_at=host.created_at,
status=info.get("status", "offline"),
carp_status=info.get("carp_status"),
status_detail=info.get("status_detail"),
)
async def _get_host_or_404(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.get("", response_model=list[HostRead])
async def list_hosts(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OpnsenseHost).order_by(OpnsenseHost.name))
hosts = result.scalars().all()
statuses = await get_statuses(hosts)
return [_to_read(host, statuses.get(host.id, {})) for host in hosts]
async def _checked_pin(base_url: str, pem: str) -> str:
"""Validate the PEM and prove the firewall really presents it."""
try:
tls_pin_service.parse_pem(pem)
except tls_pin_service.TlsPinError:
raise api_error(422, ErrorCode.TLS_PEM_INVALID)
try:
await tls_pin_service.verify_matches(base_url, pem)
except tls_pin_service.TlsPinError as exc:
logger.warning("TLS-Pin für %s abgelehnt: %s", base_url, exc)
raise api_error(422, ErrorCode.TLS_PIN_MISMATCH)
return pem.strip() + "\n"
@router.post("/tls-probe", response_model=TlsProbeResult)
async def tls_probe(payload: TlsProbeRequest):
"""Read (not trust) the certificate a firewall presents, so a human can compare
its fingerprint with the one shown on the firewall before pinning it."""
try:
return await tls_pin_service.probe(payload.base_url.rstrip("/"))
except tls_pin_service.TlsPinError as exc:
logger.warning("TLS-Probe %s fehlgeschlagen: %s", payload.base_url, exc)
raise api_error(502, ErrorCode.TLS_PROBE_FAILED)
@router.post("", response_model=HostRead, status_code=201)
async def create_host(payload: HostCreate, db: AsyncSession = Depends(get_db)):
existing = await db.scalar(select(OpnsenseHost).where(OpnsenseHost.name == payload.name))
if existing:
raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
if payload.policy_hierarchy_group_id and not await db.get(PolicyHierarchyGroup, payload.policy_hierarchy_group_id):
raise api_error(400, ErrorCode.POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN)
base_url = payload.base_url.rstrip("/")
pinned_pem = await _checked_pin(base_url, payload.tls_ca_pem) if payload.tls_ca_pem else None
host = OpnsenseHost(
name=payload.name,
base_url=base_url,
api_key=payload.api_key,
verify_tls=payload.verify_tls or bool(pinned_pem),
tls_ca_pem=pinned_pem,
enabled=payload.enabled,
cluster_name=payload.cluster_name or None,
policy_hierarchy_group_id=payload.policy_hierarchy_group_id,
)
db.add(host)
await db.flush()
opnsense_secret_store.save_secret(host.id, payload.api_secret)
await db.commit()
statuses = await get_statuses([host])
return _to_read(host, statuses.get(host.id, {}))
@router.get("/{host_id}", response_model=HostRead)
async def get_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
statuses = await get_statuses([host])
return _to_read(host, statuses.get(host.id, {}))
@router.patch("/{host_id}", response_model=HostRead)
async def update_host(host_id: uuid.UUID, payload: HostUpdate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _get_host_or_404(db, host_id)
if payload.name is not None:
existing = await db.scalar(
select(OpnsenseHost).where(OpnsenseHost.name == payload.name, OpnsenseHost.id != host_id)
)
if existing:
raise api_error(409, ErrorCode.HOST_MIT_DIESEM_NAMEN_EXISTIERT_BEREITS)
host.name = payload.name
if payload.verify_tls is not None:
host.verify_tls = payload.verify_tls
if payload.clear_tls_pin:
host.tls_ca_pem = None
elif payload.tls_ca_pem:
host.tls_ca_pem = await _checked_pin(host.base_url, payload.tls_ca_pem)
host.verify_tls = True
if payload.enabled is not None:
host.enabled = payload.enabled
if payload.clear_cluster_name:
host.cluster_name = None
elif payload.cluster_name is not None:
host.cluster_name = payload.cluster_name
if payload.clear_policy_hierarchy_group_id:
host.policy_hierarchy_group_id = None
elif payload.policy_hierarchy_group_id is not None:
if not await db.get(PolicyHierarchyGroup, payload.policy_hierarchy_group_id):
raise api_error(400, ErrorCode.POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN)
host.policy_hierarchy_group_id = payload.policy_hierarchy_group_id
await db.commit()
statuses = await get_statuses([host])
return _to_read(host, statuses.get(host.id, {}))
@router.delete("/{host_id}", status_code=204)
async def delete_host(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
# Both deployment tables FK to opnsense_hosts with ondelete=CASCADE, so
# deleting the host would silently drop OPNexus's only record of what
# it pushed there — the rule/alias itself stays live on the actual
# firewall, now completely untracked. Block until those are cleared
# (delete the rule/alias, or remove it as a target and clean up the
# deployment under "Prüfen & Push").
if await db.scalar(select(RuleDeployment.id).where(RuleDeployment.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_AKTIVE_REGEL_DEPLOYMENTS_DIESE_ZUERST_0BF41721)
if await db.scalar(select(AliasDeployment.id).where(AliasDeployment.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_AKTIVE_ALIAS_DEPLOYMENTS_DIESE_ZUERST_F4A3ED46)
for deployment_model in (PortForwardDeployment, OneToOneDeployment, OutboundNatDeployment):
if await db.scalar(select(deployment_model.id).where(deployment_model.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_NAT_DEPLOYMENTS_DIESE_ZUERST_GEZIELT_0C2A9E85)
if await db.scalar(select(PolicyHierarchyRuleDeployment.id).where(PolicyHierarchyRuleDeployment.host_id == host_id).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_NOCH_AKTIVE_POLICY_HIERARCHIE_DEPLOYMENTS_DIESE_ADD78845)
from app.models.host_operation import HostOperation
from app.services.host_maintenance_service import ACTIVE
if await db.scalar(select(HostOperation.id).where(HostOperation.host_id == host_id, HostOperation.status.in_(ACTIVE)).limit(1)):
raise api_error(409, ErrorCode.HOST_HAT_EINE_OFFENE_WARTUNGSAKTION_ZUERST_PRUEFEN_UND_8CC5FBE0)
await db.delete(host)
await db.commit()
opnsense_secret_store.delete_secret(host_id)
@router.get("/{host_id}/live-rules", response_model=list[LiveRule])
async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_live_rules(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/{host_id}/certificates", response_model=list[HostCertificate])
async def certificates(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_live_certificates(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/{host_id}/cas", response_model=list[HostCa])
async def cas(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_live_cas(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.post("/{host_id}/backups", response_model=HostBackup, status_code=201)
async def trigger_backup(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
content = await opnsense_client.download_backup(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
backup_id = backup_store.save_snapshot(host_id, content)
return next(s for s in backup_store.list_snapshots(host_id) if s["id"] == backup_id)
@router.get("/{host_id}/backups", response_model=list[HostBackup])
async def list_host_backups(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _get_host_or_404(db, host_id)
return backup_store.list_snapshots(host_id)
@router.get("/{host_id}/backups/{backup_id}")
async def download_host_backup(host_id: uuid.UUID, backup_id: str, db: AsyncSession = Depends(get_db)):
await _get_host_or_404(db, host_id)
content = backup_store.read_snapshot(host_id, backup_id)
if content is None:
raise api_error(404, ErrorCode.BACKUP_NICHT_GEFUNDEN)
return Response(
content=content,
media_type="application/xml",
headers={"Content-Disposition": f'attachment; filename="{backup_id}"'},
)
@router.get("/{host_id}/firmware-info", response_model=FirmwareInfo)
async def firmware_info(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_firmware_info(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/{host_id}/interfaces", response_model=list[InterfaceOption])
async def interfaces(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_interfaces(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/{host_id}/protocols", response_model=list[InterfaceOption])
async def protocols(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_protocols(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
@router.get("/{host_id}/aliases", response_model=list[AliasOption])
async def aliases(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await _get_host_or_404(db, host_id)
try:
return await get_aliases(host)
except OpnsenseAPIError as exc:
raise _upstream_error(host, exc) from exc
backend/app/routers/ipsec.py
+215
View File
@@ -0,0 +1,215 @@
import uuid
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, IpsecTunnelRevision
from app.models.ipsec_tunnel import IpsecTunnel
from app.models.ipsec_tunnel_deployment import IpsecTunnelDeployment
from app.schemas.ipsec import IpsecDeployApproval, IpsecDeploymentResult, IpsecTunnelCreate, IpsecTunnelEdit, IpsecTunnelRead
from app.services import ipsec_psk_store
from app.services.ipsec_service import effective_hosts, preview, revision, snapshot, workflow, IPSEC_FIELDS
from app.services.ipsec_deploy_service import deploy_tunnel, undeploy_tunnel
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/ipsec/tunnels", tags=["ipsec"])
def _load_options():
return [
selectinload(IpsecTunnel.targets),
selectinload(IpsecTunnel.groups).selectinload(DeviceGroup.hosts),
selectinload(IpsecTunnel.deployments).selectinload(IpsecTunnelDeployment.host),
]
async def _load_tunnel(db: AsyncSession, tunnel_id: uuid.UUID) -> IpsecTunnel:
# Same advisory lock as rules/aliases/NAT — serializes concurrent
# deploy-affecting changes across processes. Unlike rules there is no
# ensure_editable() gate here: this entity has no pending/revert_failed
# state to ever block editing on (see ipsec_deploy_service.py's module
# docstring) — editing is always allowed, same scope decision already
# made for rule DELETE ("already an explicit action, not gated").
await policy_lock(db)
tunnel = await db.get(IpsecTunnel, tunnel_id, options=_load_options())
if not tunnel:
raise api_error(404, ErrorCode.TUNNEL_NICHT_GEFUNDEN)
return tunnel
def _to_read(tunnel: IpsecTunnel, deployments: list[IpsecTunnelDeployment]) -> IpsecTunnelRead:
hosts_by_id = {host.id: host for host in effective_hosts(tunnel)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(tunnel, deployments)
return IpsecTunnelRead(
workflow_state=state,
workflow_counts=counts,
id=tunnel.id,
description=tunnel.description,
local_address=tunnel.local_address,
remote_address=tunnel.remote_address,
ike_version=tunnel.ike_version,
proposal=tunnel.proposal,
local_identity=tunnel.local_identity,
remote_identity=tunnel.remote_identity,
local_networks=tunnel.local_networks,
remote_networks=tunnel.remote_networks,
esp_proposal=tunnel.esp_proposal,
enabled=tunnel.enabled,
pre_shared_key_set=tunnel.pre_shared_key_set,
created_at=tunnel.created_at,
updated_at=tunnel.updated_at,
target_host_ids=[host.id for host in tunnel.targets],
target_group_ids=[g.id for g in tunnel.groups],
effective_host_ids=[h.id for h in effective_hosts(tunnel)],
deployments=[
IpsecDeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
)
for d in deployments
],
)
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.get("", response_model=list[IpsecTunnelRead])
async def list_tunnels(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(IpsecTunnel).options(*_load_options()))
tunnels = result.scalars().unique().all()
return [_to_read(tunnel, tunnel.deployments) for tunnel in tunnels]
@router.post("", response_model=IpsecTunnelRead, status_code=201)
async def create_tunnel(payload: IpsecTunnelCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
tunnel = IpsecTunnel(
**{field: getattr(payload, field) for field in IPSEC_FIELDS},
targets=hosts,
groups=groups,
)
db.add(tunnel)
await db.flush()
ipsec_psk_store.save_secret(tunnel.id, payload.pre_shared_key)
tunnel.pre_shared_key_set = True
revision(db, tunnel, "Tunnel angelegt")
await db.commit()
return _to_read(tunnel, [])
@router.put("/{tunnel_id}", response_model=IpsecTunnelRead)
async def edit_tunnel(tunnel_id: uuid.UUID, payload: IpsecTunnelEdit, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
if tunnel.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.TUNNEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_50830F70)
hosts, groups = await resolve_targets(db, payload)
if not await db.scalar(select(IpsecTunnelRevision.id).where(IpsecTunnelRevision.tunnel_id == tunnel.id).limit(1)):
revision(db, tunnel, "Vor erster Bearbeitung")
for field in IPSEC_FIELDS:
setattr(tunnel, field, getattr(payload, field))
tunnel.targets, tunnel.groups = hosts, groups
if payload.pre_shared_key is not None:
ipsec_psk_store.save_secret(tunnel.id, payload.pre_shared_key)
tunnel.pre_shared_key_set = True
from datetime import datetime, timezone
tunnel.updated_at = datetime.now(timezone.utc)
revision(db, tunnel, "Entwurf gespeichert")
await db.commit()
return _to_read(tunnel, tunnel.deployments)
@router.get("/{tunnel_id}/preview")
async def preview_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_tunnel(db, tunnel_id))
@router.get("/{tunnel_id}/revisions")
async def list_revisions(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_tunnel(db, tunnel_id)
rows = (await db.scalars(
select(IpsecTunnelRevision).where(IpsecTunnelRevision.tunnel_id == tunnel_id)
.order_by(IpsecTunnelRevision.created_at.desc(), IpsecTunnelRevision.id)
)).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{tunnel_id}/deploy", response_model=IpsecTunnelRead)
async def deploy(tunnel_id: uuid.UUID, approval: IpsecDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if not tunnel.pre_shared_key_set:
raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_PRE_SHARED_4AE1FC63)
hosts = [h for h in effective_hosts(tunnel) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.TUNNEL_HAT_KEINE_ZIEL_HOSTS)
await deploy_tunnel(db, tunnel, hosts)
await db.refresh(tunnel, attribute_names=["deployments"])
for d in tunnel.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(tunnel, tunnel.deployments)
@router.delete("/{tunnel_id}", status_code=204)
async def delete_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
try:
await undeploy_tunnel(db, tunnel)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(tunnel, attribute_names=["deployments"])
ipsec_psk_store.delete_secret(tunnel.id)
await db.delete(tunnel)
await db.commit()
@router.post("/{tunnel_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, approval: IpsecDeployApproval, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
if deployment.opnsense_child_uuid:
await opnsense_client.del_ipsec_child(deployment.host, deployment.opnsense_child_uuid)
if deployment.opnsense_local_uuid:
await opnsense_client.del_ipsec_local(deployment.host, deployment.opnsense_local_uuid)
if deployment.opnsense_remote_uuid:
await opnsense_client.del_ipsec_remote(deployment.host, deployment.opnsense_remote_uuid)
if deployment.opnsense_connection_uuid:
await opnsense_client.del_ipsec_connection(deployment.host, deployment.opnsense_connection_uuid)
if deployment.opnsense_psk_uuid:
await opnsense_client.del_ipsec_psk(deployment.host, deployment.opnsense_psk_uuid)
await opnsense_client.reconfigure_ipsec(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
backend/app/routers/logs.py
+173
View File
@@ -0,0 +1,173 @@
import uuid
from datetime import datetime
from fastapi import APIRouter, Depends, Query
from sqlalchemy import func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.log_entry import FirewallLogEntry
from app.schemas.logs import LogFilterOptions, LogSearchResult, LogSettingsRead, LogSettingsUpdate, LogStatCount, LogStats
from app.services import log_ingest_service, log_settings_service
router = APIRouter(prefix="/logs", tags=["logs"])
async def _settings_with_usage(db: AsyncSession, row) -> LogSettingsRead:
usage = await log_ingest_service.storage_usage(db)
data = LogSettingsRead.model_validate(row)
oldest = await db.scalar(select(func.min(FirewallLogEntry.timestamp)))
return data.model_copy(update={"used_bytes": usage["used_bytes"], "entries": usage["entries"], "oldest_at": oldest})
@router.get("/settings", response_model=LogSettingsRead)
async def read_log_settings(db: AsyncSession = Depends(get_db)):
return await _settings_with_usage(db, await log_settings_service.get_settings(db))
@router.put("/settings", response_model=LogSettingsRead)
async def update_log_settings(payload: LogSettingsUpdate, db: AsyncSession = Depends(get_db)):
return await _settings_with_usage(db, await log_settings_service.update_settings(db, payload))
# Aggregated over the same filter set as /logs itself (host/cluster/time
# range), not a generic analytics engine — a handful of fixed queries.
_TOP_N = 10
# Actions counted as "blocked" for the top-sources aggregate. OPNsense's
# firewall log action vocabulary is effectively pass/block/reject.
_BLOCKED_ACTIONS = ("block", "reject")
def _base_filters(
host_id: uuid.UUID | None,
action: str | None,
interface: str | None,
protocol: str | None,
q: str,
since: datetime | None,
until: datetime | None,
) -> list:
clauses = []
if host_id is not None:
clauses.append(FirewallLogEntry.host_id == host_id)
if action:
clauses.append(FirewallLogEntry.action == action)
if interface:
clauses.append(FirewallLogEntry.interface == interface)
if protocol:
clauses.append(FirewallLogEntry.protocol == protocol)
if since:
clauses.append(FirewallLogEntry.timestamp >= since)
if until:
clauses.append(FirewallLogEntry.timestamp <= until)
if q:
pattern = f"%{q}%"
clauses.append(or_(
FirewallLogEntry.label.ilike(pattern),
FirewallLogEntry.source_net.ilike(pattern),
FirewallLogEntry.destination_net.ilike(pattern),
))
return clauses
async def _cluster_host_ids(db: AsyncSession, cluster_name: str) -> list[uuid.UUID]:
result = await db.execute(select(OpnsenseHost.id).where(OpnsenseHost.cluster_name == cluster_name))
return [row[0] for row in result.all()]
@router.get("", response_model=LogSearchResult)
async def search_logs(
host_id: uuid.UUID | None = None,
cluster_name: str | None = None,
q: str = Query(default="", max_length=200),
action: str | None = None,
interface: str | None = None,
protocol: str | None = None,
since: datetime | None = None,
until: datetime | None = None,
offset: int = Query(default=0, ge=0),
limit: int = Query(default=50, ge=1, le=500),
db: AsyncSession = Depends(get_db),
):
clauses = _base_filters(host_id, action, interface, protocol, q, since, until)
if cluster_name:
clauses.append(FirewallLogEntry.host_id.in_(await _cluster_host_ids(db, cluster_name)))
total = await db.scalar(select(func.count()).select_from(FirewallLogEntry).where(*clauses))
rows = (await db.scalars(
select(FirewallLogEntry)
.where(*clauses)
.order_by(FirewallLogEntry.timestamp.desc(), FirewallLogEntry.id.desc())
.offset(offset)
.limit(limit)
)).all()
return LogSearchResult(total=total or 0, offset=offset, limit=limit, items=list(rows))
@router.get("/stats", response_model=LogStats)
async def log_stats(
host_id: uuid.UUID | None = None,
cluster_name: str | None = None,
since: datetime | None = None,
until: datetime | None = None,
db: AsyncSession = Depends(get_db),
):
clauses = _base_filters(host_id, None, None, None, "", since, until)
if cluster_name:
clauses.append(FirewallLogEntry.host_id.in_(await _cluster_host_ids(db, cluster_name)))
by_action_rows = (await db.execute(
select(FirewallLogEntry.action, func.count())
.where(*clauses)
.group_by(FirewallLogEntry.action)
)).all()
by_interface_rows = (await db.execute(
select(FirewallLogEntry.interface, func.count())
.where(*clauses)
.group_by(FirewallLogEntry.interface)
)).all()
source_count = func.count().label("source_count")
top_sources_rows = (await db.execute(
select(FirewallLogEntry.source_net, source_count)
.where(*clauses, FirewallLogEntry.action.in_(_BLOCKED_ACTIONS), FirewallLogEntry.source_net.is_not(None))
.group_by(FirewallLogEntry.source_net)
.order_by(source_count.desc())
.limit(_TOP_N)
)).all()
port_count = func.count().label("port_count")
top_ports_rows = (await db.execute(
select(FirewallLogEntry.destination_port, port_count)
.where(*clauses, FirewallLogEntry.destination_port.is_not(None))
.group_by(FirewallLogEntry.destination_port)
.order_by(port_count.desc())
.limit(_TOP_N)
)).all()
return LogStats(
by_action={(a or "unbekannt"): c for a, c in by_action_rows},
by_interface={(i or "unbekannt"): c for i, c in by_interface_rows},
top_blocked_sources=[LogStatCount(value=s, count=c) for s, c in top_sources_rows],
top_destination_ports=[LogStatCount(value=p, count=c) for p, c in top_ports_rows],
)
@router.get("/filter-options", response_model=LogFilterOptions)
async def filter_options(db: AsyncSession = Depends(get_db)):
"""Distinct action/interface/protocol values from OPNexus's own
persisted log data — replaces the old live viewer's reliance on
OPNsense's `logFiltersAction`, since that endpoint only ever reflects
one host at a time and this feature now searches across all of them."""
actions = (await db.scalars(
select(FirewallLogEntry.action).where(FirewallLogEntry.action.is_not(None)).distinct()
)).all()
interfaces = (await db.scalars(
select(FirewallLogEntry.interface).where(FirewallLogEntry.interface.is_not(None)).distinct()
)).all()
protocols = (await db.scalars(
select(FirewallLogEntry.protocol).where(FirewallLogEntry.protocol.is_not(None)).distinct()
)).all()
return LogFilterOptions(
actions=sorted(actions), interfaces=sorted(interfaces), protocols=sorted(protocols),
)
backend/app/routers/maintenance.py
+211
View File
@@ -0,0 +1,211 @@
import uuid
from datetime import datetime, timedelta, timezone
from typing import Literal
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.models.host_operation import HostOperation
from app.services import opnsense_client as client, audit_service, carp_maintenance_window_service
from app.services.host_maintenance_service import inspect_host, inspect_carp_switch, fingerprint, peers, ACTIVE
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/maintenance", tags=["host-maintenance"])
class Preview(BaseModel):
action: Literal['reboot','update','major_update']
class Execute(BaseModel):
preview_id: uuid.UUID
confirm_name: str = Field(min_length=1, max_length=120)
acknowledge_downtime: Literal[True]
# carp_enter only: optional auto-revert timer (see
# carp_maintenance_window_service) — ignored for every other action.
auto_revert_minutes: int | None = Field(default=None, ge=1, le=1440)
class Acknowledge(BaseModel):
confirm_name: str = Field(min_length=1, max_length=120)
class Cancel(BaseModel):
confirm_name: str = Field(min_length=1, max_length=120)
reason: str = Field(min_length=3, max_length=500)
async def host_or_404(db, host_id):
h=await db.get(OpnsenseHost,host_id)
if not h: raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return h
def read_operation(op):
return {"id":op.id,"host_id":op.host_id,"action":op.action,"status":op.status,"created_at":op.created_at,"expires_at":op.expires_at,"actor":op.actor,"message":op.message}
async def snapshot_or_502(db, host):
try: return await inspect_host(db,host)
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
async def carp_snapshot_or_502(db, host):
try: return await inspect_carp_switch(db,host)
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@router.get('')
async def status(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host=await host_or_404(db,host_id)
operations=(await db.scalars(select(HostOperation).where(HostOperation.host_id==host.id, HostOperation.status!='preview').order_by(HostOperation.created_at.desc()).limit(20))).all()
# Keep operation records visible while the firewall is rebooting/offline.
try: state=await inspect_host(db,host); error=None
except client.OpnsenseAPIError as exc: state=None; error=str(exc)
return {"state":state,"error":error,"operations":[read_operation(o) for o in operations]}
@router.post('/preview')
async def preview(host_id: uuid.UUID, payload: Preview, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
state=await snapshot_or_502(db,host)
if state['actions'][payload.action]: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions'][payload.action]))
op=HostOperation(host_id=host.id,host_name=host.name,actor=request.state.user.username,action=payload.action,
expires_at=datetime.now(timezone.utc)+timedelta(minutes=5),fingerprint=fingerprint(state),snapshot=state)
db.add(op);await db.commit()
return {"preview_id":op.id,"expires_at":op.expires_at,"action":op.action,"state":state}
@router.post('/execute')
async def execute(host_id: uuid.UUID, payload: Execute, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
op=await db.get(HostOperation,payload.preview_id)
if not op or op.host_id!=host_id or op.actor!=request.state.user.username: raise api_error(404, ErrorCode.WARTUNGSVORSCHAU_NICHT_GEFUNDEN)
if op.status!='preview' or op.expires_at <= datetime.now(timezone.utc): raise api_error(409, ErrorCode.VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET)
if payload.confirm_name != host.name: raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
state=await snapshot_or_502(db,host)
if state['actions'][op.action]: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions'][op.action]))
if fingerprint(state)!=op.fingerprint: raise api_error(409, ErrorCode.HOSTZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
# Consume durably BEFORE the remote request. A timeout is ambiguous, and
# replaying a reboot/update automatically could interrupt recovery.
op.status='starting'; op.started_at=datetime.now(timezone.utc); await db.commit()
await audit_service.persist(kind='host',action=op.action,status='requested',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),details={'snapshot':state})
action_fn={'reboot':client.reboot_host,'update':client.update_firmware,'major_update':client.upgrade_firmware}[op.action]
try:
result=await action_fn(host)
op.status='submitted';op.message='An Firewall übergeben. Abschluss und Erreichbarkeit prüfen; anschließend Wartung abschließen.'
except client.OpnsenseAPIError as exc:
op.status='unknown';op.message='Ausgang unklar: '+str(exc)[:800]+'. Nicht automatisch wiederholen.'
result={}
await db.commit()
await audit_service.persist(kind='host',action=op.action,status=op.status,host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details={'response':result})
return read_operation(op)
@router.post('/check-updates', status_code=202)
async def check_updates(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
from app.services.host_maintenance_service import ensure_no_maintenance
await policy_lock(db)
host=await host_or_404(db,host_id)
if not host.enabled: raise api_error(409, ErrorCode.HOST_IST_DEAKTIVIERT)
await ensure_no_maintenance(db,[host])
try:
running=await client.get_firmware_running(host)
if running.get('status')!='ready': raise api_error(409, ErrorCode.FIRMWARE_AUFTRAG_LAEUFT_BEREITS_ODER_ZUSTAND_IST_UNBEKANNT)
return await client.check_firmware_updates(host)
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@router.get('/progress')
async def progress(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host=await host_or_404(db,host_id)
try: return await client.get_firmware_update_progress(host)
except client.OpnsenseAPIError as exc: raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@router.post('/carp-preview')
async def carp_preview(host_id: uuid.UUID, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
state=await carp_snapshot_or_502(db,host)
if state['actions']['carp']: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions']['carp']))
op=HostOperation(host_id=host.id,host_name=host.name,actor=request.state.user.username,action=state['action'],
expires_at=datetime.now(timezone.utc)+timedelta(minutes=5),fingerprint=fingerprint(state),snapshot=state)
db.add(op);await db.commit()
return {"preview_id":op.id,"expires_at":op.expires_at,"action":op.action,"state":state}
@router.post('/carp-execute')
async def carp_execute(host_id: uuid.UUID, payload: Execute, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
op=await db.get(HostOperation,payload.preview_id)
if not op or op.host_id!=host_id or op.actor!=request.state.user.username or op.action not in ('carp_enter','carp_leave'):
raise api_error(404, ErrorCode.CARP_VORSCHAU_NICHT_GEFUNDEN)
if op.status!='preview' or op.expires_at <= datetime.now(timezone.utc): raise api_error(409, ErrorCode.VORSCHAU_ABGELAUFEN_ODER_BEREITS_VERWENDET)
if payload.confirm_name != host.name: raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
state=await carp_snapshot_or_502(db,host)
if state['actions']['carp']: raise api_error(409, ErrorCode.ACTION_NOT_ALLOWED, reason=' '.join(state['actions']['carp']))
if state['action']!=op.action or fingerprint(state)!=op.fingerprint: raise api_error(409, ErrorCode.CARP_ZUSTAND_WURDE_VERAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
# The OPNsense endpoint is a toggle. Consume the preview before calling it
# and never replay after a timeout, otherwise a retry could undo the switch.
op.status='starting';op.started_at=datetime.now(timezone.utc);await db.commit()
await audit_service.persist(kind='host',action=op.action,status='requested',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),details={'snapshot':state})
try:
result=await client.toggle_carp_maintenance(host)
expected='enter_maintenance' if op.action=='carp_enter' else 'leave_maintenance'
if result.get('action')!=expected:
raise client.OpnsenseAPIError('CARP-Antwort passt nicht zur angeforderten Richtung')
op.status='submitted'
op.message=('CARP-Übergabe angefordert. Prüfen, dass der Partner MASTER und dieser Host BACKUP ist.' if op.action=='carp_enter'
else 'Normalen CARP-Betrieb angefordert. Rollen und Erreichbarkeit beider Hosts prüfen.')
if op.action=='carp_enter':
await carp_maintenance_window_service.open_window(db,host,payload.auto_revert_minutes)
else:
await carp_maintenance_window_service.close_windows_for_manual_leave(db,host)
except client.OpnsenseAPIError as exc:
op.status='unknown';op.message='CARP-Ausgang unklar: '+str(exc)[:760]+'. Nicht erneut klicken; Rollen zuerst prüfen.';result={}
await db.commit()
await audit_service.persist(kind='host',action=op.action,status=op.status,host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details={'response':result})
return read_operation(op)
@router.post('/{operation_id}/acknowledge')
async def acknowledge(host_id: uuid.UUID, operation_id: uuid.UUID, payload: Acknowledge, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host=await host_or_404(db,host_id)
op=await db.get(HostOperation,operation_id)
if not op or op.host_id!=host.id: raise api_error(404, ErrorCode.WARTUNGSAKTION_NICHT_GEFUNDEN)
if payload.confirm_name!=host.name: raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
if op.status not in ACTIVE: raise api_error(409, ErrorCode.KEINE_OFFENE_WARTUNGSAKTION)
minimum_wait=5 if op.action in ('carp_enter','carp_leave') else 30
if op.started_at and (datetime.now(timezone.utc)-op.started_at).total_seconds() < minimum_wait:
raise api_error(409, ErrorCode.MAINTENANCE_WAIT_REQUIRED, seconds=minimum_wait)
if op.action in ('carp_enter','carp_leave'):
state=await carp_snapshot_or_502(db,host)
current=next((p for p in state['peers'] if p['id']==str(host.id)),{})
others=[p for p in state['peers'] if p['id']!=str(host.id)]
healthy_roles=any(p['status']=='online' and p['carp_status']=='MASTER' for p in state['peers']) and any(p['status']=='online' and p['carp_status']=='BACKUP' for p in state['peers'])
if op.action=='carp_enter' and not (state['carp_maintenance_mode'] and current.get('carp_status')=='BACKUP' and any(p['status']=='online' and p['carp_status']=='MASTER' for p in others)):
raise api_error(409, ErrorCode.ROLLENWECHSEL_NOCH_NICHT_BESTAETIGT_DIESER_HOST_MUSS_IM_4676309B)
if op.action=='carp_leave' and not (not state['carp_maintenance_mode'] and healthy_roles):
raise api_error(409, ErrorCode.NORMALBETRIEB_NOCH_NICHT_BESTAETIGT_WARTUNGSMODUS_MUSS_AUS_UND_8AF9290B)
else:
state=await snapshot_or_502(db,host)
if state['running']!='ready' or not any(p['id']==str(host.id) and p['status']=='online' for p in state['peers']):
raise api_error(409, ErrorCode.HOST_MUSS_ERREICHBAR_SEIN_UND_FIRMWARE_AUFTRAG_BEENDET_BCE1390B)
op.status='acknowledged';op.message='Abschluss manuell geprüft und bestätigt.'
await db.commit()
details={'observed_roles':state['peers']} if op.action in ('carp_enter','carp_leave') else {'observed_version':state['version']}
await audit_service.persist(kind='host',action=op.action,status='acknowledged',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details=details)
return read_operation(op)
@router.post('/{operation_id}/cancel')
async def cancel(host_id: uuid.UUID, operation_id: uuid.UUID, payload: Cancel, db: AsyncSession = Depends(get_db)):
"""Escape hatch for an operation whose claimed target state never became
true (e.g. OPNsense's CARP maintenance toggle accepted the request but the
role switch itself never completed). Unlike acknowledge, this does NOT
assert success — it only closes the record so the host/cluster stops being
blocked, and requires a human-written reason for the audit trail."""
await policy_lock(db)
host=await host_or_404(db,host_id)
op=await db.get(HostOperation,operation_id)
if not op or op.host_id!=host.id: raise api_error(404, ErrorCode.WARTUNGSAKTION_NICHT_GEFUNDEN)
if payload.confirm_name!=host.name: raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
if op.status not in ACTIVE: raise api_error(409, ErrorCode.KEINE_OFFENE_WARTUNGSAKTION)
minimum_wait=5 if op.action in ('carp_enter','carp_leave') else 30
if op.started_at and (datetime.now(timezone.utc)-op.started_at).total_seconds() < minimum_wait:
raise api_error(409, ErrorCode.MAINTENANCE_WAIT_REQUIRED, seconds=minimum_wait)
op.status='cancelled';op.message='Manuell abgebrochen, kein bestätigter Abschluss. Grund: '+payload.reason.strip()[:400]
await db.commit()
await audit_service.persist(kind='host',action=op.action,status='cancelled',host_id=str(host.id),host_name=host.name,entity_id=str(op.id),message=op.message,details={'reason':payload.reason})
return read_operation(op)
backend/app/routers/notifications.py
+61
View File
@@ -0,0 +1,61 @@
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.notification import NotificationProblem
from app.schemas.notification import (
NotificationProblemRead,
NotificationSettingsRead,
NotificationSettingsUpdate,
TestNotificationRequest,
)
from app.services import notification_delivery, notification_settings_service
router = APIRouter(prefix="/notifications", tags=["notifications"])
@router.get("/settings", response_model=NotificationSettingsRead)
async def read_settings(db: AsyncSession = Depends(get_db)):
return await notification_settings_service.get_settings(db)
@router.put("/settings", response_model=NotificationSettingsRead)
async def update_settings(payload: NotificationSettingsUpdate, db: AsyncSession = Depends(get_db)):
return await notification_settings_service.update_settings(db, payload)
@router.get("/problems", response_model=list[NotificationProblemRead])
async def list_problems(
include_resolved: bool = False,
limit: int = Query(default=100, ge=1, le=500),
db: AsyncSession = Depends(get_db),
):
clauses = [] if include_resolved else [NotificationProblem.status == "open"]
result = await db.execute(
select(NotificationProblem)
.where(*clauses)
.order_by(NotificationProblem.last_seen_at.desc())
.limit(limit)
)
return list(result.scalars().all())
@router.post("/test")
async def test_notification(payload: TestNotificationRequest, db: AsyncSession = Depends(get_db)):
settings_row = await notification_settings_service.get_settings(db)
fake_problem = NotificationProblem(
problem_type="test",
dedup_key="test",
context={"message": "Dies ist eine Testbenachrichtigung von OPNexus."},
)
try:
if payload.channel == "webhook":
await notification_delivery.send_webhook(settings_row, fake_problem, "opened")
else:
subject, body = notification_delivery.render_message(fake_problem, "opened")
await notification_delivery.send_email(settings_row, subject, body)
except notification_delivery.NotificationDeliveryError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
return {"success": True}
backend/app/routers/one_to_one.py
+271
View File
@@ -0,0 +1,271 @@
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, OneToOneRevision
from app.services.policy_service import effective_hosts, policy_lock, ensure_editable
from app.services.one_to_one_policy_service import ONE_TO_ONE_FIELDS, snapshot, revision, preview, workflow
from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment
from app.schemas.one_to_one import DeploymentResult, OneToOneCreate, OneToOneRead, OneToOneEdit, DeployApproval
from app.services.alias_service import check_alias_consistency, describe_alias_contents, ensure_aliases_settled
from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.one_to_one_deploy_service import confirm_one_to_one, deploy_one_to_one, undeploy_one_to_one, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
router = APIRouter(prefix="/one-to-one", tags=["one-to-one"])
def _load_options():
return [
selectinload(OneToOneRule.targets),
selectinload(OneToOneRule.groups).selectinload(DeviceGroup.hosts),
selectinload(OneToOneRule.deployments).selectinload(OneToOneDeployment.host),
]
async def _load_rule(db: AsyncSession, rule_id: uuid.UUID) -> OneToOneRule:
await policy_lock(db)
rule = await db.get(OneToOneRule, rule_id, options=_load_options())
if not rule:
raise api_error(404, ErrorCode.ERROR_1_1_NAT_REGEL_NICHT_GEFUNDEN)
await hydrate_hierarchy_targets(db, [rule])
return rule
def _to_read(rule: OneToOneRule, hosts: list[OpnsenseHost], deployments: list[OneToOneDeployment]) -> OneToOneRead:
hosts_by_id = {host.id: host for host in effective_hosts(rule)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(rule, deployments)
return OneToOneRead(
workflow_state=state,
workflow_counts=counts,
id=rule.id,
description=rule.description,
interface=rule.interface,
source_net=rule.source_net,
destination_net=rule.destination_net,
source_not=rule.source_not,
destination_not=rule.destination_not,
enabled=rule.enabled,
type=rule.type,
external=rule.external,
natreflection=rule.natreflection,
log=rule.log,
created_at=rule.created_at,
updated_at=rule.updated_at,
target_host_ids=[host.id for host in rule.targets],
target_group_ids=[g.id for g in rule.groups],
policy_hierarchy_group_id=rule.policy_hierarchy_group_id,
effective_host_ids=[h.id for h in effective_hosts(rule)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in deployments
],
)
@router.get("", response_model=list[OneToOneRead])
async def list_one_to_one_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OneToOneRule).options(*_load_options()))
rules = result.scalars().unique().all()
await hydrate_hierarchy_targets(db, rules)
return [_to_read(rule, rule.targets, rule.deployments) for rule in rules]
@router.post("", response_model=OneToOneRead, status_code=201)
async def create_one_to_one_rule(payload: OneToOneCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
rule = OneToOneRule(
description=payload.description,
interface=payload.interface,
source_net=payload.source_net,
destination_net=payload.destination_net,
source_not=payload.source_not,
destination_not=payload.destination_not,
enabled=payload.enabled,
type=payload.type,
external=payload.external,
natreflection=payload.natreflection,
log=payload.log,
targets=hosts,
groups=groups,
policy_hierarchy_group_id=payload.policy_hierarchy_group_id,
)
db.add(rule)
await db.flush()
revision(db, rule, "Regel angelegt")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, [])
@router.post("/{rule_id}/deploy", response_model=OneToOneRead)
async def deploy(rule_id: uuid.UUID, approval: DeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(rule) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.REGEL_HAT_KEINE_ZIEL_HOSTS)
if len(hosts) > 1:
labels = await check_interface_consistency(hosts, rule.interface)
distinct = set(labels.values())
if len(distinct) > 1:
detail = ", ".join(f"{name}={label}" for name, label in sorted(labels.items()))
raise api_error(409, ErrorCode.INTERFACE_MEANING_DIFFERS, interface=rule.interface, detail=detail)
for field_label, value in (
("Source", rule.source_net),
("Destination", rule.destination_net),
("Externes Netz", rule.external),
):
if not value:
continue
contents = await check_alias_consistency(hosts, value)
if len(set(contents.values())) > 1:
detail = describe_alias_contents(contents)
raise api_error(409, ErrorCode.ALIAS_CONTENT_DIFFERS, field_label=field_label, alias=value, detail=detail)
await ensure_aliases_settled(db, hosts, [rule.source_net, rule.destination_net, rule.external])
await deploy_one_to_one(db, rule, hosts)
await db.refresh(rule, attribute_names=["deployments"])
for d in rule.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(rule, hosts, rule.deployments)
@router.post("/{rule_id}/confirm", response_model=OneToOneRead)
async def confirm(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
await confirm_one_to_one(db, rule)
return _to_read(rule, rule.targets, rule.deployments)
@router.delete("/{rule_id}", status_code=204)
async def delete_one_to_one_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_one_to_one(db, rule)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule)
await db.commit()
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.put("/{rule_id}", response_model=OneToOneRead)
async def edit_one_to_one_rule(rule_id: uuid.UUID, payload: OneToOneEdit, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
if rule.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
if not await db.scalar(select(OneToOneRevision.id).where(OneToOneRevision.one_to_one_id == rule.id).limit(1)):
revision(db, rule, "Vor erster Bearbeitung")
for field in ONE_TO_ONE_FIELDS:
setattr(rule, field, getattr(payload, field))
rule.targets, rule.groups = hosts, groups
rule.policy_hierarchy_group_id = payload.policy_hierarchy_group_id
rule.updated_at = datetime.now(timezone.utc)
revision(db, rule, "Entwurf gespeichert")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, rule.deployments)
@router.get("/{rule_id}/preview")
async def preview_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_rule(db, rule_id))
@router.get("/{rule_id}/revisions")
async def list_revisions(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_rule(db, rule_id)
rows = (await db.scalars(select(OneToOneRevision).where(OneToOneRevision.one_to_one_id == rule_id).order_by(OneToOneRevision.created_at.desc(), OneToOneRevision.id))).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{rule_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_one_to_one_rule(deployment.host, deployment.opnsense_one_to_one_uuid)
await opnsense_client.apply_one_to_one(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/{rule_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
reverted = await revert_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
@router.get("/hosts/{host_id}/live")
async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
from app.services import opnsense_client
from app.services.nat_live_service import split_rows
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return split_rows(await opnsense_client.search_one_to_one_rules(host))
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
backend/app/routers/openvpn.py
+211
View File
@@ -0,0 +1,211 @@
import uuid
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, OpenVpnInstanceRevision
from app.models.openvpn_instance import OpenVpnInstance
from app.models.openvpn_instance_deployment import OpenVpnInstanceDeployment
from app.schemas.openvpn import (
OpenVpnDeployApproval, OpenVpnDeploymentResult, OpenVpnInstanceCreate, OpenVpnInstanceEdit, OpenVpnInstanceRead,
)
from app.services import openvpn_key_store
from app.services.openvpn_service import effective_hosts, preview, revision, snapshot, workflow, OPENVPN_FIELDS
from app.services.openvpn_deploy_service import deploy_instance, undeploy_instance
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/openvpn/instances", tags=["openvpn"])
def _load_options():
return [
selectinload(OpenVpnInstance.targets),
selectinload(OpenVpnInstance.groups).selectinload(DeviceGroup.hosts),
selectinload(OpenVpnInstance.deployments).selectinload(OpenVpnInstanceDeployment.host),
]
async def _load_instance(db: AsyncSession, instance_id: uuid.UUID) -> OpenVpnInstance:
# Same advisory lock as rules/aliases/NAT/IPsec — serializes concurrent
# deploy-affecting changes across processes. No ensure_editable() gate
# here either: this entity has no pending/revert_failed state to ever
# block editing on (see openvpn_deploy_service.py's module docstring).
await policy_lock(db)
instance = await db.get(OpenVpnInstance, instance_id, options=_load_options())
if not instance:
raise api_error(404, ErrorCode.INSTANZ_NICHT_GEFUNDEN)
return instance
def _to_read(instance: OpenVpnInstance, deployments: list[OpenVpnInstanceDeployment]) -> OpenVpnInstanceRead:
hosts_by_id = {host.id: host for host in effective_hosts(instance)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(instance, deployments)
return OpenVpnInstanceRead(
workflow_state=state,
workflow_counts=counts,
id=instance.id,
description=instance.description,
role=instance.role,
proto=instance.proto,
port=instance.port,
local=instance.local,
remote=instance.remote,
server_network=instance.server_network,
cert_refs=instance.cert_refs,
ca_refs=instance.ca_refs,
tls_key_enabled=instance.tls_key_enabled,
tls_key_mode=instance.tls_key_mode,
tls_key_material_set=instance.tls_key_material_set,
enabled=instance.enabled,
created_at=instance.created_at,
updated_at=instance.updated_at,
target_host_ids=[host.id for host in instance.targets],
target_group_ids=[g.id for g in instance.groups],
effective_host_ids=[h.id for h in effective_hosts(instance)],
deployments=[
OpenVpnDeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
)
for d in deployments
],
)
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.get("", response_model=list[OpenVpnInstanceRead])
async def list_instances(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OpenVpnInstance).options(*_load_options()))
instances = result.scalars().unique().all()
return [_to_read(instance, instance.deployments) for instance in instances]
@router.post("", response_model=OpenVpnInstanceRead, status_code=201)
async def create_instance(payload: OpenVpnInstanceCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
instance = OpenVpnInstance(
**{field: getattr(payload, field) for field in OPENVPN_FIELDS},
targets=hosts,
groups=groups,
)
db.add(instance)
await db.flush()
if payload.tls_key_material:
openvpn_key_store.save_secret(instance.id, payload.tls_key_material)
instance.tls_key_material_set = True
revision(db, instance, "Instanz angelegt")
await db.commit()
return _to_read(instance, [])
@router.put("/{instance_id}", response_model=OpenVpnInstanceRead)
async def edit_instance(instance_id: uuid.UUID, payload: OpenVpnInstanceEdit, db: AsyncSession = Depends(get_db)):
instance = await _load_instance(db, instance_id)
if instance.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.INSTANZ_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_DB923B59)
hosts, groups = await resolve_targets(db, payload)
if not await db.scalar(select(OpenVpnInstanceRevision.id).where(OpenVpnInstanceRevision.instance_id == instance.id).limit(1)):
revision(db, instance, "Vor erster Bearbeitung")
for field in OPENVPN_FIELDS:
setattr(instance, field, getattr(payload, field))
instance.targets, instance.groups = hosts, groups
if payload.tls_key_material is not None:
openvpn_key_store.save_secret(instance.id, payload.tls_key_material)
instance.tls_key_material_set = True
from datetime import datetime, timezone
instance.updated_at = datetime.now(timezone.utc)
revision(db, instance, "Entwurf gespeichert")
await db.commit()
return _to_read(instance, instance.deployments)
@router.get("/{instance_id}/preview")
async def preview_instance(instance_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_instance(db, instance_id))
@router.get("/{instance_id}/revisions")
async def list_revisions(instance_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_instance(db, instance_id)
rows = (await db.scalars(
select(OpenVpnInstanceRevision).where(OpenVpnInstanceRevision.instance_id == instance_id)
.order_by(OpenVpnInstanceRevision.created_at.desc(), OpenVpnInstanceRevision.id)
)).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{instance_id}/deploy", response_model=OpenVpnInstanceRead)
async def deploy(instance_id: uuid.UUID, approval: OpenVpnDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
instance = await _load_instance(db, instance_id)
plan = preview(instance)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if instance.tls_key_enabled and not instance.tls_key_material_set:
raise api_error(400, ErrorCode.TLS_HARDENING_IST_AKTIVIERT_ABER_ES_WURDE_NOCH_F53C81DD)
hosts = [h for h in effective_hosts(instance) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.INSTANZ_HAT_KEINE_ZIEL_HOSTS)
await deploy_instance(db, instance, hosts)
await db.refresh(instance, attribute_names=["deployments"])
for d in instance.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(instance, instance.deployments)
@router.delete("/{instance_id}", status_code=204)
async def delete_instance(instance_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
instance = await _load_instance(db, instance_id)
try:
await undeploy_instance(db, instance)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(instance, attribute_names=["deployments"])
openvpn_key_store.delete_secret(instance.id)
await db.delete(instance)
await db.commit()
@router.post("/{instance_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(instance_id: uuid.UUID, host_id: uuid.UUID, approval: OpenVpnDeployApproval, db: AsyncSession = Depends(get_db)):
instance = await _load_instance(db, instance_id)
plan = preview(instance)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in instance.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
if deployment.opnsense_instance_uuid:
await opnsense_client.del_openvpn_instance(deployment.host, deployment.opnsense_instance_uuid)
if deployment.opnsense_statickey_uuid:
await opnsense_client.del_openvpn_static_key(deployment.host, deployment.opnsense_statickey_uuid)
await opnsense_client.reconfigure_openvpn(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
backend/app/routers/outbound_nat.py
+286
View File
@@ -0,0 +1,286 @@
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, OutboundNatRevision
from app.services.policy_service import effective_hosts, policy_lock, ensure_editable
from app.services.outbound_nat_policy_service import OUTBOUND_NAT_FIELDS, snapshot, revision, preview, workflow
from app.models.outbound_nat import OutboundNatRule
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.schemas.outbound_nat import DeploymentResult, OutboundNatCreate, OutboundNatRead, OutboundNatEdit, DeployApproval
from app.services.outbound_nat_mode import require_manual_mode, host_mode
from app.services.alias_service import check_alias_consistency, describe_alias_contents, ensure_aliases_settled
from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.outbound_nat_deploy_service import confirm_outbound_nat, deploy_outbound_nat, undeploy_outbound_nat, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
router = APIRouter(prefix="/outbound-nat", tags=["outbound-nat"])
def _load_options():
return [
selectinload(OutboundNatRule.targets),
selectinload(OutboundNatRule.groups).selectinload(DeviceGroup.hosts),
selectinload(OutboundNatRule.deployments).selectinload(OutboundNatDeployment.host),
]
async def _load_rule(db: AsyncSession, rule_id: uuid.UUID) -> OutboundNatRule:
await policy_lock(db)
rule = await db.get(OutboundNatRule, rule_id, options=_load_options())
if not rule:
raise api_error(404, ErrorCode.OUTBOUND_NAT_REGEL_NICHT_GEFUNDEN)
await hydrate_hierarchy_targets(db, [rule])
return rule
def _to_read(rule: OutboundNatRule, hosts: list[OpnsenseHost], deployments: list[OutboundNatDeployment]) -> OutboundNatRead:
hosts_by_id = {host.id: host for host in effective_hosts(rule)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(rule, deployments)
return OutboundNatRead(
workflow_state=state,
workflow_counts=counts,
id=rule.id,
description=rule.description,
interface=rule.interface,
source_net=rule.source_net,
destination_net=rule.destination_net,
source_not=rule.source_not,
destination_not=rule.destination_not,
enabled=rule.enabled,
ipprotocol=rule.ipprotocol,
protocol=rule.protocol,
source_port=rule.source_port,
destination_port=rule.destination_port,
target=rule.target,
target_port=rule.target_port,
created_at=rule.created_at,
updated_at=rule.updated_at,
target_host_ids=[host.id for host in rule.targets],
target_group_ids=[g.id for g in rule.groups],
policy_hierarchy_group_id=rule.policy_hierarchy_group_id,
effective_host_ids=[h.id for h in effective_hosts(rule)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in deployments
],
)
@router.get("", response_model=list[OutboundNatRead])
async def list_outbound_nat_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(OutboundNatRule).options(*_load_options()))
rules = result.scalars().unique().all()
await hydrate_hierarchy_targets(db, rules)
return [_to_read(rule, rule.targets, rule.deployments) for rule in rules]
@router.post("", response_model=OutboundNatRead, status_code=201)
async def create_outbound_nat_rule(payload: OutboundNatCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
rule = OutboundNatRule(
description=payload.description,
interface=payload.interface,
source_net=payload.source_net,
destination_net=payload.destination_net,
source_not=payload.source_not,
destination_not=payload.destination_not,
enabled=payload.enabled,
ipprotocol=payload.ipprotocol,
protocol=payload.protocol,
source_port=payload.source_port,
destination_port=payload.destination_port,
target=payload.target,
target_port=payload.target_port,
targets=hosts,
groups=groups,
policy_hierarchy_group_id=payload.policy_hierarchy_group_id,
)
db.add(rule)
await db.flush()
revision(db, rule, "Regel angelegt")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, [])
@router.post("/{rule_id}/deploy", response_model=OutboundNatRead)
async def deploy(rule_id: uuid.UUID, approval: DeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(rule) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.REGEL_HAT_KEINE_ZIEL_HOSTS)
for host in hosts:
await require_manual_mode(host)
if len(hosts) > 1:
labels = await check_interface_consistency(hosts, rule.interface)
distinct = set(labels.values())
if len(distinct) > 1:
detail = ", ".join(f"{name}={label}" for name, label in sorted(labels.items()))
raise api_error(409, ErrorCode.INTERFACE_MEANING_DIFFERS, interface=rule.interface, detail=detail)
for field_label, value in (
("Source", rule.source_net),
("Destination", rule.destination_net),
("Ziel (Target)", rule.target),
):
if not value:
continue
contents = await check_alias_consistency(hosts, value)
if len(set(contents.values())) > 1:
detail = describe_alias_contents(contents)
raise api_error(409, ErrorCode.ALIAS_CONTENT_DIFFERS, field_label=field_label, alias=value, detail=detail)
await ensure_aliases_settled(db, hosts, [rule.source_net, rule.destination_net, rule.target])
await deploy_outbound_nat(db, rule, hosts)
await db.refresh(rule, attribute_names=["deployments"])
for d in rule.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(rule, hosts, rule.deployments)
@router.post("/{rule_id}/confirm", response_model=OutboundNatRead)
async def confirm(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
await confirm_outbound_nat(db, rule)
return _to_read(rule, rule.targets, rule.deployments)
@router.delete("/{rule_id}", status_code=204)
async def delete_outbound_nat_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_outbound_nat(db, rule)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule)
await db.commit()
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.put("/{rule_id}", response_model=OutboundNatRead)
async def edit_outbound_nat_rule(rule_id: uuid.UUID, payload: OutboundNatEdit, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
if rule.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
if not await db.scalar(select(OutboundNatRevision.id).where(OutboundNatRevision.outbound_nat_id == rule.id).limit(1)):
revision(db, rule, "Vor erster Bearbeitung")
for field in OUTBOUND_NAT_FIELDS:
setattr(rule, field, getattr(payload, field))
rule.targets, rule.groups = hosts, groups
rule.policy_hierarchy_group_id = payload.policy_hierarchy_group_id
rule.updated_at = datetime.now(timezone.utc)
revision(db, rule, "Entwurf gespeichert")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, rule.deployments)
@router.get("/{rule_id}/preview")
async def preview_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_rule(db, rule_id))
@router.get("/{rule_id}/revisions")
async def list_revisions(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_rule(db, rule_id)
rows = (await db.scalars(select(OutboundNatRevision).where(OutboundNatRevision.outbound_nat_id == rule_id).order_by(OutboundNatRevision.created_at.desc(), OutboundNatRevision.id))).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{rule_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_outbound_nat_rule(deployment.host, deployment.opnsense_outbound_nat_uuid)
await opnsense_client.apply_outbound_nat(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/{rule_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
reverted = await revert_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
@router.get("/hosts/{host_id}/live")
async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
from app.services import opnsense_client
from app.services.nat_live_service import split_rows
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return split_rows(await opnsense_client.search_outbound_nat_rules(host))
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@router.get("/hosts/{host_id}/mode")
async def outbound_mode(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return await host_mode(host)
backend/app/routers/policy_hierarchy.py
+379
View File
@@ -0,0 +1,379 @@
import uuid
from fastapi import APIRouter, Body, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.policy import PolicyHierarchyRuleRevision
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.policy_hierarchy_rule_deployment import PolicyHierarchyRuleDeployment
from app.schemas.rule import DeploymentResult
from app.schemas.policy_hierarchy import (
PolicyHierarchyDeployApproval,
PolicyHierarchyMoveDirection,
PolicyHierarchyReorder,
PolicyHierarchyRuleCreate,
PolicyHierarchyRuleEdit,
PolicyHierarchyRuleRead,
)
from app.services.alias_service import check_alias_consistency, describe_alias_contents, ensure_aliases_settled
from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_hierarchy_deploy_service import (
confirm_hierarchy_rule,
deploy_hierarchy_rule,
reassert_group_ordering,
revert_hierarchy_deployment,
undeploy_hierarchy_rule,
)
from app.services.policy_hierarchy_service import (
HIERARCHY_RULE_FIELDS,
effective_hosts,
ensure_editable,
policy_lock,
preview,
revision,
workflow,
)
router = APIRouter(prefix="/policy-hierarchy", tags=["policy-hierarchy"])
def _rule_load_options():
return [selectinload(PolicyHierarchyRule.deployments).selectinload(PolicyHierarchyRuleDeployment.host)]
async def _load_all_groups(db: AsyncSession) -> list[PolicyHierarchyGroup]:
return list((await db.scalars(
select(PolicyHierarchyGroup).options(selectinload(PolicyHierarchyGroup.hosts))
)).all())
async def _load_rule(db: AsyncSession, rule_id: uuid.UUID) -> PolicyHierarchyRule:
await policy_lock(db)
# populate_existing=True: without it, a rule already in this request's
# identity map (e.g. right after create_rule's db.add()+commit(), or a
# second _load_rule call later in the same request) is returned as-is
# by db.get() and the selectinload(deployments) option is silently
# skipped — leaving `.deployments` unloaded and triggering a
# synchronous lazy-load attempt outside any greenlet context later.
rule = await db.get(PolicyHierarchyRule, rule_id, options=_rule_load_options(), populate_existing=True)
if not rule:
raise api_error(404, ErrorCode.REGEL_NICHT_GEFUNDEN)
return rule
def _to_read(rule: PolicyHierarchyRule, all_groups: list[PolicyHierarchyGroup]) -> PolicyHierarchyRuleRead:
hosts_by_id = {h.id: h for h in effective_hosts(rule, all_groups)}
hosts_by_id.update({d.host_id: d.host for d in rule.deployments if d.host is not None})
state, counts = workflow(rule, rule.deployments, all_groups)
return PolicyHierarchyRuleRead(
workflow_state=state,
workflow_counts=counts,
id=rule.id,
hierarchy_group_id=rule.hierarchy_group_id,
rulebase=rule.rulebase,
sequence=rule.sequence,
description=rule.description,
interface=rule.interface,
direction=rule.direction,
ipprotocol=rule.ipprotocol,
protocol=rule.protocol,
action=rule.action,
source_net=rule.source_net,
source_port=rule.source_port,
destination_net=rule.destination_net,
destination_port=rule.destination_port,
enabled=rule.enabled,
created_at=rule.created_at,
updated_at=rule.updated_at,
effective_host_ids=[h.id for h in effective_hosts(rule, all_groups)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in rule.deployments
],
)
async def _get_group_or_400(db: AsyncSession, group_id: uuid.UUID) -> PolicyHierarchyGroup:
group = await db.get(PolicyHierarchyGroup, group_id)
if not group:
raise api_error(400, ErrorCode.POLICY_HIERARCHIE_GRUPPE_NICHT_GEFUNDEN)
return group
@router.get("/rules", response_model=list[PolicyHierarchyRuleRead])
async def list_rules(db: AsyncSession = Depends(get_db)):
all_groups = await _load_all_groups(db)
rules = (await db.scalars(select(PolicyHierarchyRule).options(*_rule_load_options()))).all()
return [_to_read(rule, all_groups) for rule in rules]
@router.post("/rules", response_model=PolicyHierarchyRuleRead, status_code=201)
async def create_rule(payload: PolicyHierarchyRuleCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _get_group_or_400(db, payload.hierarchy_group_id)
# New rules are appended to the end of their (group, rulebase) list;
# atomic drag-and-drop and up/down controls handle reordering later.
max_sequence = await db.scalar(
select(PolicyHierarchyRule.sequence).where(
PolicyHierarchyRule.hierarchy_group_id == payload.hierarchy_group_id,
PolicyHierarchyRule.rulebase == payload.rulebase,
).order_by(PolicyHierarchyRule.sequence.desc()).limit(1)
)
rule = PolicyHierarchyRule(
hierarchy_group_id=payload.hierarchy_group_id,
rulebase=payload.rulebase,
sequence=(max_sequence + 1) if max_sequence is not None else 0,
description=payload.description,
interface=payload.interface,
direction=payload.direction,
ipprotocol=payload.ipprotocol,
protocol=payload.protocol,
action=payload.action,
source_net=payload.source_net,
source_port=payload.source_port,
destination_net=payload.destination_net,
destination_port=payload.destination_port,
enabled=payload.enabled,
)
db.add(rule)
await db.flush()
revision(db, rule, "Regel angelegt")
await db.commit()
return _to_read(await _load_rule(db, rule.id), await _load_all_groups(db))
@router.put("/rules/{rule_id}", response_model=PolicyHierarchyRuleRead)
async def edit_rule(rule_id: uuid.UUID, payload: PolicyHierarchyRuleEdit, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
if rule.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0)
await _get_group_or_400(db, payload.hierarchy_group_id)
if not await db.scalar(select(PolicyHierarchyRuleRevision.id).where(PolicyHierarchyRuleRevision.policy_hierarchy_id == rule.id).limit(1)):
revision(db, rule, "Vor erster Bearbeitung")
for field in HIERARCHY_RULE_FIELDS:
setattr(rule, field, getattr(payload, field))
rule.hierarchy_group_id = payload.hierarchy_group_id
rule.rulebase = payload.rulebase
from datetime import datetime, timezone
rule.updated_at = datetime.now(timezone.utc)
revision(db, rule, "Entwurf gespeichert")
await db.commit()
return _to_read(rule, await _load_all_groups(db))
@router.put("/groups/{group_id}/rules/{rulebase}/order", response_model=list[PolicyHierarchyRuleRead])
async def reorder_rules(
group_id: uuid.UUID,
rulebase: str,
payload: PolicyHierarchyReorder,
db: AsyncSession = Depends(get_db),
):
"""Atomically persist a complete drag-and-drop order.
The complete sibling set and every optimistic timestamp must still
match. This prevents two administrators from silently overwriting each
other's order and avoids the partial state produced by several
sequential up/down requests.
"""
if rulebase not in ("pre", "post"):
raise api_error(404, ErrorCode.RULEBASE_NICHT_GEFUNDEN)
await policy_lock(db)
await _get_group_or_400(db, group_id)
siblings = list((await db.scalars(
select(PolicyHierarchyRule).where(
PolicyHierarchyRule.hierarchy_group_id == group_id,
PolicyHierarchyRule.rulebase == rulebase,
).options(*_rule_load_options()).order_by(PolicyHierarchyRule.sequence, PolicyHierarchyRule.id)
)).all())
if len({item.id for item in payload.items}) != len(payload.items):
raise api_error(422, ErrorCode.EINE_REGEL_DARF_IN_DER_REIHENFOLGE_NUR_EINMAL_F05DF47C)
if {rule.id for rule in siblings} != {item.id for item in payload.items}:
raise api_error(409, ErrorCode.REGELLISTE_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_ERNEUT_529D7203)
expected = {item.id: item.expected_updated_at for item in payload.items}
for rule in siblings:
ensure_editable(rule)
if any(rule.updated_at != expected[rule.id] for rule in siblings):
raise api_error(409, ErrorCode.MINDESTENS_EINE_REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_932925BD)
by_id = {rule.id: rule for rule in siblings}
from datetime import datetime, timezone
changed_at = datetime.now(timezone.utc)
for sequence, item in enumerate(payload.items):
rule = by_id[item.id]
if rule.sequence != sequence:
rule.sequence = sequence
rule.updated_at = changed_at
revision(db, rule, "Reihenfolge per Drag-and-drop geändert")
await db.commit()
all_groups = await _load_all_groups(db)
return [_to_read(await _load_rule(db, item.id), all_groups) for item in payload.items]
@router.post("/rules/{rule_id}/move", response_model=PolicyHierarchyRuleRead)
async def move_rule(rule_id: uuid.UUID, payload: PolicyHierarchyMoveDirection, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
siblings = list((await db.scalars(
select(PolicyHierarchyRule).where(
PolicyHierarchyRule.hierarchy_group_id == rule.hierarchy_group_id,
PolicyHierarchyRule.rulebase == rule.rulebase,
).order_by(PolicyHierarchyRule.sequence, PolicyHierarchyRule.id)
)).all())
index = next(i for i, r in enumerate(siblings) if r.id == rule.id)
neighbor_index = index - 1 if payload.direction == "up" else index + 1
if 0 <= neighbor_index < len(siblings):
neighbor = siblings[neighbor_index]
rule.sequence, neighbor.sequence = neighbor.sequence, rule.sequence
revision(db, rule, "Reihenfolge geändert")
await db.commit()
return _to_read(await _load_rule(db, rule_id), await _load_all_groups(db))
@router.post("/rules/{rule_id}/deploy", response_model=PolicyHierarchyRuleRead)
async def deploy(rule_id: uuid.UUID, approval: PolicyHierarchyDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
all_groups = await _load_all_groups(db)
plan = preview(rule, all_groups)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(rule, all_groups) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.REGEL_HAT_KEINE_ZIEL_HOSTS)
if len(hosts) > 1:
labels = await check_interface_consistency(hosts, rule.interface)
distinct = set(labels.values())
if len(distinct) > 1:
detail = ", ".join(f"{name}={label}" for name, label in sorted(labels.items()))
raise api_error(409, ErrorCode.INTERFACE_MEANING_DIFFERS, interface=rule.interface, detail=detail)
for field_label, value in (
("Source", rule.source_net),
("Destination", rule.destination_net),
("Source-Port", rule.source_port),
("Destination-Port", rule.destination_port),
):
if not value:
continue
contents = await check_alias_consistency(hosts, value)
if len(set(contents.values())) > 1:
detail = describe_alias_contents(contents)
raise api_error(409, ErrorCode.ALIAS_CONTENT_DIFFERS, field_label=field_label, alias=value, detail=detail)
await ensure_aliases_settled(db, hosts, [rule.source_net, rule.destination_net, rule.source_port, rule.destination_port])
await deploy_hierarchy_rule(db, rule, hosts)
await db.refresh(rule, attribute_names=["deployments"])
for d in rule.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(rule, await _load_all_groups(db))
@router.post("/rules/{rule_id}/confirm", response_model=PolicyHierarchyRuleRead)
async def confirm(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
await confirm_hierarchy_rule(db, rule)
return _to_read(rule, await _load_all_groups(db))
@router.delete("/rules/{rule_id}", status_code=204)
async def delete_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_hierarchy_rule(db, rule)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule)
await db.commit()
@router.get("/rules/{rule_id}/preview")
async def preview_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
return preview(rule, await _load_all_groups(db))
@router.get("/rules/{rule_id}/revisions")
async def list_revisions(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_rule(db, rule_id)
rows = (await db.scalars(
select(PolicyHierarchyRuleRevision).where(PolicyHierarchyRuleRevision.policy_hierarchy_id == rule_id)
.order_by(PolicyHierarchyRuleRevision.created_at.desc(), PolicyHierarchyRuleRevision.id)
)).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/rules/{rule_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approval: PolicyHierarchyDeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
all_groups = await _load_all_groups(db)
plan = preview(rule, all_groups)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_policy_hierarchy_uuid)
await opnsense_client.apply(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/rules/{rule_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(rule_id: uuid.UUID, host_id: uuid.UUID, approval: PolicyHierarchyDeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
all_groups = await _load_all_groups(db)
plan = preview(rule, all_groups)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
reverted = await revert_hierarchy_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
@router.post("/groups/{group_id}/reassert-ordering")
async def reassert_ordering(group_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""Explicit ordering repair for a group subtree.
Normal OPNexus rule/alias/NAT deploys also reassert automatically; this
action remains useful after changing the desired order or after manual
edits made directly on a firewall.
"""
await policy_lock(db)
group = await db.get(PolicyHierarchyGroup, group_id)
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
errors = await reassert_group_ordering(db, group)
if errors:
raise api_error(502, ErrorCode.ORDERING_FAILED, reason='; '.join(errors))
return {"status": "ok"}
backend/app/routers/policy_hierarchy_groups.py
+105
View File
@@ -0,0 +1,105 @@
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.policy_hierarchy_group import PolicyHierarchyGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.alias import FirewallAlias
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.port_forward import PortForwardRule
from app.schemas.policy_hierarchy import PolicyHierarchyGroupRead, PolicyHierarchyGroupWrite
from app.services.policy_hierarchy_group_service import assert_no_cycle
from app.services.policy_hierarchy_service import policy_lock
router = APIRouter(prefix="/policy-hierarchy/groups", tags=["policy-hierarchy"])
def _to_read(group: PolicyHierarchyGroup) -> PolicyHierarchyGroupRead:
return PolicyHierarchyGroupRead(
id=group.id, name=group.name, description=group.description, parent_id=group.parent_id,
created_at=group.created_at, host_ids=[h.id for h in group.hosts],
)
async def _get_or_404(db: AsyncSession, group_id: uuid.UUID) -> PolicyHierarchyGroup:
# populate_existing=True: without it, a group already in this request's
# identity map (e.g. right after create_group's db.add()+commit()) is
# returned as-is by db.get() and the `hosts` selectinload option is
# silently skipped — leaving `.hosts` unloaded and triggering a
# synchronous lazy-load attempt outside any greenlet context later.
group = await db.get(
PolicyHierarchyGroup, group_id,
options=[selectinload(PolicyHierarchyGroup.hosts)], populate_existing=True,
)
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
return group
@router.get("", response_model=list[PolicyHierarchyGroupRead])
async def list_groups(db: AsyncSession = Depends(get_db)):
groups = (await db.scalars(
select(PolicyHierarchyGroup).options(selectinload(PolicyHierarchyGroup.hosts)).order_by(PolicyHierarchyGroup.name)
)).all()
return [_to_read(g) for g in groups]
@router.post("", response_model=PolicyHierarchyGroupRead, status_code=201)
async def create_group(payload: PolicyHierarchyGroupWrite, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
if await db.scalar(select(PolicyHierarchyGroup.id).where(PolicyHierarchyGroup.name == payload.name)):
raise api_error(409, ErrorCode.GRUPPENNAME_BEREITS_VERGEBEN)
if payload.parent_id and not await db.get(PolicyHierarchyGroup, payload.parent_id):
raise api_error(400, ErrorCode.UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN)
group = PolicyHierarchyGroup(name=payload.name, description=payload.description, parent_id=payload.parent_id)
db.add(group)
await db.commit()
return _to_read(await _get_or_404(db, group.id))
@router.put("/{group_id}", response_model=PolicyHierarchyGroupRead)
async def update_group(group_id: uuid.UUID, payload: PolicyHierarchyGroupWrite, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
group = await _get_or_404(db, group_id)
duplicate = await db.scalar(select(PolicyHierarchyGroup.id).where(
PolicyHierarchyGroup.name == payload.name, PolicyHierarchyGroup.id != group_id))
if duplicate:
raise api_error(409, ErrorCode.GRUPPENNAME_BEREITS_VERGEBEN)
if payload.parent_id and not await db.get(PolicyHierarchyGroup, payload.parent_id):
raise api_error(400, ErrorCode.UEBERGEORDNETE_GRUPPE_NICHT_GEFUNDEN)
await assert_no_cycle(db, group_id, payload.parent_id)
group.name, group.description, group.parent_id = payload.name, payload.description, payload.parent_id
await db.commit()
return _to_read(await _get_or_404(db, group_id))
@router.delete("/{group_id}", status_code=204)
async def delete_group(group_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
group = await db.get(PolicyHierarchyGroup, group_id)
if not group:
raise api_error(404, ErrorCode.GRUPPE_NICHT_GEFUNDEN)
if await db.scalar(select(PolicyHierarchyGroup.id).where(PolicyHierarchyGroup.parent_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_HAT_NOCH_UNTERGRUPPEN_DIESE_ZUERST_VERSCHIEBEN_ODER_9FFFD853)
if await db.scalar(select(PolicyHierarchyRule.id).where(PolicyHierarchyRule.hierarchy_group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GRUPPE_HAT_NOCH_ZUGEORDNETE_REGELN_DIESE_ZUERST_VERSCHIEBEN_2CCBC48B)
inherited_resources = (
(FirewallAlias, "Aliase"),
(PortForwardRule, "Port-Forward-Regeln"),
(OneToOneRule, "1:1-NAT-Regeln"),
(OutboundNatRule, "Outbound-NAT-Regeln"),
)
for model, label in inherited_resources:
if await db.scalar(select(model.id).where(model.policy_hierarchy_group_id == group_id).limit(1)):
raise api_error(409, ErrorCode.GROUP_HAS_ASSIGNED_ITEMS, label=label)
# Hosts assigned directly to this group are unassigned (ondelete="SET
# NULL" on OpnsenseHost.policy_hierarchy_group_id), not blocked — a
# group with only host members and no children/rules is safe to remove.
await db.delete(group)
await db.commit()
backend/app/routers/port_forward.py
+273
View File
@@ -0,0 +1,273 @@
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, PortForwardRevision
from app.services.policy_service import effective_hosts, policy_lock, ensure_editable
from app.services.port_forward_policy_service import PORT_FORWARD_FIELDS, snapshot, revision, preview, workflow
from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment
from app.schemas.port_forward import DeploymentResult, PortForwardCreate, PortForwardRead, PortForwardEdit, DeployApproval
from app.services.alias_service import check_alias_consistency, describe_alias_contents, ensure_aliases_settled
from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.port_forward_deploy_service import confirm_port_forward, deploy_port_forward, undeploy_port_forward, revert_deployment
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets, validate_hierarchy_group
router = APIRouter(prefix="/port-forward", tags=["port-forward"])
def _load_options():
return [
selectinload(PortForwardRule.targets),
selectinload(PortForwardRule.groups).selectinload(DeviceGroup.hosts),
selectinload(PortForwardRule.deployments).selectinload(PortForwardDeployment.host),
]
async def _load_rule(db: AsyncSession, rule_id: uuid.UUID) -> PortForwardRule:
await policy_lock(db)
rule = await db.get(PortForwardRule, rule_id, options=_load_options())
if not rule:
raise api_error(404, ErrorCode.PORT_FORWARD_REGEL_NICHT_GEFUNDEN)
await hydrate_hierarchy_targets(db, [rule])
return rule
def _to_read(rule: PortForwardRule, hosts: list[OpnsenseHost], deployments: list[PortForwardDeployment]) -> PortForwardRead:
hosts_by_id = {host.id: host for host in effective_hosts(rule)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(rule, deployments)
return PortForwardRead(
workflow_state=state,
workflow_counts=counts,
id=rule.id,
description=rule.description,
interface=rule.interface,
ipprotocol=rule.ipprotocol,
protocol=rule.protocol,
source_net=rule.source_net,
source_port=rule.source_port,
destination_net=rule.destination_net,
destination_port=rule.destination_port,
target=rule.target,
local_port=rule.local_port,
natreflection=rule.natreflection,
enabled=rule.enabled,
created_at=rule.created_at,
updated_at=rule.updated_at,
target_host_ids=[host.id for host in rule.targets],
target_group_ids=[g.id for g in rule.groups],
policy_hierarchy_group_id=rule.policy_hierarchy_group_id,
effective_host_ids=[h.id for h in effective_hosts(rule)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in deployments
],
)
@router.get("", response_model=list[PortForwardRead])
async def list_port_forward_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(PortForwardRule).options(*_load_options()))
rules = result.scalars().unique().all()
await hydrate_hierarchy_targets(db, rules)
return [_to_read(rule, rule.targets, rule.deployments) for rule in rules]
@router.post("", response_model=PortForwardRead, status_code=201)
async def create_port_forward_rule(payload: PortForwardCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
rule = PortForwardRule(
description=payload.description,
interface=payload.interface,
ipprotocol=payload.ipprotocol,
protocol=payload.protocol,
source_net=payload.source_net,
source_port=payload.source_port,
destination_net=payload.destination_net,
destination_port=payload.destination_port,
target=payload.target,
local_port=payload.local_port,
natreflection=payload.natreflection,
enabled=payload.enabled,
targets=hosts,
groups=groups,
policy_hierarchy_group_id=payload.policy_hierarchy_group_id,
)
db.add(rule)
await db.flush()
revision(db, rule, "Regel angelegt")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, [])
@router.post("/{rule_id}/deploy", response_model=PortForwardRead)
async def deploy(rule_id: uuid.UUID, approval: DeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(rule) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.REGEL_HAT_KEINE_ZIEL_HOSTS)
if len(hosts) > 1:
labels = await check_interface_consistency(hosts, rule.interface)
distinct = set(labels.values())
if len(distinct) > 1:
detail = ", ".join(f"{name}={label}" for name, label in sorted(labels.items()))
raise api_error(409, ErrorCode.INTERFACE_MEANING_DIFFERS, interface=rule.interface, detail=detail)
for field_label, value in (
("Source", rule.source_net),
("Destination", rule.destination_net),
("Ziel (Target)", rule.target),
):
if not value:
continue
contents = await check_alias_consistency(hosts, value)
if len(set(contents.values())) > 1:
detail = describe_alias_contents(contents)
raise api_error(409, ErrorCode.ALIAS_CONTENT_DIFFERS, field_label=field_label, alias=value, detail=detail)
await ensure_aliases_settled(db, hosts, [rule.source_net, rule.destination_net, rule.target])
await deploy_port_forward(db, rule, hosts)
await db.refresh(rule, attribute_names=["deployments"])
for d in rule.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(rule, hosts, rule.deployments)
@router.post("/{rule_id}/confirm", response_model=PortForwardRead)
async def confirm(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
await confirm_port_forward(db, rule)
return _to_read(rule, rule.targets, rule.deployments)
@router.delete("/{rule_id}", status_code=204)
async def delete_port_forward_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_port_forward(db, rule)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule)
await db.commit()
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.put("/{rule_id}", response_model=PortForwardRead)
async def edit_port_forward_rule(rule_id: uuid.UUID, payload: PortForwardEdit, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
if rule.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0)
hosts, groups = await resolve_targets(db, payload)
await validate_hierarchy_group(db, payload.policy_hierarchy_group_id)
if not await db.scalar(select(PortForwardRevision.id).where(PortForwardRevision.port_forward_id == rule.id).limit(1)):
revision(db, rule, "Vor erster Bearbeitung")
for field in PORT_FORWARD_FIELDS:
setattr(rule, field, getattr(payload, field))
rule.targets, rule.groups = hosts, groups
rule.policy_hierarchy_group_id = payload.policy_hierarchy_group_id
rule.updated_at = datetime.now(timezone.utc)
revision(db, rule, "Entwurf gespeichert")
await db.commit()
await hydrate_hierarchy_targets(db, [rule])
return _to_read(rule, hosts, rule.deployments)
@router.get("/{rule_id}/preview")
async def preview_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_rule(db, rule_id))
@router.get("/{rule_id}/revisions")
async def list_revisions(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_rule(db, rule_id)
rows = (await db.scalars(select(PortForwardRevision).where(PortForwardRevision.port_forward_id == rule_id).order_by(PortForwardRevision.created_at.desc(), PortForwardRevision.id))).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{rule_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_port_forward_rule(deployment.host, deployment.opnsense_port_forward_uuid)
await opnsense_client.apply_port_forward(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/{rule_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
reverted = await revert_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
@router.get("/hosts/{host_id}/live")
async def live_rules(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
from app.services import opnsense_client
from app.services.nat_live_service import split_rows
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
try:
return split_rows(await opnsense_client.search_port_forward_rules(host))
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
backend/app/routers/resource_samples.py
+39
View File
@@ -0,0 +1,39 @@
import uuid
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.resource_sample import HostResourceSample
router = APIRouter(prefix="/hosts/{host_id}/resource-samples", tags=["host-resources"])
def _read(sample: HostResourceSample) -> dict:
return {
"id": str(sample.id),
"sampled_at": sample.sampled_at,
"mem_used_pct": sample.mem_used_pct,
"load_avg_1m": sample.load_avg_1m,
"disk_used_pct": sample.disk_used_pct,
"raw": sample.raw,
}
@router.get("")
async def list_samples(
host_id: uuid.UUID,
since_hours: int = Query(default=24, ge=1, le=24 * 30),
db: AsyncSession = Depends(get_db),
):
cutoff = datetime.now(timezone.utc) - timedelta(hours=since_hours)
rows = (
await db.scalars(
select(HostResourceSample)
.where(HostResourceSample.host_id == host_id, HostResourceSample.sampled_at >= cutoff)
.order_by(HostResourceSample.sampled_at.asc())
)
).all()
return [_read(r) for r in rows]
backend/app/routers/restore.py
+186
View File
@@ -0,0 +1,186 @@
import uuid
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.models.restore_operation import RestoreOperation
from app.services import backup_store, opnsense_client as client, audit_service, restore_service
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/restore", tags=["restore"])
PREVIEW_LIFETIME = timedelta(minutes=15)
class Preview(BaseModel):
backup_id: str = Field(min_length=1, max_length=255)
class Handoff(BaseModel):
confirm_name: str = Field(min_length=1, max_length=120)
class Confirm(BaseModel):
confirm_name: str = Field(min_length=1, max_length=120)
message: str | None = Field(default=None, max_length=1000)
async def host_or_404(db, host_id):
h = await db.get(OpnsenseHost, host_id)
if not h:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return h
async def op_or_404(db, host_id, operation_id):
op = await db.get(RestoreOperation, operation_id)
if not op or op.host_id != host_id:
raise api_error(404, ErrorCode.RESTORE_VORGANG_NICHT_GEFUNDEN)
return op
def read_operation(op: RestoreOperation) -> dict:
return {
"id": op.id, "host_id": op.host_id, "host_name": op.host_name, "actor": op.actor,
"backup_id": op.backup_id, "status": op.status, "created_at": op.created_at,
"expires_at": op.expires_at, "manifest": op.manifest, "warnings": op.warnings,
"handed_off_at": op.handed_off_at, "confirmed_at": op.confirmed_at,
"cancelled_at": op.cancelled_at, "message": op.message,
}
@router.get("")
async def list_operations(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await host_or_404(db, host_id)
ops = (await db.scalars(
select(RestoreOperation)
.where(RestoreOperation.host_id == host_id, RestoreOperation.status != "preview")
.order_by(RestoreOperation.created_at.desc())
.limit(20)
)).all()
return [read_operation(o) for o in ops]
@router.get("/{operation_id}")
async def get_operation(host_id: uuid.UUID, operation_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await host_or_404(db, host_id)
op = await op_or_404(db, host_id, operation_id)
return read_operation(op)
@router.post("/preview")
async def preview(host_id: uuid.UUID, payload: Preview, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await host_or_404(db, host_id)
# Best-effort fresh safety snapshot before previewing — same call the
# existing "Jetzt sichern" button already makes. If it fails we still
# let the preview proceed (the chosen backup is unaffected), just warn.
safety_warning = None
try:
content = await client.download_backup(host)
backup_store.save_snapshot(host_id, content)
except client.OpnsenseAPIError as exc:
safety_warning = f"Frische Sicherheitssicherung vor dem Restore fehlgeschlagen: {exc}"
raw = backup_store.read_snapshot(host_id, payload.backup_id)
if raw is None:
raise api_error(404, ErrorCode.SICHERUNG_NICHT_GEFUNDEN)
try:
manifest = restore_service.parse_and_redact(raw)
except restore_service.InvalidBackupError as exc:
raise api_error(400, ErrorCode(exc.code), **exc.params) from exc
warnings = restore_service.diff_against_host(manifest, host)
if safety_warning:
warnings.append(safety_warning)
op = RestoreOperation(
host_id=host.id, host_name=host.name, actor=request.state.user.username,
backup_id=payload.backup_id, expires_at=datetime.now(timezone.utc) + PREVIEW_LIFETIME,
fingerprint=restore_service.fingerprint(raw), manifest=manifest, warnings=warnings,
)
db.add(op)
await db.commit()
await audit_service.persist(
kind="restore", action="preview", status="preview", host_id=str(host.id), host_name=host.name,
entity_id=str(op.id), details={"backup_id": payload.backup_id, "manifest": manifest, "warnings": warnings},
)
return read_operation(op)
@router.post("/{operation_id}/handoff")
async def handoff(host_id: uuid.UUID, operation_id: uuid.UUID, payload: Handoff, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await host_or_404(db, host_id)
op = await op_or_404(db, host_id, operation_id)
if op.status != "preview":
raise api_error(409, ErrorCode.KEIN_OFFENER_RESTORE_VORSCHAU_VORGANG)
if op.expires_at <= datetime.now(timezone.utc):
raise api_error(409, ErrorCode.VORSCHAU_ABGELAUFEN_BITTE_ERNEUT_OEFFNEN)
if payload.confirm_name != host.name:
raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
raw = backup_store.read_snapshot(host_id, op.backup_id)
if raw is None or restore_service.fingerprint(raw) != op.fingerprint:
raise api_error(409, ErrorCode.SICHERUNG_WURDE_SEIT_DER_VORSCHAU_VERAENDERT_ODER_IST_2732C4DD)
op.status = "handed_off"
op.handed_off_at = datetime.now(timezone.utc)
op.confirm_name = payload.confirm_name
op.message = (
"Zur manuellen Wiederherstellung übergeben. Sicherung jetzt im OPNsense-Webinterface "
"importieren (System → Konfiguration → Historie/Wiederherstellen), Interface-/"
"Konsolenzuordnung prüfen, danach hier bestätigen."
)
await db.commit()
await audit_service.persist(
kind="restore", action="handoff", status="handed_off", host_id=str(host.id), host_name=host.name,
entity_id=str(op.id), message=op.message,
)
return read_operation(op)
@router.post("/{operation_id}/confirm")
async def confirm(host_id: uuid.UUID, operation_id: uuid.UUID, payload: Confirm, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await host_or_404(db, host_id)
op = await op_or_404(db, host_id, operation_id)
if op.status != "handed_off":
raise api_error(409, ErrorCode.KEIN_OFFENER_UEBERGEBENER_RESTORE_VORGANG)
if payload.confirm_name != host.name:
raise api_error(409, ErrorCode.HOSTNAMEN_EXAKT_EINGEBEN)
op.status = "verified"
op.confirmed_at = datetime.now(timezone.utc)
op.message = (
(payload.message or "").strip() or "Wiederherstellung manuell geprüft und bestätigt."
) + " Hinweis: Zentral verwaltete Regeln/Aliase/NAT für diesen Host vor dem nächsten Push neu abgleichen."
await db.commit()
await audit_service.persist(
kind="restore", action="confirm", status="verified", host_id=str(host.id), host_name=host.name,
entity_id=str(op.id), message=op.message,
)
return read_operation(op)
@router.post("/{operation_id}/cancel")
async def cancel(host_id: uuid.UUID, operation_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await host_or_404(db, host_id)
op = await op_or_404(db, host_id, operation_id)
if op.status not in ("preview", "handed_off"):
raise api_error(409, ErrorCode.VORGANG_KANN_NICHT_MEHR_ABGEBROCHEN_WERDEN)
op.status = "cancelled"
op.cancelled_at = datetime.now(timezone.utc)
await db.commit()
await audit_service.persist(
kind="restore", action="cancel", status="cancelled", host_id=str(host.id), host_name=host.name,
entity_id=str(op.id),
)
return read_operation(op)
backend/app/routers/rules.py
+261
View File
@@ -0,0 +1,261 @@
import uuid
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, RuleRevision
from app.services.policy_service import effective_hosts, snapshot, revision, preview, policy_lock, ensure_editable, RULE_FIELDS, workflow
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.schemas.rule import DeploymentResult, RuleCreate, RuleRead, RuleEdit, DeployApproval
from app.services.alias_service import check_alias_consistency, describe_alias_contents, ensure_aliases_settled
from app.services.interface_service import check_interface_consistency
from app.services.opnsense_client import OpnsenseAPIError
from app.services.rule_deploy_service import confirm_rule, deploy_rule, undeploy_rule, revert_deployment
router = APIRouter(prefix="/rules", tags=["rules"])
def _load_options():
return [
selectinload(FirewallRule.targets),
selectinload(FirewallRule.groups).selectinload(DeviceGroup.hosts),
selectinload(FirewallRule.deployments).selectinload(RuleDeployment.host),
]
async def _load_rule(db: AsyncSession, rule_id: uuid.UUID) -> FirewallRule:
await policy_lock(db)
rule = await db.get(FirewallRule, rule_id, options=_load_options())
if not rule:
raise api_error(404, ErrorCode.REGEL_NICHT_GEFUNDEN)
return rule
def _to_read(
rule: FirewallRule,
hosts: list[OpnsenseHost],
deployments: list[RuleDeployment],
) -> RuleRead:
hosts_by_id = {host.id: host for host in effective_hosts(rule)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(rule, deployments)
return RuleRead(
workflow_state=state,
workflow_counts=counts, id=rule.id,
description=rule.description,
interface=rule.interface,
direction=rule.direction,
ipprotocol=rule.ipprotocol,
protocol=rule.protocol,
action=rule.action,
source_net=rule.source_net,
source_port=rule.source_port,
destination_net=rule.destination_net,
destination_port=rule.destination_port,
enabled=rule.enabled,
created_at=rule.created_at,
updated_at=rule.updated_at,
target_host_ids=[host.id for host in rule.targets],
target_group_ids=[g.id for g in rule.groups],
effective_host_ids=[h.id for h in effective_hosts(rule)],
deployments=[
DeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
revert_deadline=d.revert_deadline,
)
for d in deployments
],
)
@router.get("", response_model=list[RuleRead])
async def list_rules(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(FirewallRule).options(*_load_options()))
rules = result.scalars().unique().all()
return [_to_read(rule, rule.targets, rule.deployments) for rule in rules]
@router.post("", response_model=RuleRead, status_code=201)
async def create_rule(payload: RuleCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
rule = FirewallRule(
description=payload.description,
interface=payload.interface,
direction=payload.direction,
ipprotocol=payload.ipprotocol,
protocol=payload.protocol,
action=payload.action,
source_net=payload.source_net,
source_port=payload.source_port,
destination_net=payload.destination_net,
destination_port=payload.destination_port,
enabled=payload.enabled,
targets=hosts,
groups=groups,
)
db.add(rule)
await db.flush()
revision(db, rule, "Regel angelegt")
await db.commit()
# All attributes we need (id, created_at, ...) were set client-side
# before commit, and `hosts`/an empty deployment list are already in
# hand — no need to touch any lazy relationship on `rule` here.
return _to_read(rule, hosts, [])
@router.post("/{rule_id}/deploy", response_model=RuleRead)
async def deploy(rule_id: uuid.UUID, approval: DeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_BESTAETIGEN_ODER_ROLLBACK_KLAEREN)
hosts = [h for h in effective_hosts(rule) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.REGEL_HAT_KEINE_ZIEL_HOSTS)
if len(hosts) > 1:
labels = await check_interface_consistency(hosts, rule.interface)
distinct = set(labels.values())
if len(distinct) > 1:
detail = ", ".join(f"{name}={label}" for name, label in sorted(labels.items()))
raise api_error(409, ErrorCode.INTERFACE_MEANING_DIFFERS, interface=rule.interface, detail=detail)
for field_label, value in (
("Source", rule.source_net),
("Destination", rule.destination_net),
("Source-Port", rule.source_port),
("Destination-Port", rule.destination_port),
):
if not value:
continue
contents = await check_alias_consistency(hosts, value)
if len(set(contents.values())) > 1:
detail = describe_alias_contents(contents)
raise api_error(409, ErrorCode.ALIAS_CONTENT_DIFFERS, field_label=field_label, alias=value, detail=detail)
await ensure_aliases_settled(db, hosts, [rule.source_net, rule.destination_net, rule.source_port, rule.destination_port])
await deploy_rule(db, rule, hosts)
await db.refresh(rule, attribute_names=["deployments"])
for d in rule.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(rule, hosts, rule.deployments)
@router.post("/{rule_id}/confirm", response_model=RuleRead)
async def confirm(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
# Mirrors the deploy endpoint below: confirm_rule mutates the same
# RuleDeployment objects already loaded onto rule.deployments (same
# session, same identity map), so it's visible immediately with no
# reload — deliberately avoiding an expire+db.get() round trip here,
# which previously surfaced a MissingGreenlet bug under pool_pre_ping
# (see rule_deploy_service history).
rule = await _load_rule(db, rule_id)
await confirm_rule(db, rule)
return _to_read(rule, rule.targets, rule.deployments)
@router.delete("/{rule_id}", status_code=204)
async def delete_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
if any(d.status in ("pending", "revert_failed") for d in rule.deployments):
raise api_error(409, ErrorCode.OFFENES_DEPLOYMENT_ZUERST_KLAEREN)
try:
await undeploy_rule(db, rule)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(rule, attribute_names=["deployments"])
await db.delete(rule)
await db.commit()
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.put("/{rule_id}", response_model=RuleRead)
async def edit_rule(rule_id: uuid.UUID, payload: RuleEdit, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
ensure_editable(rule)
if rule.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.REGEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_BF51B6B0)
hosts, groups = await resolve_targets(db, payload)
if not await db.scalar(select(RuleRevision.id).where(RuleRevision.rule_id == rule.id).limit(1)):
revision(db, rule, "Vor erster Bearbeitung")
for field in RULE_FIELDS:
setattr(rule, field, getattr(payload, field))
rule.targets, rule.groups = hosts, groups
# Association-only changes must also invalidate optimistic edits.
from datetime import datetime, timezone
rule.updated_at = datetime.now(timezone.utc)
revision(db, rule, "Entwurf gespeichert")
await db.commit()
return _to_read(rule, hosts, rule.deployments)
@router.get("/{rule_id}/preview")
async def preview_rule(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_rule(db, rule_id))
@router.get("/{rule_id}/revisions")
async def list_revisions(rule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_rule(db, rule_id)
rows = (await db.scalars(select(RuleRevision).where(RuleRevision.rule_id == rule_id).order_by(RuleRevision.created_at.desc(), RuleRevision.id))).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{rule_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_OHNE_OFFENEN_ROLLBACK_KOENNEN_06DE0ADB)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
await opnsense_client.del_rule(deployment.host, deployment.opnsense_rule_uuid)
await opnsense_client.apply(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
@router.post("/{rule_id}/deployments/{host_id}/retry-rollback", status_code=204)
async def retry_rollback(rule_id: uuid.UUID, host_id: uuid.UUID, approval: DeployApproval, db: AsyncSession = Depends(get_db)):
rule = await _load_rule(db, rule_id)
plan = preview(rule)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.DEPLOYMENT_WURDE_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_retry_rollback"]:
raise api_error(409, ErrorCode.FUER_DIESES_ZIEL_LIEGT_KEIN_WIEDERHOLBARER_ROLLBACK_VOR)
deployment = next(d for d in rule.deployments if d.host_id == host_id)
reverted = await revert_deployment(db, deployment)
await db.commit()
if not reverted:
raise api_error(502, ErrorCode.ROLLBACK_ERNEUT_FEHLGESCHLAGEN_PRUEFE_DIE_ERREICHBARKEIT_DER_FIREWALL)
backend/app/routers/scheduled_maintenance.py
+84
View File
@@ -0,0 +1,84 @@
import uuid
from datetime import datetime, timezone
from typing import Literal
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/maintenance/schedule", tags=["host-maintenance"])
class Schedule(BaseModel):
action: Literal["reboot", "update", "major_update"]
run_at: datetime
acknowledge_unattended: Literal[True]
def _read(sched: ScheduledMaintenance) -> dict:
return {
"id": sched.id, "host_id": sched.host_id, "action": sched.action, "run_at": sched.run_at,
"status": sched.status, "actor": sched.actor, "created_at": sched.created_at,
"result": sched.result, "host_operation_id": sched.host_operation_id,
}
async def _host_or_404(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.get("")
async def list_scheduled(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _host_or_404(db, host_id)
rows = (
await db.scalars(
select(ScheduledMaintenance)
.where(ScheduledMaintenance.host_id == host_id)
.order_by(ScheduledMaintenance.created_at.desc())
.limit(50)
)
).all()
return [_read(r) for r in rows]
@router.post("")
async def create_scheduled(host_id: uuid.UUID, payload: Schedule, request: Request, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
host = await _host_or_404(db, host_id)
if not host.enabled:
raise api_error(409, ErrorCode.HOST_IST_DEAKTIVIERT)
now = datetime.now(timezone.utc)
run_at = payload.run_at if payload.run_at.tzinfo else payload.run_at.replace(tzinfo=timezone.utc)
if run_at <= now:
raise api_error(422, ErrorCode.ZEITPUNKT_MUSS_IN_DER_ZUKUNFT_LIEGEN)
sched = ScheduledMaintenance(
host_id=host.id, host_name=host.name, action=payload.action, run_at=run_at,
actor=request.state.user.username, created_at=now,
)
db.add(sched)
await db.commit()
return _read(sched)
@router.delete("/{schedule_id}")
async def cancel_scheduled(host_id: uuid.UUID, schedule_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
await _host_or_404(db, host_id)
sched = await db.get(ScheduledMaintenance, schedule_id)
if not sched or sched.host_id != host_id:
raise api_error(404, ErrorCode.GEPLANTE_WARTUNG_NICHT_GEFUNDEN)
if sched.status != "pending":
raise api_error(409, ErrorCode.NUR_AUSSTEHENDE_GEPLANTE_WARTUNGEN_KOENNEN_ABGEBROCHEN_WERDEN)
sched.status = "cancelled"
await db.commit()
return _read(sched)
backend/app/routers/search.py
+306
View File
@@ -0,0 +1,306 @@
from typing import Callable
from fastapi import APIRouter, Depends, Query
from sqlalchemy import or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.sql.elements import ColumnElement
from app.database import get_db
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.ipsec_tunnel import IpsecTunnel
from app.models.managed_certificate import ManagedCertificate
from app.models.one_to_one import OneToOneRule
from app.models.openvpn_instance import OpenVpnInstance
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.policy_hierarchy_rule import PolicyHierarchyRule
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.models.wireguard_tunnel import WireguardTunnel
from app.schemas.search import SearchHit
router = APIRouter(prefix="/search", tags=["search"])
# Below this, ILIKE '%q%' on a handful of local tables is instant and a
# shorter query mostly returns noise — not a real limit on data volume.
MIN_QUERY_LENGTH = 2
# One spec = one queryable model within a result category. A category with
# several specs (NAT's port-forward/one-to-one/outbound rules; VPN's
# IPsec/OpenVPN/WireGuard) competes on relevance across all of them together
# — see _search_models — rather than giving each sub-type its own `limit`
# slots regardless of how well it actually matches.
Spec = tuple[type, list[ColumnElement], object, Callable[..., SearchHit]]
def _strip_scheme(base_url: str) -> str:
return base_url.removeprefix("https://").removeprefix("http://")
def _match_score(text: str | None, q: str) -> int:
"""Cheap relevance heuristic — no fuzzy/edit-distance matching, just
reward tighter matches so e.g. a rule *named* "vpn-uplink" outranks one
that merely mentions "vpn" in its destination net. `q` must already be
lowercased; `text` may be `None` for optional columns."""
if not text:
return 0
t = text.lower()
if t == q:
return 100
if t.startswith(q):
return 80
idx = t.find(q)
if idx == -1:
return 0
if idx == 0 or not t[idx - 1].isalnum():
return 60
return 40
async def _search_models(
db: AsyncSession,
specs: list[Spec],
q: str,
limit: int,
) -> list[SearchHit]:
like = f"%{q}%"
# Widen the per-spec candidate pool beyond `limit` so the cross-spec
# rescore below has enough to choose from — otherwise an early spec's
# weaker matches could crowd out a later spec's stronger ones just by
# being fetched first.
candidate_limit = min(max(limit * 2, 10), 30)
scored: list[tuple[int, object, Callable[..., SearchHit]]] = []
for model, columns, order_by, to_hit in specs:
rows = (
await db.scalars(
select(model).where(or_(*[c.ilike(like) for c in columns])).order_by(order_by).limit(candidate_limit)
)
).all()
for row in rows:
score = max(_match_score(getattr(row, c.key), q) for c in columns)
scored.append((score, row, to_hit))
scored.sort(key=lambda item: item[0], reverse=True)
return [to_hit(row) for _, row, to_hit in scored[:limit]]
@router.get("", response_model=list[SearchHit])
async def search(
q: str = Query(default=""),
limit: int = Query(default=5, ge=1, le=25),
db: AsyncSession = Depends(get_db),
):
q = q.strip().lower()
if len(q) < MIN_QUERY_LENGTH:
return []
hits: list[SearchHit] = []
hits += await _search_models(
db,
[
(
OpnsenseHost,
[OpnsenseHost.name, OpnsenseHost.base_url, OpnsenseHost.cluster_name],
OpnsenseHost.name,
lambda host: SearchHit(
category="hosts",
id=str(host.id),
title=host.name,
subtitle=host.cluster_name or _strip_scheme(host.base_url),
href=f"/hosts/{host.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
FirewallRule,
[FirewallRule.description, FirewallRule.source_net, FirewallRule.destination_net],
FirewallRule.updated_at.desc(),
lambda r: SearchHit(
category="rules",
id=str(r.id),
title=r.description or f"{r.source_net} → {r.destination_net}",
subtitle=f"{r.interface} · {r.action}",
href=f"/rules?rule={r.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
FirewallAlias,
[FirewallAlias.name, FirewallAlias.content, FirewallAlias.description],
FirewallAlias.updated_at.desc(),
lambda a: SearchHit(
category="aliases",
id=str(a.id),
title=a.name,
subtitle=a.content or a.description or None,
href=f"/aliases?alias={a.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
PortForwardRule,
[PortForwardRule.description, PortForwardRule.destination_net, PortForwardRule.target],
PortForwardRule.updated_at.desc(),
lambda r: SearchHit(
category="nat",
id=str(r.id),
title=r.description or f"{r.destination_net} → {r.target}",
subtitle="Port Forward",
href=f"/nat?type=port-forward&rule={r.id}",
),
),
(
OneToOneRule,
[OneToOneRule.description, OneToOneRule.source_net, OneToOneRule.external],
OneToOneRule.updated_at.desc(),
lambda r: SearchHit(
category="nat",
id=str(r.id),
title=r.description or f"{r.source_net} ↔ {r.external}",
subtitle="1:1 NAT",
href=f"/nat?type=one-to-one&rule={r.id}",
),
),
(
OutboundNatRule,
[OutboundNatRule.description, OutboundNatRule.source_net, OutboundNatRule.target],
OutboundNatRule.updated_at.desc(),
lambda r: SearchHit(
category="nat",
id=str(r.id),
title=r.description or f"{r.source_net} → {r.target}",
subtitle="Outbound NAT",
href=f"/nat?type=outbound-nat&rule={r.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
IpsecTunnel,
[IpsecTunnel.description, IpsecTunnel.remote_address, IpsecTunnel.remote_identity],
IpsecTunnel.updated_at.desc(),
lambda t: SearchHit(
category="vpn",
id=str(t.id),
title=t.description or t.remote_address,
subtitle=f"IPsec · {t.remote_address}",
href=f"/ipsec?tunnel={t.id}",
),
),
(
OpenVpnInstance,
[OpenVpnInstance.description, OpenVpnInstance.remote, OpenVpnInstance.server_network],
OpenVpnInstance.updated_at.desc(),
lambda i: SearchHit(
category="vpn",
id=str(i.id),
title=i.description,
subtitle=f"OpenVPN · {i.role}",
href=f"/openvpn?instance={i.id}",
),
),
(
WireguardTunnel,
[WireguardTunnel.server_name, WireguardTunnel.peer_name, WireguardTunnel.peer_allowed_ips],
WireguardTunnel.updated_at.desc(),
lambda t: SearchHit(
category="vpn",
id=str(t.id),
title=f"{t.server_name} · {t.peer_name}",
subtitle="WireGuard",
href=f"/wireguard?tunnel={t.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
ManagedCertificate,
[ManagedCertificate.description, ManagedCertificate.commonname],
ManagedCertificate.updated_at.desc(),
lambda c: SearchHit(
category="certificates",
id=str(c.id),
title=c.description or c.commonname,
subtitle=c.commonname or None,
href=f"/certificates?cert={c.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
DeviceGroup,
[DeviceGroup.name, DeviceGroup.description],
DeviceGroup.name,
lambda g: SearchHit(
category="groups",
id=str(g.id),
title=g.name,
subtitle=g.description or None,
href=f"/groups?group={g.id}",
),
),
],
q,
limit,
)
hits += await _search_models(
db,
[
(
PolicyHierarchyRule,
[PolicyHierarchyRule.description, PolicyHierarchyRule.source_net, PolicyHierarchyRule.destination_net],
PolicyHierarchyRule.updated_at.desc(),
lambda r: SearchHit(
category="policy-hierarchy",
id=str(r.id),
title=r.description or f"{r.source_net} → {r.destination_net}",
subtitle=f"{r.rulebase} · {r.interface}",
href=f"/policy-hierarchy?group={r.hierarchy_group_id}",
),
),
],
q,
limit,
)
return hits
backend/app/routers/services.py
+69
View File
@@ -0,0 +1,69 @@
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models import OpnsenseHost
from app.services import opnsense_client as client, audit_service
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/hosts/{host_id}/services", tags=["host-services"])
async def host_or_404(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if not host:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
@router.get("")
async def list_services(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
host = await host_or_404(db, host_id)
try:
return await client.search_services(host)
except client.OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
async def _control(host_id: uuid.UUID, name: str, action: str, fn, db: AsyncSession):
await policy_lock(db)
host = await host_or_404(db, host_id)
await ensure_no_maintenance(db, [host])
await audit_service.persist(
kind="host", action=action, status="requested",
host_id=str(host.id), host_name=host.name,
details={"service": name},
)
try:
result = await fn(host, name)
message = f"Dienst '{name}' {action} übergeben."
await audit_service.persist(
kind="host", action=action, status="accepted",
host_id=str(host.id), host_name=host.name,
message=message, details={"service": name, "response": result},
)
return result
except client.OpnsenseAPIError as exc:
await audit_service.persist(
kind="host", action=action, status="rejected",
host_id=str(host.id), host_name=host.name,
message=str(exc)[:800], details={"service": name},
)
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
@router.post("/{name}/start")
async def start_service(host_id: uuid.UUID, name: str, db: AsyncSession = Depends(get_db)):
return await _control(host_id, name, "start", client.start_service, db)
@router.post("/{name}/stop")
async def stop_service(host_id: uuid.UUID, name: str, db: AsyncSession = Depends(get_db)):
return await _control(host_id, name, "stop", client.stop_service, db)
@router.post("/{name}/restart")
async def restart_service(host_id: uuid.UUID, name: str, db: AsyncSession = Depends(get_db)):
return await _control(host_id, name, "restart", client.restart_service, db)
backend/app/routers/settings.py
+36
View File
@@ -0,0 +1,36 @@
from typing import Literal
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.dialects.postgresql import insert
from app.config import settings
from app.database import get_db
from app.models.preferences import UserPreferences
from app.models.user import AdminUser
from app.routers.auth import current_user
router=APIRouter(prefix='/settings',tags=['settings'])
class Preferences(BaseModel):
theme: Literal['light','dark','system'] | None = None
language: Literal['de','en'] | None = None
@router.get('')
async def read(user: AdminUser=Depends(current_user),db:AsyncSession=Depends(get_db)):
preferences=await db.get(UserPreferences,user.id)
return {'username':user.username,'theme':preferences.theme if preferences else 'system',
'language':preferences.language if preferences else 'de',
'deploy_confirm_window_seconds':settings.deploy_confirm_window_seconds,
'session_lifetime_seconds':settings.session_lifetime}
@router.put('')
async def update(payload:Preferences,user:AdminUser=Depends(current_user),db:AsyncSession=Depends(get_db)):
username = user.username
current = await db.get(UserPreferences, user.id)
theme = payload.theme or (current.theme if current else 'system')
language = payload.language or (current.language if current else 'de')
await db.execute(insert(UserPreferences).values(user_id=user.id,theme=theme,language=language).on_conflict_do_update(index_elements=['user_id'],set_={'theme':theme,'language':language}))
await db.commit()
return {'username':username,'theme':theme,'language':language,
'deploy_confirm_window_seconds':settings.deploy_confirm_window_seconds,
'session_lifetime_seconds':settings.session_lifetime}
backend/app/routers/system.py
+12
View File
@@ -0,0 +1,12 @@
from fastapi import APIRouter
from app.services import update_check_service
router = APIRouter(prefix="/system", tags=["system"])
@router.get("/version-check")
async def version_check() -> dict:
"""Ist eine neuere OPNexus-Version veröffentlicht? Aus, solange
UPDATE_CHECK_URL nicht gesetzt ist (dann ohne ausgehende Anfrage)."""
return await update_check_service.check()
backend/app/routers/users.py
+98
View File
@@ -0,0 +1,98 @@
import uuid
from fastapi import APIRouter, Depends
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.user import UserCreate, UserPasswordReset, UserRead, UserRoleUpdate
from app.services import auth_service, totp_service
router = APIRouter(prefix="/users", tags=["users"])
async def _admin_count(db: AsyncSession, exclude_id: uuid.UUID | None = None) -> int:
query = select(func.count()).select_from(AdminUser).where(AdminUser.role == "admin")
if exclude_id is not None:
query = query.where(AdminUser.id != exclude_id)
return await db.scalar(query)
@router.get("", response_model=list[UserRead])
async def list_users(db: AsyncSession = Depends(get_db)):
rows = (await db.scalars(select(AdminUser).order_by(AdminUser.username))).all()
return rows
@router.post("", response_model=UserRead, status_code=201)
async def create_user(payload: UserCreate, db: AsyncSession = Depends(get_db)):
existing = await db.scalar(select(AdminUser).where(AdminUser.username == payload.username))
if existing:
raise api_error(409, ErrorCode.BENUTZERNAME_BEREITS_VERGEBEN)
user = AdminUser(
username=payload.username,
password_hash=auth_service.hash_password(payload.password),
role=payload.role.value,
)
db.add(user)
await db.commit()
return user
@router.patch("/{user_id}/role", response_model=UserRead)
async def update_role(user_id: uuid.UUID, payload: UserRoleUpdate, db: AsyncSession = Depends(get_db)):
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
if target.role == "admin" and payload.role.value != "admin" and await _admin_count(db, exclude_id=target.id) == 0:
raise api_error(409, ErrorCode.MINDESTENS_EIN_ADMIN_ACCOUNT_MUSS_BESTEHEN_BLEIBEN)
target.role = payload.role.value
await db.commit()
return target
@router.post("/{user_id}/reset-2fa", response_model=UserRead)
async def reset_two_factor(user_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
"""Admin recovery for a user who lost their authenticator and recovery codes."""
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
totp_service.delete_secrets(target.id)
target.totp_enabled = False
target.totp_recovery_hashes = None
target.totp_last_step = 0
await db.commit()
return target
@router.post("/{user_id}/reset-password", response_model=UserRead)
async def reset_password(user_id: uuid.UUID, payload: UserPasswordReset, db: AsyncSession = Depends(get_db)):
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
target.password_hash = auth_service.hash_password(payload.new_password)
await db.commit()
return target
@router.delete("/{user_id}", status_code=204)
async def delete_user(
user_id: uuid.UUID,
db: AsyncSession = Depends(get_db),
current: AdminUser = Depends(current_user),
):
if user_id == current.id:
raise api_error(400, ErrorCode.DER_EIGENE_ACCOUNT_KANN_NICHT_GELOESCHT_WERDEN)
target = await db.get(AdminUser, user_id)
if target is None:
raise api_error(404, ErrorCode.BENUTZER_NICHT_GEFUNDEN)
# No separate "last admin" guard here: only an admin can reach this
# endpoint (read_only is rejected earlier, in AuthMiddleware) and the
# self-delete check above already guarantees target != current — so
# `current` is always a still-existing admin afterward, meaning the
# admin count can never reach zero this way.
await db.delete(target)
await db.commit()
backend/app/routers/wireguard.py
+223
View File
@@ -0,0 +1,223 @@
import uuid
from fastapi import APIRouter, Depends, Body
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.errors import ErrorCode, api_error
from app.database import get_db
from app.models.host import OpnsenseHost
from app.models.policy import DeviceGroup, WireguardTunnelRevision
from app.models.wireguard_tunnel import WireguardTunnel
from app.models.wireguard_tunnel_deployment import WireguardTunnelDeployment
from app.schemas.wireguard import (
WireguardDeployApproval, WireguardDeploymentResult, WireguardTunnelCreate, WireguardTunnelEdit, WireguardTunnelRead,
)
from app.services import wireguard_key_store
from app.services.wireguard_service import effective_hosts, preview, revision, snapshot, workflow, WIREGUARD_FIELDS
from app.services.wireguard_deploy_service import deploy_tunnel, undeploy_tunnel
from app.services.opnsense_client import OpnsenseAPIError
from app.services.policy_service import policy_lock
router = APIRouter(prefix="/wireguard/tunnels", tags=["wireguard"])
def _load_options():
return [
selectinload(WireguardTunnel.targets),
selectinload(WireguardTunnel.groups).selectinload(DeviceGroup.hosts),
selectinload(WireguardTunnel.deployments).selectinload(WireguardTunnelDeployment.host),
]
async def _load_tunnel(db: AsyncSession, tunnel_id: uuid.UUID) -> WireguardTunnel:
# Same advisory lock as rules/aliases/NAT/IPsec/OpenVPN — serializes
# concurrent deploy-affecting changes across processes. No
# ensure_editable() gate here either: this entity has no pending/
# revert_failed state to ever block editing on (see
# wireguard_deploy_service.py's module docstring).
await policy_lock(db)
tunnel = await db.get(WireguardTunnel, tunnel_id, options=_load_options())
if not tunnel:
raise api_error(404, ErrorCode.TUNNEL_NICHT_GEFUNDEN)
return tunnel
def _to_read(tunnel: WireguardTunnel, deployments: list[WireguardTunnelDeployment]) -> WireguardTunnelRead:
hosts_by_id = {host.id: host for host in effective_hosts(tunnel)}
hosts_by_id.update({d.host_id: d.host for d in deployments if d.host is not None})
state, counts = workflow(tunnel, deployments)
return WireguardTunnelRead(
workflow_state=state,
workflow_counts=counts,
id=tunnel.id,
server_name=tunnel.server_name,
server_public_key=tunnel.server_public_key,
server_private_key_set=tunnel.server_private_key_set,
listen_port=tunnel.listen_port,
server_tunnel_address=tunnel.server_tunnel_address,
dns=tunnel.dns,
mtu=tunnel.mtu,
peer_name=tunnel.peer_name,
peer_public_key=tunnel.peer_public_key,
peer_psk_enabled=tunnel.peer_psk_enabled,
peer_psk_set=tunnel.peer_psk_set,
peer_allowed_ips=tunnel.peer_allowed_ips,
peer_endpoint_address=tunnel.peer_endpoint_address,
peer_endpoint_port=tunnel.peer_endpoint_port,
peer_keepalive=tunnel.peer_keepalive,
enabled=tunnel.enabled,
created_at=tunnel.created_at,
updated_at=tunnel.updated_at,
target_host_ids=[host.id for host in tunnel.targets],
target_group_ids=[g.id for g in tunnel.groups],
effective_host_ids=[h.id for h in effective_hosts(tunnel)],
deployments=[
WireguardDeploymentResult(
host_id=d.host_id,
host_name=hosts_by_id[d.host_id].name if d.host_id in hosts_by_id else "?",
status=d.status,
message=d.message,
)
for d in deployments
],
)
async def resolve_targets(db, payload):
hosts = list((await db.scalars(select(OpnsenseHost).where(OpnsenseHost.id.in_(payload.target_host_ids)))).all())
groups = list((await db.scalars(select(DeviceGroup).where(DeviceGroup.id.in_(payload.target_group_ids)).options(selectinload(DeviceGroup.hosts)))).all())
if len(hosts) != len(set(payload.target_host_ids)) or len(groups) != len(set(payload.target_group_ids)):
raise api_error(400, ErrorCode.EIN_ODER_MEHRERE_ZIELE_WURDEN_NICHT_GEFUNDEN)
return hosts, groups
@router.get("", response_model=list[WireguardTunnelRead])
async def list_tunnels(db: AsyncSession = Depends(get_db)):
result = await db.execute(select(WireguardTunnel).options(*_load_options()))
tunnels = result.scalars().unique().all()
return [_to_read(tunnel, tunnel.deployments) for tunnel in tunnels]
@router.post("", response_model=WireguardTunnelRead, status_code=201)
async def create_tunnel(payload: WireguardTunnelCreate, db: AsyncSession = Depends(get_db)):
await policy_lock(db)
hosts, groups = await resolve_targets(db, payload)
tunnel = WireguardTunnel(
**{field: getattr(payload, field) for field in WIREGUARD_FIELDS},
targets=hosts,
groups=groups,
)
db.add(tunnel)
await db.flush()
wireguard_key_store.save_private_key(tunnel.id, payload.server_private_key)
tunnel.server_private_key_set = True
if payload.peer_psk_material:
wireguard_key_store.save_psk(tunnel.id, payload.peer_psk_material)
tunnel.peer_psk_set = True
revision(db, tunnel, "Tunnel angelegt")
await db.commit()
return _to_read(tunnel, [])
@router.put("/{tunnel_id}", response_model=WireguardTunnelRead)
async def edit_tunnel(tunnel_id: uuid.UUID, payload: WireguardTunnelEdit, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
if tunnel.updated_at != payload.expected_updated_at:
raise api_error(409, ErrorCode.TUNNEL_WURDE_INZWISCHEN_GEAENDERT_NEU_LADEN_UND_AENDERUNGEN_50830F70)
hosts, groups = await resolve_targets(db, payload)
if not await db.scalar(select(WireguardTunnelRevision.id).where(WireguardTunnelRevision.tunnel_id == tunnel.id).limit(1)):
revision(db, tunnel, "Vor erster Bearbeitung")
for field in WIREGUARD_FIELDS:
setattr(tunnel, field, getattr(payload, field))
tunnel.targets, tunnel.groups = hosts, groups
if payload.server_private_key is not None:
wireguard_key_store.save_private_key(tunnel.id, payload.server_private_key)
tunnel.server_private_key_set = True
if payload.peer_psk_material is not None:
wireguard_key_store.save_psk(tunnel.id, payload.peer_psk_material)
tunnel.peer_psk_set = True
from datetime import datetime, timezone
tunnel.updated_at = datetime.now(timezone.utc)
revision(db, tunnel, "Entwurf gespeichert")
await db.commit()
return _to_read(tunnel, tunnel.deployments)
@router.get("/{tunnel_id}/preview")
async def preview_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return preview(await _load_tunnel(db, tunnel_id))
@router.get("/{tunnel_id}/revisions")
async def list_revisions(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
await _load_tunnel(db, tunnel_id)
rows = (await db.scalars(
select(WireguardTunnelRevision).where(WireguardTunnelRevision.tunnel_id == tunnel_id)
.order_by(WireguardTunnelRevision.created_at.desc(), WireguardTunnelRevision.id)
)).all()
return [{"id": r.id, "created_at": r.created_at, "note": r.note, "snapshot": r.snapshot} for r in rows]
@router.post("/{tunnel_id}/deploy", response_model=WireguardTunnelRead)
async def deploy(tunnel_id: uuid.UUID, approval: WireguardDeployApproval | None = Body(default=None), db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel)
if approval and approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ENTWURF_ODER_ZIELE_WURDEN_GEAENDERT_VORSCHAU_ERNEUT_OEFFNEN)
if not tunnel.server_private_key_set:
raise api_error(400, ErrorCode.FUER_DIESEN_TUNNEL_IST_NOCH_KEIN_SERVER_PRIVATSCHLUESSEL_D7527998)
if tunnel.peer_psk_enabled and not tunnel.peer_psk_set:
raise api_error(400, ErrorCode.PEER_PRE_SHARED_KEY_IST_AKTIVIERT_ABER_ES_FCFAC129)
hosts = [h for h in effective_hosts(tunnel) if h.enabled]
if not hosts:
raise api_error(400, ErrorCode.TUNNEL_HAT_KEINE_ZIEL_HOSTS)
await deploy_tunnel(db, tunnel, hosts)
await db.refresh(tunnel, attribute_names=["deployments"])
for d in tunnel.deployments:
await db.refresh(d, attribute_names=["host"])
return _to_read(tunnel, tunnel.deployments)
@router.delete("/{tunnel_id}", status_code=204)
async def delete_tunnel(tunnel_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
try:
await undeploy_tunnel(db, tunnel)
except OpnsenseAPIError as exc:
raise api_error(502, ErrorCode.UPSTREAM_REQUEST_FAILED) from exc
await db.refresh(tunnel, attribute_names=["deployments"])
wireguard_key_store.delete_private_key(tunnel.id)
wireguard_key_store.delete_psk(tunnel.id)
await db.delete(tunnel)
await db.commit()
@router.post("/{tunnel_id}/deployments/{host_id}/remove", status_code=204)
async def remove_previous_target(tunnel_id: uuid.UUID, host_id: uuid.UUID, approval: WireguardDeployApproval, db: AsyncSession = Depends(get_db)):
tunnel = await _load_tunnel(db, tunnel_id)
plan = preview(tunnel)
if approval.preview_token != plan["token"]:
raise api_error(409, ErrorCode.ZIELE_ODER_DEPLOYMENT_WURDEN_GEAENDERT_VORSCHAU_AKTUALISIEREN)
entry = next((h for h in plan["hosts"] if h["host_id"] == str(host_id)), None)
if entry is None:
raise api_error(404, ErrorCode.DEPLOYMENT_NICHT_GEFUNDEN)
if not entry["can_remove"]:
raise api_error(409, ErrorCode.NUR_FRUEHERE_AKTIVE_ZIELE_KOENNEN_BEREINIGT_WERDEN)
deployment = next(d for d in tunnel.deployments if d.host_id == host_id)
from app.services import opnsense_client
try:
if deployment.opnsense_peer_uuid:
await opnsense_client.del_wireguard_peer(deployment.host, deployment.opnsense_peer_uuid)
if deployment.opnsense_server_uuid:
await opnsense_client.del_wireguard_server(deployment.host, deployment.opnsense_server_uuid)
await opnsense_client.reconfigure_wireguard(deployment.host)
except OpnsenseAPIError as exc:
deployment.status = "error"
deployment.message = str(exc)
await db.commit()
raise api_error(502, ErrorCode.ENTFERNEN_FEHLGESCHLAGEN_DIE_ZUORDNUNG_BLEIBT_FUER_EINEN_NEUEN_4CF90E4C) from exc
await db.delete(deployment)
await db.commit()
backend/app/schemas/__init__.py
View File
Whitespace-only changes.
backend/app/schemas/alias.py
+71
View File
@@ -0,0 +1,71 @@
import uuid
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.schemas.rule import DeployApproval, DeploymentResult # generic, reused as-is
# The OPNsense alias types this app manages centrally — content is a plain
# newline-separated list for all of these (verified against OPNsense's own
# alias model). Deliberately excludes: "internal" (auto-generated
# __if_network, not user-creatable), "external" (built-ins like
# bogons/sshlockout), and "url"/"urltable"/"urljson"/"geoip"/"dynipv6host"/
# "authgroup" (need a URL or other external config instead of/alongside a
# content list — different form shape, not yet built).
AliasType = Literal["host", "network", "port", "networkgroup", "mac", "asn"]
class AliasCreate(BaseModel):
name: str = Field(min_length=1, max_length=64, pattern=r"^[A-Za-z0-9_]+$")
type: AliasType = "host"
# One entry per line — IPs/hostnames (host), CIDRs (network), or
# numbers/ranges (port). Not validated further here: OPNsense is the
# authority on what's a legal entry for a given type, and rejects an
# invalid one at push time (surfaced as a per-host deploy error, same
# as an unknown interface today).
content: str = Field(default="", max_length=8000)
description: str = Field(default="", max_length=255)
enabled: bool = True
target_host_ids: list[uuid.UUID] = Field(default_factory=list)
target_group_ids: list[uuid.UUID] = Field(default_factory=list)
policy_hierarchy_group_id: uuid.UUID | None = None
@field_validator("name")
@classmethod
def clean_name(cls, value):
if not value.strip():
raise ValueError("Name darf nicht leer sein")
return value.strip()
@field_validator("content")
@classmethod
def clean_content(cls, value):
return "\n".join(line.strip() for line in value.splitlines() if line.strip())
class AliasRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
type: str
content: str
description: str
enabled: bool
created_at: datetime
updated_at: datetime
target_host_ids: list[uuid.UUID]
target_group_ids: list[uuid.UUID] = Field(default_factory=list)
policy_hierarchy_group_id: uuid.UUID | None = None
effective_host_ids: list[uuid.UUID] = Field(default_factory=list)
workflow_state: Literal["attention", "pending", "cleanup", "unassigned", "disabled", "draft", "synced"]
workflow_counts: dict[str, int]
deployments: list[DeploymentResult]
class AliasEdit(AliasCreate):
expected_updated_at: datetime
__all__ = ["AliasCreate", "AliasRead", "AliasEdit", "DeployApproval", "DeploymentResult"]
backend/app/schemas/carp_event.py
+16
View File
@@ -0,0 +1,16 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class CarpEventRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
host_id: uuid.UUID
host_name: str
cluster_name: str | None
from_status: str | None
to_status: str | None
detected_at: datetime
backend/app/schemas/certificates.py
+202
View File
@@ -0,0 +1,202 @@
import uuid
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
DeployStatus = Literal["draft", "success", "error"]
# ── Trust-Store (ManagedCertificate) ─────────────────────────────────────
CertAction = Literal["import", "internal", "external", "sign_csr", "import_csr", "reissue", "manual"]
# Curated subset of Trust/Cert.xml's key_type OptionValues (RSA sizes +
# EC curves) — see ManagedCertificate's docstring.
CertKeyType = Literal["2048", "3072", "4096", "prime256v1", "secp384r1", "secp521r1"]
CertDigest = Literal["sha1", "sha224", "sha256", "sha384", "sha512"]
CertType = Literal["usr_cert", "server_cert", "combined_server_client", "v3_ca"]
_DN_ACTIONS = {"internal", "external", "reissue"}
_CRT_ACTIONS = {"import", "import_csr", "manual"}
class ManagedCertificateBase(BaseModel):
host_id: uuid.UUID
action: CertAction
description: str = Field(min_length=1, max_length=255)
commonname: str = Field(default="", max_length=255)
country: str = Field(default="DE", max_length=8)
state: str = Field(default="", max_length=255)
city: str = Field(default="", max_length=255)
organization: str = Field(default="", max_length=255)
email: str = Field(default="", max_length=255)
key_type: CertKeyType = "2048"
digest: CertDigest = "sha256"
cert_type: CertType = "server_cert"
lifetime: int = Field(default=397, ge=1, le=36500)
caref: str | None = Field(default=None, max_length=64)
csr_payload: str | None = Field(default=None, max_length=20000)
crt_payload: str | None = Field(default=None, max_length=20000)
@field_validator("description")
@classmethod
def nonblank(cls, value):
if not value.strip():
raise ValueError("Feld darf nicht nur Leerzeichen enthalten")
return value.strip()
@model_validator(mode="after")
def check_action_requirements(self):
if self.action in _DN_ACTIONS and not self.commonname.strip():
raise ValueError("Für diese Aktion wird ein Common Name benötigt")
if self.action == "sign_csr" and not (self.csr_payload or "").strip():
raise ValueError("Für 'CSR signieren' wird ein CSR (PEM) benötigt")
if self.action in _CRT_ACTIONS and not (self.crt_payload or "").strip():
raise ValueError("Für diese Aktion wird ein Zertifikat (PEM) benötigt")
return self
class ManagedCertificateCreate(ManagedCertificateBase):
# Only meaningful for action="import" — see cert_key_store.py. Never
# returned by the API, only private_key_set.
private_key: str | None = Field(default=None, max_length=20000)
@model_validator(mode="after")
def check_private_key_scope(self):
if self.private_key and self.action != "import":
raise ValueError("Ein privater Schlüssel kann nur bei Aktion 'import' hinterlegt werden")
return self
class ManagedCertificateEdit(ManagedCertificateBase):
expected_updated_at: datetime
# None keeps the previously stored key unchanged (write-only, same
# pattern as IpsecTunnelEdit.pre_shared_key).
private_key: str | None = Field(default=None, max_length=20000)
@model_validator(mode="after")
def check_private_key_scope(self):
if self.private_key and self.action != "import":
raise ValueError("Ein privater Schlüssel kann nur bei Aktion 'import' hinterlegt werden")
return self
class ManagedCertificateRead(ManagedCertificateBase):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
private_key_set: bool
opnsense_uuid: str | None
status: DeployStatus
message: str | None
deployed_at: datetime | None
created_at: datetime
updated_at: datetime
# ── ACME accounts ─────────────────────────────────────────────────────
AcmeCA = Literal[
"letsencrypt", "letsencrypt_test", "zerossl", "buypass", "buypass_test", "google", "google_test", "sslcom", "custom"
]
class AcmeAccountBase(BaseModel):
host_id: uuid.UUID
name: str = Field(min_length=1, max_length=255)
description: str = Field(default="", max_length=255)
email: str = Field(default="", max_length=255)
ca: AcmeCA = "letsencrypt"
custom_ca: str = Field(default="", max_length=255)
eab_kid: str = Field(default="", max_length=255)
enabled: bool = True
@field_validator("name")
@classmethod
def nonblank(cls, value):
if not value.strip():
raise ValueError("Feld darf nicht nur Leerzeichen enthalten")
return value.strip()
@model_validator(mode="after")
def check_custom_ca(self):
if self.ca == "custom" and not self.custom_ca.strip():
raise ValueError("Für CA 'custom' wird eine Custom-CA-URL benötigt")
return self
class AcmeAccountCreate(AcmeAccountBase):
eab_hmac: str | None = Field(default=None, max_length=8192)
class AcmeAccountEdit(AcmeAccountBase):
expected_updated_at: datetime
eab_hmac: str | None = Field(default=None, max_length=8192)
class AcmeAccountRead(AcmeAccountBase):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
eab_hmac_set: bool
opnsense_uuid: str | None
status: DeployStatus
message: str | None
deployed_at: datetime | None
register_status: str | None
register_message: str | None
register_at: datetime | None
created_at: datetime
updated_at: datetime
# ── ACME certificate requests ─────────────────────────────────────────
# Prefixed OptionValues, verified against AcmeClient.xml — see
# AcmeCertificateRequest's docstring for why this matters.
AcmeKeyLength = Literal["key_2048", "key_3072", "key_4096", "key_ec256", "key_ec384"]
class AcmeCertificateRequestBase(BaseModel):
host_id: uuid.UUID
name: str = Field(min_length=1, max_length=255)
alt_names: str = Field(default="", max_length=2000)
account_id: uuid.UUID
validation_ref: str = Field(min_length=1, max_length=64)
key_length: AcmeKeyLength = "key_4096"
ocsp: bool = False
auto_renewal: bool = True
# Days SINCE LAST ISSUANCE, not days before expiry — see
# AcmeCertificateRequest's docstring. MinimumValue/MaximumValue
# verified against AcmeClient.xml's renewInterval field (0..5000).
renew_interval: int = Field(default=60, ge=0, le=5000)
enabled: bool = True
@field_validator("name", "validation_ref")
@classmethod
def nonblank(cls, value):
if not value.strip():
raise ValueError("Feld darf nicht nur Leerzeichen enthalten")
return value.strip()
class AcmeCertificateRequestCreate(AcmeCertificateRequestBase):
pass
class AcmeCertificateRequestEdit(AcmeCertificateRequestBase):
expected_updated_at: datetime
class AcmeCertificateRequestRead(AcmeCertificateRequestBase):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
opnsense_uuid: str | None
status: DeployStatus
message: str | None
deployed_at: datetime | None
sign_status: str | None
sign_message: str | None
sign_triggered_at: datetime | None
remote_status_code: int | None
remote_status_last_update: datetime | None
remote_last_update: datetime | None
created_at: datetime
updated_at: datetime
backend/app/schemas/dashboard.py
+40
View File
@@ -0,0 +1,40 @@
import uuid
from datetime import datetime
from pydantic import BaseModel
class DashboardHost(BaseModel):
id: uuid.UUID
name: str
status: str
carp_status: str | None
cluster_name: str | None
class PendingAction(BaseModel):
kind: str
status: str
entity_name: str
host_name: str
message: str | None
created_at: datetime
link: str
class ActivityItem(BaseModel):
id: uuid.UUID
created_at: datetime
actor: str
kind: str
action: str
status: str
entity_name: str | None
host_name: str | None
message: str | None
class DashboardSummary(BaseModel):
hosts: list[DashboardHost]
pending_actions: list[PendingAction]
recent_activity: list[ActivityItem]
Loaded 100 of 355 files, more files were not shown because too many files have changed in this diff. Show more