Compare commits

..
1 Commits
Author SHA1 Message Date
Julian Boldt b1a96c7563 Release v1.34.0
Release / verify (push) Waiting to run
Release / test (push) Blocked by required conditions
CI / backend (push) Blocked by required conditions
CI / frontend (push) Blocked by required conditions
CI / checks (push) Blocked by required conditions
- **Zum Update:** Das erste Update auf diese Version übernimmt auch die `docker-compose.yml` (siehe unten; braucht wie die
  CLI-Aktualisierung `cosign` und `cosign.pub`), migriert die Datenbank (Revision 0032; ein Rollback spielt die Sicherung davor
  zurück) und stellt das Backend auf einen unprivilegierten Benutzer um. Nichts davon braucht einen Handgriff.
- **Nur ein Backend führt die Hintergrunddienste aus (HA, Stufe H0):** Laufen zwei Backends an derselben Datenbank, verschickten
  beide jeden Alarm und brachen Zyklen mit `UniqueViolation` ab. Jetzt führt nur der Prozess mit der Leader-Sperre in der
  Datenbank die neun Hintergrundschleifen aus; die anderen warten in Bereitschaft und übernehmen innerhalb einer halben Minute,
  wenn der Leader ausfällt. Verliert der Leader seine Datenbankverbindung, hält er alle Schleifen sofort an (geprüft alle 5 Sekunden
  und vor jedem Durchlauf). An einer Replica startet kein Leader. Rolle in `GET /api/health` und als Metrik `opnexus_leader`.
  Zwei gleichzeitige Erststarts legen das Admin-Konto nur einmal an.
- **Umzug und Kaltreserve (HA, Stufe H1):** `opnexus backup --full --with-env --with-ca` nimmt zusätzlich die `.env` und die
  interne CA des Proxys mit, `opnexus restore DATEI --with-env --with-ca` spielt sie zurück (ohne Datenbank-Zugang, Version und
  Adressen des Zielservers; die CA bleibt dieselbe, Browser vertrauen weiter). Neu: `opnexus migrate-host BENUTZER@ZIEL` zieht
  eine Installation in einem Schritt auf einen anderen Server um (Vorprüfung, Quelle anhalten, Sicherung mit Zufalls-Passphrase,
  Restore, Kontrolle). Im Lab 90 Sekunden mit drei Firewalls. `opnexus status` zeigt das Alter der letzten Vollsicherung.
  Handbuch: „Umzug und Notfall“.
- **Betriebseinstellungen im Webinterface:** Was bisher nur in der `.env` oder gar nur im Code stand, lässt sich unter
  Einstellungen › Betriebseinstellungen ändern: Sitzungsdauer und Anmeldesperren, Bestätigungsfrist beim Ausrollen, Zeitlimit für
  Firewall-Anfragen, Messintervalle und Aufbewahrung, Intervall der Versionsprüfung, Heartbeat-Adresse, CA des Mailservers (PEM
  einfügen statt Datei einbinden), Zeitlimit Mailversand. Mit Grenzen, Standardwert, Zurücksetzen und Audit-Eintrag. Ein Wert in der
  `.env` hat weiter Vorrang und wird als „vom Server“ angezeigt. Adressen, TLS und Datenbank-Zugang bleiben bewusst in der `.env`.
- **Heartbeat für einen externen Überwacher:** Die Heartbeat-Adresse (nur `https://`) wird nach jedem erfolgreichen Prüfzyklus
  aufgerufen; bleibt der Aufruf aus, schlägt der Überwacher Alarm. Die Adresse wird nie angezeigt oder protokolliert.
- **Updates bringen die Compose-Datei mit:** Bisher tauschte `opnexus update` nur die Images; Änderungen an der
  `docker-compose.yml` (neue Variablen, Härtung) kamen bei bestehenden Installationen nie an. Jetzt übernimmt das Update die Datei
  des Zielreleases aus dem signierten Updater-Image, sichert die alte als `docker-compose.yml.bak-<version>` und legt sie beim
  Rollback zurück. Eine hier angepasste Datei wird nicht überschrieben (neue Fassung als `docker-compose.yml.new`); eigene
  Anpassungen gehören in `docker-compose.override.yml`.
- **Alarm „Platte des OPNexus-Servers fast voll“** (standardmäßig an, ab 90 %, einstellbar): eine volle Platte hält die Datenbank
  an und damit jeden anderen Alarm. Auch im Dashboard und als Metrik `opnexus_server_disk_used_percent`.
- **Härtung aus einem externen Audit:**
  - **Kein root mehr in den Containern:** Backend (uid 10001) und Frontend (`node`) laufen unprivilegiert, ohne neue Rechte (`no-new-privileges`), das Frontend zusätzlich
    ohne Capabilities. Das Backend startet über `backend/scripts/start.sh` noch kurz als root, stellt die vorhandenen Datenvolumes einmalig um und gibt dann alle Rechte ab:
    ein Update braucht keinen Handgriff, ein Rückweg auf eine ältere Version bleibt möglich. Notausgang für Sonderfälle: `OPNEXUS_RUN_AS_ROOT=1` in der `.env`.
    `wg-hub` (braucht root für WireGuard) und `gui-gateway` (läuft schon als nobody) sind unverändert. Der Proxy (Caddy) und der optionale Updater (Docker-Socket) laufen weiter wie bisher.
  - **Vollsicherung konsistent:** Ändern sich Geheimnisse, während die Sicherung läuft (jemand legt gerade ein Objekt an oder löscht eines), wird sie bis zu dreimal wiederholt;
    gelingt das nicht, bricht sie mit einer klaren Meldung ab, statt ein unstimmiges Archiv zu schreiben.
  - **Zieladress-Schutz überall:** Audit-Export (Webhook und Syslog), Offsite-S3 und der Update-Feed verbinden nur noch zu Adressen, die die Prüfung bestanden haben
    (kein Loopback, Link-Local, Metadaten-Dienst; auch nicht über DNS-Rebinding oder Weiterleitungen). Adressen im eigenen Netz bleiben erlaubt. Der Update-Feed wird beim Lesen auf 1 MB begrenzt, nicht erst danach.
- **Fehlende Basis-Rechte werden sichtbar:** Fehlen dem API-Schlüssel schon die Rechte für die Grundmessungen (Speicher, Last, Platte), schlug das ganze Sampling still fehl und der
  Rechte-Alarm konnte nie auslösen. Jetzt entsteht ein Befund „Grundwerte“ (Recht „Lobby: Dashboard“), der sich nach Korrektur von selbst schließt.
- **Installation:** Die `INSTALL.txt` im Bundle und die Anleitung auf der Website legen die `.env` jetzt mit `install -m 600` an (nur für dich lesbar), statt mit `cp`.
- **Oberfläche:** Die Aktionsknöpfe der Backup-Tabelle auf der Host-Seite werden auf schmalen Bildschirmen als Symbole gezeigt, statt abgeschnitten zu werden.
2026-10-24 18:55:10 +02:00
68 changed files with 3144 additions and 167 deletions

No files matched your search

+1
View File
@@ -9,3 +9,4 @@ backend/.pytest_cache/
.opnexus-update-state
dist/
.public-repo/
updater/docker-compose.release.yml
+46 -1
View File
@@ -29,7 +29,52 @@ in der App auf der Login-Seite, im Footer jeder Seite, auf `/about` und in
den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Saker …
ansteht): wird beim nächsten MINOR-Release festgelegt (Saker ist mit 1.33.0 vergeben).
## [1.34.0] "Griffon" — 2026-10-24
- **Zum Update:** Das erste Update auf diese Version übernimmt auch die `docker-compose.yml` (siehe unten; braucht wie die
CLI-Aktualisierung `cosign` und `cosign.pub`), migriert die Datenbank (Revision 0032; ein Rollback spielt die Sicherung davor
zurück) und stellt das Backend auf einen unprivilegierten Benutzer um. Nichts davon braucht einen Handgriff.
- **Nur ein Backend führt die Hintergrunddienste aus (HA, Stufe H0):** Laufen zwei Backends an derselben Datenbank, verschickten
beide jeden Alarm und brachen Zyklen mit `UniqueViolation` ab. Jetzt führt nur der Prozess mit der Leader-Sperre in der
Datenbank die neun Hintergrundschleifen aus; die anderen warten in Bereitschaft und übernehmen innerhalb einer halben Minute,
wenn der Leader ausfällt. Verliert der Leader seine Datenbankverbindung, hält er alle Schleifen sofort an (geprüft alle 5 Sekunden
und vor jedem Durchlauf). An einer Replica startet kein Leader. Rolle in `GET /api/health` und als Metrik `opnexus_leader`.
Zwei gleichzeitige Erststarts legen das Admin-Konto nur einmal an.
- **Umzug und Kaltreserve (HA, Stufe H1):** `opnexus backup --full --with-env --with-ca` nimmt zusätzlich die `.env` und die
interne CA des Proxys mit, `opnexus restore DATEI --with-env --with-ca` spielt sie zurück (ohne Datenbank-Zugang, Version und
Adressen des Zielservers; die CA bleibt dieselbe, Browser vertrauen weiter). Neu: `opnexus migrate-host BENUTZER@ZIEL` zieht
eine Installation in einem Schritt auf einen anderen Server um (Vorprüfung, Quelle anhalten, Sicherung mit Zufalls-Passphrase,
Restore, Kontrolle). Im Lab 90 Sekunden mit drei Firewalls. `opnexus status` zeigt das Alter der letzten Vollsicherung.
Handbuch: „Umzug und Notfall“.
- **Betriebseinstellungen im Webinterface:** Was bisher nur in der `.env` oder gar nur im Code stand, lässt sich unter
Einstellungen › Betriebseinstellungen ändern: Sitzungsdauer und Anmeldesperren, Bestätigungsfrist beim Ausrollen, Zeitlimit für
Firewall-Anfragen, Messintervalle und Aufbewahrung, Intervall der Versionsprüfung, Heartbeat-Adresse, CA des Mailservers (PEM
einfügen statt Datei einbinden), Zeitlimit Mailversand. Mit Grenzen, Standardwert, Zurücksetzen und Audit-Eintrag. Ein Wert in der
`.env` hat weiter Vorrang und wird als „vom Server“ angezeigt. Adressen, TLS und Datenbank-Zugang bleiben bewusst in der `.env`.
- **Heartbeat für einen externen Überwacher:** Die Heartbeat-Adresse (nur `https://`) wird nach jedem erfolgreichen Prüfzyklus
aufgerufen; bleibt der Aufruf aus, schlägt der Überwacher Alarm. Die Adresse wird nie angezeigt oder protokolliert.
- **Updates bringen die Compose-Datei mit:** Bisher tauschte `opnexus update` nur die Images; Änderungen an der
`docker-compose.yml` (neue Variablen, Härtung) kamen bei bestehenden Installationen nie an. Jetzt übernimmt das Update die Datei
des Zielreleases aus dem signierten Updater-Image, sichert die alte als `docker-compose.yml.bak-<version>` und legt sie beim
Rollback zurück. Eine hier angepasste Datei wird nicht überschrieben (neue Fassung als `docker-compose.yml.new`); eigene
Anpassungen gehören in `docker-compose.override.yml`.
- **Alarm „Platte des OPNexus-Servers fast voll“** (standardmäßig an, ab 90 %, einstellbar): eine volle Platte hält die Datenbank
an und damit jeden anderen Alarm. Auch im Dashboard und als Metrik `opnexus_server_disk_used_percent`.
- **Härtung aus einem externen Audit:**
- **Kein root mehr in den Containern:** Backend (uid 10001) und Frontend (`node`) laufen unprivilegiert, ohne neue Rechte (`no-new-privileges`), das Frontend zusätzlich
ohne Capabilities. Das Backend startet über `backend/scripts/start.sh` noch kurz als root, stellt die vorhandenen Datenvolumes einmalig um und gibt dann alle Rechte ab:
ein Update braucht keinen Handgriff, ein Rückweg auf eine ältere Version bleibt möglich. Notausgang für Sonderfälle: `OPNEXUS_RUN_AS_ROOT=1` in der `.env`.
`wg-hub` (braucht root für WireGuard) und `gui-gateway` (läuft schon als nobody) sind unverändert. Der Proxy (Caddy) und der optionale Updater (Docker-Socket) laufen weiter wie bisher.
- **Vollsicherung konsistent:** Ändern sich Geheimnisse, während die Sicherung läuft (jemand legt gerade ein Objekt an oder löscht eines), wird sie bis zu dreimal wiederholt;
gelingt das nicht, bricht sie mit einer klaren Meldung ab, statt ein unstimmiges Archiv zu schreiben.
- **Zieladress-Schutz überall:** Audit-Export (Webhook und Syslog), Offsite-S3 und der Update-Feed verbinden nur noch zu Adressen, die die Prüfung bestanden haben
(kein Loopback, Link-Local, Metadaten-Dienst; auch nicht über DNS-Rebinding oder Weiterleitungen). Adressen im eigenen Netz bleiben erlaubt. Der Update-Feed wird beim Lesen auf 1 MB begrenzt, nicht erst danach.
- **Fehlende Basis-Rechte werden sichtbar:** Fehlen dem API-Schlüssel schon die Rechte für die Grundmessungen (Speicher, Last, Platte), schlug das ganze Sampling still fehl und der
Rechte-Alarm konnte nie auslösen. Jetzt entsteht ein Befund „Grundwerte“ (Recht „Lobby: Dashboard“), der sich nach Korrektur von selbst schließt.
- **Installation:** Die `INSTALL.txt` im Bundle und die Anleitung auf der Website legen die `.env` jetzt mit `install -m 600` an (nur für dich lesbar), statt mit `cp`.
- **Oberfläche:** Die Aktionsknöpfe der Backup-Tabelle auf der Host-Seite werden auf schmalen Bildschirmen als Symbole gezeigt, statt abgeschnitten zu werden.
## [1.33.0] "Saker" — 2026-10-24
+2 -2
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
> **Stand:** `1.33.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
> **Stand:** `1.34.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen
@@ -68,7 +68,7 @@ Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelie
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.33.0
OPNEXUS_VERSION=1.34.0
```
```bash
+2 -2
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](docs/screenshots/readme-rules-en.webp)
> **Status:** `1.33.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
> **Status:** `1.34.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does
@@ -70,7 +70,7 @@ For access from other machines and for production use with TLS (bundled Caddy pr
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.33.0
OPNEXUS_VERSION=1.34.0
```
```bash
+10 -1
View File
@@ -17,4 +17,13 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
# Unprivilegierter Benutzer. Der Container startet über scripts/start.sh noch als
# root, stellt die Datenvolumes um und gibt die Rechte dann ab (Begründung dort).
# wg-hub und gui-gateway überschreiben den Befehl und umgehen das Skript bewusst:
# der Hub braucht root für die WireGuard-Schnittstelle, das Gateway läuft per
# Compose als nobody.
RUN groupadd --system --gid 10001 opnexus \
&& useradd --system --uid 10001 --gid 10001 --home-dir /app --shell /usr/sbin/nologin opnexus \
&& chmod +x /app/scripts/start.sh
CMD ["/app/scripts/start.sh", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+10 -1
View File
@@ -42,6 +42,13 @@ class Settings(BaseSettings):
# Quellcode gebaut; dann gibt es keine Updates per Klick (nur der Hinweis in den Einstellungen).
opnexus_image_repo: str = ""
# ── Heartbeat (app/services/heartbeat_service.py) ──────────────────
# Optionale Adresse eines externen Überwachers (healthchecks-Stil, ntfy, Uptime Kuma Push …). OPNexus ruft sie
# nach jedem erfolgreichen Prüfzyklus der Benachrichtigungen per GET auf: bleibt der Aufruf aus, ist OPNexus
# ausgefallen oder kommt nicht mehr an seine Datenbank. Nur https://, die Adresse wird nie protokolliert.
# Leer = aus.
heartbeat_url: str = ""
# ── Updates aus dem Webinterface (app/services/updater_client.py) ─────
# Optionaler Updater-Container (Compose-Profil "updater"). Das Backend hat
# nie Zugriff auf den Docker-Socket; es reicht nur "aktualisieren" /
@@ -182,7 +189,9 @@ class Settings(BaseSettings):
# entity id — see acme_secret_store.py.
acme_secret_dir: str = "/data/acme-secrets"
model_config = SettingsConfigDict(case_sensitive=False)
# env_ignore_empty: Compose passes unset variables as "" ("${X:-}"); they must mean "not set", so the value from
# the web interface (app/runtime_settings.py) applies and a number field does not fail to parse.
model_config = SettingsConfigDict(case_sensitive=False, env_ignore_empty=True)
settings = Settings()
+1
View File
@@ -191,6 +191,7 @@ class ErrorCode(str, Enum):
NOTIFICATION_CHANNEL_NAME_TAKEN = "NOTIFICATION_CHANNEL_NAME_TAKEN"
NOTIFICATION_CHANNEL_TEST_FAILED = "NOTIFICATION_CHANNEL_TEST_FAILED"
NOTIFICATION_WEBHOOK_URL_INVALID = "NOTIFICATION_WEBHOOK_URL_INVALID"
SETTING_INVALID = "SETTING_INVALID"
OVERLAY_SUBNET_LOCKED = "OVERLAY_SUBNET_LOCKED"
OVERLAY_NAME_TAKEN = "OVERLAY_NAME_TAKEN"
OVERLAY_PEER_NOT_FOUND = "OVERLAY_PEER_NOT_FOUND"
+119 -12
View File
@@ -15,6 +15,10 @@ or directly (passphrase in FULL_BACKUP_PASSPHRASE):
python -m app.full_backup create > file.opxfull
python -m app.full_backup info < file.opxfull # manifest only
python -m app.full_backup restore < file.opxfull
Files of the server itself (the .env and the proxy's internal CA) are not visible to the backend. The CLI hands them
over as a tar on stdin (``create --host-files``); they are stored under ``host/`` and only ever handed back as a tar
on stdout (``host-files``) for the CLI to place: ``restore`` never writes them.
"""
import io
import json
@@ -87,13 +91,30 @@ def _head_revision() -> str:
return ScriptDirectory.from_config(migrate._config()).get_current_head()
def create(passphrase: str, *, dump=_dump_database, revision=None) -> bytes:
if len(passphrase) < MIN_PASSPHRASE:
raise FullBackupError(f"Die Passphrase braucht mindestens {MIN_PASSPHRASE} Zeichen")
manifest = {
"format": FORMAT, "app_version": APP_VERSION, "db_revision": revision or _head_revision(),
"created_at": datetime.now(timezone.utc).isoformat(), "data": sorted(_DATA_DIRS),
}
# Die Geheimnis-Verzeichnisse ändern sich genau dann, wenn sich ein Objekt in der Datenbank ändert. Ein
# Schreibschutz über Datenbank und Dateien gleichzeitig gibt es nicht, deshalb wird eine Änderung während
# der Sicherung erkannt und die Sicherung wiederholt. (Die Verzeichnisse der Firewall-Backups und des Overlays
# gehören nicht dazu: sie ändern sich laufend und eine neue Datei darin macht die Sicherung nicht unstimmig.)
_STABLE_DIRS = tuple(name for name in _DATA_DIRS if name.endswith("secrets"))
ATTEMPTS = 3
def _fingerprint() -> frozenset:
seen = set()
for name in _STABLE_DIRS:
root = _dir(name)
if not root.is_dir():
continue
for path in root.rglob("*"):
try:
st = path.stat()
except OSError: # vom Löschen überholt: zählt als Änderung beim nächsten Vergleich
continue
seen.add((name, str(path.relative_to(root)), st.st_mtime_ns, st.st_size))
return frozenset(seen)
def _pack(manifest: dict, dump, host_files: bytes | None = None) -> bytes:
plain = io.BytesIO()
with tempfile.TemporaryDirectory() as tmp, tarfile.open(fileobj=plain, mode="w:gz") as tar:
raw = json.dumps(manifest, indent=1).encode()
@@ -107,7 +128,69 @@ def create(passphrase: str, *, dump=_dump_database, revision=None) -> bytes:
path = _dir(name)
if path.is_dir():
tar.add(path, arcname=f"data/{name}")
return backup_crypto.encrypt(plain.getvalue(), passphrase)
for member, content in _host_members(host_files):
tar.addfile(member, io.BytesIO(content) if content is not None else None)
return plain.getvalue()
_HOST_TOP = ("env", "caddy-ca")
_HOST_MAX_BYTES = 10_000_000
def _host_members(host_files: bytes | None) -> list[tuple[tarfile.TarInfo, bytes | None]]:
"""The regular files and directories of the CLI's tar, re-rooted under host/. Anything else (links, devices,
absolute or escaping paths, unknown top-level names) is refused: this tar comes from outside the container."""
if host_files is None:
return []
if len(host_files) > _HOST_MAX_BYTES:
raise FullBackupError("Die Dateien des Servers sind zu groß")
try:
src = tarfile.open(fileobj=io.BytesIO(host_files), mode="r:")
except tarfile.TarError:
raise FullBackupError("Die Dateien des Servers sind kein gültiges tar") from None
out = []
with src:
for member in src.getmembers():
parts = [p for p in Path(member.name).parts if p not in (".", "")]
if not parts:
continue
if member.name.startswith(("/", "\\")) or ".." in parts or parts[0] not in _HOST_TOP:
raise FullBackupError(f"Unzulässiger Eintrag in den Dateien des Servers: {member.name!r}")
if not (member.isfile() or member.isdir()):
raise FullBackupError(f"Unzulässiger Eintrag in den Dateien des Servers: {member.name!r}")
info = tarfile.TarInfo("host/" + "/".join(parts))
info.mtime, info.mode = member.mtime, (0o600 if member.isfile() else 0o700)
if member.isdir():
info.type = tarfile.DIRTYPE
out.append((info, None))
else:
content = src.extractfile(member).read()
info.size = len(content)
out.append((info, content))
if not out:
raise FullBackupError("Die Dateien des Servers sind leer")
return out
def create(passphrase: str, *, dump=_dump_database, revision=None, host_files: bytes | None = None) -> bytes:
if len(passphrase) < MIN_PASSPHRASE:
raise FullBackupError(f"Die Passphrase braucht mindestens {MIN_PASSPHRASE} Zeichen")
manifest = {
"format": FORMAT, "app_version": APP_VERSION, "db_revision": revision or _head_revision(),
"created_at": datetime.now(timezone.utc).isoformat(), "data": sorted(_DATA_DIRS),
}
members = _host_members(host_files) # validated before the (slow) dump
if members:
manifest["host"] = sorted({m.name.split("/")[1] for m, _ in members})
for _ in range(ATTEMPTS):
before = _fingerprint()
blob = _pack(manifest, dump, host_files)
if _fingerprint() == before:
return backup_crypto.encrypt(blob, passphrase)
raise FullBackupError(
"Während der Sicherung haben sich Geheimnisse geändert (jemand legt gerade Objekte an oder löscht sie). "
"Die Sicherung wäre in sich unstimmig; bitte gleich noch einmal versuchen."
)
def _open(blob: bytes, passphrase: str) -> tarfile.TarFile:
@@ -178,6 +261,25 @@ def restore(blob: bytes, passphrase: str, *, restore_db=_restore_database, head=
return {**manifest, "files_restored": len(restored)}
def host_files(blob: bytes, passphrase: str) -> bytes:
"""The host/ part as a tar (paths without the host/ prefix) for the CLI; empty tar when the backup has none."""
out = io.BytesIO()
with _open(blob, passphrase) as tar, tarfile.open(fileobj=out, mode="w") as dst:
_manifest(tar)
for member in tar.getmembers():
parts = Path(member.name).parts
if len(parts) < 2 or parts[0] != "host" or ".." in parts or not (member.isfile() or member.isdir()):
continue
info = tarfile.TarInfo("/".join(parts[1:]))
info.mtime, info.mode, info.type = member.mtime, member.mode, member.type
if member.isfile():
info.size = member.size
dst.addfile(info, tar.extractfile(member))
else:
dst.addfile(info)
return out.getvalue()
def _older_or_equal(revision: str | None, head: str | None) -> bool:
"""Revisions are zero-padded numbers ('0009'): a dump from an older schema is fine, the app migrates it."""
try:
@@ -188,13 +290,18 @@ def _older_or_equal(revision: str | None, head: str | None) -> bool:
def _main(argv: list[str]) -> int:
passphrase = os.environ.get("FULL_BACKUP_PASSPHRASE", "")
if len(argv) != 2 or argv[1] not in ("create", "info", "restore"):
args = argv[1:]
with_host = args[1:] == ["--host-files"] and args[0] == "create"
if not (len(args) == 1 and args[0] in ("create", "info", "restore", "host-files")) and not with_host:
print(__doc__, file=sys.stderr)
return 2
try:
if argv[1] == "create":
sys.stdout.buffer.write(create(passphrase))
elif argv[1] == "info":
if args[0] == "create":
host = sys.stdin.buffer.read(_HOST_MAX_BYTES + 1) if with_host else None
sys.stdout.buffer.write(create(passphrase, host_files=host))
elif args[0] == "host-files":
sys.stdout.buffer.write(host_files(sys.stdin.buffer.read(), passphrase))
elif args[0] == "info":
print(json.dumps(info(sys.stdin.buffer.read(), passphrase), indent=1))
else:
print(json.dumps(restore(sys.stdin.buffer.read(), passphrase), indent=1))
+226
View File
@@ -0,0 +1,226 @@
"""Leader gate: only one backend process runs the background loops (phase H0 of docs/HA-PLAN.md).
Two backends against the same database are not hypothetical: a second stack during a host move, an update that
briefly overlaps, more than one Uvicorn worker, or (later) a standby. Deploys and sweeps are already serialized by
``policy_lock``, but the loops are not: in the lab two backends sent every notification twice and aborted cycles on
``UniqueViolation``. So the loops run only in the process that holds a session-level advisory lock.
How it holds:
- The lock lives on a connection of its own that is never returned to the pool. If the process dies, PostgreSQL
closes the session and frees the lock.
- The lock can also vanish while this process lives (PostgreSQL restart, network cut, the session terminated). The
connection is therefore checked every ``check_interval`` seconds and before every loop iteration (``checkpoint``).
On the first failure every loop is cancelled at once and the process applies for the lock again.
- A process that just got the lock waits ``takeover_delay`` before starting its loops: longer than the previous
holder needs to notice its loss (one check interval plus the check timeout). Without that wait both could run an
iteration at the same moment right after a takeover.
- Losing the lock cancels the loops mid-iteration, like a hard restart would; every loop is built to survive that
(deadlines and phases live in the database, ambiguous firewall calls are reconciled). The check timeout is generous
(5 s) so a merely sluggish database does not make a single installation drop its loops.
- A database in recovery (a streaming replica) never yields a leader: advisory locks work there, the loops would
not (they write). The process stays in standby until the database is promoted.
The API is served by every process regardless of its role; writes through the API are serialized by the database
as before. ``role`` is shown by /api/health and the metrics.
"""
import asyncio
import logging
from collections.abc import Awaitable, Callable
from sqlalchemy import text
from app.database import engine
logger = logging.getLogger(__name__)
# Next to policy_lock (728194031) and the admin-role lock (728194032).
LOCK_KEY = 728194033
STARTING, STANDBY, TAKEOVER, LEADER, REPLICA = "starting", "standby", "takeover", "leader", "replica"
class LeadershipLost(Exception):
"""Raised by ``checkpoint`` in a loop of a process that is no longer the leader. Loops must not catch it."""
LoopFactory = Callable[[], Awaitable[None]]
class LeaderGate:
def __init__(self, loops: dict[str, LoopFactory], *, check_interval: float = 5.0, check_timeout: float = 5.0,
retry_interval: float = 15.0, takeover_delay: float | None = None, connect=engine.connect):
self._loops = loops
self.check_interval = check_interval
self.check_timeout = check_timeout
self.retry_interval = retry_interval
self.takeover_delay = check_interval + check_timeout + 2.0 if takeover_delay is None else takeover_delay
self._connect = connect
self._conn = None
self._conn_lock = asyncio.Lock() # one statement at a time on the lock connection
self._tasks: dict[str, asyncio.Task] = {}
self._lost = asyncio.Event()
self._last_ok = 0.0
self.role = STARTING
# ── lock connection ──────────────────────────────────────────────────
async def _close(self) -> None:
conn, self._conn = self._conn, None
if conn is None:
return
try:
await asyncio.wait_for(conn.invalidate(), self.check_timeout) # never back to the pool: it may hold the lock
except Exception:
pass
try:
await asyncio.wait_for(conn.close(), self.check_timeout)
except Exception:
pass
async def _try_acquire(self) -> str:
"""STANDBY (lock held elsewhere or database unreachable), REPLICA, or TAKEOVER (lock is ours)."""
try:
self._conn = await asyncio.wait_for(self._connect(), self.check_timeout)
async with self._conn_lock:
if await asyncio.wait_for(self._conn.scalar(text("SELECT pg_is_in_recovery()")), self.check_timeout):
await self._close()
return REPLICA
got = await asyncio.wait_for(
self._conn.scalar(text("SELECT pg_try_advisory_lock(:key)"), {"key": LOCK_KEY}), self.check_timeout)
await self._conn.commit() # leave no transaction open on a connection that lives for days
except Exception as exc:
logger.warning("Leader-Sperre: Datenbank nicht erreichbar (%s)", type(exc).__name__)
await self._close()
return STANDBY
if not got:
await self._close()
return STANDBY
self._last_ok = asyncio.get_running_loop().time()
return TAKEOVER
async def check(self) -> bool:
"""True while the lock connection answers and the database is still a primary. A False is final for this
tenure: the caller demotes."""
if self._conn is None:
return False
try:
async with self._conn_lock:
recovery = await asyncio.wait_for(self._conn.scalar(text("SELECT pg_is_in_recovery()")),
self.check_timeout)
await self._conn.commit()
except Exception as exc:
logger.error("Leader-Sperre verloren: Verbindung zur Datenbank gestört (%s)", type(exc).__name__)
self._lost.set()
return False
if recovery:
logger.error("Leader-Sperre aufgegeben: die Datenbank ist jetzt eine Replica")
self._lost.set()
return False
self._last_ok = asyncio.get_running_loop().time()
return True
async def checkpoint(self) -> None:
"""Called at the top of every loop iteration. Raises LeadershipLost unless this process is still the leader;
a recent successful check (under a second old) is reused so a draining loop does not hammer the database."""
if self.role != LEADER or self._lost.is_set():
raise LeadershipLost()
if asyncio.get_running_loop().time() - self._last_ok < 1.0:
return
if not await self.check():
raise LeadershipLost()
# ── loops ────────────────────────────────────────────────────────────
def _start_loops(self) -> None:
for name, factory in self._loops.items():
self._tasks[name] = asyncio.create_task(factory(), name=f"loop:{name}")
async def _stop_loops(self) -> None:
tasks, self._tasks = list(self._tasks.values()), {}
for task in tasks:
task.cancel()
# return_exceptions collects the loops' own CancelledError; a cancellation of THIS task still propagates.
await asyncio.gather(*tasks, return_exceptions=True)
def _restart_crashed(self) -> None:
for name, task in list(self._tasks.items()):
if not task.done() or task.cancelled():
continue
exc = task.exception()
if isinstance(exc, LeadershipLost):
self._lost.set()
continue
logger.error("Hintergrundschleife %s beendet (%s), wird neu gestartet", name,
type(exc).__name__ if exc else "ohne Fehler")
self._tasks[name] = asyncio.create_task(self._loops[name](), name=f"loop:{name}")
# ── supervisor ───────────────────────────────────────────────────────
async def _wait_or_lost(self, seconds: float) -> None:
try:
await asyncio.wait_for(self._lost.wait(), seconds)
except asyncio.TimeoutError:
pass
async def _tenure(self) -> None:
"""From the moment the lock is ours until it is lost."""
self.role = TAKEOVER
logger.info("Leader-Sperre erhalten; Hintergrunddienste starten in %.0f s", self.takeover_delay)
deadline = asyncio.get_running_loop().time() + self.takeover_delay
while (left := deadline - asyncio.get_running_loop().time()) > 0:
await self._wait_or_lost(min(left, self.check_interval))
if self._lost.is_set() or not await self.check():
return
self.role = LEADER
self._start_loops()
logger.info("Dieser Prozess führt die Hintergrunddienste aus (Leader)")
while not self._lost.is_set():
await self._wait_or_lost(self.check_interval)
if self._lost.is_set() or not await self.check():
return
self._restart_crashed()
async def run(self) -> None:
announced = None
while True:
self._lost.clear()
state = await self._try_acquire()
if state == TAKEOVER:
try:
await self._tenure()
finally:
# Order matters: no loop may outlive the lock.
await self._stop_loops()
await self._close()
self.role = STANDBY
logger.warning("Hintergrunddienste angehalten (Leader-Sperre verloren); bewerbe mich neu")
announced = None
else:
self.role = state
if announced != state:
logger.info(
"Bereitschaft: %s", "die Datenbank ist eine Replica" if state == REPLICA
else "ein anderer Prozess führt die Hintergrunddienste aus oder die Datenbank fehlt")
announced = state
await asyncio.sleep(self.retry_interval)
async def stop(self) -> None:
"""Shutdown: loops first, then the lock connection (closing it releases the lock)."""
self._lost.set()
await self._stop_loops()
await self._close()
self.role = STANDBY
# Set by main.lifespan; read by /api/health and the metrics.
gate: LeaderGate | None = None
def current_role() -> str:
return gate.role if gate is not None else STARTING
async def checkpoint() -> None:
"""For the loops in main.py. Without a gate (tests that drive a loop directly) there is nothing to check."""
if gate is not None:
await gate.checkpoint()
+44 -29
View File
@@ -10,6 +10,7 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from sqlalchemy.exc import DBAPIError
from app import runtime_settings
from app.errors import ErrorCode
from app import models # noqa: F401 - registers models with Base.metadata
@@ -65,6 +66,8 @@ from app.services import backup_schedule_service
from app.services import carp_event_service, carp_maintenance_window_service
from app.services import rule_history_service
from app.services import rule_cleanup_service
from app.services import heartbeat_service
from app import leader
# Without this, Python's root logger defaults to WARNING with no handler,
# so every logger.info() call in the 5 background loops below (CARP role
@@ -87,7 +90,7 @@ _SWEEP_INTERVAL_SECONDS = 10
# work once the table is non-trivially sized.
_LOG_PRUNE_EVERY_N_CYCLES = 20
# Same rationale as _LOG_PRUNE_EVERY_N_CYCLES, but the resource-sampling
# loop's own cadence (settings.resource_poll_interval_seconds, default 60s).
# loop's own cadence (runtime_settings.value("resource_poll_interval_seconds"), default 60s).
_RESOURCE_PRUNE_EVERY_N_CYCLES = 20
@@ -96,6 +99,7 @@ async def _deploy_revert_sweep_loop():
RuleDeployment whose confirm window passed without a human confirming
(see rule_deploy_service.sweep_expired_deployments)."""
while True:
await leader.checkpoint()
try:
async with SessionLocal() as db:
reverted = await sweep_expired_deployments(db)
@@ -164,12 +168,15 @@ async def _notification_poll_loop():
NotificationSettings.poll_interval_seconds, floored by
settings.notification_poll_min_interval_seconds."""
while True:
await leader.checkpoint()
interval = settings.notification_poll_min_interval_seconds
try:
async with SessionLocal() as db:
await notification_service.run_notification_checks(db)
row = await notification_settings_service.get_settings(db)
interval = max(row.poll_interval_seconds, settings.notification_poll_min_interval_seconds)
# Only after a cycle that went through: the external watcher must hear nothing when the checks fail.
await heartbeat_service.ping()
except Exception:
logger.exception("Notification poll failed")
try:
@@ -194,6 +201,7 @@ async def _log_ingest_poll_loop():
every _LOG_PRUNE_EVERY_N_CYCLES cycles (see constant above)."""
cycle = 0
while True:
await leader.checkpoint()
interval = settings.log_poll_min_interval_seconds
cycle += 1
try:
@@ -227,11 +235,12 @@ async def _log_ingest_poll_loop():
async def _resource_poll_loop():
"""Fourth background loop, same shape as _log_ingest_poll_loop but with
a fixed interval (settings.resource_poll_interval_seconds) — v1
a fixed interval (runtime_settings.value("resource_poll_interval_seconds")) — v1
deliberately has no admin-editable settings row for this, see
resource_monitor_service.py's module docstring."""
cycle = 0
while True:
await leader.checkpoint()
cycle += 1
try:
async with SessionLocal() as db:
@@ -242,7 +251,7 @@ async def _resource_poll_loop():
result["hosts_failed"], result["hosts_polled"],
)
if cycle % _RESOURCE_PRUNE_EVERY_N_CYCLES == 0:
deleted = await resource_monitor_service.prune_old_samples(db, settings.resource_sample_retention_days)
deleted = await resource_monitor_service.prune_old_samples(db, runtime_settings.value("resource_sample_retention_days"))
if deleted:
logger.info("Resource-Sample-Retention: %d alte Einträge gelöscht", deleted)
pruned = await rule_history_service.prune(db)
@@ -250,7 +259,7 @@ async def _resource_poll_loop():
logger.info("Regelverlauf-Retention: %d alte Einträge gelöscht", pruned)
except Exception:
logger.exception("Resource-Poll fehlgeschlagen")
await asyncio.sleep(settings.resource_poll_interval_seconds)
await asyncio.sleep(runtime_settings.value("resource_poll_interval_seconds"))
_SCHEDULED_MAINTENANCE_POLL_SECONDS = 60
@@ -262,6 +271,7 @@ async def _scheduled_maintenance_poll_loop():
A fixed 60s cadence is plenty — these are human-picked timestamps, not
a sub-minute-precision cron."""
while True:
await leader.checkpoint()
try:
async with SessionLocal() as db:
result = await scheduled_maintenance_service.fire_due(db)
@@ -295,6 +305,7 @@ async def _compliance_poll_loop():
is plenty; the first run waits a little so a restart is not a thundering herd."""
await asyncio.sleep(_COMPLIANCE_FIRST_RUN_SECONDS)
while True:
await leader.checkpoint()
try:
async with SessionLocal() as db:
await compliance_service.check_all(db)
@@ -310,6 +321,7 @@ async def _audit_export_poll_loop():
"""Seventh background loop: forwards new audit events (syslog / webhook) when configured.
Cheap when off (one settings read); keeps draining while batches come back full."""
while True:
await leader.checkpoint()
shipped = 0
try:
async with SessionLocal() as db:
@@ -326,6 +338,7 @@ async def _backup_poll_loop():
"""Eighth background loop: takes due scheduled config backups, prunes, copies offsite."""
await asyncio.sleep(90)
while True:
await leader.checkpoint()
try:
async with SessionLocal() as db:
result = await backup_schedule_service.run_due(db)
@@ -345,6 +358,7 @@ async def _rule_observe_loop():
Writes only to the database. The first look at a firewall only stores the starting point."""
await asyncio.sleep(_RULE_OBSERVE_FIRST_RUN_SECONDS)
while True:
await leader.checkpoint()
try:
async with SessionLocal() as db:
result = await rule_history_service.observe_all(db)
@@ -352,7 +366,7 @@ async def _rule_observe_loop():
logger.info("Regelverlauf: %s", result)
except Exception:
logger.exception("Regelverlauf fehlgeschlagen")
await asyncio.sleep(max(settings.rule_observe_interval_seconds, settings.rule_observe_min_interval_seconds))
await asyncio.sleep(max(runtime_settings.value("rule_observe_interval_seconds"), settings.rule_observe_min_interval_seconds))
@asynccontextmanager
@@ -362,6 +376,7 @@ async def lifespan(app: FastAPI):
# A GUI-gateway slot must never be a trusted OPNexus origin (CORS with credentials).
raise RuntimeError(f"OPNEXUS_GUI_SITES overlaps FRONTEND_ORIGIN: {', '.join(conflicts)} — refusing to start")
await migrate.run()
await runtime_settings.refresh() # values from Settings › Betrieb; refreshed in every process (leader or not)
async with SessionLocal() as db:
await auth_service.bootstrap_admin(db)
try:
@@ -376,32 +391,30 @@ async def lifespan(app: FastAPI):
if settings.allow_insecure_firewall_urls:
logger.warning("ALLOW_INSECURE_FIREWALL_URLS is on: firewall API credentials may travel over plain HTTP.")
sweep_task = asyncio.create_task(_deploy_revert_sweep_loop())
notification_task = asyncio.create_task(_notification_poll_loop())
log_ingest_task = asyncio.create_task(_log_ingest_poll_loop())
resource_task = asyncio.create_task(_resource_poll_loop())
scheduled_maintenance_task = asyncio.create_task(_scheduled_maintenance_poll_loop())
compliance_task = asyncio.create_task(_compliance_poll_loop())
audit_export_task = asyncio.create_task(_audit_export_poll_loop())
backup_task = asyncio.create_task(_backup_poll_loop())
rule_observe_task = asyncio.create_task(_rule_observe_loop())
# Only the process holding the leader lock runs these (app/leader.py); a second backend on the same database waits.
leader.gate = leader.LeaderGate({
"revert-sweep": _deploy_revert_sweep_loop,
"notifications": _notification_poll_loop,
"log-ingest": _log_ingest_poll_loop,
"resources": _resource_poll_loop,
"scheduled-maintenance": _scheduled_maintenance_poll_loop,
"compliance": _compliance_poll_loop,
"audit-export": _audit_export_poll_loop,
"backups": _backup_poll_loop,
"rule-observe": _rule_observe_loop,
})
gate_task = asyncio.create_task(leader.gate.run(), name="leader-gate")
settings_task = asyncio.create_task(runtime_settings.refresh_loop(), name="runtime-settings")
try:
yield
finally:
sweep_task.cancel()
notification_task.cancel()
log_ingest_task.cancel()
resource_task.cancel()
scheduled_maintenance_task.cancel()
compliance_task.cancel()
audit_export_task.cancel()
backup_task.cancel()
rule_observe_task.cancel()
for task in (sweep_task, notification_task, log_ingest_task, resource_task, scheduled_maintenance_task, compliance_task, audit_export_task, backup_task, rule_observe_task):
try:
await task
except asyncio.CancelledError:
pass
settings_task.cancel()
gate_task.cancel()
try:
await gate_task
except asyncio.CancelledError:
pass
await leader.gate.stop()
API_DESCRIPTION = (
@@ -517,7 +530,9 @@ app.include_router(port_forward.router, prefix="/api")
@app.get("/api/health")
async def health():
return {"status": "ok"}
# role: leader (runs the background loops), standby/takeover (another process does, or about to start),
# replica (database in recovery), starting. The HTTP status stays 200 in every role: the API works in all of them.
return {"status": "ok", "role": leader.current_role()}
app.include_router(one_to_one.router, prefix="/api")
+6 -1
View File
@@ -41,6 +41,7 @@ from sqlalchemy.engine import make_url
from sqlalchemy.ext.asyncio import AsyncConnection
import app.models
from app import runtime_settings
from app import legacy_schema_patches
from app.config import settings
from app.database import Base, engine
@@ -146,7 +147,7 @@ async def _ensure_pg_client_matches_server() -> None:
def _prune_backups(directory: Path) -> None:
dumps = sorted(directory.glob(f"{_BACKUP_PREFIX}*.dump"), key=lambda p: p.stat().st_mtime)
for old in dumps[: max(0, len(dumps) - settings.migration_backup_keep)]:
for old in dumps[: max(0, len(dumps) - runtime_settings.value("migration_backup_keep"))]:
old.unlink(missing_ok=True)
@@ -208,6 +209,10 @@ async def _migrate() -> None:
logger.info("Datenbankschema ist aktuell (Revision %s)", head)
return
try: # the kept-backups count may be set in the web interface; an old schema without the table keeps the default
await runtime_settings.refresh()
except Exception:
pass
await _backup_before_migration(current, head)
logger.info("Migriere Datenbankschema %s → %s", current, head)
async with engine.begin() as conn:
+2
View File
@@ -113,3 +113,5 @@ __all__ += ["RuleCleanupOperation", "RuleCleanupItem"]
# app.reset_2fa) otherwise fails at the first query because OpnsenseHost refers to classes nobody imported.
from app.models.host_variable import HostVariable
__all__ += ["HostVariable"]
from app.models.runtime_setting import RuntimeSetting
__all__ += ["RuntimeSetting"]
+3
View File
@@ -78,6 +78,9 @@ class NotificationSettings(Base):
# OPNexus threshold on the measured values (0 = off).
gateway_loss_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=20, server_default="20")
gateway_delay_threshold_ms: Mapped[int] = mapped_column(Integer, nullable=False, default=500, server_default="500")
# The disk OPNexus itself runs on (1.34): a full disk stops the database, and with it every alert. On by default.
server_disk_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, server_default="true")
server_disk_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=90, server_default="90")
# Resources (0 = that threshold is off). Load is per CPU thread.
disk_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=90, server_default="90")
memory_threshold_pct: Mapped[int] = mapped_column(Integer, nullable=False, default=95, server_default="95")
+18
View File
@@ -0,0 +1,18 @@
"""Values saved in Settings › Betrieb (app/runtime_settings.py); one row per changed setting."""
from datetime import datetime
from sqlalchemy import JSON, DateTime, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class RuntimeSetting(Base):
__tablename__ = "runtime_settings"
key: Mapped[str] = mapped_column(String(100), primary_key=True)
# Number or text in the unit of the spec; NULL for secrets (they live in the notification secret store).
value: Mapped[object | None] = mapped_column(JSON, nullable=True)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now(),
onupdate=func.now())
updated_by: Mapped[str | None] = mapped_column(String(150), nullable=True)
+6 -1
View File
@@ -10,7 +10,7 @@ from app.models.user import AdminUser
from app.routers.auth import current_user
from app.schemas.audit_export import AuditExportRead, AuditExportUpdate
from app.services import audit_export_service as svc
from app.services import notification_secret_store
from app.services import notification_secret_store, url_guard
router = APIRouter(prefix="/settings/audit-export", tags=["audit-export"])
@@ -29,6 +29,11 @@ async def read(db: AsyncSession = Depends(get_db), user: AdminUser = Depends(cur
@router.put("", response_model=AuditExportRead)
async def update(payload: AuditExportUpdate, db: AsyncSession = Depends(get_db), user: AdminUser = Depends(current_user)):
_require_admin(user)
if payload.webhook_url:
try:
payload.webhook_url = url_guard.validate_url(payload.webhook_url)
except ValueError as exc:
raise api_error(422, ErrorCode.NOTIFICATION_WEBHOOK_URL_INVALID, reason=str(exc)) from None
row = await svc.get_settings(db)
was_active = svc.active(row)
for field in ("syslog_enabled", "syslog_host", "syslog_port", "syslog_protocol", "webhook_enabled", "webhook_url"):
+2 -1
View File
@@ -5,6 +5,7 @@ from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.errors import ErrorCode, api_error
from app.config import settings
from app.database import get_db
@@ -58,7 +59,7 @@ def _set_session_cookie(response: Response, token: str, max_age: int | None = No
response.set_cookie(
key=session_cookie_name(),
value=token,
max_age=max_age or settings.session_lifetime,
max_age=max_age or runtime_settings.value("session_lifetime_days"),
httponly=True,
samesite="lax",
secure=settings.cookie_secure,
+49 -6
View File
@@ -1,13 +1,16 @@
from typing import Literal
from typing import Any, Literal
from fastapi import APIRouter, Depends
from pydantic import BaseModel
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.dialects.postgresql import insert
from app import runtime_settings
from app.config import settings
from app.database import get_db
from app.models.preferences import UserPreferences
from app.models.user import AdminUser
from app.routers.auth import current_user
from app.errors import ErrorCode, api_error
from app.services import audit_service
router=APIRouter(prefix='/settings',tags=['settings'])
@@ -22,8 +25,8 @@ async def read(user: AdminUser=Depends(current_user),db:AsyncSession=Depends(get
return {'username':user.username,'theme':preferences.theme if preferences else 'system',
'language':preferences.language if preferences else 'de',
'setup_done':preferences.setup_done if preferences else False,
'deploy_confirm_window_seconds':settings.deploy_confirm_window_seconds,
'session_lifetime_seconds':settings.session_lifetime}
'deploy_confirm_window_seconds':runtime_settings.value("deploy_confirm_window_seconds"),
'session_lifetime_seconds':runtime_settings.value("session_lifetime_days")}
@router.put('')
async def update(payload:Preferences,user:AdminUser=Depends(current_user),db:AsyncSession=Depends(get_db)):
@@ -35,5 +38,45 @@ async def update(payload:Preferences,user:AdminUser=Depends(current_user),db:Asy
await db.execute(insert(UserPreferences).values(user_id=user.id,theme=theme,language=language,setup_done=setup_done).on_conflict_do_update(index_elements=['user_id'],set_={'theme':theme,'language':language,'setup_done':setup_done}))
await db.commit()
return {'username':username,'theme':theme,'language':language,'setup_done':setup_done,
'deploy_confirm_window_seconds':settings.deploy_confirm_window_seconds,
'session_lifetime_seconds':settings.session_lifetime}
'deploy_confirm_window_seconds':runtime_settings.value("deploy_confirm_window_seconds"),
'session_lifetime_seconds':runtime_settings.value("session_lifetime_days")}
# ── Settings › Betrieb: values that used to need the .env (app/runtime_settings.py) ──────────────────────────
class SystemSettingsUpdate(BaseModel):
# key -> new value; null (or "") resets the setting to its default.
values: dict[str, Any] = Field(max_length=50)
@router.get('/system')
async def read_system(user: AdminUser = Depends(current_user)):
return {'settings': runtime_settings.describe()}
@router.put('/system')
async def update_system(payload: SystemSettingsUpdate, user: AdminUser = Depends(current_user),
db: AsyncSession = Depends(get_db)):
if user.role != 'admin':
raise api_error(403, ErrorCode.ADMIN_REQUIRED)
before = {item['key']: item for item in runtime_settings.describe()}
changes = {}
for key, new in payload.values.items():
spec = runtime_settings.BY_KEY.get(key)
if spec is None:
continue # rejected by save() below with a proper error
if spec.kind in ('secret_url', 'pem'): # never put the secret (or a whole certificate) into the audit trail
changes[key] = {'old': 'gesetzt' if before[key].get('is_set') else 'leer', 'new': 'leer' if not new else 'gesetzt'}
else:
changes[key] = {'old': before[key]['value'], 'new': new if new not in ('', None) else before[key]['default']}
try:
if changes:
db.add(audit_service.make_event(
kind='settings', action='system_settings', status='accepted', entity_id='system',
message='Betriebseinstellungen geändert', details={'changes': changes}))
await runtime_settings.save(db, payload.values, user.username)
except runtime_settings.SettingError as exc:
await db.rollback()
raise api_error(422, ErrorCode.SETTING_INVALID, setting=exc.key, reason=exc.reason) from None
return {'settings': runtime_settings.describe()}
+259
View File
@@ -0,0 +1,259 @@
"""Settings an admin changes in the web interface instead of the .env (Settings › Betrieb).
One registry for every tunable that is safe to change while OPNexus runs. The value in effect is, in this order:
1. the environment variable, if the spec names one and it is set to a non-empty value ("vom Server festgelegt": a
scripted installation stays in control, the UI shows the value read-only);
2. the value saved in the web interface (table ``runtime_settings``; secrets in the notification secret store);
3. the built-in default (the attribute of ``app.config.settings``).
Reads are synchronous (``value(key)``) from a cache that every backend process refreshes every few seconds
(``refresh_loop``), so a leader and a standby agree within that time and hot paths (login, every firewall call) do
not touch the database. A save refreshes the cache of the saving process at once.
What deliberately stays in the .env: addresses and TLS of OPNexus itself (a wrong value there locks everyone out of
the very interface that would fix it), the database credentials, the first admin account, the container switches
(Compose profiles) and the escape hatches (ALLOW_INSECURE_*, OPNEXUS_RUN_AS_ROOT).
"""
import asyncio
import logging
import os
import ssl
from dataclasses import dataclass
from typing import Any
from sqlalchemy import delete, select
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
from app.database import SessionLocal
from app.models.runtime_setting import RuntimeSetting
from app.services import notification_secret_store, url_guard
logger = logging.getLogger(__name__)
REFRESH_SECONDS = 15
_MAX_PEM = 65536
@dataclass(frozen=True)
class Spec:
key: str
group: str # security | operation | monitoring | integrations (UI sections)
kind: str # int | float | secret_url | pem
attr: str # attribute of app.config.settings holding the default (and the env value, via pydantic)
unit: str = "" # shown next to the field: seconds | minutes | hours | days | count
minimum: float | None = None
maximum: float | None = None
scale: int = 1 # stored/shown in `unit`; the code reads value * scale (days -> seconds)
env: str | None = None
SPECS: list[Spec] = [
# Security
Spec("session_lifetime_days", "security", "int", "session_lifetime", "days", 1, 365, scale=86400),
Spec("login_max_attempts", "security", "int", "login_max_attempts", "count", 3, 50),
Spec("login_ip_max_attempts", "security", "int", "login_ip_max_attempts", "count", 5, 500),
Spec("login_window_seconds", "security", "int", "login_window_seconds", "seconds", 60, 3600),
Spec("login_lockout_seconds", "security", "int", "login_lockout_seconds", "seconds", 60, 86400),
# Operation
Spec("deploy_confirm_window_seconds", "operation", "int", "deploy_confirm_window_seconds", "seconds", 60, 1800),
Spec("opnsense_timeout", "operation", "float", "opnsense_timeout", "seconds", 5, 120),
Spec("migration_backup_keep", "operation", "int", "migration_backup_keep", "count", 1, 50),
# Monitoring
Spec("resource_poll_interval_seconds", "monitoring", "int", "resource_poll_interval_seconds", "seconds", 30, 3600),
Spec("resource_sample_retention_days", "monitoring", "int", "resource_sample_retention_days", "days", 1, 365),
Spec("rule_observe_interval_seconds", "monitoring", "int", "rule_observe_interval_seconds", "seconds", 60, 86400),
# No env lock: Compose passes UPDATE_CHECK_INTERVAL_HOURS=12 when unset (it must: the backend before 1.34 cannot
# parse an empty value, and a rollback runs that backend with the new compose file), so the variable cannot tell
# "set by the admin" from "default". It stays the default; the web interface may override it.
Spec("update_check_interval_hours", "monitoring", "int", "update_check_interval_hours", "hours", 1, 168),
# Integrations
Spec("heartbeat_url", "integrations", "secret_url", "heartbeat_url", env="HEARTBEAT_URL"),
Spec("smtp_ca_pem", "integrations", "pem", "smtp_ca_file", env="SMTP_CA_FILE"),
Spec("notification_smtp_timeout_seconds", "integrations", "float", "notification_smtp_timeout_seconds", "seconds",
5, 120),
]
BY_KEY = {s.key: s for s in SPECS}
# Saved values (numbers as stored, in the spec's unit; secrets and PEM as text). Missing key = not saved.
_cache: dict[str, Any] = {}
class SettingError(ValueError):
def __init__(self, key: str, reason: str):
super().__init__(reason)
self.key = key
self.reason = reason
def _secret_name(key: str) -> str:
return f"runtime-{key}"
def from_server(spec: Spec) -> bool:
if spec.env and os.environ.get(spec.env, "").strip():
return True
# Text settings default to "": a non-empty value can only come from the server (env, parsed by pydantic).
return spec.kind in ("secret_url", "pem") and bool(str(getattr(settings, spec.attr) or "").strip())
def _default(spec: Spec) -> Any:
raw = getattr(settings, spec.attr)
if spec.kind in ("int", "float") and spec.scale != 1:
return type(raw)(raw / spec.scale) if spec.kind == "float" else int(raw // spec.scale)
return raw
def shown(spec: Spec) -> tuple[Any, str]:
"""(value in the spec's unit, source): source is server | ui | default."""
if from_server(spec):
return _default(spec), "server"
if spec.key in _cache:
return _cache[spec.key], "ui"
return _default(spec), "default"
def value(key: str) -> Any:
"""The value the code uses (numbers multiplied by the spec's scale)."""
spec = BY_KEY[key]
if from_server(spec) or spec.key not in _cache:
return getattr(settings, spec.attr) # tests monkeypatch settings; the env is already parsed into it
raw = _cache[spec.key]
if spec.kind == "int":
return int(raw) * spec.scale
if spec.kind == "float":
return float(raw) * spec.scale
return raw
# ── validation ───────────────────────────────────────────────────────────
def validate(spec: Spec, raw: Any) -> Any:
if spec.kind in ("int", "float"):
if isinstance(raw, bool) or not isinstance(raw, (int, float)):
raise SettingError(spec.key, "Bitte eine Zahl angeben")
if spec.kind == "int" and int(raw) != raw:
raise SettingError(spec.key, "Bitte eine ganze Zahl angeben")
number = int(raw) if spec.kind == "int" else float(raw)
if (spec.minimum is not None and number < spec.minimum) or (spec.maximum is not None and number > spec.maximum):
raise SettingError(spec.key, f"Erlaubt ist {spec.minimum:g} bis {spec.maximum:g}")
return number
if not isinstance(raw, str):
raise SettingError(spec.key, "Bitte Text angeben")
text = raw.strip()
if spec.kind == "secret_url":
try:
url = url_guard.validate_url(text, max_length=2000)
except ValueError as exc:
raise SettingError(spec.key, str(exc)) from None
if not url.startswith("https://"):
raise SettingError(spec.key, "Die Adresse muss mit https:// beginnen")
return url
if spec.kind == "pem":
if len(text) > _MAX_PEM or "-----BEGIN CERTIFICATE-----" not in text:
raise SettingError(spec.key, "Bitte ein CA-Zertifikat im PEM-Format einfügen")
try:
ssl.create_default_context(cadata=text)
except (ssl.SSLError, ValueError):
raise SettingError(spec.key, "Das Zertifikat ist nicht lesbar") from None
return text
raise SettingError(spec.key, "Unbekannte Einstellung")
# ── storage ──────────────────────────────────────────────────────────────
async def load(db: AsyncSession) -> dict[str, Any]:
rows = (await db.scalars(select(RuntimeSetting))).all()
fresh: dict[str, Any] = {}
for row in rows:
spec = BY_KEY.get(row.key)
if spec is None:
continue # a setting a newer version knew about; keep the row, ignore it
if spec.kind == "secret_url":
secret = notification_secret_store.read_secret(_secret_name(spec.key))
if secret:
fresh[spec.key] = secret
elif row.value is not None:
fresh[spec.key] = row.value
return fresh
async def refresh(db: AsyncSession | None = None) -> None:
global _cache
if db is not None:
_cache = await load(db)
return
async with SessionLocal() as session:
_cache = await load(session)
async def refresh_loop() -> None:
while True:
await asyncio.sleep(REFRESH_SECONDS)
try:
await refresh()
except Exception as exc:
logger.warning("Einstellungen konnten nicht neu gelesen werden (%s)", type(exc).__name__)
async def save(db: AsyncSession, values: dict[str, Any], actor: str) -> list[str]:
"""Validates everything first, then writes; None resets a setting to its default. Returns the changed keys."""
clean: dict[str, Any] = {}
for key, raw in values.items():
spec = BY_KEY.get(key)
if spec is None:
raise SettingError(key, "Unbekannte Einstellung")
if from_server(spec):
raise SettingError(key, f"Wird vom Server festgelegt ({spec.env} in der .env)")
clean[key] = None if raw is None or (isinstance(raw, str) and not raw.strip()) else validate(spec, raw)
for key, val in clean.items():
spec = BY_KEY[key]
if val is None:
await db.execute(delete(RuntimeSetting).where(RuntimeSetting.key == key))
if spec.kind == "secret_url":
notification_secret_store.delete_secret(_secret_name(key))
continue
stored = val
if spec.kind == "secret_url":
notification_secret_store.save_secret(_secret_name(key), val)
stored = None # the database only knows that it is set
await db.execute(
insert(RuntimeSetting).values(key=key, value=stored, updated_by=actor)
.on_conflict_do_update(index_elements=["key"], set_={"value": stored, "updated_by": actor})
)
await db.commit()
await refresh(db)
return sorted(clean)
def describe() -> list[dict]:
out = []
for spec in SPECS:
val, source = shown(spec)
item = {"key": spec.key, "group": spec.group, "kind": spec.kind, "unit": spec.unit,
"min": spec.minimum, "max": spec.maximum, "default": None, "source": source,
"env": spec.env if source == "server" else None}
if spec.kind == "secret_url":
item["value"] = None
item["is_set"] = bool(val)
elif spec.kind == "pem":
item["value"] = val if source != "server" else None # an env value is a path, not the certificate
item["is_set"] = bool(val)
else:
item["value"] = val
item["default"] = _default(spec)
out.append(item)
return out
# ── consumers ────────────────────────────────────────────────────────────
def smtp_ssl_context() -> ssl.SSLContext:
"""Trust for the mail server (and the TLS syslog of the audit export): the system store plus the private CA from
SMTP_CA_FILE or the certificate pasted in the web interface."""
spec = BY_KEY["smtp_ca_pem"]
if from_server(spec):
return ssl.create_default_context(cafile=settings.smtp_ca_file)
pem = _cache.get(spec.key)
return ssl.create_default_context(cadata=pem) if pem else ssl.create_default_context()
+5 -1
View File
@@ -16,7 +16,7 @@ ProblemType = Literal[
"backup_stale",
"backup_offsite_failed",
"compliance_finding",
"gateway_down", "resource_threshold", "service_down", "api_rights_missing",
"gateway_down", "resource_threshold", "service_down", "api_rights_missing", "server_disk",
"rollout_paused", "rollout_start_failed",
]
ChannelType = Literal["ntfy", "gotify", "telegram", "teams", "matrix"]
@@ -55,6 +55,8 @@ class NotificationSettingsRead(BaseModel):
service_down_enabled: bool = False
rollout_problems_enabled: bool = True
api_rights_enabled: bool = True
server_disk_enabled: bool = True
server_disk_threshold_pct: int = 90
gateway_alert_muted: list[str] = Field(default_factory=list)
gateway_loss_threshold_pct: int = 20
gateway_delay_threshold_ms: int = 500
@@ -114,6 +116,8 @@ class NotificationSettingsUpdate(BaseModel):
service_down_enabled: bool | None = None
rollout_problems_enabled: bool | None = None
api_rights_enabled: bool | None = None
server_disk_enabled: bool | None = None
server_disk_threshold_pct: int | None = Field(default=None, ge=50, le=99)
# 0 switches that single threshold off.
gateway_loss_threshold_pct: int | None = Field(default=None, ge=0, le=100)
gateway_delay_threshold_ms: int | None = Field(default=None, ge=0, le=60000)
+38 -15
View File
@@ -23,10 +23,11 @@ import httpx
from sqlalchemy import and_, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.config import settings
from app.models.audit import AuditEvent
from app.models.audit_export import AuditExportSettings
from app.services import notification_secret_store
from app.services import notification_secret_store, url_guard
from app.version import APP_VERSION
logger = logging.getLogger(__name__)
@@ -75,17 +76,28 @@ def _frame_tcp(lines: list[bytes]) -> bytes:
return b"".join(str(len(line)).encode() + b" " + line for line in lines)
def _send_syslog_sync(row: AuditExportSettings, lines: list[bytes]) -> None:
def _send_syslog_sync(row: AuditExportSettings, lines: list[bytes], addresses: list[str]) -> None:
"""``addresses`` are the already checked addresses of ``syslog_host``: connect to those, never to the name."""
host, port = row.syslog_host, row.syslog_port
if row.syslog_protocol == "udp":
with socket.socket(socket.AF_INET6 if ":" in (host or "") else socket.AF_INET, socket.SOCK_DGRAM) as sock:
address = addresses[0]
with socket.socket(socket.AF_INET6 if ":" in address else socket.AF_INET, socket.SOCK_DGRAM) as sock:
for line in lines:
sock.sendto(line[:8000], (host, port))
sock.sendto(line[:8000], (address, port))
return
with socket.create_connection((host, port), timeout=settings.opnsense_timeout) as raw:
last: OSError | None = None
for address in addresses:
try:
raw = socket.create_connection((address, port), timeout=runtime_settings.value("opnsense_timeout"))
break
except OSError as exc:
last = exc
else:
raise last or OSError("keine Adresse")
with raw:
if row.syslog_protocol == "tls":
context = ssl.create_default_context(cafile=settings.smtp_ca_file or None)
with context.wrap_socket(raw, server_hostname=host) as tls:
context = runtime_settings.smtp_ssl_context()
with context.wrap_socket(raw, server_hostname=host) as tls: # Zertifikat gegen den Namen, nicht die IP
tls.sendall(_frame_tcp(lines))
else:
raw.sendall(_frame_tcp(lines))
@@ -95,7 +107,13 @@ async def send_syslog(row: AuditExportSettings, events: list[dict]) -> None:
if not row.syslog_host:
raise ExportError("Kein Syslog-Ziel konfiguriert")
try:
await asyncio.to_thread(_send_syslog_sync, row, [syslog_line(e) for e in events])
addresses = await url_guard.checked_addresses(row.syslog_host, row.syslog_port)
except ValueError as exc:
raise ExportError(f"Syslog-Ziel nicht erlaubt: {exc}") from None
except OSError as exc:
raise ExportError(f"Syslog-Zustellung fehlgeschlagen ({type(exc).__name__})") from None
try:
await asyncio.to_thread(_send_syslog_sync, row, [syslog_line(e) for e in events], addresses)
except (OSError, ssl.SSLError) as exc:
raise ExportError(f"Syslog-Zustellung fehlgeschlagen ({type(exc).__name__})") from None
@@ -110,13 +128,18 @@ async def send_webhook(row: AuditExportSettings, events: list[dict]) -> None:
if not secret:
raise ExportError("Signaturschlüssel ist konfiguriert, aber nicht lesbar – bitte neu setzen")
headers["X-OPNexus-Signature"] = "sha256=" + hmac.new(secret.encode(), raw, hashlib.sha256).hexdigest()
try:
async with httpx.AsyncClient(timeout=settings.opnsense_timeout) as client:
(await client.post(row.webhook_url, content=raw, headers=headers)).raise_for_status()
except httpx.HTTPError as exc:
# The URL often is the credential: never put it in a message.
reason = f"HTTP {exc.response.status_code}" if isinstance(exc, httpx.HTTPStatusError) else type(exc).__name__
raise ExportError(f"Webhook-Zustellung fehlgeschlagen ({reason})") from None
# Die URL ist oft selbst das Zugangsdatum: das Anfrage-Log von httpx bleibt währenddessen stumm.
with url_guard.muted():
try:
url_guard.validate_url(row.webhook_url)
await url_guard.ensure_resolved_safe(row.webhook_url)
except ValueError as exc:
raise ExportError(f"Webhook-Ziel nicht erlaubt: {exc}") from None
try:
async with url_guard.guarded_client(timeout=runtime_settings.value("opnsense_timeout"), follow_redirects=False) as client:
(await client.post(row.webhook_url, content=raw, headers=headers)).raise_for_status()
except httpx.HTTPError as exc:
raise ExportError(f"Webhook-Zustellung fehlgeschlagen ({url_guard.failure_reason(exc)})") from None
def active(row: AuditExportSettings) -> bool:
+15 -8
View File
@@ -30,9 +30,10 @@ from datetime import datetime, timedelta, timezone
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerifyMismatchError
from sqlalchemy import delete, select
from sqlalchemy import delete, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.config import settings
from app.errors import ErrorCode, api_error
from app.models.user import AdminUser, Session
@@ -47,6 +48,9 @@ _DUMMY_HASH = _ph.hash(secrets.token_urlsafe(32))
SESSION_TOKEN_BYTES = 32
# Serializes the first-start account creation of several backends (policy_lock and leader.py use 728194031-33).
_BOOTSTRAP_LOCK_KEY = 728194034
# ── Password hashing ─────────────────────────────────────────────────────
@@ -158,7 +162,7 @@ class _LoginThrottle:
if not force and now - self._last_purge < self._PURGE_EVERY_SECONDS and len(self._records) < self.MAX_RECORDS:
return
self._last_purge = now
window = settings.login_window_seconds
window = runtime_settings.value("login_window_seconds")
for key in [
k for k, r in self._records.items()
if r.locked_until <= now and all(now - t >= window for t in r.failures)
@@ -179,11 +183,11 @@ class _LoginThrottle:
def _record_key(self, key: str, max_attempts: int) -> None:
now = time.monotonic()
rec = self._records.setdefault(key, _AttemptRecord())
window = settings.login_window_seconds
window = runtime_settings.value("login_window_seconds")
rec.failures = [t for t in rec.failures if now - t < window]
rec.failures.append(now)
if len(rec.failures) >= max_attempts:
rec.locked_until = now + settings.login_lockout_seconds
rec.locked_until = now + runtime_settings.value("login_lockout_seconds")
rec.failures.clear()
def retry_after(self, ip: str, username: str) -> int:
@@ -196,8 +200,8 @@ class _LoginThrottle:
def record_failure(self, ip: str, username: str) -> None:
self._purge(time.monotonic())
self._record_key(self._user_key(ip, username), settings.login_max_attempts)
self._record_key(self._ip_key(ip), settings.login_ip_max_attempts)
self._record_key(self._user_key(ip, username), runtime_settings.value("login_max_attempts"))
self._record_key(self._ip_key(ip), runtime_settings.value("login_ip_max_attempts"))
key = username.lower()
count, _ = self._failures_since_success.get(key, (0, 0.0))
self._failures_since_success[key] = (count + 1, time.monotonic())
@@ -262,7 +266,7 @@ async def create_session(
await db.rollback()
raise api_error(401, ErrorCode.TOTP_REQUIRED)
token = secrets.token_urlsafe(SESSION_TOKEN_BYTES)
expires_at = _now() + timedelta(seconds=lifetime_seconds or settings.session_lifetime)
expires_at = _now() + timedelta(seconds=lifetime_seconds or runtime_settings.value("session_lifetime_days"))
session = Session(
user_id=user_id,
token_hash=_hash_token(token),
@@ -352,9 +356,12 @@ async def finalize_login(db: AsyncSession, user: AdminUser, ip: str) -> None:
# ── Bootstrap ─────────────────────────────────────────────────────────
async def bootstrap_admin(db: AsyncSession) -> None:
"""Ensure the admin account exists."""
"""Ensure the admin account exists. Two backends starting on an empty database at once (HA, a second stack) are
serialized here; the second one then sees the account of the first."""
await db.execute(text("SELECT pg_advisory_xact_lock(:key)"), {"key": _BOOTSTRAP_LOCK_KEY})
existing = await db.execute(select(AdminUser.id).limit(1))
if existing.first() is not None:
await db.commit() # releases the lock
return
password = settings.admin_password.strip()
+2 -1
View File
@@ -32,6 +32,7 @@ from typing import Any, Awaitable, Callable
from sqlalchemy import and_, or_
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.config import settings
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -80,7 +81,7 @@ async def push_to_host(
write: Callable[[str | None], Awaitable[str]],
apply: Callable[[], Awaitable[Any]],
) -> Any:
window = timedelta(seconds=settings.deploy_confirm_window_seconds)
window = timedelta(seconds=runtime_settings.value("deploy_confirm_window_seconds"))
before = {name: getattr(existing, name) for name in _FIELDS} if existing else None
# An armed "error" row (written, apply not confirmed) holds the UNCONFIRMED payload as last_payload: a retry must
# keep the original rollback target instead of promoting that payload to it.
+61
View File
@@ -0,0 +1,61 @@
"""Heartbeat to an external watcher: who watches the watcher (docs/HA-PLAN.md, 2.7).
If OPNexus stops, every alert it would send stops with it. With HEARTBEAT_URL set, the leader's notification loop
calls that URL after each successful check cycle; a watcher outside OPNexus (healthchecks.io style, ntfy, Uptime
Kuma push monitor) raises the alarm when the calls stop. Only the leader sends (the loop runs only there), and only
after a cycle that reached the database and finished its checks: a backend that is up but cut off from PostgreSQL
must not report "fine".
The URL usually carries a secret (the check id), so it is validated like a notification target, called through the
guarded client without redirects and never written to the log.
"""
import logging
from urllib.parse import urlsplit
import httpx
from app import runtime_settings
from app.config import settings
from app.services import url_guard
from app.version import APP_VERSION
logger = logging.getLogger(__name__)
_TIMEOUT_SECONDS = 10.0
_warned = False
def configured_url() -> str | None:
"""The validated URL, or None (off or invalid; an invalid one is reported once)."""
global _warned
raw = runtime_settings.value("heartbeat_url").strip()
if not raw:
return None
try:
url = url_guard.validate_url(raw, max_length=2000)
if urlsplit(url).scheme != "https":
raise ValueError("Die Adresse muss mit https:// beginnen")
except ValueError as exc:
if not _warned:
logger.warning("HEARTBEAT_URL wird ignoriert: %s", exc)
_warned = True
return None
return url
async def ping() -> bool:
url = configured_url()
if url is None:
return False
with url_guard.muted():
try:
await url_guard.ensure_resolved_safe(url)
async with url_guard.guarded_client(timeout=_TIMEOUT_SECONDS, follow_redirects=False) as client:
(await client.get(url, headers={"User-Agent": f"OPNexus/{APP_VERSION}"})).raise_for_status()
except ValueError as exc:
logger.warning("Heartbeat nicht gesendet: %s", exc)
return False
except httpx.HTTPError as exc:
logger.warning("Heartbeat fehlgeschlagen (%s)", url_guard.failure_reason(exc))
return False
return True
+8 -1
View File
@@ -16,6 +16,7 @@ from datetime import datetime, timedelta, timezone
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app import leader
from app.models.carp_event import CarpEvent
from app.models.compliance_result import ComplianceResult
from app.models.host import OpnsenseHost
@@ -24,7 +25,7 @@ from app.models.resource_sample import HostResourceSample
from app.models.rollout import RolloutRun
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.routers.dashboard import collect_pending_actions
from app.services import compliance_service, gateway_health_service, notification_settings_service
from app.services import compliance_service, gateway_health_service, notification_checks, notification_settings_service
from app.version import APP_VERSION
CONTENT_TYPE = "text/plain; version=0.0.4; charset=utf-8"
@@ -59,6 +60,12 @@ async def render(db: AsyncSession) -> str:
out.family("opnexus_info", "OPNexus version (always 1).")
out.sample("opnexus_info", 1, version=APP_VERSION)
out.family("opnexus_leader", "1 if this backend process runs the background loops (leader lock held), else 0.")
out.sample("opnexus_leader", 1 if leader.current_role() == leader.LEADER else 0, role=leader.current_role())
disk = notification_checks.server_disk_usage()
if disk is not None:
out.family("opnexus_server_disk_used_percent", "Used share of the disk holding OPNexus' Docker volumes (this backend's view).")
out.sample("opnexus_server_disk_used_percent", round(disk[0], 1))
hosts = (await db.scalars(select(OpnsenseHost).order_by(OpnsenseHost.name))).all()
latest_sample = (
+2
View File
@@ -142,6 +142,8 @@ async def check_gateway_down(db: AsyncSession) -> dict[str, dict]:
# page-status-services: api/core/service/*; page-system-login-logout "Lobby: Dashboard": api/diagnostics/cpu_usage/*
# and api/diagnostics/system/system_swap).
API_READS: dict[str, tuple[str, str]] = {
# Die Basis-Messungen selbst (Speicher, Last, Platte, Schnittstellen): ohne sie gibt es keinen einzigen Messwert.
"base": ("Lobby: Dashboard", "/api/diagnostics/system/systemResources"),
"gateways": ("System: Gateways", "/api/routes/gateway/status"),
"services": ("Status: Services", "/api/core/service/search"),
"swap": ("Lobby: Dashboard", "/api/diagnostics/system/systemSwap"),
@@ -30,6 +30,7 @@ import httpx
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.config import settings
from app.models.notification import CHANNEL_TYPES, NotificationChannel, NotificationProblem
from app.services import notification_delivery, notification_secret_store, url_guard
@@ -224,7 +225,7 @@ async def send(channel: NotificationChannel, problem: NotificationProblem, trans
except ValueError as exc:
raise NotificationDeliveryError(f"{label}: {exc}") from None
try:
async with url_guard.guarded_client(timeout=settings.opnsense_timeout, follow_redirects=False) as client:
async with url_guard.guarded_client(timeout=runtime_settings.value("opnsense_timeout"), follow_redirects=False) as client:
resp = await client.request(method, url, json=payload, headers=headers)
resp.raise_for_status()
if ok_key and not (resp.json() or {}).get(ok_key):
@@ -2,6 +2,7 @@
whatever's currently bad. Pure detection — no delivery, no state tracking
here (see notification_dedup.py for the open/resolved diffing that turns
these snapshots into actual notifications)."""
import shutil
from collections import defaultdict
from datetime import datetime, timedelta, timezone
@@ -18,6 +19,7 @@ from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.models.port_forward_deployment import PortForwardDeployment
from app.models.rule_deployment import RuleDeployment
from app.models.scheduled_maintenance import ScheduledMaintenance
from app.config import settings
from app.services import backup_store, nat_drift_service, network_drift_service, rule_drift_service
from app.services.notification_dedup import Problems
from app.services.host_status_service import get_statuses
@@ -465,3 +467,36 @@ async def check_rollout_start_failed(db: AsyncSession) -> dict[str, dict]:
"details": _short(error.get("params")),
}
return problems
def server_disk_usage() -> tuple[float, int, int] | None:
"""(used %, used bytes, total bytes) of the file system that holds OPNexus' Docker volumes, measured on the
firewall-backup volume the backend mounts (in a standard installation the database lives on the same disk)."""
try:
usage = shutil.disk_usage(settings.opnsense_backup_dir)
except OSError:
return None
if not usage.total:
return None
return usage.used * 100.0 / usage.total, usage.used, usage.total
async def check_server_disk(db: AsyncSession) -> dict[str, dict]:
"""The disk OPNexus itself runs on. A full disk stops PostgreSQL and with it every other alert, so this one
warns while there is still room. Only the leader checks (the loop runs only there)."""
from app.services import notification_settings_service
cfg = await notification_settings_service.get_settings(db)
measured = server_disk_usage()
if measured is None:
unknown = Problems()
unknown.preserve_prefixes.add("server") # not measurable this round: leave an open problem as it is
return unknown
pct, used, total = measured
if pct < cfg.server_disk_threshold_pct:
return {}
return {"server": {
"belegt_prozent": round(pct, 1), "schwelle_prozent": cfg.server_disk_threshold_pct,
"frei_gb": round((total - used) / 1e9, 1), "gesamt_gb": round(total / 1e9, 1),
"hinweis": "Platz schaffen (z. B. alte Images mit 'docker image prune', Log-Obergrenze unter Einstellungen › Logs senken) "
"oder die Platte vergrößern. Läuft sie voll, bleibt die Datenbank stehen.",
}}
@@ -14,6 +14,7 @@ from email.message import EmailMessage
import httpx
from app import runtime_settings
from app.config import settings
from app.models.notification import NotificationProblem, NotificationSettings
from app.services import notification_secret_store, url_guard
@@ -41,6 +42,7 @@ _LABELS = {
"resource_threshold": "Ressourcen-Schwelle überschritten",
"service_down": "Dienst ausgefallen",
"api_rights_missing": "API-Rechte fehlen",
"server_disk": "Platte des OPNexus-Servers fast voll",
"rollout_paused": "Rollout angehalten",
"rollout_start_failed": "Rollout nicht automatisch gestartet",
"test": "Testbenachrichtigung",
@@ -97,7 +99,7 @@ async def send_webhook(settings_row: NotificationSettings, problem: Notification
raise NotificationDeliveryError(f"Webhook-Adresse nicht erlaubt: {exc}") from None
try:
# follow_redirects stays off: a redirect must not move the request to an address the check above never saw.
async with url_guard.guarded_client(timeout=settings.opnsense_timeout, follow_redirects=False) as client:
async with url_guard.guarded_client(timeout=runtime_settings.value("opnsense_timeout"), follow_redirects=False) as client:
with url_guard.muted():
resp = await client.post(settings_row.webhook_url, content=raw, headers=headers)
resp.raise_for_status()
@@ -112,7 +114,7 @@ def _tls_context() -> ssl.SSLContext:
"""Verifying context (certificate chain AND hostname). smtplib's own default
when no context is passed is an *unverifying* one — the connection would be
encrypted but could be impersonated."""
return ssl.create_default_context(cafile=settings.smtp_ca_file or None)
return runtime_settings.smtp_ssl_context()
def _send_email_sync(settings_row: NotificationSettings, subject: str, body: str) -> None:
@@ -130,7 +132,7 @@ def _send_email_sync(settings_row: NotificationSettings, subject: str, body: str
message["To"] = ", ".join(settings_row.smtp_to_addresses)
message.set_content(body)
timeout = settings.notification_smtp_timeout_seconds
timeout = runtime_settings.value("notification_smtp_timeout_seconds")
if settings_row.smtp_security == "ssl":
client = smtplib.SMTP_SSL(
settings_row.smtp_host, settings_row.smtp_port, timeout=timeout, context=_tls_context()
@@ -40,6 +40,7 @@ _CHECKS = [
("resource_threshold", "resource_threshold_enabled", monitor_checks.check_resource_threshold),
("service_down", "service_down_enabled", monitor_checks.check_service_down),
("api_rights_missing", "api_rights_enabled", monitor_checks.check_api_rights_missing),
("server_disk", "server_disk_enabled", notification_checks.check_server_disk),
("rollout_paused", "rollout_problems_enabled", notification_checks.check_rollout_paused),
("rollout_start_failed", "rollout_problems_enabled", notification_checks.check_rollout_start_failed),
]
@@ -27,6 +27,7 @@ async def update_settings(db: AsyncSession, payload: NotificationSettingsUpdate)
"revert_failed_enabled", "maintenance_unknown_enabled", "carp_health_enabled", "nat_drift_enabled",
"network_drift_enabled", "carp_auto_revert_failed_enabled", "scheduled_maintenance_failed_enabled", "backup_failed_enabled", "compliance_alerts_enabled",
"gateway_down_enabled", "resource_threshold_enabled", "service_down_enabled", "rollout_problems_enabled", "api_rights_enabled",
"server_disk_enabled", "server_disk_threshold_pct",
"gateway_loss_threshold_pct", "gateway_delay_threshold_ms", "disk_threshold_pct", "memory_threshold_pct",
"swap_threshold_pct", "load_per_cpu_threshold", "alert_confirm_samples", "resource_confirm_samples",
"service_watchlist",
+2 -1
View File
@@ -18,6 +18,7 @@ from typing import Any
import httpx
from app import runtime_settings
from app.config import settings
from app.models.host import OpnsenseHost
from app.services import firewall_url, opnsense_secret_store, overlay_service, tls_pin_service
@@ -60,7 +61,7 @@ def _client(host: OpnsenseHost) -> httpx.AsyncClient:
base_url=host.base_url.rstrip("/") + "/api",
auth=(host.api_key, secret),
verify=tls_pin_service.verify_argument(host.verify_tls, host.tls_ca_pem),
timeout=settings.opnsense_timeout,
timeout=runtime_settings.value("opnsense_timeout"),
# Firewalls reached through the management overlay (WireGuard hub) go via the hub's CONNECT proxy.
proxy=overlay_service.proxy_for(host.base_url),
)
@@ -110,12 +110,22 @@ async def _optional(operation, name: str, denied: list[str], ok: list[str]):
async def sample_host(db: AsyncSession, host: OpnsenseHost) -> None:
resources = await client.get_system_resources(host)
system_time = await client.get_system_time(host)
disk = await client.get_system_disk(host)
interfaces = await client.get_interface_statistics(host)
denied: list[str] = []
ok: list[str] = []
try:
resources = await client.get_system_resources(host)
system_time = await client.get_system_time(host)
disk = await client.get_system_disk(host)
interfaces = await client.get_interface_statistics(host)
ok.append("base")
except OpnsenseForbiddenError:
# Dem API-Schlüssel fehlen schon die Basis-Rechte (Dashboard): ohne Messwerte gäbe es nie einen Messpunkt und
# der Rechte-Alarm könnte nicht auslösen. Ein leerer Punkt trägt den Befund ("base" in denied); die
# Schwellenwert-Alarme sehen darin "unbekannt", nicht "in Ordnung".
db.add(HostResourceSample(host_id=host.id, sampled_at=_now(), mem_used_pct=None, load_avg_1m=None,
disk_used_pct=None, raw={"denied": ["base"], "ok": []}))
logger.warning("Resource-Sampling für Host %s: dem API-Schlüssel fehlen die Basis-Rechte (403)", host.name)
return
swap, cpu_type, gateways, services = await asyncio.gather(
_optional(client.get_system_swap(host), "swap", denied, ok),
_optional(client.get_cpu_type(host), "cpu", denied, ok),
+4 -3
View File
@@ -89,6 +89,7 @@ from sqlalchemy.orm.attributes import set_committed_value
from sqlalchemy.ext.asyncio import AsyncSession
from fastapi import HTTPException
from app import runtime_settings
from app.config import settings
from app.errors import ErrorCode, api_error
from app.models.governance import GovernanceSettings
@@ -146,13 +147,13 @@ async def _commit(db: AsyncSession) -> None:
def _window() -> timedelta:
return timedelta(seconds=settings.deploy_confirm_window_seconds)
return timedelta(seconds=runtime_settings.value("deploy_confirm_window_seconds"))
def _budget(items: int, nodes: int) -> timedelta:
"""Upper bound for the remote calls of one run: a toggle and a read-back per item, a sample, an apply and a
read per node, each at most one client timeout."""
return timedelta(seconds=(2 * items + 3 * nodes) * settings.opnsense_timeout)
return timedelta(seconds=(2 * items + 3 * nodes) * runtime_settings.value("opnsense_timeout"))
def _truthy(value) -> bool:
@@ -805,7 +806,7 @@ async def preview(db: AsyncSession, host: OpnsenseHost, *, actor: str, actor_id=
audit_service.ADMIN_ONLY_DETAILS: {"plan": plan}})))
await db.commit()
return {"preview_id": str(op.id), "expires_at": _iso(op.expires_at), "kind": kind, "plan": plan,
"max_rules": limits.max_rules, "window_seconds": settings.deploy_confirm_window_seconds}
"max_rules": limits.max_rules, "window_seconds": runtime_settings.value("deploy_confirm_window_seconds")}
async def cancel(db: AsyncSession, host: OpnsenseHost, op_id, *, actor: str) -> dict:
+8 -2
View File
@@ -13,6 +13,7 @@ from urllib.parse import quote, urlsplit
import httpx
from app.config import settings
from app.services import url_guard
class S3Error(Exception):
@@ -58,6 +59,10 @@ class S3Client:
if parts.scheme == "http" and not settings.allow_insecure_offsite:
# Credentials and (encrypted, but still) backups must not travel in the clear.
raise S3Error("Der Endpunkt muss https:// sein")
try:
url_guard.validate_url(endpoint)
except ValueError as exc: # z. B. localhost, Link-Local, Metadaten-Dienst
raise S3Error(f"Ungültiger Endpunkt: {exc}") from None
# httpx leaves a scheme's default port out of the Host header, so the signed host must not carry it either.
port = parts.port
default = {"https": 443, "http": 80}[parts.scheme]
@@ -82,10 +87,11 @@ class S3Client:
secret_key=self.secret_key, region=self.region, now=datetime.now(timezone.utc),
)
try:
async with httpx.AsyncClient(timeout=60) as client:
# Verbindet nur zu Adressen, die die Prüfung bestanden haben (auch nach DNS-Rebinding und Weiterleitungen).
async with url_guard.guarded_client(timeout=60) as client:
return await client.request(method, url, content=body if method == "PUT" else None, headers=headers)
except httpx.HTTPError as exc:
raise S3Error(f"Verbindung fehlgeschlagen ({type(exc).__name__})") from None
raise S3Error(f"Verbindung fehlgeschlagen ({url_guard.failure_reason(exc)})") from None
@staticmethod
def _check(response: httpx.Response, what: str) -> None:
+17 -10
View File
@@ -11,6 +11,7 @@ bekannte Stand bleibt erhalten, und nichts davon darf je einen Request
scheitern lassen oder länger als `_TIMEOUT_SECONDS` blockieren.
"""
import asyncio
import json
import logging
import re
from dataclasses import dataclass
@@ -19,8 +20,10 @@ from datetime import datetime, timedelta, timezone
import httpx
from sqlalchemy.ext.asyncio import AsyncSession
from app import runtime_settings
from app.config import settings
from app.models.update_check_settings import DEFAULT_FEED_URL, UpdateCheckSettings
from app.services import url_guard
from app.version import APP_VERSION
logger = logging.getLogger(__name__)
@@ -96,15 +99,19 @@ def _safe_url(value: object) -> str | None:
async def _fetch(url: str) -> tuple[str, str | None]:
async with httpx.AsyncClient(timeout=_TIMEOUT_SECONDS, follow_redirects=True, max_redirects=3) as http:
response = await http.get(
url,
headers={"Accept": "application/json", "User-Agent": f"OPNexus/{APP_VERSION}"},
)
response.raise_for_status()
if len(response.content) > _MAX_BYTES:
raise ValueError("response too large")
data = response.json()
# Geschützter Client: der Feed ist eine vom Admin einstellbare Adresse, Weiterleitungen eingeschlossen. Die
# Antwort wird beim Lesen auf _MAX_BYTES begrenzt, nicht erst danach.
async with url_guard.guarded_client(timeout=_TIMEOUT_SECONDS, follow_redirects=True, max_redirects=3) as http:
async with http.stream(
"GET", url, headers={"Accept": "application/json", "User-Agent": f"OPNexus/{APP_VERSION}"},
) as response:
response.raise_for_status()
body = bytearray()
async for chunk in response.aiter_bytes():
body += chunk
if len(body) > _MAX_BYTES:
raise ValueError("response too large")
data = json.loads(body)
tag = data["tag_name"]
if not isinstance(tag, str):
raise ValueError("tag_name is not a string")
@@ -133,7 +140,7 @@ async def check(db: AsyncSession) -> dict:
reset_cache()
_state.url = url
now = datetime.now(timezone.utc)
interval = timedelta(hours=max(1, settings.update_check_interval_hours))
interval = timedelta(hours=max(1, runtime_settings.value("update_check_interval_hours")))
fresh = _state.checked_at is not None and now - _state.checked_at < interval
backing_off = _state.error is not None and _state.attempted_at is not None and now - _state.attempted_at < _RETRY_AFTER
if not fresh and not backing_off:
+13
View File
@@ -136,6 +136,19 @@ async def ensure_resolved_safe(url: str) -> None:
raise ValueError("Der Name zeigt auf eine nicht erlaubte Adresse (lokal, Link-Local oder Metadaten-Dienst)")
async def checked_addresses(host: str, port: int | None = None) -> list[str]:
"""Addresses a non-HTTP target (syslog) may be connected to, all of them checked. Raises ValueError for a
forbidden target and OSError when the name does not resolve. The caller connects to these addresses, not to the
name, so a second answer from DNS cannot change the destination."""
bare = (host or "").strip().strip("[]")
if not bare or bare.lower() in ("localhost", "localhost.localdomain"):
raise ValueError("Diese Adresse (lokal, Link-Local oder Metadaten-Dienst) ist als Ziel nicht erlaubt")
addresses = await resolve_host(bare, port)
if not addresses or _any_forbidden(addresses):
raise ValueError("Diese Adresse (lokal, Link-Local oder Metadaten-Dienst) ist als Ziel nicht erlaubt")
return addresses
# ── connection-time guard (DNS rebinding) ─────────────────────────────────
BLOCKED_MESSAGE = "Zieladresse nicht erlaubt"
+2 -2
View File
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
Versioning scheme: see CHANGELOG.md ("Versionierung").
"""
APP_VERSION = "1.33.0"
APP_CODENAME = "Saker"
APP_VERSION = "1.34.0"
APP_CODENAME = "Griffon"
@@ -0,0 +1,31 @@
"""Einstellungen im Webinterface statt in der .env (runtime_settings) und Plattenalarm des OPNexus-Servers.
Revision ID: 0032
Revises: 0031
Create Date: 2026-10-24
"""
from alembic import op
revision = "0032"
down_revision = "0031"
branch_labels = None
depends_on = None
def upgrade() -> None:
# IF NOT EXISTS: a pre-Alembic install is bridged with create_all() first, which may already have created these.
op.execute(
"CREATE TABLE IF NOT EXISTS runtime_settings ("
" key varchar(100) PRIMARY KEY,"
" value json,"
" updated_at timestamptz NOT NULL DEFAULT now(),"
" updated_by varchar(150))"
)
op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS server_disk_enabled boolean NOT NULL DEFAULT true")
op.execute("ALTER TABLE notification_settings ADD COLUMN IF NOT EXISTS server_disk_threshold_pct integer NOT NULL DEFAULT 90")
def downgrade() -> None:
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS server_disk_threshold_pct")
op.execute("ALTER TABLE notification_settings DROP COLUMN IF EXISTS server_disk_enabled")
op.execute("DROP TABLE IF EXISTS runtime_settings")
+36
View File
@@ -0,0 +1,36 @@
#!/bin/sh
# Startet das Backend als unprivilegierter Benutzer (uid 10001).
#
# Der Container beginnt als root, damit bestehende Installationen ihre Volumes
# (bisher von root angelegt) selbst auf den Benutzer umstellen können: ein Update
# braucht dadurch keinen Handgriff, und ein Rückweg auf eine ältere Version bleibt
# möglich (root liest alles). Danach gibt `setpriv` root und alle Capabilities ab.
#
# OPNEXUS_RUN_AS_ROOT=1 lässt das Backend wie früher als root laufen (Notausgang
# für Sonderfälle, z. B. Bind-Mounts, die sich nicht umstellen lassen).
set -eu
APP_UID=10001
APP_GID=10001
if [ "$(id -u)" = "0" ] && [ "${OPNEXUS_RUN_AS_ROOT:-0}" != "1" ]; then
# Datenverzeichnisse: alles, was nicht dem App-Benutzer gehört, wird umgestellt.
# Bei jedem Start, nicht nur einmal: Befehle wie `opnexus restore` laufen als
# root und hinterlassen root-eigene Dateien.
for d in \
"${OPNSENSE_SECRET_DIR:-}" "${OPNSENSE_BACKUP_DIR:-}" "${IPSEC_PSK_SECRET_DIR:-}" \
"${OPENVPN_KEY_SECRET_DIR:-}" "${WIREGUARD_KEY_SECRET_DIR:-}" "${CERT_KEY_SECRET_DIR:-}" \
"${ACME_SECRET_DIR:-}" "${TOTP_SECRET_DIR:-}" "${NOTIFICATION_SECRET_DIR:-}" \
"${OVERLAY_DIR:-}" /backups
do
[ -n "$d" ] && [ -d "$d" ] || continue
find "$d" -xdev \( ! -user "$APP_UID" -o ! -group "$APP_GID" \) \
-exec chown -h "$APP_UID:$APP_GID" {} + 2>/dev/null || true
done
# HOME zeigt sonst noch auf /root (nicht lesbar): asyncpg/libpq suchen dort ~/.postgresql.
export HOME=/app
exec setpriv --reuid="$APP_UID" --regid="$APP_GID" --clear-groups \
--inh-caps=-all --bounding-set=-all --no-new-privs "$@"
fi
exec "$@"
+10
View File
@@ -71,6 +71,16 @@ async def _clean_tables():
await conn.execute(table.delete())
@pytest.fixture(autouse=True)
def _reset_runtime_settings():
# Saved settings live in a module-level cache (app/runtime_settings.py); the table is cleaned after each test,
# the cache must be too, or a value saved by one test changes the next.
from app import runtime_settings
runtime_settings._cache.clear()
yield
runtime_settings._cache.clear()
@pytest.fixture(autouse=True)
def _reset_login_throttle():
# The throttle is a module-level singleton (by design — it must survive
+36 -3
View File
@@ -102,7 +102,7 @@ async def test_sample_records_only_refused_reads_in_denied(client, respx_mock):
# forbidden -> denied (sorted); a server error and a missing endpoint are "unknown", no finding about rights
assert raw["denied"] == ["gateways", "services"]
assert raw["gateways"] is None and raw["services"] is None and raw["swap_used_pct"] is None
assert raw["ok"] == [] # nothing went through
assert raw["ok"] == ["base"] # only the base reading went through
@pytest.mark.asyncio
@@ -112,7 +112,7 @@ async def test_sample_denied_is_an_empty_list_when_nothing_is_refused(client, re
_mock_resource_endpoints(respx_mock)
_mock_optional_reads(respx_mock)
raw = await _sample_raw()
assert raw["denied"] == [] and raw["ok"] == ["cpu", "gateways", "services", "swap"]
assert raw["denied"] == [] and raw["ok"] == ["base", "cpu", "gateways", "services", "swap"]
# unreachable optional endpoints (connect error / timeout) are not refusals either
_mock_optional_reads(respx_mock)
respx_mock.get("https://fw1.test/api/routes/gateway/status").mock(side_effect=httpx.ReadTimeout("slow"))
@@ -120,16 +120,49 @@ async def test_sample_denied_is_an_empty_list_when_nothing_is_refused(client, re
@pytest.mark.asyncio
async def test_base_read_refusal_still_fails_the_sample(client, respx_mock):
async def test_base_read_refusal_becomes_a_finding_instead_of_a_silent_failure(client, respx_mock):
"""Fehlen schon die Basis-Rechte, gab es nie einen Messpunkt und der Rechte-Alarm konnte nicht auslösen."""
_mock_vip(respx_mock)
await _create_host(client, "fw1", "https://fw1.test")
_mock_resource_endpoints(respx_mock)
respx_mock.get("https://fw1.test/api/diagnostics/system/systemResources").respond(403)
async with SessionLocal() as db:
result = await resource_monitor_service.poll_all_hosts(db)
assert result["hosts_failed"] == 0 and result["sampled"] == 1
async with SessionLocal() as db:
sample = (await db.scalars(select(HostResourceSample))).one()
assert sample.raw == {"denied": ["base"], "ok": []}
assert sample.mem_used_pct is None and sample.disk_used_pct is None # "unbekannt", nicht "in Ordnung"
@pytest.mark.asyncio
async def test_base_read_timeout_is_still_a_failed_sample_not_a_rights_finding(client, respx_mock):
_mock_vip(respx_mock)
await _create_host(client, "fw1", "https://fw1.test")
_mock_resource_endpoints(respx_mock)
respx_mock.get("https://fw1.test/api/diagnostics/system/systemResources").respond(500)
async with SessionLocal() as db:
result = await resource_monitor_service.poll_all_hosts(db)
assert result["hosts_failed"] == 1 and result["sampled"] == 0
@pytest.mark.asyncio
async def test_base_rights_alert_opens_and_closes_again(client):
host = await _create_host(client, "fw1", "https://fw1.test")
hid = uuid.UUID(host["id"])
key = f"{hid}:base"
await _only("api_rights_enabled")
await _add_samples(hid, [{"denied": ["base"], "ok": []}] * 3)
problems = await _rights_problems()
assert list(problems) == [key]
assert problems[key]["fehlendes_recht"] == "Lobby: Dashboard" and problems[key]["endpunkt"].endswith("systemResources")
async with SessionLocal() as db:
await run_notification_checks(db)
await _clear_samples()
await _add_samples(hid, [{"denied": [], "ok": ["base"]}] * 3)
assert dict(await _rights_problems()) == {}
# ── The alert ────────────────────────────────────────────────────────────────
DENIED_GW = {"denied": ["gateways"], "ok": ["cpu", "services", "swap"], "gateways": None, "services": {"unbound": 1}, "cpu_threads": 4}
+8 -1
View File
@@ -15,10 +15,17 @@ from app.database import SessionLocal
from app.models.audit import AuditEvent
from app.models.audit_export import AuditExportSettings
from app.services import audit_export_service as svc
from app.services import notification_secret_store
from app.services import notification_secret_store, url_guard
from tests.test_api_tokens import _login
@pytest.fixture(autouse=True)
def _loopback_allowed(monkeypatch):
"""Diese Tests sprechen echte Sockets auf 127.0.0.1 an; der Zieladress-Schutz selbst steht in
tests/test_audit_export_guard.py."""
monkeypatch.setattr(url_guard, "_forbidden", lambda ip: False)
async def _event(n: int, *, age_s: float = 60, status="accepted", details=None) -> AuditEvent:
async with SessionLocal() as db:
event = AuditEvent(
+126
View File
@@ -0,0 +1,126 @@
"""Zieladress-Schutz für den Audit-Export, Offsite-S3 und den Update-Feed (1.29-Rest): Loopback, Link-Local und
Metadaten-Dienst sind als Ziel nie erlaubt, auch nicht über Weiterleitungen oder DNS-Antworten."""
import pytest
from app.models.audit_export import AuditExportSettings
from app.services import audit_export_service as svc
from app.services import s3_client, update_check_service, url_guard
def _row(**kw) -> AuditExportSettings:
row = AuditExportSettings(id=1)
for key, value in kw.items():
setattr(row, key, value)
return row
@pytest.mark.asyncio
@pytest.mark.parametrize("host", ["127.0.0.1", "localhost", "169.254.169.254", "::1", "0.0.0.0"])
async def test_syslog_refuses_forbidden_targets(host):
row = _row(syslog_host=host, syslog_port=514, syslog_protocol="udp")
with pytest.raises(svc.ExportError, match="Syslog-Ziel nicht erlaubt"):
await svc.send_syslog(row, [{"time": "t", "status": "accepted"}])
@pytest.mark.asyncio
async def test_syslog_refuses_a_name_that_resolves_to_loopback(monkeypatch):
async def fake(host, port=None):
return ["203.0.113.5", "127.0.0.1"]
monkeypatch.setattr(url_guard, "resolve_host", fake)
with pytest.raises(svc.ExportError, match="nicht erlaubt"):
await svc.send_syslog(_row(syslog_host="logs.example.test", syslog_port=514, syslog_protocol="tcp"),
[{"time": "t", "status": "accepted"}])
@pytest.mark.asyncio
async def test_syslog_connects_to_the_checked_address_not_the_name(monkeypatch):
seen = {}
async def fake(host, port=None):
return ["203.0.113.5"]
def send(row, lines, addresses):
seen["addresses"] = addresses
monkeypatch.setattr(url_guard, "resolve_host", fake)
monkeypatch.setattr(svc, "_send_syslog_sync", send)
await svc.send_syslog(_row(syslog_host="logs.example.test", syslog_port=514, syslog_protocol="tcp"),
[{"time": "t", "status": "accepted"}])
assert seen["addresses"] == ["203.0.113.5"]
@pytest.mark.asyncio
@pytest.mark.parametrize("url", ["http://127.0.0.1:9/x", "http://localhost/x", "http://169.254.169.254/latest", "ftp://example.test/x"])
async def test_webhook_refuses_forbidden_targets(url):
with pytest.raises(svc.ExportError, match="Webhook-Ziel nicht erlaubt"):
await svc.send_webhook(_row(webhook_url=url), [])
@pytest.mark.asyncio
async def test_webhook_refuses_a_rebound_name(monkeypatch):
async def fake(host, port=None):
return ["127.0.0.1"]
monkeypatch.setattr(url_guard, "resolve_host", fake)
with pytest.raises(svc.ExportError, match="nicht erlaubt"):
await svc.send_webhook(_row(webhook_url="https://hooks.example.test/a"), [])
@pytest.mark.parametrize("endpoint", ["https://127.0.0.1:9000", "https://localhost:9000", "https://169.254.169.254"])
def test_s3_endpoint_must_not_be_local(endpoint):
with pytest.raises(s3_client.S3Error, match="Ungültiger Endpunkt"):
s3_client.S3Client(endpoint=endpoint, region="", bucket="b", access_key="a", secret_key="s")
def test_s3_private_lan_endpoint_is_fine():
s3_client.S3Client(endpoint="https://10.0.0.5:9000", region="", bucket="b", access_key="a", secret_key="s")
@pytest.mark.asyncio
async def test_s3_connection_is_refused_when_the_name_resolves_to_loopback(monkeypatch):
async def fake(host, port=None):
return ["127.0.0.1"]
monkeypatch.setattr(url_guard, "resolve_host", fake)
client = s3_client.S3Client(endpoint="https://s3.example.test", region="", bucket="b", access_key="a", secret_key="s")
with pytest.raises(s3_client.S3Error, match="Zieladresse nicht erlaubt"):
await client.put("k", b"x")
@pytest.mark.asyncio
async def test_update_feed_refuses_a_rebound_name(monkeypatch):
async def fake(host, port=None):
return ["169.254.169.254"]
monkeypatch.setattr(url_guard, "resolve_host", fake)
with pytest.raises(Exception):
await update_check_service._fetch("https://updates.example.test/latest.json")
@pytest.mark.asyncio
async def test_update_feed_is_capped_while_reading_not_afterwards(monkeypatch):
"""Eine endlose Antwort wird nach _MAX_BYTES abgebrochen, ohne dass sie komplett im Speicher landet."""
import asyncio
sent = {"bytes": 0}
async def handle(reader, writer):
await reader.readuntil(b"\r\n\r\n")
writer.write(b"HTTP/1.1 200 OK\r\ncontent-type: application/json\r\ntransfer-encoding: chunked\r\n\r\n")
chunk = b"x" * 65536
try:
for _ in range(2000): # ~130 MB angeboten
writer.write(f"{len(chunk):x}\r\n".encode() + chunk + b"\r\n")
await writer.drain()
sent["bytes"] += len(chunk)
except (ConnectionError, OSError):
pass
finally:
writer.close()
server = await asyncio.start_server(handle, "127.0.0.1", 0)
port = server.sockets[0].getsockname()[1]
monkeypatch.setattr(url_guard, "_forbidden", lambda ip: False)
try:
with pytest.raises(ValueError, match="too large"):
await update_check_service._fetch(f"http://127.0.0.1:{port}/latest.json")
finally:
server.close()
assert sent["bytes"] < 10_000_000
+43
View File
@@ -140,3 +140,46 @@ def test_command_line_reads_the_passphrase_from_the_environment(data, monkeypatc
monkeypatch.setenv("FULL_BACKUP_PASSPHRASE", "")
assert fb._main(["full_backup", "create"]) == 1
assert fb._main(["full_backup", "bogus"]) == 2
def test_a_secret_changing_during_the_backup_triggers_a_new_attempt(data):
calls = []
def dump(path: Path) -> None:
calls.append(1)
if len(calls) == 1: # ein Objekt wird mitten in der Sicherung angelegt
(data / "opnsense_secrets" / "host-2.secret").write_text("API-SECRET-2")
path.write_bytes(b"PGDMP-fake-database")
blob = fb.create(PASS, dump=dump, revision="0009")
assert len(calls) == 2
for f in data.rglob("*"):
if f.is_file():
f.unlink()
fb.restore(blob, PASS, restore_db=lambda p: None, head="0012")
assert (data / "opnsense_secrets" / "host-2.secret").read_text() == "API-SECRET-2"
def test_a_backup_that_never_holds_still_is_refused_instead_of_written_inconsistent(data):
n = []
def dump(path: Path) -> None:
n.append(1)
(data / "totp_secrets" / f"u{len(n)}.secret").write_text("x")
path.write_bytes(b"PGDMP-fake-database")
with pytest.raises(fb.FullBackupError, match="Geheimnisse geändert"):
fb.create(PASS, dump=dump, revision="0009")
assert len(n) == fb.ATTEMPTS
def test_new_firewall_config_backups_do_not_count_as_a_change(data):
calls = []
def dump(path: Path) -> None:
calls.append(1)
(data / "opnsense_backups" / "host-1" / f"new-{len(calls)}.xml").write_text("<opnsense/>")
path.write_bytes(b"PGDMP-fake-database")
fb.create(PASS, dump=dump, revision="0009")
assert len(calls) == 1
+110
View File
@@ -0,0 +1,110 @@
"""H1: the full backup can carry files of the server itself (.env, the proxy's CA) for a host move.
They come from the CLI as a tar on stdin, live under host/ in the archive and only ever leave it again as a tar for
the CLI (``host-files``); ``restore`` never writes them, and nothing outside env/ and caddy-ca/ is accepted.
"""
import io
import json
import tarfile
import pytest
from app import full_backup as fb
from tests.test_full_backup import PASS, _dump, data # noqa: F401 (fixture)
def _tar(entries: dict[str, bytes | None], *, links: dict[str, str] | None = None) -> bytes:
out = io.BytesIO()
with tarfile.open(fileobj=out, mode="w") as tar:
for name, content in entries.items():
info = tarfile.TarInfo(name)
if content is None:
info.type = tarfile.DIRTYPE
tar.addfile(info)
else:
info.size = len(content)
tar.addfile(info, io.BytesIO(content))
for name, target in (links or {}).items():
info = tarfile.TarInfo(name)
info.type, info.linkname = tarfile.SYMTYPE, target
tar.addfile(info)
return out.getvalue()
HOST = _tar({
"./env": b"POSTGRES_PASSWORD=pw-of-old-server\nHEARTBEAT_URL=https://hc.example/x\n",
"./caddy-ca": None,
"./caddy-ca/local": None,
"./caddy-ca/local/root.crt": b"-----BEGIN CERTIFICATE-----CA",
"./caddy-ca/local/root.key": b"-----BEGIN EC PRIVATE KEY-----KEY",
})
def _host_tar_names(raw: bytes) -> dict[str, bytes | None]:
with tarfile.open(fileobj=io.BytesIO(raw), mode="r:") as tar:
return {m.name: (tar.extractfile(m).read() if m.isfile() else None) for m in tar.getmembers()}
def test_host_files_round_trip_and_are_encrypted(data): # noqa: F811
blob = fb.create(PASS, dump=_dump, revision="0009", host_files=HOST)
assert b"pw-of-old-server" not in blob and b"PRIVATE KEY" not in blob
assert fb.info(blob, PASS)["host"] == ["caddy-ca", "env"]
files = _host_tar_names(fb.host_files(blob, PASS))
assert files["env"].startswith(b"POSTGRES_PASSWORD=pw-of-old-server")
assert files["caddy-ca/local/root.key"] == b"-----BEGIN EC PRIVATE KEY-----KEY"
def test_restore_never_writes_host_files(data): # noqa: F811
blob = fb.create(PASS, dump=_dump, revision="0009", host_files=HOST)
before = {p for p in data.rglob("*")}
fb.restore(blob, PASS, restore_db=lambda p: None, head="0012")
created = {p for p in data.rglob("*")} - before
assert not [p for p in created if "env" in p.name or "caddy" in str(p)]
def test_a_backup_without_host_files_hands_back_an_empty_tar(data): # noqa: F811
blob = fb.create(PASS, dump=_dump, revision="0009")
assert "host" not in fb.info(blob, PASS)
assert _host_tar_names(fb.host_files(blob, PASS)) == {}
@pytest.mark.parametrize("bad", [
_tar({"etc/passwd": b"x"}), # outside env/ and caddy-ca/
_tar({"caddy-ca/../../etc/x": b"x"}), # escaping
_tar({"/env": b"x"}), # absolute
_tar({}, links={"env": "/etc/shadow"}), # symlink
b"not a tar at all",
_tar({}), # empty
])
def test_hostile_or_empty_host_files_are_refused_before_the_dump(data, bad): # noqa: F811
dumped = []
with pytest.raises(fb.FullBackupError):
fb.create(PASS, dump=lambda p: dumped.append(p), revision="0009", host_files=bad)
assert dumped == []
def test_oversized_host_files_are_refused(data, monkeypatch): # noqa: F811
monkeypatch.setattr(fb, "_HOST_MAX_BYTES", 100)
with pytest.raises(fb.FullBackupError, match="zu groß"):
fb.create(PASS, dump=_dump, revision="0009", host_files=HOST)
def test_command_line_create_with_host_files_and_extract(data, monkeypatch, capsysbinary): # noqa: F811
monkeypatch.setenv("FULL_BACKUP_PASSPHRASE", PASS)
monkeypatch.setattr(fb, "_dump_database", _dump)
monkeypatch.setattr(fb, "_head_revision", lambda: "0009")
class Stdin:
def __init__(self, raw):
self.buffer = io.BytesIO(raw)
monkeypatch.setattr("sys.stdin", Stdin(HOST))
assert fb._main(["full_backup", "create", "--host-files"]) == 0
blob = capsysbinary.readouterr().out
monkeypatch.setattr("sys.stdin", Stdin(blob))
assert fb._main(["full_backup", "host-files"]) == 0
assert "env" in _host_tar_names(capsysbinary.readouterr().out)
monkeypatch.setattr("sys.stdin", Stdin(blob))
assert fb._main(["full_backup", "info"]) == 0
assert json.loads(capsysbinary.readouterr().out)["host"] == ["caddy-ca", "env"]
assert fb._main(["full_backup", "restore", "--host-files"]) == 2 # the flag belongs to create only
+341
View File
@@ -0,0 +1,341 @@
"""H0 (docs/HA-PLAN.md): only the process holding the leader lock runs the background loops.
These run against the real test database: the lock, its loss when the session is terminated and the takeover are
PostgreSQL behaviour, not something a mock would prove.
"""
import asyncio
import logging
import httpx
import pytest
from sqlalchemy import text
from app import leader, main
from app.config import settings
from app.database import SessionLocal, engine
from app.services import auth_service, heartbeat_service, url_guard
FAST = {"check_interval": 0.2, "check_timeout": 1.0, "retry_interval": 0.2, "takeover_delay": 0.6}
class Tracker:
"""Loop bodies that record which gate is running an iteration right now."""
def __init__(self):
self.running: set[str] = set()
self.max_concurrent = 0
self.iterations: dict[str, int] = {}
def loop(self, name: str, gate_ref: dict, iteration_seconds: float = 0.05):
async def body():
while True:
await gate_ref["gate"].checkpoint()
self.running.add(name)
self.max_concurrent = max(self.max_concurrent, len(self.running))
self.iterations[name] = self.iterations.get(name, 0) + 1
try:
await asyncio.sleep(iteration_seconds)
finally:
self.running.discard(name)
return body
def _gate(tracker: Tracker, name: str, iteration_seconds: float = 0.05, **kw) -> leader.LeaderGate:
ref: dict = {}
gate = leader.LeaderGate({"work": tracker.loop(name, ref, iteration_seconds)}, **{**FAST, **kw})
ref["gate"] = gate
return gate
async def _until(predicate, timeout=10.0):
deadline = asyncio.get_running_loop().time() + timeout
while not predicate():
if asyncio.get_running_loop().time() > deadline:
raise AssertionError("condition not reached in time")
await asyncio.sleep(0.05)
async def _stop(*gates_and_tasks):
for gate, task in gates_and_tasks:
task.cancel()
try:
await task
except asyncio.CancelledError:
pass
await gate.stop()
@pytest.mark.asyncio
async def test_a_single_gate_waits_the_takeover_delay_then_runs_the_loops():
tracker = Tracker()
gate = _gate(tracker, "a")
task = asyncio.create_task(gate.run())
try:
await _until(lambda: gate.role == leader.TAKEOVER)
assert tracker.iterations == {} # nothing runs during the takeover wait
await _until(lambda: gate.role == leader.LEADER)
await _until(lambda: tracker.iterations.get("a", 0) >= 3)
finally:
await _stop((gate, task))
assert gate.role == leader.STANDBY
@pytest.mark.asyncio
async def test_a_second_gate_stays_in_standby_and_runs_nothing():
tracker = Tracker()
a, b = _gate(tracker, "a"), _gate(tracker, "b")
ta = asyncio.create_task(a.run())
try:
await _until(lambda: a.role == leader.LEADER)
tb = asyncio.create_task(b.run())
try:
await _until(lambda: b.role == leader.STANDBY)
await asyncio.sleep(1.0)
assert b.role == leader.STANDBY and "b" not in tracker.iterations
assert tracker.max_concurrent == 1
finally:
await _stop((b, tb))
finally:
await _stop((a, ta))
@pytest.mark.asyncio
async def test_a_terminated_lock_session_stops_the_loops_and_the_other_takes_over_without_overlap():
"""The review's case: the connection dies under a living process. The loops must stop before anyone else runs."""
tracker = Tracker()
a, b = _gate(tracker, "a"), _gate(tracker, "b")
ta = asyncio.create_task(a.run())
tb = None
try:
await _until(lambda: a.role == leader.LEADER and tracker.iterations.get("a", 0) > 0)
tb = asyncio.create_task(b.run())
await _until(lambda: b.role == leader.STANDBY)
pid = await a._conn.scalar(text("SELECT pg_backend_pid()"))
await a._conn.commit()
async with engine.connect() as admin:
assert await admin.scalar(text("SELECT pg_terminate_backend(:pid)"), {"pid": pid})
await _until(lambda: b.role == leader.LEADER and tracker.iterations.get("b", 0) > 0)
assert a.role != leader.LEADER
assert "a" not in tracker.running
assert tracker.max_concurrent == 1, "both gates ran an iteration at the same time"
finally:
if tb is not None:
await _stop((b, tb))
await _stop((a, ta))
@pytest.mark.asyncio
async def test_a_long_iteration_is_cancelled_when_the_lock_is_lost():
"""A notification cycle can take half a minute; it must not keep running after the lock is gone."""
tracker = Tracker()
# The old leader notices its loss only at its next check (up to 1.5 s + timeout), while the new one applies every
# 0.1 s: only the takeover delay (default: check interval + timeout + margin) keeps the two apart.
slow = {"check_interval": 1.5, "takeover_delay": None}
a = _gate(tracker, "a", iteration_seconds=60, **slow)
b = _gate(tracker, "b", iteration_seconds=60, retry_interval=0.1, **slow)
ta = asyncio.create_task(a.run())
tb = None
try:
await _until(lambda: "a" in tracker.running, timeout=15)
tb = asyncio.create_task(b.run())
await _until(lambda: b.role == leader.STANDBY)
pid = await a._conn.scalar(text("SELECT pg_backend_pid()"))
await a._conn.commit()
async with engine.connect() as admin:
await admin.scalar(text("SELECT pg_terminate_backend(:pid)"), {"pid": pid})
await _until(lambda: "a" not in tracker.running, timeout=5)
await _until(lambda: "b" in tracker.running, timeout=15)
assert tracker.max_concurrent == 1
finally:
if tb is not None:
await _stop((b, tb))
await _stop((a, ta))
@pytest.mark.asyncio
async def test_checkpoint_refuses_outside_leadership():
tracker = Tracker()
gate = _gate(tracker, "a")
with pytest.raises(leader.LeadershipLost):
await gate.checkpoint()
# Without a gate (a test driving a loop directly) the module-level checkpoint is a no-op.
leader.gate = None
await leader.checkpoint()
assert leader.current_role() == leader.STARTING
@pytest.mark.asyncio
async def test_stop_releases_the_lock_at_once():
tracker = Tracker()
a, b = _gate(tracker, "a"), _gate(tracker, "b")
ta = asyncio.create_task(a.run())
await _until(lambda: a.role == leader.LEADER)
await _stop((a, ta))
async with engine.connect() as conn:
assert await conn.scalar(text("SELECT pg_try_advisory_lock(:k)"), {"k": leader.LOCK_KEY})
await conn.execute(text("SELECT pg_advisory_unlock(:k)"), {"k": leader.LOCK_KEY})
tb = asyncio.create_task(b.run())
try:
await _until(lambda: b.role == leader.LEADER)
finally:
await _stop((b, tb))
@pytest.mark.asyncio
async def test_a_crashed_loop_is_restarted_and_lost_leadership_is_not_mistaken_for_a_crash():
starts = []
async def flaky():
starts.append(1)
await asyncio.sleep(0.05)
if len(starts) == 1:
raise RuntimeError("boom")
await asyncio.sleep(3600)
gate = leader.LeaderGate({"flaky": flaky}, **FAST)
task = asyncio.create_task(gate.run())
try:
await _until(lambda: len(starts) >= 2)
assert gate.role == leader.LEADER
finally:
await _stop((gate, task))
class _ReplicaConn:
"""A connection to a database in recovery: pg_try_advisory_lock would succeed there, so it must never be asked."""
def __init__(self):
self.statements: list[str] = []
async def scalar(self, statement, params=None):
self.statements.append(str(statement))
return True
async def commit(self):
pass
async def invalidate(self):
pass
async def close(self):
pass
@pytest.mark.asyncio
async def test_a_replica_never_becomes_leader():
conn = _ReplicaConn()
async def connect():
return conn
tracker = Tracker()
ref: dict = {}
gate = leader.LeaderGate({"work": tracker.loop("r", ref)}, connect=connect, **FAST)
ref["gate"] = gate
task = asyncio.create_task(gate.run())
try:
await _until(lambda: gate.role == leader.REPLICA)
await asyncio.sleep(0.6)
assert gate.role == leader.REPLICA and tracker.iterations == {}
assert not any("advisory" in s for s in conn.statements)
finally:
await _stop((gate, task))
@pytest.mark.asyncio
async def test_health_and_metrics_report_the_role(client):
leader.gate = None
assert (await client.get("/api/health")).json() == {"status": "ok", "role": "starting"}
tracker = Tracker()
gate = _gate(tracker, "a")
leader.gate = gate
task = asyncio.create_task(gate.run())
try:
await _until(lambda: gate.role == leader.LEADER)
assert (await client.get("/api/health")).json()["role"] == "leader"
metrics = (await client.get("/api/metrics")).text
assert 'opnexus_leader{role="leader"} 1' in metrics
finally:
await _stop((gate, task))
leader.gate = None
@pytest.mark.asyncio
async def test_bootstrap_admin_waits_for_a_concurrent_first_start():
async with engine.connect() as other:
await other.execute(text("SELECT pg_advisory_xact_lock(:k)"), {"k": auth_service._BOOTSTRAP_LOCK_KEY})
async with SessionLocal() as db:
with pytest.raises(asyncio.TimeoutError):
await asyncio.wait_for(auth_service.bootstrap_admin(db), 0.5)
await other.rollback() # releases the lock
async with SessionLocal() as db:
await asyncio.wait_for(auth_service.bootstrap_admin(db), 5)
# ── Heartbeat ────────────────────────────────────────────────────────────────
@pytest.mark.parametrize("value", ["", "http://hc.example.test/ping/abc", "https://127.0.0.1/ping", "https://localhost/x",
"not a url"])
def test_heartbeat_is_off_unless_a_valid_https_url_is_set(monkeypatch, value):
monkeypatch.setattr(settings, "heartbeat_url", value)
assert heartbeat_service.configured_url() is None
@pytest.mark.asyncio
async def test_heartbeat_pings_and_never_logs_the_url(monkeypatch, respx_mock, caplog):
url = "https://hc.example.test/ping/secret-check-id"
monkeypatch.setattr(settings, "heartbeat_url", url)
async def public(host, port=None):
return ["203.0.113.10"]
monkeypatch.setattr(url_guard, "resolve_host", public)
route = respx_mock.get(url).mock(return_value=httpx.Response(200))
caplog.set_level(logging.DEBUG)
assert await heartbeat_service.ping() is True
assert route.call_count == 1
route.mock(return_value=httpx.Response(500))
assert await heartbeat_service.ping() is False
assert "secret-check-id" not in caplog.text
@pytest.mark.asyncio
async def test_heartbeat_refuses_a_name_that_resolves_to_loopback(monkeypatch):
monkeypatch.setattr(settings, "heartbeat_url", "https://hc.example.test/ping/x")
async def rebound(host, port=None):
return ["127.0.0.1"]
monkeypatch.setattr(url_guard, "resolve_host", rebound)
assert await heartbeat_service.ping() is False
class _StopLoop(Exception):
pass
@pytest.mark.asyncio
@pytest.mark.parametrize("checks_fail", [False, True])
async def test_the_notification_loop_sends_the_heartbeat_only_after_a_successful_cycle(monkeypatch, checks_fail):
pings = []
async def checks(db):
if checks_fail:
raise RuntimeError("database gone")
async def ping():
pings.append(1)
return True
async def no_carp(db):
return []
async def stop(_):
raise _StopLoop()
leader.gate = None
monkeypatch.setattr(main.notification_service, "run_notification_checks", checks)
monkeypatch.setattr(main.heartbeat_service, "ping", ping)
monkeypatch.setattr(main.carp_event_service, "detect_and_record", no_carp)
monkeypatch.setattr(main.asyncio, "sleep", stop)
with pytest.raises(_StopLoop):
await main._notification_poll_loop()
assert pings == ([] if checks_fail else [1])
+194
View File
@@ -0,0 +1,194 @@
"""Settings › Betrieb: values that used to need the .env are set in the web interface (app/runtime_settings.py)."""
import datetime as dt
import pytest
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import NameOID
from sqlalchemy import select
from app import runtime_settings as rs
from app.config import settings
from app.database import SessionLocal
from app.models.audit import AuditEvent
from app.models.runtime_setting import RuntimeSetting
from app.services import notification_checks, notification_secret_store
from tests.test_operator_role import _operator
def _ca_pem() -> str:
key = ec.generate_private_key(ec.SECP256R1())
name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "Mail CA")])
now = dt.datetime.now(dt.timezone.utc)
cert = (x509.CertificateBuilder().subject_name(name).issuer_name(name).public_key(key.public_key())
.serial_number(1).not_valid_before(now).not_valid_after(now + dt.timedelta(days=30))
.add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
.sign(key, hashes.SHA256()))
return cert.public_bytes(serialization.Encoding.PEM).decode()
def _by_key(body) -> dict:
return {item["key"]: item for item in body["settings"]}
@pytest.fixture
def secret_dir(tmp_path, monkeypatch):
monkeypatch.setattr(settings, "notification_secret_dir", str(tmp_path))
return tmp_path
@pytest.mark.asyncio
async def test_defaults_are_listed_with_bounds_units_and_source(client):
items = _by_key((await client.get("/api/settings/system")).json())
assert set(items) == set(rs.BY_KEY)
session = items["session_lifetime_days"]
assert session == {**session, "value": 30, "default": 30, "unit": "days", "min": 1, "max": 365, "source": "default"}
assert items["heartbeat_url"]["value"] is None and items["heartbeat_url"]["is_set"] is False
@pytest.mark.asyncio
async def test_saving_changes_what_the_code_reads_and_is_audited(client):
resp = await client.put("/api/settings/system", json={"values": {
"session_lifetime_days": 7, "deploy_confirm_window_seconds": 300, "opnsense_timeout": 25.5}})
assert resp.status_code == 200, resp.text
items = _by_key(resp.json())
assert items["session_lifetime_days"]["value"] == 7 and items["session_lifetime_days"]["source"] == "ui"
assert rs.value("session_lifetime_days") == 7 * 86400 # the code keeps working in seconds
assert rs.value("deploy_confirm_window_seconds") == 300
assert rs.value("opnsense_timeout") == 25.5
async with SessionLocal() as db:
event = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "system_settings"))).one()
assert event.details["changes"]["session_lifetime_days"] == {"old": 30, "new": 7}
# null resets to the default
items = _by_key((await client.put("/api/settings/system", json={"values": {"session_lifetime_days": None}})).json())
assert items["session_lifetime_days"]["source"] == "default" and rs.value("session_lifetime_days") == 30 * 86400
@pytest.mark.asyncio
@pytest.mark.parametrize("values", [
{"deploy_confirm_window_seconds": 10}, # below the floor: the safety window must stay usable
{"login_max_attempts": 1000},
{"session_lifetime_days": 1.5},
{"session_lifetime_days": "30"},
{"no_such_setting": 1},
{"heartbeat_url": "http://hc.example.test/x"}, # https only
{"heartbeat_url": "https://127.0.0.1/x"},
{"smtp_ca_pem": "not a certificate"},
])
async def test_invalid_values_are_refused_and_nothing_is_saved(client, values):
resp = await client.put("/api/settings/system", json={"values": {"login_window_seconds": 600, **values}})
assert resp.status_code == 422 and resp.json()["detail"]["code"] == "SETTING_INVALID"
assert resp.json()["detail"]["params"]["setting"] in values
async with SessionLocal() as db:
assert (await db.scalars(select(RuntimeSetting))).all() == [] # the valid one in the same request neither
assert rs.value("login_window_seconds") == settings.login_window_seconds
@pytest.mark.asyncio
async def test_only_admins_change_settings(client):
op = await _operator(client, "settings-oper")
try:
assert (await op.get("/api/settings/system")).status_code == 200
resp = await op.put("/api/settings/system", json={"values": {"login_max_attempts": 9}})
assert resp.status_code == 403
finally:
await op.aclose()
@pytest.mark.asyncio
async def test_a_server_value_wins_and_cannot_be_overwritten(client, monkeypatch):
monkeypatch.setenv("HEARTBEAT_URL", "https://hc.example.test/from-env")
monkeypatch.setattr(settings, "heartbeat_url", "https://hc.example.test/from-env")
item = _by_key((await client.get("/api/settings/system")).json())["heartbeat_url"]
assert item["source"] == "server" and item["is_set"] is True and item["env"] == "HEARTBEAT_URL"
resp = await client.put("/api/settings/system", json={"values": {"heartbeat_url": "https://hc.example.test/ui"}})
assert resp.status_code == 422 and "HEARTBEAT_URL" in resp.json()["detail"]["params"]["reason"]
assert rs.value("heartbeat_url") == "https://hc.example.test/from-env"
@pytest.mark.asyncio
async def test_an_empty_env_value_does_not_count_as_set(client, monkeypatch, secret_dir):
monkeypatch.setenv("HEARTBEAT_URL", "")
assert (await client.put("/api/settings/system", json={"values": {"heartbeat_url": "https://hc.example.test/ui"}})).status_code == 200
assert rs.value("heartbeat_url") == "https://hc.example.test/ui"
@pytest.mark.asyncio
async def test_the_update_interval_from_an_old_compose_file_does_not_lock_the_field(client, monkeypatch):
"""Compose files before 1.34 always pass UPDATE_CHECK_INTERVAL_HOURS=12; that must stay editable."""
monkeypatch.setenv("UPDATE_CHECK_INTERVAL_HOURS", "12")
item = _by_key((await client.get("/api/settings/system")).json())["update_check_interval_hours"]
assert item["source"] == "default"
assert (await client.put("/api/settings/system", json={"values": {"update_check_interval_hours": 3}})).status_code == 200
assert rs.value("update_check_interval_hours") == 3
@pytest.mark.asyncio
async def test_the_heartbeat_url_is_a_secret(client, secret_dir):
url = "https://hc.example.test/ping/secret-id"
items = _by_key((await client.put("/api/settings/system", json={"values": {"heartbeat_url": url}})).json())
assert items["heartbeat_url"] == {**items["heartbeat_url"], "value": None, "is_set": True, "source": "ui"}
assert rs.value("heartbeat_url") == url
assert notification_secret_store.read_secret("runtime-heartbeat_url") == url
async with SessionLocal() as db:
row = await db.get(RuntimeSetting, "heartbeat_url")
events = (await db.scalars(select(AuditEvent).where(AuditEvent.action == "system_settings"))).all()
assert row.value is None # the database only knows that it is set
assert "secret-id" not in str([e.details for e in events])
await client.put("/api/settings/system", json={"values": {"heartbeat_url": ""}})
assert rs.value("heartbeat_url") == settings.heartbeat_url
assert notification_secret_store.read_secret("runtime-heartbeat_url") is None
@pytest.mark.asyncio
async def test_a_pasted_mail_ca_is_trusted_by_the_smtp_context(client):
pem = _ca_pem()
assert (await client.put("/api/settings/system", json={"values": {"smtp_ca_pem": pem}})).status_code == 200
ctx = rs.smtp_ssl_context()
assert any(c.get("subject") == ((("commonName", "Mail CA"),),) for c in ctx.get_ca_certs())
@pytest.mark.asyncio
async def test_another_process_sees_the_value_after_its_refresh(client):
async with SessionLocal() as db:
db.add(RuntimeSetting(key="login_lockout_seconds", value=900, updated_by="other-node"))
await db.commit()
assert rs.value("login_lockout_seconds") == settings.login_lockout_seconds # not yet: cache
await rs.refresh()
assert rs.value("login_lockout_seconds") == 900
@pytest.mark.asyncio
async def test_login_throttle_follows_the_saved_limit(client, anon_client):
await client.put("/api/settings/system", json={"values": {"login_max_attempts": 3}})
statuses = []
for _ in range(4):
statuses.append((await anon_client.post("/api/auth/login", json={"username": settings.admin_username,
"password": "wrong-password"})).status_code)
assert statuses[-1] == 429 and statuses.count(401) == 3
# ── server disk alert ────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_server_disk_alert_follows_the_threshold(client, monkeypatch):
monkeypatch.setattr(notification_checks, "server_disk_usage", lambda: (91.5, 915, 1000))
async with SessionLocal() as db:
problems = await notification_checks.check_server_disk(db)
assert problems["server"]["belegt_prozent"] == 91.5 and problems["server"]["schwelle_prozent"] == 90
resp = await client.put("/api/notifications/settings", json={"server_disk_threshold_pct": 95})
assert resp.status_code == 200 and resp.json()["server_disk_threshold_pct"] == 95
async with SessionLocal() as db:
assert await notification_checks.check_server_disk(db) == {}
assert (await client.put("/api/notifications/settings", json={"server_disk_threshold_pct": 20})).status_code == 422
monkeypatch.setattr(notification_checks, "server_disk_usage", lambda: None)
async with SessionLocal() as db:
unknown = await notification_checks.check_server_disk(db)
assert unknown == {} and unknown.preserve_prefixes == {"server"}
@pytest.mark.asyncio
async def test_server_disk_metric(client, monkeypatch):
monkeypatch.setattr(notification_checks, "server_disk_usage", lambda: (81.25, 8125, 10000))
assert "opnexus_server_disk_used_percent 81.2" in (await client.get("/api/metrics")).text
+11
View File
@@ -117,10 +117,14 @@ services:
TRUST_PROXY_HEADERS: ${TRUST_PROXY_HEADERS:-false}
UPDATE_CHECK_URL: ${UPDATE_CHECK_URL:-}
OPNEXUS_IMAGE_REPO: ${OPNEXUS_IMAGE_REPO:-}
# 1 = Backend wie früher als root betreiben (Notausgang, siehe backend/scripts/start.sh).
OPNEXUS_RUN_AS_ROOT: ${OPNEXUS_RUN_AS_ROOT:-0}
ALLOW_INSECURE_FIREWALL_URLS: ${ALLOW_INSECURE_FIREWALL_URLS:-false}
ALLOW_INSECURE_SMTP_AUTH: ${ALLOW_INSECURE_SMTP_AUTH:-false}
SMTP_CA_FILE: ${SMTP_CA_FILE:-}
UPDATE_CHECK_INTERVAL_HOURS: ${UPDATE_CHECK_INTERVAL_HOURS:-12}
# Optionaler Heartbeat an einen externen Überwacher (nur https://), siehe Handbuch.
HEARTBEAT_URL: ${HEARTBEAT_URL:-}
# Optionaler Updater-Dienst (Profil "updater", siehe unten). Ohne ihn
# zeigt das Webinterface nur an, wie man ihn aktiviert.
UPDATER_URL: ${UPDATER_URL:-http://updater:8080}
@@ -146,6 +150,9 @@ services:
# Zugriffstoken des Updaters (nur lesen). Leer, solange der Updater
# nicht aktiviert ist.
- updater_state:/updater:ro
# Das Backend startet als root nur, um die Volumes umzustellen, und läuft dann
# als uid 10001 (backend/scripts/start.sh). Neue Rechte kann es nicht mehr erlangen.
security_opt: ["no-new-privileges:true"]
# Bound to loopback only by default — reachable from this host, not
# the network. Set BIND_ADDRESS in .env (e.g. to 0.0.0.0, or a
# specific interface IP) for network-wide access instead of editing
@@ -173,6 +180,10 @@ services:
# not via loopback) but breaks any in-container check (e.g. the
# healthcheck below) that queries localhost.
HOSTNAME: "0.0.0.0"
# Läuft als unprivilegierter Benutzer (frontend/Dockerfile) und braucht keine
# Capabilities.
cap_drop: ["ALL"]
security_opt: ["no-new-privileges:true"]
depends_on:
backend:
condition: service_healthy
+107 -5
View File
@@ -225,6 +225,26 @@ Internet ist das nicht geeignet.
---
## Betriebseinstellungen im Webinterface
Unter **Einstellungen › Betriebseinstellungen** stellst du ein, wofür früher die `.env` des Servers nötig war:
| Bereich | Einstellungen |
|---|---|
| Anmeldung und Sitzungen | Sitzungsdauer, Fehlversuche je Konto und je Absender, Zeitfenster, Sperrdauer |
| Ausrollen und Firewall-Zugriff | Bestätigungsfrist beim Ausrollen, Zeitlimit für Firewall-Anfragen, Sicherungen vor Updates |
| Messungen und Aufbewahrung | Messintervall Systemauslastung, Aufbewahrung der Messwerte, Intervall Regelverlauf, Intervall der Versionsprüfung |
| Überwachung von außen und Mail | Heartbeat-Adresse, CA des Mailservers (PEM einfügen), Zeitlimit Mailversand |
Jedes Feld nennt Standard und erlaubten Bereich; „Auf Standard zurücksetzen“ nimmt eine Änderung zurück. Änderungen
gelten sofort (spätestens nach 15 Sekunden auch in einem zweiten Backend) und stehen im Audit-Log; die Heartbeat-
Adresse und die CA nur als „gesetzt“. Ändern dürfen nur Admins.
Steht ein Wert **in der `.env`**, gilt weiterhin dieser; das Feld ist dann gesperrt und zeigt „vom Server“. So bleiben
automatisiert eingerichtete Server steuerbar. Bewusst **nur** in der `.env` bleiben: Adressen und TLS von OPNexus
selbst (ein falscher Wert sperrt dich aus genau dieser Oberfläche aus), der Datenbank-Zugang, das erste Admin-Konto,
die Zusatzdienste (`COMPOSE_PROFILES`) und die Notausgänge (`ALLOW_INSECURE_*`, `OPNEXUS_RUN_AS_ROOT`).
## Konfiguration (`.env`)
Alle Einstellungen stehen in der `.env` neben der `docker-compose.yml`. Nach einer
@@ -243,10 +263,12 @@ Alle Einstellungen stehen in der `.env` neben der `docker-compose.yml`. Nach ein
| `BIND_ADDRESS` | Schnittstelle für Backend und Frontend (Standard `127.0.0.1`). |
| `BACKEND_PORT`, `FRONTEND_PORT` | Host-Ports (Standard 8100 und 3100), z. B. für eine zweite Instanz. |
| `OPNEXUS_IMAGE_REPO`, `OPNEXUS_VERSION` | Registry und Version der Release-Images. `OPNEXUS_VERSION` ist die „Spur“, siehe [Updates](#opnexus-aktualisieren). |
| `UPDATE_CHECK_URL`, `UPDATE_CHECK_INTERVAL_HOURS` | Hinweis auf neue Versionen, standardmäßig **aus**. Einstellbar auch im Webinterface; ein Wert in der `.env` hat Vorrang. |
| `UPDATE_CHECK_URL`, `UPDATE_CHECK_INTERVAL_HOURS` | Hinweis auf neue Versionen, standardmäßig **aus**. Beides auch im Webinterface (Versionsprüfung bzw. Betriebseinstellungen). Die Adresse in der `.env` hat Vorrang; das Intervall dort ist nur der Standard, den das Webinterface überschreiben kann. |
| `COMPOSE_PROFILES=updater` | Aktiviert den optionalen Updater für Updates per Klick. |
| `HEARTBEAT_URL` | Heartbeat-Adresse, siehe [Ausfall von OPNexus bemerken](#ausfall-von-opnexus-bemerken-heartbeat). Einfacher im Webinterface (Betriebseinstellungen); ein Wert hier hat Vorrang. |
| `OPNEXUS_RUN_AS_ROOT` | `1` lässt das Backend wie in älteren Versionen als root laufen. Normalerweise läuft es als unprivilegierter Benutzer (uid 10001) und stellt seine Datenvolumes beim Start selbst darauf um. Nur für Sonderfälle, etwa Datenverzeichnisse, die sich nicht umstellen lassen. |
| `ALLOW_INSECURE_FIREWALL_URLS` | Erlaubt `http://`-Adressen für Firewalls. Nur für isolierte Labore. |
| `SMTP_CA_FILE`, `ALLOW_INSECURE_SMTP_AUTH` | Eigene CA für den Mailserver bzw. SMTP-Anmeldung ohne Verschlüsselung (bleibt standardmäßig gesperrt). |
| `SMTP_CA_FILE`, `ALLOW_INSECURE_SMTP_AUTH` | Eigene CA für den Mailserver (einfacher: PEM in den Betriebseinstellungen einfügen; die Datei hier hat Vorrang) bzw. SMTP-Anmeldung ohne Verschlüsselung (bleibt standardmäßig gesperrt). |
---
@@ -1700,6 +1722,38 @@ mit den Rechten des Tokens: ein „Lesen“-Token sieht auch hier keine Geheimni
| `caddy_data`, `caddy_config` | Schlüssel der internen CA und Proxy-Konfiguration |
| `updater_state` | Token des Updaters (nur mit aktivem Updater) |
### Ausfall von OPNexus bemerken (Heartbeat)
Fällt OPNexus aus, schweigt auch jeder Alarm, den es senden würde. Dagegen hilft ein Überwacher außerhalb: Trage
unter **Einstellungen › Betriebseinstellungen › Überwachung von außen und Mail** die Ping-Adresse eines solchen
Dienstes ein (healthchecks.io oder selbst gehostet, ein Push-Monitor in Uptime Kuma, ein ntfy-Thema). OPNexus ruft
sie nach **jedem erfolgreichen Prüfzyklus** der Benachrichtigungen per `GET` auf, also etwa einmal pro Minute.
Bleibt der Aufruf aus, ist OPNexus ausgefallen oder erreicht seine Datenbank nicht mehr; den Alarm schlägt dann der
Überwacher. Stell dessen Karenzzeit auf ein Mehrfaches des Prüfintervalls (z. B. 5 Minuten).
Erlaubt ist nur `https://`; lokale Adressen, Link-Local und Metadaten-Dienste sind wie bei den Benachrichtigungen
ausgeschlossen. Die Adresse enthält meist einen geheimen Schlüssel: sie wird nach dem Speichern nicht mehr angezeigt
und nie protokolliert. Alternativ geht `HEARTBEAT_URL` in der `.env` (hat dann Vorrang).
**Platte des OPNexus-Servers.** Läuft die Platte voll, auf der OPNexus läuft, bleibt die Datenbank stehen und mit ihr
jeder Alarm. Der Alarm „Platte des OPNexus-Servers fast voll“ (Einstellungen › Benachrichtigungen, standardmäßig an,
ab 90 %) warnt vorher; die Metrik `opnexus_server_disk_used_percent` zeigt den Füllstand. Gemessen wird das
Dateisystem der Docker-Volumes (bei einer Standardinstallation dieselbe Platte wie die Datenbank).
### Mehrere Backends an einer Datenbank
Die Hintergrunddienste (Rücknahme unbestätigter Änderungen, Benachrichtigungen, Log-Abruf, Messwerte, geplante
Wartung und Rollouts, Sicherheitslage, Audit-Export, geplante Sicherungen, Regelverlauf) laufen immer nur in
**einem** Backend-Prozess, dem *Leader*. Er hält dafür eine Sperre in der Datenbank. Ein zweites Backend an
derselben Datenbank (etwa während eines Umzugs oder später als Reserve) beantwortet die API normal, wartet mit den
Hintergrunddiensten aber in *Bereitschaft*. Fällt der Leader aus oder verliert er die Verbindung zur Datenbank,
hält er seine Dienste sofort an; ein wartendes Backend übernimmt innerhalb einer halben Minute. An einer Datenbank, die
gerade Replica ist, startet nie ein Leader.
Welche Rolle ein Backend hat, zeigen `GET /api/health` (`"role": "leader"`, `"standby"`, `"takeover"`, `"replica"`)
und die Metrik `opnexus_leader`. Für eine einzelne Installation ändert sich nichts: ihr einziges Backend ist der
Leader, die Hintergrunddienste starten nach dem Hochfahren mit etwa 12 Sekunden Verzögerung.
### Sichern und Wiederherstellen
**Automatisch:** Der Dienst `postgres-backup` schreibt beim Start und danach alle
@@ -1714,9 +1768,47 @@ Konfigurationssicherungen) in **eine mit Passphrase verschlüsselte Datei** (`.o
wird abgefragt (oder `OPNEXUS_BACKUP_PASSPHRASE`). Auf einer frischen Installation (gleiche oder
neuere Version) spielt `./opnexus restore DATEI` alles zurück: Es prüft die Datei, fragt nach, stoppt die
Dienste, legt eine Sicherheitskopie des jetzigen Zustands an, spielt Datenbank und Volumes ein und
startet neu. Nicht enthalten: Log-Erfassung, Messwerte und `caddy_data` (die interne CA: nach einer
Wiederherstellung auf neuer Hardware importierst du das CA-Zertifikat im Browser erneut).
Datei und Passphrase getrennt vom Server aufbewahren.
startet neu. Nicht enthalten: Log-Erfassung und Messwerte. Datei und Passphrase getrennt vom Server aufbewahren.
Mit `--with-env` und `--with-ca` nimmt die Vollsicherung zusätzlich die `.env` und die interne CA des Proxys
(`caddy_data`) mit; `./opnexus restore DATEI --with-env --with-ca` spielt sie zurück. Von der `.env` übernimmt der
Restore alles **außer** den Datenbank-Zugangsdaten, der Version und den Adressen des Zielservers (`POSTGRES_*`,
`OPNEXUS_VERSION`, `OPNEXUS_IMAGE_REPO`, `OPNEXUS_SITES`, `OPNEXUS_DEFAULT_SNI`, `FRONTEND_ORIGIN`, `*_BIND_ADDRESS`,
`OPNEXUS_GUI_SITES`); er zeigt die übernommenen Namen und fragt nach. Die gesicherte Datei liegt danach vollständig
als `.env.from-backup` daneben, die vorherige als `.env.before-restore`. Mit der CA vertrauen Browser, die ihr schon
vertrauen, auch dem neuen Server. `./opnexus status` zeigt, wann die letzte Vollsicherung entstand.
### Umzug und Notfall (Kaltreserve)
Fällt der OPNexus-Server aus, laufen die Firewalls unverändert weiter; nur Verwaltung, Alarme und geplante
Wartungen stehen still. Für diesen Fall hält man eine **Kaltreserve**: eine Vollsicherung mit `.env` und CA und
einen zweiten Server, auf dem OPNexus installiert werden kann.
**Notfall (der alte Server ist weg):** neuen Server nach [Installation aus dem Release-Bundle](#installation-aus-dem-release-bundle)
mit gleicher oder neuerer Version einrichten, dann `./opnexus restore DATEI --with-env --with-ca`. Danach die Adresse
(virtuelle IP, DNS-Name) auf den neuen Server zeigen lassen. Unbestätigte Änderungen, deren Frist während des Ausfalls
abgelaufen ist, nimmt der neue Server sofort zurück: im Zweifel gilt der alte Stand.
**Geplanter Umzug (beide Server laufen):** am alten Server
```bash
./opnexus migrate-host root@neuer-server # --dir PFAD, wenn OPNexus dort woanders liegt
```
Voraussetzungen: Anmeldung per SSH-Schlüssel am Ziel, dort OPNexus installiert und gestartet (gleiche oder neuere
Version). Das Werkzeug prüft das Ziel, hält den alten Server an (ab da keine Änderungen mehr), erstellt eine
Vollsicherung mit `.env` und CA (zufällige Passphrase, die nur über die SSH-Verbindung geht), spielt sie am Ziel
zurück, vergleicht die Zahl der Firewalls und meldet die Dauer. Im Lab dauerte ein Umzug mit drei Firewalls
**90 Sekunden** von der Vorprüfung bis zum gesunden Ziel; der Rückweg auf den alten Server gut 30 Sekunden. Der alte Server bleibt angehalten, nicht gelöscht: zurück geht es mit `docker compose stop` am
Ziel und `docker compose up -d` am alten Server.
**Danach zu prüfen:**
- Firewalls, die ihre API nur bestimmten Absendern öffnen, müssen die Adresse des neuen Servers erlauben.
- Mit [Management-Overlay](#firewalls-hinter-nat-management-tunnel) wählen sich Firewalls über eine öffentliche
Adresse ein; sie muss auf den neuen Server zeigen (die Schlüssel kommen mit der Sicherung mit).
- Geöffnete Fenster der Firewall-Oberfläche über OPNexus brechen ab und werden neu geöffnet.
**Was nicht im Dump ist:**
- Die Tabellen der **Log-Erfassung** und der **Systemauslastung** (nur das Schema).
@@ -1766,6 +1858,16 @@ Patch-Releases, `1` folgt allen 1.x. Ohne Argument holt `opnexus update` die neu
Version der Spur, mit Argument wird die Spur umgestellt. Nach einem Rollback steht
die Version fest auf dem alten Stand.
**Die Compose-Datei kommt mit (ab 1.34).** `opnexus update` übernimmt auch die `docker-compose.yml` des Zielreleases,
damit neue Einstellungen der Dienste ankommen. Sie stammt aus dem signierten Updater-Image (nur mit `cosign` und
`cosign.pub`, wie die CLI selbst). Die bisherige Datei bleibt als `docker-compose.yml.bak-<version>` liegen, ein
Rollback legt sie zurück. Hast du die Datei **selbst angepasst** (sie weicht von der zuletzt übernommenen ab), bleibt
sie unverändert; die neue Fassung liegt dann als `docker-compose.yml.new` daneben. Eigene Anpassungen gehören deshalb
in eine `docker-compose.override.yml` im selben Verzeichnis: Compose liest sie automatisch mit, und Updates können die
Hauptdatei weiter austauschen. Beim ersten Update mit dieser Funktion ist noch keine Prüfsumme bekannt; dann wird
getauscht und `diff docker-compose.yml.bak-<version> docker-compose.yml` zeigt, was sich geändert hat.
`OPNEXUS_NO_COMPOSE_REFRESH=1` schaltet das Übernehmen ab.
**Rollback mit Datenbank-Wiederherstellung** bedeutet: alles steht auf dem Stand vor
dem Update, was danach geschrieben wurde, ist weg, und Log-Historie und
Messwerte sind leer. Vorher legt das Werkzeug eine Sicherheitskopie des jetzigen
+4
View File
@@ -23,5 +23,9 @@ COPY --from=builder /app/.next/static ./.next/static
# next/image nicht, sie wären nur totes Gewicht.
RUN if [ "$TARGETARCH" != "$BUILDARCH" ]; then rm -rf node_modules/@img node_modules/sharp; fi
# Läuft als unprivilegierter Benutzer des Node-Images (uid 1000); die App schreibt
# nichts in ihr eigenes Verzeichnis.
USER node
EXPOSE 3000
CMD ["node", "server.js"]
+9 -1
View File
@@ -182,7 +182,7 @@ export default function DashboardPage() {
};
}, [errorMessage, router]);
const healthProblems = (problems ?? []).filter((p) => ["host_offline", "cert_expiry", "rule_drift", "gateway_down", "resource_threshold", "service_down", "api_rights_missing"].includes(p.problem_type));
const healthProblems = (problems ?? []).filter((p) => ["host_offline", "cert_expiry", "rule_drift", "gateway_down", "resource_threshold", "service_down", "api_rights_missing", "server_disk"].includes(p.problem_type));
const gatewayOrder: Record<string, number> = { offline: 0, degraded: 1, unknown: 2, forced_down: 3, online: 4 };
const gatewayRows = (summary?.hosts ?? [])
.flatMap((host) => (host.gateways ?? []).map((gateway) => ({ host, gateway })))
@@ -354,6 +354,14 @@ export default function DashboardPage() {
<Badge tone="warning" className="shrink-0">{t("dashboard.alert.api_rights_missing")}</Badge>
</Link>
)}
{problem.problem_type === "server_disk" && (
<Link href="/settings" className="flex items-center justify-between gap-3 hover:underline">
<span className="min-w-0 truncate font-medium text-slate-900">
{t("dashboard.serverDiskLine", { pct: String(problem.context.belegt_prozent), free: String(problem.context.frei_gb) })}
</span>
<Badge tone="danger" className="shrink-0">{t("dashboard.alert.server_disk")}</Badge>
</Link>
)}
{problem.problem_type === "cert_expiry" && (
<Link href="/certificates" className="flex items-center justify-between gap-3 hover:underline">
<span className="min-w-0 truncate font-medium text-slate-900">
+11 -9
View File
@@ -401,25 +401,27 @@ export default function HostDetailPage() {
<tbody>
{backups.map((backup, index) => (
<tr key={backup.id} className="border-b border-slate-100 last:border-0">
<td className="px-5 py-3 text-slate-900">{new Date(backup.created_at).toLocaleString(locale)}</td>
<td className="px-5 py-3 text-slate-500">{formatSize(backup.size)}</td>
<td className="px-5 py-3 text-right">
<td className="px-3 py-3 text-slate-900 sm:px-5">{new Date(backup.created_at).toLocaleString(locale)}</td>
<td className="px-3 py-3 text-slate-500 sm:px-5">{formatSize(backup.size)}</td>
<td className="px-3 py-3 sm:px-5">
<div className="flex justify-end gap-x-1">
{index + 1 < backups.length && (
<Button variant="ghost" onClick={() => setDiffTarget({ older: backups[index + 1].id, newer: backup.id })}>
<Button variant="ghost" className="!px-2 sm:!px-3.5" title={t("hostDetail.changes")} aria-label={t("hostDetail.changes")} onClick={() => setDiffTarget({ older: backups[index + 1].id, newer: backup.id })}>
<GitCompare size={14} />
{t("hostDetail.changes")}
<span className="hidden sm:inline">{t("hostDetail.changes")}</span>
</Button>
)}
{!readOnly && (
<Button variant="ghost" onClick={() => handleDownloadBackup(backup.id)}>
<Button variant="ghost" className="!px-2 sm:!px-3.5" title={t("hostDetail.download")} aria-label={t("hostDetail.download")} onClick={() => handleDownloadBackup(backup.id)}>
<Download size={14} />
{t("hostDetail.download")}
<span className="hidden sm:inline">{t("hostDetail.download")}</span>
</Button>
)}
<Button variant="ghost" onClick={() => setRestoreTarget(backup.id)}>
<Button variant="ghost" className="!px-2 sm:!px-3.5" title={t("hostDetail.restore")} aria-label={t("hostDetail.restore")} onClick={() => setRestoreTarget(backup.id)}>
<RotateCcw size={14} />
{t("hostDetail.restore")}
<span className="hidden sm:inline">{t("hostDetail.restore")}</span>
</Button>
</div>
</td>
</tr>
))}
+10 -1
View File
@@ -14,6 +14,7 @@ import { OidcSection } from "@/components/OidcSection";
import { CleanupSettings } from "@/components/CleanupSettings";
import { BackupSettingsSection } from "@/components/BackupSettingsSection";
import { GuiAccessSection } from "@/components/GuiAccessSection";
import { SystemSettingsSection } from "@/components/SystemSettingsSection";
import { AuditExportSection } from "@/components/AuditExportSection";
import { APP_CODENAME, APP_MAINTAINER, APP_VERSION } from "@/lib/version";
import { translate } from "@/lib/i18n";
@@ -74,6 +75,8 @@ function NotificationsSection() {
const [serviceDown, setServiceDown] = useState(false);
const [rolloutProblems, setRolloutProblems] = useState(true);
const [apiRights, setApiRights] = useState(true);
const [serverDisk, setServerDisk] = useState(true);
const [serverDiskLimit, setServerDiskLimit] = useState(90);
const [gatewayLoss, setGatewayLoss] = useState(20);
const [gatewayDelay, setGatewayDelay] = useState(500);
const [diskLimit, setDiskLimit] = useState(90);
@@ -113,6 +116,8 @@ function NotificationsSection() {
setServiceDown(data.service_down_enabled);
setRolloutProblems(data.rollout_problems_enabled ?? true);
setApiRights(data.api_rights_enabled ?? true);
setServerDisk(data.server_disk_enabled ?? true);
setServerDiskLimit(data.server_disk_threshold_pct ?? 90);
setGatewayLoss(data.gateway_loss_threshold_pct);
setGatewayDelay(data.gateway_delay_threshold_ms);
setDiskLimit(data.disk_threshold_pct);
@@ -170,6 +175,8 @@ function NotificationsSection() {
service_down_enabled: serviceDown,
rollout_problems_enabled: rolloutProblems,
api_rights_enabled: apiRights,
server_disk_enabled: serverDisk,
server_disk_threshold_pct: serverDiskLimit,
gateway_loss_threshold_pct: gatewayLoss,
gateway_delay_threshold_ms: gatewayDelay,
disk_threshold_pct: diskLimit,
@@ -318,6 +325,7 @@ function NotificationsSection() {
[t("settings.notify.event.serviceDown"), serviceDown, setServiceDown],
[t("settings.notify.event.rolloutProblems"), rolloutProblems, setRolloutProblems],
[t("settings.notify.event.apiRights"), apiRights, setApiRights],
[t("settings.notify.event.serverDisk"), serverDisk, setServerDisk],
].map(([label, value, setter]) => (
<label key={label as string} className="flex items-center gap-2 text-sm">
<input type="checkbox" checked={value as boolean} onChange={(e) => (setter as (v: boolean) => void)(e.target.checked)} />
@@ -343,6 +351,7 @@ function NotificationsSection() {
["loadLimit", loadLimit, setLoadLimit, 0, 1000],
["confirmSamples", confirmSamples, setConfirmSamples, 1, 60],
["resourceSamples", resourceSamples, setResourceSamples, 1, 60],
["serverDiskLimit", serverDiskLimit, setServerDiskLimit, 50, 99],
] as [string, number, (v: number) => void, number, number][]).map(([key, value, setter, min, max]) => (
<label key={key} className="block text-sm">
<span className="mb-1 block font-medium text-slate-700">{t(`settings.notify.limit.${key}`)}</span>
@@ -482,5 +491,5 @@ export default function SettingsPage(){
async function saveTheme(value:Theme){setSaving(true);setError('');setNotice('');const old=theme;setTheme(value);try{setSettings(await api.saveSettings({theme:value,language}));setNotice(t('settings.appearanceSaved'));}catch(e){setTheme(old);setError(errorMessage(e));}finally{setSaving(false);}}
async function saveLanguage(value:"de"|"en"){setSaving(true);setError('');setNotice('');const old=language;setLanguage(value);try{setSettings(await api.saveSettings({theme,language:value}));setNotice(translate(value,'settings.language.saved'));}catch(e){setLanguage(old);setError(errorMessage(e));}finally{setSaving(false);}}
async function changePassword(e:React.FormEvent){e.preventDefault();setError('');setNotice('');if(password!==repeat){setError(t('settings.passwordMismatch'));return;}setSaving(true);try{await api.changePassword(current,password);setCurrent('');setPassword('');setRepeat('');setNotice(t('settings.passwordChanged'));}catch(e){setError(errorMessage(e,'settings.passwordFailed'));}finally{setSaving(false);}}
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><section className="space-y-3 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.operational')}<InfoTip text={t('settings.serverManaged')} /></h2>{settings?<dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.deployWindow')}</dt><dd>{t('settings.seconds',{n:settings.deploy_confirm_window_seconds})}</dd><dt>{t('settings.sessionLifetime')}</dt><dd>{t('settings.days',{n:Math.round(settings.session_lifetime_seconds/86400)})}</dd></dl>:<p className="text-sm text-slate-500">{t('common.loading')}</p>}</section><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><CleanupSettings/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
return <div className="mx-auto max-w-3xl space-y-6"><h1 className="text-2xl font-semibold">{t("settings.title")}</h1>{error && <p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">{error}</p>}{notice && <p role="status" className="text-sm text-emerald-700">{notice}</p>}<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.appearance')}<InfoTip text={t('settings.appearanceHint')} /></h2><div role="radiogroup" aria-label={t('settings.colorScheme')} className="flex flex-wrap gap-3">{(['light','dark','system'] as const).map((v)=><label key={v} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${theme===v?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="theme" checked={theme===v} disabled={saving || !settings} onChange={()=>saveTheme(v)}/>{t(`settings.theme.${v}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.language.label')}</h2><div role="radiogroup" aria-label={t('settings.language.label')} className="flex flex-wrap gap-3">{(['de','en'] as const).map(value=><label key={value} className={`flex cursor-pointer items-center gap-2 rounded-lg border px-4 py-3 text-sm ${language===value?'border-sky-500 bg-sky-50 text-sky-700':'border-slate-200'}`}><input type="radio" name="language" checked={language===value} disabled={saving || !settings} onChange={()=>saveLanguage(value)}/>{t(`settings.language.${value}`)}</label>)}</div></section><section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5"><h2 className="text-lg font-medium">{t('settings.account')} {settings?.username ? `· ${settings.username}`:''}</h2><form onSubmit={changePassword} className="space-y-4"><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.currentPassword')}</span><Input aria-label={t('settings.currentPassword')} type="password" autoComplete="current-password" required value={current} onChange={e=>setCurrent(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.newPassword')}</span><Input aria-label={t('settings.newPassword')} type="password" autoComplete="new-password" minLength={10} maxLength={1024} required value={password} onChange={e=>setPassword(e.target.value)}/></label><label className="block text-sm"><span className="mb-1 block font-medium text-slate-700">{t('settings.repeatPassword')}</span><Input aria-label={t('settings.repeatPassword')} type="password" autoComplete="new-password" minLength={10} required value={repeat} onChange={e=>setRepeat(e.target.value)}/></label><Button type="submit" disabled={saving}>{t('settings.changePassword')}</Button></form></section><SystemSettingsSection/><SetupAssistantSection/><TwoFactorSection/><NotificationsSection/><LogIngestSection/><OidcSection/><CleanupSettings/><BackupSettingsSection/><AuditExportSection/><GuiAccessSection/><OpnexusUpdateSection/><section className="space-y-2 rounded-xl border border-slate-200 bg-white p-5"><h2 className="flex items-center gap-2 text-lg font-medium">{t('settings.about')}<InfoTip text={t('settings.changelogNote',{file:'CHANGELOG.md'})} /></h2><dl className="grid grid-cols-2 gap-3 text-sm"><dt>{t('settings.version')}</dt><dd>{APP_VERSION} {language==='en'?'“':'„'}{APP_CODENAME}{language==='en'?'”':'“'}</dd><dt>{t('settings.development')}</dt><dd>{APP_MAINTAINER}</dd></dl></section></div>;
}
@@ -0,0 +1,229 @@
"use client";
import { Badge } from "@/components/ui/Badge";
import { Button } from "@/components/ui/Button";
import { Input } from "@/components/ui/Field";
import { useToast } from "@/components/ui/Toast";
import { InfoTip } from "@/components/ui/Tooltip";
import { useLanguage } from "@/components/LanguageProvider";
import { api, SystemSetting } from "@/lib/api";
import { useNotAdmin } from "@/lib/useReadOnly";
import { RotateCcw } from "lucide-react";
import { useEffect, useMemo, useState } from "react";
const GROUPS = ["security", "operation", "monitoring", "integrations"] as const;
type Draft = Record<string, string>;
function draftOf(items: SystemSetting[]): Draft {
const draft: Draft = {};
for (const item of items) {
draft[item.key] = item.kind === "secret_url" ? "" : item.value === null ? "" : String(item.value);
}
return draft;
}
/** Settings › Betrieb: everything that used to need the server's .env and is safe to change while OPNexus runs. */
export function SystemSettingsSection() {
const { t, errorMessage } = useLanguage();
const toast = useToast();
const readOnly = useNotAdmin();
const [items, setItems] = useState<SystemSetting[] | null>(null);
const [draft, setDraft] = useState<Draft>({});
const [saving, setSaving] = useState<string | null>(null);
const [error, setError] = useState("");
function apply(next: SystemSetting[]) {
setItems(next);
setDraft(draftOf(next));
}
useEffect(() => {
api.systemSettings().then((body) => apply(body.settings)).catch((e) => setError(errorMessage(e)));
}, [errorMessage]);
const byGroup = useMemo(() => {
const out: Record<string, SystemSetting[]> = {};
for (const item of items ?? []) (out[item.group] ??= []).push(item);
return out;
}, [items]);
function changed(item: SystemSetting): boolean {
if (item.source === "server") return false;
const value = draft[item.key] ?? "";
if (item.kind === "secret_url") return value.trim() !== "";
return value !== (item.value === null ? "" : String(item.value));
}
async function save(values: Record<string, number | string | null>, label: string) {
setSaving(label);
setError("");
try {
apply((await api.saveSystemSettings(values)).settings);
toast.show(t("settings.system.saved"), "success");
} catch (e) {
setError(errorMessage(e, "common.saveFailed"));
} finally {
setSaving(null);
}
}
function saveGroup(group: string) {
const values: Record<string, number | string | null> = {};
for (const item of byGroup[group] ?? []) {
if (!changed(item)) continue;
const raw = (draft[item.key] ?? "").trim();
values[item.key] = item.kind === "int" || item.kind === "float" ? (raw === "" ? null : Number(raw)) : raw;
}
if (Object.keys(values).length) void save(values, group);
}
function sourceBadge(item: SystemSetting) {
if (item.source === "server") {
return <Badge tone="warning">{t("settings.system.fromServer", { env: item.env ?? "" })}</Badge>;
}
if (item.source === "ui") return <Badge tone="info">{t("settings.system.changed")}</Badge>;
return null;
}
function field(item: SystemSetting) {
const label = t(`settings.system.field.${item.key}.label`);
const hint = t(`settings.system.field.${item.key}.hint`);
const locked = readOnly || item.source === "server";
const value = draft[item.key] ?? "";
const set = (v: string) => setDraft((d) => ({ ...d, [item.key]: v }));
const reset = item.source === "ui" && !readOnly && (
<Button
variant="ghost"
type="button"
title={t("settings.system.reset")}
aria-label={`${t("settings.system.reset")}: ${label}`}
disabled={saving !== null}
onClick={() => void save({ [item.key]: null }, item.key)}
>
<RotateCcw size={14} />
</Button>
);
const head = (
<span className="mb-1 flex flex-wrap items-center gap-1.5 font-medium text-slate-700">
{label}
<InfoTip text={hint} size={13} />
{sourceBadge(item)}
</span>
);
if (item.kind === "pem") {
return (
<div key={item.key} className="block text-sm sm:col-span-2">
{head}
{item.source === "server" ? (
<p className="text-slate-600">{t("settings.system.pemFromServer")}</p>
) : (
<textarea
aria-label={label}
className="h-28 w-full rounded-lg border border-slate-300 bg-white p-2 font-mono text-xs"
placeholder="-----BEGIN CERTIFICATE-----"
spellCheck={false}
disabled={locked}
value={value}
onChange={(e) => set(e.target.value)}
/>
)}
{reset && <div className="mt-1 flex justify-end">{reset}</div>}
</div>
);
}
if (item.kind === "secret_url") {
return (
<div key={item.key} className="block text-sm sm:col-span-2">
{head}
<div className="flex gap-2">
<Input
aria-label={label}
type="url"
inputMode="url"
spellCheck={false}
autoComplete="off"
disabled={locked}
placeholder={item.is_set ? t("settings.system.secretSet") : "https://"}
value={value}
onChange={(e) => set(e.target.value)}
/>
{item.is_set && item.source === "ui" && !readOnly && (
<Button variant="secondary" type="button" disabled={saving !== null} onClick={() => void save({ [item.key]: null }, item.key)}>
{t("settings.system.remove")}
</Button>
)}
</div>
</div>
);
}
const bounds = item.min !== null && item.max !== null ? t("settings.system.bounds", { min: item.min, max: item.max }) : "";
return (
<label key={item.key} className="block text-sm">
{head}
<div className="flex items-center gap-2">
<Input
type="number"
min={item.min ?? undefined}
max={item.max ?? undefined}
step={item.kind === "float" ? 0.5 : 1}
disabled={locked}
value={value}
onChange={(e) => set(e.target.value)}
/>
<span className="shrink-0 text-xs text-slate-500">{item.unit ? t(`settings.system.unit.${item.unit}`) : ""}</span>
{reset}
</div>
<span className="mt-1 block text-xs text-slate-500">
{item.default !== null && t("settings.system.defaultIs", { value: item.default })}
{bounds && ` · ${bounds}`}
</span>
</label>
);
}
return (
<section className="space-y-4 rounded-xl border border-slate-200 bg-white p-5">
<h2 className="flex items-center gap-2 text-lg font-medium">
{t("settings.operational")}
<InfoTip text={t("settings.system.intro")} />
</h2>
{!items && !error && <p className="text-sm text-slate-500">{t("common.loading")}</p>}
{items &&
GROUPS.filter((g) => byGroup[g]?.length).map((group) => {
const dirty = (byGroup[group] ?? []).some(changed);
return (
<form
key={group}
className="space-y-3 rounded-lg border border-slate-100 p-4"
onSubmit={(e) => {
e.preventDefault();
saveGroup(group);
}}
>
<p className="flex items-center gap-2 text-sm font-medium">
{t(`settings.system.group.${group}`)}
<InfoTip text={t(`settings.system.groupHint.${group}`)} />
</p>
<div className="grid gap-4 sm:grid-cols-2">{(byGroup[group] ?? []).map(field)}</div>
{!readOnly && (
<div className="flex justify-end border-t border-slate-100 pt-3">
<Button type="submit" disabled={!dirty || saving !== null}>
{t("common.save")}
</Button>
</div>
)}
</form>
);
})}
{error && (
<p role="alert" className="rounded-lg bg-red-50 p-3 text-sm text-red-700">
{error}
</p>
)}
</section>
);
}
+21 -1
View File
@@ -1298,6 +1298,20 @@ export interface UpdaterJob {
from_version: string | null;
}
export type SystemSetting = {
key: string;
group: "security" | "operation" | "monitoring" | "integrations";
kind: "int" | "float" | "secret_url" | "pem";
unit: string;
min: number | null;
max: number | null;
value: number | string | null;
default: number | null;
/** default: built-in value; ui: changed here; server: set in the server's .env (read-only here) */
source: "default" | "ui" | "server";
env: string | null;
is_set?: boolean;
};
export type UpdateCheckSettings = { enabled: boolean; feed_url: string; managed_by_server: boolean };
export interface UpdaterStatus {
/** false = Updater nicht aktiviert/erreichbar (`reason` sagt warum). */
@@ -1394,6 +1408,8 @@ export const api = {
audit: (filters: Record<string,string | number> = {}) => request<AuditPage>(`/audit?${new URLSearchParams(Object.entries(filters).filter(([,v])=>v!=="").map(([k,v])=>[k,String(v)]))}`),
dashboard: () => request<DashboardSummary>("/dashboard"),
versionCheck: () => request<VersionCheck>("/system/version-check"),
systemSettings: () => request<{ settings: SystemSetting[] }>("/settings/system"),
saveSystemSettings: (values: Record<string, number | string | null>) => request<{ settings: SystemSetting[] }>("/settings/system", { method: "PUT", body: JSON.stringify({ values }) }),
updateCheck: () => request<UpdateCheckSettings>("/system/update-check"),
updateUpdateCheck: (body: { enabled?: boolean; feed_url?: string }) => request<UpdateCheckSettings>("/system/update-check", { method: "PUT", body: JSON.stringify(body) }),
updaterStatus: () => request<UpdaterStatus>("/system/updater"),
@@ -1969,6 +1985,8 @@ export interface NotificationSettings {
service_down_enabled: boolean;
rollout_problems_enabled: boolean;
api_rights_enabled: boolean;
server_disk_enabled: boolean;
server_disk_threshold_pct: number;
gateway_loss_threshold_pct: number;
gateway_delay_threshold_ms: number;
disk_threshold_pct: number;
@@ -2027,6 +2045,8 @@ export interface NotificationSettingsUpdate {
service_down_enabled?: boolean;
rollout_problems_enabled?: boolean;
api_rights_enabled?: boolean;
server_disk_enabled?: boolean;
server_disk_threshold_pct?: number;
gateway_loss_threshold_pct?: number;
gateway_delay_threshold_ms?: number;
disk_threshold_pct?: number;
@@ -2064,7 +2084,7 @@ export type ProblemType =
| "backup_offsite_failed"
| "compliance_finding"
| "gateway_down" | "resource_threshold" | "service_down" | "api_rights_missing"
| "rollout_paused" | "rollout_start_failed";
| "rollout_paused" | "rollout_start_failed" | "server_disk";
export interface CarpEvent {
id: string;
+105 -5
View File
@@ -72,7 +72,8 @@
"resourceThreshold": "Ressourcen-Schwelle überschritten (Festplatte, Speicher, Swap, CPU-Last; standardmäßig aus)",
"serviceDown": "Überwachter Dienst ausgefallen (standardmäßig aus)",
"rolloutProblems": "Rollout angehalten oder nicht automatisch gestartet",
"apiRights": "API-Rechte fehlen (Gateway- oder Dienststatus nicht lesbar)"
"apiRights": "API-Rechte fehlen (Gateway- oder Dienststatus nicht lesbar)",
"serverDisk": "Platte des OPNexus-Servers fast voll"
},
"pollInterval": "Prüfintervall (Sekunden)",
"pollHint": "Mindestens 60 Sekunden, um die Firewalls nicht unnötig zu belasten.",
@@ -88,7 +89,8 @@
"confirmSamples": "Gateways und Dienste: Messungen in Folge",
"resourceSamples": "Ressourcen: Messungen in Folge",
"watchlist": "Überwachte Dienste",
"watchlistHint": "OPNsense-Dienstnamen, kommagetrennt. OPNexus meldet einen Dienst nur, wenn er zuvor lief und jetzt nicht mehr. Absichtlich gestoppte Dienste hier entfernen."
"watchlistHint": "OPNsense-Dienstnamen, kommagetrennt. OPNexus meldet einen Dienst nur, wenn er zuvor lief und jetzt nicht mehr. Absichtlich gestoppte Dienste hier entfernen.",
"serverDiskLimit": "OPNexus-Server: Platte voll ab (%)"
}
},
"logs": {
@@ -245,6 +247,99 @@
"teams": "Webhook-Adresse des Workflows",
"matrix": "Zugriffstoken"
}
},
"system": {
"intro": "Diese Werte standen früher nur in der .env des Servers. Steht ein Wert dort, gilt er weiter und ist hier gesperrt. Adressen, TLS und Datenbank-Zugang bleiben bewusst in der .env: ein falscher Wert würde dich aus dieser Oberfläche aussperren.",
"saved": "Einstellungen gespeichert",
"changed": "geändert",
"fromServer": "vom Server ({env})",
"reset": "Auf Standard zurücksetzen",
"remove": "Entfernen",
"secretSet": "gesetzt (zum Ändern neue Adresse eingeben)",
"pemFromServer": "Wird vom Server über SMTP_CA_FILE vorgegeben.",
"defaultIs": "Standard: {value}",
"bounds": "erlaubt {min}–{max}",
"group": {
"security": "Anmeldung und Sitzungen",
"operation": "Ausrollen und Firewall-Zugriff",
"monitoring": "Messungen und Aufbewahrung",
"integrations": "Überwachung von außen und Mail"
},
"groupHint": {
"security": "Schutz vor erratenen Passwörtern und wie lange Anmeldungen gelten.",
"operation": "Sicherheitsnetz beim Ausrollen und Geduld mit langsamen Firewalls.",
"monitoring": "Wie oft OPNexus misst und wie lange es die Daten behält. Kürzere Intervalle bedeuten mehr Anfragen an die Firewalls.",
"integrations": "Heartbeat für einen externen Überwacher und Vertrauen in einen Mailserver mit eigener CA."
},
"unit": {
"seconds": "Sekunden",
"minutes": "Minuten",
"hours": "Stunden",
"days": "Tage",
"count": "Anzahl"
},
"field": {
"session_lifetime_days": {
"label": "Sitzungsdauer",
"hint": "Wie lange eine Anmeldung gültig bleibt. Gilt für neue Anmeldungen."
},
"login_max_attempts": {
"label": "Fehlversuche je Konto",
"hint": "So viele falsche Passwörter je Konto und Absender, dann wird gesperrt."
},
"login_ip_max_attempts": {
"label": "Fehlversuche je Absender",
"hint": "So viele falsche Anmeldungen von einer Adresse (über alle Konten), dann wird gesperrt."
},
"login_window_seconds": {
"label": "Zeitfenster für Fehlversuche",
"hint": "In diesem Zeitraum werden Fehlversuche gezählt."
},
"login_lockout_seconds": {
"label": "Sperrdauer",
"hint": "So lange bleibt eine Sperre nach zu vielen Fehlversuchen bestehen."
},
"deploy_confirm_window_seconds": {
"label": "Bestätigungsfrist beim Ausrollen",
"hint": "So lange hast du nach dem Ausrollen Zeit, eine Änderung zu bestätigen. Ohne Bestätigung nimmt OPNexus sie automatisch zurück (Schutz vor Aussperren). Gilt für neue Ausrollvorgänge."
},
"opnsense_timeout": {
"label": "Zeitlimit für Firewall-Anfragen",
"hint": "Wie lange OPNexus auf eine Antwort der Firewall-API wartet. Für langsame Verbindungen (Tunnel, Mobilfunk) erhöhen."
},
"migration_backup_keep": {
"label": "Sicherungen vor Updates behalten",
"hint": "Wie viele automatische Datenbank-Sicherungen vor Schema-Änderungen aufbewahrt werden."
},
"resource_poll_interval_seconds": {
"label": "Messintervall Systemauslastung",
"hint": "Wie oft Speicher, Last, Platte, Gateways und Dienste der Firewalls gemessen werden."
},
"resource_sample_retention_days": {
"label": "Messwerte aufbewahren",
"hint": "Wie lange die Messwerte der Systemauslastung gespeichert bleiben."
},
"rule_observe_interval_seconds": {
"label": "Intervall Regelverlauf",
"hint": "Wie oft die Regeln und Zähler der Firewalls für Verlauf und Regel-Analyse gelesen werden."
},
"update_check_interval_hours": {
"label": "Intervall der Versionsprüfung",
"hint": "Wie oft nach einer neuen OPNexus-Version gefragt wird (nur wenn die Versionsprüfung an ist)."
},
"heartbeat_url": {
"label": "Heartbeat-Adresse",
"hint": "Ping-Adresse eines externen Überwachers (healthchecks.io, Uptime Kuma, ntfy). OPNexus ruft sie nach jedem erfolgreichen Prüfzyklus auf; bleibt das aus, schlägt der Überwacher Alarm. Nur https://. Die Adresse wird nicht angezeigt und nie protokolliert."
},
"smtp_ca_pem": {
"label": "CA des Mailservers",
"hint": "Nur nötig, wenn der Mailserver (oder das TLS-Syslog des Audit-Exports) ein Zertifikat einer eigenen CA nutzt. Das CA-Zertifikat im PEM-Format einfügen."
},
"notification_smtp_timeout_seconds": {
"label": "Zeitlimit Mailversand",
"hint": "Wie lange OPNexus auf den Mailserver wartet."
}
}
}
},
"nav": {
@@ -373,10 +468,12 @@
"gateway_down": "Gateway",
"resource_threshold": "Ressource",
"service_down": "Dienst",
"api_rights_missing": "API-Rechte"
"api_rights_missing": "API-Rechte",
"server_disk": "OPNexus-Platte"
},
"moreGateways": "+ {count} weitere Gateways (auf der Seite der Firewall)",
"apiRightsLine": "{name}: API-Recht fehlt ({read})"
"apiRightsLine": "{name}: API-Recht fehlt ({read})",
"serverDiskLine": "Platte des OPNexus-Servers zu {pct} % belegt ({free} GB frei)"
},
"hosts": {
"add": "Host hinzufügen",
@@ -917,7 +1014,8 @@
"HOST_CONNECTION_FAILED": "Die Firewall konnte mit diesen Angaben nicht erreicht werden: {reason}",
"ZIEL_HOST_NACH_AUSROLLEN_FEST": "Der Ziel-Host lässt sich nach dem Ausrollen nicht mehr ändern. Lösche den Eintrag (das entfernt ihn auch von der Firewall) und lege ihn für den neuen Host neu an.",
"ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET": "Dieses ACME-Konto wird von Zertifikatsanfragen verwendet; den Host kannst du erst ändern, wenn keine Anfrage es mehr nutzt.",
"VPN_DEPLOY_LAEUFT_BEREITS": "Für dieses VPN läuft gerade ein Ausrollen oder Entfernen. Warte einen Moment und versuche es erneut."
"VPN_DEPLOY_LAEUFT_BEREITS": "Für dieses VPN läuft gerade ein Ausrollen oder Entfernen. Warte einen Moment und versuche es erneut.",
"SETTING_INVALID": "Einstellung „{setting}“: {reason}"
},
"workflow": {
"attention": "Fehler prüfen",
@@ -3041,12 +3139,14 @@
"badge": "API-Rechte fehlen: {reads}",
"hint": "Der API-Benutzer von OPNexus darf diese Messwerte nicht lesen, die Alarme dazu können nicht auslösen. In OPNsense beim Benutzer oder der Gruppe unter System ▸ Zugang ergänzen:",
"read": {
"base": "Grundwerte (Speicher, Last, Platte)",
"gateways": "Gateways",
"services": "Dienste",
"swap": "Swap",
"cpu": "CPU"
},
"acl": {
"base": "Lobby: Dashboard",
"gateways": "System: Gateways",
"services": "Status: Services",
"swap": "Lobby: Dashboard",
+105 -5
View File
@@ -72,7 +72,8 @@
"resourceThreshold": "Resource threshold exceeded (disk, memory, swap, CPU load; off by default)",
"serviceDown": "Watched service stopped (off by default)",
"rolloutProblems": "Rollout paused or not started automatically",
"apiRights": "API rights missing (gateway or service status not readable)"
"apiRights": "API rights missing (gateway or service status not readable)",
"serverDisk": "Disk of the OPNexus server almost full"
},
"pollInterval": "Check interval (seconds)",
"pollHint": "At least 60 seconds, so the firewalls are not queried needlessly.",
@@ -88,7 +89,8 @@
"confirmSamples": "Gateways and services: consecutive measurements",
"resourceSamples": "Resources: consecutive measurements",
"watchlist": "Watched services",
"watchlistHint": "OPNsense service names, comma-separated. OPNexus reports a service only if it ran before and has stopped now. Remove services you stopped on purpose."
"watchlistHint": "OPNsense service names, comma-separated. OPNexus reports a service only if it ran before and has stopped now. Remove services you stopped on purpose.",
"serverDiskLimit": "OPNexus server: disk full from (%)"
}
},
"logs": {
@@ -245,6 +247,99 @@
"teams": "Workflow webhook address",
"matrix": "Access token"
}
},
"system": {
"intro": "These values used to live only in the server's .env. A value set there still applies and is locked here. Addresses, TLS and database access deliberately stay in the .env: a wrong value would lock you out of this very interface.",
"saved": "Settings saved",
"changed": "changed",
"fromServer": "from server ({env})",
"reset": "Reset to default",
"remove": "Remove",
"secretSet": "set (enter a new URL to change it)",
"pemFromServer": "Set by the server through SMTP_CA_FILE.",
"defaultIs": "Default: {value}",
"bounds": "allowed {min}–{max}",
"group": {
"security": "Sign-in and sessions",
"operation": "Rollouts and firewall access",
"monitoring": "Sampling and retention",
"integrations": "External monitoring and mail"
},
"groupHint": {
"security": "Protection against guessed passwords and how long sign-ins last.",
"operation": "The safety net when rolling out and patience with slow firewalls.",
"monitoring": "How often OPNexus measures and how long it keeps the data. Shorter intervals mean more requests to the firewalls.",
"integrations": "Heartbeat for an external watcher and trust for a mail server with its own CA."
},
"unit": {
"seconds": "seconds",
"minutes": "minutes",
"hours": "hours",
"days": "days",
"count": "count"
},
"field": {
"session_lifetime_days": {
"label": "Session lifetime",
"hint": "How long a sign-in stays valid. Applies to new sign-ins."
},
"login_max_attempts": {
"label": "Failed attempts per account",
"hint": "This many wrong passwords per account and sender, then it is locked."
},
"login_ip_max_attempts": {
"label": "Failed attempts per sender",
"hint": "This many failed sign-ins from one address (across all accounts), then it is locked."
},
"login_window_seconds": {
"label": "Window for failed attempts",
"hint": "Failed attempts are counted within this period."
},
"login_lockout_seconds": {
"label": "Lockout duration",
"hint": "How long a lock lasts after too many failed attempts."
},
"deploy_confirm_window_seconds": {
"label": "Confirmation window when rolling out",
"hint": "How long you have after a rollout to confirm a change. Without confirmation OPNexus reverts it automatically (lockout protection). Applies to new rollouts."
},
"opnsense_timeout": {
"label": "Timeout for firewall requests",
"hint": "How long OPNexus waits for the firewall API to answer. Raise it for slow links (tunnels, mobile)."
},
"migration_backup_keep": {
"label": "Backups kept before updates",
"hint": "How many automatic database backups taken before schema changes are kept."
},
"resource_poll_interval_seconds": {
"label": "Resource sampling interval",
"hint": "How often memory, load, disk, gateways and services of the firewalls are measured."
},
"resource_sample_retention_days": {
"label": "Keep resource samples",
"hint": "How long resource samples are stored."
},
"rule_observe_interval_seconds": {
"label": "Rule history interval",
"hint": "How often the firewalls' rules and counters are read for the history and the rule analysis."
},
"update_check_interval_hours": {
"label": "Version check interval",
"hint": "How often OPNexus asks for a new version (only while the version check is on)."
},
"heartbeat_url": {
"label": "Heartbeat URL",
"hint": "Ping URL of an external watcher (healthchecks.io, Uptime Kuma, ntfy). OPNexus calls it after every successful check cycle; when the calls stop, the watcher raises the alarm. https:// only. The URL is not shown and never logged."
},
"smtp_ca_pem": {
"label": "Mail server CA",
"hint": "Only needed when the mail server (or the TLS syslog of the audit export) uses a certificate from a private CA. Paste the CA certificate in PEM format."
},
"notification_smtp_timeout_seconds": {
"label": "Mail timeout",
"hint": "How long OPNexus waits for the mail server."
}
}
}
},
"nav": {
@@ -373,10 +468,12 @@
"gateway_down": "Gateway",
"resource_threshold": "Resource",
"service_down": "Service",
"api_rights_missing": "API rights"
"api_rights_missing": "API rights",
"server_disk": "OPNexus disk"
},
"moreGateways": "+ {count} more gateways (on the firewall page)",
"apiRightsLine": "{name}: API right missing ({read})"
"apiRightsLine": "{name}: API right missing ({read})",
"serverDiskLine": "Disk of the OPNexus server {pct} % full ({free} GB free)"
},
"hosts": {
"add": "Add host",
@@ -917,7 +1014,8 @@
"HOST_CONNECTION_FAILED": "The firewall could not be reached with these details: {reason}",
"ZIEL_HOST_NACH_AUSROLLEN_FEST": "The target host can no longer be changed once the entry has been deployed. Delete the entry (this also removes it from the firewall) and create it again for the new host.",
"ACME_KONTO_WIRD_VON_ANFRAGEN_VERWENDET": "This ACME account is used by certificate requests; you can only change its host once no request uses it any more.",
"VPN_DEPLOY_LAEUFT_BEREITS": "A deployment or removal is already running for this VPN. Wait a moment and try again."
"VPN_DEPLOY_LAEUFT_BEREITS": "A deployment or removal is already running for this VPN. Wait a moment and try again.",
"SETTING_INVALID": "Setting “{setting}”: {reason}"
},
"workflow": {
"attention": "Check errors",
@@ -3041,12 +3139,14 @@
"badge": "API rights missing: {reads}",
"hint": "OPNexus's API user may not read these values, so the alerts based on them cannot fire. In OPNsense, add to the user or group under System > Access:",
"read": {
"base": "Basic readings (memory, load, disk)",
"gateways": "Gateways",
"services": "Services",
"swap": "Swap",
"cpu": "CPU"
},
"acl": {
"base": "Lobby: Dashboard",
"gateways": "System: Gateways",
"services": "Status: Services",
"swap": "Lobby: Dashboard",
+2 -2
View File
@@ -1,6 +1,6 @@
// Manually bumped whenever a notable change ships — mirrors the dated
// entries already kept in the project README/CHANGELOG.md, no build
// tooling involved. Versioning scheme: see CHANGELOG.md ("Versionierung").
export const APP_VERSION = "1.33.0";
export const APP_CODENAME = "Saker";
export const APP_VERSION = "1.34.0";
export const APP_CODENAME = "Griffon";
export const APP_MAINTAINER = "Julian Boldt";
+2
View File
@@ -47,6 +47,8 @@ version="${tag#v}"
src="$(mktemp -d)"; trap 'rm -rf "$src"' EXIT
git archive "$tag" | tar -x -C "$src"
(cd "$src" && scripts/check_version.sh "${tag%%-*}")
# Die Release-Fassung der Compose-Datei ins Updater-Image: `opnexus update` übernimmt sie von dort (B18).
(cd "$src" && scripts/release_compose.sh "$version" "$repo" > updater/docker-compose.release.yml)
tags=("$version")
if [[ "$version" != *-* ]]; then # nur stabile Versionen bekommen Kurz-Tags und latest
+3 -7
View File
@@ -19,12 +19,8 @@ out="dist/$name"
rm -rf "$out" "dist/$name.tar.gz"
mkdir -p "$out"
# Standardwerte in der Compose-Datei durch die Release-Werte ersetzen.
sed -e "s|\${OPNEXUS_IMAGE_REPO:-local}|\${OPNEXUS_IMAGE_REPO:-$repo}|g" \
-e "s|\${OPNEXUS_VERSION:-latest}|\${OPNEXUS_VERSION:-$version}|g" \
docker-compose.yml > "$out/docker-compose.yml"
grep -q "OPNEXUS_IMAGE_REPO:-$repo" "$out/docker-compose.yml" || { echo "Compose-Defaults nicht ersetzt" >&2; exit 1; }
! grep -qE '^\s+build:' "$out/docker-compose.yml" || { echo "build: im Release-Compose" >&2; exit 1; }
# Standardwerte in der Compose-Datei durch die Release-Werte ersetzen (dieselbe Datei steckt im Updater-Image).
scripts/release_compose.sh "$version" "$repo" > "$out/docker-compose.yml"
# .env.example mit gesetzter Registry und Version (der Updater liest .env).
sed -e "s|^# OPNEXUS_IMAGE_REPO=.*|OPNEXUS_IMAGE_REPO=$repo|" \
@@ -37,7 +33,7 @@ cp cosign.pub "$out/"
cat > "$out/INSTALL.txt" <<TXT
OPNexus $version
1. cp .env.example .env und POSTGRES_PASSWORD (und was sonst nötig ist) eintragen
1. install -m 600 .env.example .env und POSTGRES_PASSWORD (und was sonst nötig ist) eintragen
2. docker compose pull && docker compose up -d --no-build
3. Anmelden: ./opnexus password (zeigt das Admin-Passwort, wenn ADMIN_PASSWORD leer war)
dann im Browser öffnen — der Einrichtungsassistent führt durch den Rest
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Gibt die Release-Fassung der docker-compose.yml aus: Registry und Version als Standard eingebacken, kein build:.
# Dieselbe Datei kommt ins Release-Bundle und ins Updater-Image (dort holt `opnexus update` sie ab, B18).
#
# scripts/release_compose.sh VERSION IMAGE_REPO > docker-compose.yml
set -euo pipefail
cd "$(dirname "$0")/.."
version="${1:?Usage: $0 VERSION IMAGE_REPO}"
repo="${2:?Usage: $0 VERSION IMAGE_REPO}"
out=$(sed -e "s|\${OPNEXUS_IMAGE_REPO:-local}|\${OPNEXUS_IMAGE_REPO:-$repo}|g" \
-e "s|\${OPNEXUS_VERSION:-latest}|\${OPNEXUS_VERSION:-$version}|g" docker-compose.yml)
grep -q "OPNEXUS_IMAGE_REPO:-$repo" <<<"$out" || { echo "Compose-Defaults nicht ersetzt" >&2; exit 1; }
! grep -qE '^\s+build:' <<<"$out" || { echo "build: im Release-Compose" >&2; exit 1; }
printf '%s\n' "$out"
+8 -2
View File
@@ -19,9 +19,15 @@ RUN set -e; arch="${TARGETARCH:-amd64}"; \
echo "$sha /usr/local/bin/cosign" | sha256sum -c -; \
chmod 0755 /usr/local/bin/cosign
COPY opnexus /usr/local/bin/opnexus
# Die Release-Fassung der Compose-Datei legt scripts/build_release_images.sh in den Build-Kontext; `opnexus update`
# übernimmt sie von hier (B18). Bei einem lokalen Build ohne sie bleibt /usr/share/opnexus leer. (Zwei Quellen, damit
# das Muster auch ohne Treffer gültig ist.)
COPY opnexus docker-compose.release.y[m]l /tmp/opx/
COPY server.py /app/server.py
RUN chmod 0755 /usr/local/bin/opnexus
RUN install -m 0755 /tmp/opx/opnexus /usr/local/bin/opnexus \
&& mkdir -p /usr/share/opnexus \
&& if [ -f /tmp/opx/docker-compose.release.yml ]; then install -m 0644 /tmp/opx/docker-compose.release.yml /usr/share/opnexus/docker-compose.yml; fi \
&& rm -rf /tmp/opx
WORKDIR /app
ENV PYTHONUNBUFFERED=1 OPNEXUS_DIR=/project UPDATER_STATE_DIR=/state
Binary file not shown.
Binary file not shown.
Binary file not shown.
+321 -16
View File
@@ -7,14 +7,20 @@
# bei Fehler automatisch zurückrollen
# opnexus rollback Letztes Update rückgängig machen
# opnexus backup Sofort-Backup der OPNexus-Datenbank
# opnexus backup --full [DATEI]
# opnexus backup --full [DATEI] [--with-env] [--with-ca]
# Vollsicherung: Datenbank UND alle Geheimnisse
# (Firewall-Zugänge, 2FA, VPN-/Zertifikatsschlüssel,
# Konfigurations-Backups) in einer verschlüsselten
# Datei (.opxfull). Passphrase wird abgefragt
# (oder OPNEXUS_BACKUP_PASSPHRASE).
# opnexus restore DATEI Vollsicherung zurückspielen (überschreibt Datenbank
# und Geheimnisse dieser Installation; --yes: ohne Rückfrage)
# (oder OPNEXUS_BACKUP_PASSPHRASE). Optional auch
# die .env und die CA des Proxys (für einen Umzug).
# opnexus restore DATEI [--with-env] [--with-ca] [--yes]
# Vollsicherung zurückspielen (überschreibt Datenbank
# und Geheimnisse dieser Installation)
# opnexus migrate-host ZIEL [--dir PFAD] [--yes]
# Umzug auf einen anderen Server (ssh ZIEL, dort ist
# OPNexus schon installiert): Quelle anhalten, sichern,
# am Ziel zurückspielen, prüfen. Siehe Handbuch, "Umzug".
# opnexus password [NAME] Neues Zufallspasswort für den Admin setzen und
# anzeigen (--yes: ohne Rückfrage)
# opnexus export [--out VERZEICHNIS] [--kinds a,b]
@@ -71,7 +77,7 @@ protect_env() {
# Variablen im Aufrufer (dort als local deklarieren).
read_update_state() {
local key value
for key in started prev_version prev_setting prev_db_revision target; do
for key in started prev_version prev_setting prev_db_revision target prev_compose prev_compose_sha; do
value=$(sed -n "s/^$key=//p" "$STATE_FILE" | tail -n1)
[[ "$value" =~ ^[A-Za-z0-9._-]*$ ]] || die "$STATE_FILE enthält einen ungültigen Wert für $key — er wird nicht verwendet."
printf -v "$key" '%s' "$value"
@@ -188,7 +194,7 @@ cmd_status_porcelain() {
for svc in "${SERVICES[@]}"; do echo "service.$svc=$(service_state "$svc")"; done
if [ -f "$STATE_FILE" ]; then
# Nur bekannte Schlüssel durchlassen — die Datei wird vom Skript selbst geschrieben.
grep -E '^(started|prev_version|prev_setting|prev_db_revision|target)=' "$STATE_FILE" | sed 's/^/last_update./'
grep -E '^(started|prev_version|prev_setting|prev_db_revision|target|prev_compose)=' "$STATE_FILE" | sed 's/^/last_update./'
fi
}
@@ -200,6 +206,14 @@ cmd_status() {
echo "Laufende Version: $(running_version)"
echo "DB-Revision: $(db_revision)"
for svc in "${SERVICES[@]}"; do echo "$svc: $(service_state "$svc")"; done
if [ -f "$LAST_FULL_FILE" ]; then
local t age
t=$(sed -n 's/^time=//p' "$LAST_FULL_FILE")
age=$(( ( $(date +%s) - $(date -d "$t" +%s 2>/dev/null || date +%s) ) / 86400 ))
echo "Letzte Vollsicherung: $t (vor $age Tag(en)): $(sed -n 's/^file=//p' "$LAST_FULL_FILE")"
else
echo "Letzte Vollsicherung: keine von diesem Server bekannt ('opnexus backup --full')"
fi
[ -f "$STATE_FILE" ] && { echo; echo "Letztes Update (für 'rollback'):"; sed 's/^/ /' "$STATE_FILE"; }
return 0
}
@@ -225,23 +239,126 @@ ask_passphrase() {
printf '%s' "$one"
}
LAST_FULL_FILE=".opnexus-last-full-backup"
# Dateien dieses Servers, die nicht in den Volumes des Backends liegen: die .env und die interne CA des Proxys
# (dieselbe CA = Browser, die ihr vertrauen, vertrauen auch nach einem Umzug). Als tar auf stdout.
host_files_tar() {
local with_env="$1" with_ca="$2" tmp
tmp=$(mktemp -d) || return 1
chmod 700 "$tmp"
if [ "$with_env" = 1 ]; then install -m 600 "$ENV_FILE" "$tmp/env" || { rm -rf "$tmp"; return 1; }; fi
if [ "$with_ca" = 1 ]; then
mkdir -m 700 "$tmp/caddy-ca"
# Aus dem Volume, nicht aus dem laufenden Proxy: beim Umzug ist er schon angehalten.
c run --rm --no-deps -T --entrypoint tar proxy -C /data/caddy/pki/authorities -cf - local | tar -x -C "$tmp/caddy-ca" \
|| { rm -rf "$tmp"; die "Die CA des Proxys ließ sich nicht lesen (hat der Proxy schon einmal gestartet?)."; }
fi
tar -C "$tmp" -cf - . ; local rc=$?
rm -rf "$tmp"
return $rc
}
cmd_backup_full() {
local out="${1:-./opnexus-full-$(date -u +%Y%m%d-%H%M%S).opxfull}" pass
local out="" with_env=0 with_ca=0 pass arg extra=()
for arg in "$@"; do
case "$arg" in
--with-env) with_env=1 ;;
--with-ca) with_ca=1 ;;
-*) die "Unbekannte Option: $arg" ;;
*) out="$arg" ;;
esac
done
out="${out:-./opnexus-full-$(date -u +%Y%m%d-%H%M%S).opxfull}"
[ ! -e "$out" ] || die "$out existiert schon."
c exec -T backend true >/dev/null 2>&1 || die "Das Backend läuft nicht — erst starten (docker compose up -d)."
pass="$(ask_passphrase confirm)"
( umask 077
FULL_BACKUP_PASSPHRASE="$pass" c exec -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup create > "$out.partial" ) \
|| { rm -f "$out.partial"; die "Vollsicherung fehlgeschlagen."; }
if [ "$with_env" = 1 ] || [ "$with_ca" = 1 ]; then
extra=(--host-files)
( umask 077
host_files_tar "$with_env" "$with_ca" \
| FULL_BACKUP_PASSPHRASE="$pass" c exec -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup create "${extra[@]}" > "$out.partial"
rcs=("${PIPESTATUS[@]}"); [ "${rcs[0]}" = 0 ] && [ "${rcs[1]}" = 0 ] ) || { rm -f "$out.partial"; die "Vollsicherung fehlgeschlagen."; }
else
( umask 077
FULL_BACKUP_PASSPHRASE="$pass" c exec -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup create > "$out.partial" ) \
|| { rm -f "$out.partial"; die "Vollsicherung fehlgeschlagen."; }
fi
mv "$out.partial" "$out"; chmod 600 "$out"
printf 'time=%s\nfile=%s\nenv=%s\nca=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$(readlink -f "$out")" "$with_env" "$with_ca" > "$LAST_FULL_FILE"
say "Vollsicherung geschrieben: $out ($(du -h "$out" | cut -f1))"
[ "$with_env" = 1 ] && say "Enthält die .env (Zugangsdaten dieses Servers)."
[ "$with_ca" = 1 ] && say "Enthält die CA des Proxys."
say "Ohne die Passphrase ist die Datei wertlos. Beides getrennt vom Server aufbewahren."
}
# Schlüssel der .env, die beim Zurückspielen NIE aus der Sicherung übernommen werden: die Datenbank dieses Servers
# (ihr Passwort wurde bei seiner Installation gesetzt, die Sicherung ändert es nicht), die installierte Version und
# die Adressen, unter denen DIESER Server erreichbar ist.
ENV_KEEP_KEYS="POSTGRES_PASSWORD POSTGRES_USER POSTGRES_DB OPNEXUS_VERSION OPNEXUS_IMAGE_REPO OPNEXUS_SITES OPNEXUS_DEFAULT_SNI FRONTEND_ORIGIN BIND_ADDRESS PROXY_BIND_ADDRESS GUI_BIND_ADDRESS OPNEXUS_GUI_SITES"
# Übernimmt die übrigen Schlüssel aus der gesicherten .env ($1). Schreibt die gesicherte Datei vollständig als
# .env.from-backup daneben (zum Nachsehen) und die bisherige als .env.before-restore.
merge_env_from_backup() {
local backup="$1" yes="$2" key value changed=() line
install -m 600 "$backup" "$ENV_FILE.from-backup"
while IFS= read -r line; do
[[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)=(.*)$ ]] || continue
key="${BASH_REMATCH[1]}"; value="${BASH_REMATCH[2]}"
[[ " $ENV_KEEP_KEYS " == *" $key "* ]] && continue
[ "$(env_get "$key")" = "$value" ] && grep -q "^$key=" "$ENV_FILE" && continue
changed+=("$key")
done < "$backup"
if [ "${#changed[@]}" = 0 ]; then say ".env: keine abweichenden Einstellungen in der Sicherung."; return 0; fi
say ".env: diese Einstellungen kommen aus der Sicherung: ${changed[*]}"
say "(Unverändert bleiben Datenbank-Zugang, Version und Adressen dieses Servers; die ganze gesicherte Datei liegt in $ENV_FILE.from-backup.)"
if [ "$yes" != 1 ]; then
read -rp "Übernehmen? [j/N] " ans </dev/tty
[[ "$ans" =~ ^[jJyY]$ ]] || { say ".env unverändert gelassen."; return 0; }
fi
install -m 600 "$ENV_FILE" "$ENV_FILE.before-restore"
for key in "${changed[@]}"; do
value=$(sed -n "s/^$key=//p" "$backup" | tail -n1)
env_set_literal "$key" "$value"
done
say ".env aktualisiert (vorher: $ENV_FILE.before-restore)."
}
# Wie env_set, aber für beliebige Werte (Passwörter, URLs mit | oder &): ohne sed-Ersetzung.
env_set_literal() {
local key="$1" value="$2" tmp
tmp=$(mktemp "$ENV_FILE.XXXXXX") && chmod 600 "$tmp"
if grep -q "^$key=" "$ENV_FILE"; then
awk -v k="$key" -v v="$value" 'BEGIN{FS=OFS="="} index($0, k"=")==1 {print k"="v; next} {print}' "$ENV_FILE" > "$tmp"
else
cat "$ENV_FILE" > "$tmp"; printf '%s=%s\n' "$key" "$value" >> "$tmp"
fi
mv "$tmp" "$ENV_FILE"
}
# Die gesicherte CA in das Volume des Proxys legen; die ausgestellten Zertifikate werden verworfen, damit Caddy sie
# mit dieser CA neu ausstellt. Der Proxy ist dabei gestoppt.
restore_ca() {
local dir="$1"
[ -d "$dir/local" ] || die "Die Sicherung enthält keine CA."
tar -C "$dir" -cf - local | c run --rm --no-deps -T --entrypoint sh proxy -c \
'rm -rf /data/caddy/pki/authorities/local /data/caddy/certificates/local && mkdir -p /data/caddy/pki/authorities && tar -x -C /data/caddy/pki/authorities' \
|| die "CA konnte nicht zurückgespielt werden."
say "CA des Proxys zurückgespielt (Zertifikate werden beim Start neu ausgestellt)."
}
cmd_restore() {
local file="" yes=0 pass arg
for arg in "$@"; do case "$arg" in --yes|-y) yes=1 ;; *) file="$arg" ;; esac; done
[ -n "$file" ] && [ -f "$file" ] || die "Aufruf: opnexus restore DATEI.opxfull [--yes]"
local file="" yes=0 pass arg with_env=0 with_ca=0 hosttmp=""
for arg in "$@"; do
case "$arg" in
--yes|-y) yes=1 ;;
--with-env) with_env=1 ;;
--with-ca) with_ca=1 ;;
-*) die "Unbekannte Option: $arg" ;;
*) file="$arg" ;;
esac
done
[ -n "$file" ] && [ -f "$file" ] || die "Aufruf: opnexus restore DATEI.opxfull [--with-env] [--with-ca] [--yes]"
pass="$(ask_passphrase)"
say "Prüfe die Datei …"
FULL_BACKUP_PASSPHRASE="$pass" c run --rm --no-deps -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup info < "$file" \
@@ -258,6 +375,18 @@ cmd_restore() {
say "Spiele zurück …"
FULL_BACKUP_PASSPHRASE="$pass" c run --rm --no-deps -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup restore < "$file" \
|| die "Zurückspielen fehlgeschlagen. Der vorherige Stand liegt als Dump in /backups; Dienste sind gestoppt."
if [ "$with_env" = 1 ] || [ "$with_ca" = 1 ]; then
hosttmp=$(mktemp -d) && chmod 700 "$hosttmp"
FULL_BACKUP_PASSPHRASE="$pass" c run --rm --no-deps -T -e FULL_BACKUP_PASSPHRASE backend python -m app.full_backup host-files < "$file" \
| tar -x -C "$hosttmp" || { rm -rf "$hosttmp"; die "Die Dateien des Servers (.env/CA) ließen sich nicht lesen. Datenbank und Geheimnisse sind schon zurückgespielt; Dienste sind gestoppt."; }
if [ "$with_env" = 1 ]; then
if [ -f "$hosttmp/env" ]; then merge_env_from_backup "$hosttmp/env" "$yes"; else say "Die Sicherung enthält keine .env (nicht mit --with-env erstellt)."; fi
fi
if [ "$with_ca" = 1 ]; then
if [ -d "$hosttmp/caddy-ca" ]; then restore_ca "$hosttmp/caddy-ca"; else say "Die Sicherung enthält keine CA (nicht mit --with-ca erstellt)."; fi
fi
rm -rf "$hosttmp"
fi
say "Starte die Dienste …"
c up -d backend frontend proxy >/dev/null
wait_healthy || die "Nicht alle Dienste sind gesund — 'docker compose logs' prüfen."
@@ -339,12 +468,88 @@ refresh_cli() {
fi
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
if cmp -s "$tmp" "$(readlink -f "$0")"; then rm -f "$tmp"; return 0; fi
# Im Release-Bundle (CLI neben docker-compose.yml) auch die Datei auf der Platte ersetzen: sonst liefen ein späteres
# `rollback` oder `status` mit der alten CLI, die neue Schritte (z. B. die Compose-Datei zurücklegen) nicht kennt.
# Über eine neue Datei und Umbenennen: die laufende Bash liest ihr Skript weiter aus der alten.
local self; self="$(readlink -f "$0")"
if [ -f "$(dirname "$self")/docker-compose.yml" ] && [ -w "$self" ] && [ -w "$(dirname "$self")" ]; then
cp -p "$self" "$self.bak" 2>/dev/null || true
if install -m 0755 "$tmp" "$self.new" && mv -f "$self.new" "$self"; then
say "CLI auf der Platte aktualisiert (vorherige: $(basename "$self").bak)."
fi
fi
say "Die CLI des Zielreleases (Signatur geprüft) ist neuer — starte sie."
chmod 0700 "$tmp"
export OPNEXUS_DIR="$PWD"
exec bash "$tmp" __refreshed update "$target"
}
# Die Compose-Datei des Zielreleases übernehmen (B18): `update` tauschte bisher nur Images, neue Einstellungen der Dienste
# (z. B. Härtung, neue Variablen) kamen nie an. Quelle ist das signierte Updater-Image des Ziels (wie bei refresh_cli),
# nie das Netz direkt. Wurde die Datei hier angepasst (Prüfsumme weicht von der zuletzt übernommenen ab), bleibt sie, und
# die neue Fassung liegt als docker-compose.yml.new daneben: eigene Anpassungen gehören in docker-compose.override.yml.
COMPOSE_SHA_FILE=".opnexus-compose-sha"
COMPOSE_BACKUP=""
COMPOSE_PREV_SHA=""
refresh_compose() {
local target="$1" repo ref digest tmp keep=0 cur_sha rec_sha label
[ "${OPNEXUS_NO_COMPOSE_REFRESH:-}" != 1 ] || return 0
repo=$(env_get OPNEXUS_IMAGE_REPO)
[ -n "$repo" ] || return 0
if ! command -v cosign >/dev/null 2>&1 || [ ! -f cosign.pub ]; then
say "Compose-Datei nicht abgeglichen (ohne cosign und cosign.pub wird keine Datei aus einem Image übernommen)."; return 0
fi
ref="$repo/opnexus-updater:$target"
if updater_enabled; then keep=1; fi
docker pull -q "$ref" >/dev/null 2>&1 || { say "Compose-Datei nicht abgeglichen (Updater-Image des Ziels nicht ladbar)."; return 0; }
digest=$(docker image inspect --format '{{index .RepoDigests 0}}' "$ref" 2>/dev/null || true)
if [[ "$digest" != *@sha256:* ]]; then
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
say "Compose-Datei nicht abgeglichen (Digest des Updater-Images unbekannt)."; return 0
fi
cosign verify --key cosign.pub --insecure-ignore-tlog=true "$digest" >/dev/null 2>&1 \
|| die "Signatur des Updater-Images $ref ist ungültig oder fehlt — nichts wurde verändert."
tmp=$(mktemp "${TMPDIR:-/tmp}/opnexus-compose.XXXXXX") || return 0
if ! docker run --rm --pull never --entrypoint cat "$digest" /usr/share/opnexus/docker-compose.yml > "$tmp" 2>/dev/null \
|| ! grep -q '^services:' "$tmp" || ! grep -q 'opnexus-backend:' "$tmp"; then
rm -f "$tmp"; [ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
say "Das Zielrelease liefert keine Compose-Datei mit (vor 1.34) — die vorhandene bleibt."; return 0
fi
[ "$keep" = 1 ] || docker rmi "$ref" >/dev/null 2>&1 || true
if cmp -s "$tmp" docker-compose.yml; then
rm -f "$tmp"; sha256sum docker-compose.yml | cut -d' ' -f1 > "$COMPOSE_SHA_FILE"; return 0
fi
cur_sha=$(sha256sum docker-compose.yml | cut -d' ' -f1)
rec_sha=$(cat "$COMPOSE_SHA_FILE" 2>/dev/null || true)
if [ -n "$rec_sha" ] && [ "$rec_sha" != "$cur_sha" ]; then
install -m 644 "$tmp" docker-compose.yml.new; rm -f "$tmp"
say "docker-compose.yml wurde hier angepasst und bleibt unverändert. Die Fassung von $target liegt als docker-compose.yml.new daneben;"
say "Anpassungen gehören in docker-compose.override.yml (dann übernimmt das nächste Update die Datei selbst)."
return 0
fi
label=$(running_version); label="${label:-alt}"
COMPOSE_BACKUP="docker-compose.yml.bak-$label"
[ -e "$COMPOSE_BACKUP" ] && COMPOSE_BACKUP="docker-compose.yml.bak-$label-$(date -u +%Y%m%d%H%M%S)"
COMPOSE_PREV_SHA="$rec_sha"
cp -p docker-compose.yml "$COMPOSE_BACKUP"
install -m 644 "$tmp" docker-compose.yml; rm -f "$tmp"
sha256sum docker-compose.yml | cut -d' ' -f1 > "$COMPOSE_SHA_FILE"
say "Compose-Datei von $target übernommen (vorherige: $COMPOSE_BACKUP)."
if [ -z "$rec_sha" ]; then
say "Falls du die alte Datei angepasst hattest: 'diff $COMPOSE_BACKUP docker-compose.yml' zeigt den Unterschied;"
say "eigene Anpassungen gehören in docker-compose.override.yml."
fi
}
# Beim Rollback die Compose-Datei von vorher zurücklegen (nur wenn das Update sie getauscht hatte).
restore_compose() {
[ -n "${prev_compose:-}" ] && [ -f "$prev_compose" ] || return 0
cp -p "$prev_compose" docker-compose.yml
if [ -n "${prev_compose_sha:-}" ]; then echo "$prev_compose_sha" > "$COMPOSE_SHA_FILE"; else rm -f "$COMPOSE_SHA_FILE"; fi
say "Compose-Datei von vorher wiederhergestellt ($prev_compose)."
}
cmd_update() {
require_release_mode
protect_env
@@ -366,7 +571,15 @@ cmd_update() {
OPNEXUS_VERSION="$target" c pull "${pull_services[@]}" || die "Pull fehlgeschlagen — nichts wurde verändert."
verify_signatures "$target" "${pull_services[@]}"
refresh_cli "$target"
preflight_proxy_config "$target"
refresh_compose "$target"
# In einer Subshell: scheitert die Vorprüfung, kommt die eben getauschte Compose-Datei zurück ("nichts verändert").
if ! ( preflight_proxy_config "$target" ); then
if [ -n "$COMPOSE_BACKUP" ]; then
cp -p "$COMPOSE_BACKUP" docker-compose.yml
if [ -n "$COMPOSE_PREV_SHA" ]; then echo "$COMPOSE_PREV_SHA" > "$COMPOSE_SHA_FILE"; else rm -f "$COMPOSE_SHA_FILE"; fi
fi
exit 1
fi
started=$(date -u +%Y%m%dT%H%M%SZ)
cat > "$STATE_FILE" <<STATE
@@ -375,6 +588,8 @@ prev_version=$prev_version
prev_setting=$prev_setting
prev_db_revision=$prev_rev
target=$target
prev_compose=$COMPOSE_BACKUP
prev_compose_sha=$COMPOSE_PREV_SHA
STATE
env_set OPNEXUS_VERSION "$target"
@@ -409,7 +624,7 @@ latest_pre_upgrade_dump() {
cmd_rollback() {
require_release_mode
[ -f "$STATE_FILE" ] || die "Kein Update-Zustand ($STATE_FILE) — es gibt nichts zurückzurollen."
local started prev_version prev_setting prev_db_revision target
local started prev_version prev_setting prev_db_revision target prev_compose prev_compose_sha
read_update_state
[ -n "${prev_version:-}" ] || die "$STATE_FILE ist unvollständig."
@@ -463,6 +678,7 @@ cmd_rollback() {
fi
env_set OPNEXUS_VERSION "$prev_version"
restore_compose
phase switching
local rb_targets=()
mapfile -t rb_targets < <(up_targets)
@@ -573,9 +789,97 @@ cmd_apply() {
exit "$code"
}
# ── Umzug auf einen anderen Server (HA-Plan H1: gehärtete Kaltreserve) ─────────────────────────────────────────
# Am Ziel ist OPNexus schon nach Anleitung installiert (gleiche oder neuere Version). Ablauf: Vorprüfung →
# Quelle anhalten (keine Änderungen mehr) → Vollsicherung inkl. .env und CA → am Ziel zurückspielen → prüfen.
# Die Passphrase ist zufällig, lebt nur in diesem Lauf und geht über stdin zum Ziel (nie in eine Befehlszeile).
# Die Quelle bleibt angehalten, nicht gelöscht: zurück geht es mit `docker compose up -d` hier.
remote() { ssh -o BatchMode=yes -o ConnectTimeout=10 "$MIG_TARGET" "$@"; }
cmd_migrate_host() {
local yes=0 dir="" arg t0 pass tmpfile remote_cli remote_file rv lv hosts_before hosts_after st
MIG_TARGET=""
while [ $# -gt 0 ]; do
case "$1" in
--yes|-y) yes=1 ;;
--dir) shift; dir="${1:-}" ;;
-*) die "Unbekannte Option: $1" ;;
*) MIG_TARGET="$1" ;;
esac
shift
done
[ -n "$MIG_TARGET" ] || die "Aufruf: opnexus migrate-host BENUTZER@ZIEL [--dir PFAD] [--yes]"
require_release_mode
dir="${dir:-$PWD}"
t0=$SECONDS
say "Vorprüfung …"
remote true || die "Keine SSH-Verbindung zu $MIG_TARGET (Schlüssel-Anmeldung nötig)."
remote_cli=$(remote "cd '$dir' 2>/dev/null && { [ -x ./opnexus ] && echo ./opnexus || { [ -x scripts/opnexus ] && echo scripts/opnexus; }; }") \
|| die "Auf $MIG_TARGET gibt es $dir nicht (--dir angeben)."
[ -n "$remote_cli" ] || die "Auf $MIG_TARGET liegt in $dir kein opnexus-Werkzeug: dort zuerst OPNexus installieren."
st=$(remote "cd '$dir' && $remote_cli status --porcelain") || die "Am Ziel antwortet '$remote_cli status' nicht."
rv=$(sed -n 's/^running_version=//p' <<<"$st"); lv=$(running_version)
[ -n "$rv" ] || die "Am Ziel läuft OPNexus nicht (erst starten: docker compose up -d)."
[ "$(printf '%s\n%s\n' "$lv" "$rv" | sort -V | head -n1)" = "$lv" ] \
|| die "Am Ziel läuft $rv, hier $lv: das Ziel muss gleich oder neuer sein ('$remote_cli update $lv' dort)."
grep -qE '^service\.backend=healthy$' <<<"$st" || die "Das Backend am Ziel ist nicht gesund."
hosts_before=$(psql_db -c "SELECT count(*) FROM opnsense_hosts")
say "Quelle: $lv, $hosts_before Firewall(s). Ziel $MIG_TARGET:$dir: $rv."
if [ "$yes" != 1 ]; then
echo "Danach ist OPNexus HIER angehalten und das Ziel enthält diese Installation (seine bisherigen Daten werden überschrieben)."
read -rp "Umzug starten? [j/N] " ans </dev/tty
[[ "$ans" =~ ^[jJyY]$ ]] || die "Abgebrochen."
fi
pass=$(head -c 32 /dev/urandom | base64 | tr -d '/+=' | head -c 40)
tmpfile=$(mktemp --suffix=.opxfull) && rm -f "$tmpfile"
say "Halte Proxy und Oberfläche an (ab jetzt keine Änderungen mehr) …"
c stop proxy frontend >/dev/null 2>&1 || true
say "Vollsicherung mit .env und CA …"
( export OPNEXUS_BACKUP_PASSPHRASE="$pass"
cmd_backup_full "$tmpfile" --with-env --with-ca >/dev/null ) \
|| { c up -d >/dev/null 2>&1; die "Vollsicherung fehlgeschlagen; die Quelle läuft wieder."; }
c stop >/dev/null 2>&1 || true
rm -f "$LAST_FULL_FILE" # die Datei wird gleich gelöscht, sie taugt nicht als "letzte Vollsicherung"
remote_file="/tmp/opnexus-migrate-$$.opxfull"
say "Übertrage ($(du -h "$tmpfile" | cut -f1)) …"
scp -q -o BatchMode=yes "$tmpfile" "$MIG_TARGET:$remote_file" && remote "chmod 600 '$remote_file'" \
|| { rm -f "$tmpfile"; die "Übertragung fehlgeschlagen. Die Quelle ist angehalten: 'docker compose up -d' startet sie wieder."; }
rm -f "$tmpfile"
say "Spiele am Ziel zurück …"
if ! printf '%s\n' "$pass" | remote "read -r OPNEXUS_BACKUP_PASSPHRASE; export OPNEXUS_BACKUP_PASSPHRASE; cd '$dir' && $remote_cli restore '$remote_file' --with-env --with-ca --yes; rc=\$?; rm -f '$remote_file'; exit \$rc"; then
die "Zurückspielen am Ziel fehlgeschlagen (Ausgabe oben). Die Quelle ist angehalten: 'docker compose up -d' startet sie wieder."
fi
st=$(remote "cd '$dir' && $remote_cli status --porcelain") || st=""
hosts_after=$(remote "cd '$dir' && docker compose exec -T postgres sh -c 'psql -U \"\$POSTGRES_USER\" -d \"\$POSTGRES_DB\" -Atc \"SELECT count(*) FROM opnsense_hosts\"'" 2>/dev/null | tr -d '\r')
[ "$hosts_after" = "$hosts_before" ] \
|| die "Am Ziel stehen $hosts_after statt $hosts_before Firewall(s). Bitte prüfen; die Quelle ist angehalten ('docker compose up -d' startet sie wieder)."
grep -qE '^service\.backend=healthy$' <<<"$st" || die "Das Backend am Ziel ist nach dem Zurückspielen nicht gesund."
say "Umzug fertig nach $(( SECONDS - t0 )) s: $MIG_TARGET führt OPNexus mit $hosts_after Firewall(s)."
echo
echo "Noch zu tun:"
echo " - Adresse umstellen: virtuelle IP, DNS-Namen oder Lesezeichen auf das Ziel zeigen lassen."
echo " - Firewalls, die nur bestimmte Absender an ihre API lassen, die Adresse des Ziels erlauben."
echo " - Mit Management-Overlay: der Tunnel-Endpunkt (öffentliche Adresse, UDP-Port) muss zum Ziel zeigen."
echo " - Hier bleibt alles angehalten (startet auch nach einem Neustart nicht). Zurück: am Ziel"
echo " 'docker compose stop', hier 'docker compose up -d'."
}
# Interner Neustart durch refresh_cli: Marker setzen und die eigene Temp-Datei aufräumen (bash hat sie offen).
if [ "${1:-}" = __refreshed ]; then
CLI_REFRESHED=1; shift
# Gestartet von einer älteren CLI, deren refresh_cli die Datei im Bundle noch nicht ersetzt (vor 1.34): das hier
# nachholen, sonst liefe ein späteres `rollback` mit der alten CLI.
disk="${OPNEXUS_DIR:-}/opnexus"
if [ -n "${OPNEXUS_DIR:-}" ] && [ -f "$disk" ] && [ -f "$OPNEXUS_DIR/docker-compose.yml" ] && [ -w "$OPNEXUS_DIR" ] \
&& ! cmp -s "$0" "$disk"; then
cp -p "$disk" "$disk.bak" 2>/dev/null || true
if install -m 0755 "$0" "$disk.new" && mv -f "$disk.new" "$disk"; then say "CLI auf der Platte aktualisiert (vorherige: opnexus.bak)."; fi
fi
case "$0" in */opnexus-cli.*) rm -f "$0" ;; esac
fi
@@ -588,5 +892,6 @@ case "${1:-}" in
rollback) cmd_rollback ;;
backup) if [ "${2:-}" = "--full" ]; then shift 2; cmd_backup_full "$@"; else cmd_backup; fi ;;
restore) shift; cmd_restore "$@" ;;
*) sed -n '2,32p' "$0" | sed 's/^# \{0,1\}//'; exit 1 ;;
migrate-host) shift; cmd_migrate_host "$@" ;;
*) sed -n '2,37p' "$0" | sed 's/^# \{0,1\}//'; exit 1 ;;
esac
+149
View File
@@ -0,0 +1,149 @@
"""refresh_compose / restore_compose in `opnexus` (B18): `update` takes the target release's docker-compose.yml from the
signed updater image, keeps a backup, never overwrites a file that was changed here, and `rollback` puts it back.
Runs the real functions with fake `docker` and `cosign` in front of PATH.
Run: cd updater && python3 -m unittest test_cli_compose"""
import hashlib
import os
import shutil
import stat
import subprocess
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).with_name("opnexus").resolve()
DIGEST = "registry.example/opnexus/opnexus-updater@sha256:" + "b" * 64
FAKE_DOCKER = """#!/bin/sh
echo "$@" >> "$FAKE_LOG"
case "$1" in
pull) exit 0 ;;
image) echo "$FAKE_DIGEST" ;;
run) [ -f "$FAKE_COMPOSE" ] || exit 1; cat "$FAKE_COMPOSE" ;;
compose) exit 0 ;;
esac
"""
FAKE_COSIGN = """#!/bin/sh
echo "cosign $*" >> "$FAKE_LOG"
[ "${FAKE_COSIGN_RC:-0}" = 0 ]
"""
OLD = "services:\n backend:\n image: x/opnexus-backend:1\n"
NEW = "services:\n backend:\n image: x/opnexus-backend:2\n security_opt: [\"no-new-privileges:true\"]\n"
def _exe(path: Path, text: str):
path.write_text(text)
path.chmod(path.stat().st_mode | stat.S_IXUSR)
def _sha(text: str) -> str:
return hashlib.sha256(text.encode()).hexdigest()
class ComposeRefresh(unittest.TestCase):
def setUp(self):
self.tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.tmp, ignore_errors=True)
(self.tmp / "bin").mkdir()
_exe(self.tmp / "bin" / "docker", FAKE_DOCKER)
_exe(self.tmp / "bin" / "cosign", FAKE_COSIGN)
(self.tmp / ".env").write_text("OPNEXUS_IMAGE_REPO=registry.example/opnexus\n")
(self.tmp / "cosign.pub").write_text("key")
(self.tmp / "docker-compose.yml").write_text(OLD)
self.offered = self.tmp / "offered.yml"
self.offered.write_text(NEW)
self.log = self.tmp / "calls.log"
def run_body(self, body: str, *, no_cosign=False, env_extra=None):
if no_cosign:
(self.tmp / "bin" / "cosign").unlink()
path = f"{self.tmp / 'bin'}:/usr/bin:/bin"
env = {**os.environ, "PATH": path, "OPNEXUS_DIR": str(self.tmp), "FAKE_LOG": str(self.log),
"FAKE_COMPOSE": str(self.offered), "FAKE_DIGEST": DIGEST, "TMPDIR": str(self.tmp)}
env.pop("OPNEXUS_NO_COMPOSE_REFRESH", None)
env.update(env_extra or {})
script = f'source <(sed "/^case \\"\\${{1:-}}\\" in/,\\$d" "{SCRIPT}"); {body}'
return subprocess.run(["bash", "-c", script, str(SCRIPT)], env=env, capture_output=True, text=True, timeout=30)
def refresh(self, **kw):
return self.run_body('refresh_compose 9.9.9; echo "BACKUP=$COMPOSE_BACKUP PREV=$COMPOSE_PREV_SHA"', **kw)
def compose(self) -> str:
return (self.tmp / "docker-compose.yml").read_text()
def recorded(self) -> str | None:
f = self.tmp / ".opnexus-compose-sha"
return f.read_text().strip() if f.exists() else None
def test_first_update_replaces_the_file_and_keeps_a_backup(self):
r = self.refresh()
self.assertEqual(r.returncode, 0, r.stderr)
self.assertEqual(self.compose(), NEW)
self.assertEqual((self.tmp / "docker-compose.yml.bak-alt").read_text(), OLD)
self.assertEqual(self.recorded(), _sha(NEW))
self.assertIn("BACKUP=docker-compose.yml.bak-alt", r.stdout)
self.assertIn("docker-compose.override.yml", r.stdout) # first time: point to the override file
calls = self.log.read_text().splitlines()
self.assertTrue(any(c.startswith("cosign verify") and c.endswith(DIGEST) for c in calls))
run = next(c for c in calls if c.startswith("run "))
self.assertTrue(run.endswith(f"{DIGEST} /usr/share/opnexus/docker-compose.yml")) # the verified digest, not the tag
def test_an_unchanged_file_from_the_last_update_is_replaced_quietly(self):
(self.tmp / ".opnexus-compose-sha").write_text(_sha(OLD) + "\n")
r = self.refresh()
self.assertEqual(self.compose(), NEW)
self.assertIn(f"PREV={_sha(OLD)}", r.stdout)
self.assertNotIn("diff ", r.stdout)
def test_a_file_changed_here_is_kept_and_the_new_one_offered_beside_it(self):
(self.tmp / ".opnexus-compose-sha").write_text(_sha(OLD) + "\n")
(self.tmp / "docker-compose.yml").write_text(OLD + " # my change\n")
r = self.refresh()
self.assertEqual(self.compose(), OLD + " # my change\n")
self.assertEqual((self.tmp / "docker-compose.yml.new").read_text(), NEW)
self.assertIn("BACKUP= ", r.stdout)
self.assertIn("docker-compose.override.yml", r.stdout)
def test_an_identical_file_only_records_its_checksum(self):
(self.tmp / "docker-compose.yml").write_text(NEW)
self.refresh()
self.assertEqual(self.recorded(), _sha(NEW))
self.assertFalse(list(self.tmp.glob("docker-compose.yml.bak-*")))
def test_an_old_release_without_a_compose_file_changes_nothing(self):
self.offered.unlink()
r = self.refresh()
self.assertEqual(self.compose(), OLD)
self.assertIn("vor 1.34", r.stdout)
def test_something_that_is_not_an_opnexus_compose_file_is_ignored(self):
self.offered.write_text("hello: world\n")
self.refresh()
self.assertEqual(self.compose(), OLD)
def test_without_cosign_nothing_is_taken_from_an_image(self):
r = self.refresh(no_cosign=True)
self.assertEqual(self.compose(), OLD)
self.assertIn("ohne cosign", r.stdout)
self.assertFalse(any(c.startswith("run ") for c in (self.log.read_text().splitlines() if self.log.exists() else [])))
def test_a_bad_signature_stops_before_anything_changes(self):
r = self.refresh(env_extra={"FAKE_COSIGN_RC": "1"})
self.assertNotEqual(r.returncode, 0)
self.assertEqual(self.compose(), OLD)
def test_rollback_puts_the_previous_file_and_checksum_back(self):
(self.tmp / ".opnexus-compose-sha").write_text(_sha(OLD) + "\n")
self.refresh()
r = self.run_body(f'prev_compose=docker-compose.yml.bak-alt prev_compose_sha={_sha(OLD)} restore_compose')
self.assertEqual(r.returncode, 0, r.stderr)
self.assertEqual(self.compose(), OLD)
self.assertEqual(self.recorded(), _sha(OLD))
def test_rollback_without_a_swapped_file_does_nothing(self):
r = self.run_body('prev_compose="" restore_compose; echo DONE')
self.assertIn("DONE", r.stdout)
self.assertEqual(self.compose(), OLD)
if __name__ == "__main__":
unittest.main()
+57
View File
@@ -144,3 +144,60 @@ class RefreshCli(unittest.TestCase):
if __name__ == "__main__":
unittest.main()
class RefreshCliOnDisk(RefreshCli):
"""B18: in the release bundle (CLI next to docker-compose.yml) the file on disk is replaced as well, so a later
`rollback` runs the new CLI; a repo checkout (scripts/opnexus -> updater/opnexus) is never touched."""
def run_copy(self, *, with_compose: bool):
bundle = self.tmp / "bundle"
bundle.mkdir()
copy = bundle / "opnexus"
shutil.copy2(SCRIPT, copy)
if with_compose:
(bundle / "docker-compose.yml").write_text("services: {}\n")
self.cli.write_text(NEW_CLI)
env = {**os.environ, "PATH": f"{self.tmp / 'bin'}:{os.environ['PATH']}", "OPNEXUS_DIR": str(self.tmp),
"FAKE_LOG": str(self.log), "FAKE_CLI": str(self.cli), "FAKE_DIGEST": DIGEST, "TMPDIR": str(self.tmp)}
for name in ("OPNEXUS_NO_CLI_REFRESH", "CLI_REFRESHED"):
env.pop(name, None)
body = f'source <(sed "/^case \\"\\${{1:-}}\\" in/,\\$d" "{copy}"); refresh_cli 9.9.9; echo CONTINUED'
result = subprocess.run(["bash", "-c", body, str(copy)], env=env, capture_output=True, text=True, timeout=30)
return result, copy
def test_bundle_cli_on_disk_is_replaced_with_a_backup(self):
r, copy = self.run_copy(with_compose=True)
self.assertIn("NEW-CLI __refreshed update 9.9.9", r.stdout)
self.assertEqual(copy.read_text(), NEW_CLI)
self.assertEqual(Path(str(copy) + ".bak").read_text(), SCRIPT.read_text())
self.assertTrue(os.access(copy, os.X_OK))
def test_a_cli_without_a_compose_file_next_to_it_is_left_alone(self):
r, copy = self.run_copy(with_compose=False)
self.assertIn("NEW-CLI __refreshed update 9.9.9", r.stdout)
self.assertEqual(copy.read_text(), SCRIPT.read_text())
self.assertFalse(Path(str(copy) + ".bak").exists())
class RefreshedCliInstallsItself(unittest.TestCase):
"""The first hop is started by an OLD CLI (no on-disk replacement in its refresh_cli): the new one, running from
its temp copy with __refreshed, puts itself into the bundle so a later `rollback` runs the new code."""
def test_the_refreshed_cli_replaces_the_bundle_file(self):
tmp = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, tmp, ignore_errors=True)
bundle = tmp / "bundle"
bundle.mkdir()
(bundle / "opnexus").write_text("#!/bin/bash\necho old\n")
(bundle / "docker-compose.yml").write_text("services: {}\n")
(bundle / ".env").write_text("OPNEXUS_IMAGE_REPO=registry.example/opnexus\n") # a real install always has one
running = tmp / "opnexus-cli.abc123"
shutil.copy2(SCRIPT, running)
env = {**os.environ, "OPNEXUS_DIR": str(bundle)}
r = subprocess.run(["bash", str(running), "__refreshed", "no-such-command"], env=env, capture_output=True,
text=True, timeout=30)
self.assertEqual((bundle / "opnexus").read_text(), SCRIPT.read_text())
self.assertEqual((bundle / "opnexus.bak").read_text(), "#!/bin/bash\necho old\n")
self.assertFalse(running.exists()) # the temp copy is cleaned up as before
self.assertIn("CLI auf der Platte aktualisiert", r.stdout)