Compare commits

...
1 Commits
Author SHA1 Message Date
Julian Boldt 295555cfd6 Release v1.22.0
Release / verify (push) Canceled after 0s
CI / checks (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / backend (push) Canceled after 0s
Release / test (push) Canceled after 0s
Release / publish (backend) (push) Canceled after 0s
Release / publish (frontend) (push) Canceled after 0s
Release / publish (proxy) (push) Canceled after 0s
Release / publish (updater) (push) Canceled after 0s
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
  (Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
  Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
  Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
2026-10-20 01:42:30 +02:00
31 changed files with 746 additions and 46 deletions

No files matched your search

+7
View File
@@ -31,6 +31,13 @@ den Einstellungen.
Nächste Codenamen (der Reihe nach, wenn die jeweils nächste MINOR-Version
ansteht): Merlin …
## [1.22.0] "Peregrine" — 2026-10-20
- **Standort-Variablen**: `{{NAME}}`-Platzhalter in Regeln, Aliasen, Portweiterleitungen, 1:1- und Outbound-NAT, je Host mit eigenem Wert
(Host-Seite › Standort-Variablen, listet die verwendeten Platzhalter automatisch). Je Host aufgelöst in Vorschau, Ausrollen, Zählern und
Audit; fehlt ein Wert, wird nichts gesendet (Vorschau „Variable fehlt“, Ausrollen wird vor jedem Schreiben abgelehnt, zusätzlich eine
Sperre im API-Client). Neue Tabelle `host_variables` (Migration `0016`).
## [1.21.0] "Osprey" — 2026-10-20
- **Management-Tunnel** (System › Management-Tunnel): Firewalls hinter NAT/CGNAT verwalten. Ein optionaler WireGuard-Hub auf dem
+2 -2
View File
@@ -7,7 +7,7 @@ Zentrale Verwaltung mehrerer [OPNsense](https://opnsense.org)-Firewalls: Regeln,
![Zentrale Firewall-Regeln](docs/screenshots/readme-rules-de.webp)
> **Stand:** `1.21.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
> **Stand:** `1.22.0` ist die aktuelle stabile Version. Entwickelt und getestet wurde gegen echte OPNsense-Instanzen; eine frische Installation aus dem Release-Bundle sowie Updates und Rollbacks sind durchgängig geprüft. Ecken und Kanten bitte melden.
## Funktionen
@@ -62,7 +62,7 @@ Für den Zugriff von anderen Rechnern und den Produktivbetrieb mit TLS (mitgelie
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.21.0
OPNEXUS_VERSION=1.22.0
```
```bash
+2 -2
View File
@@ -7,7 +7,7 @@ Central management for multiple [OPNsense](https://opnsense.org) firewalls: rule
![Central firewall rules](docs/screenshots/readme-rules-en.webp)
> **Status:** `1.21.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
> **Status:** `1.22.0` is the current stable release. It has been developed and tested against real OPNsense instances; a clean install from the release bundle, updates and rollbacks have been verified end to end. Please report rough edges.
## What it does
@@ -64,7 +64,7 @@ For access from other machines and for production use with TLS (bundled Caddy pr
```bash
OPNEXUS_IMAGE_REPO=git.opnexus.dev/opnexus
OPNEXUS_VERSION=1.21.0
OPNEXUS_VERSION=1.22.0
```
```bash
+2
View File
@@ -157,6 +157,8 @@ class ErrorCode(str, Enum):
API_TOKEN_LIMIT_REACHED = "API_TOKEN_LIMIT_REACHED"
OFFSITE_TEST_FAILED = "OFFSITE_TEST_FAILED"
OIDC_TEST_FAILED = "OIDC_TEST_FAILED"
VARIABLES_MISSING = "VARIABLES_MISSING"
VARIABLE_INVALID = "VARIABLE_INVALID"
OVERLAY_INVALID = "OVERLAY_INVALID"
OVERLAY_SUBNET_LOCKED = "OVERLAY_SUBNET_LOCKED"
OVERLAY_NAME_TAKEN = "OVERLAY_NAME_TAKEN"
+9 -2
View File
@@ -18,7 +18,7 @@ from app.security_middleware import OriginCheckMiddleware, SecurityHeadersMiddle
from app.config import settings
from app.database import SessionLocal
from app import migrate
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router, tokens as tokens_router, metrics as metrics_router, mcp as mcp_router, compliance as compliance_router, audit_export as audit_export_router, backups as backups_router, oidc as oidc_router, governance as governance_router, overlay as overlay_router
from app.routers import auth, hosts, logs, rules, groups, aliases, port_forward, search as search_router, users as users_router, tokens as tokens_router, metrics as metrics_router, mcp as mcp_router, compliance as compliance_router, audit_export as audit_export_router, backups as backups_router, oidc as oidc_router, governance as governance_router, overlay as overlay_router, variables as variables_router
from app.routers import one_to_one
from app.services.one_to_one_deploy_service import sweep_expired_one_to_one_deployments
from app.routers import outbound_nat
@@ -52,7 +52,7 @@ from app.routers import scheduled_maintenance as scheduled_maintenance_router
from app.services import scheduled_maintenance_service
from app.services import compliance_service
from app.services import audit_export_service
from app.services import overlay_service
from app.services import overlay_service, variables_service
from app.services import backup_schedule_service
from app.services import carp_event_service, carp_maintenance_window_service
@@ -396,6 +396,12 @@ async def _safe_validation_error(request: Request, exc: RequestValidationError):
)
@app.exception_handler(variables_service.MissingVariables)
async def _missing_variables(request: Request, exc: variables_service.MissingVariables):
"""A deploy that needs a value no target host has defined is refused before anything is written."""
return JSONResponse(status_code=422, content={"detail": {"code": ErrorCode.VARIABLES_MISSING.value, "params": {"details": str(exc)}}})
@app.exception_handler(DBAPIError)
async def _database_text_errors(request: Request, exc: DBAPIError):
"""A NUL byte in any text field cannot be stored by Postgres and used to
@@ -451,6 +457,7 @@ app.include_router(backups_router.router, prefix="/api")
app.include_router(oidc_router.router, prefix="/api")
app.include_router(governance_router.router, prefix="/api")
app.include_router(overlay_router.router, prefix="/api")
app.include_router(variables_router.router, prefix="/api")
app.include_router(logs.router, prefix="/api")
app.include_router(groups.router, prefix="/api")
app.include_router(aliases.router, prefix="/api")
+4
View File
@@ -44,3 +44,7 @@ class OpnsenseHost(Base):
policy_hierarchy_group: Mapped["PolicyHierarchyGroup | None"] = relationship(
"PolicyHierarchyGroup", back_populates="hosts"
)
# Values for {{NAME}} placeholders (services/variables_service.py); always loaded with the host.
variables: Mapped[list["HostVariable"]] = relationship(
"HostVariable", lazy="selectin", cascade="all, delete-orphan", order_by="HostVariable.name"
)
+18
View File
@@ -0,0 +1,18 @@
"""Per-host values for ``{{NAME}}`` placeholders in centrally authored objects (see services/variables_service.py)."""
import uuid
from sqlalchemy import ForeignKey, String, UniqueConstraint
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.database import Base
class HostVariable(Base):
__tablename__ = "host_variables"
__table_args__ = (UniqueConstraint("host_id", "name", name="uq_host_variable"),)
id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
host_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), ForeignKey("opnsense_hosts.id", ondelete="CASCADE"), index=True)
name: Mapped[str] = mapped_column(String(41), nullable=False)
value: Mapped[str] = mapped_column(String(255), nullable=False)
+98
View File
@@ -0,0 +1,98 @@
"""Site variables of one host: the values for {{NAME}} placeholders, and which placeholders its objects use."""
import uuid
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from app.database import get_db
from app.errors import ErrorCode, api_error
from app.models.alias import FirewallAlias
from app.models.host import OpnsenseHost
from app.models.host_variable import HostVariable
from app.models.one_to_one import OneToOneRule
from app.models.outbound_nat import OutboundNatRule
from app.models.policy import DeviceGroup
from app.models.port_forward import PortForwardRule
from app.models.rule import FirewallRule
from app.services import alias_deploy_service, one_to_one_deploy_service, outbound_nat_deploy_service
from app.services import port_forward_deploy_service, rule_deploy_service, variables_service
from app.services.policy_hierarchy_target_service import hydrate_hierarchy_targets
from app.services.policy_service import effective_hosts
router = APIRouter(prefix="/hosts/{host_id}/variables", tags=["variables"])
_KINDS = (
("rule", FirewallRule, rule_deploy_service),
("alias", FirewallAlias, alias_deploy_service),
("port_forward", PortForwardRule, port_forward_deploy_service),
("one_to_one", OneToOneRule, one_to_one_deploy_service),
("outbound_nat", OutboundNatRule, outbound_nat_deploy_service),
)
class VariablesUpdate(BaseModel):
variables: dict[str, str] = Field(default_factory=dict, max_length=200)
async def _host(db: AsyncSession, host_id: uuid.UUID) -> OpnsenseHost:
host = await db.get(OpnsenseHost, host_id)
if host is None:
raise api_error(404, ErrorCode.HOST_NICHT_GEFUNDEN)
return host
async def _usage(db: AsyncSession, host: OpnsenseHost) -> dict[str, list[dict]]:
"""name -> objects (that are deployed to this host) which use it."""
used: dict[str, list[dict]] = {}
for kind, model, service in _KINDS:
entities = (await db.execute(
select(model).options(selectinload(model.targets), selectinload(model.groups).selectinload(DeviceGroup.hosts))
)).scalars().unique().all()
await hydrate_hierarchy_targets(db, entities)
for entity in entities:
if host.id not in {h.id for h in effective_hosts(entity)}:
continue
label = getattr(entity, "description", None) or getattr(entity, "name", None) or str(entity.id)
for name in variables_service.names_in_payload(service._base_payload(entity)):
used.setdefault(name, []).append({"kind": kind, "id": str(entity.id), "label": label})
return used
async def _read(db: AsyncSession, host: OpnsenseHost) -> dict:
values = variables_service.host_values(host)
used = await _usage(db, host)
names = sorted(set(values) | set(used))
return {"variables": [
{"name": n, "value": values.get(n), "used_by": used.get(n, []), "missing": n in used and n not in values}
for n in names
]}
@router.get("")
async def read(host_id: uuid.UUID, db: AsyncSession = Depends(get_db)):
return await _read(db, await _host(db, host_id))
@router.put("")
async def replace(host_id: uuid.UUID, payload: VariablesUpdate, db: AsyncSession = Depends(get_db)):
"""Replace this host's variables with exactly the given set."""
host = await _host(db, host_id)
for name, value in payload.variables.items():
if not variables_service.NAME_RE.match(name):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Name „{name}“: Großbuchstaben, Ziffern und _, beginnend mit einem Buchstaben")
if not variables_service.VALUE_RE.match(value):
raise api_error(422, ErrorCode.VARIABLE_INVALID, reason=f"Wert von {name}: 1–255 Zeichen aus Buchstaben, Ziffern und . : / , @ + - _ Leerzeichen")
existing = {v.name: v for v in host.variables}
for name in set(existing) - set(payload.variables):
host.variables.remove(existing[name])
for name, value in payload.variables.items():
if name in existing:
existing[name].value = value
else:
host.variables.append(HostVariable(name=name, value=value))
await db.commit()
await db.refresh(host)
return await _read(db, host)
+9 -3
View File
@@ -18,7 +18,7 @@ from app.models.host import OpnsenseHost
from app.models.alias import FirewallAlias
from app.models.alias_deployment import AliasDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -27,7 +27,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(alias: FirewallAlias) -> dict:
def _build_payload(alias: FirewallAlias, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(alias)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(alias: FirewallAlias) -> dict:
return {
"enabled": "1" if alias.enabled else "0",
"name": alias.name,
@@ -38,7 +43,7 @@ def _build_payload(alias: FirewallAlias) -> dict:
async def _deploy_to_host(db: AsyncSession, alias: FirewallAlias, host: OpnsenseHost) -> AliasDeployment:
payload = _build_payload(alias)
payload = _build_payload(alias, host)
existing = await db.scalar(
select(AliasDeployment).where(AliasDeployment.alias_id == alias.id, AliasDeployment.host_id == host.id)
)
@@ -79,6 +84,7 @@ async def deploy_alias(db: AsyncSession, alias: FirewallAlias, hosts: list[Opnse
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, alias, hosts) # before anything is written
deployments = [await _deploy_to_host(db, alias, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
+10 -4
View File
@@ -9,6 +9,7 @@ import json
from app.models.policy import AliasRevision
from app.services.alias_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
ALIAS_FIELDS = ("name", "type", "content", "description", "enabled")
@@ -28,13 +29,17 @@ def revision(db, alias, note):
def preview(alias):
payload = _build_payload(alias)
deployments = {d.host_id: d for d in alias.deployments}
hosts = effective_hosts(alias)
blocked = any(d.status in ("pending", "revert_failed") for d in alias.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(alias, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_alias_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -61,6 +66,8 @@ def preview(alias):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diesen Alias, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in alias.deployments):
@@ -71,7 +78,7 @@ def preview(alias):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in alias.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -79,12 +86,11 @@ def workflow(alias, deployments):
hosts = effective_hosts(alias)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(alias)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, alias, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+4 -1
View File
@@ -88,7 +88,10 @@ def deployment_history(session, flush_context, instances):
if entity is not None and action == "deploy":
from importlib import import_module
builder = import_module(f"app.services.{kind}_deploy_service")._build_payload
details["requested_payload"] = builder(entity)
try:
details["requested_payload"] = builder(entity, host) if host is not None else builder(entity)
except Exception: # a missing variable must not break the audit write; record the unresolved intent instead
details["requested_payload"] = builder(entity)
session.add(make_event(kind=kind, action=action, status="removed" if deleted else dep.status,
entity_id=str(entity_id), entity_name=(getattr(entity, "name", None) or getattr(entity, "description", None)),
host_id=str(dep.host_id), host_name=host.name if host else None,
@@ -9,7 +9,7 @@ from app.models.host import OpnsenseHost
from app.models.one_to_one import OneToOneRule
from app.models.one_to_one_deployment import OneToOneDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -18,7 +18,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: OneToOneRule) -> dict:
def _build_payload(rule: OneToOneRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: OneToOneRule) -> dict:
return {
"description": rule.description,
"interface": rule.interface,
@@ -35,7 +40,7 @@ def _build_payload(rule: OneToOneRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: OneToOneRule, host: OpnsenseHost) -> OneToOneDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(OneToOneDeployment).where(
OneToOneDeployment.one_to_one_id == rule.id, OneToOneDeployment.host_id == host.id
@@ -78,6 +83,7 @@ async def deploy_one_to_one(db: AsyncSession, rule: OneToOneRule, hosts: list[Op
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
@@ -8,6 +8,7 @@ import json
from app.models.policy import OneToOneRevision
from app.services.one_to_one_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
ONE_TO_ONE_FIELDS = ('description', 'interface', 'source_net', 'destination_net', 'source_not', 'destination_not', 'enabled', 'type', 'external', 'natreflection', 'log')
@@ -27,13 +28,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_one_to_one_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -60,6 +65,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -70,7 +77,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -78,12 +85,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+11
View File
@@ -51,7 +51,18 @@ def _client(host: OpnsenseHost) -> httpx.AsyncClient:
)
def _refuse_unresolved_placeholders(method: str, path: str, kwargs: dict) -> None:
"""Last line of defence for site variables: a payload that still contains a {{NAME}} placeholder was never
resolved for this host and must not reach a firewall."""
if method.upper() == "GET" or "json" not in kwargs:
return
from app.services.variables_service import PLACEHOLDER
if PLACEHOLDER.search(json.dumps(kwargs["json"], default=str)):
raise OpnsenseAPIError(f"{path}: Die Anfrage enthält eine nicht aufgelöste Variable ({{{{NAME}}}}) und wird nicht gesendet")
async def _do_request(host: OpnsenseHost, method: str, path: str, **kwargs) -> httpx.Response:
_refuse_unresolved_placeholders(method, path, kwargs)
async with _client(host) as client:
try:
resp = await client.request(method, path, **kwargs)
@@ -10,7 +10,7 @@ from app.models.outbound_nat import OutboundNatRule
from app.models.outbound_nat_deployment import OutboundNatDeployment
from app.services.outbound_nat_mode import require_manual_mode
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -19,7 +19,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: OutboundNatRule) -> dict:
def _build_payload(rule: OutboundNatRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: OutboundNatRule) -> dict:
return {
"description": rule.description,
"interface": rule.interface,
@@ -39,7 +44,7 @@ def _build_payload(rule: OutboundNatRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: OutboundNatRule, host: OpnsenseHost) -> OutboundNatDeployment:
await require_manual_mode(host)
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(OutboundNatDeployment).where(
OutboundNatDeployment.outbound_nat_id == rule.id, OutboundNatDeployment.host_id == host.id
@@ -82,6 +87,7 @@ async def deploy_outbound_nat(db: AsyncSession, rule: OutboundNatRule, hosts: li
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = []
try:
for host in hosts:
@@ -8,6 +8,7 @@ import json
from app.models.policy import OutboundNatRevision
from app.services.outbound_nat_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
OUTBOUND_NAT_FIELDS = ('description', 'interface', 'source_net', 'destination_net', 'source_not', 'destination_not', 'enabled', 'ipprotocol', 'protocol', 'source_port', 'destination_port', 'target', 'target_port')
@@ -27,13 +28,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_outbound_nat_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -60,6 +65,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -70,7 +77,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -78,12 +85,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+10 -4
View File
@@ -6,6 +6,7 @@ from sqlalchemy import text
from app.models.policy import RuleRevision
from app.services.rule_deploy_service import _build_payload
from app.services import variables_service
RULE_FIELDS = ("description", "interface", "direction", "ipprotocol", "protocol", "action", "source_net", "source_port", "destination_net", "destination_port", "enabled", "log")
@@ -45,13 +46,17 @@ def ensure_editable(rule):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_rule_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -78,6 +83,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten ihre Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -89,7 +96,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -98,12 +105,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
@@ -21,7 +21,7 @@ from app.models.host import OpnsenseHost
from app.models.port_forward import PortForwardRule
from app.models.port_forward_deployment import PortForwardDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -30,7 +30,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: PortForwardRule) -> dict:
def _build_payload(rule: PortForwardRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: PortForwardRule) -> dict:
return {
"disabled": "0" if rule.enabled else "1",
"interface": rule.interface,
@@ -47,7 +52,7 @@ def _build_payload(rule: PortForwardRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: PortForwardRule, host: OpnsenseHost) -> PortForwardDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(PortForwardDeployment).where(
PortForwardDeployment.port_forward_id == rule.id, PortForwardDeployment.host_id == host.id
@@ -90,6 +95,7 @@ async def deploy_port_forward(db: AsyncSession, rule: PortForwardRule, hosts: li
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
@@ -8,6 +8,7 @@ import json
from app.models.policy import PortForwardRevision
from app.services.port_forward_deploy_service import _build_payload
from app.services import variables_service
from app.services.policy_service import effective_hosts
PORT_FORWARD_FIELDS = (
@@ -30,13 +31,17 @@ def revision(db, rule, note):
def preview(rule):
payload = _build_payload(rule)
deployments = {d.host_id: d for d in rule.deployments}
hosts = effective_hosts(rule)
blocked = any(d.status in ("pending", "revert_failed") for d in rule.deployments)
rows = []
for host in hosts:
dep = deployments.get(host.id)
try:
payload = _build_payload(rule, host)
except variables_service.MissingVariables as exc:
rows.append(variables_service.preview_row_missing(host, exc.missing[host.name], dep))
continue
before = dep.last_payload if dep else None
action = "create" if not dep or not dep.opnsense_port_forward_uuid else "update"
if before == payload and dep and dep.status in ("success", "reverted"):
@@ -63,6 +68,8 @@ def preview(rule):
warnings.append("Offene Bestätigung oder fehlgeschlagener Rollback: Push gesperrt.")
if any(r["action"] == "retained" for r in rows):
warnings.append("Frühere Ziele behalten diese Regel, bis du sie hier gezielt entfernst. Diese Löschung hat keinen automatischen Rollback.")
if any(r["action"] == "missing_variable" for r in rows):
warnings.append("Für mindestens einen Ziel-Host fehlt der Wert einer Variable ({{NAME}}). Erst auf der Host-Seite unter „Variablen“ eintragen.")
if any(r["action"] == "disabled" for r in rows):
warnings.append("Deaktivierte Hosts werden übersprungen.")
if any(d.status == "error" for d in rule.deployments):
@@ -73,7 +80,7 @@ def preview(rule):
d.host.base_url, d.host.enabled, d.host.verify_tls) for d in rule.deployments], key=lambda row: row[0])}
token = hashlib.sha256(json.dumps(state, sort_keys=True, default=str).encode()).hexdigest()
return {"token": token, "hosts": rows, "warnings": warnings,
"can_deploy": not blocked and any(r["action"] in ("create", "update") for r in rows),
"can_deploy": not blocked and not any(r["action"] == "missing_variable" for r in rows) and any(r["action"] in ("create", "update") for r in rows),
"basis": "Zuletzt von OPNexus gespeicherter Deploy-Stand; kein Live-Abgleich."}
@@ -81,12 +88,11 @@ def workflow(rule, deployments):
hosts = effective_hosts(rule)
active = [h for h in hosts if h.enabled]
by_host = {d.host_id: d for d in deployments}
payload = _build_payload(rule)
target_ids = {h.id for h in hosts}
counts = {
"targets": len(hosts),
"active_targets": len(active),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != payload),
"changes": sum(1 for h in active if h.id not in by_host or by_host[h.id].last_payload != variables_service.payload_or_none(_build_payload, rule, h)),
"pending": sum(d.status == "pending" for d in deployments),
"errors": sum(d.status in ("error", "revert_failed") for d in deployments),
"retained": sum(d.host_id not in target_ids for d in deployments),
+9 -3
View File
@@ -27,7 +27,7 @@ from app.models.host import OpnsenseHost
from app.models.rule import FirewallRule
from app.models.rule_deployment import RuleDeployment
from app.services.audit_service import tracked_rollback
from app.services import opnsense_client
from app.services import opnsense_client, variables_service
from app.services.deploy_flow import push_to_host
from app.services.opnsense_client import OpnsenseAmbiguousError, OpnsenseAPIError
@@ -36,7 +36,12 @@ def _now() -> datetime:
return datetime.now(timezone.utc)
def _build_payload(rule: FirewallRule) -> dict:
def _build_payload(rule: FirewallRule, host: OpnsenseHost | None = None) -> dict:
payload = _base_payload(rule)
return variables_service.resolve(payload, host) if host is not None else payload
def _base_payload(rule: FirewallRule) -> dict:
return {
"enabled": "1" if rule.enabled else "0",
"action": rule.action,
@@ -55,7 +60,7 @@ def _build_payload(rule: FirewallRule) -> dict:
async def _deploy_to_host(db: AsyncSession, rule: FirewallRule, host: OpnsenseHost) -> RuleDeployment:
payload = _build_payload(rule)
payload = _build_payload(rule, host)
existing = await db.scalar(
select(RuleDeployment).where(RuleDeployment.rule_id == rule.id, RuleDeployment.host_id == host.id)
)
@@ -98,6 +103,7 @@ async def deploy_rule(db: AsyncSession, rule: FirewallRule, hosts: list[Opnsense
from app.services.host_maintenance_service import ensure_no_maintenance
from app.services.policy_hierarchy_deploy_service import reassert_after_managed_change
await ensure_no_maintenance(db, hosts)
variables_service.ensure_resolvable(_build_payload, rule, hosts) # before anything is written
deployments = [await _deploy_to_host(db, rule, host) for host in hosts]
await db.flush()
for host, deployment in zip(hosts, deployments):
+78
View File
@@ -0,0 +1,78 @@
"""Site variables: ``{{NAME}}`` placeholders in centrally authored objects, filled per host.
A rule, alias or NAT entry can say ``{{LAN_NET}}`` instead of a literal network; every host that object is
deployed to supplies its own value (Host page > Variables). Resolution happens when the payload for one host is
built, so preview, deploy, the change counters and the audit all see the host's real values.
Fail closed: a placeholder without a value for a target host is never pushed. Preview marks that host
``missing_variable``, the deploy is refused before anything is written, and as a last line of defence
``opnsense_client`` refuses to send any payload that still contains a placeholder.
"""
import re
from typing import Callable
from app.models.host import OpnsenseHost
NAME_RE = re.compile(r"^[A-Z][A-Z0-9_]{0,39}$")
VALUE_RE = re.compile(r"^[\w .:/,@+\-]{1,255}$") # networks, addresses, hostnames, names, lists: no braces, quotes or newlines
PLACEHOLDER = re.compile(r"\{\{\s*([A-Z][A-Z0-9_]{0,39})\s*\}\}")
class MissingVariables(Exception):
"""One or more hosts lack values for placeholders the object uses. ``missing`` maps host name -> names."""
def __init__(self, missing: dict[str, list[str]]):
self.missing = missing
super().__init__("; ".join(f"{host}: {', '.join(names)}" for host, names in missing.items()))
def names_in(value) -> set[str]:
return set(PLACEHOLDER.findall(value)) if isinstance(value, str) else set()
def names_in_payload(payload: dict) -> set[str]:
found: set[str] = set()
for value in payload.values():
found |= names_in(value)
return found
def host_values(host: OpnsenseHost) -> dict[str, str]:
return {v.name: v.value for v in host.variables}
def resolve(payload: dict, host: OpnsenseHost) -> dict:
"""The payload with this host's values filled in; raises MissingVariables if one is not defined."""
values = host_values(host)
missing = sorted(n for n in names_in_payload(payload) if n not in values)
if missing:
raise MissingVariables({host.name: missing})
return {k: PLACEHOLDER.sub(lambda m: values[m.group(1)], v) if isinstance(v, str) else v for k, v in payload.items()}
def ensure_resolvable(builder: Callable, entity, hosts: list[OpnsenseHost]) -> None:
"""Raise MissingVariables (listing every affected host) before anything is written."""
missing: dict[str, list[str]] = {}
for host in hosts:
try:
builder(entity, host)
except MissingVariables as exc:
missing.update(exc.missing)
if missing:
raise MissingVariables(missing)
def payload_or_none(builder: Callable, entity, host: OpnsenseHost):
try:
return builder(entity, host)
except MissingVariables:
return None
def preview_row_missing(host: OpnsenseHost, names: list[str], dep) -> dict:
return {
"host_id": str(host.id), "host_name": host.name, "action": "missing_variable",
"status": dep.status if dep else None,
"message": "Variable fehlt: " + ", ".join(names), "can_remove": False,
"can_retry_rollback": False, "changes": [],
}
+2 -2
View File
@@ -6,5 +6,5 @@ convention — but both are bumped together whenever a release ships).
Versioning scheme: see CHANGELOG.md ("Versionierung").
"""
APP_VERSION = "1.21.0"
APP_CODENAME = "Osprey"
APP_VERSION = "1.22.0"
APP_CODENAME = "Peregrine"
@@ -0,0 +1,34 @@
"""Standort-Variablen: Werte für {{NAME}}-Platzhalter je Host.
Revision ID: 0016
Revises: 0015
Create Date: 2026-10-20
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision = "0016"
down_revision = "0015"
branch_labels = None
depends_on = None
def upgrade() -> None:
if sa.inspect(op.get_bind()).has_table("host_variables"):
return
op.create_table(
"host_variables",
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("host_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("name", sa.String(length=41), nullable=False),
sa.Column("value", sa.String(length=255), nullable=False),
sa.ForeignKeyConstraint(["host_id"], ["opnsense_hosts.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("host_id", "name", name="uq_host_variable"),
)
op.create_index("ix_host_variables_host_id", "host_variables", ["host_id"])
def downgrade() -> None:
op.execute("DROP TABLE IF EXISTS host_variables")
+213
View File
@@ -0,0 +1,213 @@
"""Site variables: {{NAME}} placeholders filled per host, failing closed when a value is missing."""
import json
import httpx
import pytest
from app.models.host import OpnsenseHost
from app.models.host_variable import HostVariable
from app.services import variables_service as vs
from tests.test_aliases import _alias_payload
from tests.test_port_forward import _pf_payload
from tests.test_rules import _create_host, _rule_payload
def _host(name="fw", **values):
host = OpnsenseHost(name=name, base_url="https://x", api_key="k")
host.variables = [HostVariable(name=n, value=v) for n, v in values.items()]
return host
# ── unit ─────────────────────────────────────────────────────────────────
def test_placeholders_are_found_and_resolved():
payload = {"source_net": "{{LAN_NET}}", "destination_net": "any", "description": "ssh to {{ SITE }} ({{LAN_NET}})", "enabled": "1", "port": 22}
assert vs.names_in_payload(payload) == {"LAN_NET", "SITE"}
resolved = vs.resolve(payload, _host(LAN_NET="192.168.7.0/24", SITE="Nord"))
assert resolved == {"source_net": "192.168.7.0/24", "destination_net": "any", "description": "ssh to Nord (192.168.7.0/24)", "enabled": "1", "port": 22}
assert payload["source_net"] == "{{LAN_NET}}" # the original is untouched
def test_a_missing_value_raises_naming_host_and_variable():
with pytest.raises(vs.MissingVariables) as exc:
vs.resolve({"a": "{{A}}", "b": "{{B}}"}, _host("fw-nord", A="1"))
assert exc.value.missing == {"fw-nord": ["B"]}
@pytest.mark.parametrize("text", ["lowercase {{lan}}", "{LAN_NET}", "{{ }}", "{{1ABC}}", "$LAN_NET", "{{LAN-NET}}"])
def test_things_that_are_not_placeholders_are_left_alone(text):
assert vs.names_in(text) == set()
assert vs.resolve({"x": text}, _host())["x"] == text
def test_values_are_inserted_literally_not_interpreted():
out = vs.resolve({"x": "{{A}}"}, _host(A="\\1 $& {{B}}"))
assert out["x"] == "\\1 $& {{B}}" # no regex-group or second-level expansion
def test_value_and_name_validation():
assert vs.NAME_RE.match("LAN_NET2") and not vs.NAME_RE.match("lan") and not vs.NAME_RE.match("2LAN") and not vs.NAME_RE.match("A" * 41)
for ok in ("192.168.7.0/24", "fw-nord.example.com", "10.0.0.1,10.0.0.2", "Filiale Nord", "a@b+c"):
assert vs.VALUE_RE.match(ok)
for bad in ("", "{{X}}", "a\nb", 'a"b', "a;b", "a" * 256, "<x>"):
assert not vs.VALUE_RE.match(bad)
# ── API ──────────────────────────────────────────────────────────────────
async def _mock_hosts(respx_mock, *bases):
respx_mock.get(url__regex=r".*/getVipStatus").mock(return_value=httpx.Response(200, json={"rows": []}))
# multi-host deploys compare what the interface name means on every target
respx_mock.get(url__regex=r".*/firewall/filter/getRule").mock(
return_value=httpx.Response(200, json={"rule": {"interface": {"lan": {"value": "LAN", "selected": 0}}}}))
respx_mock.get(url__regex=r".*/firewall/alias/searchItem.*").mock(return_value=httpx.Response(200, json={"rows": []}))
def _bodies(route):
return [json.loads(call.request.content) for call in route.calls]
@pytest.mark.asyncio
async def test_variables_api_roundtrip_validation_and_usage_discovery(client, respx_mock):
await _mock_hosts(respx_mock)
h1 = await _create_host(client, "fw1", "https://fw1.test")
h2 = await _create_host(client, "fw2", "https://fw2.test")
assert (await client.get(f"/api/hosts/{h1['id']}/variables")).json() == {"variables": []}
await client.post("/api/rules", json=_rule_payload([h1["id"]], description="ssh {{SITE}}", source_net="{{LAN_NET}}"))
await client.post("/api/aliases", json=_alias_payload([h2["id"]], content="{{DNS_IP}}"))
# fw1 uses LAN_NET and SITE, nothing yet defined -> both missing; fw2's alias is not fw1's business.
got = (await client.get(f"/api/hosts/{h1['id']}/variables")).json()["variables"]
assert [(v["name"], v["value"], v["missing"]) for v in got] == [("LAN_NET", None, True), ("SITE", None, True)]
assert got[0]["used_by"] == [{"kind": "rule", "id": got[0]["used_by"][0]["id"], "label": "ssh {{SITE}}"}]
put = await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"LAN_NET": "192.168.7.0/24", "EXTRA": "x"}})
assert put.status_code == 200, put.text
by_name = {v["name"]: v for v in put.json()["variables"]}
assert by_name["LAN_NET"]["missing"] is False and by_name["SITE"]["missing"] is True and by_name["EXTRA"]["used_by"] == []
# PUT replaces the whole set.
put = await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"SITE": "Nord"}})
assert {v["name"] for v in put.json()["variables"]} == {"LAN_NET", "SITE"} # LAN_NET only listed because the rule still uses it
assert next(v for v in put.json()["variables"] if v["name"] == "LAN_NET")["value"] is None
for bad in ({"lan": "1"}, {"OK": "bad;value"}, {"OK": ""}, {"OK": "{{X}}"}):
resp = await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": bad})
assert resp.status_code == 422 and resp.json()["detail"]["code"] == "VARIABLE_INVALID", bad
import uuid
assert (await client.get(f"/api/hosts/{uuid.uuid4()}/variables")).status_code == 404
@pytest.mark.asyncio
async def test_each_host_gets_its_own_values_in_preview_and_deploy(client, respx_mock):
await _mock_hosts(respx_mock)
h1 = await _create_host(client, "fw1", "https://fw1.test")
h2 = await _create_host(client, "fw2", "https://fw2.test")
await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"LAN_NET": "192.168.7.0/24", "SITE": "Nord"}})
await client.put(f"/api/hosts/{h2['id']}/variables", json={"variables": {"LAN_NET": "10.20.0.0/16", "SITE": "Sued"}})
rule = (await client.post("/api/rules", json=_rule_payload([h1["id"], h2["id"]], description="ssh {{SITE}}", source_net="{{LAN_NET}}"))).json()
preview = (await client.get(f"/api/rules/{rule['id']}/preview")).json()
rows = {r["host_name"]: r for r in preview["hosts"]}
after = lambda r: {c["field"]: c["after"] for c in r["changes"]} # noqa: E731
assert after(rows["fw1"])["source_net"] == "192.168.7.0/24" and after(rows["fw1"])["description"] == "ssh Nord"
assert after(rows["fw2"])["source_net"] == "10.20.0.0/16" and after(rows["fw2"])["description"] == "ssh Sued"
assert preview["can_deploy"] is True
routes = {b: respx_mock.post(f"https://{b}.test/api/firewall/filter/addRule").mock(return_value=httpx.Response(200, json={"result": "saved", "uuid": f"u-{b}"})) for b in ("fw1", "fw2")}
for b in ("fw1", "fw2"):
respx_mock.post(f"https://{b}.test/api/firewall/filter/apply").mock(return_value=httpx.Response(200, json={"status": "ok"}))
resp = await client.post(f"/api/rules/{rule['id']}/deploy", json={"preview_token": preview["token"]})
assert resp.status_code == 200, resp.text
sent = {b: _bodies(r)[0]["rule"] for b, r in routes.items()}
assert (sent["fw1"]["source_net"], sent["fw1"]["description"]) == ("192.168.7.0/24", "ssh Nord")
assert (sent["fw2"]["source_net"], sent["fw2"]["description"]) == ("10.20.0.0/16", "ssh Sued")
assert "{{" not in json.dumps(sent)
# The centrally stored rule keeps its placeholders; the deployment stores what was really sent.
assert (await client.get("/api/rules")).json()[0]["source_net"] == "{{LAN_NET}}"
# Changing a host's value shows up as a change for that host only.
await client.put(f"/api/hosts/{h2['id']}/variables", json={"variables": {"LAN_NET": "10.30.0.0/16", "SITE": "Sued"}})
for r in routes.values():
pass
await client.post(f"/api/rules/{rule['id']}/confirm")
again = {r["host_name"]: r["action"] for r in (await client.get(f"/api/rules/{rule['id']}/preview")).json()["hosts"]}
assert again == {"fw1": "unchanged", "fw2": "update"}
@pytest.mark.asyncio
async def test_a_missing_value_blocks_the_whole_deploy_before_anything_is_written(client, respx_mock):
await _mock_hosts(respx_mock)
h1 = await _create_host(client, "fw1", "https://fw1.test")
h2 = await _create_host(client, "fw2", "https://fw2.test")
await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"LAN_NET": "192.168.7.0/24"}})
rule = (await client.post("/api/rules", json=_rule_payload([h1["id"], h2["id"]], source_net="{{LAN_NET}}"))).json()
preview = (await client.get(f"/api/rules/{rule['id']}/preview")).json()
rows = {r["host_name"]: r for r in preview["hosts"]}
assert rows["fw2"]["action"] == "missing_variable" and "LAN_NET" in rows["fw2"]["message"] and rows["fw1"]["action"] == "create"
assert preview["can_deploy"] is False and any("Variable" in w for w in preview["warnings"])
# respx would fail on any unmocked firewall call: nothing may be sent, not even to the host that has its value.
resp = await client.post(f"/api/rules/{rule['id']}/deploy", json={"preview_token": preview["token"]})
assert resp.status_code == 422 and resp.json()["detail"]["code"] == "VARIABLES_MISSING"
assert "fw2: LAN_NET" in resp.json()["detail"]["params"]["details"] and "fw1" not in resp.json()["detail"]["params"]["details"]
assert (await client.get("/api/rules")).json()[0]["deployments"] == []
@pytest.mark.asyncio
async def test_aliases_and_port_forwards_use_variables_too(client, respx_mock):
await _mock_hosts(respx_mock)
h1 = await _create_host(client, "fw1", "https://fw1.test")
await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"DNS_IP": "10.9.9.9", "WEB": "10.7.0.20"}})
alias = (await client.post("/api/aliases", json=_alias_payload([h1["id"]], content="{{DNS_IP}}"))).json()
pf = (await client.post("/api/port-forward", json=_pf_payload([h1["id"]], target="{{WEB}}"))).json()
a_prev = (await client.get(f"/api/aliases/{alias['id']}/preview")).json()
assert {c["field"]: c["after"] for c in a_prev["hosts"][0]["changes"]}["content"] == "10.9.9.9"
p_prev = (await client.get(f"/api/port-forward/{pf['id']}/preview")).json()
assert {c["field"]: c["after"] for c in p_prev["hosts"][0]["changes"]}["target"] == "10.7.0.20"
await client.put(f"/api/hosts/{h1['id']}/variables", json={"variables": {"DNS_IP": "10.9.9.9"}})
p_prev = (await client.get(f"/api/port-forward/{pf['id']}/preview")).json()
assert p_prev["hosts"][0]["action"] == "missing_variable"
@pytest.mark.asyncio
async def test_workflow_state_does_not_crash_on_missing_values(client, respx_mock):
await _mock_hosts(respx_mock)
host = await _create_host(client, "fw1", "https://fw1.test")
await client.post("/api/rules", json=_rule_payload([host["id"]], source_net="{{NOPE}}"))
listing = (await client.get("/api/rules")).json()
assert listing[0]["workflow_state"] == "draft" and listing[0]["workflow_counts"]["changes"] == 1
@pytest.mark.asyncio
async def test_the_client_refuses_any_payload_with_an_unresolved_placeholder(client, respx_mock):
from app.models.host import OpnsenseHost as H
from app.services import opnsense_client, opnsense_secret_store
import uuid
await _mock_hosts(respx_mock)
created = await _create_host(client, "fw1", "https://fw1.test")
from app.database import SessionLocal
async with SessionLocal() as db:
host = await db.get(H, uuid.UUID(created["id"]))
with pytest.raises(opnsense_client.OpnsenseAPIError, match="Variable"):
await opnsense_client.add_rule(host, {"source_net": "{{LAN_NET}}", "action": "pass"})
with pytest.raises(opnsense_client.OpnsenseAPIError, match="Variable"):
await opnsense_client.add_rule(host, {"nested": "x {{ A_B }} y"})
@pytest.mark.asyncio
async def test_variables_follow_the_host_and_operators_may_set_them(client, respx_mock):
from tests.test_api_tokens import _login
await _mock_hosts(respx_mock)
host = await _create_host(client, "fw1", "https://fw1.test")
await client.put(f"/api/hosts/{host['id']}/variables", json={"variables": {"A": "1"}})
assert (await client.post("/api/users", json={"username": "varop", "password": "variables-pass-1", "role": "operator"})).status_code == 201
op = await _login("varop", "variables-pass-1")
try:
assert (await op.put(f"/api/hosts/{host['id']}/variables", json={"variables": {"A": "2"}})).status_code == 200
finally:
await op.aclose()
assert (await client.post("/api/users", json={"username": "varro", "password": "variables-pass-1", "role": "read_only"})).status_code == 201
ro = await _login("varro", "variables-pass-1")
try:
assert (await ro.get(f"/api/hosts/{host['id']}/variables")).status_code == 200
assert (await ro.put(f"/api/hosts/{host['id']}/variables", json={"variables": {}})).status_code == 403
finally:
await ro.aclose()
await client.delete(f"/api/hosts/{host['id']}") # variables go with the host
from app.database import SessionLocal
from sqlalchemy import select
async with SessionLocal() as db:
assert (await db.scalars(select(HostVariable))).all() == []
+21
View File
@@ -524,6 +524,27 @@ automatischen und konfigurierten.
- Die NAT-Drift-Erkennung vergleicht die Knoten eines Clusters und meldet
Abweichungen.
### Standort-Variablen
Mit **Variablen** pflegst du ein Objekt einmal und rollst es auf Firewalls mit unterschiedlichen Netzen aus.
Schreibe `{{NAME}}` (Großbuchstaben, Ziffern und `_`, beginnend mit einem Buchstaben) in ein Feld einer
**Regel**, eines **Alias**, einer **Portweiterleitung**, einer **1:1-NAT-** oder **Outbound-NAT-Regel**, etwa als
Quelle `{{LAN_NET}}` oder als Beschreibung `Zugriff {{SITE}}`. Den Wert legst du je Firewall auf der Host-Seite
unter **Standort-Variablen** fest (`192.168.7.0/24`, `Filiale Nord`, …). Die Seite listet automatisch alle
Platzhalter, die die für diesen Host bestimmten Objekte verwenden, und zeigt, welche noch ohne Wert sind.
- **Je Host aufgelöst:** Vorschau, Ausrollen, die Änderungszähler und das Audit sehen die echten Werte des
jeweiligen Hosts; ändert sich ein Wert, erscheint nur bei diesem Host eine Änderung. Das zentral gespeicherte
Objekt behält seine Platzhalter.
- **Fehlt ein Wert, wird nichts gesendet:** Die Vorschau markiert den Host mit „Variable fehlt“, das Ausrollen
wird abgelehnt, bevor irgendein Host beschickt wurde, und als letzte Absicherung sendet OPNexus keine Anfrage
an eine Firewall, die noch einen Platzhalter enthält.
- **Werte** dürfen Buchstaben, Ziffern und `. : / , @ + - _` sowie Leerzeichen enthalten (höchstens 255 Zeichen);
sie werden wörtlich eingesetzt, nie als Muster oder weiterer Platzhalter ausgewertet.
- **Nicht** unterstützt: Platzhalter in VPN-Tunneln und in der Policy-Hierarchie (dort stehen die Werte direkt im
Objekt); Variablen für Gerätegruppen (Vorgabewerte) gibt es nicht, jeder Host hat seine eigenen. Operatoren dürfen
Variablen pflegen; ob sie auch ausrollen dürfen, regelt die [Aufgabentrennung](#benutzer-rollen-und-anmeldung).
---
## Netzwerk und Host-System
+3
View File
@@ -11,6 +11,7 @@ import { Spinner } from "@/components/ui/Spinner";
import { StatusDot } from "@/components/ui/StatusDot";
import { api, AliasOption, Host, HostBackup, LiveRule, PolicyAlias, RestoreOperation, Rule } from "@/lib/api";
import { ArrowLeft, Download, ExternalLink, GitCompare, RotateCcw, ShieldCheck, ShieldOff } from "lucide-react";
import { HostVariables } from "@/components/HostVariables";
import { BackupDiffModal } from "@/components/BackupDiffModal";
import Link from "next/link";
import { useParams } from "next/navigation";
@@ -424,6 +425,8 @@ export default function HostDetailPage() {
<HostServices host={host} />
<HostVariables host={host} />
{restoreHistory && restoreHistory.length > 0 && (
<>
<h2 className="mb-3 mt-8 text-lg font-semibold text-slate-900">{t("hostDetail.restoreHistory")}</h2>
+98
View File
@@ -0,0 +1,98 @@
"use client";
import { Badge } from "@/components/ui/Badge";
import { Button } from "@/components/ui/Button";
import { Card } from "@/components/ui/Card";
import { Input } from "@/components/ui/Field";
import { InfoTip } from "@/components/ui/Tooltip";
import { useToast } from "@/components/ui/Toast";
import { api, Host, HostVariable } from "@/lib/api";
import { useLanguage } from "@/components/LanguageProvider";
import { useReadOnly } from "@/lib/useReadOnly";
import { Plus, Trash2 } from "lucide-react";
import { useEffect, useState } from "react";
/** Site variables: values for {{NAME}} placeholders that rules, aliases and NAT entries use for this firewall. */
export function HostVariables({ host }: { host: Host }) {
const { t, errorMessage } = useLanguage();
const readOnly = useReadOnly();
const toast = useToast();
const [rows, setRows] = useState<HostVariable[] | null>(null);
const [values, setValues] = useState<Record<string, string>>({});
const [added, setAdded] = useState<string[]>([]);
const [newName, setNewName] = useState("");
const [error, setError] = useState<string | null>(null);
const [saving, setSaving] = useState(false);
function apply(list: HostVariable[]) {
setRows(list);
setValues(Object.fromEntries(list.map((v) => [v.name, v.value ?? ""])));
setAdded([]);
}
useEffect(() => {
api.getHostVariables(host.id).then((r) => apply(r.variables)).catch((e) => setError(errorMessage(e, "variables.loadFailed")));
}, [host.id]);
const names = [...(rows ?? []).map((r) => r.name), ...added];
const usage = (name: string) => rows?.find((r) => r.name === name);
async function save() {
setSaving(true);
setError(null);
try {
const payload = Object.fromEntries(names.filter((n) => (values[n] ?? "").trim() !== "").map((n) => [n, values[n].trim()]));
apply((await api.setHostVariables(host.id, payload)).variables);
toast.show(t("variables.saved"), "success");
} catch (e) {
setError(errorMessage(e, "variables.saveFailed"));
} finally {
setSaving(false);
}
}
function addName() {
const name = newName.trim().toUpperCase();
if (!name || names.includes(name)) return;
setAdded([...added, name]);
setNewName("");
}
const missing = (rows ?? []).filter((r) => r.missing && !(values[r.name] ?? "").trim()).length;
return (
<section className="mt-8">
<h2 className="mb-3 flex items-center gap-2 text-lg font-semibold text-slate-900">
{t("variables.title")}
<InfoTip text={t("variables.intro")} />
{missing > 0 && <Badge tone="warning">{t("variables.missing", { n: missing })}</Badge>}
</h2>
<Card className="p-5">
{error && <p role="alert" className="mb-3 text-sm text-red-600">{error}</p>}
{rows && names.length === 0 && <p className="text-sm text-slate-500">{t("variables.empty")}</p>}
<fieldset disabled={readOnly} className="space-y-3">
{names.map((name) => {
const info = usage(name);
return (
<div key={name} className="grid items-center gap-2 sm:grid-cols-[14rem_1fr_auto]">
<code className="font-mono text-sm text-slate-800">{`{{${name}}}`}</code>
<div>
<Input value={values[name] ?? ""} onChange={(e) => setValues({ ...values, [name]: e.target.value })} placeholder={info?.missing ? t("variables.required") : ""} aria-label={name} />
{info && info.used_by.length > 0 && <p className="mt-1 text-xs text-slate-500">{t("variables.usedBy", { list: info.used_by.map((u) => u.label).slice(0, 4).join(", ") })}</p>}
</div>
<Button type="button" variant="ghost" onClick={() => { setValues({ ...values, [name]: "" }); setAdded(added.filter((n) => n !== name)); }} aria-label={t("rules.delete")}>
<Trash2 size={14} />
</Button>
</div>
);
})}
<div className="flex flex-wrap items-center gap-2 pt-2">
<Input className="max-w-[14rem]" value={newName} onChange={(e) => setNewName(e.target.value.toUpperCase())} placeholder="LAN_NET" onKeyDown={(e) => { if (e.key === "Enter") { e.preventDefault(); addName(); } }} aria-label={t("variables.newName")} />
<Button type="button" variant="outline" onClick={addName}><Plus size={14} />{t("variables.add")}</Button>
<Button type="button" onClick={save} disabled={saving} loading={saving}>{t("common.save")}</Button>
</div>
</fieldset>
</Card>
</section>
);
}
+11
View File
@@ -819,6 +819,13 @@ export interface OverlayInstructions {
host_url: string;
}
export interface HostVariable {
name: string;
value: string | null;
used_by: { kind: string; id: string; label: string }[];
missing: boolean;
}
export interface User {
id: string;
username: string;
@@ -1472,6 +1479,10 @@ export const api = {
request<{ ok: boolean }>(`/overlay/peers/${id}`, { method: "PATCH", body: JSON.stringify(data) }),
deleteOverlayPeer: (id: string) => request<void>(`/overlay/peers/${id}`, { method: "DELETE" }),
getHostVariables: (id: string) => request<{ variables: HostVariable[] }>(`/hosts/${id}/variables`),
setHostVariables: (id: string, variables: Record<string, string>) =>
request<{ variables: HostVariable[] }>(`/hosts/${id}/variables`, { method: "PUT", body: JSON.stringify({ variables }) }),
listApiTokens: () => request<ApiToken[]>("/tokens"),
createApiToken: (data: { name: string; scope: TokenScope; expires_in_days: number | null; current_password: string }) =>
request<ApiToken & { token: string }>("/tokens", { method: "POST", body: JSON.stringify(data) }),
+16
View File
@@ -603,6 +603,8 @@
"OVERLAY_SUBNET_LOCKED": "Das Overlay-Netz lässt sich nicht ändern, solange Firewalls eingetragen sind. Erst die Einträge löschen.",
"OVERLAY_NAME_TAKEN": "Dieser Name ist schon vergeben.",
"OVERLAY_PEER_NOT_FOUND": "Overlay-Eintrag nicht gefunden",
"VARIABLES_MISSING": "Es fehlen Werte für Variablen: {details}. Trage sie auf der Host-Seite unter „Variablen“ ein.",
"VARIABLE_INVALID": "Ungültige Variable: {reason}",
"OIDC_TEST_FAILED": "Der Test des Identitätsanbieters ist fehlgeschlagen: {reason}",
"OIDC_MANAGED_USER": "Dieses Konto wird vom Single Sign-on verwaltet: Rolle und Passwort ändern sich beim Identitätsanbieter.",
"OFFSITE_TEST_FAILED": "Der Test des Offsite-Ziels ist fehlgeschlagen: {reason}",
@@ -721,6 +723,7 @@
"unchanged": "Unverändert",
"disabled": "Deaktiviert · übersprungen",
"retained": "Früheres Ziel · bleibt bestehen",
"missing_variable": "Variable fehlt · übersprungen",
"missing_cert": "Kein Zertifikat/CA ausgewählt · wird übersprungen"
},
"loadFailed": "Laden fehlgeschlagen",
@@ -1569,6 +1572,19 @@
"firewallKeyHint": "Aus OPNsense: VPN › WireGuard › Instanzen. Leer lassen, um ihn zu entfernen.",
"hostHint": "Optional: welcher verwaltete Host über diesen Tunnel erreicht wird."
},
"variables": {
"title": "Standort-Variablen",
"intro": "Regeln, Aliase und NAT-Einträge können Platzhalter wie {{LAN_NET}} enthalten. Jede Firewall setzt dafür ihren eigenen Wert ein, ein Objekt bedient so viele Standorte mit unterschiedlichen Netzen. Eine Firewall ohne Wert für einen Platzhalter wird nie beschickt: die Vorschau zeigt es und das Ausrollen wird abgelehnt. Werte dürfen Buchstaben, Ziffern und . : / , @ + - _ sowie Leerzeichen enthalten.",
"loadFailed": "Die Variablen konnten nicht geladen werden",
"saved": "Variablen gespeichert",
"saveFailed": "Die Variablen konnten nicht gespeichert werden",
"empty": "Keine Platzhalter in Benutzung. Schreibe {{NAME}} (Großbuchstaben, Ziffern, _) in eine Regel, einen Alias oder NAT-Eintrag und lege den Wert hier fest.",
"missing": "{n} Wert(e) fehlen",
"required": "erforderlich",
"usedBy": "Verwendet von: {list}",
"newName": "Name der neuen Variable",
"add": "Hinzufügen"
},
"sso": {
"title": "Single Sign-on (OIDC)",
"intro": "Lässt Personen sich mit deinem Identitätsanbieter anmelden (Authentik, Keycloak, Entra ID, Google, …), statt mit einem hier gespeicherten Passwort. Die Rolle kommt bei jeder Anmeldung aus den Gruppen des Anbieters; ohne zugeordnete Gruppe kommt niemand hinein. Lokale Konten funktionieren weiter, du kommst also bei einem Ausfall des Anbieters immer mit dem Admin-Passwort hinein.",
+16
View File
@@ -603,6 +603,8 @@
"OVERLAY_SUBNET_LOCKED": "The overlay network cannot be changed while firewalls are registered. Delete the entries first.",
"OVERLAY_NAME_TAKEN": "This name is already taken.",
"OVERLAY_PEER_NOT_FOUND": "Overlay entry not found",
"VARIABLES_MISSING": "Values for variables are missing: {details}. Enter them on the host page under “Variables”.",
"VARIABLE_INVALID": "Invalid variable: {reason}",
"OIDC_TEST_FAILED": "The identity provider test failed: {reason}",
"OIDC_MANAGED_USER": "This account is managed by single sign-on: change role and password at the identity provider.",
"OFFSITE_TEST_FAILED": "The offsite target test failed: {reason}",
@@ -721,6 +723,7 @@
"unchanged": "Unchanged",
"disabled": "Disabled · skipped",
"retained": "Former target · stays in place",
"missing_variable": "Variable missing · skipped",
"missing_cert": "No certificate/CA selected · skipped"
},
"loadFailed": "Loading failed",
@@ -1569,6 +1572,19 @@
"firewallKeyHint": "From OPNsense: VPN › WireGuard › Instances. Leave empty to remove it.",
"hostHint": "Optional: which managed host is reached through this tunnel."
},
"variables": {
"title": "Site variables",
"intro": "Rules, aliases and NAT entries can contain placeholders like {{LAN_NET}}. Each firewall fills them with its own value, so one object serves many sites with different networks. A firewall without a value for a placeholder is never pushed: the preview says so and the deploy is refused. Values may contain letters, digits and . : / , @ + - _ and spaces.",
"loadFailed": "Could not load the variables",
"saved": "Variables saved",
"saveFailed": "Could not save the variables",
"empty": "No placeholders in use. Write {{NAME}} into a rule, alias or NAT entry (capital letters, digits and _) and define its value here.",
"missing": "{n} value(s) missing",
"required": "required",
"usedBy": "Used by: {list}",
"newName": "New variable name",
"add": "Add"
},
"sso": {
"title": "Single sign-on (OIDC)",
"intro": "Lets people sign in with your identity provider (Authentik, Keycloak, Entra ID, Google, …) instead of a password stored here. Roles come from the provider's groups at every login; without a mapped group nobody gets in. Local accounts keep working, so you can always sign in with the admin password if the provider is down.",
+2 -2
View File
@@ -1,6 +1,6 @@
// Manually bumped whenever a notable change ships — mirrors the dated
// entries already kept in the project README/CHANGELOG.md, no build
// tooling involved. Versioning scheme: see CHANGELOG.md ("Versionierung").
export const APP_VERSION = "1.21.0";
export const APP_CODENAME = "Osprey";
export const APP_VERSION = "1.22.0";
export const APP_CODENAME = "Peregrine";
export const APP_MAINTAINER = "Julian Boldt";