diff --git a/Reference-BUILD.de.md b/Reference-BUILD.de.md new file mode 100644 index 0000000..1241791 --- /dev/null +++ b/Reference-BUILD.de.md @@ -0,0 +1,399 @@ +# Romexis Docker Build-Prozess + +[Zurück zum README](README.de.md) | [English](BUILD.md) + +Dieses Dokument beschreibt den vollständigen Build-Prozess für die Multiarch-Romexis-Docker-Images. + +--- + +## Ziele des Builds + +Das Build-System ist darauf ausgelegt, folgende Ziele zu erfüllen: + +- reproduzierbare Docker-Builds +- wiederverwendbares Runtime-Basisimage +- native `amd64`- und `arm64`-Images +- Multiarch-Manifeste +- minimale finale Runtime-Images +- wartbare Installer-Extraktionslogik +- klare Trennung zwischen Laufzeitabhängigkeiten und Romexis-Anwendungsdateien + +--- + +## Image-Typen + +### Romexis Base Image + +Das Base Image wird aus `romexis-base/Dockerfile` gebaut. + +Es enthält gemeinsame Laufzeitabhängigkeiten: + +- Azul Zulu Java 11 Runtime mit JavaFX/OpenJFX-Unterstützung +- Microsoft SQL Server Kommandozeilenwerkzeuge +- Firebird-Clienttools und Bibliotheken +- gemeinsame Betriebssystem-Laufzeitbibliotheken + +Tags: + +```text +gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 +gitea.buchhorster.de/patrick/romexis-base-jre:11-arm64 +``` + +Das Base Image ist bewusst vom Romexis Server Image getrennt, damit es nicht für jede Romexis-Version erneut gebaut werden muss. + +### Romexis Server Image + +Das Server Image wird aus `romexis/Dockerfile` gebaut. + +Es enthält: + +- extrahierte Romexis-Serverdateien +- Romexis-Datenbank-SQL-Skripte +- native Chilkat-Laufzeitbibliothek +- Romexis Java Property Agent +- Runtime-Hilfsskripte +- Entrypoint- und Initialisierungslogik + +Architekturspezifische Tags: + +```text +gitea.buchhorster.de/patrick/romexis-server:-amd64 +gitea.buchhorster.de/patrick/romexis-server:-arm64 +``` + +Multiarch-Manifest-Tag: + +```text +gitea.buchhorster.de/patrick/romexis-server: +``` + +--- + +## Build-Argumente + +### Gemeinsame Argumente + +| Argument | Beschreibung | +|---------|--------------| +| `TARGETARCH` | Zielarchitektur, normalerweise `amd64` oder `arm64`. | +| `ROMEXIS_VERSION` | Angeforderte Romexis-Version oder Versionspräfix. | + +### Base-Image-Argumente + +| Argument | Beschreibung | +|---------|--------------| +| `IMAGE_VERSION` | OCI-Image-Label-Version des Base Images. | + +### Server-Image-Argumente + +| Argument | Beschreibung | +|---------|--------------| +| `ROMEXIS_BASE_IMAGE` | Basisimage-Referenz für die finale Romexis-Server-Stage. | +| `CHILKAT_VERSION` | Version der nativen Chilkat-Bibliothek. | + +--- + +## Versionsauflösung + +Romexis-Versionen werden in folgender Datei definiert: + +```text +romexis-payload/romexis-versions.env +``` + +Format: + +```text +6_5_3_444_203=https://content.planmeca.com/files/Planmeca_Romexis_6.5.3.444.203_Win.zip +``` + +Build-Eingabe: + +```bash +ROMEXIS_VERSION=6.5.3 +``` + +Das Dockerfile löst den neuesten passenden Eintrag auf und schreibt die tatsächlich verwendete Version nach: + +```text +/opt/romexis/version +``` + +Dadurch kann lokal oder in CI mit stabilen Versionspräfixen gebaut werden, während das Image intern die exakte Romexis-Version enthält. + +--- + +## Installer-Download und Extraktion + +Das Repository enthält keine Romexis-Binaries. + +Während der `romexis-build`-Stage: + +1. Die passende Installer-URL wird aus `romexis-versions.env` gelesen. +2. `download-romexis-installer-parts.py` lädt nur die benötigten Installerbestandteile herunter. +3. Die InstallShield-CAB-Datei wird durch `extract-and-copy-romexis-parts.sh` verarbeitet. +4. Die finale Romexis-Verzeichnisstruktur wird unter `/opt/romexis` aufgebaut. +5. SQL-Server-Initialisierungsskripte werden nach `/opt/romexis-mssql-db` kopiert. + +--- + +## Mapping-basierte Extraktion + +Die Datei: + +```text +romexis-payload/romexis-copy-map.tsv +``` + +ist die zentrale Zuordnung zwischen Installer-Komponenten und Zielverzeichnissen. + +Format: + +```text +SourceDestination +Broker_jar /opt/romexis/broker +Server_jar /opt/romexis/server +Server_Program_64bit/server/*.xml /opt/romexis/server +``` + +Das Extraktionsskript nutzt diese Datei für zwei Schritte: + +1. Ermitteln, welche Top-Level-CAB-Komponenten extrahiert werden müssen. +2. Kopieren der gemappten Dateien und Verzeichnisse an ihre Zielorte. + +Bei Glob-Mappings wie: + +```text +Server_Program_64bit/server/*.xml +``` + +wird die komplette Top-Level-Komponente `Server_Program_64bit` extrahiert, aber nur passende XML-Dateien werden in das Zielverzeichnis kopiert. + +Dadurch bleibt das Dockerfile unabhängig vom internen Romexis-Installerlayout. + +--- + +## Lokaler Build mit Docker Compose + +Verwendet wird: + +```text +docker-compose.build.yml +``` + +### Base Image bauen + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +``` + +### Romexis Server Image bauen + +```bash +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +``` + +### Lokal gebauten Stack starten + +```bash +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +### Build ohne Cache + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build --no-cache romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build --no-cache romexis +``` + +--- + +## Manueller Docker Build + +### Base Image + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg IMAGE_VERSION=11-amd64 \ + -t gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + --load \ + ./romexis-base +``` + +### Server Image + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg ROMEXIS_VERSION=6.5.3 \ + --build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + -t gitea.buchhorster.de/patrick/romexis-server:6.5.3-amd64 \ + --load \ + ./romexis +``` + +--- + + +--- + +## Payload-Build-Schicht + +Installer-Download und Extraktion finden jetzt in `romexis-payload/` statt. + +Das Payload Image enthält nur: + +```text +/opt/romexis +/opt/romexis-mssql-db +``` + +Es enthält weder Java noch Chilkat. Dadurch bleibt das Payload architekturunabhängig. + +Manueller Payload-Build: + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg ROMEXIS_VERSION=6.5.3.444.203 \ + -t gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \ + --load \ + ./romexis-payload +``` + +Das Server Image verwendet danach dieses Payload: + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg ROMEXIS_VERSION=6.5.3.444.203 \ + --build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + --build-arg ROMEXIS_PAYLOAD_IMAGE=gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \ + -t gitea.buchhorster.de/patrick/romexis-server:6.5.3.444.203-amd64 \ + --load \ + ./romexis +``` + +In CI wird das Server Image mit Buildx `--push` veröffentlicht, um den langsamen lokalen Export und das Entpacken durch `--load` zu vermeiden. + +--- + +## Aktueller CI/CD-Ablauf + +Die Drone-Pipeline folgt jetzt dieser Reihenfolge: + +```text +payload -> base amd64 -> server amd64 -> migration amd64 -> base arm64 -> server arm64 -> migration arm64 -> manifests +``` + +Payload-Rebuild-Regeln: + +- neue Version in `romexis-versions.env`: nur neues Payload Image bauen +- geänderte URL einer bestehenden Version: diese Version neu bauen +- geänderte `romexis-copy-map.tsv`, Payload-Dockerfile oder Hilfsskripte: alle Payload-Versionen neu bauen +- vorhandenes Payload ohne relevante Änderung: überspringen + +Veröffentlichte Image-Familien: + +```text +romexis-payload: +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +romexis-base-jre:11 +romexis-server:-amd64 +romexis-server:-arm64 +romexis-server: +romexis-migration-service:amd64 +romexis-migration-service:arm64 +romexis-migration-service:latest +``` + + +## CI/CD Build-Ablauf + +Die Drone-Pipeline folgt dieser Reihenfolge: + +1. `romexis-base-jre:11-amd64` bauen und veröffentlichen. +2. `romexis-server:-amd64` bauen und veröffentlichen. +3. `romexis-base-jre:11-arm64` bauen und veröffentlichen. +4. `romexis-server:-arm64` bauen und veröffentlichen. +5. Multiarch-Manifest erstellen und veröffentlichen. + +Das Publishing erfolgt bewusst seriell, um Registry-Last und parallele Upload-Probleme zu vermeiden. + +Buildx wird mit folgenden Optionen verwendet: + +```bash +--provenance=false +--sbom=false +``` + +Dies verbessert die Kompatibilität mit Registries, die OCI-Attestations nicht zuverlässig verarbeiten. + +--- + +## Empfohlene Build-Reihenfolge + +Für lokale Entwicklung: + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +Für CI: + +```text +base amd64 -> server amd64 -> base arm64 -> server arm64 -> manifest +``` + +--- + +## Build-Artefakte + +Das finale Server Image enthält: + +```text +/opt/romexis +/opt/romexis-mssql-db +/usr/lib/libchilkat.so +/opt/romexis/server/RomexisPropertyAgent.jar +/opt/init-romexis-db.sh +/opt/fix-keystore-alias.sh +/entrypoint.sh +``` + +Temporäre Installerdateien werden während des Builds entfernt. + +--- + +## Wartungshinweise + +Neue Romexis-Version hinzufügen: + +1. Installer-URL in `romexis-payload/romexis-versions.env` ergänzen. +2. Image mit neuer Version oder neuem Versionspräfix bauen. +3. `/opt/romexis/version` prüfen. +4. Container starten und Datenbankinitialisierung prüfen. +5. Client-Verbindung über konfigurierte RMI-Adresse und Ports testen. + +Wenn sich das Installerlayout ändert: + +1. `romexis-payload/romexis-copy-map.tsv` aktualisieren. +2. Server Image neu bauen. +3. Prüfen, ob alle benötigten Dateien in `/opt/romexis` vorhanden sind. +4. Dockerfile nur ändern, wenn neue Build-Werkzeuge erforderlich sind. diff --git a/Reference-BUILD.md b/Reference-BUILD.md new file mode 100644 index 0000000..65031e7 --- /dev/null +++ b/Reference-BUILD.md @@ -0,0 +1,399 @@ +# Romexis Docker Build Process + +[Back to README](README.md) | [Deutsch](BUILD.de.md) + +This document describes the complete build process for the multi-architecture Romexis Docker images. + +--- + +## Build Goals + +The build system is designed to provide: + +- reproducible Docker builds +- a reusable runtime base image +- native `amd64` and `arm64` images +- multi-architecture manifests +- minimal final runtime images +- maintainable installer extraction logic +- clear separation between runtime dependencies and Romexis application files + +--- + +## Image Types + +### Romexis Base Image + +The base image is built from `romexis-base/Dockerfile`. + +It contains shared runtime dependencies: + +- Azul Zulu Java 11 runtime with JavaFX/OpenJFX support +- Microsoft SQL Server command-line tools +- Firebird client tools and libraries +- common operating system runtime libraries + +Tags: + +```text +gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 +gitea.buchhorster.de/patrick/romexis-base-jre:11-arm64 +``` + +The base image is intentionally separated from the Romexis Server image so it does not need to be rebuilt for every Romexis version. + +### Romexis Server Image + +The server image is built from `romexis/Dockerfile`. + +It contains: + +- extracted Romexis server files +- Romexis database SQL scripts +- native Chilkat runtime library +- Romexis Java property agent +- runtime helper scripts +- entrypoint and initialization logic + +Architecture-specific tags: + +```text +gitea.buchhorster.de/patrick/romexis-server:-amd64 +gitea.buchhorster.de/patrick/romexis-server:-arm64 +``` + +Multi-architecture manifest tag: + +```text +gitea.buchhorster.de/patrick/romexis-server: +``` + +--- + +## Build Arguments + +### Common arguments + +| Argument | Description | +|---------|-------------| +| `TARGETARCH` | Target architecture, usually `amd64` or `arm64`. | +| `ROMEXIS_VERSION` | Requested Romexis version or version prefix. | + +### Base image arguments + +| Argument | Description | +|---------|-------------| +| `IMAGE_VERSION` | OCI image label version for the base image. | + +### Server image arguments + +| Argument | Description | +|---------|-------------| +| `ROMEXIS_BASE_IMAGE` | Base image reference used by the final Romexis Server stage. | +| `CHILKAT_VERSION` | Native Chilkat library version. | + +--- + +## Version Resolution + +Romexis versions are defined in: + +```text +romexis-payload/romexis-versions.env +``` + +Format: + +```text +6_5_3_444_203=https://content.planmeca.com/files/Planmeca_Romexis_6.5.3.444.203_Win.zip +``` + +Build input: + +```bash +ROMEXIS_VERSION=6.5.3 +``` + +The Dockerfile resolves the newest matching entry and writes the resolved version to: + +```text +/opt/romexis/version +``` + +This makes it possible to build using a stable prefix while still tagging the final image with the exact resolved Romexis version in CI. + +--- + +## Installer Download and Extraction + +The build does not store Romexis binaries in the repository. + +During the `romexis-build` stage: + +1. The selected installer URL is read from `romexis-versions.env`. +2. `download-romexis-installer-parts.py` downloads only the required installer parts. +3. The InstallShield CAB file is processed by `extract-and-copy-romexis-parts.sh`. +4. The final Romexis directory layout is assembled under `/opt/romexis`. +5. SQL Server initialization scripts are copied to `/opt/romexis-mssql-db`. + +--- + +## Mapping-Based Extraction + +The file: + +```text +romexis-payload/romexis-copy-map.tsv +``` + +is the central mapping between installer components and final target directories. + +Format: + +```text +SourceDestination +Broker_jar /opt/romexis/broker +Server_jar /opt/romexis/server +Server_Program_64bit/server/*.xml /opt/romexis/server +``` + +The extraction script uses this file for two steps: + +1. Determine which top-level CAB components must be extracted. +2. Copy the mapped files and directories to their final destinations. + +For glob mappings such as: + +```text +Server_Program_64bit/server/*.xml +``` + +the complete top-level component `Server_Program_64bit` is extracted, but only matching XML files are copied to the final destination. + +This keeps the Dockerfile independent from the Romexis installer layout and makes future installer changes easier to maintain. + +--- + +## Local Build with Docker Compose + +Use: + +```text +docker-compose.build.yml +``` + +### Build the base image + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +``` + +### Build the Romexis Server image + +```bash +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +``` + +### Start the locally built stack + +```bash +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +### Build without cache + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build --no-cache romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build --no-cache romexis +``` + +--- + +## Manual Docker Build + +### Base image + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg IMAGE_VERSION=11-amd64 \ + -t gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + --load \ + ./romexis-base +``` + +### Server image + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg ROMEXIS_VERSION=6.5.3 \ + --build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + -t gitea.buchhorster.de/patrick/romexis-server:6.5.3-amd64 \ + --load \ + ./romexis +``` + +--- + + +--- + +## Payload Build Layer + +Installer download and extraction now happen in `romexis-payload/`. + +The payload image contains only: + +```text +/opt/romexis +/opt/romexis-mssql-db +``` + +It does not contain Java or Chilkat. This keeps the payload architecture-independent. + +Manual payload build: + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg ROMEXIS_VERSION=6.5.3.444.203 \ + -t gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \ + --load \ + ./romexis-payload +``` + +The server image then consumes this payload: + +```bash +docker buildx build \ + --platform linux/amd64 \ + --provenance=false \ + --sbom=false \ + --build-arg TARGETARCH=amd64 \ + --build-arg ROMEXIS_VERSION=6.5.3.444.203 \ + --build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \ + --build-arg ROMEXIS_PAYLOAD_IMAGE=gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \ + -t gitea.buchhorster.de/patrick/romexis-server:6.5.3.444.203-amd64 \ + --load \ + ./romexis +``` + +In CI the server image is published with Buildx `--push` instead of `--load` to avoid the slow local image export/unpack step. + +--- + +## Current CI/CD Flow + +The Drone pipeline now follows this order: + +```text +payload -> base amd64 -> server amd64 -> migration amd64 -> base arm64 -> server arm64 -> migration arm64 -> manifests +``` + +Payload rebuild rules: + +- new version in `romexis-versions.env`: build only the new payload image +- changed URL for an existing version: rebuild that version +- changed `romexis-copy-map.tsv`, payload Dockerfile or helper scripts: rebuild all payload versions +- existing payload with no relevant change: skip + +Published image families: + +```text +romexis-payload: +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +romexis-base-jre:11 +romexis-server:-amd64 +romexis-server:-arm64 +romexis-server: +romexis-migration-service:amd64 +romexis-migration-service:arm64 +romexis-migration-service:latest +``` + + +## CI/CD Build Flow + +The Drone pipeline follows this order: + +1. Build and push `romexis-base-jre:11-amd64`. +2. Build and push `romexis-server:-amd64`. +3. Build and push `romexis-base-jre:11-arm64`. +4. Build and push `romexis-server:-arm64`. +5. Create and push the multi-architecture manifest. + +Publishing is intentionally serialized to reduce registry contention and avoid concurrent upload issues. + +Buildx is used with: + +```bash +--provenance=false +--sbom=false +``` + +This improves compatibility with registries that do not handle OCI attestations reliably. + +--- + +## Recommended Build Order + +For local development: + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +For CI: + +```text +base amd64 -> server amd64 -> base arm64 -> server arm64 -> manifest +``` + +--- + +## Build Artifacts + +The final server image contains: + +```text +/opt/romexis +/opt/romexis-mssql-db +/usr/lib/libchilkat.so +/opt/romexis/server/RomexisPropertyAgent.jar +/opt/init-romexis-db.sh +/opt/fix-keystore-alias.sh +/entrypoint.sh +``` + +Temporary installer files are removed during the build. + +--- + +## Maintenance Notes + +When adding support for a new Romexis version: + +1. Add the installer URL to `romexis-payload/romexis-versions.env`. +2. Build the image using the new version or version prefix. +3. Verify `/opt/romexis/version`. +4. Start the container and check database initialization logs. +5. Confirm client connectivity through the configured RMI host and ports. + +When the installer layout changes: + +1. Update `romexis-payload/romexis-copy-map.tsv`. +2. Rebuild the server image. +3. Confirm that all required files exist in `/opt/romexis`. +4. Keep the Dockerfile unchanged unless new build tools are required. diff --git a/Reference-DEVELOPERS.de.md b/Reference-DEVELOPERS.de.md new file mode 100644 index 0000000..fb72659 --- /dev/null +++ b/Reference-DEVELOPERS.de.md @@ -0,0 +1,369 @@ +# Entwicklerhinweise + +Dieses Dokument beschreibt den internen Aufbau des +Romexis-Docker-Projekts und erläutert, wie die einzelnen +Build-Komponenten zusammenspielen. + +------------------------------------------------------------------------ + +## Repository-Struktur + +Das Projekt ist bewusst in mehrere Schichten aufgeteilt: + +``` text +romexis-base/ + Stellt die wiederverwendbare Java- und Systemlaufzeit bereit. + +romexis/ + Erstellt das eigentliche Romexis-Server-Image aus dem offiziellen Installationspaket. + +docker-compose.yml + Startet bereits erstellte Container-Images. + +docker-compose.build.yml + Erstellt lokale Base- und Server-Images für Entwicklung und Tests. +``` + +Durch diese Trennung bleibt das Runtime-Basisimage unabhängig von +einzelnen Romexis-Versionen. + +------------------------------------------------------------------------ + +## Hauptkomponenten + +### `romexis-base/Dockerfile` + +Erstellt das wiederverwendbare Runtime-Basisimage. + +Aufgaben: + +- Bereitstellung einer Java-11-Laufzeit mit + JavaFX/OpenJFX-Unterstützung +- Installation der SQL-Server-Kommandozeilenwerkzeuge +- Installation der Firebird-Clientbibliotheken +- Einspielen aktueller Betriebssystempakete und Sicherheitsupdates +- Auslagerung gemeinsamer Abhängigkeiten aus dem eigentlichen + Server-Dockerfile + +Das Basisimage wird architekturspezifisch erstellt: + +``` text +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +``` + +### `romexis/Dockerfile` + +Erstellt das eigentliche Romexis-Server-Image. + +Hierfür wird ein dreistufiger Build verwendet: + +1. `romexis-build` + - lädt die benötigten Installationsbestandteile herunter + - extrahiert die erforderlichen CAB-Komponenten + - erstellt die endgültige Verzeichnisstruktur unter `/opt/romexis` + - lädt die native Chilkat-Bibliothek +2. `agent-build` + - kompiliert `RomexisPropertyAgent.java` + - erstellt `RomexisPropertyAgent.jar` +3. Finales Image + - basiert auf dem Romexis-Basisimage + - übernimmt die vorbereiteten Laufzeitdateien + - ergänzt Hilfsskripte und Sicherheitsrichtlinien + +### `romexis-payload/romexis-versions.env` + +Ordnet unterstützte Romexis-Versionen den offiziellen Installer-URLs zu. + +Die Schlüssel verwenden Unterstriche anstelle von Punkten: + +``` text +6_5_3_444_203=https://... +``` + +Das Dockerfile akzeptiert auch Versionspräfixe wie: + +``` text +6.5.3 +``` + +und löst diese automatisch auf die neueste passende vollständige Version +auf. + +### `romexis-payload/romexis-copy-map.tsv` + +Definiert, wie die aus dem Installer extrahierten Komponenten in die +endgültige Verzeichnisstruktur kopiert werden. + +Format: + +``` text +QuelleZiel +``` + +Beispiel: + +``` text +Server_jar /opt/romexis/server +Server_Program_64bit/server/*.xml /opt/romexis/server +``` + +Diese Datei sollte angepasst werden, wenn sich der interne Aufbau des +Romexis-Installers ändert. + +### `romexis-payload/extract-and-copy-romexis-parts.sh` + +Verwendet die Mapping-Datei, um: + +1. die benötigten CAB-Hauptkomponenten zu bestimmen, +2. ausschließlich diese Komponenten mit `unshield` zu extrahieren, +3. Dateien und Verzeichnisse an ihre endgültigen Zielorte zu kopieren. + +Dadurch bleibt das Dockerfile übersichtlich und enthält keine langen, +fest kodierten Kopierketten mehr. + +------------------------------------------------------------------------ + + +--- + +## Aktuelle Ergänzungen der Projektstruktur + +Das Repository enthält jetzt zusätzlich: + +```text +romexis-payload/ + Erstellt wiederverwendbare, versionierte, architekturunabhängige Payload Images aus dem offiziellen Romexis-Installer. + +migration-service/ + Stellt Web UI, REST API, SFTP, Validierung und Restore-Orchestrierung bereit. + +migration-client/ + Enthält den Windows Client für geführte Migrationen von Quellsystemen. +``` + +`romexis/Dockerfile` ist nicht mehr für Installer-Download und CAB-Extraktion zuständig. Es verwendet das vorbereitete Payload Image und ergänzt nur noch Runtime-spezifische Bestandteile wie Chilkat, Java Agent und Entrypoint-Skripte. + +--- + +## Payload-Image-Vertrag + +Ein Payload Image muss bereitstellen: + +```text +/opt/romexis +/opt/romexis-mssql-db +/opt/romexis/version +/opt/romexis/server/RomexisServer.jar +``` + +Es darf keine architekturspezifischen Runtime-Bibliotheken wie Chilkat enthalten. + +Änderungen an `romexis-payload/romexis-copy-map.tsv` oder den Extraktionsskripten können das Payload aller Versionen verändern. Deshalb baut CI bei Änderungen an dieser Logik alle Payload-Versionen neu. + +--- + +## Migrationsarchitektur + +Der Migration Service verwaltet Migrationsjobs als JSON-State-Dateien und stellt temporäre SFTP-Benutzer für aktive Jobs bereit. + +Wichtiges Verhalten: + +- aktive Jobs stellen SFTP-Benutzer beim Service-Start wieder her +- `completed`, `cancelled` und `failed` stellen keine SFTP-Benutzer wieder her +- Abschluss oder Abbruch eines Jobs entfernt oder deaktiviert den temporären SFTP-Benutzer +- der Server besitzt das Manifestformat +- Clients liefern Parameter, keine vollständig formatierten Manifeste + +Das Restore-Skript erwartet diesen Manifest-Vertrag: + +```json +{ + "backup": { + "file": "database/Romexis_db.bak" + } +} +``` + +--- + +## Restart-State-Datei + +Migration Service und Romexis-Container koordinieren Neustarts über: + +```text +/data/romexis_images/.romexis_restart_state +``` + +Der Migration Service schreibt eine ausstehende Neustartanforderung, der Romexis-Entrypoint startet den Romexis-Prozess neu und schreibt `success` oder `failed` zurück. Der Migration Service liest den finalen Status und entfernt die Datei. + + +## Laufzeitskripte + +### `entrypoint.sh` + +Bereitet die Romexis-Laufzeitumgebung vor. + +Aufgaben: + +- setzt `ProgramData=/programdata` +- initialisiert das persistente `sconfig` +- initialisiert das ProgramData-`sconfig` +- erzeugt die Datenbankverbindungsparameter +- bereitet statische Schlüssel und den KeyVault vor +- erstellt die benötigten Datenverzeichnisse +- aktiviert die Standardkonfiguration des Java-Property-Agents +- startet `RomexisServer.jar` + +### `init-romexis-db.sh` + +Initialisiert die SQL-Server-Datenbank. + +Aufgaben: + +- wartet auf den SQL Server +- erstellt die Datenbank, falls sie noch nicht existiert +- erstellt den Romexis-Datenbankbenutzer, falls erforderlich +- importiert die Romexis-SQL-Skripte +- setzt Linux-spezifische Datenpfade in der Datenbank + +Das Skript ist idempotent und überspringt die Initialisierung, sobald +Datenbank und Benutzer bereits vorhanden sind. + +### `RomexisPropertyAgent.java` + +Ermöglicht das Setzen von `RxProperties` über Umgebungsvariablen. + +Format: + +``` text +PROPERTY_AGENT_SET_= +``` + +Beispiel: + +``` text +PROPERTY_AGENT_SET_KEY_B_KEYVAULT_PWD_LOCATION_SERVER_PROPERTIES=true +``` + +------------------------------------------------------------------------ + +## Multi-Architektur + +Der Build unterstützt: + +- `linux/amd64` +- `linux/arm64` + +Die Architekturauswahl erfolgt über das Docker-Buildargument +`TARGETARCH`. + +Beispiele: + +``` bash +--build-arg TARGETARCH=amd64 +--build-arg TARGETARCH=arm64 +``` + +Die passende Chilkat-Bibliothek wird automatisch anhand von `TARGETARCH` +ausgewählt: + + TARGETARCH Chilkat-Architektur + ------------ --------------------- + `amd64` `x86_64` + `arm64` `aarch64` + +Das finale Basisimage wird über `ROMEXIS_BASE_IMAGE` festgelegt. + +Beispiel: + +``` bash +--build-arg ROMEXIS_BASE_IMAGE=romexis-base-jre:11-amd64 +``` + +------------------------------------------------------------------------ + +## Lokaler Entwicklungsablauf + +Empfohlener Workflow: + +``` bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +Logs anzeigen: + +``` bash +docker compose -f docker-compose.build.yml logs -f romexis +``` + +Installierte Version prüfen: + +``` bash +docker exec -it romexis-server cat /opt/romexis/version +``` + +------------------------------------------------------------------------ + +## Neue Romexis-Versionen hinzufügen + +1. Offizielle Installer-URL in `romexis-payload/romexis-versions.env` ergänzen. +2. Mit dem neuen Versionspräfix lokal bauen. +3. Die aufgelöste Version in `/opt/romexis/version` kontrollieren. +4. Container starten und Datenbankinitialisierung prüfen. +5. Verbindung eines Romexis-Clients testen. + +------------------------------------------------------------------------ + +## Installer-Mapping aktualisieren + +Wenn neue Romexis-Versionen andere InstallShield-Komponentennamen +verwenden: + +1. CAB-Inhalt untersuchen. +2. `romexis-copy-map.tsv` anpassen. +3. Server-Image neu erstellen. +4. Prüfen, ob alle benötigten Dateien unter `/opt/romexis` vorhanden + sind. +5. Extraktionslogik ausschließlich in Mapping-Datei und Hilfsskript + pflegen -- nicht im Dockerfile. + +------------------------------------------------------------------------ + +## Registry und CI + +Die Drone-Pipeline veröffentlicht: + +``` text +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +romexis-server:-amd64 +romexis-server:-arm64 +romexis-server: +``` + +Die architekturspezifischen Images werden anschließend zu einem +Multi-Architektur-Manifest zusammengeführt. + +Zur besseren Registry-Kompatibilität werden OCI-Provenance und SBOM im +CI deaktiviert: + +``` bash +--provenance=false +--sbom=false +``` + +------------------------------------------------------------------------ + +## Sicherheitshinweise + +- Im Repository werden keine Romexis-Programmdateien gespeichert. +- Das Installationspaket wird erst während des Builds heruntergeladen. +- Laufzeitgeheimnisse sollten über Docker Compose, CI-Secrets oder + einen externen Secret-Manager bereitgestellt werden. +- Standardkennwörter in den Beispiel-Compose-Dateien müssen vor einem + produktiven Einsatz geändert werden. +- Vor dem produktiven Einsatz sollten vollständige Backups erstellt + werden. diff --git a/Reference-DEVELOPERS.md b/Reference-DEVELOPERS.md new file mode 100644 index 0000000..234ce9f --- /dev/null +++ b/Reference-DEVELOPERS.md @@ -0,0 +1,357 @@ +# Developer Notes + +This document describes the internal structure of the Romexis Docker project and explains how the build components fit together. + +--- + +## Repository Design + +The project is intentionally split into the following layers: + +```text +romexis-base/ + Provides the reusable Java and system runtime. + +romexis/ + Builds the actual Romexis Server image from the official installer. + +docker-compose.yml + Runs pre-built images. + +docker-compose.build.yml + Builds local base and server images for development and testing. +``` + +This separation keeps the runtime base image independent from individual Romexis versions. + +--- + +## Main Components + +### `romexis-base/Dockerfile` + +Builds the reusable runtime base image. + +Responsibilities: + +- provide Java 11 runtime with JavaFX/OpenJFX support +- install SQL Server command-line tools +- install Firebird client libraries +- apply operating system package updates +- keep common dependencies out of the final server Dockerfile + +The base image is tagged per architecture: + +```text +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +``` + +### `romexis/Dockerfile` + +Builds the actual Romexis Server image. + +It uses a three-stage build: + +1. `romexis-build` + - downloads installer components + - extracts required CAB components + - assembles `/opt/romexis` + - downloads the native Chilkat library + +2. `agent-build` + - compiles `RomexisPropertyAgent.java` + - packages `RomexisPropertyAgent.jar` + +3. final image + - starts from the Romexis base image + - copies prepared runtime files + - adds helper scripts and policy files + +### `romexis-payload/romexis-versions.env` + +Maps supported Romexis versions to official installer URLs. + +The keys use underscores instead of dots: + +```text +6_5_3_444_203=https://... +``` + +The Dockerfile accepts a prefix such as: + +```text +6.5.3 +``` + +and resolves it to the newest matching full version. + +### `romexis-payload/romexis-copy-map.tsv` + +Defines how extracted installer components are copied to the final image. + +Format: + +```text +SourceDestination +``` + +Example: + +```text +Server_jar /opt/romexis/server +Server_Program_64bit/server/*.xml /opt/romexis/server +``` + +This file should be updated when the internal Romexis installer layout changes. + +### `romexis-payload/extract-and-copy-romexis-parts.sh` + +Uses the mapping file to: + +1. derive required top-level CAB components +2. extract only those components with `unshield` +3. copy mapped files and directories to their final destinations + +This keeps the Dockerfile small and avoids hardcoding installer layout details in shell chains. + +--- + + +--- + +## Current Project Structure Additions + +The repository now includes these additional areas: + +```text +romexis-payload/ + Builds reusable, versioned, architecture-independent payload images from the official Romexis installer. + +migration-service/ + Provides Web UI, REST API, SFTP, validation and restore orchestration. + +migration-client/ + Provides the Windows client for guided source-system migrations. +``` + +`romexis/Dockerfile` no longer owns installer download and CAB extraction. It consumes the prepared payload image and only adds runtime-specific parts such as Chilkat, the Java agent and entrypoint scripts. + +--- + +## Payload Image Contract + +A payload image must provide: + +```text +/opt/romexis +/opt/romexis-mssql-db +/opt/romexis/version +/opt/romexis/server/RomexisServer.jar +``` + +It must not contain architecture-specific runtime libraries such as Chilkat. + +Changes to `romexis-payload/romexis-copy-map.tsv` or extraction scripts can change the payload for every version. Therefore CI rebuilds all payload versions when this logic changes. + +--- + +## Migration Architecture + +The migration service manages migration jobs as JSON state files and provides temporary SFTP users for active jobs. + +Important behavior: + +- active jobs recreate SFTP users on service startup +- `completed`, `cancelled` and `failed` jobs do not recreate SFTP users +- completing or cancelling a job removes or disables the temporary SFTP user +- the server owns the manifest format +- clients provide parameters, not fully formatted manifests + +The restore script expects this manifest contract: + +```json +{ + "backup": { + "file": "database/Romexis_db.bak" + } +} +``` + +--- + +## Restart State File + +The migration service and Romexis container coordinate restarts through: + +```text +/data/romexis_images/.romexis_restart_state +``` + +The migration service writes a pending restart request, the Romexis entrypoint restarts the Romexis process and writes `success` or `failed` back. The migration service reads the final status and removes the file. + + +## Runtime Scripts + +### `entrypoint.sh` + +Prepares the Romexis runtime environment. + +Responsibilities: + +- set `ProgramData=/programdata` +- initialize persistent `sconfig` +- initialize ProgramData `sconfig` +- generate database connection properties +- prepare static keys and KeyVault +- prepare data directories +- enable the default Java property agent setting +- start `RomexisServer.jar` + +### `init-romexis-db.sh` + +Initializes the SQL Server database. + +Responsibilities: + +- wait for SQL Server +- create the database if missing +- create the Romexis database user if missing +- import Romexis SQL scripts +- set Linux data paths in the database + +The script is idempotent and skips initialization when database and user already exist. + +### `RomexisPropertyAgent.java` + +Allows setting Romexis `RxProperties` through environment variables. + +Environment variable format: + +```text +PROPERTY_AGENT_SET_= +``` + +Example: + +```text +PROPERTY_AGENT_SET_KEY_B_KEYVAULT_PWD_LOCATION_SERVER_PROPERTIES=true +``` + +--- + +## Multi-Architecture Notes + +The build supports: + +- `linux/amd64` +- `linux/arm64` + +Architecture handling is done through Docker's `TARGETARCH` build argument. + +Examples: + +```bash +--build-arg TARGETARCH=amd64 +--build-arg TARGETARCH=arm64 +``` + +The Chilkat native library is selected based on `TARGETARCH`: + +| TARGETARCH | Chilkat architecture | +|-----------|----------------------| +| `amd64` | `x86_64` | +| `arm64` | `aarch64` | + +The final base image is selected through: + +```text +ROMEXIS_BASE_IMAGE +``` + +Example: + +```bash +--build-arg ROMEXIS_BASE_IMAGE=romexis-base-jre:11-amd64 +``` + +--- + +## Local Development Workflow + +Recommended workflow: + +```bash +TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis +TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d +``` + +Check logs: + +```bash +docker compose -f docker-compose.build.yml logs -f romexis +``` + +Check installed version: + +```bash +docker exec -it romexis-server cat /opt/romexis/version +``` + +--- + +## Updating Romexis Versions + +To add a new Romexis version: + +1. Add the full installer URL to `romexis-payload/romexis-versions.env`. +2. Build locally using the new version prefix. +3. Confirm the resolved version in `/opt/romexis/version`. +4. Start the container and verify database initialization. +5. Confirm Romexis client connectivity. + +--- + +## Updating Installer Mapping + +When a new Romexis installer changes the internal InstallShield component names: + +1. Inspect the extracted CAB structure. +2. Update `romexis-copy-map.tsv`. +3. Rebuild the server image. +4. Verify that required files are present under `/opt/romexis`. +5. Keep extraction logic inside the mapping file and helper script instead of adding long copy chains to the Dockerfile. + +--- + +## Registry and CI Notes + +The Drone pipeline publishes: + +```text +romexis-base-jre:11-amd64 +romexis-base-jre:11-arm64 +romexis-server:-amd64 +romexis-server:-arm64 +romexis-server: +``` + +The architecture-specific server tags are used to create the final multi-architecture manifest. + +OCI provenance and SBOM generation are disabled in CI for better compatibility with the configured registry: + +```bash +--provenance=false +--sbom=false +``` + +--- + +## Security Notes + +- No Romexis program files are stored in the repository. +- The installer is downloaded during the build. +- Runtime secrets should be provided through Compose, CI secrets or an external secret manager. +- Default passwords in sample Compose files must be changed before production use. +- Full backups are required before production deployment. diff --git a/Reference-MIGRATION_README.md b/Reference-MIGRATION_README.md new file mode 100644 index 0000000..c1c55dd --- /dev/null +++ b/Reference-MIGRATION_README.md @@ -0,0 +1,277 @@ +# Romexis Migration Service + +This Container is a migration helper service that will be used to migrate an existing Windows-based Planmeca Romexis installation into the Docker-based Romexis Server stack. + +- **Flask Web UI / REST API** for migration management, status, validation and restore orchestration +- **SFTP upload endpoint** for large files and directory trees +- **Manifest-based migration jobs** to describe what was transferred +- **Restore hooks** for SQL Server database restore and final data placement + +The service starts, exposes a web UI, creates migration jobs, provides per-job upload credentials, accepts uploads through SFTP and shows migration state. + +--- + +## Intended Migration Flow + +```text +Existing Windows Romexis Installation + | + | 1. Create SQL Server .bak backup + | 2. Collect Romexis data directories + | 3. Generate manifest.json + | 4. Upload via SFTP/rclone/WinSCP + v +Romexis Migration Service + | + | 5. Validate uploaded files + | 6. Stop Romexis container + | 7. Restore SQL backup + | 8. Move data directories into target volumes + | 9. Start Romexis container + v +Docker-based Romexis Server +``` + +--- + +## Why SFTP Instead of Flask Uploads? + +Romexis image archives can become very large. A Flask-only upload approach would require special handling for: + +- upload timeouts +- interrupted transfers +- resume support +- progress tracking +- large numbers of files +- large single `.bak` files + +SFTP is better suited for this job because it is supported by many mature tools: + +- `rclone` (favorite tool) +- WinSCP +- FileZilla +- OpenSSH `sftp` +- PowerShell/OpenSSH +- automation scripts + +The web application remains responsible for orchestration, status and restore actions. + +--- + +## Project Structure + +```text +. +├── migration-service/ +│ ├── Dockerfile +│ ├── requirements.txt +│ ├── entrypoint.sh +│ ├── app/ +│ │ ├── app.py +│ │ ├── core/ +│ │ │ ├── config.py +│ │ │ ├── migrations.py +│ │ │ └── sftp_users.py +│ │ ├── templates/ +│ │ │ ├── base.html +│ │ │ ├── index.html +│ │ │ ├── migration.html +│ │ │ └── new_migration.html +│ │ └── static/ +│ │ └── style.css +│ ├── scripts/ +│ │ ├── restore-migration.sh +│ │ └── validate-migration.sh +│ └── ssh/ +│ └── sshd_config +│ +├── migration-client/ +│ └── romexis_migration_client.py +│ +└── docs/ + ├── examples/ + │ └── manifest.example.json + ├── MIGRATION_WORKFLOW.md + └── SECURITY.md +``` + +--- + +## Quick Start + +Build and start the migration service: + +```bash +docker compose -f docker-compose.migration.yml up -d --build +``` + +Open the web UI: + +```text +http://localhost:8080 +``` + +SFTP endpoint: + +```text +localhost:2222 +``` + +Create a migration in the web UI. The service will generate: + +- migration ID +- SFTP username +- SFTP password +- upload path + +Then upload files into the migration directory. + +--- + +## Expected Upload Layout + +Each migration gets its own incoming directory: + +```text +/incoming// +├── manifest.json +├── database/ +│ └── Romexis_db.bak +├── romexis_images/ +│ └── ... +├── romexis_ergodata/ +│ └── ... +└── romexis_cache/ + └── ... # optional +``` + +The cache directory is optional and can usually be skipped. + +--- + +## Manifest + +The `manifest.json` describes the transferred data. + +Example: + +```json +{ + "source": { + "hostname": "old-romexis-server", + "romexis_version": "6.5.3", + "database_name": "Romexis_db" + }, + "backup": { + "file": "database/Romexis_db.bak" + }, + "data": { + "images": "romexis_images", + "ergodata": "romexis_ergodata", + "cache": null + } +} +``` + +A full example is included in: + +```text +examples/manifest.example.json +``` + +--- + +## Current State + +Implemented: + +- Flask web UI +- REST API +- migration job creation +- migration state stored as JSON files +- generated per-migration credentials +- SFTP server in the same container +- upload directory preparation +- validation hook +- SQL Server restore +- Python client + +Not yet fully implemented: + +- coordinated Romexis restart through a shared state file +- checksum validation +- progress aggregation +- authentication for the web UI +- production-grade user isolation + +These pieces are intentionally separated so they can be implemented and tested step by step. + +--- + + +--- + +## Current Implementation Update + +The migration service now supports both client-driven and manual browser-based migrations. + +Implemented workflow: + +1. Create migration job. +2. Upload database backup through the Web UI or client workflow. +3. Generate `manifest.json` on the server. +4. Restore the database backup. +5. Upload `romexis_images`, `romexis_ergodata` and optionally `romexis_cache` through SFTP. +6. Confirm upload completion. +7. Validate upload. +8. Run restore workflow. +9. Request Romexis restart using the shared state file. +10. Complete migration and remove temporary SFTP access. + +Additional supported operation: + +- cancel a migration and remove the temporary SFTP user + +Final states such as `completed`, `cancelled` and `failed` do not recreate SFTP users after a service restart. + +--- + +## Server-Side Manifest Generation + +The server owns the manifest format. Clients and the Web UI should submit parameters only. + +The restore script expects: + +```json +{ + "backup": { + "file": "database/Romexis_db.bak" + } +} +``` + +For manual uploads, the manifest is generated automatically after the database backup was uploaded through the Web UI. + +--- + +## Romexis Restart Coordination + +After a successful database restore, the migration service can request a Romexis restart through: + +```text +/data/romexis_images/.romexis_restart_state +``` + +The Romexis entrypoint monitors this state file, restarts the Romexis process and writes the result back. The migration service reads the result, logs it and removes the state file. + + +## Development Notes + +The current implementation is designed as a practical foundation. It keeps the large-file transport path independent from the web application and makes it possible to test migration workflows incrementally. + +Next steps: + +- Add checksum generation and validation. +- Add authentication to the web UI. +- Add migration logs per job. +- Add a dry-run restore mode. diff --git a/Reference-MIGRATION_WORKFLOW.md b/Reference-MIGRATION_WORKFLOW.md new file mode 100644 index 0000000..35b090b --- /dev/null +++ b/Reference-MIGRATION_WORKFLOW.md @@ -0,0 +1,166 @@ +# Migration Workflow + +This document describes the target workflow for migrating an existing Romexis installation into the Docker-based Romexis Server stack. + +┌──────────────────────────────┐ +│ Existing Romexis Server │ +└──────────────┬───────────────┘ + │ + │ Romexis Migration Client + │ + ▼ + SFTP / REST API + │ + ▼ +┌──────────────────────────────┐ +│ Romexis Migration Service │ +│ │ +│ • Migration Jobs │ +│ • Upload Validation │ +│ • Restore Workflow │ +└──────────────┬───────────────┘ + │ + ▼ +┌──────────────────────────────┐ +│ Romexis Docker Container │ +│ SQL Server │ +│ Images │ +│ Ergodata │ +└──────────────────────────────┘ + +--- + + +--- + +## Updated Detailed Workflow + +```text +created + -> database_uploaded + -> database_restored + -> upload_complete + -> validated + -> restored + -> completed +``` + +Final states: + +```text +completed +cancelled +failed +``` + +### Manual browser workflow + +1. Create a migration job in the Web UI. +2. Upload the database backup in the browser. +3. The service creates `manifest.json` automatically. +4. Trigger database restore. +5. Upload file directories via SFTP. +6. Confirm upload complete. +7. Validate upload. +8. Run restore. +9. Complete migration and remove SFTP access. + +### Windows client workflow + +1. Detect Romexis installation. +2. Detect SQL Server configuration. +3. Detect data directories. +4. Create or use a migration job. +5. Create database backup. +6. Upload data via rclone/SFTP. +7. Call API endpoints to advance the migration workflow. +8. Display logs and restore status. + +### Restart after restore + +The migration service and Romexis container communicate via: + +```text +/data/romexis_images/.romexis_restart_state +``` + +The migration service writes a pending restart request. The Romexis entrypoint stops and restarts the Romexis process, writes the final status and the migration service removes the state file. + + +## Phase 1: Preparation on the Source System + +The source system is usually an existing Windows-based Romexis server. + +Required data: + +- SQL Server database backup (`.bak`) +- Romexis image directory +- Romexis ergo data directory +- optional cache directory + +The cache directory can usually be skipped because it can be regenerated. + +--- + +## Phase 2: Create Migration Job + +A migration job is created in the web UI. + +The service generates: + +- migration ID +- SFTP username +- SFTP password +- upload path + +--- + +## Phase 3: Upload Data + +Recommended tools: + +- rclone over SFTP +- WinSCP +- FileZilla +- OpenSSH SFTP + +Example using rclone: + +```bash +rclone copy ./migration-data sftp:upload \ + --transfers 8 \ + --checkers 16 \ + --progress +``` + +--- + +## Phase 4: Validate Upload + +The migration service validates: + +- manifest exists +- SQL backup exists +- image directory exists +- ergo directory exists + +Later versions should add: + +- checksum validation +- file count comparison +- expected size validation + +--- + +## Phase 5: Restore + +The restore process performs or coordinates: + +1. stop the Romexis server container +2. restore the SQL Server database backup +3. copy data directories into the target volumes +4. fix ownership and permissions +5. start the Romexis server container +6. run a final validation + +The current workflow uses explicit state transitions and keeps restart coordination separated through the shared state file. diff --git a/Security.md b/Security.md new file mode 100644 index 0000000..6ed660a --- /dev/null +++ b/Security.md @@ -0,0 +1,92 @@ +# Security + +## Repository Security + +The repository must not contain proprietary Romexis application binaries. + +Installer packages are downloaded during payload builds from official URLs. + +--- + +## Secrets + +Sensitive values should come from: + +```text +.env +Drone secrets +Docker Compose environment +external secret stores +``` + +Important values: + +```text +MSSQL_SA_PASSWORD +ROMEXIS_DB_PASSWORD +FIREBIRD_PASSWORD +MIGRATION_API_TOKEN +SFTP passwords +Registry credentials +``` + +--- + +## Migration Service Security + +Migration jobs create temporary SFTP credentials. + +Rules: + +- credentials should be unique per job +- credentials should be removed after completion +- cancelled jobs should remove credentials +- completed jobs should not recreate SFTP users on startup +- failed jobs should not recreate SFTP users unless explicitly reactivated + +--- + +## Network Exposure + +Expose only required ports. + +Typical ports: + +```text +Romexis RMI ports +MSSQL 1433, if external access is required +Firebird 3050, if external access is required +Migration Web UI 8080 +Migration SFTP 2222 +``` + +For production, place services behind appropriate firewall rules. + +--- + +## File Permissions + +Persistent directories must be writable by the relevant container users. + +Special care is required for: + +```text +/var/opt/mssql +/firebird/data +/data/romexis_images +/data/romexis_ergodata +/data/romexis_cache +``` + +--- + +## Production Notes + +Before production use: + +- change all default passwords +- restrict exposed ports +- use HTTPS/reverse proxy for the migration Web UI +- use strong API tokens +- remove test migration jobs +- verify backup/restore procedures