Private
Public Access
add Reference Files to wiki
@@ -0,0 +1,399 @@
|
|||||||
|
# Romexis Docker Build-Prozess
|
||||||
|
|
||||||
|
[Zurück zum README](README.de.md) | [English](BUILD.md)
|
||||||
|
|
||||||
|
Dieses Dokument beschreibt den vollständigen Build-Prozess für die Multiarch-Romexis-Docker-Images.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Ziele des Builds
|
||||||
|
|
||||||
|
Das Build-System ist darauf ausgelegt, folgende Ziele zu erfüllen:
|
||||||
|
|
||||||
|
- reproduzierbare Docker-Builds
|
||||||
|
- wiederverwendbares Runtime-Basisimage
|
||||||
|
- native `amd64`- und `arm64`-Images
|
||||||
|
- Multiarch-Manifeste
|
||||||
|
- minimale finale Runtime-Images
|
||||||
|
- wartbare Installer-Extraktionslogik
|
||||||
|
- klare Trennung zwischen Laufzeitabhängigkeiten und Romexis-Anwendungsdateien
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Image-Typen
|
||||||
|
|
||||||
|
### Romexis Base Image
|
||||||
|
|
||||||
|
Das Base Image wird aus `romexis-base/Dockerfile` gebaut.
|
||||||
|
|
||||||
|
Es enthält gemeinsame Laufzeitabhängigkeiten:
|
||||||
|
|
||||||
|
- Azul Zulu Java 11 Runtime mit JavaFX/OpenJFX-Unterstützung
|
||||||
|
- Microsoft SQL Server Kommandozeilenwerkzeuge
|
||||||
|
- Firebird-Clienttools und Bibliotheken
|
||||||
|
- gemeinsame Betriebssystem-Laufzeitbibliotheken
|
||||||
|
|
||||||
|
Tags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64
|
||||||
|
gitea.buchhorster.de/patrick/romexis-base-jre:11-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
Das Base Image ist bewusst vom Romexis Server Image getrennt, damit es nicht für jede Romexis-Version erneut gebaut werden muss.
|
||||||
|
|
||||||
|
### Romexis Server Image
|
||||||
|
|
||||||
|
Das Server Image wird aus `romexis/Dockerfile` gebaut.
|
||||||
|
|
||||||
|
Es enthält:
|
||||||
|
|
||||||
|
- extrahierte Romexis-Serverdateien
|
||||||
|
- Romexis-Datenbank-SQL-Skripte
|
||||||
|
- native Chilkat-Laufzeitbibliothek
|
||||||
|
- Romexis Java Property Agent
|
||||||
|
- Runtime-Hilfsskripte
|
||||||
|
- Entrypoint- und Initialisierungslogik
|
||||||
|
|
||||||
|
Architekturspezifische Tags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>-amd64
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
Multiarch-Manifest-Tag:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build-Argumente
|
||||||
|
|
||||||
|
### Gemeinsame Argumente
|
||||||
|
|
||||||
|
| Argument | Beschreibung |
|
||||||
|
|---------|--------------|
|
||||||
|
| `TARGETARCH` | Zielarchitektur, normalerweise `amd64` oder `arm64`. |
|
||||||
|
| `ROMEXIS_VERSION` | Angeforderte Romexis-Version oder Versionspräfix. |
|
||||||
|
|
||||||
|
### Base-Image-Argumente
|
||||||
|
|
||||||
|
| Argument | Beschreibung |
|
||||||
|
|---------|--------------|
|
||||||
|
| `IMAGE_VERSION` | OCI-Image-Label-Version des Base Images. |
|
||||||
|
|
||||||
|
### Server-Image-Argumente
|
||||||
|
|
||||||
|
| Argument | Beschreibung |
|
||||||
|
|---------|--------------|
|
||||||
|
| `ROMEXIS_BASE_IMAGE` | Basisimage-Referenz für die finale Romexis-Server-Stage. |
|
||||||
|
| `CHILKAT_VERSION` | Version der nativen Chilkat-Bibliothek. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Versionsauflösung
|
||||||
|
|
||||||
|
Romexis-Versionen werden in folgender Datei definiert:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/romexis-versions.env
|
||||||
|
```
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
6_5_3_444_203=https://content.planmeca.com/files/Planmeca_Romexis_6.5.3.444.203_Win.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Build-Eingabe:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ROMEXIS_VERSION=6.5.3
|
||||||
|
```
|
||||||
|
|
||||||
|
Das Dockerfile löst den neuesten passenden Eintrag auf und schreibt die tatsächlich verwendete Version nach:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis/version
|
||||||
|
```
|
||||||
|
|
||||||
|
Dadurch kann lokal oder in CI mit stabilen Versionspräfixen gebaut werden, während das Image intern die exakte Romexis-Version enthält.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installer-Download und Extraktion
|
||||||
|
|
||||||
|
Das Repository enthält keine Romexis-Binaries.
|
||||||
|
|
||||||
|
Während der `romexis-build`-Stage:
|
||||||
|
|
||||||
|
1. Die passende Installer-URL wird aus `romexis-versions.env` gelesen.
|
||||||
|
2. `download-romexis-installer-parts.py` lädt nur die benötigten Installerbestandteile herunter.
|
||||||
|
3. Die InstallShield-CAB-Datei wird durch `extract-and-copy-romexis-parts.sh` verarbeitet.
|
||||||
|
4. Die finale Romexis-Verzeichnisstruktur wird unter `/opt/romexis` aufgebaut.
|
||||||
|
5. SQL-Server-Initialisierungsskripte werden nach `/opt/romexis-mssql-db` kopiert.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Mapping-basierte Extraktion
|
||||||
|
|
||||||
|
Die Datei:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/romexis-copy-map.tsv
|
||||||
|
```
|
||||||
|
|
||||||
|
ist die zentrale Zuordnung zwischen Installer-Komponenten und Zielverzeichnissen.
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Source<TAB>Destination
|
||||||
|
Broker_jar /opt/romexis/broker
|
||||||
|
Server_jar /opt/romexis/server
|
||||||
|
Server_Program_64bit/server/*.xml /opt/romexis/server
|
||||||
|
```
|
||||||
|
|
||||||
|
Das Extraktionsskript nutzt diese Datei für zwei Schritte:
|
||||||
|
|
||||||
|
1. Ermitteln, welche Top-Level-CAB-Komponenten extrahiert werden müssen.
|
||||||
|
2. Kopieren der gemappten Dateien und Verzeichnisse an ihre Zielorte.
|
||||||
|
|
||||||
|
Bei Glob-Mappings wie:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Server_Program_64bit/server/*.xml
|
||||||
|
```
|
||||||
|
|
||||||
|
wird die komplette Top-Level-Komponente `Server_Program_64bit` extrahiert, aber nur passende XML-Dateien werden in das Zielverzeichnis kopiert.
|
||||||
|
|
||||||
|
Dadurch bleibt das Dockerfile unabhängig vom internen Romexis-Installerlayout.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lokaler Build mit Docker Compose
|
||||||
|
|
||||||
|
Verwendet wird:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docker-compose.build.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Base Image bauen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
```
|
||||||
|
|
||||||
|
### Romexis Server Image bauen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
### Lokal gebauten Stack starten
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build ohne Cache
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build --no-cache romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build --no-cache romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Manueller Docker Build
|
||||||
|
|
||||||
|
### Base Image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg IMAGE_VERSION=11-amd64 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis-base
|
||||||
|
```
|
||||||
|
|
||||||
|
### Server Image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3 \
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-server:6.5.3-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Payload-Build-Schicht
|
||||||
|
|
||||||
|
Installer-Download und Extraktion finden jetzt in `romexis-payload/` statt.
|
||||||
|
|
||||||
|
Das Payload Image enthält nur:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
```
|
||||||
|
|
||||||
|
Es enthält weder Java noch Chilkat. Dadurch bleibt das Payload architekturunabhängig.
|
||||||
|
|
||||||
|
Manueller Payload-Build:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3.444.203 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \
|
||||||
|
--load \
|
||||||
|
./romexis-payload
|
||||||
|
```
|
||||||
|
|
||||||
|
Das Server Image verwendet danach dieses Payload:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3.444.203 \
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
--build-arg ROMEXIS_PAYLOAD_IMAGE=gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-server:6.5.3.444.203-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
In CI wird das Server Image mit Buildx `--push` veröffentlicht, um den langsamen lokalen Export und das Entpacken durch `--load` zu vermeiden.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Aktueller CI/CD-Ablauf
|
||||||
|
|
||||||
|
Die Drone-Pipeline folgt jetzt dieser Reihenfolge:
|
||||||
|
|
||||||
|
```text
|
||||||
|
payload -> base amd64 -> server amd64 -> migration amd64 -> base arm64 -> server arm64 -> migration arm64 -> manifests
|
||||||
|
```
|
||||||
|
|
||||||
|
Payload-Rebuild-Regeln:
|
||||||
|
|
||||||
|
- neue Version in `romexis-versions.env`: nur neues Payload Image bauen
|
||||||
|
- geänderte URL einer bestehenden Version: diese Version neu bauen
|
||||||
|
- geänderte `romexis-copy-map.tsv`, Payload-Dockerfile oder Hilfsskripte: alle Payload-Versionen neu bauen
|
||||||
|
- vorhandenes Payload ohne relevante Änderung: überspringen
|
||||||
|
|
||||||
|
Veröffentlichte Image-Familien:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload:<version>
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
romexis-base-jre:11
|
||||||
|
romexis-server:<version>-amd64
|
||||||
|
romexis-server:<version>-arm64
|
||||||
|
romexis-server:<version>
|
||||||
|
romexis-migration-service:amd64
|
||||||
|
romexis-migration-service:arm64
|
||||||
|
romexis-migration-service:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## CI/CD Build-Ablauf
|
||||||
|
|
||||||
|
Die Drone-Pipeline folgt dieser Reihenfolge:
|
||||||
|
|
||||||
|
1. `romexis-base-jre:11-amd64` bauen und veröffentlichen.
|
||||||
|
2. `romexis-server:<version>-amd64` bauen und veröffentlichen.
|
||||||
|
3. `romexis-base-jre:11-arm64` bauen und veröffentlichen.
|
||||||
|
4. `romexis-server:<version>-arm64` bauen und veröffentlichen.
|
||||||
|
5. Multiarch-Manifest erstellen und veröffentlichen.
|
||||||
|
|
||||||
|
Das Publishing erfolgt bewusst seriell, um Registry-Last und parallele Upload-Probleme zu vermeiden.
|
||||||
|
|
||||||
|
Buildx wird mit folgenden Optionen verwendet:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--provenance=false
|
||||||
|
--sbom=false
|
||||||
|
```
|
||||||
|
|
||||||
|
Dies verbessert die Kompatibilität mit Registries, die OCI-Attestations nicht zuverlässig verarbeiten.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Empfohlene Build-Reihenfolge
|
||||||
|
|
||||||
|
Für lokale Entwicklung:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Für CI:
|
||||||
|
|
||||||
|
```text
|
||||||
|
base amd64 -> server amd64 -> base arm64 -> server arm64 -> manifest
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build-Artefakte
|
||||||
|
|
||||||
|
Das finale Server Image enthält:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
/usr/lib/libchilkat.so
|
||||||
|
/opt/romexis/server/RomexisPropertyAgent.jar
|
||||||
|
/opt/init-romexis-db.sh
|
||||||
|
/opt/fix-keystore-alias.sh
|
||||||
|
/entrypoint.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Temporäre Installerdateien werden während des Builds entfernt.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Wartungshinweise
|
||||||
|
|
||||||
|
Neue Romexis-Version hinzufügen:
|
||||||
|
|
||||||
|
1. Installer-URL in `romexis-payload/romexis-versions.env` ergänzen.
|
||||||
|
2. Image mit neuer Version oder neuem Versionspräfix bauen.
|
||||||
|
3. `/opt/romexis/version` prüfen.
|
||||||
|
4. Container starten und Datenbankinitialisierung prüfen.
|
||||||
|
5. Client-Verbindung über konfigurierte RMI-Adresse und Ports testen.
|
||||||
|
|
||||||
|
Wenn sich das Installerlayout ändert:
|
||||||
|
|
||||||
|
1. `romexis-payload/romexis-copy-map.tsv` aktualisieren.
|
||||||
|
2. Server Image neu bauen.
|
||||||
|
3. Prüfen, ob alle benötigten Dateien in `/opt/romexis` vorhanden sind.
|
||||||
|
4. Dockerfile nur ändern, wenn neue Build-Werkzeuge erforderlich sind.
|
||||||
+399
@@ -0,0 +1,399 @@
|
|||||||
|
# Romexis Docker Build Process
|
||||||
|
|
||||||
|
[Back to README](README.md) | [Deutsch](BUILD.de.md)
|
||||||
|
|
||||||
|
This document describes the complete build process for the multi-architecture Romexis Docker images.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build Goals
|
||||||
|
|
||||||
|
The build system is designed to provide:
|
||||||
|
|
||||||
|
- reproducible Docker builds
|
||||||
|
- a reusable runtime base image
|
||||||
|
- native `amd64` and `arm64` images
|
||||||
|
- multi-architecture manifests
|
||||||
|
- minimal final runtime images
|
||||||
|
- maintainable installer extraction logic
|
||||||
|
- clear separation between runtime dependencies and Romexis application files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Image Types
|
||||||
|
|
||||||
|
### Romexis Base Image
|
||||||
|
|
||||||
|
The base image is built from `romexis-base/Dockerfile`.
|
||||||
|
|
||||||
|
It contains shared runtime dependencies:
|
||||||
|
|
||||||
|
- Azul Zulu Java 11 runtime with JavaFX/OpenJFX support
|
||||||
|
- Microsoft SQL Server command-line tools
|
||||||
|
- Firebird client tools and libraries
|
||||||
|
- common operating system runtime libraries
|
||||||
|
|
||||||
|
Tags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64
|
||||||
|
gitea.buchhorster.de/patrick/romexis-base-jre:11-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
The base image is intentionally separated from the Romexis Server image so it does not need to be rebuilt for every Romexis version.
|
||||||
|
|
||||||
|
### Romexis Server Image
|
||||||
|
|
||||||
|
The server image is built from `romexis/Dockerfile`.
|
||||||
|
|
||||||
|
It contains:
|
||||||
|
|
||||||
|
- extracted Romexis server files
|
||||||
|
- Romexis database SQL scripts
|
||||||
|
- native Chilkat runtime library
|
||||||
|
- Romexis Java property agent
|
||||||
|
- runtime helper scripts
|
||||||
|
- entrypoint and initialization logic
|
||||||
|
|
||||||
|
Architecture-specific tags:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>-amd64
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
Multi-architecture manifest tag:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.buchhorster.de/patrick/romexis-server:<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build Arguments
|
||||||
|
|
||||||
|
### Common arguments
|
||||||
|
|
||||||
|
| Argument | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `TARGETARCH` | Target architecture, usually `amd64` or `arm64`. |
|
||||||
|
| `ROMEXIS_VERSION` | Requested Romexis version or version prefix. |
|
||||||
|
|
||||||
|
### Base image arguments
|
||||||
|
|
||||||
|
| Argument | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `IMAGE_VERSION` | OCI image label version for the base image. |
|
||||||
|
|
||||||
|
### Server image arguments
|
||||||
|
|
||||||
|
| Argument | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `ROMEXIS_BASE_IMAGE` | Base image reference used by the final Romexis Server stage. |
|
||||||
|
| `CHILKAT_VERSION` | Native Chilkat library version. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Version Resolution
|
||||||
|
|
||||||
|
Romexis versions are defined in:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/romexis-versions.env
|
||||||
|
```
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
6_5_3_444_203=https://content.planmeca.com/files/Planmeca_Romexis_6.5.3.444.203_Win.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Build input:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ROMEXIS_VERSION=6.5.3
|
||||||
|
```
|
||||||
|
|
||||||
|
The Dockerfile resolves the newest matching entry and writes the resolved version to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis/version
|
||||||
|
```
|
||||||
|
|
||||||
|
This makes it possible to build using a stable prefix while still tagging the final image with the exact resolved Romexis version in CI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installer Download and Extraction
|
||||||
|
|
||||||
|
The build does not store Romexis binaries in the repository.
|
||||||
|
|
||||||
|
During the `romexis-build` stage:
|
||||||
|
|
||||||
|
1. The selected installer URL is read from `romexis-versions.env`.
|
||||||
|
2. `download-romexis-installer-parts.py` downloads only the required installer parts.
|
||||||
|
3. The InstallShield CAB file is processed by `extract-and-copy-romexis-parts.sh`.
|
||||||
|
4. The final Romexis directory layout is assembled under `/opt/romexis`.
|
||||||
|
5. SQL Server initialization scripts are copied to `/opt/romexis-mssql-db`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Mapping-Based Extraction
|
||||||
|
|
||||||
|
The file:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/romexis-copy-map.tsv
|
||||||
|
```
|
||||||
|
|
||||||
|
is the central mapping between installer components and final target directories.
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Source<TAB>Destination
|
||||||
|
Broker_jar /opt/romexis/broker
|
||||||
|
Server_jar /opt/romexis/server
|
||||||
|
Server_Program_64bit/server/*.xml /opt/romexis/server
|
||||||
|
```
|
||||||
|
|
||||||
|
The extraction script uses this file for two steps:
|
||||||
|
|
||||||
|
1. Determine which top-level CAB components must be extracted.
|
||||||
|
2. Copy the mapped files and directories to their final destinations.
|
||||||
|
|
||||||
|
For glob mappings such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Server_Program_64bit/server/*.xml
|
||||||
|
```
|
||||||
|
|
||||||
|
the complete top-level component `Server_Program_64bit` is extracted, but only matching XML files are copied to the final destination.
|
||||||
|
|
||||||
|
This keeps the Dockerfile independent from the Romexis installer layout and makes future installer changes easier to maintain.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Local Build with Docker Compose
|
||||||
|
|
||||||
|
Use:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docker-compose.build.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build the base image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build the Romexis Server image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
### Start the locally built stack
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build without cache
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build --no-cache romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build --no-cache romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Manual Docker Build
|
||||||
|
|
||||||
|
### Base image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg IMAGE_VERSION=11-amd64 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis-base
|
||||||
|
```
|
||||||
|
|
||||||
|
### Server image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3 \
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-server:6.5.3-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Payload Build Layer
|
||||||
|
|
||||||
|
Installer download and extraction now happen in `romexis-payload/`.
|
||||||
|
|
||||||
|
The payload image contains only:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
```
|
||||||
|
|
||||||
|
It does not contain Java or Chilkat. This keeps the payload architecture-independent.
|
||||||
|
|
||||||
|
Manual payload build:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3.444.203 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \
|
||||||
|
--load \
|
||||||
|
./romexis-payload
|
||||||
|
```
|
||||||
|
|
||||||
|
The server image then consumes this payload:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker buildx build \
|
||||||
|
--platform linux/amd64 \
|
||||||
|
--provenance=false \
|
||||||
|
--sbom=false \
|
||||||
|
--build-arg TARGETARCH=amd64 \
|
||||||
|
--build-arg ROMEXIS_VERSION=6.5.3.444.203 \
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=gitea.buchhorster.de/patrick/romexis-base-jre:11-amd64 \
|
||||||
|
--build-arg ROMEXIS_PAYLOAD_IMAGE=gitea.buchhorster.de/patrick/romexis-payload:6.5.3.444.203 \
|
||||||
|
-t gitea.buchhorster.de/patrick/romexis-server:6.5.3.444.203-amd64 \
|
||||||
|
--load \
|
||||||
|
./romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
In CI the server image is published with Buildx `--push` instead of `--load` to avoid the slow local image export/unpack step.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current CI/CD Flow
|
||||||
|
|
||||||
|
The Drone pipeline now follows this order:
|
||||||
|
|
||||||
|
```text
|
||||||
|
payload -> base amd64 -> server amd64 -> migration amd64 -> base arm64 -> server arm64 -> migration arm64 -> manifests
|
||||||
|
```
|
||||||
|
|
||||||
|
Payload rebuild rules:
|
||||||
|
|
||||||
|
- new version in `romexis-versions.env`: build only the new payload image
|
||||||
|
- changed URL for an existing version: rebuild that version
|
||||||
|
- changed `romexis-copy-map.tsv`, payload Dockerfile or helper scripts: rebuild all payload versions
|
||||||
|
- existing payload with no relevant change: skip
|
||||||
|
|
||||||
|
Published image families:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload:<version>
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
romexis-base-jre:11
|
||||||
|
romexis-server:<version>-amd64
|
||||||
|
romexis-server:<version>-arm64
|
||||||
|
romexis-server:<version>
|
||||||
|
romexis-migration-service:amd64
|
||||||
|
romexis-migration-service:arm64
|
||||||
|
romexis-migration-service:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## CI/CD Build Flow
|
||||||
|
|
||||||
|
The Drone pipeline follows this order:
|
||||||
|
|
||||||
|
1. Build and push `romexis-base-jre:11-amd64`.
|
||||||
|
2. Build and push `romexis-server:<version>-amd64`.
|
||||||
|
3. Build and push `romexis-base-jre:11-arm64`.
|
||||||
|
4. Build and push `romexis-server:<version>-arm64`.
|
||||||
|
5. Create and push the multi-architecture manifest.
|
||||||
|
|
||||||
|
Publishing is intentionally serialized to reduce registry contention and avoid concurrent upload issues.
|
||||||
|
|
||||||
|
Buildx is used with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--provenance=false
|
||||||
|
--sbom=false
|
||||||
|
```
|
||||||
|
|
||||||
|
This improves compatibility with registries that do not handle OCI attestations reliably.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Recommended Build Order
|
||||||
|
|
||||||
|
For local development:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
For CI:
|
||||||
|
|
||||||
|
```text
|
||||||
|
base amd64 -> server amd64 -> base arm64 -> server arm64 -> manifest
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build Artifacts
|
||||||
|
|
||||||
|
The final server image contains:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
/usr/lib/libchilkat.so
|
||||||
|
/opt/romexis/server/RomexisPropertyAgent.jar
|
||||||
|
/opt/init-romexis-db.sh
|
||||||
|
/opt/fix-keystore-alias.sh
|
||||||
|
/entrypoint.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Temporary installer files are removed during the build.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Maintenance Notes
|
||||||
|
|
||||||
|
When adding support for a new Romexis version:
|
||||||
|
|
||||||
|
1. Add the installer URL to `romexis-payload/romexis-versions.env`.
|
||||||
|
2. Build the image using the new version or version prefix.
|
||||||
|
3. Verify `/opt/romexis/version`.
|
||||||
|
4. Start the container and check database initialization logs.
|
||||||
|
5. Confirm client connectivity through the configured RMI host and ports.
|
||||||
|
|
||||||
|
When the installer layout changes:
|
||||||
|
|
||||||
|
1. Update `romexis-payload/romexis-copy-map.tsv`.
|
||||||
|
2. Rebuild the server image.
|
||||||
|
3. Confirm that all required files exist in `/opt/romexis`.
|
||||||
|
4. Keep the Dockerfile unchanged unless new build tools are required.
|
||||||
@@ -0,0 +1,369 @@
|
|||||||
|
# Entwicklerhinweise
|
||||||
|
|
||||||
|
Dieses Dokument beschreibt den internen Aufbau des
|
||||||
|
Romexis-Docker-Projekts und erläutert, wie die einzelnen
|
||||||
|
Build-Komponenten zusammenspielen.
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Repository-Struktur
|
||||||
|
|
||||||
|
Das Projekt ist bewusst in mehrere Schichten aufgeteilt:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
romexis-base/
|
||||||
|
Stellt die wiederverwendbare Java- und Systemlaufzeit bereit.
|
||||||
|
|
||||||
|
romexis/
|
||||||
|
Erstellt das eigentliche Romexis-Server-Image aus dem offiziellen Installationspaket.
|
||||||
|
|
||||||
|
docker-compose.yml
|
||||||
|
Startet bereits erstellte Container-Images.
|
||||||
|
|
||||||
|
docker-compose.build.yml
|
||||||
|
Erstellt lokale Base- und Server-Images für Entwicklung und Tests.
|
||||||
|
```
|
||||||
|
|
||||||
|
Durch diese Trennung bleibt das Runtime-Basisimage unabhängig von
|
||||||
|
einzelnen Romexis-Versionen.
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Hauptkomponenten
|
||||||
|
|
||||||
|
### `romexis-base/Dockerfile`
|
||||||
|
|
||||||
|
Erstellt das wiederverwendbare Runtime-Basisimage.
|
||||||
|
|
||||||
|
Aufgaben:
|
||||||
|
|
||||||
|
- Bereitstellung einer Java-11-Laufzeit mit
|
||||||
|
JavaFX/OpenJFX-Unterstützung
|
||||||
|
- Installation der SQL-Server-Kommandozeilenwerkzeuge
|
||||||
|
- Installation der Firebird-Clientbibliotheken
|
||||||
|
- Einspielen aktueller Betriebssystempakete und Sicherheitsupdates
|
||||||
|
- Auslagerung gemeinsamer Abhängigkeiten aus dem eigentlichen
|
||||||
|
Server-Dockerfile
|
||||||
|
|
||||||
|
Das Basisimage wird architekturspezifisch erstellt:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
### `romexis/Dockerfile`
|
||||||
|
|
||||||
|
Erstellt das eigentliche Romexis-Server-Image.
|
||||||
|
|
||||||
|
Hierfür wird ein dreistufiger Build verwendet:
|
||||||
|
|
||||||
|
1. `romexis-build`
|
||||||
|
- lädt die benötigten Installationsbestandteile herunter
|
||||||
|
- extrahiert die erforderlichen CAB-Komponenten
|
||||||
|
- erstellt die endgültige Verzeichnisstruktur unter `/opt/romexis`
|
||||||
|
- lädt die native Chilkat-Bibliothek
|
||||||
|
2. `agent-build`
|
||||||
|
- kompiliert `RomexisPropertyAgent.java`
|
||||||
|
- erstellt `RomexisPropertyAgent.jar`
|
||||||
|
3. Finales Image
|
||||||
|
- basiert auf dem Romexis-Basisimage
|
||||||
|
- übernimmt die vorbereiteten Laufzeitdateien
|
||||||
|
- ergänzt Hilfsskripte und Sicherheitsrichtlinien
|
||||||
|
|
||||||
|
### `romexis-payload/romexis-versions.env`
|
||||||
|
|
||||||
|
Ordnet unterstützte Romexis-Versionen den offiziellen Installer-URLs zu.
|
||||||
|
|
||||||
|
Die Schlüssel verwenden Unterstriche anstelle von Punkten:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
6_5_3_444_203=https://...
|
||||||
|
```
|
||||||
|
|
||||||
|
Das Dockerfile akzeptiert auch Versionspräfixe wie:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
6.5.3
|
||||||
|
```
|
||||||
|
|
||||||
|
und löst diese automatisch auf die neueste passende vollständige Version
|
||||||
|
auf.
|
||||||
|
|
||||||
|
### `romexis-payload/romexis-copy-map.tsv`
|
||||||
|
|
||||||
|
Definiert, wie die aus dem Installer extrahierten Komponenten in die
|
||||||
|
endgültige Verzeichnisstruktur kopiert werden.
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
Quelle<TAB>Ziel
|
||||||
|
```
|
||||||
|
|
||||||
|
Beispiel:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
Server_jar /opt/romexis/server
|
||||||
|
Server_Program_64bit/server/*.xml /opt/romexis/server
|
||||||
|
```
|
||||||
|
|
||||||
|
Diese Datei sollte angepasst werden, wenn sich der interne Aufbau des
|
||||||
|
Romexis-Installers ändert.
|
||||||
|
|
||||||
|
### `romexis-payload/extract-and-copy-romexis-parts.sh`
|
||||||
|
|
||||||
|
Verwendet die Mapping-Datei, um:
|
||||||
|
|
||||||
|
1. die benötigten CAB-Hauptkomponenten zu bestimmen,
|
||||||
|
2. ausschließlich diese Komponenten mit `unshield` zu extrahieren,
|
||||||
|
3. Dateien und Verzeichnisse an ihre endgültigen Zielorte zu kopieren.
|
||||||
|
|
||||||
|
Dadurch bleibt das Dockerfile übersichtlich und enthält keine langen,
|
||||||
|
fest kodierten Kopierketten mehr.
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Aktuelle Ergänzungen der Projektstruktur
|
||||||
|
|
||||||
|
Das Repository enthält jetzt zusätzlich:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/
|
||||||
|
Erstellt wiederverwendbare, versionierte, architekturunabhängige Payload Images aus dem offiziellen Romexis-Installer.
|
||||||
|
|
||||||
|
migration-service/
|
||||||
|
Stellt Web UI, REST API, SFTP, Validierung und Restore-Orchestrierung bereit.
|
||||||
|
|
||||||
|
migration-client/
|
||||||
|
Enthält den Windows Client für geführte Migrationen von Quellsystemen.
|
||||||
|
```
|
||||||
|
|
||||||
|
`romexis/Dockerfile` ist nicht mehr für Installer-Download und CAB-Extraktion zuständig. Es verwendet das vorbereitete Payload Image und ergänzt nur noch Runtime-spezifische Bestandteile wie Chilkat, Java Agent und Entrypoint-Skripte.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Payload-Image-Vertrag
|
||||||
|
|
||||||
|
Ein Payload Image muss bereitstellen:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
/opt/romexis/version
|
||||||
|
/opt/romexis/server/RomexisServer.jar
|
||||||
|
```
|
||||||
|
|
||||||
|
Es darf keine architekturspezifischen Runtime-Bibliotheken wie Chilkat enthalten.
|
||||||
|
|
||||||
|
Änderungen an `romexis-payload/romexis-copy-map.tsv` oder den Extraktionsskripten können das Payload aller Versionen verändern. Deshalb baut CI bei Änderungen an dieser Logik alle Payload-Versionen neu.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Migrationsarchitektur
|
||||||
|
|
||||||
|
Der Migration Service verwaltet Migrationsjobs als JSON-State-Dateien und stellt temporäre SFTP-Benutzer für aktive Jobs bereit.
|
||||||
|
|
||||||
|
Wichtiges Verhalten:
|
||||||
|
|
||||||
|
- aktive Jobs stellen SFTP-Benutzer beim Service-Start wieder her
|
||||||
|
- `completed`, `cancelled` und `failed` stellen keine SFTP-Benutzer wieder her
|
||||||
|
- Abschluss oder Abbruch eines Jobs entfernt oder deaktiviert den temporären SFTP-Benutzer
|
||||||
|
- der Server besitzt das Manifestformat
|
||||||
|
- Clients liefern Parameter, keine vollständig formatierten Manifeste
|
||||||
|
|
||||||
|
Das Restore-Skript erwartet diesen Manifest-Vertrag:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"backup": {
|
||||||
|
"file": "database/Romexis_db.bak"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Restart-State-Datei
|
||||||
|
|
||||||
|
Migration Service und Romexis-Container koordinieren Neustarts über:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/data/romexis_images/.romexis_restart_state
|
||||||
|
```
|
||||||
|
|
||||||
|
Der Migration Service schreibt eine ausstehende Neustartanforderung, der Romexis-Entrypoint startet den Romexis-Prozess neu und schreibt `success` oder `failed` zurück. Der Migration Service liest den finalen Status und entfernt die Datei.
|
||||||
|
|
||||||
|
|
||||||
|
## Laufzeitskripte
|
||||||
|
|
||||||
|
### `entrypoint.sh`
|
||||||
|
|
||||||
|
Bereitet die Romexis-Laufzeitumgebung vor.
|
||||||
|
|
||||||
|
Aufgaben:
|
||||||
|
|
||||||
|
- setzt `ProgramData=/programdata`
|
||||||
|
- initialisiert das persistente `sconfig`
|
||||||
|
- initialisiert das ProgramData-`sconfig`
|
||||||
|
- erzeugt die Datenbankverbindungsparameter
|
||||||
|
- bereitet statische Schlüssel und den KeyVault vor
|
||||||
|
- erstellt die benötigten Datenverzeichnisse
|
||||||
|
- aktiviert die Standardkonfiguration des Java-Property-Agents
|
||||||
|
- startet `RomexisServer.jar`
|
||||||
|
|
||||||
|
### `init-romexis-db.sh`
|
||||||
|
|
||||||
|
Initialisiert die SQL-Server-Datenbank.
|
||||||
|
|
||||||
|
Aufgaben:
|
||||||
|
|
||||||
|
- wartet auf den SQL Server
|
||||||
|
- erstellt die Datenbank, falls sie noch nicht existiert
|
||||||
|
- erstellt den Romexis-Datenbankbenutzer, falls erforderlich
|
||||||
|
- importiert die Romexis-SQL-Skripte
|
||||||
|
- setzt Linux-spezifische Datenpfade in der Datenbank
|
||||||
|
|
||||||
|
Das Skript ist idempotent und überspringt die Initialisierung, sobald
|
||||||
|
Datenbank und Benutzer bereits vorhanden sind.
|
||||||
|
|
||||||
|
### `RomexisPropertyAgent.java`
|
||||||
|
|
||||||
|
Ermöglicht das Setzen von `RxProperties` über Umgebungsvariablen.
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
PROPERTY_AGENT_SET_<RXPROPERTY_NAME>=<Wert>
|
||||||
|
```
|
||||||
|
|
||||||
|
Beispiel:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
PROPERTY_AGENT_SET_KEY_B_KEYVAULT_PWD_LOCATION_SERVER_PROPERTIES=true
|
||||||
|
```
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Multi-Architektur
|
||||||
|
|
||||||
|
Der Build unterstützt:
|
||||||
|
|
||||||
|
- `linux/amd64`
|
||||||
|
- `linux/arm64`
|
||||||
|
|
||||||
|
Die Architekturauswahl erfolgt über das Docker-Buildargument
|
||||||
|
`TARGETARCH`.
|
||||||
|
|
||||||
|
Beispiele:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
--build-arg TARGETARCH=amd64
|
||||||
|
--build-arg TARGETARCH=arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
Die passende Chilkat-Bibliothek wird automatisch anhand von `TARGETARCH`
|
||||||
|
ausgewählt:
|
||||||
|
|
||||||
|
TARGETARCH Chilkat-Architektur
|
||||||
|
------------ ---------------------
|
||||||
|
`amd64` `x86_64`
|
||||||
|
`arm64` `aarch64`
|
||||||
|
|
||||||
|
Das finale Basisimage wird über `ROMEXIS_BASE_IMAGE` festgelegt.
|
||||||
|
|
||||||
|
Beispiel:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=romexis-base-jre:11-amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Lokaler Entwicklungsablauf
|
||||||
|
|
||||||
|
Empfohlener Workflow:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Logs anzeigen:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
docker compose -f docker-compose.build.yml logs -f romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
Installierte Version prüfen:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
docker exec -it romexis-server cat /opt/romexis/version
|
||||||
|
```
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Neue Romexis-Versionen hinzufügen
|
||||||
|
|
||||||
|
1. Offizielle Installer-URL in `romexis-payload/romexis-versions.env` ergänzen.
|
||||||
|
2. Mit dem neuen Versionspräfix lokal bauen.
|
||||||
|
3. Die aufgelöste Version in `/opt/romexis/version` kontrollieren.
|
||||||
|
4. Container starten und Datenbankinitialisierung prüfen.
|
||||||
|
5. Verbindung eines Romexis-Clients testen.
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Installer-Mapping aktualisieren
|
||||||
|
|
||||||
|
Wenn neue Romexis-Versionen andere InstallShield-Komponentennamen
|
||||||
|
verwenden:
|
||||||
|
|
||||||
|
1. CAB-Inhalt untersuchen.
|
||||||
|
2. `romexis-copy-map.tsv` anpassen.
|
||||||
|
3. Server-Image neu erstellen.
|
||||||
|
4. Prüfen, ob alle benötigten Dateien unter `/opt/romexis` vorhanden
|
||||||
|
sind.
|
||||||
|
5. Extraktionslogik ausschließlich in Mapping-Datei und Hilfsskript
|
||||||
|
pflegen -- nicht im Dockerfile.
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Registry und CI
|
||||||
|
|
||||||
|
Die Drone-Pipeline veröffentlicht:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
romexis-server:<version>-amd64
|
||||||
|
romexis-server:<version>-arm64
|
||||||
|
romexis-server:<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
Die architekturspezifischen Images werden anschließend zu einem
|
||||||
|
Multi-Architektur-Manifest zusammengeführt.
|
||||||
|
|
||||||
|
Zur besseren Registry-Kompatibilität werden OCI-Provenance und SBOM im
|
||||||
|
CI deaktiviert:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
--provenance=false
|
||||||
|
--sbom=false
|
||||||
|
```
|
||||||
|
|
||||||
|
------------------------------------------------------------------------
|
||||||
|
|
||||||
|
## Sicherheitshinweise
|
||||||
|
|
||||||
|
- Im Repository werden keine Romexis-Programmdateien gespeichert.
|
||||||
|
- Das Installationspaket wird erst während des Builds heruntergeladen.
|
||||||
|
- Laufzeitgeheimnisse sollten über Docker Compose, CI-Secrets oder
|
||||||
|
einen externen Secret-Manager bereitgestellt werden.
|
||||||
|
- Standardkennwörter in den Beispiel-Compose-Dateien müssen vor einem
|
||||||
|
produktiven Einsatz geändert werden.
|
||||||
|
- Vor dem produktiven Einsatz sollten vollständige Backups erstellt
|
||||||
|
werden.
|
||||||
@@ -0,0 +1,357 @@
|
|||||||
|
# Developer Notes
|
||||||
|
|
||||||
|
This document describes the internal structure of the Romexis Docker project and explains how the build components fit together.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository Design
|
||||||
|
|
||||||
|
The project is intentionally split into the following layers:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-base/
|
||||||
|
Provides the reusable Java and system runtime.
|
||||||
|
|
||||||
|
romexis/
|
||||||
|
Builds the actual Romexis Server image from the official installer.
|
||||||
|
|
||||||
|
docker-compose.yml
|
||||||
|
Runs pre-built images.
|
||||||
|
|
||||||
|
docker-compose.build.yml
|
||||||
|
Builds local base and server images for development and testing.
|
||||||
|
```
|
||||||
|
|
||||||
|
This separation keeps the runtime base image independent from individual Romexis versions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Main Components
|
||||||
|
|
||||||
|
### `romexis-base/Dockerfile`
|
||||||
|
|
||||||
|
Builds the reusable runtime base image.
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- provide Java 11 runtime with JavaFX/OpenJFX support
|
||||||
|
- install SQL Server command-line tools
|
||||||
|
- install Firebird client libraries
|
||||||
|
- apply operating system package updates
|
||||||
|
- keep common dependencies out of the final server Dockerfile
|
||||||
|
|
||||||
|
The base image is tagged per architecture:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
### `romexis/Dockerfile`
|
||||||
|
|
||||||
|
Builds the actual Romexis Server image.
|
||||||
|
|
||||||
|
It uses a three-stage build:
|
||||||
|
|
||||||
|
1. `romexis-build`
|
||||||
|
- downloads installer components
|
||||||
|
- extracts required CAB components
|
||||||
|
- assembles `/opt/romexis`
|
||||||
|
- downloads the native Chilkat library
|
||||||
|
|
||||||
|
2. `agent-build`
|
||||||
|
- compiles `RomexisPropertyAgent.java`
|
||||||
|
- packages `RomexisPropertyAgent.jar`
|
||||||
|
|
||||||
|
3. final image
|
||||||
|
- starts from the Romexis base image
|
||||||
|
- copies prepared runtime files
|
||||||
|
- adds helper scripts and policy files
|
||||||
|
|
||||||
|
### `romexis-payload/romexis-versions.env`
|
||||||
|
|
||||||
|
Maps supported Romexis versions to official installer URLs.
|
||||||
|
|
||||||
|
The keys use underscores instead of dots:
|
||||||
|
|
||||||
|
```text
|
||||||
|
6_5_3_444_203=https://...
|
||||||
|
```
|
||||||
|
|
||||||
|
The Dockerfile accepts a prefix such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
6.5.3
|
||||||
|
```
|
||||||
|
|
||||||
|
and resolves it to the newest matching full version.
|
||||||
|
|
||||||
|
### `romexis-payload/romexis-copy-map.tsv`
|
||||||
|
|
||||||
|
Defines how extracted installer components are copied to the final image.
|
||||||
|
|
||||||
|
Format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Source<TAB>Destination
|
||||||
|
```
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Server_jar /opt/romexis/server
|
||||||
|
Server_Program_64bit/server/*.xml /opt/romexis/server
|
||||||
|
```
|
||||||
|
|
||||||
|
This file should be updated when the internal Romexis installer layout changes.
|
||||||
|
|
||||||
|
### `romexis-payload/extract-and-copy-romexis-parts.sh`
|
||||||
|
|
||||||
|
Uses the mapping file to:
|
||||||
|
|
||||||
|
1. derive required top-level CAB components
|
||||||
|
2. extract only those components with `unshield`
|
||||||
|
3. copy mapped files and directories to their final destinations
|
||||||
|
|
||||||
|
This keeps the Dockerfile small and avoids hardcoding installer layout details in shell chains.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current Project Structure Additions
|
||||||
|
|
||||||
|
The repository now includes these additional areas:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-payload/
|
||||||
|
Builds reusable, versioned, architecture-independent payload images from the official Romexis installer.
|
||||||
|
|
||||||
|
migration-service/
|
||||||
|
Provides Web UI, REST API, SFTP, validation and restore orchestration.
|
||||||
|
|
||||||
|
migration-client/
|
||||||
|
Provides the Windows client for guided source-system migrations.
|
||||||
|
```
|
||||||
|
|
||||||
|
`romexis/Dockerfile` no longer owns installer download and CAB extraction. It consumes the prepared payload image and only adds runtime-specific parts such as Chilkat, the Java agent and entrypoint scripts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Payload Image Contract
|
||||||
|
|
||||||
|
A payload image must provide:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/romexis
|
||||||
|
/opt/romexis-mssql-db
|
||||||
|
/opt/romexis/version
|
||||||
|
/opt/romexis/server/RomexisServer.jar
|
||||||
|
```
|
||||||
|
|
||||||
|
It must not contain architecture-specific runtime libraries such as Chilkat.
|
||||||
|
|
||||||
|
Changes to `romexis-payload/romexis-copy-map.tsv` or extraction scripts can change the payload for every version. Therefore CI rebuilds all payload versions when this logic changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Migration Architecture
|
||||||
|
|
||||||
|
The migration service manages migration jobs as JSON state files and provides temporary SFTP users for active jobs.
|
||||||
|
|
||||||
|
Important behavior:
|
||||||
|
|
||||||
|
- active jobs recreate SFTP users on service startup
|
||||||
|
- `completed`, `cancelled` and `failed` jobs do not recreate SFTP users
|
||||||
|
- completing or cancelling a job removes or disables the temporary SFTP user
|
||||||
|
- the server owns the manifest format
|
||||||
|
- clients provide parameters, not fully formatted manifests
|
||||||
|
|
||||||
|
The restore script expects this manifest contract:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"backup": {
|
||||||
|
"file": "database/Romexis_db.bak"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Restart State File
|
||||||
|
|
||||||
|
The migration service and Romexis container coordinate restarts through:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/data/romexis_images/.romexis_restart_state
|
||||||
|
```
|
||||||
|
|
||||||
|
The migration service writes a pending restart request, the Romexis entrypoint restarts the Romexis process and writes `success` or `failed` back. The migration service reads the final status and removes the file.
|
||||||
|
|
||||||
|
|
||||||
|
## Runtime Scripts
|
||||||
|
|
||||||
|
### `entrypoint.sh`
|
||||||
|
|
||||||
|
Prepares the Romexis runtime environment.
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- set `ProgramData=/programdata`
|
||||||
|
- initialize persistent `sconfig`
|
||||||
|
- initialize ProgramData `sconfig`
|
||||||
|
- generate database connection properties
|
||||||
|
- prepare static keys and KeyVault
|
||||||
|
- prepare data directories
|
||||||
|
- enable the default Java property agent setting
|
||||||
|
- start `RomexisServer.jar`
|
||||||
|
|
||||||
|
### `init-romexis-db.sh`
|
||||||
|
|
||||||
|
Initializes the SQL Server database.
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- wait for SQL Server
|
||||||
|
- create the database if missing
|
||||||
|
- create the Romexis database user if missing
|
||||||
|
- import Romexis SQL scripts
|
||||||
|
- set Linux data paths in the database
|
||||||
|
|
||||||
|
The script is idempotent and skips initialization when database and user already exist.
|
||||||
|
|
||||||
|
### `RomexisPropertyAgent.java`
|
||||||
|
|
||||||
|
Allows setting Romexis `RxProperties` through environment variables.
|
||||||
|
|
||||||
|
Environment variable format:
|
||||||
|
|
||||||
|
```text
|
||||||
|
PROPERTY_AGENT_SET_<RXPROPERTY_NAME>=<value>
|
||||||
|
```
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
PROPERTY_AGENT_SET_KEY_B_KEYVAULT_PWD_LOCATION_SERVER_PROPERTIES=true
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Multi-Architecture Notes
|
||||||
|
|
||||||
|
The build supports:
|
||||||
|
|
||||||
|
- `linux/amd64`
|
||||||
|
- `linux/arm64`
|
||||||
|
|
||||||
|
Architecture handling is done through Docker's `TARGETARCH` build argument.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--build-arg TARGETARCH=amd64
|
||||||
|
--build-arg TARGETARCH=arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
The Chilkat native library is selected based on `TARGETARCH`:
|
||||||
|
|
||||||
|
| TARGETARCH | Chilkat architecture |
|
||||||
|
|-----------|----------------------|
|
||||||
|
| `amd64` | `x86_64` |
|
||||||
|
| `arm64` | `aarch64` |
|
||||||
|
|
||||||
|
The final base image is selected through:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ROMEXIS_BASE_IMAGE
|
||||||
|
```
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--build-arg ROMEXIS_BASE_IMAGE=romexis-base-jre:11-amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Local Development Workflow
|
||||||
|
|
||||||
|
Recommended workflow:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TARGETARCH=amd64 docker compose -f docker-compose.build.yml --profile build build romexis-base
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml build romexis
|
||||||
|
TARGETARCH=amd64 ROMEXIS_VERSION=6.5.3 docker compose -f docker-compose.build.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Check logs:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.build.yml logs -f romexis
|
||||||
|
```
|
||||||
|
|
||||||
|
Check installed version:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec -it romexis-server cat /opt/romexis/version
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Updating Romexis Versions
|
||||||
|
|
||||||
|
To add a new Romexis version:
|
||||||
|
|
||||||
|
1. Add the full installer URL to `romexis-payload/romexis-versions.env`.
|
||||||
|
2. Build locally using the new version prefix.
|
||||||
|
3. Confirm the resolved version in `/opt/romexis/version`.
|
||||||
|
4. Start the container and verify database initialization.
|
||||||
|
5. Confirm Romexis client connectivity.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Updating Installer Mapping
|
||||||
|
|
||||||
|
When a new Romexis installer changes the internal InstallShield component names:
|
||||||
|
|
||||||
|
1. Inspect the extracted CAB structure.
|
||||||
|
2. Update `romexis-copy-map.tsv`.
|
||||||
|
3. Rebuild the server image.
|
||||||
|
4. Verify that required files are present under `/opt/romexis`.
|
||||||
|
5. Keep extraction logic inside the mapping file and helper script instead of adding long copy chains to the Dockerfile.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Registry and CI Notes
|
||||||
|
|
||||||
|
The Drone pipeline publishes:
|
||||||
|
|
||||||
|
```text
|
||||||
|
romexis-base-jre:11-amd64
|
||||||
|
romexis-base-jre:11-arm64
|
||||||
|
romexis-server:<version>-amd64
|
||||||
|
romexis-server:<version>-arm64
|
||||||
|
romexis-server:<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
The architecture-specific server tags are used to create the final multi-architecture manifest.
|
||||||
|
|
||||||
|
OCI provenance and SBOM generation are disabled in CI for better compatibility with the configured registry:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--provenance=false
|
||||||
|
--sbom=false
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Notes
|
||||||
|
|
||||||
|
- No Romexis program files are stored in the repository.
|
||||||
|
- The installer is downloaded during the build.
|
||||||
|
- Runtime secrets should be provided through Compose, CI secrets or an external secret manager.
|
||||||
|
- Default passwords in sample Compose files must be changed before production use.
|
||||||
|
- Full backups are required before production deployment.
|
||||||
@@ -0,0 +1,277 @@
|
|||||||
|
# Romexis Migration Service
|
||||||
|
|
||||||
|
This Container is a migration helper service that will be used to migrate an existing Windows-based Planmeca Romexis installation into the Docker-based Romexis Server stack.
|
||||||
|
|
||||||
|
- **Flask Web UI / REST API** for migration management, status, validation and restore orchestration
|
||||||
|
- **SFTP upload endpoint** for large files and directory trees
|
||||||
|
- **Manifest-based migration jobs** to describe what was transferred
|
||||||
|
- **Restore hooks** for SQL Server database restore and final data placement
|
||||||
|
|
||||||
|
The service starts, exposes a web UI, creates migration jobs, provides per-job upload credentials, accepts uploads through SFTP and shows migration state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Intended Migration Flow
|
||||||
|
|
||||||
|
```text
|
||||||
|
Existing Windows Romexis Installation
|
||||||
|
|
|
||||||
|
| 1. Create SQL Server .bak backup
|
||||||
|
| 2. Collect Romexis data directories
|
||||||
|
| 3. Generate manifest.json
|
||||||
|
| 4. Upload via SFTP/rclone/WinSCP
|
||||||
|
v
|
||||||
|
Romexis Migration Service
|
||||||
|
|
|
||||||
|
| 5. Validate uploaded files
|
||||||
|
| 6. Stop Romexis container
|
||||||
|
| 7. Restore SQL backup
|
||||||
|
| 8. Move data directories into target volumes
|
||||||
|
| 9. Start Romexis container
|
||||||
|
v
|
||||||
|
Docker-based Romexis Server
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Why SFTP Instead of Flask Uploads?
|
||||||
|
|
||||||
|
Romexis image archives can become very large. A Flask-only upload approach would require special handling for:
|
||||||
|
|
||||||
|
- upload timeouts
|
||||||
|
- interrupted transfers
|
||||||
|
- resume support
|
||||||
|
- progress tracking
|
||||||
|
- large numbers of files
|
||||||
|
- large single `.bak` files
|
||||||
|
|
||||||
|
SFTP is better suited for this job because it is supported by many mature tools:
|
||||||
|
|
||||||
|
- `rclone` (favorite tool)
|
||||||
|
- WinSCP
|
||||||
|
- FileZilla
|
||||||
|
- OpenSSH `sftp`
|
||||||
|
- PowerShell/OpenSSH
|
||||||
|
- automation scripts
|
||||||
|
|
||||||
|
The web application remains responsible for orchestration, status and restore actions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```text
|
||||||
|
.
|
||||||
|
├── migration-service/
|
||||||
|
│ ├── Dockerfile
|
||||||
|
│ ├── requirements.txt
|
||||||
|
│ ├── entrypoint.sh
|
||||||
|
│ ├── app/
|
||||||
|
│ │ ├── app.py
|
||||||
|
│ │ ├── core/
|
||||||
|
│ │ │ ├── config.py
|
||||||
|
│ │ │ ├── migrations.py
|
||||||
|
│ │ │ └── sftp_users.py
|
||||||
|
│ │ ├── templates/
|
||||||
|
│ │ │ ├── base.html
|
||||||
|
│ │ │ ├── index.html
|
||||||
|
│ │ │ ├── migration.html
|
||||||
|
│ │ │ └── new_migration.html
|
||||||
|
│ │ └── static/
|
||||||
|
│ │ └── style.css
|
||||||
|
│ ├── scripts/
|
||||||
|
│ │ ├── restore-migration.sh
|
||||||
|
│ │ └── validate-migration.sh
|
||||||
|
│ └── ssh/
|
||||||
|
│ └── sshd_config
|
||||||
|
│
|
||||||
|
├── migration-client/
|
||||||
|
│ └── romexis_migration_client.py
|
||||||
|
│
|
||||||
|
└── docs/
|
||||||
|
├── examples/
|
||||||
|
│ └── manifest.example.json
|
||||||
|
├── MIGRATION_WORKFLOW.md
|
||||||
|
└── SECURITY.md
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
Build and start the migration service:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.migration.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Open the web UI:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://localhost:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
SFTP endpoint:
|
||||||
|
|
||||||
|
```text
|
||||||
|
localhost:2222
|
||||||
|
```
|
||||||
|
|
||||||
|
Create a migration in the web UI. The service will generate:
|
||||||
|
|
||||||
|
- migration ID
|
||||||
|
- SFTP username
|
||||||
|
- SFTP password
|
||||||
|
- upload path
|
||||||
|
|
||||||
|
Then upload files into the migration directory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Expected Upload Layout
|
||||||
|
|
||||||
|
Each migration gets its own incoming directory:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/incoming/<migration-id>/
|
||||||
|
├── manifest.json
|
||||||
|
├── database/
|
||||||
|
│ └── Romexis_db.bak
|
||||||
|
├── romexis_images/
|
||||||
|
│ └── ...
|
||||||
|
├── romexis_ergodata/
|
||||||
|
│ └── ...
|
||||||
|
└── romexis_cache/
|
||||||
|
└── ... # optional
|
||||||
|
```
|
||||||
|
|
||||||
|
The cache directory is optional and can usually be skipped.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Manifest
|
||||||
|
|
||||||
|
The `manifest.json` describes the transferred data.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"source": {
|
||||||
|
"hostname": "old-romexis-server",
|
||||||
|
"romexis_version": "6.5.3",
|
||||||
|
"database_name": "Romexis_db"
|
||||||
|
},
|
||||||
|
"backup": {
|
||||||
|
"file": "database/Romexis_db.bak"
|
||||||
|
},
|
||||||
|
"data": {
|
||||||
|
"images": "romexis_images",
|
||||||
|
"ergodata": "romexis_ergodata",
|
||||||
|
"cache": null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
A full example is included in:
|
||||||
|
|
||||||
|
```text
|
||||||
|
examples/manifest.example.json
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
Implemented:
|
||||||
|
|
||||||
|
- Flask web UI
|
||||||
|
- REST API
|
||||||
|
- migration job creation
|
||||||
|
- migration state stored as JSON files
|
||||||
|
- generated per-migration credentials
|
||||||
|
- SFTP server in the same container
|
||||||
|
- upload directory preparation
|
||||||
|
- validation hook
|
||||||
|
- SQL Server restore
|
||||||
|
- Python client
|
||||||
|
|
||||||
|
Not yet fully implemented:
|
||||||
|
|
||||||
|
- coordinated Romexis restart through a shared state file
|
||||||
|
- checksum validation
|
||||||
|
- progress aggregation
|
||||||
|
- authentication for the web UI
|
||||||
|
- production-grade user isolation
|
||||||
|
|
||||||
|
These pieces are intentionally separated so they can be implemented and tested step by step.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current Implementation Update
|
||||||
|
|
||||||
|
The migration service now supports both client-driven and manual browser-based migrations.
|
||||||
|
|
||||||
|
Implemented workflow:
|
||||||
|
|
||||||
|
1. Create migration job.
|
||||||
|
2. Upload database backup through the Web UI or client workflow.
|
||||||
|
3. Generate `manifest.json` on the server.
|
||||||
|
4. Restore the database backup.
|
||||||
|
5. Upload `romexis_images`, `romexis_ergodata` and optionally `romexis_cache` through SFTP.
|
||||||
|
6. Confirm upload completion.
|
||||||
|
7. Validate upload.
|
||||||
|
8. Run restore workflow.
|
||||||
|
9. Request Romexis restart using the shared state file.
|
||||||
|
10. Complete migration and remove temporary SFTP access.
|
||||||
|
|
||||||
|
Additional supported operation:
|
||||||
|
|
||||||
|
- cancel a migration and remove the temporary SFTP user
|
||||||
|
|
||||||
|
Final states such as `completed`, `cancelled` and `failed` do not recreate SFTP users after a service restart.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Server-Side Manifest Generation
|
||||||
|
|
||||||
|
The server owns the manifest format. Clients and the Web UI should submit parameters only.
|
||||||
|
|
||||||
|
The restore script expects:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"backup": {
|
||||||
|
"file": "database/Romexis_db.bak"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
For manual uploads, the manifest is generated automatically after the database backup was uploaded through the Web UI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Romexis Restart Coordination
|
||||||
|
|
||||||
|
After a successful database restore, the migration service can request a Romexis restart through:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/data/romexis_images/.romexis_restart_state
|
||||||
|
```
|
||||||
|
|
||||||
|
The Romexis entrypoint monitors this state file, restarts the Romexis process and writes the result back. The migration service reads the result, logs it and removes the state file.
|
||||||
|
|
||||||
|
|
||||||
|
## Development Notes
|
||||||
|
|
||||||
|
The current implementation is designed as a practical foundation. It keeps the large-file transport path independent from the web application and makes it possible to test migration workflows incrementally.
|
||||||
|
|
||||||
|
Next steps:
|
||||||
|
|
||||||
|
- Add checksum generation and validation.
|
||||||
|
- Add authentication to the web UI.
|
||||||
|
- Add migration logs per job.
|
||||||
|
- Add a dry-run restore mode.
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
# Migration Workflow
|
||||||
|
|
||||||
|
This document describes the target workflow for migrating an existing Romexis installation into the Docker-based Romexis Server stack.
|
||||||
|
|
||||||
|
┌──────────────────────────────┐
|
||||||
|
│ Existing Romexis Server │
|
||||||
|
└──────────────┬───────────────┘
|
||||||
|
│
|
||||||
|
│ Romexis Migration Client
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
SFTP / REST API
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌──────────────────────────────┐
|
||||||
|
│ Romexis Migration Service │
|
||||||
|
│ │
|
||||||
|
│ • Migration Jobs │
|
||||||
|
│ • Upload Validation │
|
||||||
|
│ • Restore Workflow │
|
||||||
|
└──────────────┬───────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌──────────────────────────────┐
|
||||||
|
│ Romexis Docker Container │
|
||||||
|
│ SQL Server │
|
||||||
|
│ Images │
|
||||||
|
│ Ergodata │
|
||||||
|
└──────────────────────────────┘
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Updated Detailed Workflow
|
||||||
|
|
||||||
|
```text
|
||||||
|
created
|
||||||
|
-> database_uploaded
|
||||||
|
-> database_restored
|
||||||
|
-> upload_complete
|
||||||
|
-> validated
|
||||||
|
-> restored
|
||||||
|
-> completed
|
||||||
|
```
|
||||||
|
|
||||||
|
Final states:
|
||||||
|
|
||||||
|
```text
|
||||||
|
completed
|
||||||
|
cancelled
|
||||||
|
failed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual browser workflow
|
||||||
|
|
||||||
|
1. Create a migration job in the Web UI.
|
||||||
|
2. Upload the database backup in the browser.
|
||||||
|
3. The service creates `manifest.json` automatically.
|
||||||
|
4. Trigger database restore.
|
||||||
|
5. Upload file directories via SFTP.
|
||||||
|
6. Confirm upload complete.
|
||||||
|
7. Validate upload.
|
||||||
|
8. Run restore.
|
||||||
|
9. Complete migration and remove SFTP access.
|
||||||
|
|
||||||
|
### Windows client workflow
|
||||||
|
|
||||||
|
1. Detect Romexis installation.
|
||||||
|
2. Detect SQL Server configuration.
|
||||||
|
3. Detect data directories.
|
||||||
|
4. Create or use a migration job.
|
||||||
|
5. Create database backup.
|
||||||
|
6. Upload data via rclone/SFTP.
|
||||||
|
7. Call API endpoints to advance the migration workflow.
|
||||||
|
8. Display logs and restore status.
|
||||||
|
|
||||||
|
### Restart after restore
|
||||||
|
|
||||||
|
The migration service and Romexis container communicate via:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/data/romexis_images/.romexis_restart_state
|
||||||
|
```
|
||||||
|
|
||||||
|
The migration service writes a pending restart request. The Romexis entrypoint stops and restarts the Romexis process, writes the final status and the migration service removes the state file.
|
||||||
|
|
||||||
|
|
||||||
|
## Phase 1: Preparation on the Source System
|
||||||
|
|
||||||
|
The source system is usually an existing Windows-based Romexis server.
|
||||||
|
|
||||||
|
Required data:
|
||||||
|
|
||||||
|
- SQL Server database backup (`.bak`)
|
||||||
|
- Romexis image directory
|
||||||
|
- Romexis ergo data directory
|
||||||
|
- optional cache directory
|
||||||
|
|
||||||
|
The cache directory can usually be skipped because it can be regenerated.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Create Migration Job
|
||||||
|
|
||||||
|
A migration job is created in the web UI.
|
||||||
|
|
||||||
|
The service generates:
|
||||||
|
|
||||||
|
- migration ID
|
||||||
|
- SFTP username
|
||||||
|
- SFTP password
|
||||||
|
- upload path
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Upload Data
|
||||||
|
|
||||||
|
Recommended tools:
|
||||||
|
|
||||||
|
- rclone over SFTP
|
||||||
|
- WinSCP
|
||||||
|
- FileZilla
|
||||||
|
- OpenSSH SFTP
|
||||||
|
|
||||||
|
Example using rclone:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rclone copy ./migration-data sftp:upload \
|
||||||
|
--transfers 8 \
|
||||||
|
--checkers 16 \
|
||||||
|
--progress
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Validate Upload
|
||||||
|
|
||||||
|
The migration service validates:
|
||||||
|
|
||||||
|
- manifest exists
|
||||||
|
- SQL backup exists
|
||||||
|
- image directory exists
|
||||||
|
- ergo directory exists
|
||||||
|
|
||||||
|
Later versions should add:
|
||||||
|
|
||||||
|
- checksum validation
|
||||||
|
- file count comparison
|
||||||
|
- expected size validation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: Restore
|
||||||
|
|
||||||
|
The restore process performs or coordinates:
|
||||||
|
|
||||||
|
1. stop the Romexis server container
|
||||||
|
2. restore the SQL Server database backup
|
||||||
|
3. copy data directories into the target volumes
|
||||||
|
4. fix ownership and permissions
|
||||||
|
5. start the Romexis server container
|
||||||
|
6. run a final validation
|
||||||
|
|
||||||
|
The current workflow uses explicit state transitions and keeps restart coordination separated through the shared state file.
|
||||||
+92
@@ -0,0 +1,92 @@
|
|||||||
|
# Security
|
||||||
|
|
||||||
|
## Repository Security
|
||||||
|
|
||||||
|
The repository must not contain proprietary Romexis application binaries.
|
||||||
|
|
||||||
|
Installer packages are downloaded during payload builds from official URLs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Secrets
|
||||||
|
|
||||||
|
Sensitive values should come from:
|
||||||
|
|
||||||
|
```text
|
||||||
|
.env
|
||||||
|
Drone secrets
|
||||||
|
Docker Compose environment
|
||||||
|
external secret stores
|
||||||
|
```
|
||||||
|
|
||||||
|
Important values:
|
||||||
|
|
||||||
|
```text
|
||||||
|
MSSQL_SA_PASSWORD
|
||||||
|
ROMEXIS_DB_PASSWORD
|
||||||
|
FIREBIRD_PASSWORD
|
||||||
|
MIGRATION_API_TOKEN
|
||||||
|
SFTP passwords
|
||||||
|
Registry credentials
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Migration Service Security
|
||||||
|
|
||||||
|
Migration jobs create temporary SFTP credentials.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- credentials should be unique per job
|
||||||
|
- credentials should be removed after completion
|
||||||
|
- cancelled jobs should remove credentials
|
||||||
|
- completed jobs should not recreate SFTP users on startup
|
||||||
|
- failed jobs should not recreate SFTP users unless explicitly reactivated
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Network Exposure
|
||||||
|
|
||||||
|
Expose only required ports.
|
||||||
|
|
||||||
|
Typical ports:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Romexis RMI ports
|
||||||
|
MSSQL 1433, if external access is required
|
||||||
|
Firebird 3050, if external access is required
|
||||||
|
Migration Web UI 8080
|
||||||
|
Migration SFTP 2222
|
||||||
|
```
|
||||||
|
|
||||||
|
For production, place services behind appropriate firewall rules.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Permissions
|
||||||
|
|
||||||
|
Persistent directories must be writable by the relevant container users.
|
||||||
|
|
||||||
|
Special care is required for:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/var/opt/mssql
|
||||||
|
/firebird/data
|
||||||
|
/data/romexis_images
|
||||||
|
/data/romexis_ergodata
|
||||||
|
/data/romexis_cache
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Production Notes
|
||||||
|
|
||||||
|
Before production use:
|
||||||
|
|
||||||
|
- change all default passwords
|
||||||
|
- restrict exposed ports
|
||||||
|
- use HTTPS/reverse proxy for the migration Web UI
|
||||||
|
- use strong API tokens
|
||||||
|
- remove test migration jobs
|
||||||
|
- verify backup/restore procedures
|
||||||
Reference in New Issue
Block a user