Skip to content

An OpenMW mod that lets you play the whole Elder Scrolls III: Morrowind as a top-down isometric game — with full online co-op or MMO modes.

C++

Packaging keeps the test characters: the server's data stays, a backup first, the zip clean

Alan Saunders12ba72914ad2
diff --git a/README.md b/README.md
index 18deed4..ac31434 100644
--- a/README.md
+++ b/README.md
@@ -34,6 +34,7 @@ parent folder (`<root>`):
crabnet\ cloned and built by the script
bundled-mods\ the third-party mods, fetched by the script
dist\Isowind\ the package, and dist\Isowind.zip beside it
+ backups\ test characters and saves, zipped before each packaging (the last five)
Then, in PowerShell, from the repository:
@@ -49,6 +50,14 @@ fetches the bundled mods and packages. Later runs are incremental; after pulling
The engine ends up in `MSVC2022_64\Release`, `Isowind.exe` in `MSVC2022_64\launcher`, and the package in
`<root>\dist\Isowind` with `<root>\dist\Isowind.zip` beside it.
+Rebuilding keeps the test characters. The server's players, cells and world (`dist\Isowind\server\CoreScripts\data`)
+and the host's `server\tes3mp-server.cfg` stay in the package folder; single-player saves are in
+`Documents\My Games\OpenMW\saves`, which packaging never touches. `make_package.ps1` still zips both into
+`<root>\backups\isowind-data-<date>-<time>.zip` first, and keeps the last five. The zip takes the server's data from
+git (the empty folders and the default ban and plugin lists), so it never carries anyone's characters. To update a
+copy elsewhere (Bottles, another PC), unzip the new `Isowind.zip` over the old folder rather than deleting the folder:
+the players stay, and only the ban list goes back to empty.
+
| Switch | What it does |
|-------------------------|------------------------------------------------------------------------------------------------|
| `-SkipDependencies` | Doesn't check or re-extract the dependency downloads. Use it on every build after the first. |
@@ -82,8 +91,11 @@ Building on Linux
`.github/workflows/linux.yml` has the recipe: the Debian/Ubuntu packages from `CI/install_debian_deps.sh`, CrabNet
at commit `19e6619`, a CMake configure that points `RakNet_INCLUDES` and `RakNet_LIBRARY_RELEASE`/`_DEBUG` at it,
then `cmake --build`. `isowind/linux/package.sh BUILD_DIR OUT_DIR [BUNDLED_MODS_DIR]` assembles the folder (it
-needs `linuxdeploy` on the PATH). On Arch, build Bullet and TinyXML in-tree (`-DOPENMW_USE_SYSTEM_BULLET=OFF
--DUSE_SYSTEM_TINYXML=OFF`), and CMake 4 wants `-DCMAKE_POLICY_VERSION_MINIMUM=3.5` for CrabNet.
+needs `linuxdeploy` on the PATH). Packaging into the same folder again keeps its server data and
+`server/tes3mp-server.cfg`, as on Windows, after backing them and `~/.local/share/openmw/saves` up into
+`~/.local/share/isowind/backups`; a release goes into a new folder, as the CI's does. On Arch, build Bullet and
+TinyXML in-tree (`-DOPENMW_USE_SYSTEM_BULLET=OFF -DUSE_SYSTEM_TINYXML=OFF`), and CMake 4 wants
+`-DCMAKE_POLICY_VERSION_MINIMUM=3.5` for CrabNet.
Running with a controller
-------------------------
diff --git a/isowind/linux/package.sh b/isowind/linux/package.sh
index bcd26b9..569ed09 100755
--- a/isowind/linux/package.sh
+++ b/isowind/linux/package.sh
@@ -1,20 +1,62 @@
#!/bin/sh
-# Assembles Isowind for Linux the way package/make_package.ps1 does for Windows: the engine with the libraries it
-# needs, the mod, the bundled mods, pixel art, profiles, server, licenses, readme, and isowind.sh to start it all.
+# Assembles Isowind for Linux the way isowind/windows/make_package.ps1 does for Windows: the engine with the
+# libraries it needs, the mod, the bundled mods, pixel art, profiles, server, licenses, readme, and isowind.sh to
+# start it all.
#
# package.sh BUILD_DIR OUT_DIR [BUNDLED_MODS_DIR]
#
# Needs linuxdeploy on PATH (it copies in every library that isn't part of the base system, so the folder runs on
# other distributions too).
+#
+# OUT_DIR is laid down afresh, except for what hosting from it made, which is the host's: the server's saved players,
+# cells and world (server/CoreScripts/data) and its settings (server/tes3mp-server.cfg), so packaging again keeps the
+# test characters. Those and the single-player saves (~/.local/share/openmw/saves) are backed up first, in
+# ~/.local/share/isowind/backups (the last five). A release is packaged into a new folder, as the CI does.
set -e
SRC=$(cd "$(dirname "$0")/../.." && pwd)
I="$SRC/isowind"
BUILD=$(cd "$1" && pwd)
OUT=$2
MODS=$3
+DATA_HOME=${XDG_DATA_HOME:-$HOME/.local/share}
-rm -rf "$OUT"
mkdir -p "$OUT"
+OUT=$(cd "$OUT" && pwd)
+# What is in OUT goes (below): only a package from here, or a new or empty folder
+if [ -n "$(ls -A "$OUT")" ] && [ ! -f "$OUT/isowind.sh" ]; then
+ echo "error: $OUT has files in it and isn't an Isowind package; give a new or empty folder" >&2
+ exit 1
+fi
+
+# Backup of what testing made (not while there is nothing yet but the server's defaults)
+backup() {
+ STATE=$(find "$OUT/server/CoreScripts/data" "$OUT/server/tes3mp-server.cfg" "$DATA_HOME/openmw/saves" -type f \
+ ! -name .gitkeep ! -path "$OUT/server/CoreScripts/data/banlist.json" \
+ ! -path "$OUT/server/CoreScripts/data/requiredDataFiles.json" 2>/dev/null | head -n 1)
+ if [ -z "$STATE" ]; then return 0; fi
+ set --
+ for f in server/CoreScripts/data server/tes3mp-server.cfg; do
+ if [ -e "$OUT/$f" ]; then set -- "$@" -C "$OUT" "$f"; fi
+ done
+ if [ -d "$DATA_HOME/openmw/saves" ]; then set -- "$@" -C "$DATA_HOME/openmw" saves; fi
+ BACKUPS="$DATA_HOME/isowind/backups"
+ mkdir -p "$BACKUPS"
+ BACKUP="$BACKUPS/isowind-data-$(date +%Y%m%d-%H%M%S).tar.gz"
+ tar -czf "$BACKUP" "$@"
+ ls "$BACKUPS" | grep -E '^isowind-data-[0-9]{8}-[0-9]{6}\.tar\.gz$' | sort -r | tail -n +6 |
+ while IFS= read -r old; do rm -f "$BACKUPS/$old"; done
+ echo "Backup: $BACKUP"
+}
+backup
+
+# Everything but the host's server data and settings goes
+find "$OUT" -mindepth 1 -maxdepth 1 ! -name server -exec rm -rf {} +
+if [ -d "$OUT/server" ]; then
+ find "$OUT/server" -mindepth 1 -maxdepth 1 ! -name CoreScripts ! -name tes3mp-server.cfg -exec rm -rf {} +
+fi
+if [ -d "$OUT/server/CoreScripts" ]; then
+ find "$OUT/server/CoreScripts" -mindepth 1 -maxdepth 1 ! -name data -exec rm -rf {} +
+fi
# Engine: engine/usr/bin with its config, settings defaults and resources beside it, libraries in engine/usr/lib
linuxdeploy --appdir "$OUT/engine" -e "$BUILD/openmw" -e "$BUILD/tes3mp-server" -e "$BUILD/openmw-iniimporter"
@@ -25,8 +67,24 @@ for look in pixelart normal; do
mkdir -p "$OUT/profiles/$look"
cp "$I/profiles/$look/"* "$I/profiles/input_v3.xml" "$OUT/profiles/$look/"
done
-mkdir -p "$OUT/server" "$OUT/tools" "$OUT/licenses"
-cp -r "$I/server/CoreScripts" "$I/server/tes3mp-server-default.cfg" "$OUT/server/"
+mkdir -p "$OUT/server/CoreScripts" "$OUT/tools" "$OUT/licenses"
+cp "$I/server/tes3mp-server-default.cfg" "$OUT/server/"
+# CoreScripts with Isowind's changes. Of its data, the repository's as git has it (the empty folders, the ban list,
+# the plugin list; not whatever a server run in the repository left there), and only what the folder doesn't have
+# yet, but for the plugin list, which goes with the bundled mods
+for f in "$I/server/CoreScripts"/* "$I/server/CoreScripts"/.[!.]*; do
+ case ${f##*/} in data) ;; *) if [ -e "$f" ]; then cp -r "$f" "$OUT/server/CoreScripts/"; fi ;; esac
+done
+DATA_FILES=$(git -C "$SRC" ls-files isowind/server/CoreScripts/data 2>/dev/null | sed 's|^isowind/server/CoreScripts/||')
+if [ -z "$DATA_FILES" ]; then
+ DATA_FILES=$(cd "$I/server/CoreScripts" && { find data -type f -name .gitkeep; find data -maxdepth 1 -type f -name '*.json'; })
+fi
+printf '%s\n' "$DATA_FILES" | while IFS= read -r f; do
+ if [ ! -e "$OUT/server/CoreScripts/$f" ] || [ "$f" = data/requiredDataFiles.json ]; then
+ mkdir -p "$OUT/server/CoreScripts/${f%/*}"
+ cp "$I/server/CoreScripts/$f" "$OUT/server/CoreScripts/$f"
+ fi
+done
cp "$I/tools/retro_textures.py" "$OUT/tools/"
cp "$SRC/LICENSE" "$OUT/licenses/GPL-3.0.txt"
cp "$SRC/AUTHORS.md" "$OUT/licenses/OpenMW-AUTHORS.md"
diff --git a/isowind/windows/build.ps1 b/isowind/windows/build.ps1
index 7c15a15..de3e95c 100644
--- a/isowind/windows/build.ps1
+++ b/isowind/windows/build.ps1
@@ -6,7 +6,10 @@
# <root>\crabnet\ the multiplayer networking library, cloned and built here
# <root>\bundled-mods\ the third-party mods (make_package.ps1 wants them); fetched from this repository's
# "bundled-mods" release (bundled-mods.zip, as the Linux CI does) when the folder is missing
-# <root>\dist\ the package
+# <root>\dist\ the package. Rebuilding keeps what testing it made (the server's players and world,
+# the host's settings); the zip never has them
+# <root>\backups\ those and the single-player saves (Documents\My Games\OpenMW\saves), zipped by
+# make_package.ps1 before each packaging; the last five are kept
# The engine builds in MSVC2022_64 in the repository, Isowind.exe in MSVC2022_64\launcher.
#
# Needs on PATH: git (Git for Windows, whose bash runs OpenMW's dependency script), cmake, and Visual Studio 2022
diff --git a/isowind/windows/make_package.ps1 b/isowind/windows/make_package.ps1
index ca21f0e..3117933 100644
--- a/isowind/windows/make_package.ps1
+++ b/isowind/windows/make_package.ps1
@@ -1,6 +1,8 @@
# Assemble the Isowind package into <root>\dist\Isowind and zip it as <root>\dist\Isowind.zip, <root> being the folder
# the repository is in (the bundled mods are there too, in bundled-mods). Run after building OpenMW (openmw,
# openmw-launcher, openmw-wizard) and Isowind.exe; build.ps1 beside this script does all of that and then this.
+# Packaging again keeps the test characters in dist\Isowind (the server's data) and backs them up in <root>\backups
+# first; the zip is always clean.
$ErrorActionPreference = 'Stop'
$src = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path # the repository
@@ -17,6 +19,48 @@ function Mirror($from, $to, [string[]]$extra = @()) {
if ($LASTEXITCODE -ge 8) { throw "robocopy $from -> $to failed ($LASTEXITCODE)" }
}
+# Copies only the files $to doesn't have yet (robocopy skips those changed, newer or older there), deleting nothing
+function CopyMissing($from, $to) {
+ & robocopy $from $to /E /XC /XN /XO /R:2 /W:2 /NFL /NDL /NJH /NJS /NP | Out-Null
+ if ($LASTEXITCODE -ge 8) { throw "robocopy $from -> $to failed ($LASTEXITCODE)" }
+}
+
+Add-Type -AssemblyName System.IO.Compression, System.IO.Compression.FileSystem
+
+# Backup of what testing made: the server's saved players, cells and world and the host's settings, which the package
+# folder keeps (below), and the single-player saves in Documents\My Games\OpenMW (OpenMW's user data; packaging never
+# touches them). Zipped into <root>\backups\isowind-data-<date>-<time>.zip all the same, the last five kept; not while
+# there is nothing yet but the server's defaults.
+$backups = Join-Path $root 'backups'
+$serverData = Join-Path $dist 'server\CoreScripts\data'
+$backupFrom = [ordered]@{ 'server/CoreScripts/data' = $serverData } # folder in the zip = folder here
+$documents = [Environment]::GetFolderPath('MyDocuments')
+if ($documents) { $backupFrom['saves'] = Join-Path $documents 'My Games\OpenMW\saves' }
+$backup = @()
+foreach ($name in $backupFrom.Keys) {
+ if (-not (Test-Path $backupFrom[$name])) { continue }
+ $base = (Get-Item $backupFrom[$name] -Force).FullName
+ $backup += @(Get-ChildItem $base -Recurse -File -Force | ForEach-Object {
+ [pscustomobject]@{ File = $_.FullName; Entry = "$name/" + $_.FullName.Substring($base.Length + 1).Replace('\', '/') }
+ })
+}
+$hostSettings = Join-Path $dist 'server\tes3mp-server.cfg'
+if (Test-Path $hostSettings) { $backup += [pscustomobject]@{ File = $hostSettings; Entry = 'server/tes3mp-server.cfg' } }
+$defaults = 'banlist.json', 'requiredDataFiles.json' | ForEach-Object { "server/CoreScripts/data/$_" }
+if ($backup | Where-Object { $_.Entry -notin $defaults -and $_.Entry -notlike '*/.gitkeep' }) {
+ New-Item -ItemType Directory -Force $backups | Out-Null
+ $backupZip = Join-Path $backups ('isowind-data-{0:yyyyMMdd-HHmmss}.zip' -f (Get-Date))
+ $archive = [IO.Compression.ZipFile]::Open($backupZip, 'Create')
+ try {
+ foreach ($file in $backup) {
+ [IO.Compression.ZipFileExtensions]::CreateEntryFromFile($archive, $file.File, $file.Entry, 'Optimal') | Out-Null
+ }
+ } finally { $archive.Dispose() }
+ Get-ChildItem $backups -Filter 'isowind-data-*.zip' | Where-Object { $_.Name -match '^isowind-data-\d{8}-\d{6}\.zip$' } |
+ Sort-Object Name -Descending | Select-Object -Skip 5 | Remove-Item -Force
+ "Backup: $backupZip"
+}
+
# Engine: the build output (OpenMW, its launcher and wizard, libraries, resources) with this project's shaders and
# layouts laid over it, so they are current even if the build didn't copy them.
Mirror $release (Join-Path $dist 'engine') @('/XF', '*.pdb', '*.ilk', '*.exp', '*.lib', 'openmw-cs.exe', '*_tests.exe',
@@ -42,10 +86,15 @@ foreach ($look in 'pixelart', 'normal') {
}
# Multiplayer server (engine\tes3mp-server.exe, started from here by Isowind.exe's "Host a server"): its settings, and
-# TES3MP's CoreScripts with Isowind's changes, without anyone's saved players or world.
+# TES3MP's CoreScripts with Isowind's changes. What hosting from the package made is the host's and stays, so a rebuild
+# keeps the test characters: the saved players, cells and world in CoreScripts\data (/XD: neither copied nor cleared
+# by /MIR) and tes3mp-server.cfg. The zip leaves them out (below); the logs go.
$server = Join-Path $dist 'server'
-# (/MIR also clears what testing the package left: saved players, the world; its logs go too)
-Mirror (Join-Path $isowind 'server\CoreScripts') (Join-Path $server 'CoreScripts')
+Mirror (Join-Path $isowind 'server\CoreScripts') (Join-Path $server 'CoreScripts') @('/XD', 'data')
+# Of the repository's data, what the folder doesn't have yet (the empty folders, the ban list), and the plugin list
+# always, as it goes with the bundled mods
+CopyMissing (Join-Path $isowind 'server\CoreScripts\data') $serverData
+Copy-Item (Join-Path $isowind 'server\CoreScripts\data\requiredDataFiles.json') $serverData -Force
if (Test-Path (Join-Path $server 'logs')) { Remove-Item -Recurse -Force (Join-Path $server 'logs') }
Copy-Item (Join-Path $isowind 'server\tes3mp-server-default.cfg') $server -Force
@@ -100,22 +149,36 @@ The top-down mod (Lua) and shaders are in the mod and pixelart folders as they a
"@ | Set-Content -Encoding utf8 (Join-Path $source 'README.txt')
# Zip: only what the package is made of, under an Isowind folder (not whatever else ended up in dist\Isowind, such
-# as a copy unzipped there to play).
+# as a copy unzipped there to play, or what playing and hosting from it left: the launcher's last choices in
+# profiles\, the host's tes3mp-server.cfg, crash dumps in server\).
$zip = Join-Path $root 'dist\Isowind.zip'
if (Test-Path $zip) { Remove-Item $zip -Force }
-Add-Type -AssemblyName System.IO.Compression, System.IO.Compression.FileSystem
+# The server's data from the repository as git has it (the empty folders, the ban list, the plugin list), not the
+# package folder's, which has the players, cells and world of whoever tested it
+$cleanData = @(& git -C $src ls-files -- 'isowind/server/CoreScripts/data')
+if ($LASTEXITCODE -ne 0 -or $cleanData.Count -eq 0) {
+ $repoData = Join-Path $isowind 'server\CoreScripts\data'
+ Write-Warning "git couldn't list the server's data files: the zip takes the .gitkeep files and data\*.json from $repoData"
+ $cleanData = @(Get-ChildItem $repoData -Recurse -File -Force |
+ Where-Object { $_.Name -eq '.gitkeep' -or ($_.DirectoryName -eq $repoData -and $_.Extension -eq '.json') } |
+ ForEach-Object { 'isowind/server/CoreScripts/data/' + $_.FullName.Substring($repoData.Length + 1).Replace('\', '/') })
+}
$archive = [IO.Compression.ZipFile]::Open($zip, 'Create')
try {
- foreach ($item in 'engine', 'licenses', 'mod', 'mods', 'pixelart', 'profiles', 'server', 'source', 'Isowind.exe', 'README.txt') {
+ foreach ($item in 'engine', 'licenses', 'mod', 'mods', 'pixelart', 'profiles\pixelart', 'profiles\normal',
+ 'server\CoreScripts', 'server\tes3mp-server-default.cfg', 'source', 'Isowind.exe', 'README.txt') {
$path = Join-Path $dist $item
$files = if (Test-Path $path -PathType Container) { Get-ChildItem $path -Recurse -File } else { Get-Item $path }
foreach ($file in $files) {
$entry = 'Isowind/' + $file.FullName.Substring($dist.Length + 1).Replace('\', '/')
- # A host's own choices (Host a server) stay theirs
- if ($entry -eq 'Isowind/server/tes3mp-server.cfg') { continue }
+ if ($entry -like 'Isowind/server/CoreScripts/data/*') { continue }
[IO.Compression.ZipFileExtensions]::CreateEntryFromFile($archive, $file.FullName, $entry, 'Optimal') | Out-Null
}
}
+ foreach ($file in $cleanData) {
+ $entry = 'Isowind/server/' + $file.Substring('isowind/server/'.Length)
+ [IO.Compression.ZipFileExtensions]::CreateEntryFromFile($archive, (Join-Path $src $file), $entry, 'Optimal') | Out-Null
+ }
} finally { $archive.Dispose() }
"Package: $dist"
"Zip: $zip ({0:N0} MB)" -f ((Get-Item $zip).Length / 1MB)