From 5d4f3063a9420a0a69b070cffa62b0485b8f621e Mon Sep 17 00:00:00 2001 From: Minseong Choi Date: Tue, 28 Jul 2026 07:45:53 +0900 Subject: [PATCH] feat(operator): make any Paper image joinable behind the forwarding proxy Velocity modern forwarding is proxy-WIDE. A backend that cannot verify the signed handshake does not degrade -- it rejects every login the proxy forwards. Until now the only backends that could verify it were the two images Felis builds itself (deploy/limbo, deploy/lobby), which read FELIS_FORWARDING_SECRET in their own entrypoints. An arbitrary Paper image a user brings does not, so it passed admission, started, reported Ready, and was UNJOINABLE. The platform's answer was to recommend the lobby image as a base for a user's own world (0018_recommended_images.sql), which was never a good base -- it carries the /menu plugin whose job is to TRANSFER a joining player away, the exact opposite of a server you mean to stay on. The fix configures forwarding from OUTSIDE the image instead of requiring it inside. The operator now injects a root `felis init-forwarding` initContainer into every user server; it writes the proxies.velocity block into config/paper-global.yml and forces online-mode=false in server.properties on the /data PVC before the main container starts. The image needs no forwarding logic of its own, so the joinable set stops being "images that self-configure forwarding" and becomes every Paper-family image the platform runs. buildStatefulSet gates the injection on the ABSENCE of the system-role label: the Felis-built system servers already consume the secret in their entrypoints and the login gate is a limbo, not Paper. It is also gated on a non-empty felis image name -- the operator Deployment passes its own image as FELIS_IMAGE, and an operator without it skips the injection rather than failing, because a cluster whose proxy is not in modern mode has nothing to configure. The init runs as root deliberately. The world volume's ownership comes from the storage provisioner and the main container runs as whatever UID its image declares, so root is the only UID that can reliably write these files; it then chmods them 0666/0777 so that non-root main container can rewrite them on boot. The privilege is bounded -- the init exits before the server container starts and the server container keeps its own UID. The alternative, an fsGroup on the pod, is noted in the code as the upgrade path if the init ever stops running as root. The writer merges rather than overwrites, both because Paper expands paper-global.yml to its full default tree on first boot and because the panel file editor may edit either file between boots. It sets proxies.velocity.* and the single online-mode key and leaves every other setting alone. It is a no-op on an empty secret, for the same reason the env var is optional: a proxy that is not in modern mode provisions no Secret, and wedging every server's init on a missing optional value would be worse than the status quo. felis-paper (deploy/paper) is the platform's plain-Paper expression of that base and 0019 seeds it recommended: same PAPER_JAR_URL the lobby build already resolves, no /menu plugin, no forwarding gate, and a correctly-escaped RCON channel so the console, the online-player list and permission commands work out of the box. 0018's row is left in place -- an admin who kept it can keep it; this only adds the better default beside it. Three fixes ride along, each of which the 1.8 path hit in practice. bootstrap pins ViaVersion's serverside-blockconnections off. ConnectionData.init() only builds its block-connection provider when Via's lowest supported protocol is below 1.13; under modern forwarding the Velocity injector reports 393, so init() returns early, blockConnectionProvider stays null, and the first 1.12.2->1.13 chunk rewrite dereferences it -- a 1.8 client takes an NPE on the first chunk it is sent and never finishes joining. Every call site is behind isServersideBlockConnections(), so switching it off skips all of them, at a cosmetic pre-1.13 cost: fences and glass panes stop drawing connected. ViaVersion ships the option ON, so a fresh install shipped that NPE. Seeding a file with this one key suffices -- Config#loadConfig parses the bundled default as the base map and merges the on-disk file over it, so every other option stays current across version bumps. The absence of "Loading block connection mappings" in the log is NOT evidence this worked: init() gates on the protocol version too, and that half fails on its own, so the line is missing either way. The config value is the only evidence, which is what the test asserts. The Velocity unit gains -Dfelis.legacy-forwarding.servers=legacy18. A protocol-47 backend sits behind ViaVersion, which strips modern forwarding's login-plugin-message when it down-translates the proxy->backend pipeline to 47 -- the packet is registered from 1.13 and has nowhere to go. Only the handshake address field survives Via, so the Felis fork forwards the named servers BungeeCord-style while every other backend keeps modern+secret untouched. v1 hardcodes the one legacy backend; rendering the list from the MinecraftServer CRs is the upgrade path. deploy/lobby's set_prop escapes the value before substituting it. The RCON password is operator-provisioned arbitrary bytes, and a '|', '\' or '&' in one corrupts a bare `sed s|...|...|` and silently kills the key -- taking the console, the online-player list and permission commands with it. deploy/paper was written with the escaping, so the lobby gets the same rather than leaving the sibling caller broken. Verified: the full Go suite passes on Windows and on Fedora 44 (go1.26.4), where TestWriteForwardingFileModes actually runs its POSIX mode assertions instead of skipping. The new tests cover the initContainer's image, root UID, world mount and secret env; the merge preserving unrelated config trees; the properties upsert including the commented-key case; and the bootstrap script both writing the Via key and still calling the function that writes it. Not verified: the initContainer has never run in a real cluster, and the felis-paper image is code-only here as the other game-stack images are -- no Go CI builds them. The ViaVersion pin is the one piece with live evidence, and that evidence is what it was written from. Before it, a client was cut within a second of "logged in with entity id" on legacy18 while the proxy logged the NPE above -- REMAP OF LEVEL_CHUNK chained into Protocol1_8To1_9's MAP_BULK_CHUNK. It was applied by hand to the running proxy on 2026-07-24 at 14:47 and only then written back into bootstrap. At 14:48:14 the same player joined real Paper 1.8.8 through the fork, issued commands, approved an op-login from in-game at 14:50:39, and held the connection until 15:30:09 -- 42 minutes. Neither session says which client version it was. The proxy never logged a protocol number. It bounds above at 1.16.4, from the viabackwards "(1.17->1.16.4) ... for 1.16 players and below" warning that fired for that player on the lobby leg, and no lower -- Via floors every handshake to the proxy's 393, so anything from 47 up is admissible. Reading Protocol1_8To1_9 in the stack as a client-version tell is backwards: that chain runs on the BACKEND leg, up-translating the 47 server's chunks to the floor. What the NPE proves is that the pin was load-bearing, not who was holding the mouse. That is one hand-run session on one host, and it is not a cell. The 393->47 leg has one now, in Felis-Legacy -- FL-009 puts a genuine protocol-47 client on a stock Paper 1.8.8 behind this proxy and flips this same option: on it, cut 0.2s after JoinGame with the fault above; off, holds. No automated test in THIS repository exercises the leg. --- bootstrap_asset_test.go | 34 +++ cmd/felis/initforwarding.go | 201 ++++++++++++++++++ cmd/felis/initforwarding_test.go | 189 ++++++++++++++++ cmd/felis/operator.go | 5 + cmd/felis/run.go | 1 + cmd/felis/run_test.go | 2 +- deploy/bootstrap.sh | 59 ++++- deploy/demo-up.sh | 10 + deploy/lobby/entrypoint.sh | 6 +- deploy/paper/Dockerfile | 57 +++++ deploy/paper/entrypoint.sh | 76 +++++++ internal/operator/builders.go | 74 ++++++- internal/operator/builders_internal_test.go | 65 +++++- internal/operator/reconciler.go | 6 +- internal/platform/workloads.go | 6 + internal/platform/workloads_test.go | 13 +- .../migrations/0019_recommended_paper.sql | 49 +++++ 17 files changed, 835 insertions(+), 18 deletions(-) create mode 100644 cmd/felis/initforwarding.go create mode 100644 cmd/felis/initforwarding_test.go create mode 100644 deploy/paper/Dockerfile create mode 100644 deploy/paper/entrypoint.sh create mode 100644 internal/store/migrations/0019_recommended_paper.sql diff --git a/bootstrap_asset_test.go b/bootstrap_asset_test.go index 3f44585..b54c803 100644 --- a/bootstrap_asset_test.go +++ b/bootstrap_asset_test.go @@ -64,6 +64,40 @@ func TestLobbyLuckPermsWiringIsConsistent(t *testing.T) { } } +// A 1.8 client joining a protocol-47 backend dies on the first chunk unless ViaVersion's +// serverside block-connection tracking is off: under modern forwarding the Velocity injector +// reports 1.13 as the lowest supported protocol, ConnectionData.init() returns early on that, +// and the 1.12.2->1.13 chunk rewrite then dereferences the provider init() never built. +// +// ViaVersion ships the option ON, so this is a correction bootstrap has to make rather than a +// default it can inherit — and nothing else in the install would notice it missing. The failure +// surfaces only when a legacy player joins, on a host that installed cleanly. +func TestBootstrapPinsViaBlockConnectionsOff(t *testing.T) { + // go:embed takes the working tree verbatim, and this repository pins no eol attribute, so + // a Windows checkout embeds CRLF. Only the assertion spanning a line break below cares. + script := strings.ReplaceAll(BootstrapScript(), "\r\n", "\n") + + const key = "serverside-blockconnections" + if !strings.Contains(script, key+": false") { + t.Errorf("bootstrap.sh never writes %s: false; a fresh install inherits ViaVersion's "+ + "default of true and NPEs the first 1.8 player to receive a chunk", key) + } + + // Writing the value is only half of it: the file has to be the one ViaVersion reads. + // Via names its data directory after the plugin in lowercase. + if !strings.Contains(script, "plugins/viaversion") { + t.Error("bootstrap.sh does not target plugins/viaversion, so whatever it writes is " + + "not the config ViaVersion loads") + } + + // Via staging and this correction have to stay welded together. If the call is dropped, + // every branch above still exists and still looks right in review. + if !strings.Contains(script, "pin_via_block_connections\n ok \"Via staged") { + t.Error("install_via_plugins no longer calls pin_via_block_connections; the jars would " + + "be staged with the option left at its default") + } +} + func readGameStackFile(t *testing.T, name string) string { t.Helper() b, err := gameStackAssets.ReadFile(name) diff --git a/cmd/felis/initforwarding.go b/cmd/felis/initforwarding.go new file mode 100644 index 0000000..7d678b4 --- /dev/null +++ b/cmd/felis/initforwarding.go @@ -0,0 +1,201 @@ +package main + +import ( + "flag" + "fmt" + "io" + "os" + "path/filepath" + "strings" + + "sigs.k8s.io/yaml" +) + +// forwardingSecretEnv is the env var the operator injects the Velocity +// modern-forwarding secret under (mirrors internal/operator.envForwardingSecret). +const forwardingSecretEnv = "FELIS_FORWARDING_SECRET" + +// defaultForwardingDataDir is the world PVC mount inside a server pod (mirrors +// internal/operator.dataMountPath). It is Paper's working directory, so its +// config/ and server.properties live under it. +const defaultForwardingDataDir = "/data" + +// fwd*Mode make the written config readable AND rewritable by the main server +// container, whose UID we do not control (an arbitrary user image). The +// initContainer runs as root (see buildStatefulSet) so it can write into a data +// volume of unknown ownership; 0666/0777 then let a non-root Paper rewrite the +// same files on boot. +// +// ponytail: relies on the initContainer running as root to write into a volume of +// unknown ownership; that is how the operator schedules it. If that ever changes, +// give the server pod an fsGroup so the shared volume is group-writable instead. +const ( + fwdFileMode os.FileMode = 0o666 + fwdDirMode os.FileMode = 0o777 +) + +// cmdInitForwarding is the felis-image initContainer entrypoint that makes an +// ARBITRARY Paper image joinable behind a modern-forwarding Velocity proxy, +// WITHOUT modifying that image: it writes the Velocity block into +// /config/paper-global.yml and forces online-mode=false in +// /server.properties before the server container starts. This is the same +// config deploy/lobby/entrypoint.sh writes for the Felis-built lobby, lifted into +// Go so it can be applied to an image Felis did not build. +// +// It is idempotent and MERGE-based: Paper expands paper-global.yml to its full +// default tree on first boot, and the panel file editor may change either file +// between boots, so it only ever sets proxies.velocity.* and the single +// online-mode key and preserves every other setting. +// +// "Preserves" means values, not formatting, and only for the YAML half: +// sigs.k8s.io/yaml round-trips through JSON, so paper-global.yml comes back with +// its keys sorted and its comments dropped. Every setting survives and Paper reads +// it back identically, but a user who annotated that file loses the annotations. +// Accepted rather than fixed: comment-faithful editing means a yaml.v3 Node walk, +// which is a lot of machinery for two keys. server.properties is edited line-wise +// and does keep its comments and ordering. +// +// An empty/unset secret is a deliberate no-op (exit 0): a cluster whose proxy is +// not in modern mode provisions no Secret, and wedging every server's init on a +// missing optional value would be worse than the pre-forwarding status quo. +func cmdInitForwarding(args []string, stdout, stderr io.Writer) int { + fs := flag.NewFlagSet("init-forwarding", flag.ContinueOnError) + fs.SetOutput(stderr) + dataDir := fs.String("data", defaultForwardingDataDir, "server data directory (Paper working dir)") + if err := fs.Parse(args); err != nil { + return 2 + } + secret := os.Getenv(forwardingSecretEnv) + if err := writePaperForwarding(*dataDir, secret); err != nil { + fmt.Fprintf(stderr, "felis init-forwarding: %v\n", err) + return 1 + } + if secret == "" { + fmt.Fprintln(stdout, "felis init-forwarding: no forwarding secret set; leaving config untouched") + } else { + fmt.Fprintln(stdout, "felis init-forwarding: wrote Velocity modern-forwarding config") + } + return 0 +} + +// writePaperForwarding writes both config surfaces (or nothing, when secret == ""). +func writePaperForwarding(dataDir, secret string) error { + if secret == "" { + return nil + } + if err := writePaperGlobal(dataDir, secret); err != nil { + return err + } + return forceServerPropertyOffline(dataDir) +} + +// writePaperGlobal merges the proxies.velocity block into config/paper-global.yml, +// creating the file and its directory when absent and preserving every other key. +func writePaperGlobal(dataDir, secret string) error { + dir := filepath.Join(dataDir, "config") + if err := os.MkdirAll(dir, fwdDirMode); err != nil { + return fmt.Errorf("create %s: %w", dir, err) + } + // MkdirAll honours the process umask (root's is typically 022 → 0755); chmod + // does not, and a non-root main container must be able to place/replace the + // file in this directory on boot. + if err := os.Chmod(dir, fwdDirMode); err != nil { + return fmt.Errorf("chmod %s: %w", dir, err) + } + + path := filepath.Join(dir, "paper-global.yml") + root := map[string]any{} + if existing, err := os.ReadFile(path); err == nil { + if err := yaml.Unmarshal(existing, &root); err != nil { + return fmt.Errorf("parse existing %s: %w", path, err) + } + if root == nil { // an empty or "null" document unmarshals to a nil map + root = map[string]any{} + } + } else if !os.IsNotExist(err) { + return fmt.Errorf("read %s: %w", path, err) + } + + setVelocity(root, secret) + out, err := yaml.Marshal(root) + if err != nil { + return fmt.Errorf("marshal %s: %w", path, err) + } + return writeFileMode(path, out) +} + +// setVelocity sets proxies.velocity.{enabled,online-mode,secret}, creating the +// intermediate maps when missing. proxies.velocity.online-mode is Paper trusting +// that the proxy verified the player as premium — distinct from server.properties +// online-mode, which must be false so the backend does not re-authenticate. +func setVelocity(root map[string]any, secret string) { + velocity := childMap(childMap(root, "proxies"), "velocity") + velocity["enabled"] = true + velocity["online-mode"] = true + velocity["secret"] = secret +} + +// childMap returns parent[key] as a map, replacing a missing or non-map value with +// a fresh one. sigs.k8s.io/yaml decodes nested objects to map[string]any (JSON +// semantics), so the assertion holds for any well-formed paper-global.yml. +func childMap(parent map[string]any, key string) map[string]any { + if m, ok := parent[key].(map[string]any); ok { + return m + } + m := map[string]any{} + parent[key] = m + return m +} + +// forceServerPropertyOffline sets online-mode=false in server.properties. A backend +// behind a modern-forwarding proxy must be offline-mode (the proxy did the Mojang +// auth); an arbitrary image defaulting to online-mode=true rejects every proxied +// login. +func forceServerPropertyOffline(dataDir string) error { + path := filepath.Join(dataDir, "server.properties") + var content []byte + if b, err := os.ReadFile(path); err == nil { + content = b + } else if !os.IsNotExist(err) { + return fmt.Errorf("read %s: %w", path, err) + } + return writeFileMode(path, upsertProperty(content, "online-mode", "false")) +} + +// upsertProperty sets key=value in a java .properties body, replacing an existing +// uncommented assignment or appending one, and leaving every other line — +// comments included — untouched. Keys sit at column 0 the way Paper writes them, +// so a "#key=" comment does not match. +func upsertProperty(content []byte, key, value string) []byte { + want := key + "=" + value + prefix := key + "=" + lines := strings.Split(string(content), "\n") + found := false + for i, ln := range lines { + if strings.HasPrefix(ln, prefix) { + lines[i] = want + found = true + } + } + if found { + return []byte(strings.Join(lines, "\n")) + } + body := string(content) + if body != "" && !strings.HasSuffix(body, "\n") { + body += "\n" + } + return []byte(body + want + "\n") +} + +// writeFileMode writes data then forces the mode, since WriteFile honours the +// umask (root's is typically 022 → 0644) but a non-root main container must be +// able to rewrite these files on boot. +func writeFileMode(path string, data []byte) error { + if err := os.WriteFile(path, data, fwdFileMode); err != nil { + return fmt.Errorf("write %s: %w", path, err) + } + if err := os.Chmod(path, fwdFileMode); err != nil { + return fmt.Errorf("chmod %s: %w", path, err) + } + return nil +} diff --git a/cmd/felis/initforwarding_test.go b/cmd/felis/initforwarding_test.go new file mode 100644 index 0000000..d6671d8 --- /dev/null +++ b/cmd/felis/initforwarding_test.go @@ -0,0 +1,189 @@ +package main + +import ( + "os" + "path/filepath" + "runtime" + "strings" + "testing" + + "sigs.k8s.io/yaml" +) + +// readYAML parses a paper-global.yml into a nested map for assertions. +func readYAML(t *testing.T, path string) map[string]any { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + var m map[string]any + if err := yaml.Unmarshal(b, &m); err != nil { + t.Fatalf("parse %s: %v", path, err) + } + return m +} + +// velocityBlock digs out proxies.velocity from a parsed paper-global.yml. +func velocityBlock(t *testing.T, root map[string]any) map[string]any { + t.Helper() + proxies, ok := root["proxies"].(map[string]any) + if !ok { + t.Fatalf("no proxies map: %v", root) + } + vel, ok := proxies["velocity"].(map[string]any) + if !ok { + t.Fatalf("no proxies.velocity map: %v", proxies) + } + return vel +} + +// An empty secret must touch nothing: a proxy that is not in modern mode +// provisions no Secret, and the init must not wedge the pod over it. +func TestWritePaperForwardingEmptySecretIsNoop(t *testing.T) { + dir := t.TempDir() + if err := writePaperForwarding(dir, ""); err != nil { + t.Fatalf("writePaperForwarding: %v", err) + } + if _, err := os.Stat(filepath.Join(dir, "config", "paper-global.yml")); !os.IsNotExist(err) { + t.Errorf("paper-global.yml should not exist, stat err = %v", err) + } + if _, err := os.Stat(filepath.Join(dir, "server.properties")); !os.IsNotExist(err) { + t.Errorf("server.properties should not exist, stat err = %v", err) + } +} + +// A fresh data dir gets a complete velocity block and an offline server.properties. +func TestWritePaperForwardingFreshDir(t *testing.T) { + dir := t.TempDir() + if err := writePaperForwarding(dir, "s3cr3t"); err != nil { + t.Fatalf("writePaperForwarding: %v", err) + } + + vel := velocityBlock(t, readYAML(t, filepath.Join(dir, "config", "paper-global.yml"))) + if vel["enabled"] != true { + t.Errorf("velocity.enabled = %v, want true", vel["enabled"]) + } + if vel["online-mode"] != true { + t.Errorf("velocity.online-mode = %v, want true", vel["online-mode"]) + } + if vel["secret"] != "s3cr3t" { + t.Errorf("velocity.secret = %v, want s3cr3t", vel["secret"]) + } + + props, err := os.ReadFile(filepath.Join(dir, "server.properties")) + if err != nil { + t.Fatalf("read server.properties: %v", err) + } + if !strings.Contains(string(props), "online-mode=false") { + t.Errorf("server.properties missing online-mode=false:\n%s", props) + } +} + +// A pre-existing paper-global.yml (as Paper expands it on first boot) must keep +// all of its other keys — clobbering them would silently reset the user's tuning +// on every restart. This is the whole reason the writer merges rather than +// overwrites. +func TestWritePaperGlobalPreservesExistingKeys(t *testing.T) { + dir := t.TempDir() + cfg := filepath.Join(dir, "config") + if err := os.MkdirAll(cfg, 0o755); err != nil { + t.Fatal(err) + } + existing := []byte("proxies:\n velocity:\n enabled: false\n secret: OLD\nchunk-loading:\n autoconfig-send-distance: true\nmisc:\n max-joins-per-tick: 5\n") + if err := os.WriteFile(filepath.Join(cfg, "paper-global.yml"), existing, 0o644); err != nil { + t.Fatal(err) + } + + if err := writePaperGlobal(dir, "NEW"); err != nil { + t.Fatalf("writePaperGlobal: %v", err) + } + + root := readYAML(t, filepath.Join(cfg, "paper-global.yml")) + vel := velocityBlock(t, root) + if vel["enabled"] != true || vel["secret"] != "NEW" { + t.Errorf("velocity not updated: %v", vel) + } + // Unrelated trees survive. + if _, ok := root["chunk-loading"].(map[string]any); !ok { + t.Errorf("chunk-loading tree lost: %v", root) + } + misc, ok := root["misc"].(map[string]any) + if !ok { + t.Fatalf("misc tree lost: %v", root) + } + // sigs.k8s.io/yaml decodes numbers via JSON, so 5 arrives as float64(5). + if misc["max-joins-per-tick"] != float64(5) { + t.Errorf("misc.max-joins-per-tick = %v, want 5", misc["max-joins-per-tick"]) + } +} + +// online-mode=false must be forced while every other property line — comments +// included — is left untouched. +func TestForceServerPropertyOfflinePreservesOthers(t *testing.T) { + dir := t.TempDir() + existing := "#Minecraft server properties\nmotd=Hello World\nonline-mode=true\ndifficulty=hard\n" + if err := os.WriteFile(filepath.Join(dir, "server.properties"), []byte(existing), 0o644); err != nil { + t.Fatal(err) + } + if err := forceServerPropertyOffline(dir); err != nil { + t.Fatalf("forceServerPropertyOffline: %v", err) + } + got, err := os.ReadFile(filepath.Join(dir, "server.properties")) + if err != nil { + t.Fatal(err) + } + s := string(got) + if strings.Contains(s, "online-mode=true") { + t.Errorf("online-mode=true not replaced:\n%s", s) + } + if !strings.Contains(s, "online-mode=false") { + t.Errorf("online-mode=false not set:\n%s", s) + } + for _, keep := range []string{"#Minecraft server properties", "motd=Hello World", "difficulty=hard"} { + if !strings.Contains(s, keep) { + t.Errorf("lost line %q:\n%s", keep, s) + } + } +} + +// upsertProperty appends when the key is absent and does not grow blank lines. +func TestUpsertPropertyAppends(t *testing.T) { + got := string(upsertProperty([]byte("motd=hi"), "online-mode", "false")) + if got != "motd=hi\nonline-mode=false\n" { + t.Errorf("append form wrong: %q", got) + } + empty := string(upsertProperty(nil, "online-mode", "false")) + if empty != "online-mode=false\n" { + t.Errorf("empty form wrong: %q", empty) + } + // A commented key must not count as present. + commented := string(upsertProperty([]byte("#online-mode=true\n"), "online-mode", "false")) + if !strings.Contains(commented, "#online-mode=true") || !strings.Contains(commented, "\nonline-mode=false\n") { + t.Errorf("comment mishandled: %q", commented) + } +} + +// The written files must be group/world writable so a non-root main container can +// rewrite them. chmod semantics are POSIX-only, so this asserts on non-Windows. +func TestWriteForwardingFileModes(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("POSIX file modes not represented on Windows") + } + dir := t.TempDir() + if err := writePaperForwarding(dir, "x"); err != nil { + t.Fatal(err) + } + for _, p := range []string{ + filepath.Join(dir, "config", "paper-global.yml"), + filepath.Join(dir, "server.properties"), + } { + fi, err := os.Stat(p) + if err != nil { + t.Fatal(err) + } + if fi.Mode().Perm() != fwdFileMode { + t.Errorf("%s mode = %o, want %o", p, fi.Mode().Perm(), fwdFileMode) + } + } +} diff --git a/cmd/felis/operator.go b/cmd/felis/operator.go index 38e2a46..fbc950c 100644 --- a/cmd/felis/operator.go +++ b/cmd/felis/operator.go @@ -4,6 +4,7 @@ import ( "flag" "fmt" "io" + "os" "felis.lolicon.best/internal/apis/felis/v1alpha1" felismetrics "felis.lolicon.best/internal/metrics" @@ -73,6 +74,10 @@ func cmdOperator(args []string, _, stderr io.Writer) int { Client: mgr.GetClient(), Scheme: mgr.GetScheme(), Prober: operator.RconProber{}, + // The operator's own image, for the forwarding-config initContainer it + // injects into user servers. The Deployment passes it as FELIS_IMAGE (see + // platform.OperatorDeployment); absent, that injection is simply skipped. + FelisImage: os.Getenv("FELIS_IMAGE"), } if err := r.SetupWithManager(mgr); err != nil { fmt.Fprintf(stderr, "felis operator: setup controller: %v\n", err) diff --git a/cmd/felis/run.go b/cmd/felis/run.go index 9652632..bdf0343 100644 --- a/cmd/felis/run.go +++ b/cmd/felis/run.go @@ -52,6 +52,7 @@ var commands = map[string]func(args []string, stdout, stderr io.Writer) int{ "setup": cmdSetup, "breakGlass": cmdBreakGlass, "bootstrap-assets": cmdBootstrapAssets, + "init-forwarding": cmdInitForwarding, "version": cmdVersion, "update": cmdUpdate, } diff --git a/cmd/felis/run_test.go b/cmd/felis/run_test.go index 40b5565..3364631 100644 --- a/cmd/felis/run_test.go +++ b/cmd/felis/run_test.go @@ -40,7 +40,7 @@ func TestRunUnknownCommand(t *testing.T) { // undocumentedCommands are routable on purpose but kept out of the usage text: they // are called by deploy/bootstrap.sh, not by a human at a prompt. Listing them here is // what makes their absence from usage a deliberate decision rather than an oversight. -var undocumentedCommands = map[string]bool{"bootstrap-assets": true} +var undocumentedCommands = map[string]bool{"bootstrap-assets": true, "init-forwarding": true} // The usage text and the dispatch table must describe the same set of commands. // diff --git a/deploy/bootstrap.sh b/deploy/bootstrap.sh index 8fef83f..9bd4a50 100644 --- a/deploy/bootstrap.sh +++ b/deploy/bootstrap.sh @@ -102,6 +102,10 @@ APT_LOCK_TIMEOUT="${APT_LOCK_TIMEOUT:-$PKG_LOCK_TIMEOUT}" # --- the game stack: proxy on the host, the two always-on backends in k3s --- FELIS_LIMBO_IMAGE="${FELIS_LIMBO_IMAGE:-felis-limbo:demo}" FELIS_LOBBY_IMAGE="${FELIS_LOBBY_IMAGE:-felis-lobby:demo}" +# Plain Paper base recommended for a user's own server (deploy/paper). Not a system +# server — forwarding is applied by the operator's init-forwarding initContainer, so it +# needs no secret. Seeded recommended in 0019_recommended_paper.sql. +FELIS_PAPER_IMAGE="${FELIS_PAPER_IMAGE:-felis-paper:demo}" # The Velocity MINOR is pinned, not discovered. PaperMC's Fill v3 groups velocity # builds by version group, and "newest across all groups" today means 4.0.0-SNAPSHOT — # an UNRELEASED proxy (the 4.0.0 group has zero published builds) that needs a Java 25 @@ -1256,8 +1260,15 @@ build_game_stack() { --build-arg LUCKPERMS_JAR_URL="$LUCKPERMS_JAR_URL" \ -t "$FELIS_LOBBY_IMAGE" "$GAME_STACK_DIR" + # Plain Paper, same MC_VERSION and PAPER_JAR_URL (no new dependency). Forwarding is the + # operator initContainer's job, so this image carries no /menu plugin and no secret gate. + log "building ${FELIS_PAPER_IMAGE} (plain Paper ${MC_VERSION}, forwarding via the operator initContainer)" + docker build -f "${GAME_STACK_DIR}/deploy/paper/Dockerfile" \ + --build-arg PAPER_JAR_URL="$PAPER_JAR_URL" \ + -t "$FELIS_PAPER_IMAGE" "$GAME_STACK_DIR" + local img - for img in "$FELIS_LIMBO_IMAGE" "$FELIS_LOBBY_IMAGE"; do + for img in "$FELIS_LIMBO_IMAGE" "$FELIS_LOBBY_IMAGE" "$FELIS_PAPER_IMAGE"; do log "importing ${img} into k3s containerd" remove_k3s_image "$img" docker save "$img" | k3s_cmd ctr images import - @@ -1384,9 +1395,45 @@ ViaVersion 5.11.0 18d19e90fc9467d68128c076630ae8700449c901402a3ef421837ce006bc8c ViaBackwards 5.11.0 b21983d561e3f92df257683f0133ab6c68ec68175e8acfd82c6231723bf83587 ViaRewind 4.1.3 2d5970d22b4711c9ab2800932326c7b08acdace25ed7c6bbb8f6ea81054962b4 EOF + pin_via_block_connections ok "Via staged; clients from 1.8 up can join under modern forwarding" } +# pin_via_block_connections turns ViaVersion's serverside block-connection tracking off. +# +# ConnectionData.init() only builds its block-connection provider when Via's lowest supported +# protocol is below 1.13. Under modern forwarding the Velocity injector reports 393 (1.13), so +# init() returns early, blockConnectionProvider stays null, and the first 1.12.2->1.13 chunk +# rewrite dereferences it. A 1.8 client on a protocol-47 backend takes an NPE on the first chunk +# it is sent and never finishes joining. Every call site in protocols/v1_12_2to1_13 is behind +# isServersideBlockConnections(), so switching the option off skips all of them. The cost is +# cosmetic and pre-1.13 only: fences and glass panes stop being drawn connected. +# +# ViaVersion's default is true, so a fresh install ships that NPE unless it is corrected here. +# Seeding a file with this one key is enough: Config#loadConfig parses the bundled +# assets/viaversion/config.yml as the base map and merges the on-disk file over it, so every +# other option still comes from the shipped default and stays current across version bumps. +# +# Do not read the absence of "Loading block connection mappings" from the log as proof this +# worked. init() gates on the protocol version as well, and under modern forwarding that half +# fails on its own — the line is missing either way. The config value is the only evidence. +pin_via_block_connections() { + local dir="${VELOCITY_DIR}/plugins/viaversion" config tmp + config="${dir}/config.yml" + ensure_velocity_directory "$dir" 0750 "$VELOCITY_USER" "$VELOCITY_USER" + tmp="$(mktemp "${VELOCITY_DIR}/.viaversion-config.XXXXXX")" + remember_temp "$tmp" + if [ ! -f "$config" ]; then + printf 'serverside-blockconnections: false\n' > "$tmp" + elif grep -qE '^serverside-blockconnections:' "$config"; then + sed -E 's/^serverside-blockconnections:.*/serverside-blockconnections: false/' "$config" > "$tmp" + else + { cat "$config"; printf 'serverside-blockconnections: false\n'; } > "$tmp" + fi + # Via rewrites this file itself on every load, so the proxy user has to own it. + atomic_install_file "$tmp" "$config" 0640 "$VELOCITY_USER" "$VELOCITY_USER" +} + install_jre() { local arch url if [ -x "${JRE_DIR}/bin/java" ]; then @@ -1539,6 +1586,14 @@ install_velocity_service() { # out of the box — not just the standalone `felis nano`. felis-api enforces the reclaim # blacklist on this route; a loopback nano would bypass it. api_ip="$(felis_internal_ip)" + # Servers that receive their forwarded identity through the handshake address (BungeeCord/legacy + # style) instead of the proxy-wide modern+secret forwarding. A protocol-47 (1.8.x) backend sits + # behind ViaVersion, which strips modern forwarding's login-plugin-message when it down-translates + # the proxy->backend pipeline to protocol 47; only the handshake field survives Via. The Felis + # fork reads this list from -Dfelis.legacy-forwarding.servers and forwards those servers legacy; + # every other backend keeps modern+secret untouched. v1 hardcodes the one legacy backend; the + # upgrade path is to have the operator render this list from the MinecraftServer CRs. + local legacy_forwarding_servers="legacy18" cat > "$VELOCITY_SERVICE" </dev/null 2>&1 || die "docker not found; cannot build images" @@ -76,6 +78,14 @@ else --build-arg PAPER_JAR_URL="$PAPER_JAR_URL" \ -t "$LOBBY_IMAGE" "$SRC_DIR" docker save "$LOBBY_IMAGE" | "$K3S" ctr images import - + + # Plain Paper recommended base — same PAPER_JAR_URL, no plugins, no secret gate. + : "${PAPER_IMAGE:=felis-paper:demo}" + log "building $PAPER_IMAGE (plain Paper $PAPER_MC_VERSION, forwarding via the operator initContainer)" + docker build -f "$SRC_DIR/deploy/paper/Dockerfile" \ + --build-arg PAPER_JAR_URL="$PAPER_JAR_URL" \ + -t "$PAPER_IMAGE" "$SRC_DIR" + docker save "$PAPER_IMAGE" | "$K3S" ctr images import - fi # 3. wire the images into the config `felis setup` reads ------------------------ diff --git a/deploy/lobby/entrypoint.sh b/deploy/lobby/entrypoint.sh index 230e6fb..88ee5ac 100644 --- a/deploy/lobby/entrypoint.sh +++ b/deploy/lobby/entrypoint.sh @@ -53,7 +53,11 @@ cd "$DATA_DIR" # set_prop KEY VALUE — replace the key's line in server.properties, or append it if absent. set_prop() { if [ -f "$PROPS" ] && grep -q "^$1=" "$PROPS"; then - sed -i "s|^$1=.*|$1=$2|" "$PROPS" + # The RCON password is operator-provisioned arbitrary bytes: a '|', '\' or '&' would + # otherwise corrupt this bare sed s||| and silently break the key. Same escaping as + # deploy/limbo — without it an unlucky password kills the console/permission channel. + esc=$(printf '%s' "$2" | sed 's/[|\\&]/\\&/g') + sed -i "s|^$1=.*|$1=${esc}|" "$PROPS" else printf '%s=%s\n' "$1" "$2" >> "$PROPS" fi diff --git a/deploy/paper/Dockerfile b/deploy/paper/Dockerfile new file mode 100644 index 0000000..16be69c --- /dev/null +++ b/deploy/paper/Dockerfile @@ -0,0 +1,57 @@ +# Felis general-purpose Paper image: plain Paper, forwarding-ready. +# +# CODE-ONLY in this repo — not built by the Go CI. This is a drop-in base for a user's +# OWN world, offered as a platform-recommended image (see +# internal/store/migrations/0019_recommended_paper.sql). It is NOT a system server: it +# carries no felis-paper /menu plugin, no LuckPerms, and no forwarding-secret gate. +# +# It writes NO Velocity forwarding config itself. The operator injects a root +# `felis init-forwarding` initContainer into every USER server (internal/operator/ +# builders.go: buildStatefulSet) that writes config/paper-global.yml + server.properties +# online-mode=false onto the /data PVC before this container starts. That external step is +# what makes an arbitrary Paper image joinable through the modern-forwarding proxy — so +# this image needs no forwarding logic of its own, and by the same mechanism ANY Paper +# image a user brings is made joinable the same way. If the initContainer is absent (no +# FELIS_IMAGE configured) Paper boots as a standalone online server: degraded, not broken. +# +# Build (deploy/bootstrap.sh does this for you; PAPER_JAR_URL comes from PaperMC's Fill v3 +# API — the SAME url the lobby build resolves, so this reuses it and adds no new dependency): +# docker build -f deploy/paper/Dockerfile \ +# --build-arg PAPER_JAR_URL=https://fill-data.papermc.io/v1/objects//paper--.jar \ +# -t felis-paper:demo . +# docker save felis-paper:demo | sudo k3s ctr images import - +# # felis.toml → recommended via 0019_recommended_paper.sql (no [velocity] key points here) +# +# The Paper version must match MC_VERSION: the login gate (LOOHP/Limbo) speaks exactly ONE +# protocol per build, and a client that passes the gate must also reach this backend. +# bootstrap resolves both Paper and Limbo from the same MC_VERSION, so they always agree. + +# 25-jre, not 21: Paper 26.2 declares java.version.minimum=25 (PaperMC Fill v3) and refuses +# to boot on anything older. +FROM eclipse-temurin:25-jre +ARG PAPER_JAR_URL +RUN set -eu; \ + if [ -z "${PAPER_JAR_URL:-}" ]; then \ + echo "ERROR: --build-arg PAPER_JAR_URL= is required" >&2; exit 1; \ + fi; \ + apt-get update && apt-get install -y --no-install-recommends curl ca-certificates; \ + mkdir -p /paper; \ + curl -fSL "$PAPER_JAR_URL" -o /paper/paper.jar; \ + apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* +COPY deploy/paper/entrypoint.sh /usr/local/bin/felis-entrypoint.sh + +# The operator mounts the world PVC at /data and the entrypoint runs Paper with it as the +# working directory, so worlds, generated config and paperclip's extracted runtime all land +# on the PVC. /paper stays the immutable image seed: the jar is never copied onto the +# volume, so the panel file editor (which sees only /data) cannot tamper with it. +WORKDIR /data + +# FELIS_GAME_PORT is the port the entrypoint pins Paper to; it MUST equal the operator's +# GamePort (internal/operator/builders.go). Default 25565 — override only in lockstep with +# the operator. +ENV FELIS_GAME_PORT=25565 +EXPOSE 25565 +# felis-entrypoint.sh writes eula.txt + server.properties, then execs `java -jar +# /paper/paper.jar --nogui` from /data. Invoked via `sh` so no +x bit is needed from the +# (Windows) build host. +ENTRYPOINT ["/bin/sh", "/usr/local/bin/felis-entrypoint.sh"] diff --git a/deploy/paper/entrypoint.sh b/deploy/paper/entrypoint.sh new file mode 100644 index 0000000..e23c6fd --- /dev/null +++ b/deploy/paper/entrypoint.sh @@ -0,0 +1,76 @@ +#!/bin/sh +# Felis general-purpose Paper server entrypoint. +# +# A plain Paper backend for a user's OWN world — NOT a system server. Unlike deploy/limbo +# and deploy/lobby it writes no Velocity forwarding config and has no secret gate: the +# operator injects a root `felis init-forwarding` initContainer that writes +# config/paper-global.yml + server.properties online-mode=false onto /data BEFORE this +# container starts, so forwarding is configured externally and this stays a drop-in Paper +# image. With no initContainer (no FELIS_IMAGE) Paper just boots standalone-online — +# degraded, not broken; a user's own world is not identity-critical, so refusing to boot +# would be the wrong failure here. +# +# What this DOES own: eula, the game-port pin, and the RCON control channel. The operator +# injects RCON_PASSWORD/RCON_PORT into every server whose spec.rcon is enabled +# (operator.buildEnv), and Paper only reads these keys from server.properties — so without +# writing them here the console, the online-player list and permission commands go dark; +# env alone does nothing. This is the exact trap the lobby entrypoint documents. +set -eu + +PORT="${FELIS_GAME_PORT:-25565}" +RUNTIME_DIR="/paper" +DATA_DIR="/data" +PROPS="server.properties" + +cd "$DATA_DIR" +printf 'eula=true\n' > eula.txt + +# set_prop KEY VALUE — replace the key's line in server.properties, or append it if absent. +# The escaping matches deploy/limbo: an RCON password is operator-provisioned arbitrary +# bytes, so a '|', '\' or '&' in the value would corrupt a bare `sed s|...|...|` and +# silently break the key (the bug the lobby's original set_prop carried). +set_prop() { + if [ -f "$PROPS" ] && grep -q "^$1=" "$PROPS"; then + esc=$(printf '%s' "$2" | sed 's/[|\\&]/\\&/g') + sed -i "s|^$1=.*|$1=${esc}|" "$PROPS" + else + printf '%s=%s\n' "$1" "$2" >> "$PROPS" + fi +} + +# Pin the port the operator's Service, readiness probe and NetworkPolicy all key off +# (GamePort). A stale persisted properties file with a different port would be unreachable +# through that fence. +set_prop server-port "$PORT" + +# RCON is the control plane's write channel (spec §8 写=RCON): the operator probes it for +# readiness and the player tally, and felis-api runs console/permission commands over it. +# Rewritten on EVERY boot from the Secret, so the value is derived state — an owner who +# edits or clobbers these lines through the panel file editor cannot lock the control plane +# out of their own server, because the next restart restores the real password. +# +# No password, no RCON: an empty enable-rcon=true would admit anything that reaches the port +# unauthenticated. Unlike the forwarding secret this is not fatal — a server without the +# write channel still serves players — so it warns and starts rather than refusing. +if [ -n "${RCON_PASSWORD:-}" ]; then + set_prop enable-rcon true + set_prop rcon.port "${RCON_PORT:-25575}" + set_prop rcon.password "$RCON_PASSWORD" + echo "felis-paper: rcon enabled on port ${RCON_PORT:-25575}" +else + set_prop enable-rcon false + echo "felis-paper: WARNING — RCON_PASSWORD is empty, so the console, the online-player" >&2 + echo " list and permission changes will be unavailable for this server. The operator" >&2 + echo " injects it from the -rcon Secret when spec.rcon.enabled is true." >&2 +fi + +echo "felis-paper: server-port=${PORT} (Velocity forwarding is applied by the init-forwarding initContainer)" +JAVA_MEMORY_ARG="" +if [ -n "${JAVA_MEMORY:-}" ]; then + JAVA_MEMORY_ARG="-Xmx${JAVA_MEMORY}" +fi +set -f +# JAVA_FLAGS is emitted by the operator as a whitespace-separated JVM argument list. The jar +# stays in the immutable image seed (/paper); only worlds/config live on the PVC (cwd). +# shellcheck disable=SC2086 +exec java $JAVA_MEMORY_ARG ${JAVA_FLAGS:-} -jar "$RUNTIME_DIR/paper.jar" --nogui "$@" diff --git a/internal/operator/builders.go b/internal/operator/builders.go index d3b5dd6..80574dd 100644 --- a/internal/operator/builders.go +++ b/internal/operator/builders.go @@ -40,6 +40,11 @@ const ( dataVolumeName = "world" dataMountPath = "/data" + // felisBinaryPath is where the felis image installs its binary; the + // forwarding-config initContainer invokes it by absolute path (matches + // platform.felisBinaryPath — the same image, the same install location). + felisBinaryPath = "/usr/local/bin/felis" + // ManagedByValue / ComponentValue are the values of the LabelManagedBy / // LabelComponent labels stamped on every per-server pod (see labelsFor). They // are exported because the platform package's minecraft-namespace @@ -184,7 +189,7 @@ func readinessProbe(server *v1alpha1.MinecraftServer) *corev1.Probe { // buildStatefulSet renders the workload for replicas in {0,1}. It is where // graceful shutdown is injected: the pod gets terminationGracePeriodSeconds and // (when enabled) a preStop RCON save+stop hook. -func buildStatefulSet(server *v1alpha1.MinecraftServer, replicas int32) (*appsv1.StatefulSet, error) { +func buildStatefulSet(server *v1alpha1.MinecraftServer, replicas int32, felisImage string) (*appsv1.StatefulSet, error) { storageSize := server.Spec.Storage.Size if storageSize == "" { storageSize = defaultStorageSize @@ -235,6 +240,15 @@ func buildStatefulSet(server *v1alpha1.MinecraftServer, replicas int32) (*appsv1 } } + // An arbitrary user Paper image does not consume FELIS_FORWARDING_SECRET, so the + // operator writes the forwarding config into the world volume for it via an + // initContainer. System servers (login/lobby) are Felis-built and handle it in + // their own entrypoints, and without a felis image name there is nothing to run. + var initContainers []corev1.Container + if felisImage != "" && server.Labels[v1alpha1.LabelSystemRole] == "" { + initContainers = append(initContainers, forwardingInitContainer(felisImage)) + } + grace := graceSeconds(server) pvc := corev1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{Name: dataVolumeName}, @@ -263,6 +277,7 @@ func buildStatefulSet(server *v1alpha1.MinecraftServer, replicas int32) (*appsv1 ObjectMeta: metav1.ObjectMeta{Labels: labelsFor(server)}, Spec: corev1.PodSpec{ TerminationGracePeriodSeconds: &grace, + InitContainers: initContainers, Containers: []corev1.Container{container}, // A Minecraft server runs untrusted user worlds and plugins and // has no business calling the K8s API, so its pod must NOT carry the @@ -335,12 +350,22 @@ func buildEnv(server *v1alpha1.MinecraftServer) []corev1.EnvVar { // the player's UUID is Mojang-verified, not offline-derived) — the pod-level fence // against bypassing the proxy is the NetworkPolicy, not this value's secrecy. // - // A user server is built from an operator-typed Dockerfile, so Felis cannot make it - // consume this; the two images Felis does build (deploy/limbo, deploy/lobby) read it - // in their entrypoints and refuse to start without it. Optional so a cluster whose - // proxy is not in modern mode — no Secret provisioned — still schedules its pods - // instead of wedging them all in CreateContainerConfigError. - env = append(env, corev1.EnvVar{ + // The Felis-built images (deploy/limbo, deploy/lobby) read this in their + // entrypoints. An arbitrary user Paper image does NOT — so the operator also runs + // a forwarding-config initContainer (see forwardingInitContainer) that writes the + // Velocity block into the shared world volume before the server starts, making a + // stock Paper image joinable without modifying it. + env = append(env, forwardingSecretEnvVar()) + return env +} + +// forwardingSecretEnvVar sources FELIS_FORWARDING_SECRET from the Secret the setup +// provisioner replicas into this namespace. Optional so a cluster whose proxy is +// not in modern mode — no Secret provisioned — still schedules its pods instead of +// wedging them all in CreateContainerConfigError; the init and lobby/limbo +// entrypoints treat an empty value as "not in modern mode" and leave config alone. +func forwardingSecretEnvVar() corev1.EnvVar { + return corev1.EnvVar{ Name: envForwardingSecret, ValueFrom: &corev1.EnvVarSource{ SecretKeyRef: &corev1.SecretKeySelector{ @@ -349,12 +374,43 @@ func buildEnv(server *v1alpha1.MinecraftServer) []corev1.EnvVar { Optional: boolPtr(true), }, }, - }) - return env + } +} + +// forwardingInitContainer writes Velocity modern-forwarding config into the shared +// world volume before the server container starts, so an arbitrary Paper image Felis +// did NOT build becomes joinable behind the proxy without being modified. It runs the +// felis image's `init-forwarding` subcommand, which merges the proxies.velocity block +// into config/paper-global.yml and forces online-mode=false in server.properties. +// +// It runs as root: the world volume's ownership is set by the storage provisioner and +// the main container runs as the user image's own UID, so root is the only UID that +// can reliably write these files and leave them rewritable by that main container. +// This is a bounded privilege — the init exits before the server container starts, and +// the server container keeps whatever (non-root) UID its image declares. +// +// Only user servers get it: the Felis-built system images (login limbo, lobby) already +// consume the secret in their own entrypoints, and the login limbo is not Paper at all. +func forwardingInitContainer(felisImage string) corev1.Container { + return corev1.Container{ + Name: "init-forwarding", + Image: felisImage, + Command: []string{felisBinaryPath, "init-forwarding"}, + Env: []corev1.EnvVar{forwardingSecretEnvVar()}, + VolumeMounts: []corev1.VolumeMount{ + {Name: dataVolumeName, MountPath: dataMountPath}, + }, + SecurityContext: &corev1.SecurityContext{ + RunAsUser: int64Ptr(0), + RunAsNonRoot: boolPtr(false), + }, + } } func boolPtr(b bool) *bool { return &b } +func int64Ptr(i int64) *int64 { return &i } + func joinFlags(flags []string) string { out := "" for i, f := range flags { diff --git a/internal/operator/builders_internal_test.go b/internal/operator/builders_internal_test.go index 4b303d1..b48bd5f 100644 --- a/internal/operator/builders_internal_test.go +++ b/internal/operator/builders_internal_test.go @@ -56,13 +56,76 @@ func TestReadinessProbeHTTPCustomPath(t *testing.T) { } } +// A user server (no system-role label) gets the forwarding-config initContainer, +// running the felis image as root and mounting the world volume. A system server +// and a build with no felis image name get none. +func TestBuildStatefulSetForwardingInitContainer(t *testing.T) { + user := &v1alpha1.MinecraftServer{} + user.Spec.Storage.Size = "1Gi" + + sts, err := buildStatefulSet(user, 1, "felis:demo") + if err != nil { + t.Fatalf("buildStatefulSet: %v", err) + } + inits := sts.Spec.Template.Spec.InitContainers + if len(inits) != 1 { + t.Fatalf("want 1 initContainer, got %d", len(inits)) + } + ic := inits[0] + if ic.Image != "felis:demo" { + t.Errorf("init image = %q, want felis:demo", ic.Image) + } + if ic.SecurityContext == nil || ic.SecurityContext.RunAsUser == nil || *ic.SecurityContext.RunAsUser != 0 { + t.Errorf("init must run as root, got %+v", ic.SecurityContext) + } + mounted := false + for _, vm := range ic.VolumeMounts { + if vm.Name == dataVolumeName && vm.MountPath == dataMountPath { + mounted = true + } + } + if !mounted { + t.Errorf("init must mount the world volume at %s, got %+v", dataMountPath, ic.VolumeMounts) + } + // The whole point of the initContainer is to write the forwarding config, which it + // cannot do without the secret: a missing Env here makes `init-forwarding` no-op and + // the server Ready-but-unjoinable — the exact silent failure the feature removes. + // Same secretKeyRef rule as the main container (optional so a non-modern proxy still + // schedules), so assert it, not just the image/root/mount above. + fwd := findEnv(ic.Env, envForwardingSecret) + if fwd == nil { + t.Fatalf("init must carry %s or it writes no forwarding config", envForwardingSecret) + } + if fwd.ValueFrom == nil || fwd.ValueFrom.SecretKeyRef == nil { + t.Fatalf("%s on init must be a secretKeyRef, got %+v", envForwardingSecret, fwd) + } + if ref := fwd.ValueFrom.SecretKeyRef; ref.Name != naming.ForwardingSecretName || ref.Key != naming.ForwardingSecretKey { + t.Errorf("init %s secretKeyRef = %s/%s, want %s/%s", envForwardingSecret, ref.Name, ref.Key, naming.ForwardingSecretName, naming.ForwardingSecretKey) + } + + // No felis image name → nothing to run. + noImg, _ := buildStatefulSet(user, 1, "") + if len(noImg.Spec.Template.Spec.InitContainers) != 0 { + t.Error("no felis image must yield no initContainer") + } + + // System server handles forwarding in its own entrypoint. + sys := &v1alpha1.MinecraftServer{} + sys.Spec.Storage.Size = "1Gi" + sys.Labels = map[string]string{v1alpha1.LabelSystemRole: "lobby"} + sysSts, _ := buildStatefulSet(sys, 1, "felis:demo") + if len(sysSts.Spec.Template.Spec.InitContainers) != 0 { + t.Error("system server must get no forwarding initContainer") + } +} + // A server with a health port also exposes it as a named container port so the // kubelet can reach it. func TestBuildStatefulSetAddsHealthPort(t *testing.T) { s := &v1alpha1.MinecraftServer{} s.Spec.Storage.Size = "1Gi" s.Spec.Startup.HealthHTTPPort = 8080 - sts, err := buildStatefulSet(s, 1) + sts, err := buildStatefulSet(s, 1, "") if err != nil { t.Fatalf("buildStatefulSet: %v", err) } diff --git a/internal/operator/reconciler.go b/internal/operator/reconciler.go index 11b9400..9a99b98 100644 --- a/internal/operator/reconciler.go +++ b/internal/operator/reconciler.go @@ -40,6 +40,10 @@ type Reconciler struct { Scheme *runtime.Scheme // Prober gates readiness on RCON reachability. Prober Prober + // FelisImage is this operator's own image, used for the forwarding-config + // initContainer injected into user servers. Empty (an operator Deployment + // without FELIS_IMAGE) disables that injection rather than failing. + FelisImage string // Now is injectable for deterministic timestamps in tests; defaults to // metav1.Now. Now func() metav1.Time @@ -98,7 +102,7 @@ func (r *Reconciler) reconcileRunning(ctx context.Context, server *v1alpha1.Mine return ctrl.Result{RequeueAfter: requeueSecret}, nil } - desired, err := buildStatefulSet(server, 1) + desired, err := buildStatefulSet(server, 1, r.FelisImage) if err != nil { // A malformed spec (e.g. bad storage quantity) is terminal until edited. r.markFailed(server, "InvalidSpec", err.Error()) diff --git a/internal/platform/workloads.go b/internal/platform/workloads.go index d7a1f55..3ed4f4f 100644 --- a/internal/platform/workloads.go +++ b/internal/platform/workloads.go @@ -375,6 +375,12 @@ func OperatorDeployment(p Params) *appsv1.Deployment { "--namespace", p.MinecraftNamespace, "--metrics-bind-address", fmt.Sprintf(":%d", operatorMetricsPort), }, + // FELIS_IMAGE names this same image so the operator can run it as the + // forwarding-config initContainer it injects into user servers (it must + // name an image to run, and its own is the one image guaranteed present). + Env: []corev1.EnvVar{ + {Name: "FELIS_IMAGE", Value: p.FelisImage}, + }, Ports: []corev1.ContainerPort{ {Name: "metrics", ContainerPort: operatorMetricsPort, Protocol: corev1.ProtocolTCP}, }, diff --git a/internal/platform/workloads_test.go b/internal/platform/workloads_test.go index ddbce60..c3934fb 100644 --- a/internal/platform/workloads_test.go +++ b/internal/platform/workloads_test.go @@ -343,9 +343,16 @@ func TestOperatorDeployment_Wiring(t *testing.T) { t.Errorf("operator must mount NO Secret volume, found %q", v.Name) } } - // And it must hold no credential env at all. - if len(c.Env) != 0 { - t.Errorf("operator must carry no env (flags-only), got %v", c.Env) + // It carries exactly one plain env — FELIS_IMAGE, for the forwarding-config + // initContainer it injects into user servers — and NO credential env: nothing + // sourced from a Secret (valueFrom), since it holds no DB URL or token. + for _, e := range c.Env { + if e.ValueFrom != nil { + t.Errorf("operator must carry no credential env, found %q with valueFrom", e.Name) + } + } + if len(c.Env) != 1 || c.Env[0].Name != "FELIS_IMAGE" || c.Env[0].Value != p.FelisImage { + t.Errorf("operator env = %v, want exactly FELIS_IMAGE=%q", c.Env, p.FelisImage) } } diff --git a/internal/store/migrations/0019_recommended_paper.sql b/internal/store/migrations/0019_recommended_paper.sql new file mode 100644 index 0000000..37922be --- /dev/null +++ b/internal/store/migrations/0019_recommended_paper.sql @@ -0,0 +1,49 @@ +-- Recommended image: felis-paper — a plain Paper base for a user's OWN server. +-- +-- SUPERSEDES the "exactly one defensible recommendation" rationale of +-- 0018_recommended_images.sql. That migration was correct WHEN WRITTEN: the only +-- joinable backends were the two images that consume FELIS_FORWARDING_SECRET in their +-- own entrypoints (deploy/limbo, deploy/lobby). Velocity modern forwarding is +-- proxy-WIDE, so a backend that cannot verify the signed handshake rejects every login +-- the proxy forwards; an arbitrary Paper image, which does not consume the secret, +-- passed admission and reported Ready but was UNJOINABLE. Of the two self-configuring +-- images only the lobby was a sensible base for a user's own server, leaving exactly one. +-- +-- THAT PREMISE NO LONGER HOLDS. The operator now injects a root `felis init-forwarding` +-- initContainer into every USER server (internal/operator/builders.go: buildStatefulSet +-- gates it on the ABSENCE of the system-role label). It writes config/paper-global.yml + +-- server.properties online-mode=false onto the /data PVC before the main container starts, +-- configuring forwarding for ANY Paper-family image EXTERNALLY — the image needs no +-- forwarding logic of its own. The joinable set is therefore no longer "the images that +-- self-configure forwarding"; it is every Paper-family user image the platform runs. The +-- honest recommended list can now grow, and this is the first entry it grows by. +-- +-- felis-paper (deploy/paper) is the platform's plain-Paper expression of that base: +-- * no felis-paper /menu plugin — the lobby carries it to TRANSFER a joining player +-- away, which is exactly wrong for a server the player means to stay and play on; +-- * no forwarding-secret gate — a user's own world is not identity-critical, so it +-- boots even before forwarding is provisioned (the login gate and lobby refuse to, +-- deliberately, because THEY authenticate the Owner); +-- * a correctly-escaped RCON control channel, so the console, the online-player list +-- and permission commands work out of the box (the operator injects RCON_PASSWORD +-- into every server whose spec.rcon is enabled). +-- It is a better "your own server" base than felis-lobby, which 0018 recommended only +-- because it was then the sole joinable option. 0018's row is left in place: an admin who +-- kept it can keep it; this migration only ADDS the better default beside it. +-- +-- REF CAVEAT (identical mechanism to 0018): felis-paper:demo is the bootstrap default +-- (FELIS_PAPER_IMAGE in deploy/bootstrap.sh and deploy/demo-up.sh), built locally and +-- imported into k3s containerd. An install that overrode that variable — or whose bootstrap +-- predates this image — will not have the ref, and the pod ImagePullBackOffs visibly in +-- server status (the loud failure, not a silent refuse-to-join). An admin clears it with +-- DELETE /images?ref=felis-paper:demo, which is unvalidated and always works. Re-adding a +-- bare local containerd tag has no API path back in — POST /images runs ValidateImageRef, +-- which requires a host-qualified reference — so, like 0018, this seed is SQL and not a +-- POST. See 0018 for the full asymmetry. +-- +-- Idempotent by ON CONFLICT DO NOTHING: migrations may re-run, and an admin who +-- deliberately disabled or re-pointed this row must not have that decision silently undone. +-- added_by records platform provenance: no human admitted this row, the platform did. +INSERT INTO image_whitelist (image_ref, source, added_by, enabled) +VALUES ('felis-paper:demo', 'recommended', 'felis-platform', true) +ON CONFLICT (image_ref) DO NOTHING;