A single HTTP 502 from fill.papermc.io aborted the entire bootstrap. Build
resolution used a one-shot curl, so one gateway blip from an upstream that
flaps was indistinguishable from a permanent failure, and the run died before
Docker, k3s or any game server was provisioned.
Pass --retry 5 --retry-delay 2 to the build-resolution fetches. 502/503/504
are already in curl's built-in transient set, so the tool had solved this; the
flags were simply never passed. papermc_latest_jar is shared by the Paper and
the Velocity resolve, so hardening it once covers both callers. The
LOOHP/Limbo CI metadata fetch feeds the same step and gets the same treatment.
Deliberately no --retry-connrefused. It only adds ECONNREFUSED to a set that
already covers this incident, and it needs curl 7.52.0 while the yum (el7)
path the script supports ships 7.29.0, where an unrecognised long option is a
parse error rather than a warning:
# centos:7, curl 7.29.0
$ curl -fsSL --retry 5 --retry-delay 2 --retry-connrefused https://example.com
curl: option --retry-connrefused: is unknown
Under set -Eeuo pipefail that exits 2 and trips the || die, so both hardened
fetches would hard-fail on a host where they used to work, each naming a cause
that is not the real one. A comment above papermc_latest_jar records this so
the flag does not come back.
The failure message was actively misleading. "no Paper build for Minecraft
26.2 (the login gate speaks only that protocol)" reads as "that Minecraft
version is unsupported", sending the reader after a version-pinning problem
that does not exist: Paper 26.2 build 60 resolved fine minutes later. Say what
is actually known instead, that the build likely exists and Fill is flapping.
Verified against a local always-502 server: curl now issues 6 requests
(1 initial + 5 retries) over 10.1s before giving up, where it previously
issued 1 and died. Verified on centos:7 that this flag set is accepted, and
against the live Fill v3 API that the resolve still returns a jar URL.
Known and deliberately unchanged: no fetch sets --max-time, so an upstream
that accepts a connection and never answers still blocks forever. --retry does
not cover that, as it fires only once a request completes with a failure. The
remaining single-shot downloads (cloudflared, the Docker GPG key and repo
list, k3s, the Temurin JRE, the Velocity jar, the Go toolchain) keep their
existing no-retry shape rather than widen this diff on a script that is about
to provision a live host.
At bootstrap there is no SMTP, so the old /setup flow was unreachable: it
requested an emailed OTP that could never arrive. Setup now records the
Owner's email address unverified (no OTP round-trip) and requires a passkey,
deferring SMTP configuration to a later Settings page. Setup completes on
email-recorded + passkey-enrolled, and the lockdown lifts on the passkey, not
on email_verified: a passkey is the Owner's only pre-SMTP login credential
(email-OTP login refuses admin accounts).
The record-email endpoint (POST /account/email) now clears email_verified in
the same write. Only VerifyEmailOTP, which proves control of the address, may
set that flag; recording a fresh unproven address must never leave a stale
email_verified=true asserting a proof the user never gave. The change strictly
tightens the invariant, so no existing reader breaks.
Remove the dead ErrEmailTaken path and its documented 409: no migration puts a
unique index on users.email and the codebase does not enforce email
uniqueness, so the unique-violation branch was unreachable and the 409 an
impossible response.
The /setup route (Setup.tsx, setEmail helper, setup i18n copy) is rewritten to
match: record-email, mandatory passkey, no skip-for-now. The SMTP settings
page and post-setup configure-SMTP nudge are deferred.
cloudflared writes the tunnel credentials JSON only at `tunnel create`. An
idempotent re-run against a tunnel that already exists — or a reset +
re-bootstrap where the old box's ~/.cloudflared was wiped but the
Cloudflare-side tunnel survived — finds no local credentials file, and the
connector crash-loops with "Tunnel credentials file doesn't exist". A tunnel
that never comes up leaves op.console unreachable, so the one-time setup link
minted just before it ages out (30-min TTL) unredeemed.
CreateTunnel now resolves the tunnel id on both paths (fresh create and
already-exists) and routes through ensureCredentials, which re-fetches the
token with `cloudflared tunnel token --cred-file` (authenticating via cert.pem,
preserving the same id / DNS / Access) when the file is absent. The secret is
written to the file, not stdout, and the file is chmod 0600 so it is not left
world-readable next to cert.pem.
The self-heal is unconditional on re-bootstrap: Setup gates on Pre.check()
(cert.pem present) before CreateTunnel, so the token re-fetch always has its
cert.pem authority.
The /setup route redeems the one-time token from `felis setup`, then walks
the Owner through email-OTP verification and passkey enrollment before
handing off to the console. It sits outside RequireAuth — the visitor
arrives without a session and the redeem is what mints one — and is
reload-safe: a spent token resumes from the surviving session via
/auth/setup/status.
Adds the Setup page and its /setup route, the setup API client methods
(redeem/status), and the en-US/zh-CN onboarding strings.
The operator console (op.console.<root>) requires internal permission
verification on top of Zero-Trust: a passkey is not access. requireExternal
now refuses any non-admin principal arriving on the admin host, before any
handler, so op.console is staff-only at the door rather than per-route —
including on the passwordless demo face where Cloudflare Access is not in
front. The gate is inert on the player console (console.<root>).
Owner first-run setup is staff onboarding, so `felis setup` mints the
one-time setup URL on op.console.<root>/setup (was console.<root>). The
passkey verifier lists both console and op.console in RPOrigins so the
one-time binding asserts on either face under the shared console.<root>
RP-ID.
Session admin-access now includes role=owner, not only admin: the owner is
a superset of admin, so excluding it left IsOwner() unreachable through a
passwordless session. No path assigns role=owner yet — this is forward
consistency.
The bootstrap summary now names console.<root> the player panel and
op.console.<root> the operator console where the Owner runs setup, fixing
text that told operators not to run setup there.
Tests: op.console door gate (non-admin refused, player console unaffected,
admin passes) and owner session admin-access; the setup-bind default-host
test follows the move to op.console.
Point Velocity's authlib (mojang.sessionserver) at the felis-api hasJoined multiplexer so
a full install federates Mojang plus the configured [[auth_source]] set out of the box,
not just the standalone `felis nano`, and ship a default LittleSkin auth_source in the
generated felis.toml (delete the block for a Mojang-only server). Velocity now owns
velocity.toml and its working tree — it migrates the config version and extracts
localizations on start — while the jar and forwarding secret stay root-owned read-only and
ReadWritePaths widens to VELOCITY_DIR; the felis-api-internal ClusterIP lookup is factored
into a felis_internal_ip helper shared by the link config and the sessionserver override.
Re-include plugins/velocity in the docker context because the felis binary now embeds all
four plugin trees for `felis bootstrap-assets game-stack`.
The owner setup URL and the limbo login link were built from the admin host
(op.console.<root>, with an op.console.localhost fallback) and a hardcoded
console.<root>, so an operator who set a custom panel_hostname got an unreachable setup
link and a wrong login target. Thread the resolved panel host (defaultPanelHostname)
through performSetupMCBind, the MC-bind TUI, and the login system-server env
(new FELIS_PANEL_HOSTNAME); the limbo plugin prefers it and keeps console.<root> only as
the fallback for an older operator whose env predates it. This also matters for security:
the only wired WebAuthn verifier is scoped to the panel host, so passkey enrollment must
land on the panel face, never op.console.
While here, the limbo login handler checks link status before minting a bind code: an
already-linked player is sent straight to the lobby instead of being shown a useless code.
go-webauthn marshals CredentialCreation/CredentialAssertion as {"publicKey": {...}},
but the panel's register (Account.tsx) and username-first login (Login.tsx) read the
options flat (options.challenge, options.user.id), so base64urlToBytes(undefined) threw
"Cannot read properties of undefined (reading 'replace')" and neither ceremony could
start. Strip the envelope in the register-begin and username-login-begin handlers via a
small unwrapPublicKey helper; discoverable login keeps the envelope because it reads
options.publicKey.* plus a top-level options.login_id. The begin tests now feed a wrapped
body and assert the handlers return it flat, so they genuinely exercise the unwrap.
A premium player and a third-party player sharing a username could not both be
online. Whichever logged in second was kicked with "You are already connected to
this proxy!" -- even though the UUID rewrite had already made them two distinct
players on the backend. Velocity's player registry is keyed on the NAME (lowercased),
not the UUID, so two identities holding one name are one player as far as the proxy
is concerned, and the reclaim invariant the rewrite buys is invisible to it.
The fix needs no plugin and no state, because Velocity honours the name in the
hasJoined RESPONSE rather than pinning the one the client sent at login-start --
established by a real login, not by reading the source. So the multiplexer hands
back a different name and the collision is simply gone.
A third-party player whose name belongs to a Mojang account now joins as
PREFIX_name (LS_steve). Everyone else keeps their own name: the rename fires only
on an actual collision, decided by asking api.mojang.com whether the name is
registered. The name's owner is never the one renamed, which is 正版优先 falling out
for free -- the identity source is never rewritten, so there is no policy to encode
and no 30-day hold to track.
The premium-name answer is cached asymmetrically, because the two directions have
very different costs. "Taken" is nearly permanent (Mojang does not recycle names) and
is trusted for a day; "free" can stop being true the moment someone buys that name,
and a stale "free" leaves a squatter holding a name its real owner has just bought,
so it is trusted for ten minutes. A lookup that fails with nothing cached fails
CLOSED -- assume premium, rename the third-party player: a Mojang outage must not
become an opportunity to hold someone else's name, and being wrong that way costs a
cosmetic prefix while being wrong the other way bounces the name's owner off the
proxy. The lookup gets its own 2s client rather than sharing the 5s auth client,
since it is a SECOND Mojang round-trip on a login that already spent one.
prefix is a required, unique, 1-4 character config field rather than something
derived from the tag, because it is player-visible and no derivation can know that
"littleskin" is meant to read LS. Two sources sharing a prefix would rewrite their
same-named players onto one name, so uniqueness is enforced case-insensitively --
the proxy folds case, and LS/ls would collide there while reading as distinct here.
Also close a pre-existing hole on the path this touches: a third-party source's
profile name was relayed verbatim, so a hostile or sloppy Yggdrasil root could put
"§4admin", an empty string, or 200 characters straight into the proxy's player list.
The name is now checked against the Minecraft username charset and a bad one is a 204,
the same way a bad UUID already was.
Verified end to end on the deploy host (Velocity 3.5.1 + Paper 26.2), both branches:
premium FLYEMOJ1 -> 195fadbd-f72e-4b9b-9f8f-f92586fe16ad, name unchanged
LittleSkin FLYEMOJ1 -> LS_FLYEMOJ1, f1b7b6ae-f250-348a-b069-a2ec0fcae668
both online at once, zero "already connected" rejections
LittleSkin FelisNyaTest01 -> joins as FelisNyaTest01, no prefix, UUID still v3
The last line is the one that matters: an ordinary third-party player collides with
nobody and keeps their name, while the rewrite that keeps identities apart still ran.
Paper's "LS_FLYEMOJ1 (formerly known as li_FLYEMOJ1) joined the game" is the other
half of it -- the rename moved the player's display name and their playerdata came
along untouched, because every server-side key is the UUID and the UUID does not
depend on the name.
Known ceiling, left alone deliberately: two players of one source whose names agree
on their first 16-len(prefix)-1 characters truncate onto the same in-game name, and a
prefixed name may itself happen to be a premium name. Both cost an "already connected"
bounce, not an identity -- the UUID rewrite does not depend on the name at all.
BREAKING CHANGE: every [[auth_source]] now requires prefix = "XX" (1-4 letters or
digits, unique across sources). An existing nano felis.toml without it fails to load
with an error naming the field, rather than silently keeping the collision.
d417efc got this backwards, in both the code comment and the installer summary.
It claimed authlib appends /session/minecraft/hasJoined itself, so the property
should be given the base URL only. Velocity does not work that way, and a real
login says so: pointed at http://127.0.0.1:8081, a Mojang login arrives at nano
as
GET /?username=FLYEMOJ1&serverId=-23ae0b50...
with no path at all. Velocity appends the query string to the property verbatim
and issues the request itself; authlib is not in the loop. nano has no route on
/, so it answers 404 and Velocity kicks the player with authservers_down.
Velocity's own default for the property is the full URL,
https://sessionserver.mojang.com/session/minecraft/hasJoined, which is the same
thing said another way.
With the full endpoint URL the same account logs straight in, so both the
nano.go header and summary_nano now print
-Dmojang.sessionserver=http://127.0.0.1:8081/session/minecraft/hasJoined
and note that the flag belongs between `java` and `-jar`.
Verified against Velocity 3.5.1 + Paper 26.2 on the deploy host: a Mojang login
reaches the backend with its real Mojang UUID unchanged, and a LittleSkin login
under the same username reaches it as UUIDv3(felisAuthNS, "littleskin:"+id) --
two different players on the backend, which is the point.
Deploying `felis nano` to a real Rocky Linux 10 host surfaced four defects that
no local check could see. Fixed together because they all sit on the same path
from `curl|bash` to a running felis-nano.service.
* docker killed the nano install on EL10. `acquire_nano_binary` pulled in
docker purely to build the binary; on Rocky 10.2 the docker-ce el10 rpms
install but dockerd refuses to start, so the install died at
`systemctl enable --now docker`. nano needs one static binary, not an image,
so the docker dependency is gone: fetch_source -> install_go_toolchain
(pinned FELIS_GO_VERSION, default 1.26.4, amd64/arm64) -> build_nano_binary.
* the built binary could not be exec'd by systemd (203/EXEC). The Go linker
renames its output out of $TMPDIR, and a same-filesystem rename carries the
source SELinux label, so `go build -o /usr/local/bin/felis` produced a binary
labelled user_tmp_t rather than bin_t. root is unconfined and could run it by
hand, which is what made this look fine, but the DynamicUser service could
not. build_nano_binary now stages the output and installs it as a fresh file
so the policy type transition labels it bin_t, with restorecon as a belt.
* re-running the installer did not converge. `systemctl enable --now` is a
no-op on an already-active unit, so a rebuilt binary was installed while the
old process kept running. Now enable + restart.
* the Velocity wiring comment in cmd/felis/nano.go was wrong. authlib appends
/session/minecraft/hasJoined itself, so -Dmojang.sessionserver takes the base
URL only, as the installer has always printed.
Also bind to loopback by default. hasJoined is unauthenticated by protocol --
authlib speaks the vanilla sessionserver dialect and sends no token -- so a
public bind is an open auth relay: anyone can point their own proxy at it and
spend this host's egress IP on Mojang until Mojang rate-limits it and the
operator's own players stop getting in. It is not an identity bypass (a caller
still needs a serverId hash bound to their own server key, which the upstream
Yggdrasil validates), but it is someone else's traffic on your address.
FELIS_NANO_LISTEN and the -listen flag now default to 127.0.0.1:8081, which a
same-host Velocity reaches unchanged; serving an off-host proxy is an explicit
opt-in. configure_nano_firewall no longer opens a port for a loopback bind, and
summary_nano prints the real bind address plus the relay warning.
Verified on the target host: installs with no docker present, service active,
binary labelled bin_t, `ss` shows LISTEN 127.0.0.1:8081, an external request is
unreachable, and an in-host request returns 204 with the login logged.
BREAKING CHANGE: felis nano defaults to 127.0.0.1:8081 instead of 0.0.0.0:8081.
A Velocity proxy on another machine must now set FELIS_NANO_LISTEN (or -listen)
to a reachable address, and should allow that port only from the proxy's IP.
bootstrap.sh now asks up front whether to install the full Felis control
plane or only Felis-nano, and grows a parallel install path for the
nano-only case.
- prompt_install_mode() runs right after OS detection and reads /dev/tty
(so it works under `curl ... | sudo bash`) offering [1] Felis / [2]
Felis-nano, default full. FELIS_INSTALL_MODE=full|nano skips the prompt
for non-interactive runs; no tty falls back to full.
- main_nano() installs only what nano needs: the felis binary (reusing
the embedded-binary / docker-build acquisition), a template felis.toml
carrying a commented [[auth_source]] example (Mojang-only until edited),
a felis-nano.service unit running `felis nano -config ... -listen ...`
under DynamicUser hardening, and a firewalld port-open for the listen
port. None of the k3s / Postgres / migrate / bundle steps run.
- write_nano_config is idempotent (leaves any existing config untouched)
and its template is valid as-is. summary_nano prints the hasJoined
endpoint and the Velocity -Dmojang.sessionserver flag, offering the
127.0.0.1 form when the proxy is on the same host.
Verified on WSL: `bash -n` clean; the emitted template loads via
config.LoadNano and the exact systemd ExecStart command serves 204 on a
miss ("Mojang + 0 third-party source(s)"); a duplicate-tag config still
exits non-zero citing "unique". Not exercised: a full main_nano run,
systemd activation of the unit, and shellcheck (unavailable in this env).
`felis nano` serves the vanilla sessionserver protocol
(GET /session/minecraft/hasJoined) as a federating multiplexer over
Mojang plus any number of third-party Yggdrasil roots, with no k3s,
Postgres, or panel — a MultiLogin-style auth front-end delivered as a
subcommand of the single felis binary rather than a separate build.
- config.LoadNano reads only [[auth_source]] blocks; it skips the
database.url / root_domain / archive requirements the full server
needs. Zero sources is valid (Mojang-only).
- Mojang is prepended in code (Identity:true), never from config, so it
is always the sole identity root. Third-party profiles are rewritten
to canonical = UUIDv3(felisAuthNS, tag+":"+nativeID).
- validateAuthSources rejects unknown keys, duplicate tags, and
scheme-less URLs — a malformed nano config fails loud at load.
- Reuses api.HasJoinedHandler with a stub Repo (no blacklist backend);
a rejected login is a 204, matching the vanilla sessionserver.
- nano.go binds the -listen flag and ignores [server] listen in config.
Verified on WSL (go1.26.4): go build/vet/test ./... green; a runtime
smoke against the template config returns 204 on a miss and logs
"Mojang + 0 third-party source(s)"; a duplicate-tag config exits
non-zero citing "unique".
Step 2 of Felis-nano: a [[auth_source]] array-of-tables (tag + full hasJoined
url, config order = priority) supplies the multiplexer's third-party Yggdrasil
roots; cmd/felis prepends Mojang as the sole code-owned identity anchor and
wires them into API.AuthSources. With no sources configured the endpoint stays
inert (204s), unchanged from step 1.
The config deliberately has no identity/trusted field: Mojang is the only source
whose self-asserted UUIDs are trusted verbatim, so no misconfiguration can
reopen the impersonation hole the per-source UUID rewrite closes. An identity=
key is an unknown key and Load rejects it. Validate adds two fail-fast guards:
unique tags (namespace collision) and a scheme-qualified url (else the source is
silently dead, never validating any login).
Round-2 mutation audit of the backup/restore data-safety surface (7 fail-open gates pinned, 1 coverage gap found and closed by the owner-gate test in 85b8a92). Reverts the Pending section and indexes c67a4d3 + 85b8a92 into the committed ledger.
handleRestoreBackup's owner-or-admin gate was not pinned by any test: the former-owner gate backstopped every non-owner case the suite exercised, so a broken owner gate would not redden. Add the mirror of the former-owner test — a released former owner (still the backup's former_owner, no longer the current owner) must get 403 — the sole subtest that fails when the owner gate is disabled. Found by the round-2 backup/restore mutation audit; production code unchanged.
The operational break-glass ops (Owner provision, OP create, halt, Sync backup) are built and oracle-verified. The fourth §B4 line item — the tarS3 archive backend — is recorded as a deliberate deferral, not a silent gap: config.Load fail-closes store=tarS3 (frozen by TestLoadRejectsUnimplementedArchiveStore), tarLocal is the tested baseline every backup/restore path uses today, and offsite/cross-cluster DR is opt-in future work. Supersedes the "S3 remains open" note in the phase-2b peer doc.
Break each load-bearing safety gate the change ledger names as "unit-tested" and
confirm the specific test turns red — passing proves GREEN, not that the test would
catch a regression. All 18 fail-open crown-jewel gates across the subsystems (auto-update
pin/no-downgrade/no-prerelease/window, passkey clone-refuse, modpack CAS + Trivy scan-gate,
cfsetup fail-closed + NodePort fence conn-count, SSE cap, OTP atomic reserve, idle stop,
startup/readiness timeout, /readyz deps, naming reservation, service-token login-pod-only)
are mutation-proven to pin behaviour. Every documented "unit-tested" claim is reconciled
one-for-one; the fence conn-count gate, previously mis-classified as integration-only, is
corrected and verified. No code changed — read-and-verify only.
A completeness re-check of the ledger backfill (in-scope pre-fad48ff
feat/fix/refactor commits vs SHAs actually cited in detail docs, excluding
the auto-generated ledger table) surfaced one backend functional commit with
no detail-doc home: 346ec68 refactor(deploy) — an 868-insertion rework of the
cloudflare-edge TUI walkthrough plus a tested cfsetup integration-runner path.
The "improved cloudflare walkthrough" subject undersold a behaviour change, so
it is folded into the existing cloudflare-tunnel-access-edge detail doc and its
INDEX row rather than left orphaned. Backend detail-doc coverage of the
pre-ledger history is now complete (0 backend orphans).
Retroactively author 15 grouped detail docs covering the backend
functional (feat/fix) commits made before the change ledger was
established (fad48ff), closing the ledger's detail-doc axis for the
pre-convention history. Each doc groups a feature's constituent commits,
lists their SHAs with subjects, and carries a backfill note stating it
was reconstructed from git history on 2026-07-07 and not independently
re-verified (current tree green at 9911b8c).
Add a Detail docs section to INDEX.md linking every detail doc (the 6
existing + 15 backfill) to the commit(s) it covers, so a doc is findable
from the index without a column on the auto-generated ledger table. Catch
the table up with the missing 9911b8c row.
Scope: backend (Go/Java/K8s) only, per the ledger's stated convention
that frontend/panel commits are the collaborator's UI work; non-functional
commits (docs/style/chore/refactor) keep their table row without a
dedicated detail doc.
Adds a break-glass console operation that snapshots a stopped world by
calling the felis-api internal face while the API is alive, rather than
rendering the backup Job locally: the Job needs felis-api deployment
coordinates the console does not hold.
The peer resolves the felis-api-internal ClusterIP Service + service
token from the control namespace, POSTs the internal backup endpoint
with the operator os_user for audit attribution, and maps 409/503/404
to friendly outcome cards. Core decision logic lives in backupnow.go
(unit-tested against a fake client + httptest); tui_backupnow.go is the
untested bubbletea glue mirroring tui_halt.go.
Detail doc + ledger row for 2ba9948: the separate felis-api-internal ClusterIP
Service that gives the login pod (and the break-glass console) a routable 8081.
The login limbo pod dials FELIS_API_BASE_URL = felis-api.<ns>.svc:8081 (the
internal face, service-token auth) to mint bind codes and poll link status, but
the only Service named felis-api is the external NodePort face and declares only
port 443. A Service answers only on its declared ports, so felis-api:8081 had no
backend and every login-pod internal call silently failed to connect.
Render a separate ClusterIP Service felis-api-internal for port 8081 and repoint
InternalAPIBaseURL at it. A second port on the NodePort Service is not an option:
Type=NodePort allocates a node port for every declared port with no per-port
opt-out, so it would publish the no-Zero-Trust internal face on every node's
external IP. A distinct ClusterIP Service keeps 8081 in-cluster only, reachable
by the login pod via DNS and by the on-node break-glass console via the
ClusterIP (exported as APIInternalServiceName / APIInternalPort).
Manifest-level fix; the live packet path is pending real-cluster verification.
Detail doc + ledger row for f2fc57c: the internal (service-token) twin of the
on-demand world backup endpoint that the break-glass console peer will call.
Add POST /api/v1/internal/servers/{name}/backup so the on-node break-glass
console can snapshot a stopped world while felis-api is alive. It goes through
the API (not direct-to-CRD like halt) because rendering the backup Job needs
deployment coordinates (FELIS_IMAGE, FELIS_BACKUP_PVC) only felis-api holds.
Service-token auth (no Principal); the middleware IS the authorization, since
the operator already has root on the node. Refactor the RWO stopped-gate,
optional-Backuper 503, async hand-off and audit+202 into a shared enqueueBackup
tail so the external (owner/admin) and internal (break-glass) faces cannot
diverge on the security-critical stopped-gate. The internal audit is attributed
to break-glass/internal so a console-initiated backup is distinguishable from an
owner self-service one.
Add POST /api/v1/servers/{name}/backup: an owner or admin snapshots a
stopped server's world into the archive store on demand, recorded as a
first-class world_backups row (reason `manual`) — restorable by the
existing restore path and expired by the reaper's retention pass, so it
never leaks as an orphan archive. This is the break-glass "Sync" op,
resolved as immediate/on-demand backup.
felis-api cannot archive in-process (the world PVC is RWO, held by the
operator StatefulSet), so the work hands off to a one-shot Kubernetes Job
(new internal/backupjob) that mounts the world PVC read-only and the
backup PVC read-write, plus the felis config Secret so it self-records
its row atomically like the reaper. The Pod mirrors restore's weak-SA
isolation (SA token un-mounted, non-root, read-only rootfs, drop ALL);
the one reviewed departure is that config-Secret mount, frozen by
jobspec_test.go. Handler answers 202 backing_up; gated on the server
being Stopped (RWO world PVC), owner-or-admin, and FELIS_IMAGE +
FELIS_BACKUP_PVC being wired (else 503 backup_unavailable).
Each request mints a unique Job name (backup-<server>-<rand>) so a repeat
on-demand backup produces a fresh archive rather than colliding with a
just-finished Job still inside its TTL window and silently no-op'ing the
retry.
Add docs/changes/ — a durable, in-repo map of every functional change and
the commit that records it, independent of git log. INDEX.md carries the
convention (each functional change gets a dated detail doc plus a ledger
row) and the full oldest-first ledger, regenerable losslessly from git.
Seed detail docs for the two changes just landed: the break-glass halt op
(c2ee21a) and the /felis migrate command (c1aa38b).
Add the in-game /felis migrate command that a player runs to open an
account migration, the entry point for handing their owned servers to
another account (§B3 inherit, scenario A). The command posts the player's
Mojang-verified UUID to the existing handleMigrateStart backend, which puts
the account into migrate mode; the player then finishes on the web console
(prove identity, name the receiving account, redeem a one-time code).
Mirrors the existing /felis claim path: requires a real player past login
limbo, acts on the caller's account rather than the current server, expects
201 Created affirming started=true (a 201 without it is a contract breach,
not a refusal), and maps the handler refusals (404 not_linked, 409
account_retired) to player-facing guidance. On success it points the player
at https://console.<root_domain>, derived from config, never hardcoded.
Compile-verified against velocity-api:3.3.0-SNAPSHOT via the podman gradle
toolchain. Closes the code-only gap named in handlers_account_migrate.go.
Add a root-gated "Halt a running server" operation to the break-glass
console. The operator picks a server from the live fleet and the console
flips that MinecraftServer CRD's spec.desiredState to Stopped via a
spec-only merge patch, disjoint from the operator's status writes, so it
cannot race or clobber reconciliation. It is the panel-independent
emergency stop for when the box still has root plus a kubeconfig.
System servers (login/lobby) are allowed but flagged: a system tag in the
picker and an explicit WARNING in the post-exit summary, since halting
login takes the shared auth front door down with no fallback. Audit is
best-effort so a halt still works with the audit sink down. Already-stopped
is a distinct no-op. The core (halt.go) is unit-tested against a real
controller-runtime fake client that applies the patch.
Old account runs /felis migrate in-game to open a migration, proves control via a
fresh web step-up (passkey forced when enrolled, else email-OTP), names the target
and mints a one-time code. The target redeems it while authenticated AS that target:
in one transaction the source's owned servers re-point to the target and the source
is retired (sessions revoked, disabled, soft-deleted), which also spends the code so
it cannot be replayed. Only server ownership moves; the mc_uuid link and web
credentials stay with the source, so migrate is not a credential-theft primitive.
- 0015 migration: account_migrations state machine (initiated -> confirmed ->
code_issued -> redeemed), one live migration per source
- Repo/PGRepo: Start/ForSource/Confirm/IssueCode/Redeem
- 8 routes (1 internal /felis side, 7 web) with openapi parity
- passkey step-up runs the same clone-signal (sign-count) check as the login door
- code bound to the named target at issue and at redeem
Quota is grandfathered at redeem: no per-target quota re-check when servers move.
Previously /readyz only verified Repo != nil && Cluster != nil — a
process-liveness check, not a dependency-health check. The spec
requires the readyz probe to verify DB, K8s API, and CRD informer
are live before declaring the pod ready.
- Repo interface gains Ping(context.Context) error
- Cluster interface gains Ping(context.Context) error
- PGRepo.Ping delegates to sql.DB.PingContext
- K8sCluster.Ping lists MinecraftServer CRDs (Limit=1) in the
configured namespace, exercising both the API and CRD informer
- handleReadyz iterates ping checks; any failure returns 503 with
the failing dependency name in the error message
- fakeRepo and fakeCluster gain configurable pingErr for hermetic
test coverage of the failure paths
New test: TestReadyzPingsDependencies verifies 200 when healthy,
503 when DB or K8s API is down.
Previously a server whose pod was ready but RCON probe kept failing
would stay in Starting phase forever. The CRD defines TimeoutSeconds
and ReadinessTimeoutSeconds but the reconciler never checked them.
- PodNotReady path: if the pod stays not-ready past timeoutSeconds
(default 300s), transition to Failed
- RCON unreachable path: if RCON stays unreachable past
readinessTimeoutSeconds (default 300s), transition to Failed
- Helper methods startupTimedOut/readinessTimedOut compare
StartRequestedAt against the respective timeout, falling back to
300s defaults when unset
- 2 new tests: ReadinessTimeoutConvertsToFailed, StartupTimeoutConvertsToFailed
Fixes the scenario where a broken backend (bad jar, crash-looping
process) would permanently occupy a Starting server slot.
flyemoji
Lemon-miaow