feat(crd): 数值字段加上下限校验,rcon.port 用 CEL 限定默认端口,文档写明 v1beta1 演进与多节点前提
This commit is contained in:
8 files changed
+186
-13
No files matched your search
@@ -101,6 +101,8 @@ spec:
|
||||
EmptySecondsBeforeStop is how long the server may sit empty before the
|
||||
operator scales it down.
|
||||
format: int32
|
||||
maximum: 604800
|
||||
minimum: 0
|
||||
type: integer
|
||||
type: object
|
||||
image:
|
||||
@@ -127,6 +129,8 @@ spec:
|
||||
description: TerminationGracePeriodSeconds is the pod grace period
|
||||
(default 300).
|
||||
format: int64
|
||||
maximum: 3600
|
||||
minimum: 0
|
||||
type: integer
|
||||
type: object
|
||||
motd:
|
||||
@@ -159,6 +163,8 @@ spec:
|
||||
port:
|
||||
description: Port is the RCON TCP port (default 25575).
|
||||
format: int32
|
||||
maximum: 65535
|
||||
minimum: 0
|
||||
type: integer
|
||||
secretRef:
|
||||
description: SecretRef points at the Secret holding the RCON password.
|
||||
@@ -172,6 +178,10 @@ spec:
|
||||
- name
|
||||
type: object
|
||||
type: object
|
||||
x-kubernetes-validations:
|
||||
- message: the allow-rcon NetworkPolicy admits only port 25575; leave
|
||||
port unset
|
||||
rule: '!has(self.port) || self.port == 0 || self.port == 25575'
|
||||
reaperExempt:
|
||||
description: ReaperExempt opts this server out of the world reaper
|
||||
entirely (spec §18).
|
||||
@@ -250,16 +260,22 @@ spec:
|
||||
(notably LOOHP/Limbo) where the felis-limbo plugin reports true
|
||||
readiness only after the first server tick.
|
||||
format: int32
|
||||
maximum: 65535
|
||||
minimum: 0
|
||||
type: integer
|
||||
readinessTimeoutSeconds:
|
||||
description: ReadinessTimeoutSeconds is the budget for the first
|
||||
successful RCON probe.
|
||||
format: int32
|
||||
maximum: 86400
|
||||
minimum: 0
|
||||
type: integer
|
||||
timeoutSeconds:
|
||||
description: TimeoutSeconds is the overall budget before the server
|
||||
is marked Failed.
|
||||
format: int32
|
||||
maximum: 86400
|
||||
minimum: 0
|
||||
type: integer
|
||||
type: object
|
||||
storage:
|
||||
|
||||
+34
-1
@@ -44,7 +44,9 @@ node's local-path storage, so a game server's pod is pinned to the node that fir
|
||||
scheduled it and cannot move when that node fails; the operator and felis-api each run
|
||||
as a single replica without leader election, so an upgrade or a node restart pauses
|
||||
wakes and stops until their pod is back. Joining k3s agents to the cluster is untested
|
||||
and gains no failover.
|
||||
and gains no failover. A multi-node shape would need, at least, storage that can follow a
|
||||
pod to another node and leader election in felis-operator (controller-runtime's
|
||||
`LeaderElection`) so a second replica can stand by.
|
||||
|
||||
## 2. Sizing
|
||||
|
||||
@@ -261,6 +263,37 @@ sudo systemctl start postgresql
|
||||
sudo k3s kubectl -n felis scale deploy/felis-api deploy/felis-operator --replicas=1
|
||||
```
|
||||
|
||||
### The MinecraftServer CRD [VM-VERIFIED]
|
||||
|
||||
Every rerun applies the CRD embedded in the `felis` binary (`felis bootstrap-assets crd`).
|
||||
It serves and stores the single version `v1alpha1`, and the apiserver refuses values the
|
||||
operator cannot act on:
|
||||
|
||||
| Field | Accepted |
|
||||
|---|---|
|
||||
| `spec.rcon.port` | unset, `0` or `25575`: the allow-rcon NetworkPolicy opens only 25575, so any other port leaves the server unprobeable |
|
||||
| `spec.startup.timeoutSeconds`, `readinessTimeoutSeconds` | 0 – 86400 |
|
||||
| `spec.startup.healthHTTPPort` | 0 – 65535 |
|
||||
| `spec.lifecycle.terminationGracePeriodSeconds` | 0 – 3600 |
|
||||
| `spec.idle.emptySecondsBeforeStop` | 0 – 604800 (the panel caps it at 86400) |
|
||||
|
||||
`0` means the operator's default throughout. An object stored before these rules keeps an
|
||||
out-of-range value until someone edits that field (CRD validation ratcheting). The operator
|
||||
reads a negative value as its default and an oversized one as written, so fix such a
|
||||
value by hand: `kubectl -n minecraft edit minecraftserver <name>`.
|
||||
|
||||
**Moving to `v1beta1` (planned, not built).** The first breaking change to the spec ships as a new
|
||||
version, in this order, each step one release:
|
||||
|
||||
1. The CRD serves `v1alpha1` and `v1beta1`, storage stays `v1alpha1`. While the two
|
||||
schemas carry the same fields, `conversion.strategy: None` suffices; a renamed or
|
||||
reshaped field needs a conversion webhook, which felis-operator would serve.
|
||||
2. Storage moves to `v1beta1`. The installer rewrites every object so etcd holds the new
|
||||
version (`kubectl get minecraftservers -A -o json | kubectl replace -f -`), then sets
|
||||
`status.storedVersions` of the CRD to `["v1beta1"]`.
|
||||
3. A later release stops serving `v1alpha1`. Felis itself reads through one Go type at a
|
||||
time, so the operator and felis-api switch in the release that moves storage.
|
||||
|
||||
## 5. Disaster recovery
|
||||
|
||||
The procedures are in §16: what a database bundle holds, restoring one on the same host,
|
||||
|
||||
@@ -140,8 +140,9 @@ message is the verbatim dial error:
|
||||
backend's `rcon.password`. Reconcile the two. [GO-TESTED that this maps to
|
||||
`RconNotReachable`.]
|
||||
- `connection refused` / `i/o timeout` → the backend has not opened the RCON
|
||||
port yet, RCON is disabled in `server.properties`, or `spec.rcon.port`
|
||||
(default 25575) is wrong. [INTEGRATION-ONLY for the live handshake.]
|
||||
port yet, RCON is disabled in `server.properties`, or the image listens on a
|
||||
port other than 25575 (the CRD accepts only that one for `spec.rcon.port`,
|
||||
operations.md §4). [INTEGRATION-ONLY for the live handshake.]
|
||||
|
||||
The per-probe timeout is a fixed 5s in code (`prober.go:45`, shortened further if
|
||||
the reconcile context has a nearer deadline). It is **not** derived from
|
||||
|
||||
@@ -19,11 +19,13 @@ require (
|
||||
k8s.io/api v0.31.3
|
||||
k8s.io/apimachinery v0.31.3
|
||||
k8s.io/client-go v0.31.0
|
||||
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340
|
||||
sigs.k8s.io/controller-runtime v0.19.3
|
||||
sigs.k8s.io/yaml v1.4.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a // indirect
|
||||
github.com/atotto/clipboard v0.1.4 // indirect
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
||||
github.com/beorn7/perks v1.0.1 // indirect
|
||||
@@ -108,7 +110,6 @@ require (
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
k8s.io/apiextensions-apiserver v0.31.0 // indirect
|
||||
k8s.io/klog/v2 v2.130.1 // indirect
|
||||
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
|
||||
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
|
||||
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
|
||||
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
|
||||
|
||||
@@ -2,6 +2,8 @@ github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk
|
||||
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
|
||||
github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ=
|
||||
github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE=
|
||||
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a h1:idn718Q4B6AGu/h5Sxe66HYVdqdGu2l9Iebqhi/AEoA=
|
||||
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
|
||||
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
|
||||
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
package v1alpha1_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"k8s.io/kube-openapi/pkg/validation/spec"
|
||||
"k8s.io/kube-openapi/pkg/validation/strfmt"
|
||||
"k8s.io/kube-openapi/pkg/validation/validate"
|
||||
"sigs.k8s.io/yaml"
|
||||
)
|
||||
|
||||
// crdSchema loads the openAPIV3Schema the cluster enforces, from the same YAML
|
||||
// `felis bootstrap-assets crd` applies. The x-kubernetes-validations (CEL) rules
|
||||
// ride along as extensions and are checked live with a server-side dry run.
|
||||
func crdSchema(t *testing.T) *spec.Schema {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("../../../../deploy/crd/felis.lolicon.best_minecraftservers.yaml")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var crd struct {
|
||||
Spec struct {
|
||||
Versions []struct {
|
||||
Schema struct {
|
||||
OpenAPIV3Schema json.RawMessage `json:"openAPIV3Schema"`
|
||||
} `json:"schema"`
|
||||
} `json:"versions"`
|
||||
} `json:"spec"`
|
||||
}
|
||||
if err := yaml.Unmarshal(raw, &crd); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(crd.Spec.Versions) != 1 {
|
||||
t.Fatalf("versions = %d, want the single v1alpha1", len(crd.Spec.Versions))
|
||||
}
|
||||
var s spec.Schema
|
||||
if err := json.Unmarshal(crd.Spec.Versions[0].Schema.OpenAPIV3Schema, &s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return &s
|
||||
}
|
||||
|
||||
// server is a minimal valid MinecraftServer with one extra spec section.
|
||||
func server(section string, body map[string]any) map[string]any {
|
||||
sp := map[string]any{"image": "paper", "subdomain": "survival"}
|
||||
if section != "" {
|
||||
sp[section] = body
|
||||
}
|
||||
return map[string]any{
|
||||
"apiVersion": "felis.lolicon.best/v1alpha1",
|
||||
"kind": "MinecraftServer",
|
||||
"metadata": map[string]any{"name": "survival"},
|
||||
"spec": sp,
|
||||
}
|
||||
}
|
||||
|
||||
func TestCRDBoundsNumericFields(t *testing.T) {
|
||||
schema := crdSchema(t)
|
||||
cases := []struct {
|
||||
name string
|
||||
section string
|
||||
body map[string]any
|
||||
ok bool
|
||||
}{
|
||||
{"defaults", "", nil, true},
|
||||
{"rcon port 0 means default", "rcon", map[string]any{"port": 0}, true},
|
||||
{"rcon port 25575", "rcon", map[string]any{"port": 25575}, true},
|
||||
{"rcon port negative", "rcon", map[string]any{"port": -1}, false},
|
||||
{"rcon port past 65535", "rcon", map[string]any{"port": 70000}, false},
|
||||
{"grace period 3600", "lifecycle", map[string]any{"terminationGracePeriodSeconds": 3600}, true},
|
||||
{"grace period negative", "lifecycle", map[string]any{"terminationGracePeriodSeconds": -1}, false},
|
||||
{"grace period past an hour", "lifecycle", map[string]any{"terminationGracePeriodSeconds": 3601}, false},
|
||||
{"startup timeout a day", "startup", map[string]any{"timeoutSeconds": 86400}, true},
|
||||
{"startup timeout negative", "startup", map[string]any{"timeoutSeconds": -5}, false},
|
||||
{"startup timeout past a day", "startup", map[string]any{"timeoutSeconds": 100000}, false},
|
||||
{"readiness timeout negative", "startup", map[string]any{"readinessTimeoutSeconds": -1}, false},
|
||||
{"readiness timeout past a day", "startup", map[string]any{"readinessTimeoutSeconds": 86401}, false},
|
||||
{"health port 8080", "startup", map[string]any{"healthHTTPPort": 8080}, true},
|
||||
{"health port past 65535", "startup", map[string]any{"healthHTTPPort": 70000}, false},
|
||||
{"health port negative", "startup", map[string]any{"healthHTTPPort": -1}, false},
|
||||
{"idle a week", "idle", map[string]any{"emptySecondsBeforeStop": 604800}, true},
|
||||
{"idle negative", "idle", map[string]any{"emptySecondsBeforeStop": -1}, false},
|
||||
{"idle past a week", "idle", map[string]any{"emptySecondsBeforeStop": 604801}, false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
// Round-trip through JSON so numbers arrive as the float64 the
|
||||
// apiserver's decoder hands the validator.
|
||||
raw, _ := json.Marshal(server(tc.section, tc.body))
|
||||
var obj any
|
||||
if err := json.Unmarshal(raw, &obj); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := validate.AgainstSchema(schema, obj, strfmt.Default)
|
||||
if tc.ok && err != nil {
|
||||
t.Fatalf("rejected a valid spec: %v", err)
|
||||
}
|
||||
if !tc.ok && err == nil {
|
||||
t.Fatal("accepted an out-of-range value")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -174,10 +174,17 @@ type MotdSpec struct {
|
||||
}
|
||||
|
||||
// RconSpec configures RCON (spec §4 spec.rcon).
|
||||
//
|
||||
// The port stays the default: the allow-rcon NetworkPolicy (internal/platform
|
||||
// netpol.go) admits the operator and felis-api on 25575 only, so any other port
|
||||
// would leave the server unprobeable and stuck in Starting.
|
||||
// +kubebuilder:validation:XValidation:rule="!has(self.port) || self.port == 0 || self.port == 25575",message="the allow-rcon NetworkPolicy admits only port 25575; leave port unset"
|
||||
type RconSpec struct {
|
||||
// Enabled must be true for readiness probing and graceful shutdown.
|
||||
Enabled bool `json:"enabled,omitempty"`
|
||||
// Port is the RCON TCP port (default 25575).
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=65535
|
||||
Port int32 `json:"port,omitempty"`
|
||||
// SecretRef points at the Secret holding the RCON password.
|
||||
SecretRef SecretKeyRef `json:"secretRef,omitempty"`
|
||||
@@ -202,14 +209,20 @@ type StorageSpec struct {
|
||||
// then the time the server has to finish its own shutdown save after SIGTERM.
|
||||
type LifecycleSpec struct {
|
||||
// TerminationGracePeriodSeconds is the pod grace period (default 300).
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=3600
|
||||
TerminationGracePeriodSeconds int64 `json:"terminationGracePeriodSeconds,omitempty"`
|
||||
}
|
||||
|
||||
// StartupSpec bounds the Starting phase (spec §5).
|
||||
type StartupSpec struct {
|
||||
// TimeoutSeconds is the overall budget before the server is marked Failed.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=86400
|
||||
TimeoutSeconds int32 `json:"timeoutSeconds,omitempty"`
|
||||
// ReadinessTimeoutSeconds is the budget for the first successful RCON probe.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=86400
|
||||
ReadinessTimeoutSeconds int32 `json:"readinessTimeoutSeconds,omitempty"`
|
||||
// HealthHTTPPort, when > 0, switches the pod readiness probe from the default
|
||||
// plain-TCP check on the game port to an HTTP GET on this container port. It
|
||||
@@ -219,6 +232,8 @@ type StartupSpec struct {
|
||||
// only after the first server tick. The operator's readiness path is otherwise
|
||||
// unchanged — with rcon disabled, passing this probe (readyReplicas >= 1) is
|
||||
// what marks the server Ready.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=65535
|
||||
HealthHTTPPort int32 `json:"healthHTTPPort,omitempty"`
|
||||
// HealthHTTPPath is the path for the HTTP readiness probe (default "/healthz"
|
||||
// when HealthHTTPPort is set).
|
||||
@@ -246,7 +261,10 @@ type IdleSpec struct {
|
||||
// AutoStopEnabled turns on idle auto-stop.
|
||||
AutoStopEnabled bool `json:"autoStopEnabled,omitempty"`
|
||||
// EmptySecondsBeforeStop is how long the server may sit empty before the
|
||||
// operator scales it down.
|
||||
// operator scales it down. The API caps what the panel sets well below the
|
||||
// schema's week.
|
||||
// +kubebuilder:validation:Minimum=0
|
||||
// +kubebuilder:validation:Maximum=604800
|
||||
EmptySecondsBeforeStop int32 `json:"emptySecondsBeforeStop,omitempty"`
|
||||
}
|
||||
|
||||
|
||||
@@ -21,14 +21,11 @@ const gamePort = operator.GamePort
|
||||
// so the policy and the server container's default RCON port are one source of
|
||||
// truth.
|
||||
//
|
||||
// LIMITATION (honestly labeled, not verifiable without a cluster): RCON is
|
||||
// per-server overridable via spec.rcon.port (internal/operator.rconPort), but this
|
||||
// is one namespace-wide policy that can open only a single port. It opens the
|
||||
// default. A server that overrides spec.rcon.port to a non-default value would have
|
||||
// its RCON port denied by this fence, so the operator's readiness prober could not
|
||||
// reach it. The supported deployment keeps the default RCON port; a per-server-port
|
||||
// deployment would need per-server NetworkPolicies, deferred until a concrete need
|
||||
// exists.
|
||||
// This is one namespace-wide policy, so it opens a single port: the default. The
|
||||
// CRD holds spec.rcon.port to unset, 0 or 25575 with a CEL rule (RconSpec in
|
||||
// internal/apis/felis/v1alpha1), so no server can move RCON behind this fence and
|
||||
// leave the operator's readiness prober outside it. Per-server ports would need
|
||||
// per-server NetworkPolicies and a relaxed rule, together.
|
||||
const rconPort = operator.DefaultRconPort
|
||||
|
||||
// serverPodSelector matches every operator-managed Minecraft server pod by the
|
||||
|
||||
Reference in new issue
Block a user