fix(auth): 公开登录门按来源限速并设全站发信上限,冷却表定期清理
This commit is contained in:
26 files changed
+1141
-64
No files matched your search
+50
-12
@@ -147,6 +147,19 @@ components:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
RateLimited:
|
||||
description: >
|
||||
This client address called the public sign-in doors faster than the per-address
|
||||
limit allows (code rate_limited); Retry-After gives the seconds until the next
|
||||
call is admitted. The address is the visitor header the install's edge writes
|
||||
([auth] client_ip_header: CF-Connecting-IP behind the Cloudflare tunnel), else
|
||||
the TCP peer; IPv6 clients share one limit per /64.
|
||||
headers:
|
||||
Retry-After:
|
||||
schema: { type: integer }
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
AccessResult:
|
||||
description: The structured access mutation succeeded; the raw RCON reply is in output.
|
||||
content:
|
||||
@@ -1875,8 +1888,8 @@ paths:
|
||||
array. It never reveals staffness: methods are computed identically for every
|
||||
resolved account (no role branch), so a staff and a player address in the same
|
||||
credential state return byte-identical bodies. passkey is offered only when a
|
||||
verifier is wired. Sends no mail and mutates nothing; not rate-limited at the app
|
||||
layer (volumetric abuse is bounded at the edge). Gated on local_auth_enabled.
|
||||
verifier is wired. Sends no mail and mutates nothing; bounded by the per-address
|
||||
sign-in rate limit (429 rate_limited). Gated on local_auth_enabled.
|
||||
x-felis-face: [external]
|
||||
x-felis-tier: public
|
||||
security: []
|
||||
@@ -1918,6 +1931,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/api/v1/auth/passkey/login/begin:
|
||||
post:
|
||||
@@ -1973,7 +1988,9 @@ paths:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
description: A passkey login for this recipient was started too recently (otp_resend_cooldown).
|
||||
description: >-
|
||||
A passkey login for this recipient was started too recently (otp_resend_cooldown);
|
||||
or this client address called the sign-in doors too often (rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
@@ -2045,6 +2062,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
'503':
|
||||
description: No passkey verifier is wired on this deployment (passkey_unavailable).
|
||||
content:
|
||||
@@ -2066,9 +2085,9 @@ paths:
|
||||
userHandle inside the signed assertion at finish. The challenge cannot be
|
||||
user-keyed, so it is stashed under login_id in a non-user-keyed store and echoed
|
||||
back at finish. Mounted Public and gated on local_auth_enabled. There is no
|
||||
recipient or principal to key a per-caller cooldown on (that volumetric limiting
|
||||
is delegated to the edge), so the server-side brake is a hard global cap on live
|
||||
challenges (429 too_many_challenges). Inert for a credential until its owner
|
||||
recipient or principal to key a per-caller cooldown on, so one client is bounded
|
||||
by the per-address sign-in rate limit (429 rate_limited) and the table by a hard
|
||||
global cap on live challenges (429 too_many_challenges). Inert for a credential until its owner
|
||||
enrolls a resident passkey; email-OTP and username-first passkey remain the
|
||||
fallbacks, so no authenticator is ever locked out.
|
||||
x-felis-face: [external]
|
||||
@@ -2114,8 +2133,9 @@ paths:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
description: >-
|
||||
Too many discoverable logins are in flight server-wide; the global cap is hit
|
||||
(too_many_challenges). No per-recipient signal is leaked — the cap is global.
|
||||
Too many discoverable logins are in flight server-wide (too_many_challenges;
|
||||
the cap is global, so no per-recipient signal leaks); or this client address
|
||||
called the sign-in doors too often (rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
@@ -2192,6 +2212,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
'503':
|
||||
description: No passkey verifier is wired on this deployment (passkey_unavailable).
|
||||
content:
|
||||
@@ -2253,7 +2275,10 @@ paths:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
description: A code for this recipient was requested too recently (otp_resend_cooldown).
|
||||
description: >-
|
||||
A code for this recipient was requested too recently (otp_resend_cooldown);
|
||||
or this client address called the sign-in doors too often (rate_limited, with Retry-After);
|
||||
or the install-wide mail budget is spent (mail_rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
@@ -2318,6 +2343,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/api/v1/auth/op-login/start:
|
||||
post:
|
||||
@@ -2373,7 +2400,10 @@ paths:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
description: A code for this recipient was requested too recently (otp_resend_cooldown).
|
||||
description: >-
|
||||
A code for this recipient was requested too recently (otp_resend_cooldown);
|
||||
or this client address called the sign-in doors too often (rate_limited, with Retry-After);
|
||||
or the install-wide mail budget is spent (mail_rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
@@ -2467,6 +2497,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/api/v1/auth/setup/redeem:
|
||||
post:
|
||||
@@ -2524,6 +2556,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/api/v1/auth/setup/status:
|
||||
get:
|
||||
@@ -2640,6 +2674,8 @@ paths:
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/api/v1/me:
|
||||
get:
|
||||
@@ -3778,7 +3814,8 @@ paths:
|
||||
description: >-
|
||||
Resend requested before the cooldown elapsed (otp_resend_cooldown); or the
|
||||
account spent its daily wrong-code budget (otp_account_locked, with
|
||||
Retry-After).
|
||||
Retry-After); or the install-wide mail budget is spent
|
||||
(mail_rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
@@ -4091,7 +4128,8 @@ paths:
|
||||
description: >-
|
||||
Resend requested before the cooldown elapsed (otp_resend_cooldown), or the
|
||||
account's daily wrong-code budget is spent (otp_account_locked, with
|
||||
Retry-After).
|
||||
Retry-After); or the install-wide mail budget is spent
|
||||
(mail_rate_limited, with Retry-After).
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: '#/components/schemas/Error' }
|
||||
|
||||
+79
-3
@@ -931,7 +931,9 @@ The series come from two processes:
|
||||
`felis_start_duration_seconds` (no Service; scrape pod-scoped, e.g. a
|
||||
PodMonitor targeting port `metrics`).
|
||||
- `felis-api` internal face `:8081/metrics` (Service `felis-api-internal`) —
|
||||
`felis_image_build_failures_total`. Unauthenticated like the probes;
|
||||
`felis_image_build_failures_total`, and the sign-in series of §17
|
||||
(`felis_mail_total`, `felis_rate_limited_total`,
|
||||
`felis_auth_otp_lockouts_total`). Unauthenticated like the probes;
|
||||
ClusterIP-only, and the external face never serves it.
|
||||
- `felis_reaper_worlds_deleted_total` is produced inside the one-shot reaper
|
||||
CronJob, which exits long before any scrape interval — without a pushgateway
|
||||
@@ -941,8 +943,10 @@ The series come from two processes:
|
||||
### Alert rules
|
||||
|
||||
`deploy/alerts/` ships ready-made rules: build failures, slow starts, node
|
||||
disk/memory thresholds, the kubelet `DiskPressure` condition, and control-plane
|
||||
database backup freshness (§16; needs node-exporter's textfile collector).
|
||||
disk/memory thresholds, the kubelet `DiskPressure` condition, control-plane
|
||||
database backup freshness (§16; needs node-exporter's textfile collector), and
|
||||
sign-in abuse: the mail budget, relay failures, throttled floods and account
|
||||
code locks (§17).
|
||||
|
||||
- Plain Prometheus: add `felis-alerts.yaml` to `rule_files`. Check and unit-test
|
||||
it standalone with `promtool check rules felis-alerts.yaml` and
|
||||
@@ -1167,6 +1171,75 @@ Skips the pre-migration snapshot (`migrate up -no-backup`). The installer warns
|
||||
loudly when it is set. Use it only when the snapshot cannot work and you have
|
||||
another backup, e.g. an external database newer than the host's `pg_dump`.
|
||||
|
||||
## 17. Sign-in refused with 429, mail budget, account code locks
|
||||
|
||||
The public sign-in doors (`/api/v1/auth/*` except logout and the op-login
|
||||
status poll) have three limits of their own. Each answers 429 with a
|
||||
`Retry-After` header and a distinct error code.
|
||||
|
||||
### `rate_limited`: one address called the doors too often
|
||||
|
||||
Each client address gets 20 calls at once, refilled at 20 a minute, shared
|
||||
across every door. A person signing in makes three or four calls, so this only
|
||||
bites scripts. IPv6 clients share one limit per /64. Refusals count in
|
||||
`felis_rate_limited_total{scope="auth_door"}`; `FelisSignInFlood` fires when
|
||||
more than 10 a minute are refused for 10 minutes.
|
||||
|
||||
The address comes from `[auth] client_ip_header`:
|
||||
|
||||
- Behind the Cloudflare tunnel it is `CF-Connecting-IP`. The edge setup writes
|
||||
it, and an install with an `access_jwt_aud` implies it. The header is
|
||||
trustworthy there because the same setup fences the panel NodePort to
|
||||
loopback, so every request reaching the API came through cloudflared.
|
||||
- Behind your own reverse proxy it is `X-Forwarded-For` (the rightmost entry,
|
||||
the one your proxy appended). Firewall the NodePort so only the proxy reaches
|
||||
it, or a direct caller can write any address it likes.
|
||||
- Unset, the TCP peer is used. Behind any proxy every visitor then shares the
|
||||
proxy's address and one limit, so **everyone gets `rate_limited` at once**.
|
||||
The `felis api` log says at start which it keys on (`sign-in rate limit keys
|
||||
on ...`). Set the header in `/etc/felis/felis.toml` and
|
||||
`/etc/felis/felis.pod.toml`, then `felis converge`.
|
||||
|
||||
### `mail_rate_limited`: the install-wide mail budget is spent
|
||||
|
||||
Every code and notice the API mails spends one token of a single budget,
|
||||
`[smtp] max_per_hour` (default 120; a quarter of it may go at once), so a flood
|
||||
cannot burn the relay's quota and get the sending account suspended. While it
|
||||
is spent, every address gets the same 429 and nothing reaches the relay.
|
||||
`felis_mail_total{result="throttled"}` counts refusals and
|
||||
`FelisMailBudgetExhausted` fires on the first one. Look at
|
||||
`felis_rate_limited_total` first: a flood shows there. If sign-ins are real,
|
||||
raise `max_per_hour` to what your relay allows.
|
||||
|
||||
`FelisMailDeliveryFailing` is the other half: the relay itself refused mail
|
||||
(`felis_mail_total{result="failed"}`, 502 `mail_undeliverable` to the caller).
|
||||
The relay's reason is in the `felis-api` log.
|
||||
|
||||
### `otp_account_locked`: ten wrong codes in 24 hours
|
||||
|
||||
Ten wrong email codes for one account within 24 hours, counted across every
|
||||
code it was sent, lock that account's email-code sign-in until 24 hours after
|
||||
the first miss. The public doors answer a locked account exactly like a wrong
|
||||
code, and the owner gets one mail saying so. Signed-in doors (email
|
||||
verification, migration step-up) answer 429 `otp_account_locked`. Passkey
|
||||
sign-in keeps working. Each lock is audited as `auth.otp.locked` and counted in
|
||||
`felis_auth_otp_lockouts_total{purpose}` (`FelisOTPAccountLocked`).
|
||||
|
||||
To lift a lock early once you have confirmed the owner locked themselves out:
|
||||
|
||||
```sh
|
||||
sudo -u postgres psql felis -c \
|
||||
"DELETE FROM otp_failure_windows WHERE user_id = (SELECT id FROM users WHERE username = '<name>');"
|
||||
```
|
||||
|
||||
### Optional: a Cloudflare rate limiting rule in front
|
||||
|
||||
The limits above live in the API, so they hold on any edge. Behind Cloudflare
|
||||
you can also stop floods before they reach the tunnel: Security → WAF → Rate
|
||||
limiting rules, match URI Path starts with `/api/v1/auth/` on the console and
|
||||
op.console hostnames, count by IP, 30 requests per 10 seconds, action Block
|
||||
for 10 seconds (the Free plan's limits).
|
||||
|
||||
---
|
||||
|
||||
## Quick reference: symptom → section
|
||||
@@ -1198,3 +1271,6 @@ another backup, e.g. an external database newer than the host's `pg_dump`.
|
||||
| `pre-migration backup failed, nothing applied` during an upgrade | §16 |
|
||||
| Undo a mistaken change / restore the control-plane database | §16 |
|
||||
| Host lost: rebuild from a database bundle | §16 |
|
||||
| Sign-in 429 `rate_limited` for everyone at once | §17 |
|
||||
| 429 `mail_rate_limited` / `FelisMailBudgetExhausted` | §17 |
|
||||
| Right code refused; `otp_account_locked` / `FelisOTPAccountLocked` | §17 |
|
||||
Reference in new issue
Block a user