refactor(account): S2 读写一次切换 —— CK/健康/重登彻底搬到账号,登录记录退化为历史

S1 加好了列,本提交把**读与写一起切**过去。这是设计里「同一份数据的读与写必须
在同一个提交里切」的要求:先切读或先切写都会出现「写账号、读记录」的中间态,
跑起来不报错,只是默默用错数据。

## 核心改动

**判定「谁有 CK」不再借道登录记录**
- `cookie_holding_filter()` / `cookie_operable_filter()` / `cookie_holding_account_ids_query()`
  / `visible_douyu_task_accounts()` 全部改为作用于 `Account`;
- `cookie_operable_filter()` 的「在途排除」仍查 `login_tasks`(那是**某次尝试**的属性),
  用 `NOT EXISTS` 子查询 + 迁移 0042 新增的 `(account_id, attempt_status)` 索引;
- 删除死代码 `latest_cookie_login_task()`;`latest_cookie()` 改为直接读账号。

**CK 写入**
- `_write_login_success()` 写 `Account.cookie / cookie_updated_at / cookie_source`;
- `_mark_attempt_failed()` 改看**账号**有没有 CK(决定是否沿用旧的失败语义);
- 检测模式的 `task.cookie = ""` **不再清空账号的 CK**(见下「行为变更」)。

**自动化作业按账号调度**
- `ck_health`:`_select_tasks` → `_select_accounts`,`db.query(Account)`;新增
  `count_remaining()`(刻意不带 LIMIT,否则积压数恒等于批次上限);
- `ck_relogin`:同样按账号选行;结算改为「账号 + 本轮尝试记录」,
  退避进度写账号;`settle_fresh_results(batch_id)` 按 `attempt_batch_id` 配对,
  `run_ck_relogin` 把 **account_ids** 交给 runner;
- 顺带让**没有任何登录历史的账号**也能被巡检/重登(以前它们永远进不了队列)。

**登录批次**
- `LoginBatchRunner` 去掉 `relogin_task_ids`,重登与登录统一走 `account_ids`;
- 批次行复用逻辑重写:只保证「同一账号不并发两行」,不再关心「哪一行持有 CK」。
  失败行复用同时看 `attempt_status`(账号有 CK 时 `_mark_attempt_failed` 刻意不动
  `status`,只看 status 会永远命不中 → 每次失败刷一行)。

**删除护栏语义变更(用户可感知)**
- 去掉「持有 CK 的记录不可删」——CK 在账号上,删历史不再影响登录态,历史终于可清理;
- **保留**「在途尝试不可删」(批次线程正拿着那条记录)。

## 行为变更(需评审)

1. **账号检测不再清空 CK**。旧代码在 check 模式写 `task.cookie = ""`,而那时记录的
   cookie 就是账号唯一凭证 —— 一次「批量检测」会把账号登录态删掉。解耦后
   「检测账号凭据」与「CK 是否有效」本就是两件事(后者归 `ck_health` 判),
   继续清空等于凭空制造不可逆的数据丢失(重登当前成功率极低)。
2. **Cookie 页的记录单位从「登录记录」变成「账号」**,所有接口的 id 入参/出参
   变成 `account_id`;`DELETE /cookies/{id}` 清的是该账号的 CK,**必写审计**。
3. 重复检测从「同一账号多行持 CK」改为「**同一个 CK 出现在多个账号上**」——
   后者才是真风险(两个账号共用一个登录态)。用 sha256 指纹分组:
   CK 是 AES-GCM + 随机 nonce 加密,**同一明文每次密文都不同**,比密文永远比不出重复。

## 修掉一个真实并发缺陷

`_write_login_success` 原先被传入 `self.db`(runner 主会话),而它是在**工作线程**里
用 `worker_db` 执行的。`emit()` 会往 `account_events` 插一行,其 `account_id` 外键对
`accounts` 那一行加**共享锁**;这把锁加在另一个会话上,紧接着本会话的
`UPDATE accounts SET cookie=...` 就被自己人挡住 —— 实测 **50 秒后
`Lock wait timeout exceeded`,登录明明成功却写不回 CK**(改造前 UPDATE 打在
`login_tasks` 上所以没撞上)。改为传 `worker_db`:事件本就该和状态变更同事务提交,
顺带修掉跨线程共用一个 Session 的隐患。

## 验证

- 全量 pytest **368 passed / 6 skipped**;迁移链 3 passed;
- 前端 `tsc -b && vite build` 通过;
- `ruff check` / `ruff format --check` 全绿(顺带修掉 main 上遗留的 F821);
- 新增/改写的关键回归:**删掉全部登录历史后账号的 CK 与健康状态仍在**、
  **无登录历史的账号也能进巡检队列**、结算不碰 CK、删除护栏改为在途语义。

仍未做(设计里排期的后续阶段):`account_lifecycle` 状态收口(S3)、事件发射点补齐(S4)、
业务事实直发事件(S5)、账号页重构(S6)。注意 `status` 目前只有模型默认值与迁移回填,
所以 `disabled/archived` 还没有任何写入方——S3 落地时要把自动化选行的状态过滤一起补上。
This commit is contained in:
yml2213
2026-10-04 14:10:13 +08:00
parent e916a7015d
commit 00125fbfc4
26 changed files with 1380 additions and 1068 deletions
@@ -16,7 +16,7 @@ from typing import Any
from sqlalchemy import or_
from sqlalchemy.orm import Session
from ....models import Account, AutomationJob, LoginTask
from ....models import Account, AutomationJob
from ...account_events_service import EVENT_CK_INVALID, emit
from ...cookie_check_service import check_douyu_cookie
from ...douyu_service import cookie_operable_filter
@@ -60,11 +60,11 @@ def classify_result(result: dict[str, Any]) -> str:
return "error"
def build_select_query(db: Session, job: AutomationJob, now: datetime):
"""构造本轮要检测记录的查询(与取行分开,便于按方言编译做回归)。
def _candidates_query(db: Session, job: AutomationJob, now: datetime):
"""本轮「待检测账号」的完整候选集(**不带 LIMIT**,供计数复用)。
口径与人工点检测一致(``cookie_operable_filter``:持有 CK 且没有在途尝试),
再加三条:距上次检测超过作业间隔、账号允许自动管理、最久没查的优先。
口径与人工点检测一致(``cookie_operable_filter``:账号有 CK 且没有在途尝试),
再加两条:距上次检测超过作业间隔、账号允许自动管理。
排序刻意不用 ``NULLS FIRST``:那是 PostgreSQL/SQLite 语法,**MySQL 会报
1064 语法错误**(生产踩过一次)。改用 ``ck_checked_at IS NULL DESC``,
@@ -72,32 +72,52 @@ def build_select_query(db: Session, job: AutomationJob, now: datetime):
"""
cutoff = now - timedelta(seconds=max(int(job.interval_seconds or 0), 60))
return (
db.query(LoginTask)
.join(Account, LoginTask.account_id == Account.id)
db.query(Account)
.filter(cookie_operable_filter())
.filter(Account.auto_managed.is_(True))
.filter(
or_(
LoginTask.ck_checked_at.is_(None),
LoginTask.ck_checked_at < cutoff,
Account.ck_checked_at.is_(None),
Account.ck_checked_at < cutoff,
)
)
.order_by(
LoginTask.ck_checked_at.is_(None).desc(),
LoginTask.ck_checked_at.asc(),
LoginTask.id.asc(),
Account.ck_checked_at.is_(None).desc(),
Account.ck_checked_at.asc(),
Account.id.asc(),
)
.limit(max(int(job.batch_size or 0), 1))
)
def _select_tasks(db: Session, job: AutomationJob, now: datetime) -> list[LoginTask]:
def build_select_query(db: Session, job: AutomationJob, now: datetime):
"""构造本轮要检测**账号**的查询(与取行分开,便于按方言编译做回归)。
解耦后**按账号选行**(2026-10-04):CK 与 ``ck_check_*`` 都在 ``accounts`` 上,
以前是 ``db.query(LoginTask).join(Account)`` —— 那样「没有登录历史的账号」
永远进不了巡检队列,而且一个账号有多行时会被重复检测。
"""
return _candidates_query(db, job, now).limit(max(int(job.batch_size or 0), 1))
def _select_accounts(db: Session, job: AutomationJob, now: datetime) -> list[Account]:
return build_select_query(db, job, now).all()
def _detect(task: LoginTask) -> dict[str, Any]:
def count_remaining(db: Session, job: AutomationJob, now: datetime) -> int:
"""剩余待检账号数(给页面提示用)。
刻意**不用** ``build_select_query`` —— 那个带 ``LIMIT batch_size``,
取 count 只会得到批次上限,完全看不出积压。
"""
return int(
_candidates_query(db, job, now).order_by(None).with_entities(Account.id).count()
or 0
)
def _detect(account: Account) -> dict[str, Any]:
try:
return check_douyu_cookie(task.cookie or "")
return check_douyu_cookie(account.cookie or "")
except Exception as exc: # noqa: BLE001 单个账号失败不影响整轮
return {
"valid": False,
@@ -111,22 +131,22 @@ def _detect(task: LoginTask) -> dict[str, Any]:
def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
now = datetime.now(UTC)
tasks = _select_tasks(db, job, now)
if not tasks:
accounts = _select_accounts(db, job, now)
if not accounts:
return {"scanned": 0, "valid": 0, "invalid": 0, "skipped": 0}
results: dict[int, dict[str, Any]] = {}
concurrency = max(min(int(job.concurrency or 1), 16), 1)
with ThreadPoolExecutor(max_workers=concurrency) as pool:
futures = {pool.submit(_detect, task): task for task in tasks}
futures = {pool.submit(_detect, account): account for account in accounts}
for future in as_completed(futures):
task = futures[future]
results[task.id] = future.result()
account = futures[future]
results[account.id] = future.result()
# 只写检测结果三件套;cookie / status / attempt_* 一律不动。
valid = invalid = errors = 0
for index, task in enumerate(tasks, start=1):
result = results.get(task.id)
for index, account in enumerate(accounts, start=1):
result = results.get(account.id)
if result is None:
continue
verdict = classify_result(result)
@@ -136,17 +156,17 @@ def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
invalid += 1
else:
errors += 1
# error 时保留该行原来的结论:查不到不等于失效,宁可维持上一次判断。
# error 时保留该账号原来的结论:查不到不等于失效,宁可维持上一次判断。
if verdict != "error":
task.ck_check_status = verdict
task.ck_check_result = {
account.ck_check_status = verdict
account.ck_check_result = {
"fish_ball": result.get("fish_ball"),
"nickname": result.get("nickname"),
"level": result.get("level"),
"message": result.get("message", ""),
"verdict": verdict,
}
task.ck_checked_at = now
account.ck_checked_at = now
# 判定失效时记一条账号事件:`ck_check_status` / `ck_checked_at` 都是**覆盖式**
# 字段,覆盖之后就再也看不出「什么时候开始失效的」。只在真的判成 invalid 时记,
@@ -154,7 +174,7 @@ def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
if verdict == "invalid":
emit(
db,
account_id=task.account_id,
account_id=account.id,
event_type=EVENT_CK_INVALID,
title="CK 检测失效",
occurred_at=now,
@@ -165,7 +185,7 @@ def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
source="login",
# 幂等键带检测时间:同一账号**再次**失效算新事件,但同一次检测
# 重跑(例如失败重试)不会重复入库。
source_ref=f"ck_invalid:{task.id}:{int(now.timestamp())}",
source_ref=f"ck_invalid:{account.id}:{int(now.timestamp())}",
)
if index % FLUSH_EVERY == 0:
@@ -173,21 +193,8 @@ def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
db.commit()
# 剩余待检数量只用于提示,不参与调度。
remaining = (
db.query(LoginTask)
.join(Account, LoginTask.account_id == Account.id)
.filter(cookie_operable_filter())
.filter(Account.auto_managed.is_(True))
.filter(
or_(
LoginTask.ck_checked_at.is_(None),
LoginTask.ck_checked_at
< now - timedelta(seconds=max(int(job.interval_seconds or 0), 60)),
)
)
.count()
)
error_ratio = errors / len(tasks) if tasks else 0.0
remaining = count_remaining(db, job, now)
error_ratio = errors / len(accounts) if accounts else 0.0
paused = False
if errors >= ERROR_PAUSE_MIN_SCANNED and error_ratio > ERROR_PAUSE_RATIO:
# 大面积「查不到」通常是斗鱼接口抖动或被风控,继续跑只会污染更多账号。
@@ -195,13 +202,13 @@ def run_ck_health(db: Session, job: AutomationJob) -> dict[str, Any]:
job.updated_at = datetime.now(UTC)
paused = True
message = f"本轮检测 {len(tasks)} 条,剩余待检 {remaining} 条"
message = f"本轮检测 {len(accounts)} 条,剩余待检 {remaining} 条"
if errors:
message += f";非鉴权类失败 {errors} 条({error_ratio:.0%})"
if paused:
message += ";失败占比过高,已自动暂停作业"
return {
"scanned": len(tasks),
"scanned": len(accounts),
"valid": valid,
"invalid": invalid,
"skipped": errors,
@@ -24,7 +24,7 @@ from ....database import SessionLocal
from ....models import Account, AutomationJob, LoginTask, User
from ....models import ProxyConfig as ProxyConfigModel
from ...account_events_service import EVENT_RELOGIN_FAILED, emit
from ...douyu_service import LOGIN_ATTEMPT_IN_FLIGHT_STATUSES, cookie_holding_filter
from ...douyu_service import cookie_holding_filter, cookie_operable_filter
from ...login_service import LoginBatchRunner, get_relogin_limits
JOB_KEY = "ck_relogin"
@@ -82,41 +82,71 @@ def build_select_query(
max_attempts: int = 2,
system_user_id: int | None = None,
):
"""构造本轮重登候选的查询(与取行分开,便于按方言编译做回归)。"""
"""构造本轮重登候选的查询(与取行分开,便于按方言编译做回归)。
解耦后**按账号选行**(2026-10-04):CK 与 ``ck_check_status`` / ``relogin_*``
都在 ``accounts`` 上。以前是 ``db.query(LoginTask).join(Account)`` —— 那样
「没有登录历史的账号」永远进不了重登队列,而重登本来只关心账号。
「在途尝试」仍然要查 ``login_tasks``(那是某次尝试的属性),由
``cookie_operable_filter`` 的 NOT EXISTS 子查询完成,走迁移 0042 新增的
``(account_id, attempt_status)`` 索引。
"""
query = (
db.query(LoginTask)
.join(Account, LoginTask.account_id == Account.id)
.filter(LoginTask.ck_check_status == "invalid")
.filter(cookie_holding_filter())
.filter(
func.coalesce(LoginTask.attempt_status, "").notin_(
LOGIN_ATTEMPT_IN_FLIGHT_STATUSES
)
)
db.query(Account)
.filter(Account.ck_check_status == "invalid")
.filter(cookie_operable_filter())
.filter(Account.auto_managed.is_(True))
.filter(LoginTask.relogin_attempts < max_attempts)
.filter(Account.relogin_attempts < max_attempts)
.filter(
or_(
LoginTask.relogin_next_at.is_(None),
LoginTask.relogin_next_at <= _cutoff(now),
Account.relogin_next_at.is_(None),
Account.relogin_next_at <= _cutoff(now),
)
)
)
# 已判定需要人工的类型不再自动重试(极验/凭证)。空值用 coalesce 兜住历史行。
# 已判定需要人工的类型不再自动重试(极验/凭证)。空值用 coalesce 兜住历史值。
query = query.filter(
func.coalesce(LoginTask.relogin_failure_kind, "").notin_(tuple(MANUAL_KINDS))
func.coalesce(Account.relogin_failure_kind, "").notin_(
tuple(sorted(MANUAL_KINDS))
)
)
if system_user_id is not None:
pass # 预留:将来若需要限制到某个归属人,在这里加过滤
return query.order_by(
LoginTask.relogin_next_at.is_(None).desc(),
LoginTask.relogin_next_at.asc(),
LoginTask.id.asc(),
Account.relogin_next_at.is_(None).desc(),
Account.relogin_next_at.asc(),
Account.id.asc(),
).limit(max(int(job.batch_size or 0), 1))
def _load_batch_state(
session: Session, batch_id: str
) -> tuple[dict[int, tuple[Account, LoginTask]], list[int]]:
"""读回某一批次产生的尝试记录,并配对其所属账号。
按 ``attempt_batch_id`` 找 —— ``_begin_attempt`` 会给每一条被本批次接管的记录
打上批次号。所以即使记录是**复用**的旧行,也能被准确对应回本轮。
"""
tasks = (
session.query(LoginTask)
.filter(LoginTask.attempt_batch_id == batch_id)
.order_by(LoginTask.id.asc())
.all()
)
state: dict[int, tuple[Account, LoginTask]] = {}
order: list[int] = []
for task in tasks:
account = session.get(Account, task.account_id)
if account is None:
continue
state[account.id] = (account, task)
order.append(account.id)
return state, order
def settle_fresh_results(
task_ids: list[int], *, now: datetime, max_attempts: int = 2
batch_id: str, *, now: datetime, max_attempts: int = 2
) -> dict[str, int]:
"""在**新会话(新事务)**里读回批次结果并结算。
@@ -130,11 +160,11 @@ def settle_fresh_results(
"""
session = SessionLocal()
try:
rows = session.query(LoginTask).filter(LoginTask.id.in_(task_ids)).all()
fresh, order = _load_batch_state(session, batch_id)
counters = settle_results(
session,
{row.id: row for row in rows},
task_ids,
fresh,
order,
now=now,
max_attempts=max_attempts,
)
@@ -152,15 +182,14 @@ def count_pending_relogin(db: Session, *, max_attempts: int = 2) -> int:
按退避窗口过滤** —— 退避是「什么时候试」的细节,不影响「还有多少要处理」。
"""
return int(
db.query(func.count(LoginTask.id))
.join(Account, LoginTask.account_id == Account.id)
.filter(LoginTask.ck_check_status == "invalid")
db.query(func.count(Account.id))
.filter(Account.ck_check_status == "invalid")
.filter(cookie_holding_filter())
.filter(Account.auto_managed.is_(True))
.filter(LoginTask.relogin_attempts < max_attempts)
.filter(Account.relogin_attempts < max_attempts)
.filter(
func.coalesce(LoginTask.relogin_failure_kind, "").notin_(
tuple(MANUAL_KINDS)
func.coalesce(Account.relogin_failure_kind, "").notin_(
tuple(sorted(MANUAL_KINDS))
)
)
.scalar()
@@ -210,11 +239,12 @@ def run_ck_relogin(db: Session, job: AutomationJob) -> dict[str, Any]:
"message": "库里没有可用用户,无法以系统身份发起重登",
}
tasks = build_select_query(db, job, now, max_attempts=max_attempts).all()
if not tasks:
accounts = build_select_query(db, job, now, max_attempts=max_attempts).all()
if not accounts:
return {"scanned": 0, "valid": 0, "invalid": 0, "skipped": 0}
task_ids = [task.id for task in tasks]
# 调度单位是**账号**(CK 在账号上);登录记录只是这次尝试的历史。
account_ids = [account.id for account in accounts]
proxy = db.query(ProxyConfigModel).first()
_, max_total_time = get_relogin_limits()
@@ -222,7 +252,7 @@ def run_ck_relogin(db: Session, job: AutomationJob) -> dict[str, Any]:
runner_db = SessionLocal()
runner = LoginBatchRunner(
db=runner_db,
account_ids=[],
account_ids=account_ids,
created_by=system_user_id,
# 自动化要对全库生效,不受「只能操作分配给自己的账号」限制。
creator_permissions=["login:view_all", "cookie:operate"],
@@ -233,7 +263,6 @@ def run_ck_relogin(db: Session, job: AutomationJob) -> dict[str, Any]:
loop=None,
concurrency=max(min(int(job.concurrency or 3), 8), 1),
mode="relogin",
relogin_task_ids=task_ids,
)
batch_id = runner.batch_id
try:
@@ -242,7 +271,7 @@ def run_ck_relogin(db: Session, job: AutomationJob) -> dict[str, Any]:
runner_db.close()
# runner 在另一个会话里提交了结果:必须用新事务读回(见 settle_fresh_results)。
counters = settle_fresh_results(task_ids, now=now, max_attempts=max_attempts)
counters = settle_fresh_results(batch_id, now=now, max_attempts=max_attempts)
parts = [
f"本轮重登 {counters['scanned']} 条",
@@ -274,58 +303,64 @@ def run_ck_relogin(db: Session, job: AutomationJob) -> dict[str, Any]:
def settle_results(
db: Session,
fresh: dict[int, LoginTask],
task_ids: list[int],
fresh: dict[int, tuple[Account, LoginTask]],
account_ids: list[int],
*,
now: datetime,
max_attempts: int = 2,
) -> dict[str, int]:
"""批次跑完后结算:成功的清零,失败的按分型决定重试还是转人工。
解耦后 **决策看「账号 + 本轮的尝试记录」**(2026-10-04):
- 成败来自这次尝试(``LoginTask.attempt_status``);
- 退避进度写在**账号**上(``Account.relogin_*``)—— 它是账号的状态,
不该随某条历史记录一起被删掉。
单独抽出来是因为这段是 P1b 的核心决策逻辑,必须能脱离真实登录做单测。
只改 ``relogin_*`` 四个字段,**不碰 CK 本体**(CK 由 runner 在成功时替换)。
``db`` 用于写账号事件(转人工时记一条);事件是旁路,写失败不影响结算。
"""
counters = {
"scanned": len(task_ids),
"scanned": len(account_ids),
"valid": 0,
"invalid": 0,
"manual": 0,
"retry": 0,
"email_quota": 0,
}
for task_id in task_ids:
task = fresh.get(task_id)
if task is None:
for account_id in account_ids:
entry = fresh.get(account_id)
if entry is None:
continue
account, task = entry
if task.attempt_status == "success":
counters["valid"] += 1
task.relogin_attempts = 0
task.relogin_next_at = None
task.relogin_failure_kind = ""
task.relogin_last_error = ""
account.relogin_attempts = 0
account.relogin_next_at = None
account.relogin_failure_kind = ""
account.relogin_last_error = ""
continue
counters["invalid"] += 1
kind = classify_failure(task.attempt_message)
task.relogin_failure_kind = kind
task.relogin_last_error = (task.attempt_message or "")[:500]
task.relogin_attempts = int(task.relogin_attempts or 0) + 1
if kind in MANUAL_KINDS or task.relogin_attempts >= max_attempts:
account.relogin_failure_kind = kind
account.relogin_last_error = (task.attempt_message or "")[:500]
account.relogin_attempts = int(account.relogin_attempts or 0) + 1
if kind in MANUAL_KINDS or account.relogin_attempts >= max_attempts:
# 转人工:极验/凭证问题重试无意义,次数用尽也不该无限撞。
counters["manual"] += 1
task.relogin_next_at = None
account.relogin_next_at = None
# 只在「转人工」时记事件:还要退避重试的不记,否则同一账号每轮都刷一条。
emit(
db,
account_id=task.account_id,
account_id=account.id,
event_type=EVENT_RELOGIN_FAILED,
title=f"自动重登转人工({kind})",
occurred_at=now,
detail={
"kind": kind,
"attempts": task.relogin_attempts,
"attempts": account.relogin_attempts,
"error": (task.attempt_message or "")[:200],
},
source="login",
@@ -335,5 +370,5 @@ def settle_results(
if kind == "email_quota":
counters["email_quota"] += 1
counters["retry"] += 1
task.relogin_next_at = _next_retry_at(kind, task.relogin_attempts, now)
account.relogin_next_at = _next_retry_at(kind, account.relogin_attempts, now)
return counters