Files
live-hub-py/web/backend/services/douyu_runner.py
T
yml2213 00125fbfc4 refactor(account): S2 读写一次切换 —— CK/健康/重登彻底搬到账号,登录记录退化为历史
S1 加好了列,本提交把**读与写一起切**过去。这是设计里「同一份数据的读与写必须
在同一个提交里切」的要求:先切读或先切写都会出现「写账号、读记录」的中间态,
跑起来不报错,只是默默用错数据。

## 核心改动

**判定「谁有 CK」不再借道登录记录**
- `cookie_holding_filter()` / `cookie_operable_filter()` / `cookie_holding_account_ids_query()`
  / `visible_douyu_task_accounts()` 全部改为作用于 `Account`;
- `cookie_operable_filter()` 的「在途排除」仍查 `login_tasks`(那是**某次尝试**的属性),
  用 `NOT EXISTS` 子查询 + 迁移 0042 新增的 `(account_id, attempt_status)` 索引;
- 删除死代码 `latest_cookie_login_task()`;`latest_cookie()` 改为直接读账号。

**CK 写入**
- `_write_login_success()` 写 `Account.cookie / cookie_updated_at / cookie_source`;
- `_mark_attempt_failed()` 改看**账号**有没有 CK(决定是否沿用旧的失败语义);
- 检测模式的 `task.cookie = ""` **不再清空账号的 CK**(见下「行为变更」)。

**自动化作业按账号调度**
- `ck_health`:`_select_tasks` → `_select_accounts`,`db.query(Account)`;新增
  `count_remaining()`(刻意不带 LIMIT,否则积压数恒等于批次上限);
- `ck_relogin`:同样按账号选行;结算改为「账号 + 本轮尝试记录」,
  退避进度写账号;`settle_fresh_results(batch_id)` 按 `attempt_batch_id` 配对,
  `run_ck_relogin` 把 **account_ids** 交给 runner;
- 顺带让**没有任何登录历史的账号**也能被巡检/重登(以前它们永远进不了队列)。

**登录批次**
- `LoginBatchRunner` 去掉 `relogin_task_ids`,重登与登录统一走 `account_ids`;
- 批次行复用逻辑重写:只保证「同一账号不并发两行」,不再关心「哪一行持有 CK」。
  失败行复用同时看 `attempt_status`(账号有 CK 时 `_mark_attempt_failed` 刻意不动
  `status`,只看 status 会永远命不中 → 每次失败刷一行)。

**删除护栏语义变更(用户可感知)**
- 去掉「持有 CK 的记录不可删」——CK 在账号上,删历史不再影响登录态,历史终于可清理;
- **保留**「在途尝试不可删」(批次线程正拿着那条记录)。

## 行为变更(需评审)

1. **账号检测不再清空 CK**。旧代码在 check 模式写 `task.cookie = ""`,而那时记录的
   cookie 就是账号唯一凭证 —— 一次「批量检测」会把账号登录态删掉。解耦后
   「检测账号凭据」与「CK 是否有效」本就是两件事(后者归 `ck_health` 判),
   继续清空等于凭空制造不可逆的数据丢失(重登当前成功率极低)。
2. **Cookie 页的记录单位从「登录记录」变成「账号」**,所有接口的 id 入参/出参
   变成 `account_id`;`DELETE /cookies/{id}` 清的是该账号的 CK,**必写审计**。
3. 重复检测从「同一账号多行持 CK」改为「**同一个 CK 出现在多个账号上**」——
   后者才是真风险(两个账号共用一个登录态)。用 sha256 指纹分组:
   CK 是 AES-GCM + 随机 nonce 加密,**同一明文每次密文都不同**,比密文永远比不出重复。

## 修掉一个真实并发缺陷

`_write_login_success` 原先被传入 `self.db`(runner 主会话),而它是在**工作线程**里
用 `worker_db` 执行的。`emit()` 会往 `account_events` 插一行,其 `account_id` 外键对
`accounts` 那一行加**共享锁**;这把锁加在另一个会话上,紧接着本会话的
`UPDATE accounts SET cookie=...` 就被自己人挡住 —— 实测 **50 秒后
`Lock wait timeout exceeded`,登录明明成功却写不回 CK**(改造前 UPDATE 打在
`login_tasks` 上所以没撞上)。改为传 `worker_db`:事件本就该和状态变更同事务提交,
顺带修掉跨线程共用一个 Session 的隐患。

## 验证

- 全量 pytest **368 passed / 6 skipped**;迁移链 3 passed;
- 前端 `tsc -b && vite build` 通过;
- `ruff check` / `ruff format --check` 全绿(顺带修掉 main 上遗留的 F821);
- 新增/改写的关键回归:**删掉全部登录历史后账号的 CK 与健康状态仍在**、
  **无登录历史的账号也能进巡检队列**、结算不碰 CK、删除护栏改为在途语义。

仍未做(设计里排期的后续阶段):`account_lifecycle` 状态收口(S3)、事件发射点补齐(S4)、
业务事实直发事件(S5)、账号页重构(S6)。注意 `status` 目前只有模型默认值与迁移回填,
所以 `disabled/archived` 还没有任何写入方——S3 落地时要把自动化选行的状态过滤一起补上。
2026-10-04 14:10:13 +08:00

190 lines
8.3 KiB
Python

"""斗鱼活动任务批次执行器(入口聚合;功能域已拆分到 douyu_runner_*.py)。"""
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor, as_completed
from sqlalchemy.orm import joinedload
from core.douyu import DouyuActivityError
from ..database import SessionLocal
from ..models import DouyuTask
from .cookie_check_service import check_douyu_cookie
from .douyu_runner_bind import BindMixin
from .douyu_runner_core import ( # noqa: F401 (douyu_batch_registry 供 routers 重导出)
DouyuBatchRunnerCore,
douyu_batch_registry,
)
from .douyu_runner_donate import DonateMixin
from .douyu_runner_gold import GoldMixin
from .douyu_runner_goods import GoodsMixin
from .douyu_runner_manual import ManualMixin
from .douyu_runner_xpd import XpdMixin
from .douyu_service import update_account_profile_from_cookie
class DouyuBatchRunner(
DouyuBatchRunnerCore,
BindMixin,
ManualMixin,
GoldMixin,
DonateMixin,
GoodsMixin,
XpdMixin,
):
"""批量执行斗鱼活动任务(功能域 Mixin 聚合 + 批次调度)。"""
def _execute_one(self, task_id: int, config: dict, total: int):
worker_db = SessionLocal()
try:
task = (
worker_db.query(DouyuTask)
.options(joinedload(DouyuTask.account))
.filter(DouyuTask.id == task_id)
.first()
)
if not task or self._stop.is_set():
return
account = task.account
self._update_task_progress(worker_db, task, "running", "执行中")
with self._counter_lock:
self._started += 1
current = self._started
self._push_log(
"info", f"[{current}/{total}] 开始: {self._account_name(account)}"
)
# CK 与健康状态都挂在账号上(2026-10-04 CK 解耦)。
cookie = account.cookie or ""
if not cookie:
self._mark_task(worker_db, task, "failed", "账号没有可用登录 Cookie")
self._push_log(
"warning", f"[{current}] {self._account_name(account)} 无 Cookie"
)
return
cookie_check = check_douyu_cookie(cookie)
account.ck_check_status = "valid" if cookie_check["valid"] else "invalid"
account.ck_check_result = {
key: cookie_check.get(key)
for key in ("fish_ball", "nickname", "level", "message")
}
account.ck_checked_at = cookie_check["checked_at"]
worker_db.commit()
if not cookie_check["valid"]:
message = f"Cookie 已失效,请重新登录:{cookie_check['message']}"
self._mark_task(worker_db, task, "failed", message)
self._push_log(
"warning", f"[{current}] {self._account_name(account)} {message}"
)
return
update_account_profile_from_cookie(account, cookie)
handler = {
"refresh_goods": self._execute_refresh_goods,
"refresh_esports_goods": self._execute_refresh_esports_goods,
"get_bind_qr": self._execute_get_bind_qr,
"confirm_bind": self._execute_confirm_bind,
"create_elite_qr": self._execute_create_elite_qr,
"query_elite_status": self._execute_query_elite_status,
"prepare_esports_bind": self._execute_prepare_esports_bind,
"get_esports_bind_qr": self._execute_get_esports_bind_qr,
"query_esports_game_name": self._execute_query_esports_game_name,
"confirm_esports_bind": self._execute_confirm_esports_bind,
"create_esports_qr": self._execute_create_esports_qr,
"query_esports_points": self._execute_query_esports_points,
"donate_esports_chicken_gift": self._execute_donate_esports_chicken_gift,
"donate_esports_firework_gift": self._execute_donate_esports_firework_gift,
"create_gold_qr": self._execute_create_gold_qr,
"donate_elite_gift": self._execute_donate_elite_gift,
"query_points": self._execute_query_points,
"lock_goods": self._execute_lock_goods,
"pay_locked_order": self._execute_pay_locked_order,
"exchange_goods": self._execute_exchange_goods,
"exchange_esports_goods": self._execute_exchange_esports_goods,
"query_game_name": self._execute_query_game_name,
"query_change_bind_time": self._execute_query_change_bind_time,
"query_limited_goods": self._execute_query_limited_goods,
"query_gold_balance": self._execute_query_gold_balance,
"query_exchange_records": self._execute_query_exchange_records,
"prefetch_csrf_token": self._execute_prefetch_csrf_token,
"get_xpd_bind_qr": self._execute_get_xpd_bind_qr,
"query_xpd_bind_info": self._execute_query_xpd_bind_info,
"confirm_xpd_bind": self._execute_confirm_xpd_bind,
"query_xpd_role": self._execute_query_xpd_role,
"refresh_xpd_goods": self._execute_refresh_xpd_goods,
"query_xpd_balance": self._execute_query_xpd_balance,
"query_xpd_fragments": self._execute_query_xpd_fragments,
"query_xpd_purchase_records": self._execute_query_xpd_purchase_records,
"exchange_xpd_goods": self._execute_exchange_xpd_goods,
}.get(task.task_type)
if handler is None:
self._mark_task(worker_db, task, "failed", "不支持的任务类型")
return
handler(worker_db, task, account, cookie, config)
self._push_log(
"success", f"[{current}] {self._account_name(account)} {task.message}"
)
except DouyuActivityError as exc:
if "task" in locals() and task:
self._mark_task(worker_db, task, "failed", str(exc))
self._push_log("warning", f"斗鱼任务失败: {exc}")
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
if "task" in locals() and task:
self._mark_task(worker_db, task, "error", str(exc))
self._push_log("error", f"斗鱼任务异常: {exc}")
finally:
worker_db.close()
def run(self):
"""执行批次任务。"""
self._push_log("info", f"斗鱼任务批次 {self.batch_id} 开始")
try:
config = self._config_info(self.db)
tasks = (
self.db.query(DouyuTask)
.filter(
DouyuTask.batch_id == self.batch_id, DouyuTask.status == "planned"
)
.order_by(DouyuTask.id.asc())
.all()
)
if not tasks:
self._push_log("warning", "没有可执行的斗鱼任务")
self._push_log("result", "")
return
for task in tasks:
task.status = "pending"
task.message = "等待执行"
self.db.commit()
for task in tasks:
self._push_task_event(task)
total = len(tasks)
with ThreadPoolExecutor(max_workers=self.concurrency) as executor:
futures = []
for task in tasks:
if self._stop.is_set():
break
futures.append(
executor.submit(self._execute_one, task.id, config, total)
)
for future in as_completed(futures):
try:
future.result()
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
self._push_log("error", f"Worker 异常: {exc}")
if self._stop.is_set():
self._push_log("warning", f"斗鱼任务批次 {self.batch_id} 已停止")
else:
self._push_log("info", f"斗鱼任务批次 {self.batch_id} 完成")
self._push_log("result", "")
finally:
self.db.close()