S1 加好了列,本提交把**读与写一起切**过去。这是设计里「同一份数据的读与写必须
在同一个提交里切」的要求:先切读或先切写都会出现「写账号、读记录」的中间态,
跑起来不报错,只是默默用错数据。
## 核心改动
**判定「谁有 CK」不再借道登录记录**
- `cookie_holding_filter()` / `cookie_operable_filter()` / `cookie_holding_account_ids_query()`
/ `visible_douyu_task_accounts()` 全部改为作用于 `Account`;
- `cookie_operable_filter()` 的「在途排除」仍查 `login_tasks`(那是**某次尝试**的属性),
用 `NOT EXISTS` 子查询 + 迁移 0042 新增的 `(account_id, attempt_status)` 索引;
- 删除死代码 `latest_cookie_login_task()`;`latest_cookie()` 改为直接读账号。
**CK 写入**
- `_write_login_success()` 写 `Account.cookie / cookie_updated_at / cookie_source`;
- `_mark_attempt_failed()` 改看**账号**有没有 CK(决定是否沿用旧的失败语义);
- 检测模式的 `task.cookie = ""` **不再清空账号的 CK**(见下「行为变更」)。
**自动化作业按账号调度**
- `ck_health`:`_select_tasks` → `_select_accounts`,`db.query(Account)`;新增
`count_remaining()`(刻意不带 LIMIT,否则积压数恒等于批次上限);
- `ck_relogin`:同样按账号选行;结算改为「账号 + 本轮尝试记录」,
退避进度写账号;`settle_fresh_results(batch_id)` 按 `attempt_batch_id` 配对,
`run_ck_relogin` 把 **account_ids** 交给 runner;
- 顺带让**没有任何登录历史的账号**也能被巡检/重登(以前它们永远进不了队列)。
**登录批次**
- `LoginBatchRunner` 去掉 `relogin_task_ids`,重登与登录统一走 `account_ids`;
- 批次行复用逻辑重写:只保证「同一账号不并发两行」,不再关心「哪一行持有 CK」。
失败行复用同时看 `attempt_status`(账号有 CK 时 `_mark_attempt_failed` 刻意不动
`status`,只看 status 会永远命不中 → 每次失败刷一行)。
**删除护栏语义变更(用户可感知)**
- 去掉「持有 CK 的记录不可删」——CK 在账号上,删历史不再影响登录态,历史终于可清理;
- **保留**「在途尝试不可删」(批次线程正拿着那条记录)。
## 行为变更(需评审)
1. **账号检测不再清空 CK**。旧代码在 check 模式写 `task.cookie = ""`,而那时记录的
cookie 就是账号唯一凭证 —— 一次「批量检测」会把账号登录态删掉。解耦后
「检测账号凭据」与「CK 是否有效」本就是两件事(后者归 `ck_health` 判),
继续清空等于凭空制造不可逆的数据丢失(重登当前成功率极低)。
2. **Cookie 页的记录单位从「登录记录」变成「账号」**,所有接口的 id 入参/出参
变成 `account_id`;`DELETE /cookies/{id}` 清的是该账号的 CK,**必写审计**。
3. 重复检测从「同一账号多行持 CK」改为「**同一个 CK 出现在多个账号上**」——
后者才是真风险(两个账号共用一个登录态)。用 sha256 指纹分组:
CK 是 AES-GCM + 随机 nonce 加密,**同一明文每次密文都不同**,比密文永远比不出重复。
## 修掉一个真实并发缺陷
`_write_login_success` 原先被传入 `self.db`(runner 主会话),而它是在**工作线程**里
用 `worker_db` 执行的。`emit()` 会往 `account_events` 插一行,其 `account_id` 外键对
`accounts` 那一行加**共享锁**;这把锁加在另一个会话上,紧接着本会话的
`UPDATE accounts SET cookie=...` 就被自己人挡住 —— 实测 **50 秒后
`Lock wait timeout exceeded`,登录明明成功却写不回 CK**(改造前 UPDATE 打在
`login_tasks` 上所以没撞上)。改为传 `worker_db`:事件本就该和状态变更同事务提交,
顺带修掉跨线程共用一个 Session 的隐患。
## 验证
- 全量 pytest **368 passed / 6 skipped**;迁移链 3 passed;
- 前端 `tsc -b && vite build` 通过;
- `ruff check` / `ruff format --check` 全绿(顺带修掉 main 上遗留的 F821);
- 新增/改写的关键回归:**删掉全部登录历史后账号的 CK 与健康状态仍在**、
**无登录历史的账号也能进巡检队列**、结算不碰 CK、删除护栏改为在途语义。
仍未做(设计里排期的后续阶段):`account_lifecycle` 状态收口(S3)、事件发射点补齐(S4)、
业务事实直发事件(S5)、账号页重构(S6)。注意 `status` 目前只有模型默认值与迁移回填,
所以 `disabled/archived` 还没有任何写入方——S3 落地时要把自动化选行的状态过滤一起补上。
470 lines
15 KiB
Python
470 lines
15 KiB
Python
"""斗鱼活动任务服务。"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import uuid
|
|
from datetime import UTC, datetime
|
|
|
|
from sqlalchemy import and_, func, select
|
|
from sqlalchemy.orm import Session
|
|
|
|
from core.douyu.activity_client import DouyuActivityClient
|
|
from core.douyu.cookie_utils import cookie_value
|
|
|
|
from ..models import Account, DouyuConfig, DouyuTask, LoginTask
|
|
|
|
SUPPORTED_DOUYU_TASK_TYPES = {
|
|
"get_bind_qr": "获取绑定二维码",
|
|
"confirm_bind": "确认绑定",
|
|
"create_elite_qr": "开通精英宝典30",
|
|
"query_elite_status": "查询精英宝典状态",
|
|
"prepare_esports_bind": "绑定电竞手册角色",
|
|
"get_esports_bind_qr": "切换电竞角色",
|
|
"query_esports_game_name": "查询最新游戏角色",
|
|
"confirm_esports_bind": "确认电竞手册绑定",
|
|
"create_esports_qr": "开通电竞手册30",
|
|
"query_esports_points": "查询电竞积分",
|
|
"donate_esports_chicken_gift": "赠送冠军鸡腿",
|
|
"donate_esports_firework_gift": "赠送冠军烟花",
|
|
"refresh_esports_goods": "刷新电竞皮肤商城",
|
|
"exchange_esports_goods": "兑换电竞皮肤",
|
|
"create_gold_qr": "充值鱼翅",
|
|
"donate_elite_gift": "赠送精英令",
|
|
"query_points": "一键查询积分",
|
|
"lock_goods": "锁定商品",
|
|
"pay_locked_order": "支付锁单",
|
|
"exchange_goods": "兑换商品",
|
|
"query_game_name": "一键获取游戏名",
|
|
"query_change_bind_time": "一键查询换绑时间",
|
|
"query_limited_goods": "一键查询限兑商品",
|
|
"query_gold_balance": "一键查询鱼翅余额",
|
|
"refresh_goods": "刷新商品列表",
|
|
"query_exchange_records": "一键查询兑换记录",
|
|
"prefetch_csrf_token": "一键获取兑换 CSRF Token",
|
|
"get_xpd_bind_qr": "生成小店绑定二维码",
|
|
"query_xpd_bind_info": "查询小店绑定信息",
|
|
"confirm_xpd_bind": "确认小店绑定",
|
|
"query_xpd_role": "查询小店绑定角色",
|
|
"refresh_xpd_goods": "刷新小店商品列表",
|
|
"query_xpd_balance": "查询小店点券余额",
|
|
"query_xpd_fragments": "查询小店扭蛋碎片",
|
|
"query_xpd_purchase_records": "查询小店兑换记录",
|
|
"exchange_xpd_goods": "兑换小店商品",
|
|
}
|
|
|
|
DOUYU_HANDBOOK_SCOPES = {"elite", "esports", "peace"}
|
|
DOUYU_HANDBOOK_TASK_TYPES = {
|
|
"elite": {
|
|
"get_bind_qr",
|
|
"confirm_bind",
|
|
"create_elite_qr",
|
|
"query_elite_status",
|
|
"create_gold_qr",
|
|
"donate_elite_gift",
|
|
"query_points",
|
|
"lock_goods",
|
|
"pay_locked_order",
|
|
"exchange_goods",
|
|
"query_game_name",
|
|
"query_change_bind_time",
|
|
"query_limited_goods",
|
|
"query_gold_balance",
|
|
"refresh_goods",
|
|
"query_exchange_records",
|
|
"prefetch_csrf_token",
|
|
},
|
|
"esports": {
|
|
"prepare_esports_bind",
|
|
"get_esports_bind_qr",
|
|
"query_esports_game_name",
|
|
"confirm_esports_bind",
|
|
"create_esports_qr",
|
|
"query_esports_points",
|
|
"query_gold_balance",
|
|
"query_change_bind_time",
|
|
"query_limited_goods",
|
|
"refresh_esports_goods",
|
|
"exchange_esports_goods",
|
|
"create_gold_qr",
|
|
"donate_esports_chicken_gift",
|
|
"donate_esports_firework_gift",
|
|
},
|
|
"peace": {
|
|
"get_xpd_bind_qr",
|
|
"query_xpd_bind_info",
|
|
"confirm_xpd_bind",
|
|
"query_xpd_role",
|
|
"refresh_xpd_goods",
|
|
"query_xpd_balance",
|
|
"query_xpd_fragments",
|
|
"query_xpd_purchase_records",
|
|
"exchange_xpd_goods",
|
|
},
|
|
}
|
|
|
|
|
|
DOUYU_CONFIG_DEFAULTS = {
|
|
"manual_id": "G4KA4Qnz4LDp7",
|
|
"rid": "9263298",
|
|
"bind_act_alias": "20260120QYOOB",
|
|
"confirm_act_alias": "20260120QYOOB",
|
|
"legacy_act_alias": "cjm",
|
|
"room_id": "9263298",
|
|
"elite_amount": 3000,
|
|
"esports_manual_id": "1W0yGkOdDbAXb",
|
|
"esports_act_alias": "20260226ZMGDG",
|
|
"esports_amount": 3000,
|
|
"esports_chicken_gift_id": "24766",
|
|
"esports_chicken_skin_id": "0",
|
|
"esports_firework_gift_id": "24767",
|
|
"esports_firework_skin_id": "3850",
|
|
"gold_pay_type": 1,
|
|
"gold_recharge_channel": "wechat_qr",
|
|
"gold_api_product_id": "111570",
|
|
"gold_api_account_template_name": "斗鱼昵称",
|
|
"gift_id": "23643",
|
|
"skin_id": "2942",
|
|
"xpd_act_alias": "20260623KDQFH",
|
|
"xpd_act_id": "46195",
|
|
"xpd_rid": "9263298",
|
|
}
|
|
|
|
DOUYU_CONFIG_FIELDS = tuple(DOUYU_CONFIG_DEFAULTS.keys())
|
|
DOUYU_ACTIVE_TASK_STATUSES = ("planned", "pending", "running")
|
|
|
|
|
|
def douyu_config_value(field: str, value):
|
|
"""读取配置值;空值自动回退到默认值。"""
|
|
default = DOUYU_CONFIG_DEFAULTS[field]
|
|
if isinstance(default, int):
|
|
try:
|
|
return int(value if value is not None else default)
|
|
except (TypeError, ValueError):
|
|
return default
|
|
text = str(value or "").strip()
|
|
return text or str(default)
|
|
|
|
|
|
def apply_douyu_config_defaults(config: DouyuConfig) -> bool:
|
|
"""补齐斗鱼配置默认值,返回是否发生变更。"""
|
|
changed = False
|
|
for field in DOUYU_CONFIG_FIELDS:
|
|
if (
|
|
field == "bind_act_alias"
|
|
and str(getattr(config, field, "") or "").strip() == "20250213NQCYX"
|
|
):
|
|
setattr(config, field, DOUYU_CONFIG_DEFAULTS[field])
|
|
changed = True
|
|
continue
|
|
normalized = douyu_config_value(field, getattr(config, field, None))
|
|
if field == "gold_recharge_channel" and normalized not in {
|
|
"wechat_qr",
|
|
"supplier_api",
|
|
}:
|
|
normalized = DOUYU_CONFIG_DEFAULTS[field]
|
|
if getattr(config, field, None) != normalized:
|
|
setattr(config, field, normalized)
|
|
changed = True
|
|
return changed
|
|
|
|
|
|
def ensure_douyu_config(db: Session) -> DouyuConfig:
|
|
"""获取单条斗鱼配置,不存在则创建。"""
|
|
config = db.query(DouyuConfig).first()
|
|
if config:
|
|
if apply_douyu_config_defaults(config):
|
|
db.commit()
|
|
db.refresh(config)
|
|
return config
|
|
config = DouyuConfig(**DOUYU_CONFIG_DEFAULTS)
|
|
db.add(config)
|
|
db.commit()
|
|
db.refresh(config)
|
|
return config
|
|
|
|
|
|
# 登录/重登尝试的在途状态:判断只看 attempt_*,与账号持有的 CK 无关。
|
|
LOGIN_ATTEMPT_IN_FLIGHT_STATUSES = ("pending", "running")
|
|
|
|
|
|
def cookie_holding_filter():
|
|
"""判定**账号**当前是否持有 CK。
|
|
|
|
CK 是账号级凭证(2026-10-04 起从 `login_tasks.cookie` 迁到 `accounts.cookie`)。
|
|
判定只看账号自己,不再看任何登录记录的状态 —— 这正是解耦的目的:
|
|
删历史、批次排队、重登失败都不该让账号凭空「没有 CK」。
|
|
"""
|
|
return and_(
|
|
Account.cookie.isnot(None),
|
|
Account.cookie != "",
|
|
)
|
|
|
|
|
|
def cookie_operable_filter():
|
|
"""判定**账号**能否基于现有 CK 再做检测或重登。
|
|
|
|
在途排除仍然要查 `login_tasks`:「正被批次处理」是**某次尝试**的属性,
|
|
CK 才是账号的属性。所以这里是「账号有 CK」+「该账号没有在途尝试」。
|
|
|
|
走 `login_tasks(account_id, attempt_status)` 索引(迁移 0042 新增),
|
|
否则每轮选行都要回表。
|
|
"""
|
|
in_flight = (
|
|
select(LoginTask.id)
|
|
.where(LoginTask.account_id == Account.id)
|
|
.where(
|
|
func.coalesce(LoginTask.attempt_status, "").in_(
|
|
LOGIN_ATTEMPT_IN_FLIGHT_STATUSES
|
|
)
|
|
)
|
|
.exists()
|
|
)
|
|
return and_(cookie_holding_filter(), ~in_flight)
|
|
|
|
|
|
def latest_cookie(db: Session, account_id: int) -> str:
|
|
"""读取账号当前 CK。"""
|
|
account = db.query(Account.cookie).filter(Account.id == account_id).first()
|
|
return (account[0] if account else "") or ""
|
|
|
|
|
|
def cookie_holding_account_ids_query(db: Session):
|
|
"""返回当前仍持有 CK 的斗鱼账号 ID 查询。"""
|
|
return db.query(Account.id).filter(cookie_holding_filter())
|
|
|
|
|
|
def visible_douyu_task_accounts(db: Session, account_ids: list[int]) -> list[Account]:
|
|
"""只保留当前仍持有 CK 的斗鱼账号。"""
|
|
if not account_ids:
|
|
return []
|
|
return (
|
|
db.query(Account)
|
|
.filter(Account.id.in_(account_ids), cookie_holding_filter())
|
|
.all()
|
|
)
|
|
|
|
|
|
def update_account_profile_from_cookie(account: Account, cookie: str) -> None:
|
|
"""从 Cookie 回填 uid/nickname。"""
|
|
profile = DouyuActivityClient.profile_from_cookie(cookie)
|
|
if profile["uid"]:
|
|
account.uid = profile["uid"]
|
|
if profile["nickname"]:
|
|
account.nickname = profile["nickname"]
|
|
|
|
|
|
def account_uid(account: Account, cookie: str) -> str:
|
|
"""优先从账号字段读取 uid,缺失时从 Cookie 中取。"""
|
|
return account.uid or cookie_value(cookie, "acf_uid")
|
|
|
|
|
|
def create_douyu_planned_tasks(
|
|
db: Session,
|
|
account_ids: list[int],
|
|
task_type: str,
|
|
handbook_scope: str,
|
|
created_by: int,
|
|
payload: dict | None = None,
|
|
) -> tuple[str, int]:
|
|
"""创建斗鱼任务记录,等待后台执行器消费。"""
|
|
if task_type not in SUPPORTED_DOUYU_TASK_TYPES:
|
|
raise ValueError("不支持的任务类型")
|
|
if handbook_scope not in DOUYU_HANDBOOK_SCOPES:
|
|
raise ValueError("无效的工作台")
|
|
if task_type not in DOUYU_HANDBOOK_TASK_TYPES[handbook_scope]:
|
|
raise ValueError("该任务不属于当前工作台")
|
|
|
|
accounts = visible_douyu_task_accounts(db, account_ids)
|
|
if (
|
|
task_type in {"refresh_goods", "refresh_esports_goods", "refresh_xpd_goods"}
|
|
and accounts
|
|
):
|
|
# 商品快照是全局数据,一个可用 CK 足够;没有 CK 时前端无法选账号创建任务。
|
|
accounts = accounts[:1]
|
|
|
|
batch_id = uuid.uuid4().hex[:12]
|
|
payload = payload or {}
|
|
for account in accounts:
|
|
db.add(
|
|
DouyuTask(
|
|
batch_id=batch_id,
|
|
account_id=account.id,
|
|
account_assigned_to=account.assigned_to,
|
|
task_type=task_type,
|
|
handbook_scope=handbook_scope,
|
|
status="planned",
|
|
message="任务已创建,等待执行",
|
|
result={"payload": payload} if payload else None,
|
|
created_by=created_by,
|
|
)
|
|
)
|
|
db.commit()
|
|
return batch_id, len(accounts)
|
|
|
|
|
|
def cleanup_orphan_douyu_tasks(
|
|
db: Session,
|
|
*,
|
|
active_batch_ids: set[str] | None = None,
|
|
batch_id: str | None = None,
|
|
statuses: tuple[str, ...] = DOUYU_ACTIVE_TASK_STATUSES,
|
|
message: str = "任务已中断(服务重启或批次丢失)",
|
|
) -> int:
|
|
"""清理没有执行器接管的斗鱼任务。"""
|
|
query = db.query(DouyuTask).filter(DouyuTask.status.in_(statuses))
|
|
if batch_id:
|
|
query = query.filter(DouyuTask.batch_id == batch_id)
|
|
elif active_batch_ids is not None and active_batch_ids:
|
|
query = query.filter(~DouyuTask.batch_id.in_(list(active_batch_ids)))
|
|
|
|
tasks = query.all()
|
|
if not tasks:
|
|
return 0
|
|
now = datetime.now(UTC)
|
|
for task in tasks:
|
|
task.status = "stopped"
|
|
task.message = message
|
|
task.finished_at = now
|
|
db.commit()
|
|
return len(tasks)
|
|
|
|
|
|
def slim_douyu_goods(goods: object) -> object:
|
|
"""保留兑换图片和列表展示需要的商品小字段。"""
|
|
if not isinstance(goods, dict):
|
|
return None
|
|
return {
|
|
key: value
|
|
for key, value in goods.items()
|
|
if key
|
|
in {
|
|
"commodityId",
|
|
"commodity_id",
|
|
"commodityName",
|
|
"name",
|
|
"webPic",
|
|
"pic",
|
|
"score",
|
|
"status",
|
|
}
|
|
}
|
|
|
|
|
|
def slim_douyu_limited_goods(goods: object) -> list[dict[str, object]]:
|
|
"""限兑列表只返回前几个商品名,避免任务列表携带完整原始数组。"""
|
|
if not isinstance(goods, list):
|
|
return []
|
|
result: list[dict[str, object]] = []
|
|
for item in goods[:5]:
|
|
if not isinstance(item, dict):
|
|
continue
|
|
result.append(
|
|
{
|
|
key: value
|
|
for key, value in item.items()
|
|
if key in {"commodityId", "commodity_id", "commodityName", "name"}
|
|
}
|
|
)
|
|
return result
|
|
|
|
|
|
def strip_douyu_raw_snapshots(value: object) -> object:
|
|
"""递归移除任务结果中的原始接口快照。"""
|
|
if isinstance(value, dict):
|
|
return {
|
|
key: strip_douyu_raw_snapshots(item)
|
|
for key, item in value.items()
|
|
if key != "raw"
|
|
}
|
|
if isinstance(value, list):
|
|
return [strip_douyu_raw_snapshots(item) for item in value]
|
|
return value
|
|
|
|
|
|
def sanitize_douyu_task_result(
|
|
result: dict | None, task_type: str, *, include_detail: bool = False
|
|
) -> dict | None:
|
|
"""列表/实时推送接口剥离原始快照与大数组;详情接口保留完整 result。"""
|
|
if not isinstance(result, dict):
|
|
return result
|
|
if include_detail:
|
|
return result
|
|
|
|
data = strip_douyu_raw_snapshots(result)
|
|
if not isinstance(data, dict):
|
|
return None
|
|
removable_keys = {
|
|
"bind_info",
|
|
"before_bind_info",
|
|
"bind_candidates",
|
|
"cooldown_bind_info",
|
|
"after_bind_info",
|
|
"activity_bind_snapshot",
|
|
"activity_bind_info",
|
|
"points_query",
|
|
"exchange_balance_query",
|
|
"query_act_aliases",
|
|
}
|
|
if task_type != "query_xpd_purchase_records":
|
|
removable_keys.add("records")
|
|
for key in removable_keys:
|
|
data.pop(key, None)
|
|
|
|
# 和平小店任务 result 内嵌 role(含 gameOpenId 等),列表接口剥离其原始快照
|
|
role = data.get("role")
|
|
if isinstance(role, dict):
|
|
data["role"] = strip_douyu_raw_snapshots(role)
|
|
|
|
goods = data.get("goods")
|
|
if isinstance(goods, list):
|
|
data.pop("goods", None)
|
|
elif isinstance(goods, dict):
|
|
data["goods"] = slim_douyu_goods(goods)
|
|
|
|
if "limited_goods" in data:
|
|
data["limited_goods"] = slim_douyu_limited_goods(data.get("limited_goods"))
|
|
|
|
if task_type not in {
|
|
"get_bind_qr",
|
|
"prepare_esports_bind",
|
|
"get_esports_bind_qr",
|
|
"get_xpd_bind_qr",
|
|
}:
|
|
data.pop("url", None)
|
|
if task_type not in {"create_elite_qr", "create_esports_qr", "create_gold_qr"}:
|
|
data.pop("pay_url", None)
|
|
return data
|
|
|
|
|
|
def douyu_task_payload(task: DouyuTask, *, include_detail: bool = False) -> dict:
|
|
"""生成 REST 与 WebSocket 共用的斗鱼任务载荷。"""
|
|
account = task.account
|
|
|
|
def isoformat(value: datetime | None) -> str | None:
|
|
if value is None:
|
|
return None
|
|
if value.tzinfo is None:
|
|
value = value.replace(tzinfo=UTC)
|
|
return value.isoformat()
|
|
|
|
return {
|
|
"id": task.id,
|
|
"batch_id": task.batch_id,
|
|
"account_id": task.account_id,
|
|
"account_username": (account.username if account else "") or "",
|
|
"account_uid": (account.uid if account else "") or "",
|
|
"account_nickname": (account.nickname if account else "") or "",
|
|
"task_type": task.task_type or "",
|
|
"handbook_scope": task.handbook_scope or "legacy",
|
|
"status": task.status or "",
|
|
"message": task.message or "",
|
|
"result": sanitize_douyu_task_result(
|
|
task.result if isinstance(task.result, dict) else None,
|
|
task.task_type or "",
|
|
include_detail=include_detail,
|
|
),
|
|
"created_by": task.created_by,
|
|
"created_at": isoformat(task.created_at),
|
|
"finished_at": isoformat(task.finished_at),
|
|
}
|