时间线从此覆盖**全部**业务事实,`douyu_tasks` 也有了可控的增长边界。
## 一、删掉收集器(它必然漏数据)
`jobs/account_events_collect.py` 连同 `_task_to_event` / `COLLECTED_TASK_TYPES` /
水位线一起删除,迁移 `20261004_0043` 清掉它的作业行。删除理由是实测出来的:
- **靠 allowlist 认业务类型,新增类型就静默丢**:原有名单只有 5 个,
而 `exchange_esports_goods`(电竞兑换)、`exchange_xpd_goods`(和平小店兑换)、
`pay_locked_order`(锁单支付)、`donate_esports_*`(电竞礼物积分)**从来没进过时间线**;
- **「任务 success」≠「业务真的发生了」**(见下);
- **字段路径靠抽样猜**:二维码充值的 `result` 里根本没有金额,收集器恒写
`amount=None`,`recharge_total` 系统性少计;
- 作业默认 `enabled=False`,生产上一直没跑。
历史**不回填**(既有结论不变):需要历史走斗鱼侧权威接口
(`query_exchange_records` / `query_xpd_purchase_records` / `query_points` 都是现成任务类型)。
## 二、业务事实直发:命名构造函数
新增 `services/business_events.py`:`record_recharge` / `record_exchange` /
`record_points_credit` / `record_bind`。每类事实的字段、标题、金额符号只定义一次
(与 S4 的 `record_account_action`、`on_ck_verdict` 同一套路)。
**9 处成功点接线**:`exchange_goods` 真兑换分支、`exchange_esports_goods`、
`exchange_xpd_goods`、`pay_locked_order`、`create_gold_qr`(二维码 + 供应商轮询 +
**供应商回调**三条路径)、`donate_elite_gift`、`donate_esports_gift`、
`confirm_bind`、`confirm_esports_bind`。
**只记真实事实**(按你的决定):`exchange_goods` 里「预约到货 / 已预兑 / 预兑确认」
三条 success 分支**不发事件** —— 它们不消耗积分也没换到东西,旧收集器只按 status
收集,于是每预约一次就生成一条金额为空的 `exchange` 把 `exchange_count` 灌大。
`confirm_bind` 里「检测到账号本来就已绑定」那条也跳过。
## 三、修掉两个真 bug(都是钱账)
1. **到账判据 `!=` → `>`**(`douyu_runner_gold.py`):余额**下降**也算「鱼翅已到账」。
批内并发(最高 10)时同账号送礼会消耗鱼翅,余额一变就被充值任务读成到账,
**往台账记一笔根本没发生的充值**。
2. **供应商回调晚于 runner 超时**:原来守卫是「任务状态没被改过才更新」,
于是 runner 先判 failed、回调随后说成功的那笔真金白银**永远不进时间线**。
改成按**回调自己的终态**判断,并补发事件。
## 四、发射顺序:一个会静默丢事件的坑
`_mark_task` 内部 `db.commit()`,而 `emit` 用 `db.begin_nested()` **只 RELEASE
SAVEPOINT、不提交外层事务**;runner 的 `finally` 只 `close()`(未提交即回滚)。
所以**在 `_mark_task` 之后发事件会被静默丢弃,且 `emit` 仍返回 True**。
所有接线一律紧贴 `_mark_task(success)` **之前**,并加了源码扫描用例钉住
(已反向验证:故意挪到之后,用例立刻报出 file:line)。
## 五、`douyu_tasks` 保留策略(新增作业,默认关闭)
`jobs/douyu_tasks_retention.py`,两档:
| 档 | 保留 |
| --- | --- |
| 噪声查询(只读看一眼就走,且前端不拿它派生账号状态) | **7 天** |
| 其余一切(业务事实 + 状态派生依赖的查询 + **以后新增的类型**) | **90 天** |
长保留档用 `task_type NOT IN (噪声名单)` 表达 → **新增类型默认长保留**,不会因为
「忘了加名单」被误删(正是收集器翻车的反面)。在途任务(planned/pending/running)
**绝不删**;分批删避免长锁;两个窗口都能在作业 `config` 里改,不必发版。
⚠️ 「前端从任务列表反推账号当前状态」那几类查询(`query_xpd_bind_info` /
`query_esports_game_name` 等)**故意留在长保留档** —— 删早了工作台账号行会变
「未查」,那是把**当前显示**搞坏。根治要靠 §3.6 的状态列拆解(后续阶段)。
作业默认 `enabled=False`,由运维决定何时打开。
## 验证
- `TZ=Asia/Shanghai`:**450 passed / 6 skipped**(S4 后 426 → 净增 24)
- `TZ=UTC`(相关文件)101 passed;ruff check/format 全绿
- 前端 build 通过、lint 0 errors
- 真实开发库:迁移已应用(head `20261004_0043`,孤儿作业行已清);
保留作业实跑确认「作业行默认关闭、SQL 在 MySQL 上正确、近期行不误删」
- **两处反向验证**:把 `record_bind` 挪到 `_mark_task` 之后 → 顺序用例报红;
修掉 `registered_jobs()` 那个**假绿用例**(它返回 dict,原来的集合推导式
遍历空字典恒成立),并在断言里加了反向确认
未做(后续):S6 账号页重构 + `bind_status` 三列拆解(做完才能把状态派生类查询
也纳入短保留)、S7 账号检测改账号维度、S8 丢 `login_tasks` 旧凭据列。
343 lines
13 KiB
Python
343 lines
13 KiB
Python
"""斗鱼任务执行器:送礼(由 douyu_runner.py 按功能域拆分)。"""
|
||
|
||
from __future__ import annotations
|
||
|
||
from datetime import UTC, datetime
|
||
from typing import TYPE_CHECKING, Any
|
||
|
||
from sqlalchemy.orm import Session
|
||
|
||
from core.douyu import DouyuActivityClient
|
||
|
||
from ..models import Account, DouyuTask
|
||
from .business_events import record_points_credit
|
||
|
||
DOUYU_GIFT_POINTS_REFRESH_TIMES = 3
|
||
DOUYU_GIFT_POINTS_REFRESH_INTERVAL = 2
|
||
|
||
|
||
class DonateMixin:
|
||
"""送礼域:精英令/电竞任务礼物赠送与积分确认。"""
|
||
|
||
if TYPE_CHECKING:
|
||
_stop: Any
|
||
|
||
def _client(self, cookie: str) -> DouyuActivityClient: ...
|
||
def _task_payload(self, task: DouyuTask) -> dict: ...
|
||
def _push_log(self, level: str, message: str) -> None: ...
|
||
def _mark_task(self, *args: Any, **kwargs: Any) -> None: ...
|
||
def _update_task_progress(self, *args: Any, **kwargs: Any) -> None: ...
|
||
def _refresh_account_points(
|
||
self,
|
||
client: DouyuActivityClient,
|
||
account: Account,
|
||
cookie: str,
|
||
*,
|
||
ctn: str | None = None,
|
||
) -> dict: ...
|
||
def _refresh_account_gold_balance(
|
||
self, client: DouyuActivityClient, account: Account
|
||
) -> dict: ...
|
||
def _refresh_esports_handbook(
|
||
self, client: DouyuActivityClient, account: Account, *, manual_id: str
|
||
) -> dict: ...
|
||
|
||
def _refresh_points_after_elite_gift(
|
||
self,
|
||
db: Session,
|
||
task: DouyuTask,
|
||
account: Account,
|
||
client: DouyuActivityClient,
|
||
cookie: str,
|
||
ctn: str | None,
|
||
result: dict,
|
||
baseline_points: int | None,
|
||
gift_count: int,
|
||
) -> dict:
|
||
"""赠送精英令后短轮询积分;1 个精英令约等于 10 积分。"""
|
||
expected_gain = max(0, gift_count) * 10
|
||
target_points = (
|
||
baseline_points + expected_gain if baseline_points is not None else None
|
||
)
|
||
result["gift_points_baseline"] = baseline_points
|
||
result["gift_points_expected_gain"] = expected_gain
|
||
result["gift_points_target"] = target_points
|
||
|
||
last_points = None
|
||
refresh_result: dict = {}
|
||
for index in range(1, DOUYU_GIFT_POINTS_REFRESH_TIMES + 1):
|
||
refresh_result = self._refresh_account_points(
|
||
client, account, cookie, ctn=ctn
|
||
)
|
||
db.commit()
|
||
last_points = refresh_result["points"]
|
||
result.update(refresh_result)
|
||
result["gift_points_refresh_count"] = index
|
||
if target_points is None or (
|
||
last_points is not None and last_points >= target_points
|
||
):
|
||
result["gift_points_confirmed"] = (
|
||
target_points is None or last_points is not None
|
||
)
|
||
return refresh_result
|
||
if index < DOUYU_GIFT_POINTS_REFRESH_TIMES:
|
||
self._update_task_progress(
|
||
db,
|
||
task,
|
||
"running",
|
||
f"赠送精英令成功,等待积分同步(当前 {last_points if last_points is not None else '-'},预期 {target_points})",
|
||
result,
|
||
)
|
||
if self._stop.wait(DOUYU_GIFT_POINTS_REFRESH_INTERVAL):
|
||
break
|
||
|
||
result["gift_points_confirmed"] = False
|
||
result["points"] = last_points
|
||
return refresh_result
|
||
|
||
def _execute_donate_esports_gift(
|
||
self,
|
||
db: Session,
|
||
task: DouyuTask,
|
||
account: Account,
|
||
cookie: str,
|
||
config: dict,
|
||
*,
|
||
gift_name: str,
|
||
config_gift_id_key: str,
|
||
config_skin_id_key: str,
|
||
):
|
||
"""赠送电竞手册任务礼物并刷新独立积分。"""
|
||
payload = self._task_payload(task)
|
||
try:
|
||
gift_count = max(
|
||
1, int(payload.get("gift_count") or payload.get("count") or 1)
|
||
)
|
||
except (TypeError, ValueError):
|
||
self._mark_task(db, task, "failed", "赠送数量必须是正整数")
|
||
return
|
||
|
||
manual_id = str(config.get("esports_manual_id") or "").strip()
|
||
gift_id = str(
|
||
payload.get("gift_id") or config.get(config_gift_id_key) or ""
|
||
).strip()
|
||
skin_id = str(
|
||
payload.get("skin_id") or config.get(config_skin_id_key) or ""
|
||
).strip()
|
||
room_id = str(payload.get("room_id") or config.get("room_id") or "").strip()
|
||
if not manual_id or not gift_id or not skin_id or not room_id:
|
||
self._mark_task(db, task, "failed", "请先完整配置电竞手册、房间和礼物参数")
|
||
return
|
||
|
||
client = self._client(cookie)
|
||
baseline_points = account.esports_points
|
||
try:
|
||
baseline = self._refresh_esports_handbook(
|
||
client, account, manual_id=manual_id
|
||
)
|
||
baseline_points = baseline["esports_points"]
|
||
db.commit()
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
self._push_log("warning", f"赠送{gift_name}前刷新电竞积分失败: {exc}")
|
||
|
||
result = client.donate_esports_gift(
|
||
gift_name=gift_name,
|
||
gift_count=gift_count,
|
||
room_id=room_id,
|
||
gift_id=gift_id,
|
||
skin_id=skin_id,
|
||
)
|
||
result.update(
|
||
{
|
||
"gift_name": gift_name,
|
||
"gift_id": gift_id,
|
||
"skin_id": skin_id,
|
||
"gift_count": gift_count,
|
||
"esports_points_baseline": baseline_points,
|
||
}
|
||
)
|
||
refresh_errors = []
|
||
try:
|
||
result.update(self._refresh_account_gold_balance(client, account))
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
refresh_errors.append(f"鱼翅余额: {exc}")
|
||
try:
|
||
points_result = self._refresh_esports_handbook(
|
||
client, account, manual_id=manual_id
|
||
)
|
||
result.update(points_result)
|
||
result["esports_points_after_gift"] = points_result["esports_points"]
|
||
result["esports_points_changed"] = (
|
||
baseline_points is not None
|
||
and points_result["esports_points"] is not None
|
||
and points_result["esports_points"] != baseline_points
|
||
)
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
refresh_errors.append(f"电竞积分: {exc}")
|
||
if refresh_errors:
|
||
result["refresh_errors"] = refresh_errors
|
||
|
||
account.esports_bind_status = "esports_gift_donated"
|
||
account.updated_at = datetime.now(UTC)
|
||
confirmed = bool(result.get("esports_points_changed"))
|
||
message = f"赠送{gift_name}成功: {gift_count}"
|
||
if account.gold_balance is not None:
|
||
message += f",鱼翅余额: {account.gold_balance}"
|
||
if account.esports_points is not None:
|
||
message += f",电竞积分: {account.esports_points}"
|
||
# 电竞礼物同样产生积分到账 —— 这一类此前**从来不在收集范围**
|
||
# (不在 COLLECTED_TASK_TYPES 里),时间线上一直是空白。
|
||
record_points_credit(
|
||
db,
|
||
account,
|
||
task_id=task.id,
|
||
gain=_esports_gift_points(result, baseline_points),
|
||
points_after=account.esports_points,
|
||
confirmed=confirmed,
|
||
source_label=gift_name,
|
||
)
|
||
self._mark_task(db, task, "success", message, result)
|
||
|
||
def _execute_donate_esports_chicken_gift(
|
||
self,
|
||
db: Session,
|
||
task: DouyuTask,
|
||
account: Account,
|
||
cookie: str,
|
||
config: dict,
|
||
):
|
||
"""赠送冠军鸡腿。"""
|
||
self._execute_donate_esports_gift(
|
||
db,
|
||
task,
|
||
account,
|
||
cookie,
|
||
config,
|
||
gift_name="冠军鸡腿",
|
||
config_gift_id_key="esports_chicken_gift_id",
|
||
config_skin_id_key="esports_chicken_skin_id",
|
||
)
|
||
|
||
def _execute_donate_esports_firework_gift(
|
||
self,
|
||
db: Session,
|
||
task: DouyuTask,
|
||
account: Account,
|
||
cookie: str,
|
||
config: dict,
|
||
):
|
||
"""赠送冠军烟花。"""
|
||
self._execute_donate_esports_gift(
|
||
db,
|
||
task,
|
||
account,
|
||
cookie,
|
||
config,
|
||
gift_name="冠军烟花",
|
||
config_gift_id_key="esports_firework_gift_id",
|
||
config_skin_id_key="esports_firework_skin_id",
|
||
)
|
||
|
||
def _execute_donate_elite_gift(
|
||
self, db: Session, task: DouyuTask, account: Account, cookie: str, config: dict
|
||
):
|
||
payload = self._task_payload(task)
|
||
gift_count = int(payload.get("gift_count") or payload.get("count") or 1)
|
||
client = self._client(cookie)
|
||
ctn = None
|
||
baseline_points = account.points
|
||
try:
|
||
ctn = client.acf_ccn(refresh_subscribe=False)
|
||
baseline_result = self._refresh_account_points(
|
||
client, account, cookie, ctn=ctn
|
||
)
|
||
db.commit()
|
||
baseline_points = baseline_result["points"]
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
self._push_log("warning", f"赠送精英令前刷新积分失败: {exc}")
|
||
result = client.donate_elite_gift(
|
||
gift_count=gift_count,
|
||
room_id=str(payload.get("room_id") or config["room_id"]),
|
||
gift_id=str(payload.get("gift_id") or config["gift_id"]),
|
||
skin_id=str(payload.get("skin_id") or config["skin_id"]),
|
||
)
|
||
result["gift_points_baseline"] = baseline_points
|
||
refresh_errors = []
|
||
try:
|
||
result.update(self._refresh_account_gold_balance(client, account))
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
refresh_errors.append(f"鱼翅余额: {exc}")
|
||
try:
|
||
result.update(
|
||
self._refresh_points_after_elite_gift(
|
||
db,
|
||
task,
|
||
account,
|
||
client,
|
||
cookie,
|
||
ctn,
|
||
result,
|
||
baseline_points,
|
||
gift_count,
|
||
)
|
||
)
|
||
except Exception as exc: # noqa: BLE001 外部接口与任务边界需要保留宽泛异常兜底
|
||
refresh_errors.append(f"积分: {exc}")
|
||
if refresh_errors:
|
||
result["refresh_errors"] = refresh_errors
|
||
account.bind_status = "gift_donated"
|
||
account.updated_at = datetime.now(UTC)
|
||
message = f"赠送精英令成功: {gift_count}"
|
||
if account.gold_balance is not None:
|
||
message += f",鱼翅余额: {account.gold_balance}"
|
||
confirmed = bool(result.get("gift_points_confirmed"))
|
||
if account.points is not None:
|
||
message += f",积分: {account.points}"
|
||
if result.get("gift_points_target") is not None and not confirmed:
|
||
message += f"(未确认涨到 {result['gift_points_target']})"
|
||
# 积分到账是业务事实。**未确认也记,但把 confirmed 明确写进 detail** ——
|
||
# 旧收集器把「预期涨幅」当既成事实写进台账(未确认时也记 +980),
|
||
# 那是把预期记成到账。这里宁可标清楚,也不假装它是确认过的数字。
|
||
# 金额用本地可等价重算的值(1 个精英令 ≈ 10 分),不再依赖 result 里的字段。
|
||
record_points_credit(
|
||
db,
|
||
account,
|
||
task_id=task.id,
|
||
gain=max(0, gift_count) * 10,
|
||
points_after=account.points,
|
||
expire_at=_points_expire_at(result),
|
||
confirmed=confirmed,
|
||
source_label="精英令",
|
||
)
|
||
self._mark_task(db, task, "success", message, result)
|
||
|
||
|
||
def _points_expire_at(result: dict) -> object:
|
||
"""从积分查询结果里取出到期时间(同样是为了给时间线留全信息)。
|
||
|
||
旧收集器走的是 `result["points_query"]["raw"]["data"]["willExpireTime"]` 这条
|
||
「猜出来」的路径;这里收敛成一个具名函数,只有它知道那段结构。
|
||
"""
|
||
query = result.get("points_query")
|
||
if not isinstance(query, dict):
|
||
return None
|
||
raw = query.get("raw")
|
||
data = raw.get("data") if isinstance(raw, dict) else None
|
||
return data.get("willExpireTime") if isinstance(data, dict) else None
|
||
|
||
|
||
def _esports_gift_points(result: dict, baseline_points: int | None) -> int | None:
|
||
"""电竞礼物带来的积分增量。
|
||
|
||
以**实际观测到的积分变化**为准(`esports_points_after_gift` - 基线),
|
||
而不是像旧收集器那样去读一个「预期涨幅」。观测不到就返回 None ——
|
||
金额宁可空着,也不要写一个猜出来的数字进台账。
|
||
"""
|
||
after = result.get("esports_points_after_gift")
|
||
if not isinstance(after, (int, float)) or not isinstance(
|
||
baseline_points, (int, float)
|
||
):
|
||
return None
|
||
delta = int(after) - int(baseline_points)
|
||
return delta if delta > 0 else None
|