后台原本靠人工逐批点「CK 检测」发现失效账号,1.2 万个账号根本盯不过来。
P1a 把「哪些账号失效了」自动化:自动挑久未检测的账号、调现成检测接口、
把结果写回 ck_check_*。**本作业不替换任何 CK**,重登仍由人工点,符合规划里
「先做只读/只提示,确认无误再做自动动作」的渐进上线原则。
实现:
- 迁移 20261002_0037:automation_jobs / automation_runs 两张表,
login_tasks 加 relogin_attempts / relogin_next_at / relogin_last_error
(为 P1b 备好),accounts 加 auto_managed(特殊账号不参与自动化);
- services/automation/scheduler.py:进程内调度线程,60s tick,状态全部落库
(重启不丢调度进度、不重复跑轮);AUTOMATION_ENABLED 默认关闭,作业开关
也默认关闭;ensure_jobs 只补缺失行、不覆盖运维改过的配置;
作业异常只记 failed,不拖垮调度线程;单容器 --workers 1,无需分布式锁;
- services/automation/jobs/ck_health.py:选行口径与人工检测一致
(cookie_holding_filter:持有 CK 且无在途尝试),叠加久未检测、
auto_managed、batch_size,按 ck_checked_at 升序(最久没查的优先);
只写 ck_check_status / ck_check_result / ck_checked_at;
- routers/automation.py:GET /jobs、PATCH /jobs/{key}、POST /jobs/{key}/run、
GET /runs;新增权限点 automation:view(运营可看)与 automation:manage
(仅超管),改配置与手动触发都写审计;
- 顺带补 POST /api/login/batch 的审计(此前批次是谁发起、多少账号无据可查);
- 前端 /automation 页:作业卡片(开关/周期/单轮条数/并发)、运行历史、
「立即跑一次」;总开关未打开、调度线程未运行时给明确提示。
验证:
- 22 条新测试覆盖调度器行为(只跑到点的、异常不抛出、下轮时间、配置不被重置)、
选行条件(在途尝试/auto_managed/间隔/batch_size)、接口与权限,其中一条
专门断言巡检后 cookie/status/message/attempt_status 一字未动;
- 全量 257 条通过;前端 tsc 与构建通过;
- 端到端用 3 个真实账号跑真实斗鱼接口:巡检结果与直接调 check_douyu_cookie
逐条一致(3/3 valid),CK 与尝试状态全程未变。
顺带修掉一处会炸生产的坑:DB 时间列读出来是 naive UTC,直接与
datetime.now(UTC) 比较会 TypeError,现统一走 _as_utc() 补时区。
136 lines
5.1 KiB
Python
136 lines
5.1 KiB
Python
"""自动化作业:作业定义表、运行记录表,以及 CK 巡检所需的退避字段。
|
||
|
||
P1a 只落地「CK 自动巡检」需要的表结构;重登退避(relogin_*)与账号级开关
|
||
(auto_managed)也一并建好,供 P1b 使用,避免之后再改一次表。
|
||
|
||
上一版登录任务把 attempt_* 与 CK 本体拆开,这里延续同一原则:巡检只写
|
||
ck_check_*,绝不碰 cookie / status / attempt_*。
|
||
"""
|
||
|
||
from collections.abc import Sequence
|
||
|
||
import sqlalchemy as sa
|
||
from alembic import op
|
||
|
||
revision: str = "20261002_0037"
|
||
down_revision: str | None = "20261001_0036"
|
||
branch_labels: str | Sequence[str] | None = None
|
||
depends_on: str | Sequence[str] | None = None
|
||
|
||
|
||
def _columns(bind, table: str) -> set[str]:
|
||
return {item["name"] for item in sa.inspect(bind).get_columns(table)}
|
||
|
||
|
||
def _tables(bind) -> set[str]:
|
||
return set(sa.inspect(bind).get_table_names())
|
||
|
||
|
||
def upgrade() -> None:
|
||
bind = op.get_bind()
|
||
|
||
existing_tables = _tables(bind)
|
||
if "automation_jobs" not in existing_tables:
|
||
op.create_table(
|
||
"automation_jobs",
|
||
sa.Column("key", sa.String(length=64), primary_key=True),
|
||
sa.Column(
|
||
"title", sa.String(length=128), nullable=False, server_default=""
|
||
),
|
||
sa.Column(
|
||
"enabled", sa.Boolean(), nullable=False, server_default=sa.text("0")
|
||
),
|
||
sa.Column(
|
||
"interval_seconds",
|
||
sa.Integer(),
|
||
nullable=False,
|
||
server_default="21600",
|
||
),
|
||
sa.Column("batch_size", sa.Integer(), nullable=False, server_default="200"),
|
||
sa.Column("concurrency", sa.Integer(), nullable=False, server_default="5"),
|
||
sa.Column("config", sa.JSON(), nullable=True),
|
||
sa.Column("last_run_at", sa.DateTime(), nullable=True),
|
||
sa.Column("next_run_at", sa.DateTime(), nullable=True),
|
||
sa.Column(
|
||
"last_status", sa.String(length=16), nullable=False, server_default=""
|
||
),
|
||
sa.Column(
|
||
"last_message", sa.String(length=512), nullable=False, server_default=""
|
||
),
|
||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||
)
|
||
if "automation_runs" not in existing_tables:
|
||
op.create_table(
|
||
"automation_runs",
|
||
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
|
||
sa.Column("job_key", sa.String(length=64), nullable=False),
|
||
sa.Column("started_at", sa.DateTime(), nullable=False),
|
||
sa.Column("finished_at", sa.DateTime(), nullable=True),
|
||
sa.Column(
|
||
"status", sa.String(length=16), nullable=False, server_default="running"
|
||
),
|
||
sa.Column("scanned", sa.Integer(), nullable=False, server_default="0"),
|
||
sa.Column("valid", sa.Integer(), nullable=False, server_default="0"),
|
||
sa.Column("invalid", sa.Integer(), nullable=False, server_default="0"),
|
||
sa.Column(
|
||
"relogin_started", sa.Integer(), nullable=False, server_default="0"
|
||
),
|
||
sa.Column(
|
||
"relogin_success", sa.Integer(), nullable=False, server_default="0"
|
||
),
|
||
sa.Column("skipped", sa.Integer(), nullable=False, server_default="0"),
|
||
sa.Column(
|
||
"message", sa.String(length=512), nullable=False, server_default=""
|
||
),
|
||
sa.Column("detail", sa.JSON(), nullable=True),
|
||
)
|
||
op.create_index("ix_automation_runs_job_key", "automation_runs", ["job_key"])
|
||
|
||
login_columns = _columns(bind, "login_tasks")
|
||
if "relogin_attempts" not in login_columns:
|
||
op.add_column(
|
||
"login_tasks",
|
||
sa.Column(
|
||
"relogin_attempts", sa.Integer(), nullable=False, server_default="0"
|
||
),
|
||
)
|
||
if "relogin_next_at" not in login_columns:
|
||
op.add_column(
|
||
"login_tasks", sa.Column("relogin_next_at", sa.DateTime(), nullable=True)
|
||
)
|
||
if "relogin_last_error" not in login_columns:
|
||
op.add_column(
|
||
"login_tasks",
|
||
sa.Column(
|
||
"relogin_last_error",
|
||
sa.String(length=512),
|
||
nullable=False,
|
||
server_default="",
|
||
),
|
||
)
|
||
|
||
account_columns = _columns(bind, "accounts")
|
||
if "auto_managed" not in account_columns:
|
||
op.add_column(
|
||
"accounts",
|
||
sa.Column(
|
||
"auto_managed",
|
||
sa.Boolean(),
|
||
nullable=False,
|
||
server_default=sa.text("1"),
|
||
),
|
||
)
|
||
|
||
|
||
def downgrade() -> None:
|
||
bind = op.get_bind()
|
||
for table in ("automation_runs", "automation_jobs"):
|
||
if table in _tables(bind):
|
||
op.drop_table(table)
|
||
login_columns = _columns(bind, "login_tasks")
|
||
for name in ("relogin_last_error", "relogin_next_at", "relogin_attempts"):
|
||
if name in login_columns:
|
||
op.drop_column("login_tasks", name)
|
||
if "auto_managed" in _columns(bind, "accounts"):
|
||
op.drop_column("accounts", "auto_managed")
|