账号不再依赖登录记录的第一步:把「账号自己的东西」搬到账号上。本提交**只加列、 只读源表**,不改任何行为,单独部署安全。 accounts 新增 15 列: - 登录态:cookie / cookie_updated_at / cookie_source - CK 健康:ck_check_status / ck_check_result / ck_checked_at - 重登进度:relogin_attempts / relogin_next_at / relogin_last_error / relogin_failure_kind - 生命周期:status / status_reason / status_changed_at / disabled_at / archived_at - 索引:ck_checked_at、status、与 (status, relogin_next_at) account_events 新增 5 列(category / subject_type / subject_key / actor_type / actor_id) 与台账复合索引 (account_id, event_type, subject_key, occurred_at) —— 支撑 「365 天同皮肤」这类需要检索客体的规则,让事件表同时承担业务台账。 login_tasks 补 (account_id, attempt_status) 索引:CK 搬到账号后,「在途尝试」只能留在 登录记录上,巡检/重登每轮都要按它过滤。 回填做成生产级: - 只读 login_tasks、分块 200 账号、幂等(只在 accounts.cookie 为空时写); - 窗口函数取「最近一条持 CK 的行」,不用 GROUP_CONCAT(受 max_len 限制会静默截断); - 自校验 fail closed:回填不完整直接抛错让应用起不来,宁可起不来也不带半份 CK 上线; - 外加值级校验,防张冠李戴(条数相等仍可能 A 账号拿到 B 账号的 CK)。 踩坑留档:cookie 是 TEXT,**不能有 server_default**(MySQL 1101: BLOB/TEXT/GEOMETRY/JSON column can not have a default value),模型与迁移两侧都要去掉。 验证: - 空库整链 upgrade head + 模型对齐 + 索引存在 → 通过; - 本机开发库实跑 0041→0042:记录侧 4 个持 CK 账号 = 账号侧 4 个,值级不一致 0, CK 仍是密文(enc:v1: 前缀),状态分布 active 4 / imported 8; - downgrade 到 0041 再 upgrade:数据完整还原(这是 CK 丢失场景的安全网); - 全量 pytest 364 passed / 6 skipped。
131 lines
5.6 KiB
Python
131 lines
5.6 KiB
Python
"""Alembic 迁移链冒烟测试(在真实 MySQL 临时库上执行)。
|
||
|
||
覆盖三件事:
|
||
1. 链完整性:只有一个头版本、头与最新迁移一致、每个修订的
|
||
down_revision 都能回溯到根(无悬挂引用、无分叉链)。
|
||
2. 全部迁移脚本可编译:防止迁移文件被改坏后部署流程才暴露。
|
||
3. 空库整链执行:迁移脚本在真实 MySQL 临时库上执行 `upgrade head`,
|
||
随后校验表/列/索引与 Base.metadata 模型元数据对齐(模型要求的不允许缺失),
|
||
并确认迁移新增的复合查询索引确实落库。
|
||
"""
|
||
|
||
import secrets
|
||
from pathlib import Path
|
||
from urllib.parse import quote_plus
|
||
|
||
from alembic import command
|
||
from alembic.config import Config
|
||
from alembic.script import ScriptDirectory
|
||
from sqlalchemy import create_engine, inspect, text
|
||
|
||
import web.backend.database as database_module
|
||
from tests.support.db import get_mysql_credentials, get_test_engine
|
||
from web.backend.database import Base
|
||
|
||
TESTS_DIR = Path(__file__).resolve().parent
|
||
PROJECT_ROOT = TESTS_DIR.parent
|
||
VERSIONS_DIR = PROJECT_ROOT / "web" / "backend" / "migrations" / "versions"
|
||
|
||
HEAD_REVISION = "20261003_0042"
|
||
|
||
# 迁移新增、但不声明在模型里的复合查询索引。
|
||
EXTRA_INDEXES = (
|
||
"ix_douyu_tasks_handbook_scope_id",
|
||
"ix_douyu_tasks_handbook_scope_task_type_id",
|
||
"ix_douyu_tasks_status_finished_at_id",
|
||
"ix_login_tasks_status_finished_at_id",
|
||
# 账号时间线按「账号 + 时间倒序」查,模型里不声明这个复合索引。
|
||
"ix_account_events_account_occurred",
|
||
# CK 搬到账号后,「在途尝试」只能留在登录记录上,巡检/重登每轮按
|
||
# (account_id, attempt_status) 过滤 —— 模型里不声明这个索引。
|
||
"ix_login_tasks_account_attempt_status",
|
||
)
|
||
|
||
|
||
def _migration_config() -> Config:
|
||
config = Config(str(PROJECT_ROOT / "alembic.ini"))
|
||
config.set_main_option(
|
||
"script_location", str(PROJECT_ROOT / "web" / "backend" / "migrations")
|
||
)
|
||
return config
|
||
|
||
|
||
class TestMigrationSmoke:
|
||
def test_chain_is_linear_and_head_matches_latest_file(self):
|
||
directory = ScriptDirectory.from_config(_migration_config())
|
||
|
||
heads = directory.get_heads()
|
||
assert len(heads) == 1, f"迁移链出现多个头版本(存在分叉): {heads}"
|
||
assert heads[0] == HEAD_REVISION, f"头版本 {heads[0]} 与最新迁移不一致"
|
||
|
||
version_files = sorted(path.name for path in VERSIONS_DIR.glob("*.py"))
|
||
revisions = list(directory.walk_revisions())
|
||
revision_ids = {revision.revision for revision in revisions}
|
||
assert len(revision_ids) == len(version_files), (
|
||
"迁移文件数量与已加载修订数量不一致(存在无法加载的脚本)"
|
||
)
|
||
for revision in revisions:
|
||
if revision.down_revision is not None:
|
||
assert revision.down_revision in revision_ids, (
|
||
f"{revision.revision} 引用了不存在的父版本 {revision.down_revision}"
|
||
)
|
||
|
||
def test_all_migration_scripts_compile(self):
|
||
for path in sorted(VERSIONS_DIR.glob("*.py")):
|
||
source = path.read_text(encoding="utf-8")
|
||
compile(source, str(path), "exec") # 语法/缩进等编译错误会在此抛出
|
||
|
||
def test_full_chain_applies_on_fresh_mysql_and_matches_models(self):
|
||
"""空库执行整条迁移链,校验表/列/索引与模型元数据对齐。"""
|
||
creds = get_mysql_credentials()
|
||
mig_db = f"douyu_login_mig_{secrets.token_hex(4)}"
|
||
mig_url = f"mysql+pymysql://{quote_plus(creds['user'])}:{quote_plus(creds['password'])}@{creds['host']}:{creds['port']}/{mig_db}?charset=utf8mb4"
|
||
|
||
admin_engine = get_test_engine()
|
||
with admin_engine.begin() as conn:
|
||
conn.execute(
|
||
text(
|
||
f"CREATE DATABASE `{mig_db}` CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;"
|
||
)
|
||
)
|
||
|
||
original_url = database_module.DATABASE_URL
|
||
try:
|
||
database_module.DATABASE_URL = mig_url
|
||
cfg = _migration_config()
|
||
cfg.set_main_option("sqlalchemy.url", mig_url)
|
||
command.upgrade(cfg, "head")
|
||
|
||
inspector = inspect(create_engine(mig_url))
|
||
db_tables = set(inspector.get_table_names())
|
||
assert db_tables
|
||
|
||
model_tables = Base.metadata.tables
|
||
missing_tables = set(model_tables) - db_tables
|
||
assert not missing_tables
|
||
|
||
all_db_indexes = set()
|
||
for name, table in model_tables.items():
|
||
db_columns = {column["name"] for column in inspector.get_columns(name)}
|
||
model_columns = set(table.columns.keys())
|
||
missing_columns = model_columns - db_columns
|
||
assert not missing_columns
|
||
|
||
db_indexes = set()
|
||
for index in inspector.get_indexes(name):
|
||
index_name = index["name"]
|
||
if index_name:
|
||
db_indexes.add(index_name)
|
||
all_db_indexes |= db_indexes
|
||
model_indexes = {index.name for index in table.indexes if index.name}
|
||
missing_indexes = model_indexes - db_indexes
|
||
assert not missing_indexes
|
||
|
||
# 迁移新增的复合查询索引(不入模型,但应存在)。
|
||
for index_name in EXTRA_INDEXES:
|
||
assert index_name in all_db_indexes, f"迁移链未创建索引 {index_name}"
|
||
finally:
|
||
database_module.DATABASE_URL = original_url
|
||
with admin_engine.begin() as conn:
|
||
conn.execute(text(f"DROP DATABASE IF EXISTS `{mig_db}`;"))
|