yml2213
00125fbfc4
S1 加好了列，本提交把**读与写一起切**过去。这是设计里「同一份数据的读与写必须
在同一个提交里切」的要求：先切读或先切写都会出现「写账号、读记录」的中间态，
跑起来不报错，只是默默用错数据。
## 核心改动
**判定「谁有 CK」不再借道登录记录**
- `cookie_holding_filter()` / `cookie_operable_filter()` / `cookie_holding_account_ids_query()`
/ `visible_douyu_task_accounts()` 全部改为作用于 `Account`；
- `cookie_operable_filter()` 的「在途排除」仍查 `login_tasks`（那是**某次尝试**的属性），
用 `NOT EXISTS` 子查询 + 迁移 0042 新增的 `(account_id, attempt_status)` 索引；
- 删除死代码 `latest_cookie_login_task()`；`latest_cookie()` 改为直接读账号。
**CK 写入**
- `_write_login_success()` 写 `Account.cookie / cookie_updated_at / cookie_source`；
- `_mark_attempt_failed()` 改看**账号**有没有 CK（决定是否沿用旧的失败语义）；
- 检测模式的 `task.cookie = ""` **不再清空账号的 CK**（见下「行为变更」）。
**自动化作业按账号调度**
- `ck_health`：`_select_tasks` → `_select_accounts`，`db.query(Account)`；新增
`count_remaining()`（刻意不带 LIMIT，否则积压数恒等于批次上限）；
- `ck_relogin`：同样按账号选行；结算改为「账号 + 本轮尝试记录」，
退避进度写账号；`settle_fresh_results(batch_id)` 按 `attempt_batch_id` 配对，
`run_ck_relogin` 把 **account_ids** 交给 runner；
- 顺带让**没有任何登录历史的账号**也能被巡检/重登（以前它们永远进不了队列）。
**登录批次**
- `LoginBatchRunner` 去掉 `relogin_task_ids`，重登与登录统一走 `account_ids`；
- 批次行复用逻辑重写：只保证「同一账号不并发两行」，不再关心「哪一行持有 CK」。
失败行复用同时看 `attempt_status`（账号有 CK 时 `_mark_attempt_failed` 刻意不动
`status`，只看 status 会永远命不中 → 每次失败刷一行）。
**删除护栏语义变更（用户可感知）**
- 去掉「持有 CK 的记录不可删」——CK 在账号上，删历史不再影响登录态，历史终于可清理；
- **保留**「在途尝试不可删」（批次线程正拿着那条记录）。
## 行为变更（需评审）
1. **账号检测不再清空 CK**。旧代码在 check 模式写 `task.cookie = ""`，而那时记录的
cookie 就是账号唯一凭证 —— 一次「批量检测」会把账号登录态删掉。解耦后
「检测账号凭据」与「CK 是否有效」本就是两件事（后者归 `ck_health` 判），
继续清空等于凭空制造不可逆的数据丢失（重登当前成功率极低）。
2. **Cookie 页的记录单位从「登录记录」变成「账号」**，所有接口的 id 入参/出参
变成 `account_id`；`DELETE /cookies/{id}` 清的是该账号的 CK，**必写审计**。
3. 重复检测从「同一账号多行持 CK」改为「**同一个 CK 出现在多个账号上**」——
后者才是真风险（两个账号共用一个登录态）。用 sha256 指纹分组：
CK 是 AES-GCM + 随机 nonce 加密，**同一明文每次密文都不同**，比密文永远比不出重复。
## 修掉一个真实并发缺陷
`_write_login_success` 原先被传入 `self.db`（runner 主会话），而它是在**工作线程**里
用 `worker_db` 执行的。`emit()` 会往 `account_events` 插一行，其 `account_id` 外键对
`accounts` 那一行加**共享锁**；这把锁加在另一个会话上，紧接着本会话的
`UPDATE accounts SET cookie=...` 就被自己人挡住 —— 实测 **50 秒后
`Lock wait timeout exceeded`，登录明明成功却写不回 CK**（改造前 UPDATE 打在
`login_tasks` 上所以没撞上）。改为传 `worker_db`：事件本就该和状态变更同事务提交，
顺带修掉跨线程共用一个 Session 的隐患。
## 验证
- 全量 pytest **368 passed / 6 skipped**；迁移链 3 passed；
- 前端 `tsc -b && vite build` 通过；
- `ruff check` / `ruff format --check` 全绿（顺带修掉 main 上遗留的 F821）；
- 新增/改写的关键回归：**删掉全部登录历史后账号的 CK 与健康状态仍在**、
**无登录历史的账号也能进巡检队列**、结算不碰 CK、删除护栏改为在途语义。
仍未做（设计里排期的后续阶段）：`account_lifecycle` 状态收口（S3）、事件发射点补齐（S4）、
业务事实直发事件（S5）、账号页重构（S6）。注意 `status` 目前只有模型默认值与迁移回填，
所以 `disabled/archived` 还没有任何写入方——S3 落地时要把自动化选行的状态过滤一起补上。
389 lines
13 KiB
Python
389 lines
13 KiB
Python
"""回归：登录任务列表分页、跨页全选删除、以及「不删在途任务」的护栏。
|
||
|
||
任务量早就过了单页能装下的规模（库里 1.4 万条、单批就 177 条），前端只能看
|
||
200 条于是「全选」其实选不到全部。分页 + ``all_matching`` 是为了让「清空失败
|
||
任务」这类操作真能一键做完。
|
||
|
||
三条必须守住：
|
||
|
||
1. **不传 page 时返回裸数组** —— Cookie 页等老调用方依赖这个契约，改了就崩；
|
||
2. **传 page 才返回 {items,total}**，total 用于跨页全选；
|
||
3. **pending/running 一律不删** —— 删掉会让批次线程找不到记录。要停批次
|
||
走 ``/stop/{batch_id}``，不是删除。
|
||
"""
|
||
|
||
import pytest
|
||
from sqlalchemy.orm import sessionmaker
|
||
|
||
import web.backend.database as database_module
|
||
from tests.support.db import get_test_engine, make_test_session
|
||
from web.backend.models import Account, LoginTask, User
|
||
from web.backend.routers.login import delete_task, delete_tasks, list_tasks
|
||
|
||
|
||
class TestTaskPagination:
|
||
def setup_method(self):
|
||
self.engine = get_test_engine()
|
||
database_module.SessionLocal.configure(bind=self.engine)
|
||
self.session = make_test_session(self.engine)
|
||
|
||
self.user = User(username="admin", password_hash="h", role="super_admin")
|
||
self.session.add(self.user)
|
||
self.session.flush()
|
||
|
||
self.account = Account(
|
||
username="分页测试账号",
|
||
password="p",
|
||
email="paged@example.com",
|
||
email_password="ep",
|
||
)
|
||
self.session.add(self.account)
|
||
self.session.flush()
|
||
|
||
# 建 12 条：7 条 failed、5 条 success，用于验证筛选与分页
|
||
self.task_ids = []
|
||
for i in range(12):
|
||
task = LoginTask(
|
||
batch_id="paged-batch",
|
||
account_id=self.account.id,
|
||
created_by=self.user.id,
|
||
status="failed" if i < 7 else "success",
|
||
message=f"任务{i}",
|
||
)
|
||
self.session.add(task)
|
||
self.session.flush()
|
||
self.task_ids.append(task.id)
|
||
self.session.commit()
|
||
|
||
def teardown_method(self):
|
||
self.session.close()
|
||
|
||
def _fresh(self):
|
||
return sessionmaker(bind=self.engine)()
|
||
|
||
def test_without_page_returns_plain_list_for_backward_compat(self):
|
||
"""老调用方（Cookie 页）拿到的必须是数组，不能是 {items,...}。"""
|
||
db = self._fresh()
|
||
try:
|
||
rows = list_tasks(db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert isinstance(rows, list)
|
||
assert len(rows) == 12
|
||
|
||
def test_with_page_returns_items_and_total(self):
|
||
db = self._fresh()
|
||
try:
|
||
data = list_tasks(page=1, page_size=5, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert isinstance(data, dict)
|
||
assert data["total"] == 12
|
||
assert data["page"] == 1
|
||
assert len(data["items"]) == 5
|
||
|
||
def test_pages_do_not_overlap_and_cover_all(self):
|
||
db = self._fresh()
|
||
try:
|
||
seen = []
|
||
for page in (1, 2, 3):
|
||
data = list_tasks(page=page, page_size=5, db=db, current=self.user)
|
||
seen.extend(item.id for item in data["items"])
|
||
finally:
|
||
db.close()
|
||
assert len(seen) == 12
|
||
assert len(set(seen)) == 12, "分页出现重复"
|
||
|
||
def test_status_filter_narrows_total(self):
|
||
"""total 必须是**筛选后**的总数，否则跨页全选会多删。"""
|
||
db = self._fresh()
|
||
try:
|
||
data = list_tasks(
|
||
page=1, page_size=5, status="failed", db=db, current=self.user
|
||
)
|
||
finally:
|
||
db.close()
|
||
assert data["total"] == 7
|
||
assert all(item.status == "failed" for item in data["items"])
|
||
|
||
def test_page_beyond_range_is_empty(self):
|
||
db = self._fresh()
|
||
try:
|
||
data = list_tasks(page=99, page_size=5, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert data["items"] == []
|
||
assert data["total"] == 12
|
||
|
||
|
||
class TestBulkDelete:
|
||
def setup_method(self):
|
||
self.engine = get_test_engine()
|
||
database_module.SessionLocal.configure(bind=self.engine)
|
||
self.session = make_test_session(self.engine)
|
||
|
||
self.user = User(username="admin2", password_hash="h", role="super_admin")
|
||
self.session.add(self.user)
|
||
self.session.flush()
|
||
self.account = Account(
|
||
username="删除测试账号",
|
||
password="p",
|
||
email="del@example.com",
|
||
email_password="ep",
|
||
)
|
||
self.session.add(self.account)
|
||
self.session.flush()
|
||
|
||
self.failed_ids = []
|
||
self.pending_ids = []
|
||
for i in range(5):
|
||
for status, bucket in (
|
||
("failed", self.failed_ids),
|
||
("pending", self.pending_ids),
|
||
):
|
||
task = LoginTask(
|
||
batch_id="del-batch",
|
||
account_id=self.account.id,
|
||
created_by=self.user.id,
|
||
status=status,
|
||
message=f"{status}{i}",
|
||
)
|
||
self.session.add(task)
|
||
self.session.flush()
|
||
bucket.append(task.id)
|
||
self.session.commit()
|
||
|
||
def teardown_method(self):
|
||
self.session.close()
|
||
|
||
def _fresh(self):
|
||
return sessionmaker(bind=self.engine)()
|
||
|
||
def _remaining(self):
|
||
db = self._fresh()
|
||
try:
|
||
return {
|
||
row[0]
|
||
for row in db.query(LoginTask.id).filter(
|
||
LoginTask.batch_id == "del-batch"
|
||
)
|
||
}
|
||
finally:
|
||
db.close()
|
||
|
||
def test_delete_by_ids(self):
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(
|
||
task_ids=",".join(str(i) for i in self.failed_ids[:3]),
|
||
db=db,
|
||
current=self.user,
|
||
)
|
||
finally:
|
||
db.close()
|
||
assert res["deleted"] == 3
|
||
remaining = self._remaining()
|
||
assert all(i not in remaining for i in self.failed_ids[:3])
|
||
|
||
def test_delete_all_matching_status(self):
|
||
"""跨页全选：删掉筛选条件下的全部 failed。"""
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(
|
||
status="failed", all_matching=True, db=db, current=self.user
|
||
)
|
||
finally:
|
||
db.close()
|
||
assert res["deleted"] == 5
|
||
remaining = self._remaining()
|
||
assert all(i not in remaining for i in self.failed_ids)
|
||
assert all(i in remaining for i in self.pending_ids)
|
||
|
||
def test_delete_all_matching_never_touches_in_flight(self):
|
||
"""核心护栏：pending/running 必须留下，否则批次线程会找不到记录。"""
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(all_matching=True, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
# 只有 5 条 failed 可删，5 条 pending 被护栏挡住
|
||
assert res["deleted"] == 5
|
||
remaining = self._remaining()
|
||
assert all(i in remaining for i in self.pending_ids)
|
||
|
||
def test_delete_without_criteria_rejected(self):
|
||
"""不给条件就拒绝 —— 绝不默认全删。"""
|
||
db = self._fresh()
|
||
try:
|
||
with pytest.raises(Exception) as exc:
|
||
delete_tasks(db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert "请指定任务ID" in str(exc.value)
|
||
|
||
def test_delete_invalid_ids_rejected(self):
|
||
db = self._fresh()
|
||
try:
|
||
with pytest.raises(Exception) as exc:
|
||
delete_tasks(task_ids="abc,xyz", db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert "无效的任务ID" in str(exc.value)
|
||
|
||
|
||
class TestDeleteGuard:
|
||
"""⚠️ 护栏语义已随 CK 解耦改变（2026-10-04）。
|
||
|
||
旧护栏：「持有 CK 的记录不可删」—— 因为那时 CK 只存在 `login_tasks.cookie`，
|
||
删历史就是删账号的登录态（2026-10-03 事故）。
|
||
|
||
新护栏：CK 已迁到 `accounts.cookie`，**删除历史不再影响任何账号的登录态**，
|
||
所以 CK 不再是护栏理由（历史终于可以自由清理）。保留的是**在途护栏**：
|
||
批次线程正拿着那条记录，删掉会让它找不到目标。
|
||
"""
|
||
|
||
def setup_method(self):
|
||
self.engine = get_test_engine()
|
||
database_module.SessionLocal.configure(bind=self.engine)
|
||
self.session = make_test_session(self.engine)
|
||
|
||
self.user = User(username="ck-admin", password_hash="h", role="super_admin")
|
||
self.session.add(self.user)
|
||
self.session.flush()
|
||
self.account = Account(
|
||
username="CK账号",
|
||
password="p",
|
||
email="ck@example.com",
|
||
email_password="ep",
|
||
cookie="valid-ck-1",
|
||
)
|
||
self.session.add(self.account)
|
||
self.session.flush()
|
||
|
||
# ① 历史：显示失败、但账号仍持有有效 CK（本次改造前会被当成垃圾删掉）
|
||
self.failed_history = self._add(status="failed", attempt_status="failed")
|
||
# ② 历史：成功
|
||
self.success_history = self._add(status="success", attempt_status="success")
|
||
# ③ 在途：批次正拿着它，不可删
|
||
self.in_flight = self._add(
|
||
status="pending", attempt_status="running", attempt_batch_id="live-batch"
|
||
)
|
||
|
||
self.other = Account(
|
||
username="CK账号2",
|
||
password="p",
|
||
email="ck2@example.com",
|
||
email_password="ep",
|
||
cookie="valid-ck-2",
|
||
)
|
||
self.session.add(self.other)
|
||
self.session.flush()
|
||
self.other_history = self._add(
|
||
status="success", attempt_status="failed", account_id=self.other.id
|
||
)
|
||
self.session.commit()
|
||
|
||
def _add(
|
||
self, *, status, attempt_status, account_id=None, attempt_batch_id="ck-batch"
|
||
):
|
||
task = LoginTask(
|
||
batch_id="ck-batch",
|
||
account_id=account_id or self.account.id,
|
||
created_by=self.user.id,
|
||
status=status,
|
||
attempt_status=attempt_status,
|
||
attempt_batch_id=attempt_batch_id,
|
||
message="x",
|
||
)
|
||
self.session.add(task)
|
||
self.session.flush()
|
||
return task.id
|
||
|
||
def teardown_method(self):
|
||
self.session.close()
|
||
|
||
def _fresh(self):
|
||
return sessionmaker(bind=self.engine)()
|
||
|
||
def _exists(self, task_id):
|
||
db = self._fresh()
|
||
try:
|
||
return (
|
||
db.query(LoginTask).filter(LoginTask.id == task_id).first() is not None
|
||
)
|
||
finally:
|
||
db.close()
|
||
|
||
def _account_cookie(self, account_id: int) -> str:
|
||
db = self._fresh()
|
||
try:
|
||
return db.query(Account).filter(Account.id == account_id).one().cookie
|
||
finally:
|
||
db.close()
|
||
|
||
def test_bulk_delete_removes_history_even_when_account_holds_cookie(self):
|
||
"""核心验收：删掉全部历史，账号的 CK 一点不受影响。"""
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(all_matching=True, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
|
||
assert not self._exists(self.failed_history)
|
||
assert not self._exists(self.success_history)
|
||
assert not self._exists(self.other_history)
|
||
# 在途那条被跳过
|
||
assert self._exists(self.in_flight)
|
||
assert res["deleted"] == 3
|
||
assert res["skipped"] == 1
|
||
# 账号的 CK 仍在 —— 这就是解耦要的效果
|
||
assert self._account_cookie(self.account.id) == "valid-ck-1"
|
||
assert self._account_cookie(self.other.id) == "valid-ck-2"
|
||
|
||
def test_bulk_delete_reports_in_flight_skipped_count(self):
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(all_matching=True, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert res["deleted"] == 3
|
||
assert res["skipped"] == 1
|
||
assert "在途" in res["message"]
|
||
|
||
def test_explicit_ids_can_delete_cookie_history_but_not_in_flight(self):
|
||
"""用户手动勾选时，历史可删，在途仍不可删。"""
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_tasks(
|
||
task_ids=f"{self.failed_history},{self.in_flight}",
|
||
db=db,
|
||
current=self.user,
|
||
)
|
||
finally:
|
||
db.close()
|
||
assert not self._exists(self.failed_history)
|
||
assert self._exists(self.in_flight)
|
||
assert res["deleted"] == 1
|
||
assert res["skipped"] == 1
|
||
assert self._account_cookie(self.account.id) == "valid-ck-1"
|
||
|
||
def test_single_delete_refuses_in_flight_record(self):
|
||
db = self._fresh()
|
||
try:
|
||
with pytest.raises(Exception) as exc:
|
||
delete_task(self.in_flight, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert "正在被批次处理" in str(exc.value)
|
||
assert self._exists(self.in_flight)
|
||
|
||
def test_single_delete_allows_cookie_history(self):
|
||
db = self._fresh()
|
||
try:
|
||
res = delete_task(self.success_history, db=db, current=self.user)
|
||
finally:
|
||
db.close()
|
||
assert res["success"] is True
|
||
assert not self._exists(self.success_history)
|
||
assert self._account_cookie(self.account.id) == "valid-ck-1"
|