refactor(结构): 分层重组 + 清除全部废弃路线
做了什么:
- 包改为分层: tcaptcha/{protocol,telemetry,solvers,ocr,adapters,models,tests}
* protocol/ client.py transport.py proxy.py
* telemetry/ crypto.py values.py key.py opcodes.py
disasm.py program.py layout.py modules.py
* solvers/ text_click.py icon_click.py icon_click_rotate.py slider.py
* ocr/ engines.py slicer.py
* adapters/ one_ms.py
- tools/ 里原本是**包运行时依赖**的 tdc_{disasm,program,layout,modules}.py 并入 telemetry/
（放在 tools/ 属于分层倒置），tools/ 只留 CLI: tdc_key_pure.py / tdc_profile.py / cdp_*.js
- crypto.py 529 -> 340 行: 切除「成功样本模板回填」与「Node 插桩兜底」两条老路；
generate_telemetry 只剩 python_values 一种模式，传别的会直接报错并说明原因
- 求解器改名分清: solver.py -> solvers/icon_click.py（client 实际在用的 YOLO+ResNet18 ONNX），
icon_click_solver.py -> solvers/icon_click_rotate.py（仅测试在用，留作对照）
- pyproject: 去掉 ultralytics/onnx（训练栈已删），补上真正在用的 onnxruntime
删除了什么:
- 雨云整条线 + dist/rainyun_qinglong（104M 旧快照）
- 模板回填: models/success_collect_template.b64、tdc_worker.js、analyze_collect_layout.py
- 「补环境」时代浏览器 JS: chrome_fetch/manual_captcha/probe_captcha/solve_in_browser/tdc_browser
- 旧 VM 探索工具 10 个(tdc_collectgen/tdc_slotmap/tdc_lift/tdc_vm_run/tdc_query/tdc_probe 等)
+ 8 个基于它们的旧 verify 脚本
- 训练/标注/评估: train_classifier/train_detector/prepare_yolo/web_labeler/collector/
siamese/evaluate/benchmark_verify
- tcaptcha/core_js/（60 个分析产物）、TOOLCHAIN_REPORT.md（整篇在描述已删工具）
- 3 个已无任何引用的 PyTorch 权重：icon_classifier.pt 43M / yolo_detector.pt 5.9M /
siamese_matcher.pt 1.6M —— 线上只跑 onnx，tcaptcha/ 117M -> 66M
为什么:
- 项目后面要专门承载「滑块 / 文字点选 / 图标点选」三种求解器；扁平包 + 运行时依赖混在
tools/ + 两条已废弃的 collect 生成路线，会让后续扩展持续踩坑
怎么验证:
- .venv/bin/python -m tcaptcha.selftest 全绿
- scripts/verify/verify_offline.py 逐槽全等 + 分段几何一致
- scripts/verify/verify_values.py 4 份真机明文 152/152 逐槽全等
- scripts/verify/send_test.py --mode python_values 1ms 签到页文字点选 errorCode=0
- 回滚点: tag backup/pre-refactor-20261012
This commit is contained in:
yml2213
@@ -0,0 +1,12 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""遥测层：collect / eks / XTEA 密钥的**纯算**还原。
|
||||
|
||||
crypto.py XTEA（教科书 32 轮）+ collect 四段组装与切分 + 遥测生成入口
|
||||
values.py 37 个语义名的取值生成器（静态画像 / 会话量 / 每次采集量）
|
||||
key.py 从 tdc.js 静态还原 4 个 XTEA 密钥字（零 Node）
|
||||
opcodes.py 按构建推导 Chaos VM 的 opcode 表（opcode 编号每份构建重排）
|
||||
disasm.py 字节码提取 + CFG 反汇编
|
||||
program.py VM 程序模型（函数体 / 闭包 / 字符串表 / 模块注册表）
|
||||
layout.py 清单（manifest）定位与 require 顺序导出
|
||||
modules.py 按字符串指纹把 37 个采集器判成语义名
|
||||
"""
|
||||
@@ -0,0 +1,338 @@
|
||||
"""
|
||||
腾讯 TDC 遥测 (collect) 纯算法还原 —— 唯一路径: **按本次会话构建纯离线生成** + 教科书 XTEA。
|
||||
|
||||
设计原则:
|
||||
* **纯离线**: 37 个语义名的取值由 `telemetry/values.py` 生成（静态画像 / 会话量 /
|
||||
每次采集量三层），collect 的明文组装 / 轨迹 / 时间戳 / XTEA 加密 / 编码全部在
|
||||
Python 内完成。零 Node、零真机明文。
|
||||
* **密钥也纯 Python**: 从 tdc.js 静态反汇编还原 4 个密钥字（见 `telemetry/key.py`）。
|
||||
* **必须用本次会话那一份构建**（实测，账本 #58/#59）: 同一 aid 下，
|
||||
"按本次构建生成" -> errorCode=0；"拿别处模板回填" -> 恒 12。
|
||||
所以历史上有过的两条老路（成功样本模板回填 / Node 插桩兜底）已整体删除。
|
||||
|
||||
本模块只负责: XTEA 本体 + collect 的四段组装/切分 + 遥测入口。
|
||||
"""
|
||||
|
||||
import base64
|
||||
import itertools
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import struct
|
||||
import urllib.parse
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
M32 = 0xFFFFFFFF
|
||||
|
||||
|
||||
logger = logging.getLogger("tcaptcha_crypto")
|
||||
|
||||
# 样本构建的 XTEA 密钥字 (未加盐)。真实样本的等价密钥 (含两个轮加盐) 见
|
||||
DEFAULT_KEY = [1448563793, 1847730015, 1733184827, 1364613714]
|
||||
|
||||
DEFAULT_USER_AGENT = (
|
||||
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/155.0.0.0 Safari/537.36"
|
||||
)
|
||||
|
||||
|
||||
def encrypt_string(plaintext: str, key: List[int] = DEFAULT_KEY) -> str:
|
||||
"""把 ASCII/Latin1 明文用教科书 XTEA 加密后转 Base64 (不足 8 字节用 \\x00 补齐)。
|
||||
|
||||
本模块只有一份 xtea_encrypt_block (教科书实现), 此函数即使用它。
|
||||
"""
|
||||
data = plaintext.encode("latin1")
|
||||
pad_len = (8 - (len(data) % 8)) % 8
|
||||
data += b"\x00" * pad_len
|
||||
|
||||
out = bytearray()
|
||||
for i in range(0, len(data), 8):
|
||||
v0, v1 = struct.unpack("<II", data[i : i + 8])
|
||||
ev0, ev1 = xtea_encrypt_block(v0, v1, key)
|
||||
out.extend(struct.pack("<II", ev0, ev1))
|
||||
|
||||
return base64.b64encode(out).decode("ascii")
|
||||
|
||||
|
||||
def extract_eks(tdc_js_code: str) -> str:
|
||||
"""从 tdc.js 中提取会话密钥信封 eks (一个较长的 Base64 字符串)。
|
||||
|
||||
注意: tdc.js 每次构建都会重命名全局变量 (见 AGENTS.md 附录 A#6), 因此**不能**写死变量名;
|
||||
这里匹配任意 `window.<随机名> = '<base64>'`, 并取长度最长的候选。
|
||||
"""
|
||||
cands = re.findall(r"window\.[A-Za-z_$][\w$]*\s*=\s*'([A-Za-z0-9+/=]{100,})'", tdc_js_code)
|
||||
if not cands:
|
||||
return ""
|
||||
return max(cands, key=len)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 标准 (教科书) XTEA: 32 轮, delta=0x9E3779B9, sum 取 mod 2^32。
|
||||
# 这是本模块**唯一**的 XTEA 实现; 需要从每份 tdc.js 里拿到的是那 4 个密钥字。
|
||||
# --------------------------------------------------------------------------- #
|
||||
XTEA_DELTA = 0x9E3779B9
|
||||
|
||||
|
||||
def xtea_encrypt_block(v0: int, v1: int, key: List[int]) -> Tuple[int, int]:
|
||||
"""教科书 XTEA 加密 (32 轮), 全 32 位无符号运算"""
|
||||
v0 &= M32
|
||||
v1 &= M32
|
||||
total = 0
|
||||
for _ in range(32):
|
||||
v0 = (v0 + (((((v1 << 4) & M32) ^ (v1 >> 5)) + v1) & M32
|
||||
^ ((total + key[total & 3]) & M32))) & M32
|
||||
total = (total + XTEA_DELTA) & M32
|
||||
v1 = (v1 + (((((v0 << 4) & M32) ^ (v0 >> 5)) + v0) & M32
|
||||
^ ((total + key[(total >> 11) & 3]) & M32))) & M32
|
||||
return v0, v1
|
||||
|
||||
|
||||
def xtea_decrypt_block(v0: int, v1: int, key: List[int]) -> Tuple[int, int]:
|
||||
"""教科书 XTEA 解密 (32 轮), 用于校验提取到的密钥"""
|
||||
v0 &= M32
|
||||
v1 &= M32
|
||||
total = (XTEA_DELTA * 32) & M32
|
||||
for _ in range(32):
|
||||
v1 = (v1 - (((((v0 << 4) & M32) ^ (v0 >> 5)) + v0) & M32
|
||||
^ ((total + key[(total >> 11) & 3]) & M32))) & M32
|
||||
total = (total - XTEA_DELTA) & M32
|
||||
v0 = (v0 - (((((v1 << 4) & M32) ^ (v1 >> 5)) + v1) & M32
|
||||
^ ((total + key[total & 3]) & M32))) & M32
|
||||
return v0, v1
|
||||
|
||||
|
||||
def xtea_encrypt_bytes(data: bytes, key: List[int]) -> bytes:
|
||||
"""XTEA 加密一段字节；不足 8 字节用 \\x00 补齐（XTEA 块对齐所需）。"""
|
||||
data = data + b"\x00" * ((8 - len(data) % 8) % 8)
|
||||
out = bytearray()
|
||||
for i in range(0, len(data), 8):
|
||||
v0, v1 = struct.unpack("<II", data[i:i + 8])
|
||||
e0, e1 = xtea_encrypt_block(v0, v1, key)
|
||||
out += struct.pack("<II", e0, e1)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# collect 的「四段加密」结构（按 docs/tdc-案例.md §5.1，真机明文已判决） #
|
||||
# --------------------------------------------------------------------------- #
|
||||
SEG_ALIGN = 24
|
||||
SEG_PAD = b" "
|
||||
|
||||
# 候选 pad = 一段空格。⚠️ **pad 可以只有 1 个空格**（段内容长 % 24 == 23 时），
|
||||
# 2026-10-12 在真机上实测到这种样本（chunk1 = 287 字节 -> pad 1）。
|
||||
# 所以**不能**用 ` {2,}` 去找 pad，那个正则会把 `{"cd":[...,0,0,0, [[1,1,12]]...` 里的
|
||||
# 单空格 pad 整段漏掉，切段结果直接崩。判据必须用长度模 24 的算术。
|
||||
_SpaceRun = re.compile(rb" +")
|
||||
|
||||
|
||||
def xtea_encrypt_segment(text: str, key: List[int],
|
||||
align: int = SEG_ALIGN, pad: bytes = SEG_PAD) -> bytes:
|
||||
"""加密 collect 的一个**数据段**：空格 pad 到 align 字节对齐后 XTEA + Base64。
|
||||
|
||||
为什么要 pad 到 24（而不是 8）：24 字节 -> 32 个 Base64 字符，且**不会产生 `=` 填充**。
|
||||
4 段密文的 Base64 直接拼接后看起来就是一个完整的 Base64 串，没有分隔符。
|
||||
实测 pad 长度落在 **1..24**（真机样本里见过 15/8/23 与 1/14/4）——
|
||||
注意 **1 个空格的 pad 是真实存在的**，别用 ` {2,}` 去找它（见 split_collect_segments）。
|
||||
"""
|
||||
data = text.encode("latin1")
|
||||
# 注意：即使 len%align==0 也要补一整个 align（真机 chunk2 内容 288 字节、
|
||||
# 288%24==0，却补了 24 个空格 -> pad 长度恒为 align - len%align，落在 1..align）
|
||||
data += pad * (align - len(data) % align)
|
||||
return base64.b64encode(xtea_encrypt_bytes(data, key))
|
||||
|
||||
|
||||
def xtea_encrypt_segment_raw(text: str, key: List[int]) -> bytes:
|
||||
"""加密 **sd 段**：不 pad 到 24，只由 XTEA 的 8 字节块补齐（真机实测如此）。"""
|
||||
data = text.encode("latin1")
|
||||
return base64.b64encode(xtea_encrypt_bytes(data, key))
|
||||
|
||||
|
||||
def build_collect(chunk1: str, trajectory: str, chunk2: str, sd: str,
|
||||
key: List[int], url_encode: bool = False) -> str:
|
||||
"""按案例 §5.1 组装 collect —— **4 段密文直接拼接，无分隔符**。
|
||||
|
||||
collect = b64(xtea(chunk1 + 空格pad到24))
|
||||
+ b64(xtea(traj + 空格pad到24))
|
||||
+ b64(xtea(chunk2 + 空格pad到24))
|
||||
+ b64(xtea(sd)) # sd 不 pad
|
||||
|
||||
段的内容边界（案例 §5.5，与真机明文逐字吻合）：
|
||||
chunk1 = '{"cd":[' + 轨迹之前的指纹值 + ','
|
||||
trajectory = '[[x,y,t],[dx,dy,dt],...,[1,1,12]]'
|
||||
chunk2 = ',' + 轨迹之后的指纹值 + '],'
|
||||
sd = '"sd":{"od":"C","ft":"..."}}'
|
||||
"""
|
||||
parts = [
|
||||
xtea_encrypt_segment(chunk1, key),
|
||||
xtea_encrypt_segment(trajectory, key),
|
||||
xtea_encrypt_segment(chunk2, key),
|
||||
xtea_encrypt_segment_raw(sd, key),
|
||||
]
|
||||
b64 = b"".join(parts).decode("ascii")
|
||||
if url_encode:
|
||||
return urllib.parse.quote(b64, safe="")
|
||||
return b64
|
||||
|
||||
|
||||
def _inside_json_string(data: bytes, pos: int) -> bool:
|
||||
"""判断字节偏移 pos 是否落在 JSON 字符串字面量内部（数未转义的双引号奇偶）。"""
|
||||
n = 0
|
||||
i = 0
|
||||
while i < pos:
|
||||
c = data[i]
|
||||
if c == 0x5C: # 反斜杠：跳过被转义的下一个字节
|
||||
i += 2
|
||||
continue
|
||||
if c == 0x22: # 双引号
|
||||
n += 1
|
||||
i += 1
|
||||
return n % 2 == 1
|
||||
|
||||
|
||||
def _candidate_pads(data: bytes) -> List[Tuple[int, int]]:
|
||||
"""候选 pad 位置：空格串 [a,b) 且满足
|
||||
① b % SEG_ALIGN == 0 ② (b-a) == SEG_ALIGN - a % SEG_ALIGN ③ 不落在 JSON 字符串内部
|
||||
|
||||
为什么要 ③：① ② 单独用**会误报** —— 实测真机 UA 串
|
||||
"Mozilla/5.0 (Macintosh…" 里偏移 47 的那个空格恰好满足
|
||||
（47 % 24 == 23 -> 期望 pad 1，48 % 24 == 0），而它只是字符串内的普通空格。
|
||||
段一定切在 JSON 记号边界上，所以真 pad 处引号计数必为偶数；用它把误报滤掉。
|
||||
"""
|
||||
out: List[Tuple[int, int]] = []
|
||||
for m in _SpaceRun.finditer(data):
|
||||
a, b = m.span()
|
||||
if b % SEG_ALIGN or (b - a) != SEG_ALIGN - a % SEG_ALIGN:
|
||||
continue
|
||||
if _inside_json_string(data, a):
|
||||
continue
|
||||
out.append((a, b))
|
||||
return out
|
||||
|
||||
|
||||
def split_collect_segments(plain: str) -> Tuple[str, str, str, str]:
|
||||
"""把一份**已解密**的 collect 明文按空格 pad 切回 4 段（去掉 pad）。
|
||||
|
||||
结构（3 份真机样本逐字节核对，2026-10-12）:
|
||||
chunk1 = '{"cd":[' + 轨迹之前的槽 + ','
|
||||
traj = '<轨迹数组>'
|
||||
chunk2 = ',' + 轨迹之后的槽 + '],'
|
||||
sd = '"sd":{...}}'
|
||||
前三段各用空格 pad 到 24 字节整数倍，sd 不 pad。
|
||||
|
||||
做法：先按模 24 算术列出**候选** pad，再对候选三元组做结构校验
|
||||
（首段以 '{"cd":[' 开头、末段以 '"sd"' 开头、去掉 pad 后是合法 JSON）。
|
||||
校验这一步不可省 —— 字符串里的普通空格同样会命中候选判据。
|
||||
"""
|
||||
data = plain.encode("latin1")
|
||||
cands = _candidate_pads(data)
|
||||
for (a1, b1), (a2, b2), (a3, b3) in itertools.combinations(cands, 3):
|
||||
if not (b1 <= a2 and b2 <= a3):
|
||||
continue
|
||||
segs = [data[0:a1], data[b1:a2], data[b2:a3], data[b3:]]
|
||||
# 段边界一定落在 JSON 记号边界上（3 份真机样本一致）：
|
||||
# chunk1 以 ',' 结尾；traj 是**一个**完整 JSON 值；chunk2 以 ',' 开头、以 '],' 结尾
|
||||
if not (segs[0].startswith(b'{"cd":[') and segs[0].endswith(b',')
|
||||
and segs[2].startswith(b',') and segs[2].endswith(b'],')
|
||||
and segs[3].startswith(b'"sd"')):
|
||||
continue
|
||||
try:
|
||||
json.loads(segs[1].decode("latin1")) # traj 必须是独立可解析的 JSON 值
|
||||
except Exception: # noqa: BLE001
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(b"".join(segs).decode("latin1"))
|
||||
except Exception: # noqa: BLE001
|
||||
continue
|
||||
if "cd" not in obj or "sd" not in obj:
|
||||
continue
|
||||
return (segs[0].decode("latin1"), segs[1].decode("latin1"),
|
||||
segs[2].decode("latin1"), segs[3].decode("latin1"))
|
||||
raise ValueError(
|
||||
f"未能切出「四段加密」结构（候选 pad {len(cands)} 个）；这份明文可能不是 4 段格式")
|
||||
|
||||
|
||||
def xtea_decrypt_bytes(data: bytes, key: List[int]) -> bytes:
|
||||
out = bytearray()
|
||||
for i in range(0, len(data) - len(data) % 8, 8):
|
||||
v0, v1 = struct.unpack("<II", data[i:i + 8])
|
||||
d0, d1 = xtea_decrypt_block(v0, v1, key)
|
||||
out += struct.pack("<II", d0, d1)
|
||||
return bytes(out)
|
||||
|
||||
_KEY_CACHE: Dict[str, List[int]] = {}
|
||||
|
||||
|
||||
def extract_xtea_key(tdc_js_code: str, use_cache: bool = True) -> List[int]:
|
||||
"""从服务端下发的 tdc.js 里提取本次构建的 XTEA 密钥（4 个 32 位字）。
|
||||
|
||||
**纯 Python 静态还原**（`tcaptcha/telemetry/key.py`）:
|
||||
反汇编字节码 -> 定位 XTEA 闭包 -> 回溯它捕获的密钥数组 -> 读出 4 个密钥字
|
||||
-> 折入轮加盐。零 Node、零 JS 运行时、零 pc 硬编码
|
||||
（opcode 表按构建推导，见 opcodes.py）。
|
||||
|
||||
密钥按构建固定（同一份脚本多次运行首块完全一致，已实测），故可缓存。
|
||||
"""
|
||||
if use_cache:
|
||||
cached = _KEY_CACHE.get(tdc_js_code)
|
||||
if cached:
|
||||
return list(cached)
|
||||
|
||||
from .key import extract_xtea_key as _pure_extract
|
||||
|
||||
key = [int(k) & M32 for k in _pure_extract(tdc_js_code)]
|
||||
if len(key) != 4:
|
||||
raise RuntimeError(f"纯 Python 静态还原出来的密钥字不是 4 个: {key!r}")
|
||||
if use_cache:
|
||||
_KEY_CACHE[tdc_js_code] = list(key)
|
||||
return list(key)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 遥测生成入口（纯离线）
|
||||
# --------------------------------------------------------------------------- #
|
||||
def generate_telemetry(
|
||||
tdc_js_code: str,
|
||||
click_points: Optional[List[Tuple[int, int]]] = None,
|
||||
aid: str = "",
|
||||
user_agent: str = DEFAULT_USER_AGENT,
|
||||
referrer: str = "https://1ms.run/console/personal/benefits/check-in",
|
||||
mode: str = "python_values",
|
||||
node_bin: str = "node",
|
||||
) -> Dict[str, object]:
|
||||
"""统一遥测生成入口 —— **纯离线**（唯一模式 `python_values`）。
|
||||
|
||||
流程: tdc.js -> 37 个语义名（清单序）-> 38 槽取值 -> 4 段明文 -> 4 段 XTEA 拼接。
|
||||
零 Node、零真机明文；密钥也由纯 Python 静态提取。实现见 `telemetry/values.py`。
|
||||
|
||||
唯一约束（实测，见 har/1ms_dianxuan/PROGRESS.md 已知 #58/#59）:
|
||||
collect **必须用本次会话那一份构建**生成 —— 同一 aid 下，
|
||||
"按本次构建生成" -> errorCode=0；"拿别处模板回填" -> 恒 12。
|
||||
|
||||
返回 {"collect", "tlg", "eks", "source", "key"}。
|
||||
"""
|
||||
if mode != "python_values":
|
||||
raise ValueError(
|
||||
f"只支持 mode='python_values'（纯离线）；收到 {mode!r}。"
|
||||
"模板回填 / Node 兜底两条老路已删除 —— 它们产生的 collect 与本次会话构建不同源，"
|
||||
"在真实 aid 上必然 errorCode=12。")
|
||||
|
||||
from .values import generate_collect
|
||||
|
||||
out = generate_collect(tdc_js_code, click_points, referrer=referrer)
|
||||
return {"collect": out["collect"], "tlg": out["tlg"], "eks": out["eks"],
|
||||
"source": out["source"], "key": out["key"]}
|
||||
|
||||
|
||||
def selfcheck() -> None:
|
||||
"""纯算法自检: XTEA 往返 + encrypt_string 一致性。"""
|
||||
for chunk in (b"hello!!", bytes(range(32)), b"x" * 24):
|
||||
dec = xtea_decrypt_bytes(xtea_encrypt_bytes(chunk, DEFAULT_KEY), DEFAULT_KEY)
|
||||
assert dec[:len(chunk)] == chunk, f"XTEA 往返失败: {chunk!r}"
|
||||
ref = base64.b64encode(
|
||||
xtea_encrypt_bytes(b"abc\x00\x00\x00\x00\x00", DEFAULT_KEY)).decode("ascii")
|
||||
assert encrypt_string("abc") == ref, "encrypt_string 与 xtea_encrypt_bytes 不一致"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
selfcheck()
|
||||
print("crypto 自检通过: XTEA 往返 + encrypt_string 一致性")
|
||||
@@ -0,0 +1,324 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
腾讯天御 TDC 混沌虚拟机 (Chaos VM) —— **纯 Python** 字节码提取 + 反汇编器
|
||||
|
||||
为什么需要它
|
||||
------------
|
||||
旧 `tools/tdc_vm_disasm.js` 依赖 Node + Babel，且用**线性扫描**解码。实测该字节码数组
|
||||
里混有数据区/不可达区，线性扫描一进入数据区就失步，产出 `UNKNOWN_7`(opcode 7 其实是空洞)、
|
||||
`UNKNOWN_5096` 这类把操作数当操作码的垃圾 —— 换句话说旧的"8151 条指令流"是坏的。
|
||||
|
||||
本工具:
|
||||
1. 纯 Python 从 tdc.js 文本里取出字节码数组 `z`（复刻其内联的 base64 解码 + 游程展开），
|
||||
不需要任何 JS 运行时;
|
||||
2. 用**可达性驱动的 CFG 反汇编**（从 @0 出发，沿 fall-through / JUMP / JUMP_IF_TRUE /
|
||||
PUSH_TRY 的 catch 目标推进），只在真正可执行的地址上解码 —— 数据区自然留空，
|
||||
不再失步。
|
||||
|
||||
opcode 表来自对 VM 源码 AST 的推导（`G = [ ... ]` 中第 i 个 handler 即 opcode i，
|
||||
操作数个数 = handler 体内 `z[H++]` 的出现次数），已固化在下方。
|
||||
|
||||
> ⚠️ **固化的表只对 `tcaptcha/core_js/tdc_sample.js` 有效。** 实测 **opcode 编号每份构建都整体重排**
|
||||
> （同一源码 pc0 的 `SET_STACK_LEN` 在 6 份构建里是 53/19/32/57/55/1）。
|
||||
> 处理其它构建请先用 `tcaptcha/tdc_opcodes.py: derive_opcode_table(js)` 推导该构建的表，再
|
||||
> `disassemble_cfg(z, names=…, arity=…, dynamic=…)` 注入。详见 VM_SPEC.md §3.1。
|
||||
|
||||
用法:
|
||||
.venv/bin/python tools/tdc_disasm.py [tdc.js] # 默认 tcaptcha/core_js/tdc_sample.js
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# opcode 表 (从 VM 源码 G 数组推导)
|
||||
# 名字: 语义 (部分沿用旧工具命名); arity: 固定操作数字节数
|
||||
# 57 = CREATE_CLOSURE: 动态 (3 + 2*capCount + paramCount)
|
||||
# --------------------------------------------------------------------------- #
|
||||
OP_NAMES: Dict[int, str] = {
|
||||
0: "PUSH_STR_CHAR", 1: "IS_EXCEPTION", 2: "DUP", 3: "XOR", 4: "EQ_STRICT",
|
||||
5: "MOD", 6: "CALL_FUNC", 8: "GET_GLOBAL_PROP", 9: "MAKE_PROP_REF",
|
||||
10: "TYPEOF", 11: "DELETE", 12: "LOGICAL_NOT", 13: "EQ_LOOSE", 14: "SET_PROP",
|
||||
15: "GREATER_THAN", 17: "SWAP", 18: "IN", 19: "RET_TRUE", 20: "GREATER_EQ",
|
||||
22: "PUSH_IMM", 23: "MUL", 25: "LOAD_REG_INDIRECT", 26: "ITER_NEXT",
|
||||
28: "BIND_THIS", 29: "PUSH_NULL", 30: "BITWISE_AND", 31: "PUSH_IMM_ARRAY",
|
||||
32: "MAKE_REG_PROP_REF", 33: "SUB", 34: "CALL_METHOD", 35: "SHR", 36: "SHL",
|
||||
37: "JUMP_IF_TRUE", 38: "FOR_IN_KEYS", 39: "ADD", 41: "NEW_METHOD",
|
||||
42: "PUSH_UNDEFINED", 45: "PUSH_FALSE", 46: "STORE_REG", 47: "ALLOC_REG",
|
||||
48: "POP_TRY", 49: "THROW", 51: "POP", 52: "DIV", 53: "SET_STACK_LEN",
|
||||
54: "REVERSE_TOP2", 55: "PUSH_EMPTY_STR", 56: "PUSH_TRY", 57: "CREATE_CLOSURE",
|
||||
59: "GET_PROP", 60: "JUMP", 61: "NEW_FUNC", 62: "BITWISE_OR", 64: "REPLACE_TOP_IMM",
|
||||
65: "SHR_UNSIGNED", 66: "PUSH_TRUE", 68: "LOAD_REG_IMM", 69: "CLEAR_EXCEPTION",
|
||||
}
|
||||
|
||||
# 固定操作数字节数 (未列出的 opcode = 0 字节)
|
||||
ARITY: Dict[int, int] = {
|
||||
0: 1, 6: 1, 17: 1, 22: 1, 31: 1, 34: 1, 37: 1, 41: 1, 47: 1, 53: 1,
|
||||
56: 2, 60: 1, 61: 1, 64: 1, 68: 1,
|
||||
}
|
||||
DYNAMIC_ARITY = {57} # 需要读操作数才能定长
|
||||
OP_JUMP = 60
|
||||
OP_JUMP_IF_TRUE = 37
|
||||
OP_PUSH_TRY = 56 # operands = (catchTarget, catchVarReg)
|
||||
OP_CREATE_CLOSURE = 57
|
||||
|
||||
_B64_ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 1. 纯 Python 取出字节码数组 z (复刻 tdc.js 内联的 I() 解码 + 游程展开)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _b64_decode_custom(text: str) -> str:
|
||||
"""复刻 tdc.js 内联的 I(A): 标准 base64 码表, 忽略非码表字符 (含换行)。"""
|
||||
g = re.sub(r"[=]+$", "", str(text))
|
||||
idx = {ch: i for i, ch in enumerate(_B64_ALPHABET)}
|
||||
out: List[str] = []
|
||||
b = 0
|
||||
n = 0
|
||||
for ch in g:
|
||||
w = idx.get(ch, -1)
|
||||
if w == -1: # ~w falsy -> 跳过 (换行/空白)
|
||||
continue
|
||||
b = (64 * b + w) if (n % 4) else w
|
||||
old = n
|
||||
n += 1
|
||||
if old % 4: # 每组 4 个字符产出 3 字节
|
||||
out.append(chr(255 & (b >> ((-2 * n) & 6))))
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _unescape_js_string(raw: str) -> str:
|
||||
"""还原 JS 字符串字面量里的转义 (重点: `\\n` -> 真换行)。
|
||||
|
||||
注意: `n` 本身是合法 base64 字符, 若把 `\\n` 当成字面 `n` 会**污染**解码结果,
|
||||
必须先把转义还原成真实字符 (非码表字符随后被解码器忽略)。
|
||||
"""
|
||||
out: List[str] = []
|
||||
i = 0
|
||||
while i < len(raw):
|
||||
c = raw[i]
|
||||
if c == "\\" and i + 1 < len(raw):
|
||||
nxt = raw[i + 1]
|
||||
out.append({"n": "\n", "r": "\r", "t": "\t", "\\": "\\",
|
||||
'"': '"', "'": "'", "/": "/"}.get(nxt, nxt))
|
||||
i += 2
|
||||
continue
|
||||
out.append(c)
|
||||
i += 1
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def extract_bytecode(js_text: str) -> List[int]:
|
||||
"""从 tdc.js 文本里取出混沌 VM 的字节码数组 z (纯 Python, 无 JS 运行时)。
|
||||
|
||||
结构 (实测): `__TENCENT_CHAOS_VM(0, function(A){...}(["<base64>",[<int>,...]]), ...)`
|
||||
这里以 `__TENCENT_CHAOS_VM` 为锚点定位 `}(["...` 实参, 不再用"最长字符串"猜测。
|
||||
"""
|
||||
anchor = js_text.rfind("__TENCENT_CHAOS_VM")
|
||||
if anchor < 0:
|
||||
raise ValueError("未找到 __TENCENT_CHAOS_VM 锚点")
|
||||
|
||||
# 找锚点之后的 `}([` -> 数组起始的 `[`
|
||||
k = js_text.find("}([", anchor)
|
||||
if k < 0:
|
||||
k = js_text.find("[", js_text.find("function", anchor))
|
||||
start = js_text.find("[", k)
|
||||
if start < 0 or start + 1 >= len(js_text) or js_text[start + 1] != '"':
|
||||
raise ValueError("未能定位字节码实参数组")
|
||||
|
||||
q1 = start + 1
|
||||
q2 = js_text.find('"', q1 + 1) # base64 字面量内不含 `"`
|
||||
if q2 < 0:
|
||||
raise ValueError("base64 字面量未闭合")
|
||||
b64 = _unescape_js_string(js_text[q1 + 1:q2])
|
||||
|
||||
# 取内层整数数组 (括号配对)
|
||||
i = q2 + 1
|
||||
while i < len(js_text) and js_text[i] in " \t\r\n,":
|
||||
i += 1
|
||||
if i >= len(js_text) or js_text[i] != "[":
|
||||
raise ValueError("base64 之后不是整数数组")
|
||||
depth = 0
|
||||
j = i
|
||||
while j < len(js_text):
|
||||
if js_text[j] == "[":
|
||||
depth += 1
|
||||
elif js_text[j] == "]":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
break
|
||||
j += 1
|
||||
# 该整数数组会混用科学计数法 (如 `1e9` / `16e3`), 需按数值解析
|
||||
ints = [int(float(x)) for x in js_text[i + 1:j].split(",") if x.strip() != ""]
|
||||
|
||||
# 3) 复刻 VM 的游程展开
|
||||
chars = _b64_decode_custom(b64) # G
|
||||
w: List[int] = ints
|
||||
|
||||
q: List[int] = []
|
||||
h = 0
|
||||
m = w.pop(0) if w else -1
|
||||
g = w.pop(0) if w else 0
|
||||
|
||||
def flush() -> None:
|
||||
nonlocal h, m, g
|
||||
while h == m:
|
||||
q.append(g)
|
||||
h += 1
|
||||
m = w.pop(0) if w else -1
|
||||
g = w.pop(0) if w else 0
|
||||
|
||||
for ch in chars:
|
||||
flush()
|
||||
q.append(ord(ch))
|
||||
h += 1
|
||||
flush()
|
||||
return q
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 2. 可达性驱动的 CFG 反汇编 (只在可执行地址解码)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _decode_one(z: List[int], pc: int, names: Optional[Dict[int, str]] = None,
|
||||
arity: Optional[Dict[int, int]] = None,
|
||||
dynamic: Optional[set] = None) -> Tuple[Optional[int], List[int], Optional[List[str]], int]:
|
||||
"""解码 pc 处一条指令 -> (opcode, operands, note, next_fallthrough_pc)。
|
||||
|
||||
返回值 (None,...) 表示该地址无法构成合法指令。
|
||||
|
||||
names/arity/dynamic 允许注入**按构建推导**的操作码表 (见 tcaptcha/tdc_opcodes.py);
|
||||
不传则用本模块固化的 core_js 表。
|
||||
"""
|
||||
if names is None:
|
||||
names = OP_NAMES
|
||||
if arity is None:
|
||||
arity = ARITY
|
||||
if dynamic is None:
|
||||
dynamic = DYNAMIC_ARITY
|
||||
if pc >= len(z):
|
||||
return None, [], None, pc
|
||||
op = z[pc]
|
||||
if op > 69 or op not in names: # 空洞 opcode / 越界 -> 非法
|
||||
return None, [], None, pc + 1
|
||||
pos = pc + 1
|
||||
operands: List[int] = []
|
||||
note: Optional[List[str]] = None
|
||||
|
||||
if op in dynamic:
|
||||
# 57: target, capCount, paramCount, [reg, reg]*capCount, [reg]*paramCount
|
||||
if pos + 3 > len(z):
|
||||
return None, [], None, pos
|
||||
target, cap, param = z[pos], z[pos + 1], z[pos + 2]
|
||||
pos += 3
|
||||
need = 2 * cap + param
|
||||
if pos + need > len(z):
|
||||
return None, [], None, pos
|
||||
operands = z[pos:pos + need]
|
||||
pos += need
|
||||
caps = [f"r{operands[2*i]}=r{operands[2*i+1]}" for i in range(cap)]
|
||||
params = [f"p{i}->r{operands[2*cap+i]}" for i in range(param)]
|
||||
note = [f"target=@{target}", f"caps=[{','.join(caps)}]", f"params=[{','.join(params)}]"]
|
||||
else:
|
||||
n = arity.get(op, 0)
|
||||
if pos + n > len(z):
|
||||
return None, [], None, pos
|
||||
operands = z[pos:pos + n]
|
||||
pos += n
|
||||
if names[op] in ("JUMP", "JUMP_IF_TRUE") and operands:
|
||||
note = [f"target=@{operands[0]}"]
|
||||
elif names[op] == "PUSH_TRY" and len(operands) == 2:
|
||||
note = [f"catchTarget=@{operands[0]}", f"catchVarReg=r{operands[1]}"]
|
||||
return op, operands, note, pos
|
||||
|
||||
|
||||
def disassemble_cfg(z: List[int], entry: int = 0, follow_closures: bool = True,
|
||||
names: Optional[Dict[int, str]] = None,
|
||||
arity: Optional[Dict[int, int]] = None,
|
||||
dynamic: Optional[set] = None) -> List[dict]:
|
||||
"""从 entry 出发按可达性反汇编; 返回按 pc 排序的指令列表。
|
||||
|
||||
names/arity/dynamic 可注入按构建推导的操作码表 (默认 = 本模块固化的 core_js 表)。
|
||||
"""
|
||||
if names is None:
|
||||
names = OP_NAMES
|
||||
if arity is None:
|
||||
arity = ARITY
|
||||
if dynamic is None:
|
||||
dynamic = DYNAMIC_ARITY
|
||||
# 控制流相关 opcode 按"语义名"定位, 不写死数值 (opcode 每份构建都重排)
|
||||
op_jump = next((o for o, n in names.items() if n == "JUMP"), OP_JUMP)
|
||||
op_jump_if = next((o for o, n in names.items() if n == "JUMP_IF_TRUE"), OP_JUMP_IF_TRUE)
|
||||
op_push_try = next((o for o, n in names.items() if n == "PUSH_TRY"), OP_PUSH_TRY)
|
||||
op_closure = next((o for o, n in names.items() if n == "CREATE_CLOSURE"), OP_CREATE_CLOSURE)
|
||||
# 终结符: 返回/抛出之后**没有 fall-through**。
|
||||
# 漏掉这一步会让控制流"穿过函数结尾"线性走下去, 把所有后续代码都算进当前子程序
|
||||
# (实测会把父函数的初始化序列误判成子函数自己的代码)。
|
||||
op_terminators = {o for o, n in names.items() if n in ("RET_TRUE", "THROW")}
|
||||
|
||||
visited: Dict[int, dict] = {}
|
||||
work = [entry]
|
||||
while work:
|
||||
pc = work.pop()
|
||||
if pc in visited or pc >= len(z) or pc < 0:
|
||||
continue
|
||||
op, operands, note, nxt = _decode_one(z, pc, names, arity, dynamic)
|
||||
if op is None:
|
||||
# 非法地址 (落在数据区): 记录但不继续
|
||||
visited[pc] = {"pc": pc, "op": None, "name": "DATA/INVALID", "operands": [], "note": None}
|
||||
continue
|
||||
visited[pc] = {"pc": pc, "op": op, "name": names[op], "operands": operands, "note": note}
|
||||
if op == op_jump:
|
||||
if operands:
|
||||
work.append(operands[0])
|
||||
continue # 无条件跳转: 无 fall-through
|
||||
if op == op_jump_if:
|
||||
if operands:
|
||||
work.append(operands[0])
|
||||
work.append(nxt) # 条件跳转: 有 fall-through
|
||||
continue
|
||||
if op == op_push_try and operands:
|
||||
work.append(operands[0]) # catch 目标可达
|
||||
if op == op_closure and follow_closures:
|
||||
# 操作数 [0] 是闭包体入口: 该段代码在闭包被调用时才执行, 但它是**代码**,
|
||||
# 必须作为遍历根加入, 否则大半个程序都算不到。
|
||||
work.append(z[pc + 1])
|
||||
if op in op_terminators: # RET_TRUE / THROW: 无 fall-through
|
||||
continue
|
||||
work.append(nxt)
|
||||
return [visited[k] for k in sorted(visited)]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
def main() -> int:
|
||||
path = sys.argv[1] if len(sys.argv) > 1 else "tcaptcha/core_js/tdc_sample.js"
|
||||
js = open(path, encoding="utf-8", errors="replace").read()
|
||||
z = extract_bytecode(js)
|
||||
insts = disassemble_cfg(z)
|
||||
invalid = [i for i in insts if i["op"] is None]
|
||||
print(f"bytecode length : {len(z)}")
|
||||
print(f"reachable instrs: {len(insts)} (invalid/data slots: {len(invalid)})")
|
||||
op_used = sorted({i["op"] for i in insts if i["op"] is not None})
|
||||
holes = [o for o in op_used if o not in OP_NAMES]
|
||||
print(f"opcodes used : {op_used}")
|
||||
print(f"illegal opcodes : {holes}")
|
||||
out = path.rsplit(".", 1)[0] + ".disasm.txt"
|
||||
with open(out, "w", encoding="utf-8") as f:
|
||||
for i in insts:
|
||||
if i["op"] is None:
|
||||
f.write(f"@{i['pc']:05d}: {i['name']}\n")
|
||||
else:
|
||||
extra = (" " + " ".join(i["note"])) if i["note"] else ""
|
||||
argc = (" " + str(i["operands"])) if (i["operands"] and not i["note"]) else ""
|
||||
f.write(f"@{i['pc']:05d}: {i['name']:<18}{extra}{argc}\n")
|
||||
print(f"written : {out}")
|
||||
for i in insts[:12]:
|
||||
print(" ", i)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,441 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
腾讯天御 TDC 混沌 VM (Chaos VM) —— XTEA 密钥的**纯 Python** 静态还原器。
|
||||
|
||||
为什么需要它
|
||||
------------
|
||||
此前要拿到某一版 tdc.js 的 XTEA 密钥, 只能把 tdc.js 丢进 Node 插桩跑一遍
|
||||
(ADDU opcode 吐密钥字)。密钥**不是字面量**, 而是 VM 运行时由字节码算出来的,
|
||||
所以必须"读指令流"而不是"搜字符串"。
|
||||
|
||||
本模块完全离线(零 Node、零 JS 运行时), 基于**正确反汇编**(tools/tdc_disasm.py)做
|
||||
**模式 + 数据流**定位, 复刻密钥构造算法。全部结论都有运行时神谕（真浏览器 TDC.getData）
|
||||
逐条对照过, 见 har/1ms_dianxuan/PROGRESS.md。
|
||||
|
||||
算法 (已证: 纯 Python 结果 == Node extract_xtea_key 逐字一致)
|
||||
------------------------------------------------------------
|
||||
1. **4 个密钥字**: 每个字由一个专门的闭包算出:
|
||||
word_i = Σ_{k=0..3} ( (ord(src_i[k]) + off_i) & 0xFF ) << (8*k)
|
||||
源串 `src_i` 是紧跟在该闭包 `CREATE_CLOSURE` 之后的 4 个 `PUSH_STR_CHAR`
|
||||
字面量; 偏移 `off_i` 是打包点的立即数; 存入下标 `i` 由闭包尾部的
|
||||
`PUSH_IMM_ARRAY / PUSH_IMM [i] / ... / SET_PROP` 给出。
|
||||
|
||||
2. **哪些闭包是真密钥**: 密钥数组会被 **XTEA 闭包捕获**。
|
||||
XTEA 轮里出现 `delta = 0x9E3779B9`; 含 delta 的那个闭包体所属的
|
||||
`CREATE_CLOSURE` 即 XTEA 闭包, 其 `caps=[r7=r<K>]` 的 `<K>` 就是密钥数组所在的
|
||||
父帧寄存器。**只有 caps 右侧等于该寄存器的密钥字闭包**才是真密钥
|
||||
(同一份脚本往往还并行算了第二个"诱饵"数组, 用不同寄存器, 必须排除)。
|
||||
|
||||
3. **轮加盐**: XTEA 轮内 `key[(sum>>>11)&3]` 查表之后会**条件性**加上一个立即数
|
||||
(salt): `GET_PROP / PUSH_IMM <salt> / ADD / ADD / XOR`。加给哪个下标由守卫条件
|
||||
`idx == ((sum>>>11)&3)` / `idx == (sum & 3)` 里的 `idx` 常量决定。
|
||||
有效密钥 = base 字按盐的下标位置相加。
|
||||
|
||||
所有定位都是**模式 + 立即数**, 不写死任何 pc (每份构建都会重排)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
if _ROOT not in sys.path: # 允许在任意 cwd 下 import tools.*
|
||||
sys.path.insert(0, _ROOT)
|
||||
|
||||
from tcaptcha.telemetry.disasm import extract_bytecode, disassemble_cfg # noqa: E402
|
||||
|
||||
from tcaptcha.telemetry.opcodes import derive_opcode_table # noqa: E402
|
||||
|
||||
M32 = 0xFFFFFFFF
|
||||
XTEA_DELTA = 0x9E3779B9 # 2654435769, XTEA 黄金比常数
|
||||
KEY_LEN = 4
|
||||
|
||||
# 走"下标表达式"时要跳过的算子 (它们构成 `(sum & 3)` / `(sum>>>11)&3`)
|
||||
_SKIP_OPS = {
|
||||
"BITWISE_AND", "SHR_UNSIGNED", "LOAD_REG_IMM", "MAKE_REG_PROP_REF",
|
||||
"GET_PROP", "SWAP", "STORE_REG", "ADD", "DUP", "LOAD_REG_INDIRECT",
|
||||
"PUSH_IMM_ARRAY", "POP",
|
||||
}
|
||||
# 这些算子紧跟在"掩码 PUSH_IMM 3"之前, 说明那个 3 是掩码而不是下标
|
||||
_MASK_NEIGHBOURS = {"BITWISE_AND", "LOAD_REG_IMM", "SHR_UNSIGNED"}
|
||||
|
||||
|
||||
class _Dis:
|
||||
"""对反汇编结果做少量索引封装, 便于按 pc 前后游走。"""
|
||||
|
||||
def __init__(self, js_text: str) -> None:
|
||||
self.z = extract_bytecode(js_text)
|
||||
# opcode 编号每份构建都重排 -> 必须先按构建把语义表推导出来
|
||||
names, arity, dynamic, diag = derive_opcode_table(js_text)
|
||||
self.names, self.arity, self.dynamic, self.diag = names, arity, dynamic, diag
|
||||
self.insts = disassemble_cfg(self.z, names=names, arity=arity, dynamic=dynamic)
|
||||
self.by_pc: Dict[int, dict] = {i["pc"]: i for i in self.insts}
|
||||
self.pcs: List[int] = sorted(self.by_pc)
|
||||
self._idx: Dict[int, int] = {pc: n for n, pc in enumerate(self.pcs)}
|
||||
|
||||
def op(self, pc: Optional[int]) -> Optional[str]:
|
||||
i = self.by_pc.get(pc) if pc is not None else None
|
||||
return i["name"] if i else None
|
||||
|
||||
def ops(self, pc: Optional[int]) -> List[int]:
|
||||
i = self.by_pc.get(pc) if pc is not None else None
|
||||
return list(i["operands"]) if i else []
|
||||
|
||||
def note(self, pc: int) -> str:
|
||||
return " ".join(self.by_pc[pc].get("note") or [])
|
||||
|
||||
def nxt(self, pc: Optional[int]) -> Optional[int]:
|
||||
n = self._idx.get(pc) if pc is not None else None
|
||||
return self.pcs[n + 1] if n is not None and n + 1 < len(self.pcs) else None
|
||||
|
||||
def prev(self, pc: Optional[int]) -> Optional[int]:
|
||||
n = self._idx.get(pc) if pc is not None else None
|
||||
return self.pcs[n - 1] if n is not None and n > 0 else None
|
||||
|
||||
def in_range(self, lo: int, hi: int) -> List[int]:
|
||||
import bisect
|
||||
a = bisect.bisect_left(self.pcs, lo)
|
||||
b = bisect.bisect_right(self.pcs, hi)
|
||||
return self.pcs[a:b]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 1. 闭包清单 / 密钥字打包点
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _closures(d: _Dis) -> List[dict]:
|
||||
"""所有 CREATE_CLOSURE -> {pc, target, caps:{dst:src}, str_after}。"""
|
||||
out: List[dict] = []
|
||||
for i in d.insts:
|
||||
if i["name"] != "CREATE_CLOSURE":
|
||||
continue
|
||||
pc = i["pc"]
|
||||
note = " ".join(i["note"] or [])
|
||||
m = re.search(r"target=@(\d+)", note)
|
||||
if not m:
|
||||
continue
|
||||
caps: Dict[int, int] = {}
|
||||
caps_part = re.search(r"caps=\[([^\]]*)\]", note)
|
||||
if caps_part:
|
||||
for dst, src in re.findall(r"r(\d+)=r(\d+)", caps_part.group(1)):
|
||||
caps[int(dst)] = int(src)
|
||||
out.append({"pc": pc, "target": int(m.group(1)), "caps": caps,
|
||||
"str_after": _string_after(d, pc)})
|
||||
return out
|
||||
|
||||
|
||||
def _string_after(d: _Dis, pc: int) -> Optional[str]:
|
||||
"""CREATE_CLOSURE 紧随其后的 `PUSH_EMPTY_STR + PUSH_STR_CHAR*` 字面量。"""
|
||||
p = d.nxt(pc)
|
||||
if p is None or d.op(p) != "PUSH_EMPTY_STR":
|
||||
return None
|
||||
chars: List[str] = []
|
||||
q = d.nxt(p)
|
||||
while q is not None and d.op(q) == "PUSH_STR_CHAR":
|
||||
chars.append(chr(d.ops(q)[0]))
|
||||
q = d.nxt(q)
|
||||
return "".join(chars)
|
||||
|
||||
|
||||
def _pack_sites(d: _Dis, lo: int, hi: int) -> List[dict]:
|
||||
"""
|
||||
在 [lo,hi] 里找"密钥字打包点":
|
||||
... CALL_METHOD[1] / PUSH_IMM c / SUB (off = -c)
|
||||
或 ... CALL_METHOD[1] / PUSH_IMM 0 / PUSH_IMM c / SUB / SUB (off = +c)
|
||||
/ PUSH_IMM 8 / LOAD_REG_IMM / MUL / SHL / BITWISE_OR
|
||||
返回 [{or_pc, off}]。
|
||||
"""
|
||||
res: List[dict] = []
|
||||
for pc in d.in_range(lo, hi):
|
||||
if d.op(pc) != "PUSH_IMM" or d.ops(pc) != [8]:
|
||||
continue
|
||||
a, b, c, e = d.nxt(pc), None, None, None
|
||||
b = d.nxt(a)
|
||||
c = d.nxt(b)
|
||||
e = d.nxt(c)
|
||||
if not (d.op(a) == "LOAD_REG_IMM" and d.op(b) == "MUL"
|
||||
and d.op(c) == "SHL" and d.op(e) == "BITWISE_OR"):
|
||||
continue
|
||||
off = _decode_offset(d, pc)
|
||||
if off is not None:
|
||||
res.append({"or_pc": e, "imm8_pc": pc, "off": off})
|
||||
return res
|
||||
|
||||
|
||||
def _decode_offset(d: _Dis, imm8_pc: int) -> Optional[int]:
|
||||
"""解出紧跟 CALL_METHOD 之后的偏移立即数 (含两种编码)。"""
|
||||
p1 = d.prev(imm8_pc) # 最后一个 SUB
|
||||
if d.op(p1) != "SUB":
|
||||
return None
|
||||
p2 = d.prev(p1)
|
||||
if d.op(p2) == "PUSH_IMM": # X - c => off = -c
|
||||
return -d.ops(p2)[0]
|
||||
if d.op(p2) == "SUB": # X - (0 - c) => off = +c
|
||||
p3 = d.prev(p2)
|
||||
p4 = d.prev(p3)
|
||||
if d.op(p3) == "PUSH_IMM" and d.op(p4) == "PUSH_IMM" and d.ops(p4) == [0]:
|
||||
return d.ops(p3)[0]
|
||||
return None
|
||||
|
||||
|
||||
def _store_index(d: _Dis, lo: int, hi: int) -> Optional[int]:
|
||||
"""闭包尾部的 `PUSH_IMM_ARRAY / PUSH_IMM [i] / MAKE_REG_PROP_REF / LOAD_REG_IMM / SET_PROP`。"""
|
||||
hits: List[int] = []
|
||||
for pc in d.in_range(lo, hi):
|
||||
if d.op(pc) != "PUSH_IMM_ARRAY":
|
||||
continue
|
||||
a, b, c, e = d.nxt(pc), None, None, None
|
||||
b = d.nxt(a)
|
||||
c = d.nxt(b)
|
||||
e = d.nxt(c)
|
||||
if (d.op(a) == "PUSH_IMM" and d.op(b) == "MAKE_REG_PROP_REF"
|
||||
and d.op(c) == "LOAD_REG_IMM" and d.op(e) == "SET_PROP"):
|
||||
hits.append(d.ops(a)[0])
|
||||
return hits[-1] if hits else None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 2. XTEA 闭包定位 + 轮加盐
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _find_xtea_closure(d: _Dis, closures: List[dict]) -> Optional[dict]:
|
||||
"""含 XTEA 常数(delta)的那个闭包体所属的 CREATE_CLOSURE。"""
|
||||
delta_pcs = [pc for pc in d.pcs
|
||||
if d.op(pc) == "PUSH_IMM" and d.ops(pc)[0] in (XTEA_DELTA, XTEA_DELTA * 32)]
|
||||
# 取"最内层"(区间最小)的闭包体 —— 外层 init 函数的体也会横跨 delta,
|
||||
# 但它不捕获密钥数组。
|
||||
best: Optional[dict] = None
|
||||
best_end: Optional[int] = None
|
||||
for cl in closures:
|
||||
end = _body_end(d, cl["pc"])
|
||||
if end is None or end < cl["target"]:
|
||||
continue
|
||||
if any(cl["target"] <= pc <= end for pc in delta_pcs):
|
||||
if best is None or best_end is None or (end - cl["target"]) < (best_end - best["target"]):
|
||||
best, best_end = cl, end
|
||||
return best
|
||||
|
||||
|
||||
def _largest_reg(regs: Dict[int, int]) -> Optional[int]:
|
||||
return max(regs.values()) if regs else None
|
||||
|
||||
|
||||
def _caps_single(caps: Dict[int, int]) -> Optional[Tuple[int, int]]:
|
||||
"""字/XTEA 闭包只捕获一个数组: 返回 (dst, src)。"""
|
||||
if len(caps) != 1:
|
||||
return None
|
||||
(dst, src), = caps.items()
|
||||
return dst, src
|
||||
|
||||
|
||||
def _defining_closure(d: _Dis, closures: List[dict], pc: int) -> Optional[dict]:
|
||||
"""含给定 pc 的**最内层**闭包(即创建 pc 处那条 CREATE_CLOSURE 的那个函数)。"""
|
||||
best: Optional[dict] = None
|
||||
best_span: Optional[int] = None
|
||||
for c in closures:
|
||||
e = _body_end(d, c["pc"])
|
||||
if e is None or e < c["target"]:
|
||||
continue
|
||||
if c["target"] <= pc <= e:
|
||||
span = e - c["target"]
|
||||
if best_span is None or span < best_span:
|
||||
best, best_span = c, span
|
||||
return best
|
||||
|
||||
|
||||
def _body_end(d: _Dis, cl_pc: int) -> Optional[int]:
|
||||
"""
|
||||
闭包体结束 pc (该体的 RET_TRUE)。约定: 闭包被创建后要么紧跟其体尾的 RET_TRUE,
|
||||
要么中间夹一条 `PUSH_IMM_ARRAY`(闭包被塞进数组时)。
|
||||
"""
|
||||
e = d.prev(cl_pc)
|
||||
if e is not None and d.op(e) == "PUSH_IMM_ARRAY":
|
||||
e = d.prev(e)
|
||||
return e if (e is not None and d.op(e) == "RET_TRUE") else None
|
||||
|
||||
|
||||
def _guard_index(d: _Dis, eq_pc: int) -> Optional[int]:
|
||||
"""
|
||||
守卫 `idx == ((sum & 3) | ((sum>>>11)&3))` 里的常量 idx。
|
||||
从 EQ_LOOSE 反向游走, 跳过下标表达式自身的算子, 第一个落在 0..3 的立即数即 idx。
|
||||
掩码 3 靠"它的后继是 BITWISE_AND / LOAD_REG_IMM / SHR_UNSIGNED"识别并跳过。
|
||||
"""
|
||||
pc = d.prev(eq_pc)
|
||||
last: Optional[str] = None
|
||||
for _ in range(24):
|
||||
if pc is None:
|
||||
return None
|
||||
nm, ops = d.op(pc), d.ops(pc)
|
||||
if nm in ("PUSH_IMM", "REPLACE_TOP_IMM") and ops:
|
||||
v = ops[0]
|
||||
if 0 <= v <= 3:
|
||||
if not (nm == "PUSH_IMM" and v == 3 and last in _MASK_NEIGHBOURS):
|
||||
return v
|
||||
elif nm not in _SKIP_OPS:
|
||||
# 出现意料之外的算子: 继续向前找, 但记录上下文
|
||||
pass
|
||||
last = nm
|
||||
pc = d.prev(pc)
|
||||
return None
|
||||
|
||||
|
||||
def _salts(d: _Dis, lo: int, hi: int) -> List[Tuple[int, int]]:
|
||||
"""
|
||||
XTEA 轮内的条件加盐: `GET_PROP / PUSH_IMM <salt> / ADD / ADD / XOR`。
|
||||
返回 [(idx, salt)] (按 (idx,salt) 去重)。
|
||||
"""
|
||||
salt_pcs: List[int] = []
|
||||
for pc in d.in_range(lo, hi):
|
||||
if d.op(pc) != "PUSH_IMM":
|
||||
continue
|
||||
if d.op(d.prev(pc)) != "GET_PROP" or d.op(d.nxt(pc)) != "ADD":
|
||||
continue
|
||||
salt_pcs.append(pc)
|
||||
if not salt_pcs:
|
||||
return []
|
||||
|
||||
# 收集所有 (JUMP_IF_TRUE, EQ_LOOSE) 守卫; block(T) 到 T 之后第一条无条件 JUMP
|
||||
guards: List[Tuple[int, int, Optional[int]]] = [] # (target, jump_pc, idx)
|
||||
for pc in d.in_range(lo, hi):
|
||||
if d.op(pc) != "JUMP_IF_TRUE":
|
||||
continue
|
||||
tgt_ops = d.ops(pc)
|
||||
if not tgt_ops:
|
||||
continue
|
||||
tgt = tgt_ops[0]
|
||||
eq = d.prev(pc)
|
||||
if d.op(eq) != "EQ_LOOSE":
|
||||
continue
|
||||
idx = _guard_index(d, eq)
|
||||
if idx is not None:
|
||||
guards.append((tgt, pc, idx))
|
||||
|
||||
def block_end(tgt: int) -> int:
|
||||
p = tgt
|
||||
while p is not None and p <= hi:
|
||||
if d.op(p) == "JUMP":
|
||||
return p
|
||||
p = d.nxt(p)
|
||||
return hi
|
||||
|
||||
out: List[Tuple[int, int]] = []
|
||||
for spc in salt_pcs:
|
||||
salt = d.ops(spc)[0]
|
||||
best: Optional[Tuple[int, int]] = None # (target, idx)
|
||||
for tgt, _j, idx in guards:
|
||||
if tgt <= spc < block_end(tgt):
|
||||
if best is None or tgt > best[0]:
|
||||
best = (tgt, idx)
|
||||
if best is not None:
|
||||
out.append((best[1], salt))
|
||||
# 去重保序
|
||||
seen = set()
|
||||
uniq: List[Tuple[int, int]] = []
|
||||
for x in out:
|
||||
if x not in seen:
|
||||
seen.add(x)
|
||||
uniq.append(x)
|
||||
return uniq
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 3. 主入口
|
||||
# --------------------------------------------------------------------------- #
|
||||
def word_from_chars(src: str, off: int) -> int:
|
||||
"""word = Σ ((ord(c) + off) & 0xFF) << (8*k) (小端打包)"""
|
||||
w = 0
|
||||
for k, ch in enumerate(src):
|
||||
w |= ((ord(ch) + off) & 0xFF) << (8 * k)
|
||||
return w & M32
|
||||
|
||||
|
||||
def extract_xtea_key_pure(js_text: str, verbose: bool = False):
|
||||
"""
|
||||
纯 Python 从 tdc.js 里还原本次构建的 XTEA 有效密钥 (4 个 32 位字)。
|
||||
|
||||
返回 dict:
|
||||
base : 4 个 base 字 (未折盐)
|
||||
salts : [(idx, salt), ...]
|
||||
key : 有效密钥
|
||||
words : [(idx, word, src, off), ...] (便于诊断)
|
||||
"""
|
||||
d = _Dis(js_text)
|
||||
closures = _closures(d)
|
||||
xtea = _find_xtea_closure(d, closures)
|
||||
if xtea is None:
|
||||
raise ValueError("未找到 XTEA 闭包 (字节码里没有 delta 常数?)")
|
||||
xs = _caps_single(xtea["caps"])
|
||||
if xs is None:
|
||||
raise ValueError("XTEA 闭包没有唯一捕获的数组, 无法确定密钥数组")
|
||||
xtea_src = xs[1] # XTEA 所在帧里, 密钥数组所在的寄存器
|
||||
|
||||
# 密钥数组是"被 XTEA 闭包捕获"的那一个。字闭包与 XTEA 闭包往往不在同一帧:
|
||||
# 父帧 G: r<R> = 密钥数组
|
||||
# ├─ 字闭包 W: caps=[r7 = 本帧 r<Wsrc>] (Wsrc ∈ {R, 诱饵寄存器})
|
||||
# └─ XTEA 闭包 : caps=[r7 = 本帧 r<xtea_src>]
|
||||
# 所以要把 W 的捕获穿一层: B = 定义 W 的那个闭包(B 帧 -> G 帧), 检查
|
||||
# B.caps[Wsrc] == xtea_src -> 该字闭包写的就是真密钥数组。
|
||||
words: Dict[int, Tuple[int, str, int]] = {}
|
||||
picked: List[str] = []
|
||||
for cl in closures:
|
||||
if cl["str_after"] is None:
|
||||
continue
|
||||
cs = _caps_single(cl["caps"])
|
||||
if cs is None:
|
||||
continue
|
||||
w_src = cs[1] # 该字闭包捕获的、定义帧里的寄存器
|
||||
definer = _defining_closure(d, closures, cl["pc"])
|
||||
if definer is not None and definer["caps"]:
|
||||
real = definer["caps"].get(w_src) == xtea_src
|
||||
else: # 同帧: 直接比
|
||||
real = (w_src == xtea_src)
|
||||
if not real:
|
||||
continue
|
||||
end = _body_end(d, cl["pc"])
|
||||
if end is None or end < cl["target"]:
|
||||
continue
|
||||
packs = _pack_sites(d, cl["target"], end)
|
||||
if not packs:
|
||||
continue
|
||||
idx = _store_index(d, cl["target"], end)
|
||||
if idx is None or not (0 <= idx < KEY_LEN):
|
||||
continue
|
||||
src, off = cl["str_after"], packs[-1]["off"]
|
||||
words[idx] = (word_from_chars(src, off), src, off)
|
||||
picked.append("%s(idx=%d,off=%+d)" % (src, idx, off))
|
||||
|
||||
if len(words) != KEY_LEN:
|
||||
raise ValueError(
|
||||
"密钥字不完整: 只定位到 %s (期望 0..3); XTEA 密钥帧寄存器 r%s"
|
||||
% (sorted(words), xtea_src))
|
||||
|
||||
base = [words[i][0] for i in range(KEY_LEN)]
|
||||
salts = _salts(d, xtea["target"], d.prev(xtea["pc"]) or xtea["target"])
|
||||
|
||||
key = list(base)
|
||||
for idx, salt in salts:
|
||||
key[idx] = (key[idx] + salt) & M32
|
||||
|
||||
report = {
|
||||
"base": base,
|
||||
"salts": salts,
|
||||
"key": key,
|
||||
"words": [(i, words[i][0], words[i][1], words[i][2]) for i in range(KEY_LEN)],
|
||||
"xtea_closure_pc": xtea["pc"],
|
||||
"key_reg": xtea_src,
|
||||
}
|
||||
if verbose:
|
||||
print("[tdc_key_pure] XTEA closure @%d key array = r%s" % (xtea["pc"], xtea_src))
|
||||
for i, w, s, o in report["words"]:
|
||||
print(" word[%d] <- %-6r off=%+d = %d" % (i, s, o, w))
|
||||
for idx, salt in salts:
|
||||
print(" salt +%d -> key[%d]" % (salt, idx))
|
||||
print(" base = %s" % base)
|
||||
print(" key = %s" % key)
|
||||
return report
|
||||
|
||||
|
||||
def extract_xtea_key(js_text: str) -> List[int]:
|
||||
"""便捷入口: 只取有效密钥 (与 crypto.extract_xtea_key 同口径)。"""
|
||||
return list(extract_xtea_key_pure(js_text)["key"])
|
||||
@@ -0,0 +1,345 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
tdc_layout.py —— 静态导出 `collect` 明文的 **cd 槽位表**（纯算, 不跑 JS, 不跑环境）。
|
||||
|
||||
背景（这条链是 2026-10-11 排查 errorCode=12 时打通的, 见 har/1ms_dianxuan/PROGRESS.md #31/#32）:
|
||||
真机成功会话的 collect 明文解出来后, 发现 **5 份同日样本的 `cd` 槽位数(38)、取值集合、`sd`
|
||||
全都一样, 但槽位顺序 5/5 互不相同** —— 也就是"哪些字段排在哪一槽"是**按构建随机化**的。
|
||||
仓库里的 success_collect_template.b64 是**某一次**的顺序, 拿它发的 collect 会被服务端
|
||||
按"本次构建的槽位表"读成自相矛盾的环境 -> errorCode=12。
|
||||
|
||||
槽位序从哪来（程序结构, 见 tools/tdc_program.py）:
|
||||
1. 顶层 @0 先把 62 个**模块工厂**按 id 填进一个数组 (registry[id] = function(module, exports, require));
|
||||
2. 其中一个模块（下称"清单模块"）返回一个**数组**, 它的构造方式是一串
|
||||
r6[r6.length] = require(N)
|
||||
也就是 **`cd` 的槽位顺序 = 这串 require 的顺序**;
|
||||
3. 另一个模块负责把这些采集器按 `.on` / `.reset` / `.get` 分到三个数组, 再另有一个模块
|
||||
遍历"有 .get() 的那些", 依次取 `get()` 的值拼成 `{"cd":[...],"sd":{...}}`。
|
||||
|
||||
所以本工具做的事:
|
||||
* 找清单模块（= push(require(N)) 次数最多的那个模块）;
|
||||
* 读出它的 require 序列 -> 槽位表 [(槽位号, 模块 id, 工厂 entry), ...];
|
||||
* 把每个槽位子模块（含其内部闭包）里的**环境属性字面量**列出来, 作为该槽的语义线索;
|
||||
* 可选: 读一份已解密的真机 collect 明文, 与槽位表并排打印, 人工/程序核对。
|
||||
|
||||
用法:
|
||||
# 只看槽位表
|
||||
.venv/bin/python python -m tcaptcha.telemetry.layout tcaptcha/core_js/tdc_sample.js
|
||||
|
||||
# 和真机明文并排核对 (明文是怎么来的: 同会话 tdc.js 密钥解密 verify 的 collect)
|
||||
.venv/bin/python python -m tcaptcha.telemetry.layout <tdc.js> --plain /tmp/real_collect_plain.txt
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Sequence, Tuple
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if str(ROOT / "tools") not in sys.path:
|
||||
sys.path.insert(0, str(ROOT / "tools"))
|
||||
|
||||
from tcaptcha.telemetry import program as tp # noqa: E402
|
||||
|
||||
# 采集器用来读环境的属性名（用于给槽位打语义标签, 不是判定依据）
|
||||
ENV_KEYS = [
|
||||
"userAgent", "platform", "languages", "language", "hardwareConcurrency", "deviceMemory",
|
||||
"colorDepth", "pixelDepth", "availHeight", "availWidth", "deviceXDPI", "logicalXDPI",
|
||||
"fontSmoothingEnabled", "appVersion", "appName", "vendor", "cookieEnabled", "timezone",
|
||||
"getTimezoneOffset", "toDataURL", "getParameter", "getExtension", "TCaptchaReferrer",
|
||||
"webdriver", "referrer", "href", "width", "height", "innerWidth", "innerHeight",
|
||||
"devicePixelRatio", "characterSet", "MAX_TEXTURE_SIZE", "WEBGL_debug_renderer_info",
|
||||
]
|
||||
|
||||
|
||||
def _fold_str_before(code: List[dict], end: int, span: int = 40) -> Optional[str]:
|
||||
"""从 end 往前折叠一个字面串（PUSH_EMPTY_STR + 连续 PUSH_STR_CHAR）。"""
|
||||
chars: List[str] = []
|
||||
i = end
|
||||
while i >= 0 and i > end - span and code[i]["name"] == "PUSH_STR_CHAR":
|
||||
chars.append(chr(code[i]["operands"][0] & 0xFF))
|
||||
i -= 1
|
||||
if i >= 0 and code[i]["name"] == "PUSH_EMPTY_STR":
|
||||
return "".join(reversed(chars))
|
||||
return None
|
||||
|
||||
|
||||
def slot_value_expr(p: tp.Program, entry: int) -> Optional[str]:
|
||||
"""**精确**取一个槽位模块"这一槽的值是什么"。
|
||||
|
||||
规则（实测形状）: 采集器模块最后做
|
||||
r3.exports.get = function () { ... return <被捕获的那个寄存器> ... }
|
||||
也就是 `get` 闭包**只返回它捕获的寄存器**。所以:
|
||||
1. 在工厂里找到被赋给属性 `get` 的那个 CREATE_CLOSURE;
|
||||
2. 从它的 caps 里取出**父寄存器号**;
|
||||
3. 该寄存器在工厂里的赋值表达式, 就是这一槽的语义。
|
||||
例: 模块 18 -> caps=[r3=r6], 工厂里 `r6 = c.navigator.userAgent`。
|
||||
"""
|
||||
code = p.function_code(entry)
|
||||
assigns = dict(p.symbolic_assignments(entry, limit=60))
|
||||
for k, inst in enumerate(code):
|
||||
if inst["name"] != "CREATE_CLOSURE":
|
||||
continue
|
||||
# 形状: <"get" 字面量>; REVERSE_TOP2; JUMP -> @site; @site: CREATE_CLOSURE
|
||||
j = k - 1
|
||||
if j >= 0 and code[j]["name"] == "JUMP":
|
||||
j -= 1
|
||||
if _fold_str_before(code, j) != "get":
|
||||
continue
|
||||
site = next((c for c in p.closures if c.pc == inst["pc"]), None)
|
||||
if site is None:
|
||||
continue
|
||||
for _child, parent in site.caps: # caps 是 (子寄存器, 父寄存器)
|
||||
expr = assigns.get(f"r{parent}")
|
||||
if expr:
|
||||
return expr
|
||||
# 兜底（覆盖更广）: 采集器模块里通常**只有一个闭包**，它捕获的那个父寄存器就是取值寄存器
|
||||
for c in p.closures:
|
||||
if p.owner(c.pc) != entry or len(c.caps) != 1:
|
||||
continue
|
||||
_child, parent = c.caps[0]
|
||||
expr = assigns.get(f"r{parent}")
|
||||
if expr:
|
||||
return expr
|
||||
return None
|
||||
|
||||
|
||||
def slot_semantics(p: tp.Program, reg: Dict[int, int], mid: int, depth: int = 2) -> List[str]:
|
||||
"""还原某个槽位子模块"到底取什么值"。
|
||||
|
||||
三条线索合并（都来自 tdc_program 的符号求值, 不跑 JS）:
|
||||
1. 模块工厂自身的赋值链, 例如 `r6 = c.navigator.userAgent`;
|
||||
2. **委派**: 有些模块只是 `r6 = r5(N)`（require 另一个模块再导出），要跟进去;
|
||||
3. **`get` 闭包内部**: canvas/WebGL 这类是延迟取值, 真正读环境在它内部创建的闭包里。
|
||||
"""
|
||||
exprs: List[str] = []
|
||||
seen: set = set()
|
||||
|
||||
def visit(m: int, d: int) -> None:
|
||||
if m in seen or d < 0 or m not in reg:
|
||||
return
|
||||
seen.add(m)
|
||||
entry = reg[m]
|
||||
for _r, v in p.symbolic_assignments(entry, limit=14):
|
||||
v = v.strip()
|
||||
if not v:
|
||||
continue
|
||||
exprs.append(v)
|
||||
mm = re.match(r"r\d+\((\d+)\)$", v) # 纯委派 r6 = r5(N)
|
||||
if mm:
|
||||
visit(int(mm.group(1)), d - 1)
|
||||
for c in p.closures: # 该模块内部创建的闭包（含 get 实现）
|
||||
if p.owner(c.pc) != entry:
|
||||
continue
|
||||
for _r, v in p.symbolic_assignments(c.target, limit=10):
|
||||
if v.strip():
|
||||
exprs.append(v.strip())
|
||||
mm = re.match(r"r\d+\((\d+)\)$", v.strip())
|
||||
if mm:
|
||||
visit(int(mm.group(1)), d - 1)
|
||||
|
||||
visit(mid, depth)
|
||||
uniq, out = set(), []
|
||||
for e in exprs:
|
||||
if e not in uniq:
|
||||
uniq.add(e)
|
||||
out.append(e)
|
||||
return out
|
||||
|
||||
|
||||
_ENV_RE = re.compile(r"c\.|screen|navigator|document|window")
|
||||
_DELEG_RE = re.compile(r"r\d+\(\d*\)\s*$")
|
||||
|
||||
|
||||
def _is_delegation(e: str) -> bool:
|
||||
"""纯模块委派 `r5(22)` / `r5()` —— 括号里的模块号每份构建重排，不能进指纹。"""
|
||||
return bool(_DELEG_RE.fullmatch(e))
|
||||
|
||||
|
||||
def static_fingerprint(p: tp.Program, reg: Dict[int, int], mid: int,
|
||||
depth: int = 4, keep: int = 3) -> str:
|
||||
"""模块的**静态语义指纹** —— 跨构建对齐的主锚点（2026-10-12 实测）。
|
||||
|
||||
为什么不用模块 id（实测证伪）：模块 id 是编译器序号，每份构建重新洗牌。
|
||||
两份构建清单交集的 28 个模块里 `id -> 语义` 完全相同的只有 **1/11**
|
||||
（模块9 在 A 是 canvas，在 B 是 webkitRTCPeerConnection）。按 id 对齐必错位。
|
||||
|
||||
构造规则（两条，都是为了抹掉"会重排"的部分、留下"不会变"的部分）:
|
||||
1. 丢掉纯模块委派 `r5(22)` —— 括号里的模块号会重排;
|
||||
2. 其余符号表达式**排序**后取前 keep 条 —— 排序消除遍历顺序差异。
|
||||
|
||||
实测（har/1ms_dianxuan/tdc.js vs live/tdc_fresh.js）:
|
||||
* 覆盖 29/37 项（比只取"环境路径"的 24/37 高不少 —— 常量槽、结构槽也有稳定形状）
|
||||
* 两份构建指纹**多重集交集 27/29**
|
||||
* 单构建内 **27/29 唯一**（只有 "Not supported | createElement(canvas)" 这一组重复）
|
||||
"""
|
||||
keepers: List[str] = []
|
||||
for e in slot_semantics(p, reg, mid, depth=depth):
|
||||
e = e.strip()
|
||||
if not e or _is_delegation(e):
|
||||
continue
|
||||
keepers.append(e)
|
||||
return " | ".join(sorted(set(keepers))[:keep])
|
||||
|
||||
|
||||
def env_fingerprint(p: tp.Program, reg: Dict[int, int], mid: int,
|
||||
depth: int = 4, keep: int = 3) -> str:
|
||||
"""只保留**读环境**的表达式（`c.`/screen/navigator/document/window）。
|
||||
|
||||
覆盖比 `static_fingerprint` 低（24/37 vs 29/37），仅在需要"这一槽读什么环境"
|
||||
这种**人类可读标签**时使用。做**对齐**请用 static_fingerprint。
|
||||
"""
|
||||
keepers: List[str] = []
|
||||
for e in slot_semantics(p, reg, mid, depth=depth):
|
||||
e = e.strip()
|
||||
if not e or _is_delegation(e) or not _ENV_RE.search(e):
|
||||
continue
|
||||
keepers.append(e)
|
||||
return " | ".join(sorted(set(keepers))[:keep])
|
||||
|
||||
|
||||
def opcode_histogram(p: tp.Program, reg: Dict[int, int], mid: int) -> str:
|
||||
"""模块的**指令直方图** —— 覆盖 37/37、跨构建多重集交集 37/37 的兜底锚点。
|
||||
|
||||
有些采集器（常量槽、纯闭包槽）的符号求值拿不到表达式（实测 8/37 项返回空串），
|
||||
但它们**函数体的 opcode 分布**是稳定的：编译器把同一份源码编成同一串指令，
|
||||
只是模块 id 换了。实测两份构建 37/37 全部匹配。
|
||||
|
||||
缺点：唯一性差一些（单构建内 20/37 唯一），所以只能当**第三级**锚点，
|
||||
在静态指纹匹配不上时兜底，并且同样要求"双向唯一"才配对。
|
||||
"""
|
||||
entry = reg.get(mid)
|
||||
if entry is None:
|
||||
return ""
|
||||
counts = Counter(inst["name"] for inst in p.function_code(entry))
|
||||
return "|".join(f"{k}:{v}" for k, v in sorted(counts.items()))
|
||||
|
||||
|
||||
def item_fingerprints(p: tp.Program, seq: Sequence[int]) -> List[str]:
|
||||
"""按清单顺序, 给每一项算出静态语义指纹（与 seq 等长）。"""
|
||||
reg = dict(p.module_registry(0))
|
||||
return [static_fingerprint(p, reg, m) for m in seq]
|
||||
|
||||
|
||||
def item_histograms(p: tp.Program, seq: Sequence[int]) -> List[str]:
|
||||
"""按清单顺序, 给每一项算出指令直方图（与 seq 等长, 兜底锚点）。"""
|
||||
reg = dict(p.module_registry(0))
|
||||
return [opcode_histogram(p, reg, m) for m in seq]
|
||||
|
||||
|
||||
def find_manifest_module(p: tp.Program) -> Tuple[int, int, List[int]]:
|
||||
"""找"清单模块": 体内 `push(require(N))` 次数最多者。
|
||||
|
||||
返回 (模块 id, 工厂 entry, require 序列)。
|
||||
"""
|
||||
reg = dict(p.module_registry(0))
|
||||
best = (0, -1, [])
|
||||
for mid, entry in reg.items():
|
||||
code = p.function_code(entry)
|
||||
seq: List[int] = []
|
||||
for k, inst in enumerate(code):
|
||||
if inst["name"] != "PUSH_IMM" or not inst["operands"]:
|
||||
continue
|
||||
if k >= 1 and code[k - 1]["name"] == "LOAD_REG_IMM" \
|
||||
and k + 1 < len(code) and code[k + 1]["name"] == "CALL_FUNC":
|
||||
seq.append(int(inst["operands"][0]))
|
||||
if len(seq) > len(best[2]):
|
||||
best = (mid, entry, seq)
|
||||
return best
|
||||
|
||||
|
||||
def transitive_strings(p: tp.Program, entry: int, depth: int = 3) -> List[str]:
|
||||
"""模块自身 + 它内部创建的闭包（递归 depth 层）里的字面串。"""
|
||||
seen, out = set(), []
|
||||
frontier = {entry}
|
||||
for _ in range(depth):
|
||||
nxt = set()
|
||||
for e in frontier:
|
||||
if e in seen:
|
||||
continue
|
||||
seen.add(e)
|
||||
for pc, s in p.strings:
|
||||
if p.owner(pc) == e:
|
||||
out.append(s)
|
||||
for c in p.closures:
|
||||
if p.owner(c.pc) == e:
|
||||
nxt.add(c.target)
|
||||
frontier = nxt
|
||||
return out
|
||||
|
||||
|
||||
def env_hints(strings: List[str]) -> List[str]:
|
||||
hits = []
|
||||
for s in strings:
|
||||
for k in ENV_KEYS:
|
||||
if k.lower() in s.lower() and k not in hits:
|
||||
hits.append(k)
|
||||
return hits
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="静态导出 collect 的 cd 槽位表")
|
||||
ap.add_argument("path")
|
||||
ap.add_argument("--plain", default=None, help="已解密的真机 collect 明文 (json 文本)")
|
||||
ap.add_argument("--json", default=None, help="把槽位表写成 JSON")
|
||||
args = ap.parse_args()
|
||||
|
||||
p = tp.load(args.path)
|
||||
reg = dict(p.module_registry(0))
|
||||
mid, entry, seq = find_manifest_module(p)
|
||||
print(f"文件 : {args.path}")
|
||||
print(f"模块总数 : {len(reg)}")
|
||||
print(f"清单模块 : registry[{mid}] = @{entry}")
|
||||
print(f"槽位序列长度 : {len(seq)} (真机 cd 槽数 = 38, 差的部分是字面量槽)")
|
||||
print(f"require 序列 : {seq}")
|
||||
|
||||
plain_cd: Optional[list] = None
|
||||
if args.plain:
|
||||
txt = open(args.plain, encoding="latin1").read().rstrip("\x00").strip()
|
||||
obj = json.loads(txt)
|
||||
plain_cd = obj.get("cd")
|
||||
print(f"真机明文 cd 槽数: {len(plain_cd)}")
|
||||
|
||||
table = []
|
||||
print("\n槽位 模块id 工厂entry 取值语义(静态还原) 真机值")
|
||||
print("-" * 112)
|
||||
for i, m in enumerate(seq):
|
||||
tgt = reg.get(m)
|
||||
expr = slot_value_expr(p, tgt) if tgt else None
|
||||
# 有些模块只是"再 require 一层"（r6 = r5(M)），跟进一层
|
||||
mm = re.match(r"r\d+\((\d+)\)$", (expr or "").strip())
|
||||
if mm and int(mm.group(1)) in reg:
|
||||
sub = slot_value_expr(p, reg[int(mm.group(1))])
|
||||
if sub:
|
||||
expr = f"{sub} [via require({mm.group(1)})]"
|
||||
if not expr:
|
||||
expr = "; ".join(slot_semantics(p, reg, m)[:2]) if tgt else ""
|
||||
val = ""
|
||||
if plain_cd is not None and i < len(plain_cd):
|
||||
v = repr(plain_cd[i])
|
||||
val = v if len(v) <= 30 else v[:30] + "..."
|
||||
sem = expr or ""
|
||||
if len(sem) > 44:
|
||||
sem = sem[:44] + "..."
|
||||
table.append({"slot": i, "module": m, "entry": tgt, "expr": expr})
|
||||
print(f" {i:2d} {m:4d} @{tgt:<7} {sem:<46} {val}")
|
||||
|
||||
if plain_cd is not None:
|
||||
print(f"\n(真机 cd 里还有 {len(plain_cd) - len(seq)} 个槽不来自 require —— "
|
||||
f"通常是字面量, 例如 cd[0] 的空串)")
|
||||
if args.json:
|
||||
Path(args.json).write_text(json.dumps({"manifestModule": mid, "entry": entry,
|
||||
"slots": table}, ensure_ascii=False, indent=1),
|
||||
encoding="utf-8")
|
||||
print(f"槽位表 -> {args.json}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,452 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
tdc_modules.py —— **按案例 §5.3 的字符串指纹识别 37 个采集器模块**。
|
||||
|
||||
> "从 v1 的全部错位到指纹方案的全部正确，关键不是更复杂的分析，
|
||||
> 而是换了一个更好的问题 —— 不再问『这个模块返回什么值』，
|
||||
> 而是问『这个模块的代码长什么样』。" —— docs/tdc-案例.md §5.3
|
||||
|
||||
为什么必须做这件事（2026-10-12 实测，推翻项目早期假设）:
|
||||
* **模块 id 每份构建重新分配**。两份构建清单交集的 28 个模块里，
|
||||
`id -> 语义` 完全相同的只有 **1/11**（模块 8 在 A 是 `navigator.platform`，
|
||||
在 B 是 canvas 指纹）。按 id 对齐必然错位。
|
||||
* 但**模块代码里的字符串集合跨构建完全不变**：采集 userAgent 的模块
|
||||
一定含 `'userAgent'`；采集时区的一定含 `'getTimezoneOffset'`。这就是不变量。
|
||||
|
||||
本模块做三件事:
|
||||
1. `module_strings()` —— 取一个模块的**可达**字符串集合（含内部闭包，递归委派）
|
||||
2. `resolve_delegate()` —— 解析**委派链**（见下）
|
||||
3. `identify()` —— 用指纹规则表把模块判成语义名
|
||||
|
||||
委派链（本项目新发现，案例未提但实测必需）:
|
||||
清单里有 6 个模块**自身不采集任何东西**，身体只是 `exports.get = require(N)`：
|
||||
项8 模块21 -> require(22)
|
||||
项9 模块24 -> require(25)
|
||||
项27 模块49 -> require(31)
|
||||
项31 模块53 -> require(0)
|
||||
它们的字符串集合只有 {'Array','Object','exports','get'} —— **零区分性**，
|
||||
必须顺着委派链跟到真正的采集器才能识别。这 6 个就是案例 §5.2 说的
|
||||
"6 个无 .get 模块"的来源之一。
|
||||
|
||||
用法:
|
||||
from tdc_modules import analyze
|
||||
info = analyze("har/1ms_dianxuan/tdc.js")
|
||||
for row in info:
|
||||
print(row["item"], row["module"], row["name"], row["strings"][:4])
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter, defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Sequence, Set, Tuple
|
||||
|
||||
from tcaptcha.telemetry import layout
|
||||
from tcaptcha.telemetry import program as tp
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 通用字符串（每个模块都有，无区分性，进指纹前必须剔除） #
|
||||
# --------------------------------------------------------------------------- #
|
||||
NOISE: Set[str] = {
|
||||
"Array", "Object", "exports", "get", "Math", "RegExp", "String", "Number",
|
||||
"JSON", "set", "reset", "on", "length", "slice", "prototype", "constructor",
|
||||
"push", "apply", "call", "bind", "toString", "valueOf", "hasOwnProperty",
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 指纹规则表（案例 §5.3）—— 每个语义名对应一组「必须出现的字符串」 #
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 形式: 语义名 -> (必需串集合, 权重提示)
|
||||
# 判定: 模块的区分串集合 **包含** 某个规则的必需串 -> 命中该规则
|
||||
# 形式: (语义名, 必需串, 禁用串)
|
||||
# 判定: required ⊆ strings 且 forbidden ∩ strings == ∅
|
||||
#
|
||||
# ⚠️ 规则必须**按 specificity 从高到低排**（必需串多的在前）。
|
||||
# 早期版本只按列表顺序取第一个命中，结果 webdriver_detect 被误判成 user_agent
|
||||
# —— 两者都只要求 1 个串，而 user_agent 恰好排在前面。
|
||||
# 给特征明显的规则补上更多必需串（如 webdriver 要求 2 个自动化标记），
|
||||
# 它们就会天然压过 user_agent 这类单串规则。
|
||||
#
|
||||
# fp_salt 之所以需要 forbidden: 它和 iframe_detect 的串集合几乎相同
|
||||
# （都是 appendChild/body/createElement/document），唯一区别是 iframe_detect
|
||||
# 多一个 contentWindow。用禁用串把两者分开。
|
||||
FINGERPRINTS: List[Tuple[str, Set[str], Set[str]]] = [
|
||||
# ---- 复合指纹（多必需串，天然压过单串规则）----
|
||||
("screen_fingerprint", {"screen", "width", "height", "availHeight", "colorDepth"}, set()),
|
||||
("webrtc_ip", {"RTCPeerConnection", "createDataChannel", "createOffer"}, set()),
|
||||
("webgl_renderer", {"UNMASKED_RENDERER_WEBGL", "WEBGL_debug_renderer_info"}, set()),
|
||||
("webgl_vendor", {"UNMASKED_VENDOR_WEBGL", "WEBGL_debug_renderer_info"}, set()),
|
||||
("webdriver_detect", {"$cdc_asdjflasutopfhvcZLmcfl_", "__webdriver_script_fn"}, set()),
|
||||
("canvas_fingerprint", {"ClientJS,org <canvas> 1.0", "#069"}, set()),
|
||||
("dom_compliance", {"::marker", "CSS"}, set()),
|
||||
("template_url", {"?rand=*", "URL"}, set()),
|
||||
("charset", {"characterSet", "charset"}, set()),
|
||||
("viewport_size", {"clientWidth", "clientHeight"}, set()),
|
||||
("time_since_load", {"getTime", "load"}, set()),
|
||||
# 轨迹采集器：真机实测其字符串集是鼠标/触摸事件名（2026-10-12 修正）。
|
||||
# 旧规则写成 {"stringify"} —— 那抓到的是另一个只做 JSON.stringify 的模块，
|
||||
# 结果「空数组」被记成 trajectory，真正的轨迹却在 window_time 名下。
|
||||
("trajectory", {"mouseMoveEvent", "touchStartEvent"}, set()),
|
||||
("struct_76_33", {"stringify"}, set()),
|
||||
# ---- 单串规则（放最后，避免压过上面的复合规则）----
|
||||
("user_agent", {"userAgent"}, set()),
|
||||
("platform", {"platform"}, set()),
|
||||
("languages", {"languages"}, set()),
|
||||
("hardware_concurrency", {"hardwareConcurrency"}, set()),
|
||||
("cookie_enabled", {"cookieEnabled"}, set()),
|
||||
("timezone_offset", {"getTimezoneOffset"}, set()),
|
||||
("itoken", {"TDC_itoken"}, set()),
|
||||
("guid_token", {r"^\d+:\d+$"}, set()),
|
||||
("referrer", {"TCaptchaReferrer"}, set()),
|
||||
("location_href", {"href"}, {"?rand=*", "TCaptchaReferrer"}),
|
||||
("iframe_detect", {"contentWindow"}, set()),
|
||||
("console_detect", {"console"}, set()),
|
||||
("touch_detect", {"ontouchstart"}, set()),
|
||||
("window_time", {"getTime"}, set()),
|
||||
("constant_98k", {"98k"}, set()),
|
||||
("screen_width", {"screen", "width"}, set()),
|
||||
("screen_height", {"screen", "height"}, set()),
|
||||
("screen_color_depth", {"colorDepth"}, set()),
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 结构兜底规则（案例 §5.3: "对于没有区分性字符串的通用模块，
|
||||
# 用 .get 函数的指令数 + entry 函数的指令数 + 特定指令来区分"）
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 形式: (语义名, 判定函数) —— 只在**字符串规则全落空**时启用。
|
||||
#
|
||||
# ⚠️ 命名纪律：下面这些名字**只描述「观测到的结构分组」，不猜测业务语义**。
|
||||
# 早期版本把 entry48/get9/call1 那组臆断成 "itoken_relay" —— 那是没有证据的猜测。
|
||||
# 现在统一用 "struct_<entry>_<get>[_call]" 这种**纯结构标签**：
|
||||
# 它诚实地表达「我们只知道它长这样，还不知道它采集什么」，
|
||||
# 同时又因为跨构建稳定，仍然可以当**对齐锚点**用。
|
||||
#
|
||||
# 实测（4 份全新在线构建 har/online/20261012-01*，2026-10-12）:
|
||||
# entry135/get121/… -> 每份构建恰好 1 个 => fp_salt（随机串每次都变，只能靠结构）
|
||||
# entry46/get9/… -> 每份构建恰好 1 个 => 稳定分组
|
||||
# entry48/get9/call1 -> 每份构建恰好 1 个 => 稳定分组
|
||||
# entry40/get9/… -> 每份构建 **2 个** => 组内无法再分，需运行时取样才能定
|
||||
STRUCT_RULES = [
|
||||
("fp_salt", lambda s: s["entry_instr"] >= 120 and s["get_instr"] >= 100),
|
||||
("struct_46_9", lambda s: s["entry_instr"] == 46 and s["get_instr"] == 9),
|
||||
("struct_48_9_call", lambda s: s["entry_instr"] == 48 and s["get_instr"] == 9
|
||||
and s["call_func"]),
|
||||
("struct_40_9", lambda s: s["entry_instr"] == 40 and s["get_instr"] == 9),
|
||||
]
|
||||
|
||||
|
||||
def struct_signature(p: tp.Program, reg: Dict[int, int], mid: int) -> Dict:
|
||||
"""取一个模块的**结构签名**（案例 §5.3 兜底用）。
|
||||
|
||||
entry_instr = 模块工厂函数体的指令数
|
||||
get_instr = `.get` 闭包（JUMP 目标）函数体的指令数
|
||||
"""
|
||||
entry = reg.get(mid)
|
||||
if entry is None:
|
||||
return {"entry_instr": -1, "get_instr": -1, "call_func": False,
|
||||
"closure_kind": ""}
|
||||
code = p.function_code(entry)
|
||||
ops = Counter(inst["name"] for inst in code)
|
||||
get_n = -1
|
||||
for inst in code:
|
||||
if inst["name"] == "JUMP" and inst["operands"]:
|
||||
try:
|
||||
get_n = len(p.function_code(inst["operands"][0]))
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
return {
|
||||
"entry_instr": len(code),
|
||||
"get_instr": get_n,
|
||||
"call_func": ops.get("CALL_FUNC", 0) > 0,
|
||||
"closure_kind": _closure_kind(p, entry),
|
||||
}
|
||||
|
||||
|
||||
def _closure_kind(p: tp.Program, entry: int) -> str:
|
||||
"""模块唯一闭包**最后写入什么** —— 用于拆开结构完全相同的模块。
|
||||
|
||||
背景（2026-10-12，4 份全新在线构建）:
|
||||
有 2 个模块 entry 都是 40 条、.get 都是 9 条、指令逐条相同，
|
||||
只差一处 JUMP 绝对地址（每构建不同，不能当特征）。
|
||||
真正的区别在**闭包体**（CREATE_CLOSURE 的 target，不是 JUMP 目标）：
|
||||
|
||||
A: new Array()[0] = "" (PUSH_EMPTY_STR; SET_PROP)
|
||||
B: new Array()[0] = 1558325408 (PUSH_IMM <随机int32>; SET_PROP)
|
||||
|
||||
实测每份构建恰好 empty×1 + rand×1 —— 稳定、可静态消歧。
|
||||
|
||||
返回: "empty" | "rand" | "const" | ""（无闭包）
|
||||
"""
|
||||
cls = [c for c in p.closures if p.owner(c.pc) == entry]
|
||||
if not cls:
|
||||
return ""
|
||||
try:
|
||||
body = p.function_code(cls[0].target)
|
||||
except Exception:
|
||||
return ""
|
||||
imms = [inst["operands"][0] for inst in body
|
||||
if inst["name"] == "PUSH_IMM" and inst["operands"]]
|
||||
if any(v > 1_000_000 for v in imms):
|
||||
return "rand"
|
||||
if any(inst["name"] == "PUSH_EMPTY_STR" for inst in body):
|
||||
return "empty"
|
||||
return "const" if imms else ""
|
||||
|
||||
|
||||
def _identify_struct(p: tp.Program, reg: Dict[int, int], mid: int) -> Optional[str]:
|
||||
"""字符串规则落空时的兜底识别。"""
|
||||
sig = struct_signature(p, reg, mid)
|
||||
for name, pred in STRUCT_RULES:
|
||||
try:
|
||||
if pred(sig):
|
||||
return name
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _clean(s: str) -> str:
|
||||
return s.strip()
|
||||
|
||||
|
||||
def module_strings(p: tp.Program, reg: Dict[int, int], mid: int,
|
||||
depth: int = 3) -> Set[str]:
|
||||
"""取一个模块的**可达**字符串集合（模块自身 + 内部闭包，递归 depth 层）。"""
|
||||
entry = reg.get(mid)
|
||||
if entry is None:
|
||||
return set()
|
||||
by_owner: Dict[int, Set[str]] = defaultdict(set)
|
||||
for pc, s in p.strings:
|
||||
by_owner[p.owner(pc)].add(s)
|
||||
|
||||
seen: Set[int] = set()
|
||||
frontier = {entry}
|
||||
out: Set[str] = set()
|
||||
for _ in range(depth):
|
||||
nxt: Set[int] = set()
|
||||
for e in frontier:
|
||||
if e in seen:
|
||||
continue
|
||||
seen.add(e)
|
||||
out |= by_owner.get(e, set())
|
||||
for c in p.closures:
|
||||
if p.owner(c.pc) == e and c.target not in seen:
|
||||
nxt.add(c.target)
|
||||
frontier = nxt
|
||||
if not frontier:
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
def resolve_delegate(p: tp.Program, reg: Dict[int, int], mid: int,
|
||||
max_hops: int = 6) -> List[int]:
|
||||
"""解析模块 m 的委派链: `exports.X = require(N)` 里的 N。
|
||||
|
||||
返回 [m, N1, N2, ...]（含自身）。取不到委派时返回 [m]。
|
||||
|
||||
形状（实测）: 模块体里出现 `LOAD_REG_IMM; PUSH_IMM N; CALL_FUNC` 且 N 在
|
||||
registry 里 -> 该模块委派给 N。有些模块 require 的是自己，要防环。
|
||||
"""
|
||||
chain: List[int] = [mid]
|
||||
seen = {mid}
|
||||
cur = mid
|
||||
for _ in range(max_hops):
|
||||
entry = reg.get(cur)
|
||||
if entry is None:
|
||||
break
|
||||
code = p.function_code(entry)
|
||||
nxt: Optional[int] = None
|
||||
for k, inst in enumerate(code):
|
||||
if inst["name"] != "PUSH_IMM" or not inst["operands"]:
|
||||
continue
|
||||
if k >= 1 and code[k - 1]["name"] == "LOAD_REG_IMM" \
|
||||
and k + 1 < len(code) and code[k + 1]["name"] == "CALL_FUNC":
|
||||
cand = int(inst["operands"][0])
|
||||
if cand in reg and cand not in seen:
|
||||
nxt = cand
|
||||
break
|
||||
if nxt is None:
|
||||
break
|
||||
chain.append(nxt)
|
||||
seen.add(nxt)
|
||||
cur = nxt
|
||||
return chain
|
||||
|
||||
|
||||
def _has(strings: Set[str], token: str) -> bool:
|
||||
"""判断 token 是否出现在字符串集合里。
|
||||
|
||||
两种模式:
|
||||
* 普通 token -> **精确**相等
|
||||
* `前缀*` 结尾 -> **子串/前缀**匹配
|
||||
|
||||
为什么需要后者（踩过的坑）: `?rand=` 在真机里是 `?rand=1519713624347`
|
||||
（带会话随机数），精确匹配永远命中不了，导致 template_url 被误判成
|
||||
location_href。凡是"带运行时随机后缀"的串都必须用前缀匹配。
|
||||
"""
|
||||
if token in strings:
|
||||
return True
|
||||
if token.endswith("*"):
|
||||
stem = token[:-1]
|
||||
return any(stem in s for s in strings)
|
||||
return False
|
||||
|
||||
|
||||
def _matches(strings: Set[str]) -> List[Tuple[str, int, Set[str]]]:
|
||||
"""列出所有命中的规则，按 specificity（必需串个数）降序。"""
|
||||
hits: List[Tuple[str, int, Set[str]]] = []
|
||||
for name, required, forbidden in FINGERPRINTS:
|
||||
if not all(_has(strings, t) for t in required):
|
||||
continue
|
||||
if any(_has(strings, t) for t in forbidden):
|
||||
continue
|
||||
hits.append((name, len(required), required))
|
||||
hits.sort(key=lambda x: -x[1])
|
||||
return hits
|
||||
|
||||
|
||||
def _identify(strings: Set[str]) -> Tuple[str, Set[str]]:
|
||||
"""用指纹规则判定语义名。返回 (名字, 命中的必需串)。
|
||||
|
||||
取**必需串最多**的那条规则（specificity 优先），而不是列表里第一条 ——
|
||||
这样 `webdriver_detect`（2 个必需串）不会被 `user_agent`（1 个）抢走。
|
||||
"""
|
||||
hits = _matches(strings)
|
||||
if not hits:
|
||||
return "unknown", set()
|
||||
return hits[0][0], hits[0][2]
|
||||
|
||||
|
||||
def disambiguate(rows: List[Dict]) -> None:
|
||||
"""给**重名**的项加稳定后缀，使 37 个锚点尽可能唯一（原地修改 name）。
|
||||
|
||||
为什么需要这一步（2026-10-12 实测，只看 har/online/ 全新样本）:
|
||||
字符串规则会把不同采集器判成同一个名字。跨 4 份新构建统计，重名是**稳定的**：
|
||||
|
||||
user_agent x2 -> entry48(委派链) / entry68(直判) 每份各 1 个 ✓ 可消歧
|
||||
window_time x3 -> entry279 / entry40 / entry48(委派链) 每份各 1 个 ✓ 可消歧
|
||||
struct_40_9 x2 -> 结构完全相同 每份各 2 个 ✗ 静态不可分
|
||||
|
||||
消歧后缀用**结构签名**（entry/get 指令数 + 是否含 CALL_FUNC），因为它是
|
||||
跨构建稳定的实测分组依据，而不是猜的语义。
|
||||
|
||||
⚠️ struct_40_9 那组每份构建固定 2 个、结构完全一致 —— 这是**静态识别的能力上限**。
|
||||
要靠运行时取样（TDC.getData 对照）才能分开。本函数会如实保留重名，
|
||||
并在 row["ambiguous_static"] 上标记，调用方据此决定是否需要运行时标定。
|
||||
"""
|
||||
groups: Dict[str, List[Dict]] = defaultdict(list)
|
||||
for r in rows:
|
||||
groups[r["name"]].append(r)
|
||||
|
||||
for name, members in groups.items():
|
||||
if len(members) <= 1:
|
||||
continue
|
||||
for r in members:
|
||||
s = r.get("struct") or {}
|
||||
tag = "e%s_g%s%s%s" % (s.get("entry_instr", "-"),
|
||||
s.get("get_instr", "-"),
|
||||
"_call" if s.get("call_func") else "",
|
||||
# 结构完全相同时，用"闭包最后写入什么"再拆一层
|
||||
("_" + s["closure_kind"]) if s.get("closure_kind") else "")
|
||||
# 同组内的 tag 是否唯一？唯一才能消歧
|
||||
r["_tag"] = tag
|
||||
tags = [r["_tag"] for r in members]
|
||||
can_split = len(set(tags)) == len(tags)
|
||||
for r in members:
|
||||
r["ambiguous_static"] = not can_split
|
||||
r["name_unique"] = "%s#%s" % (name, r["_tag"]) if can_split else name
|
||||
if can_split:
|
||||
for r in members:
|
||||
r["name"] = r["name_unique"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 主入口 #
|
||||
# --------------------------------------------------------------------------- #
|
||||
def analyze_prog(p: tp.Program, depth: int = 3) -> List[Dict]:
|
||||
"""分析一个**已加载**的 Program（供调用方复用，避免重复解析 tdc.js）。"""
|
||||
reg = dict(p.module_registry(0))
|
||||
_mid, _entry, seq = layout.find_manifest_module(p)
|
||||
|
||||
rows: List[Dict] = []
|
||||
for i, m in enumerate(seq):
|
||||
own = module_strings(p, reg, m, depth=depth)
|
||||
chain = resolve_delegate(p, reg, m)
|
||||
# 顺着委派链往下找，直到能识别出语义（或链走完）
|
||||
name, hit = _identify(own - NOISE)
|
||||
used_chain = False
|
||||
if name == "unknown" and len(chain) > 1:
|
||||
for deeper in chain[1:]:
|
||||
s = module_strings(p, reg, deeper, depth=depth)
|
||||
n2, h2 = _identify(s - NOISE)
|
||||
if n2 != "unknown":
|
||||
name, hit = n2, h2
|
||||
used_chain = True
|
||||
break
|
||||
# 字符串规则全落空 -> 启用案例 §5.3 的结构兜底（按指令数识别）
|
||||
via_struct = False
|
||||
if name == "unknown":
|
||||
n3 = _identify_struct(p, reg, m)
|
||||
if n3:
|
||||
name, hit = n3, set()
|
||||
via_struct = True
|
||||
sig = sorted((own - NOISE))
|
||||
rows.append({
|
||||
"item": i,
|
||||
"module": m,
|
||||
"name": name,
|
||||
"hit": sorted(hit),
|
||||
"delegate": chain if len(chain) > 1 else [],
|
||||
"via_delegate": used_chain,
|
||||
"via_struct": via_struct,
|
||||
"struct": struct_signature(p, reg, m),
|
||||
"ambiguous_static": False,
|
||||
"strings": sig,
|
||||
})
|
||||
disambiguate(rows)
|
||||
return rows
|
||||
|
||||
|
||||
def analyze(path: str, depth: int = 3) -> List[Dict]:
|
||||
"""分析一份 tdc.js，返回清单每一项的识别结果（按清单顺序）。"""
|
||||
return analyze_prog(tp.load(path), depth=depth)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
import argparse
|
||||
import json
|
||||
|
||||
ap = argparse.ArgumentParser(description="按字符串指纹识别 37 个采集器模块")
|
||||
ap.add_argument("path")
|
||||
ap.add_argument("--json", default=None, help="把结果写成 JSON")
|
||||
ap.add_argument("--only-unknown", action="store_true", help="只列出未识别的")
|
||||
args = ap.parse_args()
|
||||
|
||||
rows = analyze(args.path)
|
||||
unknown = [r for r in rows if r["name"] == "unknown"]
|
||||
print("文件 : %s" % args.path)
|
||||
print("清单项 : %d 已识别 %d 未识别 %d"
|
||||
% (len(rows), len(rows) - len(unknown), len(unknown)))
|
||||
print()
|
||||
print("项 模块 语义名 委派链 区分串")
|
||||
print("-" * 108)
|
||||
for r in rows:
|
||||
if args.only_unknown and r["name"] != "unknown":
|
||||
continue
|
||||
dele = "->".join(str(x) for x in r["delegate"]) if r["delegate"] else ""
|
||||
mark = " *" if r["via_delegate"] else " "
|
||||
print("%2d %-5d %s%-21s %-17s %s"
|
||||
% (r["item"], r["module"], mark, r["name"], dele,
|
||||
r["strings"][:5]))
|
||||
if args.json:
|
||||
Path(args.json).write_text(
|
||||
json.dumps(rows, ensure_ascii=False, indent=1), encoding="utf-8")
|
||||
print("\n结果 -> %s" % args.json)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,320 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
腾讯天御 TDC 混沌 VM (Chaos VM) —— **按构建推导操作码表**。
|
||||
|
||||
为什么需要它
|
||||
------------
|
||||
`telemetry/disasm.py` 里固化的 opcode 表取自**内置样本 tcaptcha/tdc_sample.js**。
|
||||
实测(5 份构建交叉验证): **opcode 编号每份构建都会整体重排** ——
|
||||
同一份源码编译出的字节码, pc0 处的 `SET_STACK_LEN` 在 core_js 是 53,
|
||||
在 root 样本是 19, 在 1ms 实时样本分别是 18 / 10 / 60。
|
||||
所以写死 opcode 表 = 换一份脚本就全错。
|
||||
|
||||
本模块的做法:
|
||||
1. 从 JS 源码里定位解释器主循环 `X[Y[Z++]]()` -> 取出**操作码处理器数组变量** `X`;
|
||||
2. 解析该数组字面量(JS 感知的括号/字符串/正则切分)得到 70 个槽位(含空洞);
|
||||
3. 把每个 handler 的源码**规范化**(去空白 + 局部变量按首次出现重命名为 v1,v2,...,
|
||||
保留 JS 关键字/内建名与属性名)得到与变量命名无关的"指纹";
|
||||
4. 用内置的 58 条参考指纹(取自内置样本 = `telemetry.disasm.OP_NAMES` 那套语义)
|
||||
反查每个槽位的语义名 —— 数组下标即该构建的 opcode。
|
||||
5. 操作数个数按语义名从参考表取 (与"handler 体内 `x[y++]` 读取次数"实测一致)。
|
||||
|
||||
这样, 只要服务端还是这台 VM(handler 语义相同), 无论 opcode 怎么重排都能对上。
|
||||
```
|
||||
经过: 5/5 本构建全部 58/58 handler 精确匹配 (har/1ms_dianxuan/PROGRESS.md)。
|
||||
```
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
if _ROOT not in sys.path:
|
||||
sys.path.insert(0, _ROOT)
|
||||
|
||||
from tcaptcha.telemetry.disasm import ARITY as _CORE_ARITY, DYNAMIC_ARITY as _CORE_DYNAMIC, OP_NAMES as _CORE_NAMES # noqa: E402
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 参考指纹: 内置样本构建的 58 个 handler 规范化源码 -> 语义名 (与 OP_NAMES 一致)
|
||||
# --------------------------------------------------------------------------- #
|
||||
_REF_BLOB = r"""PUSH_STR_CHAR function(){v1[v1.length-1]+=String.fromCharCode(v2[v3++])}
|
||||
IS_EXCEPTION function(){return!!v1}
|
||||
DUP function(){v1.push(v1[v1.length-1])}
|
||||
XOR function(){v1[v1.length-2]=v1[v1.length-2]^v1.pop()}
|
||||
EQ_STRICT function(){v1[v1.length-2]=v1[v1.length-2]===v1.pop()}
|
||||
MOD function(){v1[v1.length-2]=v1[v1.length-2]%v1.pop()}
|
||||
CALL_FUNC function(){varv1=v2[v3++],v4=v1?v5.slice(-v1):[];v5.length-=v1,v5.push(v5.pop().apply(v6,v4))}
|
||||
GET_GLOBAL_PROP function(){v1[v1.length-1]=v2[v1[v1.length-1]]}
|
||||
MAKE_PROP_REF function(){varv1=v2.pop(),v3=v2.pop();v2.push([v3[0][v3[1]],v1])}
|
||||
TYPEOF function(){v1.push(typeofv1.pop())}
|
||||
DELETE function(){varv1=v2.pop();v2.push(deletev1[0][v1[1]])}
|
||||
LOGICAL_NOT function(){v1.push(!v1.pop())}
|
||||
EQ_LOOSE function(){v1[v1.length-2]=v1[v1.length-2]==v1.pop()}
|
||||
SET_PROP function(){varv1=v2[v2.length-2];v1[0][v1[1]]=v2[v2.length-1]}
|
||||
GREATER_THAN function(){v1[v1.length-2]=v1[v1.length-2]>v1.pop()}
|
||||
SWAP function(){varv1=v2[v3++],v4=v5[v5.length-2-v1];v5[v5.length-2-v1]=v5.pop(),v5.push(v4)}
|
||||
IN function(){v1[v1.length-2]=v1[v1.length-2]inv1.pop()}
|
||||
RET_TRUE function(){return!0}
|
||||
GREATER_EQ function(){v1[v1.length-2]=v1[v1.length-2]>=v1.pop()}
|
||||
PUSH_IMM function(){v1.push(v2[v3++])}
|
||||
MUL function(){v1[v1.length-2]=v1[v1.length-2]*v1.pop()}
|
||||
LOAD_REG_INDIRECT function(){v1.push(v1[v1.pop()[0]][0])}
|
||||
ITER_NEXT function(){v1[v1.length-1].length?v1.push(v1[v1.length-1].shift(),!0):v1.push(undefined,!1)}
|
||||
BIND_THIS function(){v1.push([v2,v1.pop()])}
|
||||
PUSH_NULL function(){v1.push(null)}
|
||||
BITWISE_AND function(){v1[v1.length-2]=v1[v1.length-2]&v1.pop()}
|
||||
PUSH_IMM_ARRAY function(){v1.push([v2[v3++]])}
|
||||
MAKE_REG_PROP_REF function(){varv1=v2.pop();v2.push([v2[v2.pop()][0],v1])}
|
||||
SUB function(){v1[v1.length-2]=v1[v1.length-2]-v1.pop()}
|
||||
CALL_METHOD function(){varv1=v2[v3++],v4=v1?v5.slice(-v1):[],v1=(v5.length-=v1,v5.pop());v5.push(v1[0][v1[1]].apply(v1[0],v4))}
|
||||
SHR function(){v1[v1.length-2]=v1[v1.length-2]>>v1.pop()}
|
||||
SHL function(){v1[v1.length-2]=v1[v1.length-2]<<v1.pop()}
|
||||
JUMP_IF_TRUE function(){varv1=v2[v3++];v4[v4.length-1]&&(v3=v1)}
|
||||
FOR_IN_KEYS function(){varv1,v2=[];for(v1inv3.pop())v2.push(v1);v3.push(v2)}
|
||||
ADD function(){v1[v1.length-2]=v1[v1.length-2]+v1.pop()}
|
||||
NEW_METHOD function(){varv1=v2[v3++],v4=v1?v5.slice(-v1):[],v1=(v5.length-=v1,v4.unshift(null),v5.pop());v5.push(v6(v1[0][v1[1]],v4))}
|
||||
PUSH_UNDEFINED function(){v1.push(undefined)}
|
||||
PUSH_FALSE function(){v1.push(!1)}
|
||||
STORE_REG function(){v1[v1[v1.length-2][0]][0]=v1[v1.length-1]}
|
||||
ALLOC_REG function(){varv1=v2[v3++];v4[v1]=v4[v1]===undefined?[]:v4[v1]}
|
||||
POP_TRY function(){v1.pop()}
|
||||
THROW function(){throwv1[v1.length-1]}
|
||||
POP function(){v1.pop()}
|
||||
DIV function(){v1[v1.length-2]=v1[v1.length-2]/v1.pop()}
|
||||
SET_STACK_LEN function(){v1.length=v2[v3++]}
|
||||
REVERSE_TOP2 function(){v1.push([v1.pop(),v1.pop()].reverse())}
|
||||
PUSH_EMPTY_STR function(){v1.push("")}
|
||||
PUSH_TRY function(){v1.push([v2[v3++],v4.length,v2[v3++]])}
|
||||
CREATE_CLOSURE function(){for(varv1=v2[v3++],v4=[],v5=v2[v3++],v6=v2[v3++],v7=[],v8=0;v8<v5;v8++)v4[v2[v3++]]=v9[v2[v3++]];for(v8=0;v8<v6;v8++)v7[v8]=v2[v3++];v9.push(functionv10(){varv5=v4.slice(0);v5[0]=[this],v5[1]=[arguments],v5[2]=[v10];for(varv6=0;v6<v7.length&&v6<arguments.length;v6++)0<v7[v6]&&(v5[v7[v6]]=[arguments[v6]]);returnv11(v1,v2,v12,v5,v13,v14,v15,v16)})}
|
||||
GET_PROP function(){varv1=v2.pop();v2.push(v1[0][v1[1]])}
|
||||
JUMP function(){v1=v2[v1++]}
|
||||
NEW_FUNC function(){varv1=v2[v3++],v4=v1?v5.slice(-v1):[];v5.length-=v1,v4.unshift(null),v5.push(v6(v5.pop(),v4))}
|
||||
BITWISE_OR function(){v1[v1.length-2]=v1[v1.length-2]|v1.pop()}
|
||||
REPLACE_TOP_IMM function(){v1[v1.length-1]=v2[v3++]}
|
||||
SHR_UNSIGNED function(){v1[v1.length-2]=v1[v1.length-2]>>>v1.pop()}
|
||||
PUSH_TRUE function(){v1.push(!0)}
|
||||
LOAD_REG_IMM function(){v1.push(v1[v2[v3++]][0])}
|
||||
CLEAR_EXCEPTION function(){v1=null}"""
|
||||
|
||||
|
||||
def _reference() -> Dict[str, str]:
|
||||
out: Dict[str, str] = {}
|
||||
for line in _REF_BLOB.splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
name, _, fp = line.partition("\t")
|
||||
out[fp] = name
|
||||
return out
|
||||
|
||||
|
||||
_REF_FP2NAME = _reference()
|
||||
# 语义名 -> 操作数个数 / 是否动态长度
|
||||
_NAME_ARITY: Dict[str, int] = {}
|
||||
for _op, _nm in _CORE_NAMES.items():
|
||||
_NAME_ARITY[_nm] = _CORE_ARITY.get(_op, 0)
|
||||
_NAME_DYNAMIC = {_CORE_NAMES[o] for o in _CORE_DYNAMIC}
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# JS 感知的工具
|
||||
# --------------------------------------------------------------------------- #
|
||||
_JS_KEEP = set("""
|
||||
function var return new if else for while do typeof instanceof in of delete void this
|
||||
null true false undefined break continue switch case default throw try catch finally
|
||||
Object String Number Boolean Array Function RegExp Date Math JSON Error TypeError
|
||||
Symbol parseInt parseFloat isNaN encodeURIComponent decodeURIComponent
|
||||
Uint8Array Uint8ClampedArray Infinity NaN arguments
|
||||
""".split())
|
||||
_ID_RE = re.compile(r"[A-Za-z_$][\w$]*")
|
||||
_PREV_REGEX_OK = set("(,=[{;:!&|?+-*%~^<>")
|
||||
_OPERAND_READ_RE = re.compile(r"[A-Za-z_$][\w$]*\s*\[\s*[A-Za-z_$][\w$]*\s*\+\+\s*\]")
|
||||
|
||||
|
||||
def _skip_js_string(js: str, i: int) -> int:
|
||||
q = js[i]
|
||||
i += 1
|
||||
while i < len(js):
|
||||
c = js[i]
|
||||
if c == "\\":
|
||||
i += 2
|
||||
continue
|
||||
if c == q:
|
||||
return i + 1
|
||||
i += 1
|
||||
return i
|
||||
|
||||
|
||||
def _skip_js_regex(js: str, i: int) -> int:
|
||||
i += 1
|
||||
while i < len(js):
|
||||
c = js[i]
|
||||
if c == "\\":
|
||||
i += 2
|
||||
continue
|
||||
if c == "[":
|
||||
i += 1
|
||||
while i < len(js) and js[i] != "]":
|
||||
if js[i] == "\\":
|
||||
i += 1
|
||||
i += 1
|
||||
i += 1
|
||||
continue
|
||||
if c == "/":
|
||||
i += 1
|
||||
while i < len(js) and js[i].isalpha():
|
||||
i += 1
|
||||
return i
|
||||
i += 1
|
||||
return i
|
||||
|
||||
|
||||
def split_js_array(js: str, lb: int) -> Tuple[List[str], int]:
|
||||
"""把从 `lb`(指向 '[')起的数组字面量切成顶层元素字符串。"""
|
||||
assert js[lb] == "["
|
||||
i, depth, start, out = lb + 1, 0, lb + 1, []
|
||||
while i < len(js):
|
||||
c = js[i]
|
||||
if c in "\"'`":
|
||||
i = _skip_js_string(js, i)
|
||||
continue
|
||||
if c == "/":
|
||||
j = i - 1
|
||||
while j >= 0 and js[j].isspace():
|
||||
j -= 1
|
||||
if j < 0 or js[j] in _PREV_REGEX_OK:
|
||||
i = _skip_js_regex(js, i)
|
||||
continue
|
||||
i += 1
|
||||
continue
|
||||
if c in "([{":
|
||||
depth += 1
|
||||
i += 1
|
||||
continue
|
||||
if c in ")]}":
|
||||
if c == "]":
|
||||
if depth == 0:
|
||||
out.append(js[start:i])
|
||||
return out, i
|
||||
depth -= 1
|
||||
else:
|
||||
depth -= 1
|
||||
i += 1
|
||||
continue
|
||||
if c == "," and depth == 0:
|
||||
out.append(js[start:i])
|
||||
start = i + 1
|
||||
i += 1
|
||||
continue
|
||||
i += 1
|
||||
raise ValueError("数组字面量未闭合")
|
||||
|
||||
|
||||
def handler_array(js: str) -> List[str]:
|
||||
"""
|
||||
定位解释器主循环 `ARR[BC[PC++]]()`, 返回 `ARR` 的字面量元素(每个 = 一个 handler 源码)。
|
||||
含空洞的槽位返回空串。
|
||||
"""
|
||||
m = re.search(r"([A-Za-z_$][\w$]*)\s*\[\s*[A-Za-z_$][\w$]*\s*\[\s*[A-Za-z_$][\w$]*\s*\+\+\s*\]\s*\]\s*\(\s*\)", js)
|
||||
if not m:
|
||||
raise ValueError("未找到 VM 解释器主循环 (X[Y[Z++]]())")
|
||||
arr = m.group(1)
|
||||
best: Optional[List[str]] = None
|
||||
for mm in re.finditer(re.escape(arr) + r"\s*=\s*\[", js):
|
||||
lb = js.index("[", mm.start())
|
||||
try:
|
||||
els, _ = split_js_array(js, lb)
|
||||
except Exception:
|
||||
continue
|
||||
ne = sum(1 for e in els if e.strip())
|
||||
if best is None or ne > sum(1 for e in best if e.strip()):
|
||||
best = els
|
||||
if best is None:
|
||||
raise ValueError("未找到操作码处理器数组字面量 %s=[...]" % arr)
|
||||
return best
|
||||
|
||||
|
||||
def canonicalize(src: str) -> str:
|
||||
"""把 handler 源码规范化成与变量命名无关的指纹。"""
|
||||
out: List[str] = []
|
||||
mp: Dict[str, str] = {}
|
||||
n = 0
|
||||
i = 0
|
||||
prev = ""
|
||||
while i < len(src):
|
||||
c = src[i]
|
||||
if c in "\"'`":
|
||||
j = _skip_js_string(src, i)
|
||||
out.append(src[i:j])
|
||||
i, prev = j, "'"
|
||||
continue
|
||||
m = _ID_RE.match(src, i)
|
||||
if m:
|
||||
tok = m.group(0)
|
||||
if prev == "." or tok in _JS_KEEP:
|
||||
out.append(tok)
|
||||
else:
|
||||
if tok not in mp:
|
||||
n += 1
|
||||
mp[tok] = "v%d" % n
|
||||
out.append(mp[tok])
|
||||
i, prev = m.end(), "i"
|
||||
continue
|
||||
if not c.isspace():
|
||||
out.append(c)
|
||||
prev = c
|
||||
i += 1
|
||||
return "".join(out)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 对外: 按构建推导操作码表
|
||||
# --------------------------------------------------------------------------- #
|
||||
def derive_opcode_table(js: str) -> Tuple[Dict[int, str], Dict[int, int], set, Dict[str, str]]:
|
||||
"""
|
||||
返回 (names, arity, dynamic, diag):
|
||||
names : {opcode: 语义名}
|
||||
arity : {opcode: 固定操作数个数} (动态长度的不在其中)
|
||||
dynamic : {opcode} 需要读操作数才能定长
|
||||
diag : 诊断信息
|
||||
"""
|
||||
els = handler_array(js)
|
||||
names: Dict[int, str] = {}
|
||||
arity: Dict[int, int] = {}
|
||||
dynamic: set = set()
|
||||
unknown: List[int] = []
|
||||
slots: List[Tuple[int, str]] = []
|
||||
for op, src in enumerate(els):
|
||||
if not src.strip():
|
||||
continue
|
||||
fp = canonicalize(src)
|
||||
nm = _REF_FP2NAME.get(fp)
|
||||
if nm is None:
|
||||
unknown.append(op)
|
||||
continue
|
||||
names[op] = nm
|
||||
if nm in _NAME_DYNAMIC:
|
||||
dynamic.add(op)
|
||||
else:
|
||||
arity[op] = _NAME_ARITY.get(nm, 0)
|
||||
slots.append((op, nm))
|
||||
diag = {
|
||||
"handler_count": len(names),
|
||||
"unknown_slots": unknown,
|
||||
"table": {op: nm for op, nm in slots},
|
||||
}
|
||||
if unknown:
|
||||
raise ValueError("有 handler 指纹未能识别(服务端改了 VM 语义?): %s" % unknown[:10])
|
||||
if len(names) < 40:
|
||||
raise ValueError("推导出的 opcode 表过小(%d), 疑似解析失败" % len(names))
|
||||
return names, arity, dynamic, diag
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
_js = open(sys.argv[1] if len(sys.argv) > 1 else "tcaptcha/tdc_sample.js",
|
||||
encoding="utf-8", errors="replace").read()
|
||||
_names, _arity, _dyn, _diag = derive_opcode_table(_js)
|
||||
print("handlers: %d dynamic: %s" % (_diag["handler_count"], sorted(_dyn)))
|
||||
for _op in sorted(_names):
|
||||
print(" %3d %-18s arity=%s" % (_op, _names[_op], "dyn" if _op in _dyn else _arity.get(_op, 0)))
|
||||
@@ -0,0 +1,432 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
tdc_program.py —— TDC Chaos VM 「程序结构层」分析器。
|
||||
|
||||
它补的是 VM_SPEC.md §5/§6 自己点出的缺口:
|
||||
opcode 表 / 闭包调用约定 早已吃透, 但**模块系统 + 宿主调用序列**一直没被静态说明:
|
||||
顶层只"装配模块", 真正干活的代码在闭包被宿主调用时才跑。
|
||||
|
||||
本工具把这些做出确定规格:
|
||||
* `closures` —— 全部 CREATE_CLOSURE 站点: (pc, target, caps[(dst,src)], params[reg])
|
||||
* `entries` —— 子程序入口集合 = {0} ∪ {所有 closure target}
|
||||
* `strings` —— 从字节码折叠出的字面串 (PUSH_EMPTY_STR + 连续 PUSH_STR_CHAR)
|
||||
* `owner(pc)` —— 某 pc 属于哪个子程序 (启发式: 覆盖它的、且 ≤pc 的最大入口)
|
||||
* `call_graph` —— 子程序 → 它在自己体内创建的闭包 target (静态调用图的"创建边")
|
||||
* `xfuncs` —— 同一 target 被多个站点创建的"共享体" (VM 里的复用函数)
|
||||
|
||||
注意 (踩过的坑):
|
||||
`_decode_one` 对 CREATE_CLOSURE 的 `operands` **不含 target**(target 只写在 note 里),
|
||||
必须自己按 `target = z[pc+1]` 取, 否则会把 cap 寄存器号当成 target。
|
||||
|
||||
用法:
|
||||
.venv/bin/python python -m tcaptcha.telemetry.program tcaptcha/tdc_sample.js --summary
|
||||
.venv/bin/python python -m tcaptcha.telemetry.program <js> --seq --max 30
|
||||
.venv/bin/python python -m tcaptcha.telemetry.program <js> --find getData
|
||||
.venv/bin/python python -m tcaptcha.telemetry.program <js> --at 1725
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Sequence, Tuple
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from tcaptcha.telemetry.opcodes import derive_opcode_table # noqa: E402
|
||||
from tcaptcha.telemetry.disasm import disassemble_cfg, extract_bytecode # noqa: E402
|
||||
|
||||
MIN_STR_LEN = 3
|
||||
|
||||
|
||||
@dataclass
|
||||
class ClosureSite:
|
||||
pc: int
|
||||
target: int
|
||||
caps: List[Tuple[int, int]] = field(default_factory=list) # (新闭包寄存器, 父闭包寄存器)
|
||||
params: List[int] = field(default_factory=list) # 形参落到的寄存器号
|
||||
|
||||
|
||||
@dataclass
|
||||
class Program:
|
||||
path: str
|
||||
z: List[int]
|
||||
names: Dict[int, str]
|
||||
arity: Dict[int, int]
|
||||
dynamic: set
|
||||
insts: Dict[int, dict]
|
||||
closures: List[ClosureSite]
|
||||
strings: List[Tuple[int, str]]
|
||||
entries: List[int]
|
||||
_owner: Dict[int, int] = field(default_factory=dict)
|
||||
|
||||
# -------------------------------------------------------------- 基本查询
|
||||
def name(self, pc: int) -> str:
|
||||
i = self.insts.get(pc)
|
||||
return i["name"] if i else "?"
|
||||
|
||||
def owner(self, pc: int) -> int:
|
||||
"""pc 属于哪个子程序入口。
|
||||
|
||||
注意: 代码布局是「函数体之间夹着父函数的初始化序列 + 内联的子函数体」,
|
||||
所以**不能**用"覆盖它的最大入口 ≤pc"来判 (会把父函数续段误判给夹在中间的兄弟函数)。
|
||||
这里用**边界感知走法**: 从入口开始走控制流, 一旦走到**另一个函数入口**就停下
|
||||
(函数只能被调用进入, 不会被穿过)。
|
||||
"""
|
||||
if not self._owner:
|
||||
self._build_owners()
|
||||
return self._owner.get(pc, 0)
|
||||
|
||||
def _next_pc(self, pc: int, inst: dict) -> int:
|
||||
"""下一条 fall-through 指令的 pc。CREATE_CLOSURE 是动态长度, 不能用固定 arity。"""
|
||||
if inst["name"] == "CREATE_CLOSURE":
|
||||
capn, parn = self.z[pc + 2], self.z[pc + 3]
|
||||
return pc + 1 + 3 + 2 * capn + parn
|
||||
return pc + 1 + self.arity.get(inst["op"], 0)
|
||||
|
||||
def function_code(self, entry: int) -> List[dict]:
|
||||
"""边界感知地取某个子程序**自己的**指令序列。"""
|
||||
starts = set(self.entries)
|
||||
out: Dict[int, dict] = {}
|
||||
work = [entry]
|
||||
while work:
|
||||
pc = work.pop()
|
||||
if pc in out or pc >= len(self.z) or pc < 0:
|
||||
continue
|
||||
if pc != entry and pc in starts: # 进入别的函数体 -> 不跨越
|
||||
continue
|
||||
inst = self.insts.get(pc)
|
||||
if inst is None:
|
||||
continue
|
||||
out[pc] = inst
|
||||
name = inst["name"]
|
||||
ops = inst["operands"] or []
|
||||
if name == "JUMP":
|
||||
work.append(ops[0])
|
||||
continue
|
||||
if name == "JUMP_IF_TRUE":
|
||||
work.append(ops[0])
|
||||
work.append(self._next_pc(pc, inst))
|
||||
continue
|
||||
if name == "PUSH_TRY" and self.arity.get(inst["op"], 0) == 2:
|
||||
work.append(ops[0])
|
||||
if name == "CREATE_CLOSURE":
|
||||
work.append(self.z[pc + 1])
|
||||
if name in ("RET_TRUE", "THROW"):
|
||||
continue
|
||||
work.append(self._next_pc(pc, inst))
|
||||
return [out[k] for k in sorted(out)]
|
||||
|
||||
def _build_owners(self) -> None:
|
||||
for e in self.entries:
|
||||
for inst in self.function_code(e):
|
||||
self._owner.setdefault(inst["pc"], e)
|
||||
|
||||
def closures_of(self, entry: int) -> List[ClosureSite]:
|
||||
return [c for c in self.closures if self.owner(c.pc) == entry]
|
||||
|
||||
def call_graph(self) -> Dict[int, List[int]]:
|
||||
g: Dict[int, List[int]] = {}
|
||||
for c in self.closures:
|
||||
g.setdefault(self.owner(c.pc), []).append(c.target)
|
||||
return {k: sorted(set(v)) for k, v in g.items()}
|
||||
|
||||
def shared_bodies(self) -> Dict[int, List[int]]:
|
||||
"""target -> 创建它的站点列表 (长度>1 说明这些站点共用同一个代码体)。"""
|
||||
m: Dict[int, List[int]] = {}
|
||||
for c in self.closures:
|
||||
m.setdefault(c.target, []).append(c.pc)
|
||||
return {k: v for k, v in m.items() if len(v) > 1}
|
||||
|
||||
def module_registry(self, entry: int = 0) -> List[Tuple[int, int]]:
|
||||
"""识别顶层「模块注册表装配」模式, 返回 [(模块 id, 工厂闭包 entry), ...]。
|
||||
|
||||
模式 (源码级 `registry[id] = function(module, exports, require){...}` 的编译产物):
|
||||
DUP; PUSH_IMM [id]; REVERSE_TOP2; JUMP -> @F; @F: CREATE_CLOSURE ...; SET_PROP
|
||||
|
||||
这是整个程序的骨架: 顶层只把 228 个模块工厂按 id 填进一个数组, 入口再 drive 它。
|
||||
"""
|
||||
code = self.function_code(entry)
|
||||
pairs: List[Tuple[int, int]] = []
|
||||
for k, inst in enumerate(code):
|
||||
if inst["name"] != "CREATE_CLOSURE":
|
||||
continue
|
||||
for j in range(k - 1, max(-1, k - 4), -1):
|
||||
if code[j]["name"] != "JUMP":
|
||||
continue
|
||||
ops = code[j]["operands"] or []
|
||||
if not ops or ops[0] != inst["pc"]:
|
||||
continue
|
||||
for m in range(j - 1, max(-1, j - 6), -1):
|
||||
if code[m]["name"] == "PUSH_IMM" and code[m]["operands"]:
|
||||
# 记的是**工厂函数体入口** (CREATE_CLOSURE 的 target), 不是创建站点
|
||||
pairs.append((int(code[m]["operands"][0]), self.z[inst["pc"] + 1]))
|
||||
break
|
||||
break
|
||||
seen, out = set(), []
|
||||
for mid, tgt in sorted(pairs):
|
||||
if mid in seen:
|
||||
continue
|
||||
seen.add(mid)
|
||||
out.append((mid, tgt))
|
||||
return out
|
||||
|
||||
# ---------------------------------------------------------- 轻量符号求值
|
||||
def symbolic_assignments(self, entry: int, limit: int = 30) -> List[Tuple[str, str]]:
|
||||
"""对一个小函数(如采集器模块)做**线性符号求值**, 还原 `rN = <表达式>`。
|
||||
|
||||
只覆盖这类小模块实际用到的形状: 字面串 / BIND_THIS / MAKE_(REG_)PROP_REF /
|
||||
GET_PROP / GET_GLOBAL_PROP / NEW_FUNC / CALL_FUNC / CALL_METHOD / STORE_REG /
|
||||
LOAD_REG_IMM / REVERSE_TOP2 / DUP / 算术。
|
||||
|
||||
目的: 把"这一槽取的是什么值"变成可读路径, 例如 `c.navigator.userAgent`。
|
||||
说明: 遇到分支时按 pc 顺序近似处理, 结果用于**定位语义**, 不作为执行依据。
|
||||
"""
|
||||
regs: Dict[int, str] = {}
|
||||
out: List[Tuple[str, str]] = []
|
||||
st: List[Tuple[str, str]] = [] # ("reg", n) | ("sym", text)
|
||||
|
||||
def top() -> Tuple[str, str]:
|
||||
return st[-1] if st else ("sym", "?")
|
||||
|
||||
# 栈上元素形如 ("reg", int) | ("sym", str)：`[1]` 可能是 int,
|
||||
# 插值进 f-string 前必须统一转 str（早期版本在 CALL_* 处直接 join 会 TypeError）。
|
||||
def txt(x: Tuple[str, object]) -> str:
|
||||
return str(x[1]) if isinstance(x, tuple) else str(x)
|
||||
|
||||
def popn(n: int) -> List[Tuple[str, str]]:
|
||||
r = st[-n:] if n else []
|
||||
del st[len(st) - n:]
|
||||
return r
|
||||
|
||||
for inst in self.function_code(entry):
|
||||
n, ops = inst["name"], inst["operands"] or []
|
||||
if n == "PUSH_IMM_ARRAY":
|
||||
st.append(("reg", ops[0]))
|
||||
elif n == "PUSH_EMPTY_STR":
|
||||
st.append(("sym", ""))
|
||||
elif n == "PUSH_STR_CHAR":
|
||||
k, v = top()
|
||||
st[-1] = ("sym", v + chr(ops[0] & 0xFF))
|
||||
elif n == "PUSH_IMM":
|
||||
st.append(("sym", str(ops[0])))
|
||||
elif n == "PUSH_UNDEFINED":
|
||||
st.append(("sym", "undefined"))
|
||||
elif n == "PUSH_NULL":
|
||||
st.append(("sym", "null"))
|
||||
elif n == "PUSH_TRUE":
|
||||
st.append(("sym", "true"))
|
||||
elif n == "PUSH_FALSE":
|
||||
st.append(("sym", "false"))
|
||||
elif n == "LOAD_REG_IMM":
|
||||
st.append(("sym", regs.get(ops[0], f"r{ops[0]}")))
|
||||
elif n == "LOAD_REG_INDIRECT":
|
||||
k, v = st.pop() if st else ("sym", "?")
|
||||
st.append(("sym", regs.get(int(v), f"r{v}") if str(v).isdigit() else v))
|
||||
elif n == "BIND_THIS":
|
||||
k, v = st.pop() if st else ("sym", "?")
|
||||
st.append(("sym", f"c.{txt(('sym', v))}"))
|
||||
elif n == "MAKE_PROP_REF":
|
||||
key = txt(top())
|
||||
popn(1)
|
||||
base = txt(top())
|
||||
popn(1)
|
||||
st.append(("sym", f"{base}.{key}"))
|
||||
elif n == "MAKE_REG_PROP_REF":
|
||||
key = txt(top())
|
||||
popn(1)
|
||||
k, v = st.pop() if st else ("sym", "?")
|
||||
st.append(("sym", f"{regs.get(v, f'r{v}')}.{key}"))
|
||||
elif n == "GET_GLOBAL_PROP":
|
||||
st[-1] = ("sym", f"c.{txt(top())}")
|
||||
elif n in ("GET_PROP", "GET_ELEM"):
|
||||
pass
|
||||
elif n in ("ADD", "SUB", "MUL", "DIV", "MOD", "BIT_AND", "BIT_OR", "BIT_XOR",
|
||||
"SHL", "SHR", "SHR_U", "EQ", "EQ_STRICT", "GREATER"):
|
||||
if len(st) >= 2:
|
||||
b = txt(top())
|
||||
popn(1)
|
||||
a = txt(top())
|
||||
popn(1)
|
||||
st.append(("sym", f"({a} {n} {b})"))
|
||||
elif n in ("NEW_FUNC", "NEW_METHOD"):
|
||||
args = popn(int(ops[0]) if ops else 0)
|
||||
fn = txt(st.pop()) if st else "?"
|
||||
st.append(("sym", f"new {fn}({','.join(txt(x) for x in args)})"))
|
||||
elif n == "CALL_FUNC":
|
||||
args = popn(int(ops[0]) if ops else 0)
|
||||
fn = txt(st.pop()) if st else "?"
|
||||
st.append(("sym", f"{fn}({','.join(txt(x) for x in args)})"))
|
||||
elif n == "CALL_METHOD":
|
||||
args = popn(int(ops[0]) if ops else 0)
|
||||
tgt = txt(st.pop()) if st else "?"
|
||||
st.append(("sym", f"{tgt}({','.join(txt(x) for x in args)})"))
|
||||
elif n == "STORE_REG":
|
||||
v = txt(st.pop()) if st else "?"
|
||||
k, ref = st.pop() if st else ("sym", "?")
|
||||
if k == "reg":
|
||||
regs[ref] = v
|
||||
out.append((f"r{ref}", v))
|
||||
elif n == "DUP":
|
||||
if st:
|
||||
st.append(st[-1])
|
||||
elif n == "REVERSE_TOP2":
|
||||
if len(st) >= 2:
|
||||
st[-1], st[-2] = st[-2], st[-1]
|
||||
elif n in ("POP", "SET_PROP", "ALLOC_REG", "SET_STACK_LEN", "REPLACE_TOP_IMM"):
|
||||
if n in ("POP", "SET_PROP") and st:
|
||||
st.pop()
|
||||
if len(out) >= limit:
|
||||
break
|
||||
return out
|
||||
|
||||
def find_strings(self, pattern: str) -> List[Tuple[int, str, int]]:
|
||||
"""返回 (pc, 串, 所属子程序入口)。"""
|
||||
rx = re.compile(pattern, re.I)
|
||||
return [(pc, s, self.owner(pc)) for pc, s in self.strings if rx.search(s)]
|
||||
|
||||
|
||||
def _fold_strings(insts: Dict[int, dict]) -> List[Tuple[int, str]]:
|
||||
"""PUSH_EMPTY_STR 后跟一串 PUSH_STR_CHAR -> 还原字面串 (tdc_disasm 不做这步)。"""
|
||||
out: List[Tuple[int, str]] = []
|
||||
cur_pc: Optional[int] = None
|
||||
buf: List[str] = []
|
||||
for pc in sorted(insts):
|
||||
i = insts[pc]
|
||||
n = i["name"]
|
||||
if n == "PUSH_EMPTY_STR":
|
||||
cur_pc, buf = pc, []
|
||||
continue
|
||||
if n == "PUSH_STR_CHAR" and cur_pc is not None:
|
||||
ops = i["operands"] or []
|
||||
if ops:
|
||||
buf.append(chr(ops[0] & 0xFF))
|
||||
continue
|
||||
if cur_pc is not None:
|
||||
if len(buf) >= MIN_STR_LEN:
|
||||
out.append((cur_pc, "".join(buf)))
|
||||
cur_pc, buf = None, []
|
||||
if cur_pc is not None and len(buf) >= MIN_STR_LEN:
|
||||
out.append((cur_pc, "".join(buf)))
|
||||
return out
|
||||
|
||||
|
||||
def load(js_path: str) -> Program:
|
||||
js = open(js_path, encoding="utf-8", errors="replace").read()
|
||||
z = extract_bytecode(js)
|
||||
names, arity, dynamic, _diag = derive_opcode_table(js)
|
||||
insts = {i["pc"]: i for i in
|
||||
disassemble_cfg(z, 0, True, names=names, arity=arity, dynamic=dynamic)
|
||||
if i["op"] is not None}
|
||||
|
||||
closures: List[ClosureSite] = []
|
||||
for pc, i in insts.items():
|
||||
if i["name"] != "CREATE_CLOSURE":
|
||||
continue
|
||||
target, cap, par = z[pc + 1], z[pc + 2], z[pc + 3]
|
||||
base = pc + 4
|
||||
caps = [(z[base + 2 * k], z[base + 2 * k + 1]) for k in range(cap)]
|
||||
params = [z[base + 2 * cap + k] for k in range(par)]
|
||||
closures.append(ClosureSite(pc=pc, target=target, caps=caps, params=params))
|
||||
|
||||
strings = _fold_strings(insts)
|
||||
entries = sorted({0} | {c.target for c in closures})
|
||||
return Program(path=js_path, z=z, names=names, arity=arity, dynamic=dynamic,
|
||||
insts=insts, closures=sorted(closures, key=lambda c: c.pc),
|
||||
strings=strings, entries=entries)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- CLI
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description="TDC Chaos VM 程序结构分析")
|
||||
ap.add_argument("path", nargs="?", default=str(ROOT / "tcaptcha/tdc_sample.js"))
|
||||
ap.add_argument("--summary", action="store_true", help="总体规模")
|
||||
ap.add_argument("--seq", action="store_true", help="按子程序列出闭包站点")
|
||||
ap.add_argument("--max", type=int, default=40, help="--seq/--find 的截断条数")
|
||||
ap.add_argument("--find", default=None, help="按正则找字面串")
|
||||
ap.add_argument("--at", type=int, default=None, help="看某个 pc 属于谁")
|
||||
ap.add_argument("--shared", action="store_true", help="列出被多个站点共享的闭包体")
|
||||
ap.add_argument("--func", type=int, default=None, help="打印某个子程序**自己的**指令序列")
|
||||
ap.add_argument("--asm-out", default=None, help="--func 时把反汇编写到该文件")
|
||||
ap.add_argument("--modules", action="store_true", help="导出模块注册表 (id -> 工厂 entry)")
|
||||
args = ap.parse_args()
|
||||
|
||||
p = load(args.path)
|
||||
shown = False
|
||||
|
||||
if args.summary or not any([args.seq, args.find is not None, args.at is not None,
|
||||
args.shared, args.func is not None]):
|
||||
shown = True
|
||||
print(f"file : {p.path}")
|
||||
print(f"bytecode : {len(p.z)} ints")
|
||||
print(f"reachable instrs: {len(p.insts)}")
|
||||
print(f"closure sites : {len(p.closures)}")
|
||||
print(f"subprograms : {len(p.entries)} (0 为顶层)")
|
||||
print(f"distinct bodies : {len({c.target for c in p.closures})}")
|
||||
print(f"literal strings : {len(p.strings)} (>= {MIN_STR_LEN} 字符)")
|
||||
|
||||
if args.seq:
|
||||
shown = True
|
||||
g = p.call_graph()
|
||||
print(f"\n--- 子程序与它们创建的闭包 (前 {args.max} 个入口) ---")
|
||||
for e in p.entries[: args.max]:
|
||||
kids = g.get(e, [])
|
||||
st = [s for pc, s in p.strings if p.owner(pc) == e]
|
||||
head = ", ".join(f"{s[:22]!r}" for s in st[:4])
|
||||
print(f" @{e:05d} -> {kids}")
|
||||
if head:
|
||||
print(f" 串: {head}")
|
||||
|
||||
if args.find is not None:
|
||||
shown = True
|
||||
hits = p.find_strings(args.find)
|
||||
print(f"\n--- 字面串匹配 {args.find!r}: {len(hits)} 条 ---")
|
||||
for pc, s, own in hits[: args.max]:
|
||||
print(f" @{pc:05d} [子程序 @{own:05d}] {s[:90]!r}")
|
||||
|
||||
if args.at is not None:
|
||||
shown = True
|
||||
pc = args.at
|
||||
print(f"\n--- pc @{pc} ---")
|
||||
print(f" 指令 : {p.insts.get(pc)}")
|
||||
print(f" 所属 : @{p.owner(pc)}")
|
||||
|
||||
if args.func is not None:
|
||||
shown = True
|
||||
code = p.function_code(args.func)
|
||||
lines = []
|
||||
for x in code:
|
||||
extra = (" " + " ".join(x["note"])) if x["note"] else (
|
||||
(" " + str(x["operands"])) if x["operands"] else "")
|
||||
lines.append(f"@{x['pc']:05d}: {x['name']:<18}{extra}")
|
||||
print(f"\n--- 子程序 @{args.func} 自身指令: {len(lines)} 条 ---")
|
||||
print("\n".join(lines[: args.max]))
|
||||
if args.asm_out:
|
||||
Path(args.asm_out).write_text("\n".join(lines), encoding="utf-8")
|
||||
print(f" -> {args.asm_out}")
|
||||
|
||||
if args.modules:
|
||||
shown = True
|
||||
reg = p.module_registry(0)
|
||||
print(f"\n--- 模块注册表: {len(reg)} 个模块 (registry[id] = 工厂闭包) ---")
|
||||
for mid, tgt in reg[: args.max]:
|
||||
print(f" registry[{mid:3d}] = 工厂 @{tgt}")
|
||||
if len(reg) > args.max:
|
||||
print(f" ... 共 {len(reg)} 个")
|
||||
|
||||
if args.shared:
|
||||
shown = True
|
||||
sh = p.shared_bodies()
|
||||
print(f"\n--- 共享闭包体: {len(sh)} 个 target 被多处创建 ---")
|
||||
for t, sites in sorted(sh.items(), key=lambda kv: -len(kv[1]))[: args.max]:
|
||||
print(f" target @{t:05d} x{len(sites)} sites={sites[:8]}{'...' if len(sites) > 8 else ''}")
|
||||
|
||||
return 0 if shown else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,389 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""tcaptcha/tdc_values.py —— 为每个采集器**语义名**提供取值生成器（纯离线）。
|
||||
|
||||
背景（账本 §五 第 0 项，har/1ms_dianxuan/PROGRESS.md）:
|
||||
「纯离线生成 collect」的唯一阻塞点 = 不知道每个语义名该填什么值。
|
||||
本文件把 2026-10-12 实测到的**同会话真值**固化成生成规则。
|
||||
|
||||
真值来源（不是猜的）:
|
||||
真 Chrome + 真 1ms 页面 -> Debugger 取**同会话** tdc.js（77,954 字节，
|
||||
含 __TENCENT_CHAOS_VM）-> iframe 内 TDC.getData(true) -> 同会话 XTEA 密钥解密
|
||||
-> 按「require 序 == 槽位序」对齐，38/38 槽全覆盖。
|
||||
脚本: scripts/verify/align_session.py、scripts/verify/audit_names.py
|
||||
|
||||
会话 vs 每次采集（连续 4 次 getData 实测，这一条决定了生成器怎么分层）:
|
||||
|
||||
| 语义名 | 第1次 | 第2次 | 第3次 | 第4次 | 结论 |
|
||||
|---------------------------|-------------|-------------|-------------|-------------|---------------|
|
||||
| itoken | 641547489 | 641547489 | 641547489 | 641547489 | **会话内固定** |
|
||||
| struct_40_9 (#.._rand) | 1591912482 | 1591912482 | 1591912482 | 1591912482 | **会话内固定** |
|
||||
| window_time (#e48 委派) | 1791745224 | 1791745224 | 1791745224 | 1791745224 | **会话内固定** |
|
||||
| window_time (#e40) | 1791745331 | 1791745548 | 1791745813 | 1791745814 | 每次刷新 |
|
||||
| struct_46_9 | 1 | 2 | 3 | 4 | **每次 +1 的计数器** |
|
||||
| trajectory 点数 | 1 | 61 | 1 | 1 | 取完即清空 |
|
||||
|
||||
三类取值:
|
||||
* 静态画像 DEVICE_PROFILE —— UA / 屏幕 / WebGL / Canvas / 语言 / 时区 / 核数 …
|
||||
（已用浏览器真值交叉校验: screen.width/height/availHeight/colorDepth、
|
||||
hardwareConcurrency、timezoneOffset、languages 全部一一吻合）
|
||||
* 会话量 Session —— itoken / 会话随机盐 / 页面加载时刻
|
||||
* 每次采集 Session.next_call() —— 当前时刻、采集计数器、轨迹
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional, Sequence
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 设备画像：与真机逐项核对过（见模块 docstring 的交叉校验说明） #
|
||||
# --------------------------------------------------------------------------- #
|
||||
DEVICE_PROFILE: Dict[str, Any] = {
|
||||
"name": "Mac / Chrome 155 / 1440x2560",
|
||||
"user_agent": ("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/155.0.0.0 Safari/537.36"),
|
||||
"platform": "MacIntel",
|
||||
"languages": ["zh-CN", "zh"],
|
||||
"timezone_offset": "+08", # 真机 Date.getTimezoneOffset() == -480
|
||||
"charset": "UTF-8",
|
||||
"hardware_concurrency": 10, # 真机 navigator.hardwareConcurrency
|
||||
# ⚠️ 方向别弄反：真机 screen.width == 1440、screen.height == 2560
|
||||
# （1440x2560 竖屏；浏览器实测 screen.availHeight == 2530）
|
||||
"screen_width": 1440,
|
||||
"screen_height": 2560,
|
||||
"screen_color_depth": 24,
|
||||
# 真机 screen_fingerprint 串；字段序 = 宽-高-可用高-色深-*-*-|-*
|
||||
"screen_fingerprint": "1440-2560-2530-24-*-*-|-*",
|
||||
"webgl_vendor": "Google Inc. (Apple)",
|
||||
"webgl_renderer": "ANGLE (Apple, ANGLE Metal Renderer: Apple M4, Unspecified Version)",
|
||||
"canvas_fingerprint": ("GgoAAAANSUhEUgAAASwAAACWCAYAAATf3v8bh+ryWcfSBSLkmN3N1"
|
||||
"PQinPL7Bi3TG0qE2CogAAAABJRU5ErkJggg=="),
|
||||
"viewport_size": [360, 360], # 验证码 iframe 的可视区
|
||||
"cookie_enabled": 1,
|
||||
"touch_detect": 2, # 真机实测值（该机 maxTouchPoints == 0）
|
||||
"iframe_detect": "iframe",
|
||||
"webrtc_ip": "", # 未取到候选时为空串
|
||||
"dom_compliance": 1023,
|
||||
"constant_98k": "98k",
|
||||
"template_url": ("https://turing.captcha.gtimg.com/1/template/"
|
||||
"drag_ele.html?rand=1519713624347"),
|
||||
}
|
||||
|
||||
# 真机里恒为这些常量的槽（连续 4 次抽样一致）
|
||||
CONSTANTS: Dict[str, Any] = {
|
||||
"time_since_load": 0,
|
||||
"webdriver_detect": 0,
|
||||
"console_detect": 0,
|
||||
"fp_salt": 0,
|
||||
"struct_48_9_call": 0,
|
||||
"location_href": "",
|
||||
"struct_40_9#empty": "",
|
||||
}
|
||||
|
||||
# 真机里是空数组的槽（与轨迹区分：轨迹至少含一个点）
|
||||
EMPTY_LIST_NAME = "struct_76_33"
|
||||
|
||||
DEFAULT_REFERRER = "https://1ms.run/console/personal/benefits/check-in"
|
||||
|
||||
|
||||
def _rand_int32() -> int:
|
||||
"""[0, 2^31) 的随机整数 —— 真机观测值 1591912482 / 699129805 / 641547489 都落在这里。"""
|
||||
return random.randint(0, (1 << 31) - 1)
|
||||
|
||||
|
||||
def default_trajectory() -> List[List[int]]:
|
||||
"""无鼠标输入时的轨迹（真机实测就是 [[1,1,12]]，看着像个"无位移"哨兵点）。"""
|
||||
return [[1, 1, 12]]
|
||||
|
||||
|
||||
def trajectory_points(click_points: Sequence[Sequence[int]],
|
||||
img_w: int = 672) -> List[List[int]]:
|
||||
"""按点击点生成鼠标轨迹：首点绝对坐标 + 增量点 + 固定收尾 `[1,1,12]`。
|
||||
|
||||
形状与真机一致（3 份样本逐字核对）:
|
||||
* 首点是**绝对坐标**（含 300~600ms 初始停顿）
|
||||
* 之后全是**增量** `[dx,dy,dt]`
|
||||
* **最后一点固定是 `[1,1,12]`** —— 真机成功样本与"没动鼠标"样本都这么收尾
|
||||
（没动鼠标时整条轨迹就是 `[[1,1,12]]`）
|
||||
dt 分布按成功样本实测拟合（旧 crypto._success_style_trajectory 的统计）。
|
||||
"""
|
||||
scale = 320.0 / img_w
|
||||
targets = [(int(round(20 + x * scale)), int(round(60 + y * scale)))
|
||||
for x, y in click_points]
|
||||
cur = [random.randint(140, 220), random.randint(300, 340)]
|
||||
events: List[List[int]] = [[cur[0], cur[1], random.randint(300, 600)]]
|
||||
|
||||
def dt() -> int:
|
||||
r = random.random()
|
||||
if r < 0.50:
|
||||
return random.randint(0, 16)
|
||||
if r < 0.75:
|
||||
return random.randint(17, 24)
|
||||
if r < 0.90:
|
||||
return random.randint(25, 60)
|
||||
if r < 0.98:
|
||||
return random.randint(61, 160)
|
||||
return random.randint(161, 400)
|
||||
|
||||
for tx, ty in targets:
|
||||
seg = random.randint(14, 20)
|
||||
for i in range(1, seg + 1):
|
||||
nx = cur[0] + (tx - cur[0]) * i / seg + random.randint(-1, 1)
|
||||
ny = cur[1] + (ty - cur[1]) * i / seg + random.randint(-1, 1)
|
||||
events.append([int(nx) - cur[0], int(ny) - cur[1], dt()])
|
||||
cur = [int(nx), int(ny)]
|
||||
for _ in range(random.randint(2, 5)): # 命中后短暂驻留
|
||||
events.append([random.randint(-1, 1), random.randint(-1, 1), dt()])
|
||||
events.append([1, 1, 12]) # 固定收尾（真机如此）
|
||||
return events
|
||||
|
||||
|
||||
def rows_from_source(js_text: str) -> List[Dict]:
|
||||
"""分析 tdc.js **源码**（tdc_modules 的入口收的是路径，这里落临时文件再分析）。"""
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
fd, path = tempfile.mkstemp(suffix=".js", prefix="tdc_")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
f.write(js_text)
|
||||
return _analyze(path)
|
||||
finally:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _analyze(path: str) -> List[Dict]:
|
||||
"""惰性导入 tools/tdc_modules（它不在包里，且只做静态分析）。"""
|
||||
import sys
|
||||
from pathlib import Path as _P
|
||||
|
||||
tools = str(_P(__file__).resolve().parents[1] / "tools")
|
||||
if tools not in sys.path:
|
||||
sys.path.insert(0, tools)
|
||||
from tcaptcha.telemetry import modules as tm
|
||||
return tm.analyze(path)
|
||||
|
||||
|
||||
def generate_collect(tdc_js_code: str,
|
||||
click_points: Optional[Sequence[Sequence[int]]] = None,
|
||||
*,
|
||||
key: Optional[Sequence[int]] = None,
|
||||
sd: Optional[Dict] = None,
|
||||
session: Optional[Session] = None,
|
||||
img_w: int = 672,
|
||||
referrer: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""**纯离线**生成一颗 collect：只吃 tdc.js 源码，不需要任何真机明文。
|
||||
|
||||
流程: tdc.js -> 37 个语义名（清单序）-> 38 槽取值 -> 4 段明文 -> 4 段加密拼接。
|
||||
密钥默认用纯 Python 静态提取（`tcaptcha.telemetry.key`），零 Node。
|
||||
"""
|
||||
from tcaptcha.telemetry.crypto import build_collect, extract_eks # 局部导入避免环形依赖
|
||||
from tcaptcha.telemetry.key import extract_xtea_key_pure
|
||||
|
||||
rows = rows_from_source(tdc_js_code)
|
||||
if key is None:
|
||||
key = extract_xtea_key_pure(tdc_js_code)["key"]
|
||||
sess = session or Session(referrer=referrer or DEFAULT_REFERRER)
|
||||
if click_points:
|
||||
sess.set_trajectory(trajectory_points(click_points, img_w=img_w))
|
||||
|
||||
cd = generate_cd(rows, sess)
|
||||
plain = build_plain(cd, rows, sd)
|
||||
collect = build_collect(plain["chunk1"], plain["trajectory"],
|
||||
plain["chunk2"], plain["sd"], list(key))
|
||||
return {
|
||||
"collect": collect,
|
||||
"tlg": str(len(collect)),
|
||||
"eks": extract_eks(tdc_js_code),
|
||||
"key": list(key),
|
||||
"source": "python_values",
|
||||
"cd": cd,
|
||||
"segments": {k: len(v) for k, v in plain.items()},
|
||||
}
|
||||
|
||||
|
||||
class Session:
|
||||
"""一次验证码会话的状态。
|
||||
|
||||
会话内固定的量（itoken / 随机盐 / 页面加载时刻）在 __init__ 里定下来，
|
||||
每次采集会变的量（当前时刻 / 计数器 / 轨迹）由 next_call() 推进。
|
||||
"""
|
||||
|
||||
def __init__(self, now: Optional[float] = None, entry_age: int = 60,
|
||||
referrer: str = DEFAULT_REFERRER, profile: Optional[Dict] = None):
|
||||
self.t0 = int(now if now is not None else time.time())
|
||||
# 页面加载时刻先于会话建立（真机两次采样差 ~107 秒）
|
||||
self.loaded_at = self.t0 - entry_age
|
||||
self.itoken = [_rand_int32(), self.loaded_at] # 会话内固定，实测 4 次完全一致
|
||||
self.session_rand = _rand_int32() # struct_40_9 的"随机"槽
|
||||
self.referrer = referrer
|
||||
self.profile = dict(profile or DEVICE_PROFILE)
|
||||
self.calls = 0
|
||||
self.now = self.t0
|
||||
self.trajectory: List[List[int]] = default_trajectory()
|
||||
|
||||
def next_call(self, now: Optional[float] = None) -> "Session":
|
||||
"""进入下一次 TDC.getData()：计数器 +1、刷新当前时刻、轨迹复位由调用方决定。"""
|
||||
self.calls += 1
|
||||
self.now = int(now if now is not None else time.time())
|
||||
return self
|
||||
|
||||
def set_trajectory(self, points: Sequence[Sequence[int]]) -> "Session":
|
||||
self.trajectory = [list(p) for p in points]
|
||||
return self
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 语义名 -> 取值生成器 #
|
||||
# --------------------------------------------------------------------------- #
|
||||
def value_for(base: str, row: Dict, sess: Session) -> Any:
|
||||
"""按**基础语义名**（去掉 #结构后缀）产出一个槽的值。
|
||||
|
||||
同一基础名在一个构建里可能出现多次（user_agent×2 / window_time×2 /
|
||||
struct_40_9×2），这时用行上的**结构证据**分流，而不是靠出现次序 ——
|
||||
次序会随构建变，结构不会（见 tdc_modules.STRUCT_RULES）。
|
||||
"""
|
||||
st = row.get("struct") or {}
|
||||
p = sess.profile
|
||||
|
||||
# ---- 同组多成员：用结构 / 委派证据分流 ----
|
||||
if base == "user_agent":
|
||||
# 直采的那个返回 UA 串；走委派链的那个返回 0（真机实测）
|
||||
return 0 if row.get("via_delegate") else p["user_agent"]
|
||||
if base == "struct_40_9":
|
||||
# 闭包最后写入 "" vs 随机 int32（账本 #47 的静态消歧依据）
|
||||
return "" if st.get("closure_kind") == "empty" else sess.session_rand
|
||||
if base == "window_time":
|
||||
# 委派的那个是页面加载时刻（会话内固定），直采的那个是当前时刻（每次变）
|
||||
return sess.loaded_at if row.get("via_delegate") else sess.now
|
||||
|
||||
# ---- 会话量 / 每次采集量 ----
|
||||
if base == "itoken":
|
||||
return None # 2 槽，见 generate_cd
|
||||
if base == "trajectory":
|
||||
return sess.trajectory
|
||||
if base == "struct_46_9":
|
||||
return sess.calls # 实测 1,2,3,4 —— 每次采集 +1
|
||||
if base == "referrer":
|
||||
return sess.referrer
|
||||
|
||||
# ---- 常量 / 画像 ----
|
||||
if base in CONSTANTS:
|
||||
return CONSTANTS[base]
|
||||
if base == EMPTY_LIST_NAME:
|
||||
return []
|
||||
if base in p:
|
||||
return p[base]
|
||||
raise KeyError("没有 %r 的取值规则（新语义名? 需要补规则）" % base)
|
||||
|
||||
|
||||
def generate_cd(rows: Sequence[Dict], sess: Optional[Session] = None,
|
||||
now: Optional[float] = None) -> List[Any]:
|
||||
"""按**清单顺序**生成 cd 数组（37 项 -> 38 槽，itoken 占 2 槽）。
|
||||
|
||||
rows 来自 tools.tdc_modules.analyze()（每项含 name / struct / via_delegate）。
|
||||
每次调用会推进 sess.next_call()（模拟一次 TDC.getData）。
|
||||
"""
|
||||
s = sess or Session(now=now)
|
||||
if now is not None:
|
||||
s.next_call(now)
|
||||
elif s.calls == 0:
|
||||
s.next_call(s.t0)
|
||||
|
||||
out: List[Any] = []
|
||||
for r in rows:
|
||||
base = str(r["name"]).split("#")[0]
|
||||
if base == "itoken":
|
||||
out.extend(s.itoken) # 真机就是 2 个连续槽
|
||||
continue
|
||||
out.append(value_for(base, r, s))
|
||||
return out
|
||||
|
||||
|
||||
def trajectory_index(rows: Sequence[Dict]) -> int:
|
||||
"""轨迹在 cd 中的下标 —— 它就是 4 段加密里第 2 段的分界点。"""
|
||||
k = 0
|
||||
for r in rows:
|
||||
base = str(r["name"]).split("#")[0]
|
||||
if base == "trajectory":
|
||||
return k
|
||||
k += 2 if base == "itoken" else 1
|
||||
raise KeyError("清单里没有 trajectory 项")
|
||||
|
||||
|
||||
def build_plain(cd: Sequence[Any], rows: Sequence[Dict],
|
||||
sd: Optional[Dict] = None) -> Dict[str, str]:
|
||||
"""把 cd 组装成 4 段明文（chunk1 / trajectory / chunk2 / sd）。
|
||||
|
||||
与真机**逐字节一致**的分段写法（3 份样本核对，2026-10-12）:
|
||||
|
||||
chunk1 = '{"cd":[' + 轨迹之前的槽（逗号连接）+ ','
|
||||
traj = '<轨迹槽的 JSON>'
|
||||
chunk2 = ',' + 轨迹之后的槽（逗号连接）+ '],'
|
||||
sd = '"sd":' + json(sd) + '}'
|
||||
|
||||
分隔符一律是**无空格的逗号**（真机 JSON.stringify 风格）。
|
||||
"""
|
||||
import json
|
||||
|
||||
def js(v: Any) -> str:
|
||||
return json.dumps(v, ensure_ascii=False, separators=(",", ":"))
|
||||
|
||||
k = trajectory_index(rows)
|
||||
head = ",".join(js(v) for v in cd[:k])
|
||||
tail = ",".join(js(v) for v in cd[k + 1:])
|
||||
return {
|
||||
"chunk1": '{"cd":[' + head + ",",
|
||||
"trajectory": js(cd[k]),
|
||||
"chunk2": "," + tail + "],",
|
||||
"sd": '"sd":' + js(sd if sd is not None else {"od": "C"}) + "}",
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "tools"))
|
||||
|
||||
ap = argparse.ArgumentParser(description="按语义名生成 cd 取值（纯离线）")
|
||||
ap.add_argument("tdc_js", help="一份 tdc.js（用来取清单顺序与语义名）")
|
||||
ap.add_argument("--json", default=None)
|
||||
args = ap.parse_args()
|
||||
|
||||
from tcaptcha.telemetry import modules as tm
|
||||
rows = tm.analyze(args.tdc_js)
|
||||
sess = Session()
|
||||
cd = generate_cd(rows, sess)
|
||||
print("项 语义名 取值")
|
||||
print("-" * 78)
|
||||
k = 0
|
||||
for r in rows:
|
||||
base = str(r["name"]).split("#")[0]
|
||||
n = 2 if base == "itoken" else 1
|
||||
val = cd[k:k + n] if n > 1 else cd[k]
|
||||
k += n
|
||||
print("%-3d %-36s %s" % (r["item"], r["name"],
|
||||
json.dumps(val, ensure_ascii=False)[:34]))
|
||||
print()
|
||||
print("槽数 %d / %d 项" % (len(cd), len(rows)))
|
||||
plain = build_plain(cd, rows)
|
||||
print("分段: chunk1=%d traj=%d chunk2=%d sd=%d"
|
||||
% (len(plain["chunk1"]), len(plain["trajectory"]),
|
||||
len(plain["chunk2"]), len(plain["sd"])))
|
||||
if args.json:
|
||||
Path(args.json).write_text(json.dumps(cd, ensure_ascii=False, indent=1),
|
||||
encoding="utf-8")
|
||||
print(">>> %s" % args.json)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user