yml/hello_js_reverse_skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(结构): 分层重组 + 清除全部废弃路线

Browse Source 
做了什么:
- 包改为分层: tcaptcha/{protocol,telemetry,solvers,ocr,adapters,models,tests}
  * protocol/  client.py  transport.py  proxy.py
  * telemetry/ crypto.py  values.py  key.py  opcodes.py
               disasm.py  program.py  layout.py  modules.py
  * solvers/   text_click.py  icon_click.py  icon_click_rotate.py  slider.py
  * ocr/       engines.py  slicer.py
  * adapters/  one_ms.py
- tools/ 里原本是**包运行时依赖**的 tdc_{disasm,program,layout,modules}.py 并入 telemetry/
  （放在 tools/ 属于分层倒置），tools/ 只留 CLI: tdc_key_pure.py / tdc_profile.py / cdp_*.js
- crypto.py 529 -> 340 行: 切除「成功样本模板回填」与「Node 插桩兜底」两条老路；
  generate_telemetry 只剩 python_values 一种模式，传别的会直接报错并说明原因
- 求解器改名分清: solver.py -> solvers/icon_click.py（client 实际在用的 YOLO+ResNet18 ONNX），
  icon_click_solver.py -> solvers/icon_click_rotate.py（仅测试在用，留作对照）
- pyproject: 去掉 ultralytics/onnx（训练栈已删），补上真正在用的 onnxruntime

删除了什么:
- 雨云整条线 + dist/rainyun_qinglong（104M 旧快照）
- 模板回填: models/success_collect_template.b64、tdc_worker.js、analyze_collect_layout.py
- 「补环境」时代浏览器 JS: chrome_fetch/manual_captcha/probe_captcha/solve_in_browser/tdc_browser
- 旧 VM 探索工具 10 个(tdc_collectgen/tdc_slotmap/tdc_lift/tdc_vm_run/tdc_query/tdc_probe 等)
  + 8 个基于它们的旧 verify 脚本
- 训练/标注/评估: train_classifier/train_detector/prepare_yolo/web_labeler/collector/
  siamese/evaluate/benchmark_verify
- tcaptcha/core_js/（60 个分析产物）、TOOLCHAIN_REPORT.md（整篇在描述已删工具）
- 3 个已无任何引用的 PyTorch 权重：icon_classifier.pt 43M / yolo_detector.pt 5.9M /
  siamese_matcher.pt 1.6M —— 线上只跑 onnx，tcaptcha/ 117M -> 66M

为什么:
- 项目后面要专门承载「滑块 / 文字点选 / 图标点选」三种求解器；扁平包 + 运行时依赖混在
  tools/ + 两条已废弃的 collect 生成路线，会让后续扩展持续踩坑

怎么验证:
- .venv/bin/python -m tcaptcha.selftest                    全绿
- scripts/verify/verify_offline.py                         逐槽全等 + 分段几何一致
- scripts/verify/verify_values.py                          4 份真机明文 152/152 逐槽全等
- scripts/verify/send_test.py --mode python_values         1ms 签到页文字点选 errorCode=0
- 回滚点: tag backup/pre-refactor-20261012
This commit is contained in:
yml2213
2026-10-12 12:34:57 +08:00
parent 51dff870cf
commit 344eea2194
169 changed files with 599 additions and 83331 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tcaptcha/solvers/__init__.py
+10
View File
@@ -0,0 +1,10 @@
# -*- coding: utf-8 -*-
"""求解器层：把一道题解成点击坐标 / 滑动距离。
统一约定（后续新增题型沿用）：
solver.solve(...) -> 结果对象，带 .points / .ok / .describe()
* text_click 2404 文字点选（颜色分割 + OCR 集成指派）
* icon_click 2408 图标点选（YOLOv8 ONNX 定位 + ResNet18 ONNX 分类）
* icon_click_rotate 另一版图标点选（旋转角模板匹配，备用/对照）
* slider 2402 滑块拼图（缺口定位）
"""
tcaptcha/solvers/icon_click.py
+193
View File
@@ -0,0 +1,193 @@
# -*- coding: utf-8 -*-
"""
腾讯天御点选验证码工业级纯 ONNX 视觉求解器 (Solver v3.0 - Pure ONNX Edition)
特性:
1. 零 PyTorch / 零 Ultralytics 依赖：采用 onnxruntime 推理引擎 (从 1.5GB 骤降至 15MB)
2. 提示词解析: 垂直/水平投影切片 + ResNet18 ONNX 分类器直接预测目标序列
3. 候选图标定位: YOLOv8n ONNX + Letterbox 自适应缩放 + OpenCV cv2.dnn.NMSBoxes 精准去重
4. 全局匹配: 后验概率分布优化 + 背景干扰项抑制因子 + 全局非冲突组合搜索，输出有序像素点击坐标
"""
import io
import itertools
import logging
from pathlib import Path
from typing import List, Tuple, Union
import cv2
import numpy as np
import onnxruntime as ort
from PIL import Image
from tcaptcha.ocr.slicer import slice_prompt_image
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger(__name__)
# 权重在包根的 models/（本文件在 tcaptcha/solvers/ 下，故上跳一层）
MODEL_DIR = Path(__file__).resolve().parents[1] / "models"
YOLO_ONNX_PATH = MODEL_DIR / "yolo_detector.onnx"
CLASSIFIER_ONNX_PATH = MODEL_DIR / "icon_classifier.onnx"
class CaptchaSolver:
def __init__(self, yolo_path: Path = YOLO_ONNX_PATH, classifier_path: Path = CLASSIFIER_ONNX_PATH):
if not yolo_path.exists():
raise FileNotFoundError(f"YOLO ONNX 权重文件不存在: {yolo_path}")
if not classifier_path.exists():
raise FileNotFoundError(f"Classifier ONNX 权重文件不存在: {classifier_path}")
# 配置轻量级 CPU 推理会话
sess_options = ort.SessionOptions()
sess_options.inter_op_num_threads = 2
sess_options.intra_op_num_threads = 2
sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
self.yolo_sess = ort.InferenceSession(str(yolo_path), sess_options, providers=["CPUExecutionProvider"])
self.cls_sess = ort.InferenceSession(str(classifier_path), sess_options, providers=["CPUExecutionProvider"])
@staticmethod
def _preprocess_patch(img: Image.Image) -> np.ndarray:
"""对小图标图像进行 64x64 归一化预处理 (匹配 ResNet18 输入标准)"""
resized = img.convert("RGB").resize((64, 64), Image.Resampling.BILINEAR)
arr = np.array(resized, dtype=np.float32) / 255.0
mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
norm = (arr - mean) / std
return np.transpose(norm, (2, 0, 1)) # (3, 64, 64)
def parse_prompt_targets(self, prompt_img: Union[str, Path, bytes, Image.Image]) -> List[int]:
"""自动切片并解析提示图中 3 个目标图标的类别 ID (0~11)"""
crops = slice_prompt_image(prompt_img)
if not crops:
return []
patch_blobs = np.stack([self._preprocess_patch(c[0]) for c in crops[:3]], axis=0).astype(np.float32)
logits = self.cls_sess.run(None, {"input": patch_blobs})[0] # shape: (N, 13)
return [int(np.argmax(l[:12])) for l in logits]
def _extract_candidates(self, bg_img: Union[str, Path, bytes, Image.Image]) -> List[Tuple[int, int, Image.Image]]:
"""利用 YOLOv8n ONNX 召回背景图所有图标候选位置并去重"""
if isinstance(bg_img, Image.Image):
bg_pil = bg_img.convert("RGB")
elif isinstance(bg_img, bytes):
bg_pil = Image.open(io.BytesIO(bg_img)).convert("RGB")
else:
bg_pil = Image.open(bg_img).convert("RGB")
orig_w, orig_h = bg_pil.size
img_np = np.array(bg_pil)
# 1. Letterbox 保持宽高比填充缩放至 640x640
r = min(640.0 / orig_h, 640.0 / orig_w)
new_w, new_h = int(round(orig_w * r)), int(round(orig_h * r))
resized = cv2.resize(img_np, (new_w, new_h), interpolation=cv2.INTER_LINEAR)
dw = (640.0 - new_w) / 2.0
dh = (640.0 - new_h) / 2.0
top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
left, right = int(round(dw - 0.1)), int(round(dw + 0.1))
padded = cv2.copyMakeBorder(resized, top, bottom, left, right, cv2.BORDER_CONSTANT, value=(114, 114, 114))
blob = padded.astype(np.float32) / 255.0
blob = np.transpose(blob, (2, 0, 1))[None, ...]
# 2. ONNX 目标检测前向推理
yolo_out = self.yolo_sess.run(None, {"images": blob})[0][0].T # (8400, 5)
valid = yolo_out[yolo_out[:, 4] >= 0.03]
boxes_xywh = []
scores = []
for row in valid:
cx, cy, w, h, conf = row
x1 = (cx - w / 2.0 - dw) / r
y1 = (cy - h / 2.0 - dh) / r
boxes_xywh.append([int(x1), int(y1), int(w / r), int(h / r)])
scores.append(float(conf))
# 3. OpenCV NMS 筛选重叠边界框
indices = cv2.dnn.NMSBoxes(boxes_xywh, scores, score_threshold=0.03, nms_threshold=0.45)
raw_cands = []
if len(indices) > 0:
for idx in indices:
x, y, w, h = boxes_xywh[idx]
raw_cands.append((int(x + w / 2.0), int(y + h / 2.0)))
# 4. 空间中心点近邻聚类去重 (<=22px)
dedup_cands = []
for cx, cy in raw_cands:
if not any(np.hypot(cx - dcx, cy - dcy) < 22 for dcx, dcy in dedup_cands):
dedup_cands.append((cx, cy))
candidates = []
for cx, cy in dedup_cands:
crop_box = (max(0, cx - 32), max(0, cy - 32), min(orig_w, cx + 32), min(orig_h, cy + 32))
patch = bg_pil.crop(crop_box)
candidates.append((cx, cy, patch))
return candidates
def solve(
self,
bg_img: Union[str, Path, bytes, Image.Image],
prompt_img: Union[str, Path, bytes, Image.Image],
) -> List[Tuple[int, int]]:
"""
端到端纯 ONNX 求解验证码点选坐标
返回 3 个目标的像素坐标序列: [(x1, y1), (x2, y2), (x3, y3)]
"""
# 1. 确定提示图的目标序列 [T1, T2, T3]
target_classes = self.parse_prompt_targets(prompt_img)
if len(target_classes) < 3:
logger.warning("提示图未成功切出 3 个目标")
return []
# 2. 定位背景图所有候选图标
candidates = self._extract_candidates(bg_img)
if len(candidates) < 3:
logger.warning(f"背景图检出候选图标数量不足: {len(candidates)}")
return []
# 3. 批量推理计算候选图标的类别概率分布 (13 维)
patch_blobs = np.stack([self._preprocess_patch(c[2]) for c in candidates], axis=0).astype(np.float32)
logits = self.cls_sess.run(None, {"input": patch_blobs})[0]
exp = np.exp(logits - np.max(logits, axis=1, keepdims=True))
probs = exp / np.sum(exp, axis=1, keepdims=True)
# 4. 全局后验概率优化 (结合背景干扰项抑制因子 1 - P(distractor))
bg_probs = probs[:, 12]
clean_probs = probs[:, :12] * (1.0 - bg_probs[:, None])
num_cands = len(candidates)
best_score = -1.0
best_comb = None
cand_indices = range(num_cands)
for c1, c2, c3 in itertools.permutations(cand_indices, 3):
score = (
clean_probs[c1, target_classes[0]]
* clean_probs[c2, target_classes[1]]
* clean_probs[c3, target_classes[2]]
)
if score > best_score:
best_score = score
best_comb = (c1, c2, c3)
if best_comb is None:
used = set()
comb_list = []
for t_cls in target_classes:
cand_probs = clean_probs[:, t_cls].copy()
for u in used:
cand_probs[u] = -1.0
best_c = int(np.argmax(cand_probs))
comb_list.append(best_c)
used.add(best_c)
best_comb = tuple(comb_list)
# 5. 输出有序点击坐标
return [(candidates[idx][0], candidates[idx][1]) for idx in best_comb]
if __name__ == "__main__":
solver = CaptchaSolver()
print("✓ CaptchaSolver ONNX 引擎初始化成功！")
tcaptcha/solvers/icon_click_rotate.py
+204
View File
@@ -0,0 +1,204 @@
# -*- coding: utf-8 -*-
"""
腾讯天御『图标/图文混合点选』求解器 (subcapclass 2408 / 2401 之外的图形点选)
适用题型 (1ms.run aid=195672344 实测，2026-10-11):
提示条 `img_index=0` 是 170x50 的**目标原图条**(按点击顺序排列, 例如: "0" + 图标 + "7"),
背景图 `img_index=1` 是 672x480 的实景照片, 上面画着若干**纯黑轮廓**的符号
(目标 + 干扰项), 部分目标被**旋转/倾斜**。
与 `glyph_solver.TextClickSolver`(汉字点选 2404) 的区别:
* 2404: 指令是纯文字 ("请依次点击：本 车 憋"), 背景是**彩色**(橙黄)汉字, 用 HSV 分割 + OCR
* 本类: 指令是**图标/数字**, 提示以**图片**给出, 背景符号是**纯黑轮廓**, 且会被旋转
算法 (不重训分类器, 直接利用"提示条给的就是目标原图"这一事实):
1. 背景/提示条都按"近黑像素"分割成字形掩膜 (符号是纯黑, 背景是照片)
2. 提示条连通域 = 有序目标模板 (按 x 排序)
3. 背景连通域 = 候选符号
4. 每个目标 vs 每个候选: 二值掩膜归一化(等比缩放到正方形)后做
**旋转搜索 + IoU** 相似度 (旋转不变, 正好对付倾斜干扰)
5. 用匈牙利/贪心做**一对一**指派, 按提示顺序输出点击坐标
坐标口径: 输出与腾讯 `ans` 相同的**原图坐标系 (672x480)**。
"""
from __future__ import annotations
import io
from dataclasses import dataclass
from typing import List, Optional, Sequence, Tuple
import cv2
import numpy as np
from PIL import Image
# 近黑阈值: 符号是纯黑(<=60), 照片最暗处也远高于此; 用"三通道都暗"避免蓝色天空误检
DARK_MAX = 95
MERGE_KERNEL = 5
MIN_AREA = 80
MAX_AREA = 9000
def _to_bgr(image) -> np.ndarray:
if isinstance(image, np.ndarray):
return image
if isinstance(image, (bytes, bytearray)):
arr = np.frombuffer(image, np.uint8)
return cv2.imdecode(arr, cv2.IMREAD_COLOR)
return cv2.cvtColor(np.asarray(image.convert("RGB")), cv2.COLOR_RGB2BGR)
def glyph_mask(bgr: np.ndarray, dark_max: int = DARK_MAX, kernel: int = MERGE_KERNEL,
dilate: int = 3) -> np.ndarray:
"""近黑像素 -> 掩膜 (取三通道最大值, 只有真正的黑色轮廓会通过)。
kernel>1 时做闭运算合并同一符号的笔画; dilate>0 时再膨胀一点 (分割阶段用)。
"""
mx = bgr.max(axis=2)
m = (mx < dark_max).astype(np.uint8) * 255
if kernel > 1:
m = cv2.morphologyEx(m, cv2.MORPH_CLOSE,
cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (kernel, kernel)))
if dilate > 0:
m = cv2.dilate(m, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (dilate, dilate)))
return m
def components(mask: np.ndarray, min_area: int = MIN_AREA, max_area: int = MAX_AREA) -> List[Tuple[int, int, int, int, int]]:
n, lab, stats, _ = cv2.connectedComponentsWithStats(mask)
out = []
for i in range(1, n):
x, y, w, h, a = (int(v) for v in stats[i])
if a < min_area or a > max_area:
continue
if w < 6 or h < 6:
continue
out.append((x, y, w, h, a))
return out
def _norm(bgr: np.ndarray, box, size: int = 64) -> np.ndarray:
"""把 box 内的符号归一化成 size x size 的二值掩膜 (等比居中, 不拉伸)。
注意: 画布必须用**白色**填充 —— 黑色会被"近黑"掩膜当成字形 (踩过这个坑)。
"""
x, y, w, h, _ = box
crop = bgr[y:y + h, x:x + w]
s = max(w, h)
canvas = np.full((s, s, 3), 255, np.uint8)
canvas[(s - h) // 2:(s - h) // 2 + h, (s - w) // 2:(s - w) // 2 + w] = crop
small = cv2.resize(canvas, (size, size), interpolation=cv2.INTER_AREA)
return glyph_mask(small, dark_max=140, kernel=3, dilate=0) > 127
def _rotated(mask: np.ndarray, ang: float, size: int = 64) -> np.ndarray:
M = cv2.getRotationMatrix2D((size / 2, size / 2), ang, 1.0)
r = cv2.warpAffine(mask.astype(np.uint8) * 255, M, (size, size))
return r > 127
def _iou(a: np.ndarray, b: np.ndarray) -> float:
inter = np.logical_and(a, b).sum()
union = np.logical_or(a, b).sum()
return float(inter) / float(union) if union else 0.0
@dataclass
class IconClickResult:
points: List[Tuple[int, int]] # 原图坐标 (672x480 口径)
scores: List[float]
angles: List[float]
boxes: List[Tuple[int, int, int, int]]
target_count: int
@property
def ok(self) -> bool:
return len(self.points) == self.target_count and self.target_count > 0
@property
def min_score(self) -> float:
return min(self.scores) if self.scores else 0.0
def describe(self) -> str:
return " ".join("(%d,%d|%.2f@%d°)" % (p[0], p[1], s, a)
for p, s, a in zip(self.points, self.scores, self.angles))
class IconClickSolver:
"""提示图(目标条) + 背景图 -> 有序点击坐标。"""
def __init__(self, angles: Sequence[float] = tuple(range(-45, 46, 5)),
size: int = 64, min_area: int = MIN_AREA):
self.angles = tuple(angles)
self.size = size
self.min_area = min_area
def solve(self, bg, prompt) -> IconClickResult:
bg_bgr = _to_bgr(bg)
pr_bgr = _to_bgr(prompt)
targets = sorted(components(glyph_mask(pr_bgr), self.min_area), key=lambda b: b[0])
cands = components(glyph_mask(bg_bgr), self.min_area)
if not targets or not cands:
return IconClickResult([], [], [], [], len(targets))
tm = [_norm(pr_bgr, b, self.size) for b in targets]
cm = [_norm(bg_bgr, b, self.size) for b in cands]
# 候选旋转缓存
rot = [[(a, _rotated(c, a, self.size)) for a in self.angles] for c in cm]
# 得分矩阵: 每个目标在 (候选, 角度) 上的最大 IoU
S = np.zeros((len(tm), len(cm)), np.float32)
A = np.zeros((len(tm), len(cm)), np.float32)
for ti, t in enumerate(tm):
for ci, rs in enumerate(rot):
best = (-1.0, 0.0)
for ang, r in rs:
v = _iou(t, r)
if v > best[0]:
best = (v, ang)
S[ti, ci], A[ti, ci] = best
# 贪心一对一指派 (按全局得分从高到低)
pairs = sorted(((S[i, j], i, j) for i in range(len(tm)) for j in range(len(cm))),
key=lambda x: -x[0])
used_t, used_c = set(), set()
assign = {}
for v, i, j in pairs:
if i in used_t or j in used_c:
continue
used_t.add(i)
used_c.add(j)
assign[i] = (j, v, A[i, j])
points, scores, angs, boxes = [], [], [], []
for i in range(len(tm)):
if i not in assign:
continue
j, v, ang = assign[i]
x, y, w, h, _ = cands[j]
points.append((x + w // 2, y + h // 2))
scores.append(v)
angs.append(ang)
boxes.append((x, y, w, h))
return IconClickResult(points, scores, angs, boxes, len(tm))
def annotate(self, bg, result: IconClickResult) -> np.ndarray:
"""把结果画到背景图上 (自检用)。"""
img = _to_bgr(bg).copy()
for idx, ((px, py), sc) in enumerate(zip(result.points, result.scores), 1):
cv2.circle(img, (px, py), 26, (0, 0, 255), 3)
cv2.putText(img, str(idx), (px - 8, py + 8), cv2.FONT_HERSHEY_SIMPLEX,
1.0, (0, 255, 0), 2)
return img
if __name__ == "__main__":
import sys
bg = cv2.imread(sys.argv[1] if len(sys.argv) > 1 else "/tmp/1mscap/bg.jpg")
pr = cv2.imread(sys.argv[2] if len(sys.argv) > 2 else "/tmp/1mscap/prompt.jpg")
s = IconClickSolver()
r = s.solve(bg, pr)
print("ok=%s points=%s scores=%s angles=%s" % (r.ok, r.points, [round(x, 3) for x in r.scores], r.angles))
out = s.annotate(bg, r)
cv2.imwrite("/tmp/1mscap/annotated.jpg", out)
print("annotated -> /tmp/1mscap/annotated.jpg")
tcaptcha/solvers/slider.py
+226
View File
@@ -0,0 +1,226 @@
# -*- coding: utf-8 -*-
"""
腾讯天御『滑块拼图』求解器 (drag 型验证, 1ms.run aid=195672344 实测 2026-10-11)
题型结构 (真机 iframe `drag_ele.html` 实测):
* 背景图 `img_index=1`: 一张实景照片, 上面画着 **1~2 个缺口** (真缺口 + 干扰缺口)。
缺口 = 与拼图块**同形**的挖空, 内部是"同一张图被压暗"的结果, 外沿一圈浅色描边。
* 拼图块 `img_index=0`: 一张**精灵图**, 里面同时含拖动柄和拼图块本体。页面用
`background-position` / `background-size` 把拼图块那一块裁出来, 显示在滑轨左侧。
* 拖动柄(滑轨)与拼图块**同步横移**, 位移 1:1 (见 `har/1ms_dianxuan/PROGRESS.md`)。
为什么不用"轮廓/暗区阈值"直接找缺口:
背景照片本身有大量强边缘和暗区, 阈值法的召回/精确率都不够 (实测: 用局部变暗比
找缺口会被照片自身的暗部淹没)。但有一个**结构性**事实可以用:
"拼图块里装的是**未被压暗的原始内容**, 而缺口里装的是**同一块内容被压暗**的结果。"
于是 ZNCC(拼图块内容, 背景图窗口) 的峰值必然落在**真缺口**上; 干扰缺口内部装的是
**别处**的内容, 相关性明显更低。实测区分度 (本仓库 fixture):
真缺口 0.732
干扰缺口 0.551
拼图块当前位置 -0.136 (拼图块盖住的是完好的背景, 那里没有洞)
其余局部峰值 <= 0.28
坐标口径 (重要):
* 输入 `piece_x_image` / `piece_y_image` 是**背景图坐标系**(原始像素, 未经页面缩放);
* 输出 `dx_display` 是**页面显示像素**的横向拖动距离, 即 `dx_image * scale`,
其中 `scale = 背景图显示宽度 / 背景图原始宽度`。
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import List, Optional, Sequence, Tuple
import cv2
import numpy as np
# 拼图块精灵图/背景图的 alpha 阈值 (精灵图是 RGBA, 背景是照片)
DEFAULT_ALPHA_MIN = 128
# 局部峰值去重半径 (背景图坐标): 同一个峰附近的点算同一个候选
DEFAULT_NEIGHBOR = 20
# 纵向搜索范围 (背景图坐标): 允许上下抖 ±N 像素, 吸收取整/描边误差
DEFAULT_Y_JITTER = 3
def _to_bgr(image) -> np.ndarray:
"""把 bytes / PIL / ndarray 统一成 BGR 三通道 ndarray。"""
if isinstance(image, np.ndarray):
if image.ndim == 3 and image.shape[2] == 4:
return cv2.cvtColor(image, cv2.COLOR_BGRA2BGR)
return image
if isinstance(image, (bytes, bytearray)):
arr = np.frombuffer(image, np.uint8)
return cv2.imdecode(arr, cv2.IMREAD_COLOR)
return cv2.cvtColor(np.asarray(image.convert("RGB")), cv2.COLOR_RGB2BGR)
def sprite_region(bg_position: Sequence[float], bg_size: Sequence[float],
elem_size: Sequence[float], sprite_shape: Sequence[int]) -> Tuple[int, int, int, int]:
"""把 CSS 的 `background-position` / `background-size` 换算成精灵图里的像素区域。
参数:
bg_position: 元素上的 background-position, 形如 (-70.8333, -247.917) (CSS px)
bg_size: 元素上的 background-size, 形如 (345.06, 313.69) (CSS px, 已缩放后的整图)
elem_size: 元素的显示尺寸, 形如 (60.7143, 60.7143) (CSS px)
sprite_shape: 精灵图原始尺寸 (H, W)
返回 (x, y, w, h), 即该元素在**精灵图原始像素**里对应的矩形。
"""
sx, sy = float(bg_size[0]), float(bg_size[1])
h_sprite, w_sprite = int(sprite_shape[0]), int(sprite_shape[1])
kx, ky = w_sprite / sx, h_sprite / sy # CSS px -> 精灵图 px
x = int(round(-float(bg_position[0]) * kx))
y = int(round(-float(bg_position[1]) * ky))
w = int(round(float(elem_size[0]) * kx))
h = int(round(float(elem_size[1]) * ky))
return x, y, w, h
def piece_mask(piece, alpha_min: int = DEFAULT_ALPHA_MIN) -> np.ndarray:
"""从拼图块裁剪图里取**不透明**区域掩膜 (bool)。
精灵图是 RGBA: 只要 alpha 通道。若调用方给的是三通道图 (例如直接截屏裁的),
退化成"排除近白像素" (精灵图的空白处是纯白/全透明)。
"""
arr = np.asarray(piece)
if arr.ndim == 3 and arr.shape[2] == 4:
return arr[..., 3] > alpha_min
bgr = _to_bgr(arr)
return bgr.min(axis=2) < 245
def zncc_curve(bg_gray: np.ndarray, patch: np.ndarray, mask: np.ndarray,
y0: int, y_span: int = DEFAULT_Y_JITTER) -> Tuple[np.ndarray, int]:
"""在背景图的 y=y0±y_span 行上, 横向逐像素做零均值归一化互相关。
返回 (curve, best_y)。curve[x] 是"把拼图块左上角放在背景图 (x, best_y)"时的 ZNCC。
curve 只在 x ∈ [0, W-w] 上有定义。
"""
h, w = mask.shape
H, W = bg_gray.shape
if h > H or w > W:
raise ValueError(f"拼图块 ({w}x{h}) 比背景图 ({W}x{H}) 还大")
# 只取掩膜内的像素, 后续每列都按同一批下标取样 -> 可以向量化成一次大矩阵运算
idx_y, idx_x = np.nonzero(mask)
p = patch[idx_y, idx_x].astype(np.float32)
p = p - p.mean()
pn = float(np.sqrt((p * p).sum()))
if pn <= 1e-6:
raise ValueError("拼图块内容方差为 0, 无法做相关")
xs = np.arange(0, W - w + 1, dtype=np.int32)
curve = np.zeros(xs.shape, dtype=np.float32)
best_y, best_curve = y0, None
for y in range(max(0, y0 - y_span), min(H - h, y0 + y_span) + 1):
# 一次性把所有候选窗口的掩膜内像素取成 (len(xs), len(idx)) 矩阵
base = bg_gray[y + idx_y] # (N, W)
cols = xs[:, None] + idx_x[None, :] # (X, N)
q = base[np.arange(base.shape[0])[None, :], cols] # (X, N)
q = q.astype(np.float32)
q = q - q.mean(axis=1, keepdims=True)
denom = np.sqrt((q * q).sum(axis=1)) * pn
cur = np.where(denom > 1e-6, (q * p[None, :]).sum(axis=1) / np.maximum(denom, 1e-9), 0.0)
if best_curve is None or cur.max() > best_curve.max():
best_y, best_curve = y, cur
curve = best_curve if best_curve is not None else curve
return curve.astype(np.float64), best_y
def _peaks(curve: np.ndarray, neighbor: int, limit: int = 5) -> List[Tuple[int, float]]:
"""按分数从高到低取**互不相邻**(间距 > neighbor) 的局部峰值。"""
order = np.argsort(-curve)
picked: List[Tuple[int, float]] = []
for x in order:
xi = int(x)
if all(abs(xi - p[0]) > neighbor for p in picked):
picked.append((xi, float(curve[xi])))
if len(picked) >= limit:
break
return picked
@dataclass
class SliderResult:
"""滑块求解结果。坐标均为**背景图原始像素**, 除 dx_display 外。"""
ok: bool
dx_image: float
dx_display: float
score: float
runner_up: float
margin: float
piece_x_image: float
target_x_image: float
scale: float
y_image: int = 0
candidates: List[Tuple[int, float]] = field(default_factory=list)
def describe(self) -> str:
return (f"dx_image={self.dx_image:.1f} dx_display={self.dx_display:.1f}px "
f"score={self.score:.3f} runner_up={self.runner_up:.3f} margin={self.margin:.3f} "
f"cands={[(x, round(s, 3)) for x, s in self.candidates]}")
class SliderSolver:
"""滑块缺口求解器 (需权重为 0: 纯几何 + 相关, 不需要训练/模型文件)。"""
def __init__(self, min_score: float = 0.45, min_margin: float = 0.08,
neighbor: int = DEFAULT_NEIGHBOR, alpha_min: int = DEFAULT_ALPHA_MIN,
y_jitter: int = DEFAULT_Y_JITTER) -> None:
self.min_score = float(min_score)
self.min_margin = float(min_margin)
self.neighbor = int(neighbor)
self.alpha_min = int(alpha_min)
self.y_jitter = int(y_jitter)
def solve(self, bg, piece, piece_x_image: float, scale: float = 1.0,
piece_y_image: Optional[float] = None) -> SliderResult:
"""求拖动距离。
bg 背景图 (含缺口; 对应 `img_index=1`)
piece 拼图块裁剪图 (RGBA 精灵图里裁出来的那一块; 对应 `img_index=0`)
piece_x_image 拼图块裁剪图左边缘在背景图坐标系里的 x
scale 背景图显示宽度 / 背景图原始宽度 (页面缩放系数)
piece_y_image 拼图块裁剪图上边缘在背景图坐标系里的 y (缺省 0)
"""
bg_bgr = _to_bgr(bg)
bg_gray = bg_bgr.mean(axis=2).astype(np.float32)
cand = np.asarray(piece)
patch = _to_bgr(cand).mean(axis=2).astype(np.float32)
mask = piece_mask(cand, self.alpha_min)
if mask.sum() < 50:
return SliderResult(False, 0.0, 0.0, 0.0, 0.0, 0.0,
float(piece_x_image), float(piece_x_image), float(scale))
y0 = int(round(piece_y_image or 0))
curve, best_y = zncc_curve(bg_gray, patch, mask, y0, self.y_jitter)
peaks = _peaks(curve, self.neighbor)
if not peaks:
return SliderResult(False, 0.0, 0.0, 0.0, 0.0, 0.0,
float(piece_x_image), float(piece_x_image), float(scale), best_y)
best_x, best_s = peaks[0]
runs = peaks[1][1] if len(peaks) > 1 else 0.0
margin = best_s - runs
dx_image = float(best_x) - float(piece_x_image)
ok = bool(best_s >= self.min_score and margin >= self.min_margin)
return SliderResult(ok=ok, dx_image=dx_image, dx_display=dx_image * float(scale),
score=best_s, runner_up=runs, margin=margin,
piece_x_image=float(piece_x_image),
target_x_image=float(best_x), scale=float(scale),
y_image=best_y, candidates=peaks)
# --- 调试辅助 -------------------------------------------------------
@staticmethod
def annotate(bg, result: SliderResult, piece_w: int, piece_h: int) -> np.ndarray:
"""在背景图上标出"拼图块当前位置"与"目标位置", 用于人工核对。"""
img = _to_bgr(bg).copy()
px, py = int(round(result.piece_x_image)), int(result.y_image)
tx = int(round(result.target_x_image))
cv2.rectangle(img, (px, py), (px + piece_w, py + piece_h), (0, 255, 0), 2)
cv2.rectangle(img, (tx, py), (tx + piece_w, py + piece_h), (0, 0, 255), 2)
cv2.arrowedLine(img, (px + piece_w // 2, py + piece_h + 14),
(tx + piece_w // 2, py + piece_h + 14), (255, 255, 0), 2, tipLength=0.25)
return img
tcaptcha/solvers/text_click.py
+374
View File
@@ -0,0 +1,374 @@
# -*- coding: utf-8 -*-
"""
腾讯天御『文字点选』验证码求解器 (Tencent Turing Text-Click Solver)
适用题型: subcapclass = 2404, click_cfg.data_type = ["DynAnswerType_POS"] 且提示为
纯文字指令, 例如: "请依次点击：虫 呈 般"
与 tcaptcha.solver.CaptchaSolver (图标点选 2408: 提示是 img_index=0 的图标图) 的区别:
* 本题型 **没有提示图** (img_index=0 返回 0 字节)
* 背景图里只出现与提示完全相同的 3 个汉字 (随机位置), 不含额外干扰字
* 部分题目会做 **斜切(italic)变形**, 但实测不影响颜色分割
算法管线:
1. HSV 颜色分割 (腾讯点选汉字固定为橙黄色, H≈20 S≈185 V≈228) 提取字形掩膜
2. 形态学闭运算 + 膨胀连通同一汉字的多个偏旁, 连通域得到汉字包围盒
3. 掩膜 -> 白底黑字 -> 放大, 交给 OCR 集成(ddddocr + PP-OCRv4)做 CTC 前向
4. 取 CTC 每一时间步上目标字的概率最大值 -> (字形, 目标字) 相似度矩阵
5. 对 3! 个全排列求最优指派, 并用归一化后验概率给出本次求解的置信度
(置信度低的题目直接刷新会话, 避免提交明显不可靠的答案)
"""
from __future__ import annotations
import io
import itertools
import logging
import re
from typing import Dict, List, Optional, Sequence, Tuple
import cv2
import numpy as np
from PIL import Image
from tcaptcha.ocr.engines import EnsembleOcr
logger = logging.getLogger("tcaptcha_glyph")
# 腾讯点选汉字颜色 (实测 H≈20.5 S≈200 V≈232)。
# 关键是 **V(亮度)** 而不是色相: 暖色草地/沙地的 H 会落在 24~28、S 也不低,
# 但 V 只有 120~156, 而汉字 V≈230。早期版本用 V>=145 会把草地并进字形掩膜,
# 导致 3 个字被背景吞成 1~2 个候选 (见图 proxy/fail_2、fail_3)。
HSV_LOWER = (12, 145, 190)
HSV_UPPER = (34, 255, 255)
MIN_GLYPH_AREA = 700 # 单个汉字掩膜最小像素数
MERGE_KERNEL = 13 # 膨胀核, 用于把偏旁/离散笔画合并为同一汉字
SCALE = 3 # OCR 输入放大倍数
PAD = 24 # OCR 输入白边
_CJK_RE = re.compile(r"[\u4e00-\u9fff]")
# 滑块/图标题型的模板串关键词: 出现即说明提示里没有目标汉字
_NON_TARGET_HINTS = ("滑块", "拖动", "拼图", "请依次", "点击", "完成")
def parse_instruction(instruction: str) -> List[str]:
"""从 '请依次点击：虫 呈 般' 中解析出目标汉字序列 ['虫','呈','般']
注意: 图标点选 (2408) 与滑块 (2402/2401) 的 instruction 只有 '请依次点击：' /
'拖动下方滑块完成拼图' 这类无目标文字的模板串, 此时必须返回空列表, 否则会把
'请依次点击' 本身当成目标字。
"""
if not instruction:
return []
parts = re.split(r"[：:]", instruction, maxsplit=1)
if len(parts) == 2:
return _CJK_RE.findall(parts[1])
# 没有分隔符: 只有当整体就是一个短目标字序列 (如 '虫 呈 般') 时才接受
if any(h in instruction for h in _NON_TARGET_HINTS):
return []
chars = _CJK_RE.findall(instruction)
# 文字点选恒为 3~4 个目标字, 超过该数量必然是模板句
return chars if 0 < len(chars) <= 4 else []
class Glyph:
"""一个已定位的候选汉字字形"""
__slots__ = ("x0", "y0", "x1", "y1", "cx", "cy", "mask", "ocr_text", "confidence")
def __init__(self, x0: int, y0: int, x1: int, y1: int, mask: np.ndarray):
self.x0, self.y0, self.x1, self.y1 = x0, y0, x1, y1
self.cx = (x0 + x1) // 2
self.cy = (y0 + y1) // 2
self.mask = mask
self.ocr_text = ""
self.confidence = 0.0
@property
def point(self) -> Tuple[int, int]:
return self.cx, self.cy
def gray_image(self, thick: int = 0, scale: int = SCALE) -> np.ndarray:
"""掩膜 -> 白底黑字灰度图 -> 放大 (thick>0 时加粗笔画, 用于多尺度集成)"""
m = self.mask
if thick:
m = cv2.dilate(m, np.ones((3, 3), np.uint8), iterations=thick)
gray = 255 - m
h, w = gray.shape
side = max(h, w) + PAD
canvas = np.full((side, side), 255, np.uint8)
canvas[(side - h) // 2:(side - h) // 2 + h, (side - w) // 2:(side - w) // 2 + w] = gray
return cv2.resize(canvas, (side * scale, side * scale), interpolation=cv2.INTER_LINEAR)
# 兼容旧接口
def ocr_image(self, thick: int = 0, scale: int = SCALE) -> Image.Image:
return Image.fromarray(self.gray_image(thick=thick, scale=scale))
class SolveResult:
"""一次求解结果"""
__slots__ = ("points", "targets", "glyphs", "score_matrix", "posterior", "assignments")
def __init__(self, points, targets, glyphs, score_matrix, posterior, assignments):
self.points = points
self.targets = targets
self.glyphs = glyphs
self.score_matrix = score_matrix
self.posterior = posterior
self.assignments = assignments
@property
def text_match_ok(self) -> bool:
"""**自洽性校验**: 每个被指派的字形, 在集成得分矩阵里最优的目标字, 必须正好是它被指派的目标。
注意: 不能用单引擎的 `ocr_text` 做判断 —— 那只是给肉眼看的弱信号, 实测会把
"簿"读成"垂"、"笆"读成"芭"、"筹"读成"努"; 而**用于指派的那份集成矩阵**的行 argmax
始终与指派一致。因此这里以矩阵为唯一依据 (既可靠又不会误杀正确解)。
"""
if self.score_matrix is None or not self.assignments:
return False
if len(self.assignments) != len(self.targets):
return False
for ti, g in enumerate(self.assignments):
row = self.score_matrix[g]
if row.size == 0 or int(row.argmax()) != ti:
return False
return True
@property
def ok(self) -> bool:
return (len(self.points) == len(self.targets)
and len(self.points) > 0
and self.text_match_ok)
@property
def min_target_score(self) -> float:
if not self.assignments or self.score_matrix is None:
return 0.0
return min(float(self.score_matrix[g, ti]) for ti, g in enumerate(self.assignments))
def describe(self) -> str:
detail = ", ".join(
f"{t}<-{self.glyphs[g].ocr_text or '?'}@{self.glyphs[g].point}"
f"({self.score_matrix[g, ti]:.3f})"
for ti, (t, g) in enumerate(zip(self.targets, self.assignments))
)
status = "ALL_MATCHED" if self.text_match_ok else "MISMATCH"
return f"status={status} posterior={self.posterior:.3f} [{detail}]"
class TextClickSolver:
"""腾讯天御文字点选求解器 (颜色分割 + OCR 集成概率指派)"""
def __init__(self, recognizer: Optional[EnsembleOcr] = None, min_area: int = MIN_GLYPH_AREA,
variants: Sequence[Tuple[int, int]] = ((0, SCALE), (1, SCALE)),
min_glyph_score: float = 0.02):
self.min_area = min_area
self.variants = tuple(variants)
# 颜色分割会在照片的暖色区域误检出少量"伪字形", 用识别得分把它们剔除
self.min_glyph_score = min_glyph_score
self._recognizer = recognizer
@property
def recognizer(self) -> EnsembleOcr:
if self._recognizer is None:
self._recognizer = EnsembleOcr.default()
return self._recognizer
# ------------------------------------------------------------- detection
@staticmethod
def _mask_of(bgr: np.ndarray) -> np.ndarray:
hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, np.array(HSV_LOWER, np.uint8), np.array(HSV_UPPER, np.uint8))
# 闭运算填掉字内高光空洞
return cv2.morphologyEx(mask, cv2.MORPH_CLOSE,
cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
def detect_glyphs(self, image: bytes | np.ndarray | Image.Image) -> List[Glyph]:
"""颜色分割定位背景图中的全部汉字"""
bgr = self._to_bgr(image)
mask = self._mask_of(bgr)
dilated = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE,
(MERGE_KERNEL, MERGE_KERNEL)))
count, labels, stats, _ = cv2.connectedComponentsWithStats(dilated)
glyphs: List[Glyph] = []
for i in range(1, count):
if stats[i, cv2.CC_STAT_AREA] < self.min_area:
continue
sub = ((labels == i) & (mask > 0))
ys, xs = np.nonzero(sub)
if len(xs) == 0:
continue
x0, y0, x1, y1 = int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())
w, h = x1 - x0 + 1, y1 - y0 + 1
# 单字包围盒合理性: 过扁/过瘦/过小的多半是背景误检
if not (0.35 <= w / h <= 2.8) or w < 24 or h < 24:
continue
glyphs.append(Glyph(x0, y0, x1, y1,
(sub[y0:y1 + 1, x0:x1 + 1] * 255).astype(np.uint8)))
glyphs.sort(key=lambda g: (g.y0 // 64, g.x0))
return glyphs
def detect_glyphs_relaxed(self, image: bytes | np.ndarray | Image.Image) -> List[Glyph]:
"""放宽阈值的备用分割, 防止汉字偏暗或面积偏小导致的漏检"""
bgr = self._to_bgr(image)
hsv = cv2.cvtColor(bgr, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, np.array((8, 85, 135), np.uint8), np.array((45, 255, 255), np.uint8))
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
dilated = cv2.dilate(mask, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (MERGE_KERNEL, MERGE_KERNEL)))
count, labels, stats, _ = cv2.connectedComponentsWithStats(dilated)
glyphs: List[Glyph] = []
for i in range(1, count):
if stats[i, cv2.CC_STAT_AREA] < 350:
continue
sub = ((labels == i) & (mask > 0))
ys, xs = np.nonzero(sub)
if len(xs) == 0:
continue
x0, y0, x1, y1 = int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())
w, h = x1 - x0 + 1, y1 - y0 + 1
if not (0.28 <= w / h <= 3.5) or w < 18 or h < 18:
continue
glyphs.append(Glyph(x0, y0, x1, y1, (sub[y0:y1 + 1, x0:x1 + 1] * 255).astype(np.uint8)))
glyphs.sort(key=lambda g: (g.y0 // 64, g.x0))
return glyphs
# ---------------------------------------------------------- recognition
def char_scores(self, glyph: Glyph, targets: Sequence[str]) -> Dict[str, float]:
"""对单个字形, 返回每个目标字的集成概率 (多预处理变体取最大)"""
acc = {t: 0.0 for t in targets}
best_text, best_conf = "", 0.0
for thick, scale in self.variants:
gray = glyph.gray_image(thick=thick, scale=scale)
scores = self.recognizer.scores(gray, targets)
for t, v in scores.items():
acc[t] = max(acc[t], v)
# 用集成里最强单引擎的读数做可读性展示
try:
engine = self.recognizer.engines[-1]
chars, probs = engine.probabilities(glyph.gray_image())
idx = probs.argmax(axis=1)
seq, prev = [], -1
for i in idx:
if i != 0 and i != prev and i < len(chars):
seq.append(chars[i])
prev = i
best_text = "".join(seq)
best_conf = float(probs.max(axis=1).mean())
except Exception: # 展示信息失败不影响求解
pass
glyph.ocr_text, glyph.confidence = best_text, best_conf
return acc
# ----------------------------------------------------------------- solve
def solve(self, image: bytes | np.ndarray | Image.Image, instruction: str,
retries: int = 1) -> List[Tuple[int, int]]:
"""返回提示顺序的像素点击坐标; 失败返回空列表 (兼容旧接口)"""
res = self.solve_ex(image, instruction, retries=retries)
return res.points if res.ok else []
def solve_ex(self, image: bytes | np.ndarray | Image.Image, instruction: str,
retries: int = 1) -> SolveResult:
"""完整求解, 附带置信度(归一化后验概率)与命中明细"""
targets = parse_instruction(instruction)
empty = SolveResult([], targets, [], np.zeros((0, 0)), 0.0, [])
if len(targets) < 2:
logger.warning("提示语解析失败: %r", instruction)
return empty
for attempt in range(retries):
glyphs = self.detect_glyphs(image)
if len(glyphs) < len(targets):
glyphs = self.detect_glyphs_relaxed(image)
if len(glyphs) < len(targets):
logger.warning("第 %d 次: 检出字形 %d 个 < 目标 %d 个",
attempt + 1, len(glyphs), len(targets))
if attempt + 1 < retries:
continue
return empty
if len(glyphs) > len(targets):
logger.info("检出字形 %d 个 (目标 %d 个), 进行全局指派", len(glyphs), len(targets))
matrix = np.array([[self.char_scores(g, targets)[t] for t in targets]
for g in glyphs], dtype=np.float64) # (n, m)
# 剔除伪字形: 任何目标字都识别不出来的候选不参与指派 (不影响后验置信度)
row_max = matrix.max(axis=1) if matrix.size else np.zeros(0)
strong = [i for i in range(len(glyphs)) if row_max[i] >= self.min_glyph_score]
if len(strong) < len(targets):
# 得分整体偏低(生僻字/罕见字形)时退化为取得分最高的 m 个
order = np.argsort(-row_max)[:len(targets)]
strong = sorted(int(i) for i in order)
logger.info("可用强字形不足, 退化为得分 Top-%d 候选", len(targets))
dropped = [i for i in range(len(glyphs)) if i not in strong]
if dropped:
logger.info("剔除伪字形 %s", [glyphs[i].point for i in dropped])
glyphs = [glyphs[i] for i in strong]
matrix = matrix[strong]
# 每个字形对目标字的归一化分布 (尺度无关), 避免某个字形整体概率偏低时被忽略
eps = 1e-9
row_sums = matrix.sum(axis=1, keepdims=True)
norm = (matrix + eps) / (row_sums + eps * matrix.shape[1])
best_perm, best_score, second_score = None, -1.0, 0.0
perms = list(itertools.permutations(range(len(glyphs)), len(targets)))
probs = []
for perm in perms:
p = 1.0
for t_idx, g_idx in enumerate(perm):
p *= float(norm[g_idx, t_idx])
probs.append(p)
if p > best_score:
second_score = best_score
best_score, best_perm = p, perm
elif p > second_score:
second_score = p
total = float(sum(probs)) or 1.0
posterior = float(best_score / total)
if best_perm is None:
return empty
logger.info("题目 %s -> %s", "".join(targets),
", ".join(f"{targets[t]}<-{glyphs[g].ocr_text or '?'}@{glyphs[g].point}"
f"({matrix[g, t]:.4f})" for t, g in enumerate(best_perm)))
logger.info(" 后验置信度=%.4f (次优/最优=%.4f)", posterior,
(second_score / best_score) if best_score > 0 else 1.0)
result = SolveResult([glyphs[g].point for g in best_perm], targets, glyphs,
matrix, posterior, list(best_perm))
if best_score <= 0:
logger.warning("所有候选得分均为 0, 判定求解失败")
return SolveResult([], targets, glyphs, matrix, 0.0, list(best_perm))
return result
return empty
# ------------------------------------------------------------- utilities
@staticmethod
def _to_bgr(image: bytes | np.ndarray | Image.Image) -> np.ndarray:
if isinstance(image, np.ndarray):
return image
if isinstance(image, Image.Image):
return cv2.cvtColor(np.array(image.convert("RGB")), cv2.COLOR_RGB2BGR)
arr = np.frombuffer(image, np.uint8)
bgr = cv2.imdecode(arr, cv2.IMREAD_COLOR)
if bgr is None:
bgr = cv2.cvtColor(np.array(Image.open(io.BytesIO(image)).convert("RGB")),
cv2.COLOR_RGB2BGR)
return bgr
if __name__ == "__main__":
import sys
logging.basicConfig(level=logging.INFO, format="%(message)s")
solver = TextClickSolver()
img_path = sys.argv[1] if len(sys.argv) > 1 else "har/1ms_dianxuan/bg_3.jpg"
instr = sys.argv[2] if len(sys.argv) > 2 else "请依次点击：虫 呈 般"
res = solver.solve_ex(open(img_path, "rb").read(), instr)
print("points:", res.points)
print("confidence:", res.posterior)