604e889ca4
- spikes/：Word 转 PDF（Gotenberg）、格式识别、原句坐标、DeepSeek 抽取四项验证代码与两版提示词；样本派生产物和 .env 不进 git - tech-validation.md：完整结论。四项都可行：WPS 字体字符集要修正并改用行高调整过的字体；格式识别不需要 Docling；表格内切句待改进；抽取需第三道依据校验与规则兜底 - architecture.md：§3.5 加预处理与字体包，§3.6 证据校验改为三道，§10 回填结论 - 03 功能需求与 decisions.md（TV1）：召回 ≥ 99%、采纳准确率 ≥ 95%、页码 100%、高亮正文 ≥ 98% / 表格 ≥ 95% Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
28 lines
1.2 KiB
Python
28 lines
1.2 KiB
Python
"""把 S3 抽样结果按位置分类：表格内 / 页眉页脚带 / 正文，分别算文本保真率。"""
|
|
import json, sys, pathlib, collections
|
|
import pdfplumber
|
|
|
|
OUT = pathlib.Path(__file__).parent / "out"
|
|
pdf_dir = pathlib.Path(sys.argv[1]); val = json.loads((OUT / sys.argv[2]).read_text())
|
|
sents = {}
|
|
for f in OUT.glob("S*-sentences.json"):
|
|
for s in json.loads(f.read_text()):
|
|
sents[s["id"]] = s
|
|
tab_cache, stat = {}, collections.defaultdict(lambda: [0, 0])
|
|
for r in val["results"]:
|
|
s = sents[r["id"]]; sid = r["id"].split(":")[0]; key = (sid, s["page"])
|
|
if key not in tab_cache:
|
|
with pdfplumber.open(pdf_dir / f"{sid}.pdf") as pdf:
|
|
pg = pdf.pages[s["page"] - 1]
|
|
tab_cache[key] = ([t.bbox for t in pg.find_tables()], pg.height)
|
|
tables, h = tab_cache[key]
|
|
x0, top, x1, bottom = s["quads"][0]
|
|
if top < 0.09 * h or bottom > 0.92 * h:
|
|
cls = "页眉页脚带"
|
|
elif any(tx0 - 1 <= x0 and top >= tt - 1 and x1 <= tx1 + 1 and bottom <= tb + 1 for tx0, tt, tx1, tb in tables):
|
|
cls = "表格内"
|
|
else:
|
|
cls = "正文"
|
|
stat[cls][0] += r["in_docx_ignoring_autonumber"]; stat[cls][1] += 1
|
|
print({k: f"{ok}/{n} = {ok / n:.1%}" for k, (ok, n) in stat.items()})
|