feat(check): 文件要在、扫描件兜底(C10);响应方式存进清单并可改(C12)
- check/presence.py:「提供《授权委托书》」「须提交投标函」这类要求按标题里有没有这个文件名判断,不调模型。
- 扫描件兜底:要附证明材料的要求没找到、投标文件里有扫描件页时,改为提示级「可能在扫描件里,请人工核对」。
- 响应方式解析时识别、带依据句子存进 checklist.response_modes(迁移 0008);清单页显示、可改(写审计,
重新解析不覆盖人工改过的);查标用存下来的。新接口 PUT /projects/{id}/checklist/response-modes/{category}。
- check_item_result.layer 加 presence(迁移 0007);契约同步默认接受的两种结论。
开发集 17 份:规矩投标文件的致命问题 120 → 51,致命级召回不变(22 / 32)(docs/architecture/check-eval.md §5)。
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -60,6 +60,7 @@ def main() -> None:
|
||||
"| 分类 | 召回 |",
|
||||
"|---|---|",
|
||||
*[f"| {k} | {pct(rec[k], tot[k])} |" for k in sorted(tot) if k not in ("查标", "端到端", "致命级")],
|
||||
f"| 召回里只是「扫描件待人工核对」的 | {rec['其中只是扫描件待人工核对']} |",
|
||||
"",
|
||||
f"- 规矩的投标文件:清单 {items_clean} 条,报出问题 {findings_clean} 个;其中规矩响应的要求上 {fp}/{ok}(每 100 条 {100 * fp / ok:.1f})"
|
||||
if ok
|
||||
|
||||
@@ -129,7 +129,15 @@ async def main() -> None:
|
||||
docs = []
|
||||
for n, pdf in enumerate(sorted(d.glob("v*.pdf")), start=1):
|
||||
pages = text_layer.extract(f"{args.bid}-{pdf.stem}", pdf.read_bytes())
|
||||
docs.append(match.BidDoc(uuid4(), n, pdf.stem, text_layer.body_sentences(pages)))
|
||||
docs.append(
|
||||
match.BidDoc(
|
||||
uuid4(),
|
||||
n,
|
||||
pdf.stem,
|
||||
text_layer.body_sentences(pages),
|
||||
[p.page_no for p in pages if p.text_source == "empty"],
|
||||
)
|
||||
)
|
||||
ai_service.run_json = baseline_run_json if args.baseline else cached_run_json # type: ignore[assignment]
|
||||
if args.v1:
|
||||
match.PROMPT = ("check.match", 1)
|
||||
@@ -148,6 +156,7 @@ async def main() -> None:
|
||||
results, stats = await match.run(ctx, uuid4(), items, docs, progress)
|
||||
if decisions:
|
||||
match.apply_modes(results, items, decisions)
|
||||
match.scanned_fallback(results, items, decisions, docs)
|
||||
rows = []
|
||||
for item_id, item in all_items.items():
|
||||
if item_id not in items:
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
- 清单项 → 标注单元:清单项的招标侧句子和单元的句子有重叠就对上(和读标评分同一口径);
|
||||
- 按单元去重:同一要求在清单里有几条,只要一条报出就算召回,几条都报不算误报;
|
||||
- 问题类型按类型族比(D01 / D02 / D09 / D11 / D13 → unresponded,D03 / D07 → suspected_unmet,……);
|
||||
证明材料类(D02 / D11 / D13)报成「可能在扫描件里,请人工核对」也算标出来了(C10),另外单独计数;
|
||||
- 只算复核过的问题(决策 E1);--include-unreviewed 时也算没复核的,报告里标明;
|
||||
- 读标没抽到的问题单列「读标漏抽」:不算查标漏报,但计入端到端召回;
|
||||
- 对不上任何标注单元的问题单列「无法对照」。
|
||||
@@ -27,10 +28,10 @@ import gold # noqa: E402
|
||||
OUT = EVALS / "out" / "check"
|
||||
FAMILY = {
|
||||
"D01": {"unresponded"},
|
||||
"D02": {"unresponded"},
|
||||
"D02": {"unresponded", "manual_review"},
|
||||
"D09": {"unresponded"},
|
||||
"D11": {"unresponded"},
|
||||
"D13": {"unresponded"},
|
||||
"D11": {"unresponded", "manual_review"},
|
||||
"D13": {"unresponded", "manual_review"},
|
||||
"D16": {"unresponded"},
|
||||
"D03": {"suspected_unmet"},
|
||||
"D07": {"suspected_unmet"},
|
||||
@@ -122,6 +123,8 @@ def main() -> None:
|
||||
if good:
|
||||
for k in [*keys, "端到端", "查标"]:
|
||||
rec[k] += 1
|
||||
if all(it["finding"]["type"] == "manual_review" for it in good):
|
||||
rec["其中只是扫描件待人工核对"] += 1
|
||||
if fatal:
|
||||
sev_n += 1
|
||||
sev_ok += any(it["finding"]["severity"] == "fatal" for it in good)
|
||||
|
||||
Reference in New Issue
Block a user