feat(check): 偏离栏直读、响应错位判断；复查复用和硬预算（C2）
- deviation.py：模型判已响应的条目，证据所在表格行的偏离格写着「偏离 / 部分满足」等 就改判负偏离；照抄要求格、响应格写的是别的内容的改判「响应可能对不上」 - reuse.py：按「提示词 + 要求 + 候选句原文」算缓存键，存模型原始结论；复查时内容没变、 证据找得回的条目直接沿用，不调模型 - 每次检查送给模型的字数有上限，超出的条目列为请人工核对，写进覆盖范围 - 开发集 17 份：召回 61% → 73%，负偏离未说明 64% → 90%，规矩投标文件问题数不变 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
YANG JIANKUAN
@@ -215,4 +215,5 @@ samples/
|
||||
- 2026-11-04：查标第三轮：文件要在（按标题找文件名，不调模型）、扫描件兜底（C10，没找到的证明材料改为提示级「可能在扫描件里，请人工核对」）、响应方式存进清单并在清单页显示、可改（C12 的人工确认点，迁移 0007、0008）。开发集上规矩投标文件的致命问题 120 → 51，致命级召回不变（22 / 32）；证明材料缺失仍查不出，要靠 OCR / 多模态（S5）。见 `docs/architecture/check-eval.md` §5。
|
||||
- 2026-11-04：查标第四轮：规则检查（`check/rules.py`，规则集 2026.11-r1，不调模型）——大小写金额、分项合计、超限价、多个报价、别的项目名称、没填的地方、落款日期、投标有效期、附加条件、响应表里的未响应；文字层的句子带上单元格框，表格按单元格分行。请 Fable 从真实投标文件角度审读，按「认不准的不给致命」改了定级和取值范围。开发集 17 份上规则问题 57 / 57 查出、规矩投标文件 0 误报，但规则是看着生成器和开发词库写的，这个分数说明不了泛化，要等留出集和国动真实投标文件。见 `docs/architecture/check-eval.md` §6。
|
||||
- 2026-11-04：查标第 2 步收尾：需人工核对（份数、密封、递交、CA 签章加密、解密、保证金，从清单按主题合并，查标页勾选、按项目存，P2）；「报价」类清单项（C7，读标从清单表规则读出分项 / 单价上限）和规则 `PRICE.OVER_CEILING_ITEM`；评测集派生 4 份报价问题投标文件，21 份上规则问题 62 / 62、误报 0。见 `docs/architecture/check-eval.md` §7。
|
||||
- 下一步：查标第 3 步的 D09（响应对象错位）/ D06（负偏离未说明）提示词 v3、token 预算和复查复用；第 2 步还差分项和采购需求逐项对照（P13）。
|
||||
- 2026-11-04：查标第五轮（不调模型的后处理）：偏离栏直读（证据所在表格行的偏离格写着不满足就改判负偏离）、响应错位（照抄要求格 + 响应格写的是别的内容）；复查复用（送审内容没变的条目沿用上次的模型结论）和每次检查的硬预算。开发集 17 份召回 61% → 73%，负偏离未说明 64% → 90%，致命级 22 → 24 / 32，规矩投标文件的问题数不变（235）。见 `docs/architecture/check-eval.md` §8。
|
||||
- 下一步：送给模型的「同一行」改按单元格分行（要重跑模型），重复条目分组、致命项重问；第 2 步还差分项和采购需求逐项对照（P13）。
|
||||
|
||||
@@ -184,8 +184,28 @@
|
||||
|
||||
**评测集**：从规矩的投标文件派生 4 份只带报价问题的（`evals/check/price_defects.py`：只改一个分项单价，合价、合计、大小写跟着改，算术一致）：S01-B-p01、S04-B-p01（超总价限价 1 元 / 1%）、S10-A-p02（超采购包限价 1%）、S10-A-p03（分项单价超上限 1%）。S10 的分项上限加起来就是采购包限价，总价超限必然有分项超上限，生成器把这条一并记进金标准。21 份上规则问题 62 / 62 查出、误报 0；同样只说明规则和生成器对得上。
|
||||
|
||||
## 8. 下一步
|
||||
## 8. 第五轮：偏离栏直读、响应错位、复查复用和预算（2026-11-04）
|
||||
|
||||
**改了什么**（`check/deviation.py`，都在模型判断之后、不调模型）：
|
||||
- **偏离栏直读**：模型判「已响应 / 只写满足」的条目，看它引的证据句所在的表格行；同一行有单元格整格写着「偏离 / 负偏离 / 部分满足 / 未完全响应」这类字样，改判负偏离，证据带上那一格。只写「偏离」两字的要是识别出单元格框的格（跨页无框的行抽字时「无偏离」会被拆成「无」「偏离」，规矩的 S07-B 上出过一次）。漏掉的 D06 多数就是这种：响应格照抄要求「满足，……」、偏离格写「偏离」，提示词已经要求读整行，模型还是漏。
|
||||
- **响应错位**（D09）：模型判「已响应」、引的证据全是照抄的要求格，同一行右边的响应格写的是别的内容（「显示器升降行程：150mm」「1位」），且和要求没有一个字对 / 数字字母串重合 → 改判「响应可能对不上」，说明里写出响应格原文。笼统的「支持 / 具备」、「详见……」、只写「满足」、要求太短（汉字少于 8 个）的不判。条件很严，宁可漏。
|
||||
- **复查复用**（C2，`check/reuse.py`）：每条清单项按「提示词版本 + 要求 + 候选句和同一行的单元格原文」算缓存键，和模型那一层的原始结论（后处理之前）一起存进 `check_item_result`；复查时键相同、证据句在本次投标文件里原样找得回来的，直接沿用，不调模型；响应方式、偏离栏直读这些后处理照常重算。覆盖范围写沿用了几条。
|
||||
- **硬预算**（C2）：每次检查送给模型的字数上限 120 万字（开发集平均约 5 倍），按批次顺序分配，超出的批次不调模型，条目记「超出预算」列为请人工核对，覆盖范围另写。
|
||||
|
||||
| 指标（17 份） | 第三轮（scan） | + 偏离栏直读 | + 响应错位（本轮） |
|
||||
|---|---|---|---|
|
||||
| 查标召回 | 92 / 150（61%） | 107（71%） | **109 / 150（73%）** |
|
||||
| 致命级召回 | 22 / 32 | 24 / 32 | **24 / 32** |
|
||||
| D06 负偏离未说明 | 38 / 59（64%） | 53 / 59（90%） | 53 / 59（90%） |
|
||||
| D09 响应对象错位 | 5 / 13 | 5 / 13 | **7 / 13** |
|
||||
| 规矩的投标文件报出的问题 | 235 | 236 | **235** |
|
||||
| 模型调用 | — | 全部命中缓存 | 全部命中缓存 |
|
||||
|
||||
**读法**：两项都是确定的规则，没有改提示词、没有多花 token。响应错位报了 3 处（2 个单元），全对，规矩的投标文件上没有改判；第一版报错的 4 处（「13.56MHz」没算数字字母重合、「支持」当成别的内容、要求太短）修掉后才是这个数。偏离栏直读依赖投标文件的偏离栏写法，真实文件里偏离栏常写「无」「—」「见说明」，这些不会改判（不算不满足），只有明确写了不满足的字样才改。
|
||||
|
||||
## 9. 下一步
|
||||
|
||||
1. 第 2 步还差：分项报价表和采购需求清单逐项对照（P13）、散文写法的单价上限。
|
||||
2. 第 3 步：D09（响应对象错位）、D06（负偏离未说明）的提示词 v3，token 预算、复查复用、重复条目分组。
|
||||
2. 送给模型的「同一行」改用按单元格分行（现在还用句子重叠，跨格、合并格会串）；要重跑模型，等下一次改提示词时一起做。
|
||||
3. 第 3 步：D09（响应对象错位）、D06（负偏离未说明）的提示词 v3，token 预算、复查复用、重复条目分组。
|
||||
3. 留出集（S03、S06、S09，风格 B + C，封存的词库）和真实国动投标文件，在第 3 步冻结代码后只跑一次。
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
| 第 0 步 评测集 | 试跑完成，继续扩充 | 规范 v0.2、留出集划分已定，留出集词库和风格 C 已封存；生成、注入、渲染、离线查标、评分工具已写；开发集 7 份招标文件、14 份投标文件第一版跑完，结果和发现见 `check-eval.md` §2 |
|
||||
| 第 1 步 链路打通 | ✅ 2026-11-04 | 见下方「第 1 步落地记录」 |
|
||||
| 第 2 步 规则 | 基本完成 | 响应方式（C12）、文件要在、扫描件兜底、规则检查 `rules.py`（报价形式、形式要件、响应表）、需人工核对（`manual.py`、`manual-checks` 接口）、「报价」类清单项和分项上限（C7）。还差分项和采购需求逐项对照（P13）、散文写法的单价上限。见 `check-eval.md` §4–§7 |
|
||||
| 第 3 步 AI 响应匹配 | 进行中 | 整行上下文、提示词 v2；D09 / D06、token 预算、复查复用还没做 |
|
||||
| 第 3 步 AI 响应匹配 | 进行中 | 整行上下文、提示词 v2、偏离栏直读、响应错位、复查复用、硬预算；还差送给模型的同一行改按单元格、重复条目分组、致命项重问。见 `check-eval.md` §8 |
|
||||
| 第 4～6 步 | 未开始 | |
|
||||
|
||||
**第 1 步落地记录**（和本文设计不一致、或留到后面的，记在这里）：
|
||||
|
||||
@@ -25,7 +25,7 @@ from bid_doctor.ai import prompts
|
||||
from bid_doctor.ai import service as ai_service
|
||||
from bid_doctor.ai.providers.base import near_limit
|
||||
from bid_doctor.ai.service import OutputIncomplete, provider
|
||||
from bid_doctor.check import applicability, findings, match, rules
|
||||
from bid_doctor.check import applicability, deviation, findings, match, rules
|
||||
from bid_doctor.checklist import response_mode
|
||||
from bid_doctor.checklist import price
|
||||
from bid_doctor.checklist.extract import amount_yuan
|
||||
@@ -184,6 +184,8 @@ async def main() -> None:
|
||||
ap.add_argument("--v1", action="store_true", help="第 1 步的做法：提示词 v1、不带同一行的单元格（对比基线）")
|
||||
ap.add_argument("--no-modes", action="store_true", help="不按响应方式和「要不要回答」筛选（C12 之前的做法）")
|
||||
ap.add_argument("--rules-only", action="store_true", help="只重跑规则检查，写回已有运行的 results.json")
|
||||
ap.add_argument("--no-direct", action="store_true", help="不做偏离栏直读（对比用）")
|
||||
ap.add_argument("--no-mismatch", action="store_true", help="不做响应错位判断（对比用）")
|
||||
args = ap.parse_args()
|
||||
d = OUT / args.bid
|
||||
manifest = json.loads((d / "manifest.json").read_text())
|
||||
@@ -247,6 +249,10 @@ async def main() -> None:
|
||||
decisions = {i: applicability.decide(p, modes) for i, p in items.items()}
|
||||
items = {i: p for i, p in items.items() if decisions[i].match}
|
||||
results, stats = await match.run(ctx, uuid4(), items, docs, progress)
|
||||
if not args.no_direct:
|
||||
deviation.direct_read(results, docs)
|
||||
if not args.no_mismatch:
|
||||
deviation.mismatch_read(results, items, docs)
|
||||
if decisions:
|
||||
match.apply_modes(results, items, decisions)
|
||||
match.scanned_fallback(results, items, decisions, docs)
|
||||
|
||||
124
server/src/bid_doctor/check/deviation.py
Normal file
124
server/src/bid_doctor/check/deviation.py
Normal file
@@ -0,0 +1,124 @@
|
||||
"""偏离栏直读（m3-plan §2.1「偏离表直取」，check-eval §8）：模型判「已响应」的条目，看它引的证据句所在的表格行，
|
||||
同一行有单元格写着「偏离 / 负偏离 / 部分满足 / 未完全响应」这类字样的，改判负偏离。
|
||||
|
||||
开发集上负偏离未说明（D06）漏掉的，多数是响应格照抄了要求（「满足，……」）、偏离格写着「偏离」，
|
||||
模型只看了响应格——提示词已经要求读整行，模型还是会漏，这里用确定的规则补上。
|
||||
单元格要整格是这些字样才算（「如不满足……」这类正文不算）；改判后证据带上那一格，出处可回溯。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from uuid import UUID
|
||||
|
||||
from bid_doctor.check.match import BidDoc, Result
|
||||
from bid_doctor.document.pipeline import layout
|
||||
from bid_doctor.document.pipeline.text_layer import Sentence, norm
|
||||
|
||||
NEGATIVE = re.compile(
|
||||
r"^(未完全响应|不完全响应|部分响应|未响应|不响应|负偏离|有偏离|偏离|不满足|部分满足|未满足|不符合|部分符合)"
|
||||
r"([（(].*)?[。；]?$"
|
||||
)
|
||||
|
||||
|
||||
def negative(text: str) -> bool:
|
||||
return bool(NEGATIVE.match(norm(text)))
|
||||
|
||||
|
||||
def negative_cell(s: Sentence) -> bool:
|
||||
"""偏离格写着不满足的字样。只写「偏离」两个字的要是识别出了单元格框的格：跨页、无框线的行抽字时
|
||||
「无偏离」可能被拆成「无」和「偏离」两段（开发集 S07-B 遇到过）。"""
|
||||
t = norm(s.text)
|
||||
return negative(t) and (t not in ("偏离", "偏离。") or s.cell is not None)
|
||||
|
||||
|
||||
def direct_read(results: dict[UUID, Result], docs: list[BidDoc]) -> int:
|
||||
"""就地改判，返回改判的条数。"""
|
||||
rows: dict[int, list[Sentence]] = {}
|
||||
for d in docs:
|
||||
for page in layout.pages(d.sentences).values():
|
||||
for row in layout.rows(page):
|
||||
for c in row:
|
||||
rows[id(c)] = row
|
||||
changed = 0
|
||||
for r in results.values():
|
||||
if r.verdict not in ("responded", "no_value") or not r.evidence:
|
||||
continue
|
||||
hit: tuple[BidDoc, Sentence] | None = None
|
||||
for d, s in r.evidence:
|
||||
cell = next((c for c in rows.get(id(s), []) if c is not s and negative_cell(c)), None)
|
||||
if cell is not None:
|
||||
hit = (d, cell)
|
||||
break
|
||||
if hit is None:
|
||||
continue
|
||||
before, r.verdict = r.verdict, "negative_deviation"
|
||||
if not any(x is hit[1] for _, x in r.evidence):
|
||||
r.evidence = [*r.evidence, hit][-3:]
|
||||
r.detail = {**r.detail, "direct_read": norm(hit[1].text), "model_verdict": before}
|
||||
changed += 1
|
||||
return changed
|
||||
|
||||
|
||||
POSITIVE = re.compile(r"^(满足|完全满足|响应|完全响应|已响应|符合|同意|接受)[，,：:。；、\s]*")
|
||||
MARKER = re.compile(r"^(无偏离|正偏离|负偏离|偏离|满足|完全响应|响应|是|否|/|—+|-+)$")
|
||||
POINTER = re.compile(r"详见|参见|见附件|见第|第.{0,6}页|如下|见下|见上|附后|另附|承诺")
|
||||
GENERIC = re.compile(r"^(支持|具备|提供|有|可以|能|是|均支持|已具备|可实现|实现)[。；;]?$") # 笼统的肯定回答
|
||||
CJK = re.compile(r"[一-鿿]")
|
||||
TOKEN = re.compile(r"[A-Za-z0-9.#+/]{2,}")
|
||||
MIN_REQ_CJK = 8 # 要求太短（参数名在另一格）时没法比
|
||||
|
||||
|
||||
def _bigrams(s: str) -> set[str]:
|
||||
"""汉字字对 + 数字字母串（「13.56MHz」「IP67」）。"""
|
||||
t = "".join(CJK.findall(s))
|
||||
return {t[i : i + 2] for i in range(len(t) - 1)} | {m.lower() for m in TOKEN.findall(s)}
|
||||
|
||||
|
||||
def _similar(a: str, b: str) -> float:
|
||||
x, y = _bigrams(norm(a)), _bigrams(norm(b))
|
||||
return 2 * len(x & y) / (len(x) + len(y)) if x and y else 0.0
|
||||
|
||||
|
||||
def mismatch_read(results: dict[UUID, Result], items: dict[UUID, dict[str, object]], docs: list[BidDoc]) -> int:
|
||||
"""响应对象错位（D09）：模型判「已响应」，但引的证据全是照抄要求的那一格，同一行右边的响应格写的是别的内容
|
||||
（「满足，不额外收费」「显示器升降行程：150mm」），和要求一个字对都不重合 → 改判「未找到响应」，说明里写出
|
||||
响应格原文。只看到「满足」、写「详见……」的不算错位（前者交给 no_value，后者可能是指引）。"""
|
||||
rows: dict[int, list[Sentence]] = {}
|
||||
for d in docs:
|
||||
for page in layout.pages(d.sentences).values():
|
||||
for row in layout.rows(page):
|
||||
for c in row:
|
||||
rows[id(c)] = row
|
||||
changed = 0
|
||||
for item_id, r in results.items():
|
||||
if r.verdict != "responded" or not r.evidence:
|
||||
continue
|
||||
item = items.get(item_id) or {}
|
||||
req = f"{item.get('title') or ''}{item.get('requirement_text') or ''}"
|
||||
if len(CJK.findall(req)) < MIN_REQ_CJK:
|
||||
continue
|
||||
cells: list[tuple[BidDoc, Sentence, Sentence]] = []
|
||||
for d, s in r.evidence:
|
||||
if MARKER.match(norm(s.text)):
|
||||
continue # 证据里带的「无偏离」「满足」标记格：跳过
|
||||
row = rows.get(id(s), [])
|
||||
if POSITIVE.match(norm(s.text)) or _similar(s.text, req) < 0.5 or not s.cell:
|
||||
break # 证据里有响应格本身，或不是照抄的要求格：不判
|
||||
right = [c for c in row if c.cell and c.cell[0] >= s.cell[2] - 1 and not MARKER.match(norm(c.text))]
|
||||
if not right:
|
||||
break
|
||||
cells.append((d, s, right[0]))
|
||||
else:
|
||||
if not cells:
|
||||
continue
|
||||
d, _, resp = cells[0]
|
||||
extra = POSITIVE.sub("", norm(resp.text))
|
||||
if len(extra) < 2 or POINTER.search(extra) or GENERIC.match(extra) or _similar(extra, req) > 0:
|
||||
continue
|
||||
r.verdict = "not_found"
|
||||
r.evidence = [(d, resp)]
|
||||
r.detail = {**r.detail, "mismatch": norm(resp.text), "model_verdict": "responded"}
|
||||
r.reason_code = "response_mismatch"
|
||||
changed += 1
|
||||
return changed
|
||||
@@ -21,6 +21,8 @@ REASONS = {
|
||||
"quote_too_short": "引文太短，无法核对",
|
||||
"missing_answer": "这一条没有得到判断结果",
|
||||
"model_error": "模型服务暂时不可用",
|
||||
"over_budget": "超出本次检查的预算",
|
||||
"response_mismatch": "响应格写的是别的内容",
|
||||
}
|
||||
|
||||
|
||||
@@ -66,13 +68,23 @@ def spec(result: Result, item: dict[str, Any]) -> Spec | None:
|
||||
why = REASONS.get(result.reason_code or "", result.reason_code or "")
|
||||
text = f"模型认为投标文件{where}，但出处没能核对（{why}），按「未找到响应」处理。请人工核对。"
|
||||
return Spec("unresponded", sev, f"未找到响应：{name}", text, {**base, "evidence_status": "unverified"})
|
||||
if result.verdict == "not_found" and result.detail.get("mismatch"):
|
||||
text = (
|
||||
f"投标文件照抄了这条要求，但同一行的响应格写的是「{result.detail['mismatch'][:40]}」，"
|
||||
"和这条要求对不上，像是别的条目的内容。请核对。"
|
||||
)
|
||||
return Spec("unresponded", sev, f"响应可能对不上：{name}", text, base)
|
||||
if result.verdict == "not_found":
|
||||
text = f"在投标文件（{scope}）中没有找到对这条要求的响应。" if scope else "没有找到对这条要求的响应。"
|
||||
return Spec("unresponded", sev, f"未找到响应：{name}", text, base)
|
||||
if result.verdict == "negative_deviation":
|
||||
return Spec(
|
||||
"negative_deviation", sev, f"负偏离未说明：{name}", "投标文件写明了偏离或不满足，没有找到说明。", base
|
||||
cell = result.detail.get("direct_read")
|
||||
text = (
|
||||
f"投标文件这一行的偏离 / 响应栏写着「{cell}」，没有找到说明。"
|
||||
if cell
|
||||
else "投标文件写明了偏离或不满足，没有找到说明。"
|
||||
)
|
||||
return Spec("negative_deviation", sev, f"负偏离未说明：{name}", text, base)
|
||||
if result.verdict == "suspected_unmet":
|
||||
value = result.detail.get("response_value")
|
||||
text = (
|
||||
|
||||
@@ -19,9 +19,10 @@ from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from bid_doctor.ai import prompts
|
||||
from bid_doctor.ai import service as ai
|
||||
from bid_doctor.ai.providers.base import TransientError
|
||||
from bid_doctor.check import presence
|
||||
from bid_doctor.check import presence, reuse
|
||||
from bid_doctor.check.retrieval import Index
|
||||
from bid_doctor.core.db import TenantContext
|
||||
from bid_doctor.document.pipeline.text_layer import Sentence, norm
|
||||
@@ -31,6 +32,9 @@ CANDIDATES = 5 # 每条清单项送给模型的候选句数
|
||||
ROW_MATES = 6 # 候选句在表格里时，带上同一行的单元格（检索常命中投标人照抄要求的那一格，响应和偏离在同一行的别的格）
|
||||
BATCH = 8 # 每次调用判断的清单项数
|
||||
SPLIT_DEPTH = 2
|
||||
# 每次检查送给模型的字数上限（决策 C2 的硬预算）。开发集平均每份约 25 万字（17 份 220 万 token），
|
||||
# 定在 5 倍左右：正常文件碰不到，异常大的文件不会无限花钱；超出的条目列为「未判定」，写进覆盖范围
|
||||
MAX_INPUT_CHARS = 1_200_000
|
||||
CONCURRENCY = 4 # TODO(第 3 步)：和解析共用「供应商 × 租户」统一限流（extract.py 的同一个 TODO）
|
||||
SID = re.compile(r"^d(\d+)p(\d+)s(\d+)$")
|
||||
END_PUNCT = "。；;，,、：:）)"
|
||||
@@ -56,6 +60,9 @@ class BidDoc:
|
||||
header_footer: list[Sentence] = field(default_factory=list[Sentence]) # 页眉页脚（规则检查看页眉里的项目名称）
|
||||
|
||||
|
||||
Batch = list[tuple[UUID, dict[str, Any], dict[str, tuple["BidDoc", Sentence]]]] # 一次调用判断的清单项
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class Result:
|
||||
verdict: str
|
||||
@@ -71,6 +78,10 @@ class Stats:
|
||||
calls: int = 0
|
||||
failed_batches: int = 0
|
||||
splits: int = 0
|
||||
reused: int = 0 # 复查时沿用上次结论的条数
|
||||
over_budget: int = 0 # 超出预算没判断的条数
|
||||
input_chars: int = 0
|
||||
keys: dict[UUID, bytes] = field(default_factory=dict[UUID, bytes]) # 清单项 → 复用缓存键
|
||||
verify_failures: dict[str, int] = field(default_factory=dict[str, int])
|
||||
|
||||
def fail(self, code: str) -> None:
|
||||
@@ -220,7 +231,10 @@ async def run(
|
||||
items: dict[UUID, dict[str, Any]],
|
||||
docs: list[BidDoc],
|
||||
progress: Callable[[float], Awaitable[None]],
|
||||
prior: dict[bytes, dict[str, Any]] | None = None,
|
||||
budget: int = MAX_INPUT_CHARS,
|
||||
) -> tuple[dict[UUID, Result], Stats]:
|
||||
"""prior：上一次检查的「缓存键 → 模型原始结论」（reuse.snapshot，带 run_id），键相同的条目沿用、不调模型。"""
|
||||
stats = Stats()
|
||||
scope = [d.label for d in docs]
|
||||
index = Index([s for d in docs for s in d.sentences])
|
||||
@@ -256,19 +270,27 @@ async def run(
|
||||
rows[sid] = [short_sid(doc, m) for m in mates]
|
||||
for m in mates: # 同一行的单元格也可以引用（响应、偏离常写在那里）
|
||||
allowed.setdefault(short_sid(doc, m), (doc, m))
|
||||
fragments = [(sent_of[h][1].text, [sent_of[m][1].text for m in rows.get(h, [])]) for h in hits_of[item_id]]
|
||||
stats.keys[item_id] = k = reuse.key(PROMPT, item, fragments)
|
||||
if prior and k in prior and (old := reuse.restore(prior[k], docs, str(prior[k].get("run_id")))):
|
||||
results[item_id] = old
|
||||
stats.reused += 1
|
||||
continue
|
||||
queue.append((item_id, item, allowed))
|
||||
|
||||
sem = asyncio.Semaphore(CONCURRENCY)
|
||||
total, done = max(len(queue), 1), [0]
|
||||
|
||||
async def call(batch: list[tuple[UUID, dict[str, Any], dict[str, tuple[BidDoc, Sentence]]]], depth: int) -> None:
|
||||
keyed = [(f"I{i + 1}", item_id, item, allowed) for i, (item_id, item, allowed) in enumerate(batch)]
|
||||
|
||||
user = build_user(
|
||||
[(k, item, hits_of[item_id]) for k, item_id, item, _ in keyed],
|
||||
def render(batch: Batch) -> str:
|
||||
return build_user(
|
||||
[(f"I{i + 1}", item, hits_of[item_id]) for i, (item_id, item, _) in enumerate(batch)],
|
||||
sent_of,
|
||||
{h: rows.get(h, []) for _, item_id, _, _ in keyed for h in hits_of[item_id]},
|
||||
{h: rows.get(h, []) for item_id, _, _ in batch for h in hits_of[item_id]},
|
||||
)
|
||||
|
||||
async def call(batch: Batch, depth: int, user: str | None = None) -> None:
|
||||
keyed = [(f"I{i + 1}", item_id, item, allowed) for i, (item_id, item, allowed) in enumerate(batch)]
|
||||
user = user if user is not None else render(batch)
|
||||
try:
|
||||
async with sem:
|
||||
stats.calls += 1
|
||||
@@ -306,8 +328,23 @@ async def run(
|
||||
done[0] += len(batch)
|
||||
await progress(done[0] / total)
|
||||
|
||||
await asyncio.gather(*(call(queue[i : i + BATCH], 0) for i in range(0, len(queue), BATCH)))
|
||||
if queue and all(results[item_id].reason_code == "model_error" for item_id, _, _ in queue):
|
||||
# 预算按批次顺序分配（确定的，不受并发先后影响）；超出的批次不调模型
|
||||
system = len(prompts.get(*PROMPT).body)
|
||||
planned: list[tuple[Batch, str]] = []
|
||||
for i in range(0, len(queue), BATCH):
|
||||
batch = queue[i : i + BATCH]
|
||||
user = render(batch)
|
||||
if stats.input_chars + system + len(user) > budget:
|
||||
for item_id, _, _ in batch:
|
||||
results[item_id] = Result(
|
||||
"undetermined", "model", reason_code="over_budget", detail={"search_scope": scope}
|
||||
)
|
||||
stats.over_budget += len(batch)
|
||||
continue
|
||||
stats.input_chars += system + len(user)
|
||||
planned.append((batch, user))
|
||||
await asyncio.gather(*(call(b, 0, u) for b, u in planned))
|
||||
if planned and all(results[item_id].reason_code == "model_error" for b, _ in planned for item_id, _, _ in b):
|
||||
raise TransientError(
|
||||
"model_error", "all check.match batches failed"
|
||||
) # 一批都没成功：按基础设施错误失败，退回额度
|
||||
|
||||
64
server/src/bid_doctor/check/reuse.py
Normal file
64
server/src/bid_doctor/check/reuse.py
Normal file
@@ -0,0 +1,64 @@
|
||||
"""复查复用（m3-plan §2.1、决策 C2）：复查时，送给模型的内容（提示词版本、要求、候选句和同一行的单元格原文）
|
||||
和上一次检查完全一样的清单项，直接沿用上次的结论，不再调模型。
|
||||
|
||||
- 缓存键是送给模型那部分内容的摘要：投标文件改了的地方原文变了，键就变了，会重新判断；
|
||||
- 只沿用模型给出的结论（layer = model），证据按句子编号在本次的投标文件里找回来，找不全就不沿用；
|
||||
- 沿用的结论在 detail 里记 reused_from，检查覆盖范围里写沿用了几条。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from bid_doctor.document.pipeline.text_layer import Sentence
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from bid_doctor.check.match import BidDoc, Result
|
||||
|
||||
|
||||
def key(prompt: tuple[str, int], item: dict[str, Any], fragments: list[tuple[str, list[str]]]) -> bytes:
|
||||
"""fragments：[(候选句原文, [同一行的单元格原文])]，按检索顺序。"""
|
||||
payload = json.dumps(
|
||||
{
|
||||
"prompt": list(prompt),
|
||||
"title": item.get("title"),
|
||||
"requirement": item.get("requirement_text"),
|
||||
"categories": sorted(item.get("categories") or []),
|
||||
"spec": item.get("spec") or {},
|
||||
"fragments": fragments,
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
default=str,
|
||||
)
|
||||
return hashlib.sha256(payload.encode()).digest()
|
||||
|
||||
|
||||
def evidence_keys(r: Result) -> list[list[Any]]:
|
||||
"""证据句的位置（分册编号、页、句），存进 check_item_result.detail，复查时用来找回句子。"""
|
||||
return [[str(d.id), s.page_no, s.seq, s.text] for d, s in r.evidence]
|
||||
|
||||
|
||||
def snapshot(r: Result) -> dict[str, Any]:
|
||||
"""模型那一层的原始结论（响应方式、偏离栏直读这些后处理之前），复查时沿用的是它。"""
|
||||
return {"verdict": r.verdict, "evidence_keys": evidence_keys(r), "detail": dict(r.detail)}
|
||||
|
||||
|
||||
def restore(stored: dict[str, Any], docs: list[BidDoc], run_id: str) -> Result | None:
|
||||
"""按上次存下的证据位置在本次的投标文件里找回句子（原文也要一样），找不全返回 None。"""
|
||||
from bid_doctor.check.match import Result
|
||||
|
||||
by_pos: dict[tuple[str, int, int], tuple[BidDoc, Sentence]] = {
|
||||
(str(d.id), s.page_no, s.seq): (d, s) for d in docs for s in d.sentences
|
||||
}
|
||||
evidence = []
|
||||
for doc_id, page, seq, text in stored.get("evidence_keys") or []:
|
||||
hit = by_pos.get((doc_id, page, seq))
|
||||
if hit is None or hit[1].text != text:
|
||||
return None
|
||||
evidence.append(hit)
|
||||
status = "verified" if evidence else "none"
|
||||
detail = {**(stored.get("detail") or {}), "reused_from": run_id}
|
||||
return Result(str(stored["verdict"]), "model", status, evidence=evidence, detail=detail)
|
||||
@@ -21,6 +21,7 @@ from decimal import Decimal, InvalidOperation
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from bid_doctor.check import deviation
|
||||
from bid_doctor.check.match import BidDoc
|
||||
from bid_doctor.document.pipeline import layout
|
||||
from bid_doctor.document.pipeline.layout import column as _col
|
||||
@@ -996,7 +997,7 @@ SERIAL = re.compile(r"\d{1,3}")
|
||||
|
||||
def _negative(cell: str, head: str) -> bool:
|
||||
t = norm(cell)
|
||||
if NEG_RESP.match(t):
|
||||
if NEG_RESP.match(t) or deviation.negative(t):
|
||||
return True
|
||||
if YES_NO_HEAD.search(head) and t in ("否", "×"):
|
||||
return True
|
||||
|
||||
@@ -22,7 +22,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from bid_doctor.ai.models import AiCall
|
||||
from bid_doctor.ai.providers.base import TransientError
|
||||
from bid_doctor.billing import service as billing
|
||||
from bid_doctor.check import applicability, findings, manual, match, rules
|
||||
from bid_doctor.check import applicability, deviation, findings, manual, match, reuse, rules
|
||||
from bid_doctor.check.models import CheckItemResult, CheckRun, CheckRunDocument, Finding, FindingCitation
|
||||
from bid_doctor.check.service import RULE_SET_VERSION, open_fatal
|
||||
from bid_doctor.checklist import edit as checklist_edit
|
||||
@@ -170,6 +170,7 @@ async def _stages(ctx: TenantContext, run_id: UUID) -> None:
|
||||
)
|
||||
]
|
||||
facts = await _facts(s, ctx, r, snapshot)
|
||||
prior = await _prior(s, r)
|
||||
# 招标文件按类别的响应方式（C12）：用清单里存的（人工改过的为准）；解析早于这项功能的项目临时识别
|
||||
modes = response_mode.from_stored(stored_modes) if stored_modes else response_mode.detect(tender_sents)
|
||||
bidding = set(r.lot_ids)
|
||||
@@ -194,7 +195,10 @@ async def _stages(ctx: TenantContext, run_id: UUID) -> None:
|
||||
last[0] = pct
|
||||
await _stage(ctx, run_id, "match", pct)
|
||||
|
||||
results, stats = await match.run(ctx, run_id, items, bid_docs, progress)
|
||||
results, stats = await match.run(ctx, run_id, items, bid_docs, progress, prior)
|
||||
raw = {i: reuse.snapshot(x) for i, x in results.items() if x.layer == "model" and x.verdict != "undetermined"}
|
||||
deviation.direct_read(results, bid_docs) # 证据所在的表格行里偏离格写着「偏离」：改判负偏离
|
||||
deviation.mismatch_read(results, items, bid_docs) # 证据只是照抄的要求格、响应格写的是别的：改判未找到
|
||||
match.apply_modes(results, items, decisions)
|
||||
match.scanned_fallback(results, items, decisions, bid_docs)
|
||||
|
||||
@@ -222,7 +226,7 @@ async def _stages(ctx: TenantContext, run_id: UUID) -> None:
|
||||
by_doc = {
|
||||
d.id: d for d in (await s.execute(select(SourceDocument).where(SourceDocument.id.in_(doc_ids)))).scalars()
|
||||
}
|
||||
await _write(s, ctx, r, items, results, by_doc)
|
||||
await _write(s, ctx, r, items, results, by_doc, stats.keys, raw)
|
||||
await _write_rules(s, ctx, r, hits, facts, by_doc)
|
||||
await _carry_over(s, ctx, r, set(items))
|
||||
_stage_nowait(r, "summarize")
|
||||
@@ -336,6 +340,24 @@ async def _write_rules(
|
||||
await s.flush()
|
||||
|
||||
|
||||
async def _prior(s: AsyncSession, r: CheckRun) -> dict[bytes, dict[str, Any]]:
|
||||
"""上一次检查的「缓存键 → 模型原始结论」（复查复用，决策 C2）。"""
|
||||
if r.previous_run_id is None:
|
||||
return {}
|
||||
rows = (
|
||||
await s.execute(
|
||||
select(CheckItemResult.cache_key, CheckItemResult.detail).where(
|
||||
CheckItemResult.check_run_id == r.previous_run_id, CheckItemResult.cache_key.is_not(None)
|
||||
)
|
||||
)
|
||||
).tuples()
|
||||
return {
|
||||
bytes(k): {**d["model_raw"], "run_id": str(r.previous_run_id)}
|
||||
for k, d in rows
|
||||
if k and d and d.get("model_raw")
|
||||
}
|
||||
|
||||
|
||||
async def _write(
|
||||
s: AsyncSession,
|
||||
ctx: TenantContext,
|
||||
@@ -343,8 +365,11 @@ async def _write(
|
||||
items: dict[UUID, dict[str, Any]],
|
||||
results: dict[UUID, match.Result],
|
||||
by_doc: dict[UUID, SourceDocument],
|
||||
keys: dict[UUID, bytes],
|
||||
raw: dict[UUID, dict[str, Any]],
|
||||
) -> None:
|
||||
"""每条清单项留一行结论（C9）；没响应好的生成问题，挂招标侧和投标侧出处。阶段幂等：先清本批次的残留。"""
|
||||
"""每条清单项留一行结论（C9）；没响应好的生成问题，挂招标侧和投标侧出处。阶段幂等：先清本批次的残留。
|
||||
模型那一层的原始结论（raw）和缓存键一起存下，下次复查时内容没变就沿用（reuse.py）。"""
|
||||
await s.execute(delete(Finding).where(Finding.check_run_id == r.id))
|
||||
await s.execute(delete(CheckItemResult).where(CheckItemResult.check_run_id == r.id))
|
||||
for item_id, item in items.items():
|
||||
@@ -363,7 +388,9 @@ async def _write(
|
||||
evidence_status=res.evidence_status,
|
||||
reason_code=res.reason_code,
|
||||
bid_citation_ids=[c.id for c in bid_cites],
|
||||
detail=res.detail,
|
||||
detail={**res.detail, **({"model_raw": raw[item_id]} if item_id in raw else {})},
|
||||
cache_key=keys.get(item_id) if item_id in raw else None,
|
||||
reused_from_run_id=UUID(res.detail["reused_from"]) if res.detail.get("reused_from") else None,
|
||||
)
|
||||
)
|
||||
spec = findings.spec(res, item)
|
||||
@@ -511,7 +538,9 @@ def _coverage(
|
||||
rule_cov: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
"""检查覆盖范围：报告首页、查标页顶部都读这里（FR-M3-09、m3-plan §2.5）。"""
|
||||
undetermined = sum(x.verdict == "undetermined" for x in results.values())
|
||||
over_budget = sum(x.reason_code == "over_budget" for x in results.values())
|
||||
undetermined = sum(x.verdict == "undetermined" for x in results.values()) - over_budget
|
||||
reused = sum(bool(x.detail.get("reused_from")) for x in results.values())
|
||||
unverified = sum(x.evidence_status == "unverified" for x in results.values())
|
||||
scanned = sum(len(x["pages_without_text"]) for x in pages_info)
|
||||
checked = [f"响应完整性：正式清单 {len(items)} 条，逐条在投标文件里找响应"]
|
||||
@@ -522,6 +551,10 @@ def _coverage(
|
||||
not_checked.append(f"不在所投采购包里的 {other_lot} 条")
|
||||
if undetermined:
|
||||
not_checked.append(f"{undetermined} 条没有完成判断（模型漏答或暂时不可用），已列为「请人工核对」")
|
||||
if over_budget:
|
||||
not_checked.append(f"{over_budget} 条超出本次检查的预算没有判断（投标文件特别大），已列为「请人工核对」")
|
||||
if reused:
|
||||
checked.append(f"其中 {reused} 条和上次检查送去判断的内容完全相同，沿用上次的结论")
|
||||
if scanned:
|
||||
not_checked.append(f"{scanned} 页没有文字层（扫描件），这些页上的内容没有检查")
|
||||
for reason, n in skipped.items():
|
||||
|
||||
@@ -339,3 +339,24 @@ async def test_manual_checks_are_kept_per_project(user: User) -> None:
|
||||
assert any("列入「需人工核对」2 条" in x for x in again["coverage"]["not_checked"])
|
||||
kept = (await user.get(f"/projects/{project_id}/manual-checks")).json()["items"]
|
||||
assert [i["checked"] for i in kept] == [True, False]
|
||||
|
||||
|
||||
async def test_recheck_reuses_unchanged_model_results(user: User) -> None:
|
||||
"""复查复用（C2）：同一份投标文件再查，送给模型的内容没变，模型那一层沿用上次的结论，不再调模型。"""
|
||||
from bid_doctor.ai.models import AiCall
|
||||
from bid_doctor.core.db import TenantContext, tenant_tx
|
||||
|
||||
project_id, bid, first = await checked_project(user)
|
||||
second = await run_check(user, project_id, [bid["id"]])
|
||||
assert second["counts"]["fatal"] == first["counts"]["fatal"]
|
||||
reused = [x for x in second["coverage"]["checked"] if "沿用上次的结论" in x]
|
||||
assert reused, second["coverage"]["checked"]
|
||||
|
||||
ctx = TenantContext(tenant_id=UUID(user.tenant_id), user_id=UUID(str(user.me["user"]["id"])))
|
||||
async with tenant_tx(ctx) as s:
|
||||
from sqlalchemy import func, select
|
||||
|
||||
n = (await s.execute(select(func.count()).where(AiCall.job_id == UUID(second["id"])))).scalar_one()
|
||||
assert n == 0
|
||||
results = (await user.get(f"/check-runs/{second['id']}/item-results")).json()["items"]
|
||||
assert all(r["bid_citations"] for r in results if r["verdict"] == "responded") # 沿用的结论证据也找回来了
|
||||
|
||||
@@ -191,3 +191,91 @@ def test_manual_topics_merge_by_topic_and_skip_irrelevant() -> None:
|
||||
assert list(got) == ["DECRYPT", "SIGN_ENCRYPT", "PAPER", "SUBMIT"]
|
||||
assert got["SIGN_ENCRYPT"] == ["须用CA数字证书加密投标文件"]
|
||||
assert got["SUBMIT"][0].startswith("投标文件递交截止") # 写明截止时间的做代表
|
||||
|
||||
|
||||
def test_direct_read_turns_negative_deviation_cell_into_finding() -> None:
|
||||
"""偏离栏直读：证据句所在的表格行里偏离格写「偏离」，模型判了已响应也改判负偏离（check-eval §8）。"""
|
||||
from bid_doctor.check import deviation
|
||||
|
||||
def c(seq: int, text: str, x0: float, y: float) -> Sentence:
|
||||
s = Sentence(key=f"b:p1:s{seq}", page_no=1, seq=seq, text=text, quads=[[x0 + 2, y + 4, x0 + 90, y + 16]])
|
||||
s.cell = [x0, y, x0 + 100, y + 30]
|
||||
return s
|
||||
|
||||
req, resp, dev = c(1, "支持链路聚合", 100, 50), c(2, "满足，支持链路聚合", 200, 50), c(3, "偏离", 300, 50)
|
||||
req2, resp2, dev2 = c(4, "支持堆叠", 100, 80), c(5, "满足，支持堆叠", 200, 80), c(6, "无偏离", 300, 80)
|
||||
prose = Sentence(
|
||||
key="b:p1:s7", page_no=1, seq=7, text="如不满足上述要求，按负偏离处理。", quads=[[74, 120, 500, 132]]
|
||||
)
|
||||
d = match.BidDoc(uuid4(), 1, "技术分册", [req, resp, dev, req2, resp2, dev2, prose])
|
||||
results = {
|
||||
uuid4(): match.Result("responded", "ai", "verified", evidence=[(d, resp)]),
|
||||
uuid4(): match.Result("responded", "ai", "verified", evidence=[(d, resp2)]),
|
||||
uuid4(): match.Result("responded", "ai", "verified", evidence=[(d, prose)]),
|
||||
}
|
||||
assert deviation.direct_read(results, [d]) == 1
|
||||
first, second, third = results.values()
|
||||
assert first.verdict == "negative_deviation" and [s.text for _, s in first.evidence] == [
|
||||
"满足，支持链路聚合",
|
||||
"偏离",
|
||||
]
|
||||
assert first.detail["model_verdict"] == "responded"
|
||||
assert second.verdict == third.verdict == "responded"
|
||||
|
||||
|
||||
def test_mismatch_read_only_when_response_cell_is_unrelated() -> None:
|
||||
"""响应对象错位（D09）：证据只是照抄的要求格、同一行响应格写的是别的内容 → 未找到响应。
|
||||
指引、相关内容、只写满足的不算。"""
|
||||
from bid_doctor.check import deviation
|
||||
|
||||
def c(seq: int, text: str, x0: float, y: float) -> Sentence:
|
||||
s = Sentence(key=f"b:p1:s{seq}", page_no=1, seq=seq, text=text, quads=[[x0 + 2, y + 4, x0 + 90, y + 16]])
|
||||
s.cell = [x0, y, x0 + 100, y + 30]
|
||||
return s
|
||||
|
||||
rows = [
|
||||
("支持云管理、VLAN隔离、标准交换三种工作模式", "显示器升降行程：150mm"),
|
||||
("支持端口汇聚、端口流量统计", "满足，支持端口汇聚和端口流量统计"),
|
||||
("批量数据处理在非工作时段内完成", "满足，详见技术方案第12页"),
|
||||
("系统支持国产操作系统", "满足"),
|
||||
]
|
||||
sents, results, items = [], {}, {}
|
||||
for n, (req, resp) in enumerate(rows):
|
||||
a, b, dev = (
|
||||
c(n * 3 + 1, req, 100, 50 + n * 40),
|
||||
c(n * 3 + 2, resp, 200, 50 + n * 40),
|
||||
c(n * 3 + 3, "无偏离", 300, 50 + n * 40),
|
||||
)
|
||||
sents += [a, b, dev]
|
||||
k = uuid4()
|
||||
items[k] = {"title": req, "requirement_text": req}
|
||||
results[k] = (a, b)
|
||||
d = match.BidDoc(uuid4(), 1, "技术分册", sents)
|
||||
res = {k: match.Result("responded", "ai", "verified", evidence=[(d, a)]) for k, (a, _) in results.items()}
|
||||
assert deviation.mismatch_read(res, items, [d]) == 1
|
||||
verdicts = [r.verdict for r in res.values()]
|
||||
assert verdicts == ["not_found", "responded", "responded", "responded"]
|
||||
first = next(iter(res.values()))
|
||||
assert first.detail["mismatch"] == "显示器升降行程：150mm" and first.reason_code == "response_mismatch"
|
||||
|
||||
|
||||
async def test_over_budget_items_are_undetermined_without_model_calls() -> None:
|
||||
"""硬预算（C2）：超出预算的批次不调模型，条目记「超出预算」，覆盖范围另外写明。"""
|
||||
from bid_doctor.core.db import TenantContext
|
||||
|
||||
d, _ = bid()
|
||||
items = {
|
||||
uuid4(): {
|
||||
"title": "服务器内存不低于32GB",
|
||||
"requirement_text": "服务器内存不低于32GB",
|
||||
"categories": ["tech_param"],
|
||||
}
|
||||
}
|
||||
|
||||
async def progress(_: float) -> None:
|
||||
return None
|
||||
|
||||
ctx = TenantContext(tenant_id=uuid4(), user_id=uuid4())
|
||||
results, stats = await match.run(ctx, uuid4(), items, [d], progress, budget=10)
|
||||
(r,) = results.values()
|
||||
assert (r.verdict, r.reason_code, stats.calls, stats.over_budget) == ("undetermined", "over_budget", 0, 1)
|
||||
|
||||
Reference in New Issue
Block a user