返回 DeepSeek-Reasonix
RECOVERY_VALIDATION.md
根目录 / docs / RECOVERY_VALIDATION.md
1 # Protocol and recovery validation / 协议与恢复验证
2
3 Implementation date / 实施日期: 2026-09-05.
4
5 ## Reference scope / 参考范围
6
7 This document covers provider-protocol recovery: retries and stream
8 resumption. Transcript persistence, session versions, and concurrent writers
9 are described in [`SESSION_RECOVERY_AND_PARALLELISM.md`](SESSION_RECOVERY_AND_PARALLELISM.md)
10 and [`SESSION_OWNERSHIP.md`](SESSION_OWNERSHIP.md).
11
12 本文只覆盖 provider 协议层的恢复(重试与断流续传)。会话持久化、会话版本与
13 并发写者见上述两份文档。
14
15 Pi is pinned to `9841914c71a74d81abe07f751aefd271fd924e63`. The executable
16 comparison uses its `packages/ai/src/utils/retry.ts` `retryAssistantCall`
17 helper, with an injected zero delay. It is not an end-to-end comparison of the
18 Pi runtime, and does not measure either product's production recovery rate.
19
20 Pi 固定为上述提交;可执行对照使用其重试辅助函数,测试中等待设为零。
21 这不是完整 Pi Agent 的端到端对照,也不是线上恢复率或费用基准。
22 主会话持续等待是 Reasonix 的扩展,不是 Pi 的默认策略。
23
24 | Fault / 故障 | Pi helper requests / 请求数 | Reasonix requests / 请求数 | Outcome / 结果 |
25 | --- | ---: | ---: | --- |
26 | Two temporary service failures, then success / 两次临时服务故障后成功 | 3 | 3 | Automatically completes / 自动完成 |
27 | Exhausted quota / 配额耗尽 | 1 | 1 | Stops immediately / 立即停止 |
28 | Persistent interrupted stream / 持续断流 | 4 | 4 | Finite failure; no endless regeneration / 有限失败,不无限重生成 |
29
30 For the recoverable fixture, additional requests are 2 and manual continuation
31 is 0. Reasonix's scheduled quick backoff is 2 + 4 seconds; the persistent failure
32 fixture schedules 2 + 4 + 8 seconds. Test clocks avoid actually waiting that long.
33 Missing provider usage remains unknown: these fixtures do not establish token
34 cost, real recovery latency, or a statistically meaningful recovery percentage.
35
36 可恢复用例增加两次请求,人工继续次数为零;名义退避总时长为 6 秒。
37 持续失败用例的名义退避总时长为 14 秒。测试时钟跳过实际等待。
38 缺失的供应商用量保持未知,不把它记为零费用;这些用例不构成 token
39 成本、真实恢复耗时或有统计意义的恢复率测量。
40
41 ## Deterministic coverage / 确定性覆盖
42
43 - Compatible missing reasoning: no regeneration before the tool, and one tool
44 execution. / 兼容协议缺少 reasoning 不额外生成,工具执行一次。
45 - Mixed network and replay failures share four total attempts; cancellation
46 prevents a late completion from starting tools. / 混合失败共用四次请求上限,
47 取消后的迟到响应不能启动工具。
48 - Main conversations wait after quick retry exhaustion; subagents, planners,
49 partial streams and unknown tool outcomes cannot enter that wait.
50 / 主会话可持续等待;子任务、规划、部分断流和未知工具结果不能进入该状态。
51 - Auxiliary calls use 2/4/8-second finite backoff, aggregate usage, and suppress
52 failed partial text. / 辅助调用有限退避、汇总用量、排除失败的部分文本。
53 - A failed durable intent prevents mutation, including directory creation.
54 Verification checks every recorded target again before skipping a write.
55 Conflicts, changed symlink destinations, unavailable/replaced transports and
56 unknown evidence versions cannot prove success. / 意图持久化失败不开始写入或
57 创建目录;跳过写入前重新核验所有目标,冲突、符号链接换目标、原通道不可用或
58 被替换、未知证据版本均不能证明成功。
59 - Raw future-version write evidence survives serialization and is excluded
60 from model messages. / 未知版本原始证据往返保留,不进入正常模型消息。
61
62 Tests: `internal/agent/pi_recovery_test.go`,
63 `internal/agent/write_recovery_test.go`,
64 `internal/provider/recovery_test.go`,
65 `internal/provider/auxiliary_recovery_test.go`,
66 `internal/tool/builtin/write_recovery_test.go`, plus existing protocol,
67 checkpoint, session-generation, and frontend stream suites.
68
69 ## Validation boundaries / 验证边界
70
71 At the initial local-validation stage no live credential was available. The
72 official endpoint follow-up below supersedes that limitation, but does not
73 reproduce the reporter's exact Windows session or every custom gateway.
74 The real Serve-page check was attempted through the available in-app and Edge
75 browser channels; both blocked the loopback test URL before loading the page.
76 Frontend type and stream-state tests are separate from a real UI smoke test.
77 Native Desktop visual behavior still requires that smoke test.
78
79 初始本地验证阶段没有可用密钥;下方补充了官方端点实测,但仍未复现反馈者的
80 完整 Windows 会话,本地与官方端点通过不等同于所有中转场景均已解决。
81 尝试通过内置浏览器和 Edge 检查实际 Serve 页面时,两条通道均在加载前拦截
82 本地测试地址。前端类型与流状态测试不能替代真实界面检查,原生 Desktop
83 的视觉表现仍需完成界面冒烟验证。
84
85 ## Local checks / 本地检查结果
86
87 Passed / 已通过:
88
89 - Root module: `go test -p 1 ./... -timeout 180s`; the final affected Agent,
90 Provider, built-in tool and event-wire packages were also rerun successfully.
91 - Desktop module: `go test ./... -timeout 240s`.
92 - `make lint` and `git diff --check`.
93 - Frontend `tsc --noEmit` and `test:stream`.
94 - Targeted `-race` checks across Agent, Provider, built-in tools and Controller,
95 including cancellation, unknown writes, auxiliary retry budgets, concurrent
96 snapshots and session-generation changes.
97
98 根模块与 Desktop 全量测试、最终受影响包复测、静态检查、前端类型与流状态
99 测试均通过。恢复与取消、辅助重试、文件核验、并发快照、会话切换的定向
100 race 检查通过。真实界面与服务端验证仍受上节边界约束。
101
102 ## Anthropic compatibility follow-up / Anthropic 兼容性补充
103
104 The adapter distinguishes native Claude signature requirements from unknown
105 Anthropic-compatible gateways. Complete unsigned native non-tool history can be
106 converted to assistant text in the request view; tool activity, mixed proofs,
107 redacted data and incomplete reasoning remain protected. Gateways preserve
108 received unsigned thinking when replay is enabled, and explicit DeepSeek
109 contracts remain strict. No empty signatures are fabricated.
110
111 Additional deterministic HTTP/SSE fixtures in
112 `internal/agent/anthropic_compatibility_e2e_test.go` verify:
113
114 - Missing or unsigned thinking on a custom adaptive gateway: two HTTP requests
115 for one tool round and the final answer; one tool execution, no regeneration.
116 - A server rejection of unsigned history: three HTTP requests, one tool
117 execution, with completed-tool facts in the repaired request and original
118 reasoning retained locally.
119 - A compatible native text conversion does not claim the missing-reasoning
120 recovery incident. Adapter tests cover immutable history, idempotent
121 conversion, signed/redacted block preservation, and rejection of unsafe input.
122
123 新增测试区分原生 Claude、未知 Anthropic 网关和显式 DeepSeek 契约。模拟网关
124 缺失或返回 unsigned thinking 时,一个工具轮加最终回答共两次 HTTP 请求,工具
125 执行一次;模拟服务端拒绝旧 thinking 时,共三次请求,工具仍只执行一次,修复
126 请求携带已完成事实,本地保留原始 reasoning。兼容文本转换不消耗严格恢复预算。
127
128 These are simulated servers, not live endpoint acceptance tests. No DeepSeek,
129 Anthropic or OpenCode Go API credentials were available for this follow-up.
130 本轮为模拟服务端验证;环境未提供上述供应商密钥,不能据此宣称 #9808 的反馈者
131 端点或所有兼容网关已通过实测。
132
133 ## Official endpoint investigation (2026-09-05) / 官方端点实测
134
135 This follow-up uses a user-authorized credential only against
136 `api.deepseek.com`, on `deepseek-v4-flash` and `deepseek-v4-pro`. It covers
137 `/chat/completions`, `/responses`, and `/anthropic/v1/messages`. Credentials
138 are supplied in memory to isolated test processes; no credential file is
139 created. Only synthetic marker tools and confined temporary-file writes are
140 available. No shell, MCP, credential reader, or other provider is exposed.
141
142 本次使用用户授权的官方密钥,覆盖 Flash、Pro 及三种协议。密钥仅在测试进程中
143 传递;模型只能调用固定标记工具或临时目录内的文件写入,不提供 shell、MCP 或
144 凭据读取能力。以下区分原始服务端契约、真实模型加本地故障注入、确定性回归。
145
146 ### Raw replay contract / 原始回放契约
147
148 The initial 44 direct, non-streaming HTTP probes all returned 200. Those probes
149 reused provider-issued call IDs. A second set of 30 probes replaced call IDs,
150 with full-reasoning controls to establish that the replacement IDs themselves
151 were valid. Twenty-two returned 200; eight deliberately invalid requests
152 returned the expected 400. Both models produced the same distinctions:
153
154 | Historical input / 历史输入 | Chat | Anthropic Messages | Responses |
155 | --- | --- | --- | --- |
156 | Original call IDs; omit reasoning / 原始调用 ID,省略 reasoning | 200 | 200 | 200 |
157 | Replacement call IDs; full reasoning / 替换调用 ID,完整 reasoning | 200 | 200 | 200 |
158 | Replacement call IDs; omit reasoning / 替换调用 ID,省略 reasoning | 400 `reasoning_content` | 400 `content[].thinking` | 400 `reasoning_text` |
159 | Replacement call IDs; explicit empty field/block/lists / 替换调用 ID,显式空字段、块或列表 | 200, empty string | 200, empty thinking block | 400, empty content/summary lists |
160
161 All rejection messages require the named content to be passed back. The
162 ID-dependent difference is evidence of server behavior, not proof of its
163 internal storage/cache implementation or a durability guarantee. A successful
164 request with an original call ID is **not** sufficient evidence that missing
165 reasoning will always be accepted after restart, ID normalization, or gateway
166 translation. Do not fabricate opaque Responses items or extend Chat's empty
167 field rule to other protocols from these results.
168
169 三种协议在替换调用 ID 后均能复现真实的回放 400。原始 ID 下成功,不能证明
170 历史重载、ID 转换或网关转发后仍可省略 reasoning;服务端内部如何找回这些
171 信息未验证。Responses 的“空列表”与合法的 reasoning item 并不等价。
172
173 The official [thinking-mode guide](https://api-docs.deepseek.com/guides/thinking_mode/)
174 continues to require full historical reasoning with tools. The
175 [Anthropic compatibility guide](https://api-docs.deepseek.com/guides/anthropic_api/)
176 describes Messages compatibility. Healthy history therefore retains all
177 received proof. This investigation does not relax native Claude signatures or
178 explicit strict DeepSeek Anthropic contracts.
179
180 官方文档仍要求带工具请求完整回传历史 reasoning。正常历史继续保留真实内容;
181 本次没有放宽原生 Claude 签名要求,也没有修改显式严格 Anthropic 契约。
182
183 ### Defects found and fixed / 实测发现及修复
184
185 1. An EOF inside a JSON data line was a fatal decode error in Chat and Messages.
186 The shared stream scanner now distinguishes an unterminated JSON prefix from
187 a malformed complete event. The former uses bounded stream recovery; the
188 latter still fails. Partial tool calls never execute.
189 2. The actual Responses rejection names `reasoning_text`, which the replay-error
190 parser did not recognize. It now enters the existing bounded history repair,
191 preserving completed-tool facts and excluding invalid protocol history.
192 3. Request-only and byte-estimated usage could lose the unknown-usage flag.
193 Missing provider usage now remains unknown through estimation and aggregation;
194 request counting and known token telemetry remain available.
195 4. Strong history repair retained completed-tool names but discarded their
196 outputs, causing real models to repeat the tool or be unable to answer.
197 Recovery now includes bounded original model-visible results as escaped,
198 explicitly untrusted JSON; raw/local-only output stays excluded. Only a
199 repaired fault prefix changes; healthy requests are untouched.
200 5. The write-intent hook was attached to the outer execution context while
201 dispatch used the already prepared tool context. It is now installed on the
202 actual dispatch context after permission and preparation. A failed intent
203 checkpoint prevents the write from starting.
204
205 发现并修复五处遗漏:断流 JSON 误分类、Responses 回放错误漏识别、未知 usage
206 标记丢失、历史修复丢掉实际工具结果、写入持久化钩子未传入真正执行上下文。
207 只有故障修复视图新增有界工具结果;正常提示词、工具 schema、字段顺序与健康
208 历史不变。回归测试还验证了结果转义、RawContent 排除、持久化失败禁止写入。
209
210 Deterministic regressions: `internal/provider/stream_scanner_test.go`,
211 `internal/provider/reasoning_replay_error_test.go`,
212 `internal/agent/stream_fragment_recovery_test.go`, and
213 `internal/agent/cancel_test.go`. Live entrypoints are build-tagged `live` and
214 credential-gated. The old live missing-reasoning expectations were updated to
215 assert zero extra generation for compatible Chat/Responses turns. Independent
216 search tests now pin the supplied process credential explicitly instead of
217 silently skipping because an isolated home has no global credential file.
218
219 ### Measurements and qualification / 指标与验收结果
220
221 Worktree base: `1b4f9ae8324413d04ae272ceadb86ad49ffded2e`, plus local changes;
222 these results do not describe a published release. All paid calls were
223 sequential. Root package tests replace retry sleeps with a controllable test
224 sleeper, so the following live latency numbers exclude the production 2/4/8s
225 backoff. The proxy buffers a real upstream response before injecting a fault;
226 this is not a measurement of a naturally occurring server outage.
227
228 | Suite / 用例组 | Observed result / 结果 |
229 | --- | --- |
230 | Raw wire contract / 原始 HTTP 契约 | 74 requests: 66 HTTP 200 and 8 expected HTTP 400; 25,469 input and 1,860 output tokens reported across metered responses. Anthropic input includes cache-read/create tokens. The eight 400s have unknown usage. |
231 | Recovery matrix / 恢复矩阵 | 54 cases matched their expected outcomes: 122 client HTTP attempts, 116 official upstream requests, 46 actual marker-tool executions. Six 503s were local injection without upstream calls. |
232 | Recoverable faults / 可恢复故障 | 20/20 continued automatically, zero manual continuation, one extra attempt per fault. This includes six stream cuts, six temporary 503s, six actual server replay rejections and two single missing-thinking strict turns. |
233 | Recovery latency / 恢复耗时 | Whole recoverable scenarios: median 2.949s, range 0.994–6.591s; this includes normal tool/final requests, not just the recovery request, and excludes production retry sleeps. |
234 | Protective stops / 保护性停止 | Six cancellations executed no tool; two persistent strict Anthropic missing-thinking cases stopped after two requests and executed no tool. These are expected stops, not successful automatic recoveries. |
235 | Compatible missing reasoning / 兼容缺失 reasoning | Eight Chat/Responses missing-once/persistent cases completed with two requests and one tool execution; no reasoning regeneration. |
236 | Matrix usage / 矩阵用量 | 41,161 input and 4,347 output tokens recorded, including local estimates for incomplete requests. Twenty-four cases retained unknown-usage metadata. Extra retry tokens were not independently metered; no exact monetary total is claimed. |
237 | Conversation continuity / 连续会话 | Six Flash/Pro × protocol combinations, six user turns each, one save/load each: 72 requests, 36 tool executions, zero retries, 97,336 input / 2,053 output tokens. Healthy history, tools and settings stayed byte-stable across requests and reload. |
238 | Cache / 缓存 | Continuity aggregate: 88,320 cached / 97,336 input tokens (90.74%). A separate large-tool-output Chat test preserved its bounded stable prefix; its last two turns hit 9,472/9,611 and 9,856/9,882 tokens. Local RawContent sentinel was absent from requests. |
239 | Write effect before result checkpoint / 已写入但结果未保存 | Three protocols passed: one durable intent and one actual disk write each. Reload produced an unknown-result placeholder, not a fabricated completed result; verification prevented a second disk write. |
240 | Independent search / 独立搜索 | Search returned eight structured sources; default Chat → independent Messages search → Chat final completed. Standalone search reported one request, 17,238 input / 939 output tokens. |
241
242 Additional earlier checks passed: 20 Responses Flash/Pro tool loops, Chat and
243 Responses reasoning-removal probes, official Messages tool/history/search
244 round-trips, and cancel-after-tool save/load continuation.
245
246 真实恢复矩阵的 54 个场景均达到各自预期;其中 20 个可恢复故障自动继续,8 个
247 取消或严格协议持续缺失场景按设计停止,不能将后者算成“自动恢复成功”。主矩阵
248 没有重复执行工具。统计使用固定合成任务,不能据此估算所有真实任务的恢复率。
249 重试等待在测试中被替换,因此这些时长不是生产环境故障的真实等待时间。
250
251 A subsequent repair-boundary check found that anchoring the overlay to the
252 already stripped history could omit a trailing removed tool pair. The boundary
253 now anchors to the original source history. Completed-result facts identify
254 originating user turns, avoiding treating old work as fulfillment of a new task.
255 The deterministic follow-up asserts that the next request retains the repaired
256 view and the actual completed output.
257
258 后续还修正了历史修复边界:定位到原始历史,而不是已经删除工具轮的结果视图。
259 这样下一轮不会立即重新带回刚删除的错误协议历史;结果标记所属用户轮次,避免
260 把历史工作误当成新任务已经完成。
261
262 **Observed model variability:** the six real post-repair continuation checks
263 completed, but one Flash/Chat sample requested the read-only marker a second
264 time (five requests / two executions instead of four / one), failing the strict
265 no-repeat assertion. Three diagnostic repeats of that exact case then passed
266 without a duplicate; the added counters showed one execution before and after
267 continuation in those repeats. The first observation remains a limitation,
268 not erased by the passing repeats. No generic same-arguments deduplication was
269 added: a new request can legitimately require a fresh read. The verified
270 no-repeat disk-write result must not be generalized to every tool or model call.
271
272 补充的六组“修复后再继续”均完成任务,但其中一个 Flash/Chat 样本重复调用了
273 一次只读标记工具,未达到严格的零重复断言;随后三次定向复测没有复现。
274 该观察仍保留为边界,不因复测通过而抹去。没有按相同参数永久去重,因为用户
275 新请求可能需要重新读取。文件未重复落盘不等于所有模型都不会重复调用工具。
276
277 Final deterministic checks passed: root and Desktop module suites, targeted
278 race checks for stream/usage/replay/write-intent recovery and controller write
279 checkpoint reload, `make lint` (0 Go issues; existing repolint baseline unchanged),
280 and `git diff --check`. Live tests remain opt-in under the `live` build tag;
281 model-dependent no-repeat assertions can fail as described above.
282
283 最终根模块、Desktop 全量测试及恢复/取消/写入检查点的定向 race 检查通过,静态
284 检查与差异检查通过。真实测试受 `live` 标签保护;上述依赖模型选择的严格零重复
285 断言仍可能失败。
286
287 This is official API evidence on macOS plus controlled local faults. It does
288 not validate the reporter's complete Windows session, native Claude, arbitrary
289 custom gateways, unknown shell/MCP effects, native Desktop visual behavior, or
290 Pi/OpenCode against the same live account. Issue #9808 cannot be declared solved
291 for every deployment from these samples; no issue closure, push or release was
292 performed.
293
294 本次覆盖 macOS 下官方 API 及本地可控故障,不等同于反馈者完整 Windows 会话、
295 任意中转、原生 Claude、未知 Shell/MCP 副作用或原生界面的全面实测。没有关闭
296 issue、提交、推送或发布,不能据此宣称 #9808 的所有部署场景均已解决。
297
297 lines MARKDOWN