| 1 | Feature: Tool call lifecycle |
| 2 | Scenario: Happy path lists the current directory through a tool |
| 3 | # This executable slice asserts the public exec stream and mocked LLM border. |
| 4 | # The real Unix PTY screen slice in qa_pty.rs asserts the matching Work, |
| 5 | # statusline, idle BlueWhale, live tool-card, and settled-transcript border. |
| 6 | Given an offline CodeWhale workspace containing: |
| 7 | | path | kind | |
| 8 | | README.md | file | |
| 9 | | notes.txt | file | |
| 10 | | src | folder | |
| 11 | And the mocked LLM will request the "File" tool with: |
| 12 | | action | path | |
| 13 | | list | . | |
| 14 | And the mocked LLM will answer after the tool result: |
| 15 | | content | |
| 16 | | The directory contains README.md, notes.txt, and src/. | |
| 17 | When the user asks "list the current directory" |
| 18 | Then CodeWhale should send the user request to the mocked LLM |
| 19 | And the public tool lifecycle should show a running tool: |
| 20 | | status | marker | tool | action | input | |
| 21 | | running | [~] | File | list | . | |
| 22 | And the public tool result should return directory entries: |
| 23 | | entry | kind | |
| 24 | | README.md | file | |
| 25 | | notes.txt | file | |
| 26 | | src | folder | |
| 27 | And CodeWhale should send the tool result back to the mocked LLM |
| 28 | And the public tool lifecycle should show a completed tool: |
| 29 | | status | marker | tool | action | input | |
| 30 | | completed | ✓ | File | list | . | |
| 31 | And the public output should include "The directory contains README.md, notes.txt, and src/." |
| 32 | |
| 33 | Scenario: Unknown tool returns an error result |
| 34 | Given an offline CodeWhale workspace containing: |
| 35 | | path | kind | |
| 36 | | README.md | file | |
| 37 | And the mocked LLM will request the "missing_tool" tool with: |
| 38 | | path | |
| 39 | | . | |
| 40 | And the mocked LLM will answer after the tool result: |
| 41 | | content | |
| 42 | | I could not run the requested missing tool. | |
| 43 | When the user asks "try a missing tool" |
| 44 | Then CodeWhale should send the user request to the mocked LLM |
| 45 | And the public tool lifecycle should show a running tool: |
| 46 | | status | marker | tool | input | |
| 47 | | running | [~] | missing_tool | . | |
| 48 | And the public tool result should report an error for "missing_tool" |
| 49 | And CodeWhale should send the tool error back to the mocked LLM |
| 50 | And the public tool lifecycle should show a failed tool: |
| 51 | | status | marker | tool | input | |
| 52 | | error | [!] | missing_tool | . | |
| 53 | And the public output should include "I could not run the requested missing tool." |
| 54 | |
| 55 | Scenario: Malformed tool arguments return an error result |
| 56 | Given an offline CodeWhale workspace containing: |
| 57 | | path | kind | |
| 58 | | README.md | file | |
| 59 | And the mocked LLM will request the "File" tool with malformed arguments "{not-json" |
| 60 | And the mocked LLM will answer after the tool result: |
| 61 | | content | |
| 62 | | I could not parse the tool arguments. | |
| 63 | When the user asks "try malformed tool arguments" |
| 64 | Then CodeWhale should send the user request to the mocked LLM |
| 65 | And the public tool lifecycle should show a running tool with raw input for "File" |
| 66 | And the public tool result should report malformed arguments for "File" |
| 67 | And CodeWhale should send the malformed argument error back to the mocked LLM |
| 68 | And the public tool lifecycle should show a failed tool with raw input for "File" |
| 69 | And the public output should include "I could not parse the tool arguments." |
| 70 | |
| 71 | Scenario: A real tool error is returned to the follow-up request |
| 72 | Given an offline CodeWhale workspace containing: |
| 73 | | path | kind | |
| 74 | | README.md | file | |
| 75 | And the mocked LLM will request the "File" tool with: |
| 76 | | action | path | |
| 77 | | read | missing.txt | |
| 78 | And the mocked LLM will answer after the tool result: |
| 79 | | content | |
| 80 | | I could not read missing.txt because the file is absent. | |
| 81 | When the user asks "read the missing file" |
| 82 | Then CodeWhale should send the user request to the mocked LLM |
| 83 | And the public tool lifecycle should show a running tool: |
| 84 | | status | marker | tool | action | input | |
| 85 | | running | [~] | File | read | missing.txt | |
| 86 | And the public tool result should report a real error for "File" containing "missing.txt" |
| 87 | And CodeWhale should send the real tool error back to the mocked LLM |
| 88 | And the public tool lifecycle should show a failed tool: |
| 89 | | status | marker | tool | action | input | |
| 90 | | error | [!] | File | read | missing.txt | |
| 91 | And the public output should include "I could not read missing.txt because the file is absent." |
| 92 | |
| 93 | Scenario: An empty tool result is returned to the follow-up request |
| 94 | Given an offline CodeWhale workspace containing: |
| 95 | | path | kind | |
| 96 | | empty | folder | |
| 97 | And the mocked LLM will request the "File" tool with: |
| 98 | | action | path | |
| 99 | | list | empty | |
| 100 | And the mocked LLM will answer after the tool result: |
| 101 | | content | |
| 102 | | The directory is currently empty. | |
| 103 | When the user asks "list the empty directory" |
| 104 | Then CodeWhale should send the user request to the mocked LLM |
| 105 | And the public tool lifecycle should show a running tool: |
| 106 | | status | marker | tool | action | input | |
| 107 | | running | [~] | File | list | empty | |
| 108 | And the public tool result should be an empty list |
| 109 | And CodeWhale should send the empty tool result back to the mocked LLM |
| 110 | And the public tool lifecycle should show a completed tool: |
| 111 | | status | marker | tool | action | input | |
| 112 | | completed | ✓ | File | list | empty | |
| 113 | And the public output should include "The directory is currently empty." |
| 114 | |
| 115 | Scenario: A follow-up answer missing the expected summary is detected |
| 116 | Given an offline CodeWhale workspace containing: |
| 117 | | path | kind | |
| 118 | | README.md | file | |
| 119 | And the mocked LLM will request the "File" tool with: |
| 120 | | action | path | |
| 121 | | list | . | |
| 122 | And the mocked LLM will answer after the tool result: |
| 123 | | content | |
| 124 | | I inspected the workspace. | |
| 125 | When the user asks "summarize the current directory" |
| 126 | Then CodeWhale should send the user request to the mocked LLM |
| 127 | And the public tool lifecycle should show a running tool: |
| 128 | | status | marker | tool | action | input | |
| 129 | | running | [~] | File | list | . | |
| 130 | And the public tool result should return directory entries: |
| 131 | | entry | kind | |
| 132 | | README.md | file | |
| 133 | And CodeWhale should send the tool result back to the mocked LLM |
| 134 | And the public tool lifecycle should show a completed tool: |
| 135 | | status | marker | tool | action | input | |
| 136 | | completed | ✓ | File | list | . | |
| 137 | And the public output should include "I inspected the workspace." |
| 138 | But acceptance should report the missing expected summary "The directory contains README.md." |
| 139 |