X (formerly Twitter) ·
Repo: github.com/badlogic/lemmy… I don't consider most of this a trade secret, as the "binary" is just a semi-obfuscated Node app. All prompts are contained as plain string literals. Same for the tool schemas, etc. But what's not in there are the tool inputs and outputs.
Let's have a look at the WebSearch and Fetch tools. I asked Claude Code (CC) to research the Anthropic TypeScript SDK. It uses the WebSearch tool, passing a query to it. It then inspects the results and follows links. Cool, right? Well.
The WebSearch tool doesn't receive raw search results, but output from another LLM that summarizes the raw search results. Good way to have some prompt injection protection, I guess. The summary includes links to everything, including the api.mdf file.
Obviously, Claude won't learn the intricacies from the search summary. So it dutifully fires of a bunch of requests to the Fetch tool, pulling in more information. Like the api.md Turns out, that too goes through an LLM and won't pull in the raw docs. Whoops.
Funnily enough, the Fetch tool description includes an IMPORTANT note for CC, to use any other available MCP server to perform fetches, to lift restrictions, bypassing any prompt injection CC's own Fetch tool provides. Brilliant.
TL;DR: you likely want some dedicated "How does this API work, give me its types, etc." MCP server, to pull in API info into CC's context window. I haven't found a good one yet. 👀
@marc_krenn
The often failing Update tool is also fun. Wonder why it fails? It pretty much always fails due to indentation. Sometimes it notices (like here), but takes 10 round trips to grep, egrep, rg, and sed before figuring it out. The correct number of \t is 7.
It outputs 8 tabs for old_string. It's usually always an additional tab. Before it updates the file, it reads it (partially). The Read tool adds a prefix "\t${lineNumber} to each line it returns. And I'm 99% certain that tab in the prefix fucks it up.
I should rewrite the result of a Read command to not include that tab in the prefix and see if this decreases the insane amount of Update retries...
Passing --include-all-requests to claude-trace will make it log EVERYTHING CC requests via a call to fetch() or Https.request(). Turns out, CC does a lot of requests for one of its cosmetic features: The cute little words it shows while its thinking.
Turns out CC is firing off a request to Claude Haiku 3.5 for every. single. token. you. type. in the prompt text area. I suppose Anthropic found a free energy device to power all those useless requests to come up with words. "NEVER use a destructive word such as Terminating"
Haiku is also part of their "Make Bash Safe Again" protection layer. Before running a Bash command, Haiku is asked to determine malicious command injection. I'm not sure I'd trust an LLM with this task. YOLO I guess.
Nevermind, claude --help is lying. --dangerously-skip-permissions works outside of Docker and with an internet connection. Respect my securaiiitaayyyy
