Language servers expose semantic and diagnostic information about code for low-latency iteration during development. They provide refactor tools that are ‘language aware’: symbol renaming, ‘move to file’, ‘organise imports’. They also provide discovery tools to walk call trees and find usage sites of functions.
function build_tools( for: “humans” | “llms” | “llms_for_humans”) { /* … */ }build_tools()These are useful tools that aren’t otherwise available - while a compilation will perform a type-check, it won’t provide inlay hints or the ability to rename a function across a codebase.
There’s also a specificity to LSP tooling, you can request hover information for a single symbol, or get the errors in a single file. A compile will provide global information unrelated to your local focus. This locality allows much faster time-to-information compared to full or even incremental compilation. It is a sharper tool.
Coding agents perform better with access to rich information, automatic guard rails, and tools to debug problems efficiently. They can work faster when a tool returns information in milliseconds instead of seconds. They are cheaper when tools emit fewer tokens, especially when they’re the more relevant tokens.
Coding agents should have access to these LSP systems!
Language servers have APIs designed for cursor based (mouse or keyboard) positioning within files, because humans tend to use mice and keyboards. LLMs don’t have cursors, and are pretty poor at line and character based positioning.
As always, everything is a data representation problem and there’s a clear mismatch here.
What’s the primary modality of the LLM? Text! Lets design tools that match their modality.
Strawberry
LSP integrations tend to use line and character indexes to localise text in their tool calls:
{
"operation": "hover",
"filePath": "packages/dataflow/test/operators/multiply-schemas.test.ts",
"line": 4,
"character": 6
}While the modality is text, the input stream into the LLM is a sequence of tokens.
3504 1134 19772
Tokens in the o200k tokeniser can be up to 128 characters long. For the model to accurately keep track of character and line positioning, it needs to accumulate that information over the entire file, which it likely read in chunks, in between reasoning and edits via tool calls. There’s a substantial amount working against the model here.
Guess the line and character index of the second e in needle in this block below. As a hint, this is the whole file read.
200006 44580 38155 316 28 173781 200005 12606 815 200008 2706 2123 416 405 220 3887 2659 706 976 29082 6446 48704 1072 290 392 needle 1 306 290 8312 15145 134541 4066 943 200007
Or what about this one?
16198 176161 256 needle
Answers below:
No cheating!
If you guessed line 1, column 47 for the first, and line 2, column 4 for the second, you were right! Well done!
It’s pretty hard! Sure, these models are Turing complete and can learn anything if they believe in themselves, but is there a better way?
apply_patch
OpenAI’s Codex uses a text based format for its apply_patch tool. It supports concise file moves, additions, updates and deletions. It has a Lark grammar to enforce structural correctness of the calls.
Codex has been extensively post-trained to understand and output this format. An apply_patch call looks like this:
*** Begin Patch
*** Add File: hello.txt
+Hello world
*** Update File: src/app.rs
*** Move to: src/main.rs
@@
fn greet() {
- println!("Hi");
+ println!("Hello world");
}
@@ fn farewell() {
- println!("Goodbye");
+ println!("Fare Thee Well");
}
*** Delete File: obsolete.txt
*** End PatchWe can take advantage of their hard work and design our own ‘language server’ query format that’s idiomatic to the modality of LLMs while aligning with OpenAI’s post-training.
Queries and Diagnostics
Lets begin with the ability to make queries via hover and inlay :
? <line>for an inlay hints request for a line^ <symbol>allows for hovering over every symbol in a line, or a list of them.
Here’s a small TypeScript file with some types:
Know that it is within my power to have fancy hovering in the browser - but this has been taken away such that we may continue to empathise with the machine.
const vS = schemaNumber({ unit: volts })
const aS = schemaNumber({ unit: amperes })
type V = typeof vS['infer']
type A = typeof aS['infer']
const eq = math({ v: vS, a: aS }, `v * a`)
const res = eq.evaluate({ v: 230, a: 5 })
const wrong: string = 42This request will fetch the hover information for the V, A types and the res declaration. It will also fetch inlay hints and JSDoc content for the math function call.
*** Begin Language Server Request
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
@@ type V = typeof vS['infer']
^ V
@@ type A = typeof aS['infer']
^ A
? const eq = math({ v: vS, a: aS }, `v * a`)
^ math[docs]
@@ const res = eq.evaluate({ v: 230, a: 5 })
^ res
*** End Language Server RequestThe server responds with:
*** Begin Language Server Response
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
@@ type V = typeof vS['infer']
^ V
/*<hover:V>```typescript
type V = number & UnitOfQuantity<"volts", "Voltage">
```</hover:V>*/
@@ type A = typeof aS['infer']
^ A
/*<hover:A>```typescript
type A = number & UnitOfQuantity<"amperes", "Electric Current">
```</hover:A>*/
? const eq = math(/*<inlay>inputs:</inlay>*/{ v: vS, a: aS },/*<inlay>equation:</inlay>*/`v * a`)
^ math[docs]
/*<docs:math>
The `math` function takes an object of input variable schemas and a math string. A handle is returned with properties:
- `schema`: Containing the calculated unit.
- `evaluate`: Takes an object of the variable's values, returns the answer.
</docs:math>*/
@@ const res = eq.evaluate({ v: 230, a: 5 })
^ res
/*<hover:res>```typescript
const res: number & UnitOfQuantity<"watts", "Power">
```</hover:res>*/
@@ const wrong: string = 42
/*<diagnostics:error:wrong:ts>
Type 'number' is not assignable to type 'string'.ts(2322)
</diagnostics:error:wrong:ts>*/
*** End Language Server ResponseWhile this looks gross to a human, this is great for the LLM!
The xml-like tags unambiguously bind each hint to its source. The markdown format is well understood by LLMs, and all additional information is colocated with the textual source. The LLM is able to request the minimal information it needs, and we can inject errors and other diagnostic information automatically. The sorted context @@ lines concept works both ways, it makes it unambiguous for the parser to locate lines, and makes it unambiguous to the LLM when we inject diagnostics (like the const wrong: string = 42 line).
This ability to push faults into the model’s context is currently difficult in the first-party harnesses.
Commands
This is the spiritual successor to my copy paste adventures in apply_patch. I still think apply_patch should support copy paste, since not all of text is code. However, code should be moved and renamed with a language server.
! <symbol> RENAME_TO <name>renames a symbol on the previous line, updating every usage in the workspace.[@ <line>and@] <line>select the lines between them, inclusive.(@and@)are the exclusive variants.! MOVE_TO_FILE <path>moves the selection to a file, adding the exports and imports.
It looks like this:
*** Begin Language Server Request
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
[@ const vS = schemaNumber({ unit: volts })
@@ const eq = math({ v: vS, a: aS }, `v * a`)
! eq RENAME_TO powerEquation
@@ const res = eq.evaluate({ v: 230, a: 5 })
! res RENAME_TO power
@) const wrong: string = 42
! MOVE_TO_FILE packages/dataflow/test/fixtures/electrical-power.ts
*** End Language Server RequestAnd returns this:
*** Begin Language Server Response
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
@@ const wrong: string = 42
/*<diagnostics:error:wrong:ts>
Type 'number' is not assignable to type 'string'.ts(2322)
</diagnostics:error:wrong:ts>*/
*** File: packages/dataflow/test/fixtures/electrical-power.ts
@@ const vS = schemaNumber({ unit: volts })
@@ const powerEquation = math({ v: vS, a: aS }, `v * a`)
@@ const power = powerEquation.evaluate({ v: 230, a: 5 })
*** End Language Server ResponseSuccess and Failure
It’s almost comical to do the inverse of the experiment and ask you to ‘type out needle ’ but the triviality is kind of the point.
16198 176161 256 needle
Answer below:
No cheating!
A second needle !
Of course it’s in the last place you look, who keeps looking after they’ve found it?
We should make it easy for the models to do the right thing.
I’ve been using this language server integration for a while now, it’s working great! It’s substantially more token efficient to do language-server enabled renames and it’s much faster to not have to wait for a compile just to do a type check.
It doesn’t work every time. GPT5.5 can do some silly things sometimes. It is funny when the model gets into a loop of not being able to uniquely identify some lines for hover information, after a few tries it will apply_patch a random comment to ‘provide a unique anchor’ (and have no trouble anchoring the comment). So while the format training is helpful, it’s not perfect. Ultimately the models need to be post-trained to use tools effectively…
Closing Thoughts
Humans interact with their IDE via text selection, range edits, multiple cursors, tab based indentation, selection/file/project local find and replace, and many other tools. We should provide agents with the same capabilities, tuned for their modality.
*** End Language Server ResponseIf you would like further reading on this topic, I recommend:
- The apply_patch tool is missing something - The previous post in this series.