Fun feature I worked on for Holon. Knowledge packs: pre-compute KV cache for static docs like code guides, books etc and attach it to any new session. Compute once, reuse anywhere. Skip prefill entirely.
00:00
X (formerly Twitter) ·
Fun feature I worked on for Holon. Knowledge packs: pre-compute KV cache for static docs like code guides, books etc and attach it to any new session. Compute once, reuse anywhere. Skip prefill entirely.
00:00