Ask HN: How do you send large sets of data to an LLM
So, I am hitting limits with the amount of data I am sending to Claude via the API.
I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.
What are some good design patterns when sending large sets of data to an LLM?
I ideally need to send all the data together at it gives context to the overall prompt. Some approaches we’ve used: - Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context. - RAG where only relevant parts are included in the context. - Use an LLM with a larger context window (like Gemini pro) Yeah, I think I need to use Gemini Question, how if possible could you query rows in a table? Surely the better approach would be to have some sort of connection to table rows?