Firecrawl’s government index

· Lexifina ·

6 min read Original article ↗

On 22 October Firecrawl released a free search index of about 30 million US government pages.[1] It covers federal and state statutes, regulations, court opinions and agency guidance, and, less usually, county and city codes. An agent sends a question and gets back ranked passages, each linked to its source.

The code points at a private service, so we can only infer functionality from the results. Accordingly, federal opinions come mostly from GovInfo’s collection of court opinions. State appellate cases come mostly from CourtListener, whose court identifiers the court filter asks for by name. Local codes come from the three large code publishers (American Legal Publishing, General Code’s eCode360, and Municode), and the rest from a wide crawl of .gov sites.

Passages carry their trail of headings, which suggests documents are split along their own structure, and the citation filter accepts the same formats as Free Law Project’s open-source citation parser. Almost every ingredient is public data or open software.

Circumventing access controls

Reaching legislation is increasingly difficult. Several state legislatures refuse connections from outside the US, and California’s code sits behind a bot check. From a server in the UK, Illinois and Ohio code sites did not answer at all.

Firecrawl’s crawler presents itself as Chrome 155 on Linux, with Chrome’s encryption fingerprint, and adds a header claiming the visit came from a Google search. By default it leaves through addresses registered to Chiron Software LLC, a small Virginia Beach company, which Cox Communications announces on the internet, so lookup services list them as Cox broadband.

The same address blocks were announced by Buddy Software LLC in 2022 and by Windstream alongside Cox in 2024 and 2025, the usual pattern for leased addresses sold as ‘ISP proxies’. They are almost certainly servers, not homes. In stealth mode it leaves through T-Mobile and Verizon mobile addresses. That is how it reads the California code when an ordinary server is turned away. Firecrawl’s own response says it fetched the California, Illinois and Ohio pages through its default (‘basic’) proxy.

Statutes carry no copyright, and no one is trying to keep them private. The sites that publish them were built for people with browsers, and their defences treat a research tool and a scraper alike. Getting past them leans on the same residential and mobile proxy networks that are now the subject of litigation and takedowns. Google sued SerpApi in December 2025 for working around its bot defences,[3] and in January it disrupted IPIDEA, a residential proxy network built from apps that quietly turned phones into exit points.[4] A legal research product that depends on that layer takes on some of its risk, even when every page it reads is public.

The lasting fix is for legislatures to just publish their codes in bulk, which is how GovInfo and CourtListener made federal law easy to index in the first place.

Finding and citing

Results are pages ranked by relevance, with no date the law was in force and no version history. Firecrawl’s documentation warns that matches can include historical versions, bill text and third-party copies.[2] In my tests a Georgia statute came back from a commercial annotated copy rather than the legislature, and many local code hits were empty contents pages.

For an agent looking for leads that is acceptable. A lawyer who needs the law in force on a given date still has to open each hit and check it against the official text, as the launch post itself says.[1]

So, why bother with an offline index? The bull case is structure. Once the pages are held locally they can be joined into a graph of what cites what, which authority outranks which, and which version was in force when.

Similarly, Legora is building what it calls “a full ontology of law” with its own citator,[5] and on 19 October Harvey launched Harvey Graph, a citation graph of US case law that it says raised pass rates on its own benchmark by up to 30% and cut task cost by up to half.[6] Our own experiments point the same way: on firm-wide questions an agent with a graph used a fraction of the tokens and found answers that an agent reading files missed.[7],[8]

But the outside evidence is narrower than the launches suggest. Graph retrieval wins clearly on questions that chain several facts together and roughly ties plain retrieval on single lookups,[9] agents that simply grep raw text often beat embedding indexes,[10],[11] .

For “what does this section say today”, a live fetch of the official page is still the better answer. The value in an index of free law lies in what is built on top of it, namely the citator, the point-in-time versions and the proof that a text is the official one.[12],[13]

The most useful feature is the filter that returns every page citing a given provision. Asking for pages that cite 42 U.S.C. § 1983, or California Civil Code § 1947.12, gives a cited-by view across statutes, cases and agency pages that few free tools offer.

Cost

Search is free because it brings developers to Firecrawl’s paid scraping. The hosted search tool also asks the calling agent to state the user’s underlying goal with every query, so unless a firm has a no-retention agreement, a sentence about a client’s matter leaves with each search. I would use it for public questions and keep the facts of a matter out of the query.

On balance it is a useful free way to find US law, assembled mostly from public data and open-source parts, with a capable crawler doing the difficult work. Its weak points are the ones legal research has always had: whether the text is the official one, whether it was in force on the date that matters, and who else sees the question.

References

  1. Firecrawl, ‘Government & Legal Index’ (22 October 2026) <www.firecrawl.dev/blog/government-legal-index>
  2. Firecrawl, ‘Government Index’, documentation (accessed 24 October 2026) <docs.firecrawl.dev/features/gov>
  3. Rose Esfandiari, ‘Google Sues SerpApi for “Parasitic” Scraping and Circumvention of Protection Measures’ (IPWatchdog, 26 December 2025) <ipwatchdog.com/2025/12/26/google-sues-serpapi-parasitic-scraping-circumvention-protection-measures/>
  4. Ken Underhill, ‘Google Disrupts Major Residential Proxy Network IPIDEA’ (eSecurity Planet, 29 January 2026) <www.esecurityplanet.com/threats/google-disrupts-major-residential-proxy-network-ipidea/>
  5. Legora, ‘Legora rebuilds legal research with comprehensive data, a full ontology of law, and an AI-native citator’ (14 September 2026) <legora.com/newsroom/legora-rebuilds-legal-research-with-comprehensive-data-a-full-ontology-of-law-and-an-ai-native-citator>
  6. Harvey, ‘Improving agent accuracy with Harvey Graph’ (19 October 2026) <www.harvey.ai/blog/improving-agent-accuracy-with-harvey-graph>
  7. Alan Yahya, ‘Building full-firm knowledge graphs to dramatically speed up agents’ (Lexifina, 20 October 2026) <lexifina.com/blog/building-full-firm-knowledge-graphs-to-dramatically-speed-up-agents>
  8. Alan Yahya, ‘Knowledge graphs could help orgs respond to new laws’ (Lexifina, 22 October 2026) <lexifina.com/blog/knowledge-graphs-update-firms-for-new-laws>
  9. Dongzhe Fan, Zheyi Xue, Siyuan Liu and Qiaoyu Tan, ‘Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems’ (arXiv:2604.09666, 1 April 2026) <arxiv.org/abs/2604.09666>
  10. Zhuofeng Li and others, ‘Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction’ (arXiv:2605.05242, 3 May 2026) <arxiv.org/abs/2605.05242>
  11. Sahil Sen, Akhil Kasturi, Elias Lumer, Anmol Gulati and Vamse Kumar Subbiah, ‘Is Grep All You Need? How Agent Harnesses Reshape Agentic Search’ (arXiv:2605.15184, 14 May 2026) <arxiv.org/abs/2605.15184>
  12. Bob Ambrogi, ‘“No More Than an Ordinary Copyright Case”: 3rd Circuit Unseals Its Opinion In Thomson Reuters Case Against ROSS’ (LawSites, 12 October 2026) <www.lawnext.com/2026/10/no-more-than-an-ordinary-copyright-case-3rd-circuit-unseals-its-opinion-in-thomson-reuters-case-against-ross.html>
  13. Bob Ambrogi, ‘Alexi Fires Back at Fastcase Lawsuit with Counterclaims Alleging Anticompetitive Conduct Following Clio’s $1B Acquisition’ (LawSites, 18 January 2026) <www.lawnext.com/2026/01/alexi-fires-back-at-fastcase-lawsuit-with-counterclaims-alleging-anticompetitive-conduct-following-clios-1b-acquisition.html>