I’m Afraid of Spiders. So I Made AI Look at 2,000 of Them.

· Labqoat ·

6 min read Original article ↗

I’m afraid of spiders. A lot. 🕷️

My identification system currently consists of “the one with long legs” and “the short but big one.” There’s also “where the fuck did it go,” but that’s more of an emergency than a classification.

So I got curious: how well could AI actually identify them?

I tested nine models on the same 2,000 spider photos to see how often they got the species right.

(Looking at this many spiders was not a comfortable experience.)

This post was heavily inspired by Piotr Migdał’s post on identifying mushrooms with AI.

First, approximately ten seconds of biology

A family is a broad group of related spiders. A genus is a smaller group inside it, and a species is the specific kind of spider.

For example: Araneidae → Araneus → Araneus diadematus, the European garden spider. In that last name, Araneus is the genus.

That’s enough biology for now.

How the benchmark works

Each model got the same 2,000 photos, covering 671 species and subspecies, and a list of 20 possible names for each photo, including the correct answer.

The task was to identify the spider’s species from the photo and return the matching name from the list.

I gave each model the same possible answers so I could compare how well they distinguished the species, with fewer ambiguities in scoring. That makes this a multiple-choice identification test, and the choices themselves can help.

I built the dataset from research-grade iNaturalist observations, using the community’s species identifications as the expected answers. The species list comes from a Polish spider checklist, but the photos were taken worldwide. After filtering out label mismatches and unsuitable images, 2,183 photos remained, and I used the same 2,000-photo subset for every model.

Let’s look at a few examples

I’d call any of these “spider,” but… 😅

An orange-and-black spider facing the camera on a pale plant stem.
© Kostas Zontanos · CC BY 4.0. Resized.

According to iNaturalist, it’s a Red-bellied Jumping Spider (Philaeus chrysops).

What did the models say?

9 of 9 picked the expected species.

Model answers for Philaeus chrysops
ModelAnswerResult
Gemini 3.8 FlashPhilaeus chrysopsCorrectCorrect
GPT-6 AstraPhilaeus chrysopsCorrectCorrect
Claude Fable 5.1Philaeus chrysopsCorrectCorrect
Muse Spark 1.3*Philaeus chrysopsCorrectCorrect
GLM 5.3 FlashPhilaeus chrysopsCorrectCorrect
GPT-5.6 SolPhilaeus chrysopsCorrectCorrect
DeepSeek V4.1 FlashPhilaeus chrysopsCorrectCorrect
GPT-5.6 TerraPhilaeus chrysopsCorrectCorrect
GPT-5.6 LunaPhilaeus chrysopsCorrectCorrect

Everyone got this one right.

See the 20 choices the models received
  1. 1.Nigma walckenaeri
  2. 2.Zelotes aeneus
  3. 3.Attulus inexpectus
  4. 4.Philaeus chrysops
  5. 5.Gnaphosa lucifuga
  6. 6.Savignia frontata
  7. 7.Sittisax saxicola
  8. 8.Kishidaia conspicua
  9. 9.Evarcha laetabunda
  10. 10.Metellina mengei
  11. 11.Attulus rupicola
  12. 12.Araniella displicata
  13. 13.Leptorchestes berolinensis
  14. 14.Neon valentulus
  15. 15.Attulus terebratus
  16. 16.Marpissa pomatia
  17. 17.Cybaeus tetricus
  18. 18.Nuctenea umbratica
  19. 19.Euophrys frontalis
  20. 20.Larinioides ixobolus

* Muse Spark 1.3 uses the Contributor tier.

Selected examples; overall scores use all 2,000 photos.

Okay, who knew the spiders?

Gemini 3.8 Flash got 997 out of 2,000 correct: 49.85%, the highest score in these runs.

GPT-6 Astra followed at 47.50%, then Claude Fable 5.1 at 43.10% and Muse Spark 1.3 at 39.10%. The top two were separated by 47 photos. I’d read that as a result on this set of spiders, not a universal ranking of how much the models know about them.

Exact-species accuracy on 2,000 photos: Gemini 3.8 Flash 49.85%; GPT-6 Astra 47.50%; Claude Fable 5.1 43.10%; Muse Spark 1.3 39.10%; GLM 5.3 Flash 36.75%; GPT-5.6 Sol 33.65%; DeepSeek V4.1 Flash 26.40%; GPT-5.6 Terra 22.80%; GPT-5.6 Luna 21.50%.Exact-species accuracy on 2,000 photos: Gemini 3.8 Flash 49.85%; GPT-6 Astra 47.50%; Claude Fable 5.1 43.10%; Muse Spark 1.3 39.10%; GLM 5.3 Flash 36.75%; GPT-5.6 Sol 33.65%; DeepSeek V4.1 Flash 26.40%; GPT-5.6 Terra 22.80%; GPT-5.6 Luna 21.50%.

Same 2,000 photos for every model. Wrong and failed answers stay in the denominator.

* Muse Spark 1.3 uses the discounted Contributor tier, which allows provider training/data use.

See the exact numbers
Exact-species accuracy. Each run contains 2,000 photos.
ModelCorrectAccuracy
Gemini 3.8 Flash99749.85%
GPT-6 Astra95047.50%
Claude Fable 5.186243.10%
Muse Spark 1.3*78239.10%
GLM 5.3 Flash73536.75%
GPT-5.6 Sol67333.65%
DeepSeek V4.1 Flash52826.40%
GPT-5.6 Terra45622.80%
GPT-5.6 Luna43021.50%

Code and benchmark results on GitHub

How wrong is wrong?

Gemini 3.8 Flash missed the exact species in 1,003 photos. But in 871 of those cases, it named another spider from the same family. That’s almost 87% of its misses. In 276 cases, it even got the genus right.

GPT-6 Astra and Claude Fable 5.1 showed a similar pattern. Their exact-species scores were 47.50% and 43.10%, but their answers belonged to the correct family 92.00% and 91.45% of the time. Most of their mistakes happened between species within the same family.

Muse Spark 1.3 identified more exact species than GLM 5.3 Flash. GLM’s answers, however, belonged to the correct family slightly more often: 89.40% versus 88.70%.

That makes the errors more interesting than a simple correct-or-wrong score suggests. There’s a substantial difference between getting the broad group right and identifying the particular species.

  • Correct species
  • Same genus only
  • Same family only
  • Wrong family
  • Failed response
Each model’s 2,000 responses split into five non-overlapping groups: correct species, wrong species in the same genus, wrong genus in the same family, wrong family, and failed response. The exact counts are in the table below.Each model’s 2,000 responses split into five non-overlapping groups: correct species, wrong species in the same genus, wrong genus in the same family, wrong family, and failed response. The exact counts are in the table below.

Each answer appears in one segment. “Only” means the more specific identification was wrong. Failed responses include invalid names, refusals, empty answers, truncations, and request errors.

* Muse Spark 1.3 uses the discounted Contributor tier, which allows provider training/data use.

See the exact numbers
How close were the answers?. Each run contains 2,000 photos.
ModelCorrect speciesSame genus onlySame family onlyWrong familyFailed response
Gemini 3.8 Flash9972765951284
GPT-6 Astra9503125781564
Claude Fable 5.18623476201692
Muse Spark 1.3*7823066862260
GLM 5.3 Flash7353227312093
GPT-5.6 Sol6732646843781
DeepSeek V4.1 Flash52832871034292
GPT-5.6 Terra4562727185540
GPT-5.6 Luna43024867160645

Counts, not cumulative percentages. Each row adds up to 2,000.

Code and benchmark results on GitHub

Accuracy versus cost

Muse Spark 1.3 was the cheapest model here and still finished fourth, ahead of five more expensive models. At its Contributor pricing, the entire run cost $0.93 for 782 correct identifications.

Moving to Gemini 3.8 Flash brought that total to 997. That’s 215 more correct answers for another $26.18, an improvement of 10.75 percentage points at roughly 29 times the cost. 💸

Spending beyond that didn’t improve the results. Claude Fable 5.1 cost about twice as much as Gemini, and GPT-6 Astra cost 3.6 times as much. Both scored lower. Gemini delivered the highest accuracy here; Muse offered a cheaper trade-off if identifying fewer species was acceptable.

Accuracy plotted against estimated cost for 2,000 photos, using a logarithmic dollar axis. Gemini 3.8 Flash: 49.85% at $27.11; GPT-6 Astra: 47.50% at $97.53; Claude Fable 5.1: 43.10% at $54.19; Muse Spark 1.3: 39.10% at $0.93; GLM 5.3 Flash: 36.75% at $1.46; GPT-5.6 Sol: 33.65% at $42.89; DeepSeek V4.1 Flash: 26.40% at $8.52; GPT-5.6 Terra: 22.80% at $20.53; GPT-5.6 Luna: 21.50% at $4.99.Accuracy plotted against estimated cost for 2,000 photos, using a logarithmic dollar axis. Gemini 3.8 Flash: 49.85% at $27.11; GPT-6 Astra: 47.50% at $97.53; Claude Fable 5.1: 43.10% at $54.19; Muse Spark 1.3: 39.10% at $0.93; GLM 5.3 Flash: 36.75% at $1.46; GPT-5.6 Sol: 33.65% at $42.89; DeepSeek V4.1 Flash: 26.40% at $8.52; GPT-5.6 Terra: 22.80% at $20.53; GPT-5.6 Luna: 21.50% at $4.99.

Estimated totals for 2,000 photos. The dollar axis is logarithmic: spacing represents cost ratios.

* Muse Spark 1.3 uses the discounted Contributor tier, which allows provider training/data use.

See the exact numbers
Accuracy versus estimated cost. Each run contains 2,000 photos.
ModelCorrectAccuracyCost
Gemini 3.8 Flash99749.85%$27.11
GPT-6 Astra95047.50%$97.53
Claude Fable 5.186243.10%$54.19
Muse Spark 1.3*78239.10%$0.93
GLM 5.3 Flash73536.75%$1.46
GPT-5.6 Sol67333.65%$42.89
DeepSeek V4.1 Flash52826.40%$8.52
GPT-5.6 Terra45622.80%$20.53
GPT-5.6 Luna43021.50%$4.99

Code and benchmark results on GitHub

Is getting half wrong actually bad?

Getting almost half right is respectable when the task is telling similar species apart. The gap between recognising a family and naming the exact species is where this gets difficult.

Try telling apart. For Araniella opisthographa, NatureSpot’s identification guidance calls for examination at high magnification. That’s the sort of detail a photo can leave out.

The code and full benchmark results are on GitHub.

PS: If you actually know spiders, I’d love to hear what you think on twitter. How obvious are these mistakes to someone who knows what they’re looking at?

P.P.S. Thankfully, I have a cat in charge of spider security at home.