Can AI Identify a Fish From a Photo?

What the published numbers actually measure

Accuracy figures for automated fish identification are real, and they describe far narrower tests than the headline suggests.

One study classifying fish in underwater video reported 94.3% accuracy across 16 species. Another, photographing 9 species at a fish counter under near-constant lighting, reported 95.29%. A third, working from a 23-species dataset of over 27,000 images, reported 99.93% classification accuracy.

Read those numbers with their conditions attached and they say something more modest. Nine species. Sixteen. Twenty-three. There are more than 30,000 fish species in the world, and the third dataset was heavily skewed — around 44% of its images were a single species, and the top fifteen species accounted for 97% of everything in it. A model can score very well on a test like that while having seen almost nothing of the tail.

This is why we publish no accuracy percentage for our own photo identification feature. A number would have to come from somewhere, and any number honest enough to be useful would have to state which species, which photos and which conditions it was measured on.

Why real photographs are harder than test sets

Performance drops as soon as the model has to consider that the answer might not be on its list.

The underwater-video study makes this unusually concrete. Its 94.3% across 16 species fell to 89.0% when an “other species” class was added — the same model, the same footage, one extra option representing everything it had not been trained on. Real fishing is permanently in that second condition. Whatever you are holding might be something the model has never seen.

Conditions do the rest of the damage. That study describes its own footage in terms of murkiness, blurriness and shifting luminosity, with reef structure and moving plants confusing the background. The controlled study, by contrast, deliberately held lighting near-constant at a fish counter. Those two setups produce numbers that should never be quoted side by side without saying which was which.

There is a subtler trap the same authors flag: models over-fit their training data and then fail on data they have not seen. A system tested on the water it was trained in flatters itself. It is the machine-learning equivalent of marking your own homework.

The fish that cannot be told apart by looking

Some species are beyond photography entirely, and this is not a limitation of software but of appearance itself.

DNA barcoding of the goby genus Trimma found that 52 species defined on morphological grounds actually contained 94 distinct genetic lineages, implying roughly 42 additional cryptic species and pushing the genus estimate from about 110 towards 200. For one of them, the researchers reported no known differences in morphology or colour pattern separating the genetic groups at all.

Hybrids create a related problem closer to home. Oregon’s wildlife department publishes a bull trout identification guide that tells observers to disregard every other feature and look only at the dorsal fin: unmarked on a bull trout, marked with solid black on a brook trout, and on a hybrid marked in a way that lacks the solid black and the distinct edges. That is a real, workable field character — and it is a single fin’s pattern, frequently out of focus in a photograph taken one-handed over the water.

What professionals use instead

Identification keys are built on counting, not on overall resemblance, and that difference explains most of what a photo cannot do.

A key works as a tabulation of characters arranged in couplets, each choice leading either to a species or to the next couplet. The characters that carry the weight are things like dorsal and anal fin ray counts — described as relatively stable and straightforward to obtain — alongside scale type, mouth position, and the fish’s locality, depth and habitat. Reference databases hold fin-ray counts for thousands of species precisely because those counts are the reliable currency.

A single side-on photograph rarely delivers a countable ray series or a countable scale row. So a photo model is not doing a lower-quality version of what a taxonomist does; it is doing a different thing, inferring from overall pattern what a key resolves by measurement.

Even people get this wrong

Human identification is not the reliable baseline it feels like, which is worth knowing before treating any automated result as uniquely suspect.

A study of fish monitoring in the San Francisco Estuary tested 32 observers against 155 specimens and found trained staff misidentified fish at an average rate of 17%. Error rose for smaller fish, rarer species and less experienced observers, and untrained or part-time observers were roughly six times more likely to produce a false-positive identification. These were professionals with the animal in their hands.

Anglers do better on distinctive fish and worse on confusable ones, exactly as you would expect. Expert review of 5,390 shark photographs from a Texas tournament found 97.2% overall angler accuracy — but 76.1% for spinner shark, with blacktip and spinner the dominant confusion pair.

Community verification has a similar tail. A study scoring citizen-science records of non-native marine species found that around 24.9% of observations carrying the platform’s “research grade” label contained inaccurate or uncertain identifications, with a further 18.4% impossible to score because of poor image quality or because the taxa required molecular confirmation.

The useful conclusion is not that identification is hopeless. It is that difficulty belongs to the fish. A distinctive species photographed side-on in good light is easy for a model, an angler and a biologist alike — and a cryptic species pair is hard for all three.