Seeing like an AI

New research from a major AI company offers insight into how neural networks are able to "see."

Why it matters: Reliable computer vision is a cornerstone for AI applications like self-driving cars, but the effectiveness of neural nets in recognizing images is only matched by their impenetrability. The new research allows scientists to peer into the black box of computer vision, with implications for reducing bias and errors.

Driving the news: In a new paper about its CLARITY project, researchers from OpenAI show that "multimodal" neurons are at work within the artificial neural network of its CLIP general-purpose vision system, which is capable of recognizing images and writing text descriptions.

• Multimodal refers to a single neuron firing in response to certain photographs, sketches and even text — all different "modes" of representation — that can be classified under a single concept, like "my grandmother," or in the case of one famous neuroscience paper from 2005, the actress Halle Berry.

How it works: A majority of the artificial neurons at work in the CLIP model show evidence of this multimodality, according to Gabriel Goh, the lead researcher for OpenAI on CLARITY.

• The same neuron that fires for an image of a spider might fire for text that contains the word "spider," or even for certain patches of red and blue that CLIP can recognize from the comic book character Spiderman.
• "Both the brain and these kind of synthetic vision systems seem to be converging on a very similar kind of mode of organizing information," says Goh. "So it may turn out that these systems aren't as much of a black box as we thought."

The big picture: Beyond providing insight into how computer vision systems actually work, the CLARITY research can help AI scientists identify how those systems might be biased or even deliberately disrupted.

• "Knowing what mistakes are being made and knowing how those systems work can help us understand the kind of biases that these networks have learned from their training data in order to perform interventions in cases where a particular bias is undesirable," says Ilya Sutskever, co-founder and chief scientist at OpenAI.

The bottom line: Finding a similarity between the human brain and artificial neural networks is also a "point in favor of the idea that deep learning may go even further beyond what people think," he adds.