The smartest AI is dumb without people

• Many of today's most powerful AI systems aim to offer a convincing response to any question, regardless of accuracy.
• "If you don’t know, you should just say you don’t know rather than make something up," says Stanford researcher Percy Liang, who spoke at a Stanford event Thursday.

Liang has launched a project to evaluate the latest machine learning models on a range of factors, from accuracy to transparency.

• The goal, he said, is to create something equivalent to Consumer Reports, where people can go to understand the strengths and weaknesses of foundational AI models, such as those from Meta, Google and OpenAI.

Factuality is just one part of this picture. It also matters a great deal what basis an AI system has for providing an answer, and who benefits.

• Historically, computer systems have been designed mostly for the people using them.
• But an algorithm choosing a criminal sentence, for example, needs not only to serve the judge it's advising but also crime victims, perpetrators and society as a whole.
• Many Americans would feel, for example, that it should take into account the impact of incarcerating a significant portion of the African American adult male population.

This doesn't mean "asking a neural network to understand racism," James Landay, the co-founder of Stanford's Institute for Human-Centered AI, told a daylong gathering with reporters on Thursday.

• "It's asking the team building a system to understand racism. That’s not a question computer scientists who are generally building these systems are equipped to handle."

The big picture: For years, AI researchers have talked theoretically about responsible ways to design such systems and divide the future of work between humans and computers. However, a flood of powerful new systems is giving these questions practical urgency.

Between the lines: For all the talk of computers replacing or even replicating human activity, the most powerful use of them may be to help humans do their jobs better.

• "Just simply mimicking human beings is meager," Stanford professor Erik Brynjolfsson said at the event. "Paradoxically It’s also too hard."
• That's because computers excel at tasks where humans falter, from processing vast amounts of data to spotting patterns that even a skilled researcher might miss.

• At the same time, computers and robots still can't match humans at everything from gauging the delicate pressure needed to pick a blueberry to walking on a bumpy trail.

Zoom out: An equitable distribution of the fruits of AI will depend on whether it's used to replace humans, which tends to drive down pay, or to augment them, which drives it up, Brynjolfsson argues.

• Over the past century, we've largely used technology to make individual workers more productive, and that's raised the standard of living, he says.
• In recent years, though, we've been building more machines that substitute for humans. That's concentrated wealth further in fewer hands, while those with a high school education or less face higher rates of despair, drug abuse and suicide.
• "For most of us that’s probably not the world we want to live in," Brynjolfsson said.

Be smart: Pairing humans and computers may have advantages for society, Brynjolfsson said, but it can also get better results for businesses.

• In automating call centers, Brynjolfsson points out, you can substitute machines for people and frustrate customers. Or you can take the path of a company he is advising, Cresta, whose system monitors calls and offers suggestions to human call center workers.
• That saves companies money, maintains jobs and leads to happier customers, Brynjolfsson says: "Keeping the human in the loop seems to work a lot better."