Where high-stakes AI needs human judgment

Artificial intelligence (AI) can sort, score and make recommendations faster than people alone can. But when a decision can change someone's life, speed isn't the same as judgement.

What you need to know: As AI moves deeper into health care, education, housing and public services, organizations need to know where automation helps and where human expertise still has to lead.

• That's what Virginia Tech researchers are studying by applying AI for social good.

Why now: The rise of agentic AI means automated systems may soon make more decisions with less direct human input. The more autonomy AI gains, the more important it becomes to know where human judgment, oversight and context belong.

The story: Sanmay Das is the associate director of AI for social impact at Virginia Tech's Sanghani Center for Artificial Intelligence and Data Analytics, and he studies how AI performs in complex decisions with real human stakes.

An example: Das has recently focused on homelessness intervention services in which social workers help determine which households receive more intensive support when resources are limited.

Those choices can shape a person or family's long-term stability. They also require context, discretion and professional experience that most LLMs' broad training data may not fully capture.

• "What Sanmay does in a lot of his projects is to really think about people and impacts of automated decisions on their life or work, and what happens if you get them wrong," said Naren Ramakrishnan, director of the Sanghani Center. "That's the most crucial aspect."

The proof: Das and his team ran three experiments to compare how different decision-makers handled homelessness intervention scenarios.

• They asked non-experts to review support options for people facing homelessness.
• They studied real case workers' decisions and outcomes to understand how and when trained pros use discretionary judgement.
• They asked various AI models to make similar choices about homelessness services and analyzed their responses.

The results: AI models tended to make decisions similar to the groups of non-experts.

• Their answers also varied a lot and were inconsistent with existing scoring systems to assess vulnerability and medical frailty.

On the other hand: Trained case workers were better able to make high-value discretionary decisions, especially in ambiguous cases without a clear playbook.

• Das' research showed that when case workers allocated more intensive resources in those edge cases, they helped the households receiving support without meaningfully harming the households that did not.

In other words: AI may produce answers that seem logical to an untrained eye. But real-world judgment often depends on context the data doesn't capture and people trained to see what a model can miss.

• "AI is super useful and it can do a bunch of different things, but how you structure that interaction has all kinds of implications on the kinds of judgments and decisions you're going to get at the end of the day," Das said. "And that can have a significant impact on people's lives."

The research suggests that AI may be most useful for social good when it helps experts save time and focus their energy on the hardest calls, rather than replacing them.

Here's what else: Virginia Tech is preparing today's students to understand AI's limits.

Students in the Sanghani Center's graduate data analytics certificate program are required to take Ethics and Professionalism in Computer Science, which examines cases where tech has gone wrong, including predictive policing systems that reinforce bias.

Virginia Tech's Department of Computer Science is also launching an AI undergraduate minor this fall, available to students across all majors, focused on developing meaningful AI literacy alongside a primary field of study and understanding where humans fit into the loop.

An expert take: "Virginia Tech is committed to advancing AI that is technically sound and worthy of the public's trust," said Warren Dixon, dean of the College of Engineering.

• "Sanmay's work demonstrates why human judgment remains essential in complex decisions that affect people's lives. That human perspective and approach to research and engineering education will distinguish Virginia Tech faculty and graduates as leaders in developing and applying AI to solve society's most pressing challenges."

The takeaway: AI for social good depends on systems designed around trust, oversight and the human expertise needed to make hard calls well.

See how Virginia Tech is leading research into AI for social good.