National

Chinese tech companies have ramped up efforts to set technical standards for facial recognition, raising concerns among business competitors, political observers and humanitarian advocates.

Why it matters: China has long made a systematic effort to set international standards on data and hardware compatibility across brands so that the standards reflect how Chinese products already work — giving its domestic industries a leg up in engineering races.

But the nation's push to do so on behalf of facial recognition and surveillance industries that it built to surveil and subjugate the nation's Uighur ethnic minority is casting its standards efforts in a new, harsher light.

Driving the news: Several Chinese firms, including ZTE, Dahua and China Telecom, have sought approval at the International Telecommunication Union, a UN technical standards body, for technical standards allowing different companies' surveillance technology — including facial recognition — to work together.

China's surveillance industry has grown in no small part due to funding research and buying equipment to fuel a high-tech human rights crisis. China has sent more than 1 million Uighurs to re-education camps while developing systems that identify race using facial features. 

To be clear: Technical standards do not commit atrocities.

Facial recognition technology has legitimate uses, from unlocking phones to identifying missing persons, and to use data or software across different brands of products, there needs to be standardized hardware and file formats.

But, but, but: China will likely use the technical standards to claim a UN seal of approval for its use of its products. 

Humanitarian groups worry about a feedback loop, where standards set to create Chinese surveillance industry successes will fund more research into technologies domestically used for oppression, which can then be sold for profit to create new business success.

The big picture: China has tried to use this standards approach to corner entire industries, including telecommunications. Facial recognition techniques represent a particularly thorny part of a broader effort to control artificial intelligence standards. 

By the time complex infrastructure schemes are entrenched in poorer nations, including those in Africa and Asia, they become extremely difficult to alter.

“China offers equipment and funding to modernize those countries on the condition they use these standards, making them even harder to change,” said Tom Duesterberg, senior fellow at the Hudson Institute. 

That’s bad for technology. “One thing we don't talk about as much is how China’s rush to entrench its standards precludes other technical standards,” said Kara Frederick, fellow at the Center for a New American Security. “It stops innovation wherever China decides to stop it.” 

Between the lines: Technical standards are conventionally seen as modular schematics for emerging technologies. For instance, they explain how to encode an MP3 file so it will play on any device — not what an MP3 file should be allowed to encode. 

So a technical standards body, at least as we currently envision it, might not be an effective place to address geopolitical or humanitarian issues.

But there aren't too many other battlegrounds for challenging China on the Uighur issue. Several humanitarian groups are taking the human rights fights to the ITU.

There is a second option that some, including Frederick, specifically mention: A U.S. technology policy that funds domestic research and development, so that the U.S. remains in the conversation as China picks future standards fights.

Leaked documents show Chinese businesses are shaping UN facial recognition standards

China's move on face-recognition standards

China will likely use technical standards to claim a UN seal of approval for its use of its products.

Illustration of a person targeted by face-recognition technology

Technology

China

Never miss a major tech story

World

Surveillance

China is becoming the most formidable rival America has faced, and the risk of confrontation with Russia, North Korea or Iran still looms. Authoritarians are rising, the chaotic Afghanistan exit revealed the limits of U.S. power, and the pandemic, climate change and other critical challenges continue to pose grave threats.

From the Valley to D.C., Big Tech players like Facebook, Google and Amazon are under more scrutiny than ever as new technology develops and privacy and antitrust concerns grow in lockstep with companies’ ambitions.

New York City police until recently illegally kept a database of fingerprints from juveniles who'd been taken into custody, AP reports. 

What we know: The Legal Aid Society says the repository contained tens of thousands of youths' fingerprints. Bulletins have also been issued to the Department's 36,000 officers notifying them of the procedural change.

Under New York state law, juvenile fingerprints must be destroyed within local police departments after being sent to a statewide repository with the Division of Criminal Justice Services.

What they're saying: The NYPD says it has wiped the database since the Legal Aid Society's discovery, though it didn't admit any wrongdoing, per Spectrum News NY1.

Of note: Per AP, "Beside the fingerprint database, the Legal Aid Society’s lawyer representing juvenile and adult defendants say they’re also concerned about police use of facial recognition and DNA collection, as well as its database of suspected gang members."

NYC council votes to shut down Rikers Island


Report: NYC police illegally kept log of juveniles' fingerprints

NYPD says it's since wiped the prints from their system.

Photo: Sergi Reboredo/VW PICS/Universal Images Group via Getty Images

U.S. News

New York City

Edward Snowden on what he gave up to become a whistleblower

Former National Security Agency subcontractor Edward Snowden told "Axios on HBO" that "it was a difficult thing to come forward" and release top-secret documents about U.S. intelligence agencies' surveillance of American citizens to journalists in 2013.

Why it matters: The U.S. government does not considered Snowden a whistleblower because he did not raise his concerns through the legal process that had been established. As a result, he has lived in exile in Russia for more than six years.

Snowden's leaks prompted changes to laws and standards for American intelligence agencies and the practices of U.S.-based technology companies.

What he's saying: Snowden said he gave up a well-paying government job "spying on you" to never return home to see his family.

"Today I still live in exile, where I've been for over six years, which is actually getting close to the eight years that I spent working for the government in the intelligence community."

U.S. sues Edward Snowden for profits from new memoir

Edward Snowden tells "Axios on HBO" what he gave up to become a whistleblower

"Today I still live in exile, where I've been for over six years."

Politics

Edward Snowden

China touted emotion recognition systems as a means of crime prevention at its 2019 Public Security Expo, Financial Times reports (subscription), although experts say the tech doesn't work as advertised.

Reality check: "The science on emotion recognition is pretty bogus," ACLU senior policy analyst Jay Stanley tells Axios. A July study found that it is not possible to confidently assign emotional states to facial expressions "regardless of context, person, and culture" —  "as much of current technology tries to do."

What's happening: China says it's rolling out the tech in Xinjiang, where Uighur Muslims are kept in mass detainment camps, and in subway stations and airports to "identify criminal suspects," per FT. 

“At present only a few schools and public security bureaus have products that include this type of technology,” Zhen Wenzhuang told FT, adding that emotion recognition has "not been fully developed for commercial use" in China.

Between the lines: Even if the tech doesn't track emotions as advertised, being watched or even thinking you're being watched can still have a psychological effect and encourage people to change their behavior, as seen in workplace polling.

In the U.S., Microsoft claims that its Face API program can identify emotions like contempt, happiness and disgust. Amazon's Reokognition points out that when its API identifies someone's facial expression, it "is not a determination of the person’s internal emotional state."

Go deeper: AI is "awakening" surveillance cameras

China touts dubious emotion recognition tech

China is rolling out the tech in Xinjiang, subway stations and airports.

In this illustration, several security cameras face different directions

Most jobs are still out of reach of robots, which lack the dexterity required on an assembly line or the social grace needed on a customer service call. But in some cases, the humans doing this work are themselves being automated as if they were machines.

What's happening: Even the most vigilant supervisor can only watch over a few workers at one time. But now, increasingly cheap AI systems can monitor every employee in a store, at a call center or on a factory floor, flagging their failures in real time and learning from their triumphs to optimize an entire workforce.

A network of surveillance cameras hooked up to special software can tally the seconds of each worker's bathroom break or time each step of their work.

It can also keep workers safe, automatically detecting the absence of hard hats and gloves, for example, or people straying into the path of dangerous machines.

In some call centers, AI listens into every conversation, cataloging every word, who said it and how, and then scoring each agent.

Why it matters: Companies can use this data to juice workers' productivity and efficiency. Eventually, they could gather enough data from humans to train machines to mimic them."

How often is an employee going out to smoke a cigarette? How long a lunch are they taking? How long are they sitting in the lunchroom?" These are the questions clients want answered with AI software, says Kim Hartman, CEO of Surveillance Secure, a D.C.-area company that installs security systems.

Hartman says his company has put in video analytics for several area retailers and restaurants that wanted to monitor their employees' productivity.

In a handful of factories in the U.S., cameras have been installed over each worker's head in assembly lines as they put together car parts or electronics.

Software developed by Drishti, a Silicon Valley startup, watches these assemblers work, timing each step and checking for mistakes.

The videos let supervisors quickly figure out where something went wrong and teach a worker how to avoid repeating an error, says Drishti CEO Prasad Akella. It can also be used for training new hires.

And since AI is constantly watching the video streams, it can extract valuable data about timing and actions across the entire assembly line, which can inform new ways of assigning work.

"The most programmable machine on the planet today is still the human." 

"Employers and companies attempting to extract more value from its labor force by making that labor more efficient is nothing new," says Jess Kutch, co-founder of Coworker.org, a nonprofit that helps workers organize. A century ago, managers used stopwatches to pursue efficiency under the banner of "scientific management," or Taylorism.

But extreme monitoring enabled by new technologies can be inhumane, Kutch says.

"In low-wage work we're seeing a lot more decisions that were made by a middle manager being outsourced to an algorithm," says Aiha Nguyen of the research organization Data & Society.

"What workers are seeing, and have a fear of, is arbitrarily speeding up workplaces," Nguyen tells Axios.

The creators of AI monitoring tools argue that their software benefits employers and employees.

Drishti provides workers and supervisors with valuable feedback, Akella says. Its software can call out high performers, reward efficiency-improving creativity and even keep workers from hurting themselves.

Akella argues that employees won't be forced to work much faster and harder because turning up the heat would introduce unacceptable errors.

Call center agents monitored by AI software from CallMiner prefer being graded by an "impartial computer" over a human supervisor, says CTO Jeff Gallino.

What's next: Extensive AI-annotated video or audio data about how people work is a potential gold mine for automation developers.

Robots are still too klutzy to take over assembly lines built for humans but could learn how to put together products in a machine-only environment.

Gallino says CallMiner could use information gathered from human agents to automate the "boring" parts of customer service calls.

Go deeper: Automated management for call centers (NYT)

Automating humans with AI

In some cases, humans at work are being optimized as if they were machines.

Binoculars with 0’s and 1’s in the lenses

Future

The United States filed a lawsuit Tuesday against former CIA employee and NSA contractor Edward Snowden alleging his memoir violates non-disclosure disagreements.

The lawsuit argues Snowden did not allow the agencies to review his book before publication, as had been required by a signed agreement. While the lawsuit will not attempt to stop publication of Snowden's book book, Permanent Record, it does seek to recover all proceeds earned by Snowden because of the violation. 

Snowden leaked a series of confidential documents to journalists in 2013 while working as an NSA contractor. The information revealed previously unknown details about the state of citizen surveillance in the U.S.

"The United States’ ability to protect sensitive national security information depends on employees’ and contractors’ compliance... We will not permit individuals to enrich themselves, at the expense of the United States, without complying with their pre-publication review obligations."

Assistant Attorney General Jody Hunt of the Department of Justice’s Civil Division.

The state of play: Snowden has been on a press parade ahead of his memoir's release. In an interview, Snowden told "CBS This Morning" he would agree to return to the U.S. if he could settle on negotiations for a "fair trial."

Go deeper: Snowden says Biden warned countries not to grant him asylum

The lawsuit alleges the book's content violates Snowden's NDAs.

An announcement this week by a major spyware vendor that it aims to embrace human rights is forcing the industry, governments and civil society groups to consider whether the concepts of "human rights" and "spyware" can ever be reconciled.

The big picture: Government-grade spyware has always been abused. In June, David Kaye, the UN special rapporteur on freedom of opinion and expression, determined that commercial spyware had become so vast a problem that the world needs a moratorium on it, for companies and governments to figure out how to protect human rights.

Spyware from NSO Group, the Israel-based firm that announced the human rights initiative, was allegedly used by Saudi Arabia to spy on U.S.-based reporter Jamal Khashoggi, who was later killed by Saudi agents. Mexico also used NSO spyware to surveil government employees and researchers who backed a tax on soda. 

But even well before NSO group became a major spyware player, other products — including Gamma's FinFisher and Hacking Team's Da Vinci and Galileo products — have been embroiled in human rights debates. Ethiopia allegedly used spyware to surveil journalists, Uganda allegedly targeted opposition political figures, and Morocco allegedly targeted activists. 

Many other clients of spyware vendors have poor human rights records, including Azerbaijan, Venezuela, Uzbekistan and Sudan. 

Yes, but: It's tough to prevent abuse without oversight. Spyware vendors are loath to surveil their own clients, meaning that reporting about potential human rights abuses either comes from victims lucky enough to figure out they were being watched or from the countries themselves.

"If they don't have a mechanism of looking over governments’ shoulders, I don’t see how this has any teeth," John Scott-Railton, a senior researcher at the University of Toronto's Citizen Lab, which has done much of the research on NSO's alleged human rights abuses, told Axios.

Without that oversight, Scott-Railton isn't confident that any spyware could be safe for human rights. "If the question is, 'Is it possible to sell cyber weapons and assure they won’t be used for abuse,' I think it’s a contradiction in terms," he said. 

Amnesty International has been a persistent thorn in NSO's side, even assisting a lawsuit to force Israel to ban NSO from exporting products. But Amnesty deputy program director Danna Ingleton is optimistic that there is a way for spyware companies to align with human rights. 

That doesn't mean NSO's current plan passed Ingleton's muster, yet (see item 2). But through due diligence before making sales to regimes, honest accounting of past actions, export rules that are more transparent and engagement with civil society groups, she believes a company like NSO could get ahead of the human rights issue. 

NSO would have to be more open about its internal capabilities to flag human rights abuses as they happen. And governments would need to take an active role in restricting sales to dangerous countries. 

"The onus is on the companies. If they can't protect human rights, they need to enact safeguards," she said. "And if it's an industry that can never be in line with human rights, it's up to the state to do what it needs to do."

The bottom line: The commercial spyware industry is not going to vanish — it's too ingrained in global intelligence and law enforcement. That might mean the only way to protect human rights is to adopt rules like those NSO has announced and make them work.

Spyware's human rights dilemma

NSO Group's effort to embrace human rights poses bigger challenges for an industry known for violating those rights.

Illustration of a giant security camera watching a small person on their cell phone.

Until now, the vast majority of information collected about us has remained untouched — there was just too much to make sense of it all.

What's happening: Artificial intelligence allows data that might once have gone unnoticed to now be detected, analyzed and logged in real time. It's already started supercharging surveillance at work, in schools and in cities.

Police are experimenting with facial recognition — either to help solve specific cases, or to compare the faces of passersby against a database of suspects. This week, a U.K. court upheld real-time facial recognition there.

In New York City, software can detect crime patterns over time and across boroughs, linking events in a way humans never could.

And in schools, cameras automatically watch for unusual behavior, and microphones listen for "aggressive" sounding voices.

The big picture: Humans have monitored each other as long as we've lived in communities — to punish free riders and troublemakers.

But now, cheap, powerful machines are taking the place of human watchers, disrupting a long-held social contract.

Unlike in China, where high-tech surveillance is a tool of fear and control, systems in the West are not centralized for now, curbing the scope of data gathering.

And tech companies like Facebook and Google have perfected online versions of automated surveillance for profit, in the form of products we can no longer live without.

Details: Software can identify and track faces, skin color, clothing, tattoos, walking gait and various other physical attributes and behaviors. But it's been plagued with bias and inaccuracy problems that primarily harm people of color.

From facial expressions and body movements, AI can extrapolate emotions like happiness and anger — a process built on shaky scientific evidence.

The impact: This quiet shift from passive watching to active surveillance is chipping away at our ability to remain anonymous in physical and virtual spaces.

Blending into the crowd is no longer an option if every face in that crowd is captured, compared against a driver's license photo and logged.

Constant AI surveillance threatens to erode the all-important presumption of innocence, says Clare Garvie, a privacy expert at Georgetown Law.

How AI police surveillance treats people of color

Deep Dive: The end of anonymity

Data that might once have gone unnoticed can now be detected, analyzed and logged in real time.

Illustrated collage of a man being revealed from behind graph paper.

Deep Dive - Surveillance

Get insights on the collision of cybersecurity and policy and why it matters

Sign up for the Axios Codebook newsletter

Automation and AI

Data privacy

Axios Science

Data: SurveyMonkey online poll of 3,454 employed adults, Aug. 20–25, with a margin of error of ±2.5 points. Chart: Andrew Witherspoon/Axios

The point of surveillance is to track and influence people's behavior — in the political sphere, at school, in the subway or in the workplace. Just how new technologies affect someone's behavior and thinking at work depends on each individual, what they are doing and the goals of those who are surveilling.

The big picture: Experts predict AI-fueled mass surveillance will alter people’s behavior. But exactly how it will change our thinking isn’t well understood.

Studies of electronic monitoring in the workplace offer some insights into the psychological impact of surveillance.

The positive: In some circumstances, monitoring can be used to improve performance or train employees.

The negative: It can also focus people on optimizing for a particular goal at the expense of others, diminish the quality of someone's work or increase stress.

When the purpose of monitoring isn't clear or trusted, employees can come up with ways to avoid it, spurring managers to monitor them even more, a 2018 study of TSA workers suggests.

"Managers should be good at communicating their intent widely and clearly but also demonstrating how it will be helpful to workers," says Curtis Chan, who studies organizational management at Boston College and was an author of the TSA study along with Michel Anteby of Boston University.

Now there is data tracing our days — Slack messages, comings and going via fob swipes and social media posts.

When managers poke and prod that happenstance information or monitor employees without communicating a clear purpose in doing it, it can signal a lack of respect and trust, says psychologist Tara Behrend of George Washington University. 

"There is a sense of each party owes the other one something and this kind of surveillance can violate that," she tells Axios. 

Some people will then look for ways to reassert their autonomy, for example, by withholding effort or not helping to train a new employee. 

62% of people surveyed by Axios and SurveyMonkey said it is appropriate for an employer to routinely monitor employees using technology. 

48% said they would change their behavior if they knew their employer was monitoring them.

That's not surprising, says Behrend, but the type of data being monitored may matter (physiological measures of sweat levels, for example, versus work emails) and whether someone considers themselves as being monitored.

Society doesn't mirror the workplace: The agreements between parties are different. 

Still, Behrend says surveillance in both cases involves one party having power over the other — a dynamic that defines monitoring and its effects on behavior.

The bottom line: The basic principles of learning, motivation and trust still apply when these technologies are deployed. "We can’t just start throwing tech at things and not expect it to have profound psychological effects," says Behrend.

Methodology: These data are from a SurveyMonkey online poll conducted among employed adults ages 18 and older in the United States. Respondents were selected from the more than 2 million people who take surveys on the SurveyMonkey platform each day. Data have been weighted for age, race, sex, education and geography using the Census Bureau’s American Community Survey to reflect the demographic composition of the United States age 18 and over. The survey was conducted August 20-25, 2019 among 3,454 employed adults. The modeled error estimate for the full sample is plus or minus 2.5 percentage points.

How surveillance changes behavior

The point of surveillance is to track and influence people's behavior.

Those websites we reported on last week that target iPhone users with malware appear to have been part of China's long-running effort to monitor its Uighur population.

The big picture: The security vulnerabilities that mobile malware takes advantage of are scarce and expensive, and countries are loath to risk burning their tools by widely exposing them. 

The only scenario where it makes any economic sense to use such techniques this broadly is one involving a wealthy government trying to mount a vast surveillance effort — as with China's campaign among the Uighurs in Xinjiang.

Why it matters: No one has attempted to spread mobile malware to such a wide group before because no one has tried to surveil an entire ethnic group this way before.

Driving the news: On Thursday, Google announced it had discovered several campaigns using popular websites to indiscriminately inject malware onto iPhones. 

Subsequent reporting showed that the campaign looked to infect not only iPhones but Android devices and Windows computers as well. 

The firm Volexity found evidence those sites included Uighur news outlets and other Uighur community sites.

Background: Surveillance of the Uighurs is nothing new. "The Chinese government has long harbored suspicion about the Uighur population’s loyalty to China, confusing ethnic identity with separatism," said Sophie Richardson, China lead for Human Rights Watch. 

China has created "re-education camps" housing as many as 1.5 million Uighurs.

But in recent years, China has tightened its heavy-fisted rule of Xinjiang province with high-tech techniques.

Biometrics: China uses widespread facial recognition research to detect and track the Uighur minority, and it has roped in Western research institutions to help. Western research institutions and journals have aided in development of facial recognition to distinguish Uighur facial features using artificial intelligence. 

China tracks Uighurs through other biometrics too, including DNA. 

Digital tools: China tracks digital communications from Uighurs and stores information tapped from WiFi-enabled devices. Tourists must install a monitoring app on phones when entering the Xinjiang province that scans for Quran passages and other contraband information, and Reuters reported Thursday that China hacked telecoms to spy on Uighur travelers.

All of the information is filtered through a machine learning algorithm-fueled mobile app used to direct police activity. 

Human Rights Watch reverse-engineered the app and found a wide variety of factors play a role in determining who is treated as a suspicious individual — including whether they use their home's front or back door.

The bottom line: This is not a small undertaking. China's willingness to spend on technology to surveil Uighurs has created a niche, high-growth industry among military contractors.

Secretary of State Mike Pompeo recently described the crackdown on Uighurs as the “stain of the century.” And the White House added a Uighur American to the National Security Council to contribute to China policy.

A new window onto China's Uighur spying

A massive surveillance program is using smartphone malware in unprecedented ways.

A screen showing images of Chinese President Xi Jinping in Xinjiang

A screen showing images of Chinese President Xi Jinping in Xinjiang where a pervasive security apparatus has subdued the ethnic unrest, June 2019. Photo: Greg Baker/AFP/Getty Images

Uyghur

President Trump's crackdown on TikTok suggests that the U.S. government is starting to see the internet more like China does — as a network that countries can and should control within their borders.

The big picture: Today's global internet has split into three zones, according to many observers: The EU's privacy-focused network; China's government-dominated network; and the U.S.-led network dominated by a handful of American companies. TikTok's fate suggests China's model has U.S. fans as well.

Why it matters: As the global internet splinters further, the U.S. and China look set to enter a Cold War-style battle for the hearts and minds of global users and developing nations. In this fight, U.S. nationalism may make a weaker case to the world than the ideal of internet freedom and open networks that the U.S. once evangelized.

Driving the news: President Trump's threat to ban TikTok from the U.S. has fast-tracked an effort to force the video-sharing app's Chinese owners to sell it to a U.S. company (Microsoft is in talks).

Be smart: The move puts other foreign-owned companies on notice that the U.S. intends to favor American-owned businesses in the digital world. 

That's a giant break from a long-established bipartisan consensus that American interests are best served by a U.S. marketplace, online and off, managed as a level playing field.

China started this game — it was the first country to self-isolate from the global internet. 

China controls information flows inside its borders, and its "great firewall" keeps most information the government dislikes from entering the country. 

The Chinese government has ensured that its most widely used search and social services are domestically owned, and its laws now require any internet service to share data with the authorities.

Meanwhile, the EU's new privacy rules have created a different kind of internet zone  governed by a stringent set of standards for handling individual user data.

Last month, a major EU ruling struck down the Privacy Shield, an arrangement between the EU and the U.S. that enabled companies to transfer data into and out of EU nations without risk. 

Trump's crackdown on TikTok suggests that the U.S. is now adopting a more China-like perspective on what kinds of actions governments should take toward internet companies, including:

picking winners and losers among companies based on the whims of leaders;

insisting that companies make data available to the government for its needs — as Attorney General William Barr's argument for weak encryption does;

threatening to punish private platforms unless they support political leaders' preferred form of speech regulation.

Yes, but: Concerns over TikTok's willingness to share U.S. user data with the Chinese government are real.

TikTok says it keeps all such data outside of China and the reach of the Chinese government's laws. 

Many security experts view all Chinese-owned businesses as vulnerable to government interference and intelligence demands.

Our thought bubble: During the Cold War, America's reputational advantage over the Soviet Union as a rule-of-law democracy was regularly undercut by U.S. operations to overthrow governments abroad and American failures to guarantee human rights at home. 

The U.S. is now playing by China's internet rules

China believes in nationalist control over digital networks, and increasingly, so does the White House.

Sections

Axios Local

Axios gets you smarter, faster with news & information that matters.

About

Subscribe

All Surveillance stories