IBM, Amazon and Microsoft step back from face recognition
In one week, three of the biggest companies in the field paused or ended police sales. Where I agree, and why a pause isn't a policy.
This week, in the middle of protests across the US after the killing of George Floyd, three large companies changed course on face recognition. On Monday IBM’s CEO Arvind Krishna wrote to Congress that IBM would no longer offer general-purpose face recognition or analysis software. On Wednesday Amazon announced a one-year moratorium on police use of Rekognition. On Thursday Microsoft said it won’t sell face recognition to US police departments until there’s a federal law governing it.
I’ve written here many times about where I think the line is, so I’ll keep this short.
I agree with the direction. Police use of face recognition is the hardest case: people haven’t consented, the stakes are someone’s freedom, and the documented accuracy gaps across demographic groups mean the errors land unevenly. Doing this in the middle of protests, when the fear is that police could identify people at demonstrations, makes the decision easier to understand.
These are pauses from companies for which police sales are a tiny part of revenue. IBM had already been winding down this business. Amazon’s moratorium is one year. Microsoft says it doesn’t currently sell to US police anyway. None of this constrains the companies for which police sales are the main business, like Clearview, or the many smaller vendors who sell to law enforcement quietly. If anything, it gives them less competition.
A pause is also not a policy. Microsoft’s statement is the most useful one, because it asks for federal law. That’s what’s actually needed: rules that apply to everyone, that say which uses require consent, what accuracy has to be demonstrated and on whom, what can be retained, and what recourse people have. Leaving it to each company’s ethics team means the least careful company sets the market.
And the conversation keeps treating “face recognition” as one thing. It isn’t. Unlocking your own phone, a consenting attendee skipping a line, a police officer matching a protest photo against a database of everyone, and a company scraping billions of photos are different acts with different risks, even if the model inside is similar. A law that bans the technology outright would sweep up the harmless uses. A law that regulates uses, based on consent and scope, would be both stricter where it matters and less damaging where it doesn’t.
At Amanda we’ve tried to build as if that law already existed. I hope it gets written soon.