San Francisco banned face recognition. Customers asked if we're next
The ban covers city agencies, not private companies like ours. It still changed a lot of sales conversations this week. Where I think the line should be.
San Francisco’s Board of Supervisors voted on Tuesday to ban city agencies, including the police, from using face recognition. It’s the first major US city to do it. The ordinance also requires agencies to get approval before buying other surveillance technology.
It doesn’t apply to private businesses, so it doesn’t directly affect a company like Amanda. By Wednesday I’d still had three conversations with customers and prospects who wanted to know whether they could keep using us, whether their attendees would object, and whether this was the start of something that would reach them.
I want to be clear about where I stand, since I run engineering at a face recognition company and people reasonably assume I’d be against the ban. I’m not. I think it’s mostly right.
The problem with police use of face recognition is that the people being scanned haven’t agreed to anything. A camera on a street or a feed from a protest gets matched against a database of mugshots or driver’s license photos. Error rates are measurably higher for some demographic groups, which MIT’s Joy Buolamwini and others have shown in commercial systems. A false match in that setting can mean police at your door. And even a perfectly accurate system changes what it means to be in public, since anywhere with a camera becomes somewhere you can be identified. None of that is a model accuracy problem you can fix by training harder.
The use we build for is different in the ways that matter. People opt in, knowingly, usually by providing their own photo. They match against a small list of registered attendees, not a watch list. The data exists for the length of an event. The benefit goes to the person being recognized: they skip a line. If an attendee doesn’t want to use it, they walk to the desk like before.
So the line I’d draw is less about the technology and more about consent and scope. Is the person being identified choosing it? Is the matching limited to a set they know they’re in? Does the benefit go to them? Is there an easy alternative? Government surveillance fails all four. Unlocking your phone passes all four. Conference check-in, done properly, passes all four.
I’d actually like to see regulation that says this explicitly for private companies too, because right now the rules are “whatever you can get away with,” and some companies in this industry are getting away with a lot. Clear rules would help the companies trying to do this responsibly. We lose deals to competitors who will do things we won’t, and it would be nice if the law caught up.