Face ID: the 3D face is finally consumer hardware

Apple announced the iPhone X yesterday. No home button, no Touch ID. You unlock it by looking at it.

The part of the keynote I rewatched was the TrueDepth camera. A flood illuminator lights your face with infrared so it works in the dark. A dot projector throws more than 30,000 invisible infrared dots onto your face. An infrared camera reads how the dot pattern is distorted by the shape of your face, and from that the phone computes a depth map. A neural network, running on a new part of the chip Apple calls the Neural Engine, compares that depth map and an infrared image against the model of your face it enrolled. Apple says the chance a random person could unlock your phone is about 1 in 1,000,000, compared with 1 in 50,000 for Touch ID.

Structured light isn’t new. It’s the same principle as the original Kinect, and Apple bought PrimeSense, the company behind that sensor, in 2013. What’s new is making it small enough for the notch at the top of a phone and reliable enough to be the only way into the device for hundreds of millions of people.

I have a personal reaction to this. In 2013 my job was taking one flat selfie and guessing a 3D face from it, fitting a template mesh to a few dozen landmarks. The depth was always a guess, which is why the side view of our faces was always the weakest. Now the phone just measures it, with tens of thousands of real depth points, every time you look at it.

Animoji, the talking emoji, uses the same data. It tracks more than 50 facial muscle movements and drives a cartoon unicorn or a pile of poop with your expressions in real time. It looks like a gimmick in the keynote, but it’s the most sophisticated consumer face tracking ever shipped, and it’ll be sitting in millions of pockets by Christmas. Developers will get access to the face mesh through ARKit, so I expect a flood of apps that do what we were trying to do four years ago, only much better and with a fraction of the effort.

Two things I’m curious about. First, security researchers will attack it immediately with masks and photos and twins, and Apple itself says twins and young siblings may fool it. Second, the social side. A phone that recognizes your face every few minutes normalizes face recognition in a way nothing else has. People who would be uneasy about cameras identifying them on the street will now do it hundreds of times a day, and like it. That change in what people are comfortable with may end up mattering more than the hardware.