Soon, Your Car May Be Able to Read Your Expressions
By JOHN R. QUAIN APRIL 6, two thousand seventeen
When you shop for cat food online, Amazon knows who you are. When you look for the best airfares for summer vacation, Google knows who you are. Soon, when you get behind the wheel, your car may recognize you, too.
Using cameras with facial recognition software and other biometric indicators, automakers are looking to personalize the driving practice with cars that stare back at you, calmly adjusting seats and driving modes. They may even anticipate your wants and desires by playing your beloved music based on your mood. And it’s not only about convenience, but also about the possibility of improving safety and security.
“It’s not just about personalization,” said Zachary Bolton, a systems and technology engineer at Continental Automotive Group, explaining the sophistication of such systems. “We can use the gleam, the twinkle in your eye to determine precisely where you’re looking.” Engineers can then dynamically adjust the so-called human machine interface, putting critical information, say, about a stalled car up ahead, or the fact that you are about to exceed the speed limit, directly in a driver’s line of look on the dashboard or in a display on the windshield. Conversely, by tracking downward eye movements, the car could “see” that a driver was dispersed and sound a warning.
Continental has already demonstrated in-car systems that permit drivers to register their faces using something as elementary as a driver’s license picture. An interior infrared camera is used to overcome potential obstacles like sunglasses, which would stymie a conventional movie camera. Putting the camera in the center instrument cluster also helps to pinpoint the driver’s eyes, even if she’s wearing a hat. The largest technical challenge — glare caused by sunlight — can be filtered out using machine learning, Mr. Bolton said.
Once the car knows who you are, systems in vehicles like the Chrysler Portal concept car would automatically adjust the seat for maximum convenience, select a driving mode (for example, one driver likes to let the car do most of the work; another likes taking control in sport mode) and suggest a destination based on the owner’s past behavior.
Watching a driver’s face can also give a car significant clues about that person’s state of mind. For several years, carmakers like Ford and tech companies like Intel have been interested in determining whether a driver is glad or sad. Depending on your mood, a car could switch its tune, playing a bouncy Beach Boys song and switching the interior lighting to improve your attitude.
Honda’s NeuV concept car, for example, has a large customizable LCD dashboard and a cloud-connected, onboard computer that uses artificial intelligence to interact with drivers. NeuV employs what the company coyly refers to as an “emotion engine” to grease the wheels of the conversation, and its automated individual assistant can read “facial skin vibrations” to help it isolate the driver’s voice and better understand spoken directives.
There are practical reasons as well, designers say, for detecting a driver’s emotional state: A tranquil driver is a safer driver. So cars that recognize when you are becoming angry and thus prone to road rage could potentially quell annoying bells and chimes in the car and play some mellow jazz to alleviate you.
By substituting keys and remote control fobs, biometrics like facial recognition could also make cars more difficult to steal. In its prototype FF ninety one sport utility vehicle, the electrical car start-up Faraday Future uses an outer camera mounted in the door framework to detect the car’s possessor and automatically unlock the vehicle. Such technologies can create fresh security challenges, however.
In this digital age, our faces are everywhere: in online company profiles, on Twitter accounts, even tagged in friends’ Facebook accounts. Finding an photo to print out and foil a car’s facial recognition system would not be very difficult.
Fortunately, engineers have devised high-tech countermeasures. Stereoscopic movie cameras, for example, can tell the difference inbetween a vapid pic and a three-dimensional object. Continental’s cameras measure the distance of reflected light off various parts of a person’s face, ensuring that it is a real object rather than a high-resolution shot of the owner’s visage.
“Some systems have added blink detection and aliveness detection,” said Steve Grobman, chief technology officer for security at Intel. But he acknowledged that it was still possible to thwart such technology. “We had 3-D masks printed that we ordered and were able to trick some systems,” he said. But he said most thieves would be unlikely to go to such extreme lengths to steal a car.
“It all depends on the level of accuracy you need,” said Yoni Heilbronn, chief marketing officer of Argus Cyber Security, which works with automakers to short-circuit hacking threats. “Retina scans are even better than facial recognition” as a potential solution, he said, “but by adding another level of authentication you lose some of the convenience.”
On the other arm, high-tech personalization could be used not only to create amenities for single owners, but also to instantly adapt a vehicle to suit a multitude of drivers. Valets, for example, could be automatically prevented from accessing individual information in a navigation system or driving quicker than, say, thirty miles per hour. In a ride-sharing situation, such systems could also be used to quickly tailor a car’s interior to the physical characteristics of different drivers and passengers.
Rental cars would be lighter to operate and safer, Mr. Bolton of Continental said. “If I know where your head is and where your eyes are,” he said, “I can adjust the position of the steering wheel and the mirrors so you don’t have to fumble looking for the right buttons.”
Even airbags could be fine-tuned, reducing the power of their deployment depending on the size and position of a driver or youthful passenger.
Some elements of the personalized driving practice are already coming to cars. By the end of the year, Ford plans to add Amazon’s Alexa individual assistant to some of its cars, said Dave Hatton, manager of Ford’s mobile applications for connected vehicles. It will not only permit personalized music stations to play with a voice guideline, but also enable drivers to bounce chores like adding items to an existing grocery list with just a few words.
Such convenience may come with some trade-offs on privacy.
“It’s a phat concern,” said John Simpson, privacy project director at Consumer Watchdog, a nonprofit advocacy group. “All that data is in some database without your consent or skill about how it’s going to be used,” he said, adding that there was little if any current government regulation to curtail such use.
Using traffic-tracking programs like Waze, some consumers have already signaled their preference for convenience in comeback for providing up some information like their location. Programmers also point out that such services are optional: You don’t have to let the company track you — but then you may get stuck in traffic for forty five minutes.
Today, basic biometric technologies like facial recognition software are used for everything from signing into Windows laptops to thwarting toilet paper thieves in Beijing. Fingerprint readers are commonly used to unlock smartphones. As consumers become more habitual to such systems, the introduction of the technology in vehicles may seem like a natural evolution rather than a creepy intrusion.
And it could be joy. Consider the entertainment and social media repercussions in the vein of James Corden’s “Carpool Karaoke” segments. A built-in camera could record and broadcast your singalong on Facebook or Twitter — assuming the car was in autonomous driving mode. Of course, drivers could grab quick selfies on the road, too.
“It’s a novel ideal,” Mr. Bolton said, “but recall that an infrared camera makes your face look a little like a skeleton, so it’s not that flattering.”
A version of this article emerges in print on April 7, 2017, on Page B6 of the Fresh York edition with the headline: Your Car May Soon Be Able to Read Your Face. Order Reprints | Today’s Paper | Subscribe
We’re interested in your feedback on this page. Tell us what you think.