Interview with George Brostoff: How SensibleVision’s face recognition tech will help Android phones compete with Apple’s FaceID

0
3D Dace Recognition Data - SensibleVision

Smartphones have become one of the most important tools in our lives, and consumers store critical information such as contacts, documents, images, and passwords on their phones, sometimes without much protection. While the use of a fingerprint reader has been pretty popular since the launch of the Apple iPhone 5s, new forms of biometric authentication systems have emerged this year. Samsung debuted the iris scanning implementation in its flagship smartphones, the Galaxy S8, S8+, and the Galaxy Note 8. Apple introduced Face ID, its own version of 3D face scanning and recognition, on the ultra-popular iPhone X. Face ID uses a dot projector for 3D face mapping, a flood illuminator, and an infrared camera to read the dot pattern. The whole mechanism is a bit slower than a regular fingerprint reader, and you can’t add multiple faces like you can with fingers.

While face unlock feature isn’t completely new to Android (it was present in the Galaxy Nexus which was launched six years ago), Android smartphone manufacturers have shown a renewed interest in face recognition after the launch the iPhone X. The question, however, remains whether Android phones can implement a face recognition and unlocking system that can match (or even exceed) Apple’s FaceID system? Not every smartphone brand has the engineering might or R&D budget of Apple, so how can your average Android phone maker implement fast and secure face recognition? SensibleVision is a data protection company headquartered in Florida. The company develops 3D face scanning authentication technology among many other things. SensibleVision is now working with various smartphone makers to bring 3D face scanning technology to Android smartphones.

SensibleVision Logo

We had a chance to conduct an email interview with the SensibleVision CEO George Brostoff to learn about the underlying technology, how soon can we expect 3D face scanning in Android smartphones, and if they can deliver an experience that can match or exceed Apple’s FaceID technology. Before we get to that, here’s some information about Brostoff and his past achievements:

George Brostoff is the founder and CEO of SensibleVision, a leader in 3D face scanning authentication technology, headquartered in Cape Coral, Florida. He has founded three successful tech companies, holds seven patents and grew up working in a family business.

Prior to SensibleVision, Brostoff co-founded Ensure Technologies in 1997, a pioneer in the use of RF badges for computer security. In 1981, he founded Symplex, a manufacturer of the first data compression networking devices. He also transitioned Symplex from a startup to a profitable company and positioned it for an initial public offering.

Based on his contributions to the business community, Brostoff was selected as Entrepreneur of the Year by the Detroit Free Press and Michigan Technology Council. He also received the Governor of Michigan Certificate of Appreciation for Outstanding Business Achievement. He is also a graduate of the University of Michigan.

George Brostoff - Sensible Vision CEO

Q. How exactly does SensibleVision’s technology work, how accurate is it, and how does it fit in the current smartphone ecosystem?

A. With 12 years of “in the field” refinement, SensibleVision uses AI and neural networks for both face detection and face recognition. Unlike face recognition engines where there is the classic trade-off between False Acceptances Rate (FAR) to False Rejections Rate (FRR), our multi-patented approach was specifically developed for device authentication and has a FAR of one in 20 million and FRR approaching 0.

Our patented auto learning techniques are key to providing a transparent user experience that virtually eliminates False Rejections. SensibleVision technologies allow for consistent performance with standard RGB cameras with 2D recognition.

Our 3D face recognition solutions first shipped commercially in 2013 as a result of a strategic partnership with Intel, making spoofing virtually impossible. One of the keys is that our solution works well in all lighting situations, from the brightest, outdoor sunlight (a problem for some 3D solutions) as well as in settings with no light.

But our technologies go far beyond using the face to unlock a phone. Apple has made a nice start by allowing users to transparently unlock the iPhone X with FaceID, providing a strong anti-spoofing solution. But in the end, it’s simply an alternative to TouchID. By going way beyond fingerprints and passwords, SensibleVision’s technology and patents provide everything Apple offers, plus functionality that Apple has missed

  1. Personalisation – the ability to automatically manage multiple face profiles. Hand your phone to your child and only their apps appear.
  2. Continuous Security – our solution knows when the authorised user is no longer present and will instantly secure the phone and its data.
  3. More secure and better user experience – delivers faster and smoother unlock than Apple’s “Gaze Detection.”
  4. Fastest multi-factor security – banks and many other Enterprise-level applications specify and rely on multi-factor security. SensibleVision’s patented approach combines the scan of the face with a second identifier. The user has many options including using a fingerprint sensor, drawing a gesture, touching a secret shape, which when combined with the 3D face recognition, provides a virtually spoof-proof authentication process.
Why this is important:

How people use their smartphones and tablets today is fundamentality different from how they traditionally interact with PCs. We access our computers perhaps 5-20 times a day. It’s common for most of us to interact with our mobile devices ten times as frequently – as much as 50 to 100 times a day or more. Equally important is the fact that each “session” often lasts just a few seconds rather than 30-60 minutes which is typical when we are sitting at our PCs. 3D face authentication is perfectly aligned with the way we use our devices in the current workplace.

In addition, using passwords or their alternatives, such as fingerprints, solutions which are not transparent or continuous, frustrates the user and also leaves the device at risk. The key takeaways about SensibleVision’s innovative simultaneous multi-factor authentication is that it is:

  1. Fast – <100ms
  2. Transparent – automatic, nothing for the user to do
  3. Secure – no false acceptances or photo/video spoofing
  4. Continuous after the unlock – full time security protection
  5. Personalised – adapts the device to the current user

Q. When will your solution hit Android smartphones?

A. Fall 2018

Q. What do you think of OnePlus 5T‘s Face Unlock feature? It’s already received critical acclaim for being practical. How do you strike the balance between face recognition implementations that are either fast or secure?

A. This is nothing new. We have been doing this for years. It even appears that they may be using several patented technologies. The key question is how does it work under difficult lighting and with dark skinned people?

For example, the Forbes demo video is under ideal operating conditions (office lighting). The open question is: what happens in low light, back light and in direct sunlight?

The research would suggest that people are more inclined to use a solution that is fast and easy rather than focusing on security. That said, there are certain interactions where security is critically important such as when conducting financial transactions or sharing personal data. SensibleVision’s solution delivers on both requirements – it is fast, transparent and secure.

Q. Regarding the OnePlus 5T, you mentioned that “it even appears that they may be using several patented technologies” What did you mean by that? Are they using a patented technology without paying for it?

A. We have not licensed our patents to OnePlus or their supplier. From the video on Forbes, they are likely infringing at the very least on our illumination patent. We have a legal firm that handles these type of matters, so I cannot comment more specifically. The benefits we bring to the OEMs in this area include:

1. A strong patent portfolio that can offer better performance
2. Excellent protection from the threat of infringement from other groups as we have been shipping in the market since 2006.

For more on the developing story, click here to read our independent report.

Q. What do you make of the Vietnamese security firm claiming to defeat Face ID? The Apple’s biometric hardware was supposed to be water tight. How important is software when it comes to defeating hackers?

A. It’s a valid concern. Anything created by a human can be broken by a human. I expect that the real weakness for the iPhone X spoofing will be in bright sunlight where it will be the easiest to defeat. Bkav has been testing face authentication solutions for years and has defeated every vendor they have tested. Our previous press releases have noted they have not included our solutions in their tests.

I am not saying we can’t be defeated, but I do suggest our techniques and recommendations – like fast and simple simultaneous multi-factor security – really covers a lot of critical bases.

The average smartphone user isn’t as concerned with security as CEOs, high ranking government officials, or other high net worth individuals. Since that holds true for the majority of smartphone users, do you think the more expensive and cumbersome face recognition-based biometric security implementations could replace simpler and more intuitive ones such as fingerprint sensors?

I would suggest a well-done face authentication solution is faster, simpler and more intuitive than a fingerprint. It can be not only secure, but fully transparent to the user. Simplicity and user transparency are key factors to user acceptance and compliance. The cost reason for 3D cameras is not just security, but the active lighting. This allows a properly developed solution to work in side, back and no light conditions.

Q. Since you’ve told us that SensibleVision’s technology and patents offer personalisation and the fastest multi-factor security – something that’s missing in Apple’s FaceID – we would like to know if and how Google plans to support these features in the current or future versions of Android, considering how software implementation could be a bit of a problem in the Android ecosystem?

A. We can’t comment on Google’s plans. However, Android OEMs are looking to provide equal and greater functionality than what Apple supports. Therefore, they are looking to include these innovations which improve both security and the user experience.

Q. Where is the face information saved and how is it kept secure?

A. It depends on the OEM. At minimum, all the OEM’s we are working with or in discussions with are keeping all the face biometric templates in the Trust Zone. This is equal to Apple’s method for securing the biometric templates. Where the phone hardware supports it, both the camera interface, our authentication technology and the templates are in the trust zone.

Q. What is your view on the backdoors that some brands keep open on their devices (for whatever reason)?

A. As a security professional, I feel this is very bad. We always recommend our solutions and the solutions of our partners that follow best practices, period. That said, this is their decision and the decision of their customers who buy these unsecure products.

In a similar fashion, we strongly discourage our partners who consider 2D recognition to not claim any anti-spoofing technology.

Q. How exactly does SensibleVision’s technology work without any kind of depth sensor (since you’ve mentioned that it can work consistently with standard RGB cameras) and even without light (or even very bright light)?

A. Great question. We had a patent issued years ago covering the use of the screen as an illuminator and automatically turning up the screen brightness temporarily. We have notified several companies about their infringement on our patent. In practice this technique works really well.

For bright sunlight, we look at multiple regions of the face so we can extract detail where it is present.

Q. What kind of devices do you see SensibleVision’s technology being used in the future?

A. 3DVerify can be used on smartphones, tablets and laptops. We are already working with groups putting cameras into intelligent speakers (like Alexa). I see this happening much sooner than other form factors like watches. Also, automotive identification and authentication within the cabin is a really big area.

Q. Which (and how many) brands are currently working with your company and how satisfied are they with your solutions? How easy or hard is it for companies to implement your technology in their upcoming products or prototypes?

A. We can’t disclose the names of the companies, but they include Tier 1 and large Tier 2 companies as well as some smaller but significant Chinese vendors. The implementation ease varies by the Trust Zone requirements. Otherwise, it’s a matter of optimising our solution to their chosen 3D camera. From our end, 3-4 months to production is how long it is currently taking. As our solutions are tied to hardware, the actual device release dates are further out.

Q. Is your company working with specific camera hardware manufacturers for facial recognition?

A. Yes, we can’t disclose who we are working with other than Infineon and Mantis (and their module makers). Other sensors and camera vendors can’t currently be disclosed publicly. These solutions include both Time of Flight and Structured Light.

Q. Since SensibleVision uses AI and Neural Networks for face detection and recognition, what kind of performance improvement do you think processors with dedicated cores for machine learning or AI will make when compared to the ones lacking such specialised cores?

A. We agree that using dedicated AI chips will improve performance and hence adoption, following on the model of GPUs moving graphics-intensive tasks off the main CPU with the following caveat. The face authentication use case for standard phones and IoT devices is not overly taxing. There are only a few users to be recognised (now Apple is only doing one!). This reduces the complexity so that the most advanced neural networks and other AI techniques are not necessarily required to deliver this functionality. This can save cost and allow the use of less expensive hardware without compromising performance.

Of course, having the hardware allows us to use our most advanced techniques, but for current smartphones use cases, this is overkill. That said, our neural networks for anti-spoofing and IR reflectance truly benefit from the additional hardware. In short, for today’s use cases, advanced hardware is not required, but Apple with its chipset has set a marketing bar that almost requires the use of special routines and hardware at least on high end phones.

Q. Is SensibleVision’s technology compatible with Microsoft’s Windows Hello system so that it can be used in Windows-based devices?

A. We have been shipping a Windows compatible solution since 2006 (we even supported XP and support Win 10 today). We don’t currently integrate into Windows Hello as it does not support key critical features such as Continuous Security or Simultaneous Multifactor.

Q. What do you think about the reliability and ease-of-use of other types of biometric authentication systems such as iris or retinal scanning, something that Samsung has been using in its smartphones?

A. A solution has to be both secure and convenient (virtually transparent to the user) for long term acceptance. The anti-spoofing must not delay authentication and must also be truly difficult to work around. This has been a historic challenge for all biometrics. This is where the 3D cameras come into play for face authentication – we can be highly resistant to spoofing attacks yet not introduce a delay. In our case, we use multiple techniques including depth contours, dimensional perspective and IR reflectance among others.

For the user experience, it must be as close to no security as possible. I have always been skeptical of Samsung’s eye scans as they require a user to align their eyes in a box. If you are a government agency and can force your employees to use a solution, this type of non-transparent security can work. For most business and virtually all consumers, this is way too difficult a process to achieve adoption.

Q. What do you make of inherently insecure face recognition-based biometric security systems are touted as the best thing since sliced bread just because the biometric security implementations are relatively faster or easier to implement or use?

A. Our perspective is that. As read in my last answer, we feel that ease of use and security are the two top drivers for adoption of a face recognition solution. We have been dealing with the poor performance and/or poor security for years, often from major vendors such as Google and Samsung. This has hampered our efforts. Until early 2017, most smartphone OEMs would not even consider face recognition for serious security. We are thankful that Apple entered the market as this took the “concerns” away from the smartphone OEMs. We encourage all our partners to at least place the biometric templates into the trust zone.

Q. Do you think normal users can be taught the advantages of features like multi-factor security, something that SensibleVision’s solutions provide?

A. Yes, we have seen users adopt this solution quite easily in a variety of settings. Once they understand the process and see the value it delivers in terms of quick and secure access, they are fans.

Buying options

Asif heads the editorial team at MySmartPrice. He has more than six years of experience in reporting consumer technology, and has been quoted in various esteemed publications, including TheVerge, TWiT, and SamMobile. Asif has immense interest in CPUs, GPUs, mobile chipsets, camera sensors, and apps. Asif can be tagged as one of the most patient team members as we often pick his brains when it comes to learn about complex tech topics like camera sensors.