(by Pao - Charubutr Asavaroengchai, Data Scientist at Omise)

Through cloud-base data storage and deep learning, Omise FacePay is here to revolutionize the traditional retail store by enabling an all-in-one solution that offers next level customer experiences. Omise FacePay takes on an AI approach to provide seamless customer recognition in stores.

The technology behind

Artificial Intelligence (AI) is among the game changing technologies that is sure to impact global economy, the employment market and the way we live our daily lives. Through the years, AI has gone through incredible boost, enhancing its ability in doing new things and doing them better and better. This is primarily due to the advancement of Deep Learning neural network.

Photo Caption: Deep Learning takes inspiration from the brain in a way that it is composed of many layers of neurons, all connected to every other neuron in the next layer.

With an effective learning algorithm and boost in computational power due primarily to usage of parallel processing of graphics processing units (GPU), deep learning has been able to make incredible leaps in one of the grand quests of computer science, image recognition.

Unlike other tasks that could be solved by writing explicit instructions, image recognition is infinitely more complicated as there are variations of a single object; shapes, sizes, colors and orientations. Before Deep learning was implemented, a large team of image recognition specialists were needed to build many filters for different shapes, sizes, colors and orientations to anticipate what an object may look like, and would only achieve moderate accuracies. With Deep learning, the challenge in image recognition has been “solved”, in one of the main open dataset called IMAGENET. Deep learning algorithm has been able to achieve super-human results by learning from over 1.2 million images of objects from 1000 categories.

At the moment of this writing, billions of dollars has been put into application of Deep learning to create self-driving cars which can understand its environment, detect lanes, cars, bicycles, traffic signs and other things to take the correct anticipatory actions for safe driving. Medicine is also about to be disrupted, Deep learning algorithm has been able to make diagnostic at par or better than expert physicians at many tasks, such as diagnosing diabetic retinopathy, skin cancer and breast cancer.

Face recognition, once considered a “solved” problem, went through a massive boost in accuracy through application of Deep learning algorithms.

Image recognition in retail

Physical stores have now reached saturation point, and in some cases, have declined in recent years due primarily to the rise of e-commerce businesses. In addition to accessibility, e-commerce is offering consumers a higher level of personalization and convenience. Consider this example. When you visit amazon.com you get a personalized page and product recommendations that are based on your preferences and history, plus you can pay for your purchase through just one click. On the other hand, visit a large chain store and you get treated like a number. You get an extra loyalty card to add to the dozen already in your wallet just to get the discounts, and to pay? that’s an extra card or some cash.

Omise FacePay can be easily integrated to platforms with decent cameras and internet connection. Customers visiting a FacePay enabled store will be greeted by their names and any product offering can be personalized to their likings. The Omise FacePay system is always learning. This means that as more data gets accumulated, the system’s accuracy gets better and better.

Photo Caption: FacePay cloud platform API manages secured database and image data storage

User Identification

Omise FacePay enables fast identification of users from a pool of registered profiles. Each user will have one or more photo(s) of them taken for identification. However, only one photo is required to be sent into FacePay API from local devices (such as an Android phone or IOT cameras). The FacePay API will return the most likely match from the dataset of registered users, or no match found. FacePay identification system is always learning, whenever a match is not found, the latest photo with confirmed identity through secondary mean (such as phone number) will be added to the user’s identity gallery.

Verification

Omise FacePay verification service does 1:1 user verification, for example, if a user has claimed to be an individual who owns a stated bank account, the user may have had a photo taken during time of registration. When the user comes to the bank again to make a withdraw, FacePay verification system will check whether it is the same person as one who originally register. Very much like hand writing verification but the process can be done seamlessly and in case of fraud, a photographic evidence is kept in the system.

How we started

The idea of Omise FacePay sprung when two of the Omise team members, Pao (Data Scientist) and Saroj (Developer) participated in the HUBBA Stadium:Startup Battle ‘Redesign Gas Station/Café Amazon Experience’ in late November last year. The event was co-hosted by Express Solutions, a new business unit under PTT Public Company Limited, the national oil and gas conglomerate (Thailand).

Omise FacePay is currently in prototype. Here's a few snaps from the soft launch in April 2017.

Photo Caption: (L-R) Charubutr Asavaroengchai (Data Scientist at Omise), Bhanubhak Nunbhakdi (Machine Learning Developer at Omise), Arweemas Sirisaengtaksin (Project Manager at Espress Solutions, PTT Plc), Saroj Saengphongumphai (Developer at Omise), Kaweewut Temphuwapat (Team Lead of Espress Solutions, PTT Pcl) and Pasarn Choatkaew (Business Development at Omise)

Photo Caption: Pao giving a talk on the technology behind Omise FacePay and how it can be integrated into different types of business in addition to retail

Photo Caption: Event attendees creating a customer profile with Saroj. They enter their name, input their beverage preference(s) and take a photo of themselves to create a profile.

Photo Caption: After registering a profile with Omise FacePay, entrants move on to the next station to test out the customer recognition on Omise FacePay.

Photo Caption: Customers with a registered profile are greeted by their name, and staff are able give beverage suggestions

Testing out Omise FacePay customer recognition at The Amazon's Embrace, PTT Plc Headquarters.