Computer Vision solution for a cosmetic retailer with Recommendation System

Project background

Not so long ago the sale of cosmetic products depended mostly on the physical store experience: the customer needed to try out different shades or beauty products before they could find the right one that suits them best.

Retailers now have to meet the growing expectations of the customers for engaging experiences in stores. That’s why the consumers’ experience in beauty stores is now greatly transformed by technology, namely by diverse mobile opportunities: starting with chatbots and finishing with AI-powered computer vision solutions for retail to pick up needed products fast.   

 Our client, a cosmetic retailer, has a variety of beauty items, including products for all skin types, under-eye care, and decorative cosmetics. They were looking for innovative AI mobile solutions that could analyze the customer’s face and type of skin, and then offer the best-suited product and, thus, improve their customers’ experience.  

Business needs

The client’s beauty company has a wide range of beauty products and customers in the United States. The company needed to train a large staff of consultants to serve everyone. Therefore, to optimize processes and minimize human labor, an idea of an app appeared. This solution should help end customers choose the right cosmetics themselves. Thanks to this AI-powered computer vision solution for retail the number of dissatisfied customers should decrease, and the costs of hiring and training new consultants should be optimized.

Product features

1. Face analysis. Analyzes collected user data received from a photo (selfie)
2. Recommendation Based on the collected data, this computer vision solution for retail makes recommendations on available beauty products
3. Helpful hints. Offers hints and tips for the buyers to simplify the selection of items in the catalog

Solution

CHI Software developed a mobile application based on Computer Vision and photo face recognition, highlighting their expertise in computer vision development services.

In this computer vision case study we want to show how it works:

– A user takes a selfie

– Using the detectors based on the histograms of forfeited gradients from the Dlib library, the program can accurately determine the user’s face on the photo

– Using the same approach, we were able to train a system that can detect 126 feature points on the user’s face

– The app then analyzes the unique facial features of a user and adopts the user profile

– The applied AI technologies then help select the needed skincare

– Our system allows to perform a detailed/smart analysis of the user’s face

– The app now provides personalized skincare and beauty recommendations with the help of ChatGPT API  integration.

Our technology stack

• C++
• Open CV
• Dlib
• Caffe
• Swift 4.2
• Alamofire
• REST
• AVFoundation
• CoreGraphics
• CoreImage
• CoreAnimation
• Multithreading
• UserDefaults
• Firebase analytics
• SafariServices (API)
• PHP 7.2.12
• MySQL
• MariaDB (AI)
• AWS EC2, S3, RDS

Key achievements delivered

The client’s new mobile-based application helps users not only to navigate better in the range of items but also to buy cosmetics according to skin type and preferences. The selection of cosmetic products has become a simple and convenient automated process that increases consumers’ loyalty.