Image recognition service

Project background

In Japan, many people send their children between the ages of 0 and 5 to preschool. Children are there on weekdays from morning until 3-6 pm, while their parents work. Parents want to be aware of how their children spend time at preschool.

The conditions for children at preschool institutions are monitored through regular reporting established by these institutions.    

Still, some parents want more than mere formalized reporting. Photos of children at preschool are a good way to confirm a child’s condition. The goal of this image recognition software development  project was to help educators to use face recognition technology in providing visual information about a child’s condition.

The biggest challenge was to design and apply neural networks that work on mobile devices with good accuracy and performance.

Business needs

In Japan, many parents send their children between the ages of 0 and 5 to preschool. Children are there on weekdays from morning until 3-6 pm, while their parents work. Parents want to be aware of how their children spend time in preschool. The preschool managers are eager to cover parents’ needs to be sure that everything is fine with their kids. The conditions for children in preschool institutions are monitored through regular reporting established by the institutions.  

Still, parents want more than mere formalized reporting. Photos with children in preschool are a good way to confirm a child’s condition. The biggest challenge was to design and apply neural networks that work on mobile devices with good accuracy and performance. 

The goal of this image recognition software development project was to help educators to use face recognition technologies in the following:   

1. Simplify the interaction of educators  with  the recipients (parents) of the child depicted in the photo.

2. Display the previous number of photos and the photos of each child sent previously .

3. During the shooting, mark the children whose photo it is forbidden to take.

Product features

Within the proof of concept framework, two main procedures were implemented: 

1. Procedure for registering a child in the app is the following. A teacher can:

– Take a photo from the camera;  

– Take a photo from the gallery;  

– Label a user (a child): First name  and Last name  

– Save the labeling;  

– Cancel the labeling process;  

– See the list of users. 

2. Photo processing procedure allows a teacher to:

– Take a photo from the camera (points the camera at the user);  

– Take a photo from the gallery;  

– See the labeling near the user (labels taken from the local database);  

– Take pictures of the user;  

– Cancel if there are no required users (children) for reporting.

The main peculiarity of this image recognition software development project is that all the logic of the system was implemented on the mobile device, without a back-end part. 

Solution

The CHI Software Development team was involved in this project from the very beginning, commencing with a product discovery phase and creating detailed documentation on how to develop an image recognition app for the Japanese market.

We solved the following issues: 

– Registration of children (the ability to take a photo of one child and save it to the database on the device);

– Processing of the faces in the photo in real-time;

– Thanks to computer vision development services, comparing the detected face images with the face images in the database and labeling the found faces.  

– We had limitations on the accuracy of the F1 measure not lower than 85% and the speed of the system not exceeding 3/4 of a second. 

Before actual image recognition software development, we conducted research, tried, and tested more than 10 neural networks for obtaining landmarks and face embeddings. Most of them had limitations that didn’t allow us to run them locally on the device. But at the end of the day, we succeeded.

The peculiarity of the application is that all the logic of the system was implemented on the mobile device, without a back-end part. 

Our technology stack

• PyTorch mobile
• Tensorflow-lite-vision
• Tensorflow-lite-meta
• Kotlin-stdlib
• yolov5n
• mobileNetV3_face_land
• moibleFaceNet_m1sm

Client values

1. Using facial recognition technologies, the CHI Software team helped preschool workers to produce fast reports for parents.
2. In the POC phase, we developed functionality for registering children and searching for faces in real-time.
3. We managed to achieve the processing and recognition of 5 faces at the same time and the speed of 3/4 second in real-time.