Building automation system
The company of our client belongs to the construction industry leaders in Japan. The main focus of the company is automated reporting software.
AI-powered indoor positioning system
The aim of the project is to create a custom Indoor Positioning System (IPS) to have information about a specific place and at a particular time at a construction site. One of the client's main requirements is to build a solution avoiding the common GPS technology. Therefore, we are using Computer Vision, accelerometer, and gyroscope sensor data for more accurate navigation inside the building.
Project background
By definition, an Indoor Positioning System (IPS) is a network of devices used to locate people or objects where GPS and other satellite technologies lack precision or fail entirely, such as inside multistory buildings, airports, alleys, parking garages, and underground locations. Thus, the most popular solution is one with GPS.
Indoor Positioning System’s functionality is applicable not only in construction. The scope of this solution covers:
– banks
– logistics
– robotic production and more.
– smart home with technology to support the disabled (Ambient assisted living, AAL)
– location-based services (LBS)
– medical service
– environmental monitoring
– rescue services
Our client, a construction leader in Japan, was looking for a solution to have information about what was filmed in a specific place and time at his construction site. They wanted to see what this or that place in the room looked like, let’s say, some years back, and analyze what has changed.
However, it did not suit our client’s needs. We were tasked to create a custom solution, an indoor trajectory determination system using Computer Vision, accelerometer, and gyroscope sensor data without GPS.
- Duration: April 2022 – Ongoing
- Location: Japan
- Industry: Construction
- Services:
- POC development, Back-end development, Custom software development
Business needs
The Indoor Positioning System or IPS is a local system for finding a location inside buildings and closed structures, where a satellite navigation system is practically inaccessible.
Our client wants to have information about a specific place at a specific time at his construction site. They need to see what this or that place in the room looked like, for example, a year ago, and analyze what has changed.
IPS functionality can be useful not only in the construction industry. The full scope of this solution includes:
– location-based services (LBS)
– smart home with technology to support the disabled (Ambient assisted living, AAL)
– medical service
– environmental monitoring
– rescue services
– museums
– banks
– logistics
– robotic production
– underground construction
Product features
- Video processing. The user uploads a video to the server.
- Room layout. After loading and processing the video, the trajectory where this video was filmed is displayed on the room layout (the entire route is displayed)
- Video frames. The user can select a point on the diagram and see the corresponding video frames
- Date and time. The user can set the date/time for the “point” on the map and if there is another video in the database
- Room view mode. The user can switch to the room view mode for the specified date
Solution
As input, CHI Software team has a video of a construction site inside the building. The video is shot on a 360-degree camera that shoots in all directions. The camera has sensors such as an accelerometer and a gyroscope.
– One of our tasks is to restore the trajectory and display it on the map of the room for which the video was filmed, using data from the sensors of the 360-degree camera.
– To correct the trajectory obtained with the help of sensors, special markers are placed on the walls of the room. One more task is to detect and recognize these markers and apply the information received from the markers to correct the trajectory. This allows us to achieve fairly accurate results.
– Obtaining a trajectory from sensor data is not an easy task. Sensors have errors and accumulate them. Our AI/ML team is doing a lot of work to eliminate these errors, filtering data and smoothing out the noise.
– At the current stage, we added video data to sensor data, namely ArUco markers, which were selected for the current task and generated in a special way.
Our technology stack
- Python
- OpenCV
- Pandas
- NumPy
- SciPy
- scikit-kinematics
- AWS S3
- AWS Lambda
- AWS ECS
Employee testimonial
Yuriy Chaban ML Engineer
The main goal of the indoor location project was to find a trajectory of movements without GPS. This is useful for navigation inside the buildings. For this purpose we use data from inertial sensors like the gravitometer, accelerometer and magnetometer in the hands of a walking person. It was a challenge because there was a very strong noise from steps and unpredictable movements of the hands and the device with sensors while walking, which did not allow us to use the known algorithms. But we managed to build an algorithm for calculating a trajectory with sufficient accuracy, as well as an algorithm for correcting the trajectory by specifying control points through which it should pass.