Development of methods for determining the anthropometric data of the human body based on real images using machine learning

Project leader: Azat Absadyk

Development and optimization of neural networks for accurate extraction of anthropometric data from images, in order to minimize the number of returns of clothing in online stores due to size mismatches, as well as reduce the associated environmental impacts.

• Definition of an algorithm for building a three-dimensional model of the human body from photographs using machine learning.
• Development of a synthetic dataset for determining the silhouette of a person from a photograph, taking into account clothing
• Building a semantic segmentation model to determine the shape of a person’s silhouette from a photograph.
• Development of a three-dimensional model of the human body with variable body parts using the BlendShape technology.

• Determining the silhouette of a person from real photos from the front and side.
• Determination of key points of human joints from real photographs from the front and side.
• Machine learning plugins for web service.
• Determination of anthropometric characteristics from real photographs.
• Building a three-dimensional virtual model of the client based on real photos.
• Web service for determining the anthropometric characteristics of a person from photographs.

1. Collecting data for training neural networks.
2. Development of an algorithm for determining the silhouette of a person based on real photos using machine learning.
3. Development of an algorithm for determining the key points of the body joints based on real photos using machine learning.
4. Creation of a three-dimensional template of the human body, modified for anthropometric data.
5. Development of a web application for determining anthropometric data from real photographs.

A dataset was created using the new NVIDIA Omniverse technology. With this technology, you can generate synthetic data with a very realistic picture and the ability to get different scenarios, which reduces manual work. Synthetically obtained a data set of more than 14,000 images. Training on this dataset showed the applicability of the model to real photos.

To determine the appropriate model, the main architectures of convolutional neural networks were modified and tested with pre-trained semantic segmentation models as an encoder. For training, a synthetically created data set was used. Out of 16 models, 1 model showed good results and applicability to real drawings.

Absadyk A. M.T.Sc,

Web of Science ResearcherID: IXN-3525-2023

ORCID – 0009-0003-6566-8693

Scopus ID – 57480517400

Funding amount: 18 979 301,5