Virtual Try-On Solution

Client

The client, a well-known eyewear brand, operates a wide network of retail stores and offers high-quality, stylish optical products, including prescription glasses and sunglasses.

Business Challenge

The client has built a best-in-class Virtual Try-On (VTO) solution and wanted to make it even better by allowing customers (many of whom need their current glasses to see) to keep their real glasses on while shopping with VTO. The challenge was to develop a mobile application, initially for iOS, capable of erasing real glasses in real-time and replacing them with augmented reality (AR) glasses in a live video feed.

This idea, once deemed impossible due to technological constraints before 2021, became achievable through collaboration with Coherent Solutions.

Solution

Coherent Solutions collaborated closely with the client, leading to the development of a revolutionary app that enables real-time removal of glasses from a user’s face on a mobile device. Key achievements include its ability to operate in real-time, adapt to diverse lighting conditions, and effectively track and erase glasses of different shapes, colors, and sizes.

To bring this innovation to life, the team executed several crucial steps:

• Neural Network Research: Conducted research to identify optimal neural network architectures.
• Comprehensive Dataset: Created a dataset with ~10,000 annotated images for semantic segmentation and object detection and creatively applied current VTO assets to enhance training sets and testing models.
• Advanced Training Pipelines: Built training pipelines for chosen neural network models.
• Model Deployment: Deployed various models, including eyeglasses tracking, eyeglasses frame semantic segmentation, and inpainting to remove glasses.
• Mobile App Development: Designed and built a mobile application to integrate and showcase these models, ensuring real-time performance on mobile devices (30 FPS).
• User testing and refinement: Responded to feedback in customer studies to minimize artifacts, while increasing processing efficiency.

The app is currently in the final stages of preparation for its production launch on iOS.

Results

The collaboration between the client and Coherent Solutions began with an ambitious goal: to enhance a Virtual Try-On solution with advanced features that initially seemed beyond reach. However, the Coherent Solutions team’s expertise and dedication transformed this ambitious vision into reality. The result is one of the most technologically advanced apps in the industry that has already significantly boosted the client’s brand recognition and driven sales by attracting a broader audience to their products.