AR Mobile App for Vivienne Sabo

We developed a mobile application for choosing, testing, and purchasing cosmetics using an AR camera in real-time.

The Vivienne Sabo — cosmetics brand is available in 15 countries, including Russia, Belarus, Ukraine, and Kazakhstan. Their Cabaret mascara won the hearts of millions of women. The brand works with beauty bloggers and influencers and is constantly looking for new and creative approaches to client interaction.

Problems

An ordinary store and mobile version of Vivienne Sabo website didn’t cover all marketing tasks, so the goal was not just to create a convenient way to buy cosmetics for current customers, but also to attract new ones, due to the popular AR-mechanics.

To improve interaction with the application, we developed 2 custom scenarios:

1. The user is already a brand-savvy customer of the company. Our focus was on the advantages of buying through the app. In this case, we focused on the purchase advantages of our app: promotions, personal recommendations, discounts, and we lead them to the purchase.

2. Users not familiar with the brand and/or its products encounter an enticing AR feature which generates interest and excitement.

AR Mode and Icons

We developed a simple and clear interface for the AR mode mainly focusing on visual images.

To keep the interface simple, we avoided text and focused on icons. We chose friendly, intuitive icons understandable to every woman.

Development and Technologies

The client already had a Magento web version of the store. We developed a requirements specification for the store’s developers. When they were done, we integrated an API into our application. Now both web and mobile clients are managed from one administrative panel.

For data collection and output we used Firebase + BigQuery +DataStudio.

Developing Textures

It was important for the client that cosmetics were virtualized as precisely as possible and looked natural. Our guiding principle was the question: “Would a woman share this photo on social media?”

The texturing stage was not easy. To best convey qualities like gloss, matte, glitter, and shine, textures had to be drawn with complicated overflows and glitter.

For lip gloss we prepared a separate glare texture overlaid on other products. For eye products, every texture was drawn as an illustration. We sketched more than 30 options for each category until we found the ideal one.

Balance of Technologies and Makeup

While working on the AR component, we had to find a balance between the cost of implementation and the quality of facial recognition and overlay. The best solution was to license a third-party framework, in this case, dlib + OpenCV.

While developing the larger textures for facial powders and concealers, we were faced with the problem of limited library capacity. We could not reduce the quality of the textures, so we imposed some limitations by dividing devices into three groups:

1. Devices on which the overlay would be available only in photos.
2. Devices on which the AR would be available, but without “big textures” (face powder, color tone).
3. Devices on which everything is available.

We did several additional experiments with different technologies and found a solution using OpenGL for image rendering that increased productivity tenfold. We plan to implement it in future releases.