Wildbytes leverages ftrack

Wildbytes pioneers live facial projection mapping with ftrack

The brief

We were tasked with helping ftrack to sell its creative project management solution by talking to the customers that were using it on the front lines. We interviewed Interactive Experiential Agency, Wildbytes, about their use of ftrack in concocting and executing the first-ever live facial projection mapping performance in association with the launch of Kat Von D’s latest makeup collab with Sephora.


The story

Wildbytes recently leveraged the power of ftrack to tackle a project that blended technology with artistry; the launch show of a new beauty line from Sephora and tattoo artist, Kat Von D. Concocting and executing the first-ever live facial projection mapping performance, they were able to showcase stunning images which were displayed across Kat Von D’s face as she unveiled the line on a stage in Spain. ftrack was a crucial ally in bringing the performance to life.

Sephora approached Wildbytes about the Kat Von D Beauty line, asking the agency to pitch one of its wild ideas to drive huge attention to the debut. Wildbytes, in turn, suggested the concept of a live facial projection mapping performance to anchor the campaign, using Von D’s own face to display memorable and truly beautiful ideas. “Sephora was in love with the idea, and even more importantly, they were bold enough to take the risk and jump into such an ambitious project," says Julio Obelleiro, co-founder and managing partner.

Wildbytes began with two 3D scans of Von D, capturing her likeness using structured light and photogrammetry to develop an incredibly realistic 3D model, complete with skin effects. From there, Wildbytes 3D printed a bust of her face, using it during testing and development of the projection mapping effects, albeit without the added challenge of a live, moving face. Following that, the team needed to launch into the colourful, kaleidoscopic imagery that would be projected.

Obelleiro’s team worked on the campaign for four months. An entire new pipeline had to be created to power the performance, including the facial tracking, with five cameras following Von D’s every movement at 240 frames per second. “We developed a framework that included the real-time face tracking system and a 3D real-time rendering engine, which projected the created content onto the face and adapted to its movements in real-time,” says Obelleiro. “Tons of computer vision and rendering algorithms were needed to achieve the final results and get the system to work well.”

ftrack proved to be a vital tool across the Sephora project. “Even before we launched into actual production, ftrack was having a huge impact in the early R&D phase,” he explains. “ftrack’s focus on the visual side of things was key for us. Its toolset also helped significantly in streamlining the production timeline and the review process. And this was a very fast-paced project, so the impact was huge.”

“Visual project management tools are a must for production studios, and honestly, the decision is not easy if you haven’t used any software of this sort before,” he adds. “But after having used ftrack for such challenging project and achieved such success, we would definitely recommend it.”

"Visual project management tools are a must for production studios, and honestly, the decision is not easy if you haven’t used any software of this sort before. But after having used ftrack for such challenging project and achieved such success, we would definitely recommend it."

Obelleiro says the team fully intends to use ftrack again in future project, and there could be many more of them on the way: Wildbytes is further refining the facial projection mapping tech to be even more dynamic, and they hope to push into the music industry, as well as working on further augmented reality experiences, smart mirror installations for retail stores, and a variety of other out-of-the-box and immersive ideas that stick with people long after the moment of contact.

Interested in working with us?

We’d love to hear from you.