Machine learning meets art to reflect a broader community
Originally posted on Medium by Ripon DeLeon & Jason Wittenbach
Last month, we had the chance to immerse ourselves among Charlottesville, VA’s leading civic organizers, entrepreneurs, tech innovators, and futurist visionaries at the 2018 Tom Tom Founders Festival. As a co-sponsor of the Festival’s Applied Machine Learning Summit, our team at Capital One wanted to do more than just talk about machine learning — we sought to create an interactive experience showcasing a real-live example of the technology in an engaging, participatory way, while also highlighting some of the area’s amazing local artists.
A central theme of Tom Tom is “learn[ing] to see your world in a new way.” We took inspiration from that ethos to create a pop-up art gallery that would further explore the idea of viewing the world from a new perspective, powered by a machine learning technique called style transfer.
Enter our Style Transfer Gallery, where festival participants were invited to have their picture taken, which was then superimposed into the style and technique of each featured artist — Laura Wooten, Theodore Taylor, Shannon Wright, and Brandon Robertson — and projected among an array of digital screens. This created a real-time, visual tapestry of participants while also preserving a history of previous festival-goers and providing a unique lens into the community.
Under the hood, we used a machine learning technique called “neural style transfer.” This model takes inspiration from what we know about how the human brain processes visual information in order to teach the algorithm how to create images in the style of a particular work of art. Training for a single painting’s style takes hours, but once complete, the model can transform a new image into that style in a fraction of a second. This allowed us to process video collected from a camera in real-time, so participants could see themselves moving and interacting as if they were in the painting, before snapping a self-portrait to be added to the Gallery.
At Capital One, we believe that leveraging the power of machine learning can help us interact with our customers in a more natural, seamless, and accessible way — from how we’re able to look out for our customers and their finances, to our ability to continuously enhance their digital experiences — and the Style Transfer Gallery is a further embodiment of this sentiment. The gallery celebrated and elevated visitors as both artwork and artist, and it was critical for us to create a space that could present and treat these works accordingly.
Our hope is that through the collection of this artwork, a new window into the Tom Tom community emerged.