This week, Facebook has announced several new AI-powered art filters that are sure to keep users occupied. These filters will soon come to the Facebook mobile application and when they do, users will be able to overlay photos and video—and even live broadcasts—with artistic styles.
For example, you can take a picture of a cityscape and then have it rendered as though it was painted by Rembrandt.
Facebook calls this new AI feature “style transfer” and, in fact, it is nothing new. The Russian mobile app Prisma originally popularized this style earlier this year and Facebook originally showcased a prototype version—working with live video—in October. So far, though, the feature is only available in Facebook’s main app in Ireland with no due date, yet, for the United States (just that it is coming “soon”).
But while the feature—which also connotes Facebook’s interest in adding more camera features to its mobile app functionality—is certainly fun to play with, it is actually indicative of far more complex software; and what Facebook could be capable of in the future.
This feature—along with many AI applications—utilize deep neural networks. These systems are actually a class of computer programs which intend to mimic the actions of neurons in the human brain. These neurons can be trained to recognize common patterns after observing massive amounts of data. Deep neural networks can be used for just about anything: from voice recognition in digital assistants (which we have seen for awhile now) to helping you categorize your holiday photos via Google Photos.
These programs are complicated and they require massive processing power. Typically, programs that utilize this kind of AI require connection to the internet because they send your request to a distant server farm to do the whole computation and then transmit it back to you. Somehow, though, Facebook has managed to make these new filters available using only the neural networks locally in your phone.
So how do they do it?
Facebook has created a completely new deep learning architecture, which they call “Caffe2go.” In brief, the software implements 20 frames per second using only your smartphone’s hardware. More importantly, though, Facebook CTO Mike Schroepfer explains that this is only the beginning. He comments that using Caffe2go “We can recognize gesture-based controls, where the computer can see where you’re pointing and activate different styles or commands. We can recognize facial expressions and perform related actions, like putting a ‘yay’ filter over your selfie when you smile.”