Here is the latest from Julia Computing

Parallel Neural Styles on Video Powered by Azure

17 Feb 2017 | Ranjan Anantharaman

The above video was generated with the help of an algorithm called Neural Styles. It is an algorithm based on neural networks that is used to learn artistic styles and happens to be commonly used in apps like Prisma to beautify images. The algorithm extracts features from a “style image” and then applies it to a “content image”. Since a video is basically a collection of frames, we were able to apply our algorithm to every frame. This is the original video which we took as input to our model:

In a previous blog post, we described the model in neural styles and outlined the training process. We have now scaled our algorithm from individual images to video, using Julia’s built-in parallel primitives that make it trivially easy it is to scale algorithms to multiple nodes.

We ran the transformation on an Azure Data Science VM. The Azure DSVM is a pre-built image with many data science libraries included. It contains an installation of JuliaPro, and includes over a 100 major Julia packages. It is by far the easiest way to get started with running Julia code on the Azure ecosystem.

Our VM came with an Intel(R) Xeon(R) CPU E5-2660 with 8 physical cores and 56 GB RAM. We first extracted the frames from the video with VideoIO.jl, and then processed them in parallel with 8 Julia worker processes. Julia has several high level primitives for distributed computing, and we used the parallel map pmap function to split the images amongst the workers and subsequently texturize them. For larger videos, exactly the same code would be used to run this on a cluster of multiple physical or virtual servers.

@everywhere function f(a)
        texturize(a, "fire", "style")

pmap(x -> f(x), frames_from_video)

After the parallel processing, we stitched the images back together via ffmpeg to form a video.

The first video of this article was the output of passing the original video through a model we call “fire”, due to the fiery texture that was transferred. Here’s the result of passing through another model called “frost”:

Recent posts

Newsletter December 2018
04 Dec 2018 | Julia Computing
Researchers Use BioJulia to Develop a New Single-Cell RNA Sequencing Method
03 Dec 2018 | Julia Computing
Racefox Uses Julia to Provide Digital Sports Coaching
03 Dec 2018 | Julia Computing
Get the latest news about Julia delivered to your inbox.
Need help with Julia?
We provide products, training and consulting to make Julia successful in your organization. Email us: [email protected]
Contact us
Julia Computing, Inc. was founded with a mission to make Julia easy to use, easy to deploy and easy to scale. We operate out of Boston, London and Bangalore and we serve customers worldwide.
© 2015-2018 Julia Computing, Inc. All Rights Reserved.