Technology from the WUT used in the series BBC America
Comixify – initially an application for converting movies and photos into comics, now also the VFX Platform, supporting video post-production, created and developed by a team associated with the Faculty of Electronics and Information Technology of the Warsaw University of Technology - was used in the creation of "The Watch" series based on Terry Pratchetts’ books.
The "VFX Comixify platform” was used to convert recorded scenes into animations. A British studio, Lola Post Production, which deals with the post-production of the series, used the technology of style transfer, i.e. special filters that apply a specific comic-style image of animation to the recorded video. This technology takes advantage of the experience gained from the "previously released version". One only needs to upload a movie or photos and choose a style to be able to watch the animation generated in this way.
Original technology
-‘We have developed our own Style Transfer technology, based on machine- learning (Deep Learning Generative Adversarial Networks)’ - explains Paweł Andruszkiewicz - a co-founder of Comixify, a graduate of our Faculty of Electronics and Information Technology.
- ‘In this way, we are able to create a style inspired by a given artist based on representative painted or drawn works. We can then apply this style to any video sequence while maintaining the highest quality.’
The method used in Style Transfer is based on two different neural networks: A generator (current styling model) and a discriminator (auxiliary model, the purpose of which is to distinguish whether a given work was created by artificial intelligence or by a cartoonist).
- ‘Such a pair of neural networks, trained simultaneously, is the basis of the learning’ - emphasizes Paweł Andruszkiewicz. – ‘Additionally, to improve and significantly speed up the learning process, we use samples containing pairs of images: the original photo and its version copied by an artist.’
Stylizations in "The Watch"
The effects of the collaboration between Comixify and Lola Post Production can be seen in two episodes of "The Watch" - the fifth and the sixth. The style transfer was used, among others to the comic conversion of the wizard and styling the frames of the archival film watched by the characters.
What does it look like? Selected scenes can be seen on the Comixify blog
Lola Post Production, looking for cutting edge technologies that optimize the costs of video post-production, quickly appreciated the Style Transfer offered by Comixify. After talks with Lola Posta and BBC America, it was finally agreed that the technology would be used to modify the content in episodes of the first season of the series.
More than 25,000 frames were used for the needs of BBC America, which were restyled on the "VFX platform".
The producers took advantage of existing styles and added minor modifications to the final post-production process.
The work on the platform itself is extremely intuitive, and the ten-second material is converted into less than 120 seconds.
The “Watch” series is a fantasy police drama inspired by Terry Pratchett's book collection - “Disc World”, set in the fictional city of Ankh-Morpork. The first season has of 8 episodes. Each of them focuses on a different puzzle solved by the Municipal Police, led by Samuel Vimes.
The premiere of the series took place on January 3, 2021.
The Comixify team consists of:
* Tomasz Trzciński, DSc, PhD - the CEO (Assistant Professor at the Faculty of Electronics and Information Technology at WUT, the Institute of Computer Science, the Division of Computer Graphics),
* Maciej Pęśko - the CTO (graduate of the Faculty of Electronics and Information Technology at the Warsaw University of Technology),
* Adam Svystun – a programmer (graduate of the Faculty of Electronics and Information Technology at the Warsaw University of Technology),
* Paweł Andruszkiewicz – an AI expert (graduate of the Faculty of Electronics and Information Technology at the Warsaw University of Technology),
* Bolesław Michalski – COO, and CMO,
* Przemysław Rokita , DSc, PhD - the scientific advisor (Assistant Professor at the Faculty of Electronics and Information Technology at WUT, the Institute of Computer Science, the Head of the Division of Computer Graphics),