First launched in 2004, the Microsoft Research PhD Program has so far supported more than 200 PhD students from more than 18 countries and 51 institutions. The Warsaw University of Technology has become one of them this year.

The 16 applications selected for funding (including from universities such as Cambridge and Oxford) include a research project called “Low Shot Realistic Human Rendering from Partial Information”.

A novel idea

The authors of the application and scientific supervisors of the project are Tomasz Trzciński, Ph.D., D.Sc., Eng. of the Institute of Computer Science Department of Computer Graphics within the Faculty of Electronics and Information Technology and one of our alumni, Marek Kowalski, Ph.D., Eng., currently working for Microsoft.

“We want our project to use generative neural network models to create high quality human visualizations and human motion animations,” Doctor Trzciński explains. “Our research challenge is to reduce the volume of data needed to render such algorithms and achieve quality that would be indistinguishable from the effects delivered with traditional graphics generation methods.”

Why does it matter?

Video games and movies increasingly require that quality human animations be created. The solutions used by technologies such as VR (Virtual Reality), AR (Augmented Reality) or CGI (Computer-Generated Imaging) are time consuming, expensive and hard to scale.

The first step to create a digital human is usually 3D scanning of the face and body of a real human performing various activities. Specialist equipment is needed to obtain such scans and then these are further processed by graphic artists. But this does not end here. To achieve realistic model animation, an actor is needed who will perform the desired motion sequences at a special motion capture studio. This is how some iconic movie characters such as Gollum of “The Lord of the Rings”, Na-vi of “Avatar” or Hulk (played by Mark Ruffalo) of the “Avengers” franchise were created.

“Our goal is to facilitate the creation of avatars by using two approaches: Partial Information and Low-Shot Learning,” says Doctor Trzciński.

How is it supposed to work?

The purpose is simple: to generate realistically-looking videos (e.g. featuring a full face with a full range of facial expressions and natural delivery of lines) out of partial information (e.g. a fragment of a face) and basic and readily available data without using sophisticated systems that require lots of repetitions (under one described method a human must use 122 expressions and 50 sentences and be recorded by 40 cameras).

Who will get the scholarship?

Recruitment of candidates for PhD student to join the program is going on until July 31, 2020. For more details, please refer to the advertisement and our website.