In such an internet, known as web3, decentralized applications operate on blockchain technology. Instead of focusing on information, as it is in the case of web2, the emphasis is placed on users. The key values and objectives are to ensure privacy, security, and full ownership of data. A shared virtual reality will enable the interpenetration of different worlds with the aid of created avatars. The tools for this invention are being developed by Paweł Andruszkiewicz, Maciej Pęśko, and Adam Svystun, graduates of the WUT Faculty of Electronics and Information Technology.

Comics, Artificial Intelligence and Terry Pratchett

A few years ago, while they were still students, they developed an algorithm for turning videos into comics, assisted by Prof. Tomasz Trzciński, and supported by Prof. Przemysław Rokita. Comixify would automatically select the most interesting and important frames from the uploaded video, and then arrange them, adjust to frames, and graphically process them. All thanks to artificial intelligence. All you had to do was submit a video to the specifically created page, make a few clicks, and you'd get a comic book.

The first publication about the project was released towards the end of 2018, and the avalanche followed after that. Comixify was a huge sensation in the world of science, business, entertainment and in the media worldwide, not just in Poland. The creators continually improved their idea, adding new features, gaining funding sources, connections, as well as prizes, appreciation, and recognition. This led, among other things, to cooperation on the Terry Pratchett book-based BBC America television series "The Watch". There, the machine learning-based style transfer technology created by the WUT team was applied. This allows you to replicate a particular style based on the drawings or paintings and then incorporate it into the video.

A still from the sixth episode of the "The Watch” series. One can see the style transfer technology used here, photo: Comixify blog

The road to success

Soon afterwards, the creators of the innovative comics technology began working with the American-Czech start-up Alter. It is a team that develops advanced solutions to support the development of the Internet of the future and the growth of a unified virtual world using metaverse technology.

Paweł Andruszkiewicz, Maciej Pęśko and Adam Svystun eventually joined Alter, where they now face new challenges as its employees.

– We have created the world's best avatar technology to meet the needs of developers – says Jon Slimak, Alter's Chief Operations Officer. – We have the essentials the market needs, and the Comixify team is the key to our vision.

– Avatars are the future of how we connect online, but the technology is still new – adds Robin Raszka, CEO, and the founder of the company. – Building the next generation of digital spaces and creating user-friendly, great-looking 3D content from scratch currently costs a lot of money and time. We're changing it.

And our graduates are helping to do that.

How are avatars created?

– The most important product that Alter is building is Alter SDK, a cross-platform toolkit for developers, that allows users to easily add support for three-dimensional avatars to their own applications – explains Paweł Andruszkiewicz.

The Alter SDK consists of three basic modules: a real-time 3D avatar generation system (based on the original rendering engine), a facial motion capture module, and the Avatar Designer – an interface and components that make it simple for users to create their own avatars.

The solution allows users to create avatars in a variety of formats, e.g., only the head or torso with clothing. It also offers a huge selection of different accessories. Another avatar format—the full character—will be available soon as well. The system already enables the creation of avatars for people of different ages and genders, and users are not limited to creating only human avatars.

– Alter places a strong emphasis on building a system that will be as developer-friendly as possible – simple to use and integrable with any application or game – says Adam Svystun. – Our tools are very easy to configure (For instance, we've created a JSON configuration file we've named Avatar Matrix that lets you define avatars from the level of this file.) The SDK was created with characteristic for web3 interoperability in mind from the very beginning.

Paweł, Maciek and Adam are primarily involved in the development of the so-called mocap4face, a system for capturing facial motion. This module enables precise observation of the facial expression through the use of artificial neural networks and advanced image processing. It tracks over 40 different expressions.

– The main task of our team is to develop this component of the technology stack, among other things, by improving training data, and developing the neural network architecture that is being used and the training process – explains Maciej Pęśko. – We focus on both improving the quality of the facial motion capture module and optimizing the performance of this module across all supported platforms.

Facial motion capture system tracks over 40 different expressions

If the project continues to develop at this rate, the world of avatars will soon become part of our daily lives.