Skip to content Search

In recent years, neural rendering has emerged as a prominent and prolific field within computer graphics research. One groundbreaking innovation in this domain is the concept of Neural Radiance Fields (NeRFs), which has revolutionized 3D modeling. NeRFs enable the creation of intricate, high-fidelity 3D scenes from novel viewpoints using only a sparse set of images and corresponding camera positions. This neural network architecture leverages connections between these images and foundational computer graphics techniques like ray tracing to produce realistic scenes.

NeRF

In NeRFs, the scene is represented using fully connected network architectures. These networks process 5D coordinates comprising camera positions and spatial locations to generate the color and volume density of each point within the scene. The loss function of NeRFs draws inspiration from conventional volume rendering techniques, incorporating color information for every ray traversing the scene directly into the neural network’s weights.

While NeRF architecture excels at generating high-quality renders of new viewpoints in static scenes, it encounters challenges, primarily due to the time-intensive process of encoding object shapes into neural network weights. Training and inference with NeRF models can be prohibitively time-consuming, limiting their applicability in real-time scenarios.

Gaussian Splatting

In contrast, Gaussian Splatting (GS) offers comparable render quality with faster training and inference times. GS achieves this by dispensing with neural networks and instead encoding object information into a set of Gaussian distributions. Gaussian Splatting model 3D scene by a collection of 3D Gaussians defined by a position (mean), covariance matrix, opacity, and color represented via spherical harmonics (SH). GS algorithm creates the radiance field representation by a sequence of optimization steps of 3D Gaussian parameters (i.e., position, covariance, opacity, and SH colors). The key to the efficiency of GS is the rendering process, which uses projections of Gaussian components. These Gaussians can be utilized similarly to classical meshes, allowing for the swift development of models, especially for dynamic scenes. However, conditioning GS is challenging, requiring a large number of Gaussian components.

Both NeRFs and GS present distinct advantages and drawbacks. Our team’s primary objective is to develop new representations for both NeRFs and Gaussian Splatting to address a fundamental challenge in neural rendering.

– Przemysław Spurek

Research Team Leader


Przemysław Spurek

Przemysław Spurek is the leader of the Neural Rendering research team at IDEAS NCBR and a researcher in the GMUM group operating at the Jagiellonian University in Krakow. In 2014, he defended his PhD in machine learning and information theory. In 2023, he obtained his habilitation degree and became a university professor. He has published articles at prestigious international conferences such as NeurIPS, ICML, IROS, AISTATS, ECML. He co-authored the book Głębokie uczenie. Wprowadzenie [Deep Learning. Introduction] – a compendium of knowledge about the basics of AI. He was the director of PRELUDIUM, SONATA, OPUS and SONATA BIS NCN grants. Currently, his research focuses mainly on neural rendering, in particular NeRF and Gaussian Splatting models.

  • 2023 scientific award of the Rector of the Jagiellonian University
  • Paper Award at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023)
  • Contributed talks at NeurIPS 2022 Workshop on Meta-Learning 2022

  • SONATA BIS, Modele generatywne 3D oparte na reprezentacji NeRF [Generative 3D models based on NeRF representation] (awarded)
  • OPUS, Hypernetworks methods in Meta-Learning (Polish National Centre of Science Grant No. 2021/43/B/ST6/01456).
  • OPUS, Generating real-life images by AutoEncoder based models (Polish National Centre of Science Grant No. 2019/33/B/ST6/00894).
  • SONATA Clustering algorithm, which uses generalized Gaussian distribution and non-normal distributions (Polish National Centre of Science Grant No. 2015/19/D/ST6/01472).
  • PRELUDIUM The memory center (Polish National Centre of Science Grant No. 2013/09/N/ST6/01178).

Other research groups and teams

  • Algorithms in Autonomous UAVs Unmanned aerial vehicles (UAVs), also commonly known as drones, are becoming increasingly common in various aspects of life.
    Karol Pieniący
  • Sequential Decision Making We believe that the development of techniques for effective analysis and decision-making in sequences will lead to the creation of intelligent and autonomous systems. This will translate into many practical solutions, ranging from controlling robots or autonomous vehicles to multi-step decision or deductive procedures, such as mathematical proofs.
    Piotr Miłoś
  • Systems Security and Data Privacy Several machine learning applications involve issues where privacy plays a special role.
    Stefan Dziembowski