Evaluate Neural Rendering Methods for XR/AI Applications​

Keywords

Computer Graphics, Augmented Reality, Neural Rendering, Generative AI, Rapid Prototyping

Project Surpervisors

Daniel Filonik, Christian Sandor, Huyen Nguyen.

Emails: daniel.filonik@universite-paris-saclay.fr, christian.sandor@universite-paris-saclay.fr, thi-thuong-huyen.nguyen@universite-paris-saclay.fr

Description

The recent advances in machine learning have inspired and enabled an abundance of novel methods in computer graphics. Collectively these approaches are sometimes referred to as “neural rendering methods”. Applications of these methods are vast, and they can outperform classic shaders in fidelity and performance. For example, they include:

• Geometry/Texture Compression
• Generative Materials
• Generative Geometries
• Light Simulation/Physically Based Rendering
• Post-Processing/Effects

The fundamental idea behind neural rendering is: Rather than compute an exact solution for an expensive computation, use machine learning to approximate it. This way, the goal is that computations which were previously too expensive for real-time applications, can now be performed more efficiently.

Efficiency is especially important in resource constrained computing environment, such as portable extended reality (XR) devices. Due to the engineering challenges in portable devices, they are often limited in terms of their compute capabilities. At the same time, XR applications have very high requirements in terms of accuracy and realism. The better the quality of the rendering, the more convincing and seamless the integration between virtual and real-world objects.

Through this project, we want to explore novel rendering approaches and foster expertise in our team. As part of this, the intern will:

• Review the influx of novel neural rendering methods.
• Evaluate the feasibility of implementing these methods for XR/AI applications.

In the process of this project, the intern will produce proof-of-concept prototypes to explore the viability of neural rendering methods for XR/AI applications. The goal is to implement two or more comparable methods, and rigorously test and compare their performance. This will allow the intern to gather valuable experience working with XR/AI technologies. Furthermore, we anticipate the publication of a literature review, which will provide the foundation for future research, and a first prototype to demonstrate the research direction.

The goal is to produce a literature review to cover the current state-of-the-art of neural rendering methods. This will be a valuable resource for researchers who are looking to get an overview of the rapidly growing research area. It will also allow the research intern to gain experience with the process of academic publishing.

Apply

To express your interest, please send your application materials to Daniel Filonik, cc Huyen Nguyen, Christian Sandor:

• CV
• Transcript of Records
• Link to portfolio (github, personal homepage, etc.)