Mathematical methods for image synthesis

Numerical methods for computer graphics 2020

Lecturers: Nicolas Bonneel, Julie Digne, David Coeurjolly

This class will expose mathematical tools encountered at various stages of the image synthesis pipeline. These methods are widely used in computer graphics research.

Optimal transport (Nicolas)

Monge and Kantorovich formulations
Differential geometry of optimal transport
Numerical methods (Sinkhorn algorithm, Power Diagrams, ...)
Applications to images and rendering
Lecture notes: PDF version, PPTX version
Readings and project

Monte-Carlo methods for 3d rendering (Nicolas, part I)

The rendering equation: a Fredholm equation of the second kind
Lecture notes: PDF version, PPTX version
Readings and project

Monte-Carlo methods for 3d rendering and geometry(David, part II)

Importance sampling, Monte-Carlo convergence rate
Quasi-Monte Carlo
Lecture notes: Web
Monte Carlo for Geometry Processing
Lecture notes (continued): PDF

Image and video processing -- mostly with Poisson equation! (Nicolas)

Variational formulation / PDE
Numerical methods
Applications (image stitching, content-aware copy/paste, video filtering, gradient based rendering, fluid simulation, ...)
Other cool non-Poisson image and video processing techniques
Lecture notes: PDF version, PPTX version
Readings and project

Spectral mesh processing (Julie)

A Fourier basis on surface meshes, Laplace Beltrami definition
Application to mesh processing: denoising, compression
Shape correspondence linearization through functional maps
Lecture notes: PDF version
Readings and project

Around Laplace-Beltrami (David)

Intrinsic Vector Field Design
Robustness on geometry processing and Laplace-Beltrami on non-standard meshes
Lecture notes: Directory (pdf/pptx/key)
Readings

Markov random fields (Julie)

Definition, graphical model formulation
Maximum a posteriori inference: belief propagation and graph cuts
Application to Segmentation and Texture Synthesis
Lecture notes: PDF version
Readings and project

Machine Learning for Graphics and Vision (Julie)

Statistical tools for machine learning, Maximum likelihood and Maximum a posteriori estimator, supervised and unsupervised learning
K-means and Expectation Maximization for Gaussian mixture models.
Support Vector Machine, Kernel trick, Random Forests and boosting
Neural networks for image synthesis (Autoregressive neural networks, Variational auto-encoders, generative adversarial networks)
Readings and project

Prerequisites: Knowledge of basic geometric data structures and basic linear algebra. Programming.

Evaluation: Students will be graded on one short project related to the course, implemented in the language students feel the most comfortable with, and one article reading. Both the project and chosen paper will be presented at the end of the semester. If a project is taken from Nicolas' part, the reading should be taken from Julie and vice-versa.

Bio:
Nicolas Bonneel and Julie Digne are junior CNRS researchers at LIRIS lab, David Coeurjolly is a senior CNRS researcher at LIRIS.

Nicolas Bonneel develops mathematical tools for computer graphics, with particular attention to optimal transport, with applications to geometry, video processing, and rendering. After obtaining his PhD at INRIA Sophia-Antipolis in 2009, and years of post-docs at UBC (Vancouver), INRIA Nancy and Harvard University (Cambridge), he joined LIRIS in the geometry teams GéoMod and M2Disco in 2014, and joined École Polytechnique as a part-time assistant professor in 2019. He nows leads the ORIGAMI computer graphics team of LIRIS.

Julie Digne focuses her research on geometry processing and applied math. She obtained her PhD, advised by Jean-Michel Morel, in 2010 in applied mathematics at Ecole Normale Supérieure de Cachan. She graduated from ENS Cachan (Master's degree: Mathématiques Vision Apprentissage). Her PhD thesis was awarded the Jacques Hadamard Foundation for Mathematics PhD award. In 2010-2012, she was a post-doc at INRIA in collaboration with Caltech. She joined the GéoMod (now ORIGAMI) team at LIRIS, University of Lyon in 2012.

David Coeurjolly graduated from the Ecole Normale Superieure and the Université Claude Bernard of Lyon, France, in 2000 and received the Ph.D. Degree in digital geometry in 2002 from the Universiteé Lumière Lyon 2. In 2003, he obtained a permanent re- search position (Chargé de Recherche CNRS) in the LIRIS laboratory, CNRS UMR 5205 Lyon. In 2011, he got a Senior tenured researcher position (Directeur de Recherche CNRS) at the same institute. He led the m2DisCo research team in LIRIS from October 2011 to January 2015. From January 2017, he is director of the Fédération Informatique de Lyon (CNRS). In 2009, he received a Bronze Medal from CNRS that recognized his activities in Digital Geometry. His present research interests include geometry processing, digital and computational geometry, Monte Carlo rendering and computer graphics.