Nicolas Bonneel

CNRS - LIRIS liris logo

Équipe Origami
Batiment Nautibus, Bureau 246
23-25, Avenue Pierre de Coubertin
69622 Villeurbanne Cedex (France)

Email : nicolas.bonneel@liris.cnrs.fr
Phone : (+33)426234448

I am a research director at CNRS (Lyon, France) where I lead the ORIGAMI computer graphics team, and I am part-time assistant professor at École Polytechnique (Paris, France). I did a two-year postdoc at Harvard University, working with Hanspeter Pfister until January 2014, after a one-year postdoc at INRIA Nancy, with Bruno Levy until January 2012, working on anisotropic remeshing of surfaces. I previously did a one year postdoc at UBC in Vancouver, supervised by Michiel van de Panne, working on displacement interpolation. I received my PhD in September 2009 at INRIA Sophia Antipolis, supervised by George Drettakis. I am particularly interested in optimal transport, video processing, geometry, realistic rendering.

 

I am happy to share our Siggraph Asia paper, accepted to the conference track. It allows to learn sampling patterns from examplars (e.g. Sobol', Poisson disk, Rank 1...) using a diffusion model. All convolutions are performed on the grid, thanks to an optimal transport assignment of scatter samples to grid samples.
I also got a survey of optimal transport applications to computer graphics recently published at Computer Graphics Forum / Eurographics.
On an unrelated note, you may want to check the (fun) books I generated in english and in french.
I also just got promoted to the rank of Research Director at CNRS.
I can now share our Siggraph paper, MatBuilder, mentioned below: a paper about sampling points with arbitrary uniformity constraints (stratification, (0, m, s)-net constraints, ...) on projections. A short Twitter thread to summarize it here in English, or here in French.
I'm also happy our long-term collaboration with Brown University led to a Computers & Graphics paper on video-based rendering, that is, being able to navigate inside a set of input videos. See our video here !
Finally, we release a tech report about generating nicer optimal transport interpolations between meshes by symmetrizing the semi-discrete optimal transport problem. With the explanatory Twitter thread by Bruno Levy :)
A couple of new papers!
- a Eurographics paper about rendering participating media from multiple viewpoints at the same time.
- A conditionally accepted SIGGRAPH paper about sampling, which I can't disclose right now but more details are coming soon, and I am very excited about it!
Long time since my last update! And thus, several interesting papers and preprints released in the meantime.
- a SIGGRAPH Asia paper about getting better uniformity out of Sobol sequences.
- a Eurographics paper about the perceptual quality of BRDFs.
- a JMIV paper about learning the optimal transport ground metric from displacement interpolation results.
- an arxiv preprint about generating very fast optimal transport barycenters using deep learning.
- an arxiv preprint about synthesizing novel views from a set of input videos (i.e., "Video-Based Rendering").
My contract at École Polytechnique has been renewed for 2 more years.
First, I am happy to announce our second baby, Lila, is born on January 9th, 2020.

We also had two papers accepted at SIGGRAPH this year (though in this COVID-19 context, the conference will be held remotely):
- a paper on the replicability of computer graphics source codes, with a cool website, https://replicability.graphics where you can contribute by reviewing the replicability of new papers.
- a paper on sliced optimal transport sampling, or how to solve the optimal location problem in high dimension using a sliced optimal transport approach, or how to maintain social distancing in high-dimensional spaces ;) This sampling improves the convergence of quasi-Monte Carlo integral estimates over the best samplers to date, and can be used for rendering.

We finally released a preprint on estimating geodesic ground metrics in optimal transport distances given distributions at intermediate times during a displacement interpolation.
As of September 2019, I am now a quarter-time Assistant Professor at École Polytechnique (LIX laboratory), where I will teach a computer graphics course, in addition to my CNRS position at LIRIS in Lyon.
I now also lead the Origami computer graphics group at LIRIS (to be officially launched in January 2020). I seize the opportunity to welcome Julien Lacombe now a Ph.D. student I am supervising with Julie Digne on optimal transport and machine learning !

Finally, Dominic Schuhmacher and Florian Heinemann have successfully integrated my adapted network simplex code into their R package so R users can now benefit from a fast exact discrete optimal transport solver !
I am happy to see our paper on sliced optimal transport accepted to SIGGRAPH this year ! This computes a fast n-dimensional optimal transport approximation of point clouds via 1-d slices, but assuming the point clouds do not have the same cardinalities. For each slice, this consists in solving an alignment problem for which we provide a fast algorithm. We also apply this technique to improve the well-known ICP algorithm to register point clouds. We released the code and video as well !

I am also welcoming a couple of Masters students: Guillaume Cherreau who I am co-supervising with Julie Digne and Filippo Santambrogio, and Julien Lacombe who I am also co-supervising with Julie.

I'll also be part of Dmitry Kuzovkin's PhD thesis committee on June 21st, in Rennes.
We have released a number of useful codes and benchmarks:
- Wasserstein Dictionary Learning code: Our C++ implementation, to perform dictionary learning using optimal transport.
- My old adaptation of LEMON's network simplex is now improved, and benchmarked against CPLEX and others. It solves discrete optimal transport problems of measures involving about 60k diracs in 6 minutes (using a quadratic cost ; speed is affected by the ground cost). It is orders of magnitude faster than CPLEX's network simplex on large problems. The nice github page explains everything. This was done in the context of our funded ANR project ROOT
- My path tracer. It is of little value in term of rendering performance, but is useful to easily render scenes for papers (e.g., just drag'n drop an obj file, drag'n drop a .seg mesh segmentation file, drag'n drop an envmap, and you'll see it nicely rendered, colored and ready to import in your paper). Many images on the github page!

We have also released a tech report on adapting the Mumford-Shah functional (often used in image processing) for mesh processing.
I am also just back from Bologna, Italy, where I have presented our optimal transport regression works at the SIAM Conference on Image Science.
I'm first happy to announce the birth of my son Evan, born on November 19th, 2017.

Our paper on dictionary learning using optimal transport (preprint) just got accepted to the SIAM Journal on Imaging Sciences. It consists in finding a set of histograms whose combinations (in the sense of optimal transport, i.e., based on warpings) well approximate an input database of histograms, with various applications in image and word processing.

We are releasing an excellent preprint on arxiv about doing dictionary learning in the Wasserstein space. I also explained (in french!) mass transport principles and our last Siggraph article as a vulgarization paper in the Interstices journal from INRIA.

Starting from September, we are welcoming Beatrix Fulop-Balogh as a PhD student on image-based-rendering related problems, who I will co-supervise with Julie Digne. I will also be part of Hristina Hristova's Ph.D committee in October.
We got a full paper accepted at Eurographics 2017 on how to make image processing temporally consistent for multi-camera setups, as well as a state-of-the-art report on intrinsic decompositions also at Eurographics. This state-of-the-art is accompanied by a website, called ARAP, that provides ground truth data on realistic synthetic 3D scenes for intrinsic decompositions, depth, segmentation, matte, positions, and shadowing, with multiple lighting conditions. We also got a 3DOR paper accepted on semantic mesh editing.

Eurographics 2017 will be held in our city, Lyon, at the end of this month and I'm helping with the organization. I will be kept busy with the presentation of our paper and STAR, but I am also co-chairing, with Belen Masia, the Doctoral Consortium -- a Eurographics event aimed at having senior researchers mentor PhD students during an afternoon. Eurographics will welcome three invted superstars: Frédo Durand, Jos Stam and Bruno Lévy. Please register soon!

I am also still looking for a good PhD student interested in geometry processing, math and optimal transport to be co-supervised by Bruno Lévy (see, superstar above!) and me. I am currently co-supervising with Julie Digne two interns from ENS: Agathe Herrou on optimal transport (also co-supervised by Bruno) and Beatrix Fulop-Balogh on 3d animation capture.
My project on inverse problems in optimal transportation got funded by the ANR (french version of the NSF). I am thus pleased to welcome Matthieu Heitz as my new PhD student, also co-supervised by G. Peyré, M. Cuturi. and D. Coeurjolly. I will be looking for a computational geometry-oriented PhD student to start in September 2017, co-supervised by B. Lévy.
Speaking of PhD students, I was co-supervising Vincent Léon (with G. Lavoué and J-P. Vandeborre) who will be defending his thesis on mesh part retrieval on Nov. 10th in Lille.

I am also a new associate editor for The Visual Computer, international program committee member of the Siggraph Asia 2016 Posters&shorts program, Pacific Graphics 2016, the International Conference on Computational Photography (ICCP 2017), the International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM+), and I am still in the organizing committee of Eurographics 2017 which will take place in Lyon. Speaking of which, I will be co-chairing the doctoral consortium of Eurographics with Belen Masia -- the Call for submissions in coming soon, but feel free to ask me questions in the meantime!

We also got a short paper accepted at Pacific Graphics on interactive multi-label video segmentation using multi-cuts. I also recently presented our recent Siggraph work at the PICOF (Problèmes Inverses, Contrôle et Optimisation de Formes) conference, and will be presenting it again at a workshop on optimal transport on Oct. 11th-12th at INRIA in Paris. Also started teaching at ENS a class on mathematical methods for image synthesis.
Our new SIGGRAPH paper is out! Check the Youtube Video and Project page. We present a technique to efficiently compute barycentric coordinates for histograms using optimal transport theory. In collaboration with Gabriel Peyré and Marco Cuturi.
Speaking of SIGGRAPH, I have also been selected by the french GDR IGRV and AFIG to fly to SIGGRAPH and give feedback to the french community during the Journées AFIG (Grenoble, in November) and an event organized by the french SIGGRAPH chapter. So, see you in Anaheim in July for SIGGRAPH, in Grenoble in November for the journées AFIG, and in Paris for the SIGGRAPH France event! :)
We had a few exciting paper accepted! In particular, I am currently in Japan to present our SIGGRAPH Asia paper on temporal consistency (YouTube) and we also had a nice ICCV paper accepted on image and mesh segmentation (YouTube)!
I have also parallelized my network simplex code for exact optimal transport computations.
I will be on Guillaume Tartavel PhD committee at Telecom ParisTech tomorrow for his defense on texture synthesis (Signal and Image Processing). I will also join Marco Cuturi at Kyoto University from April 12th to 21st. Marco invited me to collaborate and give a seminar on video processing.
I have now left Harvard (and many thanks to the Visual Computing Group for these fantastic two years!) and joined CNRS as a permanent researcher. A couple of new papers got in:
- Our paper on our super-fast sliced mass transport is now published in the Journal of Mathematical Imaging and Vision
- The Mu-8 project was extended to a paper in BMC Proceedings -- an online open-access journal in biomed. You can check our nice interactive protein visualization here!
- We published a normal reconstruction method that works on time-lapse video sequences, at Pacific Graphics (Computer Graphics Forum), with Fangyang Shen, Kalyan Sunkavalli, Szymon Rusinkiewicz, Hanspeter Pfister and Xin Tong.
- Our SIGGRAPH Asia (ACM Trans. on Graphics) paper got accepted, with Kalyan Sunkavalli, James Tompkin, Deqing Sun, Sylvain Paris and Hanspeter Pfister! We allow for the interactive intrinsic decomposition (i.e., a shading*reflectance decomposition) of videos.
I also started teaching some programming labs here ; gave a couple of talks (at MPI in Erlangen, for a mass transportation workshop in Paris, here, and soon for another mass transportation workshop in Toulouse...), and despite my lack of updates on this webpage, everything is doing great!
New tech report on sliced transportation using both the Radon approach and random projections ; co-authored with Julien Rabin and Gabriel Peyré, and advised by Hanspeter ; it is also a bit more mathematical.
We also received an honorable mention for the BioVis contest for our protein visualization tool Mu-8 (pronounce "Mutate") with Johnathan Mercer, Balaji Pandian and Alexander Lex, under Hanspeter's supervision and in the context of his CS171 visualization class.
The end of the semester is approaching for the CS109 - Data Science class I am TF'ing, and Math232a (by Mboyo Esole) and Math231a (by Hiro Lee Tanaka) that I'm both auditing.

Finally, I'm also joining CNRS as a junior research scientist in January in Lyon, between two geometry teams at LIRIS: GeoMod and m2Disco. I'm thus no longer looking for positions ! :)
I also created a small webpage with a few efficient mass transport (/Earth Mover's Distance/1st Mallow distance/Wasserstein/Monge Kantorovich/Kantorovich Rubinstein/Hitchcock Koopmans/"distance du cantonnier" in French!...) codes I reimplemented or used. Tell me what you think!
Don't miss our new technical report on sliced mass transportation!
Also, don't miss my talk at Siggraph, presenting our video color grading method, on Monday, July 22nd, at 10am in the Anaheim Convention Center.
Our ACM SIGGRAPH paper Example-Based Video Color Grading (Nicolas Bonneel, Kalyan Sunkavalli, Sylvain Paris and Hanspeter Pfister) is accepted !
Our method makes use of cool differential geometry to color grade videos in a temporally consistent way. And speaking of differential geometry... the term is ending for Hanspeter's CS171 visualization class that I'm TA'ing and for the differential geometry classes I'm auditing (Math230B by S-T.Yau and Math132 by A.Cotton-Clay).

End of the term for CS 205 , and new tech report available on BSDF sharing.
Our paper Variational Anisotropic Surface Meshing with Voronoi Parallel Linear Enumeration is now online. Check it out if you are in need of fast and parallel Voronoi diagrams, possibly in very high dimension, or simply want efficient anisotropic remeshing.
I am now a teaching assistant for CS 205 - Computing Foundations for Computational Science at Harvard for this semester (this course is an introduction to parallel thinking and programming for computational sciences), and auditing S-C. Lau's Math230A differential geometry class.
Our IMR'12 remeshing paper got accepted (Variational Anisotropic Surface Meshing with Voronoi Parallel Linear Enumeration, Bruno Levy and Nicolas Bonneel). The pdf is not online yet.
I was also happy to be head teaching fellow for CS 171 - Visualization at Harvard this past semester.
I'm happy to announce that, on February 1st, I will be starting a new postdoc at Harvard with Hanspeter Pfister.

Harvard Logo

For older news : I was invited to give a talk for an annual meeting of the LIMA project in July 2011 - a project bringing together research teams working in graphics around Lyon and Grenoble (FR). I also presented our 2010 ACM Trans. App. Perception paper at APGV in August 2011. I also presented a demo of our VMV 2011 paper at the "Palais de la Découverte" in Paris (a science museum) in October. And finally, I went to Hong Kong to present our Displacement Interpolation paper at Siggraph Asia 2011 in December !

Our paper Displacement Interpolation Using Lagrangian Mass Transport has been conditionally accepted for publication at Siggraph Asia (ACM Trans. on Graphics).
I was ranked 5th for a full time research position at INRIA over approximately 125 candidates holding PhDs in various computer science disciplines. This does not allow me to get the position (there are 2 positions), and I'm thus looking for career opportunities, research positions in faculties or interesting companies. My research statement mainly involves Wasserstein spaces and mass transport - or how to deal with probability distributions (such as BRDFs, color histograms, character pose distributions, descriptor distributions for vision applications, etc.) in a principled way. I see this space of distributions as a Riemmanian manifold (see for example the excellent book from Vilani), where geodesics can be computed, where projection makes sense etc. : in short, where differential geometry makes sense. This is inspired by our last work (and to a lesser extend, by our EGSR paper on hair reflectance acquisition) where I realized that most blending operations performed on probability distributions just don't make sense and give rather poor results, while Wasserstein spaces solve these issues and give an exciting research direction!

For a more applied research and development industry, my skills (non-exhaustively) involve computer graphics, applied maths, C++/Matlab coding. To a lesser extent, computer vision, machine learning, mobile development on Android, and other programming languages (Java, Fortran etc.).
We released our last work on displacement interpolation as a technical report. This is an excellent topic to work on... Check the video and the project page!

After the Siggraph deadline, I spent a few days with some friends in the largest wind tunnel in the world, at Paraclete XP, NC, trained by the skydiving World Champions: the Team4Speed. Check the awesome video we did!
I just accepted to be part of the International Program Committee for the Eurographics Symposium on Rendering (EGSR) 2011 and for the Graphics and Virtual Reality (GRVR) 2011 international conference.
Best Paper Award! We got the Best Paper Award at VMV for Proxy Guided Texture Synthesis for Rendering Natural Scenes over 82 submitted papers including 43 accepted papers!.
I will also start a new postdoc at INRIA Nancy with Bruno Levy from December 8th 2010.

New website! Our technical report has also been accepted as a paper to appear at VMV'2010.
We released a great technical report on rendering sketched 3D scenes using photographs... Check this out here! You can also directly see the video on YouTube here.
PostDoc at the University of British Columbia, Vancouver (Canada), until November 2010.
My PhD thesis is now online. Download it now!
I just defended my PhD thesis "Audio and Visual Rendering with Perceptual Foundations". I will be a postdoctoral researcher at the University of British Columbia starting from December 2009 to November 2010, supervised by Michiel van de Panne.

Jury for my PhD defense

I've been interviewed by the New Scientist on the Harmonic Fluids paper this year at Siggraph! Check the article!