ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016)
 
Wasserstein Barycentric Coordinates: Histogram Regression Using Optimal Transport

Nicolas Bonneel Gabriel Peyré Marco Cuturi
Univ. Lyon, CNRS, LIRIS CNRS, Univ. Paris-Dauphine Kyoto University

Our Wasserstein projection framework can be used to automatically color grade an input photo (a) using a database of stylized color histograms, with samples shown in (b). We propose to compute the optimal transport barycenter of these stylized palettes that can approximate best the original palette, and use that barycenter to carry out color transfer without large color distortions as shown in (c), where the modified image and the barycentric palette are represented. That barycentric palette is parameterized using only the weights appearing in the captions of figure (b). Other applications include inferring reflectance functions or missing geometry.


Abstract
This article defines a new way to perform intuitive and geometrically faithful regressions on histogram-valued data. It leverages the theory of optimal transport, and in particular the definition of Wasserstein barycenters, to introduce for the first time the notion of barycentric coordinates for histograms. These coordinates take into account the underlying geometry of the ground space on which the histograms are defined, and are thus particularly meaningful for applications in graphics to shapes, color or material modification. Beside this abstract construction, we propose a fast numerical optimization scheme to solve this backward problem (finding the barycentric coordinates of a given histogram) with a low computational overhead with respect to the forward problem (computing the barycenter). This scheme relies on a backward algorithmic differentiation of the Sinkhorn algorithm which is used to optimize the entropic regularization of Wasserstein barycenters. We showcase an illustrative set of applications of these Wasserstein coordinates to various problems in computer graphics: shape approximation, BRDF acquisition and color editing.

  
@article{BTSSPP15,        
  author = {Nicolas Bonneel and Gabriel Peyr{\'e} and Marco Cuturi                   
  title = {Wasserstein Barycentric Coordinates: Histogram Regression Using Optimal Transport},       
  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016)},      
  volume = {35},        
  number = {4},         
  year = {2016}
}

   
Paper
PDF (41 MB) 		  Paper (low res.)
PDF (1 MB) 		  C++ Code
ZIP (40 KB)


 
  Supplemental Video
MP4 (96 MB) | YouTube


Acknowledgements
The work of G. Peyré has been supported by the European Research Council (ERC project SIGMA-Vision). N. Bonneel thanks Adobe for software donations. M. Cuturi gratefully acknowledges the support of JSPS young researcher A grant 26700002.

We thank the authors of the images, from Flickr users: Joe Giordano, taquiman, Tom Babich (Fig. 1 and 7), Michael Villavicencio, Rod Waddington (Fig. 1), Paul Stevenson, d pham, Rolands Lakis, NeilsPhotography, Shruti Biyani (Fig. 7, row 1), Ree Dexter, Richard P J Lambert (Fig. 7, row 2), Chris Sorge, Susanne Nilsson, Peggy2012CREATIVELENZ, Axel (Fig. 7, row 3), Yuri Samoilov, Neil Piddock, William Matthews, Luca Sartoni, Erik Drost (Fig. 7, row 4), Nick Kenrick, Gary Millar (Fig. 8). Used with permission or under CC BY-NA licence.
Copyright by the authors, 2016. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics: http://dx.doi.org/10.1145/2897824.2925918
Zip icon adapted from tastic mimetypes by Untergunter, CC BY-NA-SA 3.0 licence.