ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2015) |
||
Blind Video Temporal Consistency |
Nicolas Bonneel | James Tompkin | Kalyan Sunkavalli | Deqing Sun | Sylvain Paris | Hanspeter Pfister |
LIRIS - CNRS | Harvard University SEAS | Adobe |
Abstract | |
Extending image processing techniques to videos is a non-trivial
task; applying processing independently to each video frame often
leads to temporal inconsistencies, and explicitly encoding temporal
consistency requires algorithmic changes. We describe a more
general approach to temporal consistency. We propose a gradientdomain
technique that is blind to the particular image processing
algorithm. Our technique takes a series of processed frames that
suffers from flickering and generates a temporally-consistent video
sequence. The core of our solution is to infer the temporal regularity
from the original unprocessed video, and use it as a temporal
consistency guide to stabilize the processed sequence. We formally
characterize the frequency properties of our technique, and demonstrate,
in practice, its ability to stabilize a wide range of popular
image processing techniques including enhancement and stylization
of color and tone, intrinsic images, and depth estimation.
|
@article{BTSSPP15, |
|
Supplemental Video MP4 (93 MB) | YouTube |
Acknowledgements | |
We thank Eugene Hsu and Ivaylo Boyadzhiev for their white balance
implementation [Hsu et al. 2008], Xue Bai and Kang In Kim for
their suggestions, Pierre Jasmin of RE:Vision for his expertise in
processing sequences with DE:Flicker [RE:Vision 2015], and Adobe
for their donation.
We thank the authors of the video footage: G. Mougin (Fig. 1), Unbound (Fig. 4), Millgate (Fig. 5 and 9), G. Henkel (Fig. 6), S. Pyeatte (Zina Nicole Lahr, Fig. 7 and 9), B. Fitzgerald (Fig. 9), P. Shin (Fig. 9) and B. Bourgon (Fig. 8). Used with permission or under CC BY-NA licence. |
|
Copyright by the authors, 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics: http://dx.doi.org/10.1145/2661229.2661253 | |
Zip icon adapted from tastic mimetypes by Untergunter, CC BY-NA-SA 3.0 licence. | |
This work was partially supported by NSF grants NSF CGV-1110955, with the work also sponsored by the Air Force Research Laboratory and DARPA Memex program. |