Speaker: Dr Nikola Kovachki (None / Other)
Date: 18th Jul 2024
Title: Function Space Diffusion for Video Modeling
Event: (DMLW01) International workshop on diffusions in machine learning: foundations, generative models, and optimisation
Abstract: We present a generalization of score-based diffusion models to function space by perturbing functional data via a Gaussian process at multiple scales. We obtain an appropriate notion of score by defining densities with respect to Guassian measures and generalize denoising score matching. We then define the generative process by integrating a function-valued Langevin dynamic. We show that the corresponding discretized algorithm generates samples at a fixed cost that is independent of the data discretization. As an application for such a model, we formulate video generation as a sequence of joint inpainting and interpolation problems defined by frame deformations. We train an image diffusion model using Gaussian process inputs and use it to solve the video generation problem by enforcing equivariance with respect to frame deformations. Our results are state-of-the-art for video generation using models trained only on image data.
👉 Want to learn more about the talks in this stream?
🔗| Seminar Link: www.newton.ac....
🔗| Workshop Link: www.newton.ac....
-------------------
FOLLOW US
🌐| Website: www.newton.ac.uk
🎥| Main Channel: / @isaacnewtoninstitute
🐦| Twitter: / newtoninstitute
💬| Facebook: / newton.institute
📷| Instagram: / isaacnewtoninstitute
🔗| LinkedIn: / isaac-newton-institute...
SEMINAR ROOMS
🥇| INI Seminar Room 1: / @iniseminarroom1
🥈| INI Seminar Room 2: / @iniseminarroom2
🛰️| INI Satellite Events: / @inisatellite
ABOUT
The Isaac Newton Institute is a national and international visitor research institute. It runs research programmes on selected themes in mathematics and the mathematical sciences with applications over a wide range of science and technology. It attracts leading mathematical scientists from the UK and overseas to interact in research over an extended period.
👉 Learn more about us and our events here: www.newton.ac.uk
Негізгі бет Dr. Nikola Kovachki | Function Space Diffusion for Video Modeling
Пікірлер