Multi Marginal Temporal Schrödinger Bridge for Video Generation from Static Unpaired Data

This project explores cutting-edge techniques in video generation using multi-marginal temporal Schrödinger bridges.

For detailed information about this research project, methodologies, results, and interactive demos, please visit our comprehensive project website:

🔗 Visit the complete project documentation

Key Contributions

• Multi-marginal Optimal Transport: Extension of classical optimal transport to multiple marginals for temporal consistency
• Schrödinger Bridge Framework: Application of diffusion-based bridges for video generation
• Static-to-Dynamic Translation: Novel approach to generate videos from unpaired static images
• Temporal Coherence: Maintaining consistency across video frames through optimal transport

Technical Approach

The method leverages multi-marginal optimal transport theory combined with Schrödinger bridge formulations to:

1. Bridge Static Distributions: Connect multiple static image distributions through optimal transport plans
2. Temporal Interpolation: Generate smooth temporal transitions using diffusion processes
3. Unpaired Learning: Train on static images without requiring paired video data
4. Dynamic Synthesis: Produce temporally coherent video sequences

For complete technical details, experimental results, code implementation, and interactive demonstrations, please visit:

📖 Full Project Documentation - mmdsbm.notion.site