YangLing0818/Diffusion-Models-Papers-Survey-Taxonomy

Catalogue

Algorithm Taxonomy

Sampling-Acceleration Enhancement

• Learning-Free Sampling
• SDE Solver
• ODE Solver
• Learning-Based Sampling
• Optimized Discretization
• Knowledge Distillation
• Truncated Diffusion

Likelihood-Maximization Enhancement

• Noise Schedule Optimization
• Reverse Variance Learning
• Exact Likelihood Computation

Data with Special Structures

• Data with Manifold Structures
• Known Manifolds
• Learned Manifolds
• Data with Invariant Structures
• Discrete Data

Diffusion with (Multimodal) LLM

• Simple Combination
• Deep Collaboration

Diffusion with DPO/RLHF

Application Taxonomy

• Computer Vision
• Image Super Resolution, Inpainting and Translation
• Semantic Segementation
• Video Generation
• 3D Generation
• Anomaly Detection
• Object Detection
• Natural Language Processing
• Temporal Data Modeling
• Time-Series Imputation
• Time-Seires Forecasting
• Waveform Signal Processing
• Multi-Modal Learning
• Text-to-Image Generation
• Text-to-3D Generation
• Scene Graph/Layout to Image Generation
• Text-to-Audio Generation
• Text-to-Motion Generation
• Text-to-Video Generation/Editting
• Robust Learning
• Data Purification
• Generating Synthetic Data for Robust Learning
• Molecular Graph Modeling
• Material Design
• Medical Image Reconstruction

Connections with Other Generative Models

• Variational Autoencoder
• Generative Adversarial Network
• Normalizing Flow
• Autoregressive Models
• Energy-Based Models

Algorithm Taxonomy

1. Efficient Sampling

1.1 Learning-Free Sampling

1.1.1 SDE Solver

Score-Based Generative Modeling through Stochastic Differential Equations

Adversarial score matching and improved sampling for image generation

Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

Gotta Go Fast When Generating Data with Score-Based Models

Elucidating the Design Space of Diffusion-Based Generative Models

Generative modeling by estimating gradients of the data distribution

Structure-Guided Adversarial Training of Diffusion Models

1.1.2 ODE Solver

Improving Diffusion-Based Image Synthesis with Context Prediction

gDDIM: Generalized denoising diffusion implicit models

Elucidating the Design Space of Diffusion-Based Generative Models

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Step

Pseudo Numerical Methods for Diffusion Models on Manifolds

Fast Sampling of Diffusion Models with Exponential Integrator

Poisson flow generative models

Improving Diffusion-Based Image Synthesis with Context Prediction

Cross-Modal Contextualized Diffusion Models for Text-Guided Visual Generation and Editing

Structure-Guided Adversarial Training of Diffusion Models

Consistency Flow Matching: Defining Straight Flows with Velocity Consistency

Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening

MMaDA: Multimodal Large Diffusion Language Models

Revolutionizing reinforcement learning framework for diffusion large language models

1.2 Learning-Based Sampling

1.2.1 Optimized Discretization

GENIE: Higher-Order Denoising Diffusion Solvers

Learning fast samplers for diffusion models by differentiating through sample quality

1.2.2 Knowledge Distillation

Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed

Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening

1.2.3 Truncated Diffusion

Truncated Diffusion Probabilistic Models

2. Improved Likelihood

2.1. Noise Schedule Optimization

Cross-Modal Contextualized Diffusion Models for Text-Guided Visual Generation and Editing

Improved denoising diffusion probabilistic models

Variational diffusion models

2.2. Reverse Variance Learning

Improved denoising diffusion probabilistic models

Stable Target Field for Reduced Variance Score Estimation in Diffusion Models

2.3. Exact Likelihood Computation

Score-Based Generative Modeling through Stochastic Differential Equations

Maximum likelihood training of score-based diffusion models

A variational perspective on diffusion-based generative models and score matching

Score-Based Generative Modeling through Stochastic Differential Equations

Maximum Likelihood Training for Score-based Diffusion ODEs by High Order Denoising Score Matching

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

Improving Diffusion-Based Image Synthesis with Context Prediction

3. Data with Special Structures

3.1. Data with Manifold Structures

3.1.1 Known Manifolds

Riemannian Score-Based Generative Modeling

Riemannian Diffusion Models

3.1.2 Learned Manifolds

Diffusion priors in variational autoencoders

Hierarchical text-conditional image generation with clip latents

High-resolution image synthesis with latent diffusion models

Improving Diffusion-Based Image Synthesis with Context Prediction

3.2. Data with Invariant Structures

Permutation invariant graph generation via score-based generative modeling

Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations DiGress: Discrete Denoising diffusion for graph generation

Learning gradient fields for molecular conformation generation Graphgdp: Generative diffusion processes for permutation invariant graph generation

SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation

Protein-Ligand Interaction Prior for Binding-aware 3D Molecule Diffusion Models

Graphusion: Latent Diffusion for Graph Generation

3.3 Discrete Data

Structured Denoising Diffusion Models in Discrete State-Spaces

Vector Quantized Diffusion Model with CodeUnet for Text-to-Sign Pose Sequences Generation

Deep Unsupervised Learning using Non equilibrium Thermodynamics.

A Continuous Time Framework for Discrete Denoising Models

MMaDA: Multimodal Large Diffusion Language Models

Revolutionizing reinforcement learning framework for diffusion large language models

4. Diffusion with (Multimodal) LLM

4.1. Simple Combination

Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning

RealCompo: Dynamic Equilibrium between Realism and Compositionality Improves Text-to-Image Diffusion Models

4.2. Deep Collaboration

VideoTetris: Towards Compositional Text-To-Video Generation

4. Diffusion with DPO/RLHF

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation

Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening

MMaDA: Multimodal Large Diffusion Language Models

Revolutionizing reinforcement learning framework for diffusion large language models

Application Taxonomy

1. Computer Vision

• Conditional Image Generation (Image Super Resolution, Inpainting, Translation, Manipulation)
• Improving Diffusion-Based Image Synthesis with Context Prediction
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models
• Image Super-Resolution via Iterative Refinement
• High-Resolution Image Synthesis with Latent Diffusion Models
• Repaint: Inpainting using denoising diffusion probabilistic models.
• Palette: Image-to-image diffusion models.
• Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models
• Cascaded Diffusion Models for High Fidelity Image Generation.
• Conditional image generation with score-based diffusion models
• Unsupervised Medical Image Translation with Adversarial Diffusion Models
• Score-based diffusion models for accelerated MRI
• Solving Inverse Problems in Medical Imaging with Score-Based Generative Models
• MR Image Denoising and Super-Resolution Using Regularized Reverse Diffusion
• Sdedit: Guided image synthesis and editing with stochastic differential equations
• Soft diffusion: Score matching for general corruptions
• Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training
• ControlNet: Adding Conditional Control to Text-to-Image Diffusion Models
• Image Restoration with Mean-Reverting Stochastic Differential Equations
• SpaText: Spatio-Textual Representation for Controllable Image Generation
• Break-A-Scene: Extracting Multiple Concepts from a Single Image
• Improving Diffusion-Based Image Synthesis with Context Prediction
• Cross-Modal Contextualized Diffusion Models for Text-Guided Visual Generation and Editing
• RealCompo: Dynamic Equilibrium between Realism and Compositionality Improves Text-to-Image Diffusion Models
• Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
• EditWorld: Simulating World Dynamics for Instruction-Following Image Editing
• IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation
• Consistency Flow Matching: Defining Straight Flows with Velocity Consistency
• Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow
• Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpening
• MMaDA: Multimodal Large Diffusion Language Models
• Revolutionizing reinforcement learning framework for diffusion large language models

• Semantic Segmentation
• Label-Efficient Semantic Segmentation with Diffusion Models.
• Decoder Denoising Pretraining for Semantic Segmentation.
• Diffusion models as plug-and-play priors

• Video Generation
• Flexible Diffusion Modeling of Long Videos
• Video diffusion models
• Diffusion probabilistic modeling for video generation
• MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model.
• Cross-Modal Contextualized Diffusion Models for Text-Guided Visual Generation and Editing
• Stable video diffusion: Scaling latent video diffusion models to large datasets
• I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models
• Lumiere: A space-time diffusion model for video generation
• VideoTetris: Towards Compositional Text-To-Video Generation

• 3D Generation
• 3d shape generation and completion through point-voxel diffusion
• Diffusion probabilistic models for 3d point cloud generation
• A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
• Let us Build Bridges: Understanding and Extending Diffusion Generative Models.
• LION: Latent Point Diffusion Models for 3D Shape Generation
• Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
• RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation
• HOLODIFFUSION: Training a 3D Diffusion Model using 2D Images
• Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures
• DiffRF: Rendering-Guided 3D Radiance Field Diffusion
• DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models
• 3D Neural Field Generation using Triplane Diffusion
• Semantic Score Distillation Sampling for Compositional Text-to-3D Generation
• Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis

• Anomaly Detection
• AnoDDPM: Anomaly Detection With Denoising Diffusion Probabilistic Models Using Simplex Noise
• Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models.

• Object Detection
• DiffusionDet: Diffusion Model for Object Detection

2. Natural Language Processing

• Structured denoising diffusion models in discrete state-spaces
• Diffusion-LM Improves Controllable Text Generation.
• Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
• DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
• MMaDA: Multimodal Large Diffusion Language Models
• Revolutionizing reinforcement learning framework for diffusion large language models

3. Temporal Data Modeling

• Time Series Imputation
• CSDI: Conditional score-based diffusion models for probabilistic time series imputation
• Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models
• Neural Markov Controlled SDE: Stochastic Optimization for Continuous-Time Data

• Time Series Forecasting
• Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting