🎨 MAIST

Imagine A Reference: MLLM-Augmented Versatile Image Stylization

Accepted at ACM Multimedia 2026

Overview · Highlights · Method · Results · Quick Start · Citation

🌟 Overview

MAIST is a multimodal framework for reference-guided image stylization. Given a style image and a content text prompt, MAIST first constructs a semantically aligned imagined exemplar and then transfers its structure-aware style information throughout diffusion sampling.

🎯 TL;DR: MAIST follows an Imagine-then-Infuse pipeline: an MLLM helps interpret the user's intent, while dedicated infusion and alignment modules preserve content structure and expressive style.

✨ Highlights

• 🎛️ Flexible dual conditioning. Control image content with natural-language prompts while providing style through a reference image.
• 🧠 MLLM-assisted imagination. Qwen2.5-VL jointly understands the style image and content prompt to produce an enriched description for exemplar generation.
• 🧩 Structure-aware stylization. AdaInf injects enriched textual semantics, while CaVA aligns global structure and selected style tokens during sampling.
• ⚡ Multiple diffusion backbones. The demo supports both Stable Diffusion v1.5 and SDXL.
• 🖥️ Interactive demo. A Gradio interface is included for the released style-image + content-text setting.

🧠 Method

Stage	Description
🌌 Imagine	Qwen2.5-VL interprets the style image and content prompt, produces an enriched description, and guides a frozen diffusion model to create an imagined exemplar.
🎨 Infuse	AdaInf injects enriched textual semantics into the latent representation, while CaVA aligns global structure and selected style tokens throughout diffusion sampling.

🚧 Current release: the repository provides the style image + content text prompt setting through a Gradio demo.

📸 Results

The figure above presents qualitative comparisons with representative state-of-the-art image stylization methods across diverse content and style conditions.

🚀 Quick Start

1. 📦 Clone the repository

git clone https://github.com/Vincotto/MAIST.git
cd MAIST

2. 🤖 Prepare Qwen2.5-VL with Ollama

ollama pull qwen2.5vl:latest

3. 🪄 Launch the Gradio demo

python app.py

The first run may download the selected Stable Diffusion backbone.

Backbone	Resolution used in the paper
stable-diffusion-v1-5/stable-diffusion-v1-5	512 × 512
stabilityai/stable-diffusion-xl-base-1.0	1024 × 1024

Experiments were conducted on a single 48 GB GPU.

📚 Citation

Citation information and the BibTeX entry will be added after publication.

🙏 Acknowledgements

This project builds upon PyTorch, Diffusers, Gradio, Ollama, Qwen2.5-VL, Stable Diffusion v1.5, and SDXL.