Technology

Art

LILAC: A Layer-Based Framework for Multi-Concept Customization of Diffusion Models Without Joint Training

[Submitted on 6 Jul 2026]

1h ago· 2 min readenInsight

technology science artificial intelligence computer vision

Summary

This paper introduces LILAC (Layer-Wise Independent LoRAs and Cascaded Conditioning), a framework for multi-concept customization of text-to-image diffusion models. Unlike existing methods that fuse independently trained concept adapters in a shared weight space (causing identity confusion and style bleeding), LILAC composes concepts as separate image layers with cascaded conditioning. Each subject is conditioned on the frozen composite of previously placed subjects, with exactly one adapter active at a time, preventing parameter-level interference. The approach requires no joint training, scales linearly with the number of concepts, and is backbone-agnostic. Applied on Qwen-Image-Edit under the Orthogonal Adaptation protocol, LILAC achieves an ArcFace detection rate of 0.861 compared to 0.745 for the original method.

Source

bskyLILAC: A Layer-Based Framework for Multi-Concept Customization of Diffusion Models Without Joint Trainingarxiv.org

Key quotes

· 3 pulled

We show that composing concepts as separate image layers, instead of merging their adapters in a shared weight space, avoids parameter-level interference.

LILAC composes the adapters without any joint training, scales linearly with the number of concepts, and is backbone-agnostic.

Each subject is conditioned on the frozen composite of previously placed subjects, with exactly one adapter active at a time, therefore identities never interfere at the parameter level.

Snippet from the RSS feed

Personalizing text-to-image diffusion models to render several specific subjects in a coherent image remains challenging: the model must preserve each subject's identity while keeping the scene spatially and visually coherent. Methods that fuse independen

You might also wanna read

Qwen-Image-Layered: Diffusion Model for Image Decomposition into Semantic Layers for Enhanced Editability

Qwen-Image-Layered is a novel diffusion model that decomposes single RGB images into multiple semantic layers to enable inherent editability

huggingface.co·6mo ago

Training-Free Single-Image Diffusion Model Achieves Fast, High-Quality Generation

This paper presents a training-free approach to single-image diffusion models. Instead of training a neural network on a single image (which

arxiv.org·1mo ago

iLLaDA: An 8B Masked Diffusion Language Model Trained with Bidirectional Attention

The paper introduces iLLaDA, an 8-billion parameter masked diffusion language model trained from scratch with fully bidirectional attention,

arxiv.org·12d ago

Iris: Pure C Implementation of Flux 2 Image Generation Model Inference Pipeline

Iris is a pure C implementation of an inference pipeline for image generation models, specifically designed for Flux 2 models. The project p

github.com·5mo ago

Research Study: Effectiveness of Adaptive Merging for Recycling LoRA Modules from Public Repositories

This research paper examines the effectiveness of adaptive merging methods for recycling LoRA (Low-Rank Adaptation) modules from public repo