K-StyleLoRA: Information-Guided Image Generation via Selective Feature Learning

Soyoung Lee, Hyoungseo Cho, Myungjoo Kang, YoungJoon Yoo; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 4465-4472

Despite remarkable advances in image generation, existing diffusion models struggle to capture diverse cultural aesthetics. While Low-Rank Adaptation (LoRA) enables efficient fine-tuning, conventional approaches lack semantic awareness. They apply uniform adaptations across all features, leading to suboptimal cultural representation. To address these limitations, we introduce K-StyleLoRA, a novel framework that leverages CLIP's cross-modal capabilities for cultural image generation. Our approach introduces two key contributions. First, CLIP-Guided Information Gating dynamically modulates LoRA adaptations based on cultural relevance scores, selectively enhancing relevant features while suppressing irrelevant ones. Second, Cultural Semantic Loss provides additional semantic guidance by optimizing CLIP-based similarity to cultural concepts. Extensive experiments on Korean traditional culture show superior cultural fidelity while maintaining generation quality and diversity. In particular, K-StyleLoRA demonstrates exceptional cultural transfer capability on generic prompts that require implicit cultural understanding. Our method achieves a Cultural Similarity Score of 0.274, representing a 9.6% improvement over the vanilla SDXL baseline (0.250). Our framework establishes semantic-aware adaptation as a powerful paradigm for cultural representation. This scalable approach can be extended to diverse cultural contexts and generation tasks beyond Korean aesthetics.