Fast Personalized Text to Image Synthesis with Attention Injection

doi:10.1109/ICASSP48485.2024.10447042

	Fast Personalized Text to Image Synthesis with Attention Injection
	Yuxuan Zhang 1; Yiren Song 1; Jinpeng Yu2 ; Han Pan 1; Zhongliang Jing 1
	2024-04
会议录名称	ICASSP 2024 - 2024 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
ISSN	1520-6149
页码	6195-6199
发表状态	已发表
DOI	10.1109/ICASSP48485.2024.10447042
摘要	Currently, personalized image generation methods mostly require considerable time to finetune and often overfit the concept resulting in generated images that are similar to custom concepts but difficult to edit by prompts. We propose an effective and fast approach that could balance the text-image consistency and identity consistency of the generated image and reference image. Our method can generate personalized images without any fine-tuning while maintaining the inherent text-to-image generation ability of diffusion models. Given a prompt and a reference image, we merge the custom concept into generated images by manipulating cross-attention and self-attention layers of the original diffusion model to generate personalized images that match the text description. Comprehensive experiments highlight the superiority of our method. © 2024 IEEE.
会议录编者/会议主办者	The Institute of Electrical and Electronics Engineers Signal Processing Society
关键词	Personalized Text-to-Image Generation Computer Vision Deep Learning Diffusion models
会议名称	49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
会议地点	Seoul, Korea, Republic of
会议日期	14-19 April 2024
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20242416240132
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/354941
专题	信息科学与技术学院_硕士生
作者单位	1.Shanghai Jiao Tong University 2.ShanghaiTech University
推荐引用方式 GB/T 7714	Yuxuan Zhang,Yiren Song,Jinpeng Yu,et al. Fast Personalized Text to Image Synthesis with Attention Injection[C]//The Institute of Electrical and Electronics Engineers Signal Processing Society:Institute of Electrical and Electronics Engineers Inc.,2024:6195-6199.