TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion
2024-01
发表期刊SENSORS (IF:3.4[JCR-2023],3.7[5-Year])
ISSN1424-8220
EISSN1424-8220
卷号24期号:1
发表状态已发表
DOI10.3390/s24010020
摘要

In the field of image fusion, the integration of infrared and visible images aims to combine complementary features into a unified representation. However, not all regions within an image bear equal importance. Target objects, often pivotal in subsequent decision-making processes, warrant particular attention. Conventional deep-learning approaches for image fusion primarily focus on optimizing textural detail across the entire image at a pixel level, neglecting the pivotal role of target objects and their relevance to downstream visual tasks. In response to these limitations, TDDFusion, a Target-Driven Dual-Branch Fusion Network, has been introduced. It is explicitly designed to enhance the prominence of target objects within the fused image, thereby bridging the existing performance disparity between pixel-level fusion and downstream object detection tasks. The architecture consists of a parallel, dual-branch feature extraction network, incorporating a Global Semantic Transformer (GST) and a Local Texture Encoder (LTE). During the training phase, a dedicated object detection submodule is integrated to backpropagate semantic loss into the fusion network, enabling task-oriented optimization of the fusion process. A novel loss function is devised, leveraging target positional information to amplify visual contrast and detail specific to target objects. Extensive experimental evaluation on three public datasets demonstrates the model’s superiority in preserving global environmental information and local detail, outperforming state-of-the-art alternatives in balancing pixel intensity and maintaining the texture of target objects. Most importantly, it exhibits significant advantages in downstream object detection tasks. © 2023 by the authors.

关键词Behavioral research Decision making Deep learning Image enhancement Image fusion Object detection Object recognition Pixels Textures Deep learning Detection tasks Down-stream High-level vision task High-level visions Infrared and visible image Objects detection Target driven Target object Vision transformer
URL查看原文
收录类别EI ; SCI
语种英语
WOS研究方向Chemistry ; Engineering ; Instruments & Instrumentation
WOS类目Chemistry, Analytical ; Engineering, Electrical & Electronic ; Instruments & Instrumentation
WOS记录号WOS:001140690300001
出版者Multidisciplinary Digital Publishing Institute (MDPI)
EI入藏号20240215365529
EI主题词Vision
EI分类号461.4 Ergonomics and Human Factors Engineering ; 723.2 Data Processing and Image Processing ; 912.2 Management ; 971 Social Sciences
原始文献类型Journal article (JA)
引用统计
正在获取...
文献类型期刊论文
条目标识符https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348633
专题信息科学与技术学院
信息科学与技术学院_硕士生
通讯作者Liu, Shijian
作者单位
1.ShanghaiTech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China
2.Chinese Acad Sci, Shanghai Inst Tech Phys, Shanghai 200083, Peoples R China
3.Chinese Acad Sci, Key Lab Infrared Syst Detect & Imaging Technol, Shanghai 200083, Peoples R China
4.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
第一作者单位信息科学与技术学院
第一作者的第一单位信息科学与技术学院
推荐引用方式
GB/T 7714
Lu, Siyu,Ye, Xiangzhou,Rao, Junmin,et al. TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion[J]. SENSORS,2024,24(1).
APA Lu, Siyu,Ye, Xiangzhou,Rao, Junmin,Li, Fanming,&Liu, Shijian.(2024).TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion.SENSORS,24(1).
MLA Lu, Siyu,et al."TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion".SENSORS 24.1(2024).
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
个性服务
查看访问统计
谷歌学术
谷歌学术中相似的文章
[Lu, Siyu]的文章
[Ye, Xiangzhou]的文章
[Rao, Junmin]的文章
百度学术
百度学术中相似的文章
[Lu, Siyu]的文章
[Ye, Xiangzhou]的文章
[Rao, Junmin]的文章
必应学术
必应学术中相似的文章
[Lu, Siyu]的文章
[Ye, Xiangzhou]的文章
[Rao, Junmin]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。