ShanghaiTech University Knowledge Management System
TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion | |
2024-01 | |
发表期刊 | SENSORS (IF:3.4[JCR-2023],3.7[5-Year]) |
ISSN | 1424-8220 |
EISSN | 1424-8220 |
卷号 | 24期号:1 |
发表状态 | 已发表 |
DOI | 10.3390/s24010020 |
摘要 | In the field of image fusion, the integration of infrared and visible images aims to combine complementary features into a unified representation. However, not all regions within an image bear equal importance. Target objects, often pivotal in subsequent decision-making processes, warrant particular attention. Conventional deep-learning approaches for image fusion primarily focus on optimizing textural detail across the entire image at a pixel level, neglecting the pivotal role of target objects and their relevance to downstream visual tasks. In response to these limitations, TDDFusion, a Target-Driven Dual-Branch Fusion Network, has been introduced. It is explicitly designed to enhance the prominence of target objects within the fused image, thereby bridging the existing performance disparity between pixel-level fusion and downstream object detection tasks. The architecture consists of a parallel, dual-branch feature extraction network, incorporating a Global Semantic Transformer (GST) and a Local Texture Encoder (LTE). During the training phase, a dedicated object detection submodule is integrated to backpropagate semantic loss into the fusion network, enabling task-oriented optimization of the fusion process. A novel loss function is devised, leveraging target positional information to amplify visual contrast and detail specific to target objects. Extensive experimental evaluation on three public datasets demonstrates the model’s superiority in preserving global environmental information and local detail, outperforming state-of-the-art alternatives in balancing pixel intensity and maintaining the texture of target objects. Most importantly, it exhibits significant advantages in downstream object detection tasks. © 2023 by the authors. |
关键词 | Behavioral research Decision making Deep learning Image enhancement Image fusion Object detection Object recognition Pixels Textures Deep learning Detection tasks Down-stream High-level vision task High-level visions Infrared and visible image Objects detection Target driven Target object Vision transformer |
URL | 查看原文 |
收录类别 | EI ; SCI |
语种 | 英语 |
WOS研究方向 | Chemistry ; Engineering ; Instruments & Instrumentation |
WOS类目 | Chemistry, Analytical ; Engineering, Electrical & Electronic ; Instruments & Instrumentation |
WOS记录号 | WOS:001140690300001 |
出版者 | Multidisciplinary Digital Publishing Institute (MDPI) |
EI入藏号 | 20240215365529 |
EI主题词 | Vision |
EI分类号 | 461.4 Ergonomics and Human Factors Engineering ; 723.2 Data Processing and Image Processing ; 912.2 Management ; 971 Social Sciences |
原始文献类型 | Journal article (JA) |
引用统计 | 正在获取...
|
文献类型 | 期刊论文 |
条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348633 |
专题 | 信息科学与技术学院 信息科学与技术学院_硕士生 |
通讯作者 | Liu, Shijian |
作者单位 | 1.ShanghaiTech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China 2.Chinese Acad Sci, Shanghai Inst Tech Phys, Shanghai 200083, Peoples R China 3.Chinese Acad Sci, Key Lab Infrared Syst Detect & Imaging Technol, Shanghai 200083, Peoples R China 4.Univ Chinese Acad Sci, Beijing 100049, Peoples R China |
第一作者单位 | 信息科学与技术学院 |
第一作者的第一单位 | 信息科学与技术学院 |
推荐引用方式 GB/T 7714 | Lu, Siyu,Ye, Xiangzhou,Rao, Junmin,et al. TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion[J]. SENSORS,2024,24(1). |
APA | Lu, Siyu,Ye, Xiangzhou,Rao, Junmin,Li, Fanming,&Liu, Shijian.(2024).TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion.SENSORS,24(1). |
MLA | Lu, Siyu,et al."TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion".SENSORS 24.1(2024). |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 |
修改评论
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。