Personalized Saliency and Its Prediction
Xu, Yanyu1,2; Gao, Shenghua1; Wu, Junru3; Li, Nianyi4; Yu, Jingyi1
2019-12
Source PublicationIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN0162-8828
EISSN1939-3539
Volume41Issue:12Pages:2975-2989
Status已发表
DOI10.1109/TPAMI.2018.2866563
Abstract

Nearly all existing visual saliency models by far have focused on predicting a universal saliency map across all observers. Yet psychology studies suggest that visual attention of different observers can vary significantly under specific circumstances, especially a scene is composed of multiple salient objects. To study such heterogenous visual attention pattern across observers, we first construct a personalized saliency dataset and explore correlations between visual attention, personal preferences, and image contents. Specifically, we propose to decompose a personalized saliency map (referred to as PSM) into a universal saliency map (referred to as USM) predictable by existing saliency detection models and a new discrepancy map across users that characterizes personalized saliency. We then present two solutions towards predicting such discrepancy maps, i.e., a multi-task convolutional neural network (CNN) framework and an extended CNN with Person-specific Information Encoded Filters (CNN-PIEF). Extensive experimental results demonstrate the effectiveness of our models for PSM prediction as well their generalization capability for unseen observers.

KeywordObservers Saliency detection Feature extraction Visualization Semantics Predictive models Image color analysis Universal saliency personalized saliency multi-task learning convolutional neural network
URL查看原文
Indexed BySCI
Language英语
Funding ProjectUS National Science Foundation[HCC-1319598] ; US National Science Foundation[IIS-1422477]
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS IDWOS:000498677600014
PublisherIEEE COMPUTER SOC
WOS KeywordMODEL
Original Document TypeArticle
Citation statistics
Cited Times:8[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttps://kms.shanghaitech.edu.cn/handle/2MSLDSTB/28257
Collection信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_虞晶怡组
信息科学与技术学院_PI研究组_高盛华组
Corresponding AuthorGao, Shenghua
Affiliation1.ShanghaiTech Univ, Shanghai 201210, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
3.Texas A&M Univ, College Stn, TX 77843 USA
4.Univ Delaware, Newark, DE 19716 USA
First Author AffilicationShanghaiTech University
Corresponding Author AffilicationShanghaiTech University
First Signature AffilicationShanghaiTech University
Recommended Citation
GB/T 7714
Xu, Yanyu,Gao, Shenghua,Wu, Junru,et al. Personalized Saliency and Its Prediction[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2019,41(12):2975-2989.
APA Xu, Yanyu,Gao, Shenghua,Wu, Junru,Li, Nianyi,&Yu, Jingyi.(2019).Personalized Saliency and Its Prediction.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,41(12),2975-2989.
MLA Xu, Yanyu,et al."Personalized Saliency and Its Prediction".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 41.12(2019):2975-2989.
Files in This Item: Download All
File Name/Size DocType Version Access License
10.1109@TPAMI.2018.2(1254KB)期刊论文作者原稿开放获取UnknownView Download
Related Services
Usage statistics
Scholar Google
Similar articles in Scholar Google
[Xu, Yanyu]'s Articles
[Gao, Shenghua]'s Articles
[Wu, Junru]'s Articles
Baidu academic
Similar articles in Baidu academic
[Xu, Yanyu]'s Articles
[Gao, Shenghua]'s Articles
[Wu, Junru]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Xu, Yanyu]'s Articles
[Gao, Shenghua]'s Articles
[Wu, Junru]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 10.1109@TPAMI.2018.2866563.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.