RGBD Based Gaze Estimation via Multi-Task CNN

	RGBD Based Gaze Estimation via Multi-Task CNN
	Lian, Dongze1 ; Zhang, Ziheng1 ; Luo, Weixin1 ; Hu, Lina1 ; Wu, Minye1 ; Li, Zechao 2; Yu, Jingyi1 ; Gao, Shenghua1
	2019
会议录名称	THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE
页码	2488-2495
发表状态	已发表
摘要	This paper tackles RGBD based gaze estimation with Convolutional Neural Networks (CNNs). Specifically, we propose to decompose gaze point estimation into eyeball pose, head pose, and 3D eye position estimation. Compared with RGB image-based gaze tracking, having depth modality helps to facilitate head pose estimation and 3D eye position estimation. The captured depth image, however, usually contains noise and black holes which noticeably hamper gaze tracking. Thus we propose a CNN-based multi-task learning framework to simultaneously refine depth images and predict gaze points. We utilize a generator network for depth image generation with a Generative Neural Network (GAN), where the generator network is partially shared by both the gaze tracking network and GAN-based depth synthesizing. By optimizing the whole network simultaneously, depth image synthesis improves gaze point estimation and vice versa. Since the only existing RGBD dataset (EYEDIAP) is too small, we build a large-scale RGBD gaze tracking dataset for performance evaluation. As far as we know, it is the largest RGBD gaze dataset in terms of the number of participants. Comprehensive experiments demonstrate that our method outperforms existing methods by a large margin on both our dataset and the EYEDIAP dataset.
会议录编者/会议主办者	Association for the Advancement of Artificial Intelligence
关键词	Generative adversarial networks Large dataset Image enhancement Neural networks Black holes Depth image Eye position Gaze estimation Gaze point estimations Gaze tracking Head Pose Estimation Large margins
会议名称	33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Annual Conference on Innovative Applications of Artificial Intelligence, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019
出版地	2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA
会议地点	Honolulu, HI, United states
会议日期	January 27, 2019 - February 1, 2019
收录类别	CPCI ; CPCI-S ; EI
语种	英语
资助项目	NSFC[61502304]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000485292602062
出版者	ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE
EI入藏号	20203509101183
EI主题词	Eye tracking
EI分类号	723.2 Data Processing and Image Processing ; 723.4 Artificial Intelligence
原始文献类型	Proceedings Paper
引用统计	被引频次：25[WOS] [WOS记录] [WOS相关记录]
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/80215
专题	信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_PI研究组_高盛华组信息科学与技术学院_博士生
通讯作者	Gao, Shenghua
作者单位	1.ShanghaiTech Univ, Shanghai, Peoples R China 2.Nanjing Univ Sci & Technol, Nanjing, Jiangsu, Peoples R China
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Lian, Dongze,Zhang, Ziheng,Luo, Weixin,et al. RGBD Based Gaze Estimation via Multi-Task CNN[C]//Association for the Advancement of Artificial Intelligence. 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA:ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE,2019:2488-2495.