Deeper Attention-Based Network for Structured Data

X Wu, Y Fan, W Peng, H Pang, Y Luo - International Conference of …, 2020 - Springer
X Wu, Y Fan, W Peng, H Pang, Y Luo
International Conference of Pioneering Computer Scientists, Engineers and …, 2020Springer
Deep learning methods are applied into structured data and in typical methods, low-order
features are discarded after combining with high-order featuresfor prediction tasks.
However, in structured data, ignorance of low-order features may cause the low prediction
rate. To address this issue, in this paper, deeper attention-based network (DAN) is
proposed. With DAN method, to keep both low-and high-order features, attention average
pooling layer was utilized to aggregate features of each order. Furthermore, by shortcut …
Abstract
Deep learning methods are applied into structured data and in typical methods, low-order features are discarded after combining with high-order featuresfor prediction tasks. However, in structured data, ignorance of low-order features may cause the low prediction rate. To address this issue, in this paper, deeper attention-based network (DAN) is proposed. With DAN method, to keep both low- and high-order features, attention average pooling layer was utilized to aggregate features of each order. Furthermore, by shortcut connections from each layer to attention average pooling layer, DAN can be built extremely deep to obtain enough capacity. Experimental results show DAN has good performance and works effectively.
Springer
Showing the best result for this search. See all results