In recent years, the amount of Web services has increased dramatically, and service discovery aiming to help users identify appropriate services matching their requirements thus becomes increasingly important. Many studies based on machine learning techniques have been reported to improve the performance of service discovery. A major obstacle in Web service discovery is the data sparsity in service descriptions. Towards this issue, in this paper, we propose a novel approach based on enriched neural topic model (NTM) and attentional Bi-LSTM. To alleviate the data sparsity issue, we enrich the semantics of each word in service descriptions and queries using external knowledge sources like Wikipedia and combine NTM and the attention mechanism to minimize the noise brought in the enrichment process. Experiments conducted on a real-world dataset show that our approach outperforms several state-of-the-art methods.