In response to the rapid increase in e-commerce, distribution services, and smart technology, businesses need to solve the vehicle routing problem (VRP) and its variants to improve service quality and cut operating costs. In this paper, we introduce the VRP with time windows and time costs (VRPTWTC), which is a new but practical VRP variant that adds time windows of customers and time costs of service constraints into the optimization of the traditional VRP problem. Furthermore, we propose a masking algorithm and combine it into our proposed end-to-end and data-driven deep reinforcement learning framework to solve the VRPTWTC. Experimental results show a shorter tour length and reasonable processing time than benchmarks both on a simulated dataset and a real enterprise-level dataset. Besides, violation of time window and total time cost, which is one of the most important criterions in our problem formulation, never occurred in our experiment under the real road connection conditions when using a real map API. To the best of our knowledge, our framework is the first to effectively and efficiently solve a VRP variant problem with practical time windows and time costs constraints with a real map API to calculate routes of VRP considering real road connection conditions on an enterprise-level dataset.