Autonomous driving decision-making is a great challenge in complex traffic environment, and the deep reinforcement learning (DRL) can contribute to the more intelligent strategy. In the autonomous driving scenarios with DRL algorithms, sufficient exploration to the traffic environment is vital for constructing the state spaces, training the driving decision model and transferring to a new environment. In this paper, three different noise modes are presented to investigate the performance of noisy exploration and generalization in self-driving tasks. Extensive experiments indicate that the noisy exploration is not necessary for the easy traffic environments, and the correlated noisy exploration is an effective technique in generalizing to complex traffic environments, while the uncorrelated noisy exploration may result in a counter-productive effect in inertial autonomous driving system.