The paper investigates the issue of physical layer security in spectrum overlay networks. First, in order to enhance security performance of the primary system, the secondary users are motivated to act as friendly jamming nodes with the compensation of access to the primary system's spectrum opportunistically. Considering the non-altruistic or rational attribute in practice, we propose a cooperative jamming approach based on Stackelberg game, while primary users acting as the game leader, and secondary users constituting the follower. Under the proposed framework, we design a new mechanism that the secondary users transmit jamming signals according to a pre-determined probability, in order to maximize their own data rate priced by the invested power. To be specific, the more accessible spectrum opportunities, the larger possibility that secondary users would jam. For primary users, their traffic load can be tuned so as to squeeze spectrum holes for secondary access. Therefore, the considered system is dynamic and either primary users or secondary users can operate in the channel. We employ continuous-time Markov chain to explicitly model the system's evolutionary behaviors. Under the proposed scheme, the optimal strategies of primary and secondary users, which are jointly referred to as Stackelberg equilibrium, are analyzed in details. Finally, we demonstrate the performance of the proposed scheme by numerical simulations.