Software Defined Networking (SDN) is a new paradigm which allows the network devices to be more programmable and easily controlled. SDN can be applied to improve the flexibility and performance of mobile networks. As a key problem of SDN-based mobile networks, the mobility management still cannot simultaneously support various different scenarios and has the limitations of high signaling cost and handover delay. In order to overcome these drawbacks of existing related protocols, we propose a SDN-based mobility management scheme named H-SMM by constructing a hierarchical architecture in control plane to support intra-domain and inter-domain handover scenarios at the same time. Then we design handover procedures to simplify intra-domain handover and optimize inter-domain handover respectively. In addition, the communication initiation procedure is also provided. Qualitative analysis shows H-SMM is an overall mobility management scheme. Calculative analysis indicates that H-SMM has lower signaling cost. Simulation results illustrate that the handover delay and the end to end delay of first packet in communication initiation are lower than contrast schemes. Moreover, our evaluation also shows that the above performance advantages of H-SMM will be more evident with the increasing of domain numbers and distance between domains.