We consider a dynamic server allocation problem over parallel queues with randomly varying connectivity and server switchover delay between the queues. At each time slot, the server decides either to stay with the current queue or switch to another queue based on the current connectivity and the queue length information. Switchover delay occurs in many telecommunications applications and is a new modeling component of this problem that has not been previously addressed. We show that the simultaneous presence of randomly varying connectivity and switchover delay changes the system stability region and the structure of optimal policies. In the first part of this paper, we consider a system of two parallel queues, and develop a novel approach to explicitly characterize the stability region of the system using state-action frequencies which are stationary solutions to a Markov decision process formulation. We then develop a frame-based dynamic control (FBDC) policy, based on the state-action frequencies, and show that it is throughput optimal asymptotically in the frame length. The FBDC policy is applicable to a broad class of network control systems and provides a new framework for developing throughput-optimal network control policies using state-action frequencies. Furthermore, we develop simple myopic policies that provably achieve more than 90% of the stability region. In the second part of this paper, we extend our results to systems with an arbitrary finite number of queues. In particular, we show that the stability region characterization in terms of state-action frequencies and the throughput optimality of the FBDC policy follows for the general case. Furthermore, we characterize an outer bound on the stability region and an upper bound on sum throughput and show that a simple myopic policy can achieve this sum-throughput upper bound in the corresponding saturated system. Finally, simulation results show that the myopic policies may achieve the full stability region and are more delay efficient than the FBDC policy in most cases.