In this paper, we consider the problem of assessing when the control of a vehicle can be safely transferred from an automated driving system to the driver. We propose a method based on a description of the driver's capabilities to maneuver the vehicle, which is defined as a subset of the vehicle's state space and called the driver controllability set (DCS). Since drivers' capabilities vary among individuals, the DCS is updated online during manual driving. By identifying the limits of the individual driver's normal driving envelope, we find the estimated bounds of the DCS. Using a vehicle model and reachability analysis, we assess whether the states of the vehicle start and remain within the DCS during the transition to manual driving. Only if the states are within the DCS is the transition to manual driving classified as safe. We demonstrate the estimation of the DCS for four drivers based on the data collected with real vehicles in highway and city driving. Experiments on transitions to manual driving are also conducted with real vehicles. Results show that the proposed method can be implemented with a real system to classify transitions from automated to manual driving.