Cloud radio access networks (C-RANs) have been regarded in recent times as a promising concept in future 5G technologies where all DSP processors are moved into a central base band unit (BBU) pool in the cloud, and distributed remote radio heads (RRHs) compress and forward received radio signals from mobile users to the BBUs through radio links. In such dynamic environment, automatic decision-making approaches, such as artificial intelligence based deep reinforcement learning (DRL), become imperative in designing new solutions. In this paper, we propose a generic framework of autonomous cell activation and customized physical resource allocation schemes to balance energy consumption and QoS satisfaction in wireless networks. We formulate the cell activation problem as a Markov decision process and set up a revised reinforcement learning model based on K-means clustering and anchor-graph hashing to satisfy the QoS requirements of users and to achieve low energy consumption with the minimum number of active RRHs under varying traffic demand and user mobility. Extensive simulations are conducted to show the effectiveness of our proposed solution compared with existing schemes.