Automated Web Browsing tools, such as Selenium and headless browsers, are used to collect traffic traces from networked applications, with which statistical models describing the traffic are obtained. However, we show that traces from Selenium and headless browsers have markedly different traffic characteristics than human generated traces, with potential impact on the quality of the obtained models. To overcome this limitation, we propose RLBrowse, an automated web automation framework that imitates human browsing habits by separating web automation from the browser using reinforcement learning. By separating the browser and automation tool, RLBrowse improves on 9 out of the 13 traffic trace features tested. The distribution of packet sizes in a trace improves the most, with a nearly 400 % improvement. We test RLBrowse by collecting a corpus of network packet traces on a set of human-navigated website browsing sessions, and by RLBrowse and Selenium. In the subsequent analysis, we identify key differences in the resulting packet traces.