The emergence of 6 G requires efficient management of heterogeneous networks and computational resources to achieve targeted end-to-end network automation, with slice orchestration as a key feature. Despite opportunities offered by the recent advances in network virtualization and distributed cloud infrastructures, these developments introduce complexity in the context of multi-domain networks. This paper presents a distributed horizontal scaling method that leverages deep reinforcement learning (DRL) to enhance network function orchestration (NFO) with intelligent scaling decisions, facilitating seamless cross-domain information exchange. Firstly, we develop a DRL agent designed to handle fluctuating traffic loads and generate scaling actions tailored for the considered network function (NF). The trained DRL agent is then integrated into a multi-domain message exchange scaling framework with traffic prediction capabilities. Moreover, a simulation testbed is developed to manipulate multi-domain topologies, customize network slices, and enable precise per-slice and per-domain scaling decisions. Our DRL-based solution outperforms the Horizontal Pod Autoscaling (HPA) heuristic used by Kubernetes, improving resource utilization and reducing data rate losses.