Transformer-based models have gained increasing popularity for time-series prediction; however, in specific applications such as residential heating systems, static contextual data of buildings is crucial to effectively capture and learn complex environmental dynamics. This paper presents a novel transformer-based model that adapts the contextual meta-data of residential buildings, generalizing across diverse environments. The model integrates temporal data with adaptive embedding of building-specific contextual meta-data such as geographic locations and building characteristics to dynamically learn and adapt to the variations. These adaptive context embeddings allow the model to comprehensively understand how different buildings respond to environmental changes over time. Initial results show improved accuracy and reliability in indoor temperature predictions of residential buildings, demonstrating the model’s potential to optimize district heating systems across a diverse array of residential buildings. This proposed model provides a basis for developing proactive heat management systems in buildings.