The building energy demand for heating and cooling is changing due to climate change. The adoption of climate change adaptation measures at the building scale aims at limiting heating and cooling demands. In previous studies on adaptation measures little attention has been paid to lightweight semi-portable buildings, which are increasingly used to temporarily house the growing number of small households (1–2 persons) in peripheral and derelict areas. In this paper the impact of passive climate adaptation measures and building orientation on heating and cooling demands is assessed for a detached, lightweight, semi-portable residential building by means of building energy simulations (BES), considering two climate scenarios for the Netherlands: current climate and a future climate (2050). The results show that the most efficient adaptation measure consists in a combination of exterior solar shading and an increase of thermal resistance of the building envelope, which reduces the annual heating and cooling demand – averaged over eight building orientations – by 11% for the current climate and 15% for the future climate. The impact of building orientation varies according to the climate scenario. Compared to the average over the eight orientations considered, the annual cooling demand for a single orientation varies between about −31% and +22% and between about −24% and +18% for the current and future climate, respectively. For the case without adaptation measures, optimizing the building orientation leads to annual total energy savings of about 4% for the current and 3% for the future climate.