This paper focused on energy consumption and carbon emission for heating and cooling during a building’s operation stage, and examined the energy effects of using Cross Laminated Timber (CLT) as an alternative building material to reinforced concrete (RC) in China’s 31 key cities located in different climate zones. The authors designed two seven-story residential buildings, which were constructed with RC framed and CLT systems, separately. This was followed by simulating the energy consumption using commercialized software IESTM under the different climate zones and calculating the carbon emissions. Comparisons were made between RC and CLT systems buildings on the basis of simulation data. The results show that the estimated energy consumption and carbon emission in CLT buildings are much lower than that of RC buildings in all studied cities, which indicates that CLT systems have good potential in reducing carbon emission and saving energy consumption compared to RC. The energy consumptions and carbon emissions in both concrete and CLT buildings are closely related to the climate zones. Buildings in Severe Cold and Cold Regions consumed the most energy and released more carbon. At the national level, the estimated energy consumption at the operation stage, in the studied building with RC frames and CLT system was approximately 465.1 MJ/m2 and 332.6 MJ/m2 per annum, respectively. Despite vast differences in China’s climate zones, the effects of energy saving and carbon reduction potentials of CLT buildings show little relationship to the climate zone. CLT buildings may result in a weighted 29.4% energy saving, which equals 24.6% carbon reductions, compared with RC buildings at the operation stage at national level, although it may vary in different climate zones.