In Japan, there has been an increase in the number of buildings built using cross-laminated timber (CLT) in order to utilize the abundant forest resources in the country. However, no studies have evaluated the environmental impact of the construction of CLT buildings in Japan. This study evaluates the environmental impacts from the start of construction to the completion of a real CLT building in Kumamoto city, Kyushu region, southern Japan. We investigated the input of the materials and energy used in the construction of the building. The environmental impact categories evaluated include climate change, ozone layer depletion, eutrophication, acidification, and photochemical oxidation. We found that the concrete used for the foundations, and the cement-based soil stabilizer used for ground reinforcement accounted for 42% of the greenhouse gas (GHG) emissions. The construction site was previously used as a seedbed field, necessitating ground reinforcement. Furthermore, the large foundations were designed in order to raise the low height of the wooden structure from the ground level. Developing and applying methods with lower environmental impacts for ground reinforcement and building foundations is recommended. In addition, we found that by using biomass-derived electricity in CLT manufacturing, the environmental impacts of CLT manufacturing could be reduced, thus reducing the environmental impacts of the entire building. The biogenic carbon fixed in the wooden parts during the building usage accounted for 32% of the total GHG emissions of the building construction. Since this biogenic carbon will be released to the atmosphere at the end-of-life stage of the building, a long-term usage of the CLT buildings and/or reuse of the CLT is recommended.