Construction of operational control rules for an earth-to-air heat exchanger through transfer reinforcement learning

In recent years, earth-to-air heat exchanger (EAHE) systems, which is a method of pre-cooling and pre-heating outdoor air with earth-to-air heat, have been attracting attention as one of the technologies to achieve ZEB. However, at the operational phase, in order to achieve both energy saving and suppression of dew condensation control, EAHE control methods such as the timing or amount of outdoor air introduction have not been established. Recently, research on operational control by reinforcement learning (RL) has become popular and has attracted attention in the field of air conditioning control. RL is effective even in cases where future states are difficult to predict, such as EAHE. In previous studies, the unsteady CFD analysis method proposed by the authors made it possible to evaluate the annual energy savings and dew condensation in EAHE in detail. In addition, it was clarified that the RL, which uses the same CFD method as a simulator, can establish a control law that achieves both energy-saving effects and prevent indoor air pollution by suppressing dew condensation. On the other hand, RL requires a huge number of trials to construct the control law.
Therefore, the purpose of this study is to improve the learning speed and control performance. First, we adopt transfer learning (TL), which reuses a model pre-trained in RL for training in a new environment. Next, we verify the effectiveness of using this transfer reinforcement learning (TRL) as a control method for EAHE. The result showed that TRL achieved better control performance and faster learning speed than RL. In addition, it was suggested that EAHE with insufficient actual measurement data may be efficiently controlled from the first year of operation by directly using the control law established in advance. It was confirmed that RL performs well in terms of energy efficiency and air quality maintenance.