Abstract: The traditional power load combination model uses the historical data prediction error in a rolling and fixed-length time window to perform sub-model variable weights. However, the window length cannot be adaptively adjusted according to the latest environmental characteristics, resulting in the loss of effective information or the introduction of outdated information, thereby reducing the accuracy of short-term load forecasting. In this study, a twin-delay deep deterministic policy gradient（TD3） was used to construct the agent, and a variable weight combination forecasting strategy with an adaptive variable time window length was designed. By establishing the target and related constraints with the lowest error in a short-term load forecasting scenario, the input state, action, and reward mechanism of the agent are designed such that the agent can quickly converge and make an optimal decision to accurately adjust the length of the time window. Consequently, the combination model responds to the time window guided by the agent in real time, and the optimal weighting method is used to realize an accurate variable weight combination of the submodels. Finally, real power load data from a certain area in northern China were used to verify the effectiveness and superiority of the proposed strategy.