Abstract: Development and utilization degree of unconventional water resources in China are relatively low at present, and water resources allocation is complicated with a variety of uncertainties. Aimed at this problem, this paper applies the method of reinforcement learning to water resources allocation, and develops an optimal allocation model of Python coding with the objective function of maximizing the economic benefit. This model is applied in a case study of Beijing, taking the unconventional water resources into account, and focusing on comparison of different allocation schemes and their interval scales of economic profits. Results show it predicts the overall profits under different water inflow conditions satisfactorily, giving larger values than those predicted using the two-stage stochastic programming method. And utilizing the unconventional water resources would play a significant role in alleviating the city’s water shortage.