Abstract: The pricing mechanism of electricity spot market is one of the key issues in market design, which interacts with the transaction behavior of generators. The design of pricing mechanism needs to consider possible transaction behavior of generators. However, bidding strategy of generators may vary a lot under different pricing mechanisms. To solve this nested problem systematically, two papers of different focuses were formed. As the first part, this paper discussed the applicability of reinforcement learning in bidding optimization of generators. Considering the two-stage process of system marginal price (SMP) and zonal marginal price (ZMP), the bi-level optimization models of generators under three kinds of pricing mechanisms of locational marginal price (LMP), SMP and ZMP were constructed. Then, a multi-agent reinforcement learning (MARL) method combining win or learn fast and policy hill-climbing (WoLF-PHC) algorithm was proposed to solve the model iteratively. In the bi-level models, bidding decision-making models of generators served as the upper layers, following with the market clearing model as lower layer. The interactive data were composed of the bidding information optimized by the decision-making layers and the market clearing information calculated by the clearing layer. The bidding strategies of generators were optimized through continuous interaction. Finally, taking IEEE 39 system as an example, four typical load scenarios were selected to optimize the bidding of generators under three pricing mechanisms. The results show that the proposed model and algorithm can effectively solve the optimal bidding strategy of generators and reach the market equilibrium results.