Based on the continuum theory

the dilatancy angle is related to the confining stress and plastic shear strain. Due to the tunnel excavation

the confining stress of rock mass decreases from the initial stress to the support force

while the plastic shear strain increases. Thus the dilatancy coefficient varies nonlinear. In this study

based on the unified strength failure criterion and non-associated flow rule

the potential plastic zone was divided into a finite number of concentric rings according to the equal decreased confining stress. A finite difference method was proposed by considering the effects of the intermediate principal stress and nonlinear dilatancy. Then the accuracy of this method was further verified using example analysis. The parameter analysis was conducted to study the effects of the intermediate principal stress

the critical softening parameter and the support load on the dilatancy angle in the plastic zone of rock mass. It was found that the peak dilatancy increased with the increase of the intermediate principal stress. The variation rate of dilatancy angle became slowly with the increase of the critical softening parameter. The intermediate principal stress and the critical softening parameter showed corporate effects on the variation of dilatancy angle in the plastic zone. The dilatancy angle at the tunnel wall increased with the increase of the support load. The double shear strength criterion should be used cautiously in the tunnel because the calculated displacement at the tunnel wall is small. Additionally

when the Mohr-Coulomb strength criterion is used

it is proper to consider the bearing potential of rock mass.