To consider the effect of nonlinear changes of the shear strength of unsaturated soil in the two dimensions of net normal stress and matric suction on the passive earth pressure

a calculation method for the passive earth pressure on the back of retaining walls for unsaturated backfill is proposed based on the nonlinear shear strength envelope shell and the classical Rankine earth pressure theory. First

employing the generalized Mohr-Coulomb failure criterion

we applied the variable tangent technique to determine equivalent cohesion

equivalent friction angle

and their correlations at arbitrary points on the envelope. Subsequently

through iterative analysis combining Mohr’s circle stress states

we determined the equivalent cohesion and friction angle under passive limit equilibrium conditions. Finally

we derived Rankine passive earth pressure expressions for unsaturated soils that maintain formal consistency with classical soil mechanics formulations. The method’s reliability was validated through comparative case studies with established models. Parametric analysis revealed that increasing matric suction and overburden pressure enhances equivalent cohesion and passive pressure

while reducing equivalent friction angle. In the case study

the envelope shell model yielded lower passive earth pressures than the plane model at low matric suction levels

but higher pressures at elevated suction levels.