Stress state of supporting structure and the retained soil was analyzed with elastoplastic FEM in foundation pit; and the value of limiting state function of retained soil was calculated. Based on the nodes and meshs

with “artificial participation” optimization

the shape and position of potential bulge fracture plane in foundation pit were searched with the DIJKSTRA optimization algorithm; and the relative safety factor was calculated. Large scale numeric calculation and in situ measurements show that the end point of potential fracture plane is insensitive to the insertion ratio of supporting pile

but

the start point of potential fracture plane is close to the supporting pile when the insertion of pile is increasing; and that the method of potential bulge fracture plane searching proposed in this paper is not restrict to the man-made fracture plane shape and limiting earth pressure assumption. The in-situ measurements show the availability and utilization of method of the potential bulge fracture plane searching based on the DIJKSTRA algorithm in composite optimization.