A three-dimensional soften model is established combined with statistical mechanics to take the heterogeneities on mesoscopic scale into consideration. Two sets of numerical tests are undertaken by using a numerical code RFPA3D to analyze the failure process of rocks subjected to uniaxial compression. Numerical specimens are prepared to investigate the influence of the geometry effects on the macro response of the rocks. One set of numerical tests are conducted without constraint and there are no friction between the loading plates and the specimens. Another set of numerical tests compared with laboratory experiments are conducted to simulate the geometry effect by taking end constraint effect into consideration. Numerical results show that the geometry of specimen influences both the peak strength and failure mode. The peak strength decreases as the ratio of length to width of the specimen increases; and it inclines to a certain value when the ratio exceeds 3.0. When the ratio is small

the failure of the specimens are caused by tensile fracture propagating the top and the bottom of specimens; while the ratio is larger enough

specimens tend to be in shear failure mode. End constraint effect is one of the key factors that lead to geometry effects. The plate restricts the lateral deformation and results in lateral stress in the middle of the specimen

which leads to tensile failure. However

geometry effects still exist even with smooth plates. The transition of the ductile failure mode to brittle mode can be found as the ratio of the length to the width of the specimens increase.