The displacement of residual oil by water flooding in porous media is an important mechanism of enhanced oil recovery in many sandstone reservoirs. Nonetheless, our basic understanding of the influence of complex pore geometries of natural porous media on fluid distribution is still incomplete. Herein, two-phase flow simulations were performed to investigate the pore-scale dynamics of imbibition in a heterogeneous sandstone rock sample. Furthermore, the relationship between residual oil distribution and pore structure parameters was quantitatively characterized based on a pore-throat segmentation method. The findings suggest that the pore-scale displacement and snap-off processes have a strong dependence on the coordination number and aspect ratio. The entrapment and remobilization of oil clusters were also analyzed under continuous and discontinuous displacement modes. In addition, a new quantitative method to evaluate the displacement potential and mobilization pattern of remaining oil was presented and discussed. Statistical analysis revealed that the development of sub-pathways and the suppression of snap-off are responsible for the decrease in residual oil saturation with increasing capillary number during water injection. Moreover, the connected residual oil clusters trapped in pores with high coordination number prefer to be displaced and produced. Finally, the displacement modes with different capillary numbers under different initial oil distributions were evaluated to explain the effect of pore structure. By incorporating these correlations of displacement events with pore-throat geometry, existing predictive models can be improved, which could be helpful for the fine tapping of highly disconnected remaining oil in sandstone reservoirs.