Graphene oxide (GO), a derivative of graphene, is a novel carbon material that has attracted a lot of attention in the field of membrane materials as its ability to achieve layer-by-layer stacking and the formation of nanochannels between the lamellae makes it excellent for selective separation of substances. In this paper, the separation mechanism of the GO membrane is summarized. According to the different separation substances, the separation mechanism of graphene oxide membrane is reviewed from two aspects of metal ions and organic pollutants. Next, the preparation methods of graphene oxide membranes is introduced, such as spin-coating, vacuum filtration, dip-coating, spraying, and layer-by-layer self-assembly, followed by a review on the structural regulation of GO. Finally, this paper concludes with an overview of the potential development prospects and challenges of GO membranes.