Brain asymmetry, involving structural and functional differences between the two hemispheres, is a major organizational principle of the human brain. The structural and functional connectivity within each hemisphere defines the hemispheric network or connectome. Elucidating left-right differences of the hemispheric network provides opportunities for brain asymmetry exploration. This review examines the asymmetry in the hemispheric white matter and functional network to assess health and brain disorders. In this article, the brain asymmetry in structural and functional connectivity including network topologies of healthy individuals, involving brain cognitive systems and the development trend, is highlighted. Moreover, the abnormal asymmetry of the hemispheric network related to cognition changes in brain disorders, such as Alzheimer’s disease, schizophrenia, autism spectrum disorder, attention deficit hyperactivity disorder, and bipolar disorder, is presented. This review suggests that the hemispheric network is highly conserved for measuring human brain asymmetries and has potential in the study of the cognitive system and brain disorders.