Sonodynamic therapy (SDT), as a novel non-invasive strategy for eliminating tumor, has the advantages of deeper tissue penetration, fewer side effects, and better patient compliance, compared with photodynamic therapy (PDT). In SDT, ultrasound was used to activate sonosensitizer to produce cytotoxic reactive oxygen species (ROS), induce the collapse of vacuoles in solution, and bring about irreversible damage to cancer cells. In recent years, much effort has been devoted to developing highly efficient sonosensitizers which can efficiently generate ROS. However, the traditional organic sonosensitizers, such as porphyrins, hypericin, and curcumins, suffer from complex synthesis, poor water solubility, and low tumor targeting efficacy which limit the benefits of SDT. In contrast, inorganic sonosensitizers show good in vivo stability, controllable physicochemical properties, ease of achieving multifunctionality, and high tumor targeting, which greatly expanded their application in SDT. In this review, we systematically summarize the nanomaterials which act as the carrier of molecular sonosensitizers, and directly produce ROS under ultrasound. Moreover, the prospects of inorganic nanomaterials for SDT application are also discussed.