Solid-state Li-metal batteries with solid-state electrolytes have attracted increasing attention due to their high energy density and intrinsically high safety. Among diverse available solid-state electrolytes, cellulose-based solid polymer electrolytes (CSPEs) are particularly attractive and have showcased great promise because of their multiple merits including abundant reserves, abundant polar groups, chemical stability and high flexibility. This review surveys currently-developed solid electrolytes based on modified cellulose and its composites with diverse organic and inorganic fillers. Common preparation methods for solid electrolyte membranes are discussed in detail, followed by a sequential overview of various modification and compositing strategies for improving Li-ion transport in CSPEs, and a summary of the current existing challenges and future prospects of CSPEs to achieve high-performance solid batteries.