Sugar transportation and sugar-to-starch metabolism are considered important processes in seed development and embryo viability. A few plant SWEET proteins acting as sugar transporters have been reported to function in inflorescence and/or seed development. Here, we identified seven members of the 21 OsSWEET genes in rice that play essential roles in sugar transportation and sugar-to-starch conversion in seed development. Nineteen OsSWEET genes exhibiting different expression patterns during inflorescence and seed development were knocked out individually by CRISPR/Cas9. One third of the mutants showed decreased fertile pollen viability and shriveled mature caryopses, resulting in weakened seed traits. Grain fill-related genes but not representative grain shape-regulating genes showed attenuated expression in the mutants. Seed of each of these mutants accumulated more sucrose, glucose or fructose but less starch. Among all OsSWEET genes, OsSWEET4 and OsSWEET11 had major effects on caryopsis development. The sugar-to-starch metabolic pathway was significantly altered in ossweet11 mutants based on differential expression analysis in RNA sequencing assays, confirming that OsSWEET11 functions as a sugar transporter with a key role in seed development. These results help to decipher the multiple functions of OsSWEET genes and to show how they might be used in genetic improvement of rice.