Yttria-stabilized zirconia (YSZ) has been used as a thermal barrier coating (TBC) material in gas turbines for several decades. Although continuous efforts have been made to develop novel TBC materials that can work at a higher temperature, no single material other than YSZ has all the desired attributes for the TBCs. In this paper, we report the in-situ synthesis of quasi-binary GdNbO₄/Gd₃NbO₇ composites based on the simple Gd₂O₃–Nb₂O₅ binary phase diagram. The fracture toughness of these quasi-binary composites is remarkably enhanced compared with the value predicted by the rule of mixtures because the ferroelastic domain switching is more activated due to the residual stress in the quasi-binary composites, which triggers more crack defections due to the enlarged process zone. Additionally, the Gd₃NbO₇ phase provides a low thermal conductivity due to the substantial chemical inhomogeneity, which diffuses phonons. Gd₃NbO₇/GdNbO₄ exhibits a balanced thermal conductivity of 1.6 W/(m·K) at 1073 K and a toughness value of 2.76 MPa·m^0.5, and these values are among the best comprehensive properties that have been obtained for new TBC materials. The work demonstrates a feasible approach of designing a new TBC material with balanced properties and can be easily fabricated.