Low quality (Q) factor is often the limiting factor for high performance carbon nanotube (CNT) resonators. The most commonly used approach to enhance the Q factor of CNTs is to reduce/eliminate the intrinsic defects. Herein, we show surprisingly that hole defects of suitable size and position are able to enhance the Q factor of CNT, which strongly contradicts to the common notion that the presence of defects promote intrinsic dissipation via defects dissipation. By analyzing the strain distribution, we find that such abnormal enhancement in Q factor of defected CNT originates from a coupling competition mechanism between the atomic mismatch around defected atoms and the thermoelastic damping. Although the presence of holes will introduce an additional defect dissipation source, suitable holes are capable of reducing the energy dissipation arisen from the thermoelastic damping, through changing the spatial strain field of defected CNT. This coupling competition mechanism provides a new route for designing high performance CNT resonators via defects engineering.