Mesh-based image warping techniques typically represent image deformation using linear functions on triangular meshes or bilinear functions on rectangular meshes. This enables simple and efficient implementation, but in turn, restricts the representation capability of the deformation, often leading to unsatisfactory warping results. We present a novel, flexible polygonal finite element (poly-FEM) method for content-aware image warping. Image deformation is represented by high-order poly-FEMs on a content-aware polygonal mesh with a cell distribution adapted to saliency information in the source image. This allows highly adaptive meshes and smoother warping with fewer degrees of freedom, thus significantly extending the flexibility and capability of the warping representation. Benefiting from the continuous formulation of image deformation, our poly-FEM warping method is able to compute the optimal image deformation by minimizing existing or even newly designed warping energies consisting of penalty terms for specific transformations. We demonstrate the versatility of the proposed poly-FEM warping method in representing different deformations and its superiority by comparing it to other existing state-of-the-art methods.