Traditional image resizing methods usually work in pixel space and use various saliency measures. The challenge is to adjust the image shape while trying to preserve important content. In this paper weperform image resizing in feature space using the deep layers of a neural network containing rich important semantic information. We directly adjust the imagefeature maps, extracted from a pre-trained classification network, and reconstruct the resized image using neural-network based optimization. This novel approachleverages the hierarchical encoding of the network, and in particular, the high-level discriminative power of its deeper layers, that can recognize semantic regions and objects, thereby allowing maintenance of their aspect ratios. Our use of reconstruction from deep features results in less noticeable artifacts than use of image-space resizing operators. We evaluate our method on benchmarks, compare it to alternative approaches, and demonstrate its strengths on challenging images.