Device-free Passive (DfP) detection has received increasing attention for its ability to support various pervasive applications. Instead of relying on variable Received Signal Strength (RSS), most recent studies rely on finer-grained Channel State Information (CSI). However, existing methods have some limitations, in that they are effective only in the Line-Of-Sight (LOS) or for more than one moving individual. In this paper, we analyze the human motion effect on CSI and propose a novel scheme for Robust Passive Motion Detection (R-PMD). Since traditional low-pass filtering has a number of limitations with respect to data denoising, we adopt a novel Principal Component Analysis (PCA)-based filtering technique to capture the representative signals of human motion and extract the variance profile as the sensitive metric for human detection. In addition, existing schemes simply aggregate CSI values over all the antennas in MIMO systems. Instead, we investigate the sensing quality of each antenna and aggregate the best combination of antennas to achieve more accurate and robust detection. The R-PMD prototype uses off-the-shelf WiFi devices and the experimental results demonstrate that R-PMD achieves an average detection rate of 96.33% with a false alarm rate of 3.67%.