A deep-learning-based feature extraction has recently been proposed for HyperSpectral Images (HSI) classification. A Deep Belief Network (DBN), as part of deep learning, has been used in HSI classification for deep and abstract feature extraction. However, DBN has to simultaneously deal with hundreds of features from the HSI hyper-cube, which results into complexity and leads to limited feature abstraction and performance in the presence of limited training data. Moreover, a dimensional-reduction-based solution to this issue results in the loss of valuable spectral information, thereby affecting classification performance. To address the issue, this paper presents a Spectral-Adaptive Segmented DBN (SAS-DBN) for spectral-spatial HSI classification that exploits the deep abstract features by segmenting the original spectral bands into small sets/groups of related spectral bands and processing each group separately by using local DBNs. Furthermore, spatial features are also incorporated by first applying hyper-segmentation on the HSI. These results improved data abstraction with reduced complexity and enhanced the performance of HSI classification. Local application of DBN-based feature extraction to each group of bands reduces the computational complexity and results in better feature extraction improving classification accuracy. In general, exploiting spectral features effectively through a segmented-DBN process and spatial features through hyper-segmentation and integration of spectral and spatial features for HSI classification has a major effect on the performance of HSI classification. Experimental evaluation of the proposed technique on well-known HSI standard data sets with different contexts and resolutions establishes the efficacy of the proposed techniques, wherein the results are comparable to several recently proposed HSI classification techniques.