Urban energy simulation is critical for understanding and managing energy performance in cities. In this research, we design a novel framework called DeepRadiation, to enable automatic urban environmental performance prediction. By incorporating deep learning strategies, DeepRadiation predicts solar radiation on an urban scale using just panoramic streetscape images without any 3D modeling and simulation. New York City was chosen as the case study for this research. DeepRadiation is comprised of three different deep learning models organized into two stages. The first stage, named DeepRadiation modeling, serves as the framework's brain. At this stage, solar radiation analysis was performed using a Pix2Pix model, a type of conditional generative adversarial networks (GANs). After extracting GIS data and performing energy simulation analysis to prepare the dataset, the Pix2Pix model was trained on 10000 paired panoramic depth images of streetscapes with only building blocks and related panoramic images of streetscapes with only solar radiation analysis. Two GAN generator evaluation measures named qualitative evaluation and quantitative evaluation were used to validate the trained Pix2Pix model. Both demonstrated high levels of accuracy (qualitative evaluation: 93%, quantitative evaluation: 89%). DeepRadiation application as the DeepRadiation's sescond stage is the framework's eyes. At this stage, two convolutional neural network (CNN) models (DeepLabv3 and MiDaS) were used to perform computer vision tasks on panoramic streetscape images, such as semantic segmentation and depth estimation. The DeepRadiation application stage allows urban designers, architects, and urban policymakers to use the DeepRadiation framework and experience the final output via augmented reality.