Generation of circular dichroism (CD) beyond the UV region is of great interest in developing chiral sensors and chiroptical devices. Herein, we demonstrate a simple and versatile method for fabrication of plasmonic oligomers with strong CD response in the visible and near IR spectral range. The oligomers were fabricated by triggering the side-by-side assembly of cysteine-modified gold nanorods. The modified nanorods themselves did not exhibit obvious plasmonic CD signals; however, the oligomers show strong CD bands around the plasmon resonance wavelength. The sign of the CD band was dictated by the chirality of the absorbed cysteine molecules. By adjusting the size of the oligomers, the concentration of chiral molecules, and/or the aspect ratio of the nanorods, the CD intensity and spectral range were readily tunable. Theoretical calculations suggested that CD of the oligomers originated from a slight twist of adjacent nanorods within the oligomer. Therefore, we propose that the adsorbed chiral molecules are able to manipulate the twist angles between the nanorods and thus modulate the CD response of the oligomers.