This paper reports on the impact of γ-ray irradiation (10, 100 kGy) on melt-quenched Ag+-doped phosphate glass and the effects of subsequent thermal processing leading to the production of plasmonic Ag nanocomposites. The γ-irradiated glasses were characterized alongside the pristine by differential scanning calorimetry (DSC), Raman spectroscopy, electron paramagnetic resonance (EPR) spectroscopy, optical absorption, and photoluminescence (PL) spectroscopy. DSC characterization showed consistent glass transition temperatures (Tg) before and after γ-irradiation whereas the crystallization temperatures tended to decrease with increasing γ-ray dose. However, a lack of alteration of the glass network structure was supported by Raman spectroscopy. Room temperature EPR spectra clearly showed the formation of phosphorus oxygen hole center (POHC) defects in the undoped host, in addition to another doublet likely associated with a P3 defect. The presence of paramagnetic silver species encompassing Ag2+ and 107/109Ag0 atoms was also indicated in the silver-activated glass together with POHC defects. Optical absorption spectra were also consistent with the presence of various radiation-induced centers. Further analyzing the glass absorption edge via Tauc plots suggested the formation of electron center (EC) defects in γ-irradiated samples wherein the silver-doped glass exhibited decreasing band gap energies with increasing γ-ray dose. The PL characterization showed the silver-related radio-PL effect was induced exhibiting broad band emission with two maxima around 500 and 625 nm stemming from various molecular clusters. Emission decay analyses revealed that the longer wavelength emission exhibited slower decay. The highest radiation dose of 100 kGy however resulted in weaker emission and faster decay kinetics attributed to energy transfer between the luminescent silver species and POHC defects. Finally subjecting the γ-irradiated Ag-doped glasses to heat treatment near the Tg at 490 °C led to the development of the surface plasmon resonance of Ag nanoparticles (NPs) and the vanishing of the clusters luminescence. The presence of the matrix-related EC defects was deemed accountable for the thermally induced reduction and consequent precipitation of Ag NPs making the plasmonic glasses attractive for photonic applications such as nonlinear optics.