• Only a small fraction of observed Active Galactic Nuclei display large-scale radio emission associated with jets, yet these radio-loud AGN have become increasingly important in models of galaxy evolution. In determining the dynamics and energetics of the radio sources over cosmic time, a key question concerns what happens when their jets switch off. The resulting `remnant' radio-loud AGN have been surprisingly evasive in past radio surveys, and therefore statistical information on the population of radio-loud AGN in their dying phase is limited. In this paper, with the recent developments of LOFAR and the VLA, we are able to provide a systematically selected sample of remnant radio-loud AGN in the Herschel-ATLAS field. Using a simple core-detection method, we constrain the upper limit on the fraction of remnants in our radio-loud AGN sample to 9 per cent, implying that the extended lobe emission fades rapidly once the core/jets turn off. We also find that our remnant sample has a wide range of spectral indices ($-1.5\leqslant \alpha^{1400}_{150}\leqslant -0.5$), confirming that the lobes of some remnants may possess flat spectra at low frequencies just as active sources do. We suggest that, even with the unprecedented sensitivity of LOFAR, our sample may still only contain the youngest of the remnant population.