• We present XMM-Newton observations of a complete sample of five archetypal young radio-loud AGN, also known Gigahertz Peaked Spectrum (GPS) sources. They are among the brightest and best studied GPS/CSO sources in the sky, with radio powers in the range L_{5GHz}=10^{43-44} erg/s and with 4 sources having measured kinematic ages of 570 to 3000 yrs. All sources are detected, and have 2-10 keV luminosities from 0.5 to 4.8x10^{44} erg/s. In comparison with the general population of radio galaxies, we find that: 1) GPS galaxies show a a range in absorption column densities similar to other radio galaxies. We therefore find no evidence that GPS galaxies reside in significantly more dense circumnuclear environment, such that they could be hampered in their expansion. 2) The ratio of radio to X-ray luminosity is significantly higher than for classical radio sources. This is consistent with an evolution scenario in which young radio sources are more efficient radio emitters than large extended objects at a constant accretion power. 3) Taking the X-ray luminosity of radio sources as a measure of ionisation power, we find that GPS galaxies are significantly underluminous in their [OIII]_{5007 Angstrom}, including a weak trend with age. This is consistent with the fact that the Stroemgren sphere should still be expanding in these young objects. This would mean that here we are witnessing the birth of the narrow line region of radio-loud AGN.