As of today, hundreds of hot Jupiters have been found, yet the inflated radii of a large fraction of them remain unexplained. It is still unclear whether a single inflation mechanism is enough to explain the entire distribution of radii, or whether a combination of them is needed. We seek to understand the relationship of hot Jupiter radii with stellar irradiation and mass. We also aim to find the intrinsic physical scatter in their radii caused by unobservable parameters. By constructing a hierarchical Bayesian model, we infer the probabilistic relation between planet radius, mass and incident flux for a sample of 286 gas giants. We separately incorporate the observational uncertainties of the data and the intrinsic physical scatter in the population. This allows us to treat the intrinsic physical scatter in radii (due to latent parameters such as the heavy element fraction) as a parameter to be inferred. We find that the planetary mass plays a key role in the inflation extent, with planets in the range between 0.37 - 0.98 MJ showing the most inflated radii. At higher masses, the radius response to incident flux begins to decrease. Below a threshold of 0.37 +- 0.03 MJ we find that giant exoplanets as a population are unable to maintain inflated radii greater than 1.4 RJ but instead exhibit smaller sizes as the incident flux is increased beyond 10^6 Wm-2. We also find that below 1 MJ, there is a cutoff point at high incident flux beyond which we find no more inflated planets, and that this cutoff point decreases as the mass decreases. At incident fluxes higher than 1.6*10^6 Wm -2 and in a mass range 0.37 - 0.98 MJ, we find no evidence for a population of non-inflated hot Jupiters. Our study demonstrates the importance of population-level analysis to grasp the underlying properties of exoplanets.