The Milky Way (MW) dwarf spheroidal (dSph) satellites are known to be the most dark-matter (DM) dominated galaxies with estimates of dark to baryonic matter reaching even above one hundred. It comes from the assumption that dwarfs are dynamically supported by their observed velocity dispersions. However their spatial distributions around the MW is not at random and this could challenge their origin, previously assumed to be residues of primordial galaxies accreted by the MW potential. Here we show that alternatively, dSphs could be the residue of tidal dwarf galaxies (TDGs), which would have interacted with the Galactic hot gaseous halo and disk. TDGs are gas-rich and have been formed in a tidal tail produced during an ancient merger event at the M31 location, and expelled towards the MW. Our simulations show that low-mass TDGs are fragile to an interaction with the MW disk and halo hot gas. During the interaction, their stellar content is progressively driven out of equilibrium and strongly expands, leading to low surface brightness feature and mimicking high dynamical M/L ratios. Our modeling can reproduce the properties, including the kinematics, of classical MW dwarfs within the mass range of the Magellanic Clouds to Draco. An ancient gas-rich merger at the M31 location could then challenge the currently assumed high content of dark matter in dwarf galaxies. We propose a simple observational test with the coming GAIA mission, to follow their expected stellar expansion, which should not be observed within the current theoretical framework.