• We use femtosecond x-ray diffraction to study the structural response of charge and orbitally ordered Pr$_{1-x}$Ca$_x$MnO$_3$ thin films across a phase transition induced by 800 nm laser pulses. By investigating the dynamics of both superlattice reflections and regular Bragg peaks, we disentangle the different structural contributions and analyze their relevant time-scales. The dynamics of the structural and charge order response are qualitatively different when excited above and below a critical fluence $f_c$. For excitations below $f_c$ the charge order and the superlattice is only partially suppressed and the ground state recovers within a few tens of nanosecond via diffusive cooling. When exciting above the critical fluence the superlattice vanishes within approximately half a picosecond followed by a change of the unit cell parameters on a 10 picoseconds time-scale. At this point all memory from the symmetry breaking is lost and the recovery time increases by many order of magnitudes due to the first order character of the structural phase transition.