Glassiness is ubiquitous and diverse in characteristics in nature. Understanding their differences and classification remains a major scientific challenge. Here, we show that scaling of magnetic memories with time can be used to classify magnetic glassy materials into two distinct classes. The systems studied are high-temperature superconductor-related materials, spin-orbit Mott insulators, frustrated magnets, and dilute magnetic alloys. Our bulk magnetization measurements reveal that most densely populated magnets exhibit similar memory behavior characterized by a relaxation exponent of 1-n ~ 0.6(1). This exponent is different from 1-n ~ 1/3 of dilute magnetic alloys that was ascribed to their hierarchical and fractal energy landscape and is also different from 1-n=1 of the conventional Debye relaxation expected for a spin solid, a state with long range order. Furthermore, our systematic study on dilute magnetic alloys with varying magnetic concentration exhibits crossovers among the two glassy states and spin solid.