The Jagiellonian Positron Emission Tomograph (J-PET) was constructed as a prototype of the cost-effective scanner for the simultaneous metabolic imaging of the whole human body. Being optimized for the detection of photons from the electron-positron annihilation with high time- and high angular-resolution, it constitutes a multi-purpose detector providing new opportunities for studying the decays of positronium atoms. Positronium is the lightest purely leptonic object decaying into photons. As an atom bound by a central potential it is a parity eigenstate, and as an atom built out of an electron and an anti-electron it is an eigenstate of the charge conjugation operator. Therefore, the positronium is a unique laboratory to study discrete symmetries whose precision is limited in principle by the effects due to the weak interactions expected at the level of (~10$^{-14}$) and photon-photon interactions expected at the level of (~10$^{-9}$). The J-PET detector enables to perform tests of discrete symmetries in the leptonic sector via the determination of the expectation values of the discrete-symmetries-odd operators, which may be constructed from the spin of ortho-positronium atom and the momenta and polarization vectors of photons originating from its annihilation. In this article we present the potential of the J-PET detector to test the C, CP, T and CPT symmetries in the decays of positronium atoms.