• The iron chalcogenide Fe$_{1+y}$Te$_{1-x}$Se$_{x}$ on the Te-rich side is known to exhibit the strongest electron correlations among the Fe-based superconductors, and is non-superconducting for $x$ < 0.1. In order to understand the origin of such behaviors, we have performed ARPES studies of Fe$_{1+y}$Te$_{1-x}$Se$_{x}$ ($x$ = 0, 0.1, 0.2, and 0.4). The obtained mass renormalization factors for different energy bands are qualitatively consistent with DFT + DMFT calculations. Our results provide evidence for strong orbital dependence of mass renormalization, and systematic data which help us to resolve inconsistencies with other experimental data. The unusually strong orbital dependence of mass renormalization in Te-rich Fe$_{1+y}$Te$_{1-x}$Se$_{x}$ arises from the dominant contribution to the Fermi surface of the $d_{xy}$ band, which is the most strongly correlated and may contribute to the suppression of superconductivity.