The distant (z~3) radio-loud AGN (=high-z radio galaxies, HzRGs) are hosted by some of the most massive galaxies living in the centre of (proto-)clusters. They are the perfect laboratory for studying simultaneously both the radiatively driven and jet-driven feedback at around Cosmic Noon. To disentangle the interactions between different components in the interstellar to circumgalactic medium (ISM to CGM), I exploit the state-of-the-art integral field spectrographs (IFS) through my PhD: NIRSpec/IFU, MUSE, and ALMA. Using MUSE, I mapped 100s kpc Ly-alpha nebulae around HzRGs and linked them to other quasar species. Through analysis of absorbing gas, I found evidence of CGM enrichment at z~4.5 by AGN feedback. The NIRSpec/IFU on board JWST unveils the ISM in sub-kpc resolution. The warm ionised gas shows multiple components and complex morphologies which could be related to outflow, shocks, and companions. For one of the HzRGs, I discovered its inefficiency in radiatively driven feedback even close to the central AGN. I will also show the ALMA observed [CII] line which indicates rotation structures, but its broad line width may also trace multi-phase outflows. Last but not least, spatially resolved cold dust in ALMA Band8 pinpoint the young stars in the hosts. The richer information brought by the combination of these IFS will revolutionise our understanding of the evolution of distant galaxies.