Because nitrogen and oxygen are nearly transparent to infrared, and abundance is irrelevant if a gas does not interact with the radiation in question.
A molecule absorbs infrared only if the vibration changes its dipole moment. N₂ and O₂ are two identical atoms, symmetric, no dipole change, so they are effectively invisible to outgoing longwave radiation regardless of how much there is.
CO₂ bends and stretches asymmetrically, so it absorbs strongly around 15 micrometres, which is close to the peak of Earth's emission.
So the 99 percent is not doing nothing because it is weak. It is doing nothing because it cannot couple to that energy at all. Concentration only matters among the gases that can.