Temperature-driven spectral weight transfer in doped magnetic insulators