Collective instability and physics of the anomalous Doppler resonance in fusion plasmas