Check again for anything loose. Then check for voice coil rubbing in the magnetic gap. Check that the driver cone seems to be centred in it's travel. Try playing it on it's side, well all four sides to see if the buzz goes away.
At worst the voice coil may have overheated and broken the glued joint to the VC former. This will probably make the driver scrap.
