A raging wildfire drove all 11,000 residents from the town of Los Alamos on Wednesday and briefly set fire to a building inside the biggest US nuclear weapons laboratory, where high explosives and plutonium are bunkered in disaster-proof shelters.