The former Tory leader says it will mean "a terrible state of lost jobs", arguing the data behind the decision should have been scrutinised more carefully