Gregg Wallace hits out at 'handful' of accusers