Wavering public support puts cracks in political subway deal: James