It seems a philosophical error to think that an argument by implied consent which justifies hypothecated taxation to pay for infrastructure also justifies taxation for the other two categories of government spending.
Put another way, people can think that all taxation is theft, or some taxation is theft, or no taxation is theft. We agree that there is a good argument against the "all" version of the proposition. But the "some" version is very much still out there.