Hermits are a vital unit for adding tankiness to spider formations. Without them spiders are made of paper. Swarms of hermits can also make for pretty great cliff assaulters as already mentioned.
They feel like they suck because their "handling characteristics", so to speak, are bad. The're slow, clumsy, turn their turret slowly, shot inaccurate, individually deal little damage, individually are easy to kill, etc.
But the're very cost efficient and that makes up for all of that.