Noo,you do not have to be wealthy,just a bit insane
I moved to my current location on the verge of bankruptcy,a small diminutive cottage but has just over the acre on the back
for the first year I did not do anything and it was all overgrown,6ft of nettles and thistles,basically a bit of waste land really.
It is now 6 yrs on and although it is really hard work,I love it,you would not regret it when you do find somewhere,the entire lifestyle is just so vastly different to that of the rat race so to speak,if I were you I would get
and make your dream come true,go for it!!