If that's all that's keeping you back, you should check if the city/state you wish to live in has what's usually called a homestead exemption. Many places will reduce property tax if you meet a few criteria. The criteria varies, but it usually revolves around being on a fixed income and living on the property.

Florida has property tax, but they don't have an estate tax, which probably influences people to retire there...