Just to chip in, in both England and Scotland, even to build a log cabin on your own land needs planing permission. If you can find cheap land it will almost certainly be agricultural or a bit of woodland; chances of getting planing permission, zero. As long as you move it from time to time (so it is not regarded as permanent) you
might get away with a caravan. There could be an alternative though; Scoraig
http://www.scoraig.com/ Although there's a track of sorts, there is no vehicle access. Only way in or out is by boat. If you contact them, you could say the man who sold them the steel sink suggested it.
This is the only permanent community of this type that I know of, but there may be others out there with a lower profile. Try looking at old Ordinance Survey maps showing coastal communities with no road access. Then check with a new map and see which ones are no longer on it.
Remember though, remote abandoned communities were abandoned for a reason.