Unfortunately, I believe the FAA initially ruled that homeowners don't own the airspace over their properties, and this is where there have been issues.
To anyone who says anything along the lines of "if you're doing x,y, or z in your yard or house then you should probably rethink it", what gives you the right to basically tell someone how to conduct themselves on/in their property? I also don't do anything out of the ordinary, but I shouldn't have to possibly change my daily routine because of someone elses hobby.
If I decide to operate my RC car on the road and it gets crushed by an automobile, that's on me. In my opinion, if I decide to fly a drone over my neighbors house and they decide to drop it because it's interfering with what they're doing there, to me that's the equivalent of driving the RC car in the road.
Also, one of my hobbies is RC, and I like the idea of responsible drone usage, not the current wild-west atmosphere.