Actually, who owns the airspace does matter. If it's not on his property, it's not trespassing. As to "it's coming down"...what are you going to do? Fire a gun in city limits? Accurately throw a rock 100 feet and knock it out of the air? What major league team do you pitch for?
A sling shot you can get at a sporting good store would take it out and not violate discharge in city limits laws but it might other laws. Maybe the victim could show it as a warning. The drone owner might get the hint and leave the residence alone.