I see your point, but it might be a tad unfair to good sight-readers. A really good sight-reader can not only run down the notes but also imbue the performance with some musicality, which in the terms you describe must mean that he/she is understanding it, at least to a significant degree, while reading it. So it really isn't just a simple matter of reading without understanding.
As an analogy, I have seen actors who could take a monologue they've never seen before and deliver an actual dramatic performance with it. As with the written music, this would have to mean that they're understanding the material to a degree while they're sight-reading it, because if they weren't, they wouldn't be able to deliver anything that was at all dramatically cogent.
I think most of us would agree that being able to sit with a written piece or part and work on it is likely to yield better understanding and performance. (Actors would generally tell you the same thing.) The thing is, some playing situations just don't allow you the time to do that. Those situations demand that you read both quickly and convincingly.
The argument for sight-reading has always been more a practical one than a strictly musical one. I don't think I've ever heard anyone maintain that you play a piece better when sight-reading it than when you've learned it well. The main value of sight-reading is that it allows you to deliver a musical performance when time is highly limited. This is a very valuable skill for those who need it. Not everyone has an urgent need for it, however.
Keep in mind also that a good sight-reader can also do "regular" reading, by definition, but a "regular" reader won't necessarily be a good sight-reader. Working on your sight-reading, if you choose to do that, is not going to get in the way of any other work you want to do on the music.