Do you remove the strings all at once and replace all at once or do you change them out one at a time? One at a time, starting with the highest/lightest string, doesn't affect the neck relief as much as taking them all off and replacing them all at once.
What's your definition of lightest? Smallest gauge? Or lowest tension?
