Because the reason why intervals are referred to as "perfect" has nothing to do with whether they are or are not invariable between major and minor scales. They are called that because they were traditionally considered to be the most consonant-sounding of the intervals, with seconds, thirds, sixths, and sevenths being considered less consonant.
You could also look at it another way, though this is hardly an "official" reason: A second is, in a sense, just the inverse of a seventh (just as a fourth is the inverse of a fifth). If a seventh is major/minor, why shouldn't a second be?
Yeah... What he said, lol...