Sorry, I mean I can tell the difference when I HEAR a minor vs major chord. What I cannot do is if you say "Key of Dmaj", I cannot tell you from memory which chords are major, minor, diminished, etc.
Google "harmonized major scale" or Diatonic harmony. It's not rocket science and there's only one sequence to memorize. It's same same for every Major key/scale.
MAJOR, minor, minor, MAJOR, MAJOR, minor, diminshed. That's it. That's diatonic harmony and in this case diatonic means within the diatonic (also called the Ionian mode) scale sequence of whole and half steps. Whole, Whole, half, Whole, Whole, Whole, half. This is all easier to see sync up on a piano keyboard.
Using roman numerals where uppercase means major and lowercase minor, I, ii, iii, IV, V, vi, viiº
In the key of D as per your example, Dmaj, Emin, Fmin, Gmaj, Amaj, Bmin, Cdim.
In G with --------------------------->, Gmaj, Amin, Bmin, Cmaj, Dmaj, Emin, Fdim.
See how they line up? Once you can remember the the fifth of D major is an A, because it's a chord built on the 5th, you instantly default to knowing that in the KEY of D, an A CHORD, is major.
Now when you get to song harmony, this all can go out the window. For example, in minor keys it's common to substitute a major chord for the V or IV chord. Chords can be "borrowed" from other keys etc. This is a case or memorizing the SONG's chord sequence but as to how you posited your issue. If you know you're playing in key of G and they say, "Go down to D." It's almost certainly a major chord. If they say go up to B, it's almost certainly minor.
HTH,
Mugre