| Really, I have no idea how the Tascam works. I've used a Portastudio to cut scratch tracks once in my entire life. After a little bit of hunting, it seems that all eight input channels have preamps. Assuming the Trim knobs are set for zero (ie: Line), the XLR inputs are expecting a -10dBu signal, and the TRS inputs are expecting a +4dBu signal. I don't know whether these are switching pots that will take the preamp out of the chain if they're set for Line operation, but I can't find anything else in the manual about line operation.
Furthermore, I skimmed through the ART Pro Channel manual and can't find a nominal output spec on either the 1/4" or XLR outputs. It seems that the XLR can output hotter (+27dBu) than the 1/4" (+22dBu), but that doesn't really tell you what either outputs under optimal condition. It's a guessing game at this point.
So, I would suggest trying both XLR and 1/4" cabling, both with the Trim set for Line, and see which gives you a lower noise floor and optimal gain structure. If the gain structure is where it should be, you'll get the OL light flashing on that input on the loudest notes. If it's going off regularly, though, it's too hot. | 
