Are you saying it would be better to go from my Zoom into a direct box?
Most passive DIs drop the level at the XLR output by 10 to 20dB to be close to typical microphone levels. Active DIs may or may not do this. I have seen some active DIs that actually boost the signal level.
The primary task of a direct box is to lower the impedance. The level only matters with mixers that cannot tolerate high levels at the XLR inputs. That would typically be mixers with no gain trim on the channels.