Correct, the change in weight (moving mass) is insignificant. The issue is the binders used in the cone pulp, there are many different binders and additives used. It's probably ok, but maybe not always ok.
It's the effect on the stiffness that might change, due to deterioration of the binders and additives of the cone pulp.
It's probably going to be ok, but speaker cones aren't made entirely of paper. There are all kinds of additives that are used along with the binders in order to control stiffness. Some of these may be broken down by solvents (including water) in paints, dyes and markers.