Well I've just had another look at it and it seems the nut on the output jack is a bit loose which is probably what's causing that problem. Even when I tighten it, it seems to become loose after a while, I'll have a go at tightening it with a wrench or something and see if that does the trick. This is one of the boss pedals that had a metal/silver screw instead of the current black plastic ones, and there's no black rubber in it.
I've checked everything under the hood and it seems fine. The 'check' led comes on everytime (it wasn't a status LED back then). But sometimes the effects just doesn't come on, I'm hoping it's not something to do with the switch itself because it's one of the clear ones that seem more sought after than the black

(must be some ebay boss fad).