I remember those Pangborn basses - very cool. I was thinking about those as I read through this thread. That's a great looking bass.It can be done. This is a Pangborn fretless bass from the early 80's that has an aluminium fingerboard. The fingerboard was recently reattached and has been in place for about a year without problems. I don't recall the type of adhesive that was used though. Pangborn is long gone, but Vigier still makes both guitars and basses using fingerboards made of "delta metal", which is some kind of alloy. Why not shoot Patrice Vigier an e-mail and ask him what he would recommend. I don't think aluminimum is very practical for the reasons others have outlined in depth.
Really rare and unusual these days. You should definitely review it in the basses forum if you haven't done so already.