Here's a tip...
To stop giffgaff telling your phone that there is a voice mail message using messages and the home screen of your phone you can ask giffgaff to send you a text letting you know of a voice mail message.
This means you don't get an annoying voice mail symbol on your home screen.
This can be done by phoning 443 at standard rates and it is in the settings.
I actually MUCH prefer the proper voicemail icon which giffgaff does by default, as with the text message system (which I used to have on o2) I had a habit of deleting the text message with every intention of checking my voicemail later, but then forgot!
At least with the voicemail symbol, it sits there reminding you until you actually listen to the message and delete it (then clears itself). I like.
I agree with the suggestion of using hullomail
I have set it up with one of the more advanced features, which emails me my voicemails as soon as they are received. This means I can access and listen to my voice mail on my PC, as well as my phone. On the phone you can use both the Hullomail visual voicemail app (easily found in the Market) or even access it in the Android Gmail client (or the HTC Email client I imagine).
I've tried looking for a solution to your issue changing the voicemail number, but I can't recreate it and can't find any other mention of it (even on the usually useful XDA forum). Where did you buy it from?
If you are running Android 1.6 or 2.2 then you can change the number to 443 as per http://community.giffgaff.com/t5/Learn-giffgaff-To