It appears that once used the information is no longer retained by giffgaff's systems, at least it was the case 12 months ago
I had adult content enabled, turned it off, and 48 hrs later tried to re-enable it, I had to enter my driver number once again.
As regards the information they can access with your driver number, not a lot, without another step/authorisation code from the dvla + driver, even insurance companies need that extra step to access your driving history/convictions/ vehicle classes
In the case of the driver number, everything giffgaff require is in the number, your DOB is part of your driver number
xxxxx/ ymmddy /xxxxx / xx , they have no need for anything further,
(anecdotally one post a month or two back stated they'd removed the bar with someone else's DL number)
I cannot confirm if the same is true of passport details, but I'd imagine the same information is also in the passport in a similar format