Today's Service Outage

[gaffer]

As you very likely to be already aware, today we've suffered a major service outage - it was caused by major water damage to the computer network in a our data centre and it took nearly 8 hours fix.

Firstly I'd like to offer my apologies for the interruption in service and thank all of our members for their patience whilst we fixed the issue.

The service is largely back up and running now - the exception is new activations which are currently being queued and our service teams are working on a fix.

Some of you have asked about compensation - we're going to focus on fixing the issues we've had and make sure we have a stable service first - and then we'll sit down think of ways to make it up to you.

We'll continue to post updates on the support community so click here for the latest news.

Thanks once again for your patience.

Mike Fairman

CEO giffgaff

Update 21:15

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

In response to a few of the issues raised...

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can. 

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.

[helenl]

It is interesting the number of people who are cross with GG for not having back up systems in place to stop this kind of thing happening but haven't got back up plans themselves. Water mains burst, these things happen. GG kept everyone updated as to what the position is. Thank you GG. I don't think compensation is necessary.

[suranja]

Crappy service, no direct answer from agents. phone is out since Friday 12 noon. 

[charliemars]

I still cannot access into 3G network in my iPhone....should I do anything with my phone after your resolve ?

[pnp]

hi im a newby i registered yesterday @ 9.12 am and still no service can anybody advise what to do,,

 i have a signal but thats all

  thanks pnp

[pinklady1988]

Finally got my credit back last night, just a shame agents have problems reading which needs sorting out.

[whiteknight69]

thanks giffgaff for letting your customers know what problems you face in suppling us the customer with a great and pleasant service. yes i couldn't send texts when the server was down. But trust me, i didn't think my life would come to an end. As for as compensation. well your giving me that everyday,with a fantastic service already and great packages on offer. so all you people that are seeking compensation i cant write here what i want to say. so ill put this, in your job nothing ever goes wrong, you never make a wrong decision, yeah right. so come on give these guys and girls a round of applause for a great job so far thanks giffgaff.

[nancym]

Once again GiffGaff offers a very different customer experience. This level of communication when things go wrong is exactly why I'm a fan of this network. As others have pointed out, there are many service providers that we use for different utilities and I have only ever had the experience of being utterly in the dark when something crashes like this. The normal experience is that you lose service; you don't know of its network wide or something to do with your phone or account; you call the helpline number; if you're lucky there's a 5 second recorded message saying "some customers have experienced a loss of service and we are experiencing a very high volume of calls as a result, if your call is not urgent, please phone back later.". Call back 2 hours later and you get exactly the same. With GiffGaff I have an RSS feed from the service updates board to my work Outlook so I knew before I tried to send a text that there was a problem and I didn't even have a moment of wondering what the problem was, leaving me free to get on with my day rather than waste time phoning someone in India who doesn't know that a burst water pipe has fried the servers! I'm absolutely with those who say that the way to make it up to us is to look at what it would take to build additional redundancy into the system and/or make it possible to make the switch to back-up much swifter. Do you, for example, do practice drills on this kind of event? Don't waste money on compensating, just let us know what you are doing so that we can have confidence that this is very unlikely to happen again. Whilst i do have things in perspective and i didnt die in a ditch because your kit got wet, it is annoying not to have access to my phone and I'd prefer not to lose service for a day again!

[shirleyrose9]

Cheers mike super just great wow what a net work sends me to a whole new level the wheel man on the moon aw sorry mr armstrong we lost contact on giff gaff

[andycowl]

Sorry, no excuses a company like Giffgaff should have a business continuity plan, with the current virtualisation products available replication between sites should enable services to be brought back on line in a matter of an hour. If colleges and universities can do it why can't a business like giffgaff ?

[celinajohnson2]

my sevice is still off when will it be back on