Get Help
Community

Today's Service Outage

handy giff-staffer

As you very likely to be already aware, today we've suffered a major service outage - it was caused by major water damage to the computer network in a our data centre and it took nearly 8 hours fix.

 

Firstly I'd like to offer my apologies for the interruption in service and thank all of our members for their patience whilst we fixed the issue.

 

The service is largely back up and running now - the exception is new activations which are currently being queued and our service teams are working on a fix.

 

Some of you have asked about compensation - we're going to focus on fixing the issues we've had and make sure we have a stable service first - and then we'll sit down think of ways to make it up to you.

 

We'll continue to post updates on the support community so click here for the latest news.

 

Thanks once again for your patience.

 

Mike Fairman

 

CEO giffgaff

 

 

Update 21:15

 

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

 

In response to a few of the issues raised...

 

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can. 

 

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

 

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.



853 Comments
newcomer

I must admit I find many of these comments amusing - Having your network go down can be very inconvenient but it happens - I was without Internet for an entire weekend recently because BT had a fault on their exchange so considering GiffGaff recovered in 8 hours is a credit to them. As for backups - people seem to be confusing backups with contingency. I'm almost certain backups exist as I imagine the 8 hours were spent restoring the system from those backups. What they clearly lack is contingency whereby secondary servers kick in when the primary ones go down. Unfortunately contingency costs money and when you consider what you are getting for your £10 goody bag then its no surprise they can't afford contingency - its not ideal but in this world you get what you pay for. If you want contingency pay through your nose with the big networks - if you want to save money then accept that there are going to be bad days from time to time - but at the end of the day I'd rather suffer a day of inconvenience than pay out an extra £120 per year to have the contingency reassurance.

 

 

beginner
Mike I think how you handle the aftermath of this outage will have an important impact on the future growth trajectory of GG. I must admit that earlier today I had resolved to leave GG. A mobile phone is too important to modern life to fail for such a long period of time. So really, I would like an explanation as to how the disaster recovery systems did not kick in. This will help me determine whether this is likely to happen again.
handy giff-staffer

Update 21:15

 

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

 

In response to a few of the issues raised...

 

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can. 

 

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

 

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.

 

Regards

 

Mike Fairman

 

CEO giffgaff

novice
Giffgaff's reputation is in the gutter after today. Itll have to do something pretty big to recover after today. I recently got a talk mobile sim as a back up. I'm glad I did.
newcomer
Think we should all have a sponsored mobile switch off day for a charity!!
newcomer

Was quite nice actually not being able to recieve texts or calls today...........but seriously it was good to get service updates throughout the day.I have had dealings with much bigger and more established companies that dont keep there customers as informed when a problem occurs. :)........makes a nice change thumbs up giffgaff.

beginner
Never mind about compensation Mike, these things happen...
newcomer
To Mike the CEO of giffgaff, well done for stepping forward and apologising to everyone. I personally have been very impressed by the fact that you have managed to restore the service as it is and kept us all updated. You and the team deserve a medal for what must have been and probably is gonna be a big headache of a day. As for those of you who demand compensation for todays outtage, you should all be disappointed with yourselfs. you made a bad situation worse for the giffgaff team. If i were them i would have made sure that your services were still down. Its people like you that giffgaff doesnt need and id happily wave goodbye to you. Once again huge well done to the giffgaff team, you really deserve it.
beginner

Thanks for the update. glad it is sorted!:smileyhappy:

oracle

Hi Mike

Appreciate the update .. Thank You :smileyhappy:

 

Hopefully things will be back to normal very soon. :smileyhappy: