Get Help
Community

Today's Service Outage

handy giff-staffer

As you very likely to be already aware, today we've suffered a major service outage - it was caused by major water damage to the computer network in a our data centre and it took nearly 8 hours fix.

 

Firstly I'd like to offer my apologies for the interruption in service and thank all of our members for their patience whilst we fixed the issue.

 

The service is largely back up and running now - the exception is new activations which are currently being queued and our service teams are working on a fix.

 

Some of you have asked about compensation - we're going to focus on fixing the issues we've had and make sure we have a stable service first - and then we'll sit down think of ways to make it up to you.

 

We'll continue to post updates on the support community so click here for the latest news.

 

Thanks once again for your patience.

 

Mike Fairman

 

CEO giffgaff

 

 

Update 21:15

 

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

 

In response to a few of the issues raised...

 

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can. 

 

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

 

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.



853 Comments
newcomer

Coming from a major media provider myself I know the importance of backup systems, redundant backup systems and eliminating single point of failures - right down to having equipment spread across different geographic sites. That is the price the company I work for has to pay in 'insurance' to prevent such a major outage... This area of investment with regards to GiffGaff appears to be lacking... We could all say well we are getting a cheap service - which is true... But when it comes to my mobile phone it is very important that I have a service that I can rely upon. In 3 months I have had 2 months where I have not been able to buy a goody bag (thanks to GiffGaff website malfunction) causing me personal outage aside from todays issues... That's 3 months of pretty poor service... I signed up to GiffGaff as it seemed a good value service at the time - but now I question my wisdom and when the cracks start appearing the savings we all rave about have been paid for by a lack of backup systems and a proper customer support service.... I will carry on with GiffGaff for a while but I am prepared to pay more for more reliability which I suspect will mean picking another provider unless GiffGaff can assure me that proper backup systems are planned to be put in place. I would much prefer GiffGaff invest in preventive measures and concentrate upon providing a reliable service (including website function as that is the only way to topup or buy goody bags) rather than compensating everyone...

 

trainee

Strange enough this issue didn't really effect me apart from not being able to send texts, though calls can data was working :)

Thanks anyway mike for posting this, would never expect this from other nethworks, but as many people say, its always good to have a backup :)

newcomer
Well done to the GG team for the way they have handled what must have been one of their worst days. We all appreciate the hard work and keeping us updated despite technical limitations. There may be questions around business continuity plans but that is for another day. I believe we all feel part of the team as giffgaff members and as such we are proud of the service you provide and the way you handled yourselves today ...good effort.
newcomer
Hey ho, been a nightmare day without mobile, data etc.., but these things happen, no bull s***t excuses, CEO speaking up, sure the GiffGaff team will do everything possible to make sure it does not happen again and also compensate everyone for what has been a very frustrating Friday. But for me this has been the one & only problem I've had with GiffGaff and the water damage is probably out of the hands of any network, big or small.
newcomer

Hey dear giffgaff-team,

 

I like giffgaff and i recommended it to a lot of friends. However, my number is also very important for my professional environment... and I missed a lot of very important calls and it seriously damaged business. This can not happen!

trainee

It's not compensation culture, it's good customer service. When I go to a restaurant and have a crap meal I didn't order delivered late, if I am told "sorry about that, dinner's on us and why don't you have a cold beer", I'll go back.

 

(This happened to some colleagues at a curry restaurant recently - actually the food was quite good, it was just too late for most of them to finish it. They were told "sorry the food was so late, it's on the house, please consider coming back." They will, and so will I.  

 

I usually travel on an unreliable train service provided by #Scotfail. For several days this week my wife was in Europe on business and I had to be back before 6p.m. to pick up the kids and avoid keeping the after school club staff late. Given #scotfail have cancelled the train after it was due about five times in the last ten weeks, I was nervous. But I'd also recently persuaded my wife to move to giffgaff so she and my daughter could talk free. 

 

Anyway, there she is over in Italy travelling via London and giffgaff go down completely. Luckily she was able to call from Heathrow and it looks like she was unaffected. But the timing of this giffgaff outage couldn't have been worse for us. If something had gone wrong with her trip from Italy, or my train had failed, would have been up sh!t creek without a paddle...

 

So for the loss of the best part of a day's goodybag, as well a certain amount of stress worrying about what might happen if me or my wife really needed our phones to work, yes, I reckon we need to see some compensation. Apart from today's episode - and a bad episode porting my number - I quite like giffgaff so far, but there might be a Ting in the UK soon, so giffgaff need to do something that will make p*ssed off or unsettled customers feel good about them again. 

 

I also want to know a lot more about giffgaff's Disaster Recovery infrastructure and plans. The goodybags and service terms may be good value. Good value but occasionally down for most of the day is just not acceptable. Short of major multi-site catastrophe, giffgaff are now a significant national service provider and need infrastructure that can just take over seamlessly in the event of anything up to and including total loss of  a facility.  I don't know how many customers giffgaff have but there were close on 300,000 messages on the community item discussing this issue this afternoon - and that happened even though many people could not log on. So it cannot be giffgaff do not have the resources to implement a Disaster Recovery contingency. 

 

For myself, I'll give gifffgaff another chance for now but if a similar event were to occur again in the next six months or so, I'd seriously consider whether I can afford it. The core reason for me to have a mobile in the first place has always been to be able to get in touch in the event of urgent problems. Being able to browse the web, use Twitter or chat to a friend on my phone is pure cake icing. An eight hour outage because a digger went through a single water pipe - or whatever happened - must not be allowed to happen again. 

 

cheers,

 

John  

 

newcomer
As bad as today was for the whole community, long term I don't think it will harm the credibility of what has been built here! I for one will still convert as many as I can. As they say "today's news is tomorrows chip paper" or some technical alternative to this! :-)
newcomer

I have recently joined giffgaff and have been very happy, but I must admit such a long outage is seriously worrying. Luckily I was in the office and could use a landline, but what about people who were relying on their mobiles/smartphones to do their job today? As already mentioned by several others, it is seriously worrying that a backup system/disaster recovery plan has not kicked in. I understand that competing on price means cutting unnecessary cost, however having a reliable service is a must for any operator and I hope that the reason of the outage was not savings or insufficient backup systems.

newcomer

Nice one Mike - very honest and sensible approach from all of you today.

 

To all the people moaning - you are paying for the cheapest mobile service in the country - feel free to go elsewhere. Doubt you will though !!

MAN UP PEOPLE!!, Sure its an inconvience but hey you're alive and hopefully well.