Today's Service Outage

gaffer · ‎16-03-2012

As you very likely to be already aware, today we've suffered a major service outage - it was caused by major water damage to the computer network in a our data centre and it took nearly 8 hours fix.

Firstly I'd like to offer my apologies for the interruption in service and thank all of our members for their patience whilst we fixed the issue.

The service is largely back up and running now - the exception is new activations which are currently being queued and our service teams are working on a fix.

Some of you have asked about compensation - we're going to focus on fixing the issues we've had and make sure we have a stable service first - and then we'll sit down think of ways to make it up to you.

We'll continue to post updates on the support community so click here for the latest news.

Thanks once again for your patience.

Mike Fairman

CEO giffgaff

Update 21:15

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

In response to a few of the issues raised...

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can.

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.

redux · ‎16-03-2012

Thanks for keeping most people updated, Mike

I didn't receive the email

Well, not that one anyway. Only some over the top rants by one of your colleagues whining that she couldn't phone me, like it's my fault I've had no giffgaff service for a while.

I'm still not sure why she insisted on phoning, as the firm's complaints procedure doesn't mention phone calls at all, and if it does go to another party to sort out they want written evidence

robwiz · ‎16-03-2012

This outage couldn't have happened at a worse time for me. I drove 100 miles to meet a family member and bring her home. We were both calling and texting each other to arrange where to meet and getting no response, simultaneously fearing the worst. Not a good experience.

gboi · ‎16-03-2012

Like many on here, i appreciated the info being put on the website to let people know whats happening. why is everyone on about moving to other networks? are you all mad. this could have happened to anyone! compensation for not being able to use for phone for 8 hours. some people need to get a life if you want to go to another network and get a crap signal and be charged a fortune then off you go i say!!!

what did people do before mobile phones?? think we all need to get a grip!

Technology has issues at times even when back up sysytems are in place. GiffGaff have handled this issue extremley well. they put up info about the issue and kept people updated as much as possible.

HORAHH TO GIFFGAFF

nuroptics · ‎17-03-2012

Unimpressed.

Gifgaf - a network you can't speak to. No obvious phone number to ring (say from a colleges phone) to get the basic prerecorded message stating a problem. So I need to get internet access to get answers (quite useless when your onsite all day trying to have meetings).

Your data center? Is this a slight exaggeration?

50p compensation to all? Too expensive to Gifgaf, an insult to your customers (finishing work at 23.30pm as opposed to 5.00pm is worth a hell of a lot more than 50p)

Way forward? Full unbiased details of the outage. Accept fault where applicable to try and regain your customers confidence in this network. Install a system to alert your customers of a fault.

/rant

ramon_0404 · ‎17-03-2012

Sorry but I got to drop this... Hell is water mains doing there? Why have ammunition near open fire? Don't get it. Don't worry GG, it's not you. It's the system that's broke...

andy0 · ‎17-03-2012

ramon_0404 wrote:
Sorry but I got to drop this... Hell is water mains doing there? Why have ammunition near open fire? Don't get it. Don't worry GG, it's not you. It's the system that's broke.

yeah, what prize idiot designed a world full of buildings that need water and electricity at the same time?

mcrosby78 · ‎17-03-2012

So long as backup systems are now implemented so this issue can't happen again, I'll be happy. :-)

simonrawson · ‎17-03-2012

I think this sort if incident is all a risk. It's likelihood is remote, so giffgaff/you/I tolerate it. I don't have a problem with that. But recovery seems to have taken a long time. Not sure that's ok.

poochop · ‎17-03-2012

I normally don't like repeating what others have said unless I strongly agree but good job on reporting , apologising and general openness. Higher profile service providers have had higher impact outages with poorer communication. I don't necessarily agree compensation required but if you choose too, I agree with the Help the Heroes.

djrich04 · ‎17-03-2012

Many thanks for the apology & explanation of the recent water related problem. I am most impressed, with my previous service provider, I never got either. These things happen sometimes, but we can work around the problem, so good luck with getting it all back to normal.