Today's Service Outage

[gaffer]

As you very likely to be already aware, today we've suffered a major service outage - it was caused by major water damage to the computer network in a our data centre and it took nearly 8 hours fix.

Firstly I'd like to offer my apologies for the interruption in service and thank all of our members for their patience whilst we fixed the issue.

The service is largely back up and running now - the exception is new activations which are currently being queued and our service teams are working on a fix.

Some of you have asked about compensation - we're going to focus on fixing the issues we've had and make sure we have a stable service first - and then we'll sit down think of ways to make it up to you.

We'll continue to post updates on the support community so click here for the latest news.

Thanks once again for your patience.

Mike Fairman

CEO giffgaff

Update 21:15

Thanks for everyone's comments - especially those appreciating our way of keeping you up to date!

In response to a few of the issues raised...

Porting - those that were scheduled for today may have been delayed - we're continuing to work on these as fast as we can. 

Texts - we think that for most of the outage, inbound text messages should have been delivered. Whether those that failed initially will ultimately be delivered will be determined largely by the re-try policies of the network from which they were sent. In other words, you may or may not get them depending on which network sent them.

Root cause - we will of course investigate this and report on the conclusions. Our data centre does have an uninterruptable power supply and backup generators but its possible that even if these worked the water damage to the network gear was sufficient to bring the service down.

[oscar3red]

I've only been with giffgaff for 2 months and yes I've had problems today BUT what a welcome change to receive an apology so quickly and honestly. Now I know I've done the right thing moving to Giffgaff Thanks again for keeping us all informed

[reeltoreel]

Thanks for the updates through the day, Sh*t happens sometimes but its how you deal with it that counts.

[billysiv]

No one can predict a burst water main. But my major gripe is that my girlfriend has to stand on a chair in her bedroom to text me and yet the network provider reckons her area is well served. Maybe getting better coverage in blackspots would be a better use of your time than trying to cope with events beyond your control. I hope the water company will compensate Giffgaff as it is their fault.

[seeley777]

To everyone that is moaning about how hard done by you are, get over it. Before you start whining that you couldn't use your phone for a few hours and you couldn't update your facebook status, boo hoo. Problems happen, it is as simple as that.

You got updated on what was happening so you knew there was a problem.

If you don't like it, leave. But considering how amazingly cheap the giff gaff guys are, you going to have a hard job finding an alternative that gives you anywhere near what giff gaff gives you for a tenner, so I highly doubt you will.

You don't need compensation, you need to get a life and accept that life was fine before mobiles etc and a few hours without is not the end of the world.

Thank you to the giff gaff team for keeping us updated and sorting the problem as fast as you did.

[beedoc]

Well done to everyone who got the service back up and running. In situations like this it would be beneficial if there was an area on the Website (or an emergency website) where only valid updates on the status of the problem we're displayed. whilst everyone had a right to put in their tu'pencd worth it tended to cloud the real valid information coming from those who were in touch with the problem. So would prefer to see Status updates, with no comments from the general community. Whilst its not obvious which elements of the GiffGaff network went down it seems to have involved servers do it would be nice to know what efforts GG will be putting into using services in "the Cloud" on various geographically split application servers. Hindsight "has 20-20 vision" so these are observations based on what happened today. We can all learn from this failure and design in more resiliency but it's how we adapt to the failure that makes us better.

[brijoolz]

Thanks for keeping us informed.

[pinklady1988]

Seriously water damage? Is this a joke? All these southerners complaining about water shortages yet they still have enough to criple a whole network for a day. End of the day water damage or something else giffnaff has serious outages every few months, far more than any other network nationally. If you want to make money from your network giffnaff is the network you want, If you actually want to use your phone then this isnt the network to join. You can get cheaper payg rates on networks with call centres(the excuse giffnaff gave for not having them was to be cheaper) there are now networks that offer truely unlimited that dont cry when you use it, hide caps or say your effecting other users all for a similar price and every other network is nationally more reliable(cant comment on local issues because i dont live everywhere). Compensation? Giving people the credit on their phone as cash along with a pac code would be compensation enough, because if people stay this compensation could become costly every few months.

[trivvie]

Thanks for all your hard work in trying to get the issues sorted.  More importantly thanks for keeping us updated.

The unforseen happens to all types of utility services and you can't be blamed for it.  Conversely give yourselves Kudos for the way you handled it.

We don't compensation from the water board or the electricity or gas companies if their are outages with them, so please don't even begin to think of compensating your customers, just review what happened and consider what DR systems you need to minimise the damage if it happens again.

Your loyal customers will stay with you ........ and thank all of you for your hard and stressful day today

[michaelburgess]

Yes it was a little annoying, but I do believe that the network will learn from this. Giffgaff is in its infancy and things like this are to be expected. Sometimes it takes a harsh lesson like this to learn from and build upon. Like someone else touched upon, I actually quite enjoyed having no signal as I was able to focus on my work more today! For anyone complaining that they've missed vitally important calls etc, you sound like business users and should really be looking at business packages, not a consumer online-only network. The age old adage of "you get what you pay for" stands true - if you're not happy, go and pay o2 >£20 for an equivalent package!

[faybijou]

Yup! I could have done with my phone today. Nope! Not multi-million pound deals that i missed out on, or working somewhere that won't let me make a single important, personal call on their landline but will let me sit all day making calls and sending texts from my mobile phone; I have work to do I don't have time to play telephone operator. Just something personal, but I used a land-line. Where I work has several back-up servers (is that right - bit of a technophobe here?) but still the whole server/internet/system has been down twice this year for between 4 hours and three days.

I now pay half what I did with my last provider for double the calls, unlimited texts and unlimited data (I had 10mb per month before). I am NOT embarrassed that I convinced my other half to come to Giffgaff - he made an informed choice. You know - like he has his own brain to make his own decisions? We both agree it's one of those things. Three years ago our electric was out for 4 days - and I was having to go out and ask the guys digging up the road for any updates. I didn't have a choice about that; but you know - it happens!

Giffgaff I love you. We love you. I personally would like compensation to go on improving the servers (?) or to McMillan Nurses.

I had no intention of writing all that. What I really wanted to say is that for the last half hour I've been reading the comments on this thread....... and had the best laugh I've had in a long time.  
 I mean really people? Get a life. And use the fekkin' spell check before you press "post".