Forum Downtime

G

Graham Smith

Guest
Dear VOC Member/Vincent Enthusiast

On Saturday 20th February 2010 at approximately 2020hrs GMT, our web host experienced a hardware failure as a result of an annual fire system inspection at their Data Centre. An inadvertent release of Inergen was triggered in the Data Centre environment. This release was triggered by an actuator that was not removed as required in the Data Centre's pre-test checklist.

Because of this, the entire Data Centre was impacted at some level, however, not all servers were affected. This error was not the result of a mistake by our web host or thier employees, our hardware or systems. The mistake was triggered by an error from an external contractor. With that said, our web host take their responsibilities seriously and did everything they could to restore our systems as quickly as possible.

There were multiple servers impacted by this outage and some of the servers had damaged hardware replaced while others needed to be completely restored from backups. The majority of servers that were affected are now back online.

We apologise for any inconvenience or frustration this outage may have caused you.

The Executive Committee of the Vincent H.R.D. Owners Club
 

Vic Youel

Well Known and Active Forum User
Non-VOC Member
Graham,

I'm rather surprised that the club's web space provider does not have a physical back-up at a second geographical location with a mirrored server system to provide continuity in such circumstances. I has assumed that would be the norm these days as a contingency for any web services provision. Suggest that the VOC contingencies are reviewed for backing up valuable data.

Some organisations even utilise three separate pysical locations as a contingency.

Vic
 
V

VOC Machine Registrar

Guest
Valuable Data

I take Vic's point about contingencies for preserving valuable data.

However with regard to the Forum we are not dealing with valuable data. And before anyone takes this as sarcasm or a jibe at the quality of members contributions, let me make it clear that this is not my intention.
In all seriousness, what would we have lost if we lost the entire Forum?
Yes it would be a pain and a disappointment, but when all said and done we could start afresh.
The Forum is not the receptacle of truly valuable or irreplaceable club data. Such material is held elsewhere and is backed up.

As I understand it the Host does run a normal mirrored back-up system in case of drive failure, but that mirrored system is in the same locality and would therefore be equally affected by the problem.
As soon as you start talking about multiple geographical site provision, then the cost goes up. It is at this point you have to put a value on the data; is the value of the data worth the perpetual increased cost?
The Forum is a chat shop, a place convenient to some of our members to exchange views and ideas; are these views and ideas so valuable that we have to archive them for ever?
The Forum is not even the Club's Official organ of communication; only 60% of VOC members list internet availability on their application form.

Let me just stress: Vic's comments are not lost on me. Quite the contrary. The protection and preservation of the clubs truly valuable data figures high amongst my priorities. It is just that the Forum content does not figure in the "valuable" category.

Along with everyone elses, my thanks go to Graham for the work he puts into the Forum.
 
Last edited by a moderator:
G

Graham Smith

Guest
Graham,

I'm rather surprised that the club's web space provider does not have a physical back-up at a second geographical location with a mirrored server system to provide continuity in such circumstances. I has assumed that would be the norm these days as a contingency for any web services provision. Suggest that the VOC contingencies are reviewed for backing up valuable data.

Some organisations even utilise three separate pysical locations as a contingency.

Vic

Yes Vic you're right - there are plenty of other hosts who provide different things to our current provider. In fact, our existing provider even offers a mirrored server facility.

However these facilities cost money, and whilst it was annoying our forum was out of action for a couple of days, it's not as if we're reliant on it to run our business. I really don't think paying for a mirrored server would be money well spent or offer value for money to our Members.

Mind you - following your two most recent posts on where we're going wrong with our IT and what we could do to improve things, you will note in the March edition of MPH we're currently looking for a new Computer Officer.

Would you like me to propose you? :)
 
G

Graham Smith

Guest
As I understand it the Host does run a normal mirrored back-up system in case of drive failure, but that mirrored system is in the same locality and would therefore be equally affected by the problem.

Although our server was one that was affected, the thing that really screwed them up was a problem with a server that deals with many of the back-end processes at the data centre. The failure of this server prevented our server (and others) from operating properly.

It wasn't that likely we were ever going to lose anything, as I occasionally back-up the files and database at home. This is in addition to the backups the data centre makes.
 

John Appleton

Well Known and Active Forum User
VOC Member
Hi Graham, I think we are all pleased to be back in action and I would like to say that I appreciate your freely given efforts to ensure that we get the best service for the best price . You obviously spend a lot of your own time making sure all runs smoothly, and must feel disgruntled when criticised by somebody stating the bleeding obvious about something they suddenly know all about after the event. Keep up the good work. John
 

ossie

Well Known and Active Forum User
VOC Member
thanks graham

well done lads had us worried glad i didnt start pushing the wrong computer keys and really messed it up.
OSSIE
 

Alan J

Well Known and Active Forum User
VOC Member
Not to worry, Graham, what is 2 days in the grand scheme of things? Anyway, I went on to my other bike sites for my evening "fix".
 
Top