Cogent Network Redundancy

Jakey Rolling

Have Internet via a local Wireless ISP, down since yesterday evening.

Normally it's down to a local outage, however this time my traces stop at a Cogent router 149.11.x.x in Dublin.

For a short while I got a message when trying to access this device " a submarine cable between Ireland and England has been cut, we hope to have a backup route operational soon".

WTF? Surely there are multiple redundant routes from Dublin to London that all ISPs would avail of as standard. Or are Cogent working some cheapskate model with no redundancy?

Anyone out there have inside info on this?

gouche

It's one of the main links operated by Hibernia Networks that has gone down.

According to Cogent it's a double subsea fibre outage.
Not an easy thing to fix.

There are of course multiple fibre links into Ireland.
Cogent, afaik, only lease one or two, hence the major outage at the minute.

Your wireless ISP may only have one fibre backhaul due to cost.

Villain

They have two cables and both are down at present, one was having maintenance completed when the other failed, the current ETR is Saturday.

gouche

Latest update from Cogent:

Welcome to the Cogent Communications status page. Customers in the Dublin and Belfast region of Ireland may be experiencing a loss of network connectivity. This is due to a double fiber cut in our vendors network. The emergency repairs in the Irish sea are still ongoing, however there was some delay due to equipment failure on the cable ship. The ETR for this repair is 11/22 Saturday afternoon, weather conditions permitting. The second break which started at 11/19/14 at 22:26 GMT, our fiber vendor has confirmed the subsea cable is damage and a ship is in route. There is no ETR at this time. The master case for this outage is HD5941711.

Jakey Rolling

gouche wrote: »

It's one of the main links operated by Hibernia Networks that has gone down.

According to Cogent it's a double subsea fibre outage.
Not an easy thing to fix.

There are of course multiple fibre links into Ireland.
Cogent, afaik, only lease one or two, hence the major outage at the minute.

Your wireless ISP may only have one fibre backhaul due to cost.

Thanks, didn't have visibility of that.

Have a fair idea of the repair process, I used to work on Cable & Wireless cable laying ships in the Far East. Wishing them fair weather!

I see my own ISP is back up using an alternative backhaul, they had just changed to Cogent a few months ago so have reverted to previous provider.

Villain

Cogent are back now, on an alternative route.

Impetus

It was reported that a company called Cogent cut two underwater cables in the Irish Sea which caused large scale disruption to broadband and other networked services today. There are other suggestions that the root of the problem is at Hibernian Atlantic. (In any event, given the incompetence of the way in which the country is run generally speaking (eg Comreg) , companies should have their own alternative routes for plan B backup.)

Irrespective Points arising:

1. Comreg should investigate and take action against Cogent/Hibernia Atl on behalf of their victim customers and distribute the proceeds of the fine to the victims of Cogent’s/Hibernia’s incompetence (if proven) – rather than diverting it into the bottomless pit of “government” (eg Irish Water and Eircode/ fake postcode) privacy intrusion and waste).

2. Ireland has the cheapest interconnectivity internationally of any OECD country. There is no excuse for poor quality service.

3. Ireland needs a greater diversity of fiber routes to the rest of the world (aside from running a string or two to GB, with a can at either end) – (a) to provide redundancy and (b) to make it more difficult for the three/four letter agencies to steal information from Irish based companies and to provide a more bullet-proof connectivity to companies such as Google & Co and (c) to provide more responsive internet access in terms of lower ping times and data fill speeds.

Ireland is in an ideal location between the US and mainland Europe to be a reliable transit point and a cloud/server space in the Euro-time zone. It cannot allow a material %age of its infrastructure to be taken out by an “incompetent electrician” working for what looks to me to be an incompetent “service provider”/re-seller who does not have redundant backup. There is a 50 billion EUR market in “cloud services” which up for grabs after Snowden & Co. Either have a clean, neutral, reliable cloud or store everything under your own control. Cloud offers enormous global flexibility to corporate and private users. The issue is surrounded by trust. The Chinese, Europeans and most of the rest of the world don’t trust N America.

Do you trust your ISP?

http://www.irishexaminer.com/breakingnews/business/cut-cables-to-blame-for-interruption-to-dublin-web-services-651758.html

daffy_duc

Impetus wrote: »

Blahblahblah incoherent babbling

Hibernia own two of the submarine cables between Ireland and the UK.
Hibernia own one of the submarine cables between Ireland and the US.

There are at least 10 other submarine cables operated by other companies connecting Ireland to the rest of the world.

Cogent purchased connectivity from Hibernia.

Neither of these companies caused the cable cuts. Its in the interests of both companies to get the cables repaired.

They are all private companies, the government has nothing to do with this.
There are no 'victims' as you want to call them.

Submarine fibre cuts are a frequent occurrence. Fishing trawlers, ships dragging anchors, natural events can all cause cuts to these cables.
This is nothing new and usually no-one notices since there is enough diversity.

Basically, this is the 1 cable cut that made the news. There were many more last week that didn't.

There are at least 3 submarine cable repair ships that operate between the North Atlantic and Denmark, and they're pretty much constantly busy repairing damaged cables.

Put your tinfoil hat away.

ED E

FWIW this happened same time last year on another set further south (C&W), links failover and they order the ships get moving. Its not great, but its not catastrophic.

We could probably do with more transit in terms of bandwidth as a whole, but there are enough links for redundancy for a comparatively tiny island nation.