eircom backhaul problems

MyPeopleDrankTheSoup

eircom backhaul problems have been going on too long. for the past month, at peak times and Sundays (especially today) the bandwidth has been crap. the bottleneck seems to be the backhaul and using tracert it seems to be their London servers.

if you do a speedtest, it connects to the closest server so speeds are usually great. it's only when you're streaming stuff from the US, it's terrible.

anyone finding the same?

primalux

Yes, absolute crap today.

ZephyrSurfer

It's ridiculous that such an issue could
a) As extreme as it is
b) Persist as long as it has

Crazy carry on there.

swiftblade

Youtube is unusable tonight. As in, videos won't even load.

Dailymotion and other sites are fine.

I'd love to hear eircom try and talk their way out of this one.

_dof_

georgiecasey wrote: »

eircom backhaul problems have been going on too long. for the past month, at peak times and Sundays (especially today) the bandwidth has been crap. the bottleneck seems to be the backhaul and using tracert it seems to be their London servers.

if you do a speedtest, it connects to the closest server so speeds are usually great. it's only when you're streaming stuff from the US, it's terrible.

anyone finding the same?

Yes, it's bad tonight, I was trying to lad a stream hosted by livestream.com and won't load when connecting directly, but when I turn on zenmate in chrome to go via a VPN, it loads perfectly.

Your point about the backhaul problems in London seem to hit the nail on the head. I tried a few speedtests, in one I selected Gloucester server, and it was working fine, giving me full download speeds, which for me is 26Mb/sec, but when I select the one in London, it drops to 8Mb/sec, and pings were taking twice as long, 121ms

Zatoichi

Any of you changed the default ericom DNS to a public one to see if it helps?

_dof_

Zatoichi wrote: »

Any of you changed the default ericom DNS to a public one to see if it helps?

I changed to the google one fairly soon after I got efibre cos the eircom one was soooo slow, it was taking ages to load web pages, displaying "Waiting for <whatever hostname>" at the bottom of the chrome page...

Changing to the goolge DNS improved it, but even with that sometimes it takes ages to load some pages..

Update:
I can stream HD on youtbe without buffering at the moment, so it seems the immediate problem with youtube is solved but it's very hit and miss.

Livestream still not loading the stream without loading zenmate VPN though

_dof_

and not livestream is working again....

I hate these intermittent problems, even when they happen frequently, because you know when you call up tech support the problem won't be there at the time....

Zatoichi

I get sub 30ms in most UK servers and around 65Mbps, except for the bottom London server on speedtest. Get a ping of 112ms and barely 10Mbps. Even with that I can't even get a youtube video running at all.

MyPeopleDrankTheSoup

surely if it is just a london server congestion issue, it's an easy fix, just upgrade the hardware in London. it's been like this for at least a month. bad sign from eircom and future investement that they haven't bothered to fix this.

conorkvngh

So would anyone be able to tell me how I can slightly fix this? Or is there no fix?

_dof_

conorkvngh wrote: »

So would anyone be able to tell me how I can slightly fix this? Or is there no fix?

I downloaded the zenmate extension for chrome brower, it gives you a VPN, and you can select the location for the far endpoint of the VPN, I selected London as the location. So Zenmates sets itself up as a proxy and sends all my browser traffic through its server in London.

That route must avoid the congested part that Eircom's traffic normally goes through becuase it works find when I enable Zenmate. When I turn it off again, and try to connect directly, it slows down or hangs, with buffering in streams etc.

MyPeopleDrankTheSoup

ah cheers _dof_, i nevr thought of that. i'm connected to a squid proxy on my french OVH server and getting 50Megabits down and perfect gamepass stream! crazy that it's way more stable than eircom!

ED E

georgiecasey wrote: »

surely if it is just a london server congestion issue, it's an easy fix, just upgrade the hardware in London. it's been like this for at least a month. bad sign from eircom and future investement that they haven't bothered to fix this.

Not necessarily theirs, ISPs networks only go so far until your traffic hits a Tier 1.

MyPeopleDrankTheSoup

it's 5am and i can't load stackoverflow.com through eircom. works no problem via my VPN.

i checked DNS, and both my server and PC resolve it to IP of 198.252.206.16

something seriously wrong with their network.

ekimiam

thats it, im finding an alternative on tuesday ! they have been useless for far to long

Spear

This sounds more like a peering saturation issue rather than backhaul. If it were the latter everything would be impacted.

mememo

This is a joke today i cant even connect to some of my favorite sites without using a VPN I'm going to demand Eircom wave my broadband fees for this month tomorrow when i phone them

jantheman91

Can't load youtube videos in 360p with 90 download.

It's actually ridiculous.

Cutting corners again, typical Eircom.

Mayo Yid

jantheman91 wrote: »

.

Cutting corners again, typical Eircom.

And you know this, how?

loyatemu

Eircom peer with Google in Dublin: https://www.inex.ie/ixp/peering-matrix - YouTube traffic would use this link (interestingly UPC don't peer with Google at INEX).

Just because a particular SpeedTest server in London is giving slow results is meaningless, that server is hosted on someone else's network and the latency issue could be anywhere.

Having said all that, I've found access to all sorts of sites (including the likes of Amazon) desperately slow through Eircom this last week, so there clearly is an issue somewhere.

ED E

loyatemu wrote: »

Eircom peer with Google in Dublin: https://www.inex.ie/ixp/peering-matrix - YouTube traffic would use this link (interestingly UPC don't peer with Google at INEX).

Just because a particular SpeedTest server in London is giving slow results is meaningless, that server is hosted on someone else's network and the latency issue could be anywhere.

Having said all that, I've found access to all sorts of sites (including the likes of Amazon) desperately slow through Eircom this last week, so there clearly is an issue somewhere.

Googles CDN for youtube isnt necessarily in Dublin though. More likely france due to cheaper power rates.

INEX is pretty small fry nowadays, check our their usage stats. Most of the time the links are pretty idle.

Hank Scorpio

It's so annoying. Check out how bad their network is

Certain apps/games the dl speed is like 5kb/s , I'm on 24 meg

Spear

nuxxx wrote: »

It's so annoying. Check out how bad their network is

Certain apps/games the dl speed is like 5kb/s , I'm on 24 meg

That's fairly meaningless in this context. It's common for backbone switches to drop or deprioritize ICMP traffic like that, it doesn't mean there's an issue.

ED E

Lines 7 8 and 9 point to the culprit.

That said, in a problem scenario ICMP (Ping and traceroutes) are the first packets to be dropped so its not a hard and fast way to diagnose.

ItHurtsWhenIP

Spear wrote: »

That's fairly meaningless in this context. It's common for backbone switches to drop or deprioritize ICMP traffic like that, it doesn't mean there's an issue.

You are bang on there. The only truly meaningful Packet Loss result is what is given at the final hop because as you say intermediate switches and firewalls will typically ignore ICMP.

Hank Scorpio

Is this meaningless too? I mean the two points where the program says PL occurs is where the latency spikes. From Dublin to Eircoms node in London and onto France again

Spear

nuxxx wrote: »

Is this meaningless too? I mean the two points where the program says PL occurs is where the latency spikes. From Dublin to Eircoms node in London and onto France again

The jump in latency is far more likely to be a reflection of the physical distance there, since it jumps with the hope between Dublin and London. But again you can see two hops completely discarding ICMP traffic.

Hank Scorpio

Spear wrote: »

The jump in latency is far more likely to be a reflection of the physical distance there, since it jumps with the hope between Dublin and London. But again you can see two hops completely discarding ICMP traffic.

When eircoms network was going OK over the summer there wasn't jumps in latency like that or any PL, they were all consistently in the mid twenties 24/7

ItHurtsWhenIP

nuxxx wrote: »

Is this meaningless too? I mean the two points where the program says PL occurs is where the latency spikes. From Dublin to Eircoms node in London and onto France again

I'm only guessing here, but from years of looking at PingPlotter graphs in a large corporate environment, I'm very familiar with reading the graphs. Ping Plotter documentation also points this out.

My guess is that Hop 4 is in Ireland as Hop 6 is in the UK, so there is a switch at Hop 5 there that is configured to drop ICMP (as pings consumes CPU cycles and the switch has more important things to be doing than replying to pings). Similarly Hop 8 is in France so there must be another switch at Hop 7 doing the same thing.

The increase in latency is simply down to the greater "distance" that the ping is traveling as it traverses international borders.

Hank Scorpio

I accept the point that switches are capable of dropping icmp packets, my initial point was outside of peak hours my pings are much lower and consistently in the 20s as I have fastpath enabled on my line. The above server for example pings 25ms offpeak but during peak hours anywhere from 60-200. I don't want to drag the thread off topic

Hopefully they can fix the issues, its so frustrating not being able to game on a stable connection past 6pm everyday

There's alot of people complaining here http://community.eircom.net/t5/Broadband/bd-p/32

Spear

If Eircom have saturated their peering, possibly to Level3 in this case, then a large carrier like Level3 would never refuse the possibility of adding more peering capacity. Which only leaves on other side of the equation as being at fault.

loyatemu

ED E wrote: »

Googles CDN for youtube isnt necessarily in Dublin though. More likely france due to cheaper power rates.

INEX is pretty small fry nowadays, check our their usage stats. Most of the time the links are pretty idle.

doesn't matter where geographically YouTube's servers are - Google and Eircom's networks interconnect at INEX, that's probably the route the traffic takes. After that its in Google's network which obviously is global.

72Gb/s currently running across INEX - hardly idle.

willycat

Look at this: twitter.com/michaelcdever/status/526874277333987329

Thanks Eircom!

ED E

loyatemu wrote: »

doesn't matter where geographically YouTube's servers are - Google and Eircom's networks interconnect at INEX, that's probably the route the traffic takes. After that its in Google's network which obviously is global.

72Gb/s currently running across INEX - hardly idle.

Do google own their own fibre into ireland? Pretty sure they're buying from L3/Telia or maybe C&W(Now vodafone). So their local peering probably doesnt matter.

MyPeopleDrankTheSoup

just an update to this, it still bad but not as bad as last week

i noticed a lot of the slow sites were akamai hosted. NFL gamepass and facebook seem to sometimes use them for images

when i visit http://www.akamai.com/ I get a strange 'Duplicate headers received from server' on eircom. works fine from my VPN.

Spear

georgiecasey wrote: »

just an update to this, it still bad but not as bad as last week

i noticed a lot of the slow sites were akamai hosted. NFL gamepass and facebook seem to sometimes use them for images

when i visit http://www.akamai.com/ I get a strange 'Duplicate headers received from server' on eircom. works fine from my VPN.

I get the same on UPC. Seems the worlds largest CDN can't maintain their own site. Not the best bit of self advertising.

_dof_

It was pretty slow yesterday for me, I tried to download a programme on my Sky box On Demand service, and it was so slow, that I tried an experiment. I configured a wifi hotspot on my 3G mobile phone and reconfigured the Sky box to connect to that instead, and lo and behold, it speeded up considerably. Changed it back to the efibre modem's wifi network, and it slowed down again.

It toook about 6 hours to download an episode of around 500 MB using efibre.

MyPeopleDrankTheSoup

Spear wrote: »

I get the same on UPC. Seems the worlds largest CDN can't maintain their own site. Not the best bit of self advertising.

wow that's strange. like the website works perfect from both my London and French servers.

Spear

georgiecasey wrote: »

wow that's strange. like the website works perfect from both my London and French servers.

Different geo location means being balanced to a different set of edge clusters, since all CDNs will geo balance traffic.

dalta5billion

Spear wrote: »

Different geo location means being balanced to a different set of edge clusters, since all CDNs will geo balance traffic.

As we've been reporting to Eircom over on their subforum, it appears Akamai sometimes return European IP's that they don't announce at INEX to Irish users. These can be located anywhere in Europe, but in all cases they seem to get routed through gtt.net's routers in Dublin, rather than via Eircom's own London or Amsterdam routers. This link appears to be saturated at peak times, but we don't know whose fault exactly it is.

Any traffic going directly through INEX to Akamais own infrastructure seems to be unaffected by this congestion, as far as I can see.

leigh852

For the first time in well over a month it seems perfect this evening. YouTube is buffering at full speed where usually I find at this time of evening its unusable. Anybody else seen improvements?

_dof_

leigh852 wrote: »

For the first time in well over a month it seems perfect this evening. YouTube is buffering at full speed where usually I find at this time of evening its unusable. Anybody else seen improvements?

Not for me. On Youtube, it's just about able to buffer enough at 720p, but not at 1080p.\\

On livestream.com it's very slow, buffering constantly even at SD.

mememo

_dof_ wrote: »

Not for me. On Youtube, it's just about able to buffer enough at 720p, but not at 1080p.\\

On livestream.com it's very slow, buffering constantly even at SD.

Same here i can get 720p running but not 1080p

Barry5714

Big improvement this week for the first time in a couple of months for me, usually slows to a crawl from 7pm but is staying full speed now. Wonder how long it will last????

dalta5billion

Same for me again tonight, 720p barely manages, 1080p buffers all the time.

_dof_

I just spotted the following response from eircom:
community.eircom.net/t5/Service-Updates-and/Response-to-customer-posts-on-recent-broadband-speeds-queries/m-p/83939#M229

Response to customer posts on recent broadband speeds queries - November Thursday 06 2014

To our customers

We are aware that recently some customers have reported difficulties with video streaming and accessing content from some sites hosted externally to our network, such as YouTube.com for example.

This is due in part to the huge growth in data volumes and data traffic that is transported over our network, which has exceeded our forecasted growth.

We are making a number of improvements to our international connectivity which will add significant capacity and this work should be completed in the next two or three weeks.

Many of you posting to this forum on this issue will see improvements as this work has completed. If you continue to have speed and have followed all troubleshooting procedures, you should call our tech support team on 1890260 260.


We will keep you posted on this issue here but if you have any queries on this, or any other issue please feel free to post us on the forum here, contact us on Twitter @eircom or call you can also chat with one of our online Chat agents at Eircom.ie.

Eircom Customer Care