What parameters should Eir line engineer check during fault finding?

TomOnBoard

I've been having an ongoing Packet Loss issue since my VDSL service was installed 6 months ago. In the course of the fault finding process, Eir lads came to the house on 3 occasions to check the line, as they were tasked through the Pure Telecom service desk.

On each occasion, The engineer plugged in a Megger and then pointed to the values of 3 things on it-
attainable speed (112k)
FEC (0)
and CRC (0)

as evidence that everything on the connection was OK, and the problem was with the provider. These checks were done for a couple of minutes after the Megger was switched on and made its connection.

Are these the only 3 values that would need to be tested in such a situation and is a couple of minutes enough to catch problems? I had expected more exhaustive testing from an engineer with such a piece of equipment, in order to troubleshoot a problem that has evaded a solution for months. The lads were only focused on those 3 things and didn't really want to get into anything by way of ping tests, latency, packet loss etc.

ED E

On the phone so this answer can't be as thorough as always.

1. Engineer, no. Field service technician. Some have 15yrs teclo experience, some only stopped digging footpaths for sierra last week.

2. Ping/PL/IP are a level above him, he works between you and the DSLAM primarily, an ip level issue can occur at the aggregation or handoff to pure which he doesn't understand nor care about.

3. Metrics
Up down atten, up down snrm, per channel PSD, fec, crc, line capacitance, RF graphing.

Tests include what his meter(JDSU/heritage) can read and also testing from the MTAU module in the cabinet (primarily what they call a MELT test).


They can apply what the old timers call a "golden modem" for extended testing but good luck getting them to bother.

What Id suggest:
A. Confirm the packet loss is to *the first hop* between you and the public net, if its not then Pure need to sort their crap out.
B. Post VDSL stats here for review, ideally after 5 mins uptime and then a few hours.

TomOnBoard

Thanks for the replies. It great to get some knowledgeable thinking into this ongoing issue.:):):)

I'm still working on A. in the above response. When traceroute fails, it is showing as unable to access route points earlier rather than later in the trace, but I need to do a bit more work on that.

In relation to B., there are a few pages of xDSL stats available on the router. Most of these don't change. However, I am posting here a sheet I put together of the stats that do change at four different points in the uptime of the Router, i.e at 5 minutes, 37 minutes, 93 minutes and 361 minutes.

Note, the PPP connection had to be restarted at the 93 minutes mark, as packet loss had risen up to 60% over a 10 minute period. At all other times, before and after this restart, packet loss of between 3 and 15% was happening.

TomOnBoard

Since I posted the analysis from 2 days ago, the Packet Loss has been ranging from 3-10%. At one point today, it went as high as 20% for a time not long before there was a PPP re-start (not initiated by me).

The DSL connection has now been up for over 2 days. Following on from the readings I posted 2 days ago as requested, here's the latest set of readings from the xDSL stats where values change over time:

Uptime: 56:40:25
Noise Margin (Up/Down) 9.5/16.6
FEC (Up/Down) 2608/2533
ReXmt (Up/Down) 766/423
ReXmtCor (Up/Down) 739/380
minEFTR (Up/Down) 20473/92145
ErrFreeBits (Up/Down) 63681688/286915495
ErrS Packet Send 30680
ErrS Stat Send 15430

ED E

Are CRCs counted?

The figures are climbing but not hugely but if ErrS is Error seconds then its churning packets all right.


What device did Pure give you?

TomOnBoard

Up and Down CRC are being displayed. They've been at 0 all the time.

The ErrS values come from a section headed Vectoring which also contains:

Vectoring stats
ErrS Packet Send
31026
ErrS Packet Drop
0
ErrS Stat Send
15513
ErrS Stat Drop
0
Direction Mode
0
Rx BitSwap Mode
0
Disable VN Mode
0

I'm afraid I don't understand what 'churning packets' means.

The Router is a Technicolor MediaAccess TG588v v2

Thanks

ED E

Not sure how limited the UI is but if you have the option to disable vectoring try that for a few hours and see what PL levels are like.

TomOnBoard

I'm afraid I can't find any option to disable vectoring.

TomOnBoard

Hi Ed,

What does churning packets mean in his context? And how bad is that value?

Thanks

kevinbad2010

ED E wrote: »

Tests include what his meter(JDSU/heritage) can read

If we bought a JDSU/heritage would it be possible to test our own lines?

ED E

kevinbad2010 wrote: »

If we bought a JDSU/heritage would it be possible to test our own lines?

Wrong model, but as an example....
http://www.ebay.com/itm/JDSU-Viavi-DSAM-6000B-xt-Digital-CATV-Meter-w-CableLabs-BPI-Cert-3-DSAM-6000-/391800959686

I doubt you're that into xDSL.

ED E

TomOnBoard wrote: »

Hi Ed,

What does churning packets mean in his context? And how bad is that value?

Thanks

TBH the values are presented differently to Huawei/Zyx gear and Im not sure Im reading it correctly, but climbing values added to extreme PL (anything over 0.5% is crap) would correlate as a PHY issue.

The high horse brigade

kevinbad2010 wrote: »

If we bought a JDSU/heritage would it be possible to test our own lines?

No, you need to disconnect from the exchange at tree cab to properly test the line to your house.

TomOnBoard

kevinbad2010 wrote: »

If we bought a JDSU/heritage would it be possible to test our own lines?

Holy Thread Hijack, Batman!

TomOnBoard

I restarted the PPP connection just over 2 days ago and miraculously had no Packet Loss at all for almost 48 hours. Then the PPP restarted (not triggered by me) at midday (when it was exactly 2 days up, so the restart was probably based on a timer somewhere) and Packet Loss is back up to 10-20%. The Broadband link is still up and reading that it has been connected for over 2 days.

The start of this state of PL exactly matches with the start of the new PPP connection. This is the kind of thing that is happening all the time. When PL is very bad and the connection becomes unusable, I regularly have to stop and restart the PPP connection and in perhaps 9 out of 10 such restarts, PL drops significantly and the connection becomes more usable again.

I am almost certain that something that is out of whack somewhere back in the network is causing this problem. However, I simply don;t know enough of the technical stuff to be able to figure out what is going on. My basic understanding of DSl and I.P. tells me that if the problem was being caused by a bad physical connection in/between my house and the Eir cabinet, why would the level of PL change with different PPP connections? Eir have been out many times, and done the basic checks I outlined earlier, but found no fault.

Pure Telecom have told me they wont do any more since Eir charged them for the 3rd call-out to my house in June. Basically they want me to move to another provider. I am so frustrated and annoyed with their approach which had appeared helpful until Eir raised that charge (which they then tried to bill me for). So much evidence points away from the last mile and toward the core network/ interconnects and they simply can't or wont get to the bottom of it.

I know that the best advice would probably be to just suck it up and move provider. However, a stubbornness coupled with annoyance at the way they are dismissing me is pushing me to get to the bottom of this.

Does anyone out there have enough knowledge to be able to throw light on what might be happening here? I'd be very grateful for any such useful guidance at this point. Thanks also for previous helpful replies.

ED E

The fact that several re-dials change nothing then suddenly another one fixes things might suggest some of their RAS nodes have issues and you eventually get lucky and get a good one.

Physical faults would change when a re-sync occurs, not when sync stays up but the PPP tunnel on top is reset.

TomOnBoard

ED E wrote: »

The fact that several re-dials change nothing then suddenly another one fixes things might suggest some of their RAS nodes have issues and you eventually get lucky and get a good one.

Physical faults would change when a re-sync occurs, not when sync stays up but the PPP tunnel on top is reset.

Thanks Ed.

In fact every time the PPP is restarted (without touching the DSL link itself) it affects PL, usually bringing it way down to acceptable levels (as in <10%) but occasionally pushing it up beyond acceptable levels (usallly >10 but could be as high as 40-70%).

So using your logic, which makes sense to me, it appears that whatever this issue is impacts most of the RAS nodes to which I might possibly connect and not just a few. That would imply that it is the interconnection between the Eir infrastructure and the BT Wholesale infrastructure that contains some peculiarity. But then, I'm asking why loads of other users are not experiencing this? That piece doesn't make sense. BTW, where would those RAS nodes be located? are they likely to be in the local cabinet, back in the exchange or in a server farm somewhere. Obviously the closer they are to my location, the greater is the likelihood that the fault lies there as fewer users would be involved. As far as I know, most people local to me are on either Eir or Vodafone- based services.

Anyway, I'm not likely to get a response or any progress from Pure at this stage. I have complained to them that they have simply blamed the problem on 'something' in my house, even though Eir say its not, and we haven't moved on from that in over 2 months now.

At least I have some understanding of networking. I wonder how do the many people who don't know how to do the things I'm doing such as doing the ongoing analysis on a 24/7 basis, restarting the PPP etc. manage. They must have to put up with very poor quality service with daily router reboots to get even a basic service.:mad::eek::mad:

ED E

Eir operate like 9-12 RAS clusters nationally, Pure might only have one or two at regional POPs.


You could go the COMREG route but changing operator might be better for your sanity.

TomOnBoard

ED E wrote: »

You could go the COMREG route but changing operator might be better for your sanity.

I certainly see the sense in your advice. However, isn't it a dreadful reflection on the management / governing of our country that the most sensible advice is to avoid using the so-called regulator when a telco fails its customers. Like so-called regulation of other sectors of the country's operation, toothless organisations have been set up with highly paid public sector officials who cannot discharge their responsibilities to the people. Clearly Comreg is but a small symptom of this issue- how have we all suffered from similar toothless regulation of banks etc not so long ago.

Anyway, that rant will do little to resolve my problem. I suppose I'll have to start the hunt for a replacement. Now to establish how Vodafone versus Eir compare in my area. I suspect that, as they will both be using the FTTC service available at my local cabinet, the question will once again revolve around quality of service in/from their respective core networks.

benmeade

Also with pure and just like you have been suffering ongoing problems.
I first noticed speed dropping around 8pm in the evening, next morning it would be back at full speed 65mb. It has become so regular that i just give up using the internet in the evening if speeds are down below 4-5mb.
The weird thing for me is that the upload speed is rarely affected despite the download speed flat lining!!

Just like you i dont want to change provider i want them to fix it. Also have had 2 eircom lads call and say all is ok but if only they called after 8 when its crap.
Also pure tryed to pass the eir call out fee on to me even though i never requested it.

Pure have told me i and released from my contract and no penalty applies if i go elsewhere.
Thing is it was FAR TO EASY to be released from my contract, usually this is never an option when your tied in. Makes me wonder what pure are aware of that they dont want to tell me or admit that they cant repair.??

To add to the equation my next door neighbour is with sky and also suffers a carbon copy of my issues with dropping download speeds.

Siro is an option but their idea of an install is far from tidy.

TomOnBoard

Very interesting similarities in treatment by Pure tech support!

One of the things I found very useful in tracing my problem was the use of the Broadband Quality Monitor (BQM) that is available at www.thinkbroadband.com. Basically, assuming your link isn't being restarted every couple of hours, the BQM monitors the external I.P. Address over a 24 hour period (until the I.P. Address changes unless you have a static I.P. Address) and presents the results of continuous ping tests that can help to isolate the time of day at which various network delays are occurring.

While others on here will be able to give you a much more useful technical response, I see the main difference in our situations from your description as being:

a) my link can go bad and back to good at any time of day/night
b) my link issues are entirely caused by packet loss; yours might be more typical of time of day 'contention' where other subscribers' usage is saturating the parts of the link that you are sharing. If you run the BQM, it will greatly help you to figure out if packets are being delayed or lost entirely.

Are you and your Sky neighbour both on an old telephone line linked back to an Eir Fibre-enabled cabinet? If so, the fact that your next door Sky neighbour is a carbon copy tells me that whatever yeer both being connected to and/or the exchange behind it are saturated at the bad times of day, so everyone is trying to squeeze their downloads through a link that is not big enough.

Has your neighbour complained to Sky and what kind of response did s/he get?

benmeade

Isnt it interesting that the general spiel is that fibre cant be contended. To be fair the slow down might be earlier in the day and its when i want to use it is about 8pm.

Pure have never actively picked up the phone to contact me about the issue. Ive always followed up as to if they know how to resolve the problem. They have said im not the only one with drops in download speed.
Sky have also never called my neighbour back despite telling him they would. There is a bigger picture and i smell a rat.

TomOnBoard

Perhaps a better term would be 'congestion' as I don't see providers using the 'contention ratio' metric any more for fibre based services.

My use of the term is in the context of there being some blockage at a point between the ports at the cabinet to which both you and your neighbour's phone lines are terminated and the core network which provides you with the actual connection out to the Internet. This could be at the point where the pipe leaves the exchange for example rather than in your house. As I understand it, that could explain why the Eir line techs found no faults: they're only testing from your Phone point to the local cabinet/exchange, rather than from the cabinet back to the Provider's core network.

The problem I found with Pure is that they appeared unwilling/incapable of establishing where along the route a problem might be occurring, had little or no proactive management of the 'end to end' fault finding across all the companies (in my case KN, Eir, BT Wholesale) involved. As the provider they should clearly establish where a problem lies, rather than passing such a failure off as 'it has to be on your end' or similar with no evidence at all for saying that.

Much of the blame for this lies in the way the market is regulated here and the fact that Eir own the last mile infrastructure and can do mostly what they want, only helping competitor providers grudgingly and sparingly. The amount of subliminal marketing that one of the Eir lads who came to the house tried on was exhausting, basically echoing what I'm saying about the disconnects between the various companies, and more than implying that if I was with Eir, they would take ownership of all aspects of the service and my problem would have been solved months before. And he was probably correct.

So basically any company (other than Eir) who sells a service that uses the Eir phone lines to get to your house will be equally dependent on the pieces of Eir infrastructure over which they have no control. The further back in the network that you must go before the service breaks out of the Eir network, the greater will be that dependency.

The 'rat' here may well be that both Pure and Sky know well where the problem lies and can do SFA about it.

ED E

Your line is contended 1:1. Its dedicated to you. Thats better than Virgins cable, its better than the FTTH products from OpenEir/SIRO.

From cab to agg in the exchange there's abundant bandwidth. In theory its possible it would congest but in practice thats not happening.


The issues you're probably seeing are on national transit (eg from OpenEir in WTD to Sky in Belfast) or peering out to the wider net. I've got 240Mb, and I get it. But I dont get it to Frankfurt, or NY. Nobody does.

With Pure I suspect the issue is a little more severe.