Is VoIP ready for the real world?

JimmySmith

Blaster99 wrote:

I doubt that ATA supports it. You can try it with the X-Lite soft phone. It should be downloadable from here: http://www.tucows.com/get/309984_117238. You need a headset as well. There are instructions on Blueface's support site for how to set it up. You should disable the codecs other than iLBC.

I haven't been able to find a line with packet loss, so I haven't been able to do any meaningful testing. If you have a line without packet loss, G.711 gives the best quality as it's uncompressed. G.729a and iLBC much the same.

Your Linksys probably gives call stats on ongoing calls, so you can see if you're getting packet loss during calls as a way of troubleshooting the problem.

Thanks for that.
I'd like to be able to use the phone without the pc on though. i think i'll stivk to using voip for outgoing calls only for now.

Blaster99

I didn't mean for you to permanently switch to the soft phone. You obviously bought the PAP for a reason. But if you do some testing with the softphone and find that the iLBC codec works more reliably for you, then you at least have some data to work on. What's your ISP?

JimmySmith

Useing Netsource at the moment for voip. Its not amazing but works ok for the home phone.
I also have i have ICE comms wireless. Thats the one i want to use the VOIP on, so i dont have to pay line rental. The plan is that as soon as voip is reliable i ditch the landline. I understand that wireless has some packet loss so thats why the interest in which codec handle packet loss given that bandwidth is not a concern, 2Mb/512K up.
ICE used to be a great service but its been extremely flakey the last month or so. Massive slowdowns and huge packet loss. This has also been happening for other ICE users i know over the last month. Which reminds me i must post a thread on ICE.

Blaster99

These are my ping stats to sip.blueface.ie right now:

Statistics for sip.blueface.ie:
Packets: sent=500, rcvd=462, error=0, lost=38 (7% loss) in 49.640521 sec
RTTs of replies in ms: min/avg/max: 37.597 / 39.636 / 53.737

G.711 works poorly from my IP phone, lots of noticable dropouts. Using X-Lite with iLBC works better but it's difficult to compare them for sound quality as I'm using a phone in one case and headset in another and X-Lite always sounds a bit iffy in my opinion.

Skype-to-Skype around this time sounded amazingly good, like in the next room. SkypeOut at this time had no quality problems other than that the far end sounded like they were in a barrel. But maybe that's just the way a telephone sounds when you're comparing it with a wideband codec.

The ping stats for the Skype user:
Packets: sent=500, rcvd=480, error=0, lost=20 (4% loss) in 49.923004 sec
RTTs of replies in ms: min/avg/max: 47.125 / 115.754 / 353.109

And here are some stats for other SIP providers:

Statistics for smart076.ie:
Packets: sent=500, rcvd=498, error=0, lost=2 (0% loss) in 49.916410 sec
RTTs of replies in ms: min/avg/max: 15.194 / 17.089 / 28.458

Statistics for sip.broadtalk.ie:
Packets: sent=500, rcvd=486, error=0, lost=14 (2% loss) in 49.916176 sec
RTTs of replies in ms: min/avg/max: 15.277 / 39.933 / 258.026

Statistics for sip.voipcheap.com:
Packets: sent=500, rcvd=467, error=0, lost=33 (6% loss) in 49.326238 sec
RTTs of replies in ms: min/avg/max: 25.748 / 28.680 / 154.815

These ping tests were done with

hrping -s 100 -n 500 -l 32

I would suggest that anyone thinking of going VoIP with something like G.711 or G.729 should run a lot of ping tests to a selection of VoIP providers.

Once I have the G.711 -> iLBC transcoder setup, I'll post some more results.

WillieFlynn

Blaster99 wrote:

The real world is not packet loss free and is not full of stable pings and packets arriving in the right order. Skype is significantly better at dealing with the internet connections I've thrown at it and that's presumably down to the codec. I will continue to maintain that stuff like G.729a is geared towards managed networks with QoS etc and Skype (iLBC/iSAC) deals with the real world.

I found that VoIP providers like blueface, work much better than Skype; to the point that I no longer use skype. (BTW my ISP is netsource, my brother also uses blueface with no problems from Finland)

Also every smart broadband customer, is using VoIP for normal landline calls whether they know it or not. Then you have BT in the UK, where they are replacing all phone lines with VoIP. So the common VoIP protocols, do cut it in the real world.

It is possible that under some conditions skype works better. The main advantage which skype has, is that it can get through firewalls in companies with out having to get premission.

Blaster99

"works better", what does that mean?

Freddie59

Blaster99 wrote:

"works better", what does that mean?

I would presume it means of a higher quality. When Skype-Skype is used between two PCs it's FM quality sound. It can't be beaten. But not everyone wants to sit at, or use the phone via, a PC.

No-one's knocking Skype - merely giving Blueface the credit they deserve (take a bow lads). As I've said already, I have it and it works perfectly. As I'm a domestic customer, I presume that would be classed as the 'real world'?

aaronc

Blaster99 wrote:

These ping tests were done with

hrping -s 100 -n 500 -l 32

I would suggest that anyone thinking of going VoIP with something like G.711 or G.729 should run a lot of ping tests to a selection of VoIP providers.

Ping tests are not the best way to test a connection for VoIP except to give an idea of the latency of a connection and a very rough gauge of packet loss. If your ping reposne times are less than 150ms or 200ms then it's unlikely you will notice anything on your call. With regards loss you'll often find a connection getting 0% ping loss but if you load it up with RTP traffic you can find up to 5% loss.

I'm not familair with hrping but I assume what that line is saying is send an ICMP Ping every 100ms with a 32 byte payload 500 times. The payload is a bit small for VoIP but ignoring that the test is only 50 seconds which probably isn't enough to give a connection the workout it will typically get from a real call.

The call audio test is of course the best measure but if you want to get metrics on a connection ideally you'd send UDP packets at intervals of around 20 to 30ms with payloads of 80 to 160 bytes. If you've got a Windows machine you can test your connection using a little app we whipped up precisley for this purpose:

http://www.blueface.ie/downloads/BlueFaceNetQualityMonitor-0.3.0.msi

Aaron

Blaster99

aaronc wrote:

Ping tests are not the best way to test a connection for VoIP except to give an idea of the latency of a connection and a very rough gauge of packet loss. If your ping reposne times are less than 150ms or 200ms then it's unlikely you will notice anything on your call. With regards loss you'll often find a connection getting 0% ping loss but if you load it up with RTP traffic you can find up to 5% loss.

I would disagree about not noticing anything. Last night, when I had about 7% packet loss, I could very clearly hear it. As I was making the calls, I checked the phone status which has an absolute packet loss counter, and every time I heard a dropout, the counter went up. I have never suffered any real problems from latency, but I don't think I've ever seen it worse than 150ms. I could also tell from the pinging that the packet loss would typically occur in groups. Perhaps this is normal. On the phone, the packet loss counter always jumped by perhaps 5 packets at a time.

Tonight when I have a lot less or no packet loss, the line quality is excellent.

aaronc wrote:

I'm not familair with hrping but I assume what that line is saying is send an ICMP Ping every 100ms with a 32 byte payload 500 times. The payload is a bit small for VoIP but ignoring that the test is only 50 seconds which probably isn't enough to give a connection the workout it will typically get from a real call.

The reason hrping is much better than ping is that ping sends a packet once a blue moon. With hrping you can fire packets much more often to get a better picture of what's going on. I appreciate that ping has limitations but it's the best I had at hand. I had to set the packet size low to get Smart's SIP gateway to respond to the pings, to get the comparative numbers.

I sent 500 packets to not have to spend a lifetime gathering ping stats and I incidently ran these tests a few times and I got pretty consistent numbers back from the SIP gateways I tested with. In the case of Blueface, I'm fairly sure the problem is not with Eircom but with the transit between Eircom and Blueface. When you get the INEX stuff up and going, I would expect that to be resolved. With other networks my experience has been that the problem is typically either access contention or in internal backhaul and no INEX peering in the world is going to solve that.

aaronc wrote:

The call audio test is of course the best measure but if you want to get metrics on a connection ideally you'd send UDP packets at intervals of around 20 to 30ms with payloads of 80 to 160 bytes. If you've got a Windows machine you can test your connection using a little app we whipped up precisley for this purpose:

You da man. I ran it there and it's v cool app! It confirmed something I suspected, which is that the packet loss occurs more frequently outbound than inbound in my case. I enabled silence suppression last night and the incoming audio sounded a lot better. I'm guessing the outbound packet loss was echoed back to me as I was listening and when I switched on silence suppression, there was no traffic from me and the phone reported a lot less packet loss as well. Don't know if that makes sense or not. I incidently got less packet loss with G.729 as well, presumably because it's sending less or smaller packets, and therefore it sounded better. So using G.729 helps, but it's just a workaround and doesn't really solve the underlying issue in my opinion.

Another thing is that your app is reporting significantly higher jitter than my phone does. My phone shows jitter around 8ms, your app shows it at 30ms. I don't know what the story is there. Maybe this is the power of hardware accelerated codecs.

aaronc

Blaster99 wrote:

Another thing is that your app is reporting significantly higher jitter than my phone does. My phone shows jitter around 8ms, your app shows it at 30ms. I don't know what the story is there. Maybe this is the power of hardware accelerated codecs.

The app does not subtract the correct interarrival arrival time from the actual arrival interval so a packet that arrives correctly spaced at 20ms will show as having a jitter of 20ms. This keeps the jitter lines above 0 and makes the graphs easier to read to be more correct jitter on the graphs should be called interarrival time. The app tries to send out 20ms spaced packets so if you subtract 20ms from the jitter readings it should be more in line with what other tools will report.

The measurements should not be relied upon to be too accurate though given that the app is entirely dependent on the PC resources. The measurements should be used as a rough gauge and the most important one are packet loss and jitter discards. Generally if there's no red or orange lines showing up on the graphs the calls on a connection will be ok.

Aaron

JimmySmith

I might try that app later on.
aaron, i've tried your echo test before, which i think you get by calling 331 or somthing like that.
Its a great app, but you should really have it let you speak for about 30 seconds and then have it played back to you. Its very hard to judge the quality when its played back about 1 second after you speak because you're speaking and listening at the same time.

I'm still looking for the best voip provider, and will be checking quality often to see how the technology is going, but will be holding onto the eircom line as well for now i think.

aaronc

JimmySmith wrote:

I might try that app later on.
aaron, i've tried your echo test before, which i think you get by calling 331 or somthing like that.
Its a great app, but you should really have it let you speak for about 30 seconds and then have it played back to you. Its very hard to judge the quality when its played back about 1 second after you speak because you're speaking and listening at the same time.

I'm still looking for the best voip provider, and will be checking quality often to see how the technology is going, but will be holding onto the eircom line as well for now i think.

The echo test is on 301 (most Asterisk based providers will have it on this number). The echo test is mainly designed to measure latency not quality. When you speak if you hear your voice back almost instantaneously then you have low latency. if you wait for a few seconds and then you hear your voice you've got very high latency. You'll hear your latency shoot up if you try a donwload at the same time as the echo test.

Aaron

JimmySmith

aaronc wrote:

The echo test is on 301 (most Asterisk based providers will have it on this number). The echo test is mainly designed to measure latency not quality. When you speak if you hear your voice back almost instantaneously then you have low latency. if you wait for a few seconds and then you hear your voice you've got very high latency. You'll hear your latency shoot up if you try a donwload at the same time as the echo test.

Aaron

I see.
Wouldnt it be a good idea then to allow the user to test quality too?

Blaster99

aaronc wrote:

The measurements should not be relied upon to be too accurate though given that the app is entirely dependent on the PC resources. The measurements should be used as a rough gauge and the most important one are packet loss and jitter discards. Generally if there's no red or orange lines showing up on the graphs the calls on a connection will be ok.

When does a jitter discard occur? Is that once jitter exceeds a threshold?

aaronc

JimmySmith wrote:

I see.
Wouldnt it be a good idea then to allow the user to test quality too?

Yes, you just call somebody.

Aaron

aaronc

Blaster99 wrote:

When does a jitter discard occur? Is that once jitter exceeds a threshold?

Yes. All VoIP devices will typically operate with a jitter buffer that allow the packets to arrive out of order or with different inter-arrival spacings (jitter). If a jitter buffer is not used every packet would have to arrive exactly when the audio device is expecting it or it would be discarded. With a jitter buffer there is a window in which the packet can arrive and be put into the correct sequence to be passed to the audio device. The catch is if the jitter buffer is too large you introduce latency into the call and you get the annoying talkover where both parties pause and then start speaking at once.

In our app the jitter buffer is set at 10 x frame size, currently working out to about 2 seconds. This is very generous and most devices would typically have dynamic jitter buffer with a maximum size of 200ms.

Aaron

Blaster99

Assuming that what this test tool is telling me is correct, why is it testing against 82.195.148.216? Seeing as that is hosted by Hosting365, it's not terribly representative is it?

Freddie59

Blaster99 wrote:

Assuming that what this test tool is telling me is correct, why is it testing against 82.195.148.216? Seeing as that is hosted by Hosting365, it's not terribly representative is it?

You don't work for another VOIP outfit or Eircom by any chance, do you? You seem to be extremely knowledgable about it.:)

aaronc

Blaster99 wrote:

Assuming that what this test tool is telling me is correct, why is it testing against 82.195.148.216? Seeing as that is hosted by Hosting365, it's not terribly representative is it?

nslookup www.blueface.ie (depending on your ISP this will generally be using INEX). We don't want the tests to interfere with our real calls in anyway. The test is designed to give a rough measure of a broadband connection's suitability for VoIP and not specifically measure whether an ISP has greater contention on their internet or INEX links.

Aaron

WillieFlynn

Blaster99 wrote:

"works better", what does that mean?

When I said that blueface works better than skype, what I mean is that there is less lag, jitter, breakups, etc. In other words it has sufficient quality that you can forget that it is not a normal landline phone.

The problem I found with skype, was that it would work fine for say twenty minuits then it would breakup for 30 seconds before working again. Also found on some calls that the latency was such that you tended find you were talking across the person at the other end. Overall I found skype usable most of the time, but was constantly reminded that it wasn't a normal phone.

JimmySmith

aaronc wrote:

Yes, you just call somebody.

Aaron

I see - Now i understand why Blueface are having call quality issues as compared to broadtalk etc. Possibly braodtalk etc know how to test properly.

What you have described is not a test at all. I really hope you weren't serious.

You should understand that that calling someboby lets you know how they sound to you (downstream) only, not how you sound to them (upstream).

By what your are suggesting i would have to ask the person on the other side of the phone call what it sounds like. Not a good idea. They could be deaf, have earwax, have faulty equipment and report faults and problems that are on their end and nothing to do with the connection between myslef and blueface.

A better test would be to call blueface and blueface to play your voice back to you giving you time (about 10 seconds at a time should be enough) to actually listen to how your voice sounds being played back.

Now using this method its hard to tell if there are problems whether they are happening going up or down but it does give you an indication as to the quality of the link between yourself and blueface. Regardless of whether the problem is up or down, if a punter gets a bad quality call here then there is definitely some work to be done before they consider voip as their only phone.