Net1 major outage

Dellas

Anyone also off ? Its been down for 4 hours now and their website is down.

rugbyman

Yes, Net1 has been down since 7 pm. repairs currently taking place. earliest repair estimated 1.30 am.

Net1

All of the network back up with the exception of parts of our network in Carrickmacross.
On our backup system at present - core cisco router failure due to high levels of RF on Mount Oriel from a new radio station on the adjacent mast

DMC

Wow. i105107 causing the interference?

I presume you will be switching back to the core once a replacement is in place, so there will be downtime then?

Net1

These are RF Related problems - ie - ethernet cables acting as antennae - just to make it clear as people have being making all kinds of assumptions on other forums.
TBH our network was never designed to have 5kW on the adjacent mast ! - we are now mid way through the corrective measures which is painstaking

Our core router should be back online by midnight after its "ferrite bead" circuit board modifications today.

Although these problems only manifested themselves after the turning on of the new station, we must fully point out that their equipment is fully working within spec (we have checked!!!).

Our problems are merely a consequence of High RF levels around 100MHz being so close to the ethernet frequency - meaning we have had to run huge chunks of the network on 10 base T.


549kHz in Monaghan does not cause a problem even though the power is much hgher - infact the same gear is used there t suppy their link stream !

Sponge Bob

Ouch !

My sympathies , I remember what the MW transmiter in Tullamore does to phone lines for 5 miles around .

DMC

Nice one, thanks for the update.

There used to be a phonebox on Farrell Street in Kells that had Atlantic 252 for hold music...

watty

Now I know why I like Shielded TP Cat5e. with shield earthed only one end.

one or two turn of cable in a ferrite clip on clamp. Maplins have them for mains cable.

bigpaddy2004

Sponge Bob wrote: »

Ouch !

My sympathies , I remember what the MW transmiter in Tullamore does to phone lines for 5 miles around .

Is that TX still operating or is it shut down?

DMC

Tullamore on 567kHz was closed last St. Patricks' Day, but think it was hoped that it might be used again for a national service, licensed by the BCI.

Sponge Bob

I doubt if a green minister would ever permit such a power hog to go live on his watch

Fuzzy Clam

DMC wrote: »

Tullamore on 567kHz was closed last St. Patricks' Day, but think it was hoped that it might be used again for a national service, licensed by the BCI.

I don't think the transmitter will ever be used again for envionment and efficiency reasons but the mast could be.

Net1

DMC - That any better now ???
A lot done and still a little bit more to go

Getting up there was a task in itself this evening - thought I was in the Alps

But bed becons !!!

Why can't we do these things during the day

DMC

Mount Oriel - deceptively exposed.

Not in a position to tell you right this moment, I will tell you later today.

rav1410

Been pulling my hair out all night blaming my router for loss of connection :P

Alls fine now

bigpaddy2004

Net1 wrote: »

DMC -

Why can't we do these things during the day

NET1...its all good fun!! You just cant beat being up an icy mast in the black cold dead of night trying to terminate something tricky.

If you dont go insane in the process....you'll have time in years to come to laugh at it.!

crawler

Shudder!!!

Brings back bad bad bad memories

DMC

Net1 wrote: »

DMC - That any better now ???
A lot done and still a little bit more to go

Trying it now, here is what I'm getting...

C:\Documents and Settings\User>tracert www.rte.ie

Tracing route to www.rte.ie [89.207.56.94]
over a maximum of 30 hops:

  1     2 ms    <1 ms    <1 ms  192.168.1.1
  2    12 ms    13 ms    14 ms  unalloc-80-249-253-1.net1.ie [80.249.253.1]
  3    31 ms    19 ms    19 ms  gw.screebogue2.net1.ie [80.249.248.33]
  4    30 ms    24 ms    30 ms  core-rtr.orielcore.net1.ie [80.249.249.193]
  5    42 ms    44 ms    49 ms  ge-2-2-0-2421.ycr1.dub.cw.net [208.175.245.49]
  6     *       41 ms    50 ms  so-5-0-0-bcr1.lnd.cw.net [166.63.163.141]
  7    43 ms    44 ms    42 ms  xe-4-3-0.xcr1.lnd.cw.net [195.2.25.5]
  8    53 ms    48 ms    49 ms  tiscali-gw.lnd.cw.net [195.2.3.62]
  9    62 ms    59 ms    60 ms  so-0-0-0.dub10.ip.tiscali.net [89.149.187.38]
 10    53 ms    54 ms    44 ms  rte-dub1.ip.tiscali.net [213.200.67.50]
 11    47 ms    49 ms    59 ms  89.207.56.94

Trace complete.

C:\Documents and Settings\User>tracert www.theregister.co.uk

Tracing route to www.theregister.co.uk [212.100.234.54]
over a maximum of 30 hops:

  1     1 ms    <1 ms    <1 ms  192.168.1.1
  2    19 ms    14 ms    14 ms  unalloc-80-249-253-1.net1.ie [80.249.253.1]
  3    17 ms    29 ms    19 ms  gw.screebogue2.net1.ie [80.249.248.33]
  4    24 ms    29 ms    28 ms  core-rtr.orielcore.net1.ie [80.249.249.193]
  5    39 ms    40 ms    66 ms  ge-2-2-0-2421.ycr1.dub.cw.net [208.175.245.49]
  6    41 ms    46 ms    53 ms  so-5-0-0-bcr1.lnd.cw.net [166.63.163.141]
  7    52 ms    39 ms     *     pos1-0-16-ahr1.lo1.cw.net [166.63.161.190]
  8    39 ms    44 ms    50 ms  64.69.188.82
  9    88 ms   194 ms   201 ms  212.62.19.98
 10   150 ms   181 ms   202 ms  vlan903.core3.lon2.rackspace.com [83.138.150.17]

 11    47 ms    54 ms    63 ms  core3-core1.lon2.rackspace.net [83.138.150.160]

 12    41 ms    52 ms    49 ms  g1-2.aggr1a.lon.rackspace.net [83.138.138.67]
 13    45 ms    47 ms    44 ms  212.100.234.54

Trace complete.

I also have IRC open now, it will tell me if something disconnects

DMC

About 20 mins of downtime there before 6pm, all ok atm.

rav1410

DMC wrote: »

About 20 mins of downtime there before 6pm, all ok atm.

Same

Dellas

All ok now. Thanks for the updates and keep up the good service

Net1

Thanks everyone - We were only running with one peer for most of the day (C&W) - all ok from about 17:55

What heve we learned - cisco switches don't like RF !!! - Tracing to RTE should now be much better !

racing route to www.rte.ie [89.207.56.94]

over a maximum of 30 hops:

1 14 ms 7 ms 8 ms gw.dundalk1.net1.ie [80.249.243.1]

2 13 ms 9 ms 8 ms telecity-rtr.oriel-telecity.link.net1.ie [79.171.79.254]

3 9 ms 9 ms 8 ms deg-gw-1.rte.ie [193.242.111.42]

4 10 ms 13 ms 14 ms 89.207.56.94



Trace complete.

Net1

Could any users give us some feed back on the Dublin server result @ speedtest.net

Regards

Falconire

Here are 3 speed tests from Slane area fro one of our customers

DMC

As requested, it hasn't missed a beat since.

C:\Documents and Settings\User>tracert www.rte.ie

Tracing route to www.rte.ie [89.207.56.94]
over a maximum of 30 hops:

  1     1 ms     1 ms     1 ms  192.168.1.1
  2    11 ms     9 ms    10 ms  unalloc-80-249-253-1.net1.ie [80.249.253.1]
  3    32 ms    11 ms    11 ms  gw.screebogue2.net1.ie [80.249.248.33]
  4    18 ms    18 ms    25 ms  core-rtr.orielcore.net1.ie [80.249.249.193]
  5    29 ms    21 ms    39 ms  telecity-rtr.oriel-telecity.link.net1.ie [79.171.79.254]
  6    23 ms    37 ms    27 ms  deg-gw-1.rte.ie [193.242.111.42]
  7    33 ms    38 ms    31 ms  89.207.56.94

Trace complete.