Recent Bank of Ireland outage, no SSL Certificate

02-07-2023 1:56pm #1

Joe Public was seriously inconvenienced for about a day recently by the outage of Banking365, confronted with the message of no SSL Certificate. I think Joe Public deserves to know what might have happened.

For folk who no little or nothing I’ll put it simply that in order for safe encrypted transfers to take place between you, and the bank in this case, an SSL Certificate needs to be in place. For anyone curious, SSL stands for Secure Sockets Layer, and the secure bit is obvious to anyone. These need to be renewed periodically as things can change, this can be done through management software or a human being in the organisation. In any case a human would/should be overseeing that all this is in order.

For anyone interested encryption here works by a pair of keys, a public and private key, the pairing or handshake happens with every transaction you do through likes of Banking 365. Happens with your WhatsApp messages too, we are informed. Data cannot be seen without both keys, and the private key should be kept secret to the organisation’s system.

There are a number of possibilities of why a site, as happened with Banking365, displayed No SSL Cert. One is that the cert was let expire by lack of oversight, plain carelessness at a most basic level. Are we to believe BOI let this happen, by not having proper management of it in place on a software and human level? It’s possible.

Another possibility is that a human revoked the certified on an emergency basis because it got compromised, ie leaked out to a bad actor. This would be a serious matter with a lot of implications.

Third possibility is a malfunction of the software managing the certificate. A fourth possibility is someone purposefully revoked cert to take the site offline for sone reason unknown, like pulling the fuel cut off valve in an airline to stop the engine.

Anyone else have thoughts on this?

Crash · 02-07-2023 2:03pm

First one didn't stack up right? Browser will display expired cert error in that case, not no cert.

02-07-2023 2:06pm

https://www.boards.ie/discussion/comment/120795371#Comment_120795371

Exactly, none of scenario 1, 2 or 4 would have resulted in a No SSL Cert error. Someone needs to Joey Prickly Meteorology.

02-07-2023 2:09pm

https://www.boards.ie/discussion/comment/120795371#Comment_120795371

I can’t recall exactly what the message was, I had no urgent transactions to complete. I’ll see if I have a screenshot of it.

Crash · 02-07-2023 2:13pm

At a guess, considering what I've seen happen in the past is likely that it's very much an "infrastructure as code" release process, they probably had a release, for whatever reason there was some discrepancy around how staging and prod managed credential deployment (or they were identical, but a pre-step was missed) and they pushed out their production stack with a cert missing. Then scrambled to roll back their release process, or fix the release bug and redeploy.

I don't work in fintech, but regardless of whether you're banking or not, the end result is downed product for a chunk of time. Even in my industry it'd be a pretty unpleasant post mortem and awkward customer questions.

02-07-2023 2:19pm

Found the BOI thread. Another user left a screenshot of invalid SSL.

02-07-2023 2:22pm

https://www.boards.ie/discussion/comment/120795410#Comment_120795410

Fits the category of the third possibility, and most likely what happened.

L1011 · 06-07-2023 6:08pm

It wasn't "no" SSL cert, it was an expired SSL cert. There is a HUGE difference.

tnegun · 06-07-2023 7:01pm

Agree the way the opening post is written you'd think they operated over HTTP for a period. Did anyone actually look at the cert that was presented details? That would give a proper indication of what happend.

06-07-2023 10:17pm

https://www.boards.ie/discussion/comment/120814874#Comment_120814874

I was working from memory only, hadn’t the app to hand… all I knew was it an a ssl issue that was preventing people gaining access to bank app. I didn’t need to access my own account at the time but I was watching on Twitter tons of people not being able to make payments. It certainly was rather an inconvenience that a lot of people missed important payments.

lukin · 13-10-2024 2:27pm

Hi, I am getting the certificate invalid error message on

http://www.365online.com

LambshankRedemption · 13-10-2024 3:02pm

https://www.boards.ie/discussion/comment/122714786#Comment_122714786

365online.com is resolving to 107.162.173.100 which is based in the US.

Looks a DNS hack perhaps.

cython · 13-10-2024 3:21pm

https://www.boards.ie/discussion/comment/122714887#Comment_122714887

That address appears to be owned by F5 who provide enterprise load balancing solutions, etc. so I'd not assume a hack. Maybe a change gone awry over the weekend though. Website also indicates maintenance was to take place overnight. https://www.bankofireland.com/it-maintenance/

The cert issuance actually looks OK, in being a valid CA, and to a subdomain of 365online.com, and over 6 weeks old, but it may be a cert intended for internal use (glb could likely mean global load balancer) bubbling up to the user in error.

LambshankRedemption · 13-10-2024 3:27pm

https://www.boards.ie/discussion/comment/122714941#Comment_122714941

Yeah I know F5. Could well be a change gone wrong then.

lukin · 13-10-2024 3:35pm

Ridiculous that a supposedly reputable organisation can allow this to happen.

LambshankRedemption · 13-10-2024 3:43pm

https://www.boards.ie/discussion/comment/122714965#Comment_122714965

Just this past Friday I had someone hassling me on MS Teams to approve his Change Request. He was getting really worked up about it. I took great pleasure in telling him there is a Change Freeze on Fridays. The thinking being, if a change breaks anything, your staff are in work on the Friday to fix it. not hungover or still partying on Saturday morning.

I agree though. One SSL mistake as per a few months ago is unlucky. A second is negligence.