New security measures on links

Goodshape

Boards.ie notice wrote:

This is all managed by Google, so it's not a list we have direct access to (and therefore it's not like we're going to be blacklisting sites on it ourselves or anything Orwellian like that).

So the fact that the blacklisting is being outsourced to the massive multinational corporation that you've got no control over... makes it less Orwellian?!

I'm not protesting really, just thought the choice of words there was amusingly unfortunate.

Biggins

I'm just glad we're not China where a lot more links might be actually blocked without question - never mind getting warnings about sites!

Tallon

Wait till sopa kicks in...

tricky D

Copy Link Location function now broken. Can't even just strip out this: http://www.boards.ie/out?f=7&url= bit as the chars in the target URL have been hexed - why??? Great, just great.:mad:

Boards.ie: Paddy

tricky D wrote: »

Copy Link Location function now broken. Can't even just strip out this: http://www.boards.ie/out?f=7&url= bit as the chars in the target URL have been hexed - why??? Great, just great.:mad:

The URL has to be encoded so that it can be passed as part of another URL. If we don't do that we can't pass a URL as part of another URL.

If you want to decode what we encode you can use this:

http://urldecoder.net/

Past in the URL and hit decode and you'll get the plaintext version of the URL.

Alternatively you could just follow the link and then copy and paste the URL from the address bar.

Steve

Why do this in the first place though?

Surely if boards takes it upon itself to 'protect' users from harmful links then it also leaves itself open to the system failing and the subsequent fallout?

If it ain't broke etc..

Monotype

I wonder is there any monetary benefit for boards or google in this?

I'm sure google definitely wouldn't pass up the opportunity to gather whatever information they can from people using this service.

I think the mods have been doing an excellent service in filtering out bad links over the years. In the rare case that a bad link does show up, I would be far more confident placing my trust in their hands than google's.
Most of the time that someone posts something 'bad', it's due to copyright or perhaps explicit material - these would need individual moderation anyway.

Edit:

Steve wrote: »

Why do this in the first place though?

Surely if boards takes it upon itself to 'protect' users from harmful links then it also leaves itself open to the system failing and the subsequent fallout?

If it ain't broke etc..

I'm guessing it's the recent increase in the use of url shortener sites being used. I'd rather see every last one of these blocked than what's there now.

Boards.ie: Paddy

Steve wrote: »

Surely if boards takes it upon itself to 'protect' users from harmful links then it also leaves itself open to the system failing and the subsequent fallout?

If it ain't broke etc..

The system works in such a way that if google's safe browsing API is down we will just assume the link is safe and forward you on. So there is nothing that can break that is out of our control.

Boards.ie: Paddy

Monotype wrote: »

I wonder is there any monetary benefit for boards or google in this?

There is no monetary benefit to either party. No personally identifiable information is passed to google by us. Google also do not get any firm idea of how many people are visiting a link as lookups to the safe browsing API are cached on our side for 30 minutes.

Gordon

Steve wrote: »

If it ain't broke etc..

If it ain't broke.. don't try to improve anything?!

Steve

Boards.ie: Paddy wrote: »

The system works in such a way that if google's safe browsing API is down we will just assume the link is safe and forward you on. So there is nothing that can break that is out of our control.

My point exactly Paddy

User: I followed a link from boards and it broke my computer, you promised you'd protect me!!

Boards: Google safesearch was down so we let it through anyway...

If anything, if you want you protect both users and boards, assert a page with "you are now leaving boards.ie, anything you view from now on is at your own risk" as do many sites.
It's a bit lame though, we're all grown ups as far as the internet is concerned, and aside from the tech aspects - and more relative to site feedback, why should boards attempt to nanny us and leave itself open to failure of such?

Orion

I agree with the above. I think this is an added layer that is both unnecessary and opening boards to risk as Steve said. I don't need boards to protect me - that's what local AV/anti-spyware is for. Chrome even has this protection built in. I don't need any website to do it for me tbh.

Doesn't this qualify as a disguised link? Is this the reason for the change in that policy? /cynic

Steve

Monotype wrote: »

I'm guessing it's the recent increase in the use of url shortener sites being used. I'd rather see every last one of these blocked than what's there now.

I agree, that's worthy of discussion as to whether it should be actionable by mods sitewide. It's already banned in sigs.

Boards.ie: Danny

Orion wrote: »

I agree with the above. I think this is an added layer that is both unnecessary and opening boards to risk as Steve said. I don't need boards to protect me - that's what local AV/anti-spyware is for. Chrome even has this protection built in. I don't need any website to do it for me tbh.

That's all well and good for you but only 34% of our users are using Chrome. Added with the latest Firefox versions which do the same thing Chrome does (via the same method - the Google lookup) and there are still roughly 50% of our users who are not getting the benefits from browser-based checking.

Orion wrote: »

Doesn't this qualify as a disguised link? Is this the reason for the change in that policy? /cynic

Your cynicism can be put away on this one, the discussion on the removal of the rule banning disguised links in sigs has been going on for quite some time; long before this was even considered.

Steve, we're not promising a site is safe and neither is your browser. Do you think if Chrome couldn't look up a site it'd deny you access to it? Of course not, it lets you through anyway. If it flags a site with a warning, it's for being "potentially harmful" not definitely harmful. It isn't an exact science but it's a good warning system for potential malware sites for our users, and our guests. It's also not just trapping malware, it's trapping phishing sites too which many AVs overlook.

If you want to send a link to a friend, send them on the full URL - you don't have to be a boards user or strip out the URL or decode it, just copy and paste the full link.

Monotype wrote: »

I'm guessing it's the recent increase in the use of url shortener sites being used. I'd rather see every last one of these blocked than what's there now.

Or maybe, just maybe, we're actually trying to do something helpful for the community. I know, I'm shocked too :eek:

Orion

Boards.ie: Danny wrote: »

That's all well and good for you but only 34% of our users are using Chrome. Added with the latest Firefox versions which do the same thing Chrome does (via the same method - the Google lookup) and there are still roughly 50% of our users who are not getting the benefits from browser-based checking.

34%? Really? Didn't realise that Chrome had gone that high.

Not really the point tho. I don't expect a site to check these things for me. Common sense applies in browsing. I'd rather hover over a link and see if I want to go there than be forced to click on it to see if it's a valid link.

Boards.ie: Danny wrote: »

Your cynicism can be put away on this one, the discussion on the removal of the rule banning disguised links in sigs has been going on for quite some time; long before this was even considered.

Fairy nuff

Boards.ie: Danny wrote: »

If you want to send a link to a friend, send them on the full URL - you don't have to be a boards user or strip out the URL or decode it, just copy and paste the full link.

Not going to happen. There's no way I will paste a boards.ie link to a different site to anyone. I'd get the actual url and post that. And that's not cos I don't like/trust boards - I do obviously. But the principal applies. If I search on google I can right-click a link in the results page and copy/paste - it doesn't have google.com?f=999&url=blahblahblah - it has the actual link. Yet they can give a warning without that referral part.

Boards.ie: Danny wrote: »

Or maybe, just maybe, we're actually trying to do something helpful for the community. I know, I'm shocked too :eek:

Nobody's doubting the motives The path to hell though ...

Boards.ie: Danny

Orion wrote: »

34%? Really? Didn't realise that Chrome had gone that high.

Not really the point tho. I don't expect a site to check these things for me. Common sense applies in browsing. I'd rather hover over a link and see if I want to go there than be forced to click on it to see if it's a valid link.

So hover over a link and you can see the full URL - the difference is it's no longer in your status bar, but appears in the title popup instead. You do not need to decrypt a URL-encoded link to see where it will bring you, nor do you need to click it either.

Not going to happen. There's no way I will paste a boards.ie link to a different site to anyone. I'd get the actual url and post that. And that's not cos I don't like/trust boards - I do obviously. But the principal applies. If I search on google I can right-click a link in the results page and copy/paste - it doesn't have google.com?f=999&url=blahblahblah - it has the actual link. Yet they can give a warning without that referral part.

I've right clicked on a link in a search result just now (searched for "Help me") and clicked copy and this is the URL I get:

http://www.google.ie/url?sa=t&rct=j&q=help%20me&source=web&cd=3&ved=0CH0QFjAC&url=http%3A%2F%2Fhelpmehelpme.com%2F&ei=EQ0aT_LdIM66hAet1oDCDA&usg=AFQjCNFpf5fSIxwsppSTMxSlXKQV9mI6qg

Google do the same thing, and have done for years. There is a slight difference in that hovering over a link on Google results shows the URL in the status bar, but as soon as a click is registered the URL will change to one like the above. Either way, copying and pasting is the same for both sites - funky looking URLs.

Nobody's doubting the motives The path to hell though ...

I don't know about that, there was the suggestion we were making money, possibly farming out data to Google. I wonder what other wild ideas will appear here over time

Sarky

It feels terribly unnecessary to me, but then I've been internetting for years. Those kids will have to find out about hidden links and Tubgirl sooner or later.

Orion

Boards.ie: Danny wrote: »

So hover over a link and you can see the full URL - the difference is it's no longer in your status bar, but appears in the title popup instead. You do not need to decrypt a URL-encoded link to see where it will bring you, nor do you need to click it either.

I've ignored tooltips for so long it's second nature. They can be too easily manipulated as this, ironically, proves. I purely go on the status bar.

Boards.ie: Danny wrote: »

I've right clicked on a link in a search result just now (searched for "Help me") and clicked copy and this is the URL I get:

http://www.google.ie/url?sa=t&rct=j&q=help%20me&source=web&cd=3&ved=0CH0QFjAC&url=http%3A%2F%2Fhelpmehelpme.com%2F&ei=EQ0aT_LdIM66hAet1oDCDA&usg=AFQjCNFpf5fSIxwsppSTMxSlXKQV9mI6qg

Ok you got me - google was a bad example. But at least the actual end-link is in the status bar.

Boards.ie: Danny wrote: »

I don't know about that, there was the suggestion we were making money, possibly farming out data to Google. I wonder what other wild ideas will appear here over time

Nobody on this thread implied moneymaking as a cause. Even though it's obviously the reason :pac:

Orwellian was mentioned - and tbh not completely unreasonably. Google have been known to blacklist sites in a certain country at the behest of the Government of that country. Do No Evil only applies when it suits the market prospects. SOPA/PIPA being dumped is only a blip - they'll be back better, faster, stronger. And then can we still rely on Google's filtering?

Inbox

I keep getting 404 errors now when I click any link.

Boards.ie: Danny

Orion wrote: »

I've ignored tooltips for so long it's second nature. They can be too easily manipulated as this, ironically, proves. I purely go on the status bar.

As the Google example shows, it's not that hard to manipulate a status bar either.

Orion wrote: »

Nobody on this thread implied moneymaking as a cause. Even though it's obviously the reason :pac:

Monotype was wondering if there were monetary benefits for either party, I picked this up as an implication.

Orion wrote: »

Orwellian was mentioned - and tbh not completely unreasonably. Google have been known to blacklist sites in a certain country at the behest of the Government of that country. Do No Evil only applies when it suits the market prospects. SOPA/PIPA being dumped is only a blip - they'll be back better, faster, stronger. And then can we still rely on Google's filtering?

The Pirate Bay and other blacklisted sites from Google search results still show up as safe on the lookup API. Google aren't filtering anything, they are providing an advisory; users are free to ignore it if they want

Boards.ie: Danny

Inbox wrote: »

I keep getting 404 errors now when I click any link.

File a bug report in Site Development

Orion

Missed Monotypes post and Paddy's reply - must have been the righteous outrage. I think I'll just write an angry letter - you got an address?

I was actually talking about google's entry to China. The blacklisted sites so that they didn't even appear in the search results. Who's to say that they won't add sites to the unsafe filter for whatever reason. I'm a fan of google but they're not as pure driven snow as they want to appear.

Monotype

I was doubtful about the money aspect but I would still be curious about any advantages to google.

Boards.ie: Paddy wrote: »

No personally identifiable information is passed to google by us. Google also do not get any firm idea of how many people are visiting a link as lookups to the safe browsing API are cached on our side for 30 minutes.

So are you saying that after a new link is checked, it gets passed back to boards and then we get forwarded?
- As opposed to boards handing us over to google who would forward us on?

That doesn't sound so bad then, although the link changing is still a nuisance.

Boards.ie: Paddy

Monotype wrote: »

I was doubtful about the money aspect but I would still be curious about any advantages to google.

Google's motivation for offering this service is probably fairly simple. A safer web means more people using the web for more things, which inevitably means more money for google.

You can find more details on the service here:

http://code.google.com/apis/safebrowsing/

It's a publically available API, not some special deal between boards and google.

Monotype wrote: »

So are you saying that after a new link is checked, it gets passed back to boards and then we get forwarded?
- As opposed to boards handing us over to google who would forward us on?

That doesn't sound so bad then, although the link changing is still a nuisance.

The exact flow we use is as follows:

- You click on a link to an external site
- Your request first goes to our outbound link page
- We look the link up in our local cache to see if it's dangerous
- If we have no record in our cache we send a request to google's API from our servers to find out if the link is potentially dangerous.
- Based on what comes back from google we either send you on your way directly or display a warning message informing you that the link may be dangerous

At no time are you in direct communication with google. At no time does google know who you are. All google sees is a request coming from our IP using our API key.

I do understand the issue some people have with not being able to see the link in the status bar when they hover over a link. Having investigated google's own approach to this I see that they switch out the link with javascript when you click on it. You could say that this is more disingenuous than our approach as the link you see in your status bar is not the link you actually visit, however it is smoother from a user experience perspective.

We will investigate what's involved in implementing that type of approach however for the meantime you can take the tooltip text as indicative of the actual URL you will be taken to.

Monotype

Boards.ie: Paddy wrote: »

The exact flow we use is as follows:

- You click on a link to an external site
- Your request first goes to our outbound link page
- We look the link up in our local cache to see if it's dangerous
- If we have no record in our cache we send a request to google's API from our servers to find out if the link is potentially dangerous.
- Based on what comes back from google we either send you on your way directly or display a warning message informing you that the link may be dangerous

At no time are you in direct communication with google. At no time does google know who you are. All google sees is a request coming from our IP using our API key.

Sounds good. Thanks for clarifying.

tricky D

Gordon wrote: »

If it ain't broke.. don't try to improve anything?!

Unfortunately it is not an improvement. There are 3 things now broken for me, since this has been implemented :

I've been getting 404s on the filtered link where the real link is actually good.
The real link on this is also good but results in Flickr's homepage, which is not Flickr's 404 page, when filtered http://www.boards.ie/vbulletin/showpost.php?p=76732248&postcount=112
Right click functionality is broken which is a basic UI no no.

So it is not a case of improvement but a case of it is definitely broken now in 3 instances, so far.

Boards.ie: Danny

tricky D wrote: »

Unfortunately it is not an improvement. There are 3 things now broken for me, since this has been implemented :

I've been getting 404s on the filtered link where the real link is actually good.
The real link on this is also good but results in Flickr's homepage, which is not Flickr's 404 page, when filtered http://www.boards.ie/vbulletin/showpost.php?p=76732248&postcount=112
Right click functionality is broken which is a basic UI no no.

So it is not a case of improvement but a case of it is definitely broken now in 3 instances, so far.

1) You don't get a 404 when a link malware, you get it when something on our side is broke - i.e. it's a bug.
2) Once again, a bug.
3) Right click functionality is not broken, you can copy the link. Sure, it'll link to boards.ie which will redirect the user to that site, but I'm not going to rehash old ground (i.e. Google does the same thing).

We can't be everywhere, checking the millions of links on Boards.ie to see if they're working or not - we need users to report issues to Site Development so we can investigate and fix instead of coming in here and going negative because something doesn't work as expected.

tricky D

Boards.ie: Danny wrote: »

1) You don't get a 404 when a link malware, you get it when something on our side is broke - i.e. it's a bug.
2) Once again, a bug.
3) Right click functionality is not broken, you can copy the link. Sure, it'll link to boards.ie which will redirect the user to that site, but I'm not going to rehash old ground (i.e. Google does the same thing).

Pedantically it's not broken, but practially, it most certainly is. Just because Google do it, does not make it any less annoying, which is bad UIing.

Boards.ie: Danny

tricky D wrote: »

Pedantically it's not broken

I assure you, it is. If you don't get to the link but can via a copy-text and paste, it is broken and it's broken on our side. It is a bug.

I'm not going to debate you on the UI aspect, suffice to say you've not emailed Google relating to their poor UI choice.

tricky D

Boards.ie: Danny wrote: »

I assure you, it is. If you don't get to the link but can via a copy-text and paste, it is broken and it's broken on our side. It is a bug.

I'm not going to debate you on the UI aspect, suffice to say you've not emailed Google relating to their poor UI choice.

Apols, I didn't make it clear that I was talking about point 3.

There is no debate on the UI aspect, it is a very very basic no-no as it alters the behaviour which the user expects. That the all powerful Google choose to do it and whether I have got in touch with them is irrelevant to the matter at hand, on this site, Boards.ie.

Snowbat

Boards.ie: Danny wrote: »

That's all well and good for you but only 34% of our users are using Chrome. Added with the latest Firefox versions which do the same thing Chrome does (via the same method - the Google lookup) and there are still roughly 50% of our users who are not getting the benefits from browser-based checking.

Firefox has used Google Safe Browsing lookups since 3.0 (not just the latest versions) and Safari since 3.2. Microsoft and Opera have maintained their own similar lookup services since IE 7.0 and Opera 9.1. I really doubt vulnerable browser versions account for more than 5%, never mind 50% (and a fair chunk of those vulnerable browsers are likely protected by URL scanners in AV products anyway). Care to provide some stats including browser version numbers?

This system might have been useful a couple of years ago but not now.

It is creepy and it breaks the way most people check links before clicking.

Please reconsider.

Boards.ie: Paddy

We've taken all your feedback on board and you will now see the original URL in your status bar when you hover over a link. The link will still take you through our security check on the way out but it should be all but invisible to you.

We've also taken a look at the way we parse URLs and it should be much more compliant with the relevant RFCs. If you spot a link that doesn't resolve properly please let us know about it in the Site Development forum.

pickarooney

Boards.ie: Paddy wrote: »

We've taken all your feedback on board and you will now see the original URL in your status bar when you hover over a link. The link will still take you through our security check on the way out but it should be all but invisible to you.

We've also taken a look at the way we parse URLs and it should be much more compliant with the relevant RFCs. If you spot a link that doesn't resolve properly please let us know about it in the Site Development forum.

An excellent compromise in my view. Good work.