dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
2436

guppy_fish
Premium Member
join:2003-12-09
Palm Harbor, FL

1 recommendation

guppy_fish

Premium Member

BBR now requires Referer Hppt header or no soup for you!!

FYI

»Images in Firefox not displaying?

Justin, the site owner started out claiming no changes were made, turns out he did making changes that thumbnails and attachments won't display now. He made the change as he said other sites were leeching images from BBR

I did a little checking and forcing users to have Referer HTTP header enabled seems to open those up to a number of vulnerabilities which while on BBR might not mater, but its not safe for general browsing.

This seemed like a good topic to bring up in this forum ...

So, what do the experts have to say?
TheWiseGuy
Dog And Butterfly
MVM
join:2002-07-04
East Stroudsburg, PA

1 recommendation

TheWiseGuy

MVM

The page you linked to in the other thread was about security problems that a web site would need to worry about in the referer sent from a browser.

My understanding is the main concern with a referer is you send a site info about where you were if you click on a link to get to the site. Obviously this can lead to some tracking and information being given out. If you are at a site and concerned about a destination site knowing you clicked a link on the previous site to get reach the destination site, you can copy and paste the link.

Probably the best solution if you are concerned about your privacy is to use a add on that allows you to default to no referer but whitelist dslreports. Of course that would still allow a destination site to know you came from dslreports, if you clicked a link on the dslreports site but you could of course get into te habit of copy and pasting links on dslreports.

sivran
Vive Vivaldi
Premium Member
join:2003-09-15
Irving, TX

sivran to guppy_fish

Premium Member

to guppy_fish
As TheWiseGuy noted I don't think there are any client-side worries about Referer aside from a minor privacy issue. Privoxy or Proxomitron can easily take care of it.

antdude
Matrix Ant
Premium Member
join:2001-03-25
US

antdude to guppy_fish

Premium Member

to guppy_fish
I always disable referers unless the web sites require it. I mentioned them in here years ago: »Do you block Web browsers' referrers? and »Some embedded YouTube videos require referrers? ...
antdude

antdude to TheWiseGuy

Premium Member

to TheWiseGuy
said by TheWiseGuy:

The page you linked to in the other thread was about security problems that a web site would need to worry about in the referer sent from a browser.

My understanding is the main concern with a referer is you send a site info about where you were if you click on a link to get to the site. Obviously this can lead to some tracking and information being given out. If you are at a site and concerned about a destination site knowing you clicked a link on the previous site to get reach the destination site, you can copy and paste the link.

Probably the best solution if you are concerned about your privacy is to use a add on that allows you to default to no referer but whitelist dslreports. Of course that would still allow a destination site to know you came from dslreports, if you clicked a link on the dslreports site but you could of course get into te habit of copy and pasting links on dslreports.

Is there a white list addon in Mozilla's web browsers like SeaMonkey?
TheWiseGuy
Dog And Butterfly
MVM
join:2002-07-04
East Stroudsburg, PA

3 recommendations

TheWiseGuy

MVM

said by antdude:

Is there a white list addon in Mozilla's web browsers like SeaMonkey?

:)
First thing I checked before I initially posted

»addons.mozilla.org/en-us ··· ver=23.0

It appears Refcontrol does the job. Seems to get a reasonable rating.

»addons.mozilla.org/en-us ··· c=search

antdude
Matrix Ant
Premium Member
join:2001-03-25
US

2 edits

antdude

Premium Member

said by TheWiseGuy:

said by antdude:

Is there a white list addon in Mozilla's web browsers like SeaMonkey?

:)
First thing I checked before I initially posted

»addons.mozilla.org/en-us ··· ver=23.0

It appears Refcontrol does the job. Seems to get a reasonable rating.

»addons.mozilla.org/en-us ··· c=search

I don't use Firefox. I use SeaMonkey: »addons.mozilla.org/en-us ··· ver=2.20 and »addons.mozilla.org/en-us ··· c=search (still not supported).

Cartel
Intel inside Your sensitive data outside
Premium Member
join:2006-09-13
Chilliwack, BC

Cartel to guppy_fish

Premium Member

to guppy_fish
»[FireFox] VirusTotal not working with Firefox

Virus total wont work either.

Thanks for the heads up on ref control...works great
TheWiseGuy
Dog And Butterfly
MVM
join:2002-07-04
East Stroudsburg, PA

1 recommendation

TheWiseGuy to antdude

MVM

to antdude
Sorry, I misunderstood. It appears it does not work with Seamonkey 2.2 only with 2.0.x

Rocky67
Pencil Neck Geek
Premium Member
join:2005-01-13
Orange, CA

1 recommendation

Rocky67 to guppy_fish

Premium Member

to guppy_fish
I've been using Change Referrer Button, »addons.mozilla.org/en-US ··· -button/, with FX for several years with good results.

antdude
Matrix Ant
Premium Member
join:2001-03-25
US

antdude

Premium Member

said by Rocky67:

I've been using Change Referrer Button, »addons.mozilla.org/en-US ··· -button/, with FX for several years with good results.

Prefbar can do that too which I use. It is just annoying to know when the web sites will require it and has no white listings.
19579823 (banned)
An Awesome Dude
join:2003-08-04

19579823 (banned) to guppy_fish

Member

to guppy_fish

 

quote:
Justin, the site owner started out claiming no changes were made, turns out he did making changes that thumbnails and attachments won't display now. He made the change as he said other sites were leeching images from BBR
Well I guess he is trying to save his bandwidth... I wondered why he did that!

Justin just made all his images check the referrer to see if you are viewing it ON THIS SITE right?


EDIT:

I guess not,i just viewed my avatar in MY BROWSER and it worked (I put this url in my address bar http://i.dslr.net/nav/w50/33/61/853361.gif)

justin
..needs sleep
Mod
join:1999-05-28
2031
Billion BiPAC 7800N
Apple AirPort Extreme (2011)

1 edit

5 recommendations

justin to guppy_fish

Mod

to guppy_fish

Re: BBR now requires Referer Hppt header or no soup for you!!

The referrer check is for post image attachments above a minimum size, not all images everywhere, and is done because robots do not obey robots.txt, and I am uncomfortable with every picture people post getting stored by google and bing image search engines (and hordes of others), and people like to hotlink stuff external to the site which is just rude.

If you are worried about sending referrer header lines then get some security software that has enough smarts to whitelist domains instead of just suppressing referrer headers in all cases.

The change isn't a nefarious trick to undermine your security, instead you're just seeing a consequence of altering the standard behavior of your browser.

If you didn't see the effect here, you might see the same issue at other sites. At different times dslreports.com has used referrer checks, it really depends on how toxic the environment outside the site is.

referrer checking works to improve your privacy and therefore security not decrease it. Unless of course you like every forum attached image appearing in google images, searchable for colors, faces and so on?
EdmundGerber
join:2010-01-04

1 recommendation

EdmundGerber to TheWiseGuy

Member

to TheWiseGuy
said by TheWiseGuy:

It appears Refcontrol does the job. Seems to get a reasonable rating.

Thanks for that. Just about everyday we have to find workarounds to deal with the increasing weirdness of this site.
Mele20
Premium Member
join:2001-06-05
Hilo, HI

1 recommendation

Mele20 to justin

Premium Member

to justin
The problem is that there is NO extension for SeaMonkey and I doubt there is for Opera 12.15. And IE 10 on Windows 8? An extension for it? Bah. RefControl only works on Fx.

I have always had Opera set to block referer header for all the years I've used it. Myabe there is an Opera extension and I'll look but Opera extensions, in general, suck just like the childish extensions for Chrome (one reason I stopped using Iron browser a year or more ago).

I actually like IE 10 on Windows 8 which continues to surprise me as I hated IE 7, thought IE 8 was ok but a not a browser I'd want to really use much. I couldn't install IE 9 on my XP computer so I don't know about it and was really surprised by IE 10 64 bit on Windows 8 Desktop. But I doubt there is any extension to white list this site for referrer.

As for SeaMonkey, I use it a fair amount but I suppose not here anymore. Plus, I keep Opera 12.15 running in the taskbar all the time and I use it here a a fair amount, but I can't now (unless I can find an extension to white list this site). There may be one for Opera 15 but I will never upgrade to that abomination which bears no resemblance to the Opera we loyal users have loved since its inception.

I don't upload photos of myself and I no longer have friends who don't understand (and refuse to learn) how to protect MY privacy on the net even if they don't care about their own. The internet has proven to be the greatest leveler I have encountered in my entire lifetime as to who you can trust and who you can count as a true friend. So, while I laud your concern about google, etc having forum images in their images pages and searchable for faces it doesn't affect me. I also no longer communicate (except when absolutely necessary and that is extremely seldom) with friends, relatives, etc by email because they can't trusted with my privacy as they are all ignorant about privacy on the net and refuse to learn.

sivran
Vive Vivaldi
Premium Member
join:2003-09-15
Irving, TX

sivran

Premium Member

I thought you ran Proxo. What do you need a browser add-on for? Referer whitelisting should be trivial with Proxo or Privoxy.

Trihexagonal5
join:2004-08-29
US

Trihexagonal5 to guppy_fish

Member

to guppy_fish
I was wondering why I couldn't see screenshots in the UNIX forum thread with Firefox. I tried it with Seamonkey this morning and could see them so I knew something was up.

I'm using the Change Referrer Button extension with Firefox, if I switch it to the #2 setting I can see the screenshots.

therube
join:2004-11-11
Randallstown, MD

2 recommendations

therube to Mele20

Member

to Mele20
RefControl 0.8.16 can & does work in SeaMonkey.

It may not work "out of the box", but with a change to it's install.rdf, bumping the max version number, or by using other work-arounds, it will work.

From install.rdf:

<em:targetApplication>
<Description>
<em:id>{92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a}</em:id>
<em:minVersion>2.0</em:minVersion>
<em:maxVersion>2.29.*</em:maxVersion>
</Description>
</em:targetApplication>
 

Tested on the image on this page, »[Seamonkey] Security Exception window missing on 2.20 on XP Pro
System

to guppy_fish

Anon

to guppy_fish
This topic has been un-stickied by Wildcatboy See Profile
Mele20
Premium Member
join:2001-06-05
Hilo, HI

Mele20 to therube

Premium Member

to therube

Re: BBR now requires Referer Hppt header or no soup for you!!

It doesn't matter as it doesn't work as advertised in Fx so I won't bother fixing it to work on SeaMonkey.

It forces me to first go to a site and THEN APPLY "Block" as the option. It refused to apply BLOCK as default to all sites not listed in Options. So, it is a worthless piece of junk. Maybe it only works on Fx 24?
85160670 (banned)
"If U know neither the enemy nor yoursel
join:2013-09-17
Edmonton, AB

85160670 (banned)

Member

Click for full size
"IF" I use IE10 just with per-session cookies & PCFlank result is positive

norwegian
Premium Member
join:2005-02-15
Outback

norwegian to justin

Premium Member

to justin
At first I thought you needed a pat on the back for your comments, and you and the team do good work for us all to visit.

Bear with me though, and don't take it the wrong way:
"Stop Google and Bing and other image search engines from storing images etc?
Sounds good, but to the fact you utilize Google and others to help sponsor the site, it's it a touch controversial?
Wouldn't you be questioned on that and the contracts you have with your sponsors.

I applaud the work you all do for us as I've mentioned, but no other site is affected by referrer blocks until now for me, so what bought on this moral dilemma all of a sudden you feel obliged to think this is better for us?

Like I said, don't take it the wrong way.

justin
..needs sleep
Mod
join:1999-05-28
2031
Billion BiPAC 7800N
Apple AirPort Extreme (2011)

1 recommendation

justin

Mod

with no referrer checks bots sweep in and vacuum up all images for their silly media searches copyright searches or just archiving everything forever.

so I prefer that doesn't happen and since the bots are stupid, they do not correctly match their attachment fetching with the topics they are fetching: their referrer lines are empty.

google shows ads to anon visitors yes (although I imagine anyone who worries about referrer lines also adblocks everything which actually as a 100% ad supported site isn't doing me any favors anyway) but there is no requirement to let them Hoover up all media on the site in exchange for this service.
19579823 (banned)
An Awesome Dude
join:2003-08-04

19579823 (banned) to guppy_fish

Member

to guppy_fish

 

No there isnt mate!

Those stupid bots are BANDWIDTH WASTERS and they are quite intrusive!!!! (Its good to try and stop them) -- One way of doing that is by MAKING ALL THE BASES MEMBERS ONLY! (All links members only) Maybe members wouldnt have problems then,seems like some are now..

justin
..needs sleep
Mod
join:1999-05-28
2031
Billion BiPAC 7800N
Apple AirPort Extreme (2011)

2 recommendations

justin to norwegian

Mod

to norwegian

Re: BBR now requires Referer Hppt header or no soup for you!!

norwegian See Profile because you asked clearly and accurately, rather than berating the policy in a knee jerk reaction, I spent some time thinking about whether I can serve both the privacy/anti-bot rules that I want, while still letting people run with no referrer lines if they really must.

The first way would be to authenticate all /r0/download URLs but that url is built for speed and I don't want to add any weight to it just for that reason. However what I can do is look in the cookie line for a dslr cookies that would indicate you're logged in, and exempt those requests from getting checked.

So that is what I did. Of course the first time a bot works out that you can squeeze more juice out of screen scraping sites by becoming a member, I'll have to roll this back

norwegian
Premium Member
join:2005-02-15
Outback

norwegian

Premium Member


Thank you for looking more into it, and on your first point relative to speed, this site has always been one of the fastest loading sites, and I applaud you for trying to keep it that way.

On bots creating accounts and logging in, I know that must be a real pain in the rear.
I wish there was a more manageable way for the Internet on a whole to handle those, or someone came up with the software to run parallel or as a plug-in to really assist webmasters.
We can dream - "one day".

On your modification it did work when using RefControl in Firefox, and I thank you for the change.
However the tool does have an option for Forge, (send the root of this site).
That also works when not logged in - and I can live with that if the mod does not work for you and the site long term.

Fingers crossed and being a member it will work fine, while still allowing anon posters the benefits too, in hopes they aren't deterred from being a member long term and keeping your site up.
(If they are anon and blocking referrers, that is their own concern).
Hopefully that doesn't sound too pompous either.

pike
Premium Member
join:2001-02-01
Washington, DC

pike to justin

Premium Member

to justin
said by justin:

Of course the first time a bot works out that you can squeeze more juice out of screen scraping sites by becoming a member, I'll have to roll this back

This doesn't affect me but.. out of curiosity, how difficult would it be to add the member's user "level" to the cookie data, and only exempt users with a level of 1 or greater from this check? I would think this would exclude bots because they don't post or contribute anything that would elevate their level.

justin
..needs sleep
Mod
join:1999-05-28
2031

justin

Mod

possible but I prefer to wait till a problem is actually imminent before adding those type of things.
19579823 (banned)
An Awesome Dude
join:2003-08-04

19579823 (banned) to justin

Member

to justin

 

quote:
...Of course the first time a bot works out that you can squeeze more juice out of screen scraping sites by becoming a member, I'll have to roll this back
Well lets hope they dont mate -- We know you dont like putting these restrictions on...

So far all these years BOTS DONT REGISTER on any site,so it shouldnt be a problem
BlitzenZeus
Burnt Out Cynic
Premium Member
join:2000-01-13

BlitzenZeus

Premium Member

If there is a human behind them they can, and do on boards that don't allow anonymous posting. You might not see them, but there is spam from registered new accounts which gets quickly cleaned up by users flagging it, and mods deleting them. They are just gone, no moved message, not just closed leaving the spam behind, gone.

All it takes is a throw away e-mail address, and the time to find a username not in use. I've never really followed up on it, but I flag the topic, along with the profile for spamming. You can thank the mods for not even seeing most of the spam on this site, and the insomniac members who catch it.