dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
2
share rss forum feed

JPedroT

join:2005-02-18
kudos:1
reply to FirebirdTN

Re: Block web crawlers?

You create the robots.txt file with a text editor and upload it to your site, just like any other file.

Its not hard, it will at least stop your stuff from being indexed in the normal search engines.
--
"Perl is executable line noise, Python is executable pseudo-code."