|reply to FirebirdTN |
Re: Block web crawlers?
You create the robots.txt file with a text editor and upload it to your site, just like any other file.
Its not hard, it will at least stop your stuff from being indexed in the normal search engines.
"Perl is executable line noise, Python is executable pseudo-code."