Topic: Hiding a Website from Search Engines (Page 1 of 1)

has anyone experience with such a task?
i'm running a big online forum for students, and I need it do be as invisible as possible. any advices would be very much appreciated!

The Space Between Us | My Blog: lukas.grumet.at

robots.txt ?

not all search engines obey robots.

1. You can make it not reachable by providing a special link to access the forum (like do not use index pages)
2. Make it only viewable for registered users, this way search engines won't index the contents

--
Regards,
Elias

Make them use Firefox and a special Greasemonkey userscript. The userscript can use AJAX to retrieve a URL to a PHP script that creates a random key every day; the userscript then creates an iframe to the forum, passing that key through the query string -- and without the key, the forum won't load at all.

Probably either too much security or not enough, but I think it'd work fine, assuming that students don't distribute the Greasemonkey script. Perhaps it can also require password entry from students.

Or, if you're looking for a more cross-browser solution, do all that, but with a user-entered password instead of a userscript; the key-sending server script can then be set up to take that password.

----------------------
dA

(Edited by DavidJCobb on 03-27-2009 20:54)

Access to the forum needs to be protrected somehow : captcha, random keyword input, user login, http auth, ... you name it.

lallous: you're right about the the robots.txt. Also I skimmed through the original post and actually answered to the question posted in the title of this thread, instead of the more specific question ... although a robots.txt file could do the trick too ... provided the crawlers do follow the instructions.

DavidJCobb: Requiring people to use ONE browser is so last century. Please. We're finally out of the " This browser works best in IE " days. Let's not fall for that again.

quote:

---

poi said:

DavidJCobb: Requiring people to use ONE browser is so last century. Please. We're finally out of the " This browser works best in IE " days. Let's not fall for that again.

---

I realized that immediately after I posted, hence the edit I'd made:

quote:

---

DavidJCobb said:

Or, if you're looking for a more cross-browser solution, do all that, but with a user-entered password instead of a userscript; the key-sending server script can then be set up to take that password.

---

----------------------

dA