Topic: How can i stop google putting my site in the search engine?? Pages that link to <a href="https://ozoneasylum.com/backlink?for=22989" title="Pages that link to Topic: How can i stop google putting my site in the search engine??" rel="nofollow" >Topic: How can i stop google putting my site in the search engine??\

 
Author Thread
H][RO
Bipolar (III) Inmate

From: Australia
Insane since: Oct 2002

IP logged posted posted 08-19-2004 07:58 Edit Quote

Hi alll...

I have a domain that i use for testing sites... there is no site setup at the domain. mydomain.com.

when i test sites i create a subdomain for them so i can get to them easily. so site1.mydomain.com etc etc..

Now somehow google is finding these sites and putting them in the search engine!!! but i dont want them too... I only want them to put the proper location up when the size goes live.

For starters how is google even finding the pages? I have not submitted it to any search engines, and there are no links anywhere else pointing to these test sites , so how can google find it without any reference?

Secondly how can i stop it, i want to be able to have sites there and being ignored by google....

Thanks all..

tntcheats
Bipolar (III) Inmate

From: BC, Canada
Insane since: Jun 2004

IP logged posted posted 08-19-2004 09:28 Edit Quote

I believe you can put <meta name="robots" content="noindex,nofollow"> between your head tags and that'll keep almost all spiders from indexing your website.

Then you can go to http://services.google.com:8882/urlconsole/controller and remove each URL from the listings.

It could be finding the pages because you have gone to them with the Google Toolbar installed--I believe that this can happen, correct me if I'm wrong.

-- http://www.google.com/remove.html#exclude_pages

-----------------------------------------------------
funny websites | funny signatures | funny jokes

H][RO
Bipolar (III) Inmate

From: Australia
Insane since: Oct 2002

IP logged posted posted 08-19-2004 12:19 Edit Quote

Oh.. i do use the google toolbar, i didnt know it did that. It must monitor everywhere u go then?

tntcheats
Bipolar (III) Inmate

From: BC, Canada
Insane since: Jun 2004

IP logged posted posted 08-19-2004 12:40 Edit Quote

I'm not positive, but I believe it does.

However it's not really personal data that it stores, just the URL of websites. Because it tries to find information like PR on every website you see. (This is assuming you have the advanced features turned on)

-----------------------------------------------------
funny websites | funny signatures | funny jokes

HZR
Bipolar (III) Inmate

From: Cold Sweden
Insane since: Jul 2002

IP logged posted posted 08-19-2004 15:42 Edit Quote

The easiest way, if you have the ability to do it, is to create a robots.txt file in your server root with the following content

code:
User-agent: *
Disallow: /


This means that robots are forbidden to fetch anything from this domain.

H][RO
Bipolar (III) Inmate

From: Australia
Insane since: Oct 2002

IP logged posted posted 08-20-2004 00:46 Edit Quote

Yeh i have looked that stuff up, i was mainly wondering how they were finding the site.

Actually with that robot file, does that have to go in the root of each subdomain? Im not sure how search engines treat subdomains since they are normally just directories stored in the main domain..

tntcheats
Bipolar (III) Inmate

From: BC, Canada
Insane since: Jun 2004

IP logged posted posted 08-20-2004 01:12 Edit Quote

I would probably include it in each subdomain, just as an extra precaution.

-----------------------------------------------------
funny websites | funny signatures | funny jokes

Tyberius Prime
Paranoid (IV) Mad Scientist with Finglongers

From: Germany
Insane since: Sep 2001

IP logged posted posted 08-20-2004 02:28 Edit Quote

Each and every subdomain needs it's own robots.txt. All the spiders do is ask for
$currentDomain/robots.txt - how should it find the 'main' subdomain? And subdomains once were meant to actually point to different machines - that they point to different subdirectories is more or less an 'accident' of modern day webhosting.

so long,

->Tyberius Prime

H][RO
Bipolar (III) Inmate

From: Australia
Insane since: Oct 2002

IP logged posted posted 08-20-2004 07:31 Edit Quote

Yeh i thought that might be the case... thanks everyone!



Post Reply
 
Your User Name:
Your Password:
Login Options:
 
Your Text:
Loading...
Options:


« BackwardsOnwards »

Show Forum Drop Down Menu