Preserved Topic: spiders or spyders (Page 1 of 1)

ok, I'm getting ready to setup my website at its new home at runningwolf. What i'm wanting to know is if anyone knows if google and the major search engines still use bots to crawl a website. I'm going to use php/mysql to create links from table entries of text documents. Since I won't actually have a file named page1.html, I'm wondering how to get my pages listed on the various search engines. At one time, the bots would follow links in a file and report these links back to the search engine. So, how do I get a spider to crawl my stuff?

Thanks

spiders do indeed still crawl and I'm interested in the answers to a few of your questions myself actually. I've set up a search engine that crawls a site and have yet to check the effectiveness of it on a databases PHP system but it's next on the list.

spiders historically do not do well with the scenario you're laying out here but they are an ever-changing thing and they don't let on so.... hopefully someone more up_in_it_than_me can let us know

Jason

well, the probably most effective solution is to use a
htaccess file to 'redirect' your pages.
So when the spider accesse yourdomain.com/something/1
it actually get's something/page.php?id=1. Of course all the links have to be made this way.

That concept has been working very good for the Emperor
Maniac (V) Mad Scientist with Finglongers

From: Cell 53, East Wing
Insane since: Jul 2001

And they are running nicely at the GN too Have a look at the links section and the forums for examples of this.

Some search engines (like Google) will now index a dynamic page and possibly the next level down but won't index everything. Use mod_rewrite to make the links more search engine friendly.

___________________
Emps

FAQs: Emperor