Topic: Robots and SEO and the such...

Wow, haven't been on here in quite a while...

Okay, so I'm once again doing the whole new site search engine submission etc. process. I have a few quick questions about "safe urls" and robots.txt....

Here's the site: www.bptparts.com

I'm using OsCommerce (an opensource cart system) for the whole thing. From what I know, it uses Session ID's in the urls. Is this a taboo thing for some engines or what? I mean, so far, for some keyword phrases I've managed first listing on Yahoo and MSN and Hotbot. I'm quite pleased about this. Thing is, I only seem to see the main page of the site showing up. I have even put in dynamic keyword creation on product specific pages to have other pages appear, yet I still don't see these pages acting as entry pages...

Is this a result of the url being composed the way it is? (take a look and see if you would...) And would I use robots.txt to direct search engines to other pages?

yeah, you're kinda screwed for Search Engine friendliness there. Search Engines, for the most part, index pages like "index.php" and "product_info.php" but almost all of the pages on your site are assembled from database in the disguise of "product_info.php+a_buncha_foo_to_id_product_and_build_page" and that's not a good thing.

Something you could do relatively easily to have a bunch of keywords indexed is to hardcode some links into index.php somewhere that link to:

• http://bptparts.com/cart/allprods.php
• http://bptparts.com/cart/allprods.php?&page=2
• http://bptparts.com/cart/allprods.php?&page=3

those should be easy to follow (I think) if they are hardcoded into an indexed page like index.php...

<!-- information //-->

<!--snipped a buncha crap //-->

    <td class="boxText">

    <a href="http://bptparts.com/cart/shipping.php?osCsid=260abf89b828f31eb47f47d4a51e3be2">Shipping & Returns</a>

    <br>

    <a href="http://bptparts.com/cart/privacy.php?osCsid=260abf89b828f31eb47f47d4a51e3be2">Privacy Notice</a>

    <br>

    <a href="http://bptparts.com/cart/conditions.php?osCsid=260abf89b828f31eb47f47d4a51e3be2">Conditions of Use</a>

    <br>

    <a href="http://bptparts.com/cart/contact_us.php?osCsid=260abf89b828f31eb47f47d4a51e3be2">Contact Us</a>

    <br>

    <a href="http://bptparts.com/cart/allprods.php?osCsid=260abf89b828f31eb47f47d4a51e3be2">View All Products</a>

    {{{INSERT LINKS HEREABOUTS MAYBE?}}}

    </td>

<!--snipped more crap //-->

It's a start anyhow



edit: code was all on one line

[This message has been edited by JKMabry (edited 02-25-2004).]

grats42: I think you are going to struggle. Really a session ID only needs to be generated if the user is logged in and doesn't accept cookies and even if search engines accept query strings (or fully index sites with query stirngs) they might think it is a different page (one of the reasons that search engines tend not to index dyanmic URLs).

You might be able to clean up the URL using htaccess and then pass the session ID through the query string but I'm not sure how easy it would be to edit osCommerce to work properly with these URLs (i.e. getting it to produce them all the time, etc.).

___________________
Emps

The Emperor dot org

Yeah, I hear what you're both saying. That's exactly my issue.

On the session Id/url thing - session ID's are only used if they need to be logged in and don't accept cookies? So, if I tell OsCommerce to use cookies instead (which I think I can do) that may help take care of that whole issue.

Okay, actually, I just went and did precisely that and now I'm getting urls that look more like this:
http://bptparts.com/cart/product_info.php?cPath=21&products_id=76

http://bptparts.com/cart/index.php?cPath=25

Hopefully that will help out with my problem?? I've found different responses to this question on whether or not the major search engines will deal with these as separate pages or not. They all have different titles and content, its just whether or not they actually choose to spider these links, as you say...





[This message has been edited by grats42 (edited 02-25-2004).]