Topic awaiting preservation: Catching visits from search engine hits (referrer question) (Page 1 of 1)

Sorry for the possibly confusing post title, but I tried to make it as descriptive as possible. Sorry also if this has been asked before, but I did a search and couldn't find what I was looking for.

Anyway, what I'd like to do is this: I'd like to determine if people are arriving at my site from a search engine after searching for a particular word. In my case, this particular word is "liminality" (which also happens to be the name of my site). Now, in the case of Google I know I can check to see if the referrer is "http://www.google.com/search?query=liminality", which would indicate a search on the single term "liminality." Is there a way, though, to check to see if "liminality" is part of a multi-term search? Like checking for "what is liminality?" or "liminality help" etc.?

OK, bear with me... here's where I'm going to get a little crazy. Is there a way to catch any and all search engine hit on "liminality" or do I have to go through each search engine that I want to check for?

I ask these questions because I get over a hundred hits a week from searches on "liminality," and I occasionally get an e-mail from someone telling me that they find my site while searching for the definition of liminality. What I'd like to do is catch these hits and offer the visitor information on "liminality" hopefully tailored to what they are looking for. The idea is that I will give them what they are looking for and show them how it ties in to my site, (hopefully) getting them interested in seeing more. After all, for every visitor that stays to look around, I've most likely got a hundred or so who say, "Ugh, personal site, next" and just skip back to the search engine. I don't want to limit this to Google because eight of the top ten search engines (Google, Yahoo, AOL, MSN, Lycos, LookSmart, AltaVista, AlltheWeb) list my site in the #1 spot for "liminality."

So, is this a JS question, or should I be thinking server side. I notice that my logs are able to pick out search engine hits on liminality without regard for the particular engine, so maybe there's a server side solution?

I really have no idea how to do this, but I do know the basics about referrers, so I thought I'd toss the question in here. Any and all help will be appreciated.

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

OK then, server side it is (I almost forgot I can move threads... I think this is the first time I've ever done so).

InI: I must admit that I am still a bit in the dark here. I understand what you are saying, but I'm not sure how I would go about doing this. Any tips or resources I might be able to look at? I really wouldn't even know where to start. I'll try to Google for some stuff, but I'm not even sure what to search for...

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

This reminds me of something I've come across a few months ago at the-emperor.org, a PHP script that monitors Google rankings. Let me fish for the exact link..... here:

http://www.the-emperor.org/wiki/u_ranker/

From the little info given on the page, it seems to work very differently from the way InI suggests and is possibly quite a bit more complicated, but if you ask the guy behind the project he might prove cooperative and grant you a look at the source code...


[edit: this doesn't cover anything but Google, though, and if you're looking for something more general (as I just saw in your initial post...), InI's most probably is the way to go. But U Ranker might be worth a look, anyway, I guess ]

(Edited by kuckus on 12-30-2004 23:25)

OK, that's really odd... I am positive I replied to InI's last post, but now it's not there. Whatever.

Long story short, thank you InI (and kuckus) for the advice. I've been messing around with the ideas you through out (I was actually thinking along the same lines, but wasn't sure how to snag the referrer... turns out to be a rather simple process). I'll post more when I make progress.

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

(Edited by Suho1004 on 12-30-2004 23:56)

For a former client I wrote a PHP code snippet several months ago to track the keywords of AFAIR Google, Yahoo, MSN, Altavista, Hotbot and probably Excite too. As soon as I'll be back home, that is around Jan. 2, I could post it here.

Actually it's fairly easy, but boring to do as you have to try several searches on all the search engines you want to monitor and take care of their local variants. My script may not fully support all variants of Google

I'm already well on my way to a simple script, and I'm covering as many search engines as possible. I don't know if the path I'm currently on is the best way to do it, though, and I'd be interest in seeing your script, poi, just to see how you approached the problem.

I've got a bunch of other things to take care of before I finish and test the script, so it may be a bit before I post anything.

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

IIRC I split the referer, extract the domain name then look if it matches the name/url of a search engine and finally extract the correct parameter.
My script also handles the researches done via Google's Directories and returns the path.

I have a working script now, but I haven't implemented it yet--I have to write the actual content I'm going to be using now that the coding is done.

What I did was just search the refer(r)er string for common search page names (like "search," "web," "results," etc., followed by ?) and then search for "liminality." With only five page names I've managed to cover at least ten search engines, most likely more.

Something that I'm wondering about, though... do you think I could just get away with search for a question mark in the referrer? I mean, is it possible to get a referrer with a question mark and the term "liminality" in it that isn't a search engine result? Wait, let me rephrase that: I'm sure it's possible, but how likely do you think it is? Since this script is going to be run every time a request is made for any page on my site, the simpler it is the better.

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

I suppose that if you get a referer with the term "liminality" in the query string, there's all the chances that it's a stop Google whores script. You know ,the sort of script some bloggers use to vanish one of the goals of the comment spam.

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

I meant something like that. That is a little script redirecting to a page without boosting its PR in Google.

Thanks for the ideas, guys. Sorry about not getting back to you sooner on this... I'm still a little sketchy with my Asylum visits and posts, as you can tell.

InI, yes, that helps. That was pretty much what I was thinking, but I wanted to get other (and perhaps more knowledgeable) opinions. Thanks.

poi: Right, I see what you're saying. If I read this correctly, I would get a referrer with a question mark and "liminality" in the query if a person followed a link to my site that was run through one of these scripts, right?

There are only a very few sites that I comment on (actually, only two that I can think of), and I don't think I've ever seen either turn up in my referrers, except on very rare occasion. I think what I'll do for now is go with the question mark/liminality combination and see how it goes. I still haven't written the actual custom content yet, so until that's done, this will remain theoretical. Once I get my act together I'll let you know so you can check it out.

Thanks again for the advice and the extra brainpower!

___________________________
Suho: www.liminality.org | Cell 270 | Sig Rotator | the Fellowship of Sup | "Hooray for linguistic idiots and yak milk!"

Sorry for not dropping in sooner - esp. as my name was mentioned

Seems like InI has it all in hand.

What I do is use the Google web services to grab the rank for that specific search term and store all the information in a database so I can plot changes over time, etc.

---------
If i was doing what you were thinking of I'd:

Grab the GET or POST VARS

Run it through php->parseurl()

Check the search engine (if I wanted to be fancy I'd check what engine it was and then I'd know what to look for)*

Grab the goodies out of the querystring using php->parsestr

--
Anyway just a few thoughts - I look forward to seeing what you come up with.

--------
* Check your referers - it is likely that the number of engines is relatively limited and most of them draw their results from similar engines so it shouldn't take much tweaking to get good coverage.

___________________
Emps

The Emperor dot org | Justice for Pat Richard | FAQs: Emperor | Site Reviews | Reception Room

if I went 'round saying I was an Emperor just because some moistened bint had lobbed a scimitar at me, they'd put me away!