Preserved Topic: search engine software

We're running IIS 4.0 at my company and I'm not to thrilled with it's ability to search for documents on our Intranet. Does anyone know how to tweak it perform better searches?

For instance is there a way to automatically prohibit searching the "_vti" or rather hidden folders that Frontpage creates? (don't flame me, it wasn't my choice to allow Frontpage to be used here)

Here is a typical reply. I searched for 'pcb':

<BLOCKQUOTE><FONT face="Verdana, Arial">quote:</font><HR>1. index.html
Abstract: vti_encoding:SR

Wow, this seems like a little bit of a choir. Well tweaking it... you mean when they preform a search via the microsoft search system? The one coded by the devil? Or are you looking for a search over a webpage?

I think that you could achomplish great things with Perl/ASP/PHP could be faster? I don't know for sure. Perl would be my choice, it works well for things like that. You would need to do a search. Simply pattern match for the meta characters. That should not be too much hassel, simple search for and extract the keywords. Search would be more difficult, I am not sure on the exact way to do it, but it should be as easy as openning and looking for specific file names. It would get trouble some if you need to do thing differently.

One solution I would have would be to connect perl to a database. The database would contain the location of all files as well at the meta information. At the end of each night a perl script would re-index all of the information. This way it stays semi easy. When you index you input to the database the file name, its location, and its meta information.

This doesn't appear to be a really easy task. I don't know about tweaking anything M$... That seems like a bitch of a task to me.

-mage-

at work here we have a sloppy intranet ...I would like to be able to create an engine that scans for words in the html (especially meta tags)

We use SWISH-E, which is free. The prog builds an index of documents (very efficiently) that your queryies run against. On our sloppy corporate intranet, I run it every day @ 05:00. Takes around 15 seconds to index about a megabyte of text (on crap hardware).

You'll need a form that can query the index. The scripts link on the above page will get you started. I took one of the ones listed and rewrote it in PHP, and use that (around 25 lines of PHP, not too complicated).

And its F R E E.

(There's also a swish 2 and a swish++ out there, but I haven't used them and won't comment. swish-e works great for me. swish++ can index word documents too, which might be appealing to you.)

There's also mnoGoSearch