Preserved Topic: News-script or similar... (Page 1 of 1)

Hi gurus :-)
Need input/help pretty quick...

What I need:
A JSP or ASP function that visits select remote websites and collects all links that are new since yesterday. (I must be able to get at the source-code and modify it)

What I have seen: All those newsscripts that watches select newssites and sorts the new news under "sport", "tech", "politics"... and presents the links on your site.

What I can't do right now: Code it from scratch, or buy a service from a company that wont let me host the code.

I need to know if there are any freebies out there that are any good, alternativley, any scripts that you can buy and modify (it's for a big client, so there is a budget...). I don't know which language yet, but probably JSP/Java.

Greatful for any tips, even if it's just a tutorial.
Thnx for your time!
Dan

I don't even want to get involved with the legal ramifications of content stealing... so we will avoid that issue entirely. So I will assume you have partnerships formed with the other places for content sharing.

I think if you are going to be going with Java/JSP you will need to write it from scratch. If you want to do it in another language you will need to do it from scratch too.

The process is not a hard on, it is a simple act of find an replace. The find part which will tend to be a pain in the ass unless you have an XML version which is easily parsable.

I think the easiest way to do this would be to run it as a service on your machine, and then simply connect grab data parse data, and then write the data to the appropriate files.

I would recommend using XML and having your partners use XML as well, because this will make coding a bit easiers, as well as any future ventures you may want to take this to.

I think the service would be good if you are looking for speed in getting it online, but I would have to say that with the level of customization needed, it would probabally be just as good to write it yourself.

But then again Max or Linear might have some excellent links.

Thank you for your thoughts and tips warmage.
Just want to make it clear, this is in no way associated with content stealing, it's part of an admin function for a large legit job for a client that works with this stuff.

We are looking into it deeper now, but as it seems today, we have no possibillity to access raw data (no db-acess) from some of the sources that we need to check daily for updates, also, no way to change their current technology.
On our side we will be working with XML though.
Dan

From this point if there is no raw data to access then you would have to look for patterns.

You said java/jsp you could get the regular expression that follows Perl 5 reg expressions for that, they are seperate classes, and I believe they are provided open source, but it may require a quick google or hotbot search to find it. Most websites relating to java reg expressions will direct you to that site.

I am guessing there would be some simple way to find the news and parse it. But finding the patterns is the hard part.

If you could give us some example pages you would like parsed we could probabally pick something out for you.

I believe that you have to programming knowledge to undertake this (?) and I am not sure what level of help you might require, or if you are just looking for oppionions and / or links to ready made scripts.

Well, I'm not going to program this, I'm doing preliminary research at this point to find out what possibillities are available to use as a base to work with. Unfortunatley I can't show you any urls at the moment since we are in a bidding situation.
We'll see what comes out of this.
/Thnx, Dan

Get your content here

The scripting involves parsing the XML and outputting HTML formatted the way you like. I'd recommend coding it yourself also.

Nice tip linear!
I'll take a good look at that link.
Thanx!
Dan