Closed Thread Icon

Topic awaiting preservation: performance issues using nusoap with repeated calls Pages that link to <a href="https://ozoneasylum.com/backlink?for=26440" title="Pages that link to Topic awaiting preservation: performance issues using nusoap with repeated calls" rel="nofollow" >Topic awaiting preservation: performance issues using nusoap with repeated calls\

 
Author Thread
Pugzly
Paranoid (IV) Inmate

From: 127.0.0.1
Insane since: Apr 2000

posted posted 08-13-2005 00:36

Greets

I'm using nusoap to convert addresses to latitude and longitude coordinates via http://www.geocode.us. I'm basically just running through a loop, doing the lookup, then dumping the coordinates to my db.

When I get the first lookup, it finishes fairly quickly (under .025 seconds). But subsequent lookups take about 15 seconds for EACH lookup. Curious to know if anyone has used nusoap before and had this issue, of if you notice an issue with my code:


code:
include($_SERVER['DOCUMENT_ROOT'] . '/i/constants.php');
startTimer();
include('nusoap.php');
set_time_limit(0); // just for testing
$records = 0;
$soapclient = new soapclient("http://geocoder.us/service/soap");

$info = mysql_query("select * from psor_offenders where addressLat = '' and addressLong = ''");
while ($rows=mysql_fetch_array($info)){
	$mapData = '';
	$address = $rows[address] . " " . $rows[city] . " " . $rows[state] . " " . $rows[zip];
	// echo "address assembly at " . showtimer() . "<br />\n";
	$address2check = array('geocode'=> $address);
	// echo "address check at " . showtimer() . "<br />\n";
	$mapData = $soapclient->call("geocode", $address2check, "", "");
	// echo "mapData at " . showtimer() . "<br />\n";
	$sql = "update psor_offenders set addressLat = '" . $mapData[0]['lat'] . "', addressLong = '" . $mapData[0]['long'] . "' where id = '$rows[id]'";
	// echo "SQL query string assembled at " . showtimer() . " ($sql)<br />\n";
	mysql_query($sql);
	// echo "SQL query finished at " . showtimer() . "<br />\n";
	// echo "ID " . $rows[id] . " is done at " . showtimer() . "<br /><br />\n";
	$records++;
	flush();
}

I've got several thousand lookups to do initially, plus sporadic lookups daily. I'd prefer it not take the rest of my life to do the lookups!  

stopTimer();
echo $totaltime . " seconds to assemble $records records\n";



norm
Paranoid (IV) Inmate

From: [s]underwater[/s] under-snow in Juneau
Insane since: Sep 2002

posted posted 08-13-2005 01:34

I bet it has nothing to do with nuSoap and everything to do with geocoder's soap server. Head over to xmethods and try something that loops through a series of calls to one of their services and see if you have the same problem.

bitdamaged
Maniac (V) Mad Scientist

From: 100101010011 <-- right about here
Insane since: Mar 2000

posted posted 08-13-2005 06:02

hmm.. I'm really out of my element, but if you're using SOAP can't you do all of these in one call?



.:[ Never resist a perfect moment ]:.

Pugzly
Paranoid (IV) Inmate

From: 127.0.0.1
Insane since: Apr 2000

posted posted 08-13-2005 14:25

Not sure I understand - I still need to loop through my db results....

BTW - 3.5+ hours for it to do 861 lookups.... :-(

WarMage
Maniac (V) Mad Scientist

From: Rochester, New York, USA
Insane since: May 2000

posted posted 08-13-2005 14:45

They might have something in place on the server which throttles back the requests you can make. I would see if you could pull back 1 giant resultset in a single server call, instead of trying to hit there server hundreds of times. It is really inefficient to make the call so many times.

Dan @ Code Town

Pugzly
Paranoid (IV) Inmate

From: 127.0.0.1
Insane since: Apr 2000

posted posted 08-13-2005 15:26

I was kind of thinking the same thing (throttling). I sent them an email asking for some info. I can upgrade to a commercial account, which isn't a problem, provided that the throttling is not in place. Ultimately, I'll need to make a one-time call for about 30,000 lookups, and then probably a couple dozen daily after that......

A single call, eh? Let me see if I can pull that off.

bitdamaged
Maniac (V) Mad Scientist

From: 100101010011 <-- right about here
Insane since: Mar 2000

posted posted 08-13-2005 21:40

Here's my basic SOAP knowlege (samples from the Busy Developers Guide to SOAP).

Your basic request looks like:

code:
<?xml version="1.0"?>
<SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance">
   <SOAP-ENV:Body>
      <m:getStateName xmlns:m="http://www.soapware.org/">
         <statenum xsi:type="xsd:int">41</statenum>
         </m:getStateName>
      </SOAP-ENV:Body>
    </SOAP-ENV:Envelope>



I'm pretty sure that m:get statement can be multiple statements allowing you to wrap all your calls into one SOAP statement.



.:[ Never resist a perfect moment ]:.

(Edited by bitdamaged on 08-13-2005 21:43)

WarMage
Maniac (V) Mad Scientist

From: Rochester, New York, USA
Insane since: May 2000

posted posted 08-14-2005 16:57

XSLT yum!

Dan @ Code Town

Pugzly
Paranoid (IV) Inmate

From: 127.0.0.1
Insane since: Apr 2000

posted posted 08-15-2005 14:35

Well, I set my script to sleep for 20 seconds between each call, and as soon as I did that, the performance increased greatly. I was seeing my calls complete in .25 seconds.

I also received a reply from geocoder.us saying they did throttle the free access to every 15 seconds.

bit - thanks for the info. I sat at Barnes & Noble for four hours on Saturday reading about SOAP. Interesting stuff....

« BackwardsOnwards »

Show Forum Drop Down Menu