Topic awaiting preservation: performance issues using nusoap with repeated calls

Greets

I'm using nusoap to convert addresses to latitude and longitude coordinates via http://www.geocode.us. I'm basically just running through a loop, doing the lookup, then dumping the coordinates to my db.

When I get the first lookup, it finishes fairly quickly (under .025 seconds). But subsequent lookups take about 15 seconds for EACH lookup. Curious to know if anyone has used nusoap before and had this issue, of if you notice an issue with my code:

include($_SERVER['DOCUMENT_ROOT'] . '/i/constants.php');
startTimer();
include('nusoap.php');
set_time_limit(0); // just for testing
$records = 0;
$soapclient = new soapclient("http://geocoder.us/service/soap");

$info = mysql_query("select * from psor_offenders where addressLat = '' and addressLong = ''");
while ($rows=mysql_fetch_array($info)){
	$mapData = '';
	$address = $rows[address] . " " . $rows[city] . " " . $rows[state] . " " . $rows[zip];
	// echo "address assembly at " . showtimer() . "<br />\n";
	$address2check = array('geocode'=> $address);
	// echo "address check at " . showtimer() . "<br />\n";
	$mapData = $soapclient->call("geocode", $address2check, "", "");
	// echo "mapData at " . showtimer() . "<br />\n";
	$sql = "update psor_offenders set addressLat = '" . $mapData[0]['lat'] . "', addressLong = '" . $mapData[0]['long'] . "' where id = '$rows[id]'";
	// echo "SQL query string assembled at " . showtimer() . " ($sql)<br />\n";
	mysql_query($sql);
	// echo "SQL query finished at " . showtimer() . "<br />\n";
	// echo "ID " . $rows[id] . " is done at " . showtimer() . "<br /><br />\n";
	$records++;
	flush();
}

I've got several thousand lookups to do initially, plus sporadic lookups daily. I'd prefer it not take the rest of my life to do the lookups!  

stopTimer();
echo $totaltime . " seconds to assemble $records records\n";

I bet it has nothing to do with nuSoap and everything to do with geocoder's soap server. Head over to xmethods and try something that loops through a series of calls to one of their services and see if you have the same problem.

hmm.. I'm really out of my element, but if you're using SOAP can't you do all of these in one call?



.:[ Never resist a perfect moment ]:.

Not sure I understand - I still need to loop through my db results....

BTW - 3.5+ hours for it to do 861 lookups.... :-(

They might have something in place on the server which throttles back the requests you can make. I would see if you could pull back 1 giant resultset in a single server call, instead of trying to hit there server hundreds of times. It is really inefficient to make the call so many times.

Dan @ Code Town

I was kind of thinking the same thing (throttling). I sent them an email asking for some info. I can upgrade to a commercial account, which isn't a problem, provided that the throttling is not in place. Ultimately, I'll need to make a one-time call for about 30,000 lookups, and then probably a couple dozen daily after that......

A single call, eh? Let me see if I can pull that off.

Here's my basic SOAP knowlege (samples from the Busy Developers Guide to SOAP).

Your basic request looks like:

<?xml version="1.0"?>
<SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance">
   <SOAP-ENV:Body>
      <m:getStateName xmlns:m="http://www.soapware.org/">
         <statenum xsi:type="xsd:int">41</statenum>
         </m:getStateName>
      </SOAP-ENV:Body>
    </SOAP-ENV:Envelope>

I'm pretty sure that m:get statement can be multiple statements allowing you to wrap all your calls into one SOAP statement.



.:[ Never resist a perfect moment ]:.

(Edited by bitdamaged on 08-13-2005 21:43)

XSLT yum!

Dan @ Code Town

Well, I set my script to sleep for 20 seconds between each call, and as soon as I did that, the performance increased greatly. I was seeing my calls complete in .25 seconds.

I also received a reply from geocoder.us saying they did throttle the free access to every 15 seconds.

bit - thanks for the info. I sat at Barnes & Noble for four hours on Saturday reading about SOAP. Interesting stuff....