Re: The length requested for storage allocation is out of range

[Scott Klement <sk@xxxxxxxxxxxxxxxx>]

hi Kim,

> The error is occurring in http_url_post_xml. Here's the statement that
> I'm using - cTimeOut is set at 600. Could that be the problem

I don't see how a timeout could cause a problem with storage allocation?!


> - I'm not giving enough time to receive the entire response? The
> response of this web service can be pretty slow.

The timeout isn't the total download time, it's the amount of "lull" 
that can occur.

HTTPAPI is sitting in a loop, reading bytes off of the network, and (in 
your case) handing them to the XML parser.  As long as it continues to 
receive data, it's happy.  It doesn't matter how slowly that data 
appears, as long as it receives a byte every so often, it's happy.

In your case, if no bytes at all are received for 10 minutes, it'll 
signal a timeout.  To me, that's an absurdly long timeout.  Even in 
dialup days, we typically got transfer rates of at least 33.6k/sec -- 
33600 bits per second, or approx 4000 bytes/sec.

You have your timeout to allow 1 byte per 10 minutes.

I guess if you have to wait 10 minutes for the program to process on the 
remote end (maybe it's running a long SQL query, during which nothing is 
sent) your timeout makes sense...   but I just can't see how it could 
possibly affect storage allocation.


> The first time this error occurred the content-length was 10833517, and
> only 9846465 was received. The second time it happened the
> content-length was 10164047, and 9857785 was received (I isolated the
> response portion of each debug file into a text file to determine the
> size of the response received).

Unless it's receiving the whole document in a single lump, I don't see 
why the content-length would matter.

If you want to debug this yourself, I suggest this strategy:

1) When it blows up, find the precise statement of HTTPAPI that it blows 
up on.

2) Find out the size of the storage it's allocating.  Is it larger than 
16mb?  If so, why is HTTPAPI asking for so much?  Or is it less than 0, 
in which case there's a more severe error.

3) Trade the code back, and find out how it calculated the storage size. 
  Is there anything it could've done better?


> I'm only keeping a week's worth of debug logs, but the next largest
> content-length I've received was 5921033, and it was received in its
> entirety without a problem. So I guess I'm assuming it's erroring out
> whenever I receive a response of a certain size.

AFAIK, HTTPAPI itself doesn't care about the size of the total response. 
However, the XML parsing routines, will need to allocate enough memory 
for the contents of a whole XML element -- this is where you're more 
likely to have a problem.


> As for reproducing the problem - I suppose I could send you the response
> that I have in the debug file with the identifying information removed,
> and I can doctor it up so that it is the correct content? Would that
> help?

If HTTPAPI is crashing, I doubt you're getting the whole response in the 
debug file.

What you could do, however, is use http_url_post() (instead of post_xml) 
so that HTTPAPI saves the response to disk.  Does that produce the same 
error?  If not, try parsing the XML document with http_parse_xml_stmf(). 
  Does that produce the same error?  I'm guessing it would.

Then send me the XML file....  then I can use it to reproduce the 
problem, and figure out how to work around it in the code.
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------