RE: http_url_post speed concerns

["Greg Prior" <gprior@xxxxxxxxxxxxxx>]

Sender: "Greg Prior" <gprior@xxxxxxxxxxxxxx>

Aaron,

The http header does not include your xml document.  The header is the
part that you pass to (or receive from) the server that describes what
to do.  On a post it looks something like:

POST /cgi-bin/pgmname HTTP/1.0
Content-Type: text/xml
Content-Length: 1745


Each line is terminated by CRLF with the last line having 2 CRLF in a
row.  Your data follows the header.  Scott said that he was using a
different method to get the data that follows the header.

The RFC for HTTP specifies that the header ends with x'0d0a0d0a'.  After
that Scott's code does not look for it.

greg


> -----Original Message-----
> From: Bartell, Aaron L. (TC) [mailto:ALBartell@xxxxxxxxxxxxxx]
> Sent: Friday, June 13, 2003 1:13 PM
> To: 'ftpapi@xxxxxxxxxxxxx'
> Subject: RE: http_url_post speed concerns
> 
> Sender: "Bartell, Aaron L. (TC)" <ALBartell@xxxxxxxxxxxxxx>
> 
> >Why?  Because it makes the code easier to write that way :)
> 
> Ha!  I was afraid you were going to say that.  I have written my own
> processes to do the receive, but I switched over to yours because I
didn't
> know how to do the algorithm to figure out where the end of the
request
> was
> (Buck posted it once to the RPG list, but I never did take the time to
> implement it
http://archive.midrange.com/rpg400-l/200211/msg00525.html).
> 
> >The reason the code was easier to write this way is that it needs to
read
> until it encounters a x'0d0a0d0a' (CRLF twice) from the HTTP server.
> 
> I am not sure if it is safe to write it this way, because you could
> potentially have that sequence of characters in the content of the
POST
> response.
> 
> >...and on average, will be reading less than 200 bytes.
> 
> How did you come up with that average?  I may be an exception, but I
am
> passing XML back and forth, and I am easily past 1000 bytes going out
and
> coming into each web service.  Oh, don't you just love XML;-)  It
looks
> like
> I would have to have my program that is hosting the web service
specify
> that
> info was chunked for it to come across as a "document" per your
comment
> about recvdoc() being much faster?
> 
> I hear ya on wanting to get help, I haven't had one ounce of outside
help
> yet for the RPG XML parser I am working on, of course I have only had
that
> open for a couple months and I don't have it in CVS yet (as you
pointed
> out
> Scott :-).  I wont have time to punch out any helpful code for a
couple
> weeks (conferences) but will give it a shot after I am done with
those.
> 
> Aaron Bartell
> 
> -----Original Message-----
> From: Scott Klement [mailto:klemscot@xxxxxxxxxxxx]
> Sent: Friday, June 13, 2003 12:41 PM
> To: 'ftpapi@xxxxxxxxxxxxx'
> Subject: Re: http_url_post speed concerns
> 
> 
> Sender: Scott Klement <klemscot@xxxxxxxxxxxx>
> 
> On Fri, 13 Jun 2003, Bartell, Aaron L. (TC) wrote:
> >
> > I have a program making use of the http_url_post sub proc from the
HTTP
> > API's and have some concerns about speed. I have gone through the
code
> and
> > have a question as to why only one byte of data is being read from
the
> > socket connection at a time?  Take a look at this line of code "eval
> > wwRec = recv(peSock: wwPos: 1: 0)".
> 
> Why?  Because it makes the code easier to write that way :)   You're
> right, it won't perform very well, but the recvresp() routine is only
> used once during the HTTP conversation, and on average, will be
reading
> less than 200 bytes.   So, making it as fast as possible wasn't a
priority
> for me.   By contrast, it was important to be that recvdoc() be
efficient,
> since that could be used to handle many megabytes during transfer.
That's
> where most of the time will be.
> 
> The reason the code was easier to write this way is that it needs to
read
> until it encounters a x'0d0a0d0a' (CRLF twice) from the HTTP server.
The
> length of the data between the start and the CRLF sequence is
completely
> variable.  It could be 0 bytes, it could be many bytes...  no way to
know
> how much to read.
> 
> You could read the data into a larger buffer (say 1024A) and then scan
> that buffer for the x'0d0a0d0a' sequence, however, what if the length
> happened to end up so that part of that string was in one read, and
part
> of it was in another?   So you'd have to add code to handle that
> situation.   Then, what if the server sends more data after the
> x'0d0a0d0a' sequence, and it ends up appearing in your 1k buffer
> after the CRLF sequence?   You'd have to save that data, and use it
> instead of a direct call to recv() in any/all subprocedures that could
> read from the socket after calling recvresp().
> 
> >
> > Any clarification is welcome. . .
> >
> 
> To sum up:  You're right, reading one at a time is slow and
inefficient.
> However, it's much easier to code that way, and that routine is only a
> small part of the conversation.
> 
> Solution:   If you'd like to re-write the recvresp and/or ssl_recvresp
> procedures to be more efficient, you're welcome to do so.  Assuming
> that your code looks like it'll work properly in all circumstances,
> I'd be happy to include it in the distribution of HTTPAPI.
> 
> Remember, this is an open-source project.  It only survives if people
are
> willing to contribute back to the project.    I do my best to help you
> guys out, but the only way that I'll get any benefit from giving you
guys
> this code is if you contribute something back!!
> 
>
-----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubsribe from the list send
mail
> to majordomo@xxxxxxxxxxxxx with the body: unsubscribe ftpapi
mymailaddr
>
-----------------------------------------------------------------------
>
-----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubsribe from the list send
mail
> to majordomo@xxxxxxxxxxxxx with the body: unsubscribe ftpapi
mymailaddr
>
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubsribe from the list send mail
to majordomo@xxxxxxxxxxxxx with the body: unsubscribe ftpapi mymailaddr
-----------------------------------------------------------------------