[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: HTTPAPI and setsockopt()
Scott: Thanks for your response. The list site seemed to have gone 'dark' for a few weeks and if this was due to health concerns, I hope your recent response indicates that all is well with you. The httpapi suite and your continued support have been of great benefit to us and I'm sure to everyone on this list. The fact that you can do this and hold another (paying) job is amazing. In the interim I was able to make the code modifications I wanted and the results have been most satisfactory. In answer to your questions:
* We discovered through analyzing a trace (wireshark displaying iSeries comm. trace) of a web service we are consuming that there was a consistent 18ms delay after each SOAP_Action post that then received an ACK before the actual xml post request was sent. This was not the behavior shown through SoapUi when I called the same service from my desktop - it sent the Soap_Action packet and the xml post request one after another with no delay or ACK in between. 18ms is not a lot of time - we only cared because we call this service (it puts address information into a national standard format) several times a year in a large batch with several hundred thousand records and it could take more than a day to process. Finding and eliminating the delay would be a huge win. We were able to get a good communications person at IBM interested in the problem and he suggested that we might try turning on TCP_NoDelay which is off by default. I did this via the setsockopt() and the performance improvement was dramatic (about 10x). Apparently, the web service request suffered from an implementation of the Nagle algorithm which as I understand it, serves to consolidate data into packets to make the TCP transmission more efficient in cases where a small payload would waste resources (like Telnet session key strokes). I'm not sure why our request fell into this category but I can't argue with the performance when we disabled it (by indicating TCP_NoDelay on). I just added an optional parm to http_persist_open() and below to indicate if it should be set. You already had a constant for the TCP_NoDelay option and I believe any non-zero value will turn it on using setsockopt().
* I will heed your warning about modifying the code and will isolate the mods from the standard LIBHTTP objects so that we can upgrade to a newer version in the future. Thanks for all the pointers and support. I thought you would appreciate an explanation concerning why we were looking into it and that it had a happy ending.
Thanks, Brian
-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: Friday, December 13, 2013 12:44 AM
To: HTTPAPI and FTPAPI Projects
Subject: Re: HTTPAPI and setsockopt()
Hello Brian,
Can you explain why you want to call setsockopt()? This seems like a
very strange thing to do. HTTPAPI is intended to perform all of the
network code for you so you don't have to write it yourself. But, you seem to want to do the network code yourself.
If you change the socket option, then (depending on what you change) you
could easily make HTTPAPI stop working entirely. And, if you can't
figure out where to insert the call to setsockopt, will you be able to figure out how to adjust the routines in HTTPAPI so that they are coded properly for the new socket options?
If you truly do want to figure out how to modify/handle the network code in HTTPAPI, I can explain it a little bit. But, be aware that once you've made the modifications it will be up to you to support it
henceforth. When you go rewriting a portion of HTTPAPI, it will not be
practical for me to provide support to you.
Having said all of that, HTTPAPI is designed such that there are different modules for different communications methods. The idea is that all of the network code for plain TCP is in one place, all of the network code for SSL TCP is in another place, etc. I can add more communications methods just by writing another communications module.
There is a copybook called COMM_H that contains a generic interface to a 'communications driver'. Everything in COMM_H is accessed by pointer, so by changing the pointers to point to objects in one of the communication modules (which currently are just plain TCP and SSL TCP
modules) HTTPAPI will call a different driver.
The actual code for each communication module is in COMMTCPR4 for plain TCP, or COMMSSLR4 for SSL over TCP. Each of these modules has a routine named COMMxxx_NEW (where xxx is the communication method, TCP or SSL) that sets up all of the pointers in COMM_H so that HTTPAPI will call
it's communication routines. It can also reserve additional memory for
features that are specific to it's communication method and tack those on to the end of the communication structure so it can keep track of all
of that. Each module also has a COMMxxx_CLEANUP routine that is used
to shut down the communication module and free up the memory it's
using. The other routines in the modules are used for the various bits
of communication that HTTPAPI does... connecting, disconnecting, resolving host names, reading data in stream, line and block mode.
writing data in stream, line and block mode, etc
If you want to change the socket options, then you would need to change it in one of those modules. Where you'd do that would depend on which
option you are changing, and what it affects. If it affects the
behavior of reading/writing data from the socket, you'd probably want to change it in the connect routine, but you'd almost certainly need to update the code in all of the routines to handle the new mode appropriately.
Hope that helps.
On 11/21/2013 1:10 PM, Beaton Brian wrote:
> I would like to call setsockopt() to influence the behavior of a
> program we are using to consume a web service with http_persist_open(),
> followed by http_persist_post() called within a loop. I believe I am
> clear on how to call the setsockopt routine correctly, I just don't see
> where this code would be inserted. I see prototype definitions for the
> http_persist routines, but can't find any actual source code for them
> in LIBHTTP. Any help appreciated and apologies if this type of
> question has already been asked - I didn't find anything in the archive
> when I searched. Thanks, BB
>
>
>
> ----------------------------------------------------------------------
> - This is the FTPAPI mailing list. To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> ----------------------------------------------------------------------
> -
-----------------------------------------------------------------------
This is the FTPAPI mailing list. To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------