[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: httpapi speed



Peter:   Your issue sounds very similar to one we encountered.  Comparison of the wireshark caps between iSeries and the PC would be needed to confirm if the issues are the same but I will share my experience in case they match.  You can view my original post in the forum archive dated 10/8/2013 and titled "HTTP performance issue vs. soapUi".  Here is an excerpt of the issue which sounds similar to your situation:

  I am using the  http_persist_open() and http_persist_post() APIs to
   consume a web service running on a Windows server in our local domain.
   It works correctly but I notice a dramatic performance difference
   between the call from our iSeries using the APIs vs. soapUi from my
   desktop.  The soapUi calls are 10x faster.  Upon examination of
   WireShark captures from both sessions, I see that the iSeries call
   waits for (and receives) an ACK from the Windows server after the TCP
   segment that contains the url and SoapAction  and User-Agent values,
   etc.  before sending the http Post request with the xml tags (the
   actual request segment).  There is a fair amount of delay here and with
   the soapUi call, the client does not wait between these segments and
   just sends them one after another - resulting in much better
   performance.  I am looking to explain why the iSeries version of the
   call is waiting for the ACK and if there is a way to configure my
   session to act like the soapUi version.

I did not get a response on that post, but after working with a very good IBM comm person, the reason for the delay was discovered and it turned out that the iSeries call was suffering from the implementation of the Nagle algorithm [ http://en.wikipedia.org/wiki/Nagle's_algorithm ] which introduces a delay under certain circumstances and is turned on by default within httpapi.  The solution for us was to disable this delay by using a TCP_No_Delay option to setsockopt() and this resulted in a dramatic improvement which then matched the PC soapUi performance.  If you determine that this might be your situation, I can share the code changes we made.  Scott gave me some good advice and cautions about messing with this which you can find in the forum archive dated 12/17/2013 and titled "HTTPAPI and setsockopt()".  Hope it helps,  Brian

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Sawatzki, Peter
Sent: Friday, May 30, 2014 9:32 AM
To: 'HTTPAPI and FTPAPI Projects'
Subject: RE: httpapi speed

Yes, all three (AS/400, PC and webserver) are on the same subnet and switch.

I'm currently tracing on the webserver with wireshark and what I see is that httpapi request do not end requests with CRLF CRLF as is defined in RFC for http 1.1. If I modify httpapi source to add a CRLF in the request, the response time from the server is around 8 times faster (but not quite as fast as the PC performance).

Peter

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Julius Kaj
Sent: Friday, May 30, 2014 3:09 PM
To: 'HTTPAPI and FTPAPI Projects'
Subject: RE: httpapi speed

Just to ensure that the problem isn't with the network itself:

Is your IBM i connected to the same switch as your PC?

How do ping times and trace route compare between the IBM i and your PC?

IBM i:
PING '192.168.101.5'
TRACEROUTE '192.168.101.5'

Your PC:
PING 192.168.101.5
TRACERT 192.168.101.5

Have you tried to do a COMM trace on your IBM I to capture the low-level communication? See anything suspecious?
STRCOMTRC RMTIP('192.168.101.5')
Do your thing
ENDCOMTRC
PRTCOMTRC
DLTCOMTRC
Print out the spool file
(I believe the procedure is something like that, but it has been some time since I've done this..)


Have you tried to do the same from your PC (using a program like Wireshark or similar) and compared?



-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Sawatzki, Peter
Sent: 30. maj 2014 14:30
To: 'HTTPAPI and FTPAPI Projects'
Subject: RE: httpapi speed

Haha, that's where I'm coming from.

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Charles Wilt
Sent: Friday, May 30, 2014 2:16 PM
To: HTTPAPI and FTPAPI Projects
Subject: Re: httpapi speed

Try running it without debug turned on.

Charles


On Fri, May 30, 2014 at 7:45 AM, Sawatzki, Peter <Sawatzki@xxxxxxxxxxxx>
wrote:

> I added http_dmsg calls before and after comm_lineread and it seems
> that the delay is during reading the first line returned from the server:
>
> HTTPAPI Ver 1.25beta2 released 2012-03-06 OS/400 Ver V6R1M0
>
> 2014-05-30-13.15.01.574000 New iconv() objects set, PostRem=819.
> PostLoc=0. ProtRem=819. ProtLoc=0
> 2014-05-30-13.15.01.555000 http_persist_open(): entered
> 2014-05-30-13.15.02.173000 http_long_ParseURL(): entered
> 2014-05-30-13.15.02.187000 DNS resolver retrans: 2
> 2014-05-30-13.15.02.187000 DNS resolver retry  : 2
> 2014-05-30-13.15.02.187000 DNS resolver options: x'00000136'
> 2014-05-30-13.15.02.187000 DNS default domain: WESTFALIA.NET
> 2014-05-30-13.15.02.187000 DNS server found: 192.168.101.2
> 2014-05-30-13.15.02.187000 DNS server found: 192.168.101.14
> 2014-05-30-13.15.02.189000 http_persist_get(): entered
> 2014-05-30-13.15.02.190000 http_persist_req(GET) entered.
> 2014-05-30-13.15.02.190000 http_long_ParseURL(): entered
> 2014-05-30-13.15.02.190000 do_oper(GET): entered
> 2014-05-30-13.15.02.207000 There are 0 cookies in the cache GET
> /ping.htm HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
>
> 2014-05-30-13.15.02.208000 recvresp(): entered
> 2014-05-30-13.15.02.208000 before comm_lineread
> HTTP/1.1 200 OK
>
> 2014-05-30-13.15.02.491000 after comm_lineread
> 2014-05-30-13.15.02.491000 before comm_lineread
> Content-Type: text/html
>
> 2014-05-30-13.15.02.491000 after comm_lineread
> 2014-05-30-13.15.02.491000 before comm_lineread
> Last-Modified: Fri, 30 May 2014 07:45:28 GMT
>
> 2014-05-30-13.15.02.491000 after comm_lineread
> 2014-05-30-13.15.02.491000 before comm_lineread
> Accept-Ranges: bytes
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
> ETag: "a895b20db7bcf1:0"
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
> Server: Microsoft-IIS/7.5
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
> X-Powered-By: ASP.NET
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
> Date: Fri, 30 May 2014 11:15:02 GMT
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
> Content-Length: 4
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 before comm_lineread
>
>
> 2014-05-30-13.15.02.492000 after comm_lineread
> 2014-05-30-13.15.02.492000 SetError() #13: HTTP/1.1 200 OK
> 2014-05-30-13.15.02.492000 recvresp(): end with 200
> 2014-05-30-13.15.02.493000 recvdoc parms: identity 4
> 2014-05-30-13.15.02.501000 header_load_cookies() entered
> 2014-05-30-13.15.02.504000 recvdoc(): entered
> 2014-05-30-13.15.02.504000 SetError() #0:
> pong
> 2014-05-30-13.15.02.504000 http_persist_get(): entered
> 2014-05-30-13.15.02.504000 http_persist_req(GET) entered.
> 2014-05-30-13.15.02.504000 http_long_ParseURL(): entered
> 2014-05-30-13.15.02.505000 do_oper(GET): entered
> 2014-05-30-13.15.02.505000 There are 0 cookies in the cache GET
> /ping.htm HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
> -----Original Message-----
> From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:
> ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Sawatzki, Peter
> Sent: Friday, May 30, 2014 12:01 PM
> To: 'HTTPAPI and FTPAPI Projects'
> Subject: RE: httpapi speed
>
> Scott,
>
> I implemented the changes you suggested in my test program but
> unfortunatly the performance didn't impoved at all.
>
> To further eliminate any other issues I may have I used a toString
> procedure that does nothing (just returns the length) and I'm
> reuqsting a simple static htm file that has only "pong" as a contents
> (4 bytes). So I'm simply calling "http://192.168.101.5/ping.htm";.
>
> Here is a log file of the requests, it seems that most of the time is
> spent receiving the data from the web server ("recvresp() entered":
>
>
> HTTPAPI Ver 1.25beta2 released 2012-03-06 OS/400 Ver V6R1M0
>
>
> 2014-05-30-11.48.35.494000 http_persist_open(): entered
> 2014-05-30-11.48.35.494000 http_long_ParseURL(): entered
> 2014-05-30-11.48.35.494000 DNS resolver retrans: 2
> 2014-05-30-11.48.35.494000 DNS resolver retry  : 2
> 2014-05-30-11.48.35.494000 DNS resolver options: x'00000136'
> 2014-05-30-11.48.35.494000 DNS default domain: WESTFALIA.NET
> 2014-05-30-11.48.35.494000 DNS server found: 192.168.101.2
> 2014-05-30-11.48.35.494000 DNS server found: 192.168.101.14
> 2014-05-30-11.48.35.495000 http_persist_get(): entered
> 2014-05-30-11.48.35.495000 http_persist_req(GET) entered.
> 2014-05-30-11.48.35.495000 http_long_ParseURL(): entered
> 2014-05-30-11.48.35.495000 do_oper(GET): entered
> 2014-05-30-11.48.35.495000 There are 0 cookies in the cache GET
> /ping.htm
> HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
>
> 2014-05-30-11.48.35.495000 recvresp(): entered
> HTTP/1.1 200 OK
> Content-Type: text/html
> Last-Modified: Fri, 30 May 2014 07:45:28 GMT
> Accept-Ranges: bytes
> ETag: "a895b20db7bcf1:0"
> Server: Microsoft-IIS/7.5
> X-Powered-By: ASP.NET
> Date: Fri, 30 May 2014 09:48:35 GMT
> Content-Length: 4
>
>
> 2014-05-30-11.48.35.696000 SetError() #13: HTTP/1.1 200 OK
> 2014-05-30-11.48.35.696000 recvresp(): end with 200
> 2014-05-30-11.48.35.696000 recvdoc parms: identity 4
> 2014-05-30-11.48.35.696000 header_load_cookies() entered
> 2014-05-30-11.48.35.696000 recvdoc(): entered
> 2014-05-30-11.48.35.696000 SetError() #0:
> pong
> 2014-05-30-11.48.35.696000 http_persist_get(): entered
> 2014-05-30-11.48.35.696000 http_persist_req(GET) entered.
> 2014-05-30-11.48.35.696000 http_long_ParseURL(): entered
> 2014-05-30-11.48.35.696000 do_oper(GET): entered
> 2014-05-30-11.48.35.696000 There are 0 cookies in the cache GET
> /ping.htm
> HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
>
> 2014-05-30-11.48.35.696000 recvresp(): entered
> HTTP/1.1 200 OK
> Content-Type: text/html
> Last-Modified: Fri, 30 May 2014 07:45:28 GMT
> Accept-Ranges: bytes
> ETag: "a895b20db7bcf1:0"
> Server: Microsoft-IIS/7.5
> X-Powered-By: ASP.NET
> Date: Fri, 30 May 2014 09:48:35 GMT
> Content-Length: 4
>
>
> 2014-05-30-11.48.35.912000 SetError() #13: HTTP/1.1 200 OK
> 2014-05-30-11.48.35.912000 recvresp(): end with 200
> 2014-05-30-11.48.35.912000 recvdoc parms: identity 4
> 2014-05-30-11.48.35.912000 header_load_cookies() entered
> 2014-05-30-11.48.35.912000 recvdoc(): entered
> 2014-05-30-11.48.35.912000 SetError() #0:
> pong
> 2014-05-30-11.48.35.912000 http_persist_get(): entered
> 2014-05-30-11.48.35.912000 http_persist_req(GET) entered.
> 2014-05-30-11.48.35.912000 http_long_ParseURL(): entered
> 2014-05-30-11.48.35.912000 do_oper(GET): entered
> 2014-05-30-11.48.35.912000 There are 0 cookies in the cache GET
> /ping.htm
> HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
>
> 2014-05-30-11.48.35.912000 recvresp(): entered
> HTTP/1.1 200 OK
> Content-Type: text/html
> Last-Modified: Fri, 30 May 2014 07:45:28 GMT
> Accept-Ranges: bytes
> ETag: "a895b20db7bcf1:0"
> Server: Microsoft-IIS/7.5
> X-Powered-By: ASP.NET
> Date: Fri, 30 May 2014 09:48:35 GMT
> Content-Length: 4
>
>
> 2014-05-30-11.48.36.115000 SetError() #13: HTTP/1.1 200 OK
> 2014-05-30-11.48.36.115000 recvresp(): end with 200
> 2014-05-30-11.48.36.115000 recvdoc parms: identity 4
> 2014-05-30-11.48.36.115000 header_load_cookies() entered
> 2014-05-30-11.48.36.115000 recvdoc(): entered
> 2014-05-30-11.48.36.115000 SetError() #0:
> pong
> 2014-05-30-11.48.36.115000 http_persist_get(): entered
> 2014-05-30-11.48.36.115000 http_persist_req(GET) entered.
> 2014-05-30-11.48.36.115000 http_long_ParseURL(): entered
> 2014-05-30-11.48.36.115000 do_oper(GET): entered
> 2014-05-30-11.48.36.115000 There are 0 cookies in the cache GET
> /ping.htm
> HTTP/1.1
> Host: 192.168.101.5
> User-Agent: http-api/1.24
>
> -----Original Message-----
> From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:
> ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Scott Klement
> Sent: Wednesday, May 28, 2014 7:25 PM
> To: HTTPAPI and FTPAPI Projects
> Subject: Re: httpapi speed
>
> Peter,
>
> When you're connecting to the same server repeatedly, I would
> recommend using the persistent APIs.  They have the ability to create
> a single connection, and then re-use that connection for multiple
> requests.  This should greatly improve performance.
>
>
> D comm            s               *
>
>
>   /free
>
>       comm = http_persist_open( url );
>
>       if comm = *null;
>          http_crash();
>       endif;
>
>       for x = 1 to 1000;
>
>          rci = http_persist_get( comm: url: 0: %paddr( toString ));
>
>          if rci <> 1;
>           // handle error
>          endif;
>
>       endfor;
>
>       http_persist_close(comm);
>
>
> On 5/28/2014 5:24 AM, Sawatzki, Peter wrote:
> > I'm unsure whether we are doing something wrong: we are using
> > httpapi
> successfully in several different projects however I'm wondering why
> the
> AS/400 is much slower than for example a simple PC that issues a request.
> >
> > To nail this down I wrote a simple web service that can be called
> > with a GET and that returns a fixed string. The get is called with
> > this code
> >
> > Rci = http_url_get_raw( Url : 0 : %PAddr( toString ) );
> >
> > in a loop 1000 times.
> >
> > On the PC (client) side I have a simple Delphi app that does
> >
> >   For i:= 1 to 1000 do Idhttp.get(Url);
> >
> > Where URL is
> > http://192.168.101.27/dwhsvc/dwhsvc.dll/datasnap/rest/TServermethods
> > /p
> > ing
> >
> > And the service responds with "pong". 192.168.101.27 is located in
> > the
> same subnet as the AS/400.
> >
> > The AS/400 takes 210 seconds to issue 1000 calls, the PC takes less
> > than
> two seconds (or 13 seconds for 10,000 calls).
> >
> > Any idea what I'm doing wrong on the AS/400 side ?
> >
> > Peter
> >
> >
> >
> > --------------------------------------------------------------------
> > --
> > - This is the FTPAPI mailing list.  To unsubscribe, please go to:
> > http://www.scottklement.com/mailman/listinfo/ftpapi
> > --------------------------------------------------------------------
> > --
> > -
> >
>
> ----------------------------------------------------------------------
> - This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> ----------------------------------------------------------------------
> -
> ----------------------------------------------------------------------
> - This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> ----------------------------------------------------------------------
> -
> ----------------------------------------------------------------------
> - This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> ----------------------------------------------------------------------
> -
>
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------