[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: httpapi speed



The interesting thing is that the delay is when reading the very first character from the web server in response to the request. It seems that the server thinks the request is not yt completely sent.

I noticed that *requests* that Fiddler2 sends to a web server ends with OD 0A 0D 0A while http does only send 0D 0A tot he web server.

I used this info to add an additional CRLF tot he request header and then the response from the server is almost immediate.

I added this to do_oper() just before sending the request chain:

0469.01 c                   eval      wwReqChain = wwReqChain + CRLF                            
0470.00                                                                                         
0471.00  *********************************************************                              
0472.00  *  Send request chain & get response                                                   
0473.00  *********************************************************                              
0474.00 c                   eval      rc = SendReq( peComm                                      
0475.00 c                                         : %addr(wwReqChain)+VARPREF                  

If I make this change my performance is better - 210 Seconds before the change, now 1000 requests take only 50 seconds. But looking at the stats it seems that every five or so requests still have a delay.

Peter

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Sawatzki, Peter
Sent: Friday, May 30, 2014 1:45 PM
To: 'HTTPAPI and FTPAPI Projects'
Subject: RE: httpapi speed

I added http_dmsg calls before and after comm_lineread and it seems that the delay is during reading the first line returned from the server:

HTTPAPI Ver 1.25beta2 released 2012-03-06
OS/400 Ver V6R1M0

2014-05-30-13.15.01.574000 New iconv() objects set, PostRem=819. PostLoc=0. ProtRem=819. ProtLoc=0
2014-05-30-13.15.01.555000 http_persist_open(): entered
2014-05-30-13.15.02.173000 http_long_ParseURL(): entered
2014-05-30-13.15.02.187000 DNS resolver retrans: 2
2014-05-30-13.15.02.187000 DNS resolver retry  : 2
2014-05-30-13.15.02.187000 DNS resolver options: x'00000136'
2014-05-30-13.15.02.187000 DNS default domain: WESTFALIA.NET
2014-05-30-13.15.02.187000 DNS server found: 192.168.101.2
2014-05-30-13.15.02.187000 DNS server found: 192.168.101.14
2014-05-30-13.15.02.189000 http_persist_get(): entered
2014-05-30-13.15.02.190000 http_persist_req(GET) entered.
2014-05-30-13.15.02.190000 http_long_ParseURL(): entered
2014-05-30-13.15.02.190000 do_oper(GET): entered
2014-05-30-13.15.02.207000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24


2014-05-30-13.15.02.208000 recvresp(): entered
2014-05-30-13.15.02.208000 before comm_lineread
HTTP/1.1 200 OK

2014-05-30-13.15.02.491000 after comm_lineread
2014-05-30-13.15.02.491000 before comm_lineread
Content-Type: text/html

2014-05-30-13.15.02.491000 after comm_lineread
2014-05-30-13.15.02.491000 before comm_lineread
Last-Modified: Fri, 30 May 2014 07:45:28 GMT

2014-05-30-13.15.02.491000 after comm_lineread
2014-05-30-13.15.02.491000 before comm_lineread
Accept-Ranges: bytes

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread
ETag: "a895b20db7bcf1:0"

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread
Server: Microsoft-IIS/7.5

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread
X-Powered-By: ASP.NET

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread
Date: Fri, 30 May 2014 11:15:02 GMT

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread
Content-Length: 4

2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 before comm_lineread


2014-05-30-13.15.02.492000 after comm_lineread
2014-05-30-13.15.02.492000 SetError() #13: HTTP/1.1 200 OK
2014-05-30-13.15.02.492000 recvresp(): end with 200
2014-05-30-13.15.02.493000 recvdoc parms: identity 4
2014-05-30-13.15.02.501000 header_load_cookies() entered
2014-05-30-13.15.02.504000 recvdoc(): entered
2014-05-30-13.15.02.504000 SetError() #0:
pong
2014-05-30-13.15.02.504000 http_persist_get(): entered
2014-05-30-13.15.02.504000 http_persist_req(GET) entered.
2014-05-30-13.15.02.504000 http_long_ParseURL(): entered
2014-05-30-13.15.02.505000 do_oper(GET): entered
2014-05-30-13.15.02.505000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Sawatzki, Peter
Sent: Friday, May 30, 2014 12:01 PM
To: 'HTTPAPI and FTPAPI Projects'
Subject: RE: httpapi speed

Scott,

I implemented the changes you suggested in my test program but unfortunatly the performance didn't impoved at all. 

To further eliminate any other issues I may have I used a toString procedure that does nothing (just returns the length) and I'm reuqsting a simple static htm file that has only "pong" as a contents (4 bytes). So I'm simply calling "http://192.168.101.5/ping.htm";.

Here is a log file of the requests, it seems that most of the time is spent receiving the data from the web server ("recvresp() entered":


HTTPAPI Ver 1.25beta2 released 2012-03-06
OS/400 Ver V6R1M0


2014-05-30-11.48.35.494000 http_persist_open(): entered
2014-05-30-11.48.35.494000 http_long_ParseURL(): entered
2014-05-30-11.48.35.494000 DNS resolver retrans: 2
2014-05-30-11.48.35.494000 DNS resolver retry  : 2
2014-05-30-11.48.35.494000 DNS resolver options: x'00000136'
2014-05-30-11.48.35.494000 DNS default domain: WESTFALIA.NET
2014-05-30-11.48.35.494000 DNS server found: 192.168.101.2
2014-05-30-11.48.35.494000 DNS server found: 192.168.101.14
2014-05-30-11.48.35.495000 http_persist_get(): entered
2014-05-30-11.48.35.495000 http_persist_req(GET) entered.
2014-05-30-11.48.35.495000 http_long_ParseURL(): entered
2014-05-30-11.48.35.495000 do_oper(GET): entered
2014-05-30-11.48.35.495000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24


2014-05-30-11.48.35.495000 recvresp(): entered
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Fri, 30 May 2014 07:45:28 GMT
Accept-Ranges: bytes
ETag: "a895b20db7bcf1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 30 May 2014 09:48:35 GMT
Content-Length: 4


2014-05-30-11.48.35.696000 SetError() #13: HTTP/1.1 200 OK
2014-05-30-11.48.35.696000 recvresp(): end with 200
2014-05-30-11.48.35.696000 recvdoc parms: identity 4
2014-05-30-11.48.35.696000 header_load_cookies() entered
2014-05-30-11.48.35.696000 recvdoc(): entered
2014-05-30-11.48.35.696000 SetError() #0:
pong
2014-05-30-11.48.35.696000 http_persist_get(): entered
2014-05-30-11.48.35.696000 http_persist_req(GET) entered.
2014-05-30-11.48.35.696000 http_long_ParseURL(): entered
2014-05-30-11.48.35.696000 do_oper(GET): entered
2014-05-30-11.48.35.696000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24


2014-05-30-11.48.35.696000 recvresp(): entered
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Fri, 30 May 2014 07:45:28 GMT
Accept-Ranges: bytes
ETag: "a895b20db7bcf1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 30 May 2014 09:48:35 GMT
Content-Length: 4


2014-05-30-11.48.35.912000 SetError() #13: HTTP/1.1 200 OK
2014-05-30-11.48.35.912000 recvresp(): end with 200
2014-05-30-11.48.35.912000 recvdoc parms: identity 4
2014-05-30-11.48.35.912000 header_load_cookies() entered
2014-05-30-11.48.35.912000 recvdoc(): entered
2014-05-30-11.48.35.912000 SetError() #0:
pong
2014-05-30-11.48.35.912000 http_persist_get(): entered
2014-05-30-11.48.35.912000 http_persist_req(GET) entered.
2014-05-30-11.48.35.912000 http_long_ParseURL(): entered
2014-05-30-11.48.35.912000 do_oper(GET): entered
2014-05-30-11.48.35.912000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24


2014-05-30-11.48.35.912000 recvresp(): entered
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Fri, 30 May 2014 07:45:28 GMT
Accept-Ranges: bytes
ETag: "a895b20db7bcf1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 30 May 2014 09:48:35 GMT
Content-Length: 4


2014-05-30-11.48.36.115000 SetError() #13: HTTP/1.1 200 OK
2014-05-30-11.48.36.115000 recvresp(): end with 200
2014-05-30-11.48.36.115000 recvdoc parms: identity 4
2014-05-30-11.48.36.115000 header_load_cookies() entered
2014-05-30-11.48.36.115000 recvdoc(): entered
2014-05-30-11.48.36.115000 SetError() #0:
pong
2014-05-30-11.48.36.115000 http_persist_get(): entered
2014-05-30-11.48.36.115000 http_persist_req(GET) entered.
2014-05-30-11.48.36.115000 http_long_ParseURL(): entered
2014-05-30-11.48.36.115000 do_oper(GET): entered
2014-05-30-11.48.36.115000 There are 0 cookies in the cache GET /ping.htm HTTP/1.1
Host: 192.168.101.5
User-Agent: http-api/1.24

-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: Wednesday, May 28, 2014 7:25 PM
To: HTTPAPI and FTPAPI Projects
Subject: Re: httpapi speed

Peter,

When you're connecting to the same server repeatedly, I would recommend using the persistent APIs.  They have the ability to create a single connection, and then re-use that connection for multiple requests.  This should greatly improve performance.


D comm            s               *


  /free

      comm = http_persist_open( url );

      if comm = *null;
         http_crash();
      endif;

      for x = 1 to 1000;

         rci = http_persist_get( comm: url: 0: %paddr( toString ));

         if rci <> 1;
          // handle error
         endif;

      endfor;

      http_persist_close(comm);


On 5/28/2014 5:24 AM, Sawatzki, Peter wrote:
> I'm unsure whether we are doing something wrong: we are using httpapi successfully in several different projects however I'm wondering why the AS/400 is much slower than for example a simple PC that issues a request.
>
> To nail this down I wrote a simple web service that can be called with 
> a GET and that returns a fixed string. The get is called with this 
> code
>
> Rci = http_url_get_raw( Url : 0 : %PAddr( toString ) );
>
> in a loop 1000 times.
>
> On the PC (client) side I have a simple Delphi app that does
>
>   For i:= 1 to 1000 do Idhttp.get(Url);
>
> Where URL is
> http://192.168.101.27/dwhsvc/dwhsvc.dll/datasnap/rest/TServermethods/p
> ing
>
> And the service responds with "pong". 192.168.101.27 is located in the same subnet as the AS/400.
>
> The AS/400 takes 210 seconds to issue 1000 calls, the PC takes less than two seconds (or 13 seconds for 10,000 calls).
>
> Any idea what I'm doing wrong on the AS/400 side ?
>
> Peter
>
>
>
> ----------------------------------------------------------------------
> - This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> ----------------------------------------------------------------------
> -
>

-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------