[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: HTTPAPI performance suggestions



Don't see why one program couldn't send the REQUEST and process the RESPONSE

You mentioned this is causing lots of IO's on the IBM

Are the programs setting on *INLR causing the files to opened/closed on each call?

In the HTTP 1.1 protocol, all connections are considered "persistent" by default.  The idea is you make one request (its very much like sending a
command) and when the request is done, you're left at the point where you can make another request (much like returning to the command line), you can then keep making requests until you're done, at which point you disconnect.

HTTPAPI, therefore provides a set of routines called http_persist_open(), http_persist_xxx (where xxx are the different 
request types) and http_persist_close().   You can use these to connect, 
run multiple requests, and then disconnect if desired. This is the "persistent" approach.



www.pilotpen.us
Chris Hayden - Sr. Systems Analyst
Office: (904) 645-9999 ext.1252
Mobile: 904-654-4089
Pilot Corporation of America
3855 Regent Blvd, Jacksonville, Florida 32224 United States

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. 
-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Venter.Derick@xxxxxxxxxxxxxx
Sent: Wednesday, August 10, 2016 4:01 AM
To: HTTPAPI and FTPAPI Projects <ftpapi@xxxxxxxxxxxxxxxxxxxxxx>
Subject: HTTPAPI performance suggestions

Hello  everyone.

Hope someone can give us some suggestions to improve performance.

We are using HTTPAPI libraries to consume Corticon Rule Engine (Business decision making) web services on the IBM.

We are generating +- 185 000 XML's daily in our production environment. 
This is causing lots of IO's on the IBM. 

We generate and store the Request XML's in RPG on a folder in the IFS. The path and file name to the Request XML is then passed to a RPG function to call the web service and store the Response XML in the same IFS folder. We are generating folders per hour per day to keep the amount of XML files per folder to a minimum and to assist with archiving. Another RPG function then reads the Response XML file.

Is there maybe a faster way of doing this. The thing is we need history of the Request and Response XML files for at least 2 weeks. I am thinking of maybe storing the XML data in a table and not a .xml file. 

Any ideas

Derick Venter
Applications Developer IV

Systems Integration
Tel: +27 (13) 247 2816 Fax: +27 (0) 86 573 2274
Cell: +27 (0) 83 458 6599
Email: derick.venter@xxxxxxxxxx
www.gijima.com



From:   Scott Klement <sk@xxxxxxxxxxxxxxxx>
To:     HTTPAPI and FTPAPI Projects <ftpapi@xxxxxxxxxxxxxxxxxxxxxx>
Date:   2016/08/06 02:10 AM
Subject:        Re: HTTPAPI -Header Parser
Sent by:        ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx



Hi Athar,

It's unusual for an application to need information from the headers, that's why you haven't seen much discussion in the archives.  Normally header data is not needed for anything except httpapi, but in the rare occasion that it is, the application usually just needs a single header, so they just use http_header() to grab that one.

Can you explain why you'd like to iterate through all of them? That seems like a strange thing to do.

Your comment about searching for CRLF,CRLF is a good point. HTTPAPI should definitely NOT be looking at data after the HTTP headers. 
Though, I don't think it does read past the CRLF,CRLF, because the calling routine only passes the headers to parse_resp_chain, it does not pass the entire response document. The variable names might be a bit misleading, however.

-SK


On 8/5/2016 1:01 PM, Athar Iqbal wrote:
>     Hi,
>
>
>     I am new to this forum, so please forgive me if I ask something that
>     has already been asked. I did go through the archives and didn't 
find
>     any information.
>
>
>     My question is on header parser function in HTTPAPI. Currently,
>     header_parse() function reads all the headers and puts them in the 
data
>     structure which is available to the HEADERR4 module only.  In order 
to
>     get the header information, we need to use http_header() function.
>     Also, header_parse() goes through the complete response object to 
pull
>     headers from it. Wouldn't it be faster, if it looks for <CRLF><CRLF>
>     and stop the search after that?
>
>
>     I am developing a generic framework to consume web services offered 
by
>     different businesses such MelissaData or D&B. Developers on my team 
may
>     or may not know all the headers coming back in the response object. 
I
>     want to iterate though the all the header and put them in the list 
and
>     return it back to the calling procedure.
>
>
>     Challenge I am coming across is that there is no iterator that can 
get
>     me all headers without specifying name. Also, could someone please
>     explain, what is best use of USERDATA pointer in http_xproc()
>
>
>     If someone has come across something like this, could you please 
share.
>     If not, I would like to contribute to HTTPAPI by enhancing existing
>     code and send it back to the team to see if it is acceptable to 
include
>     in package.
>
>
>     Again, I apologize in advance, if this has been already discussed 
and
>     if you can point me to some resource.
>
>
>     Athar
>
>
>     DISCLAIMER:
>     This e-mail is intended for the use of the addressee(s) only and may
>     contain privileged, confidential, or proprietary information that is
>     exempt from disclosure under law. If you are not the intended
>     recipient, please do not read, copy, use or disclose the contents of
>     this
>     communication to others. Please notify the sender that you have
>     received this e-mail in error by replying to the e-mail. Please then
>     delete the e-mail and destroy any copies of it. Thank you.
>
>
>
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------

-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------


-------------------------------------------------------------------------------------
This e-mail is subject to the Columbus Stainless [Pty] Ltd Email Legal Notices available at: http://www.columbus.co.za/EmailLegalNotice.htm.
-------------------------------------------------------------------------------------

This e-mail message has been scanned for Viruses and Content and cleared by MailMarshal
-------------------------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------