[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Suggestions for the HTTP API (and Questions)

At 12:50 AM 12/24/2003 -0600, you wrote:
In my opinion, you should try not to make changes to the API unless you
intend to contribute them back to the project.  The reason I say that is
that now each time you run a new version of HTTPAPI, you'll have to make
those same changes before you can use it.   And, if someone else changes
the code, your changes could conflict.

A painful truth, as I have just discovered :> Once 1.9 is mainstream I'll see if my apps can seamlessly use the exit program, otherwise I may have to bear this burden at least as far as peHeaders is concerned :<

This is an open-source project, and we'd appreciate it if you'd contribute
your changes by either posting them to the mailing list (if they're small)
or putting them on-line and providing a link (if they're large)

If you cannot put them on-line, you can send them to me, and I'll put them

Will do...

It's my firm belief that people should not need to understand HTTP headers
to use HTTPAPI.

For the more advanced programmers, I added the ability to specify an "exit
proc" (call-back function) so that they can specify any header that they
like.  This functionality will be available in version 1.9.  (1.9 was not
released yet, but you can get the beta version here:
http://www.scottklement.com/httpapi/beta/httpapi.savf  the code was
written back in October)

This should satisfy your requirement as well as protecting backwards and
forwards compatibility.

Our case is perhaps a bit different than the norm, where our users will be interfacing to the API for a very wide variety of applications, and our users will not want (indeed may not be capable) to write their own exit programs for each case :< If I can find a way to 'soft-code' the exit program I'll do it.... (I certainly do not relish the thought of manually re-merging changes at each new version)

> 2) Have https_init() clear wkEnvH when it fails (except in the case where
> it has already been initialized).  For those who use named activation
> groups, not clearing it results in an inability to retry the operation
> after the problem which caused it to fail has been fixed.

Please post (or send me) your code for this.  It sounds like a bug that
should be fixed...

I added the following line:

c eval wkEnvH = *NULL

Before every return -1, except for the first... I'll attach the routine as I have it... Because it's a global variable it retains its value, and if it failed because of no authority, or an app not registered, it means you cannot retry the init within the same activation group because the first error check knocks you out. (with CGI applications it's especially a problem in that it's not that easy to just reclaim the activation group)

> 3) Have the automatic registration function register the application ID as
> a client rather than a server (the default).  I have a code snippet which
> does this FWIW...

When I wrote the registration code I was on a V4R5 box where there was
only one type of registration with the DCM.  In V5 they changed it to have
both client & server separately.

I'd like to see your code -- particularly if it doesn't break
compatibility with V4R5.  (Though, we can use /if defined() and friends to
make it compatible if necessary)

Ahh, I wondered if that was the case. I'll attach my changes here as well... It all works fine as a server, I just made the change as I was trying to overcome the authority issue mentioned below, and the IBM guy I was talking to refused to accept that this was not the cause of the problem :<

> 1) How does everyone deal with the fact that authority is needed for the
> certificate store file in order to establish an SSL connection? I looked
> up gsk_secure_soc_open and it says you need to have authority to it, but an
> IBMer that I spoke to said that they use GSKit for their client side secure
> FTP and the users do not need authority to those files. In fact he
> insisted that no authority was needed (I tested it thoroughly later and
> determined that yes it was (as documented)). Have you all found this to be
> the case? If not would you be able to let me know some info on your config?

Huh?  The documentation for gsk_secure_soc_open() clearly states that no
special authority is required.

>From the V5R2 information center: "Service Program Name:
   QSYS/QSOSSLSR  Default Public Authority: *USE"  "Authorities: No
   authorization is required."

The only caveat that I know of is that the user must have authority to
read the stream files in the IFS where the certificates and keys are
stored by the digital certificate manager...   I granted them access using
the standard QShell CHOWN command.

It's been a long time since I did that and my mind is a little fuzzy, but
I think all I did was:
    STRQSH CMD('chown -R 755 /QIBM/UserData/ICSS/Cert')

Sorry, my bad, the function is: gsk_secure_soc_init. under authorities it states:

Authorization of *R (allow access to the object) to the certificate store file and its associated files is required. Authorization of *X (allow use of the object) to each directory of the path name of the certificate store file and its associated files is required.

It is the caveat you mentioned which is presenting the problem in the case of one of our clients. Their program can be run by any number of users, and they don't want to have to grant authority to every one, but they don't want to set it to public either. They talked to an IBMer who said they shouldn't need to grant that authority. He made his claim based on the fact that at V5R2 IBM has a FTP SSL client which does not require any authority changes to make it work... But if you've had to grant that same authority as I did I'll have to assume the IBM guy didn't know what he was talking about. FWIW, I would also suggest amending the readme file to mention that this might be necessary, it was probably my biggest stumbling block in getting my example programs to work when I first tried the API.

> 2) The 1.8 version removed the 'connection: close' from the requests.  This
> broke an existing app for me where the program hangs on the select in
> ssl_recvdoc if the connection close is not specified.  I can provide a
> sample URL and postdata if someone wants to try it.  I was able to work
> around it (by specifying connection: close in my peHeaders parm), but it
> makes me worry as to how to determine when it is needed and when it is
> not.  I did notice the content length returned when it was missing was 306
> versus 700 when that header was provided, so I don't know to what extent it
> is something strange happening on the remote end.

According to RFC 2616, "connection: close" is not required and removing it
should not break anything.

I removed it because people were finding that data was getting truncated
with certain (IMHO, buggy) versions of Microsoft IIS.

Let me see your URL, though.  If removing connection:close is breaking
something, I'd like to see if I can determine when and why so we can
account for it in the future.

The URL is:


The post data is: (though I suppose anything would do)


And they are one example of the folks who require special headers:

MIME-Version: 1.0
Content-type: application/PTI21
Content-transfer-encoding: text
Request-number: 1
Document-type: Request

> 3) When connecting to one site with POST (and it returns chunked output) it
> returns an error code of 200 instead of 100, which causes the call to
> fail.  This is as a result of the post data being empty.  Is this something
> the remote server should handle or should the API not expect the
> 100-continue if the postdata length was zero?

Hmmm... a post with no data? Why not just do a GET in that case?

At any rate, 200 means "success" and, in fact, HTTPAPI should *never*
return 200 -- that would be a bug.

Here's the code I'm looking at from the end of do_post():

c                   if        rc<1 or wwFinRC=200
c                   return    rc
c                   else
c                   callp     SetError(wwErrorNo: wwErrorMsg)
c                   return    wwFinRC
c                   endif

Since rc can only be -1, 0 or 1, I don't see how it could ever return 200?
Please tell me how to reproduce this error.

Certainly in this case using a GET would be the prudent thing to do, but this is something one of our clients tried :< Which makes me wonder... is it even legal to specify parms on the URL if it's a post? Anyway, in this case if you specify some random post data the call does work... Here's the url:


It should yield a small XML document saying the merchant IP is incorrect. The 'problem' occurs in do_post right after the first call to RespProc:

1921.00 c                   eval      rc = RespProc(peSock: wwRespChain:
1922.00 c                                    %size(wwRespChain): 2: *ON)
1923.00 c                   if        rc<0
1924.00 c                   return    rc
1925.00 c                   endif
1926.00 c                   if        rc>0 and rc<>100
1927.00 c                   callp     SetError(HTTP_PST417:'Received HTTP '+
1928.00 c                              %editc(rc:'L') + 'during post request')
1929.00 c                   return    -1
1930.00 c                   endif

on line 1926 it does not account for a return code of 200, which is what the remote server returns if there is no post data... Hope this helps... I now realize my description may have been slightly inaccurate :< the return code is ultimately -1, but as a result of a 200 response code.

Thanks for your input!

Thank you for your comprehensive response!

Take Care,

      * https_init():  Initialize https (HTTP over SSL/TLS) protocol
      *     peAppID = application ID that you registered program as
      *        in the Digital Certificate Manager
      * MDH 20031008: When the initialization fails, it does not
      *               reset the environment handle, which means subsequent
      *               attempts will always fail even if the underlying
      *               problem was fixed.  Changed it to clear the
      *               environment handle on error
     P https_init      B                   export
     D https_init      PI            10I 0
     D  peAppID                     100A   const

     D rc              S             10I 0

     c                   callp     debug_msg('https_init(): entered')

     c                   if        wkEnvH <> *NULL
     c                   callp     SetError(HTTP_GSKENVI: 'SSL environment'+
     c                             ' was already initialized!')
     c                   return    -1
     c                   endif

     c                   eval      rc = gsk_environment_open(wkEnvh)
     c                   if        rc <> GSK_OK
     c                   callp     SetError(HTTP_GSKENVO: 'gsk_env_open: '+
     c                               ssl_error(rc))
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   endif

     C* tell OS/400 which application ID we use:
     c                   eval      rc = gsk_attribute_set_buffer(
     c                              wkEnvh: GSK_OS400_APPLICATION_ID:
     c                              %trimr(peAppID): 0)
     c                   if        rc <> GSK_OK
     c                   callp     SetError(HTTP_GSKAPPID:'Setting ID: ' +
     c                               ssl_error(rc))
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   endif

     C* tell GSKit that we're a client app:
     c                   eval      rc = gsk_attribute_set_enum(wkEnvh:
     c                               GSK_SESSION_TYPE: GSK_CLIENT_SESSION)
     c                   if        rc <> GSK_OK
     c                   callp     SetError(HTTP_GSKSTYP: 'Setting ' +
     c                             'session type: ' + ssl_error(rc))
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   endif

     C* Tell GSKit that we want passthru authentication:
     C*   FIXME: Using 'AUTH_PASSTHRU' means that we can start
     C*     even if some elements of the certificate chain are invalid.
     C*     We should really be checking to see what, if anything, is
     C*     not valid, and let the user know!
     c                   eval      rc = gsk_attribute_set_enum(wkEnvh:
     c                               GSK_CLIENT_AUTH_TYPE:
     c                               GSK_CLIENT_AUTH_PASSTHRU)
     c                   if        rc <> GSK_OK
     c                   callp     SetError(HTTP_GSKATYP: 'Setting ' +
     c                             'auth type: ' + ssl_error(rc))
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   endif

     C* Initialize the SSL environment.  After this, secure sessions
     C*   can be created!
     c                   eval      rc = gsk_environment_init(wkEnvh)
     c                   if        rc <> GSK_OK
     c                   if        rc = GSK_AS400_ERROR_NOT_REGISTERED
     c                   callp     SetError(HTTP_NOTREG: 'Application ' +
     c                             'is not registered with DCM!')
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   else
     c                   callp     SetError(HTTP_GSKATYP: 'gsk_env_init: '+
     c                                         ssl_error(rc))
      * MDH 20031008: Reset environment handle
     c                   eval      wkEnvH = *NULL
     c                   return    -1
     c                   endif
     c                   endif

     c                   return    0
     P                 E

     C* Number of control keys:
     c                   eval      p_NumKeys = %addr(wwBuf)
      * MDH 20031218 Make it three so that we can set it to a client
      *              app
     c*zapped            eval      wwNumKeys = 2
     c                   eval      wwNumKeys = 3

     C* First key is "limit CA" which we set to '0'
     c                   eval      p_RGAP_DS1 = %addr(wwBuf) + 4
     c                   eval      RGAP_DS1_VarRecLen = 13
     c                   eval      RGAP_DS1_AppCtrlKey = RGAP_LIMITCA
     c                   eval      RGAP_DS1_DataLen = 1
     c                   eval      %subst(RGAP_DS1_Data:1:1) = peLimitCA

     C* Next key is "replace" which we set to '1' so we can
     C*    run this code each time the program runs without
     C*    getting an error.
     c                   eval      p_RGAP_DS1= %addr(wwBuf) + 17
     c                   eval      RGAP_DS1_VarRecLen = 13
     c                   eval      RGAP_DS1_AppCtrlKey = RGAP_REPLACE
     c                   eval      RGAP_DS1_DataLen = 1
     c                   eval      %subst(RGAP_DS1_Data:1:1) = '1'

     C* Next key is "Application type" which we set to '2' so that
     C*    it knows that we are running as a client application
     c                   eval      p_RGAP_DS1= %addr(wwBuf) + 30
     c                   eval      RGAP_DS1_VarRecLen = 13
     c                   eval      RGAP_DS1_AppCtrlKey = RGAP_APPTYPE
     c                   eval      RGAP_DS1_DataLen = 1
     c                   eval      %subst(RGAP_DS1_Data:1:1) = '2'