[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: HTTPAPI to SSL Website



Scott:

Well, that works just dandy! Amazing what happens when you use the RIGHT CODE! I guess a little humility is good for me now and then...

Follow-up to comments on comments:

GETTING ONLY THE LAST "GET" REQUEST IN THE DEBUG FILE: Upon further review, on this pass it appears to have only gotten the LAST request into the file, like you would expect. The debug file that I sent on 6/26 (if you still have it) APPEARED to have both requests in it, but unfortunately, I don't remember what the code looked like for that one.

I'm not going to lose any sleep over it, though, because otherwise this thing (HTTPAPI) works BEAUTIFUL.

Thank you again. I'll try to have more intelligent questions as I proceed with this!

Jon Juracich

________________________________________
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Juracich, Jon [Jon.Juracich@xxxxxxxxxxx]
Sent: Wednesday, July 02, 2008 9:29 AM
To: HTTPAPI and FTPAPI Projects
Subject: RE: HTTPAPI to SSL Website

Scott:

Well, the reason that I think I'm running beta code but really aren't is that I'm an idiot! I copy/pasted the RSTLIB command from your instructions to install the LIBHTTP library. Of course, the SAVF name is different, but I didn't change that. Duh.  I'll fix that and let you know how it works. Sorry about being stupid!

Comments on comments.

HTTPS_INIT(). OK - I was under the impression that I had to call HTTPS_INIT (even without parms) to initialize SSL. Since I already had a *SYSTEM store, etc. set up on our machine (from learning how to do CA 5250 SSL), I decided to go ahead and try it.

USING %PADDR() TO SPECIFY A CALLBACK: Cool. This is cut / paste from one of your examples, but I'll change it in the next prototype.

GETTING ONLY THE LAST "GET" REQUEST IN THE DEBUG FILE: Actually, I was only getting the FIRST request - the second request is the one that vanished. This seemed to be inconsistent, as it would sometimes show up, and sometimes not. Adding a second "http_debug_sts = *ON" SEEMED before the second HTTP request SEEMED to make a difference. Sometimes. Like you said, it could be part of the timing issues that you noted.

MY INCOMING ROUTINE: Like you said,  copy / paste of your routine, But I will implement your suggestions. Does INCOMING get called for each "chunk" of data that gets returned?

TIMED OUT MESSAGES and cookie control: Good ideas. I'll implement the COOKIE control.

Thank you again for your help on this, and I apologize for being an IDIOT and installing the wrong code. As you pointed out in your article this month, copy / paste can be a classic trap - and one that bit me in the butt again!

Jon Juracich

________________________________________
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Scott Klement [sk@xxxxxxxxxxxxxxxx]
Sent: Tuesday, July 01, 2008 7:26 PM
To: HTTPAPI and FTPAPI Projects
Subject: Re: HTTPAPI to SSL Website

Hi Jon,

This is the very first line of the http_debug.txt file that you sent
this afternoon:

    HTTPAPI Ver 1.23 released 2008-04-24

The version I want you to test with is this one:

    HTTPAPI Ver 1.24beta2 released 2008-07-01

Please make sure you're downloading and installing the copy from the
BETA directory on my web site -- not the current release.  You need to
try the beta version of the code.

http://www.scottklement.com/httpapi/beta/

That explains why the bug in my "beta1" wasn't visible in your earlier
debug file -- you weren't running the updated code!  It also explains
why you continue to get the same error, while mine is fixed...

Anyway, I ran your cflc_prot3 program, and I didn't get the timed out
error.  I do have some comments on it, however:

HTTPS_INIT:

You said you were surprised I didn't call https_init().  The
https_init() routine is only required if you need to associate your
program with a particular profile in the i5/OS Digital Certificate
Manager (DCM).  The advantage of using a profile this way is that you
can control exactly which certificate authorities your application
trusts, as well as supply a client-side certificate.   However, if
you're happy with trusting all CA's, and you don't need client
certificates (almost nobody does) then you can omit the call to
https_init().

One big advantage to leaving the https_init() off is that you don't have
to go into the DCM and create an application profile.  (You still have
to set up the *SYSTEM certificate store, but you can skip the
application profile).   That makes it a lot easier to distribute your
app.  For example, when you sent me your code, I had to either remove
the https_init() from your code, or I had to set up a new app profile in
my DCM named CADOJ_CFLC_RQSSHPVERIFY -- and since this is just a one-off
test (for me, that is) it seems silly to have to do that.


USING %PADDR() TO SPECIFY A CALLBACK

Starting in V4R4 (I think?) it's not necessary to do %paddr('INCOMING').
  When you have the name in quotes like that, it'll provide the address
of an actual export, much like extproc('INCOMING').   However, with V4R4
you can do %paddr(Incoming) and it won't be case-sensitive, and it will
determine the address from the Incoming prototype instead of the actual
ILE export.  I mainly like that it's not case-sensitive :)


GETTING ONLY THE LAST "GET" REQUEST IN THE DEBUG FILE:

Yes, the debug-file is per request.  However you could modify your code
to look like this:

        if %parms() >= 1;
          http_debug_sts = *ON;
          http_debug(http_debug_sts:'/tmp/log1.txt');
        endif;

And then before the 2nd call to http_url_get_raw(), you could code this:

        http_debug(http_debug_sts:'/tmp/log2.txt');

That way, the first request goes to a file named "log1" and the second
one goes to a file named "log2" so that you can go back and look at them
both.  There's still some weirdness with the timing of http_debug(),
though, I'll have to fix that for the next release (or beta)

But, as long as you only call http_debug() once per HTTP request (GET or
POST) it should work.



YOUR INCOMING ROUTINE:

Your incoming routine could be made much simpler if you used a VARYING
field for retdata.   I realize that you're just copying my code -- but
if I were to write that code again today, my incoming routine would look
like this:

      P incoming        B
      D incoming        PI            10I 0
      D   descriptor                  10I 0 value
      D   data                      8192A   options(*varsize)
      D   datalen                     10I 0 value
       /FREE
        retdatav += %subst(data:1:datalen);
        return datalen;
       /END-FREE
      P                 E

(where retdatav is a VARYING field).  It's just so much simpler...

TIMED OUT MESSAGES

I added code like this to go at the end of the program (with "wait"
defined as a 1A field):

        if %scan('timed': retdata) <> 0;
            dsply 'timed out message' ' ' wait;
        endif;

This immediately tells me if I got the timed out message. (I didn't)

I also added the following to the beginning of your program:

        if %parms() >= 2;
           http_use_cookies(*OFF);
        else;
           http_use_cookies(*ON);
        endif;

So this works very similarly to your code for enabling/disabling the
debug log.  When I pass two parms, it tells HTTPAPI to disable cookies
-- when cookies are disabled, the JSESSION isn't transmitted to the DOJ
website, and the DSPLY kicks in and says 'timed out message'.  When
cookies are turned on, the JSESSION is transmitted properly, and I don't
get the message.

I'm completely convinced at this point (from my tests) that the problem
is exactly what I originally suspected...  the DOJ web site is treating
the JSESSIONID cookie as case sensitive, in defiance of the standards.
(But since IE and Firefox both preserve the case, this doesn't cause
them any woes).  My updates to HTTPAPI cause HTTPAPI to also preserve
case (despite that it shouldn't be required according to the standards)
and that fixes the problem.



Juracich, Jon wrote:
> Scott:
>
> Alright, here we go...
>
> I installed the new beta LIBHTTP code prior to doing any of this.
>
> When I tried your sample program verbatim, I got the same results that I had been getting previously. That is, the second page that comes back is actually the "first" page with "<li>Your session has timed out.</li>" embedded in it.
>
> I also noticed that you're not initializing SSL, which I found interesting...
>
> When I tried MY program, dumping the output the debug file, the second fetch doesn't seem to be dumped to the DEBUG file. It looks like it tried to go get it, but the server returns 0 bytes available, or that DEBUG "magically" got turned off.
>
> Thinking that it might be caused by using a variable (rather than hard-coding) the URL, I tried that too (hard coding). Same result.
>
> I've included the files that your program generated, the DEBUG log from my program, and the source from my program. There's a little extra "stuff" in my program, as I'm trying to learn / build / think about finalized code / etc. as I go...
>
> From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Juracich, Jon [Jon.Juracich@xxxxxxxxxxx]
> Sent: Tuesday, July 01, 2008 3:44 PM
> To: HTTPAPI and FTPAPI Projects
> Subject: RE: HTTPAPI to SSL Website
>
> Scott:
>
> Thanks for continuing to look at this!
>
> After I had sent the program / described what it was doing (yesterday, actually), I changed it to perform a GET, rather than a POST. I did that based on watching the packets with the Firefox TamperData add in. Since I was using the non-beta version of your code, it did the same thing (session timeout).
>
> I'll get the beta code & try it again.
>
> Thanks again for your help on this - I really, REALLY appreciate it!
>
> Jon Juracich, IBM Certified Specialist: RPG IV Developer
> Sr. Staff Consultant, IBM Solutions Practice
>
> Affiliated, Inc.
> (614)889-6555, ext 355
>
> www.aresgrp.com
>
> Affiliated helps growing & Mid-Market organizations identify, evaluate, and implement solutions that use new & existing technology & resources that improve operational efficiency or increase revenue. Both make your organization more profitable and more successful.
> ________________________________________
> From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx [ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Scott Klement [sk@xxxxxxxxxxxxxxxx]
> Sent: Tuesday, July 01, 2008 2:59 PM
> To: HTTPAPI and FTPAPI Projects
> Subject: Re: HTTPAPI to SSL Website
>
> Hi Jon,
>
> I tried to write a quick program to test your CFLC web site, I did got
> it working (maybe... I'm not exactly sure what the program is supposed
> to do).  I noticed the following:
>
> a) You were sending a POST request (instead of a GET) for the
> addEnrolee.do page -- which might've caused the dispatch variable to not
> be found by the server, since the server was expecting a GET request.
>
> b) When I ran my own copy of the program I got a Cookie error in the
> debug log.  (This error wasn't in your log, which I can't explain!)  I
> traced the error to a bug in the "case-sensitivity" update that I sent
> you on June 26.
>
> c) So I fixed the bug in the case-sensitivity code, and now my program
> works!  Here's the code I'm testing with:
>
>
>       H DFTACTGRP(*NO) ACTGRP(*NEW) BNDDIR('LIBHTTP/HTTPAPI')
>
>        /copy httpapi_h
>
>       D rc              s             10i 0
>
>        /free
>         http_debug(*ON);
>
>         rc=http_url_get('https://webapps.doj.ca.gov/cflc/index.jsp'
>                        : '/tmp/deleteme');
>         if (rc<>1);
>           http_crash();
>         endif;
>
>         rc=http_url_get('https://webapps.doj.ca.gov/cflc/addEnrolee.do'
>                        + '?dispatch=enrollmentGuidelines'
>                        : '/tmp/deleteme');
>         if (rc<>1);
>           http_crash();
>         endif;
>
>         *inlr = *on;
>
>        /end-free
>
> To get my updated HTTPAPI (with the fix I made to the bug in the
> case-sensitivity code) please download the latest copy from
> http://www.scottklement.com/httpapi/beta
>
> I'm still confused about why the bug didn't show up in your log file
> (with a cookie error message).  Did you notice the problem and fix it on
> your end already?  That's all I can think of...  but it seems unlikely
> you'd do that without mentioning it... so I'm puzzled.
>
>
> Juracich, Jon wrote:
>> I don't know that I'm having a problem with the SSL part, per se.
>> It's more like what I'm trying to send (the session cookie value, for
>> the second page), or maybe what I'm not sending. I've used a couple
>> of the tools available for Firefox (Firebug and TamperData) but I
>> don't really know how to read what I'm seeing. Using TamperData, it
>> LOOKS like Firefox is doing a straight GET to retrieve the second
>> page. Does that involve sending anything up to the server, other than
>> the "GET" request? If it does, and HTTPAPI doesn't do that, I would
>> think that I would have to "send" something with the cookie value.
>> Maybe.
>>
>> I've also used the HTTPAPI debug tool to dump the communications out
>> to a file. That, I can kind of read (but not necessarily completely
>> understand!).
>
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list.  To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------

-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------