Re: Buffer size?

[Scott Klement <sk@xxxxxxxxxxxxxxxx>]

Hi David,

>
> 1. I'm an old hand on the AS/400 but an acolyte in RPG and learning
> as I go. Apologies for any obvious no-no's.
>

What surprised me was not that you are "an old dog learning new tricks" 
(which is commonplace) but rather than you seem to have picked the %STR 
BIF out of a hat.

None of my examples use that (because it's not an appropriate tool) so 
it was a big surprise to see it used, here.

Just to be clear, the problem with %STR() is related to the fact that 
you can potentially get 'garbage' (unwanted characters, not part of the 
XML stream) by using this on the value returned by HTTPAPI.

>
> 2. The length of the XML stream that I'm accepting is potentially
> large.
>

What is meant by "the stream" in this sentence?  For example, if you had 
this:

<mydoc>
    <elem1>...10000 bytes here...</elem1>
    <elem1>...10000 bytes here...</elem1>
    <elem1>...10000 bytes here...</elem1>
</mydoc>

This XML document is more than 30000 bytes long, right?  But, the 
largest element ("tag") of the document is only 10000 bytes long.  When 
you say "stream" are you referring to the total size of the document, or 
are you referring to the maximum size of a particular element?

HTTPAPI's XML parser can handle a total document size that's gigabytes 
long... maybe even terabytes long.... Honestly, I'm not sure what the 
limit is.

But for the size of a single element, the limit is smaller.  With 
HTTP_XmlReturnPtr(*OFF) the limit per element is 65535.  (But you can 
have thousands and thousands of elements that are that long without 
problem.)

With HTTP_XmlReturnPtr(*ON) the limit is just under 16 mb per element. 
Again... emphasis on "per element", not the entire document.

Typically, something like XmlReturnPtr(*ON) is only used when a single 
element is going to have tons of data... for example, sometimes folks 
will embed an entire picture or PDF document inside a single XML tag, 
and base64 encode it, so the contents of that one element ("tag") might 
be megabytes long.

For normal XML documents, something like <name>Scott Klement</name> 
would never have enything in excess of 64k, so it doesn't matter.

In your code, you are copying the data to a 32767A varying string -- so 
even if you coded your program to use the pointer correctly, it's going 
to discard anything in excess of 32767 in a particular element.


> I found some old references to examples where a pointer has
> been used in cases where this was so, thus thought using a pointer
> might facilitate easier handling.

You are welcome to use pointers if you understand them and can maintain 
your code that way.  It doesn't matter to me.

My objection is to the %STR BIF, not to pointers.

>
> Regarding the length, since I may need to parse a large stream , what
> approach would you recommend ? From the log, I see the stream being
> cut into chunks of size 8192, however without experimentation, I'm
> unsure if this means this is going to be handled through multiple
> submissions to my 'incoming ' routine of if the stream will appear
> seamless within the RPG, in which case, how to access the data.
>

You are confusing two different things, here.  The 8192 long buffer is 
used internally within HTTPAPI to optimize reading from the disk or 
network. It has nothing to do with the length of data that gets passed 
to your 'incoming' procedure.
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------