[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Ftpapi] Ideas for New HTTPAPI functions
Hi Scott,
The advantage of the first idea is its flexibility. You want to pass the
POST data as a buffer (pointer)? No problem. Just create a
"BufferReader" with the related methods and you are done.
The idea was inspired by Java. Of course the different readers and
writers are different "classes" but they all share a common interface
with the following methods:
Readers:
open()
read()
close()
Writers:
open()
write()
close()
HTTPAPI calls the methods of the interface by pointers. If I had to do
it, I would create modules or even service programs for each reader/writer.
Regards,
Thomas.
Am 27.02.2017 um 06:22 schrieb Scott Klement:
> Thomas,
>
> Lots of good ideas, here. I also like the original idea -- at first, it
> sounded complicated, but the more that I think about it, the more I like
> it.
>
> I will definitely consider these ideas, thanks for the feedback.
>
> Everyone else: Please look over my ideas and the ones that Thomas sent,
> and give me your thoughts. Which sounds the best to you?
>
> -SK
>
>
> On 2/26/2017 11:52 AM, Thomas Raddatz wrote:
>> Hi Scott,
>>
>> Now I understand your intention better. Here is another idea to consider.
>>
>> What about stripping the procedure interface down to the following
>> parameters and using a type prefix similar to the URL protocol like this:
>>
>> http_req(Method: Url: ReceivedData: [SendData);
>>
>> Possible type identifiers:
>>
>> string:// - The data that follows the type identifier is sent
>> to the server.
>>
>> file:// - The value following the type identifier specifies
>> a file name. The data of the file identified by
>> that file name is send to the server.
>>
>> The default type identifier is 'string://'.
>>
>> ReceivedData
>>
>> Data that is received from the server. By default the received data is
>> stored as a string value in the "ReceivedData" field. The 'file://' type
>> identifier can be used to save the data to the specified file.
>>
>> Examples:
>>
>> http_req('GET': 'http://example.com/test.txt':
>> 'file:///home/joe/receivedData.txt');
>>
>> http_req('GET': 'http://example.com/test.txt':
>> receiveBuffer);
>>
>> "receiveBuffer" could be empty, to use the 'string://' default value or
>> could specify 'string://'.
>>
>> SendData
>>
>> Data that is send as POST data to the server. By default the content of
>> field "SendData" is sent to the server. The 'file'//' type identifier
>> can be used to read the data from the specified file.
>>
>> Examples:
>>
>> http_req('POST': 'http://example.com/test.txt':
>> receiveBuffer:
>> sendBuffer);
>>
>> http_req('POST': 'http://example.com/test.txt':
>> receiveBuffer:
>> 'My POST data');
>>
>> http_req('POST': 'http://example.com/test.txt':
>> 'file:///home/joe/receivedData.txt':
>> 'string://My POST data');
>>
>> http_req('POST': 'http://example.com/test.txt':
>> 'file:///home/joe/receivedData.txt':
>> 'file:///home/joe/postData.txt');
>>
>> Just another idea to make it easy and to stay flexible.
>>
>> Regards,
>>
>> Thomas.
>>
>> Am 26.02.2017 um 09:08 schrieb Scott Klement:
>>> Thomas,
>>>
>>> I have not designed new "raw" routines because I think the existing
>>> http_persist_req() routine provides that functionality already.
>>> Passing a fd doesn't make sense, since I do not have a callback option!
>>> HTTPAPI internally manages the file when writing so there's no need for
>>> an fd to be passed along.
>>>
>>> Your idea of providing http_StmfWriter/http_StringWriter, etc would make
>>> sense for the "raw" routines, but I am looking for really simple
>>> solutions, and this seems to add a level of complexity. Someone who
>>> wants this level of complexity can achieve it by using the "raw" type of
>>> routines...
>>>
>>> -SK
>>>
>>>
>>> On 2/25/17 8:57 AM, Thomas Raddatz wrote:
>>>> Hi Scott,
>>>>
>>>> I agree with you and I like your idea of changing/enhancing HTTPAPI.
>>>>
>>>> When I understood you correctly, then the procedure interface shall be
>>>> changed to that:
>>>>
>>>> http_req(Method: Url: [RecvStmf: [RecvString: [SendStmf: [SendString);
>>>>
>>>> But what about the "file/socket descriptor" option that we now have
>>>> with
>>>> the "raw/raw2" methods?
>>>>
>>>> Following your idea that would lead to the following method signature:
>>>>
>>>> http_req(Method: Url:
>>>> [RecvStmf: [RecvString: [RecvFD: [SendStmf: [SendString: [SendFD);
>>>>
>>>> Even without the additional FD parameters the user has to know that he
>>>> must not use all parameters but just a subset. He has to know that he
>>>> must specify "RecvStmf" exclusive or "RecvString" and that there are
>>>> two
>>>> more optional parameters which are used exclusive or, too.
>>>>
>>>> Also not very intuitive from my point of view.
>>>>
>>>> Specific methods names are also not an option, because that would
>>>> led to
>>>> a whole bunch of methods like this:
>>>>
>>>> http_reqString() - Receive string
>>>> http_reqStringString() - Receive string, send string
>>>> http_reqStringStmF() - Receive string, send stream file
>>>> http_reqStmf() - Receive stream file
>>>> http_reqStmfString() - Receive stream file, send string
>>>> http_reqStmfStmF() - Receive stream file, send stream file
>>>>
>>>> Now imagine what happened when we would add the "file/socket
>>>> descriptor"
>>>> parameters. Most ugly approach.
>>>>
>>>> But what about the following:
>>>>
>>>> http_req(Method: Url: Writer: Reader);
>>>>
>>>> "Writer" could be:
>>>>
>>>> StringWriter - Stores the data in a string
>>>> StmfWriter - Stores the data in a stream file
>>>> DescriptorWriter - Writes the data to a given file/socket descriptor
>>>>
>>>> "Reader" could be:
>>>>
>>>> StringReader - Reads the data from a string
>>>> StmfReader - Reads the data from a stream file
>>>> DescriptorReader - Reads the data from a given file/socket
>>>> descriptor
>>>>
>>>> HTTPAPI could expose the readers and writers as a large strings (data
>>>> structures) with a type field at the first position. So the procedure
>>>> interface of http_req() could be similar to this:
>>>>
>>>> d http_req...
>>>> d pr
>>>> d i_method 10a varying const
>>>> d i_url 1024a varying const
>>>> d i_genericReader...
>>>> d 5000a options(*varsize)
>>>> d i_genericWriter...
>>>> d 5000a options(*varsize: *nopass)
>>>>
>>>> A user could call http_req() like this:
>>>>
>>>> http_req('GET': 'http://...': http_StringWriter(responseData));
>>>>
>>>> For POST data it could look like this:
>>>>
>>>> http_req('POST': 'http://...':
>>>> http_StmfWriter(filename: ccsid: replace):
>>>> http_StringReader('Hello World, my post data'));
>>>>
>>>> And for raw data it could go like this:
>>>>
>>>> http_req('POST': 'http://...':
>>>> http_RawWriter(%paddr('myOpen'): %paddr('write'): %paddr('close')):
>>>> http_RawReader(%paddr('myOpen'): %paddr('read'): %paddr('close')));
>>>>
>>>> Perhaps it might be useful to add a optional "userData" parameter to
>>>> http_RawWriter() and http_RawReader().
>>>>
>>>> http_req() could first check the "type" field of the data structure
>>>> created by e.g. http_StmfWriter() and then set up the open(), write()
>>>> and close() procedures accordingly:
>>>>
>>>> d genRdWr ds qualified based(pGenRdWr)
>>>> d type 10a
>>>>
>>>> pGenRdWr = %addr(i_genericReader);
>>>>
>>>> select;
>>>> when (genRdWr.type = '*STRING');
>>>> reader.pOpen = %paddr('StringReader_open');
>>>> reader.pRead = %paddr('StringReader_read');
>>>> reader.pClose = %paddr('StringReader_close');
>>>> when (genRdWr.type = '*STMF');
>>>> reader.pOpen = %paddr('StmfReader_open');
>>>> reader.pRead = %paddr('StmfReader_read');
>>>> reader.pClose = %paddr('StmfReader_close');
>>>> when (genRdWr.type = '*FD');
>>>> reader.pOpen = %paddr('GenericRawReader_open');
>>>> reader.pRead = %paddr('GenericRawReader_read');
>>>> reader.pClose = %paddr('GenericRawReader_close');
>>>> when (genericSndRcv.type = '???');
>>>> ...
>>>> other;
>>>> // send escape message
>>>> endsl;
>>>>
>>>> StringReader:
>>>>
>>>> Most likely open(open() and close() do nothing and read() returns the
>>>> whole string at once. But of course it could also initialize an
>>>> internal
>>>> "offset" field in open() and then return the string chunk by chunk in
>>>> read().
>>>>
>>>> StmfReader:
>>>>
>>>> I assume that this reader could just pass the open, read and close
>>>> requests to the Unix APIs open(), read() and close().
>>>>
>>>> GenericRawReader:
>>>>
>>>> Most likely this reader has to use a user defined open procedures and
>>>> could, in a lot of cases, use the Unix APIs read() and close() if the
>>>> user did not specify these procedures at http_RawReader().
>>>>
>>>> The writer procedures would work similar to the reader procedures
>>>> that I
>>>> just explained.
>>>>
>>>> Regards,
>>>>
>>>> Thomas.
>>>>
>>>> Am 24.02.2017 um 23:54 schrieb Scott Klement:
>>>>> Everyone,
>>>>>
>>>>> For a long time, I've been wanting to find a better way to interface
>>>>> with the HTTP protocol that HTTPAPI provides... currently, most people
>>>>> are using these routines:
>>>>>
>>>>> http_url_get(), http_url_get_raw(), http_url_get_xml()
>>>>>
>>>>> http_url_post(), http_url_post_raw(), http_url_post_xml()
>>>>>
>>>>> While these provide access to the GET and POST methods, I see several
>>>>> big problems with them:
>>>>>
>>>>> 1) There are too many parameters for things like timeout, user agent,
>>>>> soap action, etc. To me, this makes the code awkward.
>>>>>
>>>>> 2) There is no really easy way to simply get the result as a
>>>>> string. My
>>>>> original design in 2001 was that a callback approach would be very
>>>>> versatile -- and it is -- because you can code any logic you want for
>>>>> where to store the data when it comes in. But in perhaps 95% of the
>>>>> cases, you want it in a string or in a stream file. So having to code
>>>>> the logic to write to a string every time seems cumbersome.
>>>>>
>>>>> 3) Only supporting GET/POST is limiting, especially with REST.
>>>>>
>>>>> 4) Automatically calling an XML parser is kinda neat, but not really
>>>>> required anymore, since RPG now has one built-in. Plus, XML is fading
>>>>> in popularity in favor of JSON. People get confused when they see XML
>>>>> routines but no routines for other formats like JSON, MIME or
>>>>> URL-encoded forms. They think HTTPAPI is limited, when in fact, it
>>>>> was
>>>>> designed to be as versatile as possible.
>>>>>
>>>>> So I was thinking of a new interface that, really, is one core
>>>>> routine:
>>>>>
>>>>> http_req() = make an HTTP request.
>>>>>
>>>>>
>>>>> Parameters would be:
>>>>>
>>>>> method = HTTP method to use (GET, POST, PUT, DELETE, etc)
>>>>> URL = the URL, of course
>>>>>
>>>>>
>>>>> Where to put the response... two parmeters for that:
>>>>>
>>>>> RecvStmf = If you want the response put into a stream file (IFS) then
>>>>> you pass this, otherwise use *OMIT
>>>>> RecvString = If you want the response put into a string, you pass
>>>>> this,
>>>>> Otherwise *OMIT
>>>>>
>>>>>
>>>>> Where to get request body from.. only used for POST/PUT
>>>>> methods,
>>>>> otherwise you can leave them off
>>>>>
>>>>> SendStmf = stream file to send request body from, or *omit to send
>>>>> from
>>>>> a string
>>>>> SendString = string to send request body from, or *omit to send from
>>>>> a stmf
>>>>>
>>>>> For body the body and the response, you can either use one or the
>>>>> other,
>>>>> not both. This might lead to confusion? But, it surely would be
>>>>> simpler than what we have now. It'd look like this:
>>>>>
>>>>> req = '<something>some request to send</something>';
>>>>> http_setOption('content-type': 'text/xml');
>>>>> rc = http_req('POST': 'http://whatever': *omit: resp: *omit: req);
>>>>>
>>>>> // resp now contains the response document (XML, JSON, HTML,
>>>>> whatever it
>>>>> is.)
>>>>>
>>>>> The http_setOption() routine allows you to set any of the options that
>>>>> might've been extra parameters before.
>>>>>
>>>>> We could also have some "simpler" wrappers around this routine for
>>>>> working with strings and/or stream files. I'm thinking that the
>>>>> wrappers would (a) send an exception message for errors, so rc isn't
>>>>> needed, (b) only accept one parameter for data to send, and one to
>>>>> receive. There'd be two wrappers, one for using strings, one for
>>>>> using
>>>>> files.
>>>>>
>>>>> Resp = http_string('GET': 'http://whatever');
>>>>>
>>>>> Resp = http_string('POST': 'https://foo.bar': dataToSend);
>>>>>
>>>>> http_stmf('GET': 'http://whatever': '/home/scott/myresponse.xml');
>>>>>
>>>>> http_stmf('PUT': 'http://foo': '/tmp/response.dat':
>>>>> '/tmp/request.dat');
>>>>>
>>>>>
>>>>> All the old routines would be retained, of course, so existing
>>>>> programs
>>>>> would work without problems. Instead of using the automatic XML
>>>>> parsing, you'd just receive your data to a string, and then parse the
>>>>> string afterwards. (using RPG's built-ins, or HTTPAPIs
>>>>> http_parse_xml_String routine, or SQL's XML stuff) Likewise for
>>>>> JSON you
>>>>> could use YAJL, SQL, or your favorite 3rd party tool. It's a string,
>>>>> right? You can feed it into anything.
>>>>>
>>>>> Let me know your thoughts!
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ftpapi mailing list
>>>>> Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
>>>>> http://scottklement.com/mailman/listinfo/ftpapi
>>>>>
>>>> _______________________________________________
>>>> Ftpapi mailing list
>>>> Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
>>>> http://scottklement.com/mailman/listinfo/ftpapi
>>>>
>>> _______________________________________________
>>> Ftpapi mailing list
>>> Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
>>> http://scottklement.com/mailman/listinfo/ftpapi
>>>
>> _______________________________________________
>> Ftpapi mailing list
>> Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
>> http://scottklement.com/mailman/listinfo/ftpapi
>>
>
> _______________________________________________
> Ftpapi mailing list
> Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
> http://scottklement.com/mailman/listinfo/ftpapi
>
_______________________________________________
Ftpapi mailing list
Ftpapi@xxxxxxxxxxxxxxxxxxxxxx
http://scottklement.com/mailman/listinfo/ftpapi