Re: question on ftp_list - get *next 100?

[Scott Klement <klemscot@xxxxxxxxxxxx>]

Sender: Scott Klement <klemscot@xxxxxxxxxxxx>



On Tue, 15 Oct 2002, Richard B Baird wrote:
>
> if    ftp_list(ftp: '': 100: %addr(Files): FilesFound) < 0
>
> but there are more to list, but I have no way of knowing for sure how many.

The "FilesFound" parameter tells you how many, doesn't it?

>
> I'd like to avoid making the array big enough to satisfy any number that
> might be returned.

Maybe you should use the FilesFound parameter to allocate chunk of memory
large enough to support any number of files, then?

>
> so my question (and maybe you, Scott are the best one to answer this),
>
> is there a way to modify FTP_list to say:  list 100 file names, starting at
> file name 101? or 201?  (i.e. passing in a position to field)
>

Sure it's possible...   There are two ways (that I can think of) to
implement such a thing, and both of them have problems.

Let's examine them, shall we?

1)  Each time you call FTP_list, it could allocate it's own temporary
    array to store the file list into.   It could either make the array
    large enough for "any" number of files to fit into it, or it could
    dynamically change the size of its array using the ALLOC / REALLOC
    op-codes.  Then, each time you called it, it could copy a section of
    it's array into your array.

    Problems:
      a) You end up allocating more memory this way, because both FTPAPI
         and your code are allocating space for the filenames.  Remember,
         you didn't want to have an array large enough for more than 100
         elements in the first place... now you need two arrays, one
         with more than 100 elements, and another with your original
         100 elements.   What have you saved?
      b) Because the initial call would need to do the network code,
         allocate memory, etc, you'd need seperate functions to
         start the list, read sections of the list, and clean up
         the memory.  in other words, you'd have to restructure it
         into FTP_OpenList, FTP_ReadList, FTP_CloseList.

2) The other alternative would be to ask the remote FTP server each time
     you called FTP_List to return the files.   If you asked for a list
     starting with number 201 (for example) then it would simply ignore
     the first 200 files returned, and start copying them into your array
     at position 201.

    Problems:
     1) You have to transfer the data from the FTP server twice, and
         this can potentially be slow.   Especially if the directory
         in question has a LOT of files (which, ironically, is where
         this type of function is the most useful)

     2) Because you'd be asking the FTP server more than once, the
         list may not end up being the same.   For example, if a
         file got deleted by another process between your first listing
         and your second listing, you might end up with a file which used
         to be in position 201 getting moved down to position 200.  This
         means that you could end up skipping a file.   Likewise, if
         a file was added in the middle, you could end up duplicating
         a file by moving it from 200 to 201.

It seems to me, that if your goal is to do it the way we're discussing as
#1, it'd be just as easy to do that inside your code by using FTP_listraw
and then just doing your own ALLOC/REALLOC/etc.

And if you wanted to use method #2, why not just call FTP_list, get the
size of the result, then call FTP_list again with a large enough array?

Or, you could make everything easier by using FTP_listraw in conjunction
with a file.  Simply write the entries to a file and not worry about it
overflowing an array...   Make it so that you'll never run out of space.

If you want to incorporate all of this into FTPAPI itself instead of your
application, feel free to write up the code, test it, and send it to me
and I'll include it in further distributions, and give you credit in
the source code.

That's the idea behind open-source, yes?   I give you my code for free,
and you test it and expand it and improve it for me as well as everyone
else...   as a community, we make all of our lives better :)


-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubsribe from the list send mail
to majordomo@xxxxxxxxxxxxx with the body: unsubscribe ftpapi mymailaddr
-----------------------------------------------------------------------