[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Finding the size of a file prior to download

Hi Ian,
The HTTP protocol sends (amongst other things) a bunch of HTTP "headers" before sending the actual data. One of the headers is "content-length" and is followed by the total size (in bytes) of the document that's being sent. The only time when this header isn't present is when using "chunked" transfers. (Chunked is designed for situations when the total length of the document isn't known when the download begins. For example, when the document is being generated by a program on-the-fly, you may be downloading it as the document is still being generated, so the total size isn't known when you begin.)
The reason you'd use the 'HEAD' operation with WinInet is because HEAD will send you the headers without the data. So you can get the content-length header (and know the total size) without needing to begin the actual download. (Which you could then do in a subsequent step.)
This won't help you for chunked transfers, though. The size simply isn't known up-front when doing chunked transfers. 

-SK


On 9/15/2014 2:43 PM, Ian Patterson wrote:

Mike,

Thanks for the quick response.

I have gone through all of your suggestions already prior to leaning on the
httpapi community.
My program needs to do this in real time so don't want to go down the ftp
route. It's programmatical so can't 'see' the remote folder prior to
download.
My downloads are in real time so its mainly just to give the user some idea
if the download may take some time. It has no effect on the actual download.
But thanks.

Looking at the Windows API (WinInet) - (similar to httpapi) -  they suggest
using the http protocol 'HEAD' (instead of 'GET') but have no idea where
that fits in.

At the end of the day it’s a 'nice to have' but won't affect the
functionality of the system.

Thanks again
Regards
Ian Patterson -----Original Message----- 
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx
[mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Mike Krebs
Sent: 15 September 2014 19:43
To: HTTPAPI and FTPAPI Projects
Subject: RE: Finding the size of a file prior to download

You already figured out the problem...
If it is not chunked, you can parse the header for the file size. If it is
chunked the file size is determined on the fly.

So, an idea or two. Is the file sitting somewhere that you can see? Maybe
you can FTP or even http browse the directory and see the files and their
sizes? Obviously this would mean that the server would need that directory
in the ftp server config or for http would need some files to NOT be present
and some directives to be turned on. For Apache, see this page for that:

https://wiki.apache.org/httpd/DirectoryListings

If the file is being built on the fly, there just is not a good solution
except to turn off chunking.

I don't recall and don't have time to look it up but maybe you can "lie" to
the server and tell it you are using http 1.0 and see if it will buffer the
file for you. Then you could get the size in the header at the start of the
download.


-----Original Message-----
From: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx
[mailto:ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Ian Patterson
Sent: Monday, September 15, 2014 1:21 PM
To: 'HTTPAPI and FTPAPI Projects'
Subject: Finding the size of a file prior to download

We use httpapi to download files (binary and SSL).
It would be useful to know what the file size is prior to downloading it.

We use GET with and without chunking but can't see a method to establish the
download file size before the download.
Especially with chunked downloads where the content length is the chunk.

Any ideas anyone?
Regards Ian Patterson 
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------

-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------



-----------------------------------------------------------------------
This is the FTPAPI mailing list.  To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------