peter pilsl wrote:
>
> How do you transfer huge files (2-18Gigs) between two hosts?
> It needs to be secure and it must offer a REGET-possibility (continue
> download after the transfer was interrupted)
>
> My first attempt was simply using https, which turned out to be
> difficult, cause apache does not support big files. One can recompile it
> to include big-file support, but there seem to be several problems and I
> really dont want to go out to 10machines and recompile apache just for
> this purpose. (and one would need to recompile all modules as well,
> which always is a pain in the back)
>
> Second attempt was scp, which does not support REGET.
>
> Third attempt was ftp via a ssh-tunnel. I didnt get it to work.
> (ssh -L 4021:localhost.21 user@remote.host -> passive -> dir ->
> connection refused) and I remember always having troubles ftp via ssh.
>
> Fourth attempt was psftp (there is a unix-port of putty), which supports
> REGET (great tool : psftp) but does not support big files either. 2GB
> ist the limit here as well.
>
>
> Fifth attempt was rsync via ssh, which takes a *very* long time to
> determine the needed delta-pieces and then fails anyway (there was
> enough space on the targetdrive ...)
> write failed on "/extra/monthly.imap.00-06-15.tgz": No space left on device
> rsync error: error in file IO (code 11) at receiver.c(305)
> rsync: connection unexpectedly closed (77 bytes read so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(189)
>
> I tried with different checksum-blocksizes, but always get the same result.
> Rsync is an overkill anyway, cause it assumes possible changes all over
> the file and not only at the end.
>
>
>
> My last attempt will be to invoke dd or split to split the file to
> chunks<2GB and transfer them seperately. On the other hand: we almost
> have 2005 and I'm sure there is a solution to do it in once and I just
> dont think about it by now.
>
> thnx,
> peter
>
>
>
>
How about using something like gtk-gnutella, or the torrent over ssl?
Both of these support very large files (eg. DVD's, distro's) and give
the capability to serve the files from multiple machines as well as
restartability and automatic reassembly.