orgone wrote:
> I recently rsync'd around 2.8TB between a RHE server (jfs fs) and a
> Netapps system. Did a 'du -sk' against each to verify the transfers:
>
> 2894932960 sources total, KB
> 2751664496 destination total, KB
>
> That's a 140GB discrepancy. Subsequent verbose rsyncs have turned up
> nothing that was not originally transferred.
>
> I often note similar behaviour with smaller transfers between servers
> with similar OS/fs combos and have always seen it to come extent with
> transfers between systems of any type. It's just that the usual
> discrepancies in this case are magnified greatly by the sheer volume of
> data. Needless to say, 140GB going missing would be a bit of a problem
> and it's not much fun picking through 2.8TB for MIA data.

Rsync has a "-c" option for producing checksums, I imagine that would
give me some reassurance that the transfer ocurred correctly. There is
also the "-v" verbose option as you noted.

To be certain I'd consider checksumming all the files on each system
(e.g. something like find mydirectory -exec sum {} \; > sysname.sums)
and use diff to compare the results. If really paranoid I'd use md5sum
instead of sum. I imagine this will take considerable time on 2.8TB so
I'd try it on small subsets first :-)