Re: Combining large files
> cat file1 file2 file3 > new_file
If they're numbered sequentially, you could get away with:
cat file* >new_file
This assumes the names are 01 to 20 and NOT 1 to 20. Since the default
sort is by ASCII sequence. Assuming an ASCII based platform and other
defaults are in place. I join mpg and other files this way all the time.
This also assumes the new_file name differs enough from the file? name
that it doesn't get included in the cat portion. And that no other
extraneous files get grabbed by the wild card.
Otherwise DOS's:
copy file1/b+file2/b+file3/b new_file
under linux is roughly equal to:
cat file1 file2 file3 >new_file
You do NOT need to step it up like this:
cat file1 file2 >new_file1
cat new_file1 file3 >new_file2
cat new_file2 file4 >new_file1
cat new_file1 file5 >new_file
rm new_file1 new_file2
That would be very wasteful and slow. But I've known a few sadists in my
day who enjoyed typing and would do it that way.
One other limitation of sorts is that 32 bit processors are likely to
limit the maximum size of files to 2.1G. So you may be limited in only
forming your 20G file on a x86-64 or other 64+ bit platform.
HTH,
Shadow_7
|