zip a tree file-wise?

Hi,

I want to do a backup on a directory tree. It appears to be a weak
solution to use 'tar czf' because I guess if a few bytes are broken of
the archive it want be readable at all anymore?! What I merely do is
to copy the whole directory tree to my backup disk. I wonder now if
one could zip the files while copying and keep the directory and file
structure intact. Just turning every file into a zipped version. How
would one do it?
I dont want to use gzip -r because then I need the full orignal space
on the target disk.
I thought of using find with the exec option but then I could not
figure out how to create the directory tree at the destination.
Does anyone know a trick?

Thanks,

Tobias

In article <1ju5u0dvta5df1efh4apue7kr2np1skbov@4ax.com> (Mon, 10 Jan 2005
23:08:20 +0100), Tobias Schenk wrote:

> I want to do a backup on a directory tree. It appears to be a weak
> solution to use 'tar czf' because I guess if a few bytes are broken of
> the archive it want be readable at all anymore?!

Since it's a single (tar) file being compressed, data beyond the error
might be unrecoverable.

> What I merely do is to copy the whole directory tree to my backup disk.

'cp -a SOURCE TARGET' ?

> I wonder now if one could zip the files while copying and keep the
> directory and file structure intact.

infozip? Decompressing ZIP files with errors is still a problem.

> I thought of using find with the exec option but then I could not
> figure out how to create the directory tree at the destination.
> Does anyone know a trick?

'cp --parents' ?

--
"Only vigorous government enforcement and judicially imposed restraints
will have any real chance to restrain Microsoft's ongoing unlawful
conduct."
-- Ed Black. Computer and Communications Industry Association. 5/29/03.

Tobias Schenk <schenk_remove_this_@physik.tu-berlin.de>:

> Hi,
>
> I want to do a backup on a directory tree. It appears to be a weak
> solution to use 'tar czf' because I guess if a few bytes are broken of
> the archive it want be readable at all anymore?!

That's right - the problem is however due to the gzip compression,
which is used by giving a "z" to tar. If you'd use bzip2 (i),
you would be able to recover a broken archive.

But I'd rather use cpio with CRC information.

Alexander Skwar
--
A prig is a fellow who is always making you a present of his opinions.
-- George Eliot
-------------------------------------------------------------------------

On Mon, 10 Jan 2005 15:20:15 -0700, Hamilcar Barca
<hamilcar@tld.always.invalid> wrote:

>In article <1ju5u0dvta5df1efh4apue7kr2np1skbov@4ax.com> (Mon, 10 Jan 2005
>23:08:20 +0100), Tobias Schenk wrote:
>
>> I want to do a backup on a directory tree. It appears to be a weak
>> solution to use 'tar czf' because I guess if a few bytes are broken of
>> the archive it want be readable at all anymore?!
>
>Since it's a single (tar) file being compressed, data beyond the error
>might be unrecoverable.
>
>> What I merely do is to copy the whole directory tree to my backup disk.
>
>'cp -a SOURCE TARGET' ?
>
>> I wonder now if one could zip the files while copying and keep the
>> directory and file structure intact.
>
>infozip? Decompressing ZIP files with errors is still a problem.
>
>> I thought of using find with the exec option but then I could not
>> figure out how to create the directory tree at the destination.
>> Does anyone know a trick?
>
>'cp --parents' ?


zipfix is NOT the final answer, or even the best one. If you're
really worried about damage to the archive file being reparable, look
into rar. The 'repair file, try harder' options are the single
advantage rar has over the other archive formats, IMHO.

Mike-

--
If you can keep your head while those around you are losing theirs...
You may have a great career as a network administrator ahead!
--
Please note - Due to the intense volume of spam, we have installed
site-wide spam filters at catherders.com. If email from you bounces,
try non-HTML, non-encoded, non-attachments,

Hi,

thanks for your answers so far. But I think I didnt make my point
clear enough.
What I finally want is a complete copy of my data tree, but every file
is zipped (for its own). Clearly, without zipping its cp -a or
tar cpsf - /data | tar xf -

What I would like is a method that pipes every file through a gzip,
compressing it and putting it then where it should be.
Something like cp -az .
I could do a cp -a and afterwards a
find /backup -type f -exec gzip '{}' ';' or similar. But for this I
already need the maximum space of du -hs /data.

I hope this makes it a bit clear. Nevertheless I thank yor for the
hints.

Tobias

Tobias Schenk <schenk_remove_this_@physik.tu-berlin.de> wrote:
> What I finally want is a complete copy of my data tree, but every file
> What I would like is a method that pipes every file through a gzip,
> compressing it and putting it then where it should be.

Not tested, but something like this should do it:

cd TOP_OF_DATA_TREE
find . ! -type f -depth |
cpio -pdm BACKUP_TREE
find . -type f |
while read F
do
gzip -f <"$F" >BACKUP_TREE/"$F".gz
chmod --reference="$F" BACKUP_TREE/"$F".gz
done


Caveats:
* Hard links will generate multiple copies of the same file
* Symbolic links will point to the uncompressed file name instead
of the gzipped one
* Files that are already compressed will still be re-compressed
* Error checking is pretty non-existent

In the interests of brevity (and, uh, laziness) I've omitted fixes to
these issues.

Chris

On Thu, 13 Jan 2005 at 10:06 GMT, Tobias Schenk wrote:
> Hi,
>
> thanks for your answers so far. But I think I didnt make my point
> clear enough.
> What I finally want is a complete copy of my data tree, but every file
> is zipped (for its own). Clearly, without zipping its cp -a or
> tar cpsf - /data | tar xf -
>
> What I would like is a method that pipes every file through a gzip,
> compressing it and putting it then where it should be.
> Something like cp -az .
> I could do a cp -a and afterwards a
> find /backup -type f -exec gzip '{}' ';' or similar. But for this I
> already need the maximum space of du -hs /data.

source=$1
dest=$2
cd $source || exit 5

## create directory hierarchy
find . -type d |
while read dir
do
mkdir -p "$dest/$dir"
done

## gzip files to new location
find . -type f |
while read file
do
gzip -c "$file" > "$dest/$file.gz"
done


--
Chris F.A. Johnson http://cfaj.freeshell.org/shell
================================================== =================
My code (if any) in this post is copyright 2004, Chris F.A. Johnson
and may be copied under the terms of the GNU General Public License

Chris F.A. Johnson wrote:
> On Thu, 13 Jan 2005 at 10:06 GMT, Tobias Schenk wrote:
>
>>Hi,
>>
>>thanks for your answers so far. But I think I didnt make my point
>>clear enough.
>>What I finally want is a complete copy of my data tree, but every file
>>is zipped (for its own). Clearly, without zipping its cp -a or
>>tar cpsf - /data | tar xf -
>>
>>What I would like is a method that pipes every file through a gzip,
>>compressing it and putting it then where it should be.
>>Something like cp -az .
>>I could do a cp -a and afterwards a
>>find /backup -type f -exec gzip '{}' ';' or similar. But for this I
>>already need the maximum space of du -hs /data.
>
>
> source=$1
> dest=$2
> cd $source || exit 5
>
> ## create directory hierarchy
> find . -type d |
> while read dir
> do
> mkdir -p "$dest/$dir"
> done
>
> ## gzip files to new location
> find . -type f |
> while read file
> do
> gzip -c "$file" > "$dest/$file.gz"
> done

What about device special files and anything else that's neither
directory nor regular file? I'd suggest adding:

find . ! -type d -a ! -type f |
while read file
do
cp -dp "$file" "$dest/$file"
done

The above is untested.

On Thu, 13 Jan 2005 at 18:21 GMT, John-Paul Stewart wrote:
> Chris F.A. Johnson wrote:
>> On Thu, 13 Jan 2005 at 10:06 GMT, Tobias Schenk wrote:
>>
>>>Hi,
>>>
>>>thanks for your answers so far. But I think I didnt make my point
>>>clear enough.
>>>What I finally want is a complete copy of my data tree, but every file
>>>is zipped (for its own). Clearly, without zipping its cp -a or
>>>tar cpsf - /data | tar xf -
>>>
>>>What I would like is a method that pipes every file through a gzip,
>>>compressing it and putting it then where it should be.
>>>Something like cp -az .
>>>I could do a cp -a and afterwards a
>>>find /backup -type f -exec gzip '{}' ';' or similar. But for this I
>>>already need the maximum space of du -hs /data.
>>
>>
>> source=$1
>> dest=$2
>> cd $source || exit 5
>>
>> ## create directory hierarchy
>> find . -type d |
>> while read dir
>> do
>> mkdir -p "$dest/$dir"
>> done
>>
>> ## gzip files to new location
>> find . -type f |
>> while read file
>> do
>> gzip -c "$file" > "$dest/$file.gz"
>> done
>
> What about device special files and anything else that's neither
> directory nor regular file? I'd suggest adding:
>
> find . ! -type d -a ! -type f |
> while read file
> do
> cp -dp "$file" "$dest/$file"
> done

Good point.

source=$1
dest=$2
cd $source || exit 5

find . |
while read file
do
if [ -d "$file" ]
then
mkdir -p "$dest/$file"
elif [ -f "$file" ]
then
case $file in
*gz) cp -pd "$file" "$dest/$file" ;;
*) gzip -c "$file" > "$dest/$file.gz" ;;
esac
else
cp -dp "$file" "$dest/$file"
fi
done


--
Chris F.A. Johnson http://cfaj.freeshell.org/shell
================================================== =================
My code (if any) in this post is copyright 2004, Chris F.A. Johnson
and may be copied under the terms of the GNU General Public License

On 13 Jan 2005 20:42:09 GMT, "Chris F.A. Johnson"
<cfajohnson@gmail.com> wrote:
Hi,

thank you all for your extensive help. :-)
I will try your suggestions asap.

Bye,
Tobias