mod rewrite for 404's -- moving directories on complex config

Hi,

I have many directories from years ago on a fileserver. Many of them
are indexed on Google. I have five webservers reading from one
fileserver for the data. I'd like to keep copies of the most recent
data on the webservers and through mod_rewrite tell it to use local
copies instead of reading through the fileserver. I would like to move
most of my older directories into an "archived" directory.

If I move most of my older directories into an archived directory, the
Google listings will then 404. I could put a 404 doc up redirecting to
the index page, but Google will drop the link, which I would like to
avoid.

I could add a new RewriteCond / RewriteRule (below) for each new
directory. But, it would need to be added in the httpd.conf file for
each of the webservers and restarted. With the number of new
directories I add, this would be infeasible.

Does anyone know of a way to accomplish what I am trying to do? I do
not need the older directories in the "archived" directory to be
accessible or rewritten, I just don't want to lose the search engine
listings from them. And, I want to reduce the strain on the fileserver
for these files.

I also don't know of a way to say: rewrite everything locally unless
it is in the /dir1/archived directory and if there is a 404, the
directory probably existed at some point, redirect that to the index
and give a 200 status code.

Does anyone have any suggestions? I tried mod_cache but it doesn't
seem all that speedy.

Thanks,
Dave


RewriteEngine on
RewriteCond %{REQUEST_URI} ^/dir1/dir2
RewriteRule ^/dir1/dir2/(.*)$ /path/to/dir1-localcache/dir2/$1

On Apr 27, 3:35 pm, Davers <dszos...@gmail.com> wrote:
> Hi,
>
> I have many directories from years ago on a fileserver. Many of them
> are indexed on Google. I have five webservers reading from one
> fileserver for the data. I'd like to keep copies of the most recent
> data on the webservers and through mod_rewrite tell it to use local
> copies instead of reading through the fileserver. I would like to move
> most of my older directories into an "archived" directory.
>
> If I move most of my older directories into an archived directory, the
> Google listings will then 404. I could put a 404 doc up redirecting to
> the index page, but Google will drop the link, which I would like to
> avoid.
>
> I could add a new RewriteCond / RewriteRule (below) for each new
> directory. But, it would need to be added in the httpd.conf file for
> each of the webservers and restarted. With the number of new
> directories I add, this would be infeasible.
>
> Does anyone know of a way to accomplish what I am trying to do? I do
> not need the older directories in the "archived" directory to be
> accessible or rewritten, I just don't want to lose the search engine
> listings from them. And, I want to reduce the strain on the fileserver
> for these files.
>
> I also don't know of a way to say: rewrite everything locally unless
> it is in the /dir1/archived directory and if there is a 404, the
> directory probably existed at some point, redirect that to the index
> and give a 200 status code.
>
> Does anyone have any suggestions? I tried mod_cache but it doesn't
> seem all that speedy.
>
> Thanks,
> Dave
>
> RewriteEngine on
> RewriteCond %{REQUEST_URI} ^/dir1/dir2
> RewriteRule ^/dir1/dir2/(.*)$ /path/to/dir1-localcache/dir2/$1

you could try the !-f and !-d conditions to provoke the rewrite if and
only if the files arent present locally:
http://httpd.apache.org/docs/2.0/rew...ml#redirect404
or under "retrieve missing data from in tranet" a little durther on
within this section:
http://httpd.apache.org/docs/2.0/rew...d.html#content

they could both be adapted i think.