mod_rewrite question with directory indexing

I have a script that I use as a template for all the files I serve up.
I used mod_rewrite to apply this script to all html files that are
requested. However, I'm running into one issues - with directory
indexing. I have my server setup to look for index.html as the
directory index. If you go to http://server.com/directory/ it looks
for http://server.com/directory/index.html. The problem is, the
rewrite rule for the template script always gets run. I only want it
to apply if there exists a file. So if http://server.com/directory has
no index.html, I want it to just list the directory contents, not run
the template script. I've played with RewriteCond with the -f and -d
switches to test for existence of the index.html file but that doesn't
seem to quite work. Anyone have a similar setup/solution? Thanks

Q