mod rewrite: %20 -> space -> 400 Bad Request

Hello,

I'm using mod_rewrite with httpd 1.3.34 and am having some difficulties
with it when the URL that is getting rewritten has spaces (encoded as
%20) in it. Example:
http://simpy.com/user/otis/search/foo%20bar

The error I get is "400 bad Request":

Bad Request
Your browser sent a request that this server could not understand.
The request line contained invalid characters following the protocol
string.

My rewrite rule looks like this:

RewriteRule ^/user/(.*)/search/(.*)$
/simpy/User.do?username=$1&q=$2 [P,NE]


And what I see in my access_log is:
... "GET /simpy/User.do?username=otis&q=foo bar HTTP/1.1" ...


The problem seems to be that space in "foo bar".
I believe I need to tell httpd/rewrite module _not_ to decode the %20
from the original URL.

Does anyone know how I could configure mod_rewrite so that the %20 gets
preserved instead of converted to spaces?

Any help would be greatly appreciated.

Thanks.

OtisUsenet wrote:
> My rewrite rule looks like this:
>
> RewriteRule ^/user/(.*)/search/(.*)$
> /simpy/User.do?username=$1&q=$2 [P,NE]

You're using the Proxy-Flag for proxyPass here, but it looks like the
protocoll + host is missing in the substitution.

Try

RewriteRule ^/user/([^/]+)/search/([^/]*)$
/simpy/User.do?username=$1&q=$2 [L]

--
Robert

Robert Ionescu wrote:
> OtisUsenet wrote:
> > My rewrite rule looks like this:
> >
> > RewriteRule ^/user/(.*)/search/(.*)$
> > /simpy/User.do?username=$1&q=$2 [P,NE]
>
> You're using the Proxy-Flag for proxyPass here, but it looks like the
> protocoll + host is missing in the substitution.
>
> Try
>
> RewriteRule ^/user/([^/]+)/search/([^/]*)$
> /simpy/User.do?username=$1&q=$2 [L]

Hi,

Thanks for ([^]*) regexp, that one is indeed better!

However, substituting L in place of P doesn't work for me. L just
means this is the last rule, and from reading mod_rewrite docs, it
looks like using P implies that, too.

Maybe this is important to say - the above RewriteRule (without
http://server bits) works with my setup (Apache in front of Jetty
(listening on port 8009 (AJP13))), it's just that %20 in the request
URL gets converted to spaces, and that causes a "400 Bad Request"
error, which is what I'm trying to fix.

Any idea how I can avoid that conversion?
For example, this is what is currently happening:

http://example.com/user/johnny/search/apache%20httpd

gets translated to:

http://example.com/myapp/User.do?use...ohnny&q=apache httpd

NOTE those spaces between "apache" and "httpd". They cause the 400 Bad
Request error.


What I want is for this to happen:

http://example.com/user/johnny/search/apache%20httpd

gets translated to:

http://example.com/myapp/User.do?use...apache%20httpd

NOTE how the %20 should be left between "apache" and "httpd". That is
what I want. I don't want mod_rewrite to take the initial %20 and
change it to a space.

Thanks!

"OtisUsenet" <otis_usenet@yahoo.com> schreef in bericht
news:1132092398.966780.219000@f14g2000cwb.googlegr oups.com...
> However, substituting L in place of P doesn't work for me. L just
> means this is the last rule, and from reading mod_rewrite docs, it
> looks like using P implies that, too.
It does [at least] more than that, it also makes your server go and get that
[redirected] page for you.

> Maybe this is important to say - the above RewriteRule (without
> http://server bits) works with my setup (Apache in front of Jetty
> (listening on port 8009 (AJP13))),
That's about a proxy ... just like the [P] implies.

> 's just that %20 in the request
> URL gets converted to spaces, and that causes a "400 Bad Request"
> error, which is what I'm trying to fix.

> Any idea how I can avoid that conversion?
> For example, this is what is currently happening:
> http://example.com/user/johnny/search/apache%20httpd
> gets translated to:
> http://example.com/myapp/User.do?use...ohnny&q=apache httpd
> NOTE those spaces between "apache" and "httpd". They cause the 400 Bad
> Request error.
> What I want is for this to happen:
> http://example.com/user/johnny/search/apache%20httpd
> gets translated to:
> http://example.com/myapp/User.do?use...apache%20httpd
> NOTE how the %20 should be left between "apache" and "httpd". That is
> what I want. I don't want mod_rewrite to take the initial %20 and
> change it to a space.
A space in the query string should be escaped to a + ...
.... just to avoid breaking the request where space is a seperator ...
.... dropping the NoEscape flag might be a good thing:
RewriteRule ^/user/(.*)/search/(.*)$
/simpy/User.do?username=$1&q=$2 [L]
On failure try [PT,L]

Gray cells has started to hybernate must stop now ...

HansH

OtisUsenet wrote:
> Robert Ionescu wrote:
>> OtisUsenet wrote:
>>> My rewrite rule looks like this:
>>>
>>> RewriteRule ^/user/(.*)/search/(.*)$
>>> /simpy/User.do?username=$1&q=$2 [P,NE]
>> You're using the Proxy-Flag for proxyPass here, but it looks like the
>> protocoll + host is missing in the substitution.
>>
>> Try
>>
>> RewriteRule ^/user/([^/]+)/search/([^/]*)$
>> /simpy/User.do?username=$1&q=$2 [L]
>
> However, substituting L in place of P doesn't work for me. L just
> means this is the last rule, and from reading mod_rewrite docs, it
> looks like using P implies that, too.

Well, the P-flag forces proxy throughput, this means the substitution
will turn out in a new http request, but this time with the unescaped space.

Browser requests via GET: /user/otti/search/foo%20bar
So, the space here is escaped.

Mod_rewrite acts:
init rewrite engine with requested uri /user/otti/search/foo bar

applying pattern '^/user/(.*)/search/(.*)$' to uri
'/user/otti/search/boo bar'

rewrite /user/otti/search/foo bar ->
/test/index3.php?username=otti&q=boo bar

split uri=/test/index3.php?username=otti&q=boo bar ->
uri=/test/index3.php, args=username=otti&q=boo bar

forcing proxy-throughput with http://localhost:8080/test/index3.php

go-ahead with proxy request proxy:http://localhost:8080/test/index3.php [OK]

Since there is no protocol+host specified in the rewriteRule, it looks
like apache prefixes the URL with the current protocol+host
automatically. This proxy-throughput terminates the process with a 400
bad request, because mod_proxy requests via proxyPass now
via GET: /test/index3.php?username=otti&q=foo bar (unescaped). And
AFAICS this unescaped request by mod_proxy is causing the error.

Consider also
http://issues.apache.org/bugzilla/show_bug.cgi?id=29554
http://issues.apache.org/bugzilla/show_bug.cgi?id=34602

--
Robert

"HansH" <hans@niet.op.het.net> schreef in bericht
news:437a6821$0$11079$e4fe514c@news.xs4all.nl...
> "OtisUsenet" <otis_usenet@yahoo.com> schreef in bericht
> news:1132092398.966780.219000@f14g2000cwb.googlegr oups.com...
> > What I want is for this to happen:
> > http://example.com/user/johnny/search/apache%20httpd
> > gets translated to:
> > http://example.com/myapp/User.do?use...apache%20httpd
> > NOTE how the %20 should be left between "apache" and "httpd". That is
> > what I want. I don't want mod_rewrite to take the initial %20 and
> > change it to a space.
> A space in the query string should be escaped to a + ...
> ... just to avoid breaking the request where space is a seperator ...
> ... dropping the NoEscape flag might be a good thing:
> RewriteRule ^/user/(.*)/search/(.*)$
> /simpy/User.do?username=$1&q=$2 [L]
> On failure try [PT,L]
>
> Gray cells has started to hybernate must stop now ...
.... resuming

You cannot keep apache from unescaping ...
.... you then need to escape the value within the querystring
.... strictly a %20 for a space is incorrect, but it does work.
This should do
RewriteMap escape int:escape
RewriteRule ^/user/(.*)/search/(.*)$
/simpy/User.do?username=${escape:$1}&q=${escape:$2} [L]
add PT flag if needed


HansH