URI::InvalidURIError (bad URI(is not URI?):

[technion]

Hi,

I know you've seen similar errors, I'm hoping this is somewhat different.
(I'm trying to fetch a virus infected website so please be careful with these links)

Consider the following:

irb(main):021:0> HTTParty.get("https://therapie-schorndorf.de/%2001.zip")
Traceback (most recent call last):
       12: from /usr/bin/irb:23:in `<main>'
       11: from /usr/bin/irb:23:in `load'
       10: from /usr/lib/ruby/gems/2.6.0/gems/irb-1.2.4/exe/irb:11:in `<top (required)>'
        9: from (irb):21
        8: from /usr/lib/ruby/gems/2.6.0/gems/httparty-0.18.1/lib/httparty.rb:627:in `get'
        7: from /usr/lib/ruby/gems/2.6.0/gems/httparty-0.18.1/lib/httparty.rb:508:in `get'
        6: from /usr/lib/ruby/gems/2.6.0/gems/httparty-0.18.1/lib/httparty.rb:594:in `perform_request'
        5: from /usr/lib/ruby/gems/2.6.0/gems/httparty-0.18.1/lib/httparty/request.rb:159:in `perform'
        4: from /usr/lib/ruby/gems/2.6.0/gems/httparty-0.18.1/lib/httparty/request.rb:301:in `handle_host_redirection'
        3: from /usr/lib/ruby/2.6.0/uri/common.rb:234:in `parse'
        2: from /usr/lib/ruby/2.6.0/uri/rfc3986_parser.rb:73:in `parse'
        1: from /usr/lib/ruby/2.6.0/uri/rfc3986_parser.rb:67:in `split'
URI::InvalidURIError (bad URI(is not URI?): "https://www.therapie-schorndorf.de/ 01.zip")

This doesn't appear to be directly an issue with Ruby's parser:

irb(main):024:0> URI.parse("https://therapie-schorndorf.de/%2001")
=> #<URI::HTTPS https://therapie-schorndorf.de/%2001>

But more importantly, it doesn't appear to be an issue with the space. This is fine:

irb(main):025:0> HTTParty.get("https://google.com/%2001.zip")

I seem to be going in circles getting to the bottom of what exactly is invalid about this. I'm sure this used to work as I have hardcoded URIs in my app and my app worked a few months back. Any assistance appreciated.

[luiseugenio]

Did you try to remove the %20?

HTTParty.get("https://therapie-schorndorf.de/2001.zip")

[technion]

Can confirm without that space it works fine.

However, many other URLs with a space also work fine. This works (of course it's a 404):

HTTParty.get("https://github.com/jnunemaker/httparty/issu%20es")

I've also noted this gives you a DNS lookup error - which appears to suggest it parsed the URI fine:

 HTTParty.get("https://therapie-schorndorf.com/%2001.zip")

I've tried a series of alternatives but can't find a clear correlation as to what is wrong with the given URI.

[luiseugenio]

Can confirm without that space it works fine.

However, many other URLs with a space also work fine. This works (of course it's a 404):

HTTParty.get("https://github.com/jnunemaker/httparty/issu%20es")

I've also noted this gives you a DNS lookup error - which appears to suggest it parsed the URI fine:

 HTTParty.get("https://therapie-schorndorf.com/%2001.zip")

I've tried a series of alternatives but can't find a clear correlation as to what is wrong with the given URI.

I think there is a correlation between the space (%20) being right after /. It doesn't make much sense for this path to exist this way.

[technion]

Unfortunately this is also a bad URI:

HTTParty.get("https://therapie-schorndorf.de/a%2001.zip")

[luiseugenio]

Unfortunately this is also a bad URI:

HTTParty.get("https://therapie-schorndorf.de/a%2001.zip")

@technion did you try something like this:

url = "https://therapie-schorndorf.de/a%2001.zip"
encoded_url = URI.encode(url)
HTTParty.get(encoded_url)

It works for me.

[technion]

That "works" but i I believe that's the wrong URL. When run against my own domain and reviewing my own web server logs, it's encoded the % to %25 and then literally searched the file system for a file named a%2001.zip as opposed to "a 01.zip".