Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix encoding issues #17

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix encoding issues #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee47367
Do not force content string encodings, keep them as they are
Oct 2, 2014
8ed777b
Bump version
Oct 2, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions lib/simple-rss.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,13 +151,13 @@ def clean_tag(tag)
tag.to_s.gsub(':','_').intern
end

def unescape(content)
if content.respond_to?(:force_encoding) && content.force_encoding("binary") =~ /([^-_.!~*'()a-zA-Z\d;\/?:@&=+$,\[\]]%)/n then
CGI.unescape(content).gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
else
content.gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
end
end
def unescape(content)
if content =~ /([^-_.!~*'()a-zA-Z\d;\/?:@&=+$,\[\]]%)/ then
CGI.unescape(content)
else
content
end.gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
end
end

class SimpleRSSError < StandardError
Expand Down
4 changes: 2 additions & 2 deletions simple-rss.gemspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Gem::Specification.new do |s|
s.name = "simple-rss"
s.version = "1.2.3"
s.version = "1.3.2"
s.version = "#{s.version}-alpha-#{ENV['TRAVIS_BUILD_NUMBER']}" if ENV['TRAVIS']
s.date = "2010-07-06"
s.date = "2014-10-03"
s.summary = "A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. It is designed to be backwards compatible with the standard RSS parser, but will never do RSS generation."
s.email = "[email protected]"
s.homepage = "http://github.com/cardmagic/simple-rss"
Expand Down
11 changes: 11 additions & 0 deletions test/base/base_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# -*- coding: utf-8 -*-
require 'test_helper'
class BaseTest < Test::Unit::TestCase
def setup
@rss09 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss09.rdf')
@rss20 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss20.xml')
@rss20_utf8 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss20_utf8.xml')
@media_rss = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/media_rss.xml')
@atom = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/atom.xml')
end
Expand Down Expand Up @@ -69,4 +71,13 @@ def test_atom
def test_bad_feed
assert_raise(SimpleRSSError) { SimpleRSS.parse(open(File.dirname(__FILE__) + '/../data/not-rss.xml')) }
end

def test_rss_utf8
assert_equal 2, @rss20_utf8.items.size
assert_equal "SC5 Blog", @rss20_utf8.title
assert_equal Encoding::UTF_8, @rss20_utf8.title.encoding
item = @rss20_utf8.items.first
assert_equal "Mitä asiakkaamme ajattelevat meistä?", item.title
assert_equal Encoding::UTF_8, item.title.encoding
end
end
61 changes: 61 additions & 0 deletions test/data/rss20_utf8.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
>

<channel>
<title>SC5 Blog</title>
<atom:link href="http://sc5.io/blog/feed/" rel="self" type="application/rss+xml" />
<link>http://sc5.io/blog</link>
<description>HTML5 expertise</description>
<lastBuildDate>Wed, 01 Oct 2014 09:52:48 +0000</lastBuildDate>
<language>fi-FI</language>
<sy:updatePeriod>hourly</sy:updatePeriod>
<sy:updateFrequency>1</sy:updateFrequency>
<generator>http://wordpress.org/?v=4.0</generator>
<item>
<title>Mitä asiakkaamme ajattelevat meistä?</title>
<link>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/</link>
<comments>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/#comments</comments>
<pubDate>Mon, 29 Sep 2014 13:19:37 +0000</pubDate>
<dc:creator><![CDATA[Saskia Salomaa]]></dc:creator>
<category><![CDATA[HTML5]]></category>
<category><![CDATA[Organization]]></category>

<guid isPermaLink="false">http://sc5.io/blog/?p=2818</guid>
<description><![CDATA[<p>Toteutimme yhdessä MicroMedian kanssa asiakastyytyväisyystutkimuksen, jossa tiedustelimme asiakkaidemme näkemyksiä toimintatavoistamme. Voimme ylpeänä kertoa, että asiakkaamme tykkäsivät työmme jäljestä. Mutta mihin asiakamme olivat meissä erityisen tyytyväisiä? Vastauksissa korostuivat työntekijöidemme osaaminen ja ammattitaito. Lisäksi kehuttiin kykyämme nähdä... <a class="read-more" href="http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/">Read More</a></p>
<p>The post <a rel="nofollow" href="http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/">Mitä asiakkaamme ajattelevat meistä?</a> appeared first on <a rel="nofollow" href="http://sc5.io/blog">SC5 Blog</a>.</p>
]]></description>
<wfw:commentRss>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>Tribute to Jyväskylä, City of Lights</title>
<link>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/</link>
<comments>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/#comments</comments>
<pubDate>Thu, 25 Sep 2014 10:36:20 +0000</pubDate>
<dc:creator><![CDATA[Arto Liukkonen]]></dc:creator>
<category><![CDATA[JavaScript]]></category>
<category><![CDATA[Jyväskylä]]></category>
<category><![CDATA[Mobile]]></category>
<category><![CDATA[Video]]></category>
<category><![CDATA[Web apps]]></category>
<category><![CDATA[city of lights]]></category>
<category><![CDATA[internet of things]]></category>
<category><![CDATA[iot]]></category>
<category><![CDATA[raspberry pi]]></category>

<guid isPermaLink="false">http://sc5.io/blog/?p=2886</guid>
<description><![CDATA[<p>In Finland the winters are dark so artificial, energy-efficient lights are needed. Good lighting not only boosts enjoyment and the quality of life, but can be a form of art itself. Jyväskylä is a forerunner... <a class="read-more" href="http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/">Read More</a></p>
<p>The post <a rel="nofollow" href="http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/">Tribute to Jyväskylä, City of Lights</a> appeared first on <a rel="nofollow" href="http://sc5.io/blog">SC5 Blog</a>.</p>
]]></description>
<wfw:commentRss>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
</channel>
</rss>