Encoding issues with SQLite

[gpoo]

As reported by @MishiR in issue #72, it seems to be encoding issues when storing the data on SQLite.

sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError)
You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str).
It is highly recommended that you instead just switch your application to Unicode strings
[SQL: u'INSERT INTO messages (message_id, mailing_list_url, mailing_list, first_date,
                              first_date_tz, arrival_date, arrival_date_tz, subject,
                              message_body, is_response_of, mail_path)
 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)']
[parameters: (
 u'<[email protected]<mailto:[email protected]>>',
 'https://mail.gnome.org/archives/gnome-devel-list',
 None,
'1999-12-25 18:09:52.000000',
 3600.0,
 '2000-05-11 23:18:34.000000',
 None,
 u'Re: Solaris_x86, libgtop and kstat...',
 u"Markus D\xf6hr <[email protected]<mailto:[email protected]>> writes:\n\n> I'm tryin' to compile gnome-utils-1.0.50 on a Intel SunOS 5.7 after\n> following the excellent document ... (1367 characters truncated) ... 't list -lkstat as a dependency -\nI'll fix this in LibGTop asap.\n\n-- \nMartin Baulig - [email protected]<mailto:[email protected]> - http://www.home-of-linux.org<http://www.home-of-linux.org/>\n\n", 'Markus D\xf6hr\'s message of "Thu, 23 Dec 1999 03:55:25 +0100"',
 None)]

The command line executed was:

$ sudo mlstats --db-driver=sqlite --db-name=gnome.db https://mail.gnome.org/archives/gnome-devel-list/

[gpoo]

Nothing related, but sudo is not required and should not be used.

[MishiR]

Hi,

If I do not use sudo:

mlstats --db-driver=sqlite --db-name=gnome.db https://mail.gnome.org/archives/gnome-devel-list/

I get an error:

Unknown URL or directory: https://mail.gnome.org/archives/gnome-devel-list/. Skipping.
0 messages analysed
0 messages stored in database gnome.db
0 messages ignored by the parser
INFO: Everything seems to be ok.

/BR

Mehvish Rashid

From: Germán Poo-Caamaño [mailto:[email protected]]
Sent: 14 September 2016 01:35
To: MetricsGrimoire/MailingListStats
Cc: Mehvish.Rashid; Mention
Subject: Re: [MetricsGrimoire/MailingListStats] Encodoing issues with SQLite (#73)

Nothing related, but sudo is not required and should not be used.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/73#issuecomment-246870378, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANq6cmw69HauaXzlGTsLfSufICljToMkks5qp0ElgaJpZM4J8Sbe.

[gpoo]

Maybe you are running the command in a directory where your user does not permissions.

Nevertheless, I have not had time to look at the issue. I will try to replicate it.