Support icalendar v6 #1361

dotlambda · 2024-10-01T20:25:48Z

Describe the bug
Icalendar v6 was released: https://github.com/collective/icalendar/blob/main/README.rst#version-6-with-zoneinfo
Now running pytest yields test failures.

If applicable: Stack Trace

============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.3.2, pluggy-1.5.0
rootdir: /build/source
configfile: pyproject.toml
plugins: hypothesis-6.108.5
collected 315 items / 5 deselected / 310 selected                              

tests/backend_test.py .F.FF...FFFF....F.............                     [  9%]
tests/cal_display_test.py .........                                      [ 12%]
tests/cli_test.py .FFFF...F.xX.x..FF....F...........F.F.FF.              [ 25%]
tests/configwizard_test.py ..                                            [ 26%]
tests/controller_test.py .....FF.....                                    [ 30%]
tests/event_test.py ..F.FF..FFFFFFF...F..F.........F..F.......F......... [ 47%]
...                                                                      [ 48%]
tests/icalendar_test.py ..FF.F.                                          [ 50%]
tests/khalendar_test.py .......F..FFF..F..FFF.........                   [ 60%]
tests/khalendar_utils_test.py FFF....F.....F...F...FFFF.F..FF.........   [ 72%]
tests/parse_datetime_test.py ........................................... [ 86%]
.....                                                                    [ 88%]
tests/settings_test.py ............                                      [ 92%]
tests/terminal_test.py ...                                               [ 93%]
tests/ui/test_calendarwidget.py .....                                    [ 94%]
tests/ui/test_editor.py ....                                             [ 96%]
tests/ui/test_widgets.py .                                               [ 96%]
tests/utils_test.py ........                                             [ 99%]
tests/vdir_test.py .                                                     [ 99%]
tests/vtimezone_test.py ..                                               [100%]

=================================== FAILURES ===================================
...

=========================== short test summary info ============================
FAILED tests/backend_test.py::test_event_rrule_recurrence_id - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_event_rrule_recurrence_id_reverse - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_event_rrule_recurrence_id_update_with_exclude - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_event_delete - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_this_and_prior - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_event_rrule_this_and_future - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_event_rrule_this_and_future_multi_day_shift - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/backend_test.py::test_two_calendars_same_uid - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/cli_test.py::test_simple - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_simple_color - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_days - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_notstarted - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_invalid_calendar - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_list - AssertionError: assert False
FAILED tests/cli_test.py::test_search - AssertionError: assert False
FAILED tests/cli_test.py::test_import_proper_invalid_timezone - assert not SystemExit(1)
FAILED tests/cli_test.py::test_edit - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_new_interactive - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_new_interactive_extensive - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/cli_test.py::test_issue_1056 - assert not AttributeError("'zoneinfo.ZoneInfo' object has no attribute '_tz...
FAILED tests/controller_test.py::TestImport::test_import - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/controller_test.py::TestImport::test_mix_datetime_types - khal.exceptions.FatalError: 'zoneinfo.ZoneInfo' object has no attribute 'lo...
FAILED tests/event_test.py::test_raw_dt - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_update_simple - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_add_url - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_do_not_save_empty_location - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_do_not_save_empty_description - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_do_not_save_empty_url - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_remove_existing_location_if_set_to_empty - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_remove_existing_description_if_set_to_empty - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_remove_existing_url_if_set_to_empty - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_update_remove_categories - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_transform_event - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_dt_two_tz - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_event_raw_UTC - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_multi_uid - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/event_test.py::test_remove_instance_from_recuid - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/icalendar_test.py::test_split_ics - AttributeError: module 'icalendar' has no attribute 'windows_to_olson'
FAILED tests/icalendar_test.py::test_split_ics_random_uid - AttributeError: module 'icalendar' has no attribute 'windows_to_olson'
FAILED tests/icalendar_test.py::test_windows_timezone - AttributeError: module 'icalendar' has no attribute 'windows_to_olson'
FAILED tests/khalendar_test.py::TestCollection::test_insert - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_get - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_change - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_update_event - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_search - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_delete_two_events - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_delete_recuid - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_test.py::TestCollection::test_invalid_timezones - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'
FAILED tests/khalendar_utils_test.py::TestExpand::test_expand_dt - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestExpand::test_expand_dtb - AttributeError: '_tzicalvtz' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestExpand::test_expand_dttz - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestExpand::test_expand_invalid_exdate - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestExpandNoRR::test_expand_dtr_exdatez - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_until_notz - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_event_exdate_dt - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_event_exdates_dt - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_event_exdatesl_dt - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_event_exdates_remove - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestSpecial::test_event_dt_rrule_invalid_until2 - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestRDate::test_simple_rdate - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
FAILED tests/khalendar_utils_test.py::TestRDate::test_rrule_and_rdate - AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
= 61 failed, 246 passed, 5 deselected, 2 xfailed, 1 xpassed, 177 warnings in 7.66s =

To Reproduce
Install icalendar v6 and run pytest.

OS, version, khal version and how you installed it:

The output of khal --version: khal, version 0.11.3
Installation method: Nixpkgs
python version: 3.12
OS: NixOS
The versions of your other python packages: icalendar 6.0.0

The text was updated successfully, but these errors were encountered:

heull001 · 2024-10-08T11:50:00Z

Failures don't only occur while running tests. "khal new -i" stops while saving with the following stacktrace:

Traceback (most recent call last):
File ".../khal/envkhal/bin/khal", line 8, in
sys.exit(main_khal())
^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/khal/cli.py", line 393, in new
new_func(
File ".../khal/envkhal/lib/python3.12/site-packages/khal/controllers.py", line 367, in new_interactive
event = new_from_dict(
^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/khal/controllers.py", line 423, in new_from_dict
collection.insert(event)
File ".../khal/envkhal/lib/python3.12/site-packages/khal/khalendar/khalendar.py", line 224, in insert
event.href, event.etag = self._storages[calendar].upload(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/khal/khalendar/vdir.py", line 210, in upload
if not isinstance(item.raw, str):
^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/khal/khalendar/event.py", line 385, in raw
timezone = create_timezone(tzinfo, self.start)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../khal/envkhal/lib/python3.12/site-packages/khal/khalendar/event.py", line 903, in create_timezone
for one, two in iter(tz._tzinfos.items()) # type: ignore
^^^^^^^^^^^
AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'

If it takes more time to support icalendar 6.0.0, the requirement should be changed to icalendar<6.0.0 as a workaround as quickly as possible, because at the moment khal is broken for everyone who installed it via pip.

WhyNotHugo · 2024-10-10T17:51:17Z

As a short-term solution, we can pin icalendar<6.0.0. This fixes the issue or those installing from source, but brings a new problem to downstream distributions.

WhyNotHugo · 2024-10-10T17:52:41Z

The documentation for icalendar doesn't list any breaking changes on 6.0.0.

I have no idea what APIs changed and how to fix them; we're going to need to ask upstream.

WhyNotHugo · 2024-10-10T17:56:00Z

With the following dependencies:

atomicwrites==1.4.1
click==8.1.7
click-log==0.4.0
configobj==5.0.8
icalendar==6.0.0
-e git+https://github.com/pimutils/khal.git@106e4d013e33640cccccbed823c948f41027e2d0#egg=khal
python-dateutil==2.9.0.post0
pytz==2024.1
pyxdg==0.28
six==1.16.0
typing_extensions==4.12.2
tzdata==2024.2
tzlocal==5.2
urwid==2.6.15
wcwidth==0.2.13

khal new -i works fine for me.

WhyNotHugo · 2024-10-10T17:57:05Z

See: #1362

WhyNotHugo · 2024-10-10T18:03:47Z

My bad; the changes are described in the README, as linked by op above: https://github.com/collective/icalendar/blob/main/README.rst#version-6-with-zoneinfo

The pytz timezone implementation had a localize() function which seems to be equivalent to datetime.replace(tz=...).

WhyNotHugo · 2024-10-10T18:06:42Z

Some of our usages of the localize function pass a date instance. In this case, it's not as simple as using replace, we need to convert the date into a datetime first. This isn't trivial, because datetime is a subtype of date. Some functions declare taking a date instance as parameter but receive a datetime.

Never mind, we don't need to special-case this in any way; .replace(tzinfo=...) works fine.

LionyxML · 2024-10-25T01:08:43Z

@dotlambda / @WhyNotHugo

I don't know if this is related, but I am getting some errors on the stack showing zoneinfo errors.

Using khal, version 0.11.3 installed with pipx.

I can't see any of my recurring .ical items, either daily or weekly.

When i can khal calendar I get a giant stack with stuff like this:

nknown exception happened.
Traceback (most recent call last):
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/khalendar.py", line 398, in _update_vevent
    update(event.raw, href=href, etag=etag, calendar=calendar)
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/backend.py", line 246, in update
    self._update_impl(vevent, href, calendar)
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/backend.py", line 367, in _update_impl
    dtstartend = expand_vevent(vevent, href)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/icalendar.py", line 315, in expand
    dtstartl = set(rrule)
               ^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/icalendar.py", line 256, in sanitize_datetime
    date = events_tz.localize(date)
           ^^^^^^^^^^^^^^^^^^
AttributeError: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
warning: Skipping CALENDAR/1dbccc760be0f54dc8bf683697b7413c3279fd2f.ics: 'zoneinfo.ZoneInfo' object has no attribute 'localize'
warning: This event will not be available in khal.

Similarly, I can't create anything with time on datetime range.

 khal new -i
calendar: [email protected]
summary: delete weekly
datetime range: 2024-10-10 08:00     
timezone [America/Sao_Paulo]: 
description (or 'None'): None
Traceback (most recent call last):
  File "/home/rmj/.local/bin/khal", line 8, in <module>
    sys.exit(main_khal())
             ^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/cli.py", line 393, in new
    new_func(
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/controllers.py", line 367, in new_interactive
    event = new_from_dict(
            ^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/controllers.py", line 423, in new_from_dict
    collection.insert(event)
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/khalendar.py", line 224, in insert
    event.href, event.etag = self._storages[calendar].upload(event)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/vdir.py", line 210, in upload
    if not isinstance(item.raw, str):
                      ^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/event.py", line 385, in raw
    timezone = create_timezone(tzinfo, self.start)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/rmj/.local/pipx/venvs/khal/lib/python3.12/site-packages/khal/khalendar/event.py", line 903, in create_timezone
    for one, two in iter(tz._tzinfos.items())  # type: ignore
                         ^^^^^^^^^^^
AttributeError: 'zoneinfo.ZoneInfo' object has no attribute '_tzinfos'

The same thing happens with

datetime range: 2024-10-10 08:00
datetime range: 2024-10-10 8:00     
datetime range: 2024-10-10 08:00 2024-10-11 08:00

I can only create new entries if no time part of the date is provided.

How can I help debug this?

For timezone functionalities icalendar can use both pytz and zoneinfo, preferring zoneinfo. We rely on pytz for several attributes, stick to it. Fixes pimutils#1361. Should obsolete pimutils#1363.

iamleot · 2024-10-31T13:25:04Z

I have shared a possible patch to address that via #1364.

Now all tests passes for me with icalendar>=6.0.0.

LionyxML · 2024-11-06T23:52:46Z

Funny addon, if I edit my google calendar calendar with the iphone calendar app, it shows alright with khal.

dotlambda mentioned this issue Oct 2, 2024

python312Packages.icalendar: 5.0.13 -> 6.0.0 NixOS/nixpkgs#328227

Merged

13 tasks

iamleot linked a pull request Oct 31, 2024 that will close this issue

Adjust for and require icalendar>=6.0.0 #1364

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support icalendar v6 #1361

Support icalendar v6 #1361

dotlambda commented Oct 1, 2024

heull001 commented Oct 8, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024 •

edited

Loading

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024 •

edited

Loading

LionyxML commented Oct 25, 2024 •

edited

Loading

iamleot commented Oct 31, 2024

LionyxML commented Nov 6, 2024

Support icalendar v6 #1361

Support icalendar v6 #1361

Comments

dotlambda commented Oct 1, 2024

heull001 commented Oct 8, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024 • edited Loading

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024

WhyNotHugo commented Oct 10, 2024 • edited Loading

LionyxML commented Oct 25, 2024 • edited Loading

iamleot commented Oct 31, 2024

LionyxML commented Nov 6, 2024

WhyNotHugo commented Oct 10, 2024 •

edited

Loading

WhyNotHugo commented Oct 10, 2024 •

edited

Loading

LionyxML commented Oct 25, 2024 •

edited

Loading