forked from celery/celery
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Changelog
4219 lines (2756 loc) · 133 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
================
Change history
================
.. contents::
:local:
.. _version-2.2.4:
2.2.4
=====
:release-date: 2011-02-19 12:00 AM CET
* celeryd: 2.2.3 broke error logging, resulting in tracebacks not being logged.
* AMQP result backend: Polling task states did not work properly if there were
more than one result message in the queue.
* ``TaskSet.apply_async()`` and ``TaskSet.apply()`` now supports an optional
``taskset_id`` keyword argument (Issue #331).
* The current taskset id (if any) is now available in the task context as
``request.taskset`` (Issue #329).
* SQLAlchemy result backend: `date_done` was no longer part of the results as it had
been accidentally removed. It is now available again (Issue #325).
* SQLAlchemy result backend: Added unique constraint on `Task.task_id` and
`TaskSet.taskset_id`. Tables needs to be recreated for this to take effect.
* Fixed exception raised when iterating on the result of ``TaskSet.apply()``.
* Tasks Userguide: Added section on choosing a result backend.
.. _v224-fixes:
.. _version-2.2.3:
2.2.3
=====
:release-date: 2011-02-12 16:00 PM CET
.. _v223-fixes:
Fixes
-----
* Now depends on Kombu 1.0.3
* Task.retry now supports a ``max_retries`` argument, used to change the
default value.
* `multiprocessing.cpu_count` may raise :exc:`NotImplementedError` on
platforms where this is not supported (Issue #320).
* Coloring of log messages broke if the logged object was not a string.
* Fixed several typos in the init script documentation.
* A regression caused `Task.exchange` and `Task.routing_key` to no longer
have any effect. This is now fixed.
* Routing Userguide: Fixes typo, routers in :setting:`CELERY_ROUTES` must be
instances, not classes.
* :program:`celeryev` did not create pidfile even though the
:option:`--pidfile` argument was set.
* Task logger format was no longer used. (Issue #317).
The id and name of the task is now part of the log message again.
* A safe version of ``repr()`` is now used in strategic places to ensure
objects with a broken ``__repr__`` does not crash the worker, or otherwise
make errors hard to understand (Issue #298).
* Remote control command ``active_queues``: did not account for queues added
at runtime.
In addition the dictionary replied by this command now has a different
structure: the exchange key is now a dictionary containing the
exchange declaration in full.
* The :option:`-Q` option to :program:`celeryd` removed unused queue
declarations, so routing of tasks could fail.
Queues are no longer removed, but rather `app.amqp.queues.consume_from()`
is used as the list of queues to consume from.
This ensures all queues are available for routing purposes.
* celeryctl: Now supports the `inspect active_queues` command.
.. _version-2.2.2:
2.2.2
=====
:release-date: 2011-02-03 16:00 PM CET
.. _v222-fixes:
Fixes
-----
* Celerybeat could not read the schedule properly, so entries in
:setting:`CELERYBEAT_SCHEDULE` would not be scheduled.
* Task error log message now includes `exc_info` again.
* The `eta` argument can now be used with `task.retry`.
Previously it was overwritten by the countdown argument.
* celeryd-multi/celeryd_detach: Now logs errors occuring when executing
the `celeryd` command.
* daemonizing cookbook: Fixed typo ``--time-limit 300`` ->
``--time-limit=300``
* Colors in logging broke non-string objects in log messages.
* ``setup_task_logger`` no longer makes assumptions about magic task kwargs.
.. _version-2.2.1:
2.2.1
=====
:release-date: 2011-02-02 16:00 PM CET
.. _v221-fixes:
Fixes
-----
* Eventlet pool was leaking memory (Issue #308).
* Deprecated function ``celery.execute.delay_task`` was accidentally removed,
now available again.
* ``BasePool.on_terminate`` stub did not exist
* celeryd detach: Adds readable error messages if user/group name does not
exist.
* Smarter handling of unicode decod errors when logging errors.
.. _version-2.2.0:
2.2.0
=====
:release-date: 2011-02-01 10:00 AM CET
.. _v220-important:
Important Notes
---------------
* Carrot has been replaced with `Kombu`_
Kombu is the next generation messaging framework for Python,
fixing several flaws present in Carrot that was hard to fix
without breaking backwards compatibility.
Also it adds:
* First-class support for virtual transports; Redis, Django ORM,
SQLAlchemy, Beanstalk, MongoDB, CouchDB and in-memory.
* Consistent error handling with introspection,
* The ability to ensure that an operation is performed by gracefully
handling connection and channel errors,
* Message compression (zlib, bzip2, or custom compression schemes).
This means that `ghettoq` is no longer needed as the
functionality it provided is already available in Celery by default.
The virtual transports are also more feature complete with support
for exchanges (direct and topic). The Redis transport even supports
fanout exchanges so it is able to perform worker remote control
commands.
.. _`Kombu`: http://pypi.python.org/pypi/kombu
* Magic keyword arguments pending deprecation.
The magic keyword arguments were responsibile for many problems
and quirks: notably issues with tasks and decorators, and name
collisions in keyword arguments for the unaware.
It wasn't easy to find a way to deprecate the magic keyword arguments,
but we think this is a solution that makes sense and it will not
have any adverse effects for existing code.
The path to a magic keyword argument free world is:
* the `celery.decorators` module is deprecated and the decorators
can now be found in `celery.task`.
* The decorators in `celery.task` disables keyword arguments by
default
* All examples in the documentation have been changed to use
`celery.task`.
This means that the following will have magic keyword arguments
enabled (old style):
.. code-block:: python
from celery.decorators import task
@task
def add(x, y, **kwargs):
print("In task %s" % kwargs["task_id"])
return x + y
And this will not use magic keyword arguments (new style):
.. code-block:: python
from celery.task import task
@task
def add(x, y):
print("In task %s" % add.request.id)
return x + y
In addition, tasks can choose not to accept magic keyword arguments by
setting the `task.accept_magic_kwargs` attribute.
.. admonition:: Deprecation
Using the decorators in :mod:`celery.decorators` emits a
:class:`PendingDeprecationWarning` with a helpful message urging
you to change your code, in version 2.4 this will be replaced with
a :class:`DeprecationWarning`, and in version 3.0 the
:mod:`celery.decorators` module will be removed and no longer exist.
Similarly, the `task.accept_magic_kwargs` attribute will no
longer have any effect starting from version 3.0.
* The magic keyword arguments are now available as `task.request`
This is called *the context*. Using thread-local storage the
context contains state that is related to the current request.
It is mutable and you can add custom attributes that will only be seen
by the current task request.
The following context attributes are always available:
===================================== ===================================
**Magic Keyword Argument** **Replace with**
===================================== ===================================
`kwargs["task_id"]` `self.request.id`
`kwargs["delivery_info"]` `self.request.delivery_info`
`kwargs["task_retries"]` `self.request.retries`
`kwargs["logfile"]` `self.request.logfile`
`kwargs["loglevel"]` `self.request.loglevel`
`kwargs["task_is_eager` `self.request.is_eager`
**NEW** `self.request.args`
**NEW** `self.request.kwargs`
===================================== ===================================
In addition, the following methods now automatically uses the current
context, so you don't have to pass `kwargs` manually anymore:
* `task.retry`
* `task.get_logger`
* `task.update_state`
* `Eventlet`_ support.
This is great news for I/O-bound tasks!
To change pool implementations you use the :option:`-P|--pool` argument
to :program:`celeryd`, or globally using the
:setting:`CELERYD_POOL` setting. This can be the full name of a class,
or one of the following aliases: `processes`, `eventlet`, `gevent`.
For more information please see the :ref:`concurrency-eventlet` section
in the User Guide.
.. admonition:: Why not gevent?
For our first alternative concurrency implementation we have focused
on `Eventlet`_, but there is also an experimental `gevent`_ pool
available. This is missing some features, notably the ability to
schedule ETA tasks.
Hopefully the `gevent`_ support will be feature complete by
version 2.3, but this depends on user demand (and contributions).
.. _`Eventlet`: http://eventlet.net
.. _`gevent`: http://gevent.org
* Python 2.4 support deprecated!
We're happy^H^H^H^H^Hsad to announce that this is the last version
to support Python 2.4.
You are urged to make some noise if you're currently stuck with
Python 2.4. Complain to your package maintainers, sysadmins and bosses:
tell them it's time to move on!
Apart from wanting to take advantage of with-statements, coroutines,
conditional expressions and enhanced try blocks, the code base
now contains so many 2.4 related hacks and workarounds it's no longer
just a compromise, but a sacrifice.
If it really isn't your choice, and you don't have the option to upgrade
to a newer version of Python, you can just continue to use Celery 2.2.
Important fixes can be backported for as long as there is interest.
* `celeryd`: Now supports Autoscaling of child worker processes.
The :option:`--autoscale` option can be used to configure the minimum
and maximum number of child worker processes::
--autoscale=AUTOSCALE
Enable autoscaling by providing
max_concurrency,min_concurrency. Example:
--autoscale=10,3 (always keep 3 processes, but grow to
10 if necessary).
* Remote Debugging of Tasks
``celery.contrib.rdb`` is an extended version of :mod:`pdb` that
enables remote debugging of processes that does not have terminal
access.
Example usage:
.. code-block:: python
from celery.contrib import rdb
from celery.task import task
@task
def add(x, y):
result = x + y
rdb.set_trace() # <- set breakpoint
return result
:func:`~celery.contrib.rdb.set_trace` sets a breakpoint at the current
location and creates a socket you can telnet into to remotely debug
your task.
The debugger may be started by multiple processes at the same time,
so rather than using a fixed port the debugger will search for an
available port, starting from the base port (6900 by default).
The base port can be changed using the environment variable
:envvar:`CELERY_RDB_PORT`.
By default the debugger will only be available from the local host,
to enable access from the outside you have to set the environment
variable :envvar:`CELERY_RDB_HOST`.
When `celeryd` encounters your breakpoint it will log the following
information::
[INFO/MainProcess] Got task from broker:
tasks.add[d7261c71-4962-47e5-b342-2448bedd20e8]
[WARNING/PoolWorker-1] Remote Debugger:6900:
Please telnet 127.0.0.1 6900. Type `exit` in session to continue.
[2011-01-18 14:25:44,119: WARNING/PoolWorker-1] Remote Debugger:6900:
Waiting for client...
If you telnet the port specified you will be presented
with a ``pdb`` shell::
$ telnet localhost 6900
Connected to localhost.
Escape character is '^]'.
> /opt/devel/demoapp/tasks.py(128)add()
-> return result
(Pdb)
Enter ``help`` to get a list of available commands,
It may be a good idea to read the `Python Debugger Manual`_ if
you have never used `pdb` before.
.. _`Python Debugger Manual`: http://docs.python.org/library/pdb.html
* Events are now transient and is using a topic exchange (instead of direct).
The `CELERYD_EVENT_EXCHANGE`, `CELERYD_EVENT_ROUTING_KEY`,
`CELERYD_EVENT_EXCHANGE_TYPE` settings are no longer in use.
This means events will not be stored until there is a consumer, and the
events will be gone as soon as the consumer stops. Also it means there
can be multiple monitors running at the same time.
The routing key of an event is the type of event (e.g. `worker.started`,
`worker.heartbeat`, `task.succeeded`, etc. This means a consumer can
filter on specific types, to only be alerted of the events it cares about.
Each consumer will create a unique queue, meaning it is in effect a
broadcast exchange.
This opens up a lot of possibilities, for example the workers could listen
for worker events to know what workers are in the neighborhood, and even
restart workers when they go down (or use this information to optimize
tasks/autoscaling).
.. note::
The event exchange has been renamed from "celeryevent" to "celeryev"
so it does not collide with older versions.
If you would like to remove the old exchange you can do so
by executing the following command::
$ camqadm exchange.delete celeryevent
* `celeryd` now starts without configuration, and configuration can be
specified directly on the command line.
Configuration options must appear after the last argument, separated
by two dashes::
$ celeryd -l info -I tasks -- broker.host=localhost broker.vhost=/app
* Configuration is now an alias to the original configuration, so changes
to the original will reflect Celery at runtime.
* `celery.conf` has been deprecated, and modifying `celery.conf.ALWAYS_EAGER`
will no longer have any effect.
The default configuration is now available in the
:mod:`celery.app.defaults` module. The available configuration options
and their types can now be introspected.
* Remote control commands are now provided by `kombu.pidbox`, the generic
process mailbox.
* Internal module `celery.worker.listener` has been renamed to
`celery.worker.consumer`, and `.CarrotListener` is now `.Consumer`.
* Previously deprecated modules `celery.models` and
`celery.management.commands` have now been removed as per the deprecation
timeline.
* [Security: Low severity] Removed `celery.task.RemoteExecuteTask` and
accompanying functions: `dmap`, `dmap_async`, and `execute_remote`.
Executing arbitrary code using pickle is a potential security issue if
someone gains unrestricted access to the message broker.
If you really need this functionality, then you would have to add
this to your own project.
* [Security: Low severity] The `stats` command no longer transmits the
broker password.
One would have needed an authenticated broker connection to receive
this password in the first place, but sniffing the password at the
wire level would have been possible if using unencrypted communication.
.. _v220-news:
News
----
* The internal module `celery.task.builtins` has been removed.
* The module `celery.task.schedules` is deprecated, and
`celery.schedules` should be used instead.
For example if you have::
from celery.task.schedules import crontab
You should replace that with::
from celery.schedules import crontab
The module needs to be renamed because it must be possible
to import schedules without importing the `celery.task` module.
* The following functions have been deprecated and is scheduled for
removal in version 2.3:
* `celery.execute.apply_async`
Use `task.apply_async()` instead.
* `celery.execute.apply`
Use `task.apply()` instead.
* `celery.execute.delay_task`
Use `registry.tasks[name].delay()` instead.
* Importing `TaskSet` from `celery.task.base` is now deprecated.
You should use::
>>> from celery.task import TaskSet
instead.
* New remote control commands:
* `active_queues`
Returns the queue declarations a worker is currently consuming from.
* Added the ability to retry publishing the task message in
the event of connection loss or failure.
This is disabled by default but can be enabled using the
:setting:`CELERY_TASK_PUBLISH_RETRY` setting, and tweaked by
the :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY` setting.
In addition `retry`, and `retry_policy` keyword arguments have
been added to `Task.apply_async`.
.. note::
Using the `retry` argument to `apply_async` requires you to
handle the publisher/connection manually.
* Periodic Task classes (`@periodic_task`/`PeriodicTask`) will *not* be
deprecated as previously indicated in the source code.
But you are encouraged to use the more flexible
:setting:`CELERYBEAT_SCHEDULE` setting.
* Built-in daemonization support of celeryd using `celeryd-multi`
is no longer experimental and is considered production quality.
See :ref:`daemon-generic` if you want to use the new generic init
scripts.
* Added support for message compression using the
:setting:`CELERY_MESSAGE_COMPRESSION` setting, or the `compression` argument
to `apply_async`. This can also be set using routers.
* `celeryd`: Now logs stacktrace of all threads when receiving the
`SIGUSR1` signal. (Does not work on cPython 2.4, Windows or Jython).
Inspired by https://gist.github.com/737056
* Can now remotely terminate/kill the worker process currently processing
a task.
The `revoke` remote control command now supports a `terminate` argument
Default signal is `TERM`, but can be specified using the `signal`
argument. Signal can be the uppercase name of any signal defined
in the :mod:`signal` module in the Python Standard Library.
Terminating a task also revokes it.
Example::
>>> from celery.task.control import revoke
>>> revoke(task_id, terminate=True)
>>> revoke(task_id, terminate=True, signal="KILL")
>>> revoke(task_id, terminate=True, signal="SIGKILL")
* `TaskSetResult.join_native`: Backend-optimized version of `join()`.
If available, this version uses the backends ability to retrieve
multiple results at once, unlike `join()` which fetches the results
one by one.
So far only supported by the AMQP result backend. Support for memcached
and Redis may be added later.
* Improved implementations of `TaskSetResult.join` and `AsyncResult.wait`.
An `interval` keyword argument have been added to both so the
polling interval can be specified (default interval is 0.5 seconds).
A `propagate` keyword argument have been added to `result.wait()`,
errors will be returned instead of raised if this is set to False.
.. warning::
You should decrease the polling interval when using the database
result backend, as frequent polling can result in high database load.
* The PID of the child worker process accepting a task is now sent as a field
with the `task-started` event.
* The following fields have been added to all events in the worker class:
* `sw_ident`: Name of worker software (e.g. celeryd).
* `sw_ver`: Software version (e.g. 2.2.0).
* `sw_sys`: Operating System (e.g. Linux, Windows, Darwin).
* For better accuracy the start time reported by the multiprocessing worker
process is used when calculating task duration.
Previously the time reported by the accept callback was used.
* `celerybeat`: New built-in daemonization support using the `--detach`
option.
* `celeryev`: New built-in daemonization support using the `--detach`
option.
* `TaskSet.apply_async`: Now supports custom publishers by using the
`publisher` argument.
* Added :setting:`CELERY_SEND_TASK_SENT_EVENT` setting.
If enabled an event will be sent with every task, so monitors can
track tasks before the workers receive them.
* `celerybeat`: Now reuses the broker connection when applying
scheduled tasks.
* The configuration module and loader to use can now be specified on
the command line.
For example::
$ celeryd --config=celeryconfig.py --loader=myloader.Loader
* Added signals: `beat_init` and `beat_embedded_init`
* :signal:`celery.signals.beat_init`
Dispatched when :program:`celerybeat` starts (either standalone or
embedded). Sender is the :class:`celery.beat.Service` instance.
* :signal:`celery.signals.beat_embedded_init`
Dispatched in addition to the :signal:`beat_init` signal when
:program:`celerybeat` is started as an embedded process. Sender
is the :class:`celery.beat.Service` instance.
* Redis result backend: Removed deprecated settings `REDIS_TIMEOUT` and
`REDIS_CONNECT_RETRY`.
* CentOS init script for :program:`celeryd` now available in `contrib/centos`.
* Now depends on `pyparsing` version 1.5.0 or higher.
There have been reported issues using Celery with pyparsing 1.4.x,
so please upgrade to the latest version.
* Lots of new unit tests written, now with a total coverage of 95%.
.. _v220-fixes:
Fixes
-----
* `celeryev` Curses Monitor: Improved resize handling and UI layout
(Issue #274 + Issue #276)
* AMQP Backend: Exceptions occurring while sending task results are now
propagated instead of silenced.
`celeryd` will then show the full traceback of these errors in the log.
* AMQP Backend: No longer deletes the result queue after successful
poll, as this should be handled by the
:setting:`CELERY_AMQP_TASK_RESULT_EXPIRES` setting instead.
* AMQP Backend: Now ensures queues are declared before polling results.
* Windows: celeryd: Show error if running with `-B` option.
Running celerybeat embedded is known not to work on Windows, so
users are encouraged to run celerybeat as a separate service instead.
* Windows: Utilities no longer output ANSI color codes on Windows
* camqadm: Now properly handles Ctrl+C by simply exiting instead of showing
confusing traceback.
* Windows: All tests are now passing on Windows.
* Remove bin/ directory, and `scripts` section from setup.py.
This means we now rely completely on setuptools entrypoints.
.. _v220-experimental:
Experimental
------------
* Jython: celeryd now runs on Jython using the threaded pool.
All tests pass, but there may still be bugs lurking around the corners.
* PyPy: celeryd now runs on PyPy.
It runs without any pool, so to get parallel execution you must start
multiple instances (e.g. using :program:`celeryd-multi`).
Sadly an initial benchmark seems to show a 30% performance decrease on
pypy-1.4.1 + JIT. We would like to find out why this is, so stay tuned.
* :class:`PublisherPool`: Experimental pool of task publishers and
connections to be used with the `retry` argument to `apply_async`.
The example code below will re-use connections and channels, and
retry sending of the task message if the connection is lost.
.. code-block:: python
from celery import current_app
# Global pool
pool = current_app().amqp.PublisherPool(limit=10)
def my_view(request):
with pool.acquire() as publisher:
add.apply_async((2, 2), publisher=publisher, retry=True)
.. _version-2.1.4:
2.1.4
=====
:release-date: 2010-12-03 12:00 PM CEST
.. _v214-fixes:
Fixes
-----
* Execution options to `apply_async` now takes precedence over options
returned by active routers. This was a regression introduced recently
(Issue #244).
* `celeryev` curses monitor: Long arguments are now truncated so curses
doesn't crash with out of bounds errors. (Issue #235).
* `celeryd`: Channel errors occurring while handling control commands no
longer crash the worker but are instead logged with severity error.
* SQLAlchemy database backend: Fixed a race condition occurring when
the client wrote the pending state. Just like the Django database backend,
it does no longer save the pending state (Issue #261 + Issue #262).
* Error email body now uses `repr(exception)` instead of `str(exception)`,
as the latter could result in Unicode decode errors (Issue #245).
* Error e-mail timeout value is now configurable by using the
:setting:`EMAIL_TIMEOUT` setting.
* `celeryev`: Now works on Windows (but the curses monitor won't work without
having curses).
* Unit test output no longer emits non-standard characters.
* `celeryd`: The broadcast consumer is now closed if the connection is reset.
* `celeryd`: Now properly handles errors occurring while trying to acknowledge
the message.
* `TaskRequest.on_failure` now encodes traceback using the current filesystem
encoding. (Issue #286).
* `EagerResult` can now be pickled (Issue #288).
.. _v214-documentation:
Documentation
-------------
* Adding :ref:`contributing`.
* Added :ref:`guide-optimizing`.
* Added :ref:`faq-security` section to the FAQ.
.. _version-2.1.3:
2.1.3
=====
:release-date: 2010-11-09 17:00 PM CEST
.. _v213-fixes:
* Fixed deadlocks in `timer2` which could lead to `djcelerymon`/`celeryev -c`
hanging.
* `EventReceiver`: now sends heartbeat request to find workers.
This means :program:`celeryev` and friends finds workers immediately
at startup.
* celeryev cursesmon: Set screen_delay to 10ms, so the screen refreshes more
often.
* Fixed pickling errors when pickling :class:`AsyncResult` on older Python
versions.
* celeryd: prefetch count was decremented by eta tasks even if there
were no active prefetch limits.
.. _version-2.1.2:
2.1.2
=====
:release-data: TBA
.. _v212-fixes:
Fixes
-----
* celeryd: Now sends the `task-retried` event for retried tasks.
* celeryd: Now honors ignore result for
:exc:`~celery.exceptions.WorkerLostError` and timeout errors.
* celerybeat: Fixed :exc:`UnboundLocalError` in celerybeat logging
when using logging setup signals.
* celeryd: All log messages now includes `exc_info`.
.. _version-2.1.1:
2.1.1
=====
:release-date: 2010-10-14 14:00 PM CEST
.. _v211-fixes:
Fixes
-----
* Now working on Windows again.
Removed dependency on the pwd/grp modules.
* snapshots: Fixed race condition leading to loss of events.
* celeryd: Reject tasks with an eta that cannot be converted to a time stamp.
See issue #209
* concurrency.processes.pool: The semaphore was released twice for each task
(both at ACK and result ready).
This has been fixed, and it is now released only once per task.
* docs/configuration: Fixed typo `CELERYD_SOFT_TASK_TIME_LIMIT` ->
:setting:`CELERYD_TASK_SOFT_TIME_LIMIT`.
See issue #214
* control command `dump_scheduled`: was using old .info attribute
* :program:`celeryd-multi`: Fixed `set changed size during iteration` bug
occurring in the restart command.
* celeryd: Accidentally tried to use additional command line arguments.
This would lead to an error like:
`got multiple values for keyword argument 'concurrency'`.
Additional command line arguments are now ignored, and does not
produce this error. However -- we do reserve the right to use
positional arguments in the future, so please do not depend on this
behavior.
* celerybeat: Now respects routers and task execution options again.
* celerybeat: Now reuses the publisher instead of the connection.
* Cache result backend: Using :class:`float` as the expires argument
to `cache.set` is deprecated by the memcached libraries,
so we now automatically cast to :class:`int`.
* unit tests: No longer emits logging and warnings in test output.
.. _v211-news:
News
----
* Now depends on carrot version 0.10.7.
* Added :setting:`CELERY_REDIRECT_STDOUTS`, and
:setting:`CELERYD_REDIRECT_STDOUTS_LEVEL` settings.
:setting:`CELERY_REDIRECT_STDOUTS` is used by :program:`celeryd` and
:program:`celerybeat`. All output to `stdout` and `stderr` will be
redirected to the current logger if enabled.
:setting:`CELERY_REDIRECT_STDOUTS_LEVEL` decides the log level used and is
:const:`WARNING` by default.
* Added :setting:`CELERYBEAT_SCHEDULER` setting.
This setting is used to define the default for the -S option to
:program:`celerybeat`.
Example:
.. code-block:: python
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
* Added Task.expires: Used to set default expiry time for tasks.
* New remote control commands: `add_consumer` and `cancel_consumer`.
.. method:: add_consumer(queue, exchange, exchange_type, routing_key,
**options)
:module:
Tells the worker to declare and consume from the specified
declaration.
.. method:: cancel_consumer(queue_name)
:module:
Tells the worker to stop consuming from queue (by queue name).
Commands also added to :program:`celeryctl` and
:class:`~celery.task.control.inspect`.
Example using celeryctl to start consuming from queue "queue", in
exchange "exchange", of type "direct" using binding key "key"::
$ celeryctl inspect add_consumer queue exchange direct key
$ celeryctl inspect cancel_consumer queue
See :ref:`monitoring-celeryctl` for more information about the
:program:`celeryctl` program.
Another example using :class:`~celery.task.control.inspect`:
.. code-block:: python
>>> from celery.task.control import inspect
>>> inspect.add_consumer(queue="queue", exchange="exchange",
... exchange_type="direct",
... routing_key="key",
... durable=False,
... auto_delete=True)
>>> inspect.cancel_consumer("queue")
* celerybeat: Now logs the traceback if a message can't be sent.
* celerybeat: Now enables a default socket timeout of 30 seconds.
* README/introduction/homepage: Added link to `Flask-Celery`_.
.. _`Flask-Celery`: http://github.com/ask/flask-celery
.. _version-2.1.0:
2.1.0
=====
:release-date: 2010-10-08 12:00 PM CEST
.. _v210-important:
Important Notes
---------------
* Celery is now following the versioning semantics defined by `semver`_.
This means we are no longer allowed to use odd/even versioning semantics
By our previous versioning scheme this stable release should have
been version 2.2.
.. _`semver`: http://semver.org
* Now depends on Carrot 0.10.7.
* No longer depends on SQLAlchemy, this needs to be installed separately
if the database result backend is used.
* django-celery now comes with a monitor for the Django Admin interface.
This can also be used if you're not a Django user. See
:ref:`monitoring-django-admin` and :ref:`monitoring-nodjango` for more information.
* If you get an error after upgrading saying:
`AttributeError: 'module' object has no attribute 'system'`,
Then this is because the `celery.platform` module has been
renamed to `celery.platforms` to not collide with the built-in
:mod:`platform` module.
You have to remove the old :file:`platform.py` (and maybe
:file:`platform.pyc`) file from your previous Celery installation.
To do this use :program:`python` to find the location
of this module::