Release v2.2.0

Algorithm

DiscreteDecisionTransformer, a Decision Transformer implementation for discrete action-space, has been finally implemented in v2.2.0! The reduction results with Atari 2600 are available here.

import d3rlpy

dataset, env = d3rlpy.datasets.get_cartpole()

dt = d3rlpy.algos.DiscreteDecisionTransformerConfig(
    batch_size=64,
    num_heads=1,
    learning_rate=1e-4,
    max_timestep=1000,
    num_layers=3,
    position_encoding_type=d3rlpy.PositionEncodingType.SIMPLE,
    encoder_factory=d3rlpy.models.VectorEncoderFactory([128], exclude_last_activation=True),
    observation_scaler=d3rlpy.preprocessing.StandardObservationScaler(),
    context_size=20,
    warmup_tokens=100000,
).create()

dt.fit(
    dataset,
    n_steps=100000,
    n_steps_per_epoch=1000,
    eval_env=env,
    eval_target_return=500,
)

Enhancement

• Expose action_size and action_space options for manual dataset creation #338
• FrameStackTrajectorySlicer has been added.

Refactoring

• Typing check of numpy is enabled. Some parts of codes differentiate data types of numpy arrays, which is checked by mypy.

Bugfix

• Device error at AWAC #341
• Invalid batch.intervals #346
  • ⚠️ This fix is important to retain the performance of Q-learning algorithms since v1.1.1.