Global Workspace Module

Let's again remember this diagram:

We will now focus on the Global Workspace module bim_gw.modules.gw.GlobalWorkspace.

A previous chapter showed how to use the global workspace.

The GW module can encode multimodal representations into a common latent space.

It uses encoders and decoders (ev, el, dv, dl in the figure) to translate the unimodal representations to the GW state and back.

The encoder and decoder class used for the translation is defined by each module's module.domain_specs.workspace_encoder_cls and module.domain_specs.workspace_decoder_cls attributes.

By default, the encoders and decoders from this file are used.

These modules can be configured using the configuration values

• global_workspace.hidden_size to set the hidden dimensions of the encoder or decoder
• global_workspace.n_layers to set the depth of the encoder and decoders
• global_workspace.z_size to set the dimension of the GW representation.

Logging

When logging during training, the module calls:

• unimodal_module.log_domain for the "original" at the beginning of training (in the on_train_start PytorchLightning hook).
• unimodal_module.log_domain_from_latent for the translations, cycles, and demi-cycles.

Up next

← Unimodal modules | Downstream tasks →