question regarding the forward -backward training loop

[sorobedio]

Hello,

I have a question regarding the forward and backward loop. I'm trying to update it to the Lightning style, but I'm having trouble understanding a particular part. Could you please help clarify it for me?

micro = batch[i : i + self.microbatch].to(dist_util.dev()) micro = self.get_first_stage_encoding(micro).detach() micro_cond = { k: v[i : i + self.microbatch].to(dist_util.dev()) for k, v in cond.items() }

assumed 'microbatch' refers to a subset of input data, equivalent to a mini-batch, so let's define microbatch = batch.

The conditioning data is a dictionary where each element matches the batch size. The size of v should depend on the user's conditioning data, not batch_size. We should enforce len(cond) = self.microbatch if self.microbatch > 0; otherwise, len(cond) should match the batch size ==len(micro) or micro.shape[0].