why grouping different rows improves L2 cache hit in the LayerNormalization tutorial?

[andy90]

In the tutorial of LayerNormalization, the author tries to group different rows together to accumulate the gradients of w and b. But I don't see why this could improve the L2 cache hit. Moreover, the tutorial tries to assign neighboring rows to different groups. This seems to me will decrease the L2 cache hit.