Softmax layer (page 437)

[ashwinshetgaonkar]

Here you have not defined the softmax layer while building the Sequential model.

Is is taken by default when we choose loss to be:nn.CrossEntropyLoss()?

[rasbt]

Great question. Yes, you are right, it's done by default. This is one of the unintuitive parts about PyTorch, but under the hood, it takes care of the softmax. I am currently writing a blog article to discuss this in more detail.

What's a bit unintuitive about that is that the binary cross entropy loss (BCELoss) accepts probabilities (after activation), and there is a version that accepts the logits: BCEWithLogitsLoss

However, with the multiclass cross entropy loss (unlike the binary version) there is no WithLogits version. In fact, the CrossEntropyLoss is the multiclass equivalent of BCEWithLogitsLoss. And NLLLoss is the multiclass equivalent of BCELoss.

I have a little explanation here from my deep learning class if useful:

• https://github.com/rasbt/stat453-deep-learning-ss21/blob/main/L08/code/cross-entropy-pytorch.ipynb
• L8.4 Logits and Cross Entropy (06:47) https://www.youtube.com/watch?v=icQaFxKa_J0
• L8.5 Logistic Regression in PyTorch – Code Example (19:02) https://www.youtube.com/watch?v=6igMArA6k3A