-
-
Notifications
You must be signed in to change notification settings - Fork 34
Gotchas
Patrick Kidger edited this page Oct 20, 2019
·
1 revision
-
Think about axis ordering and about how to save stuff for backward. (See here.)
-
Every C++ function defining the forward/backward passes of an
autograd.Function
should start by detaching everytorch.Tensor
that they are given. First of all, this is because there is no point keeping track of gradients when our implementations typically use a lot of in-place operations. Secondly, not doing this raises some errors as the autograd framework tries to point this fact out to us.