From PyTorch 1.4 tutorial
Outline
 Tensor
 torch.autograd.backward
 If the result node is scalar
 If the result node is a vector
[PyTorch] Note 02: Autograd auto derivation
In PyTorch, the core of all neural networks is autograd package
1 Tensor

torch.Tensor is the core class of this autograd

A Tensor tensor usually records the following properties:
 Data: stored data information
 requires_grad: if it is set to True, it means that the Tensor needs derivation
 grad: the gradient value of this Tensor needs to be reset to zero every time when calculating the backward, otherwise the gradient value will be accumulated all the time
 grad_fn: the leaf node (i.e. independent variable) is usually None, and only the grad of the result node (i.e. dependent variable)_ FN is valid to indicate which type of gradient function is.
 is_leaf: used to indicate whether the Tensor is a leaf node.
See Detailed explanation of Python autograd, backward
 requires_grad=True
 Track all operations of the corresponding tensor
 Default requirements_ Grad is flame
a = torch.randn(2, 2) a = ((a * 3) / (a  1)) print(a.requires_grad) a.requires_grad_(True) print(a.requires_grad) #output False True
 . detach() method
 Prevent the tensor from being tracked
 Call The detach() method separates it from the calculation history and prevents its future calculation records from being tracked
 with torch.no_grad():
 To prevent tracing history (and memory usage), code blocks can be wrapped in with torch no_ Grad (): medium
 grad_fn
 Only dependent variables have properties
x=torch.tensor(1.0,requires_grad=True) y=torch.tensor(2.0,requires_grad=True) z=x+y print(x,y,z) #output tensor(1., requires_grad=True) tensor(2., requires_grad=True) tensor(3., grad_fn=<AddBackward0>)
2 torch.autograd.backward
Source code interface
torch.autograd.backward( tensors, grad_tensors=None, retain_graph=None, create_graph=False)
 Parameter meaning
 Tensor: the tensor used to calculate the gradient
torch.autograd.backward(tensor) and tensor Backward() is written equivalently  grad_tensors: used when calculating the gradient of the matrix. In fact, it is also a tensor. Generally, the shape needs to be consistent with the previous tensor
 retain_graph: usually, after calling backward once, pytorch will automatically destroy the calculation graph. Therefore, if you want to call backward repeatedly for a variable, you need to set this parameter to True
 create_graph: when set to True, it can be used to calculate higherorder gradients
 Tensor: the tensor used to calculate the gradient
2.1 if the result node is scalar
 Scalar can be understood as onedimensional
 Just apply backward directly
 Reference examples are as follows:
x=torch.tensor(3.0,requires_grad=True) y=torch.tensor(7.0,requires_grad=True) z=x+y z.backward() #Returns the gradient of x,y, and the value of z print(x.grad,y.grad,z) #output tensor(1.) tensor(1.) tensor(10., grad_fn=<AddBackward0>)
2.2 if the result is vector
 Vector can be understood as highdimensional and multidimensional
 Referring to the Chinese documents of pytorch and Zhihu's articles, I feel that Zhihu's articles are better understood, but they are essentially the same, that is, a tensor is introduced that is the same as the previous tensor tensor
 Reference examples are as follows:
x=torch.ones(4,requires_grad=True) #x=[x1,x2,x3,x4] #print(x.type()) torch.FloatTensor z=x+2 #z=[x1+2,x2+2,x3+2,x4+2] #If all incoming are 1 z.backward(torch.ones_like(z)) #If the incoming is set by yourself #z.backward(torch.Tensor([1,2,3,4])) #z=[x1+2,2(x2+2),3(x3+2),4(x4+2)] note the matching of types. Tensor defaults to torch Floattensor type #z.backward(torch.tensor([1.,2.,3.,4.])) #z=[x1+2,2(x2+2),3(x3+2),4(x4+2)] print(x.grad) #output tensor([1., 1., 1., 1.]) #tensor([1., 2., 3., 4.])
In the process of writing, I found that tensor and tensor were different
 torch.Tensor
 Short for default tensor type (torch.FlaotTensor)
 torch.tensor
 Tensor is created according to the following data, and the tensor type is inferred according to the data.
See [PyTorch] the difference between tensor and tensor
The next section describes how to build neural networks
Welcome criticism and correction, learn and make progress together!!!