thumbnail: https://image.zhangxiann.com/...
toc: true
date: 2020/2/5 20:39:20
disqusId: zhangxian
categories:
 PyTorch
tags:
 AI
 Deep Learning
Code of this chapter:
 https://github.com/zhangxiann/PyTorch_Practice/blob/master/lesson1/tensor_introduce1.py
 https://github.com/zhangxiann/PyTorch_Practice/blob/master/lesson1/tensor_introduce1.py
Tensor concept
Tensor is tensor in Chinese. Tensor means a multidimensional array, which is a highdimensional extension of scalar, vector and matrix.
Scalars can be called 0dimensional tensors, vectors can be called 1dimensional tensors, matrices can be called 2dimensional tensors, and RGB images can represent 3dimensional tensors. You can think of tensors as multidimensional arrays.
<!more>
<div align="center"></div>
Tensor and Variable
Before PyTorch 0.4.0, torch The data type variable exists in the autograd package, which is mainly used to encapsulate Tensor for automatic derivation. Variable mainly contains the following attributes.
 data: packaged Tensor.
 Grad: gradient of data.
 grad_fn: the Function used to create Tensor is the key to automatic derivation, because the derivative can be calculated according to the recorded Function.
 requires_grad: indicates whether a gradient is required. Not all tensors need to calculate the gradient.
 is_leaf: indicates whether the leaf node (tensor) is. The concept of leaf node will be used in the calculation diagram, which will be described in detail later.
<div align="center"></div>
After PyTorch 0.4.0, Variable was incorporated into Tensor. In later versions of Tensor, in addition to the five attributes of the above Variable, there are three other attributes.
 dtype: data type of tensor, such as torch FloatTensor，torch.cuda.FloatTensor.
 Shape: the shape of the tensor. E.g. (64, 3, 224, 224)
 Device: the device where the tensor is located (CPU/GPU). GPU is the key to accelerating computing
<div align="center"></div>
For dtype, PyTorch provides nine data types, which are divided into three categories: float (16 bit, 32bit, 64 bit), integer (unsigned8bit, 8bit, 16 bit, 32bit, 64 bit) and Boolean. The most commonly used type of model parameters and data is float32bit. The commonly used type of label is integer64bit.
<div align="center"></div>
Method created by Tensor
Create Tensor directly
torch.tensor()
torch.tensor(data, dtype=None, device=None, requires_grad=False, pin_memory=False)
 Data: data, which can be list or numpy
 dtype: data type, which is consistent with data by default
 Device: device, cuda/cpu
 requires_grad: do you need gradients
 pin_memory: whether it is stored in lock page memory
Code example:
arr = np.ones((3, 3)) print("ndarray Data type:", arr.dtype) # Create data stored in GPU # t = torch.tensor(arr, device='cuda') t= torch.tensor(arr) print(t)
Output is:
ndarray Data type: float64 tensor([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]], dtype=torch.float64)
torch.from_numpy(ndarray)
Create tensor from numpy. The tensor created by this method shares memory with the original ndarray. When one of the data is modified, the other will also be changed.
<div align="center"></div>
Code example:
arr = np.array([[1, 2, 3], [4, 5, 6]]) t = torch.from_numpy(arr) # If you modify the array, the tensor will also be modified # Print ("\ nmodify arr") # arr[0, 0] = 0 # print("numpy array: ", arr) # print("tensor : ", t) # If you modify the tensor, the array will also be modified print("\n modify tensor") t[0, 0] = 1 print("numpy array: ", arr) print("tensor : ", t)
Output is:
modify tensor numpy array: [[1 2 3] [ 4 5 6]] tensor : tensor([[1, 2, 3], [ 4, 5, 6]], dtype=torch.int32)
Create Tensor based on numeric values
torch.zeros()
torch.zeros(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create all 0 tensors according to size
 size: shape of tensor
 Out: output tensor. If out is specified, torch The tensor returned by zeros () and out point to the same address
 Layout: layout form in memory, including striped and sparse_coo et al. Set to sparse when it is a sparse matrix_ Coo can reduce memory consumption.
 Device: device, cuda/cpu
 requires_grad: do you need gradients
Code example:
out_t = torch.tensor([1]) # out is formulated here t = torch.zeros((3, 3), out=out_t) print(t, '\n', out_t) # id is the memory address. Final t and out_t is the same memory address print(id(t), id(out_t), id(t) == id(out_t))
The output is:
tensor([[0, 0, 0], [0, 0, 0], [0, 0, 0]]) tensor([[0, 0, 0], [0, 0, 0], [0, 0, 0]]) 2984903203072 2984903203072 True
torch.zeros_like
torch.zeros_like(input, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format)
Function: create full 0 tensor according to input shape
 Input: create a total 0 tensor with the same shape as input
 dtype: data type
 Layout: layout form in memory, including striped and sparse_coo et al. Set to sparse when it is a sparse matrix_ Coo can reduce memory consumption.
Similarly, there is the creation method of full 1 tensor: torch ones()，torch.ones_like().
torch.full()，torch.full_like()
torch.full(size, fill_value, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create tensor of custom value
 size: the shape of the tensor, such as (3,3)
 fill_value: the value of each element in the tensor
Code example:
t = torch.full((3, 3), 1) print(t)
Output is:
tensor([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]])
torch.arange()
torch.arange(start=0, end, step=1, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create isometric 1dimensional tensor. Note that the interval is [start, end].
 start: the starting value of the sequence
 End: end value of sequence, open interval, end value cannot be obtained
 step: sequence tolerance, the default is 1
Code example:
t = torch.arange(2, 10, 2) print(t)
Output is:
tensor([2, 4, 6, 8])
torch.linspace()
torch.linspace(start, end, steps=100, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create 1dimensional tensor with equal score. Interval is [start, end]
 start: the starting value of the sequence
 End: end value of sequence
 steps: sequence length (number of elements)
Code example:
# t = torch.linspace(2, 10, 5) t = torch.linspace(2, 10, 6) print(t)
Output is:
tensor([ 2.0000, 3.6000, 5.2000, 6.8000, 8.4000, 10.0000])
torch.logspace()
torch.logspace(start, end, steps=100, base=10.0, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create 1dimensional tensor of logarithmic mean. The value range is [start, end], and the bottom is base.
 start: the starting value of the sequence
 End: end value of sequence
 steps: sequence length (number of elements)
 Base: the base of the logarithmic function. The default value is 10
Code example:
# t = torch.linspace(2, 10, 5) t = torch.linspace(2, 10, 6) print(t)
Output is:
tensor([ 2.0000, 3.6000, 5.2000, 6.8000, 8.4000, 10.0000])
torch.eye()
torch.eye(n, m=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: create unit diagonal matrix (2dimensional tensor), and the default is square matrix
 n: Number of matrix rows. Usually, only n is set as a square matrix.
 m: Number of matrix columns
Create Tensor based on probability
torch.normal()
torch.normal(mean, std, *, generator=None, out=None)
Function: generate normal distribution (Gaussian distribution)
 Mean: mean
 std: standard deviation
There are four modes:

mean is scalar and std is scalar. You need to set the size.
Code example:
# mean: scalar std: scalar # You need to set size here t_normal = torch.normal(0., 1., size=(4,)) print(t_normal)
Output is:
tensor([0.6614, 0.2669, 0.0617, 0.6213])
 mean is scalar and std is tensor

mean is tensor and std is scalar
Code example:
# mean: tensor std: scalar mean = torch.arange(1, 5, dtype=torch.float) std = 1 t_normal = torch.normal(mean, std) print("mean:{}\nstd:{}".format(mean, std)) print(t_normal)
Output is:
mean:tensor([1., 2., 3., 4.]) std:1 tensor([1.6614, 2.2669, 3.0617, 4.6213])
The mean value of the four sampling distributions is different, but the variance is 1.

mean is tensor and std is tensor
Code example:
# mean: tensor std: tensor mean = torch.arange(1, 5, dtype=torch.float) std = torch.arange(1, 5, dtype=torch.float) t_normal = torch.normal(mean, std) print("mean:{}\nstd:{}".format(mean, std)) print(t_normal)
Output is:
mean:tensor([1., 2., 3., 4.]) std:tensor([1., 2., 3., 4.]) tensor([1.6614, 2.5338, 3.1850, 6.4853])
Among them, 1.6614 is sampled from the normal distribution $N(1,1) $, and so on.
torch.randn() and torch randn_ like()
torch.randn(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: generate standard normal distribution.
 size: shape of tensor
torch.rand() and torch rand_ like()
torch.rand(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: generate uniform distribution on interval [0, 1).
torch.randint() and torch randint_ like()
randint(low=0, high, size, *, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Function: generate integer uniform distribution on interval [low, high].
 size: shape of tensor
torch.randperm()
torch.randperm(n, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False)
Function: generate random arrangement from 0 to n1. Commonly used to generate indexes.
 n: Length of tensor
torch.bernoulli()
torch.bernoulli(input, *, generator=None, out=None)
Function: generate Bernoulli distribution (01 distribution, twopoint distribution) with input as probability
 input: probability value
reference material
If you think this article is helpful to you, you might as well like it to give me more motivation to write good articles.
My article will be first published on the official account. Welcome to scan the code to pay attention to my official account classmate Zhang Xian.
<div align="center"></div>