[PyTorch learning notes] 1.2 introduction to tensor

thumbnail: https://image.zhangxiann.com/...
toc: true
date: 2020/2/5 20:39:20
disqusId: zhangxian
categories:

  • PyTorch

tags:

  • AI
  • Deep Learning

Code of this chapter:

Tensor concept

Tensor is tensor in Chinese. Tensor means a multidimensional array, which is a high-dimensional extension of scalar, vector and matrix.

Scalars can be called 0-dimensional tensors, vectors can be called 1-dimensional tensors, matrices can be called 2-dimensional tensors, and RGB images can represent 3-dimensional tensors. You can think of tensors as multidimensional arrays.

<!--more-->

<div align="center"></div>

Tensor and Variable

Before PyTorch 0.4.0, torch The data type variable exists in the autograd package, which is mainly used to encapsulate Tensor for automatic derivation. Variable mainly contains the following attributes.

  • data: packaged Tensor.
  • Grad: gradient of data.
  • grad_fn: the Function used to create Tensor is the key to automatic derivation, because the derivative can be calculated according to the recorded Function.
  • requires_grad: indicates whether a gradient is required. Not all tensors need to calculate the gradient.
  • is_leaf: indicates whether the leaf node (tensor) is. The concept of leaf node will be used in the calculation diagram, which will be described in detail later.

<div align="center"></div>

After PyTorch 0.4.0, Variable was incorporated into Tensor. In later versions of Tensor, in addition to the five attributes of the above Variable, there are three other attributes.

  • dtype: data type of tensor, such as torch FloatTensor,torch.cuda.FloatTensor.
  • Shape: the shape of the tensor. E.g. (64, 3, 224, 224)
  • Device: the device where the tensor is located (CPU/GPU). GPU is the key to accelerating computing

<div align="center"></div>

For dtype, PyTorch provides nine data types, which are divided into three categories: float (16 bit, 32-bit, 64 bit), integer (unsigned-8-bit, 8-bit, 16 bit, 32-bit, 64 bit) and Boolean. The most commonly used type of model parameters and data is float-32-bit. The commonly used type of label is integer-64-bit.

<div align="center"></div>

Method created by Tensor

Create Tensor directly

torch.tensor()

torch.tensor(data, dtype=None, device=None, requires_grad=False, pin_memory=False)
  • Data: data, which can be list or numpy
  • dtype: data type, which is consistent with data by default
  • Device: device, cuda/cpu
  • requires_grad: do you need gradients
  • pin_memory: whether it is stored in lock page memory

Code example:

arr = np.ones((3, 3))
print("ndarray Data type:", arr.dtype)
# Create data stored in GPU
# t = torch.tensor(arr, device='cuda')
t= torch.tensor(arr)
print(t)

Output is:

ndarray Data type: float64
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)

torch.from_numpy(ndarray)

Create tensor from numpy. The tensor created by this method shares memory with the original ndarray. When one of the data is modified, the other will also be changed.

<div align="center"></div>

Code example:

arr = np.array([[1, 2, 3], [4, 5, 6]])
t = torch.from_numpy(arr)

# If you modify the array, the tensor will also be modified
# Print ("\ nmodify arr")
# arr[0, 0] = 0
# print("numpy array: ", arr)
# print("tensor : ", t)

# If you modify the tensor, the array will also be modified
print("\n modify tensor")
t[0, 0] = -1
print("numpy array: ", arr)
print("tensor : ", t)

Output is:

modify tensor
numpy array:  [[-1  2  3]
 [ 4  5  6]]
tensor :  tensor([[-1,  2,  3],
        [ 4,  5,  6]], dtype=torch.int32)

Create Tensor based on numeric values

torch.zeros()

torch.zeros(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create all 0 tensors according to size

  • size: shape of tensor
  • Out: output tensor. If out is specified, torch The tensor returned by zeros () and out point to the same address
  • Layout: layout form in memory, including striped and sparse_coo et al. Set to sparse when it is a sparse matrix_ Coo can reduce memory consumption.
  • Device: device, cuda/cpu
  • requires_grad: do you need gradients

Code example:

out_t = torch.tensor([1])
# out is formulated here
t = torch.zeros((3, 3), out=out_t)
print(t, '\n', out_t)
# id is the memory address. Final t and out_t is the same memory address
print(id(t), id(out_t), id(t) == id(out_t))

The output is:

tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]]) 
 tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]])
2984903203072 2984903203072 True

torch.zeros_like

torch.zeros_like(input, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format)

Function: create full 0 tensor according to input shape

  • Input: create a total 0 tensor with the same shape as input
  • dtype: data type
  • Layout: layout form in memory, including striped and sparse_coo et al. Set to sparse when it is a sparse matrix_ Coo can reduce memory consumption.

Similarly, there is the creation method of full 1 tensor: torch ones(),torch.ones_like().

torch.full(),torch.full_like()

torch.full(size, fill_value, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create tensor of custom value

  • size: the shape of the tensor, such as (3,3)
  • fill_value: the value of each element in the tensor

Code example:

t = torch.full((3, 3), 1)
print(t)

Output is:

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])

torch.arange()

torch.arange(start=0, end, step=1, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create isometric 1-dimensional tensor. Note that the interval is [start, end].

  • start: the starting value of the sequence
  • End: end value of sequence, open interval, end value cannot be obtained
  • step: sequence tolerance, the default is 1

Code example:

t = torch.arange(2, 10, 2)
print(t)

Output is:

tensor([2, 4, 6, 8])

torch.linspace()

torch.linspace(start, end, steps=100, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create 1-dimensional tensor with equal score. Interval is [start, end]

  • start: the starting value of the sequence
  • End: end value of sequence
  • steps: sequence length (number of elements)

Code example:

# t = torch.linspace(2, 10, 5)
t = torch.linspace(2, 10, 6)
print(t)

Output is:

tensor([ 2.0000,  3.6000,  5.2000,  6.8000,  8.4000, 10.0000])

torch.logspace()

torch.logspace(start, end, steps=100, base=10.0, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create 1-dimensional tensor of logarithmic mean. The value range is [start, end], and the bottom is base.

  • start: the starting value of the sequence
  • End: end value of sequence
  • steps: sequence length (number of elements)
  • Base: the base of the logarithmic function. The default value is 10

Code example:

# t = torch.linspace(2, 10, 5)
t = torch.linspace(2, 10, 6)
print(t)

Output is:

tensor([ 2.0000,  3.6000,  5.2000,  6.8000,  8.4000, 10.0000])

torch.eye()

torch.eye(n, m=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: create unit diagonal matrix (2-dimensional tensor), and the default is square matrix

  • n: Number of matrix rows. Usually, only n is set as a square matrix.
  • m: Number of matrix columns

Create Tensor based on probability

torch.normal()

torch.normal(mean, std, *, generator=None, out=None)

Function: generate normal distribution (Gaussian distribution)

  • Mean: mean
  • std: standard deviation

There are four modes:

  1. mean is scalar and std is scalar. You need to set the size.

    Code example:

    # mean: scalar std: scalar
    # You need to set size here
    t_normal = torch.normal(0., 1., size=(4,))
    print(t_normal)

    Output is:

    tensor([0.6614, 0.2669, 0.0617, 0.6213])
  2. mean is scalar and std is tensor
  3. mean is tensor and std is scalar

    Code example:

    # mean: tensor std: scalar
    mean = torch.arange(1, 5, dtype=torch.float)
    std = 1
    t_normal = torch.normal(mean, std)
    print("mean:{}\nstd:{}".format(mean, std))
    print(t_normal)

    Output is:

    mean:tensor([1., 2., 3., 4.])
    std:1
    tensor([1.6614, 2.2669, 3.0617, 4.6213])

    The mean value of the four sampling distributions is different, but the variance is 1.

  4. mean is tensor and std is tensor

    Code example:

    # mean: tensor std: tensor
    mean = torch.arange(1, 5, dtype=torch.float)
    std = torch.arange(1, 5, dtype=torch.float)
    t_normal = torch.normal(mean, std)
    print("mean:{}\nstd:{}".format(mean, std))
    print(t_normal)

    Output is:

    mean:tensor([1., 2., 3., 4.])
    std:tensor([1., 2., 3., 4.])
    tensor([1.6614, 2.5338, 3.1850, 6.4853])

    Among them, 1.6614 is sampled from the normal distribution $N(1,1) $, and so on.

torch.randn() and torch randn_ like()

torch.randn(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: generate standard normal distribution.

  • size: shape of tensor

torch.rand() and torch rand_ like()

torch.rand(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: generate uniform distribution on interval [0, 1).

torch.randint() and torch randint_ like()

randint(low=0, high, size, *, generator=None, out=None,
dtype=None, layout=torch.strided, device=None, requires_grad=False)

Function: generate integer uniform distribution on interval [low, high].

  • size: shape of tensor

torch.randperm()

torch.randperm(n, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False)

Function: generate random arrangement from 0 to n-1. Commonly used to generate indexes.

  • n: Length of tensor

torch.bernoulli()

torch.bernoulli(input, *, generator=None, out=None)

Function: generate Bernoulli distribution (0-1 distribution, two-point distribution) with input as probability

  • input: probability value

reference material


If you think this article is helpful to you, you might as well like it to give me more motivation to write good articles.

My article will be first published on the official account. Welcome to scan the code to pay attention to my official account classmate Zhang Xian.

<div align="center"></div>

Tags: Algorithm AI neural networks Pytorch Deep Learning

Posted by sean14592 on Wed, 11 May 2022 06:42:57 +0300