Let’s throw some “Torch” on Tensor Operations

5 min readJan 22, 2021

Discussing on 5 Basic most used Tensor Operations

Deep learning allows us to carry out a very wide range of complicated tasks. In order to carry out our tasks effectively, we need a tool which is flexible. Pytorch gives us this option because of its simplicity. It provides accelerated operations using GPU’s (Graphical Processing Units). Due to this reason of Pytorch being a high performance library that it has gained its popularity. The below notebook consists of some essential functions which are very useful in carrying out tensor operations. These operations are used for multi-dimensional tensors & for arithmetic operations.

General Ops — Inverse
Creation Ops — Complex
Arithmetic Ops — Transpose
Mutating Ops — Add
Reduction Ops — Amax

We will discuss the examples of these 5 Basic functions & observe the errors. Before we begin, let’s install and import PyTorch

# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html# Import torch and other required modules
import torch

1. General Operations — Inverse Function

First function we will be using is the ‘Inverse’ function.

a = torch.randn(2,3,3)
print(a)
torch.inverse(a)

The above 'randn' function has created 3X3 square matrix with 2 outer most rows. The 'inverse' function then takes the inverse of the individual elements of the matrix

a = torch.rand(4,4)
print(a)
torch.inverse(a)

The above is 4X4 square matrix where each element is inversed using the inverse function.

a = torch.rand(4,3)
print(a)
torch.inverse(a)RuntimeError: A must be batches of square matrices, but they are 3 by 4 matrices

The third example given above throws an error. ‘Inverse’ function gives the inverse of the individual elements of the matrix. The error is due to the fact that the matrix is not square. By changing the dimensions, one can obtain the correct result.

Inverse function is very useful for performing an inverse function on the Pytorch neural network

2. Creation Operations — Complex Function

Lets use functions for initializing working on tensor data by creating vector or matrix. Here we will be using complex function.

In order to get the resultant complex64, we need to input float32 type.

real = torch.tensor([2,1], dtype=torch.float32)
imag = torch.tensor([2,3], dtype=torch.float32)
a= torch.complex(real, imag)
a.dtype

Here below, we have created ‘real’ & ‘imag’ named tensors using rand function. Using the ‘complex’ function we have joined the two tensors & formed a single equation having real & imaginary numbers

real = torch.rand(2,3)
imag = torch.rand(1,3)
print(real)
print(imag)
x = torch.complex(real, imag,)
print(x)

Here, in the below example, instead of creating a complex tensor with two values ‘real’ & ‘imag’ data, it is trying to create a single complex tensor. We could see the above error just because of missing a single square bracket that would have given us the required result.

real = torch.tensor(2., 4)
imag = torch.tensor(7., 3)
x = torch.complex(real, imag,)
xTypeError: tensor() takes 1 positional argument but 2 were given

We can use the above function for creating complex tensors consisting of real & imaginary data.

3. Mutating Tensor Data — Transpose Function

Here we will use transpose function to work on mutating tensor data for making our operations easy.

a = torch.rand(2,3,5)
print(a)
torch.transpose(a,1,2)

From the outer most row1, we have transposed all the elements of the first row.

a = torch.rand(2,5)
print(a)
torch.transpose(a, -1, 0)

Here, in the above case, we are giving the first dimension & the second dimension to be transposed.

a = torch.rand(2,5)
print(a)
torch.transpose(a)TypeError: transpose() received an invalid combination of arguments - got (Tensor), but expected one of:  * (Tensor input, name dim0, name dim1)  * (Tensor input, int dim0, int dim1)

While using the transpose function on the tensor data, we also have to pass the dimensions in order to clarify which dimensions need to be transposed. The above function would have worked accurately if we have used ‘t’ instead of ‘transpose’ function.

We can use ‘transpose’ function when we have to tranpose the given dimensions of tensor data while specifying whichn dimensions need to be transposed

Arithematic Operations — Add Function

Lets perform some arithematic operations — add function on our tensor data.

a = torch.randn(10)
print(a)
torch.add(a,5)

The second attribute (5 in above case) should be an integer which must be added to the tensor data (a in above case). The resultant will be the sum of two.

a = torch.rand(5)
b = torch.rand(5)
print(a)
print(b)
torch.add(a,b)

‘Add’ function computes the sum of two tensor data of the same dimensions & gives the result in the same dimension.

a = torch.rand(10)
b = torch.rand(5)
torch.add(a,b)RuntimeError: The size of tensor a (10) must match the size of tensor b (5) at non-singleton dimension 0

While performing any arithematic operations in tensor, we need to take care that the dimensions of the input tensors matches each other.

‘Add’ function can be used to add any two given tensors, or also to add a tensor data with a given number.

Reduction Operations — Amax Function

Using certain Reduction operations — amax. These will help in performing statistical operations on our tensor data. Here, in the below example, ‘amax’ function is used to give the maximum element in the each dimension, where ‘-1’ shows the dimension to be reduced.

a = torch.rand(3,2)
print(a)
torch.amax(a, dim = -1)

Also, in below case, ‘amax’ function gives us the maximum value in the tensor data for each for each slice.

a = torch.rand(5)
print(a)
torch.amax(a, dim=-2)

In the below case, the dimensions of amax function varies from -1 to 0. Hence, ‘dim’ attribute must be within this range.

a = torch.tensor([[3,2], [1,2], [4,7],[6,5]])
print(a)
torch.amax(a, dim = 1)IndexError: Dimension out of range (expected to be in range of [-1, 0], but got -2)

Conclusion

In this topic, we have covered the necessary functions from creation of tensor data to carrying out arithematic operations. We have also carried out mutating operations which are very much necessary for the general understanding.

Follow the below link for the complete codes in my github account: https://github.com/Anchit13/5-Basic-Tensor-Operations/blob/main/01-tensor-operations.ipynb

Reference Links

Provide links to your references and other interesting articles about tensors

Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf