PyTorch torch.zeros Function | Online Tutorial

Pytorch torch Reference Manual

torch.zeros is a function in PyTorch used to create a tensor filled with zeros. It creates a tensor with the specified shape, with all elements initialized to 0.

This is one of the most common ways to initialize tensors in deep learning, often used to create bias vectors, placeholders, or initialize model parameters.

Function Definition

torch.zeros(*size, dtype=None, device=None, requires_grad=False, pin_memory=False)

Parameters:

*size (int): The shape of the tensor, e.g., 3, (3, 4), (2, 3, 4), etc.
dtype (torch.dtype, optional): Specifies the data type of the tensor; defaults to torch.float32.
device (torch.device, optional): Specifies the device on which the tensor will be stored.
requires_grad (bool, optional): Whether to compute gradients for this tensor.
pin_memory (bool, optional): Whether to use pinned (page-locked) memory.

Return Value:

torch.Tensor: Returns a tensor filled with zeros.

Usage Examples

Below are some examples using the torch.zeros function.

Example 1: Create a 1D Zero Tensor

Example

import torch

# Create a 1D zero tensor with 5 elements
x = torch.zeros(5)

print(x)

Output:

tensor([0., 0., 0., 0., 0.])

In this example, we created a 1D zero tensor with 5 elements.

Example 2: Create a 2D Zero Tensor

Example

import torch

# Create a 3x4 zero tensor (matrix)
x = torch.zeros(3, 4)

print(x)
print(x.shape)

Output:

tensor([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]])
torch.Size([3, 4])

In this example, we created a 2D zero tensor with 3 rows and 4 columns.

Example 3: Create a Zero Tensor with Integer Type

Example

import torch

# Create a zero tensor with integer type
x = torch.zeros(3, 3, dtype=torch.int32)

print(x)
print(x.dtype)

Output:

tensor([[0, 0, 0], [0, 0, 0], [0, 0, 0]], dtype=torch.int32)

In this example, we created a zero tensor with integer type, overriding the default float32 with int32.

Example 4: Create a Zero Tensor Requiring Gradients

Example

import torch

# Create a zero tensor that requires gradients
x = torch.zeros(3, requires_grad=True)

print(x.requires_grad)

Output:

True

In this example, we created a zero tensor that requires gradient computation, commonly used for learnable parameters during neural network training.

Example 5: Create a Zero Tensor on CUDA Device

Example

import torch

# Check if CUDA is available
if torch.cuda.is_available():
# Create a zero tensor on the CUDA device
x = torch.zeros(3, 4, device='cuda')
print(x.device)
else:
print("CUDA is not available")

Output:

cuda:0

In this example, we checked if CUDA is available and then created a zero tensor on the GPU.

Difference Between torch.zeros and torch.zeros_like

torch.zeros(*size): Creates a zero tensor based on the specified dimensions.
torch.zeros_like(input): Creates a zero tensor with the same shape, dtype, and device as the input tensor.

Use Cases

The torch.zeros function is commonly used in the following scenarios:

Initialize bias vectors: In neural networks, biases are often initialized to zero.
Create placeholders: Used as input placeholders in dynamic computation graphs.
Fill arrays: Create an array initialized to zero, which is later filled with data.
Mathematical operations: Used in accumulation operations where zero is the starting value.

Notes

By default, torch.zeros creates a tensor of type float32.
If you need a zero tensor with the same shape as another tensor, use torch.zeros_like().
Zero tensors are commonly used for initialization in deep learning, but in some cases, they may lead to vanishing gradient problems.

Pytorch torch Reference Manual

YouTip