YouTip

## PyTorch `torch.randn_like` Tutorial In PyTorch, creating tensors with the same shape as an existing tensor is a common pattern, especially when initializing weights, adding noise, or setting up dummy data. The `torch.randn_like` function is a convenient utility that creates a new tensor filled with random numbers from a **standard normal distribution** (mean of 0 and variance of 1), matching the shape and properties of a given input tensor. --- ## Introduction to `torch.randn_like` Instead of manually querying the shape of an existing tensor and passing it to `torch.randn`, you can use `torch.randn_like`. This function automatically reads the shape of the input tensor and returns a new tensor of the same size, populated with random values sampled from: $$\mathcal{N}(0, 1)$$ By default, the returned tensor will also inherit the data type (`dtype`), device (CPU/CUDA), and layout of the input tensor, unless explicitly overridden. --- ## Syntax and Parameters ### Syntax ```python torch.randn_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) ``` ### Parameters | Parameter | Type | Description | | :--- | :--- | :--- | | **`input`** | *Tensor* | The prototype tensor whose shape and properties will be copied. | | **`dtype`** | *torch.dtype* (Optional) | The desired data type of the returned tensor. If `None` (default), inherits the `dtype` of `input`. | | **`layout`** | *torch.layout* (Optional) | The desired layout of the returned tensor. If `None` (default), inherits the layout of `input`. | | **`device`** | *torch.device* (Optional) | The desired device of the returned tensor (e.g., `'cpu'`, `'cuda'`). If `None` (default), inherits the device of `input`. | | **`requires_grad`** | *bool* (Optional) | If `True`, autograd will record operations on the returned tensor. Default is `False`. | | **`memory_format`** | *torch.memory_format* (Optional) | The desired memory format of the returned tensor. Default is `torch.preserve_format`. | --- ## Code Examples ### Example 1: Basic Usage This example demonstrates how to create a random tensor with the same shape as a 2x2 integer tensor. ```python import torch # Create a 2x2 input tensor input_tensor = torch.tensor([[1, 2], [3, 4]]) # Create a tensor of the same shape filled with standard normal random values random_tensor = torch.randn_like(input_tensor, dtype=torch.float32) print("Input Tensor:") print(input_tensor) print("\nGenerated Random Tensor:") print(random_tensor) ``` **Output:** ```text Input Tensor: tensor([[1, 2], [3, 4]]) Generated Random Tensor: tensor([[-0.1245, 1.4302], [-0.5891, 0.0234]]) ``` *Note: Because the input tensor was of integer type (`torch.int64`), we explicitly specified `dtype=torch.float32` since standard normal values are floating-point numbers.* --- ### Example 2: Inheriting Device and Type Properties When working with GPUs, `torch.randn_like` is highly efficient because it automatically places the new tensor on the same device as the input tensor. ```python import torch # Check if CUDA is available and set the device device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # Create a tensor on the selected device x = torch.empty(3, 3, dtype=torch.float64, device=device) # Create a random tensor inheriting the same device (GPU/CPU) and dtype (float64) y = torch.randn_like(x) print(f"Tensor y device: {y.device}") print(f"Tensor y dtype: {y.dtype}") print(f"Tensor y shape: {y.shape}") ``` --- ### Example 3: Adding Gaussian Noise to a Tensor A common use case for `torch.randn_like` is adding Gaussian noise to an image or feature map during model training. ```python import torch # Simulate a batch of 1 image with 3 channels, 4x4 pixels clean_image = torch.ones(1, 3, 4, 4) # Define noise scale (standard deviation) noise_level = 0.1 # Generate noise with the same shape as the image noise = torch.randn_like(clean_image) * noise_level # Add noise to the image noisy_image = clean_image + noise print("Noisy Image Tensor:") print(noisy_image) # Displaying the first channel ``` --- ## Key Considerations 1. **Data Type Compatibility**: If your input tensor is an integer type (e.g., `torch.int32` or `torch.int64`), calling `torch.randn_like(input)` without specifying `dtype` will result in a `RuntimeError` because normal distribution values are continuous floats. Always override the `dtype` to a floating-point type (e.g., `torch.float32` or `torch.float64`) when working with integer inputs. 2. **Performance**: Using `torch.randn_like(input)` is cleaner and more readable than writing `torch.randn(input.size(), dtype=input.dtype, device=input.device)`. It also helps prevent device mismatch errors in multi-GPU setups.