YouTip

## PyTorch torch.broadcast_shapes In PyTorch, `torch.broadcast_shapes` is a utility function used to compute the resulting shape of broadcasting multiple tensor shapes together. Unlike operations such as addition or multiplication which perform element-wise computations on actual tensors, `torch.broadcast_shapes` only operates on the shape tuples themselves. It returns the final broadcasted shape **without** allocating memory or creating actual tensor objects. This makes it highly efficient for shape validation and dry-running tensor operations. --- ## Syntax & Definition ```python torch.broadcast_shapes(*shapes) ``` ### Parameters * **`*shapes`** *(iterable of ints or torch.Size)*: A variable number of shapes (tuples, lists, or `torch.Size` objects) representing the dimensions of the tensors you wish to broadcast. ### Return Value * **`torch.Size`**: The resulting shape after broadcasting the input shapes together. ### Exceptions * **`RuntimeError`**: Raised if the input shapes are incompatible for broadcasting according to PyTorch's broadcasting rules. --- ## Broadcasting Rules Recap Two shapes are compatible for broadcasting if, starting from the trailing (rightmost) dimension: 1. The dimensions are equal, OR 2. One of the dimensions is `1`, OR 3. One of the dimensions does not exist (one shape has fewer dimensions than the other). --- ## Code Examples The following examples demonstrate how to use `torch.broadcast_shapes` under different scenarios. ```python import torch # Example 1: Broadcasting multiple 2D shapes # (3, 1) and (1, 4) can be broadcasted to (3, 4) result1 = torch.broadcast_shapes((3, 1), (1, 4), (3, 4)) print("(3, 1) + (1, 4) + (3, 4) ->", result1) # Output: torch.Size([3, 4]) # Example 2: Broadcasting shapes with different number of dimensions (ND) # The 1D shape (5,) and 2D shape (1, 5) are prepended with 1s to match the 3D shape (3, 1, 5) result2 = torch.broadcast_shapes((5,), (1, 5), (3, 1, 5)) print("(5,) + (1, 5) + (3, 1, 5) ->", result2) # Output: torch.Size([3, 1, 5]) # Example 3: Passing a single shape # If only one shape is provided, it returns that shape as a torch.Size object result3 = torch.broadcast_shapes((2, 3)) print("(2, 3) ->", result3) # Output: torch.Size([2, 3]) # Example 4: Handling incompatible shapes (Error Handling) # (3,) and (4,) cannot be broadcasted because their trailing dimensions are different and neither is 1. try: result4 = torch.broadcast_shapes((3,), (4,)) except RuntimeError as e: print("Broadcasting Error:", e) # Output: Broadcasting Error: Shape mismatch: objects cannot be broadcast to a single shape ``` --- ## Practical Considerations ### 1. Performance Optimization When writing custom PyTorch layers or complex loss functions, you often need to verify if the input tensors can be mathematically combined. Instead of performing a dummy operation like `(a + b).shape` (which allocates memory and performs unnecessary computations), use `torch.broadcast_shapes(a.shape, b.shape)`. It is a pure-metadata operation and executes instantly. ### 2. Equivalent to `torch.broadcast_tensors` Shape The shape returned by `torch.broadcast_shapes(*shapes)` is identical to the shape of the tensors returned by `torch.broadcast_tensors(*tensors)`. ```python # These two approaches yield the same shape information: shape_only = torch.broadcast_shapes((1, 3), (3, 1)) t1, t2 = torch.zeros(1, 3), torch.zeros(3, 1) b1, b2 = torch.broadcast_tensors(t1, t2) assert shape_only == b1.shape == b2.shape ```