YouTip

## PyTorch `torch.xlogy` API Reference `torch.xlogy` is a PyTorch utility function that computes $x \cdot \log(y)$ element-wise. A key feature of this function is how it handles boundary conditions: **if $x$ is 0, it returns 0**, even if $y$ is 0 or negative (which would normally result in `NaN` or undefined values in standard logarithmic calculations). This behavior is particularly useful in machine learning for computing loss functions like cross-entropy and Kullback-Leibler (KL) divergence, where $0 \log(0)$ is mathematically defined as $0$ by convention. --- ## Syntax & Parameters ### Function Signature ```python torch.xlogy(input, other, *, out=None) -> Tensor ``` ### Parameters | Parameter | Type | Description | | :--- | :--- | :--- | | **`input`** | `Tensor` or `Scalar` | The multiplier ($x$). Can be a PyTorch Tensor or a Python scalar. | | **`other`** | `Tensor` or `Scalar` | The argument of the logarithm ($y$). Can be a PyTorch Tensor or a Python scalar. | | **`out`** | `Tensor` (Optional) | The output tensor where the result will be written. | ### Return Value * Returns a `Tensor` containing the element-wise evaluation of $x \cdot \log(y)$. * The output data type is promoted according to PyTorch's type promotion rules (typically floating-point). --- ## Code Examples ### 1. Basic Usage This example demonstrates a simple element-wise calculation of $x \cdot \log(y)$ using two 1D tensors. ```python import torch # Define input tensors x and y x = torch.tensor([1.0, 2.0, 3.0]) y = torch.tensor([2.0, 3.0, 4.0]) # Compute x * log(y) result = torch.xlogy(x, y) print("Result:") print(result) # Output: tensor([0.6931, 2.1972, 4.1589]) ``` ### 2. Handling Zero Inputs ($0 \log(0)$) In standard mathematics, $0 \cdot \log(0)$ is undefined because $\log(0) = -\infty$. However, `torch.xlogy` safely evaluates this to `0.0`. ```python import torch # Standard calculation: 0 * log(0) x = torch.tensor([0.0, 0.0, 1.0]) y = torch.tensor([0.0, 2.0, 0.0]) result = torch.xlogy(x, y) print("Result with zero handling:") print(result) # Output: tensor([0.0000, 0.0000, -inf]) # Note: 0 * log(0) -> 0.0 # 0 * log(2) -> 0.0 # 1 * log(0) -> -inf ``` ### 3. Broadcasting Support `torch.xlogy` supports element-wise broadcasting rules. You can pass a tensor and a scalar, or tensors of different but compatible shapes. ```python import torch # Multiplying a tensor by a scalar log argument x = torch.tensor([1.0, 2.0, 3.0]) scalar_y = 5.0 result_scalar = torch.xlogy(x, scalar_y) print("Scalar broadcasting result:") print(result_scalar) # Output: tensor([1.6094, 3.2189, 4.8283]) # Broadcasting with 2D and 1D tensors matrix_x = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) vector_y = torch.tensor([2.0, 3.0]) result_matrix = torch.xlogy(matrix_x, vector_y) print("\nMatrix broadcasting result:") print(result_matrix) # Output: # tensor([[0.6931, 2.1972], # [2.0794, 4.3944]]) ``` --- ## Key Considerations & Mathematical Properties 1. **Mathematical Convention**: $$\text{xlogy}(x, y) = \begin{cases} 0 & \text{if } x = 0 \\ x \log(y) & \text{otherwise} \end{cases}$$ This is extremely helpful in information theory and probability calculations (e.g., Shannon Entropy: $- \sum p_i \log(p_i)$), where some probabilities $p_i$ might be exactly $0$. 2. **In-place Variant**: If you want to perform the operation in-place and modify the `input` tensor directly, you can use `input.xlogy_(other)`. 3. **Negative Values**: If $y < 0$ and $x \neq 0$, the result will be `NaN` (Not a Number) because the natural logarithm of a negative number is undefined in real numbers.