YouTip

## PyTorch torch.addcmul The `torch.addcmul` function in PyTorch performs an element-wise product of two tensors (`tensor1` and `tensor2`), multiplies the result by a scalar multiplier (`value`), and adds it to the input tensor (`input`). This operation is highly optimized and commonly used in custom optimization algorithms (like Adam or RMSprop), neural network layers, and physics simulations where fused multiply-add operations are required. --- ## Mathematical Formula The operation is defined mathematically as: $$\text{out}_i = \text{input}_i + \text{value} \times (\text{tensor1}_i \times \text{tensor2}_i)$$ --- ## Syntax ```python torch.addcmul(input, tensor1, tensor2, *, value=1, out=None) -> Tensor ``` ### Parameter Descriptions | Parameter | Type | Description | | :--- | :--- | :--- | | `input` | `Tensor` | The tensor to be added to. | | `tensor1` | `Tensor` | The first tensor in the element-wise multiplication. | | `tensor2` | `Tensor` | The second tensor in the element-wise multiplication. | | `value` | `Number` (Optional) | The scalar multiplier for $\text{tensor1} \times \text{tensor2}$. Default is `1`. | | `out` | `Tensor` (Optional) | The output tensor where the result will be stored. | --- ## Code Examples ### Example 1: Basic Usage with 1D Tensors This example demonstrates the basic element-wise calculation using 1D tensors. ```python import torch # Create input tensors input_tensor = torch.tensor([1.0, 2.0, 3.0]) tensor1 = torch.tensor([2.0, 3.0, 4.0]) tensor2 = torch.tensor([3.0, 4.0, 5.0]) # Execute: input + value * (tensor1 * tensor2) # Calculation: # [1.0 + 1 * (2.0 * 3.0), 2.0 + 1 * (3.0 * 4.0), 3.0 + 1 * (4.0 * 5.0)] result = torch.addcmul(input_tensor, tensor1, tensor2, value=1) print("Result:") print(result) ``` **Output:** ```text Result: tensor([ 7., 14., 23.]) ``` --- ### Example 2: Using a Custom Scalar Multiplier (`value`) You can scale the product of `tensor1` and `tensor2` by specifying the `value` parameter. ```python import torch input_tensor = torch.tensor([10.0, 10.0, 10.0]) tensor1 = torch.tensor([2.0, 2.0, 2.0]) tensor2 = torch.tensor([3.0, 4.0, 5.0]) # Execute: input + 0.5 * (tensor1 * tensor2) # Calculation: # [10.0 + 0.5 * (2.0 * 3.0), 10.0 + 0.5 * (2.0 * 4.0), 10.0 + 0.5 * (2.0 * 5.0)] result = torch.addcmul(input_tensor, tensor1, tensor2, value=0.5) print("Result with value=0.5:") print(result) ``` **Output:** ```text Result with value=0.5: tensor([13., 14., 15.]) ``` --- ### Example 3: In-place Operation (`addcmul_`) If you want to modify the `input` tensor in-place without allocating new memory, use the in-place version `addcmul_`. ```python import torch input_tensor = torch.tensor([1.0, 1.0, 1.0]) tensor1 = torch.tensor([2.0, 3.0, 4.0]) tensor2 = torch.tensor([5.0, 5.0, 5.0]) # Perform in-place addition and multiplication input_tensor.addcmul_(tensor1, tensor2, value=2) print("Modified input_tensor:") print(input_tensor) ``` **Output:** ```text Modified input_tensor: tensor([21., 31., 41.]) ``` --- ## Important Considerations 1. **Broadcasting Support**: The shapes of `input`, `tensor1`, and `tensor2` must be broadcastable to a common shape. If their shapes do not match exactly, PyTorch will attempt to broadcast them according to standard broadcasting rules. 2. **Data Types**: The data types of `input`, `tensor1`, and `tensor2` must be compatible. If they are different, PyTorch will attempt to promote them to a common type (e.g., promoting `float32` and `float64` to `float64`).