TensorFlow Example – Regression Problem |

\\n\\n

What is a Regression Problem?

\\n\\n

A regression problem is an important class of problems in machine learning, where the goal is to predict continuous numerical outputs. Unlike classification problems (which predict discrete categories), regression problems predict values within a real number range.

\\n\\n

Common Regression Problem Examples

\\n\\n

\\n
• House price prediction: predicting price based on features such as house area and location
\\n
• Stock price prediction: predicting future stock prices based on historical data
\\n
• Temperature prediction: predicting future temperature based on meteorological data
\\n

\\n\\n

---

\\n\\n

Basic Workflow for Solving Regression Problems with TensorFlow

\\n\\n

\\n\\n

---

\\n\\n

Hands-on: Boston Housing Price Prediction

\\n\\n

1. Prepare the Data

\\n\\n

We will use the classic Boston Housing dataset, which contains 506 samples, each with 13 features:

\\n\\n

from tensorflow.keras.datasets import boston_housing\\n\\n# Load data\\n\\n(train_data, train_targets),(test_data, test_targets)= boston_housing.load_data()\\n\\n# Data normalization (important step)\\n\\n mean = train_data.mean(axis=0)\\n\\n train_data -= mean\\n\\n std = train_data.std(axis=0)\\n\\n train_data /= std\\n\\ntest_data -= mean\\n\\n test_data /= std\\n

\\n\\n

2. Build the Model

\\n\\n

from tensorflow.keras import models\\n\\nfrom tensorflow.keras import layers\\n\\ndef build_model():\\n\\n model = models.Sequential([\\n\\n layers.Dense(64, activation='relu', input_shape=(train_data.shape,)),\\n\\n layers.Dense(64, activation='relu'),\\n\\n layers.Dense(1)# The output layer does not require an activation function\\n\\n])\\n\\nreturn model\\n

\\n\\n

Model Architecture Explanation

\\n\\n

\\n
• Input layer: corresponds to 13 features
\\n
• Two hidden layers: each with 64 neurons, using ReLU activation function
\\n
• Output layer: 1 neuron (predicting house price), no activation function
\\n

\\n\\n

---

\\n\\n

3. Compile the Model

\\n\\n

model = build_model()\\n\\n model.compile(optimizer='rmsprop',\\n\\n loss='mse',# Mean Squared Error\\n\\n metrics=['mae'])# Mean Absolute Error\\n

\\n\\n

Key Parameter Explanation

\\n\\n

\\n
• optimizer: optimizer, controls the learning process\\n
  \\n
  • rmsprop: default choice suitable for most problems
  \\n
  \\n
\\n
• loss: loss function, commonly used for regression problems\\n
  \\n
  • mse (Mean Squared Error): Mean Squared Error
  \\n
  \\n
\\n
• metrics: evaluation metrics\\n
  \\n
  • mae (Mean Absolute Error): Mean Absolute Error
  \\n
  \\n
\\n

\\n\\n

---

\\n\\n

4. Train the Model

\\n\\n

history = model.fit(train_data, train_targets,\\n\\n epochs=100,\\n\\n batch_size=16,\\n\\n validation_split=0.2)\\n

\\n\\n

Parameter Explanation

\\n\\n

\\n
• epochs: number of training rounds
\\n
• batch_size: size of each data batch
\\n
• validation_split: proportion of validation set
\\n

\\n\\n

---

\\n\\n

5. Evaluate the Model

\\n\\n

# Evaluate on the test set\\n\\n test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)\\n\\nprint(f"Test set MAE: {test_mae_score}")\\n

\\n\\n

Interpretation of Evaluation Metrics

\\n\\n

\\n
• MAE (Mean Absolute Error): average absolute difference between predicted and actual values\\n
  \\n
  • For example, MAE=2.5 means the average prediction deviation is $25,000
  \\n
  \\n
\\n
• MSE (Mean Squared Error): penalizes larger errors more heavily
\\n

\\n\\n

---

\\n\\n

6. Use the Model for Prediction

\\n\\n

# Make predictions on new data\\n\\n sample = test_data# Take the first sample from the test set\\n\\n prediction = model.predict(sample.reshape(1, -1))\\n\\nprint(f"Predict prices: {prediction}, Actual prices: {test_targets}")\\n

\\n\\n

---

\\n\\n

Model Optimization Techniques

\\n\\n

1. Adjust Network Architecture

\\n\\n

# Deeper networks may perform better\\n\\ndef build_deeper_model():\\n\\n model = models.Sequential([\\n\\n layers.Dense(128, activation='relu', input_shape=(train_data.shape,)),\\n\\n layers.Dense(64, activation='relu'),\\n\\n layers.Dense(32, activation='relu'),\\n\\n layers.Dense(1)\\n\\n])\\n\\nreturn model\\n

\\n\\n

2. Use K-Fold Cross-Validation

\\n\\n

from sklearn.model_selection import KFold\\n\\nk =4\\n\\n kf = KFold(n_splits=k)\\n\\nfor train_index, val_index in kf.split(train_data):\\n\\n# Split into training and validation sets\\n\\n partial_train_data = train_data\\n\\n partial_train_targets = train_targets\\n\\n val_data = train_data\\n\\n val_targets = train_targets\\n\\n# Train and evaluate models\\n\\n model = build_model()\\n\\n model.fit(partial_train_data, partial_train_targets,\\n\\n epochs=100, batch_size=16, verbose=0)\\n\\n val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\\n\\nprint(f"Validation MAE: {val_mae}")\\n

\\n\\n

3. Add Regularization to Prevent Overfitting

\\n\\n

from tensorflow.keras import regularizers\\n\\nmodel = models.Sequential([\\n\\n layers.Dense(64, activation='relu',\\n\\nkernel_regularizer=regularizers.l2(0.001),\\n\\ninput_shape=(train_data.shape,)),\\n\\n layers.Dense(64, activation='relu',\\n\\nkernel_regularizer=regularizers.l2(0.001)),\\n\\n layers.Dense(1)\\n\\n])\\n

\\n\\n

---

\\n\\n

Common Issues and Solutions

\\n\\n

Issue 1: Unstable Model Performance

\\n\\n

\\n
• Cause: small dataset size or randomness in initialization
\\n
• Solution: increase dataset size or use K-fold cross-validation
\\n

\\n\\n

Issue 2: Low Training Error but High Test Error

\\n\\n

\\n
• Cause: overfitting
\\n
• Solution: add Dropout layers or L2 regularization
\\n

\\n\\n

Issue 3: Predicted Values Deviate Significantly from Actual Values

\\n\\n

\\n
• Cause: data not standardized or unreasonable network architecture
\\n
• Solution: check data preprocessing steps, adjust network depth and width
\\n