YouTip

Before using Selenium, we need to install it first. The following are the detailed steps to install Selenium. Selenium supports multiple programming languages, but this tutorial series will use Python as an example. If you haven't installed Python yet, you can follow these steps to install it: 1. Visit the (https://www.python.org/). 2. Download the Python installer suitable for your operating system. 3. Run the installer, and make sure to check the "Add Python to PATH" option during the installation process. 4. After installation is complete, open the command line (Command Prompt or PowerShell on Windows, Terminal on macOS and Linux), enter `python --version` or `python3 --version` to check if Python is installed successfully. For more about Python, refer to: (#) After installing Python, we can use `pip` to install Selenium. `pip` is Python's package management tool used to install and manage Python libraries. Open the command line and enter the following command to install Selenium: pip install selenium If you are using Python3, you may need to use `pip3`: pip3 install selenium After installation, you can check if Selenium is installed successfully by running the following command: pip show selenium If the Selenium version information is displayed, the installation was successful. ### Download Browser Driver Selenium needs to control browsers through browser drivers. Different browsers require different drivers. Here are the download addresses for common browser drivers: * **Chrome**: (https://sites.google.com/chromium.org/driver/), for ChromeDriver documentation see: (#). * **Firefox**: (https://github.com/mozilla/geckodriver/releases) * **Edge**: (https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/) Download the driver that matches your browser version, and add its path to the system's environment variables, or place the driver file in the same directory as your Python script. * * * ## Using Selenium After installing Selenium and the browser driver, we can start writing automation scripts. The following is a simple example showing how to use Selenium to open a webpage and get the page title. ### 1、Import Selenium First, we need to import the `webdriver` module from Selenium in the Python script: from selenium import webdriver ### 2、Start Browser Next, we need to start a browser instance. This series will use Chrome as an example. driver = webdriver.Chrome(executable_path='path/to/chromedriver') Select the browser and initialize the WebDriver: ## Example from selenium import webdriver # Use Chrome browser driver = webdriver.Chrome(executable_path='/path/to/chromedriver') # Or use Firefox browser # driver = webdriver.Firefox(executable_path='/path/to/geckodriver') # Or use Edge browser # driver = webdriver.Edge(executable_path='/path/to/msedgedriver') **Starting from Selenium 4, there are changes in the way browser drivers are managed: Selenium 4 attempts to automatically detect the browser version installed on the system and download the corresponding driver, which means users no longer need to manually download and set the driver path unless they need a specific version of the driver.** ## Example from selenium import webdriver driver = webdriver.Chrome()# If using other browsers like Firefox, modify accordingly When in China's network environment, the automatic driver detection and download requires a different network environment, so it is recommended to manually download the driver and then specify the driver path. In Selenium 4, the driver path is no longer set directly in webdriver.Chrome, but through introducing a Service object. This avoids deprecation warnings and ensures proper loading of the driver. For example: ## Example from selenium import webdriver from selenium.webdriver.chrome.service import Service as ChromeService service = ChromeService(executable_path="PATH_TO_DRIVER") options = webdriver.ChromeOptions() driver = webdriver.Chrome(service=service, options=options) If you have placed `chromedriver` in the system path, or in the same directory as your Python script, you can omit the `executable_path` parameter: driver = webdriver.Chrome() ### 3、Open Webpage Use the `get` method to open a webpage: driver.get('https://www.baidu.com') ### 4、Get Page Title We can use the `title` attribute to get the title of the current page: print(driver.title) ### 5、Close Browser After completing the operation, remember to close the browser: driver.quit() ### 6、Complete Example Combining the above steps, the complete code is as follows: ## Example from selenium import webdriver from selenium.webdriver.chrome.service import Service as ChromeService # Set the correct driver path service = ChromeService(executable_path="/TUTORIAL/Downloads/chromedriver-mac-arm64/chromedriver") options = webdriver.ChromeOptions() driver = webdriver.Chrome(service=service, options=options) # Open a website driver.get("https://www.baidu.com") # Get page title print(driver.title) # Close browser driver.quit() After the above code executes successfully, it will open the browser and open the webpage, as shown below: !(#) After opening, it will output the webpage title: Baidu, who knows ### Use window.stop() to Force Stop Page Loading Sometimes a webpage loads too slowly, which can cause exceptions. To successfully scrape data, you can let the page load for a few seconds and then execute the JavaScript code window.stop() to force stop the page loading. ## Example from selenium import webdriver from selenium.webdriver.chrome.service import Service as ChromeService # Set the correct driver path service = ChromeService(executable_path="/TUTORIAL/Downloads/chromedriver-mac-arm64/chromedriver") options = webdriver.ChromeOptions() driver = webdriver.Chrome(service=service, options=options) # Open a website driver.get("https://www.baidu.com") # Wait for page to load time.sleep(2)# You can adjust the wait time as needed # Force stop page loading driver.execute_script("window.stop();") # Get page title print(driver.title) # Close browser driver.quit() * * * ## Common Operations In addition to opening webpages and getting titles, Selenium supports many other operations. Here are some common operation examples: ### 1、Find Element You can use the `find_element` method to find elements on the page. For example, find an input box and enter text: ## Example input_element = driver.find_element('name','q') input_element.send_keys('Selenium') ### 2、Click Button You can use the `click` method to click a button: ## Example button_element = driver.find_element('name','btnK') button_element.click() ### 3、Wait for Page to Load Sometimes page elements don't load immediately. We can use `WebDriverWait` to wait for elements to appear: ## Example from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC wait = WebDriverWait(driver,10) element = wait.until(EC.presence_of_element_located((By.NAME,'q')))