YouTipPython3.x Python subprocess Module
\n\nsubprocess is a module in the Python standard library used for creating and managing child processes.
subprocess allows you to execute external commands within a Python program and interact with these commands.
Through the subprocess module, you can execute system commands, call other programs, and obtain their output or error information.
Why use the subprocess module?
\n\nIn Python, sometimes we need to execute some system commands or call other programs to complete specific tasks. For example, you might need to run a shell command, start an external application, or interact with a command-line tool. The subprocess module provides a secure and flexible way to handle these needs.
Compared to older methods like os.system() or os.popen(), the subprocess module offers more powerful features and better control capabilities. It allows you to manage the input, output, and error streams of child processes more finely and can handle more complex scenarios.
\n\n
1. Executing External Commands
\n\nsubprocess.run() is one of the most commonly used functions in the subprocess module. It can execute an external command and wait for the command to complete. Here is a simple example:
Example
\n\nimport subprocess\n\n# Execute a simple shell command\nresult = subprocess.run(['ls', '-l'], capture_output=True, text=True)\n\n# Print the command's output\nprint(result.stdout)\nIn this example, subprocess.run() executes the ls -l command and captures the output into result.stdout.
2. Handling Input and Output
\n\nThe subprocess module allows you to control the input, output, and error streams of the child process. You can pass data to the standard input of the child process or read data from the standard output and standard error of the child process. Here is an example:
Example
\n\nimport subprocess\n\n# Execute a command and pass input to the child process\nresult = subprocess.run(['grep', 'python'], input='hellon pythonn world', capture_output=True, text=True)\n\n# Print the command's output\nprint(result.stdout)\nIn this example, subprocess.run() executes the grep python command and passes the string 'hellonpythonnworld' as input to the child process.
3. Handling Errors
\n\nThe subprocess module also allows you to handle errors from the child process. If the child process returns a non-zero exit status code, subprocess.run() will throw a CalledProcessError exception. You can get the exit status code of the child process by checking result.returncode.
Example
\n\nimport subprocess\n\ntry:\n result = subprocess.run(['ls', 'nonexistent_file'], capture_output=True, text=True, check=True)\nexcept subprocess.CalledProcessError as e:\n print(f"Command failed with return code {e.returncode}")\n print(f"Error output: {e.stderr}")\nIn this example, subprocess.run() executes the ls nonexistent_file command. Since the file does not exist, the command fails and throws a CalledProcessError exception.
\n\n
Advanced Usage of the subprocess Module
\n\n1. Using the Popen Class
\n\nThe subprocess.Popen class provides a lower-level interface, allowing you to control the child process more flexibly. You can use Popen to start a child process and run it in the background, or interact with it.
Example
\n\nimport subprocess\n\n# Start a child process\nprocess = subprocess.Popen(['ping', 'google.com'], stdout=subprocess.PIPE, text=True)\n\n# Read the output of the child process\nwhile True:\n output = process.stdout.readline()\n if output == '' and process.poll() is not None:\n break\n if output:\n print(output.strip())\n\n# Get the exit status code of the child process\nreturn_code = process.poll()\nprint(f"Process finished with return code {return_code}")\nIn this example, subprocess.Popen starts a ping google.com command and runs it in the background. The program reads the output of the child process through a loop and obtains its exit status code after the child process ends.
2. Using Pipes
\n\nThe subprocess module allows you to connect multiple commands together using pipes. You can use the output of one command as the input for another command.
Example
\n\nimport subprocess\n\n# Connect two commands using a pipe\np1 = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)\np2 = subprocess.Popen(['grep', 'py'], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)\n\n# Get the final output\noutput = p2.communicate()\nprint(output)\nIn this example, the output of the ls -l command is passed to the grep py command, and the final output includes files or directories containing py.
\n\n
Common Methods, Classes, and Parameters of the subprocess Module
\n\nThe following describes the common methods, classes, and parameters of the Python subprocess module, including functional descriptions and examples:
\n\nsubprocess Module Core Methods
\n\n| Method | \nDescription | \nExample | \n
|---|---|---|
subprocess.run() | \nExecute command and wait for completion (recommended) | \nsubprocess.run(["ls", "-l"], capture_output=True, text=True) | \n
subprocess.Popen() | \nCreate child process (low-level control) | \nproc = subprocess.Popen(["ping", "google.com"], stdout=subprocess.PIPE) | \n
subprocess.call() | \nExecute command and return exit code (legacy) | \nexit_code = subprocess.call(["python", "--version"]) | \n
subprocess.check_call() | \nExecute command, throw exception on failure | \nsubprocess.check_call(["git", "commit"]) | \n
Minimal subprocess.check_output() | \nExecute command and return output (legacy) | \noutput = subprocess.check_output(, text=True) | \n
subprocess.CompletedProcess Object Attributes (Return object of run() method)
\n\n| Attribute | \nDescription | \n
|---|---|
args | \nExecuted command argument list | \n
| returncode | \nProcess exit status code (0 indicates success) | \n
stdout | \nStandard output content (if capture_output is set) | \n
stderr | \nStandard error content (if capture_output is set) | \n
subprocess.Popen Class Common Methods/Attributes
\n\n| Method/Attribute | \nDescription | \nExample | \n
|---|---|---|
poll() | \nCheck if process has terminated (returns None if running) | \nif proc.poll() is None: print("Running") | \n
wait() | \nBlock and wait for process to end | \nproc.wait() | \n\n
communicate() | \nInteractive input/output | \nstdout, stderr = proc.communicate(input="data") | \n
terminate() | \nSend termination signal (SIGTERM) | \nproc.terminate() | \n
kill() | \nForce terminate process (SIGKILL) | \nproc.kill() | \n
stdin | \nProcess standard input stream | \nproc.stdin.write("input") | \n
stdout | \nProcess standard output stream | \nprint(proc.stdout.read()) | \n
stderr | \nProcess standard error stream | \nerrors = proc.stderr.read() | \n
Common Parameter Descriptions (Applicable to run() and Popen())
\n\n| Parameter | \nDescription | \nExample Value | \n
|---|---|---|
args | \nCommand (list or string) | \n["ls", "-l"] or "ls -l" | \n
stdMinimal in | \nStandard input configuration | \nsubprocess.PIPE (pipe), None (inherit) | \n
stdout | \nStandard output configuration | \nsubprocess.PIPE, open('log.txt', 'w') | \n
stderr | \nStandard error configuration | \nsubprocess.STDOUT (merge to stdout) | \n
shell | \nWhether to execute via Shell | \nTrue (supports string commands) | \n
cwd | \nWorking directory path | \n"/tmp" | \n
env | \nCustom environment variables | \n{"PATH": "/usr/bin"} | \n
timeout | \nTimeout duration (seconds) | \n30 | \n
text | \nWhether input/output is string (not bytes) | \nTrue | \n
Examples
\n\nExecute command and capture output:
\n\nExample
\n\nresult = subprocess.run(["echo", "Hello"], capture_output=True, text=True)\nprint(result.stdout) # Output: "Hellon"\nExecute complex command via Shell:
\n\nsubprocess.run("grep 'error' log.txt | wc -l", shell=True, check=True)\nGet output stream in real-time:
\n\nExample
\n\nproc = subprocess.Popen(["tail", "-f", "log.txt"], stdout=subprocess.PIPE)\nwhile True:\n line = proc.stdout.readline()\n if not line: break\n print(line.decode().strip())\nTimeout control:
\n\nExample
\n\ntry:\n subprocess.run(["sleep", "10"], timeout=5)\nexcept subprocess.TimeoutExpired:\n print("Command timed out!")\n