Running a shell command from Python is one of those practical skills that can instantly make your scripts more powerful. Think about it: whether you want to automate a backup, call a command-line tool, check system information, or interact with Git, Docker, or a package manager, knowing how to execute shell command and get output in Python is essential. The good news is that Python provides several reliable ways to do this, and once you understand the differences between them, you can choose the safest and most efficient method for your task.
In this guide, you will learn how to run shell commands in Python, capture their output, handle errors, and avoid common security mistakes. We will explore the most useful modules and functions, including subprocess, os.Which means system, and os. popen, while focusing on the best practices that professional developers use in real projects.
Why Run Shell Commands from Python?
Python is already excellent for automation, data processing, web development, and scripting. Still, many systems and tools are designed to be used from the command line. To give you an idea, you might need to:
- Run a system utility such as
ls,ping, ordf - Call a version control command like
git status - Execute a container command such as
docker ps - Start a build process using
makeornpm - Retrieve output from a command-line tool for further processing
Instead of manually switching between your terminal and your Python script, you can let Python handle the command execution directly. This makes your workflow faster, more repeatable, and easier to automate.
The main goal is not just to run a command, but also to capture its output so your program can use it. That is why understanding how to execute shell command and get output in Python is more useful than simply knowing how to trigger a command That's the part that actually makes a difference..
The Best Module for Running Shell Commands: subprocess
When most Python developers need to run external commands, the first choice should usually be the subprocess module. Worth adding: it is part of Python’s standard library, so you do not need to install anything extra. It is also flexible, secure when used correctly, and capable of handling input, output, and error streams And it works..
The subprocess module gives you several functions, but the most commonly used ones are:
subprocess.run()subprocess.Popen()subprocess.check_output()subprocess.call()
Each has its own strengths, but subprocess.run() is the best starting point for most beginners because it is simple and modern It's one of those things that adds up..
Using subprocess.run() to Execute a Command and Get Output
The subprocess.run() function is designed to run a command, wait for it to finish, and return a result object. This result object contains useful information such as the return code, standard output, and standard error Simple, but easy to overlook..
Here is a basic example:
import subprocess
result = subprocess.run(
["echo", "Hello from Python"],
capture_output=True,
text=True
)
print(result.stdout)
In this example:
["echo", "Hello from Python"]is the command and its argumentscapture_output=Truetells Python to capture standard output and standard errortext=Trueconverts the output from bytes to a normal string
The output will be:
Hello from Python
This is one of the cleanest ways to execute a shell command and get its output in Python.
Understanding stdout, stderr, and Return Code
Every command-line program has three important communication channels:
- Standard output (
stdout): the normal output of the command - Standard error (
stderr): error messages or diagnostic information - Return code: a number that indicates whether the command succeeded or failed
A return code of 0 usually means success. Any non-zero value usually means something went wrong.
You can inspect all of these using subprocess.run():
import subprocess
result = subprocess.run(
["python", "--version"],
capture_output=True,
text=True
)
print("Output:", result.stdout)
print("Error:", result.stderr)
print("Return code:", result.returncode)
This gives you much more control than simply printing the result. In real applications, checking the return code is often just as important as reading the output No workaround needed..
Running Commands with shell=True
Sometimes people use shell=True when they want to run a command through the system shell. For example:
import subprocess
result = subprocess.run(
"echo Hello from shell",
shell=True,
capture_output=True,
text=True
)
print(result.stdout)
This works, but it should be used carefully. When shell=True is enabled, the command string is interpreted by the shell. That can be convenient for pipelines or shell features, but it also introduces security risks if any part of the command comes from user input.
To give you an idea, this pattern can be dangerous:
user_input = input("Enter filename: ")
subprocess.run(f"cat {user_input}", shell=True)
If the user enters something malicious, the shell may execute unintended commands. This is known as command injection Nothing fancy..
Safer Alternative: Pass a List of Arguments
A safer approach is to avoid shell=True and pass the command as a list:
import subprocess
result = subprocess.run(
["cat", "example.txt"],
capture_output=True,
text=True
)
print(result.stdout)
This method is usually safer because the arguments are not interpreted by the shell in the same way. If you are working with filenames, usernames, or any external input, this is the recommended approach.
Using subprocess.check_output() for Simple Output Capture
If your main goal is simply to get the output of a command and you do not need as much control, subprocess.check_output() can be very convenient Still holds up..
Example:
import subprocess
output = subprocess.check_output(["echo", "Hello"], text=True)
print(output)
This function runs the command and returns its standard output directly. Even so, if the command fails, it raises a CalledProcessError. That means you should handle exceptions if there is any chance the command might fail.
Here is a safer version:
import subprocess
try:
output = subprocess.check_output(
["python", "--version"],
text=True,
stderr=subprocess.STDOUT
)
print(output)
except subprocess.CalledProcessError as e:
print("Command failed with return code", e.
This function is useful for short scripts where you only care about the output and expect the command to succeed.
## Using `subprocess.Popen()` for More Control
When you need more advanced control, such as interacting with a process while it is still running, `subprocess.Popen()` is the right tool. It starts a process and gives you an object that you can use to monitor or communicate with it.
Basic example:
```python
import subprocess
process = subprocess.Which means popen(
["echo", "Hello from Popen"],
stdout=subprocess. PIPE,
stderr=subprocess.
stdout, stderr = process.communicate()
print("Output:", stdout)
print("Error:", stderr)
Beyond the fundamental APIs, real-world scripts often need to handle input streams, environment configuration, or working directory changes. These patterns are straightforward once you
understand how to configure the subprocess call. Let's look at a few common scenarios.
### Providing Input to a Process
Sometimes a command expects data on standard input. You can supply it directly through the `input` parameter of `subprocess.run()`:
```python
import subprocess
result = subprocess.run(
["grep", "error"],
input="info: ok\nerror: disk full\nwarning: low memory\n",
capture_output=True,
text=True
)
print(result.stdout) # error: disk full
This avoids the need to open a separate pipe or write temporary files. For Popen, you would use process.Consider this: communicate(input="... ") instead Simple, but easy to overlook..
Setting Environment Variables
Child processes inherit the current environment by default. Worth adding: to add or override variables, pass a dictionary to env. A common pattern is to copy `os.
import os
import subprocess
custom_env = os.environ.copy()
custom_env["MY_APP_MODE"] = "testing"
subprocess.run(
["python", "script.py"],
env=custom_env,
check=True
)
If you provide a completely new dictionary without copying, the child process will see only those variables. That can be useful in tightly controlled environments, but it often leads to confusing failures when basic tools cannot be found.
Changing the Working Directory
The cwd parameter lets you run a command as if it had been started from a different directory:
import subprocess
result = subprocess.run(
["ls", "-l"],
cwd="/tmp",
capture_output=True,
text=True
)
print(result.stdout)
It's cleaner than embedding cd /tmp && ...Because of that, in a shell string, and it keeps your command arguments separate from directory changes. It also works consistently across platforms That's the whole idea..
Capturing Output While Preserving Order
When you redirect both stdout and stderr to the same pipe using stderr=subprocess.STDOUT, the combined output preserves the order in which the child process wrote to each stream. This is helpful for logging:
result = subprocess.run(
["make", "all"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True
)
print(result.stdout)
Keep in mind that buffering inside the child process can still affect when lines appear, especially for programs that detect whether they are writing to a terminal.
A Note on Timeouts
Long-running or hung processes can stall your script indefinitely. The timeout parameter raises a TimeoutExpired exception if the command does not finish in time:
import subprocess
try:
subprocess.run(
["sleep", "10"],
timeout=2,
check=True
)
except subprocess.TimeoutExpired:
print("Command took too long and was terminated")
For Popen, you would call process.wait(timeout=...Think about it: ) and handle the exception yourself, potentially calling process. kill() afterward And that's really what it comes down to..
Conclusion
The subprocess module gives you a spectrum of tools, from the simple convenience of subprocess.Think about it: run() to the fine-grained control of Popen(). Now, the key takeaways are straightforward: prefer argument lists over shell strings to avoid injection risks, use check=True or explicit exception handling when failures matter, and configure input, environment, and working directory through the dedicated parameters rather than shell workarounds. With these patterns in hand, you can integrate external commands into your Python programs safely and predictably, whether you are writing a quick utility script or a production automation pipeline That's the part that actually makes a difference..
Worth pausing on this one.