Listing files in a directory is one of the most fundamental tasks in Python scripting and application development. Whether you are building a data processing pipeline, organizing a media library, or writing a simple automation script, the ability to scan a folder and retrieve file names efficiently is essential. 4. Python offers several modules to accomplish this, ranging from the classic os module to the modern, object-oriented pathlib library introduced in Python 3.Understanding the strengths and nuances of each approach allows developers to write cleaner, faster, and more maintainable code.
The Modern Standard: Using pathlib
Since Python 3.4, the pathlib module has been the recommended way to handle filesystem paths. But it treats paths as objects rather than strings, providing a more intuitive and cross-platform API. For developers starting new projects, this should be the default choice Worth knowing..
The primary method for listing directory contents is Path.iterdir(). It yields Path objects for each entry in the directory, allowing immediate access to properties like name, suffix, and size without further string manipulation.
from pathlib import Path
# Define the target directory
target_dir = Path(".") # Current directory
# Iterate over entries
for entry in target_dir.iterdir():
print(entry.name)
This snippet prints the name of every file and subfolder in the current working directory. Because entry is a Path object, you can easily filter results. Here's one way to look at it: to list only files (excluding directories), you can use the is_file() method:
for entry in target_dir.iterdir():
if entry.is_file():
print(f"File: {entry.name}, Size: {entry.stat().st_size} bytes")
Recursive Listing with rglob and glob
One of pathlib's most powerful features is pattern matching. On top of that, the glob() method matches patterns in the current directory, while rglob() (recursive glob) traverses the entire directory tree. This is incredibly useful for finding specific file types, such as all Python scripts or images The details matter here. Surprisingly effective..
# Find all .txt files in current directory only
txt_files = target_dir.glob("*.txt")
# Find all .py files recursively in subdirectories
python_files = target_dir.rglob("*.py")
for py_file in python_files:
print(py_file.relative_to(target_dir))
Using relative_to() helps display clean paths relative to your project root, which is ideal for logging or user-facing output.
The Classic Approach: The os Module
Before pathlib, the os module was the standard for filesystem interaction. It remains widely used in legacy codebases and is perfectly valid for simple scripts. Its functions operate on string paths, which some developers find familiar and lightweight Not complicated — just consistent..
The most common function is os.listdir(). It returns a list of strings representing the names of entries in the directory The details matter here..
import os
entries = os.listdir(".")
for entry in entries:
print(entry)
Important Note: os.listdir() returns only the base names, not full paths. To get the full path or check file attributes, you must join the directory path with the entry name using os.path.join().
import os
dir_path = ".Think about it: "
for entry in os. listdir(dir_path):
full_path = os.On the flip side, path. In practice, join(dir_path, entry)
if os. path.isfile(full_path):
print(f"File: {entry}")
elif os.path.
### Walking the Tree with `os.walk()`
For recursive directory traversal, `os.On top of that, walk()` is the workhorse. That's why it generates the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at the directory top (including top itself), it yields a 3-tuple: `(dirpath, dirnames, filenames)`.
Counterintuitive, but true.
```python
import os
for root, dirs, files in os.On top of that, walk(". So naturally, "):
level = root. replace(".Now, ", ""). count(os.sep)
indent = " " * 4 * level
print(f"{indent}{os.path.
This function is highly efficient for deep directory scans because it yields results incrementally rather than building a massive list in memory all at once.
## High-Performance Scanning: `os.scandir()`
Introduced in Python 3.Which means scandir()` is a directory iteration function that returns an iterator of `os. 5, `os.DirEntry` objects. Think about it: it is significantly faster than `os. listdir()` when you need file type or attribute information because the `DirEntry` objects expose this data without requiring additional system calls (syscalls) in many operating systems.
If you are scanning directories with thousands of files and need to check `is_file()` or `is_dir()`, `os.scandir()` is the performance winner among the standard library's string-based tools.
```python
import os
with os.scandir(".") as entries:
for entry in entries:
if entry.is_file():
print(f"File: {entry.On the flip side, name} ({entry. stat().st_size} bytes)")
elif entry.is_dir():
print(f"Dir: {entry.
Using the `with` statement ensures the underlying file descriptor is closed promptly. Plus, the `entry. path.In practice, stat()` call on a `DirEntry` object is often cached, making it much faster than calling `os. getsize()` on a string path.
## Filtering and Practical Patterns
In real-world applications, raw listing is rarely the end goal. You typically need to filter by extension, size, modification date, or name pattern.
### Filtering by Extension
With `pathlib`, filtering is expressive and readable:
```python
from pathlib import Path
images = Path(".jpg")
# Or multiple extensions
media_files = [f for f in Path(".").jpg", ".That said, rglob("*. rglob("*") if f.Because of that, lower() in {". png", ".suffix.").mp4", ".
With `os` or `glob` module (a separate utility module for Unix-style pathname pattern expansion):
```python
import glob
# glob.glob returns a list of strings (paths)
jpg_files = glob.glob("**/*.jpg", recursive=True)
Filtering by Modification Time
Finding files modified in the last 24 hours is a common administrative task.
import time
from pathlib import Path
now = time.time()
day_ago = now - 86400 # 24 hours in seconds
recent_files = [
f for f in Path(".rglob("*")
if f.is_file() and f.In real terms, "). stat().
for f in recent_files:
print(f"{f.name} - Modified: {time.ctime(f.stat().st_mtime)}")
Handling Errors and Permissions
Filesystem operations are prone to PermissionError, FileNotFoundError, and OSError. solid code anticipates these. When using os.walk() or pathlib.rglob(), an error in a subdirectory can halt the entire iteration if not handled Surprisingly effective..
With os.walk(), you can modify the dirnames list in-place to prevent descending into problematic folders, or wrap the loop in a try-except block.
import os
for root, dirs, files in os.That said, startswith('. walk(".And )
dirs[:] = [d for d in dirs if not d. ')]
try:
for file in files:
print(os."):
# Example: Skip hidden directories (starting with .path.
denied: {os.path.join(root, file)}")
except OSError as e:
print(f"OS error while processing {root}: {e}")
# Using the onerror callback with os.walk for centralized error handling
def walk_error_handler(exception):
print(f"Skipping {exception.filename} due to {exception.strerror}")
for root, dirs, files in os.walk(".", onerror=walk_error_handler):
# Optionally prune directories you don't want to descend into
dirs[:] = [d for d in dirs if not d.startswith('.And ') and not d == '__pycache__']
for file in files:
try:
full_path = os. On top of that, path. join(root, file)
# Perform any needed checks here
if os.path.
# pathlib also offers a way to handle errors via try/except around iteration
from pathlib import Path
def safe_rglob(root_path, pattern="*"):
for entry in Path(root_path).rglob(pattern):
try:
yield entry
except PermissionError:
print(f"Permission denied: {entry}")
except OSError as e:
print(f"OS error accessing {entry}: {e}")
for p in safe_rglob(".", "*.txt"):
if p.is_file():
print(p)
## Performance Tips Recap
- **Prefer `os.scandir()`** when you only need a shallow listing and want to avoid extra `stat()` calls.
- **make use of `pathlib`** for readable, chainable operations; its `rglob()` is convenient for recursive searches.
- **Cache `stat()` results** if you need multiple attributes (size, mtime, mode) from the same entry.
- **Prune directories early** (modify `dirs` in `os.walk` or skip unwanted paths in `pathlib`) to reduce I/O.
- **Handle errors gracefully** with `try/except` blocks or the `onerror` callback in `os.walk` to prevent a single faulty directory from aborting the whole walk.
## Conclusion
Choosing the right filesystem traversal tool depends on the balance between performance, readability, and error‑handling needs. For high‑performance, low‑level scans, `os.scandir()` paired with manual filtering is unbeatable. When code clarity and expressive pattern matching matter more, `pathlib` offers a modern, object‑oriented interface that integrates well with Python’s standard libraries. Regardless of the approach, always anticipate permission issues and transient errors, prune unnecessary branches early, and reuse `stat()` information when possible. By following these patterns, you can efficiently and safely process thousands of files without sacrificing maintainability.