Copying files from one location to another is a fundamental operation in almost every Python automation script, data processing pipeline, or system administration tool. In real terms, whether you are building a backup utility, organizing a dataset, or deploying configuration files, understanding the nuances of the standard library modules available for this task is essential. Python offers several dependable ways to handle file system operations, primarily through the shutil module for high-level tasks and the os module for lower-level control, with the modern pathlib module providing an object-oriented approach that improves readability and cross-platform compatibility.
The Primary Tool: The shutil Module
The shutil (shell utilities) module is the standard, "batteries-included" way to copy files in Python. It provides high-level operations that abstract away the complexities of opening file handles, reading chunks, and writing bytes manually. For most use cases, this is the module you should reach for first.
Using shutil.copy() for Basic Duplication
The most common function is shutil.Plus, copy(src, dst). This function copies the file content and the file’s permission mode (read/write/execute bits) to the destination. It does not copy metadata such as creation time, modification time, or extended attributes.
If the destination (dst) is a directory, the file is copied into that directory using the original filename. If dst is a full file path, the file is copied and potentially renamed.
import shutil
import os
source_file = 'data/report.txt'
destination_dir = 'backup/'
# Ensure destination exists
os.makedirs(destination_dir, exist_ok=True)
# Copy the file
shutil.copy(source_file, destination_dir)
print(f"Copied {source_file} to {destination_dir}")
Key Behavior: If a file with the same name already exists in the destination, shutil.copy() will silently overwrite it. This is a critical detail to remember for data integrity.
Using shutil.copy2() for Metadata Preservation
When building backup tools or synchronization scripts, preserving metadata is often a requirement. shutil.copy2(src, dst) functions identically to copy() but attempts to preserve all file metadata. Under the hood, it calls copystat() after copying the content But it adds up..
On Windows, this copies file creation time, modification time, and attributes. On POSIX systems (Linux, macOS), it copies permission bits, last access time, last modification time, and flags Small thing, real impact..
# Preserves timestamps and permissions
shutil.copy2('data/report.txt', 'backup/report.txt')
Best Practice: Default to copy2() for archival or backup scripts. Use copy() only when you explicitly want to reset timestamps or save minimal system call overhead.
Using shutil.copyfile() for Stream Control
shutil.On the flip side, it copies the *contents* of the file from srctodstbut **requiresdst to be a complete target filename** (not a directory). copyfile(src, dst) operates at a slightly lower level. It does not copy metadata or permission bits; the destination file gets default permissions based on the current umask.
This changes depending on context. Keep that in mind.
This is useful when you need precise control over the output filename or when piping data between file-like objects isn't necessary, but you want to avoid the metadata overhead of copy2() Surprisingly effective..
# Destination MUST be a filename, not a directory
shutil.copyfile('source.log', 'archive/log_2023_10_01.log')
Using shutil.copyfileobj() for Memory Efficiency
For extremely large files (multi-gigabyte datasets), reading the entire file into memory is impossible. shutil.Here's the thing — copyfileobj(fsrc, fdst[, length]) copies data between file objects in chunks. And that's what lets you control the buffer size, keeping memory usage constant regardless of file size.
with open('huge_dataset.bin', 'rb') as fsrc:
with open('backup/huge_dataset.bin', 'wb') as fdst:
# Copy in 1MB chunks (default is 16KB)
shutil.copyfileobj(fsrc, fdst, length=1024*1024)
This method gives you the granular control needed for high-performance I/O operations without loading the full file into RAM And that's really what it comes down to. And it works..
The Modern Approach: pathlib (Python 3.4+)
Since Python 3.4, the pathlib module has offered an object-oriented filesystem path interface. And while pathlib Path objects do not have a direct . copy() method (a deliberate design choice to keep the API focused on path manipulation), they integrate easily with shutil Surprisingly effective..
The modern, Pythonic pattern combines pathlib for path resolution and shutil for the action.
from pathlib import Path
import shutil
src = Path('project/config.yaml')
dst_dir = Path('/etc/myapp/')
# Resolve paths (handles symlinks, relative paths)
src_resolved = src.resolve()
dst_dir.mkdir(parents=True, exist_ok=True) # Create dir if missing
# Use shutil with Path objects (auto-converted to strings in Py3.6+)
shutil.copy2(src_resolved, dst_dir / src_resolved.name)
print(f"Configuration deployed to {dst_dir}")
Why prefer this?
- Cross-platform paths:
Pathhandles Windows backslashes and POSIX forward slashes automatically. - Readability:
dst_dir / src.nameis instantly readable. - Safety: Methods like
.resolve(),.exists(), and.is_file()allow for strong pre-flight checks before copying.
Handling Directories Recursively
Often, the requirement is not a single file but an entire directory tree. shutil provides copytree() for this exact scenario Simple, but easy to overlook..
import shutil
shutil.copytree('source_folder', 'destination_folder', dirs_exist_ok=True)
dirs_exist_ok=True(Python 3.8+): By default,copytreeraises an error if the destination exists. This flag allows merging trees, similar tocp -rin Unix.ignoreparameter: Accepts a callable (likeshutil.ignore_patterns('*.pyc', '__pycache__')) to skip specific files during the copy.copy_function: Allows you to specifyshutil.copy,shutil.copy2, or a custom function to control how individual files are copied.
solid Error Handling and Pre-flight Checks
Production code cannot assume the source exists or the destination is writable. Wrapping copy operations in try...except blocks and checking preconditions prevents silent failures or cryptic tracebacks.
Common Exceptions to Catch
| Exception | Cause |
|---|---|
FileNotFoundError |
Source file or parent directory of destination does not exist. |
PermissionError |
Insufficient read permissions on source or write permissions on destination. |
IsADirectoryError |
Using copyfile() with a directory as source or destination. Here's the thing — |
SameFileError |
Source and destination resolve to the same file (raised by shutil. That said, copy2/copy). |
OSError / IOError |
Generic system errors (disk full, network drive disconnected, etc.). |
A Production-Ready Wrapper Function
Here is a pattern for a reusable, safe copy function:
import shutil
from pathlib import Path
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def safe_copy_file(source: Path, destination: Path, preserve_metadata: bool = True) -> bool:
"""
Copies a file safely with logging