Introduction

shell-lib is designed to simplify the writing of shell-like scripts:

Python syntax: Write scripts in readable Python, freeing from complex shell command syntax.
Reliable error handling: Use Python’s exception to manage command failure. If a command fails, by default, it raises a subprocess.CalledProcessError exception. For commands that may fail, user can also only check the exit-code.
Cross-platform compatibility: Write a single script that works across Linux, macOS, and Windows platforms.
Rich ecosystem integration: Easily integrate with both the CLI tool and Python library ecosystems.
Lightweight and portable: Only use Python standard library.
Well tested: Consistent and reliable behavior on different platforms and Python versions.
Friendly: Readers can understand the code without consulting the documentation. If writers use modern IDEs, then almost no need to consult the documentation.

It provides some functions that make writing shell-like scripts easy:

Executing external command: Conveniently invoke external commands.
File and directory operations: Provide a consistent and intuitive file system operations API, that clearly distinguish between file and directory operations.
User interactions: Do common interactions with one line of code.
Obtaining system information: Get common system information for shell-like scripts.

This module was co-created with Google Gemini.

#!/usr/bin/python3
from shell_lib import sh

PROJECT_PATH = "my_project"
FILE = "hello.txt"

sh.create_dir(PROJECT_PATH)
# sh.cd() context manager restores the previous working directory when
# exiting the code block, even if an exception raised within the code block.
with sh.cd(PROJECT_PATH):
   sh(f"echo 'Hello, World!' > {FILE}")
   print(f"File size: {sh.get_file_size(FILE)} bytes")
sh.remove_dir(PROJECT_PATH)

There is a demo script at the bottom of this page.

API - File and directory operations

Path parameters can be str, bytes or pathlib.Path object.

sh.home_dir() -> Path: Get the current user’s home directory, a pathlib.Path object.
sh.path(path) -> Path: Convert a str/bytes path to a pathlib.Path object. Can utilize the rich features of pathlib module.
sh.create_dir(path, *, exist_ok=False): Create directory, make all intermediate-level directories needed to contain the leaf directory.
sh.remove_file(path, *, ignore_missing=False): Remove a file.
sh.remove_dir(path, *, ignore_missing=False): Recursively remove a directory.
sh.clear_dir(path) -> None: Clear the contents of a directory.
sh.copy_file(src, dst, *, remove_existing_dst=False): Copy a file.
sh.copy_dir(src, dst, *, remove_existing_dst=False): Copy a directory.
sh.move_file(src, dst, *, remove_existing_dst=False): Move a file.
sh.move_dir(src, dst, *, remove_existing_dst=False): Move a directory.
sh.rename_file(src, dst): Rename a file.
sh.rename_dir(src, dst): Rename a directory.
sh.list_dir(path): List all entry names within a directory.
sh.walk_dir(path, top_down=True): A generator that traverses a directory tree, yield a tuple(directory_path, file_name).
sh.cd(path: str|bytes|Path|None): Change the working directory. Can be used as a context manager.
sh.split_path(path): os.path.split() alias.
sh.join_path(*paths): os.path.join() alias.
sh.path_exists(path) -> bool: Check if a path exists.
sh.is_file(path) -> bool: Check if a path is a file, or a symlink pointing to a file.
sh.is_dir(path) -> bool: Check if a path is a directory, or a symlink pointing to a directory.
sh.is_link(path) -> bool: Check if a path is a symlink.
sh.get_file_size(path) -> int: Get file size.
sh.get_path_info(path) -> PathInfo: Retrieve detailed information about an existing file or directory:

>>> sh.get_path_info('/usr/bin/')  # directory
PathInfo(path=/usr/bin/, size=69632, ctime=2025-09-16 08:18:21.992288,
mtime=2025-09-16 08:18:21.992288, atime=2025-09-17 09:43:29.210739,
is_dir=True, is_file=False, is_link=False,
is_readable=True, is_writable=False, is_executable=True)

>>> sh.get_path_info('/usr/bin/python3')  # file
PathInfo(path=/usr/bin/python3, size=8021824, ctime=2025-08-29 13:12:47.657879,
mtime=2025-08-15 01:47:21, atime=2025-09-16 15:42:37.201079,
is_dir=False, is_file=True, is_link=True,
is_readable=True, is_writable=False, is_executable=True)

API - Shell command executions

sh.which(cmd: str|bytes|Path) -> str|bytes|None: Return the path to an executable if the given cmd was called. Searching in PATH environment variable, NOT searching in the current directory. Return None if no executable is found.

There are several ways to run an external command:

from shell_lib import sh
# 1️⃣ sh(command: str)
# Use shell=True, can use shell features like pipe (|) and redirection (>).
# Support t-string quoting on Python 3.14+.
sh("ls shell_lib")

# 2️⃣ sh.safe_run(command: list[str])
# Use shell=False, can't use shell features.
# It only accepts a list of str to prevent shell injection. Use this method
# when the command contains external input.
# Don't support t-string quoting on Python 3.14+.
sh.safe_run(["ls", "shell_lib"])

from shell_lib.powershell import pwsh
# 3️⃣ pwsh(command: str)
# Run a PowerShell command.
# Support t-string quoting on Python 3.14+.
pwsh("pip list --outdated | Select-Object -Skip 2 | ForEach-Object { pip install -U $_.Split()[0] }")

# 4️⃣ pwsh.run_file(command: list[str])
# Run a PowerShell script. command[0] is the path, the others are arguments.
# Don't support t-string quoting on Python 3.14+.
pwsh.run_file(["a.ps1"])
pwsh.run_file(["a.ps1", "-param1", "value1", "-param2", "value2"])

To prevent shell injection attack:

# User inputs "flask; rm -rf something"
pack = input("Please input PyPI package name:")

# ✅ sh.safe_run() naturally prevents shell injection, but it doesn't
# support shell features like pipe (|) and redirection (>).
sh.safe_run(["pip", "install", pack])  # recommend using this

# pwsh.run_file() also prevents shell injection, but it's only used
# for running a PowerShell script.
pwsh.run_file(["a.ps1", pack])  # pack is an argument for a.ps1

# For sh() and pwsh(), on Python 3.14+, you may use t-string to quote
# argument, to prevent shell injection.
sh(t"pip install {pack}")
pwsh(t"pip install {pack}")

# For sh() and pwsh(), on Python 3.13-, use quote_*() function to quote.
from shell_lib import quote_sh
sh(f"pip install {quote_sh(pack)}")

from shell_lib.powershell import quote_pwsh
pwsh(f"pip install {quote_pwsh(pack)}")

sh(command, *, text=True, input=None, timeout=None, alternative_title=None, print_output=True, fail_on_error=True)

The parameters are the same for sh(), sh.safe_run(), pwsh(), pwsh.run_file(), except the first command parameter.

Parameters:

text (bool) – For stdin/stdout/stderr, True uses system locale encoding to encode/decode, False uses bytes.
input (str|bytes|None) – Passed to Popen.communicate() and thus to the subprocess’s stdin. If used it must be a byte sequence, or a string if text is true.
timeout (int|float|None) –
Timeout in seconds, it’s internally passed on to Popen.communicate(). If the timeout expires, the child process will be killed. The subprocess.TimeoutExpired exception will be raised.
alternative_title (str|None) – Print this title instead of the command. Used for commands containing sensitive information.
print_output (bool) – True streams stdout and stderr to the console. False, stdout and stderr are saved in return value’s .stdout / .stderr attributes.
fail_on_error (bool) – True raises a subprocess.CalledProcessError on failure. False doesn’t raise exception, user need to check return value’s .returncode attribute to see if it has failed.

Returns:

A subprocess.CompletedProcess object.

API - User interactions

sh.ask_choice(title: str, *choices: str) -> int: Display a menu and get a 1-based index from the user’s choice.
sh.ask_yes_no(title: str) -> bool: Ask user to answer yes or no.
sh.ask_regex_input(title: str, pattern: str, *, print_pattern: bool = False) -> re.Match: Ask user to input a string, and verify it with a regex pattern.
sh.ask_password(title: str = "Please input password") -> str: Ask user to input a password, not echo on screen. No need to add : at the end of title.
sh.pause(msg: str|None = None) -> None: Prompt the user to press any key to continue.
sh.exit(exit_code: int = 0): Exit the script with a specified exit code.

API - Get system information

sh.get_preferred_encoding() -> str: Get the preferred encoding, used for subprocess output or files that don’t specify an encoding. If Python UTF-8 mode is enabled, return utf-8. Otherwise, return the system locale encoding.
sh.get_locale_encoding() -> str: Get the system locale encoding. It’s not affected by Python UTF-8 mode.
sh.get_filesystem_encoding() -> str: Get the encoding used to convert between str filenames and bytes filenames. On Windows, return utf-8, unless use legacy mode, see PEP-529.
sh.get_env(key: str, default: str|None = None) -> str|None: Get an environment variable. It’s more reliable than os.getenv().
sh.set_env(key: str, value: str|None, /) or sh.set_env(dict: Dict[str, str|None], /): Set environment variable(s), None means delete, the changes affect the current process and subprocesses. Can also be used as a context manager for automatically restore.
sh.get_hostname(hostname_type=sh.HOSTNAME_TYPE_Host) -> str: Get the host name. hostname_type can be sh.HOSTNAME_TYPE_Host or sh.HOSTNAME_TYPE_FQDN.
sh.get_username() -> str: Get the current username. On POSIX, if running a script with sudo, see the docstring to learn how to get the username.
sh.is_elevated() -> bool: If the script is running with elevated (admin/root) privilege.
sh.is_os(os_mask: int) -> bool: Test whether it’s the OS specified by the parameter.

# os_mask can be:
sh.OS_Windows
sh.OS_Cygwin
sh.OS_Linux
sh.OS_macOS
sh.OS_Unix
sh.OS_Unix_like  # It's (OS_Linux | OS_macOS | OS_Unix | OS_Cygwin)

# Support bit OR (|) combination:
if sh.is_os(sh.OS_Linux | sh.OS_macOS):
    ...
elif sh.is_os(sh.OS_Windows):
    ...

API - Top level context manager

with sh: is a top-level context manager.

Its main purpose is, if sh(), sh.safe_run(), pwsh() or pwsh.run_file() fails, exit the entire script, and return the error exit-code from the command.

If you don’t need this, don’t use it.

with sh:
   ...
   sh("command that may fail")
   ...

Demo script

#!/usr/bin/python3
import os
from shell_lib import sh
# shell-lib demo script: Build and install cpython on Linux
# Need to install build dependencies first:
# https://devguide.python.org/getting-started/setup-building/#install-dependencies

# Input Python version
m = sh.ask_regex_input('Please input Python version to install (such as 3.13.7)',
                       r'\s*(((\d+)\.(\d+))\.\d+)\s*')
ver = m.group(1)
ver_2 = m.group(2)
ver_info = int(m.group(3)), int(m.group(4))

# Variables
work_dir = sh.home_dir() / 'build_python'
xz_filename = sh.path(f'Python-{ver}.tar.xz')
compile_dir = f'Python-{ver}'
install_dir = sh.path(f'/opt/python{ver_2}')
url = f'https://www.python.org/ftp/python/{ver}/Python-{ver}.tar.xz'

# Check existing installed Python
msg = (f'Install path `{install_dir}` is existing, '
       f'overwrite install(yes) or exit(no)?')
if install_dir.is_dir() and not sh.ask_yes_no(msg):
    sh.exit()

# Build options
config = f'./configure --prefix={install_dir}'
optimize = sh.ask_choice('Please choose build options',
                         'PGO + LTO (very slow)',
                         'LTO (slow)',
                         'No optimization',
                         'Debug build')
if optimize == 1:
    config += ' --enable-optimizations --with-lto'
elif optimize == 2:
    config += ' --with-lto'
elif optimize == 3:
    pass
elif optimize == 4:
    config += ' --with-pydebug'

if ver_info >= (3, 13) and sh.ask_yes_no("Build Free-threaded build?"):
    config += ' --disable-gil'

sh.create_dir(work_dir, exist_ok=True)
with sh.cd(work_dir):
    if not xz_filename.is_file() or sh.get_file_size(xz_filename) == 0:
        sh(f'wget --no-proxy -O {xz_filename} {url}')

    password = sh.ask_password('Please input sudo password')
    sh(f'sudo -S rm -rf {compile_dir}', input=password)
    sh(f'tar -xvf {xz_filename}', print_output=False)

    with sh.cd(compile_dir):
        # Compile
        with sh.set_env('CFLAGS', '-O2'):
            sh(config, print_output=False)
        sh('make clean')
        sh(f'make -j{os.cpu_count()}')
        sh.pause('Please check for missing modules')

        # Install
        sh(f'sudo -S rm -rf {install_dir}', input=password)
        sh(f'sudo -S make install', input=password)

    if sh.ask_yes_no('Run unit-tests? (very slow)'):
        sh(f'{install_dir}/bin/python{ver_2} -m test', fail_on_error=False)

    if sh.ask_yes_no('Remove building directory?'):
        sh(f'sudo -S rm -rf {compile_dir}', input=password)