Python Packages: Directory Organization

Welcome to Python Packages! If modules are filing cabinets, packages are entire filing rooms - they organize multiple modules into hierarchical directory structures.

What is a Package?

A package is a directory containing Python modules and a special __init__.py file. Packages allow you to organize related modules into a hierarchical structure.

my_package/
├── __init__.py      # Makes this a package
├── module1.py       # Module in the package
├── module2.py       # Another module
└── subpackage/      # Nested package
    ├── __init__.py
    └── submodule.py

Creating Your First Package

Let’s create a simple package structure:

calculator/
├── __init__.py
├── basic.py
├── advanced.py
└── constants.py

# calculator/__init__.py
"""Calculator package for mathematical operations."""

# Import key functions to make them easily accessible
from .basic import add, subtract, multiply, divide
from .advanced import power, sqrt, factorial

__version__ = "1.0.0"
__author__ = "Your Name"

# Define what's available when someone imports the package
__all__ = ['add', 'subtract', 'multiply', 'divide', 'power', 'sqrt', 'factorial']

# calculator/basic.py
"""Basic arithmetic operations."""

def add(a, b):
    """Add two numbers."""
    return a + b

def subtract(a, b):
    """Subtract b from a."""
    return a - b

def multiply(a, b):
    """Multiply two numbers."""
    return a * b

def divide(a, b):
    """Divide a by b."""
    if b == 0:
        raise ValueError("Cannot divide by zero!")
    return a / b

# calculator/advanced.py
"""Advanced mathematical operations."""
import math

def power(base, exponent):
    """Calculate base raised to exponent."""
    return base ** exponent

def sqrt(number):
    """Calculate square root."""
    if number < 0:
        raise ValueError("Cannot take square root of negative number!")
    return math.sqrt(number)

def factorial(n):
    """Calculate factorial of n."""
    if not isinstance(n, int) or n < 0:
        raise ValueError("Factorial is only defined for non-negative integers!")
    return math.factorial(n)

# calculator/constants.py
"""Mathematical constants."""

PI = 3.141592653589793
E = 2.718281828459045
PHI = 1.618033988749895  # Golden ratio
TAU = 2 * PI  # Tau (2π)

Now let’s use this package:

# main.py
import calculator

# Use functions imported in __init__.py
result1 = calculator.add(5, 3)
result2 = calculator.power(2, 3)
result3 = calculator.sqrt(16)

print(f"5 + 3 = {result1}")
print(f"2³ = {result2}")
print(f"√16 = {result3}")
print(f"π ≈ {calculator.PI}")

# Import specific modules
from calculator import basic, advanced

result4 = basic.multiply(6, 7)
result5 = advanced.factorial(5)

print(f"6 × 7 = {result4}")
print(f"5! = {result5}")

The `init.py` File

The __init__.py file serves several purposes:

1. Makes a Directory a Package

Without __init__.py, Python treats the directory as a regular folder, not a package.

2. Package Initialization

Code in __init__.py runs when the package is imported:

# my_package/__init__.py
print("My package is being imported!")

# Initialize package-level variables
__version__ = "1.0.0"

# Import submodules
from . import submodule1, submodule2

3. Controls Package Interface

Use __all__ to define what’s exported:

# my_package/__init__.py
from .module1 import function1, Class1
from .module2 import function2

__all__ = ['function1', 'Class1', 'function2']  # Public API

Package Structure Patterns

Basic Package Structure

mypackage/
├── __init__.py
├── core.py          # Core functionality
├── utils.py         # Utility functions
└── exceptions.py    # Custom exceptions

Complex Package with Subpackages

ecommerce/
├── __init__.py
├── products/
│   ├── __init__.py
│   ├── models.py
│   └── validators.py
├── orders/
│   ├── __init__.py
│   ├── models.py
│   └── processors.py
├── payments/
│   ├── __init__.py
│   ├── stripe.py
│   └── paypal.py
└── utils/
    ├── __init__.py
    └── helpers.py

Application Package Structure

myapp/
├── __init__.py
├── config.py        # Configuration
├── database.py      # Database connections
├── models.py        # Data models
├── views.py         # View handlers
├── controllers.py   # Business logic
├── utils.py         # Utilities
└── tests/
    ├── __init__.py
    ├── test_models.py
    └── test_views.py

Importing from Packages

Absolute Imports

# From the package root
from ecommerce.products.models import Product
from ecommerce.orders.processors import OrderProcessor

# From current package
from .models import User
from ..utils.helpers import format_date

Relative Imports

# In ecommerce/orders/models.py
from . import processors          # Same package
from ..products import models     # Parent package
from ..utils.helpers import *     # Parent's sibling

Best Practices for Imports

# Good: Absolute imports (preferred)
from mypackage.utils import helper_function
import mypackage.core as core

# Good: Relative imports within package
from . import sibling_module
from .sibling_module import some_function
from ..parent_module import parent_function

# Avoid: Ambiguous relative imports
from .. import *  # Unclear what's imported

Practical Examples

Example 1: Web Framework Package

# webframework/
# ├── __init__.py
# ├── app.py
# ├── routing.py
# ├── templates.py
# └── static.py

# webframework/__init__.py
from .app import WebApp
from .routing import Route

__version__ = "1.0.0"
__all__ = ['WebApp', 'Route']

# webframework/app.py
from .routing import Router

class WebApp:
    def __init__(self):
        self.router = Router()

    def route(self, path):
        def decorator(func):
            self.router.add_route(path, func)
            return func
        return decorator

    def run(self, host='localhost', port=8000):
        print(f"Running on http://{host}:{port}")
        # Implementation would go here

# webframework/routing.py
class Router:
    def __init__(self):
        self.routes = {}

    def add_route(self, path, handler):
        self.routes[path] = handler

    def get_handler(self, path):
        return self.routes.get(path)

Example 2: Data Science Package

# datascience/
# ├── __init__.py
# ├── preprocessing/
# │   ├── __init__.py
# │   ├── cleaning.py
# │   └── scaling.py
# ├── models/
# │   ├── __init__.py
# │   ├── linear.py
# │   └── neural.py
# └── visualization/
#     ├── __init__.py
#     └── plots.py

# datascience/__init__.py
from .preprocessing import clean_data, scale_features
from .models import LinearRegression, NeuralNetwork
from .visualization import plot_data

__all__ = ['clean_data', 'scale_features', 'LinearRegression',
           'NeuralNetwork', 'plot_data']

# datascience/preprocessing/__init__.py
from .cleaning import clean_data
from .scaling import scale_features

__all__ = ['clean_data', 'scale_features']

Example 3: Game Engine Package

# gameengine/
# ├── __init__.py
# ├── core/
# │   ├── __init__.py
# │   ├── game.py
# │   └── entity.py
# ├── graphics/
# │   ├── __init__.py
# │   ├── renderer.py
# │   └── sprites.py
# ├── physics/
# │   ├── __init__.py
# │   ├── collision.py
# │   └── forces.py
# └── audio/
#     ├── __init__.py
#     └── sound.py

# gameengine/__init__.py
from .core.game import Game
from .core.entity import Entity
from .graphics.renderer import Renderer

__all__ = ['Game', 'Entity', 'Renderer']

Package Initialization Patterns

Lazy Loading

# mypackage/__init__.py
def __getattr__(name):
    """Lazy load modules when accessed."""
    if name == 'expensive_module':
        from . import expensive_module
        return expensive_module
    raise AttributeError(f"module '{__name__}' has no attribute '{name}'")

Conditional Imports

# mypackage/__init__.py
try:
    import optional_dependency
    HAS_OPTIONAL = True
except ImportError:
    HAS_OPTIONAL = False

if HAS_OPTIONAL:
    from .optional_features import advanced_function

Plugin System

# mypackage/__init__.py
import importlib
import pkgutil

# Auto-discover plugins
def load_plugins():
    """Load all plugins in the plugins subpackage."""
    plugins = {}
    for finder, name, ispkg in pkgutil.iter_modules(__path__):
        if name.startswith('plugin_'):
            module = importlib.import_module(f'{__name__}.{name}')
            plugins[name] = module
    return plugins

plugins = load_plugins()

Namespace Packages

For packages split across multiple directories:

# No __init__.py needed for namespace packages
mylib/
├── utils/
│   └── helpers.py
└── models/
    └── user.py

Python 3.3+ supports namespace packages without __init__.py.

Best Practices

1. Use Clear Package Names

# Good
import data_processing
import web_scraping
import machine_learning

# Bad
import dp
import ws
import ml

2. Organize by Functionality

# Good: Group related modules
auth/           # Authentication
api/            # API endpoints
models/         # Data models
utils/          # Utilities

# Bad: Random organization
stuff/
random/
misc/

3. Use `all` in `init.py`

# __init__.py
from .module1 import Class1, function1
from .module2 import Class2

__all__ = ['Class1', 'function1', 'Class2']  # Explicit public API

4. Document Your Packages

"""
My Package

A comprehensive package for doing amazing things.

Modules:
    - core: Core functionality
    - utils: Utility functions
    - api: API interfaces

Usage:
    import mypackage
    result = mypackage.do_something()
"""

__version__ = "1.0.0"
__author__ = "Your Name"

5. Handle Dependencies Properly

# __init__.py
try:
    import pandas
    HAS_PANDAS = True
except ImportError:
    HAS_PANDAS = False

def read_csv(file_path):
    if not HAS_PANDAS:
        raise ImportError("pandas is required for CSV operations")
    # Implementation

Common Package Patterns

API Package

# api_package/
# ├── __init__.py
# ├── client.py
# ├── models.py
# └── exceptions.py

# __init__.py
from .client import APIClient
from .models import User, Post
from .exceptions import APIError

__all__ = ['APIClient', 'User', 'Post', 'APIError']

CLI Tool Package

# cli_tool/
# ├── __init__.py
# ├── commands/
# │   ├── __init__.py
# │   ├── init.py
# │   └── build.py
# └── utils.py

# __init__.py
from .commands import init_command, build_command

def main():
    # CLI entry point
    pass

Library Package

# mylib/
# ├── __init__.py
# ├── core.py
# ├── extensions.py
# └── contrib/
#     ├── __init__.py
#     └── plugins.py

# __init__.py
from .core import main_function
from .extensions import extra_function

__all__ = ['main_function', 'extra_function']

Practice Exercises

Exercise 1: Math Library Package

Create a mathlib package with:

basic/ - add, subtract, multiply, divide
advanced/ - power, sqrt, log, trigonometry
constants/ - π, e, etc.
Proper __init__.py files with __all__

Exercise 2: Blog Package

Create a blog package with:

models/ - Post, Comment, User classes
views/ - functions to display content
utils/ - helper functions for formatting
Database simulation with in-memory storage

Exercise 3: Image Processing Package

Create an imageproc package with:

filters/ - blur, sharpen, grayscale
transform/ - resize, rotate, crop
io/ - load/save different formats
Command-line interface for batch processing

Exercise 4: Task Management Package

Create a taskmanager package with:

models/ - Task, Project, User
storage/ - save/load tasks (JSON, CSV)
ui/ - console interface for managing tasks
Export functionality for reports

Summary

Packages organize modules into directory hierarchies:

Basic Package Structure:

mypackage/
├── __init__.py      # Package marker and initialization
├── module1.py       # Module
└── submodule/       # Subpackage
    ├── __init__.py
    └── module2.py

Key Concepts:

__init__.py makes a directory a package
Use __all__ to control the public API
Absolute imports: from mypackage.module import function
Relative imports: from .module import function

Best Practices:

Clear, descriptive package names
Logical organization by functionality
Proper documentation and version info
Explicit public APIs with __all__

Next: Import System - mastering Python’s import mechanics! 📦

Popular Topics

Categories