File Handling: Reading and Writing Files

Welcome to the world of persistent data! Files allow your programs to save data between runs. Think of files as digital filing cabinets - you can store information and retrieve it later.

Why File Handling Matters

Real-Life Benefits

1. Data Persistence: Save user data, settings, game progress
2. Data Exchange: Read/write CSV, JSON, XML files
3. Logging: Record program activity and errors
4. Configuration: Store program settings
5. Backup: Save important data safely

Code Without Files (Bad)

# Data disappears when program ends!
user_scores = {"Alice": 100, "Bob": 85, "Charlie": 92}
print("Game Over! Final scores:")
for name, score in user_scores.items():
    print(f"{name}: {score}")
# When program restarts, all data is gone!

Code With Files (Good)

import json

# Save data to file
user_scores = {"Alice": 100, "Bob": 85, "Charlie": 92}
with open("scores.json", "w") as file:
    json.dump(user_scores, file)

# Later, load data from file
with open("scores.json", "r") as file:
    loaded_scores = json.load(file)

print("Loaded scores from file:")
for name, score in loaded_scores.items():
    print(f"{name}: {score}")

Method 1: Reading Text Files

Basic File Opening and Closing

# Open file for reading (default mode)
file = open("example.txt", "r")

# Do something with file
content = file.read()
print(content)

# Always close the file!
file.close()

The with Statement (Best Practice)

# Automatic file closing - no need to call close()
with open("example.txt", "r") as file:
    content = file.read()
    print(content)
# File is automatically closed here

Reading Methods

# Method 1: read() - read entire file as string
with open("story.txt", "r") as file:
    entire_content = file.read()
    print(entire_content)

# Method 2: readline() - read one line at a time
with open("story.txt", "r") as file:
    line1 = file.readline()
    line2 = file.readline()
    print(f"First line: {line1.strip()}")
    print(f"Second line: {line2.strip()}")

# Method 3: readlines() - read all lines as list
with open("story.txt", "r") as file:
    all_lines = file.readlines()
    print(f"File has {len(all_lines)} lines")
    print(f"Third line: {all_lines[2].strip()}")

# Method 4: Iterate through lines
with open("story.txt", "r") as file:
    for line_number, line in enumerate(file, 1):
        print(f"Line {line_number}: {line.strip()}")

Handling File Not Found

filename = "nonexistent.txt"

try:
    with open(filename, "r") as file:
        content = file.read()
        print(content)
except FileNotFoundError:
    print(f"Error: File '{filename}' not found!")
    print("Please check the filename and try again.")

Method 2: Writing Text Files

Writing to Files

# Write mode - creates new file or overwrites existing
with open("output.txt", "w") as file:
    file.write("Hello, World!\n")
    file.write("This is a new line.\n")
    file.write("Python file handling is easy!")

# Append mode - adds to existing file
with open("output.txt", "a") as file:
    file.write("\nThis line was appended!")

Writing Multiple Lines

lines = [
    "First line of text",
    "Second line of text",
    "Third line of text"
]

with open("multiline.txt", "w") as file:
    # Method 1: Write each line individually
    for line in lines:
        file.write(line + "\n")

    # Method 2: Use writelines() (no automatic newlines)
    # file.writelines(line + "\n" for line in lines)

Writing Formatted Data

# Write a simple report
students = [
    ("Alice", 95, "A"),
    ("Bob", 87, "B"),
    ("Charlie", 92, "A")
]

with open("report.txt", "w") as file:
    file.write("Student Report\n")
    file.write("=" * 30 + "\n")
    file.write("15")
    file.write("-" * 30 + "\n")

    for name, score, grade in students:
        file.write("15")

Method 3: Working with CSV Files

Reading CSV Files

import csv

# Read CSV file
with open("data.csv", "r") as file:
    csv_reader = csv.reader(file)

    # Read header
    header = next(csv_reader)
    print(f"Header: {header}")

    # Read data rows
    for row in csv_reader:
        print(row)

Writing CSV Files

import csv

# Sample data
employees = [
    ["Name", "Department", "Salary"],
    ["Alice", "Engineering", 75000],
    ["Bob", "Marketing", 65000],
    ["Charlie", "Engineering", 80000]
]

with open("employees.csv", "w", newline="") as file:
    csv_writer = csv.writer(file)

    # Write all rows
    for row in employees:
        csv_writer.writerow(row)

CSV with Dictionaries

import csv

# Writing with DictWriter
employees = [
    {"name": "Alice", "department": "Engineering", "salary": 75000},
    {"name": "Bob", "department": "Marketing", "salary": 65000},
    {"name": "Charlie", "department": "Engineering", "salary": 80000}
]

with open("employees_dict.csv", "w", newline="") as file:
    fieldnames = ["name", "department", "salary"]
    writer = csv.DictWriter(file, fieldnames=fieldnames)

    writer.writeheader()
    for employee in employees:
        writer.writerow(employee)

# Reading with DictReader
with open("employees_dict.csv", "r") as file:
    reader = csv.DictReader(file)
    for row in reader:
        print(f"{row['name']} works in {row['department']} and earns ${row['salary']}")

Method 4: Working with JSON Files

JSON Basics

JSON (JavaScript Object Notation) is a popular format for data exchange.

import json

# Python data
data = {
    "name": "Alice",
    "age": 25,
    "hobbies": ["reading", "coding", "gaming"],
    "address": {
        "street": "123 Main St",
        "city": "New York",
        "zipcode": "10001"
    }
}

# Convert to JSON string
json_string = json.dumps(data, indent=2)
print(json_string)

Writing JSON Files

import json

# Data to save
user_profile = {
    "username": "alice_dev",
    "email": "alice@example.com",
    "settings": {
        "theme": "dark",
        "notifications": True,
        "language": "en"
    },
    "stats": {
        "login_count": 42,
        "last_login": "2024-01-15"
    }
}

# Write to JSON file
with open("user_profile.json", "w") as file:
    json.dump(user_profile, file, indent=2)

print("User profile saved to user_profile.json")

Reading JSON Files

import json

# Read from JSON file
try:
    with open("user_profile.json", "r") as file:
        loaded_profile = json.load(file)

    print("Loaded user profile:")
    print(f"Username: {loaded_profile['username']}")
    print(f"Email: {loaded_profile['email']}")
    print(f"Theme: {loaded_profile['settings']['theme']}")
    print(f"Login count: {loaded_profile['stats']['login_count']}")

except FileNotFoundError:
    print("User profile file not found!")

Method 5: File and Directory Operations

Checking File Existence

import os

filename = "example.txt"

# Check if file exists
if os.path.exists(filename):
    print(f"File '{filename}' exists")
else:
    print(f"File '{filename}' does not exist")

# Check if it's a file or directory
if os.path.isfile(filename):
    print(f"'{filename}' is a file")
elif os.path.isdir(filename):
    print(f"'{filename}' is a directory")

Working with Directories

import os

# Create directory
os.makedirs("data/reports", exist_ok=True)

# List directory contents
files = os.listdir("data")
print("Files in data directory:")
for file in files:
    print(f"  {file}")

# Get file information
if os.path.exists("example.txt"):
    file_size = os.path.getsize("example.txt")
    modified_time = os.path.getmtime("example.txt")
    print(f"File size: {file_size} bytes")
    print(f"Last modified: {modified_time}")

File Path Operations

import os

# Current working directory
current_dir = os.getcwd()
print(f"Current directory: {current_dir}")

# Join paths (cross-platform)
data_file = os.path.join("data", "user_data.json")
print(f"Data file path: {data_file}")

# Get file extension
filename = "document.pdf"
name, extension = os.path.splitext(filename)
print(f"Name: {name}, Extension: {extension}")

# Get absolute path
relative_path = "data/settings.json"
absolute_path = os.path.abspath(relative_path)
print(f"Absolute path: {absolute_path}")

Real-World Examples

Example 1: Simple Note-Taking App

def save_note(filename, note):
    """Save a note to a file."""
    with open(filename, "a") as file:
        from datetime import datetime
        timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
        file.write(f"[{timestamp}] {note}\n")
    print(f"Note saved to {filename}")

def read_notes(filename):
    """Read all notes from a file."""
    try:
        with open(filename, "r") as file:
            notes = file.readlines()
            if notes:
                print(f"Notes from {filename}:")
                for i, note in enumerate(notes, 1):
                    print(f"{i}. {note.strip()}")
            else:
                print("No notes found.")
    except FileNotFoundError:
        print(f"Notes file '{filename}' not found.")

def search_notes(filename, keyword):
    """Search notes containing a keyword."""
    try:
        with open(filename, "r") as file:
            notes = file.readlines()
            matching_notes = [note.strip() for note in notes if keyword.lower() in note.lower()]

            if matching_notes:
                print(f"Notes containing '{keyword}':")
                for note in matching_notes:
                    print(f"  {note}")
            else:
                print(f"No notes found containing '{keyword}'.")
    except FileNotFoundError:
        print(f"Notes file '{filename}' not found.")

# Usage
notes_file = "my_notes.txt"

save_note(notes_file, "Started learning Python file handling")
save_note(notes_file, "Created a note-taking app")
save_note(notes_file, "File handling is very useful")

read_notes(notes_file)
search_notes(notes_file, "python")

Example 2: Configuration File Manager

import json
import os

class ConfigManager:
    def __init__(self, config_file="config.json"):
        self.config_file = config_file
        self.defaults = {
            "theme": "light",
            "language": "en",
            "font_size": 12,
            "auto_save": True
        }
        self.config = self.load_config()

    def load_config(self):
        """Load configuration from file or create with defaults."""
        if os.path.exists(self.config_file):
            try:
                with open(self.config_file, "r") as file:
                    return json.load(file)
            except json.JSONDecodeError:
                print("Warning: Config file corrupted. Using defaults.")
                return self.defaults.copy()
        else:
            # Create config file with defaults
            self.save_config(self.defaults)
            return self.defaults.copy()

    def save_config(self, config=None):
        """Save configuration to file."""
        if config is None:
            config = self.config
        with open(self.config_file, "w") as file:
            json.dump(config, file, indent=2)

    def get(self, key, default=None):
        """Get a configuration value."""
        return self.config.get(key, default)

    def set(self, key, value):
        """Set a configuration value."""
        self.config[key] = value
        self.save_config()

    def reset(self):
        """Reset configuration to defaults."""
        self.config = self.defaults.copy()
        self.save_config()

# Usage
config = ConfigManager()

print(f"Current theme: {config.get('theme')}")
print(f"Font size: {config.get('font_size')}")

# Change settings
config.set("theme", "dark")
config.set("font_size", 14)

print(f"Updated theme: {config.get('theme')}")
print(f"Updated font size: {config.get('font_size')}")

Example 3: CSV Data Analyzer

import csv
from collections import defaultdict

def analyze_sales_data(filename):
    """Analyze sales data from CSV file."""
    sales_by_product = defaultdict(float)
    sales_by_month = defaultdict(float)
    total_sales = 0

    try:
        with open(filename, "r") as file:
            reader = csv.DictReader(file)

            for row in reader:
                product = row["product"]
                month = row["month"]
                sales = float(row["sales"])

                sales_by_product[product] += sales
                sales_by_month[month] += sales
                total_sales += sales

        # Display results
        print(f"Total sales: ${total_sales:,.2f}")
        print("\nSales by product:")
        for product, sales in sorted(sales_by_product.items(), key=lambda x: x[1], reverse=True):
            print(f"  {product}: ${sales:,.2f}")

        print("\nSales by month:")
        for month, sales in sorted(sales_by_month.items()):
            print(f"  {month}: ${sales:,.2f}")

    except FileNotFoundError:
        print(f"Error: File '{filename}' not found.")
    except KeyError as e:
        print(f"Error: Missing column '{e}' in CSV file.")

# Create sample data
sample_data = [
    ["product", "month", "sales"],
    ["Laptop", "Jan", "1200.50"],
    ["Mouse", "Jan", "25.99"],
    ["Laptop", "Feb", "1350.00"],
    ["Keyboard", "Feb", "89.99"],
    ["Mouse", "Feb", "32.50"]
]

with open("sales_data.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerows(sample_data)

# Analyze the data
analyze_sales_data("sales_data.csv")

Error Handling Best Practices

Comprehensive File Error Handling

def safe_file_operation(filename, operation="read", data=None):
    """Safely perform file operations with comprehensive error handling."""
    try:
        if operation == "read":
            with open(filename, "r") as file:
                return file.read()
        elif operation == "write":
            with open(filename, "w") as file:
                file.write(data)
                return True
        elif operation == "append":
            with open(filename, "a") as file:
                file.write(data)
                return True
        else:
            raise ValueError(f"Unknown operation: {operation}")

    except FileNotFoundError:
        print(f"Error: File '{filename}' not found.")
        return None
    except PermissionError:
        print(f"Error: Permission denied for file '{filename}'.")
        return None
    except IsADirectoryError:
        print(f"Error: '{filename}' is a directory, not a file.")
        return None
    except OSError as e:
        print(f"OS Error: {e}")
        return None
    except Exception as e:
        print(f"Unexpected error: {e}")
        return None

# Usage
content = safe_file_operation("example.txt", "read")
if content is not None:
    print("File read successfully!")
else:
    print("File operation failed.")

File Encoding

Handling Different Encodings

# Specify encoding for international characters
with open("german_text.txt", "w", encoding="utf-8") as file:
    file.write("Schöne Grüße aus Deutschland!")

# Read with correct encoding
with open("german_text.txt", "r", encoding="utf-8") as file:
    content = file.read()
    print(content)

Performance Tips

Reading Large Files

# ❌ Bad: Load entire file into memory
# with open("large_file.txt", "r") as file:
#     content = file.read()  # May crash with large files

# ✅ Good: Process line by line
with open("large_file.txt", "r") as file:
    for line in file:
        # Process one line at a time
        process_line(line.strip())

Writing Large Amounts of Data

# ✅ Good: Write in chunks
data_chunks = ["chunk1", "chunk2", "chunk3"]

with open("large_output.txt", "w") as file:
    for chunk in data_chunks:
        file.write(chunk + "\n")
        # Optional: Force write to disk
        # file.flush()

Practice Exercises

Exercise 1: Personal Journal

Create a journaling application that:

• Add new journal entries with timestamps
• View all entries
• Search entries by keyword
• Save/load journal from file

Exercise 2: Todo List Manager

Build a todo list that:

• Add/remove/complete tasks
• Save todo list to JSON file
• Load todo list from file
• Display tasks by status (pending/completed)

Exercise 3: Student Grade Tracker

Create a grade tracking system:

• Add students and their grades
• Calculate averages and letter grades
• Save/load data as CSV
• Generate reports

Exercise 4: Simple Database

Build a simple database system:

• Store records (like users, products)
• Save/load data as JSON
• Search records by criteria
• Update/delete records

Exercise 5: Log File Analyzer

Create a log analyzer that:

• Read server log files
• Count different types of log entries
• Find errors and warnings
• Generate summary reports

Summary

File handling enables data persistence and exchange:

Reading Files:

• open(filename, "r") - read mode
• file.read() - entire file
• file.readline() - one line
• file.readlines() - all lines as list

Writing Files:

• open(filename, "w") - write mode (overwrites)
• open(filename, "a") - append mode (adds to end)
• file.write(text) - write string

File Formats:

• Text: Plain text files
• CSV: Comma-separated values
• JSON: Structured data format

Best Practices:

• Use with statement for automatic closing
• Handle file errors appropriately
• Consider file encoding
• Process large files line by line

Next: Error Handling - making your programs robust! 🛡️