Processes vs Threads

Understanding concurrency fundamentals with psutil

Learning Objectives

By the end of this lesson, you will be able to:

• Understand the difference between processes and threads
• Use psutil to monitor system processes
• Identify when to use processes vs threads
• Understand the basic concepts of concurrency

What is a Process?

A process is an instance of a running program. Each process has its own memory space, file handles, and system resources. Processes are isolated from each other by the operating system.

Let’s explore processes using the psutil library:

import psutil
import os
import time

# Get current process information
current_process = psutil.Process()

print(f"Process ID (PID): {current_process.pid}")
print(f"Process Name: {current_process.name()}")
print(f"Parent Process ID: {current_process.ppid()}")
print(f"Memory Info: {current_process.memory_info()}")
print(f"CPU Usage: {current_process.cpu_percent()}%")

Process ID (PID): 53108
Process Name: python3
Parent Process ID: 53107
Memory Info: pmem(rss=132276224, vms=885882880, shared=35389440, text=31588352, lib=0, data=243585024, dirty=0)
CPU Usage: 0.0%

System-wide Process Information

# Get all running processes
processes = []
for proc in psutil.process_iter(['pid', 'name', 'memory_percent']):
    try:
        processes.append(proc.info)
    except psutil.NoSuchProcess:
        pass

# Sort by memory usage and show top 5
top_processes = sorted(processes, key=lambda x: x['memory_percent'], reverse=True)[:5]

print("Top 5 processes by memory usage:")
for proc in top_processes:
    print(f"PID: {proc['pid']:<6} Name: {proc['name']:<20} Memory: {proc['memory_percent']:.1f}%")

Top 5 processes by memory usage:
PID: 53074  Name: deno                 Memory: 3.5%
PID: 53108  Name: python3              Memory: 3.2%
PID: 53107  Name: python3              Memory: 2.4%
PID: 294    Name: systemd-journald     Memory: 1.7%
PID: 3264   Name: fwupd                Memory: 1.0%

What is a Thread?

A thread is a lightweight execution unit within a process. Threads within the same process share memory space and resources, making communication between them easier but also introducing potential issues like race conditions.

import threading
import time

def worker_function(name, delay):
    """A simple worker function that simulates some work"""
    print(f"Thread {name} starting...")
    time.sleep(delay)
    print(f"Thread {name} finished after {delay} seconds")

# Get information about the current thread
main_thread = threading.current_thread()
print(f"Main thread name: {main_thread.name}")
print(f"Main thread ID: {main_thread.ident}")
print(f"Active thread count: {threading.active_count()}")

Main thread name: MainThread
Main thread ID: 123313093158720
Active thread count: 8

Creating and Managing Threads

# Create some threads
threads = []
for i in range(3):
    thread = threading.Thread(
        target=worker_function, 
        args=(f"Worker-{i}", i + 1),
        name=f"WorkerThread-{i}"
    )
    threads.append(thread)

# Start all threads
for thread in threads:
    thread.start()

# Wait for all threads to complete
for thread in threads:
    thread.join()

print("All threads completed!")

Thread Worker-0 starting...Thread Worker-1 starting...

Thread Worker-2 starting...
Thread Worker-0 finished after 1 seconds
Thread Worker-1 finished after 2 seconds
Thread Worker-2 finished after 3 seconds
All threads completed!

Process vs Thread Comparison

Let’s create a practical example to demonstrate the differences:

import multiprocessing
import threading
import time
import os

def cpu_intensive_task(n):
    """A CPU-intensive task for demonstration"""
    result = 0
    for i in range(n):
        result += i * i
    return result

def io_intensive_task(duration):
    """An I/O-intensive task (simulated with sleep)"""
    time.sleep(duration)
    return f"Task completed after {duration} seconds"

# Function to run with multiprocessing
def process_worker(task_id, n):
    start_time = time.time()
    result = cpu_intensive_task(n)
    end_time = time.time()
    print(f"Process {task_id} (PID: {os.getpid()}) completed in {end_time - start_time:.2f} seconds")
    return result

# Function to run with threading
def thread_worker(task_id, duration):
    start_time = time.time()
    result = io_intensive_task(duration)
    end_time = time.time()
    print(f"Thread {task_id} completed in {end_time - start_time:.2f} seconds")
    return result

# Demonstrate threading for I/O-bound tasks
print("=== Threading Example (I/O-bound) ===")
start_time = time.time()

threads = []
for i in range(3):
    thread = threading.Thread(target=thread_worker, args=(i, 1))
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()

print(f"Total time with threading: {time.time() - start_time:.2f} seconds")

=== Threading Example (I/O-bound) ===
Thread 0 completed in 1.00 seconds
Thread 1 completed in 1.00 seconds
Thread 2 completed in 1.00 seconds
Total time with threading: 1.00 seconds

Memory and Resource Usage

Let’s examine how processes and threads use system resources:

import psutil
import threading
import multiprocessing
import time

def monitor_resources():
    """Monitor system resources"""
    process = psutil.Process()
    
    print(f"Memory usage: {process.memory_info().rss / 1024 / 1024:.2f} MB")
    print(f"CPU usage: {process.cpu_percent()}%")
    print(f"Number of threads: {process.num_threads()}")
    # print(f"Number of file descriptors: {process.num_fds()}")
    print("---")

print("Initial resource usage:")
monitor_resources()

# Create some threads
def dummy_thread_work():
    time.sleep(2)

threads = []
for i in range(5):
    thread = threading.Thread(target=dummy_thread_work)
    threads.append(thread)
    thread.start()

print("After creating 5 threads:")
monitor_resources()

# Clean up
for thread in threads:
    thread.join()

print("After threads completed:")
monitor_resources()

Initial resource usage:
Memory usage: 126.27 MB
CPU usage: 0.0%
Number of threads: 14
---
After creating 5 threads:
Memory usage: 126.27 MB
CPU usage: 0.0%
Number of threads: 19
---
After threads completed:
Memory usage: 126.27 MB
CPU usage: 0.0%
Number of threads: 14
---

When to Use Processes vs Threads

Use Processes when:

1. CPU-bound tasks - Tasks that require intensive computation
2. Fault isolation - You want to isolate failures
3. True parallelism - You need to utilize multiple CPU cores
4. Different programming languages - Communicating between different systems

Use Threads when:

1. I/O-bound tasks - Tasks that wait for file reads, network requests, etc.
2. Shared state - You need to share data between concurrent operations
3. Lightweight concurrency - You need many concurrent operations with low overhead
4. Responsive interfaces - Keeping a GUI responsive while doing background work

Practical Example: System Monitor

Let’s create a simple system monitor that demonstrates these concepts:

import psutil
import threading
import time
from datetime import datetime

class SystemMonitor:
    def __init__(self):
        self.monitoring = False
        self.stats = []
    
    def collect_stats(self):
        """Collect system statistics"""
        while self.monitoring:
            stats = {
                'timestamp': datetime.now().strftime('%H:%M:%S'),
                'cpu_percent': psutil.cpu_percent(interval=1),
                'memory_percent': psutil.virtual_memory().percent,
                'disk_usage': psutil.disk_usage('/').percent if hasattr(psutil, 'disk_usage') else 0,
                'process_count': len(psutil.pids())
            }
            self.stats.append(stats)
            time.sleep(1)
    
    def start_monitoring(self, duration=5):
        """Start monitoring for specified duration"""
        self.monitoring = True
        self.stats = []
        
        # Start monitoring thread
        monitor_thread = threading.Thread(target=self.collect_stats)
        monitor_thread.daemon = True
        monitor_thread.start()
        
        # Let it run for specified duration
        time.sleep(duration)
        self.monitoring = False
        
        # Wait for thread to finish
        monitor_thread.join()
        
        return self.stats
    
    def display_stats(self):
        """Display collected statistics"""
        if not self.stats:
            print("No statistics collected")
            return
        
        print("System Statistics:")
        print("Time     | CPU%  | Memory% | Processes")
        print("-" * 40)
        for stat in self.stats:
            print(f"{stat['timestamp']} | {stat['cpu_percent']:5.1f} | {stat['memory_percent']:7.1f} | {stat['process_count']:9d}")

# Use the system monitor
monitor = SystemMonitor()
print("Starting system monitoring for 5 seconds...")
monitor.start_monitoring(5)
monitor.display_stats()

Starting system monitoring for 5 seconds...
System Statistics:
Time     | CPU%  | Memory% | Processes
----------------------------------------
20:17:06 |   5.1 |    17.3 |        98
20:17:08 |   1.0 |    17.3 |        98
20:17:10 |   1.0 |    17.3 |        98

Key Takeaways

1. Processes are isolated instances of programs with their own memory space
2. Threads are lightweight execution units within processes that share memory
3. CPU-bound tasks benefit from multiprocessing
4. I/O-bound tasks benefit from multithreading
5. psutil is an excellent library for system monitoring and process management

Exercises

1. Process Explorer: Create a script that lists all running processes and their memory usage
2. Thread Pool: Implement a simple thread pool to handle multiple I/O operations
3. Resource Monitor: Build a real-time system resource monitor using threading

Additional Resources

• psutil Documentation
• Python Threading Module
• Python Multiprocessing Module
• Real Python - Concurrency

---

Next: Lesson 2: Multiprocessing and Multithreading