Python vs Shell Learning Guide: Beginner to

Intermediate
Introduction and Overview
Learning Python and shell scripting simultaneously provides a comprehensive foundation for
automation, system administration, and software development. Python is a versatile, high-level
programming language known for its simplicity and readability[1][2], while shell scripting excels
at system administration tasks and command-line automation[3][4].

Comparison Overview
Aspect Python Shell

Syntax Clean, readable, indentation-based[1] Command-oriented, pipe-friendly[3]

Learning
Gentler for beginners[2] Steeper, many commands to memorize[5]
Curve

General programming, data analysis, web System administration, text processing,

Use Cases
development[2] automation[4]

Performance Higher execution efficiency[5] Lower due to process overhead[5]

Cross-
Excellent portability[5] Platform
platform

Error Handling Robust exception handling[6] Limited error handling capabilities[7]

Chapter 1: Getting Started - Basic Concepts

1.1 Variables and Data Types

Python Variables and Data Types

# Python - Dynamic typing, no declaration needed

name = "John Doe" # String
age = 25 # Integer
height = 5.9 # Float
is_student = True # Boolean
fruits = ["apple", "banana"] # List
person = {"name": "John", "age": 25} # Dictionary

# Check data type

print(type(name)) # <class 'str'>
Python supports various data types including integers, floats, strings, booleans, lists, tuples,
sets, and dictionaries[1][8]. Variables are dynamically typed and don't require explicit
declaration.

Shell Variables

#!/bin/bash
# Shell - Everything is treated as string by default
name="John Doe"
age=25
readonly PI=3.14159 # Read-only variable

# Variable usage - note the $ prefix

echo "Name: $name"
echo "Age: $age"
echo "Next year: $((age + 1))"

# Arrays in shell
fruits=("apple" "banana" "orange")
echo "First fruit: ${fruits[0]}"
echo "All fruits: ${fruits[@]}"

Shell variables are primarily strings, with special syntax for arithmetic operations[4][7]. Arrays
are supported but with different syntax than Python.

1.2 Input and Output

Python Input/Output

# Python input/output
name = input("Enter your name: ")
age = int(input("Enter your age: "))

print(f"Hello {name}, you are {age} years old")

print("Multiple", "values", sep="-", end="!\n")

# Formatted output
print("Age in 5 years: {}".format(age + 5))
print(f"Age in 10 years: {age + 10}")

Shell Input/Output

#!/bin/bash
# Shell input/output
echo "Enter your name:"
read name

echo "Enter your age:"

read age
 echo "Hello $name, you are $age years old"
echo -n "This doesn't add newline"
echo -e "This supports\tescapes\nlike tabs and newlines"

# Using command line arguments

echo "Script name: $0"
echo "First argument: $1"
echo "All arguments: $@"

Chapter 2: Control Flow - Conditionals and Loops

2.1 Conditional Statements

Python Conditionals

# Python if-elif-else
age = 20

if age >= 18:

print("You are an adult")
elif age >= 13:
print("You are a teenager")
else:
print("You are a child")

# Multiple conditions
score = 85
if score >= 90 and score <= 100:
grade = "A"
elif 80 <= score < 90: # Pythonic range check
grade = "B"
else:
grade = "C"

# Ternary operator
status = "adult" if age >= 18 else "minor"

Python uses clean, readable syntax with if, elif, and else keywords[9][10]. Conditions support
logical operators like and, or, and not.

Shell Conditionals

#!/bin/bash
# Shell if-elif-else
age=20

if [ $age -ge 18 ]; then

echo "You are an adult"
elif [ $age -ge 13 ]; then
echo "You are a teenager"
else
 echo "You are a child"
fi

# Multiple conditions with logical operators

score=85
if [ $score -ge 90 ] && [ $score -le 100 ]; then
grade="A"
elif [ $score -ge 80 ] && [ $score -lt 90 ]; then
grade="B"
else
grade="C"
fi

# String comparisons
if [ "$name" = "John" ]; then
echo "Hello John!"
fi

# File tests
if [ -f "[Link]" ]; then
echo "File exists"
fi

Shell conditionals use test operators like -eq, -gt, -lt for numbers and =, != for strings[11][7]. The
syntax requires specific spacing and brackets.

2.2 Loops

Python Loops

# For loops
for i in range(5):
print(f"Count: {i}")

# Iterating over collections

fruits = ["apple", "banana", "orange"]
for fruit in fruits:
print(f"I like {fruit}")

# While loops
count = 0
while count < 5:
print(f"While count: {count}")
count += 1

# List comprehensions (advanced)

squares = [x**2 for x in range(10) if x % 2 == 0]
print(squares) # [0, 4, 16, 36, 64]

Python offers elegant iteration with for and while loops, plus advanced features like list
comprehensions[9][12].
Shell Loops

#!/bin/bash
# For loops
for i in {1..5}; do
echo "Count: $i"
done

# Iterating over arrays

fruits=("apple" "banana" "orange")
for fruit in "${fruits[@]}"; do
echo "I like $fruit"
done

# While loops
count=0
while [ $count -lt 5 ]; do
echo "While count: $count"
count=$((count + 1))
done

# Until loops (unique to shell)

counter=0
until [ $counter -ge 5 ]; do
echo "Until counter: $counter"
counter=$((counter + 1))
done

# C-style for loop

for ((i=0; i<5; i++)); do
echo "C-style: $i"
done

Shell provides multiple loop types including for, while, until, and C-style loops[11][13].

Chapter 3: Functions and Code Organization

3.1 Functions

Python Functions

# Basic function
def greet(name, age=18):
"""Function with default parameter and docstring"""
return f"Hello {name}, you are {age} years old"

# Function with multiple return values

def calculate_stats(numbers):
return sum(numbers), len(numbers), sum(numbers)/len(numbers)

# Using the functions

message = greet("Alice", 25)
 print(message)

total, count, average = calculate_stats([1, 2, 3, 4, 5])

print(f"Total: {total}, Count: {count}, Average: {average}")

# Lambda functions (anonymous)

square = lambda x: x**2
print(square(5)) # 25

# Function as argument
def apply_operation(numbers, operation):
return [operation(x) for x in numbers]

result = apply_operation([1, 2, 3, 4], lambda x: x**2)

print(result) # [1, 4, 9, 16]

Python functions support default parameters, multiple return values, docstrings, and lambda
functions[14].

Shell Functions

#!/bin/bash
# Basic function
greet() {
local name=$1
local age=${2:-18} # Default value
echo "Hello $name, you are $age years old"
}

# Function with return value

calculate_sum() {
local a=$1
local b=$2
echo $((a + b)) # Return via echo
}

# Using functions
message=$(greet "Alice" 25)
echo "$message"

sum_result=$(calculate_sum 10 20)
echo "Sum: $sum_result"

# Function with exit status

check_file() {
if [ -f "$1" ]; then
return 0 # Success
else
return 1 # Failure
fi
}

if check_file "[Link]"; then

echo "File exists"
 else
echo "File not found"
fi

Shell functions use positional parameters ($1, $2) and return values through echo or exit status[11]
[15].

3.2 Modules and Libraries

Python Modules

# Creating a module (save as math_utils.py)

def add(a, b):
"""Add two numbers"""
return a + b

def multiply(a, b):

"""Multiply two numbers"""
return a * b

PI = 3.14159

# Using the module (in another file)

import math_utils
from math_utils import add, PI
import math_utils as math

result1 = math_utils.add(5, 3)
result2 = add(10, 20) # Direct import
result3 = [Link](4, 5) # Alias

# Built-in modules
import os, sys, datetime
import json, csv, re

# Package structure
from [Link] import function_name

Python's module system allows for organized, reusable code with packages and imports[16][17]
[18].

Shell Function Libraries

#!/bin/bash
# Creating a function library (save as [Link])
calculate_area() {
local length=$1
local width=$2
echo $((length * width))
}

log_message() {
 echo "[$(date)] $1" >> /var/log/[Link]
}

# Using the library (in another script)

#!/bin/bash
source ./[Link] # or . ./[Link]

area=$(calculate_area 10 5)
log_message "Calculated area: $area"

# System-wide library (add to ~/.bashrc)

if [ -f ~/bin/[Link] ]; then
source ~/bin/[Link]
fi

Shell libraries are created by sourcing script files containing functions[15][19].

Chapter 4: File Handling and Text Processing

4.1 File Operations

Python File Handling

# Writing to files
with open("[Link]", "w") as file:
[Link]("Hello, World!\n")
[Link]("This is line 2\n")

# Reading from files

with open("[Link]", "r") as file:
content = [Link]() # Read entire file
print(content)

# Reading line by line

with open("[Link]", "r") as file:
for line_num, line in enumerate(file, 1):
print(f"Line {line_num}: {[Link]()}")

# Appending to files
with open("[Link]", "a") as file:
[Link]("Appended line\n")

# Error handling
try:
with open("[Link]", "r") as file:
content = [Link]()
except FileNotFoundError:
print("File not found!")
except IOError:
print("Error reading file!")

# JSON handling
import json
 data = {"name": "John", "age": 30}
with open("[Link]", "w") as file:
[Link](data, file, indent=2)

with open("[Link]", "r") as file:

loaded_data = [Link](file)

Python provides robust file handling with context managers (with statement) and
comprehensive error handling[20][21][22].

Shell File Operations

#!/bin/bash
# Writing to files
echo "Hello, World!" > [Link]
echo "This is line 2" >> [Link] # Append

# Reading files
cat [Link] # Display entire file
head -n 5 [Link] # First 5 lines
tail -n 5 [Link] # Last 5 lines

# Reading line by line

while IFS= read -r line; do
echo "Processing: $line"
done < [Link]

# File tests and permissions

if [ -f "[Link]" ]; then
echo "File exists"
ls -l [Link] # Show permissions
chmod 755 [Link] # Change permissions
fi

# File information
wc -l [Link] # Count lines
wc -w [Link] # Count words
du -h [Link] # File size

# Backup and copy

cp [Link] [Link]
mv [Link] [Link]

Shell excels at file operations with powerful built-in commands and simple syntax[23][24][25].

4.2 Text Processing

Python Text Processing

import re

text = "The quick brown fox jumps over the lazy dog"

# String manipulation
print([Link]()) # UPPERCASE
print([Link]()) # lowercase
print([Link]("fox", "cat")) # Replace text
print([Link]()) # Split into words

# Regular expressions
pattern = r'\b\w{4}\b' # 4-letter words
matches = [Link](pattern, text)
print(matches) # ['over', 'lazy']

# Advanced text processing

lines = [
"apple,5,red",
"banana,3,yellow",
"orange,8,orange"
]

# Parse CSV-like data

for line in lines:
fruit, count, color = [Link](',')
print(f"{fruit}: {count} ({color})")

# File processing with regex

def process_log_file(filename):
pattern = r'(\d{4}-\d{2}-\d{2}) (\w+): (.+)'
with open(filename, 'r') as file:
for line in file:
match = [Link](pattern, [Link]())
if match:
date, level, message = [Link]()
print(f"Date: {date}, Level: {level}, Message: {message}")

Python offers comprehensive string manipulation and powerful regular expression support[14].

Shell Text Processing

#!/bin/bash
text="The quick brown fox jumps over the lazy dog"

# String manipulation
echo "$text" | tr '[:lower:]' '[:upper:]' # UPPERCASE
echo "$text" | tr '[:upper:]' '[:lower:]' # lowercase
echo "${text/fox/cat}" # Replace first occurrence
echo "${text//fox/cat}" # Replace all occurrences

# Text processing with AWK

echo "apple,5,red
 banana,3,yellow
orange,8,orange" | awk -F',' '{print $1 ": " $2 " (" $3 ")"}'

# Regular expressions with grep

echo "$text" | grep -oE '\b\w{4}\b' # Find 4-letter words

# Advanced text processing with sed

echo "$text" | sed 's/\b\w\{4\}\b/[&]/g' # Bracket 4-letter words

# Log file processing

process_log() {
local logfile="$1"
# Extract entries with ERROR level
grep "ERROR" "$logfile" | while read -r line; do
date=$(echo "$line" | cut -d' ' -f1)
level=$(echo "$line" | cut -d' ' -f2 | tr -d ':')
message=$(echo "$line" | cut -d' ' -f3-)
echo "Date: $date, Level: $level, Message: $message"
done
}

# Word frequency count

word_frequency() {
tr ' ' '\n' < "$1" | tr '[:upper:]' '[:lower:]' | \
sort | uniq -c | sort -nr | head -10
}

Shell provides powerful text processing tools like sed, awk, grep, and tr for efficient text
manipulation[23][26][27].

Chapter 5: Error Handling and Debugging

5.1 Error Handling

Python Exception Handling

# Basic try-except
try:
number = int(input("Enter a number: "))
result = 10 / number
print(f"Result: {result}")
except ValueError:
print("Invalid input! Please enter a number.")
except ZeroDivisionError:
print("Cannot divide by zero!")
except Exception as e:
print(f"An unexpected error occurred: {e}")
else:
print("Operation completed successfully!")
finally:
print("This always executes")

# Custom exceptions
 class CustomError(Exception):
"""Custom exception class"""
def __init__(self, message):
[Link] = message
super().__init__([Link])

def validate_age(age):
if age < 0:
raise CustomError("Age cannot be negative")
if age > 150:
raise CustomError("Age seems unrealistic")
return True

# Using custom exceptions

try:
age = -5
validate_age(age)
except CustomError as e:
print(f"Validation error: {e}")

# File handling with exceptions

def safe_file_read(filename):
try:
with open(filename, 'r') as file:
return [Link]()
except FileNotFoundError:
print(f"File '{filename}' not found")
return None
except PermissionError:
print(f"Permission denied for '{filename}'")
return None
except IOError as e:
print(f"IO error: {e}")
return None

Python provides comprehensive exception handling with try-except-finally blocks and custom
exceptions[6][28][29].

Shell Error Handling

#!/bin/bash
# Enable error handling
set -e # Exit on any error
set -u # Exit on undefined variables
set -o pipefail # Exit on pipe failures

# Basic error checking

check_file() {
local file="$1"
if [ ! -f "$file" ]; then
echo "Error: File '$file' not found" >&2
return 1
fi
return 0
 }

# Usage with error checking

if check_file "[Link]"; then
echo "File exists, processing..."
cat "[Link]"
else
echo "Cannot proceed without the file"
exit 1
fi

# Function with error handling

safe_copy() {
local source="$1"
local dest="$2"

if [ ! -f "$source" ]; then
echo "Error: Source file '$source' not found" >&2
return 1
fi

if ! cp "$source" "$dest" 2>/dev/null; then

echo "Error: Failed to copy '$source' to '$dest'" >&2
return 1
fi

echo "Successfully copied '$source' to '$dest'"

return 0
}

# Trap for cleanup

cleanup() {
echo "Cleaning up..."
rm -f /tmp/temp_file$$
}
trap cleanup EXIT

# Error logging
log_error() {
echo "[$(date)] ERROR: $1" >> /var/log/script_errors.log
}

# Command with error checking

if ! command -v python3 >/dev/null 2>&1; then
log_error "Python3 is not installed"
exit 1
fi

Shell error handling relies on exit codes, conditional checks, and the trap command for
cleanup[7].
5.2 Debugging Techniques

Python Debugging

# Debug with print statements

def calculate_average(numbers):
print(f"Debug: Input numbers = {numbers}") # Debug info
total = sum(numbers)
print(f"Debug: Total = {total}")
count = len(numbers)
print(f"Debug: Count = {count}")
average = total / count
print(f"Debug: Average = {average}")
return average

# Using logging for better debugging

import logging
[Link](level=[Link], format='%(levelname)s: %(message)s')

def process_data(data):
[Link](f"Processing data: {data}")
try:
result = data * 2
[Link](f"Result calculated: {result}")
return result
except Exception as e:
[Link](f"Error processing data: {e}")
raise

# Using pdb debugger

import pdb

def complex_function(x, y):

pdb.set_trace() # Debugger breakpoint
result = x + y
intermediate = result * 2
final = intermediate - 1
return final

# Debug with assertions

def divide(a, b):
assert b != 0, "Division by zero is not allowed"
assert isinstance(a, (int, float)), "First argument must be a number"
assert isinstance(b, (int, float)), "Second argument must be a number"
return a / b

# Interactive debugging techniques

def debug_info(obj):
print(f"Type: {type(obj)}")
print(f"Dir: {dir(obj)}")
if hasattr(obj, '__dict__'):
print(f"Attributes: {obj.__dict__}")
Python offers multiple debugging approaches including print statements, logging, pdb
debugger, and introspection tools[6][28][30].

Shell Debugging

#!/bin/bash
# Enable debugging modes
set -x # Print commands as they execute
set -v # Print script lines as they are read

# Debug function
debug() {
if [ "$DEBUG" = "1" ]; then
echo "DEBUG: $1" >&2
fi
}

# Usage
DEBUG=1
debug "Starting script execution"

# Conditional debugging
process_file() {
local file="$1"
debug "Processing file: $file"

if [ ! -f "$file" ]; then
debug "File not found: $file"
return 1
fi

local lines=$(wc -l < "$file")

debug "File has $lines lines"

# Process the file

cat "$file"
}

# Function tracing
trace_function() {
echo "Entering function: ${FUNCNAME[1]}" >&2
echo "Arguments: $@" >&2
}

example_function() {
trace_function "$@"
local arg1="$1"
local arg2="$2"
echo "Processing $arg1 and $arg2"
}

# Error tracing
error_handler() {
echo "Error occurred in ${BASH_SOURCE[1]} at line ${BASH_LINENO[0]}"
echo "Function: ${FUNCNAME[1]}"
 exit 1
}
trap error_handler ERR

# Performance timing
time_function() {
local start_time=$(date +%s.%N)
"$@" # Execute the command
local end_time=$(date +%s.%N)
local duration=$(echo "$end_time - $start_time" | bc)
echo "Function '$1' took $duration seconds"
}

# Usage: time_function some_command arg1 arg2

Shell debugging uses built-in options like set -x, custom debug functions, and tracing
techniques.

Chapter 6: Advanced Topics

6.1 Object-Oriented Programming (Python)

# Basic class definition

class Person:
"""A class representing a person"""

# Class variable
species = "Homo sapiens"

def __init__(self, name, age):

"""Initialize a Person instance"""
[Link] = name # Instance variable
[Link] = age

def introduce(self):
"""Instance method"""
return f"Hi, I'm {[Link]} and I'm {[Link]} years old"

def have_birthday(self):
"""Modify instance state"""
[Link] += 1
return f"Happy birthday! {[Link]} is now {[Link]}"

@classmethod
def from_birth_year(cls, name, birth_year):
"""Class method - alternative constructor"""
from datetime import datetime
age = [Link]().year - birth_year
return cls(name, age)

@staticmethod
def is_adult(age):
"""Static method - doesn't access instance or class"""
return age >= 18
 # Inheritance
class Student(Person):
def __init__(self, name, age, student_id):
super().__init__(name, age) # Call parent constructor
self.student_id = student_id
[Link] = []

def enroll(self, course):

[Link](course)
return f"{[Link]} enrolled in {course}"

def introduce(self): # Method overriding

parent_intro = super().introduce()
return f"{parent_intro}. My student ID is {self.student_id}"

# Usage
person = Person("Alice", 30)
print([Link]())

student = Student("Bob", 20, "S123456")

print([Link]())
print([Link]("Python Programming"))

# Class methods and static methods

young_person = Person.from_birth_year("Charlie", 2005)
print(Person.is_adult(young_person.age))

Python's OOP features include classes, inheritance, method overriding, class methods, and
static methods[31][32][33].

6.2 Advanced Shell Features

Arrays and Associative Arrays

#!/bin/bash
# Indexed arrays
fruits=("apple" "banana" "orange")
fruits[3]="grape"

echo "All fruits: ${fruits[@]}"

echo "Number of fruits: ${#fruits[@]}"
echo "First fruit: ${fruits[0]}"

# Iterating over array

for fruit in "${fruits[@]}"; do
echo "I like $fruit"
done

# Array manipulation
new_fruits=("${fruits[@]}" "kiwi" "mango") # Append elements
filtered_fruits=("${fruits[@]/apple/}") # Remove elements

# Associative arrays (Bash 4+)

 declare -A person
person["name"]="John"
person["age"]=30
person["city"]="New York"

echo "Name: ${person[name]}"

echo "Age: ${person[age]}"

# Iterate over associative array

for key in "${!person[@]}"; do
echo "$key: ${person[$key]}"
done

Regular Expressions

#!/bin/bash
# Pattern matching with regex
text="Email: [Link]@[Link] Phone: 555-1234"

# Using =~ operator for regex matching

email_pattern='[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}'
phone_pattern='[0-9]{3}-[0-9]{4}'

if [[ $text =~ $email_pattern ]]; then

echo "Found email: ${BASH_REMATCH[0]}"
fi

# Advanced pattern matching

validate_input() {
local input="$1"
local pattern="$2"

if [[ $input =~ $pattern ]]; then

return 0
else
return 1
fi
}

# Usage
if validate_input "test@[Link]" "$email_pattern"; then
echo "Valid email address"
fi

# String manipulation with patterns

filename="[Link]"
echo "Extension: ${filename##*.}" # pdf
echo "Basename: ${filename%.*}" # document

path="/home/user/documents/[Link]"
echo "Directory: ${path%/*}" # /home/user/documents
echo "Filename: ${path##*/}" # [Link]

Shell provides powerful pattern matching and string manipulation capabilities[34][35][36].

Chapter 7: Practical Applications and Projects

7.1 System Administration Scripts

Python System Script

#!/usr/bin/env python3
import os
import shutil
import psutil
import logging
from datetime import datetime

# Setup logging
[Link](
level=[Link],
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[
[Link]('system_monitor.log'),
[Link]()
]
)

class SystemMonitor:
def __init__(self):
[Link] = [Link](__name__)

def check_disk_space(self, threshold=80):

"""Check disk space usage"""
disk_usage = psutil.disk_usage('/')
used_percent = (disk_usage.used / disk_usage.total) * 100

if used_percent > threshold:

[Link](f"Disk usage is {used_percent:.1f}%")
return False
else:
[Link](f"Disk usage is {used_percent:.1f}%")
return True

def check_memory(self, threshold=80):

"""Check memory usage"""
memory = psutil.virtual_memory()
used_percent = [Link]

if used_percent > threshold:

[Link](f"Memory usage is {used_percent:.1f}%")
return False
else:
[Link](f"Memory usage is {used_percent:.1f}%")
return True

def backup_directory(self, source, destination):

"""Backup a directory"""
try:
 timestamp = [Link]().strftime("%Y%m%d_%H%M%S")
backup_name = f"backup_{timestamp}"
full_destination = [Link](destination, backup_name)

[Link](source, full_destination)
[Link](f"Backup completed: {full_destination}")
return True
except Exception as e:
[Link](f"Backup failed: {e}")
return False

def run_health_check(self):
"""Run complete system health check"""
results = {
'disk_space': self.check_disk_space(),
'memory': self.check_memory(),
'timestamp': [Link]().isoformat()
}
return results

# Usage
if __name__ == "__main__":
monitor = SystemMonitor()
health_results = monitor.run_health_check()
print(f"System health check results: {health_results}")

Shell System Script

#!/bin/bash
# System monitoring and backup script

# Configuration
DISK_THRESHOLD=80
MEMORY_THRESHOLD=80
LOG_FILE="/var/log/system_monitor.log"
BACKUP_DIR="/backup"

# Logging function
log_message() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

# Check disk space

check_disk_space() {
local usage=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')

if [ "$usage" -gt "$DISK_THRESHOLD" ]; then

log_message "WARNING: Disk usage is ${usage}%"
return 1
else
log_message "INFO: Disk usage is ${usage}%"
return 0
fi
}
# Check memory usage
check_memory() {
local usage=$(free | awk 'NR==2{printf "%.0f", $3*100/$2}')

if [ "$usage" -gt "$MEMORY_THRESHOLD" ]; then

log_message "WARNING: Memory usage is ${usage}%"
return 1
else
log_message "INFO: Memory usage is ${usage}%"
return 0
fi
}

# Backup function
backup_directory() {
local source="$1"
local destination="$2"
local timestamp=$(date +%Y%m%d_%H%M%S)
local backup_name="backup_${timestamp}"

if [ ! -d "$source" ]; then
log_message "ERROR: Source directory '$source' does not exist"
return 1
fi

mkdir -p "$destination"

if cp -r "$source" "${destination}/${backup_name}"; then

log_message "INFO: Backup completed: ${destination}/${backup_name}"
return 0
else
log_message "ERROR: Backup failed"
return 1
fi
}

# Main health check

run_health_check() {
log_message "Starting system health check"

local disk_ok=true
local memory_ok=true

if ! check_disk_space; then
disk_ok=false
fi

if ! check_memory; then
memory_ok=false
fi

if $disk_ok && $memory_ok; then

log_message "INFO: All systems normal"
return 0
else
log_message "WARNING: System issues detected"
 return 1
fi
}

# Command line interface

case "${1:-health}" in
"health")
run_health_check
;;
"backup")
if [ $# -lt 3 ]; then
echo "Usage: $0 backup <source> <destination>"
exit 1
fi
backup_directory "$2" "$3"
;;
*)
echo "Usage: $0 {health|backup <source> <destination>}"
exit 1
;;
esac

7.2 Data Processing Scripts

Python Data Processing

#!/usr/bin/env python3
import csv
import json
import re
from collections import defaultdict, Counter
from datetime import datetime

class LogAnalyzer:
def __init__(self, log_file):
self.log_file = log_file
[Link] = []
[Link] = defaultdict(int)

def parse_log_line(self, line):

"""Parse a single log line"""
# Example: [Link] - - [25/Dec/[Link] +0000] "GET /[Link] HTTP/1.1
pattern = r'(\d+\.\d+\.\d+\.\d+).*?\[(.*?)\] "(\w+) (.*?) HTTP.*?" (\d+) (\d+)'
match = [Link](pattern, line)

if match:
ip, timestamp, method, path, status, size = [Link]()
return {
'ip': ip,
'timestamp': timestamp,
'method': method,
'path': path,
'status': int(status),
'size': int(size)
 }
return None

def load_logs(self):
"""Load and parse log file"""
with open(self.log_file, 'r') as file:
for line_num, line in enumerate(file, 1):
entry = self.parse_log_line([Link]())
if entry:
[Link](entry)
else:
print(f"Warning: Could not parse line {line_num}")

def analyze_traffic(self):
"""Analyze traffic patterns"""
ip_counts = Counter(entry['ip'] for entry in [Link])
status_counts = Counter(entry['status'] for entry in [Link])
method_counts = Counter(entry['method'] for entry in [Link])

return {
'total_requests': len([Link]),
'unique_ips': len(ip_counts),
'top_ips': ip_counts.most_common(10),
'status_codes': dict(status_counts),
'methods': dict(method_counts)
}

def find_errors(self):
"""Find error entries (4xx, 5xx status codes)"""
errors = [entry for entry in [Link] if entry['status'] >= 400]
return errors

def export_results(self, output_file, format='json'):

"""Export analysis results"""
results = self.analyze_traffic()

if format == 'json':
with open(output_file, 'w') as file:
[Link](results, file, indent=2)
elif format == 'csv':
with open(output_file, 'w', newline='') as file:
writer = [Link](file)
[Link](['Metric', 'Value'])
[Link](['Total Requests', results['total_requests']])
[Link](['Unique IPs', results['unique_ips']])

[Link](['', '']) # Empty row

[Link](['Top IPs', 'Count'])
for ip, count in results['top_ips']:
[Link]([ip, count])

# Usage
if __name__ == "__main__":
analyzer = LogAnalyzer('[Link]')
analyzer.load_logs()
results = analyzer.analyze_traffic()
 analyzer.export_results('analysis_results.json')

print(f"Analyzed {results['total_requests']} requests")

print(f"Found {results['unique_ips']} unique IP addresses")

Shell Data Processing

#!/bin/bash
# Log analysis script

LOG_FILE="[Link]"
OUTPUT_DIR="analysis_output"
DATE=$(date +%Y%m%d)

# Create output directory

mkdir -p "$OUTPUT_DIR"

# Function to analyze log file

analyze_logs() {
local logfile="$1"
local output_prefix="$2"

echo "Analyzing log file: $logfile"

# Total requests
total_requests=$(wc -l < "$logfile")
echo "Total requests: $total_requests" > "${output_prefix}_summary.txt"

# Unique IP addresses
unique_ips=$(awk '{print $1}' "$logfile" | sort -u | wc -l)
echo "Unique IPs: $unique_ips" >> "${output_prefix}_summary.txt"

# Top 10 IP addresses
echo "Top 10 IP addresses:" > "${output_prefix}_top_ips.txt"
awk '{print $1}' "$logfile" | sort | uniq -c | sort -rn | head -10 >> "${output_prefi

# HTTP status codes

echo "HTTP Status Codes:" > "${output_prefix}_status_codes.txt"
awk '{print $9}' "$logfile" | sort | uniq -c | sort -rn >> "${output_prefix}_status_c

# Error analysis (4xx and 5xx)

echo "Error Analysis (4xx and 5xx):" > "${output_prefix}_errors.txt"
grep -E ' [45][0-9]{2} ' "$logfile" | awk '{print $1, $9}' | sort | uniq -c | sort -r

# Traffic by hour
echo "Traffic by hour:" > "${output_prefix}_hourly.txt"
awk '{
# Extract hour from timestamp [DD/MMM/YYYY:HH:MM:SS +ZONE]
match($4, /[0-9]{2}/)
hour = substr($4, RSTART, 2)
hours[hour]++
}
END {
for (hour in hours) {
printf "%02d:00 %d\n", hour, hours[hour]
 }
}' "$logfile" | sort >> "${output_prefix}_hourly.txt"

# Most requested URLs

echo "Top 10 requested URLs:" > "${output_prefix}_top_urls.txt"
awk '{print $7}' "$logfile" | sort | uniq -c | sort -rn | head -10 >> "${output_prefi
}

# Function to generate HTML report

generate_html_report() {
local prefix="$1"
local html_file="${prefix}_report.html"

cat > "$html_file" << EOF

<!DOCTYPE html>
<html>
<head>
<title>Log Analysis Report - $DATE</title>
<style>
body { font-family: Arial, sans-serif; margin: 20px; }
.section { margin-bottom: 30px; }
pre { background-color: #f4f4f4; padding: 10px; border-radius: 5px; }
</style>
</head>
<body>
<h1>Log Analysis Report</h1>
<p>Generated on: $(date)</p>

<div class="section">
<h2>Summary</h2>
<pre>$(cat "${prefix}_summary.txt")</pre>
</div>

<div class="section">
<h2>Top IP Addresses</h2>
<pre>$(cat "${prefix}_top_ips.txt")</pre>
</div>

<div class="section">
<h2>HTTP Status Codes</h2>
<pre>$(cat "${prefix}_status_codes.txt")</pre>
</div>

<div class="section">
<h2>Hourly Traffic</h2>
<pre>$(cat "${prefix}_hourly.txt")</pre>
</div>
</body>
</html>
EOF

echo "HTML report generated: $html_file"

}

# Main execution
if [ ! -f "$LOG_FILE" ]; then
 echo "Error: Log file '$LOG_FILE' not found"
exit 1
fi

output_prefix="${OUTPUT_DIR}/analysis_${DATE}"
analyze_logs "$LOG_FILE" "$output_prefix"
generate_html_report "$output_prefix"

echo "Analysis complete. Results saved in: $OUTPUT_DIR"

Learning Path and Best Practices

Beginner Level (A1-A2)

Start with fundamentals:
Python: Variables, data types, basic I/O, simple conditionals and loops[1][2]
Shell: Basic commands, variables, simple scripts, file operations[3][4]
Practice projects:
Simple calculators
File organizers
Basic text processors

Intermediate Level (B1-B2)

Build on basics:
Python: Functions, modules, file handling, exception handling, basic OOP[37][14]
Shell: Functions, arrays, advanced text processing with sed/awk, error handling[11][13]
Practice projects:
Log analyzers
System monitors
Data converters

Advanced Level (C1-C2)

Master complex concepts:
Python: Advanced OOP, decorators, generators, metaclasses, async programming[38][33]
Shell: Regular expressions, associative arrays, complex pipelines, libraries[34][35]
Best Practices

Python Best Practices

1. Follow PEP 8 style guidelines
2. Use virtual environments for project isolation
3. Write docstrings for functions and classes
4. Handle exceptions appropriately
5. Use list comprehensions for concise code
6. Leverage built-in functions and libraries

Shell Best Practices

1. Use proper shebang lines (#!/bin/bash)
2. Quote variables to handle spaces
3. Check exit codes for error handling
4. Use functions for code reuse
5. Enable strict mode (set -euo pipefail)

6. Use meaningful variable names

When to Use Each Language

Choose Python when:
Building complex applications[39][40]
Need cross-platform compatibility[5]
Working with data structures and algorithms
Requiring extensive libraries and frameworks
Error handling is critical
Choose Shell when:
System administration tasks[4][39]
Simple file operations and text processing
Chaining existing command-line tools
Quick automation scripts
Working primarily on Unix/Linux systems
Conclusion
Both Python and shell scripting are valuable skills that complement each other. Python excels in
complex logic, data processing, and application development, while shell scripting is unmatched
for system administration and quick automation tasks. Learning both languages simultaneously
provides a comprehensive toolkit for automation and development work.
The key to mastering both is consistent practice, starting with simple projects and gradually
building complexity. Use Python for tasks requiring robust error handling and complex data
structures, and use shell scripting for system-level operations and text processing tasks.
By following this guide and practicing the provided examples, you'll develop proficiency in both
languages and understand when to apply each one effectively in real-world scenarios.