File Downloader Library - System Design
1. Overview
A standalone, reusable file downloader library designed for iOS/macOS platforms with support for
large files, background downloads, and concurrent operations with a clean, thread-safe API.
2. Architecture Diagram
┌───────────────────────────────────────────────────
│ Client Application │
└───────────────────────────────────────────────────
│
▼
┌───────────────────────────────────────────────────
│ FileDownloader (Public API) │
│
┌───────────────────────────────────────────────────
│
│ │ + download(url, to: destination) -> DownloadTask │ │
│ │ + pause(taskId) / resume(taskId) / cancel(taskId) │ │
│ │ + setMaxConcurrentDownloads(count) │ │
│ │ + addProgressListener(taskId, listener) │ │
│ │ + setConfiguration(config) │ │
│
└───────────────────────────────────────────────────
│
└───────────────────────────────────────────────────
│
┌─────────────────────────── ─────────────────────── ┼
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐
┌──────────────────┐
│ Download Manager │ │ Task Queue │ │ Config Manager │
│ │◄───│ │ │ │
│ - Orchestration │ │ - Priority Queue │ │ - Retry Policy │
│ - State Machine │ │ - Concurrency │ │ - Timeout │
│ - Retry Logic │ │ Control │ │ - Headers │
└──────────────────┘ └──────────────────┘
└──────────────────┘
│ │
▼ ▼
┌───────────────────────────────────────────────────
│ Download Engine │
│ ┌──────────────┐ ┌──────────────┐
┌────────────────────┐ │
│ │ URL Session │ │ Chunk Manager│ │ Background Session │ │
│ │ Handler │ │ │ │ Support │ │
│ └──────────────┘ └──────────────┘
└────────────────────┘ │
└───────────────────────────────────────────────────
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐
┌──────────────────┐
│ Storage Layer │ │ Cache Manager │ │ Persistence Layer│
│ │ │ │ │ │
│ - File Writer │ │ - Memory Cache │ │ - State Storage │
│ - Temp Storage │ │ - Disk Cache │ │ - Metadata DB │
│ - Cleanup │ │ - LRU Eviction │ │ - Resume Data │
└──────────────────┘ └──────────────────┘
└──────────────────┘
3. Component Design
3.1 Core Components
┌───────────────────────────────────────────────────
│ DownloadTask │
├───────────────────────────────────────────────────
│ Properties: │
│ - taskId: UUID │
│ - url: URL │
│ - destination: URL │
│ - state: DownloadState │
│ - progress: DownloadProgress │
│ - priority: Priority │
│ - metadata: [String: Any] │
│ - resumeData: Data? │
│ - error: Error? │
├───────────────────────────────────────────────────
│ Methods: │
│ + pause() │
│ + resume() │
│ + cancel() │
│ + addProgressObserver(observer) │
│ + setPriority(priority) │
└───────────────────────────────────────────────────
┌───────────────────────────────────────────────────
│ DownloadState Machine │
├───────────────────────────────────────────────────
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Pending │─────▶│Preparing│─────▶│Downloading│ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ │ │ │
│ │ ┌─────────┐ ▼ │
│ └─────────────▶│ Failed │ ┌─────────┐ │
│ └─────────┘ │ Paused │ │
│ ▲ └─────────┘ │
│ │ │ │
│ ┌─────────┐ ▼ │
│ │Cancelled│ ┌─────────┐ │
│ └─────────┘ │Completed│ │
│ └─────────┘ │
└───────────────────────────────────────────────────
3.2 Download Manager Architecture
┌───────────────────────────────────────────────────
│ DownloadManager │
├───────────────────────────────────────────────────
│ Components: │
│ │
│ ┌─────────────────────┐ ┌──────────────────────┐ │
│ │ Task Registry │ │ Execution Engine │ │
│ │ ┌─────────────────┐ │ │ ┌────────────────┐ │ │
│ │ │ Active Tasks │ │ │ │ Worker Pool │ │ │
│ │ ├─────────────────┤ │ │ ├────────────────┤ │ │
│ │ │ Pending Queue │ │───▶│ │ Thread Safety │ │ │
│ │ ├─────────────────┤ │ │ ├────────────────┤ │ │
│ │ │ Completed Tasks │ │ │ │ Concurrency │ │ │
│ │ └─────────────────┘ │ │ │ Limiter │ │ │
│ └─────────────────────┘ │ └────────────────┘ │ │
│ └──────────────────────┘ │
│ │
│ ┌─────────────────────┐ ┌──────────────────────┐ │
│ │ Priority Manager │ │ Retry Manager │ │
│ │ ┌─────────────────┐ │ │ ┌────────────────┐ │ │
│ │ │ Priority Queue │ │ │ │ Retry Policy │ │ │
│ │ ├─────────────────┤ │ │ ├────────────────┤ │ │
│ │ │ Task Scheduler │ │ │ │ Backoff Logic │ │ │
│ │ └─────────────────┘ │ │ │ Max Attempts │ │ │
│ └─────────────────────┘ │ └────────────────┘ │ │
│ └──────────────────────┘ │
└───────────────────────────────────────────────────
4. Detailed Functional Requirements
4.1 Core Features (As Specified)
✅ Standalone Reusable Library
✅ Start Download
✅ Pause & Resume
✅ Cancel
✅ Download Progress Reporting
✅ Parallel Downloads
✅ Limit Number of Parallel Downloads
✅ Support Large Files
✅ Background Download Support
4.2 Additional Features (Enhanced)
Advanced Download Management
Chunked Downloads: Split large files into chunks for parallel downloading
Range Requests: Support HTTP range headers for resumable downloads
Multi-part Downloads: Download file segments from multiple sources
Download Prioritization: Queue management with priority levels
Bandwidth Throttling: Control download speed to manage bandwidth
Smart Retry: Exponential backoff with jitter for failed downloads
Auto-Resume: Automatically resume interrupted downloads
Progress & Monitoring
Granular Progress: Bytes downloaded, total size, speed, ETA
Network Speed Monitoring: Real-time download speed tracking
Progress Persistence: Save progress for app restarts
Download History: Track completed, failed, and cancelled downloads
Analytics Integration: Hooks for tracking download metrics
File Management
Integrity Verification: MD5/SHA checksum validation
Automatic Decompression: Support for zip, gzip, tar files
File Move/Copy Options: Flexible destination handling
Temporary File Management: Clean up incomplete downloads
Duplicate Detection: Avoid re-downloading existing files
Storage Space Checking: Pre-flight storage availability checks
Network Optimization
Connection Pooling: Reuse connections for efficiency
DNS Prefetching: Optimize connection establishment
IPv6 Support: Full IPv6 compatibility
Proxy Support: HTTP/HTTPS/SOCKS proxy configuration
Custom Headers: Support for authentication and custom headers
Cookie Management: Handle cookies for authenticated downloads
5. API Design
5.1 Swift API Interface
swift
// MARK: - Main API
public class FileDownloader {
// Singleton instance
public static let shared = FileDownloader()
// Configuration
public func configure(_ configuration: DownloaderConfiguration)
// Core Operations
@discardableResult
public func download(
from url: URL,
to destination: URL? = nil,
priority: DownloadPriority = .normal,
headers: [String: String]? = nil,
metadata: [String: Any]? = nil
) -> DownloadTask
// Batch Operations
public func downloadBatch(
_ requests: [DownloadRequest],
completion: @escaping BatchCompletionHandler
) -> BatchDownloadTask
// Control Operations
public func pause(taskId: String)
public func resume(taskId: String)
public func cancel(taskId: String)
public func pauseAll()
public func resumeAll()
public func cancelAll()
// Query Operations
public func getTask(id: String) -> DownloadTask?
public func getAllTasks() -> [DownloadTask]
public func getActiveTasks() -> [DownloadTask]
public func getPendingTasks() -> [DownloadTask]
// Configuration
public func setMaxConcurrentDownloads(_ count: Int)
public func setBandwidthLimit(_ bytesPerSecond: Int?)
public func setRetryPolicy(_ policy: RetryPolicy)
}
// MARK: - Configuration
public struct DownloaderConfiguration {
public var maxConcurrentDownloads: Int = 3
public var timeoutInterval: TimeInterval = 60
public var allowsCellularAccess: Bool = true
public var sessionConfiguration: URLSessionConfiguration = .default
public var retryPolicy: RetryPolicy = .default
public var cachePolicy: CachePolicy = .default
public var storagePolicy: StoragePolicy = .default
public var backgroundSessionIdentifier: String?
}
// MARK: - Download Task
public protocol DownloadTaskProtocol {
var id: String { get }
var url: URL { get }
var state: DownloadState { get }
var progress: DownloadProgress { get }
func pause()
func resume()
func cancel()
func addProgressHandler(_ handler: @escaping ProgressHandler)
func addCompletionHandler(_ handler: @escaping CompletionHandler)
}
// MARK: - Progress Tracking
public struct DownloadProgress {
public let bytesDownloaded: Int64
public let totalBytes: Int64
public let downloadSpeed: Double // bytes per second
public let estimatedTimeRemaining: TimeInterval?
public var percentComplete: Double {
return totalBytes > 0 ? Double(bytesDownloaded) / Double(totalBytes) : 0
}
}
// MARK: - Callbacks
public typealias ProgressHandler = (DownloadProgress) -> Void
public typealias CompletionHandler = (Result<URL, DownloadError>) -> Void
public typealias BatchCompletionHandler = ([DownloadResult]) -> Void
5.2 Objective-C Bridge API
objc
// For Objective-C compatibility
@interface FDLFileDownloader : NSObject
+ (instancetype)sharedDownloader;
- (FDLDownloadTask *)downloadFromURL:(NSURL *)url
toDestination:(nullable NSURL *)destination
completion:(FDLCompletionHandler)completion;
- (void)pauseTaskWithId:(NSString *)taskId;
- (void)resumeTaskWithId:(NSString *)taskId;
- (void)cancelTaskWithId:(NSString *)taskId;
@end
6. Implementation Details
6.1 Thread Safety Architecture
┌───────────────────────────────────────────────────
│ Thread Safety Design │
├───────────────────────────────────────────────────
│ │
│ Main Queue (UI Thread) │
│ ┌────────────────────────────────────────────┐ │
│ │ • API calls │ │
│ │ • Progress callbacks │ │
│ │ • Completion handlers │ │
│ └────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ Serial Queue (Coordinator) │
│ ┌────────────────────────────────────────────┐ │
│ │ • State management │ │
│ │ • Task registry updates │ │
│ │ • Priority queue management │ │
│ └────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ Concurrent Queue (Workers) │
│ ┌────────────────────────────────────────────┐ │
│ │ • Download operations │ │
│ │ • File I/O │ │
│ │ • Network requests │ │
│ └────────────────────────────────────────────┘ │
│ │
│ Synchronization Primitives: │
│ • DispatchSemaphore for resource limiting │
│ • os_unfair_lock for critical sections │
│ • Atomic properties for state flags │
│ • Actor pattern for Swift concurrency │
└───────────────────────────────────────────────────
6.2 Large File Handling Strategy
┌───────────────────────────────────────────────────
│ Large File Download Strategy │
├───────────────────────────────────────────────────
│ │
│ 1. File Size Detection │
│ └─> HEAD request to get Content-Length │
│ │
│ 2. Chunking Decision │
│ ├─> If size > 10MB: Use chunked download │
│ └─> If size ≤ 10MB: Single download │
│ │
│ 3. Chunk Management │
│ ├─> Calculate optimal chunk size (2-5MB) │
│ ├─> Create range requests │
│ └─> Parallel chunk downloads │
│ │
│ 4. Assembly Process │
│ ├─> Write chunks to temp files │
│ ├─> Merge on completion │
│ └─> Verify integrity │
│ │
│ 5. Memory Management │
│ ├─> Stream writing (no full file in memory) │
│ ├─> Memory mapped files for large operations │
│ └─> Automatic memory pressure handling │
└───────────────────────────────────────────────────
6.3 Background Download Implementation
swift
// Background Session Configuration
class BackgroundDownloadManager {
private lazy var backgroundSession: URLSession = {
let config = URLSessionConfiguration.background(
withIdentifier: "com.app.downloader.background"
)
config.isDiscretionary = false
config.sessionSendsLaunchEvents = true
config.shouldUseExtendedBackgroundIdleMode = true
return URLSession(
configuration: config,
delegate: self,
delegateQueue: nil
)
}()
// Handle app termination/suspension
func handleEventsForBackgroundURLSession(
identifier: String,
completionHandler: @escaping () -> Void
){
// Reconnect to background session
// Process completed downloads
// Call completion handler
}
}
7. Database Schema
7.1 Core Tables
sql
-- Download Tasks Table
CREATE TABLE download_tasks (
id TEXT PRIMARY KEY,
url TEXT NOT NULL,
destination_path TEXT NOT NULL,
state INTEGER NOT NULL,
priority INTEGER DEFAULT 0,
bytes_downloaded INTEGER DEFAULT 0,
total_bytes INTEGER,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
completed_at TIMESTAMP,
error_message TEXT,
retry_count INTEGER DEFAULT 0,
resume_data BLOB,
metadata TEXT -- JSON
);
-- Download Chunks Table (for large files)
CREATE TABLE download_chunks (
id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
chunk_index INTEGER NOT NULL,
start_byte INTEGER NOT NULL,
end_byte INTEGER NOT NULL,
bytes_downloaded INTEGER DEFAULT 0,
state INTEGER NOT NULL,
temp_file_path TEXT,
FOREIGN KEY (task_id) REFERENCES download_tasks(id)
);
-- Download History Table
CREATE TABLE download_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT NOT NULL,
event_type TEXT NOT NULL, -- started, paused, resumed, completed, failed
timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
details TEXT -- JSON
);
-- Indexes
CREATE INDEX idx_tasks_state ON download_tasks(state);
CREATE INDEX idx_tasks_priority ON download_tasks(priority);
CREATE INDEX idx_chunks_task ON download_chunks(task_id);
8. Error Handling & Recovery
8.1 Error Types
swift
public enum DownloadError: Error {
// Network Errors
case networkUnavailable
case timeout
case connectionLost
case invalidURL
case httpError(statusCode: Int)
// File System Errors
case insufficientStorage
case fileWriteError
case destinationNotWritable
case fileAlreadyExists
// State Errors
case invalidState
case taskNotFound
case alreadyDownloading
// Validation Errors
case checksumMismatch
case corruptedFile
case sizeMismatch
// Configuration Errors
case maxRetriesExceeded
case quotaExceeded
}
8.2 Retry Strategy
┌───────────────────────────────────────────────────
│ Retry Strategy │
├───────────────────────────────────────────────────
│ │
│ Exponential Backoff with Jitter: │
│ │
│ Retry 1: 1s + random(0-1s) │
│ Retry 2: 2s + random(0-2s) │
│ Retry 3: 4s + random(0-4s) │
│ Retry 4: 8s + random(0-8s) │
│ Retry 5: 16s + random(0-16s) │
│ │
│ Conditions for Retry: │
│ • Network timeout │
│ • 5xx server errors │
│ • Connection failures │
│ • Partial content errors │
│ │
│ No Retry Conditions: │
│ • 4xx client errors (except 408, 429) │
│ • Insufficient storage │
│ • Invalid URL │
│ • User cancellation │
└───────────────────────────────────────────────────
9. Testing Strategy
9.1 Unit Testing
swift
class DownloadManagerTests: XCTestCase {
func testConcurrentDownloadLimit() { }
func testPauseResumeDownload() { }
func testLargeFileChunking() { }
func testRetryMechanism() { }
func testBackgroundDownload() { }
func testErrorRecovery() { }
func testMemoryPressure() { }
func testDiskSpaceHandling() { }
}
9.2 Integration Testing
swift
class IntegrationTests: XCTestCase {
func testRealNetworkDownload() { }
func testBackgroundSessionHandling() { }
func testAppTerminationRecovery() { }
func testCellularDataRestriction() { }
}
9.3 Performance Testing
swift
class PerformanceTests: XCTestCase {
func testLargeFilePerformance() { }
func testConcurrentDownloadPerformance() { }
func testMemoryUsage() { }
func testDiskIOPerformance() { }
}
10. Monitoring & Analytics
10.1 Metrics to Track
swift
struct DownloadMetrics {
// Performance Metrics
let averageDownloadSpeed: Double
let peakDownloadSpeed: Double
let totalBytesDownloaded: Int64
let averageFileSize: Int64
// Reliability Metrics
let successRate: Double
let failureRate: Double
let retryRate: Double
let averageRetryCount: Double
// User Behavior Metrics
let pauseResumeCount: Int
let backgroundDownloadCount: Int
let averageDownloadDuration: TimeInterval
// Error Metrics
let errorDistribution: [DownloadError: Int]
let timeoutRate: Double
let networkErrorRate: Double
}
11. Performance Optimizations
11.1 Memory Optimization
Stream-based file writing
Memory-mapped files for large operations
Automatic cache purging on memory warnings
Lazy loading of metadata
11.2 Network Optimization
HTTP/2 multiplexing
Keep-alive connections
Compression support (gzip, deflate)
Smart chunk size calculation based on bandwidth
11.3 Disk I/O Optimization
Asynchronous file operations
Batch writes for efficiency
Temporary file cleanup
Pre-allocation for large files
12. Security Considerations
12.1 Security Features
SSL/TLS certificate pinning
Integrity verification (checksums)
Secure temporary file handling
Sandboxed file access
URL validation and sanitization
12.2 Privacy
No tracking without consent
Local-only metadata storage
Secure credential handling
Anonymized analytics
13. Platform-Specific Considerations
13.1 iOS Specific
Background fetch API integration
Low power mode detection
Cellular data restrictions
App extension support
13.2 macOS Specific
Finder integration
Quick Look support
Spotlight indexing
Power Nap compatibility
14. Migration & Versioning
14.1 Version Migration
swift
protocol MigrationProtocol {
func migrate(from oldVersion: String, to newVersion: String)
func validateMigration() -> Bool
}
14.2 Backward Compatibility
Maintain API compatibility
Graceful degradation
Feature detection
Legacy format support
15. Example Usage
15.1 Basic Download
swift
let downloader = FileDownloader.shared
// Simple download
let task = downloader.download(from: url) { result in
switch result {
case .success(let fileURL):
print("Downloaded to: \(fileURL)")
case .failure(let error):
print("Download failed: \(error)")
}
}
// With progress tracking
task.addProgressHandler { progress in
print("Progress: \(progress.percentComplete * 100)%")
print("Speed: \(progress.downloadSpeed) bytes/sec")
print("ETA: \(progress.estimatedTimeRemaining ?? 0) seconds")
}
15.2 Advanced Usage
swift
// Configure downloader
let config = DownloaderConfiguration()
config.maxConcurrentDownloads = 5
config.allowsCellularAccess = false
config.retryPolicy = .exponentialBackoff(maxRetries: 3)
downloader.configure(config)
// Batch download with priority
let requests = [
DownloadRequest(url: url1, priority: .high),
DownloadRequest(url: url2, priority: .normal),
DownloadRequest(url: url3, priority: .low)
]
downloader.downloadBatch(requests) { results in
results.forEach { result in
print("Download result: \(result)")
}
}
// Background download
let backgroundTask = downloader.download(
from: largeFileURL,
options: [.allowBackground, .discretionary]
)
16. Conclusion
This File Downloader Library design provides a robust, scalable, and efficient solution for handling file
downloads in iOS/macOS applications. The architecture emphasizes:
Modularity: Clear separation of concerns with well-defined components
Scalability: Support for concurrent downloads with intelligent resource management
Reliability: Comprehensive error handling and recovery mechanisms
Performance: Optimized for both small and large file downloads
Extensibility: Plugin architecture for custom features
User Experience: Background support and progress tracking
The design can be extended with additional features like P2P downloads, torrent support, or cloud
storage integration while maintaining the core architecture's integrity.
