Storage and Paths

Multi Host supports multiple storage backends for uploaded content. This documentation covers configuration for local filesystem storage, object storage services, and hybrid approaches.

Storage Architecture

The storage system separates concerns into distinct layers:

• Upload Reception - Temporary storage during upload processing
• Original Storage - Permanent storage for uploaded files
• Thumbnail Storage - Generated derivatives and cached content
• Public Access - Serving mechanism for user requests

Each layer can use different storage backends depending on performance, cost, and operational requirements.

Directory Structure

A typical local installation uses this structure:

/var/www/html/                  # Application root
├── uploads/                    # Temporary upload processing
├── storage/
│   ├── originals/             # Original uploaded files
│   │   ├── 2025/
│   │   │   ├── 01/
│   │   │   └── 02/
│   │   └── ...
│   └── thumbnails/            # Generated thumbnails
│       ├── small/
│       ├── medium/
│       └── large/
├── cache/                      # Application cache
└── logs/                       # Application logs

Date-based subdirectories prevent filesystem performance issues from too many files in a single directory. Most filesystems slow significantly beyond 10,000-50,000 files per directory.

Local Filesystem Storage

Basic Configuration

$config['storage_driver'] = 'local';
$config['storage_path'] = '/var/www/storage/originals/';
$config['storage_url'] = 'https://example.com/storage/originals/';
$config['thumbnail_path'] = '/var/www/storage/thumbnails/';
$config['thumbnail_url'] = 'https://example.com/storage/thumbnails/';
$config['temp_path'] = '/var/www/uploads/';

Path Configuration Details

| Setting | Purpose | Security Notes | |---------|---------|----------------| | storage_path | Original file storage | Consider outside document root | | storage_url | Public URL for originals | May route through delivery script | | thumbnail_path | Generated thumbnail storage | Can be publicly accessible | | thumbnail_url | Public URL for thumbnails | Direct web server access typical | | temp_path | Upload processing temporary files | Should be outside document root |

Filesystem Permissions

The web server user needs appropriate access:

# Set ownership
chown -R www-data:www-data /var/www/storage/
chown -R www-data:www-data /var/www/uploads/

# Set permissions
chmod 755 /var/www/storage/
chmod 755 /var/www/storage/originals/
chmod 755 /var/www/storage/thumbnails/
chmod 700 /var/www/uploads/  # More restrictive for temp files

Avoid 777 permissions—they create security vulnerabilities even if they appear to solve permission problems.

Storage Outside Document Root

For enhanced security, place original uploads outside the web-accessible directory:

$config['storage_path'] = '/var/uploads/originals/';
$config['storage_url'] = 'https://example.com/image/';

With this configuration, requests to /image/ route through a delivery script that validates access and streams the file. This prevents direct access to stored files and enables access control, logging, and hotlink protection.

The delivery script pattern:

// Simplified example - actual implementation more complex
$file = validateAndResolvePath($_GET['id']);
if ($file && userCanAccess($file)) {
    header('Content-Type: ' . $file['mime']);
    readfile($file['path']);
} else {
    http_response_code(404);
}

Object Storage Integration

Amazon S3

$config['storage_driver'] = 's3';
$config['s3_key'] = 'AKIAIOSFODNN7EXAMPLE';
$config['s3_secret'] = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY';
$config['s3_region'] = 'us-east-1';
$config['s3_bucket'] = 'my-image-bucket';
$config['s3_path_prefix'] = 'uploads/';
$config['s3_url'] = 'https://my-image-bucket.s3.amazonaws.com/';

S3-Compatible Services

Services like MinIO, Backblaze B2, DigitalOcean Spaces, and Wasabi use S3-compatible APIs:

$config['storage_driver'] = 's3';
$config['s3_key'] = 'your-access-key';
$config['s3_secret'] = 'your-secret-key';
$config['s3_region'] = 'auto';  // Or specific region
$config['s3_bucket'] = 'your-bucket';
$config['s3_endpoint'] = 'https://s3.us-west-001.backblazeb2.com';
$config['s3_url'] = 'https://your-bucket.s3.us-west-001.backblazeb2.com/';

The s3_endpoint setting directs requests to the alternative service instead of AWS.

Object Storage Considerations

Latency: Object storage adds latency compared to local filesystem. This affects upload processing time but typically not user-facing delivery when using CDN.

Consistency: Some object storage services have eventual consistency. Newly uploaded files may not be immediately available. Plan for this in upload workflows.

Costs: Object storage charges for storage volume, requests, and egress bandwidth. Model costs based on your usage patterns before committing.

Direct Upload: For large files, consider browser-direct uploads to object storage using pre-signed URLs. This bypasses your application server entirely.

Hybrid Storage Strategies

Combine storage backends for optimal cost and performance:

Hot/Cold Storage

$config['storage_driver'] = 'hybrid';
$config['hot_storage'] = [
    'driver' => 'local',
    'path' => '/var/www/storage/hot/',
    'max_age' => 7  // days
];
$config['cold_storage'] = [
    'driver' => 's3',
    'bucket' => 'archive-bucket'
];

Recently uploaded files stay on fast local storage. Older files migrate to cheaper object storage automatically.

Separate Thumbnail Storage

$config['storage_driver'] = 's3';
$config['thumbnail_driver'] = 'local';
$config['thumbnail_path'] = '/var/www/storage/thumbnails/';

Keep frequently-accessed thumbnails on local SSD while storing originals in object storage.

CDN Integration

Content delivery networks cache and serve files from edge locations worldwide:

Pull-Based CDN

Configure your CDN to pull from your storage URLs:

$config['storage_url'] = 'https://example.com/storage/';
$config['cdn_url'] = 'https://cdn.example.com/storage/';

The CDN fetches files from your origin on first request, then serves from cache. Update URLs in responses to use the CDN domain.

Push-Based CDN

Some configurations push files to CDN storage during upload:

$config['cdn_push_enabled'] = true;
$config['cdn_push_endpoint'] = 'https://api.cdn.example.com/';
$config['cdn_push_key'] = 'your-api-key';

Push approaches ensure content is available at edges before users request it, eliminating cold-start latency.

Cache Invalidation

When files are deleted or replaced, invalidate CDN caches:

$config['cdn_invalidation_enabled'] = true;
$config['cdn_invalidation_endpoint'] = 'https://api.cdn.example.com/purge';

Plan invalidation strategies for your update patterns. Aggressive invalidation maintains consistency but may increase CDN costs.

Path Patterns

Date-Based Paths

$config['path_pattern'] = 'date';  // Creates YYYY/MM/ structure

Files organize by upload date, distributing across directories automatically.

Hash-Based Paths

$config['path_pattern'] = 'hash';  // Uses first characters of filename hash
$config['hash_depth'] = 2;  // Creates ab/cd/ structure

Hash distribution spreads files evenly regardless of upload timing, useful for archival systems.

User-Based Paths

$config['path_pattern'] = 'user';  // Creates user_id/ structure

Organizes by uploader, simplifying per-user backup or migration operations.

Storage Quotas

Global Quotas

$config['storage_quota_enabled'] = true;
$config['storage_quota_total'] = 1099511627776;  // 1 TB
$config['storage_quota_warning'] = 0.8;  // Warn at 80%

Per-User Quotas

$config['user_quota_enabled'] = true;
$config['user_quota_default'] = 104857600;  // 100 MB
$config['user_quota_include_thumbnails'] = false;

Quota calculations can include or exclude generated thumbnails. Most configurations exclude them since users don't directly control thumbnail creation.

Frequently Asked Questions

Can I migrate between storage backends?

Yes, migration tools copy files between backends while updating database references. Plan for downtime or read-only periods during migration. Test thoroughly with a subset before full migration.

How do I handle storage failures?

Implement monitoring for storage health and capacity. For critical deployments, consider redundant storage with failover. Object storage services typically provide built-in redundancy.

What's the best storage option for high traffic?

Object storage with CDN handles high traffic efficiently. The CDN absorbs request load while object storage provides scalable capacity. Local storage works for moderate traffic but requires more capacity planning.

How do I optimise storage costs?

Use lifecycle policies to move old content to cheaper storage tiers. Enable compression where appropriate. Implement deduplication if your use case has many identical uploads. Monitor and eliminate orphaned files.

Can thumbnails use different storage than originals?

Yes, configure separate drivers for each. A common pattern keeps thumbnails on fast local storage for quick access while storing larger originals in object storage.