Image Plugin Architecture
Deep dive into the Image Plugin's architecture, powered by Sharp.
Overview
The Image Plugin provides high-performance image processing capabilities through Sharp, a fast Node.js library based on libvips.
Key Technologies:
- Sharp: Core image processing engine
- libvips: High-performance image processing library (C++)
- Node.js streams: Efficient memory usage for large images
Plugin Structure
plugins/image/
├── package.json # Plugin metadata
├── tsconfig.json # TypeScript configuration
├── README.md # Plugin documentation
├── bin/
│ └── cli.js # Standalone CLI entry
└── src/
├── index.ts # Plugin exports
├── register.ts # Plugin registration
├── types.ts # Type definitions
├── utils/
│ ├── sharp-helpers.ts # Sharp utility functions
│ ├── validation.ts # Input validation
│ └── metadata.ts # Image metadata extraction
└── commands/
├── resize.ts # Resize command
├── convert.ts # Format conversion
├── optimize.ts # Optimization
├── crop.ts # Cropping
├── rotate.ts # Rotation
├── blur.ts # Blur effects
├── sharpen.ts # Sharpening
└── composite.ts # Image composition
Core Dependencies
Sharp Library
Why Sharp?
- Performance: 4-10x faster than ImageMagick/GraphicsMagick
- Memory efficient: Streaming processing, handles large images
- Modern formats: WebP, AVIF, HEIF support
- Quality: High-quality resampling algorithms
Installation:
npm install sharp
Basic Usage:
import sharp from "sharp";
await sharp("input.jpg")
.resize(1920, 1080)
.webp({ quality: 85 })
.toFile("output.webp");
libvips
Sharp is built on libvips, a C library:
Features:
- Fast image processing
- Low memory footprint
- Lazy evaluation and pipelines
- Threading and SIMD optimization
Official Documentation:
https://libvips.github.io/libvips/
Command Architecture
Command Pattern
All image commands follow this pattern:
import type { Command, CommandContext } from "@mediaproc/core";
import sharp from "sharp";
export const resizeCommand: Command = {
name: "resize",
description: "Resize images to specified dimensions",
options: [
{
name: "width",
alias: "w",
type: "number",
description: "Target width in pixels",
},
{
name: "height",
alias: "h",
type: "number",
description: "Target height in pixels",
},
{
name: "fit",
type: "string",
choices: ["cover", "contain", "fill", "inside", "outside"],
default: "cover",
description: "How to fit the image",
},
{
name: "format",
alias: "f",
type: "string",
choices: ["jpg", "png", "webp", "avif", "tiff"],
description: "Output format",
},
],
async handler(context: CommandContext) {
const { input, options, logger } = context;
// Validate options
validateResizeOptions(options);
// Process each input file
const results = await Promise.all(
input.map((file) => resizeImage(file, options, logger)),
);
// Return summary
return {
success: true,
processed: results.filter((r) => r.success).length,
failed: results.filter((r) => !r.success).length,
};
},
};
Processing Pipeline
Each command uses Sharp's fluent API:
async function resizeImage(
input: string,
options: ResizeOptions,
logger: Logger,
): Promise<ProcessResult> {
try {
const pipeline = sharp(input);
// 1. Read metadata
const metadata = await pipeline.metadata();
logger.info(`Processing: ${input} (${metadata.width}x${metadata.height})`);
// 2. Build pipeline
pipeline
.resize({
width: options.width,
height: options.height,
fit: options.fit,
withoutEnlargement: options.noEnlarge,
})
.rotate() // Auto-rotate based on EXIF
.withMetadata(); // Preserve metadata
// 3. Apply format conversion
if (options.format) {
applyFormatOptions(pipeline, options.format, options);
}
// 4. Generate output path
const output = generateOutputPath(input, options);
// 5. Execute pipeline
await pipeline.toFile(output);
logger.success(`Created: ${output}`);
return { success: true, output };
} catch (error) {
logger.error(`Failed: ${input} - ${error.message}`);
return { success: false, error: error.message };
}
}
Format Handling
Supported Formats
The Image Plugin supports all formats Sharp provides:
| Format | Extension | Read | Write | Notes |
|---|---|---|---|---|
| JPEG | .jpg | ✅ | ✅ | Lossy, universal support |
| PNG | .png | ✅ | ✅ | Lossless, transparency |
| WebP | .webp | ✅ | ✅ | Modern, efficient |
| AVIF | .avif | ✅ | ✅ | Best compression, slower |
| GIF | .gif | ✅ | ✅ | Animation support |
| TIFF | .tiff | ✅ | ✅ | Professional, uncompressed |
| HEIF | .heic | ✅ | ✅ | Apple ecosystem |
| SVG | .svg | ✅ | ❌ | Vector (rasterized on read) |
| ✅ | ❌ | First page only |
Format-Specific Options
Each format has specific quality settings:
function applyFormatOptions(
pipeline: sharp.Sharp,
format: string,
options: any,
): void {
switch (format) {
case "jpg":
case "jpeg":
pipeline.jpeg({
quality: options.quality ?? 85,
progressive: options.progressive ?? true,
mozjpeg: options.mozjpeg ?? true,
});
break;
case "png":
pipeline.png({
quality: options.quality ?? 100,
compressionLevel: options.compression ?? 9,
progressive: options.progressive ?? true,
});
break;
case "webp":
pipeline.webp({
quality: options.quality ?? 85,
lossless: options.lossless ?? false,
effort: options.effort ?? 4,
});
break;
case "avif":
pipeline.avif({
quality: options.quality ?? 85,
lossless: options.lossless ?? false,
effort: options.effort ?? 4,
});
break;
case "tiff":
pipeline.tiff({
quality: options.quality ?? 100,
compression: options.compression ?? "lzw",
});
break;
}
}
Resize Strategies
Fit Options
Sharp provides multiple resize strategies:
enum FitMode {
// Cover entire area, crop if needed
COVER = "cover",
// Contain within area, letterbox if needed
CONTAIN = "contain",
// Fill entire area, stretch if needed
FILL = "fill",
// Fit inside, never enlarge
INSIDE = "inside",
// Fit outside, always fill
OUTSIDE = "outside",
}
Visual Examples:
Original: 1000x500
Target: 800x800
COVER: [========] (crops to fill)
[========]
CONTAIN: [-- --] (letterbox)
[--------]
[-- --]
FILL: [========] (stretches)
[========]
INSIDE: [--------] (fits, no enlarge)
[ ]
OUTSIDE: [========] (ensures fill)
[========]
Position Options
Control how images are positioned when resizing:
type Position =
| "center" // Default
| "top"
| "bottom"
| "left"
| "right"
| "top left"
| "top right"
| "bottom left"
| "bottom right"
| "entropy" // Auto-detect important area
| "attention"; // Auto-detect faces/features
Smart Cropping:
// Entropy-based (high contrast areas)
sharp(input).resize(800, 600, {
fit: "cover",
position: "entropy",
});
// Attention-based (faces, text)
sharp(input).resize(800, 600, {
fit: "cover",
position: "attention",
});
Performance Optimization
Streaming Processing
Sharp uses streams for memory efficiency:
import { pipeline } from "stream/promises";
async function processStream(input: string, output: string) {
await pipeline(
fs.createReadStream(input),
sharp().resize(1920, 1080).webp({ quality: 85 }),
fs.createWriteStream(output),
);
}
Benefits:
- Constant memory usage regardless of image size
- Can process images larger than available RAM
- Efficient for high-throughput scenarios
Parallel Processing
Process multiple images in parallel:
async function processBatch(
files: string[],
options: ProcessOptions,
concurrency: number = 4,
): Promise<Result[]> {
const chunks = chunkArray(files, concurrency);
const results: Result[] = [];
for (const chunk of chunks) {
const batchResults = await Promise.all(
chunk.map((file) => processImage(file, options)),
);
results.push(...batchResults);
}
return results;
}
Caching
Cache processed images to avoid redundant work:
import crypto from "crypto";
function getCacheKey(input: string, options: any): string {
const optionsHash = crypto
.createHash("md5")
.update(JSON.stringify(options))
.digest("hex");
return `${input}-${optionsHash}`;
}
async function getCached(key: string): Promise<string | null> {
const cachePath = path.join(".cache", `${key}.jpg`);
if (await fileExists(cachePath)) {
return cachePath;
}
return null;
}
Metadata Handling
Reading Metadata
Extract comprehensive image metadata:
async function getMetadata(input: string): Promise<ImageMetadata> {
const image = sharp(input);
const metadata = await image.metadata();
return {
// Dimensions
width: metadata.width,
height: metadata.height,
aspectRatio: metadata.width / metadata.height,
// Format
format: metadata.format,
space: metadata.space, // Color space
channels: metadata.channels,
depth: metadata.depth, // Bit depth
// EXIF data
exif: metadata.exif,
orientation: metadata.orientation,
// Size
size: metadata.size,
// Color
hasAlpha: metadata.hasAlpha,
hasProfile: metadata.hasProfile,
// Animation
isAnimated: metadata.pages > 1,
pages: metadata.pages,
};
}
Preserving Metadata
Keep EXIF, IPTC, and XMP data:
sharp(input)
.resize(1920, 1080)
.withMetadata({
// Preserve orientation
orientation: metadata.orientation,
// Custom metadata
exif: {
IFD0: {
Copyright: "MediaProc 2026",
Artist: "Your Name",
},
},
})
.toFile(output);
EXIF Orientation
Handle EXIF orientation automatically:
// Auto-rotate based on EXIF
sharp(input)
.rotate() // Reads EXIF orientation tag
.resize(1920, 1080)
.toFile(output);
Color Space Handling
Color Spaces
Sharp supports multiple color spaces:
- sRGB: Standard RGB (web, most displays)
- RGB16: 16-bit RGB
- CMYK: Print color space
- LAB: Perceptual color space
- Grayscale: Single channel
Color Conversion
Convert between color spaces:
// Convert to grayscale
sharp(input).grayscale().toFile(output);
// Convert to specific space
sharp(input).toColorspace("lab").toFile(output);
// Normalize to sRGB
sharp(input).toColorspace("srgb").toFile(output);
Image Operations
Available Operations
The Image Plugin supports all Sharp operations:
Resizing:
resize()- Change dimensionsextend()- Add paddingextract()- Crop regiontrim()- Remove borders
Transformations:
rotate()- Rotate imageflip()- Flip verticallyflop()- Flip horizontallyaffine()- Affine transform
Effects:
blur()- Gaussian blursharpen()- Sharpen edgesmedian()- Median filternegate()- Invert colorsnormalize()- Normalize contrastmodulate()- Adjust brightness/saturation/huetint()- Apply color tint
Composition:
composite()- Layer imagesoverlayWith()- Add overlay
Operation Chaining
Chain operations in a pipeline:
await sharp(input)
// 1. Resize
.resize(1920, 1080, { fit: "cover" })
// 2. Rotate
.rotate(90)
// 3. Sharpen
.sharpen()
// 4. Adjust colors
.modulate({
brightness: 1.1,
saturation: 0.9,
})
// 5. Convert format
.webp({ quality: 85 })
// 6. Save
.toFile(output);
Error Handling
Common Errors
Handle Sharp-specific errors:
async function processImage(input: string, options: any) {
try {
await sharp(input).resize(options.width, options.height).toFile(output);
} catch (error) {
if (error.message.includes("Input file is missing")) {
throw new Error(`File not found: ${input}`);
}
if (
error.message.includes("Input buffer contains unsupported image format")
) {
throw new Error(`Unsupported image format: ${input}`);
}
if (error.message.includes("Image dimensions exceed maximum")) {
throw new Error(`Image too large: ${input}`);
}
// Generic error
throw new Error(`Processing failed: ${error.message}`);
}
}
Validation
Validate inputs before processing:
async function validateImage(file: string): Promise<void> {
// Check if file exists
if (!(await fileExists(file))) {
throw new Error(`File not found: ${file}`);
}
// Check if readable
if (!(await isReadable(file))) {
throw new Error(`Cannot read file: ${file}`);
}
// Validate format
const metadata = await sharp(file).metadata();
if (!metadata.format) {
throw new Error(`Invalid image format: ${file}`);
}
// Check dimensions
if (metadata.width > 50000 || metadata.height > 50000) {
throw new Error(`Image dimensions too large: ${file}`);
}
}
Testing
Unit Tests
Test individual commands:
import { resizeCommand } from "../commands/resize";
describe("Resize Command", () => {
it("should resize image", async () => {
const context = {
input: ["test.jpg"],
options: { width: 1920, height: 1080 },
logger: mockLogger,
};
const result = await resizeCommand.handler(context);
expect(result.success).toBe(true);
expect(result.processed).toBe(1);
});
});
Integration Tests
Test with actual images:
import sharp from "sharp";
describe("Image Processing", () => {
it("should produce correct output", async () => {
await processImage("input.jpg", {
width: 800,
height: 600,
format: "webp",
});
const metadata = await sharp("output.webp").metadata();
expect(metadata.width).toBe(800);
expect(metadata.height).toBe(600);
expect(metadata.format).toBe("webp");
});
});