Image Plugin Architecture

Deep dive into the Image Plugin's architecture, powered by Sharp.

Overview

The Image Plugin provides high-performance image processing capabilities through Sharp, a fast Node.js library based on libvips.

Key Technologies:

  • Sharp: Core image processing engine
  • libvips: High-performance image processing library (C++)
  • Node.js streams: Efficient memory usage for large images

Plugin Structure

plugins/image/ ├── package.json # Plugin metadata ├── tsconfig.json # TypeScript configuration ├── README.md # Plugin documentation ├── bin/ │ └── cli.js # Standalone CLI entry └── src/ ├── index.ts # Plugin exports ├── register.ts # Plugin registration ├── types.ts # Type definitions ├── utils/ │ ├── sharp-helpers.ts # Sharp utility functions │ ├── validation.ts # Input validation │ └── metadata.ts # Image metadata extraction └── commands/ ├── resize.ts # Resize command ├── convert.ts # Format conversion ├── optimize.ts # Optimization ├── crop.ts # Cropping ├── rotate.ts # Rotation ├── blur.ts # Blur effects ├── sharpen.ts # Sharpening └── composite.ts # Image composition

Core Dependencies

Sharp Library

Why Sharp?

  • Performance: 4-10x faster than ImageMagick/GraphicsMagick
  • Memory efficient: Streaming processing, handles large images
  • Modern formats: WebP, AVIF, HEIF support
  • Quality: High-quality resampling algorithms

Installation:

npm install sharp

Basic Usage:

import sharp from "sharp";

await sharp("input.jpg")
  .resize(1920, 1080)
  .webp({ quality: 85 })
  .toFile("output.webp");

libvips

Sharp is built on libvips, a C library:

Features:

  • Fast image processing
  • Low memory footprint
  • Lazy evaluation and pipelines
  • Threading and SIMD optimization

Official Documentation:
https://libvips.github.io/libvips/


Command Architecture

Command Pattern

All image commands follow this pattern:

import type { Command, CommandContext } from "@mediaproc/core";
import sharp from "sharp";

export const resizeCommand: Command = {
  name: "resize",
  description: "Resize images to specified dimensions",

  options: [
    {
      name: "width",
      alias: "w",
      type: "number",
      description: "Target width in pixels",
    },
    {
      name: "height",
      alias: "h",
      type: "number",
      description: "Target height in pixels",
    },
    {
      name: "fit",
      type: "string",
      choices: ["cover", "contain", "fill", "inside", "outside"],
      default: "cover",
      description: "How to fit the image",
    },
    {
      name: "format",
      alias: "f",
      type: "string",
      choices: ["jpg", "png", "webp", "avif", "tiff"],
      description: "Output format",
    },
  ],

  async handler(context: CommandContext) {
    const { input, options, logger } = context;

    // Validate options
    validateResizeOptions(options);

    // Process each input file
    const results = await Promise.all(
      input.map((file) => resizeImage(file, options, logger)),
    );

    // Return summary
    return {
      success: true,
      processed: results.filter((r) => r.success).length,
      failed: results.filter((r) => !r.success).length,
    };
  },
};

Processing Pipeline

Each command uses Sharp's fluent API:

async function resizeImage(
  input: string,
  options: ResizeOptions,
  logger: Logger,
): Promise<ProcessResult> {
  try {
    const pipeline = sharp(input);

    // 1. Read metadata
    const metadata = await pipeline.metadata();
    logger.info(`Processing: ${input} (${metadata.width}x${metadata.height})`);

    // 2. Build pipeline
    pipeline
      .resize({
        width: options.width,
        height: options.height,
        fit: options.fit,
        withoutEnlargement: options.noEnlarge,
      })
      .rotate() // Auto-rotate based on EXIF
      .withMetadata(); // Preserve metadata

    // 3. Apply format conversion
    if (options.format) {
      applyFormatOptions(pipeline, options.format, options);
    }

    // 4. Generate output path
    const output = generateOutputPath(input, options);

    // 5. Execute pipeline
    await pipeline.toFile(output);

    logger.success(`Created: ${output}`);
    return { success: true, output };
  } catch (error) {
    logger.error(`Failed: ${input} - ${error.message}`);
    return { success: false, error: error.message };
  }
}

Format Handling

Supported Formats

The Image Plugin supports all formats Sharp provides:

FormatExtensionReadWriteNotes
JPEG.jpgLossy, universal support
PNG.pngLossless, transparency
WebP.webpModern, efficient
AVIF.avifBest compression, slower
GIF.gifAnimation support
TIFF.tiffProfessional, uncompressed
HEIF.heicApple ecosystem
SVG.svgVector (rasterized on read)
PDF.pdfFirst page only

Format-Specific Options

Each format has specific quality settings:

function applyFormatOptions(
  pipeline: sharp.Sharp,
  format: string,
  options: any,
): void {
  switch (format) {
    case "jpg":
    case "jpeg":
      pipeline.jpeg({
        quality: options.quality ?? 85,
        progressive: options.progressive ?? true,
        mozjpeg: options.mozjpeg ?? true,
      });
      break;

    case "png":
      pipeline.png({
        quality: options.quality ?? 100,
        compressionLevel: options.compression ?? 9,
        progressive: options.progressive ?? true,
      });
      break;

    case "webp":
      pipeline.webp({
        quality: options.quality ?? 85,
        lossless: options.lossless ?? false,
        effort: options.effort ?? 4,
      });
      break;

    case "avif":
      pipeline.avif({
        quality: options.quality ?? 85,
        lossless: options.lossless ?? false,
        effort: options.effort ?? 4,
      });
      break;

    case "tiff":
      pipeline.tiff({
        quality: options.quality ?? 100,
        compression: options.compression ?? "lzw",
      });
      break;
  }
}

Resize Strategies

Fit Options

Sharp provides multiple resize strategies:

enum FitMode {
  // Cover entire area, crop if needed
  COVER = "cover",

  // Contain within area, letterbox if needed
  CONTAIN = "contain",

  // Fill entire area, stretch if needed
  FILL = "fill",

  // Fit inside, never enlarge
  INSIDE = "inside",

  // Fit outside, always fill
  OUTSIDE = "outside",
}

Visual Examples:

Original: 1000x500 Target: 800x800 COVER: [========] (crops to fill) [========] CONTAIN: [-- --] (letterbox) [--------] [-- --] FILL: [========] (stretches) [========] INSIDE: [--------] (fits, no enlarge) [ ] OUTSIDE: [========] (ensures fill) [========]

Position Options

Control how images are positioned when resizing:

type Position =
  | "center" // Default
  | "top"
  | "bottom"
  | "left"
  | "right"
  | "top left"
  | "top right"
  | "bottom left"
  | "bottom right"
  | "entropy" // Auto-detect important area
  | "attention"; // Auto-detect faces/features

Smart Cropping:

// Entropy-based (high contrast areas)
sharp(input).resize(800, 600, {
  fit: "cover",
  position: "entropy",
});

// Attention-based (faces, text)
sharp(input).resize(800, 600, {
  fit: "cover",
  position: "attention",
});

Performance Optimization

Streaming Processing

Sharp uses streams for memory efficiency:

import { pipeline } from "stream/promises";

async function processStream(input: string, output: string) {
  await pipeline(
    fs.createReadStream(input),
    sharp().resize(1920, 1080).webp({ quality: 85 }),
    fs.createWriteStream(output),
  );
}

Benefits:

  • Constant memory usage regardless of image size
  • Can process images larger than available RAM
  • Efficient for high-throughput scenarios

Parallel Processing

Process multiple images in parallel:

async function processBatch(
  files: string[],
  options: ProcessOptions,
  concurrency: number = 4,
): Promise<Result[]> {
  const chunks = chunkArray(files, concurrency);
  const results: Result[] = [];

  for (const chunk of chunks) {
    const batchResults = await Promise.all(
      chunk.map((file) => processImage(file, options)),
    );
    results.push(...batchResults);
  }

  return results;
}

Caching

Cache processed images to avoid redundant work:

import crypto from "crypto";

function getCacheKey(input: string, options: any): string {
  const optionsHash = crypto
    .createHash("md5")
    .update(JSON.stringify(options))
    .digest("hex");

  return `${input}-${optionsHash}`;
}

async function getCached(key: string): Promise<string | null> {
  const cachePath = path.join(".cache", `${key}.jpg`);

  if (await fileExists(cachePath)) {
    return cachePath;
  }

  return null;
}

Metadata Handling

Reading Metadata

Extract comprehensive image metadata:

async function getMetadata(input: string): Promise<ImageMetadata> {
  const image = sharp(input);
  const metadata = await image.metadata();

  return {
    // Dimensions
    width: metadata.width,
    height: metadata.height,
    aspectRatio: metadata.width / metadata.height,

    // Format
    format: metadata.format,
    space: metadata.space, // Color space
    channels: metadata.channels,
    depth: metadata.depth, // Bit depth

    // EXIF data
    exif: metadata.exif,
    orientation: metadata.orientation,

    // Size
    size: metadata.size,

    // Color
    hasAlpha: metadata.hasAlpha,
    hasProfile: metadata.hasProfile,

    // Animation
    isAnimated: metadata.pages > 1,
    pages: metadata.pages,
  };
}

Preserving Metadata

Keep EXIF, IPTC, and XMP data:

sharp(input)
  .resize(1920, 1080)
  .withMetadata({
    // Preserve orientation
    orientation: metadata.orientation,

    // Custom metadata
    exif: {
      IFD0: {
        Copyright: "MediaProc 2026",
        Artist: "Your Name",
      },
    },
  })
  .toFile(output);

EXIF Orientation

Handle EXIF orientation automatically:

// Auto-rotate based on EXIF
sharp(input)
  .rotate() // Reads EXIF orientation tag
  .resize(1920, 1080)
  .toFile(output);

Color Space Handling

Color Spaces

Sharp supports multiple color spaces:

  • sRGB: Standard RGB (web, most displays)
  • RGB16: 16-bit RGB
  • CMYK: Print color space
  • LAB: Perceptual color space
  • Grayscale: Single channel

Color Conversion

Convert between color spaces:

// Convert to grayscale
sharp(input).grayscale().toFile(output);

// Convert to specific space
sharp(input).toColorspace("lab").toFile(output);

// Normalize to sRGB
sharp(input).toColorspace("srgb").toFile(output);

Image Operations

Available Operations

The Image Plugin supports all Sharp operations:

Resizing:

  • resize() - Change dimensions
  • extend() - Add padding
  • extract() - Crop region
  • trim() - Remove borders

Transformations:

  • rotate() - Rotate image
  • flip() - Flip vertically
  • flop() - Flip horizontally
  • affine() - Affine transform

Effects:

  • blur() - Gaussian blur
  • sharpen() - Sharpen edges
  • median() - Median filter
  • negate() - Invert colors
  • normalize() - Normalize contrast
  • modulate() - Adjust brightness/saturation/hue
  • tint() - Apply color tint

Composition:

  • composite() - Layer images
  • overlayWith() - Add overlay

Operation Chaining

Chain operations in a pipeline:

await sharp(input)
  // 1. Resize
  .resize(1920, 1080, { fit: "cover" })

  // 2. Rotate
  .rotate(90)

  // 3. Sharpen
  .sharpen()

  // 4. Adjust colors
  .modulate({
    brightness: 1.1,
    saturation: 0.9,
  })

  // 5. Convert format
  .webp({ quality: 85 })

  // 6. Save
  .toFile(output);

Error Handling

Common Errors

Handle Sharp-specific errors:

async function processImage(input: string, options: any) {
  try {
    await sharp(input).resize(options.width, options.height).toFile(output);
  } catch (error) {
    if (error.message.includes("Input file is missing")) {
      throw new Error(`File not found: ${input}`);
    }

    if (
      error.message.includes("Input buffer contains unsupported image format")
    ) {
      throw new Error(`Unsupported image format: ${input}`);
    }

    if (error.message.includes("Image dimensions exceed maximum")) {
      throw new Error(`Image too large: ${input}`);
    }

    // Generic error
    throw new Error(`Processing failed: ${error.message}`);
  }
}

Validation

Validate inputs before processing:

async function validateImage(file: string): Promise<void> {
  // Check if file exists
  if (!(await fileExists(file))) {
    throw new Error(`File not found: ${file}`);
  }

  // Check if readable
  if (!(await isReadable(file))) {
    throw new Error(`Cannot read file: ${file}`);
  }

  // Validate format
  const metadata = await sharp(file).metadata();

  if (!metadata.format) {
    throw new Error(`Invalid image format: ${file}`);
  }

  // Check dimensions
  if (metadata.width > 50000 || metadata.height > 50000) {
    throw new Error(`Image dimensions too large: ${file}`);
  }
}

Testing

Unit Tests

Test individual commands:

import { resizeCommand } from "../commands/resize";

describe("Resize Command", () => {
  it("should resize image", async () => {
    const context = {
      input: ["test.jpg"],
      options: { width: 1920, height: 1080 },
      logger: mockLogger,
    };

    const result = await resizeCommand.handler(context);

    expect(result.success).toBe(true);
    expect(result.processed).toBe(1);
  });
});

Integration Tests

Test with actual images:

import sharp from "sharp";

describe("Image Processing", () => {
  it("should produce correct output", async () => {
    await processImage("input.jpg", {
      width: 800,
      height: 600,
      format: "webp",
    });

    const metadata = await sharp("output.webp").metadata();

    expect(metadata.width).toBe(800);
    expect(metadata.height).toBe(600);
    expect(metadata.format).toBe("webp");
  });
});


Learn More

Found an issue? Help us improve this page.

Edit on GitHub →