GMS-Zip - High Performance archive/zip replacement

A high-performance, concurrency-safe replacement for the standard archive/zip.

GMS-Zip is designed for high-load applications where speed, security, and interface flexibility are critical. It treats file systems, memory buffers, and network streams as first-class citizens, allowing you to manipulate archives without managing temporary files or complex buffers.

At a Glance

Why switch from the standard library?

Feature	archive/zip (StdLib)	GMS-Zip
Concurrency	Single-threaded	Parallel (Multi-core)
Performance	Baseline	~4x Faster (See Benchmarks)
Security	Vulnerable to Zip Slip	Native Protection
I/O Source	File / ReaderAt	Polymorphic (File, URL, Buffer)
Encryption	Legacy ZipCrypto (Read-only)	AES-256 & ZipCrypto (R/W)
Editing	Append only	Modify / Rename / Remove
Control	No cancellation	Context-aware

Key Capabilities

• Cloud-Native I/O: Stream archives directly from HTTP/S3 to the client without touching the disk.
• No External Dependencies The core library relies strictly on the Go standard library.
• Zero-Overhead Abstraction: Unified API for files, bytes, and streams using Source and Sink interfaces.
• The Archiver Pattern: No global state side-effects. Configure passwords and codecs once, reuse everywhere.
• Resilience: "Best Effort" strategy collects all errors (errors.Join) instead of failing on the first file.
• Legacy Support: Automatically handles Zip64, NTFS timestamps, Unix permissions, and CP866 (DOS) encodings.

---

Performance Benchmarks

GMS-Zip utilizes a fixed worker pool, memory pooling (sync.Pool), and a Zero-Allocation pipeline. It achieves massive speedups while maintaining allocation parity with the standard library.

Environment: Intel Core i5-12400F (6 cores, 12 threads)

Compression Speed

Scenario	Standard Lib	GMS-Zip (Sequential)	GMS-Zip (Parallel)	Speedup
1,000 Small Files	14.3 ms	15.0 ms	3.9 ms	3.7x faster
10 Medium Files (100MB)	179.8 ms	177.9 ms	42.2 ms	4.3x faster

Memory Efficiency

Metric	Standard Lib	GMS-Zip (Sequential)	GMS-Zip (Parallel)	Impact
Allocations	12,016 op	12,078 op	12,568 op	< 5% overhead
Memory Usage	0.48 MB/op	1.71 MB/op	13.8 MB/op	Bounded by worker count

Note: GMS-Zip trades a fixed amount of buffer memory (per worker) to saturate CPU cores, ensuring the GC remains idle even under heavy load.

---

Installation

go get rtlabs.tech/gms-zip

Usage Guide

The One-Liners (Simple)

For 90% of use cases, use the static helpers with safe defaults (Deflate compression, auto-detection).

package main

import "rtlabs.tech/gms-zip"

func main() {
  // Archive a directory to a file
  gmszip.ArchiveDir("data/images", gmszip.ToFilePath("backup.zip"))

  // Extract specific files
  gmszip.Unzip(
    gmszip.FromFilePath("backup.zip"),
    "restored/",
    gmszip.WithFromDir("images"), // Extract only this folder
  )

  // Read single file content directly into memory
  data, _ := gmszip.ReadFile(gmszip.FromFilePath("logs.zip"), "error.log")
}

The Archiver (Configured)

Use NewArchiver when you need specific settings (encryption, compression levels) isolated from the rest of your app.

func main() {
  // Create a reusable configuration
  archiver := gmszip.NewArchiver(
    gmszip.WithArchivePassword("secure-password-123"),
    gmszip.WithCompression(gmszip.ZStandard, zstd.SpeedBestCompression),
    // gmszip.WithWorkers(4), // Optional: Limit concurrency
  )

  ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
  defer cancel()

  // Use it for multiple operations
  err := archiver.UnzipWithContext(ctx, gmszip.FromFilePath("encrypted.zip"), "out/")
}

Modifying Archives (Edit Mode)

GMS-Zip allows surgical modifications to existing archives (Edit-in-place logic).

func main() {
  archive := gmszip.NewZip()
    
  // Load existing metadata (lightweight, doesn't read contents yet)
  src, _ := os.Open("app.zip")
  archive.LoadFromFile(src)
  src.Close()

  // Modify structure
  archive.Remove("logs/")
  archive.Rename("config.dev.json", "config.json")
    
  // Add new file dynamically (Lazy execution)
  archive.AddLazy("db.dump", func() (io.ReadCloser, error) {
    return exec.Command("pg_dump", "db").StdoutPipe()
  })

  // Write the new version
  out, _ := os.Create("app_v2.zip")
  archive.WriteTo(out)
}

Advanced: Polymorphic I/O

You can perform operations on any Source (File, URL, Buffer) using the unified API.

// Calculate SHA256 of a remote ZIP without storing it on disk
func HashRemoteZip(url string) string {
  h := sha256.New()
    
  // GMS-Zip handles the HTTP range requests or buffering automatically
  gmszip.UseSource(gmszip.FromURL(url, http.DefaultClient), 
    func(r io.Reader, _ io.ReaderAt, _ int64) error {
      _, err := io.Copy(h, r)
      return err
    },
  )
  return hex.EncodeToString(h.Sum(nil))
}

---

Configuration & Options

Functional Options

Configure operations per-file or per-archive:

• WithName("new.txt"): Rename file inside the archive.
• WithCompression(method, level): Override compression.
• WithEncryption(method, password): Set AES-256 or ZipCrypto.
• WithWorkers(n): Set number of parallel workers.
• WithProgress(callback): track progress of operations.

Filters

Selectively process files:

• WithOnly([]*File): Whitelist specific files.
• WithFromDir("folder"): Restrict operation to a specific directory.
• WithExcludeDir("folder"): Blacklist a directory.

Sorting Strategies

• SortDefault: Preserves insertion order.
• SortAlphabetical: Deterministic order (A-Z).
• SortSizeDescending: Ensures heavy files don't block the output queue in parallel mode.
• SortZIP64Optimized: Buckets files by size to optimize Zip64 header overhead.

Error Handling

GMS-Zip uses structured error handling.

• Bulk operations return errors.Join.
• Specific errors are wrapped in *FileError for introspection.

Error	Description
ErrFormat	Not a valid ZIP archive (invalid signatures).
ErrPasswordMismatch	Incorrect password or missing password.
ErrChecksum	CRC-32 integrity check failed.
ErrInsecurePath	Zip Slip detected: file path attempts to escape destination.
ErrDuplicateEntry	A file with this name already exists in the archive.
ErrAlgorithm	Compression method not supported (e.g., LZMA without plugin).
ErrFileNotFound	Requested entry is missing. Wraps fs.ErrNotExist.
ErrFilenameTooLong	Filename exceeds the ZIP limit of 65,535 bytes.
ErrResourceLimit	Extraction exceeded defined limits.
ErrNotImplemented	Code path is not implemented.

Copyright and License

Code and Documentation Copyright © 1994-2026, Stephen Kapp and Reaper Technologies Limited. All Rights Reserved.