fastjsond 1.0.2

fastjsond

High-performance JSON parser for D.

Built on simdjson, the SIMD-accelerated JSON parser.

Overview

fastjsond provides two APIs:

• Native API: Zero-copy parsing with maximum performance
• std.json-compatible API: Drop-in replacement with faster execution

SIMD instruction sets (AVX2, SSE4.2, NEON) are auto-detected at runtime.

Features

• ⚡ High Performance: 7-20x faster than std.json on typical workloads
• 🔒 Zero-Copy: Native API returns string slices directly into the JSON buffer
• 🔄 Drop-in Replacement: fastjsond.std is API-compatible with std.json
• 🛡️ Type-Safe: Strong typing with JsonType, JsonError, and Result<T> types
• 🧵 Thread-Safe std API: JSONValue is immutable and thread-safe after creation
• 📦 No Dependencies: Self-contained with embedded simdjson
• 🎯 @nogc Support: Native API works in @nogc contexts
• 🔍 Comprehensive Error Handling: Detailed error codes and exception support
• 🚀 SIMD Optimized: Auto-detects and uses best SIMD instructions available

Installation

Add to your dub.json:

"dependencies": {
    "fastjsond": "~>1.0.2"
}

Or with dub.sdl:

dependency "fastjsond" version="~>1.0.2"

Building from Source

git clone https://github.com/federikowsky/fastjsond.git
cd fastjsond
make lib    # Build static library
make test   # Run tests

Quick Start

Drop-in Replacement for std.json

import fastjsond.std;  // Change import from std.json

auto json = parseJSON(`{"name": "Aurora", "version": 2}`);
string name = json["name"].str;
long ver = json["version"].integer;

Native Zero-Copy API

import fastjsond;

auto parser = Parser.create();
auto doc = parser.parse(`{
    "name": "Aurora",
    "version": 2,
    "features": ["fast", "safe"]
}`);

if (!doc.valid) {
    writeln("Error: ", doc.errorMessage);
    return;
}

// Zero-copy: getString returns slice into original buffer
const(char)[] name = doc.root["name"].getString;
long ver = doc.root["version"].getInt;

// Iteration
foreach (feature; doc.root["features"]) {
    writeln(feature.getString);
}

API Reference

Native API (fastjsond)

Parser

// Create parser with default capacity (~4GB)
auto parser = Parser.create();

// Create parser with custom max capacity
auto parser = Parser(1024 * 1024);  // 1MB max

// Parse JSON (multiple overloads)
auto doc = parser.parse(jsonString);
auto doc = parser.parse(cast(const(char)[]) json);
auto doc = parser.parse(cast(const(ubyte)[]) json);

// Parse with pre-padded buffer (for maximum performance)
auto doc = parser.parsePadded(paddedBuffer);

// Check if parser is valid
if (parser.valid) { ... }

Document

// Check if parsing succeeded
if (doc.valid) {
    auto root = doc.root;
}

// Get error information
if (!doc.valid) {
    JsonError err = doc.error;           // Error code
    string msg = doc.errorMessage;       // Human-readable message
}

// Access root value
auto root = doc.root;

// Convenience: direct indexing into root
auto name = doc["name"];        // If root is object
auto first = doc[0];            // If root is array

Value - Type Checking

// Get JSON type
JsonType t = value.type();  // null_, bool_, int64, uint64, double_, string_, array, object

// Type checking methods
if (value.isNull()) { ... }
if (value.isBool()) { ... }
if (value.isInt()) { ... }
if (value.isUint()) { ... }
if (value.isDouble()) { ... }
if (value.isNumber()) { ... }  // int, uint, or double
if (value.isString()) { ... }
if (value.isArray()) { ... }
if (value.isObject()) { ... }

Value - Extraction

// Throwing extraction (throws JsonException on error)
bool   b = value.getBool();
long   n = value.getInt();
ulong  u = value.getUint();
double d = value.getDouble();

// Throwing extraction (throws JsonException on error)
const(char)[] s = value.getString();  // Zero-copy, throws on error

// Safe extraction with Result<T> (no exceptions)
if (auto result = value.tryBool()) {
    bool b = result.value;
} else {
    JsonError err = result.error;
}

if (auto result = value.tryInt()) {
    long n = result.value;
}

if (auto result = value.tryString()) {
    const(char)[] s = result.value;  // Zero-copy
}

// Get value or default
long n = value.tryInt().valueOr(0);
const(char)[] s = value.tryString().valueOr("");

Value - Object Access

// Get field by key (throws if not found)
auto field = root["key"];

// Check if field exists
if (root.hasKey("optional")) {
    auto opt = root["optional"];
}

// Get number of fields
size_t count = root.objectSize();

// Iteration
foreach (const(char)[] key, val; root) {
    writeln(key, ": ", val);
}

Value - Array Access

// Get element by index (throws if out of bounds)
auto item = root["items"][0];

// Get array length
size_t len = root["items"].length;
// or
size_t len = root["items"].$;  // opDollar alias

// Iteration
foreach (item; root["items"]) {
    writeln(item);
}

// Iteration with index
foreach (size_t i, item; root["items"]) {
    writeln(i, ": ", item);
}

Value - Utilities

// Convert to string (for debugging)
string str = value.toString();  // Returns JSON-like representation

std.json-Compatible API (fastjsond.std)

import fastjsond.std;

// Parse JSON (throws JSONException on error)
auto json = parseJSON(jsonString);
auto json = parseJSON(cast(const(char)[]) json);

// Type checking
JSONValue.Type t = json.type();  // null_, string_, integer, uinteger, float_, array, object, true_, false_
if (json.isNull()) { ... }

// Value accessors
string name = json["name"].str;
long count = json["count"].integer;
ulong ucount = json["count"].uinteger;
double price = json["price"].floating;
bool flag = json["flag"].boolean;

// Array/Object access
JSONValue[] arr = json["items"].array;
JSONValue[string] obj = json["config"].object;

// Operators
auto field = json["key"];        // Object field
auto item = json[0];             // Array element
if (auto ptr = "optional" in json) { ... }  // Check existence
size_t len = json.length;        // Array/Object length

// Iteration
foreach (ref item; json["items"]) { ... }
foreach (size_t i, ref item; json["items"]) { ... }
foreach (string key, ref val; json) { ... }

// Serialization
string jsonStr = toJSON(json);
string pretty = toJSON(json, true);  // Pretty print
string custom = toPrettyJSON(json, "  ");  // Custom indent

Types

import fastjsond;

// JSON type enum
enum JsonType : ubyte {
    null_, bool_, int64, uint64, double_, string_, array, object
}

// Error codes
enum JsonError : ubyte {
    none, capacity, memalloc, tapeError, depthError,
    stringError, numberError, utf8Error, incorrectType,
    indexOutOfBounds, noSuchField, // ... and more
}

// Exception type
class JsonException : Exception {
    JsonError error;
}

// Result type for safe extraction
struct Result(T) {
    bool ok();
    bool hasError();
    JsonError error();
    T value();              // Throws if error
    T valueOr(T default);   // Returns default if error
}

Module Functions

import fastjsond;

// Validate JSON without full parse (faster for validation-only)
JsonError err = validate(jsonString);
if (err == JsonError.none) {
    // Valid JSON
}

// Get required padding for parsePadded()
size_t padding = requiredPadding();  // Returns 64 (SIMDJSON_PADDING)

// Get active SIMD implementation
string impl = activeImplementation();  // "haswell", "westmere", "arm64", "fallback", etc.

Error Handling

Native API

import fastjsond;

// Option 1: Check document validity
auto doc = parser.parse(json);
if (!doc.valid) {
    writeln("Error: ", doc.errorMessage);
    writeln("Code: ", doc.error);
    return;
}

// Option 2: Use Result types (no exceptions)
auto result = doc.root["key"].tryString;
if (!result.ok) {
    writeln("Error: ", result.error);
} else {
    auto value = result.value;
}

// Option 3: Let it throw
try {
    auto value = doc.root["missing"]["key"].getString;
} catch (JsonException e) {
    writeln("Error: ", e.error, " - ", e.msg);
}

// More Result<T> examples:

// Chaining multiple optional fields
if (auto name = doc.root["user"].tryString) {
    if (auto email = doc.root["email"].tryString) {
        writeln("User: ", name.value, " <", email.value, ">");
    }
}

// Using valueOr() for defaults
long count = doc.root["count"].tryInt.valueOr(0);
const(char)[] title = doc.root["title"].tryString.valueOr("Untitled");

// Checking specific error types
if (auto result = doc.root["data"].tryInt) {
    processData(result.value);
} else if (result.error == JsonError.incorrectType) {
    writeln("Expected integer, got: ", doc.root["data"].type);
} else if (result.error == JsonError.noSuchField) {
    writeln("Field 'data' not found");
}

// Pattern: Process array with error handling
foreach (item; doc.root["items"]) {
    if (auto id = item["id"].tryInt) {
        if (auto name = item["name"].tryString) {
            processItem(id.value, name.value);
        } else {
            writeln("Item ", id.value, " missing name");
        }
    }
}

// getString() throws JsonException on error (coherent with other get*() methods)
try {
    const(char)[] str = value.getString();
    // Use str...
} catch (JsonException e) {
    // Handle error - use tryString() for explicit error handling in @nogc contexts
}

std API

import fastjsond.std;

try {
    auto json = parseJSON(input);
    auto name = json["name"].str;
} catch (JSONException e) {
    writeln("Error: ", e.msg);
}

Thread Safety

Native API:

• Parser is NOT thread-safe - use one per thread (thread-local recommended)
• Document is NOT thread-safe - owned by creating thread
• Value is NOT thread-safe - borrows from Document

std API:

• JSONValue is thread-safe after creation (immutable data)

Recommended Pattern:

// Thread-local parser for maximum efficiency
static Parser tlsParser;

void processRequest(const(char)[] json) {
    if (tlsParser is null) {
        tlsParser = Parser.create();
    }
    auto doc = tlsParser.parse(json);
    // Process doc (all access in same thread)
}

Performance

Benchmarked on MacBook Pro M4 (10-core, 16GB RAM):

Payload	std.json	fastjsond	Speedup
1 MB	8.45 ms	0.59 ms	14x
10 MB	82 ms	6.8 ms	12x
100 MB	818 ms	126 ms	6.5x

Error detection:

Error Type	std.json	fastjsond	Speedup
Invalid syntax	0.75 ms	0.008 ms	93x
Invalid escapes	0.86 ms	0.004 ms	210x

For comprehensive benchmark results, see benchmarks/README.md.

String Lifetime

Native API strings are borrowed references into the original JSON buffer. They are valid only while the Document exists.

Important: getString() throws JsonException on error (coherent with other get*() methods). Use tryString() for @nogc contexts or explicit error handling.

// Incorrect: reference invalid after doc goes out of scope
const(char)[] getName() {
    auto doc = parser.parse(`{"name": "test"}`);
    return doc.root["name"].getString;  // ⚠️ Dangling reference
}

// Correct: copy the string
string getName() {
    auto doc = parser.parse(`{"name": "test"}`);
    return doc.root["name"].getString.idup;  // ✓ Safe: copied to GC heap
}

// Better: use tryString() for error handling
string getName() {
    auto doc = parser.parse(`{"name": "test"}`);
    if (auto result = doc.root["name"].tryString) {
        return result.value.idup;  // ✓ Safe with error handling
    }
    return "";
}

// Alternative: use std API (auto-copies, thread-safe)
import fastjsond.std;
string getName() {
    auto json = parseJSON(`{"name": "test"}`);
    return json["name"].str;  // ✓ Safe: all strings copied
}

Migration Guide

From std.json

fastjsond.std is a drop-in replacement for std.json. Simply change your import:

// Before
import std.json;
auto json = parseJSON(`{"name": "test"}`);
string name = json["name"].str;

// After
import fastjsond.std;
auto json = parseJSON(`{"name": "test"}`);
string name = json["name"].str;  // Identical API!

Benefits:

• ✅ Same API - no code changes needed
• ✅ 1.5-2.5x faster parsing
• ✅ Thread-safe after creation
• ✅ Better error messages

Differences:

• Exception type is JSONException (same as std.json)
• All strings are copied (not zero-copy like native API)
• No streaming support (same as std.json)

From fastjsond v1.0.0/v1.0.1

Breaking Changes in v1.0.2:

1. `getString()` behavior changed:

• Before: Returned null on error (due to @nogc constraint)
• After: Throws JsonException on error (consistent with other get*() methods)

Migration:

   // Before (v1.0.0/v1.0.1)
   const(char)[] str = value.getString();
   if (str is null) {
       // Handle error
   }
   
   // After (v1.0.2)
   try {
       const(char)[] str = value.getString();
       // Use str...
   } catch (JsonException e) {
       // Handle error
   }
   
   // Or use tryString() for @nogc contexts (unchanged)
   if (auto result = value.tryString) {
       const(char)[] str = result.value;
   }

1. No other breaking changes - all other APIs remain compatible.

From Native API to std API

If you're using the native API and want to switch to the std API for easier migration:

// Native API (zero-copy, maximum performance)
import fastjsond;
auto parser = Parser.create();
auto doc = parser.parse(json);
const(char)[] name = doc.root["name"].getString;  // Zero-copy!

// std API (copies data, easier to use)
import fastjsond.std;
auto json = parseJSON(json);
string name = json["name"].str;  // Copied to GC heap

When to use each:

• Native API: Performance-critical code, zero-copy needed, @nogc contexts
• std API: Easier migration, thread-safety needed, simpler lifetime management

Building

Requirements

• D Compiler: LDC 1.35+ (recommended) or DMD 2.105+
• C++ Compiler: clang++ or g++ (C++17)

Make Targets

Target	Description
make lib	Build libfastjsond.a
make test	Run all tests
make bench	Run benchmarks
make clean	Clean artifacts

Documentation

• Technical Specifications — Complete API reference with examples
• Benchmarks — Comprehensive performance analysis
• simdjson — Underlying parser library

Contributing

Contributions are welcome. Please ensure:

1. Tests pass (make test)
2. Benchmarks do not regress
3. Code follows D style guidelines

Future Testing Improvements

We welcome contributions for:

• Property-based testing: Using libraries like dproperty to generate random valid JSON and verify parsing correctness
• Fuzz testing: Testing with malformed inputs to improve robustness
• Performance regression tests: Automated benchmarks to catch performance regressions

License

MIT License — see LICENSE for details.