requests 0.1.5

dlang-requests

HTTP requests library with goals:

• small memory footprint
• performance
• simple, high level API

API docs: Wiki

In simplest scenario you just need to fetch and process document from remote site. In this case you can call getContent

import std.stdio;
import std.algorithm;
import requests;

pragma(lib, "ssl");
pragma(lib, "crypto");

void main() {
    writeln(
        getContent("https://httpbin.org/")
        .splitter('\n')
        .filter!("a.length>0")
        .count
    );
}

getContent returns Buffer, filled with data. Buffer looks like Appender!ubyte (it have method data()), but also support many Range operations.

When you need access to response code, you have to use Request and Response structures:

Request rq = Request();
Response rs = rq.get("https://httpbin.org/");
assert(rs.code==200);

For anything other than default, you can configure Request structure for keep-alive, redirects, headers, or for different io-buffer and maximum sizes of response headers and body.

For example to authorize with Basic authorization use next code:

    rq = Request();
    rq.authenticator = new BasicAuthentication("user", "passwd");
    rs = rq.get("http://httpbin.org/basic-auth/user/passwd");

You can use requests in parallel tasks (but you can't share single Request structure between threads):

import std.stdio;
import std.parallelism;
import std.algorithm;
import std.string;
import core.atomic;

immutable auto urls = [
    "http://httpbin.org/stream/10",
    "https://httpbin.org/stream/20",
    "http://httpbin.org/stream/30",
    "https://httpbin.org/stream/40",
    "http://httpbin.org/stream/50",
    "https://httpbin.org/stream/60",
    "http://httpbin.org/stream/70",
];

void main() {    
    defaultPoolThreads(5);

    shared short lines;
    
    foreach(url; parallel(urls)) {
        atomicOp!"+="(lines, getContent(url).splitter("\n").count);
    }
    assert(lines == 287);
}

Here is short descrition of some Request options:

Usage example:

 auto rq = Request();
 auto rs = rq.get("http://httpbin.org/");
 writeln(rs.responseBody.data!string);

 rq.keepAlive = true;
 rs = rq.post("http://httpbin.org/post", `{"a":"b", "c":[1,2,3]}`, "application/json");
 assert(rs.code==200);

 auto f = File("tests/test.txt", "rb");
 rs = rq.post("http://httpbin.org/post", f.byChunk(3), "application/octet-stream");
 assert(rs.code==200);
 auto data = parseJSON(rs.responseBody.data).object["data"].str;
 assert(data=="abcdefgh\n12345678\n");
 f.close();

For Windows users

Requests distributed with binary ssl and crypto libraries. These libararies were downloaded from https://slproweb.com/products/Win32OpenSSL.html (full version) and converted using "implib /system" http://ftp.digitalmars.com/bup.zip. If you know better way to link windows libraries, please, let me know.