requests 0.1.6

dlang-requests

HTTP requests library with goals:

• small memory footprint
• performance
• simple, high level API

API docs: Wiki

In simplest scenario you just need to fetch and process document from remote site. In this case you can call getContent

import std.stdio;
import std.algorithm;
import requests;

pragma(lib, "ssl");
pragma(lib, "crypto");

void main() {
    writeln(
        getContent("https://httpbin.org/")
        .splitter('\n')
        .filter!("a.length>0")
        .count
    );
}

getContent returns Buffer, filled with data. Buffer looks like Appender!ubyte (it have method data()), but also support many Range operations.

If you have to send some parameters with request, then you can use next simple inrerface:

    string name = "user";
    r = getContent("https://httpbin.org/get", "name", name, "age", 42);

The simple way to upload content is postContent() API call.

Posting to forms using x-form/x-www-form-urlencoded:

postContent("http://httpbin.org/post", queryParams("first", "a", "second", 2));

Using multipart/form-data (also upload files to forms, see examples in unitests for HTTPRequest):

MultipartForm form;
form.add(formData("greeting", cast(ubyte[])"hello"));
postContent("http://httpbin.org/post", form);

If you do not need upload to form, then you can post from range(see more examples in section HTTPRequest).

postContent("http://httpbin.org/post", `{"a":"b", "c":1}`, "application/json");

When you need access to response code, you have to use Request and Response structures:

Request rq = Request();
Response rs = rq.get("https://httpbin.org/");
assert(rs.code==200);

For anything other than default, you can configure Request structure for keep-alive, redirects, headers, or for different io-buffer and maximum sizes of response headers and body.

For example to authorize with Basic authorization use next code:

    rq = Request();
    rq.authenticator = new BasicAuthentication("user", "passwd");
    rs = rq.get("http://httpbin.org/basic-auth/user/passwd");

You can use requests in parallel tasks (but you can't share single Request structure between threads):

import std.stdio;
import std.parallelism;
import std.algorithm;
import std.string;
import core.atomic;

immutable auto urls = [
    "http://httpbin.org/stream/10",
    "https://httpbin.org/stream/20",
    "http://httpbin.org/stream/30",
    "https://httpbin.org/stream/40",
    "http://httpbin.org/stream/50",
    "https://httpbin.org/stream/60",
    "http://httpbin.org/stream/70",
];

void main() {    
    defaultPoolThreads(5);

    shared short lines;
    
    foreach(url; parallel(urls)) {
        atomicOp!"+="(lines, getContent(url).splitter("\n").count);
    }
    assert(lines == 287);
}

Here is short descrition of some Request options:

Usage example:

 auto rq = Request();
 auto rs = rq.get("http://httpbin.org/");
 writeln(rs.responseBody.data!string);

 rq.keepAlive = true;
 rs = rq.post("http://httpbin.org/post", `{"a":"b", "c":[1,2,3]}`, "application/json");
 assert(rs.code==200);

 auto f = File("tests/test.txt", "rb");
 rs = rq.post("http://httpbin.org/post", f.byChunk(3), "application/octet-stream");
 assert(rs.code==200);
 auto data = parseJSON(rs.responseBody.data).object["data"].str;
 assert(data=="abcdefgh\n12345678\n");
 f.close();

For Windows users

If building or testing under windows fails with message about ssl, then inslall libssl32.lib and ssleay32.lib into dmd or ldc2 library path. You can find this libs on https://github.com/ikod/dlang-requests/tree/master/lib/win-i386. These libararies were downloaded from https://slproweb.com/products/Win32OpenSSL.html (full version) and converted using "implib /system" http://ftp.digitalmars.com/bup.zip. If you know better way to link windows libraries, please, let me know.