sttp: the Scala HTTP client you always wanted!

Welcome!

sttp client is an open-source library which provides a clean, programmer-friendly API to describe HTTP requests and how to handle responses. Requests are sent using one of the backends, which wrap other Scala or Java HTTP client implementations. The backends can integrate with a variety of Scala stacks, providing both synchronous and asynchronous, procedural and functional interfaces.

Backend implementations include ones based on akka-http, async-http-client, http4s, OkHttp, and HTTP clients which ship with Java. They integrate with Akka, Monix, fs2, cats-effect, scalaz and ZIO.

Here’s a very quick example of sttp client in action:

import sttp.client._

val query = "http language:scala"
val sort: Option[String] = None

// the `query` parameter is automatically url-encoded
// `sort` is removed, as the value is not defined
val request = basicRequest.get(
  uri"https://api.github.com/search/repositories?q=$query&sort=$sort")

implicit val backend = HttpURLConnectionBackend()
val response = request.send()

// response.header(...): Option[String]
println(response.header("Content-Length"))

// response.body: by default read into an Either[String, String] 
// to indicate failure or success 
println(response.body)              

// alternatively, if you prefer to pass the backend explicitly, instead
// of using implicits, you can also call:
val sameResponse = backend.send(request)

For more examples, see the usage examples section. To start using sttp client in your project, see the quickstart. Or, browse the documentation to find the topics that interest you the most!

Other sttp projects

sttp is a family of Scala HTTP-related projects, and currently includes:

  • sttp client: this project
  • sttp tapir: Typed API descRiptions
  • sttp model: simple HTTP model classes (used by client & tapir)

Sponsors

Development and maintenance of sttp client is sponsored by SoftwareMill, a software development and consulting company. We help clients scale their business through software. Our areas of expertise include backends, distributed systems, blockchain, machine learning and data analytics.

SoftwareMill

Try sttp client in your browser!

Table of contents

Quickstart

The main sttp client API comes in a single jar, with a single transitive dependency on the sttp model. This also includes a default, synchronous backend, which is based on Java’s HttpURLConnection.

To integrate with other parts of your application, you’ll often need to use an alternate backend (but what’s important is that the API remains the same!). See the section on backends for a short guide on which backend to choose, and a list of all implementations.

Using sbt

The basic dependency which provides the API and the default synchronous backend is:

"com.softwaremill.sttp.client" %% "core" % "2.3.0"

sttp client is available for Scala 2.11, 2.12 and 2.13, and requires Java 8, as well as for Dotty (Scala 3) 0.23.

sttp client is also available for Scala.js 1.0. Note that not all modules are compatible and there are no backends that can be used on both. The last version compatible with Scala.js 0.6 was 2.2.1.

Using Ammonite

If you are an Ammonite user, you can quickly start experimenting with sttp by copy-pasting the following:

import $ivy.`com.softwaremill.sttp.client::core:2.3.0`
import sttp.client.quick._
quickRequest.get(uri"http://httpbin.org/ip").send()

Importing the quick object has the same effect as importing sttp.client._, plus defining an implicit synchronous backend (implict val backend = HttpURLConnectionBackend()), so that sttp can be used right away.

If the default HttpURLConnectionBackend for some reason is insufficient, you can also use one based on OkHttp:

import $ivy.`com.softwaremill.sttp.client::okhttp-backend:2.3.0`
import sttp.client.okhttp.quick._
quickRequest.get(uri"http://httpbin.org/ip").send()

Imports

Working with sttp is most convenient if you import the sttp.client package entirely:

import sttp.client._

This brings into scope the starting point for defining requests and some helper methods. All examples in this guide assume that this import is in place.

And that’s all you need to start using sttp client! To create and send your first request, import the above, type basicRequest. and see where your IDE’s auto-complete gets you! Here’s a simple request, using the synchronous backend:

import sttp.client._

implicit val backend = HttpURLConnectionBackend()
val response = basicRequest
  .body("Hello, world!")  
  .post(uri"https://httpbin.org/post?hello=world").send()

println(response.body)            

Next, read on about the how sttp client works or see some examples.

How sttp client works

Describe the request

This first step when using sttp client is describing the request that you’d like to send.

A request is represented as an immutable data structure of type RequestT (as in Request Template). The basic request is provided as the basicRequest value, in the sttp.client package. It can be refined using one of the available methods, such as .header, .body, .get(Uri), .responseAs, etc.

A RequestT value contains both information on what to include in the request, but also how to handle the response body.

To start describing a request, import the sttp client package and customise basicRequest:

import sttp.client._
val myRequest: Request[_, _] = ??? // basicRequest.(...)

An alternative to importing the sttp.client._ package, is to extend the sttp.client.SttpApi trait. That way, multiple integrations can be grouped in one object, thus reducing the number of necessary imports.

Send the request

Once the request is described as a value, it can be sent. To send a request, you’ll need to have an implicit value of type SttpBackend in scope.

The backend is where most of the work happens: the request is translated to a backend-specific form; a connection is opened, data sent and received; finally, the backend-specific response is translated to sttp’s Response, as described in the request.

A backend can be synchronous, that is, sending a request can be a blocking operation. When invoking myRequest.send(), you’ll get a value of type Response[T]. Backends can also be asynchronous, and evaluate the send operation eagarly or lazily. For example, when using the Akka backend, myRequest.send() will return a Future[Response[T]]: an eagerly-evaluated, asynchronous result. When using a Monix backend, you’ll get back a Task[Response[T]]: a lazily-evaluated, but also non-blocking and asynchronous result.

Backends manage the connection pool, thread pools for handling responses, depending on the implementation provide various configuration options, and optionally support streaming and websockets. They typically need to be created upon application startup, and closed when the application terminates.

For example, the following sends a synchronous request, using the default JVM backend:

import sttp.client._
val myRequest: Request[String, Nothing] = ???
implicit val backend = HttpURLConnectionBackend()
val response = myRequest.send()

Alternatively, if you prefer to pass the backend explicitly, instead of using implicits, you can also send the request the following way:

import sttp.client._
val myRequest: Request[String, Nothing] = ???
val backend = HttpURLConnectionBackend()
val response = backend.send(myRequest)

Next steps

Read more about:

Goals of the project

  • provide a simple, discoverable, no-surprises, reasonably type-safe API for making HTTP requests and reading responses
  • separate definition of a request from request execution
  • provide immutable, easily modifiable data structures for requests and responses
  • support multiple execution backends, both synchronous and asynchronous
  • provide support for backend-specific request/response streaming
  • minimum dependencies

See also the blog posts:

Non-goals of the project

  • implement a full HTTP client. Instead, sttp client wraps existing HTTP clients, providing a consistent, programmer-friendly API. All network-related concerns such as sending the requests, connection pooling, receiving responses are delegated to the chosen backend
  • provide ultimate flexibility in defining the request. While it’s possible to define most valid HTTP requests, e.g. some of the less common body chunking approaches aren’t available

How is sttp different from other libraries?

  • immutable request builder which doesn’t impose any order in which request parameters need to be specified. Such an approach allows defining partial requests with common cookies/headers/options, which can later be specialized using a specific URI and HTTP method.
  • support for multiple backends, both synchronous and asynchronous, with backend-specific streaming support
  • URI interpolator with context-aware escaping, optional parameters support and parameter collections

Community

If you have a question, or hit a problem, feel free to ask on our gitter channel!

Or, if you encounter a bug, something is unclear in the code or documentation, don’t hesitate and open an issue on GitHub.

We are also always looking for contributions and new ideas, so if you’d like to get into the project, check out the open issues, or post your own suggestions!

Usage examples

All of the examples are available in the sources in runnable form.

POST a form using the synchronous backend

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "core" % "2.3.0")

Example code:

import sttp.client._

val signup = Some("yes")

val request = basicRequest
  // send the body as form data (x-www-form-urlencoded)
  .body(Map("name" -> "John", "surname" -> "doe"))
  // use an optional parameter in the URI
  .post(uri"https://httpbin.org/post?signup=$signup")

implicit val backend = HttpURLConnectionBackend()
val response = request.send()

println(response.body)
println(response.headers)

GET and parse JSON using the akka-http backend and json4s

Required dependencies:

libraryDependencies ++= List(
  "com.softwaremill.sttp.client" %% "akka-http-backend" % "2.3.0",
  "com.softwaremill.sttp.client" %% "json4s" % "2.3.0",
  "org.json4s" %% "json4s-native" % "3.6.0"
)

Example code:

import scala.concurrent.Future
import sttp.client._
import sttp.client.akkahttp._
import sttp.client.json4s._

import scala.concurrent.ExecutionContext.Implicits.global

case class HttpBinResponse(origin: String, headers: Map[String, String])

implicit val serialization = org.json4s.native.Serialization
implicit val formats = org.json4s.DefaultFormats
val request = basicRequest
  .get(uri"https://httpbin.org/get")
  .response(asJson[HttpBinResponse])

implicit val backend = AkkaHttpBackend()
val response: Future[Response[Either[ResponseError[Exception], HttpBinResponse]]] =
  request.send()

for {
  r <- response
} {
  println(s"Got response code: ${r.code}")
  println(r.body)
  backend.close()
}

GET and parse JSON using the ZIO async-http-client backend and circe

Required dependencies:

libraryDependencies ++= List(
  "com.softwaremill.sttp.client" %% "async-http-client-backend-zio" % "2.3.0",
  "com.softwaremill.sttp.client" %% "circe" % "2.3.0",
  "io.circe" %% "circe-generic" % "0.12.1"
)

Example code:

import sttp.client._
import sttp.client.circe._
import sttp.client.asynchttpclient.zio._
import io.circe.generic.auto._
import zio._
import zio.console.Console

object GetAndParseJsonZioCirce extends zio.App {

  override def run(args: List[String]): ZIO[ZEnv, Nothing, ExitCode] = {

    case class HttpBinResponse(origin: String, headers: Map[String, String])

    val request = basicRequest
      .get(uri"https://httpbin.org/get")
      .response(asJson[HttpBinResponse])

    // create a description of a program, which requires two dependencies in the environment:
    // the SttpClient, and the Console
    val sendAndPrint: ZIO[Console with SttpClient, Throwable, Unit] = for {
      response <- SttpClient.send(request)
      _ <- console.putStrLn(s"Got response code: ${response.code}")
      _ <- console.putStrLn(response.body.toString)
    } yield ()

    // provide an implementation for the SttpClient dependency; other dependencies are
    // provided by Zio
    sendAndPrint.provideCustomLayer(AsyncHttpClientZioBackend.layer()).fold(_ => ExitCode.failure, _ => ExitCode.success)
  }
}

POST and serialize JSON using the Monix async-http-client backend and circe

Required dependencies:

libraryDependencies ++= List(
  "com.softwaremill.sttp.client" %% "async-http-client-backend-monix" % "2.3.0",
  "com.softwaremill.sttp.client" %% "circe" % "2.3.0",
  "io.circe" %% "circe-generic" % "0.12.1"
)

Example code:

import sttp.client._
import sttp.client.circe._
import sttp.client.asynchttpclient.monix._
import io.circe.generic.auto._
import monix.eval.Task

case class Info(x: Int, y: String)

val postTask = AsyncHttpClientMonixBackend().flatMap { implicit backend =>
  val r = basicRequest
    .body(Info(91, "abc"))
    .post(uri"https://httpbin.org/post")

  r.send()
    .flatMap { response =>
      Task(println(s"""Got ${response.code} response, body:\n${response.body}"""))
    }
    .guarantee(backend.close())
}

import monix.execution.Scheduler.Implicits.global
postTask.runSyncUnsafe()

Test an endpoint which requires multiple query parameters

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "core" % "2.3.0")

Example code:

import sttp.client._
import sttp.client.testing._

implicit val backend = SttpBackendStub.synchronous
  .whenRequestMatches(_.uri.paramsMap.contains("filter"))
  .thenRespond("Filtered")
  .whenRequestMatches(_.uri.path.contains("secret"))
  .thenRespond("42")

val parameters1 = Map("filter" -> "name=mary", "sort" -> "asc")
println(
  basicRequest
    .get(uri"http://example.org?search=true&$parameters1")
    .send()
    .body)

val parameters2 = Map("sort" -> "desc")
println(
  basicRequest
    .get(uri"http://example.org/secret/read?$parameters2")
    .send()
    .body)

Open a websocket using the high-level websocket interface and ZIO

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "async-http-client-backend-zio" % "2.3.0")

Example code:

import sttp.client._
import sttp.client.asynchttpclient.zio._
import sttp.client.ws.WebSocket
import sttp.model.ws.WebSocketFrame
import zio.{App => ZApp, _}
import zio.console.Console

object WebsocketZio extends ZApp {
  def useWebsocket(ws: WebSocket[Task]): ZIO[Console, Throwable, Unit] = {
    def send(i: Int) = ws.send(WebSocketFrame.text(s"Hello $i!"))
    val receive = ws.receiveText().flatMap(t => console.putStrLn(s"RECEIVED: $t"))
    send(1) *> send(2) *> receive *> receive *> ws.close
  }

  // create a description of a program, which requires two dependencies in the environment:
  // the SttpClient, and the Console
  val sendAndPrint: ZIO[Console with SttpClient, Throwable, Unit] = for {
    response <- SttpClient.openWebsocket(basicRequest.get(uri"wss://echo.websocket.org"))
    _ <- useWebsocket(response.result)
  } yield ()

  override def run(args: List[String]): ZIO[ZEnv, Nothing, ExitCode] = {
    // provide an implementation for the SttpClient dependency; other dependencies are
    // provided by Zio
    sendAndPrint.provideCustomLayer(AsyncHttpClientZioBackend.layer()).fold(_ => ExitCode.failure, _ => ExitCode.success)
  }
}

Open a websocket using the high-level websocket interface and Monix

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "async-http-client-backend-monix" % "2.3.0")

Example code:

import monix.eval.Task
import sttp.client._
import sttp.client.ws.{WebSocket, WebSocketResponse}
import sttp.model.ws.WebSocketFrame
import sttp.client.asynchttpclient.monix.{AsyncHttpClientMonixBackend, MonixWebSocketHandler}

object WebsocketMonix extends App {
  import monix.execution.Scheduler.Implicits.global

  def useWebsocket(ws: WebSocket[Task]): Task[Unit] = {
    def send(i: Int) = ws.send(WebSocketFrame.text(s"Hello $i!"))
    val receive = ws.receiveText().flatMap(t => Task(println(s"RECEIVED: $t")))
    send(1) *> send(2) *> receive *> receive *> ws.close
  }

  val websocketTask: Task[Unit] = AsyncHttpClientMonixBackend().flatMap { implicit backend =>
    val response: Task[WebSocketResponse[WebSocket[Task]]] = basicRequest
      .get(uri"wss://echo.websocket.org")
      .openWebsocketF(MonixWebSocketHandler())

    response
      .flatMap(r => useWebsocket(r.result))
      .guarantee(backend.close())
  }

  websocketTask.runSyncUnsafe()
}

Stream request and response bodies using fs2

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "async-http-client-backend-fs2" % "2.3.0")

Example code:

import sttp.client._
import sttp.client.asynchttpclient.fs2.AsyncHttpClientFs2Backend

import cats.effect.{ContextShift, IO}
import cats.instances.string._
import fs2.{Stream, text}

implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)

def streamRequestBody(implicit backend: SttpBackend[IO, Stream[IO, Byte], NothingT]): IO[Unit] = {
  val stream: Stream[IO, Byte] = Stream.emits("Hello, world".getBytes)

  basicRequest
    .streamBody(stream)
    .post(uri"https://httpbin.org/post")
    .send()
    .map { response => println(s"RECEIVED:\n${response.body}") }
}

def streamResponseBody(implicit backend: SttpBackend[IO, Stream[IO, Byte], NothingT]): IO[Unit] = {
  basicRequest
    .body("I want a stream!")
    .post(uri"https://httpbin.org/post")
    .response(asStreamAlways[Stream[IO, Byte]])
    .send()
    .flatMap { response =>
      response.body
        .chunks
        .through(text.utf8DecodeC)
        .compile
        .foldMonoid
    }
    .map { body => println(s"RECEIVED:\n$body") }
}

val effect = AsyncHttpClientFs2Backend[IO]().flatMap { implicit backend =>
  streamRequestBody.flatMap(_ => streamResponseBody).guarantee(backend.close())
}

effect.unsafeRunSync()

Retry a request using ZIO

Required dependencies:

libraryDependencies ++= List("com.softwaremill.sttp.client" %% "async-http-client-backend-zio" % "2.3.0")

Example code:

import sttp.client._
import sttp.client.asynchttpclient.zio.AsyncHttpClientZioBackend

import zio.{ZIO, Schedule}
import zio.clock.Clock
import zio.duration._

AsyncHttpClientZioBackend()
  .flatMap { implicit backend =>
    val localhostRequest = basicRequest
      .get(uri"http://localhost/test")
      .response(asStringAlways)

    val sendWithRetries: ZIO[Clock, Throwable, Response[String]] = localhostRequest
      .send()
      .either
      .repeat(
        Schedule.spaced(1.second) *>
          Schedule.recurs(10) *>
          Schedule.recurWhile(result => RetryWhen.Default(localhostRequest, result))
      )
      .absolve

    sendWithRetries.ensuring(backend.close().catchAll(_ => ZIO.unit))
  }

Model classes

sttp model is a stand-alone project which provides a basic HTTP model, along with constants for common HTTP header names, media types, and status codes.

The basic model classes are: Header, Cookie, CookieWithMeta, MediaType, Method, StatusCode and Uri. The .toString methods of these classes returns a representation as in a HTTP request/response. See the ScalaDoc for more information.

Companion objects provide methods to construct model class instances, following these rules:

  • .parse(serialized: String): Either[String, ModelClass]: returns an error message or an instance of the model class
  • .unsafeApply(values): creates an instance of the model class; validates the input values and in case of an error, throws an exception. An error could be e.g. that the input values contain characters outside of the allowed range
  • .safeApply(...): Either[String, ModelClass]: same as above, but doesn’t throw exceptions. Instead, returns an error message or the model class instance
  • .apply(...): ModelClass: creates the model type, without validation, and without throwing exceptions

Moreover, companion objects provide constants and/or constructor methods for well-know model class instances. For example, there’s StatusCode.Ok, Method.POST, MediaType.ImageGif and Header.contentType(MediaType).

These constants are also available as traits: StatusCodes, MediaTypes and HeaderNames.

The model also contains aggregate/helper classes such as Headers and MultiQueryParams

Example with objects:

import sttp.client._
import sttp.model._

object Example {
  val request = basicRequest.header(Header.contentType(MediaType.ApplicationJson))
    .get(uri"https://httpbin.org")

  implicit val backend = HttpURLConnectionBackend()
  val response = request.send()
  if (response.code == StatusCode.Ok) println("Ok!")
}

Example with traits:

import sttp.client._
import sttp.model._

object Example extends HeaderNames with MediaTypes with StatusCodes {
  val request = basicRequest.header(ContentType, ApplicationJson.toString)
    .get(uri"https://httpbin.org")

  implicit val backend = HttpURLConnectionBackend()
  val response = request.send()
  if (response.code == Ok) println("Ok!")
}

For more information see

URIs

A request can only be sent if the request method & URI are defined. To represent URIs, sttp comes with a Uri case class, which captures all of the parts of an address.

To specify the request method and URI, use one of the methods on the request definition corresponding to the name of the desired HTTP method: .post, .get, .put etc. All of them accept a single parameter, the URI to which the request should be sent (these methods only modify the request definition; they don’t send the requests).

The Uri class is immutable, and can be constructed by hand, but in many cases the URI interpolator will be easier to use.

URI interpolator

Using the URI interpolator it’s possible to conveniently create Uri instances, for example:

import sttp.client._
import sttp.model._

val user = "Mary Smith"
val filter = "programming languages"

val endpoint: Uri = uri"http://example.com/$user/skills?filter=$filter"

assert(endpoint.toString ==
  "http://example.com/Mary%20Smith/skills?filter=programming+languages")

Note the uri prefix before the string and the standard Scala string-embedding syntax ($user, $filter).

Any values embedded in the URI will be URL-encoded, taking into account the context (e.g., the whitespace in user will be %-encoded as %20D, while the whitespace in filter will be query-encoded as +). On the other hand, parts of the URI given as literal strings (not embedded values), are assumed to be URL-encoded and thus will be decoded when creating a Uri instance.

All components of the URI can be embedded from values: scheme, username/password, host, port, path, query and fragment. The embedded values won’t be further parsed, with the exception of the : in the host part, which is commonly used to pass in both the host and port:

import sttp.client._

// the embedded / is escaped
println(uri"http://example.org/${"a/b"}")
// http://example.org/a%2Fb

// the embedded / is not escaped
println(uri"http://example.org/${"a"}/${"b"}")
// http://example.org/a/b

// the embedded : is not escaped
println(uri"http://${"example.org:8080"}")
// http://example.org:8080

Both the Uri class and the interpolator can be used stand-alone, without using the rest of sttp. Conversions are available both from and to java.net.URI; Uri.toString returns the URI as a String.

Optional values

The URI interpolator supports optional values for hosts (subdomains), query parameters and the fragment. If the value is None, the appropriate URI component will be removed. For example:

val v1 = None
val v2 = Some("v2")
println(uri"http://example.com?p1=$v1&p2=v2")
// http://example.com?p2=v2

println(uri"http://$v1.$v2.example.com")
// http://v2.example.com

println(uri"http://example.com#$v1")
// http://example.com

Maps and sequences

Maps, sequences of tuples and sequences of values can be embedded in the query part. They will be expanded into query parameters. Maps and sequences of tuples can also contain optional values, for which mappings will be removed if None.

For example:

val ps = Map("p1" -> "v1", "p2" -> "v2")
println(uri"http://example.com?$ps&p3=p4")
// http://example.com?p1=v1&p2=v2&p3=p4

Sequences in the host part will be expanded to a subdomain sequence, and sequences in the path will be expanded to path components:

val params = List("a", "b", "c")
println(uri"http://example.com/$params")
// http://example.com/a/b/c

Special cases

If a string containing the protocol is embedded at the very beginning, it will not be escaped, allowing to embed entire addresses as prefixes, e.g.: uri"$endpoint/login", where val endpoint = "http://example.com/api".

This is useful when a base URI is stored in a value, and can then be used as a base for constructing more specific URIs.

All features combined

A fully-featured example:

import sttp.client._
val secure = true
val scheme = if (secure) "https" else "http"
val subdomains = List("sub1", "sub2")
val vx = Some("y z")
val paramMap = Map("a" -> 1, "b" -> 2)
val jumpTo = Some("section2")
println(uri"$scheme://$subdomains.example.com?x=$vx&$paramMap#$jumpTo")
// https://sub1.sub2.example.com?x=y+z&a=1&b=2#section2

Request definition basics

As mentioned in the quickstart, the following import will be needed:

import sttp.client._

This brings into scope basicRequest, the starting request. This request can be customised, each time yielding a new, immutable request definition (unless a mutable body is set on the request, such as a byte array). As the request definition is immutable, it can be freely stored in values, shared across threads, and customized multiple times in various ways.

For example, we can set a cookie, String -body and specify that this should be a POST request to a given URI:

val request = basicRequest
    .cookie("login", "me")
    .body("This is a test")
    .post(uri"http://endpoint.com/secret")

The request parameters (headers, cookies, body etc.) can be specified in any order. It doesn’t matter if the request method, the body, the headers or connection options are specified in this sequence or another. This way you can build arbitrary request templates, capturing all that’s common among your requests, and customizing as needed. Remember that each time a modifier is applied to a request, you get a new immutable object.

There’s a lot of ways in which you can customize a request, which are covered in this guide. Another option is to just explore the API: most of the methods are self-explanatory and carry scaladocs if needed.

Using the modifiers, each time we get a new request definition, but it’s just a description: a data object; nothing is sent over the network until the send() method is invoked.

Query parameters and URIs

Query parameters are specified as part of the URI, to which the request should be sent. The URI can only be set together with the request method (using .get(Uri), .post(Uri), etc.).

The URI can be created programatically (by calling methods on the Uri class), or using the uri interpolator, which also allows embedding (and later escaping) values from the environment. See the documentation on creating URIs for more details.

Sending a request

A request definition can be created without knowing how it will be sent. But to send a request, a backend is needed. A default, synchronous backend based on Java’s HttpURLConnection is provided out-of-the box.

To invoke the send() method on a request description, an implicit value of type SttpBackend needs to be in scope:

implicit val backend = HttpURLConnectionBackend()
val response: Identity[Response[Either[String, String]]] = request.send()

The default backend doesn’t wrap the response into any container, but other asynchronous backends might do so. See the section on backends for more details.

Alternatively, if you prefer to pass the backend explicitly, instead of using implicits, you can also send the request the following way:

val backend = HttpURLConnectionBackend()
val response = backend.send(request)

Note

Only requests with the request method and uri can be sent. If trying to send a request without these components specified, a compile-time error will be reported. On how this is implemented, see the documentation on the type of request definitions.

Initial requests

sttp provides two initial requests:

  • basicRequest, which is an empty request with the Accept-Encoding: gzip, deflate header added. That’s the one that is most commonly used.
  • emptyRequest, a completely empty request, with no headers at all.

Both of these requests will by default read the response body into a UTF-8 String. How the response body is handled is also part of the request definition. See the section on response body specifications for more details on how to customize that.

Debugging requests

sttp comes with builtin request to curl converter. To convert request to curl invocation use .toCurl method.

For example:

basicRequest.get(uri"http://httpbin.org/ip").toCurl
// res2: String = "curl -L --max-redirs 32 -X GET 'http://httpbin.org/ip'"

Note that the Accept-Encoding header, which is added by default to all requests (Accept-Encoding: gzip, deflate) is filtered out from the generated command, so that when running a request from the command line, the result has higher chance of being human-readable, and not compressed.

Headers

Arbitrary headers can be set on the request using the .header method:

import sttp.client._

basicRequest.header("User-Agent", "myapp")

As with any other request definition modifier, this method will yield a new request, which has the given header set. The headers can be set at any point when defining the request, arbitrarily interleaved with other modifiers.

While most headers should be set only once on a request, HTTP allows setting a header multiple times. That’s why the header method has an additional optional boolean parameter, replaceExisting, which defaults to true. This way, if the same header is specified twice, only the last value will be included in the request. If previous values should be preserved, set this parameter to false.

There are also variants of this method accepting a number of headers:

import sttp.client._
import sttp.model._

basicRequest.header(Header("k1", "v1"), replaceExisting = false)
basicRequest.header("k2", "v2")
basicRequest.header("k3", "v3", replaceExisting = true)
basicRequest.headers(Map("k4" -> "v4", "k5" -> "v5"))
basicRequest.headers(Header("k9", "v9"), Header("k10", "v10"), Header("k11", "v11"))

Common headers

For some common headers, dedicated methods are provided:

import sttp.client._

basicRequest.contentType("application/json")
basicRequest.contentType("application/json", "iso-8859-1")
basicRequest.contentLength(128)
basicRequest.acceptEncoding("gzip, deflate")

See also documentation on setting cookies and authentication.

Cookies

Cookies sent in requests are key-value pairs contained in the Cookie header. They can be set on a request in a couple of ways. The first is using the .cookie(name: String, value: String) method. This will yield a new request definition which, when sent, will contain the given cookie.

Cookies are currently only available on the JVM.

Cookies can also be set using the following methods:

import sttp.client._
import sttp.model._

basicRequest
  .cookie("k1", "v1")
  .cookie("k2" -> "v2")
  .cookies("k3" -> "v3", "k4" -> "v4")
  .cookies(Seq(CookieWithMeta("k5", "k5"), CookieWithMeta("k6", "k6")))

Cookies from responses

It is often necessary to copy cookies from a response, e.g. after a login request is sent, and a successful response with the authentication cookie received. Having an object response: Response[_], cookies on a request can be copied:

import sttp.client._

implicit val backend = HttpURLConnectionBackend()
val loginRequest = basicRequest
    .cookie("login", "me")
    .body("This is a test")
    .post(uri"http://endpoint.com")
val response = loginRequest.send()

basicRequest.cookies(response)

Or, it’s also possible to store only the sttp.model.CookieWithMeta objects (a sequence of which can be obtained from a response), and set the on the request:

import sttp.client._

implicit val backend = HttpURLConnectionBackend()
val loginRequest = basicRequest
    .cookie("login", "me")
    .body("This is a test")
    .post(uri"http://endpoint.com")
val response = loginRequest.send()
val cookiesFromResponse = response.cookies

basicRequest.cookies(cookiesFromResponse)

Authentication

sttp supports basic, bearer-token based authentication and digest authentication. Two first cases are handled by adding an Authorization header with the appropriate credentials.

Basic authentication, using which the username and password are encoded using Base64, can be added as follows:

import sttp.client._

val username = "mary"
val password = "p@assword"
basicRequest.auth.basic(username, password)

A bearer token can be added using:

val token = "zMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F"
basicRequest.auth.bearer(token)

Digest authentication

This type of authentication works differently. In its assumptions it is based on an additional message exchange between client and server. Due to that a special wrapping backend is need to handle that additional logic.

In order to add digest authentication support just wrap other backend as follows:

val myBackend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()
new DigestAuthenticationBackend[Identity, Nothing, NothingT](myBackend)

Then only thing which we need to do is to pass our credentials to the relevant request:

val secureRequest = basicRequest.auth.digest(username, password)

It is also possible to use digest authentication against proxy:

val secureProxyRequest = basicRequest.proxyAuth.digest(username, password)

Both of above methods can be combined with different values if proxy and target server use digest authentication.

To learn more about digest authentication visit wikipedia

Also keep in mind that there are some limitations with the current implementation:

  • there is no caching so each request will result in an additional round-trip (or two in case of proxy and server)
  • authorizationInfo is not supported
  • scalajs supports only md5 algorithm

OAuth2

You can use sttp with OAuth2. Looking at the OAuth2 protocol flow, sttp might be helpful in the second and third step of the process:

  1. (A)/(B) - Your UI needs to enable the user to authenticate. Your application will then receive a callback from the authentication server, which will include an authentication code.
  2. (C)/(D) - You need to send a request to the authentication server, passing in the authentication code from step 1. You’ll receive an access token in response (and optionally a refresh token). For example, if you were using GitHub as your authentication server, you’d need to take the values of clientId and clientSecret from the GitHub settings, then take the authCode received in step 1 above, and send a request like this:
import sttp.client.circe._
import io.circe._
import io.circe.generic.semiauto._

val authCode = "SplxlOBeZQQYbYS6WxSbIA"
val clientId = "myClient123"
val clientSecret = "s3cret"
case class MyTokenResponse(access_token: String, scope: String, token_type: String, refresh_token: Option[String])
implicit val tokenResponseDecoder: Decoder[MyTokenResponse] = deriveDecoder[MyTokenResponse]
implicit val myBackend = HttpURLConnectionBackend()

val tokenRequest = basicRequest
    .post(uri"https://github.com/login/oauth/access_token?code=$authCode&grant_type=authorization_code")
    .auth
    .basic(clientId, clientSecret)
    .header("accept","application/json")
val authResponse = tokenRequest.response(asJson[MyTokenResponse]).send()
val accessToken = authResponse.body.map(_.access_token)
  1. (E)/(F) - Once you have the access token, you can use it to request the protected resource from the resource server, depending on its specification.

Body

Text data

In its simplest form, the request’s body can be set as a String. By default, this method will:

  • use the UTF-8 encoding to convert the string to a byte array
  • if not specified before, set Content-Type: text/plain
  • if not specified before, set Content-Length to the number of bytes in the array

A String body can be set on a request as follows:

import sttp.client._
basicRequest.body("Hello, world!")

It is also possible to use a different character encoding:

import sttp.client._
basicRequest.body("Hello, world!", "utf-8")

Binary data

To set a binary-data body, the following methods are available:

import sttp.client._

val bytes: Array[Byte] = ???
basicRequest.body(bytes)

import java.nio.ByteBuffer
val byteBuffer: java.nio.ByteBuffer = ???
basicRequest.body(byteBuffer)

import java.io.ByteArrayInputStream
val inputStream: ByteArrayInputStream = ???
basicRequest.body(inputStream)

If not specified before, these methods will set the content type to application/octet-stream. When using a byte array, additionally the content length will be set to the length of the array (unless specified explicitly).

Note

While the object defining a request is immutable, setting a mutable request body will make the whole request definition mutable as well. With InputStream, the request can be moreover sent only once, as input streams can be consumed once.

Uploading files

To upload a file, simply set the request body as a File or Path:

import sttp.client._

import java.io.File
basicRequest.body(new File("data.txt"))

import java.nio.file.Path
basicRequest.body(Path.of("data.txt"))

Note that on JavaScript only a Web/API/File is allowed.

As with binary body methods, the content type will default to application/octet-stream, and the content length will be set to the length of the file (unless specified explicitly).

See also multi-part and streaming support.

Form data

If you set the body as a Map[String, String] or Seq[(String, String)], it will be encoded as form-data (as if a web form with the given values was submitted). The content type will default to application/x-www-form-urlencoded; content length will also be set if not specified.

By default, the UTF-8 encoding is used, but can be also specified explicitly:

import sttp.client._
basicRequest.body(Map("k1" -> "v1"))
basicRequest.body(Map("k1" -> "v1"), "utf-8")
basicRequest.body("k1" -> "v1", "k2" -> "v2")
basicRequest.body(Seq("k1" -> "v1", "k2" -> "v2"), "utf-8")

Custom body serializers

It is also possible to set custom types as request bodies, as long as there’s an implicit BodySerializer[B] value in scope, which is simply an alias for a function:

type BodySerializer[B] = B => BasicRequestBody

A BasicRequestBody is a wrapper for one of the supported request body types: a String/byte array or an input stream.

For example, here’s how to write a custom serializer for a case class, with serializer-specific default content type:

import sttp.client._
import sttp.model.MediaType
case class Person(name: String, surname: String, age: Int)

// for this example, assuming names/surnames can't contain commas
implicit val personSerializer: BodySerializer[Person] = { p: Person =>
  val serialized = s"${p.name},${p.surname},${p.age}"
  StringBody(serialized, "UTF-8", Some(MediaType.TextCsv))
}

basicRequest.body(Person("mary", "smith", 67))

See the implementations of the BasicRequestBody trait for more options.

Multipart requests

To set a multipart body on a request, the multipartBody method should be used (instead of body). Each body part is represented as an instance of Part[BasicRequestBody], which can be conveniently constructed using multipart methods coming from the sttp.client package.

A single part of a multipart request consist of a mandatory name and a payload of type:

  • String
  • Array[Byte]
  • ByteBuffer
  • InputStream
  • Map[String, String]
  • Seq[(String, String)]

To add a file part, the multipartFile method (also from the com.softwaremill.sttp package) should be used. This method is overloaded and supports File/Path objects on the JVM, and Web/API/File on JS.

The content type of each part is by default the same as when setting simple bodies: text/plain for parts of type String, application/x-www-form-urlencoded for parts of key-value pairs (form data) and application/octet-stream otherwise (for binary data).

The parts can be specified using either a Seq[Multipart] or by using multiple arguments:

import sttp.client._

basicRequest.multipartBody(Seq(multipart("p1", "v1"), multipart("p2", "v2")))
basicRequest.multipartBody(multipart("p1", "v1"), multipart("p2", "v2"))

For example:

import sttp.client._
import java.io._

val someFile = new File("/sample/path")

basicRequest.multipartBody(
  multipart("text_part", "data1"),
  multipartFile("file_part", someFile), // someFile: File
  multipart("form_part", Map("x" -> "10", "y" -> "yes"))
)

Customising part meta-data

For each part, an optional filename can be specified, as well as a custom content type and additional headers. For example:

import sttp.client._
import java.io._

val logoFile = new File("/sample/path/logo123.jpg")
val docFile = new File("/sample/path/doc123.doc")
basicRequest.multipartBody(
  multipartFile("logo", logoFile).fileName("logo.jpg").contentType("image/jpg"),
  multipartFile("text", docFile).fileName("text.doc")
)

Streaming

Some backends (see backends summary) support streaming bodies. If that’s the case, you can set a stream of the supported type as a request body using the streamBody method, instead of the usual body method.

Note

Here, streaming refers to (usually) non-blocking, asynchronous streams of data. To send data which is available as an InputStream, or a file from local storage (which is available as a File or Path), no special backend support is needed. See the documenttation on setting the request body.

For example, using the akka-http backend, a request with a streaming body can be defined as follows:

import sttp.client._

import akka.stream.scaladsl.Source
import akka.util.ByteString

val chunks = "Streaming test".getBytes("utf-8").grouped(10).to(Iterable)
val source: Source[ByteString, Any] = Source.apply(chunks.toList.map(ByteString(_)))

basicRequest
  .streamBody(source)
  .post(uri"...")

Note

A request with the body set as a stream can only be sent using a backend supporting exactly the given type of streams.

It’s also possible to specify that the response body should be a stream.

The type of request definitions

All request definitions have type RequestT[U, T, S] (RequestT as in Request Template). If this looks a bit complex, don’t worry, what the three type parameters stand for is the only thing you’ll hopefully have to remember when using the API!

Going one-by-one:

  • U[_] specifies if the request method and URL are specified. Using the API, this can be either type Empty[X] = None, meaning that the request has neither a method nor an URI. Or, it can be type Id[X] = X (type-level identity), meaning that the request has both a method and an URI specified. Only requests with a specified URI & method can be sent.
  • T specifies the type to which the response will be read. By default, this is Either[String, String]. But it can also be e.g. Array[Byte] or Unit, if the response should be ignored. Response body handling can be changed by calling the .response method. With backends which support streaming, this can also be a supported stream type. See response body specifications for more details.
  • S specifies the stream type that this request uses. Most of the time this will be Nothing, meaning that this request does not send a streaming body or receive a streaming response. So most of the time you can just ignore that parameter. But, if you are using a streaming backend and want to send/receive a stream, the .streamBody or response(asStream[S]) will change the type parameter.

There are two type aliases for the request template that are used:

  • type Request[T, S] = RequestT[Id, T, S]. A sendable request.
  • type PartialRequest[T, S] = RequestT[Empty, T, S]

As basicRequest, the starting request, by default reads the body into a Either[String, String], its type is:

basicRequest: PartialRequest[Either[String, String], Nothing]

Responses

Responses are represented as instances of the case class Response[T], where T is the type of the response body. When sending a request, the response will be returned in a wrapper. For example, for asynchronous backends, we can get a Future[Response[T]], while for the default synchronous backend, the wrapper will be a no-op, Id, which is the same as no wrapper at all.

If sending the request fails, either due to client or connection errors, an exception will be thrown (synchronous backends), or an error will be represented in the wrapper (e.g. a failed future).

Note

If the request completes, but results in a non-2xx return code, the request is still considered successful, that is, a Response[T] will be returned. See response body specifications for details on how such cases are handled.

Response code

The response code is available through the .code property. There are also methods such as .isSuccess or .isServerError for checking specific response code ranges.

Response headers

Response headers are available through the .headers property, which gives all headers as a sequence (not as a map, as there can be multiple headers with the same name).

Individual headers can be obtained using the methods:

import sttp.model._
import sttp.client._
implicit val backend = HttpURLConnectionBackend()
val request = basicRequest
    .get(uri"https://httpbin.org/get")
val response = request.send()

val singleHeader: Option[String] = response.header(HeaderNames.Server)
val multipleHeaders: Seq[String] = response.headers(HeaderNames.Allow)

There are also helper methods available to read some commonly accessed headers:

val contentType: Option[String] = response.contentType
val contentLength: Option[Long] = response.contentLength

Finally, it’s possible to parse the response cookies into a sequence of the CookieWithMeta case class:

import sttp.model._

val cookies: Seq[CookieWithMeta] = response.cookies

If the cookies from a response should be set without changes on the request, this can be done directly; see the cookies section in the request definition documentation.

Obtaining the response body

The response body can be obtained through the .body: T property. T is the body deserialized as specified in the request description - see the next section on response body specifications.

Response body specification

By default, the received response body will be read as a Either[String, String], using the encoding specified in the Content-Type response header (and if none is specified, using UTF-8). This is of course configurable: response bodies can be ignored, deserialized into custom types, received as a stream or saved to a file.

The default response.body will be a:

  • Left(errorMessage) if the request is successful, but response code is not 2xx.
  • Right(body) if the request is successful and the response code is 2xx.

How the response body will be read is part of the request description, as already when sending the request, the backend needs to know what to do with the response. The type to which the response body should be deserialized is the second type parameter of RequestT, and stored in the request definition as the request.response: ResponseAs[T, S] property.

Basic response specifications

To conveniently specify how to deserialize the response body, a number of as[Type] methods are available. They can be used to provide a value for the request description’s response property:

import sttp.client._

basicRequest.response(asByteArray)

When the above request is completely described and sent, it will result in a Response[Either[String, Array[Byte]]] (where the left and right correspond to non-2xx and 2xx status codes, as above). Other possible response descriptions are:

def ignore: ResponseAs[Unit, Nothing]
def asString: ResponseAs[Either[String, String], Nothing]
def asStringAlways: ResponseAs[String, Nothing]
def asString(encoding: String): ResponseAs[Either[String, String], Nothing]
def asStringAlways(encoding: String): ResponseAs[String, Nothing]
def asByteArray: ResponseAs[Either[String, Array[Byte]], Nothing]
def asByteArrayAlways: ResponseAs[Array[Byte], Nothing]
def asParams: ResponseAs[Either[String, Seq[(String, String)]], Nothing]
def asParams(encoding: String): ResponseAs[Either[String, Seq[(String, String)]], Nothing]
def asFile(file: File): ResponseAs[Either[String, File], Nothing]
def asFileAlways(file: File): ResponseAs[File, Nothing]
def asPath(path: Path): ResponseAs[Either[String, Path], Nothing]
def asPathAlways(path: Path): ResponseAs[Path, Nothing]

def asEither[L, R, S](onError: ResponseAs[L, S], 
                      onSuccess: ResponseAs[R, S]): ResponseAs[Either[L, R], S]
def fromMetadata[T, S](f: ResponseMetadata => ResponseAs[T, S]): ResponseAs[T, S]

Hence, to discard the response body, the request description should include the following:

import sttp.client._

basicRequest.response(ignore)

And to save the response to a file:

import sttp.client._
import java.io._

val someFile = new File("some/path")
basicRequest.response(asFile(someFile))

Note

As the handling of response is specified upfront, there’s no need to “consume” the response body. It can be safely discarded if not needed.

Custom body deserializers

It’s possible to define custom body deserializers by taking any of the built-in response descriptions and mapping over them. Each ResponseAs instance has map and mapWithMetadata methods, which can be used to transform it to a description for another type (optionally using response metadata, such as headers or the status code). Each such value is immutable and can be used multiple times.

Note

Alternatively, response descriptions can be modified directly from the request description, by using the request.mapResponse(...) and request.mapResponseRight(...) methods (which is available, if the response body is deserialized to an either). That’s equivalent to calling request.response(request.response.map(...)), that is setting a new response description, to a modified old response description; but with shorter syntax.

As an example, to read the response body as an int, the following response description can be defined (warning: this ignores the possibility of exceptions!):

import sttp.client._

val asInt: ResponseAs[Either[String, Int], Nothing] = asString.mapRight(_.toInt)

basicRequest
  .get(uri"http://example.com")
  .response(asInt)

To integrate with a third-party JSON library, and always parse the response as a json (regardless of the status code):

def parseJson(json: String): Either[JsonError, JsonAST] = ...
val asJson: ResponseAs[Either[JsonError, JsonAST], Nothing] = asStringAlways.map(parseJson)

basicRequest
  .response(asJson)

A number of JSON libraries are supported out-of-the-box, see json support.

Using the fromMetadata combinator, it’s possible to dynamically specify how the response should be deserialized, basing on the response status code and response headers. The default asString, asByteArray response descriptions use this method to return a Left in case of non-2xx responses, and a Right otherwise.

A more complex case, which uses Circe for deserializing JSON, choosing to which model to deserialize to depending on the status code, can look as following:

import sttp.client._
import sttp.model._
import sttp.client.circe._
import io.circe._
import io.circe.generic.semiauto._

sealed trait MyModel
case class SuccessModel(name: String, age: Int) extends MyModel
case class ErrorModel(message: String) extends MyModel
implicit val successModelDecoder: Decoder[SuccessModel] = deriveDecoder[SuccessModel]
implicit val errorModelDecoder: Decoder[ErrorModel] = deriveDecoder[ErrorModel]

val myRequest: Request[Either[ResponseError[io.circe.Error], MyModel], Nothing] =
  basicRequest
    .get(uri"https://example.com")
    .response(fromMetadata { meta =>
      meta.code match {
        case StatusCode.Ok => asJson[SuccessModel]
        case _             => asJson[ErrorModel]
      }
    })

Streaming

If the backend used supports streaming (see backends summary), it’s possible to receive responses as a stream. This can be described using the following methods:

def asStream[S]: ResponseAs[Either[String, S], S] = ResponseAsStream[S, S]()
def asStreamAlways[S]: ResponseAs[S, S] = ResponseAsStream[S, S]()

For example, when using the Akka backend:

import akka.stream.scaladsl.Source
import akka.util.ByteString
import scala.concurrent.Future
import sttp.client._
import sttp.client.akkahttp._

implicit val sttpBackend: SttpBackend[Future, Source[ByteString, Any], NothingT] = AkkaHttpBackend()

val response: Future[Response[Either[String, Source[ByteString, Any]]]] =
  basicRequest
    .post(uri"...")
    .response(asStream[Source[ByteString, Any]])
    .send()

Note

Unlike with non-streaming response handlers, each streaming response should be entirely consumed by client code.

Exceptions

HTTP requests might fail in a variety of ways! There are two basic types of failures that might occur:

  • network-level failure, such as the invalid/unroutable hosts, inability to establish a TCP connection, or broken sockets
  • protocol-level failure, represented as 4xx and 5xx responses

The first type of failures is represented by exceptions, which are thrown when sending the request (using request.send()) or opening a websocket (request.openWebsocket(handler)). The second type of failure is represented as a Response[T], with the appropriate response code. The response body might depend on the status code; by default the response is read as a Either[String, String], where the left side represents protocol-level failure, and the right side: success.

Note

Exceptions might also be thrown when deserializing the response body - depending on the specification on how to handle response bodies. The built-in handlers return Either instead of throwing exceptions, but custom one are free to do otherwise.

Exceptions might be thrown directly (Identity synchronous backends), or returned in a backend-specific wrapper: a failed effect (other backends). Backends will try to categorise these exceptions into a SttpClientException, which has two subclasses:

  • ConnectException: when a connection (tcp socket) can’t be established to the target host
  • ReadException: when a connection has been established, but there’s any kind of problem receiving the response (e.g. a broken socket)

In general, it’s safe to assume that the request hasn’t been sent in case of connect exceptions. With read exceptions, the target host might or might have not received and processed the request.

Websockets

Apart from streaming, backends (see backends summary) can also optionally support websockets. Websocket requests are described exactly the same as regular requests, starting with basicRequest, adding headers, specifying the request method and uri.

The difference is that openWebsocket(handler) should be called instead of send(), given an instance of a backend-specific websocket handler. Refer to documentation of individual backends for details on how to instantiate the handler.

As with regular requests, instead of calling request.openWebsocket(handler) and using an implicit backend instance, it is also possible to call backend.openWebsocket(request, handler).

If creating the websocket handler is a side-effecting operation (and the handler is wrapped with an effects wrapper), the openWebsocketF(handler) can be used.

After opening a websocket, an sttp.client.ws.WebSocketResponse instance is returned, wrapped in a backend-specific effects wrapper, such as Future, IO, Task or no wrapper for synchronous backends. If the protocol upgrade hasn’t been successful, the request will fail with an error (represented as an exception or a failed effects wrapper).

In case of success, WebSocketResponse contains:

  • the headers returned when opening the websocket
  • a handler-specific and backend-specific value, which can be used to interact with the websocket, or somehow representing the result of the connection

Websocket handlers

Each backend which supports websockets, does so through a backend-specific websocket handler. Depending on the backend, this can be an implementation of a “low-level” Java listener interface, a “high-level” interface build on top of these listeners, or a backend-specific Scala stream.

The type of the handler is determined by the third type parameter of SttpBackend.

Streaming websockets

The following backends support streaming websockets:

Using the high-level websocket interface

The high-level, “functional” interface to websockets is available when using the following backends and handlers:

  • Monix and MonixWebSocketHandler from the appropriate package
  • ZIO and ZioWebSocketHandler from the appropriate package
  • fs2 and sttp.client.asynchttpclient.fs2.Fs2WebSocketHandler.

Note

The listeners created by the high-level handlers internally buffer incoming websocket events. In some implementations, when creating the handler, a bound can be specified for the size of the buffer. If the bound is specified and the buffer fills up (as can happen if the messages are not received, or processed slowly), the websocket will error and close. Otherwise, the buffer will potentially take up all available memory.

When the websocket is open, the WebSocketResponse will contain an instance of sttp.client.ws.WebSocket[F], where F is the backend-specific effects wrapper, such as IO or Task. This interface contains two methods, both of which return computations wrapped in the effects wrapper F (which typically is lazily-evaluated description of a side-effecting, asynchronous process):

  • def receive: F[Either[WebSocketEvent.Close, WebSocketFrame.Incoming]] which will complete once a message is available, and return either information that the websocket has been closed, or the incoming message
  • def send(f: WebSocketFrame, isContinuation: Boolean = false): F[Unit], which should be used to send a message to the websocket. The WebSocketFrame companion object contains methods for creating binary/text messages. When using fragmentation, the first message should be sent using finalFragment = false, and subsequent messages using isContinuation = true.

There are also other methods for receiving only text/binary messages, as well as automatically sending Pong responses when a Ping is received.

If there’s an error, a failed effects wrapper will be returned, containing one of the sttp.client.ws.WebSocketException exceptions, or a backend-specific exception.

Example usage with the Monix variant of the async-http-client backend:

import monix.eval.Task
import monix.reactive.Observable
import java.nio.ByteBuffer
import sttp.client._
import sttp.client.ws.{WebSocket, WebSocketResponse}
import sttp.model.ws.WebSocketFrame
import sttp.client.asynchttpclient.monix.MonixWebSocketHandler
import sttp.client.asynchttpclient.WebSocketHandler
import monix.execution.Scheduler.Implicits.global

implicit val backend: SttpBackend[Task, Observable[ByteBuffer], WebSocketHandler] = ???

val response: Task[WebSocketResponse[WebSocket[Task]]] = basicRequest
  .get(uri"wss://echo.websocket.org")
  .openWebsocketF(MonixWebSocketHandler())

response.flatMap { r =>
  val ws: WebSocket[Task] = r.result
  val send = ws.send(WebSocketFrame.text("Hello!"))
  val receive = ws.receiveText().flatMap(t => Task(println(s"RECEIVED: $t")))
  send.flatMap(_ => receive).flatMap(_ => ws.close)
}

Using the low-level websocket interface

Given a backend-native low-level Java interface, you can lift it to a web socket handler using WebSocketHandler.fromListener (from the appropriate package). This listener will receive lifecycle callbacks, as well as a callback each time a message is received. Note that the callbacks will be executed on the network thread, so make sure not to run any blocking operations there, and delegate to other executors/thread pools if necessary. The value returned in the WebSocketResponse will be a backend-native instance.

The types of the handlers, low-level Java interfaces and resulting websocket interfaces are, depending on the backend implementation:

  • sttp.client.asynchttpclient.WebSocketHandler / org.asynchttpclient.ws.WebSocketListener / org.asynchttpclient.ws.WebSocket
  • sttp.client.okhttp.WebSocketHandler / okhttp3.WebSocketListener / okhttp3.WebSocket
  • sttp.client.httpclient.WebSocketHandler / java.net.http.WebSocket.Listener / java.net.http.WebSocket

JSON

Adding support for JSON (or other format) bodies in requests/responses is a matter of providing a body serializer and/or a response body specification. Both are quite straightforward to implement, so integrating with your favorite JSON library shouldn’t be a problem. However, there are some integrations available out-of-the-box.

Each integration is available as an import, which brings the implicit BodySerializers and asJson methods into scope. Alternatively, these values are grouped intro traits (e.g. sttp.client.circe.SttpCirceApi), which can be extended to group multiple integrations in one object, and thus reduce the number of necessary imports.

Following data class will be used through the next few examples:

case class RequestPayload(data: String)
case class ResponsePayload(data: String)

Circe

JSON encoding of bodies and decoding of responses can be handled using Circe by the circe module. To use add the following dependency to your project:

"com.softwaremill.sttp.client" %% "circe" % "2.3.0"

This module adds a body serialized, so that json payloads can be sent as request bodies. To send a payload of type T as json, a io.circe.Encoder[T] implicit value must be available in scope. Automatic and semi-automatic derivation of encoders is possible by using the circe-generic module.

Response can be parsed into json using asJson[T], provided there’s an implicit io.circe.Decoder[T] in scope. The decoding result will be represented as either a http/deserialization error, or the parsed value. For example:

import sttp.client._
import sttp.client.circe._

implicit val backend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()

import io.circe.generic.auto._
val requestPayload = RequestPayload("some data")

val response: Identity[Response[Either[ResponseError[io.circe.Error], ResponsePayload]]] =
  basicRequest
    .post(uri"...")
    .body(requestPayload)
    .response(asJson[ResponsePayload])
    .send()

Arbitrary JSON structures can be traversed by parsing the result as io.circe.Json, and using the circe-optics module.

Json4s

To encode and decode json using json4s, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "json4s" % "2.3.0"
"org.json4s" %% "json4s-native" % "3.6.0"

Note that in this example we are using the json4s-native backend, but you can use any other json4s backend.

Using this module it is possible to set request bodies and read response bodies as case classes, using the implicitly available org.json4s.Formats (which defaults to org.json4s.DefaultFormats), and by bringing an implicit org.json4s.Serialization into scope.

Usage example:

import sttp.client._
import sttp.client.json4s._

implicit val backend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()

val requestPayload = RequestPayload("some data")

implicit val serialization = org.json4s.native.Serialization
implicit val formats = org.json4s.DefaultFormats

val response: Identity[Response[Either[ResponseError[Exception], ResponsePayload]]] =
  basicRequest
    .post(uri"...")
    .body(requestPayload)
    .response(asJson[ResponsePayload])
    .send()

spray-json

To encode and decode JSON using spray-json, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "spray-json" % "2.3.0"

Using this module it is possible to set request bodies and read response bodies as your custom types, using the implicitly available instances of spray.json.JsonWriter / spray.json.JsonReader or spray.json.JsonFormat.

Usage example:

import sttp.client._
import sttp.client.sprayJson._
import spray.json._

implicit val backend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()

implicit val payloadJsonFormat: RootJsonFormat[RequestPayload] = ???
implicit val myResponseJsonFormat: RootJsonFormat[ResponsePayload] = ???

val requestPayload = RequestPayload("some data")

val response: Identity[Response[Either[ResponseError[Exception], ResponsePayload]]] =
  basicRequest
    .post(uri"...")
    .body(requestPayload)
    .response(asJson[ResponsePayload])
    .send()

play-json

To encode and decode JSON using play-json, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "play-json" % "2.3.0"

To use, add an import: import sttp.client.playJson._.

Resilience

Resilience covers areas such as retries, circuit breaking and rate limiting.

sttp client doesn’t have the above built-in, as these concepts are usually best handled on a higher level. Sending a request (that is, invoking myRequest.send() using an implicit backend that is in scope), can be viewed as a:

  • () => Response[T] function for synchronous backends
  • () => Future[Response[T]] for Future-based asynchronous backends
  • IO[Response[T]]/Task[Response[T]] process description

All of these are lazily evaluated, and can be repeated. Such a representation allows to integrate the send() side-effect with a stack-dependent resilience tool. There’s a number of libraries that implement the above mentioned resilience functionalities, hence there’s no sense for sttp client to reimplement any of those. That’s simply not the scope of this library.

Still, the input for a particular resilience model might involve both the result (either an exception, or a response) and the original description of the request being sent. E.g. retries can depend on the request method; circuit-breaking can depend on the host, to which the request is sent; same for rate limiting.

Retries

Here’s an incomplete list of libraries which can be used to manage retries in various Scala stacks:

sttp client contains a default implementation of a predicate, which allows deciding if a request is retriable: if the body can be sent multiple times, and if the HTTP method is idempotent. This predicate is available as RetryWhen.Default and has type (Request[_, _], Either[Throwable, Response[_]]) => Boolean.

See also the retrying using ZIO example, as well as an example of a very simple retrying backend wrapper.

Note that some backends also have built-in retry mechanisms, e.g. akka-http or OkHttp (see the builder’s retryOnConnectionFailure method).

Circuit breaking

Rate limiting

Java libraries

OpenAPI

sttp-client request definitions can be automatically generated from openapi .yaml specifications using the scala-sttp code generator, included in the openapi-generator project.

For scala-sttp’s generator’s configuration options refer to: https://openapi-generator.tech/docs/generators/scala-sttp.

Standalone setup

This is the simplest setup which relies on calling openapi-generator manually and generating a complete sbt project from it.

First, you will need to install/download openapi-generator. Follow openapi-generator’s official documentation on how to do this.

Keep in mind that the scala-sttp generator is available only since v5.0.0-beta.

Next, call the generator with the following options:

openapi-generator-cli generate \
  -i petstore.yaml \
  --generator-name scala-sttp \
  -o samples/client/petstore/

Sbt managed

In this setup openapi-generator is plugged into sbt project through the sbt-openapi-generator plugin. Sttp requests and models are automatically generated upon compilation.

To have your openapi descriptions automatically turned into classes first define a new module in your project:

lazy val petstoreApi: Project = project
  .in(file("petstore-api"))
  .settings(
    openApiInputSpec := s"${baseDirectory.value.getPath}/petstore.yaml",
    openApiGeneratorName := "scala-sttp",
    openApiOutputDir := baseDirectory.value.name,
    libraryDependencies ++= Seq(
      "com.softwaremill.sttp.client" %% "core" % "2.3.0",
      "com.softwaremill.sttp.client" %% "json4s" % "2.3.0",
      "org.json4s" %% "json4s-jackson" % "3.6.8"
    )
  )

As this will generate code into petstore-api/src you might want to add this folder to the .gitignore.

Since this plugin is still in a very early stage it requires some additional configuration.

First we need to connect generation with compilation. Add following line into petstore module settings:

    (compile in Compile) := ((compile in Compile) dependsOn openApiGenerate).value,

Now we have to attach our generated source code directory into cleaning process. Add following line into petstore module settings:

    cleanFiles += baseDirectory.value / "src"

Last but not least we need to tell openapi-generator not to generate whole project but only the source files (without the sbt build file): Add following line into petstore module settings:

    openApiIgnoreFileOverride := s"${baseDirectory.in(ThisBuild).value.getPath}/openapi-ignore-file",

and create openapi-ignore-file file in project’s root directory with following content:

*
**/*
!**/src/main/scala/**/*

Final petstore module configuration:

lazy val petstoreApi: Project = project
  .in(file("petstore-api"))
  .settings(
    openApiInputSpec := s"${baseDirectory.value.getPath}/petstore.yaml",
    openApiGeneratorName := "scala-sttp",
    openApiOutputDir := baseDirectory.value.name,
    openApiIgnoreFileOverride := s"${baseDirectory.in(ThisBuild).value.getPath}/openapi-ignore-file",
    libraryDependencies ++= Seq(
      "com.softwaremill.sttp.client" %% "core" % "2.3.0",
      "com.softwaremill.sttp.client" %% "json4s" % "2.3.0",
      "org.json4s" %% "json4s-jackson" % "3.6.8"
    ),
    (compile in Compile) := ((compile in Compile) dependsOn openApiGenerate).value,
    cleanFiles += baseDirectory.value / "src"
  )

Full demo project is available on github.

Additional notes

Although recent versions of the IntelliJ IDEA IDE come with “OpenApi Specification” plugin bundled into it, this plugin doesn’t seem to support latest versions of generator and so, it is impossible to generate sttp bindings from it.

Supported backends

sttp supports a number of synchronous and asynchronous backends. It’s the backends that take care of managing connections, sending requests and receiving responses: sttp defines only the API to describe the requests to be send and handle the response data. Backends do all the heavy-lifting.

Choosing the right backend depends on a number of factors: whether you are using sttp to explore some data, or is it a production system; are you using a synchronous, blocking architecture or an asynchronous one; do you work mostly with Scala’s Future, or maybe you use some form of a Task abstraction; finally, if you want to stream requests/responses, or not.

Which one to choose?

  • for simple exploratory requests, use the synchronous HttpURLConnectionBackend, or HttpClientSyncBackend if you are on Java11.
  • if you have Akka in your stack, use Akka backend
  • otherwise, if you are using Future, use the AsyncHttpClientFutureBackend Future backend
  • finally, if you are using a functional effect wrapper, use one of the “functional” backends, for ZIO, Monix, Scalaz, cats-effect or fs2.

Each backend has three type parameters:

  • F[_], the effects wrapper for responses. That is, when you invoke send() on a request description, do you get a Response[_] directly, or is it wrapped in a Future or a Task?
  • S, the type of supported streams. If Nothing, streaming is not supported. Otherwise, the given type can be used to send request bodies or receive response bodies.
  • WS_HANDLER, the type of supported websocket handlers. If NothingT, websockets are not supported. Otherwise, websocket connections can be opened, given an instance of the handler

Below is a summary of all the JVM backends; see the sections on individual backend implementations for more information:

Class Response wrapper Supported stream type Supported websocket handlers
HttpURLConnectionBackend None (Identity) n/a n/a
TryHttpURLConnectionBackend scala.util.Try n/a n/a
AkkaHttpBackend scala.concurrent.Future akka.stream.scaladsl.Source[ByteString, Any] akka.stream.scaladsl.Flow[Message, Message, _]
AsyncHttpClientFutureBackend scala.concurrent.Future n/a sttp.client.asynchttpclient.WebSocketHandler
AsyncHttpClientScalazBackend scalaz.concurrent.Task n/a sttp.client.asynchttpclient.WebSocketHandler
AsyncHttpClientZioBackend zio.Task zio.stream.Stream[Throwable, Byte] sttp.client.asynchttpclient.WebSocketHandler
AsyncHttpClientMonixBackend monix.eval.Task monix.reactive.Observable[ByteBuffer] sttp.client.asynchttpclient.WebSocketHandler
AsyncHttpClientCatsBackend F[_]: cats.effect.Async n/a sttp.client.asynchttpclient.WebSocketHandler
AsyncHttpClientFs2Backend F[_]: cats.effect.Async fs2.Stream[F, Byte] sttp.client.asynchttpclient.WebSocketHandler
OkHttpSyncBackend None (Identity) n/a sttp.client.okhttp.WebSocketHandler
OkHttpFutureBackend scala.concurrent.Future n/a sttp.client.okhttp.WebSocketHandler
OkHttpMonixBackend monix.eval.Task monix.reactive.Observable[ByteBuffer] sttp.client.okhttp.WebSocketHandler
Http4sBackend F[_]: cats.effect.Effect fs2.Stream[F, Byte] n/a
HttpClientSyncBackend None (Identity) n/a sttp.client.httpclient.WebSocketHandler
HttpClientFutureBackend scala.concurrent.Future n/a sttp.client.httpclient.WebSocketHandler
HttpClientMonixBackend monix.eval.Task monix.reactive.Observable[ByteBuffer] sttp.client.httpclient.WebSocketHandler
HttpClientZioBackend zio.RIO[Blocking, *] zio.stream.ZStream[Blocking, Throwable, Byte] sttp.client.httpclient.WebSocketHandler
FinagleBackend com.twitter.util.Future n/a n/a

The backends work with Scala 2.11, 2.12 and 2.13 (with some exceptions for 2.11). Moreover, HttpURLConnectionBackend, AsyncHttpClientFutureBackend, AsyncHttpClientZioBackend, HttpClientSyncBackend, HttpClientFutureBackend and HttpClientZioBackend are additionally built with Dotty (Scala 3).

There are also backends which wrap other backends to provide additional functionality. These include:

  • TryBackend, which safely wraps any exceptions thrown by a synchronous backend in scala.util.Try
  • OpenTracingBackend, for OpenTracing-compatible distributed tracing. See the dedicated section.
  • PrometheusBackend, for gathering Prometheus-format metrics. See the dedicated section.
  • slf4j backends, for logging. See the dedicated section.

In addition, there are also backends for Scala.JS:

Class Response wrapper Supported stream type Supported websocket handlers
FetchBackend scala.concurrent.Future n/a n/a
FetchMonixBackend monix.eval.Task monix.reactive.Observable[ByteBuffer] n/a

And a backend for scala-native:

Class Response wrapper Supported stream type Supported websocket handlers
CurlBackend None (Identity) n/a n/a

Finally, there are third-party backends:

Starting & cleaning up

In case of most backends, you should only instantiate a backend once per application, as a backend typically allocates resources such as thread or connection pools.

When ending the application, make sure to call backend.close(), which results in an effect which frees up resources used by the backend (if any). If the effect wrapper for the backend is lazily evaluated, make sure to include it when composing effects!

Note that only resources allocated by the backends are freed. For example, if you use the AkkaHttpBackend() the close() method will terminate the underlying actor system. However, if you have provided an existing actor system upon backend creation (AkkaHttpBackend.usingActorSystem), the close() method will be a no-op.

Synchronous backends

There are several synchronous backend implementations. Sending a request using these backends is a blocking operation, and results in a sttp.client.Response[T].

Using HttpURLConnection

The default synchronous backend, available in the main jar for the JVM.

To use, add an implicit value:

implicit val sttpBackend = HttpURLConnectionBackend()

This backend works with all Scala versions. A Dotty build is available as well.

Using OkHttp

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "okhttp-backend" % "2.3.0"

Create the backend using:

import sttp.client.okhttp.OkHttpSyncBackend

implicit val sttpBackend = OkHttpSyncBackend()

or, if you’d like to instantiate the OkHttpClient yourself:

import sttp.client.okhttp.OkHttpSyncBackend
import okhttp3._

val okHttpClient: OkHttpClient = ???
implicit val sttpBackend = OkHttpSyncBackend.usingClient(okHttpClient)

This backend depends on OkHttp and fully supports HTTP/2.

Using HttpClient (Java 11+)

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "httpclient-backend" % "2.3.0"

Create the backend using:

import sttp.client.httpclient.HttpClientSyncBackend

implicit val sttpBackend = HttpClientSyncBackend()

or, if you’d like to instantiate the HttpClient yourself:

import sttp.client.httpclient.HttpClientSyncBackend
import java.net.http.HttpClient
val httpClient: HttpClient = ???
implicit val sttpBackend = HttpClientSyncBackend.usingClient(httpClient)

This backend is based on the built-in java.net.http.HttpClient available from Java 11 onwards, works with all Scala versions. A Dotty build is available as well.

Streaming

Synchronous backends don’t support non-blocking streaming.

Websockets

The HttpURLConnection-based backend doesn’t support websockets.

OkHttp and HttpClient backends support websockets by wrapping a low-level Java interface:

  • sttp.client.okhttp.WebSocketHandler, or
  • sttp.client.httpclient.WebSocketHandler

Akka backend

This backend is based on akka-http. To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "akka-http-backend" % "2.3.0"

A fully asynchronous backend. Sending a request returns a response wrapped in a Future. There are also other Future-based backends, which don’t depend on Akka.

Note that you’ll also need an explicit dependency on akka-streams, as akka-http doesn’t depend on any specific akka-streams version. So you’ll also need to add, for example:

"com.typesafe.akka" %% "akka-stream" % "2.5.31"

Next you’ll need to add an implicit value:

import sttp.client.akkahttp._
implicit val sttpBackend = AkkaHttpBackend()

or, if you’d like to use an existing actor system:

import sttp.client.akkahttp._
import akka.actor.ActorSystem

val actorSystem: ActorSystem = ???
implicit val sttpBackend = AkkaHttpBackend.usingActorSystem(actorSystem)

This backend supports sending and receiving akka-streams streams of type akka.stream.scaladsl.Source[ByteString, Any].

To set the request body as a stream:

import sttp.client._

import akka.stream.scaladsl.Source
import akka.util.ByteString

val source: Source[ByteString, Any] = ???

basicRequest
  .streamBody(source)
  .post(uri"...")

To receive the response body as a stream:

import scala.concurrent.Future
import sttp.client._
import sttp.client.akkahttp._

import akka.stream.scaladsl.Source
import akka.util.ByteString

implicit val sttpBackend = AkkaHttpBackend()

val response: Future[Response[Either[String, Source[ByteString, Any]]]] =
  basicRequest
    .post(uri"...")
    .response(asStream[Source[ByteString, Any]])
    .send()

Testing

Apart from testing using the stub, you can create a backend using any HttpRequest => Future[HttpResponse] function, or an akka-http Route.

That way, you can “mock” a server that the backend will talk to, without starting any actual server or making any HTTP calls.

If your application provides a client library for its dependants to use, this is a great way to ensure that the client actually matches the routes exposed by your application:

import sttp.client.akkahttp._
import akka.http.scaladsl.server.Route
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer

val route: Route = ???
implicit val system: ActorSystem = ???
implicit val materializer: ActorMaterializer = ActorMaterializer()

val backend = AkkaHttpBackend.usingClient(system, http = AkkaHttpClient.stubFromRoute(route))

Websockets

The Akka backend supports websockets, where the websocket handler is of type akka.stream.scaladsl.Flow[Message, Message, _]. That is, when opening a websocket connection, you need to provide the description of a stream, which will consume incoming websocket messages, and produce outgoing websocket messages. For example:

import akka.Done
import akka.stream.scaladsl.Flow
import akka.http.scaladsl.model.ws.Message
import sttp.client._
import sttp.client.ws.WebSocketResponse
import scala.concurrent.Future
import sttp.client.akkahttp._

implicit val backend : AkkaHttpBackend = ???
val flow: Flow[Message, Message, Future[Done]] = ???
val response: Future[WebSocketResponse[Future[Done]]] =
    basicRequest.get(uri"wss://echo.websocket.org").openWebsocket(flow)

In this example, the given flow materialises to a Future[Done], however this value can be arbitrary and depends on the shape and definition of the message-processing stream. The Future[WebSocketResponse] will complete once the websocket is established and contain the materialised value.

Future-based backends

There are several backend implementations which are scala.concurrent.Future-based. These backends are asynchronous, sending a request is a non-blocking operation and results in a response wrapped in a Future.

Apart from the ones described below, also the Akka backend is Future-based.

Class Supported stream type Websocket support
AkkaHttpBackend akka.stream.scaladsl.Source[ByteString, Any] akka-streams
AsyncHttpClientFutureBackend n/a wrapping a low-level Java interface
OkHttpFutureBackend n/a wrapping a low-level Java interface
HttpClientFutureBackend (Java11+) n/a wrapping a low-level Java interface

Using async-http-client

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-future" % "2.3.0"

And some imports:

import sttp.client._
import sttp.client.asynchttpclient.future.AsyncHttpClientFutureBackend

This backend depends on async-http-client and uses Netty behind the scenes.

Next you’ll need to add an implicit value:

implicit val sttpBackend = AsyncHttpClientFutureBackend()

or, if you’d like to use custom configuration:

import org.asynchttpclient.AsyncHttpClientConfig

val config: AsyncHttpClientConfig = ???
implicit val sttpBackend = AsyncHttpClientFutureBackend.usingConfig(config)

or, if you’d like to use adjust the configuration sttp creates:

import org.asynchttpclient.DefaultAsyncHttpClientConfig

val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???
implicit val sttpBackend = AsyncHttpClientFutureBackend.usingConfigBuilder(adjustFunction, sttpOptions)

or, if you’d like to instantiate the AsyncHttpClient yourself:

import org.asynchttpclient.AsyncHttpClient

val asyncHttpClient: AsyncHttpClient = ???  
implicit val sttpBackend = AsyncHttpClientFutureBackend.usingClient(asyncHttpClient)

Using OkHttp

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "okhttp-backend" % "2.3.0"

and some imports:

import sttp.client.okhttp.OkHttpFutureBackend
import scala.concurrent.ExecutionContext.Implicits.global

Create the backend using:

implicit val sttpBackend = OkHttpFutureBackend()

or, if you’d like to instantiate the OkHttpClient yourself:

import okhttp3.OkHttpClient

val asyncHttpClient: OkHttpClient = ???  
implicit val sttpBackend = OkHttpFutureBackend.usingClient(asyncHttpClient)

This backend depends on OkHttp and fully supports HTTP/2.

Using HttpClient (Java 11+)

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "httpclient-backend" % "2.3.0"

and some imports:

import sttp.client.httpclient.HttpClientFutureBackend
import scala.concurrent.ExecutionContext.Implicits.global

Create the backend using:

implicit val sttpBackend = HttpClientFutureBackend()

or, if you’d like to instantiate the HttpClient yourself:

import java.net.http.HttpClient

val client: HttpClient = ???  
implicit val sttpBackend = HttpClientFutureBackend.usingClient(client)

This backend is based on the built-in java.net.http.HttpClient available from Java 11 onwards, works with all Scala versions. A Dotty build is available as well.

Streaming

The Akka backend supports streaming using akka-streams.

Other backends don’t support non-blocking streaming.

Websockets

The Akka backend supports websockets through a high-level, streaming, akka-streams-based interface.

Other backends support websockets by wrapping the appropriate low-level Java interface.

Monix backends

There are several backend implementations which are monix.eval.Task-based. These backends are asynchronous. Sending a request is a non-blocking, lazily-evaluated operation and results in a response wrapped in a Task.

Using async-http-client

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-monix" % "2.3.0"

This backend depends on async-http-client, uses Netty behind the scenes and supports effect cancellation.

Next you’ll need to define a backend instance as an implicit value. This can be done in two basic ways:

  • by creating a Task, which describes how the backend is created, or instantiating the backend directly. In this case, you’ll need to close the backend manually
  • by creating a Resource, which will instantiate the backend and close it after it has been used

A non-comprehensive summary of how the backend can be created is as follows:

import sttp.client.asynchttpclient.monix.AsyncHttpClientMonixBackend
import sttp.client._

AsyncHttpClientMonixBackend().flatMap { implicit backend => ??? }

// or, if you'd like the backend to be wrapped in cats-effect Resource:
AsyncHttpClientMonixBackend.resource().use { implicit backend => ??? }

// or, if you'd like to use custom configuration:
import org.asynchttpclient.AsyncHttpClientConfig
val config: AsyncHttpClientConfig = ???
AsyncHttpClientMonixBackend.usingConfig(config).flatMap { implicit backend => ??? }

// or, if you'd like to use adjust the configuration sttp creates:
import org.asynchttpclient.DefaultAsyncHttpClientConfig
val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???
AsyncHttpClientMonixBackend.usingConfigBuilder(adjustFunction, sttpOptions).flatMap { implicit backend => ??? }

// or, if you'd like to instantiate the AsyncHttpClient yourself:
import org.asynchttpclient.AsyncHttpClient
val asyncHttpClient: AsyncHttpClient = ???  
implicit val sttpBackend = AsyncHttpClientMonixBackend.usingClient(asyncHttpClient)

Using OkHttp

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "okhttp-backend-monix" % "2.3.0"

Create the backend using:

import sttp.client.okhttp.monix.OkHttpMonixBackend

OkHttpMonixBackend().flatMap { implicit backend => ??? }

// or, if you'd like the backend to be wrapped in cats-effect Resource:
OkHttpMonixBackend.resource().use { implicit backend => ??? }

// or, if you'd like to instantiate the OkHttpClient yourself:
import okhttp3._
val okHttpClient: OkHttpClient = ???
implicit val sttpBackend = OkHttpMonixBackend.usingClient(okHttpClient)

This backend depends on OkHttp and fully supports HTTP/2.

Using HttpClient (Java 11+)

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "httpclient-backend-monix" % "2.3.0"

Create the backend using:

import sttp.client.httpclient.monix.HttpClientMonixBackend

HttpClientMonixBackend().flatMap { implicit backend => ??? }

// or, if you'd like the backend to be wrapped in cats-effect Resource:
HttpClientMonixBackend.resource().use { implicit backend => ??? }

// or, if you'd like to instantiate the HttpClient yourself:
import java.net.http.HttpClient
val httpClient: HttpClient = ???
implicit val sttpBackend = HttpClientMonixBackend.usingClient(httpClient)

This backend is based on the built-in java.net.http.HttpClient available from Java 11 onwards.

Streaming

The Monix backends support streaming. The type of supported streams in this case is Observable[ByteBuffer]. That is, you can set such an observable as a request body (using the async-http-client backend as an example, but any of the above backends can be used):

import sttp.client._
import sttp.client.asynchttpclient.monix._

import java.nio.ByteBuffer
import monix.reactive.Observable

AsyncHttpClientMonixBackend().flatMap { implicit backend =>
  val obs: Observable[ByteBuffer] =  ???

  basicRequest
    .streamBody(obs)
    .post(uri"...")
    .send()
}

And receive responses as an observable stream:

import sttp.client._
import sttp.client.asynchttpclient.monix._

import java.nio.ByteBuffer
import monix.eval.Task
import monix.reactive.Observable
import scala.concurrent.duration.Duration

AsyncHttpClientMonixBackend().flatMap { implicit backend =>
  val response: Task[Response[Either[String, Observable[ByteBuffer]]]] =
    basicRequest
      .post(uri"...")
      .response(asStream[Observable[ByteBuffer]])
      .readTimeout(Duration.Inf)
      .send()
    response
}

Websockets

The Monix backend supports:

  • high-level, “functional” websocket interface, through the sttp.client.asynchttpclient.monix.MonixWebSocketHandler
  • low-level interface by wrapping a low-level Java interface, sttp.client.asynchttpclient.WebSocketHandler

See websockets for details on how to use the high-level and low-level interfaces.

cats-effect backend

The Cats Effect backend is asynchronous. It can be created for any type implementing the cats.effect.Concurrent typeclass, such as cats.effect.IO. Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. There’s a transitive dependency on cats-effect.

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-cats" % "2.3.0"

You’ll need the following imports and implicits to create the backend:

import sttp.client._
import sttp.client.asynchttpclient.cats.AsyncHttpClientCatsBackend
import cats.effect._

// an implicit `cats.effect.ContextShift` in required to create the backend; here, for `cats.effect.IO`:
implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)

This backend depends on async-http-client, uses Netty behind the scenes and supports effect cancellation.

Alternatively, the http4s backend can also be created for a type implementing the cats-effect’s Effect typeclass, and supports streaming as in fs2.

Next you’ll need to define a backend instance as an implicit value. This can be done in two basic ways:

  • by creating an effect, which describes how the backend is created, or instantiating the backend directly. In this case, you’ll need to close the backend manually
  • by creating a Resource, which will instantiate the backend and close it after it has been used

A non-comprehensive summary of how the backend can be created is as follows:

// the type class instance needs to be provided explicitly (e.g. `cats.effect.IO`). 
// the effect type must implement the Concurrent typeclass
AsyncHttpClientCatsBackend[IO]().flatMap { implicit backend => ??? }

or, if you’d like to use a custom configuration:

import org.asynchttpclient.AsyncHttpClientConfig
val config: AsyncHttpClientConfig = ???
AsyncHttpClientCatsBackend.usingConfig[IO](config).flatMap { implicit backend => ??? }

or, if you’d like to use adjust the configuration sttp creates:

import org.asynchttpclient.DefaultAsyncHttpClientConfig

val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???
AsyncHttpClientCatsBackend.usingConfigBuilder[IO](adjustFunction, sttpOptions).flatMap { implicit backend => ??? }

or, if you’d like the backend to be wrapped in cats-effect Resource:

AsyncHttpClientCatsBackend.resource[IO]().use { implicit backend => ??? }

or, if you’d like to instantiate the AsyncHttpClient yourself:

import org.asynchttpclient.AsyncHttpClient

val asyncHttpClient: AsyncHttpClient = ???  
implicit val sttpBackend = AsyncHttpClientCatsBackend.usingClient[IO](asyncHttpClient)

Streaming

This backend doesn’t support non-blocking streaming.

Websockets

The backend supports websockets by wrapping a low-level Java interface, sttp.client.asynchttpclient.WebSocketHandler.

fs2 backend

The fs2 backend is asynchronous. It can be created for any type implementing the cats.effect.Async typeclass, such as cats.effect.IO. Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. There’s a transitive dependency on cats-effect.

Using async-http-client

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-fs2" % "2.3.0"

And some imports:

import sttp.client.asynchttpclient.fs2.AsyncHttpClientFs2Backend
import cats.effect._
import sttp.client._

// an implicit `cats.effect.ContextShift` in required to create an instance of `cats.effect.Concurrent`
// for `cats.effect.IO`:
implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)

This backend depends on async-http-client and uses Netty behind the scenes.

Next you’ll need to define a backend instance as an implicit value. This can be done in two basic ways:

  • by creating an effect, which describes how a backend is created, or instantiating the backend directly. In this case, you’ll need to close the backend manually
  • by creating a Resource, which will instantiate the backend and close it after it has been used

A non-comprehensive summary of how the backend can be created is as follows:

AsyncHttpClientFs2Backend[IO]().flatMap { implicit backend => ??? }

or, if you’d like to use a custom configuration:

import org.asynchttpclient.AsyncHttpClientConfig

val config: AsyncHttpClientConfig = ???
AsyncHttpClientFs2Backend.usingConfig[IO](config).flatMap { implicit backend => ??? }

or, if you’d like to use adjust the configuration sttp creates:

import org.asynchttpclient.DefaultAsyncHttpClientConfig

val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???
AsyncHttpClientFs2Backend.usingConfigBuilder[IO](adjustFunction, sttpOptions).flatMap { implicit backend => ??? }

or, if you’d like the backend to be wrapped in cats-effect Resource:

AsyncHttpClientFs2Backend.resource[IO]().use { implicit backend => ??? }

or, if you’d like to instantiate the AsyncHttpClient yourself:

import org.asynchttpclient.AsyncHttpClient

val asyncHttpClient: AsyncHttpClient = ???  
implicit val sttpBackend = AsyncHttpClientFs2Backend.usingClient[IO](asyncHttpClient)

Using HttpClient (Java 11+)

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "httpclient-backend-fs2" % "2.3.0"

And some imports:

import sttp.client.httpclient.fs2.HttpClientFs2Backend
import cats.effect._
import sttp.client._

// an implicit `cats.effect.ContextShift` is required to create a concurrent instance for `cats.effect.IO`,
// as well as a `cats.effect.Blocker` instance. Note that you'll probably want to use a different thread
// pool for blocking.
implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)
val blocker = Blocker.liftExecutionContext(scala.concurrent.ExecutionContext.global)

Create the backend using:

import sttp.client.httpclient.fs2.HttpClientFs2Backend
HttpClientFs2Backend[IO](blocker).flatMap { implicit backend => ??? }

or, if you’d like the backend to be wrapped in cats-effect Resource:

HttpClientFs2Backend.resource[IO](blocker).use { implicit backend => ??? }

or, if you’d like to instantiate the HttpClient yourself:

import java.net.http.HttpClient
val httpClient: HttpClient = ???
implicit val sttpBackend = HttpClientFs2Backend.usingClient[IO](httpClient, blocker)

This backend is based on the built-in java.net.http.HttpClient available from Java 11 onwards.

Streaming

The fs2 backend supports streaming for any instance of the cats.effect.Effect typeclass, such as cats.effect.IO. If IO is used then the type of supported streams is fs2.Stream[IO, Byte].

Requests can be sent with a streaming body like this:

import sttp.client._
import sttp.client.asynchttpclient.fs2.AsyncHttpClientFs2Backend
import fs2.Stream

val effect = AsyncHttpClientFs2Backend[IO]().flatMap { implicit backend =>
  val stream: Stream[IO, Byte] = ???

  basicRequest
    .streamBody(stream)
    .post(uri"...")
    .send()
}

Responses can also be streamed:

import sttp.client.asynchttpclient.fs2.AsyncHttpClientFs2Backend
import fs2.Stream
import scala.concurrent.duration.Duration

val effect = AsyncHttpClientFs2Backend[IO]().flatMap { implicit backend =>
  val response: IO[Response[Either[String, Stream[IO, Byte]]]] =
    basicRequest
      .post(uri"...")
      .response(asStream[Stream[IO, Byte]])
      .readTimeout(Duration.Inf)
      .send()

  response
}

Websockets

The fs2 backend supports:

  • high-level, “functional” websocket interface, through the sttp.client.asynchttpclient.fs2.Fs2WebSocketHandler or sttp.client.httpclient.fs2.Fs2WebSocketHandler
  • low-level interface by wrapping a low-level Java interface, sttp.client.asynchttpclient.WebSocketHandler or sttp.client.httpclient.WebSocketHandler
  • streaming - see below

See websockets for details on how to use the high-level and low-level interfaces.

Streaming websockets

There are additionally high-level helpers collected in sttp.client.asynchttpclient.fs2.Fs2Websockets which provide means to run the whole websocket communication through an fs2.Pipe. Example for a simple echo client:

import sttp.client.ws._
import sttp.model.ws.WebSocketFrame
import sttp.client.asynchttpclient.fs2._
import sttp.client.impl.fs2._
import sttp.client.asynchttpclient.WebSocketHandler
import cats.implicits._

implicit val backend: SttpBackend[IO, fs2.Stream[IO, Byte], WebSocketHandler] = ???
basicRequest
  .get(uri"wss://echo.websocket.org")
  .openWebsocketF(Fs2WebSocketHandler[IO]())
  .flatMap { response =>
    Fs2WebSockets.handleSocketThroughTextPipe(response.result) { in =>
      val receive = in.evalMap(m => IO(println("Received")))
      val send = fs2.Stream("Message 1".asRight, "Message 2".asRight, WebSocketFrame.close.asLeft)
      send merge receive.drain
    }
  }

Scalaz backend

The Scalaz backend is asynchronous. Sending a request is a non-blocking, lazily-evaluated operation and results in a response wrapped in a scalaz.concurrent.Task. There’s a transitive dependency on scalaz-concurrent.

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-scalaz" % "2.3.0"

This backend depends on async-http-client and uses Netty behind the scenes.

Next you’ll need to add an implicit value:

import sttp.client._
import sttp.client.asynchttpclient.scalaz.AsyncHttpClientScalazBackend

AsyncHttpClientScalazBackend().flatMap { implicit backend => ??? }

// or, if you'd like to use custom configuration:
import org.asynchttpclient.AsyncHttpClientConfig
val config: AsyncHttpClientConfig = ???

AsyncHttpClientScalazBackend.usingConfig(config).flatMap { implicit backend => ??? }

// or, if you'd like to use adjust the configuration sttp creates:
import org.asynchttpclient.DefaultAsyncHttpClientConfig
val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???

AsyncHttpClientScalazBackend.usingConfigBuilder(adjustFunction, sttpOptions).flatMap { implicit backend => ??? }

// or, if you'd like to instantiate the AsyncHttpClient yourself:
import org.asynchttpclient.AsyncHttpClient
val asyncHttpClient: AsyncHttpClient = ???

implicit val sttpBackend = AsyncHttpClientScalazBackend.usingClient(asyncHttpClient)

Streaming

This backend doesn’t support non-blocking streaming.

Websockets

The backend supports websockets by wrapping a low-level Java interface, sttp.client.asynchttpclient.WebSocketHandler.

ZIO backends

The ZIO backends are asynchronous. Sending a request is a non-blocking, lazily-evaluated operation and results in a response wrapped in a zio.Task. There’s a transitive dependency on zio.

Using async-http-client

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "async-http-client-backend-zio" % "2.3.0"

This backend depends on async-http-client, uses Netty behind the scenes and supports effect cancellation. This backend works with all Scala versions. A Dotty build is available as well.

Next you’ll need to define a backend instance as an implicit value. This can be done in two basic ways:

  • by creating a Task which describes how the backend is created, or instantiating the backend directly. In this case, you’ll need to close the backend manually
  • by creating a TaskManaged, which will instantiate the backend and close it after it has been used

A non-comprehensive summary of how the backend can be created is as follows:

import sttp.client._
import sttp.client.asynchttpclient.zio.AsyncHttpClientZioBackend

AsyncHttpClientZioBackend().flatMap { implicit backend => ??? }

// or, if you'd like the backend to be wrapped in a Managed:
AsyncHttpClientZioBackend.managed().use { implicit backend => ??? }

// or, if you'd like to use custom configuration:
import org.asynchttpclient.AsyncHttpClientConfig
val config: AsyncHttpClientConfig = ???
AsyncHttpClientZioBackend.usingConfig(config).flatMap { implicit backend => ??? }

// or, if you'd like to use adjust the configuration sttp creates:
import org.asynchttpclient.DefaultAsyncHttpClientConfig
val sttpOptions: SttpBackendOptions = SttpBackendOptions.Default  
val adjustFunction: DefaultAsyncHttpClientConfig.Builder => DefaultAsyncHttpClientConfig.Builder = ???

AsyncHttpClientZioBackend.usingConfigBuilder(adjustFunction, sttpOptions).flatMap { implicit backend => ??? }

// or, if you'd like to instantiate the AsyncHttpClient yourself:
import org.asynchttpclient.AsyncHttpClient
import zio.Runtime
val asyncHttpClient: AsyncHttpClient = ???
val runtime: Runtime[Any] = ???
implicit val backend = AsyncHttpClientZioBackend.usingClient(runtime, asyncHttpClient)

Using HttpClient (Java 11+)

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "httpclient-backend-zio" % "2.3.0"

Create the backend using:

import sttp.client.httpclient.zio.HttpClientZioBackend

HttpClientZioBackend().flatMap { implicit backend => ??? }

// or, if you'd like the backend to be wrapped in a Managed:
HttpClientZioBackend.managed().use { implicit backend => ??? }

// or, if you'd like to instantiate the HttpClient yourself:
import java.net.http.HttpClient
val httpClient: HttpClient = ???
implicit val sttpBackend = HttpClientZioBackend.usingClient(httpClient)

This backend is based on the built-in java.net.http.HttpClient available from Java 11 onwards.

ZIO environment

As an alternative to effectfully or resourcefully creating backend instances, ZIO environment can be used. In this case, a type alias is provided for the service definition:

package sttp.client.asynchttpclient.zio
type SttpClient = Has[SttpBackend[Task, Stream[Throwable, Byte], WebSocketHandler]]

// or, when using Java 11 & HttpClient

package sttp.client.httpclient.zio
type SttpClient = Has[SttpBackend[BlockingTask, ZStream[Blocking, Throwable, Byte], NothingT]]

The lifecycle of the SttpClient service is described by ZLayers, which can be created using the .layer/.layerUsingConfig/… methods on AsyncHttpClientZioBackend / HttpClientZioBackend.

The SttpClient companion object contains effect descriptions which use the SttpClient service from the environment to send requests or open websockets. This is different from sttp usage with other effect libraries (which use an implicit backend when .send()/.openWebsocket() is invoked on the request), but is more in line with how other ZIO services work. For example:

import sttp.client._
import sttp.client.asynchttpclient.zio._
import zio._
val request = basicRequest.get(uri"https://httpbin.org/get")

val send: ZIO[SttpClient, Throwable, Response[Either[String, String]]] = 
  SttpClient.send(request)

Example using websockets:

import sttp.client._
import sttp.client.ws._
import sttp.client.asynchttpclient.zio._
import zio._
val request = basicRequest.get(uri"wss://echo.websocket.org")

val open: ZIO[SttpClient, Throwable, WebSocketResponse[WebSocket[Task]]] = 
  SttpClient.openWebsocket(request)

Streaming

The ZIO based backends support streaming using zio-streams. The following example is using the AsyncHttpClientZioBackend backend, but works similarly with HttpClientZioBackend.

The type of supported streams is Stream[Throwable, Byte]. To leverage ZIO environment, use the SttpClient object to create request send/websocket open effects.

Requests can be sent with a streaming body:

import sttp.client._
import sttp.client.asynchttpclient.zio._

import zio.stream._

val s: Stream[Throwable, Byte] =  ???

val request = basicRequest
  .streamBody(s)
  .post(uri"...")

SttpClient.send(request)

And receive response bodies as a stream:

import sttp.client._
import sttp.client.asynchttpclient.zio._

import zio._
import zio.stream._

import scala.concurrent.duration.Duration

val request =
  basicRequest
    .post(uri"...")
    .response(asStream[Stream[Throwable, Byte]])
    .readTimeout(Duration.Inf)

val response: ZIO[SttpClient, Throwable, Response[Either[String, Stream[Throwable, Byte]]]] = SttpClient.send(request)

Websockets

The ZIO backend supports:

  • high-level, “functional” websocket interface, through the ZioWebSocketHandler from the appropriate package
  • low-level interface by wrapping a low-level Java interface, WebSocketHandler from the appropriate package

See websockets for details on how to use the high-level and low-level interfaces. Websockets opened using the SttpClient.openWebsocket and SttpStreamsClient.openWebsocket (leveraging ZIO environment) always use the high-level interface.

Testing

The ZIO backends also support a ZIO-familiar way of configuring stubs as well. In addition to the usual way of creating a stand-alone stub, you can also define your stubs as effects instead:

import sttp.client._
import sttp.model._
import sttp.client.asynchttpclient._
import sttp.client.asynchttpclient.zio._
import sttp.client.asynchttpclient.zio.stubbing._

val stubEffect = for {
  _ <- whenRequestMatches(_.uri.toString.endsWith("c")).thenRespond("c")
  _ <- whenRequestMatchesPartial { case r if r.method == Method.POST => Response.ok("b") }
  _ <- whenAnyRequest.thenRespond("a")
} yield ()

val responseEffect = stubEffect *> SttpClient.send(basicRequest.get(uri"http://example.org/a")).map(_.body)

responseEffect.provideLayer(AsyncHttpClientZioBackend.stubLayer) // Task[Either[String, String]]

The whenRequestMatches, whenRequestMatchesPartial, whenAnyRequest are effects which require the SttpClientStubbing dependency. They enrich the stub with the given behavior.

Then, the stubLayer provides both an implementation of the SttpClientStubbing dependency, as well as a SttpClient which is backed by the stub.

Http4s backend

This backend is based on http4s (blaze client) and is asynchronous. To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "http4s-backend" % "2.3.0"

Add some imports as well:

import cats.effect._
import sttp.client.http4s._
import scala.concurrent._

// an implicit `cats.effect.ContextShift` is required to create an instance of `cats.effect.Concurrent` 
// for `cats.effect.IO`,  as well as a `cats.effect.Blocker` instance. 
// Note that you'll probably want to use a different thread pool for blocking.
implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
val blocker: cats.effect.Blocker = Blocker.liftExecutionContext(ExecutionContext.global)

The backend can be created for any type implementing the cats.effect.ConcurrentEffect typeclass, such as cats.effect.IO. Moreover, an implicit ContextShift will have to be in scope as well.

If a blocker instance is not available, a new one can be created, and the resource definition can be chained, e.g. as follows:

implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global) // or another instance
Blocker[IO].flatMap(Http4sBackend.usingDefaultClientBuilder[IO](_)).use { implicit backend => ... }

Sending a request is a non-blocking, lazily-evaluated operation and results in a wrapped response. There’s a transitive dependency on http4s.

There are also other cats-effect-based backends, which don’t depend on http4s.

Please note that:

  • the backend does not support SttpBackendOptions,that is specifying proxy settings (proxies are not implemented in http4s, see this issue), as well as configuring the connect timeout
  • the backend does not support the RequestT.options.readTimeout option

Instead, all custom timeout configuration should be done by creating a org.http4s.client.Client[F], using org.http4s.client.blaze.BlazeClientBuilder[F] and passing it to the appropriate method of the Http4sBackend object.

The backend supports streaming using fs2. For usage details, see the documentation on streaming using fs2.

The backend doesn’t support websockets.

Twitter future (Finagle) backend

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "finagle-backend" % "2.3.0"

Next you’ll need to add an implicit value:

import sttp.client.finagle.FinagleBackend
implicit val sttpBackend = FinagleBackend()

This backend depends on finagle, and offers an asynchronous backend, which wraps results in Twitter’s Future.

Please note that:

  • the backend does not support SttpBackendOptions, that is specifying proxy settings (proxies are not implemented in http4s, see this issue), as well as configuring the connect timeout
  • the backend does not support non-blocking streaming or websockets.

JavaScript (Fetch) backend

A JavaScript backend implemented using the Fetch API and backed via Future.

This is the default backend, available in the main jar for JS. To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %%% "core" % "2.3.0"

And add an implicit value:

implicit val sttpBackend = FetchBackend()

Timeouts are handled via the new AbortController class. As this class only recently appeared in browsers you may need to add a polyfill.

As browsers do not allow access to redirect responses, if a request sets followRedirects to false then a redirect will cause the response to return an error.

Note that Fetch does not pass cookies by default. If your request needs cookies then you will need to pass a FetchOptions instance with credentials set to either RequestCredentials.same-origin or RequestCredentials.include depending on your requirements.

Node.js

Running sttp in a node.js will require downloading modules that implement the various classes and functions used by sttp, usually available in browser. At minima, you will need replacement for fetch, AbortController and Headers. To achieve this, you can either use npm directly, or the scalajs-bundler sbt plugin if you use sbt :

npm install --save node-fetch
npm install --save abortcontroller-polyfill
npm install --save fetch-headers

You then need to load the modules into your runtime. This can be done in your main method as such :

val g = scalajs.js.Dynamic.global.globalThis
g.fetch = g.require("node-fetch")
g.require("abortcontroller-polyfill/dist/polyfill-patch-fetch")
g.Headers = g.require("fetch-headers")

Streaming

Streaming support is provided via FetchMonixBackend. Note that streaming support on Firefox is hidden behind a flag, see ReadableStream for more information.

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %%% "monix" % "2.3.0"

An example of streaming a response:

import sttp.client._
import sttp.client.impl.monix._

import java.nio.ByteBuffer
import monix.eval.Task
import monix.reactive.Observable

implicit val sttpBackend = FetchMonixBackend()

val response: Task[Response[Observable[ByteBuffer]]] =
  sttp
    .post(uri"...")
    .response(asStream[Observable[ByteBuffer]])
    .send()

Note

Currently no browsers support passing a stream as the request body. As such, using the Fetch backend with a streaming request will result in it being converted into an in-memory array before being sent. Response bodies are returned as a “proper” stream.

Curl backend

A Scala Native backend implemented using Curl.

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %%% "core" % "2.3.0"

and initialize one of the backends:

implicit val sttpBackend = CurlBackend()
implicit val sttpTryBackend = CurlTryBackend()

You need to have an environment with Scala Native setup with additionally installed libcrypto (included in OpenSSL) and curl in version 7.56.0 or newer.

Opentracing backend

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "opentracing-backend" % "2.3.0"

This backend depends on opentracing, a standardized set of api for distributed tracing.

The opentracing backend wraps any other backend, but it’s useless without a concrete distributed tracing implementation. To obtain instance of opentracing backend:

OpenTracingBackend(wrappedBackend, tracer)

Where tracer is an interface which can be implemented by any compatible library. See examples below.

The backend obtains the current trace context using default spans’s propagation mechanisms.

There is an additional method exposed to override default operation id:

import sttp.client._
import sttp.client.opentracing.OpenTracingBackend._

basicRequest
  .get(???)
  .tagWithOperationId("register-user")

There is an additional method exposed to customize generated span:

import sttp.client._
import sttp.client.opentracing.OpenTracingBackend._

basicRequest
  .get(???)
  .tagWithTransformSpan(_.setTag("custom-tag", "custom-value").setOperationName("new-name").log("my-event"))

Integration with jaeger

Using with jaeger tracing

Add following dependency:

libraryDependencies += "io.jaegertracing" % "jaeger-client" % "1.4.0"

Create an instance of tracer:

import io.opentracing.Tracer
import io.jaegertracing.Configuration
import io.jaegertracing.Configuration.ReporterConfiguration
import io.jaegertracing.Configuration.SamplerConfiguration

def initTracer(serviceName: String ): Tracer = {
  val samplerConfig = SamplerConfiguration.fromEnv().withType("const").withParam(1)
  val reporterConfig = ReporterConfiguration.fromEnv().withLogSpans(true)
  val config = new Configuration(serviceName).withSampler(samplerConfig)
                    .withReporter(reporterConfig)
  config.getTracer()
}

For more details about integration with jaeger click here

Integration with brave

Using with brave tracing

Add following dependency:

libraryDependencies += "io.opentracing.brave" % "brave-opentracing" % "0.37.2"
// and for integrationw with okHttp:
libraryDependencies += "io.zipkin.reporter2" % "zipkin-sender-okhttp3" % "2.15.2" 

Create an instance of tracer:

import io.opentracing.Tracer
import zipkin2.reporter.AsyncReporter
import zipkin2.reporter.okhttp3.OkHttpSender
import brave.propagation.{ExtraFieldPropagation, B3Propagation}
import brave.Tracing
import brave.opentracing.BraveTracer
import java.util.Arrays

def initTracer(zipkinUrl: String, serviceName: String): Tracer = {
  // Configure a reporter, which controls how often spans are sent
  val sender = OkHttpSender.create(zipkinUrl)
  val spanReporter = AsyncReporter.create(sender)

  // If you want to support baggage, indicate the fields you'd like to
  // whitelist, in this case "country-code" and "user-id". On the wire,
  // they will be prefixed like "baggage-country-code"
  val propagationFactory = ExtraFieldPropagation.newFactoryBuilder(B3Propagation.FACTORY)
                                  .addPrefixedFields("baggage-",
                                          Arrays.asList("country-code", "user-id"))
                                  .build()

  // Now, create a Brave tracing component with the service name you want to see in
  // Zipkin (the dependency is io.zipkin.brave:brave).
  val braveTracing = Tracing.newBuilder()
                        .localServiceName(serviceName)
                        .propagationFactory(propagationFactory)
                        .spanReporter(spanReporter)
                        .build()

  // use this to create an OpenTracing Tracer
  BraveTracer.create(braveTracing)
}

For more details about integration with brave click here

zio-telemetry opentracing backend

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "zio-telemetry-opentracing-backend" % "2.3.0"

This backend depends on zio-opentracing.

The opentracing backend wraps a Task based ZIO backend and yields a backend of type SttpBackend[RIO[OpenTracing, *], Nothing, WS_HANDLER]. The yielded effects are of type RIO[OpenTracing, *] which mean they can be a child of a other span created in your ZIO program.

Here’s how you construct ZioTelemetryOpenTracingBackend. I would recommend wrapping this is in ZLayer

new ZioTelemetryOpenTracingBackend(zioBackend)

Additionally you can add tags per request by supplying a ZioTelemetryOpenTracingTracer

import sttp.client._
import zio._
import zio.telemetry.opentracing._
import sttp.client.ziotelemetry.opentracing._

implicit val zioBackend: SttpBackend[Task, Nothing, NothingT] = ???

def sttpTracer: ZioTelemetryOpenTracingTracer = new ZioTelemetryOpenTracingTracer {
    def before[T](request: Request[T, Nothing]): RIO[OpenTracing, Unit] =
      OpenTracing.tag("span.kind", "client") *>
      OpenTracing.tag("http.method", request.method.method) *>
      OpenTracing.tag("http.url", request.uri.toString()) *>
      OpenTracing.tag("type", "ext") *>
      OpenTracing.tag("subtype", "http")
    
    def after[T](response: Response[T]): RIO[OpenTracing, Unit] =
      OpenTracing.tag("http.status_code", response.code.code)
}

new ZioTelemetryOpenTracingBackend[NothingT](zioBackend, sttpTracer)

Prometheus backend

To use, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "prometheus-backend" % "2.3.0"

and some imports:

import sttp.client.prometheus._

This backend depends on Prometheus JVM Client. Keep in mind this backend registers histograms and gathers request times, but you have to expose those metrics to Prometheus e.g. using prometheus-akka-http.

The Prometheus backend wraps any other backend, for example:

import sttp.client.akkahttp._
implicit val sttpBackend = PrometheusBackend(AkkaHttpBackend())

It gathers request execution times in Histogram. It uses by default sttp_request_latency name, defined in PrometheusBackend.DefaultHistogramName. It is possible to define custom histograms name by passing function mapping request to histogram name:

import sttp.client.akkahttp._
implicit val sttpBackend = PrometheusBackend(AkkaHttpBackend(), requestToHistogramNameMapper = request => Some(HistogramCollectorConfig(request.uri.host)))

You can disable request histograms by passing None returning function:

import sttp.client.akkahttp._
implicit val sttpBackend = PrometheusBackend(AkkaHttpBackend(), requestToHistogramNameMapper = _ => None)

This backend also offers Gauge with currently in-progress requests number. It uses by default sttp_requests_in_progress name, defined in PrometheusBackend.DefaultRequestsInProgressGaugeName. It is possible to define custom gauge name by passing function mapping request to gauge name:

import sttp.client.akkahttp._
implicit val sttpBackend = PrometheusBackend(AkkaHttpBackend(), requestToInProgressGaugeNameMapper = request => Some(CollectorConfig(request.uri.host)))

You can disable request in-progress gauges by passing None returning function:

import sttp.client.akkahttp._
implicit val sttpBackend = PrometheusBackend(AkkaHttpBackend(), requestToInProgressGaugeNameMapper = _ => None)

Logging using slf4j

There are three backend wrappers available, which log request & response information using a slf4j Logger. To see the logs, you’ll need to use an slf4j-compatible logger implementation, e.g. logback, or use a binding, e.g. log4j-slf4j.

To use the backend wrappers, add the following dependency to your project:

"com.softwaremill.sttp.client" %% "slf4j-backend" % "2.3.0"

The following backend wrappers are available:

import sttp.client._
import sttp.client.logging.slf4j._
val delegateBackend: SttpBackend[Identity, Nothing, NothingT] = ???

Slf4jLoggingBackend[Identity, Nothing, NothingT](delegateBackend)
Slf4jTimingBackend[Identity, Nothing, NothingT](delegateBackend)
Slf4jCurlBackend[Identity, Nothing, NothingT](delegateBackend)

The logging backend logs DEBUG-level logs when a request is started, completes successfully, and ERROR-level logs when it results in an exception.

The timing backend logs INFO-level logs when a request completes successfully or with an exception, together with the number of seconds and milliseconds that the request took.

The curl backend logs DEBUG-level logs when a request completes successfully or with an exception, together with the curl command that can be issued to reproduce the request.

Example usage:

import sttp.client._
import sttp.client.logging.slf4j.Slf4jTimingBackend

implicit val backend = Slf4jTimingBackend[Identity, Nothing, NothingT](HttpURLConnectionBackend())
basicRequest.get(uri"https://httpbin.org/get").send()

To create a customised logging backend, see the section on custom backends.

Custom backends, logging, metrics

It is also entirely possible to write custom backends (if doing so, please consider contributing!) or wrap an existing one. One can even write completely generic wrappers for any delegate backend, as each backend comes equipped with a monad for the response type. This brings the possibility to map and flatMap over responses.

Possible use-cases for wrapper-backend include:

  • logging
  • capturing metrics
  • request signing (transforming the request before sending it to the delegate)

See also the section on resilience which covers topics such as retries, circuit breaking and rate limiting.

Request tagging

Each request contains a tags: Map[String, Any] map. This map can be used to tag the request with any backend-specific information, and isn’t used in any way by sttp itself.

Tags can be added to a request using the def tag(k: String, v: Any) method, and read using the def tag(k: String): Option[Any] method.

Backends, or backend wrappers can use tags e.g. for logging, passing a metric name, using different connection pools, or even different delegate backends.

Listener backend

The sttp.client.listener.ListenerBackend can make it easier to create backend wrappers which need to be notified about request lifecycle events: when a request is started, and when it completes either successfully or with an exception. This is possible by implementing a sttp.client.listener.RequestListener. This is how e.g. the slf4j backend is implemented.

A request listener can associate a value with a request, which will then be passed to the request completion notification methods.

A side-effecting request listener, of type RequestListener[Identity, L], can be lifted to a request listener RequestListener[F, L] given a MonadError[F], using the RequestListener.lift method.

Backend wrappers and redirects

By default redirects are handled at a low level, using a wrapper around the main, concrete backend: each of the backend factory methods, e.g. HttpURLConnectionBackend() returns a backend wrapped in FollowRedirectsBackend.

This causes any further backend wrappers to handle a request which involves redirects as one whole, without the intermediate requests. However, wrappers which collects metrics, implements tracing or handles request retries might want to handle every request in the redirect chain. This can be achieved by layering another FollowRedirectsBackend on top of the wrapper. Only the top-level follow redirects backend will handle redirects, other follow redirect wrappers (at lower levels) will be disabled.

For example:

import sttp.client._
import sttp.client.ws._
import sttp.client.monad._
class MyWrapper[F[_], S, WS_HANDLER[_]] private (delegate: SttpBackend[F, S, WS_HANDLER])
  extends SttpBackend[F, S, WS_HANDLER] {

  def send[T](request: Request[T, S]): F[Response[T]] = ???

  def openWebsocket[T, WS_RESULT](
      request: Request[T, S],
      handler: WS_HANDLER[WS_RESULT]
    ): F[WebSocketResponse[WS_RESULT]] = ???

  def close(): F[Unit] = ???

  def responseMonad: MonadError[F] = ???
}

object MyWrapper {
  def apply[F[_], S, WS_HANDLER[_]](
    delegate: SttpBackend[F, S, WS_HANDLER]): SttpBackend[F, S, WS_HANDLER] = {
    // disables any other FollowRedirectsBackend-s further down the delegate chain
    new FollowRedirectsBackend(new MyWrapper(delegate))
  }
}

Logging backend wrapper

A good example on how to implement a logging backend wrapper is the slf4j backend wrapper implementation. It uses the ListenerBackend to get notified about request lifecycle events, and logs messages created using sttp.client.logging.LogMessages.

To adjust the logs to your needs, or to integrate with your logging framework, simply copy the code and modify as needed.

Example metrics backend wrapper

Below is an example on how to implement a backend wrapper, which sends metrics for completed requests and wraps any Future-based backend:

import sttp.client._
import sttp.client.monad._
import sttp.client.ws._
import sttp.client.akkahttp._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util._
// the metrics infrastructure
trait MetricsServer {
  def reportDuration(name: String, duration: Long): Unit
}

class CloudMetricsServer extends MetricsServer {
  override def reportDuration(name: String, duration: Long): Unit = ???
}

// the backend wrapper
class MetricWrapper[S](delegate: SttpBackend[Future, S, NothingT],
                       metrics: MetricsServer)
    extends SttpBackend[Future, S, NothingT] {

  override def send[T](request: Request[T, S]): Future[Response[T]] = {
    val start = System.currentTimeMillis()

    def report(metricSuffix: String): Unit = {
      val metricPrefix = request.tag("metric").getOrElse("?")
      val end = System.currentTimeMillis()
      metrics.reportDuration(metricPrefix + "-" + metricSuffix, end - start)
    }

    delegate.send(request).andThen {
      case Success(response) if response.is200 => report("ok")
      case Success(response)                   => report("notok")
      case Failure(t)                          => report("exception")
    }
  }

  override def openWebsocket[T, WS_RESULT](
      request: Request[T, S],
      handler: NothingT[WS_RESULT]
    ): Future[WebSocketResponse[WS_RESULT]] = {
    delegate.openWebsocket(request, handler) // No websocket support due to NothingT
  }

  override def close(): Future[Unit] = delegate.close()

  override def responseMonad: MonadError[Future] = delegate.responseMonad
}

// example usage
implicit val backend = new MetricWrapper(
  AkkaHttpBackend(),
  new CloudMetricsServer()
)

basicRequest
  .get(uri"http://company.com/api/service1")
  .tag("metric", "service1")
  .send()

See also the Prometheus backend for an example implementation.

Example retrying backend wrapper

Handling retries is a complex problem when it comes to HTTP requests. When is a request retryable? There are a couple of things to take into account:

  • connection exceptions are generally good candidates for retries
  • only idempotent HTTP methods (such as GET) could potentially be retried
  • some HTTP status codes might also be retryable (e.g. 500 Internal Server Error or 503 Service Unavailable)

In some cases it’s possible to implement a generic retry mechanism; such a mechanism should take into account logging, metrics, limiting the number of retries and a backoff mechanism. These mechanisms could be quite simple, or involve e.g. retry budgets (see Finagle’s documentation on retries). In sttp, it’s possible to recover from errors using the responseMonad. A starting point for a retrying backend could be:

import sttp.client._
import sttp.client.monad._
import sttp.client.ws._

class RetryingBackend[F[_], S](
    delegate: SttpBackend[F, S, NothingT],
    shouldRetry: RetryWhen,
    maxRetries: Int)
    extends SttpBackend[F, S, NothingT] {

  override def send[T](request: Request[T, S]): F[Response[T]] = {
    sendWithRetryCounter(request, 0)
  }

  private def sendWithRetryCounter[T](request: Request[T, S],
                                      retries: Int): F[Response[T]] = {
    val r = responseMonad.handleError(delegate.send(request)) {
      case t if shouldRetry(request, Left(t)) && retries < maxRetries =>
        sendWithRetryCounter(request, retries + 1)
    }

    responseMonad.flatMap(r) { resp =>
      if (shouldRetry(request, Right(resp)) && retries < maxRetries) {
        sendWithRetryCounter(request, retries + 1)
      } else {
        responseMonad.unit(resp)
      }
    }
  }

  override def openWebsocket[T, WS_RESULT](
      request: Request[T, S],
      handler: NothingT[WS_RESULT]
    ): F[WebSocketResponse[WS_RESULT]] = {
    delegate.openWebsocket(request, handler) // No websocket support due to NothingT
  }

  override def close(): F[Unit] = delegate.close()

  override def responseMonad: MonadError[F] = delegate.responseMonad
}

Example backend with circuit breaker

“When a system is seriously struggling, failing fast is better than making clients wait.”

There are many libraries that can help you achieve such a behavior: hystrix, resilience4j, akka’s circuit breaker or monix catnap to name a few. Despite some small differences, both their apis and functionality are very similar, that’s why we didn’t want to support each of them explicitly.

Below is an example on how to implement a backend wrapper, which integrates with circuit-breaker module from resilience4j library and wraps any backend:

import io.github.resilience4j.circuitbreaker.{CallNotPermittedException, CircuitBreaker}
import sttp.client.monad.MonadError
import sttp.client.ws.WebSocketResponse
import sttp.client.{Request, Response, SttpBackend}
import java.util.concurrent.TimeUnit

class CircuitSttpBackend[F[_], S, W[_]](
    circuitBreaker: CircuitBreaker,
    delegate: SttpBackend[F, S, W]
    )(implicit monadError: MonadError[F]) extends SttpBackend[F, S, W] {

  override def send[T](request: Request[T, S]): F[Response[T]] = {
    CircuitSttpBackend.decorateF(circuitBreaker, delegate.send(request))
  }

  override def openWebsocket[T, WS_RESULT](
      request: Request[T, S],
      handler: W[WS_RESULT]
  ): F[WebSocketResponse[WS_RESULT]] =
        CircuitSttpBackend.decorateF(circuitBreaker, delegate.openWebsocket(request, handler))

  override def close(): F[Unit] = delegate.close()

  override def responseMonad: MonadError[F] = delegate.responseMonad
}

object CircuitSttpBackend {

  def decorateF[F[_], T](
      circuitBreaker: CircuitBreaker,
      service: => F[T]
  )(implicit monadError: MonadError[F]): F[T] = {
    monadError.flatMap(monadError.unit(())) { _ =>
        if (!circuitBreaker.tryAcquirePermission()) {
          monadError.error(CallNotPermittedException
                                .createCallNotPermittedException(circuitBreaker))
        } else {
          val start = System.nanoTime()
          try {
            monadError.handleError(monadError.map(service) { r =>
              circuitBreaker.onSuccess(System.nanoTime() - start, TimeUnit.NANOSECONDS)
              r
            }) {
              case t =>
                circuitBreaker.onError(System.nanoTime() - start, TimeUnit.NANOSECONDS, t)
                monadError.error(t)
            }
          } catch {
            case t: Throwable =>
              circuitBreaker.onError(System.nanoTime() - start, TimeUnit.NANOSECONDS, t)
              monadError.error(t)
          }
        }
    }
  }
}

Example backend with rate limiter

“Prepare for a scale and establish reliability and HA of your service.”

Below is an example on how to implement a backend wrapper, which integrates with rate-limiter module from resilience4j library and wraps any backend:

import io.github.resilience4j.ratelimiter.RateLimiter
import sttp.client.monad.MonadError
import sttp.client.ws.WebSocketResponse
import sttp.client.{Request, Response, SttpBackend}

class RateLimitingSttpBackend[F[_], S, W[_]](
    rateLimiter: RateLimiter,
    delegate: SttpBackend[F, S, W]
    )(implicit monadError: MonadError[F]) extends SttpBackend[F, S, W] {

  override def send[T](request: Request[T, S]): F[Response[T]] = {
    RateLimitingSttpBackend.decorateF(rateLimiter, delegate.send(request))
  }

  override def openWebsocket[T, WS_RESULT](
      request: Request[T, S],
      handler: W[WS_RESULT]
  ): F[WebSocketResponse[WS_RESULT]] = delegate.openWebsocket(request, handler)

  override def close(): F[Unit] = delegate.close()

  override def responseMonad: MonadError[F] = delegate.responseMonad
}

object RateLimitingSttpBackend {

  def decorateF[F[_], T](
      rateLimiter: RateLimiter,
      service: => F[T]
  )(implicit monadError: MonadError[F]): F[T] = {
    monadError.flatMap(monadError.unit(())){ _=>
        try {
          RateLimiter.waitForPermission(rateLimiter)
          service
        } catch {
          case t: Throwable =>
            monadError.error(t)
        }
    }
  }
}

Example new backend

Implementing a new backend is made easy as the tests are published in the core jar file under the tests classifier. Simply add the follow dependencies to your build.sbt:

"com.softwaremill.sttp.client" %% "core" % "2.3.0" % Test classifier "tests"

Implement your backend and extend the HttpTest class:

import sttp.client._
import sttp.client.testing.{ConvertToFuture, HttpTest}
import scala.concurrent.Future

class MyCustomBackendHttpTest extends HttpTest[Future] {
  override implicit val convertToFuture: ConvertToFuture[Future] = ConvertToFuture.future
  override implicit lazy val backend: SttpBackend[Future, Nothing, NothingT] = ??? //new MyCustomBackend()
}

You can find a more detailed example in the sttp-vertx repository.

Custom backend wrapper using cats

When implementing a backend wrapper using cats, it might be useful to import:

import sttp.client.impl.cats.implicits._

from the cats integration module. The module should be available on the classpath when using the cats async-http-client backend. The object contains implicits to convert a cats MonadError into the sttp MonadError, as well as a way to map the effects wrapper used with the .mapK extension method for the backend.

Testing

If you need a stub backend for use in tests instead of a “real” backend (you probably don’t want to make HTTP calls during unit tests), you can use the SttpBackendStub class. It allows specifying how the backend should respond to requests matching given predicates.

You can also create a stub backend using akka-http routes.

Creating a stub backend

An empty backend stub can be created using the following ways:

  • by calling .stub on the “real” base backend’s companion object, e.g. AsyncHttpClientZioBackend.stub or HttpClientMonixBackend.stub
  • by using one of the factory methods SttpBackendStub.synchronous or SttpBackendStub.asynchronousFuture, which return stubs which use the Identity or standard Scala’s Future response wrappers without streaming support
  • by explicitly giving the response wrapper monad and supported streams type, e.g. SttpBackendStub[Task, Observable[ByteBuffer]](TaskMonad)
  • by specifying a fallback/delegate backend, see below

Some code which will be reused among following examples:

import sttp.client._
import sttp.model._
import sttp.client.testing._
import java.io.File
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global  

case class User(id: String)

Specifying behavior

Behavior of the stub can be specified using a combination of the whenRequestMatches and thenRespond methods:

implicit val testingBackend = SttpBackendStub.synchronous
  .whenRequestMatches(_.uri.path.startsWith(List("a", "b")))
  .thenRespond("Hello there!")
  .whenRequestMatches(_.method == Method.POST)
  .thenRespondServerError()

val response1 = basicRequest.get(uri"http://example.org/a/b/c").send()
// response1.body will be Right("Hello there")

val response2 = basicRequest.post(uri"http://example.org/d/e").send()

It is also possible to match requests by partial function, returning a response. E.g.:

implicit val testingBackend = SttpBackendStub.synchronous
  .whenRequestMatchesPartial({
    case r if r.uri.path.endsWith(List("partial10")) =>
      Response("Not found", StatusCode.NotFound)

    case r if r.uri.path.endsWith(List("partialAda")) =>
      // additional verification of the request is possible
      assert(r.body == StringBody("z", "utf-8"))
      Response.ok("Ada")
  })

val response1 = basicRequest.get(uri"http://example.org/partial10").send()
// response1.body will be Right(10)

val response2 = basicRequest.post(uri"http://example.org/partialAda").send()

This approach to testing has one caveat: the responses are not type-safe. That is, the stub backend cannot match on or verify that the type of the response body matches the response body type requested.

Another way to specify the behaviour is passing response wrapped in the result monad to the stub. It is useful if you need to test a scenario with a slow server, when the response should be not returned immediately, but after some time. Example with Futures:

implicit val testingBackend = SttpBackendStub.asynchronousFuture
  .whenAnyRequest
  .thenRespondWrapped(Future {
    Thread.sleep(5000)
    Response(Right("OK"), StatusCode.Ok, "", Nil, Nil)
  })

val responseFuture = basicRequest.get(uri"http://example.org").send()

The returned response may also depend on the request:

implicit val testingBackend = SttpBackendStub.synchronous
  .whenAnyRequest
  .thenRespondWrapped(req =>
    Response(Right(s"OK, got request sent to ${req.uri.host}"), StatusCode.Ok, "", Nil, Nil)
  )

val response = basicRequest.get(uri"http://example.org").send()

You can define consecutive raw responses that will be served:

implicit val testingBackend:SttpBackendStub[Identity, Nothing, NothingT] = SttpBackendStub.synchronous
  .whenAnyRequest
  .thenRespondCyclic("first", "second", "third")

basicRequest.get(uri"http://example.org").send()       // Right("OK, first")       // Right("OK, first")
basicRequest.get(uri"http://example.org").send()       // Right("OK, second")       // Right("OK, second")
basicRequest.get(uri"http://example.org").send()       // Right("OK, third")       // Right("OK, third")
basicRequest.get(uri"http://example.org").send()       // Right("OK, first")

Or multiple Response instances:

implicit val testingBackend:SttpBackendStub[Identity, Nothing, NothingT] = SttpBackendStub.synchronous
  .whenAnyRequest
  .thenRespondCyclicResponses(
    Response.ok[String]("first"),
    Response("error", StatusCode.InternalServerError, "Something went wrong")
  )

basicRequest.get(uri"http://example.org").send()       // code will be 200       // code will be 200
basicRequest.get(uri"http://example.org").send()       // code will be 500       // code will be 500
basicRequest.get(uri"http://example.org").send()       // code will be 200

Simulating exceptions

If you want to simulate an exception being thrown by a backend, e.g. a socket timeout exception, you can do so by throwing the appropriate exception instead of the response, e.g.:

implicit val testingBackend = SttpBackendStub.synchronous
  .whenRequestMatches(_ => true)
  .thenRespond(throw new SttpClientException.ConnectException(new RuntimeException))

Adjusting the response body type

If the type of the response body returned by the stub’s rules (as specified using the .whenXxx methods) doesn’t match what was specified in the request, the stub will attempt to convert the body to the desired type. This might be useful when:

  • testing code which maps a basic response body to a custom type, e.g. mapping a raw json string using a decoder to a domain type
  • reading a classpath resource (which results in an InputStream) and requesting a response of e.g. type String

The following conversions are supported:

  • anything to () (unit), when the response is ignored
  • InputStream and Array[Byte] to String
  • InputStream and String to Array[Byte]
  • InputStream, String and Array[Byte] to custom types through mapped response specifications

Example: returning JSON

For example, if you want to return a JSON response, simply use .withResponse(String) as below:

implicit val testingBackend = SttpBackendStub.synchronous
  .whenRequestMatches(_ => true)
  .thenRespond(""" {"username": "john", "age": 65 } """)

def parseUserJson(a: Array[Byte]): User = ???

val response = basicRequest.get(uri"http://example.com")
  .response(asByteArrayAlways.map(parseUserJson))
  .send()

In the example above, the stub’s rules specify that a response with a String-body should be returned for any request; the request, on the other hand, specifies that response body should be parsed from a byte array to a custom User type. These type don’t match, so the SttpBackendStub will in this case convert the body to the desired type.

Note that no conversions will be attempted for streaming response bodies.

Example: returning a file

If you want to return a file and have a response handler set up like this:

val destination = new File("path/to/file.ext")
basicRequest.get(uri"http://example.com").response(asFile(destination))

Then set up the mock like this:

val fileResponseHandle = new File("path/to/file.ext")
SttpBackendStub.synchronous
  .whenRequestMatches(_ => true)
  .thenRespond(fileResponseHandle)

the File set up in the stub will be returned as though it was the File set up as destination in the response handler above. This means that the file from fileResponseHandle is not written to destination.

If you actually want a file to be written you can set up the stub like this:

import org.apache.commons.io.FileUtils
import cats.effect.IO
import sttp.client.impl.cats.implicits._
import sttp.client.monad.MonadError

val sourceFile = new File("path/to/file.ext")
val destinationFile = new File("path/to/file.ext")
SttpBackendStub(implicitly[MonadError[IO]])
  .whenRequestMatches(_ => true)
  .thenRespondWrapped { _ =>
    FileUtils.copyFile(sourceFile, destinationFile)
    IO(Response(Right(destinationFile), StatusCode.Ok, ""))
  }

Delegating to another backend

It is also possible to create a stub backend which delegates calls to another (possibly “real”) backend if none of the specified predicates match a request. This can be useful during development, to partially stub a yet incomplete API with which we integrate:

implicit val testingBackend =
  SttpBackendStub.withFallback[Identity,Nothing,Nothing,NothingT](HttpURLConnectionBackend())
    .whenRequestMatches(_.uri.path.startsWith(List("a")))
    .thenRespond("I'm a STUB!")

val response1 = basicRequest.get(uri"http://api.internal/a").send()
// response1.body will be Right("I'm a STUB")

val response2 = basicRequest.post(uri"http://api.internal/b").send()

Testing WebSockets

Stub methods whenRequestMatches and whenAnyRequest also allow specifying behavior when openWebsocket is called, using thenRespondWebSocket or thenHandleOpenWebSocket.

The first behavior, thenRespondWebSocket is best suited when the high-level websocket interface is used, i.e. when the client code manipulates WebSocket[F] to implement the websocket-handling logic.

In this case, the handler that is passed to the openWebsocket call is ignored, and the test double WebSocket implementation is returned.

For example:

import sttp.client.ws._
import zio.{App => _, _}
import sttp.client.asynchttpclient.zio._

val testWebSocket: WebSocket[Task] = ???

implicit val testingBackend =
  AsyncHttpClientZioBackend.stub
    .whenAnyRequest
    .thenRespondWebSocket(testWebSocket)

for {
  handler <- ZioWebSocketHandler() // using sttp's "high-level" websockets
  openResponse <- testingBackend.openWebsocket(basicRequest.get(uri"wss://some.uri"), handler)
  webSocket = openResponse.result // interaction with WebSocket is interesting in this case
  message <- webSocket.receive
} yield message

WebSocketStub

WebSocketStub allows easy creation of stub WebSocket instances. Such instances wrap a state machine that can be used to simulate simple WebSocket interactions. The user sets initial responses for receive calls as well as logic to add further messages in reaction to send calls. SttpBackendStub has a special API that accepts a WebSocketStub and builds a WebSocket instance out of it.

For example:

import sttp.model.ws._

val backend = SttpBackendStub.synchronous[Identity]
val webSocketStub = WebSocketStub
  .withInitialIncoming(
    List(WebSocketFrame.text("Hello from the server!"))
  )
  .thenRespondS(0) {
    case (counter, tf: WebSocketFrame.Text) => (counter + 1, List(WebSocketFrame.text(s"echo: ${tf.payload}")))
    case (counter, _)                       => (counter, List.empty)
  }

backend.whenAnyRequest.thenRespondWebSocket(webSocketStub)

There is a possiblity to add error responses as well. If this is not enough, using a custom implementation of the WebSocket trait is recommended.

Using the client-provided handler when testing WebSockets

Akka backend works a bit differently in regard to WebSockets. In this case the user defines a Flow[Message, Message, Mat] to define WebSocket client behavior.

The thenHandleOpenWebSocket method allows specifying stub behavior, which uses the given client-flow to simulate client-server interaction in tests.

When the websocket behavior is captured by the handler (as is the case when using Akka), we can use the value passed by the user to couple it with simulated server-side behavior, and run the whole interaction.

For example:

import akka.stream.scaladsl._
import akka.Done
import akka.http.scaladsl.model.ws._
import akka.stream._
import akka.actor.ActorSystem
import sttp.client.akkahttp._
import scala.concurrent.Await
import scala.concurrent.duration._

implicit val system = ActorSystem("sttp-ws-test")
implicit val materializer = ActorMaterializer()

// it should say Hi! and 42
val behaviorToTest: Flow[Message, Message, Future[Done]] = ???
// setup test double
var received = List.empty[String]
val testFlow: Flow[Message, Message, Future[Done]] => Future[Done] = clientFlow => {
  val ((outQueue, flowCompleted), inQueue) = Source
    .queue(1, OverflowStrategy.fail)
    .viaMat(clientFlow)(Keep.both)
    .toMat(Sink.queue())(Keep.both)
    .run()

  def recordInboundMessage: Future[Unit] =
    inQueue.pull().flatMap {
      case None      => recordInboundMessage
      case Some(msg) => Future.successful { received = msg.asTextMessage.getStrictText :: received }
    }

  (for {
    _ <- recordInboundMessage
    _ <- outQueue.offer(TextMessage("Give me a number"))
    _ <- recordInboundMessage
    _ = outQueue.complete()
    _ <- outQueue.watchCompletion()
  } yield ()).flatMap(_ => flowCompleted)
}

implicit val backend = AkkaHttpBackend.stub.whenAnyRequest
  .thenHandleOpenWebSocket(testFlow)

// code under test with test doubles
Await.ready(backend
  .openWebsocket(basicRequest.get(uri"wss://echo.websocket.org"), behaviorToTest)
  .flatMap(_.result), Duration.Inf)
// assertions can be performed
assert(received.reverse == List("Hi!", "42"))

Timeouts

sttp supports read and connection timeouts:

  • Connection timeout - can be set globally (30 seconds by default)
  • Read timeout - can be set per request (1 minute by default)

How to use:

import sttp.client._
import scala.concurrent.duration._

// all backends provide a constructor that allows to specify backend options
implicit val backend = HttpURLConnectionBackend(
  options = SttpBackendOptions.connectionTimeout(1.minute))

basicRequest
  .get(uri"...")
  .readTimeout(5.minutes) // or Duration.Inf to turn read timeout off
  .send()

SSL

SSL handling can be customized (or disabled) when creating a backend and is backend-specific.

Depending on the underlying backend’s client, you can customize SSL settings as follows:

  • HttpUrlConnectionBackend: when creating the backend, specify the customizeConnection: HttpURLConnection => Unit parameter, and set the hostname verifier & SSL socket factory as required
  • akka-http: when creating the backend, specify the customHttpsContext: Option[HttpsConnectionContext] parameter. See akka-http docs
  • async-http-client: create a custom client and use the setSSLContext method
  • OkHttp: create a custom client modifying the SSL settings as described on the wiki

Proxy support

sttp library by default checks for your System proxy properties (docs):

Following settings are checked:

  1. socksProxyHost and socksProxyPort (default: 1080)
  2. http.proxyHost and http.proxyPort (default: 80)
  3. https.proxyHost and https.proxyPort (default: 443)

Settings are loaded in given order and the first existing value is being used.

Otherwise, proxy values can be specified manually when creating a backend:

import sttp.client._

implicit val backend = HttpURLConnectionBackend(
  options = SttpBackendOptions.httpProxy("some.host", 8080))

basicRequest
  .get(uri"...")
  .send() // uses the proxy

Or in case your proxy requires authentication (supported by the JVM backends):

import sttp.client._

SttpBackendOptions.httpProxy("some.host", 8080, "username", "password")

Redirects

By default, sttp follows redirects.

If you’d like to disable following redirects, use the followRedirects method:

import sttp.client._

basicRequest.followRedirects(false)

If a request has been redirected, the history of all followed redirects is accessible through the response.history list. The first response (oldest) comes first. The body of each response will be a Left(message) (as the status code is non-2xx), where the message is whatever the server returned as the response body.

Redirecting POST requests

If a POST or PUT request is redirected, by default it will be sent unchanged to the new address, that is using the original body and method. However, most browsers and some clients issue a GET request in such case, without the body.

To enable this behavior, use the redirectToGet method:

import sttp.client._

basicRequest.redirectToGet(true)

Note that this only affects 301 Moved Permanently and 302 Found redirects. 303 See Other redirects are always converted, while 307 Temporary Redirect and 308 Permanent Redirect never.

Other Scala HTTP clients

Also, check the comparison by Marco Firrincieli on how to implement a simple request using a number of Scala HTTP libraries.