Mike Slinn
ScalaCourses.com

For-comprehensions With Futures

— Draft —

Published 2014-09-22. Last modified 2015-01-08.
Time to read: 4 minutes.

We first discussed for-comprehensions in the For-Loops and For-Comprehensions lecture earlier in this course. This lecture continues the discussion of Scala Futures by exploring how for-comprehensions work with Futures.

The sample code for this lecture is provided in courseNotes/src/main/scala/multi/futures/FutureFor.scala and uses the factorial and readUrlFuture methods introduced in previous lectures, found in courseNotes/src/main/scala/multi/package.scala.

To get the results of a Future without blocking, use a combinator like map and flatMap as we will discuss in this lecture and the next, or use a for-expression.

When a Future is used as a generator in a for-expression, the value of the successful Future will be assigned to the left-hand variable, and will remain available for the remainder of the for-expression. This is true for for-loops and for-comprehensions.

We'll discuss how to handle failed Futures in the next lecture. Let's first try printing out the first 200 characters of a URL using a for-loop and a Future.

Scala REPL
scala> for {
  value <- readUrlFuture("https://scalacourses.com", 200)
} println(s"For loop: value=$value")

<!DOCTYPE html>
<html lang="en">
<head>
  <title>Welcome to ScalaCourses.com</title>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  <script src='/webjars/jquery/1.9.1/jqu

Notice that the REPL prompt appeared, then a pause, then the output of the future appeared. This is not a bug.

Notice what happens with a for-comprehension instead of a for-loop:

Scala REPL
scala> for {
  value <- readUrlFuture("https://scalacourses.com", 200)
} yield value
res0: scala.concurrent.Future[String] = scala.concurrent.impl.Promise$DefaultPromise@5523cc24

As is the case with all monads, a for-comprehension returns the same monadic type as was supplied by the generators. We learned in the For-Loops and For-Comprehensions lecture that is how for-comprehensions and map/flatmap works. This is why a completed Future is returned from the for-comprehension, containing the value, implemented as a Promise subclass known as DefaultPromise. Inside the for-comprehension, the Future's value is available.

Let’s explore all of this in much more detail.

Sequential Execution

As a quick reminder, here is a for-comprehension that uses Lists instead of Futures:

Scala REPL
scala> val list: List[BigInt] = for {
  x <- List(factorial(123))
  y <- List(factorial(234)) if y/10>x
  z <- List(factorial(345)) x%2==BigInt(0)
} yield x + y + z
list=List(242156386507923 ...really long output...)

The expression to the right of the <- symbol, known as a generator, creates bindings for the variable on its left. This for-comprehension has 3 generators, one for each of x, y and z. If any generator fails the following generators will be suppressed. The definition of what constitutes failure is dependant on the monadic type of the generator.

Lists 'fail' when they consist of the empty list (Nil), so if x evaluates to Nil, the generators for y and z will not be computed and Nil will be yielded as the value of the for-comprehension. Similarly, if y evaluates to Nil or the guard y/10>x evaluates false, the generator for z will not be computed for that value of y and that iteration of the for-comprehension will not yield a value. Also, if z evaluates to Nil or x is an even number, the iteration for that value of z will not yield a value.

Here is another for-comprehension that uses Options:

Scala REPL
scala> val option: Option[BigInt] = for {
  x <- Option(factorial(123))
  y <- Option(factorial(234)) if y/10>x
  z <- Option(factorial(345)) if x%2==BigInt(0)
} yield x + y + z
option=Some(242156386507923 ...really long output...)

Options fail when they return None, so:

  • If x evaluates to None, the generators for y and z will not be computed and option will be set to Nil.
  • Similarly, if y evaluates to None or the guard y/10>x evaluates false, the generator for z will not be computed and the loop for that value of y will be suppressed. Once again, option will be set to Nil.
  • Finally, if z evaluates to None, or x is an even number, the iteration for that value of z will not yield a value, and option will be set to Nil.

Following is an example of a Scala for-comprehension that uses Futures. This for-comprehension evaluates each of the terms x, y and z sequentially. I'll show you a similar for-comprehension that computes the value of the terms in parallel shortly.

Scala REPL
scala> val futureResult: Future[BigInt] = for {
  x <- Future(factorial(123))
  y <- Future(factorial(234)) if y/10>x
  z <- Future(factorial(345)) x%2==BigInt(0)
} yield x + y + z
futureResult andThen {
  case Success(r) ⇒
    println(s"Success: $r")

  case Failure(ex) ⇒
    println(s"Failure: ${ex.getMessage}")
}
Future value=242156386507923 ...really long output...

This example is like the preceding example, but creates a future for each generator. Each generator runs sequentially! The generators do not run simultaneously! Using the futures in this way probably does not do what you want. See the Parallel Execution section below to see how to run the futures simultaneously.

As you know, Futures fail when they complete with a Throwable. The result of this for-comprehension is a new Future, called future1, containing the result of adding the values of the other Future values together after they each run to completion, one after the other.

If the Future that generates x fails, the Future that would have generated the value for y will not be invoked. Similarly, if the Future that generates y fails, the Future that would have generated the value for z will not be invoked. If any generator fails the yield statement will not be executed, and the result of the for-comprehension will be the Throwable from the failed Future wrapped inside a new failed Future.

We can run this program by typing:

Shell
$ sbt "runMain multi.futures.ForCompSerial1"
Success: 4272888918078 ... really long output

Here is another way to write the above, without the intermediate futureResult variable.

Scala code
(for {
    x <- Future(factorial(123))
    y <- Future(factorial(234)) if y/10>x
    z <- Future(factorial(345)) x%2==BigInt(0)
  } yield x + y + z) andThen {
    case Success(r) ⇒
      println(s"Success: $r")

    case Failure(ex) ⇒
      println(s"Failure: ${ex.getMessage}")
  }

Output is the same.

You can run this program by typing:

Shell
$ sbt "runMain multi.futures.ForCompSerial2"

Output is the same as the previous example.

Equivalent flatMap/map Code

The Scala compiler rewrites the above as follows; the complete program is provided as ForComp3. This version is somewhat simpler because it does not have any guards, which would be implemented via filterWith method invocations.

Scala code
Future(factorial(123)).flatMap { x ⇒
  Future(factorial(234)).flatMap { y ⇒
    Future(factorial(345)).map { z ⇒ x + y + z }
  }
} andThen {
  case Success(value) ⇒
    println(s"Success: $value")

  case Failure(ex) ⇒
    println(s"Failure: ${ex.getMessage}")
}

As you can see, the body of the yield statement in the for-comprehension is used as the body of a function passed to map, and the preceding assignments are used to form enclosing flatMaps. This formulation is probably a little harder to read, so I encourage you to use for-comprehensions instead of nested flatMaps / map.

You can run this program by typing:

Shell
$ sbt "runMain multi.futures.ForCompSerial3"

Parallel Execution

All that is necessary in order to cause multiple futures to be candidates for simultaneous execution is to initiate them prior to referencing them as generators in a for-comprehension. They will all execute as soon as free threads are available to be allocated to the Futures so they can begin evaluation.

Scala code
val futureX: Future[BigInt] = Future(factorial(123))
val futureY: Future[BigInt] = Future(factorial(234))
val futureZ: Future[BigInt] = Future(factorial(345))

val futureResult: Future[BigInt] = for {
  x <- futureX
  y <- futureY if y / 10 > x
  z <- futureZ if x % 2 == BigInt(0)
} yield x + y + z

futureResult andThen {
  case Success(value) ⇒
    println(s"Success: $value")

  case Failure(ex) ⇒
    println(s"Failure: ${ex.getMessage}")
}

You can run this program by typing:

Shell
$ sbt "runMain multi.futures.ForCompParallel"

Output is the same as was produced by the previous examples.

Previous lecture:

Next lecture:
* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.