Published 2014-04-13.
Last modified 2016-10-24.
Time to read: 7 minutes.
This lecture discusses short examples and exercises of for-loops and for-comprehensions.
This lecture provides short examples and exercises for the For-Loops and For-Comprehensions lecture.
The sample code for this lecture can be found in
courseNotes/
.
Exercise – Sum of Case Class Property Matching a Criteria
The case class Trade
has three properties:
case class Trade(porfolio: String, price: Double, risk: String)
We can make a List
of specific Trade
instances:
val trades = List( Trade("fund1", 123.4, "AA"), Trade("fund2", 234.5, "A"), Trade("fund3", -23.45, "A") )
Your task is to use a for-comprehension to select only those trades that have risk "A"
, and
sum
the price
.
Solution
scala> case class Trade(porfolio: String, price: Double, risk: String) defined class Trade
scala> val trades = List( | Trade("fund1", 123.4, "AA"), | Trade("fund2", 234.5, "A"), | Trade("fund3", -23.45, "A") | ) trades: List[Trade] = List(Trade(fund1,123.4,AA), Trade(fund2,234.5,A), Trade(fund3,-23.45,A))
First let’s walks through all the items in trades
and select the items that have the risk property set to "A"
.
scala> for { trade <- trades if trade.risk=="A" } yield trade.price res0: List[Double] = List(234.5, -23.45)
Now let’s sum the resulting List
.
scala> (for { trade <- trades if trade.risk=="A" } yield trade.price).sum res1: Double = 211.05
for-Comprehension of Trys
This example takes advantage of two facts:
-
The value returned by a for-comprehension is wrapped in the same type as the generators were, or a common supertype.
This means that the body of the
yield
statement below is wrapped in aTry
, even though you did not explicitly specify it. If you thought there were only twoTry
invocations in this code example, you were wrong: there are three! -
Within a
for
yield
statement, or the body of afor
loop, the generated values are directly available (they are unwrapped from the monadic container within the for-loop body).
scala> import util.Try import util.Try
scala> def tryCompute(a: Int, b: Int): Try[Int] = Try { a / b } tryCompute: (a: Int, b: Int)scala.util.Try[Int]
scala> def tryFor(w: Int, x: Int, y: Int, z: Int): Try[Int] = for { | result1 <- tryCompute(w, x) | result2 <- tryCompute(y, z) | } yield result1 / result2 tryFor: (w: Int, x: Int, y: Int, z: Int)scala.util.Try[Int]
scala> tryFor(2, 1, 4, 2) res0: scala.util.Try[Int] = Success(1)
scala> tryFor(2, 0, 4, 2) // fails in first generator res2: scala.util.Try[Int] = Failure(java.lang.ArithmeticException: / by zero)
scala> tryFor(2, 1, 4, 0) // fails in second generator res2: scala.util.Try[Int] = Failure(java.lang.ArithmeticException: / by zero)
scala> tryFor(0, 1, 0, 2) // fails in yield body res1: scala.util.Try[Int] = Failure(java.lang.ArithmeticException: / by zero)
To run this code example, type:
$ sbt "runMain ForTry"
Wish You Were Here!
This example uses a for-comprehension to simulate travelers sending postcards from various cities to relatives.
The domain model simply consists of a Postcard
case class.
There are 3 lists: locations
, relatives
and travellers
.
A method called writePostCards
creates a List[String]
by explicitly permuting the lists to
create a Postcard
for each combination, and then invoking Postcard.generate
on each Postcard
.
The resulting strings are then concatenated with a newline inserted between them by invoking mkString
.
object ForFun2 extends App { case class Postcard(state: String, from: String, to: String) { def generate: String = s"Dear $to,\n\nWish you were here in $state!\n\nLove, $from\n\n" }
val locations = List("Bedrock", "Granite City") val relatives = List("Barney", "Betty") val travellers = List("Wilma", "Fred")
def writePostCards(locations: List[String], travellers: List[String], relatives: List[String]): List[String] = for { sender <- travellers recipient <- relatives state <- locations } yield Postcard(state, sender, recipient).generate
val postcards: List[String] = writePostCards(locations, travellers, relatives) val output = postcards.mkString("\n") println(output) }
You can run this example by typing:
$ sbt "runMain ForFun2" Dear Barney,
Wish you were here in Bedrock!
Love, Wilma
Dear Barney,
Wish you were here in Granite City!
Love, Wilma
Dear Betty,
Wish you were here in Bedrock!
Love, Wilma
Dear Betty,
Wish you were here in Granite City!
Love, Wilma
Dear Barney,
Wish you were here in Bedrock!
Love, Fred
Dear Barney,
Wish you were here in Granite City!
Love, Fred
Dear Betty,
Wish you were here in Bedrock!
Love, Fred
Dear Betty,
Wish you were here in Granite City!
Love, Fred
Pulling Apart a List of Maps
This code is provided in the ForFun1
object in
courseNotes/
.
Let’s use a for-comprehension to pick apart a List
of Map
s,
similar to what you might have to contend with when receiving a JSON response from a web service.
First we will create some test data:
scala> val selectedKeys = Map("selectedKeys" -> Seq("one", "two", "three")) selectedKeys: scala.collection.immutable.Map[String,Seq[String]] = Map(selectedKeys -> List(one, two, three))
scala> val otherKeys = Map("otherKeys" -> Seq("four", "five")) otherKeys: scala.collection.immutable.Map[String,Seq[String]] = Map(otherKeys -> List(four, five))
scala> val list: List[Map[String, Seq[String]]] = List(selectedKeys, otherKeys) list: List[Map[String,Seq[String]]] = List(Map(selectedKeys -> List(one, two, three)), Map(otherKeys -> List(four, five)))
The test data is now ready in list
.
Curly braces must be used to enclose the generators if the for-comprehension contains more than one generator.
Notice that the second and third generators are converted to List
s.
This is done so result
will also be a List
.
scala> val result = for { | data <- list | selectedKeysSeq <- data.get("selectedKeys").toList | id <- selectedKeysSeq.toList | } yield id scala> res6: List[String] = List(one, two, three)
Let’s rewrite this, showing types, to clarify what is happening:
val result2: List[String] = for { data: Map[String, Seq[String]] <- list selectedKeysSeq: Seq[String] <- data.get("selectedKeys").toList id: String <- selectedKeysSeq.toList } yield id
Let’s break this down:
-
The line
data <- list
causes a looping action to be invoked on each element oflist
. A temporary variable calleddata
of typeMap[String, Seq[String]]
is created for each loop iteration, and it receives the current element oflist
. -
The next line (
selectedKeysSeq <- data.get("selectedKeys").toList
) pulls out the value associated with the key"selectedKeys"
from thedata
map, which isSeq("one", "two", "three")
and starts an inner loop. Each element of theSeq
is assigned to a new temporary variable calledselectedKeysSeq
. -
The next line (
id <- selectedKeysSeq.toList
) iterates through each item inselectedKeysSeq
and assigns it to a temporary variable calledid
. -
The last line forms a new collection consisting of each of the values of that
id
had assumed. The collection type is determined by the collection type of each of the previous statements, which must be compatible. In this case, they are allList
s.
So, this means that equivalent code is:
list.flatMap { data: Map[String, Seq[String]] => data.get("selectedKeys").toList.flatMap { selectedKeysSeq: Seq[String] => selectedKeysSeq } }
Without types it would look like this:
list.flatMap { data => data.get("selectedKeys").toList.flatMap { selectedKeysSeq => selectedKeysSeq } }
Which form would you rather write?
Exercise – Transforming groupBy Output
The sample code below defines a Map
created by a groupBy
:
scala> val map = Map( "selectedKeys" -> Seq("one", "two", "three"), "otherKeys" -> Seq("two", "one") ) map: scala.collection.immutable.Map[String,Seq[String]] = Map(selectedKeys -> List(one, two, three), otherKeys -> List(four, five))
scala> val inversion = map.groupBy(_._1) inversion: scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,Seq[String]]] = HashMap(otherKeys -> Map(otherKeys -> List(two, one)), selectedKeys -> Map(selectedKeys -> List(one, two, three)))
How can you transform inversion
into the following?
Map[String,Seq[String]]] = Map(otherKeys -> List(two, one), selectedKeys -> List(one, two, three))
Solutions
This solution is clumsy.
inversion.map { x => val value = x._2(x._1) x._1 -> value }
This solution is more elegant.
inversion.values.flatten.toMap
Let’s consider how the second solution works.
In the following, squint a little at the returned containers.
It will probably seem a bit confusing, because Scala will sometimes return a collection trait instead of the collection type.
First let’s notice inversion
’s type (Map[String, Map[String, Seq[String]]]
).
In other words, inversion is a Map
of Map
s,
and the innermost Map
is a Map
of String
and an ordered collection of String
.
scala> inversion res3: scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,Seq[String]]] = HashMap(otherKeys -> Map(otherKeys -> List(two, one)), selectedKeys -> Map(selectedKeys -> List(one, two, three)))
Now we pull out all the values.
Notice that the returned type is shown as an Iterable
,
which is a superclass of MapOps
.
scala> inversion.values res0: Iterable[scala.collection.immutable.Map[String,Seq[String]]] = View(<not computed>)
You could force a List
if you like; this would not affect the result:
scala> inversion.values.toList res1: List[scala.collection.immutable.Map[String,Seq[String]]] = List(Map(otherKeys -> List(two, one)), Map(selectedKeys -> List(one, two, three)))
Now we collapse each inner container – thereby transforming the inner Map
s into tuples.
This results in a collection of tuples:
scala> inversion.values.flatten res2: Iterable[(String, Seq[String])] = View(<not computed>)
Here we can see the forced type produces the same result, although the types are again different.
scala> inversion.values.toList.flatten res3: List[(String, Seq[String])] = List((otherKeys,List(two, one)), (selectedKeys,List(one, two, three)))
Now transform the collection of tuples into a Map
:
scala> inversion.values.flatten.toMap res4: scala.collection.immutable.Map[String,Seq[String]] = Map(otherKeys -> List(two, one), selectedKeys -> List(one, two, three))
Finally, the forced type returns exactly the same result:
scala> inversion.values.toList.flatten.toMap res5: scala.collection.immutable.Map[String,Seq[String]] = Map(otherKeys -> List(two, one), selectedKeys -> List(one, two, three))
These solutions are provided in
courseNotes/
.
You can run these solutions by typing:
$ sbt "runMain solutions.GroupBy"
Exercise – Complete This Program
This exercise exposes you to a more comprehensive Scala program, longer than most of the code examples we have previously seen. It brings together many concepts into one program.
Your task is to fill in the blanks of this incomplete program, provided as ResourceAuthorization.scala
.
It builds a collection of users that require collection of resources, and also builds collections of resources that specific users can access.
It then creates some users and resources, and assigns access for specific users to specific resources.
The program concludes by printing out the results of several authentication attempts.
Notice how isUserAuthorized
is written using a for-comprehension.
You should learn this Scala idiom.
If authenticate
fails, then user
does not receive a value and the remainder of the for-comprehension terminates.
Similarly, if findResourceById
fails, resource
is not assigned a value and the yield
statement does not execute.
Thus the only way that token
will not receive the value None
is if isUserAuthorized
and findResourceById
both succeed and return Some[User]
and Some[Resource]
, respectively.
package solutions
object ResourceAuthorization extends App { import collection._ import java.util.Date
case class User(name: String, password: String, userId: String) case class Resource(name: String, id: Long) case class AuthorizationToken(user: User, resource: Resource, expires: Date=new Date(System.currentTimeMillis))
val users = mutable.HashMap.empty[String, User] val resources = mutable.HashMap.empty[Long, Resource] val resourceAccess = mutable.HashMap.empty[User, List[Resource]]
/** Add the given user to the users HashMap */ def add(user: User): User = { // what goes here? user }
/** Add the given resource to the resources HashMap */ def add(resource: Resource): Resource = { span class='bg_yellow'>// what goes here? resource }
/** Use a for-comprehension to write this method * @return Some User if the user with the given userId and password is returned by findUserById. */ def authenticate(userId: String, password: String): Option[User] = ???
/** Add the given resource to the list of resources assigned to the user in resourceAccess */ def authorize(user: User, resource: Resource): Unit = ???
/** @return Some[Resource] if an entry for resourceId is found in resources */ def findResourceById(resourceId: Long): Option[Resource] = ???
/** @return Some[User] if an entry for userId is found in users */ def findUserById(userId: String): Option[User] = ???
def isUserAuthorized(userId: String, password: String, resourceId: Long): Option[AuthorizationToken] = { val token = for { user <- authenticate(userId, password) resource <- findResourceById(resourceId) } yield AuthorizationToken(user, resource) token }
val fred = add(User("Fred Flintstone", "Yabbadabbadoo", "fflintstone")) val wilma = add(User("Wilma Flintstone", "Friendship", "wflintstone")) val barney = add(User("Barney Rubble", "Huyuk", "brubble")) val betty = add(User("Betty Rubble", "Youtoo", "betty"))
val work = add(Resource("Work Cave", 1)) val flintstoneHome = add(Resource("Flintstone home", 2)) val rubbleHome = add(Resource("Rubble home", 3))
authorize(fred, flintstoneHome) authorize(wilma, flintstoneHome) authorize(fred, work) authorize(barney, rubbleHome) authorize(betty, rubbleHome) authorize(barney, work)
/** For-comprehensions would not be as convenient here */ def report(userId: String, password: String, resourceId: Long): Unit = ???
report("fflintstone", "Yabbadabbadoo", 1) report("betty", "Youtoo", 3) report("fflintstone", "ForgotPassword", 1) report("nobody", "nopass", 9) }
Output from report
should be as follows.
Notice that user names are displayed if possible, otherwise the user id is displayed.
Also, resource names are displayed if possible, otherwise the resource id is displayed.
Fred Flintstone and password Yabbadabbadoo can access resource Work Cave until Sat Apr 19 11:39:30 PDT 2014. Betty Rubble and password Youtoo can access resource Rubble home until Sat Apr 19 11:39:30 PDT 2014. Fred Flintstone and password ForgotPassword can not access resource Work Cave. User with id nobody and password nopass can not access resource with Id 9.
Hint
The report method should be written using for-loop that employs a series of generators which use the map/orElse
idiom.
Solution
/** Add the given user to the users HashMap */ def add(user: User): User = { users.put(user.userId, user) user } /** Add the given resource to the resources HashMap */ def add(resource: Resource): Res = { resources.put(resource.id, resource) resource } /** Use a for-comprehension to write this method * @return Some User if the user with the given userId and password is returned by findUserById. */ def authenticate(userId: String, password: String): Option[User] = for { user <- findUserById(userId) if user.password == password } yield user
Given a User
and a Resource
, Authorize
adds the resource to the user’s authorized list of resources.
/** Add the given resource to the list of resources assigned to the user in resourceAccess */ def authorize(user: User, resource: Resource): Unit = { val oldUserResources: List[Resource] = resourceAccess.getOrElse(user, Nil) val updatedUserResources: List[Resource] = resource +: oldUserResources resourceAccess.put(user, updatedUserResources) }
As discussed in the
Collections Overview lecture,
writing resourceAccess.getOrElse(user, Nil)
is preferred over writing resourceAccess.get(user).getOrElse(Nil)
.
Recall that the Immutable Collections lecture introduced the +:
operator for lists;
it appends a new item to an existing list.
Methods whose name ends in a colon are right-associative, so this could be rewritten as oldUserResources.+:(resource)
.
/** @return Some[Resource] if an entry for resourceId is found in resources */ def findResourceById(resourceId: Long): Option[Resource] = resources.get(resourceId) /** @return Some[User] if an entry for userId is found in users */ def findUserById(userId: String): Option[User] = users.get(userId)
Output is formatted here.
This for-loop uses the map/orElse
idiom.
/** Format output */ def report(userId: String, password: String, resourceId: Long): Unit = { for { resourceName <- findResourceById(resourceId).map(_.name).orElse(Some(s"resource with Id $resourceId")) userName <- findUserById(userId).map(_.name).orElse(Some(s"User with id $userId")) msg <- isUserAuthorized(userId, password, resourceId).map { authToken => s"$userName and password $password can access resource $resourceName until ${authToken.expires}." }.orElse { Some(s"$userName and password $password can not access resource $resourceName.") } } println(msg) }
To run this solution, type:
$ sbt "runMain solutions.ResourceAuthorization"
Either Became Right-Biased With Scala 2.12
We learned about Either
and its subclasses Left
and Right
in the
Either, Left and Right lecture of the
Introduction to Scala course.
Prior to Scala 2.12, Either
was not right-biased, which means that neither the Left
nor
the Right
subclasses was favored.
As a result, Either
did not provide the necessary methods to support monadic operations such as
flatMap
, map
, filter
, etc, although the Left
and Right
projections did.
For a given Either
instance, you could write for-expressions and for-comprehensions that specified the appropriate bias (left or right).
Starting with Scala 2.12 you can also use the default right-bias.
To understand how Either’s right bias works, and why it is dangerous, lets work through a few examples.
First we define an immutable variable of type Either[String, Int]
.
scala> val e: Either[String, Int] = Right(1) e: Either[String,Int] = Right(1)
Prior to Scala 2.12, Either
was unbiased, so map
and flatMap
were not supported and therefor for-comprehensions could not be written.
scala> for (x <- e) yield x <console>:13: error: value map is not a member of Either[String,Int] for (x <- e) yield x
Scala 2.12+ yields a result instead of throwing an exception:
scala> for (x <- e) yield x res0: scala.util.Either[String,Int] = Right(1)
The right
and left
properties are of type Right
and Left
, respectively.
Those types always were monads and therefore could be transformed by map
, flatMap
and for-comprehensions.
scala> for (x <- e.right) yield x res5: scala.util.Either[String,Int] = Right(1)
scala> for (x <- e.left) yield x res6: scala.util.Either[String,Int] = Right(1)
Short-Circuiting
Starting with Scala 2.12, Either
became a right-biased monad,
which means that the Right
value is transformed by map
,
flatMap
and for-comprehensions, and the Left
value is used to short-circuit the iteration.
This is similar to how Option
works: a Some
value is transformed by map
,
flatMap
and for-comprehensions, and a None
value short-circuits iteration.
To show you how this works, here are three instances of Either
, one Right
and two Left
s.
scala> val a: Either[Int, Int] = Right(1) a: Either[Int,Int] = Right(1)
scala> val b: Either[Int, Int] = Left(2) b: Either[Int,Int] = Left(2)
scala> val c: Either[Int, Int] = Left(3) c: Either[Int,Int] = Left(3)
scala> val d: Either[Int, Int] = Right(4) d: Either[Int,Int] = Right(4)
First we can loop over a generator that returns a Right
;
the result of for-expression is also a Right
,
and the yield expression is evaluated for each value of the generator.
scala> val r1 = for { x <- a } yield x r1: scala.util.Either[Int,Int] = Right(1)
Next we can introduce an inner loop, whose generator is a Left
; the resulting for-expression is also a Left
.
The yield statement is also evaluated for each value of the two generators.
scala> val r2 = for { x <- a y <- b } yield y r2: scala.util.Either[Int,Int] = Left(2)
Now another inner loop is introduces whose generator is also a Left
;
again, the resulting for-expression is also a Left
.
scala> val r3 = for { x <- a y <- b z <- c } yield z r3: scala.util.Either[Int,Int] = Left(2)

Danger, Will Robinson!
The first generator that returns a Left
stops for for-expression from looping through any remaining generators,
and establishes the values of the for-expression intermediate variables in the yield
expression.
Thus b
, with value Left(2)
, causes all remaining generators to also return the values of b
.
scala> val r4 = for { x <- a y <- b z <- c w <- d } yield x + y*10 + z*100 + w*1000 r4: scala.util.Either[Int,Int] = Left(2)
With right-biased Either, performing a computation in the body of a for-comprehension is dangerous because variables may have unexpected values when a generator short-circuits, and there is no way of detecting this inside the for-comprehension. This problem manifests even when the computation is performed in the loop.
scala> val r5 = for { x <- a y <- b z <- c w <- d result = x + y*10 + z*100 + w*1000 } yield result r5: scala.util.Either[Int,Int] = Left(2)
Once the Left
value is returned the situation is easily detectable.
One way to process the result is with a partial function, discussed in the
Partial Functions lecture.
scala> r5 match { case Left(trash) => println(s"Failure: $trash") case Right(value) => println(s"Success: $value") } Failure: 2
fold
is often the preferred way to handle an Either
;
notice that the failure case is handled first, followed by the success case.
scala> r5.fold( trash => println(s"Success: $trash"), value => println(s"Failure: $value") ) Failure: 2
swap
Either
’s swap method returns a copy of the Either
instance with the left and right properties swapped.
scala> for (x <- e.swap.left) yield x res3: scala.util.Either[Int,String] = Left(1)
scala> for (x <- e.swap.right) yield x res4: scala.util.Either[Int,String] = Left(1)
left and right projections
In summary: do not use Either
’s left
or right
projections in a
for-expression unless .toOption
is used.
Otherwise, the behavior of Either
’s right
and left
projections is probably not what you want.
To see why, let’s test just one Either
instance.
scala> val r1a = for { | x <- a.right | } yield x r1a: Product with Serializable with scala.util.Either[Int,Int] = Right(1)
scala> val r1b = for { | x <- a.left | } yield x r1b: Product with Serializable with scala.util.Either[Int,Int] = Right(1)
Now two Either
instances:
scala> val r2a = for { | x <- a.right | y <- b.right | } yield y r2a: scala.util.Either[Int,Int] = Left(2)
scala> val r2b = for { | x <- a.right | y <- b.left | } yield y r2b: scala.util.Either[Int,Int] = Left(2)
scala> val r2c = for { | x <- a.left | y <- b.right | } yield y r2c: scala.util.Either[Int,Int] = Right(1)
scala> val r2d = for { | x <- a.left | y <- b.left | } yield y r2d: scala.util.Either[Int,Int] = Right(1)
Now with all three Either
instances:
scala> val r3a = for { | x <- a.right | y <- b.right | z <- c.right | } yield z r3a: scala.util.Either[Int,Int] = Left(2)
scala> val r3b = for { | x <- a.right | y <- b.right | z <- c.left | } yield z r3b: scala.util.Either[Int,Int] = Left(2)
scala> val r3c = for { | x <- a.right | y <- b.left | z <- c.right | } yield z r3c: scala.util.Either[Int,Int] = Left(3)
scala> val r3d = for { | x <- a.right | y <- b.left | z <- c.left | } yield z r3d: scala.util.Either[Int,Int] = Left(3)
scala> val r3e = for { | x <- a.left | y <- b.right | z <- c.right | } yield z r3e: scala.util.Either[Int,Int] = Right(1)
scala> val r3f = for { | x <- a.left | y <- b.right | z <- c.left | } yield z r3f: scala.util.Either[Int,Int] = Right(1)
scala> val r3g = for { | x <- a.left | y <- b.left | z <- c.right | } yield z r3g: scala.util.Either[Int,Int] = Right(1)
scala> val r3h = for { | x <- a.left | y <- b.left | z <- c.left | } yield z r3h: scala.util.Either[Int,Int] = Right(1)
Now let’s try with toOption
.
This behavior is much more intuitive.
scala> val r4a = for { | x <- a.right.toOption | } yield x r4a: Option[Int] = Some(1)
scala> val r4b = for { | x <- a.left.toOption | } yield x r4b: Option[Int] = None
scala> val r4c = for { | x <- a.right.toOption | y <- b.right.toOption | } yield y r4c: Option[Int] = None
scala> val r4d = for { | x <- a.right.toOption | y <- b.left.toOption | } yield y r4d: Option[Int] = Some(2)
scala> val r4e = for { | x <- a.left.toOption | y <- b.right.toOption | } yield y r4e: Option[Int] = None
scala> val r4f = for { | x <- a.left.toOption | y <- b.left.toOption | } yield y r4f: Option[Int] = None
You can run this example by typing:
$ sbt "runMain EitherFun"
© Copyright 1994-2024 Michael Slinn. All rights reserved.
If you would like to request to use this copyright-protected work in any manner,
please send an email.
This website was made using Jekyll and Mike Slinn’s Jekyll Plugins.