Scala – sort/sortWith/sortBy functions

In this post, we are going to see how we can use scala sort related function with examples.

Assume that we have a list of fruits and we want to sort the fruit name in different order.

Refer the below code to know how to do that.



object SortExample {


  def main(args: Array[String]): Unit = {

    val fruits = List("Water Melon", "Banana", "Carrot", "Apple", "Plums", "Orange")

    //To sort a list by ascending order
    println(fruits.sorted)

    //To sort a list by descending order
    println(fruits.sorted.reverse)

    //To sort a list by length
    println(fruits.sortWith(_.length > _.length))

    //To sort a list by length(descending)
    println(fruits.sortWith(_.length < _.length))

  }
}

The output of the above code is given below,


List(Apple, Banana, Carrot, Orange, Plums, Water Melon)
List(Water Melon, Plums, Orange, Carrot, Banana, Apple)
List(Water Melon, Banana, Carrot, Orange, Apple, Plums)
List(Apple, Plums, Banana, Carrot, Orange, Water Melon)

Now, We will see how we can use sortBy function to sort a list.

The definition of the sortBy function is given below,



def sortBy[B](f: (A) ⇒ B)(implicit ord: math.Ordering[B]): List[A]
Sorts this Seq according to the Ordering which results from transforming an implicitly given Ordering with a transformation function.


Refer the below code, where we use sortBy function to sort the employee objects list in three different ways.


object SortByExample {

  case class Employee(name: String,
                      age: Int,
                      designation: String,
                      salary: Int)

  def main(args: Array[String]): Unit = {

    val employees = List(Employee("Bala", 31, "Developer", 500000),
      Employee("John", 33, "Tester", 200000),
      Employee("Bob", 35, "Tech Lead", 300000),
      Employee("Woods", 30, "Manager", 400000),
      Employee("Robert", 30, "Tester", 200000),
      Employee("Jon", 30, "Tester", 100000),
      Employee("Paul", 26, "Sales Person", 100000),
      Employee("Jason", 25, "Sales Executive", 100000),
      Employee("Jill", 25, "Sales Executive", 200000),
      Employee("Steve", 27, "Product Owner", 500000))

    employees
      .sortBy(_.age)
      .foreach(println(_))

    employees
      .sortBy(x => (x.salary, x.age))
      .foreach(println(_))

    val orderForDesignation = Map("Product Owner" -> 0, "Manager" -> 1, "Tech Lead" -> 2, "Developer" -> 3, "Tester" -> 4)

    employees
      .sortBy(x => (orderForDesignation.get(x.designation).orElse(Some(10)), x.name))
     .foreach(println(_))

  }

}

We have Employee case class and it contains name, salary, designation and age fields and we create a list with different details, then we call sortBy function and pass the age field to sort the list by age.

The output looks like below,


Employee(Jason,25,Sales Executive,100000)
Employee(Jill,25,Sales Executive,200000)
Employee(Paul,26,Sales Person,100000)
Employee(Steve,27,Product Owner,500000)
Employee(Woods,30,Manager,400000)
Employee(Robert,30,Tester,200000)
Employee(Jon,30,Tester,100000)
Employee(Bala,31,Developer,500000)
Employee(John,33,Tester,200000)
Employee(Bob,35,Tech Lead,300000)

Now we want to sort the list by salary and then age. To do that we have to pass the salary and age fields to sortBy function in the right order.
The output looks like below,


Employee(Jason,25,Sales Executive,100000)
Employee(Paul,26,Sales Person,100000)
Employee(Jon,30,Tester,100000)
Employee(Jill,25,Sales Executive,200000)
Employee(Robert,30,Tester,200000)
Employee(John,33,Tester,200000)
Employee(Bob,35,Tech Lead,300000)
Employee(Woods,30,Manager,400000)
Employee(Steve,27,Product Owner,500000)
Employee(Bala,31,Developer,500000)


Now, we want to sort it by designation but follow a different order. The order should be like this “Product Owner -> Manager -> Tech Lead -> Developer -> Tester.

To do that we have to maintain a Map contains designation to order mapping and this Map to be used in sortBy function and we also use name as secondary sorting.

The output looks like below,


Employee(Steve,27,Product Owner,500000)
Employee(Woods,30,Manager,400000)
Employee(Bob,35,Tech Lead,300000)
Employee(Bala,31,Developer,500000)
Employee(John,33,Tester,200000)
Employee(Jon,30,Tester,100000)
Employee(Robert,30,Tester,200000)
Employee(Jason,25,Sales Executive,100000)
Employee(Jill,25,Sales Executive,200000)
Employee(Paul,26,Sales Person,100000)


Advertisements

Scala – Currying function

A Currying function is a function which has multiple parameter list but can be called with few arguments and yields another function which accepts the missing parameters as its arguments.

Consider that we have a list names and want to add prefix and suffix to each name. Let’s see how we can do this with a curried function.

Scala-Currying

In the above example, the function “addPrefixSuffix” is a curried function which accepts three arguments such as prefix, suffix and name. So in the “main” function, I pass only two arguments to this so it will yield another function to accept the missing parameter(name).

Scala – Higher Order Function

Higher order Function is a functional programming principle that takes other functions as arguments or returns a function as output.

In this post, we are going to see that with an example.

Assume that we have a list of string and want to iterate it and append and prepend $ symbol to each value and collect the modified values as list. Refer the below example to know how we can do that with the higher-order function principle.

Scala Higher order function

In the above example, “modifyList” is a higher order function as it takes the “format” function as an argument. The “format” function is performing the append and prepend operation to each value. This example is pretty simple one and we can do it without a higher order function.

‘Library Fine’ problem in Hackerrank – solution in Scala

Problem Statement:

The Head Librarian at a library wants you to make a program that calculates the fine for returning the book after the return date. You are given the actual and the expected return dates. Calculate the fine as follows:

If the book is returned on or before the expected return date, no fine will be charged, in other words fine is 0.

If the book is returned in the same month as the expected return date, Fine = 15 Hackos × Number of late days

If the book is not returned in the same month but in the same year as the expected return date, Fine = 500 Hackos × Number of late months

If the book is not returned in the same year, the fine is fixed at 10000 Hackos.

Refer the below link to know more about this problem. https://www.hackerrank.com/challenges/library-fine

Solution in Scala


import java.util.{Calendar, Scanner}

object Solution {

    def main(args: Array[String]) {
       val scanner: Scanner = new Scanner(System.in)

    //Actual Returned Date
    val aDate: Int = scanner.nextInt
    val aMonth: Int = scanner.nextInt
    val aYear: Int = scanner.nextInt

    //Due Date
    val dDate: Int = scanner.nextInt
    val dMonth: Int = scanner.nextInt
    val dYear: Int = scanner.nextInt

    val isValidData: Boolean = ((aDate >= 1 && aDate = 1 && dDate = 1 && aMonth = 1 && dMonth = 1 && aYear = 1 && dYear <= 3000))

    var fineAmount: Int = 0
    if (isValidData) {
      val actualCalendar: Calendar = Calendar.getInstance()
      actualCalendar.set(aYear, aMonth, aDate)

      val dCalendar: Calendar = Calendar.getInstance()
      dCalendar.set(dYear, dMonth, dDate)

      if ((actualCalendar.getTime == dCalendar.getTime) || actualCalendar.getTime.before(dCalendar.getTime)) {
        fineAmount = 0
      }
      else if (actualCalendar.getTime.after(dCalendar.getTime) && aYear == dYear) {
        fineAmount = if ((aMonth == dMonth)) 15 * (aDate - dDate) else 500 * (aMonth - dMonth)
      }
      else {
        fineAmount = 10000
      }
    }
    println(fineAmount)
    }
}

Scala: Enum Creation Examples

Enum is used for creating a group of constants like the days of week, colors and etc.

In this post, I am going to show how to create enums in Scala.

There are two ways,
1. Using Scala Enumeration
2. Using Scala Traits

Using Scala Enumeration

Scala has Enumerations class which can be extended to create an Enum. Check the below example.


object ScalaEnumObject {

  def main(args: Array[String]) {
    println("Event:")
    Event.values foreach println
  }
  //Extends Enumeration class
  object Event extends Enumeration {
    type Event = Value
    val CREATE, READ, UPDATE, REMOVE = Value
  }
}

The output looks below,


Event:
CREATE
READ
UPDATE
REMOVE

Using Scala Traits
We can use Trait to create Enums. Trait is similar to Java interface. So we can create a Trait and then create the case object which extends that Trait for each Enum values.

Check the below example.


object ScalaEnumObject {

  def main(args: Array[String]) {
    println("Directions:")
    val directions = List(EAST,WEST,SOUTH,NORTH)
    directions.foreach(direction => println (direction.name))
  }

   //Using Trait
  sealed trait Direction {
    def name: String
  }

  case object EAST extends Direction {
    val name = "E"
  }

  case object WEST extends Direction {
    val name = "W"
  }

  case object SOUTH extends Direction {
    val name = "S"
  }

  case object NORTH extends Direction {
    val name = "N"
  }
}

The output looke below,


Directions:
E
W
S
N

Web app and RESTful services using MEAN stack and back end with Spark + Scala

I have developed a MEAN stack application which shows the San Francisco Food inspections details.
Source: Food Inspection(Use Food Inspections – LIVES Standard)

I have used Spark, Scala, MongoDB, NodeJs, AngularJs to do this.

My spark job reads the input CSV data contains food inspection details and processes it and stores the data in MongoDB as collections. I have allFoodInspection and filterFoodInspection collections here. The first one has all the data and the second one has the business name, the unique risk category and number of risk’s committed.

My MEAN stack REST layer reads the data from Mongodb and processes and exposes the data and the Web Layer uses the data and display it and use the data for drawing a chart.

Let us see how we can execute this.

  1. Follow the steps given in this post to install scala, sbt and spark in your machine if you are using Ubuntu. Refer my another post to know how to install these. How to install Scala, SBT and Spark in Ubuntu
  2. Clone the git repository https://github.com/dkbalachandar/sf-food-inspection-spark.git and go inside of sf-inspection-spark folder and run ‘sbt assembly’ to create a far jar with all the dependencies.Here I have used spark 2.0.2 and scala 2.11.8 (Spark 2.0.2 version is compatible with scala 2.11.x version).
    If you don’t use the compatible version then you will end up with lots of errors.
  3. Copy the ../sf-food-inspection-spark/target/scala-2.11/sf-food-inspection-spark-assembly-1.0.jar to /usr/local/spark folder
  4. Download Food_Inspections_-_LIVES_Standard.csv from https://data.sfgov.org/browse?q=food+inspection and move it to /usr/local/spark folder
  5. Install Mongodb with the below steps
    
     sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
     echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
     sudo apt-get update
     sudo apt-get install -y mongodb-org
     sudo service mongod start
    
    

    Run the spark job with the below command

    
    bin/spark-submit --class com.spark.SFFoodInspectionAnalysis --master local sf-food-inspection-spark-assembly-1.0.jar file:///usr/local/spark/Food_Inspections_-_LIVES_Standard.csv 
    
    
  6. Then check the Mongo Db and check the collections and make sure that the data are getting inserted and availableOpen up a terminal window and type ‘mongo’ and enter. It will open a shell window. Then use the below commands to verify the data
    
      show dbs
      use sfFood
      show collections
      db.allFoodInspection.find()
      db.filterFoodInspection.find()
    
    
  7. Clone the git repository https://github.com/dkbalachandar/sf-food-inspection-web.git and go inside of sf-food-inspection-web folder, then run below commands to build and run the application
    
      npm install
      node server.js
    
    
  8. Open the http://localhost:8081 and check the page. I have used the data and created a table and display a chart with the details.

Please are the some of the screenshots taken from the application

How to install Scala, SBT and Spark in Ubuntu

Install Scala 2.11.8 and SBT


sudo apt-get remove scala-library scala
sudo wget www.scala-lang.org/files/archive/scala-2.11.8.deb
sudo dpkg -i scala-2.11.8.deb
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823
sudo apt-get update
sudo apt-get install sbt


Install Spark 2.0.2


sudo wget http://d3kbcqa49mib13.cloudfront.net/spark-2.0.2-bin-hadoop2.7.tgz
sudo chmod -R 755 spark-2.0.2-bin-hadoop2.7.tgz
sudo gunzip spark-2.0.2-bin-hadoop2.7.tgz
sudo tar -xvf spark-2.0.2-bin-hadoop2.7.tar
sudo mv spark-2.0.2-bin-hadoop2.7 spark
sudo mv spark /usr/local/spark

Open bashrc file and add spark_home and update PATH  
sudo vi ~/.bashrc
  export SPARK_HOME="/usr/local/spark"
  export PATH=$PATH:$SPARK_HOME/bin

source ~/.bashrc