Scala: first impressions

Posts you want to find years later go here.

Scala: first impressions

Postby Jonathan » Thu Sep 18, 2014 2:18 pm

I needed to parse an XML file to produce an output file yesterday, so I decided to use Scala as an exercise in learning it.

Code: Select all
import scala.xml._
import scala.math._

object HelloWorld {
  def main(args: Array[String]) {
    println("graph New_Eden {");
    println("    node [shape=point];");
    val xml = XML.loadFile("/home/jonathan/Downloads/mapsolarsystems.xml")
    val records = (xml \\ "RECORDS" \\ "RECORD").filter(x => (x \ "securityClass").text != "")
    val systems = records.map( x => ((x \ "solarSystemName").text, (x \ "x").text.toDouble, (x \ "y").text.toDouble, (x \ "z").text.toDouble))
    val output = systems.map(base => {
      val name = base._1
      val x = base._2
      val y = base._3
      val z = base._4
      (name, (systems
   .filter(subrecord =>
   name != subrecord._1)
     .map(pair => {
       val x2 = pair._2
       val y2 = pair._3
       val z2 = pair._4
       (pair._1, sqrt(pow(x-x2,2)+pow(y-y2,2)+pow(z-z2,2))/(9.4605284*pow(10,15)))
     }).filter(outrec =>
   outrec._2 <= 2.0)
       ))
    })
    output.foreach(r => {
      r._2.foreach(A => {
   println("\"" + r._1 + "\" -- \"" + A._1 + "\" [label=" + A._2 + " headlabel=\"" + A._1 + "\" len=" + A._2 + "];")
      })
    })
    println("}");
  }
}


The good news was I didn't have to install anything but `apt-get install scala` and an Emacs package manager (!) so I could grab a Scala mode Emacs package. The base capabilities of the language were plenty for this little toy. The other good news is that it runs 100x faster than the piece of software (graphviz) which consumes the output, so it is by definition sufficiently fast (10-20 seconds on a 6.6MB input, depending on the compilation time).

The ugly:
Having a compiler barf at you when you do something stupid is pretty great, but the errors are very non-intuitive. Also, at first I pretty consistently generated a runtime error on a .toDouble call, the text of which was zero help.
The syntax for creating tuples is simple and "right". The syntax for consuming tuples as arguments to functions seems backasswards. I'm still not convinced there isn't a more elegant way to do it lurking somewhere in the language, but I couldn't find it while drinking a Double Mountain IRA last night.
The abomination that is the }) really needs to be dragged out into the street and hanged.
At first I tried producing my output with a map, but evidently that is a no-no. Something about side effects? Not sure.
I would have never in a thousand years come up with that XML parsing syntax, but it's starting to grow on me. I worry about how expensive it is.

ToDo:
I need to control the number of digits printed out in the label.
I need to limit the number of taillabels to the number of systems.
Jonathan
Grand Pooh-Bah
 
Posts: 6031
Joined: Tue Sep 19, 2006 7:45 pm
Location: Portland, OR

Re: Scala: first impressions

Postby Jonathan » Wed Sep 24, 2014 2:17 am

You can handle tuples as arguments using 'case' syntax, which is weird but workable.
You can put the whole map block into curly braced, eliminating the }), which is good.
Jonathan
Grand Pooh-Bah
 
Posts: 6031
Joined: Tue Sep 19, 2006 7:45 pm
Location: Portland, OR

Re: Scala: first impressions

Postby quantus » Wed Sep 24, 2014 7:06 pm

Why Scala as opposed to any of the mountain of other scripting languages? I gotta imagine that python or ruby would have very nice support for XML.
Have you clicked today? Check status, then: People, Jobs or Roads
User avatar
quantus
Tenth Dan Procrastinator
 
Posts: 4653
Joined: Fri Jul 18, 2003 2:09 am
Location: San Jose, CA

Re: Scala: first impressions

Postby Jonathan » Wed Sep 24, 2014 7:49 pm

Because I "know" Python and Ruby. I don't know Scala. It isn't like creating a jump map of Eve star systems is important to my work, but Apache Spark might be.
Jonathan
Grand Pooh-Bah
 
Posts: 6031
Joined: Tue Sep 19, 2006 7:45 pm
Location: Portland, OR

Re: Scala: first impressions

Postby Jonathan » Sun Dec 14, 2014 8:10 pm

Code: Select all
import scala.xml._
import scala.math._

object HelloWorld {
  @inline def calc_distance(x1: Double, y1: Double, z1: Double, x2: Double, y2: Double, z2: Double): Double = {
    return sqrt(pow(x1-x2,2)+pow(y1-y2,2)+pow(z1-z2,2))
  }
  def main(game: Array[String]) {
    //Iterable("Hek","Perimeter", "Jita","Lasleinur","Pator","Promised Land","5-N2EY","L-HV5C","New Caldari","Huola","Zaimeth","Raravath","Amarr","Dal","Amamake")
    println("graph New_Eden {");
    println("    node [shape=hexagon fontcolor=blue width=0.85 regular=true fixedsize=true];");
    //println("    size =\"512,512\"");
    val xml = XML.loadFile("/home/jonathan/Downloads/mapsolarsystems.xml")
    val records = (xml \\ "RECORDS" \\ "RECORD").filter(x => (x \ "securityClass").text != "")
    val systems = records.map( x => ((x \ "solarSystemName").text, (x \ "x").text.toDouble, (x \ "y").text.toDouble, (x \ "z").text.toDouble))

    val fn: PartialFunction[(String,Seq[(String,Double,Double,Double,Double)]), (String,Seq[(String,Double,Double,Double,Double)])] = {
      case x if game.exists(_ == x._1) => x
    }

    val universe = systems.map(base => {
      val name = base._1
      val x = base._2/(9.4605284*pow(10,15))
      val y = base._3/(9.4605284*pow(10,15))
      val z = base._4/(9.4605284*pow(10,15))
      (name, (systems
   .filter(subrecord =>
   name != subrecord._1)
     .map(pair => {
       val x2 = pair._2/(9.4605284*pow(10,15))
       val y2 = pair._3/(9.4605284*pow(10,15))
       val z2 = pair._4/(9.4605284*pow(10,15))
       (pair._1, x2, y2, z2, calc_distance(x, y, z, x2, y2, z2))
     }).sortWith(_._5 > _._5)).filter(outrec => (((outrec._5 <= 2.0) && (outrec._5 >= 1.5)) || ((game.exists(_ == outrec._1) && (outrec._5 >= 1.0) && outrec._5 <= 4.0)))
      ))    
    })

    val gamesystems = universe.collect(fn)
    val adjacents:Seq[String] = gamesystems.flatMap{
      case(name, dest) =>
   dest.map{x => x._1}
    }
    val output = gamesystems.union(universe.collect{
      case (name, dest) if adjacents.exists(_ == name) => (name, dest)
    })
    .map{
      case(name, dest) =>
   (name, dest.filter{
     case(drillname,x,y,z,drilldist) =>
         ((game.exists(_ == drillname)
      && game.exists(_ == name))
          || ((drilldist >= 1.9) && (drilldist <= 2.0)))
   })
    }
    .flatMap{
      case(name, dest) =>
   dest.map{
     case(drillname,x,y,z,drilldist) =>
       val names = Vector(name, drillname).sorted
       (names(0), names(1),drilldist)
   }
    }.distinct
    output.foreach{
   case(name,drillname,drilldist) =>
     val w = if(drilldist > 2.0) { " penwidth=5" } else { "" }
//     if(i==0) {
//       println ("     \"" + r._1 + "\" -- \"" + drillname + "\" [taillabel=\"" + r._1 + "\" label=" + dist + " len=" + drilldist + "]");
//     } else {
     println("    \"" + name + "\" -- \"" + drillname + "\" [len=" + drilldist + w + "];")
    }
    println("}");
  }
}

Implemented all the stuff on the old ToDo list and cleaned it up a bit.

Dicking around with flatMap made me realize I really haven't ever tried to program systems using higher-order functions since college.

Anyway, much of the increase in LOC here is trying to prune away significant parts of the universe. In this case, I'm eliminating all links between system except those in a narrow shell between 1.9 & 2.0 light years (with exceptions for key systems specified via command line argument) to get something approximating a human-legible graph.

ToDo List 2.0:
Collapse the systems adjacent to the key systems before calculating any links between anything.
Jonathan
Grand Pooh-Bah
 
Posts: 6031
Joined: Tue Sep 19, 2006 7:45 pm
Location: Portland, OR


Return to The Vault

Who is online

Users browsing this forum: No registered users and 1 guest

cron