Course Review: Functional Programming Principles in Scala

A BRIEF Absence

When I created in this blog, I’d certainly intended on writing in it more often than I have, but a funny thing happened in between then and now: I got a chance to write a Spark job at work in Scala. Now, I’ve had to cobble together some Scala code a few times for various projects, mostly controllers that then delegate to Java classes, but this was the first time I really had to get down and dirty into a functional mindset and think about multi-stage processing of data.

I’ve got to say, it was a bit of a revelation. Having gotten into software engineering from a web development background, I absolutely loved how much more control over data structures you got in Java, how much more formalized and important design patterns were, but there was also something that really bothered me about it. If writing code is like, well, writing, working in Java was like penning a technical manual. You can’t be clever in the same ways you can in more expressive languages (which, granted, can be both a gift and a curse).

With Scala, though, it’s like taking I love about Java and just for fun merging it with everything love about JavaScript and getting a language that’s fast (runs on the JVM after all), easy to develop in, equally suited to functional and OO idioms, and frankly a hell of a lot of fun to code in. If writing code is like driving a car, Java is like the family mini-van and Scala is a Ferrari.

So, I never got this blog off the ground the way I intended, because I’ve been putting most of my free time (at least, my non-wife, friends, and cats free time) into learning Scala and improving my FP-foo. Specifically, I’ve been working through Coursera’s new “Functional Programming in Scala” certification, which is done in conjunction with the École Polytechnique Fédérale de Lausanne. I realized that if I wanted to be a serious Scala developer (and I really want to be), that I needed to take a couple of months and really put some focus into learning the language.

Course Review

If you’re going to learn Scala, who better to learn it from than it’s creator, the brilliant Martin Odersky. Odersky teaches all of this course and most of the next in the certification, and I have to say, for someone as multi-talented and as deeply knowledgeable as he is, he does a pretty great job of keeping the course squarely focused on the fundamentals. He doesn’t over simplify or treat the audience like children, nor does he act like a bored star prof who somehow got stuck teaching CS101 to the new undergrads. He’s patient, explains things, and covers a lot of the basics of Functional Programming.

I should mention, I spoke to a colleague who’d also taken the course, and the colleague mentioned feeling that it was a bit slow for his tastes. I think if you have a background in Functional Programming you’ll probably end up keeping the recording speeded up (even with some rudimentary knowledge of Scala and a little background attempting to incorporate functional elements into my JavaScript, I ended up watching a lot of the videos sped up), but I don’t think being thorough is something you can fault a course for.

At one point, Odersky mentions his course design being pretty heavily influenced by Structures and Interpretations of Computer Programming, and I think that’s probably a pretty fair assessment. I read through a bit of SICP a year or two ago, and there are a lot of similarities with this course — some of the examples are straight out of the book, and as an example, the course spends a lot of time using lists to illustrate the basics of functional datastructures, which I remember also being prominent in SICP.

In the end, I found the course to be extremely helpful, though I should mention that in terms of learning Scala, the course doesn’t quite get you all the way there, even by the lowest possible bar of learning all the basics of the language’s features. You’ll cover pattern matching, the list data structure, general functional data structure principles, call by name vs call by value, the substitution model of programming, as well as a few other topics, but other important aspects such as for comprehensions, generators, monads (and how flatMap and map relate to them), streams, etc., aren’t really explored until the second course. This isn’t to say that the course wasn’t wonderful and didn’t bring up a lot of ideas that have made me a better programmer, just that it’s worth continuing, and going into the course with the understanding that it’s not exactly the best fit if you’re looking for the shortest possible way to get up to speed on Scala.

Load Testing with Vegeta

The Big Jump

We had one of those good news/bad news situations at work recently: we learned that we’d be signing a major media site that will end up bringing us something like a 50% increase in traffic (awesome news!) and that they’d start sending us traffic as soon as this week (scary!).

In an ideal world, we’d throttle the traffic from this new site and slowly ramp up, allowing us to see how our system performs under the additional load. We use AWS autoscaling for our api servers, so we have some flexibility, but there are some underlying aspects that don’t scale — we have a fixed number of MySQL instances, for example, and our log rolling has always been a pain point so there’s no guarantees whether it will scale gracefully.

The issue is that the traffic comes from a 3rd party JavaScript library, and there’s not really a graceful way to make a throttling decision on the client side (at least not one that gives us an easy ability to change the throttling rate). The best we could do is implement a request filter that quickly looks up the throttling rate of the site the request is from (there’s some caching involved, but I’m already on a bit of a tangent) and then ‘short circuits’ the request so to speak and sends back a jsonp response that disables our JavaScript.

Ultimately, this means that even in the best case, we’re going to see a 50% jump in initial calls, even if those calls immediately return a disable response. So that means we needed a load testing solution!

Continue reading Load Testing with Vegeta