Bash vs Ruby

I was recently asked by a colleague if I could help out with a bash script that would scp a file to a remote server and then move the local file if the scp was successful. I have dabbled with bash for many years now, but I never went deep enough to do the more advanced stuff, so I said I couldn’t help him.

However, I did manage to learn Ruby (with a lot less effort than that of learning bash) and it has excellent support for running bash commands. So, I hacked together a script that would do the same thing in Ruby. Here’s what it looks like:

files = ['file1.txt', 'file2.txt'];
destinations = ['', '/will/fail.txt'];

files.each_with_index do |file, index| 
  destination = destinations[index];
  puts "Moving #{file} to somehost:#{destination}";
  `scp #{file} somehost:#{destination}`;
  if ($?.success?)
    puts "Success, moving file.";
    `mv #{file} 'success'`;
  else
    puts "Failed, file remains.";
  end
end

If you don’t know any Ruby, I’ll just walk you through it. First I define two lists, a list of files and a list of destinations. I then iterate over the list of files with the .each_with_index method. Inside the pipes, I define the item that I will get on each iteration as “file” and the index as “index”. I then fetch the destination from the list of destinations, print the file name and destination and then I run scp. Note that you have to put #{} around the variable names in the command. The back tick means “run this on the command line”.

After the scp has finished, I check the result with $?.success?. This seems a bit arcane and there might be a better way to do it (suggestions are welcome in the comments), but it works. If successful, I then move the file locally, else I print a message and let the file remain.

When you present this to a bash guy, there’s going to be resistance. Things like “I have to learn a new language” and “bash is readily available, Ruby is not” will probably be mentioned. However, given that the syntax of Ruby is so straightforward, I think there are huge benefits in terms of readability and maintenance. Regarding the “availability” argument, it’s very fast and easy to install Ruby, if it’s not already there. If you have a Mac, try starting the terminal and typing “irb”, you might be surprised. Final argument, learning a new language (especially Ruby) is fun!

What do you think, should we stick with bash for command line “apps”, or is the investment in something like Ruby worth it? Your comments are much appreciated!

Posted by Viktor Nordling

Learning Languages

I’ve been looking at a number of different sites for tips on how to learn new languages and here’s a summary of what I’ve found.

To get you started, here’s a list of 10 puzzle websites to sharpen your programming skills. I would like to strongly recommend Project Euler, which I used for learning Ruby.

I did try to solve the problems again, this time using Scala. I got stuck doing this, because my solutions were so ugly and the “official” solutions seemed a bit complex. This was a while ago and looking at some example solutions, I think it’s just a matter of understanding some new concepts, like what a Stream[int] is and how zip works.

Another site from the list I want to point out is Python Challenge. It’s a spin off from a popular website called Not Pron, which is a puzzle website that has kept me busy for many many hours. Anyway, after just an hour or two of solving problems from the Python Challenge, I had learnt quite a few Python concepts, and this is as a complete beginner before I started. Highly recommended!

When it comes to Scala, there are lots of other resources. The Scala School and the Scala Koens should get you (and me) started.

If you have any other tips for how to learn a language, please write a comment below!

Posted by Viktor Nordling

Using Scala to Test Your Java Projects

In this technical blog coder-in-residence Viktor Nordling describes a sample project he has developed to showcase the ScalaTest test framework.

The Mission

I’ve been planning on starting to learn Scala by developing tests for my existing Java projects using the Scala language (following the advice of Java Posse’s Dick Wall). I originally found this article on StackOverflow but it didn’t really answer my questions. I then read an excellent blog article from ex-colleague Frederik Tyboni that descibed exactly what I wanted to do. Unfortunately for alot of readers his article is in Swedish. So here follows a rough translation of what he wrote — along with an example project that I have developed and posted on GitHub. My goal is to get you started with ScalaTest in minutes!

For the impatient, the example project is here

Integrating ScalaTest into Your Existing Maven Build

Assuming you are using Maven, all you need to do is first add the dependencies for Scala, JUnit and ScalaTest, like so:

    org.scala-lang
    scala-library
    2.9.1
    test


    junit
    junit
    4.8.1
    test


    org.scalatest
    scalatest_2.9.1
    1.6.1
    test

Secondly, we add the maven-scala-plugin so that we can compile Scala code:

    org.scala-tools
    maven-scala-plugin
    2.15.0
    
        
            
                testCompile
            
            
                
                    -make:transitivenocp
                    -dependencyfile
                    ${project.build.directory}/.scala_dependencies
                
            
        
    

Now we tell the surefire-test-plugin to include **Spec files when executing tests:

    org.apache.maven.plugins
    maven-surefire-plugin
    2.6
    
        false
        true
        
            **/*Spec.*
            **/*Test.*
        
     

Finally, use the build-helper-plugin to tell Maven to include source files under src/test/scala when compiling the tests.

    org.codehaus.mojo
    build-helper-maven-plugin
    
        
            generate-test-sources
            generate-test-sources
            
                add-test-source

            
            
                
                    src/test/scala
                
            
        
    

All we have to do now is to add a test. If you want to, you can use JUnit. In this way, you can use the more expressive syntax of Scala to write your tests, while still being in the familiar land of JUnit.

Or, you can take one step further and try a Scala testing framework. Here’s an example using ScalaTest, which is a test framework that integrates with JUnit and TestNG. It supports BDD if you’re into that and will make your tests more expressive than standard JUnit tests.

package com.cubeia

import org.junit.runner.RunWith
import org.scalatest.matchers.MustMatchers
import org.scalatest.FlatSpec
import org.scalatest.junit.JUnitRunner

@RunWith(classOf[JUnitRunner])
class CalculatorSpec extends FlatSpec with MustMatchers {

  "Calculator" must "multiply two values" in {
    val calculator = new Calculator;
    calculator.multiply(3, 5) must equal (15)
  }
}

The Calculator class is written in Java, but tested from Scala. Again, you source files for this if you clone my example project on GitHub.

That’s all! Happy Scala hacking!

—
Viktor is a freelance software engineer who occasionally finds coding serenity at the Agile Digital office.

Using Structure101 to Improve the Design of Your Code

In this technical blog coder-in-residence Viktor Nordling describes how Structure101 is an awesome (albeit expensive) tool for significantly improving code quality (and why you should use it).

Background

I recently started working on a greenfield project. As you all know it’s always fun and games in the beginning, but after some time things usually start to get a bit messy. After a year or so it can seem that starting a new project would be alot more fun than maintaining the inexplicably horrible legacy project you’ve been working on. This time I was determined not to let this happen. In order to prevent code from becoming “instant legacy” you need a way to get an overview of what the code structure actually looks like, and a way to measure if it’s getting better or worse. Structure101 is an analysis tool from Headway Software that does just that.

Actual Architecture

By this point I already had the basic structure of the code in place, so I was really curious as to what Structure101 would say about it. Here’s a screenshot of what Structure101 reported about the first version of my code:

This is an architecture diagram, which Structure101 generates based on how the code actually hangs together. Experience suggests that this is likely to be markedly different from the architecture that an architect may have at some point in time optimistically drawn in a fancy modelling tool. One layer in the diagram is only allowed to use layers below it. The arrows in the diagrams indicate violations against the architecture.

If you open up the interfaces layer (package) you see the following diagram:

Fixing the Flaws

So we have four violations in total, two from the interfaces layer. There are a number of techniques to fix a violation. These include:

1. Moving the dependant to the dependee’s package (or vice versa);
2. Extracting an interface from the dependee’s class and placing in a layer below the dependant’s layer; or
3. Refactoring the code so that the dependency no longer exists.

In the case of the StateChanger above, I could simply move it to the states package. It actually felt like it belonged there more than it belonged in interfaces.

In case of the notifier, it depended on a Player class in entities. I extracted an interface from this class and placed it in the interfaces package.

The dependency from entities to states actually pointed out a code smell. I was shortcutting my state pattern when I scheduled the start of a new hand. I knew when I wrote it that it was wrong, but this violation pushed me to actually fix it.

The dependency from entities to util was similar. I had made an instance variable in a class default to a NullNotifier (which was in the util package), in order to be able to test my code without having to inject this from the test. I solved this by creating a static utility method in my test code that injected mock objects into my class under test.

Once this was done my software architecture in Structure101 looked like this:

Keeping it Clean

Once you’ve reached a point where you have an architecture that you like, the key point is to keep it this way. In my case, I was the only programmer on the project however I knew that more developers would be joining soon.

In order for others in your team to adhere to the architecture you’ve carefully laid out, it’s vital that they actually know (in a pragmatic way) what that architecture is. Structure101 comes with a neat plugin (for Eclipse and IntelliJ) which will show you the architecture diagram inside your IDE. And here’s the pragmatic part — if you introduce a dependency that violates the software architecture the compilation will actually fail. Here’s a screenshot of what that looks like:

If you select the “Architecture” tab, you see the violation graphically in the diagram:

You can also get the Structure101 Build component (including an Ant and Maven plugin) which will check for violations during the build phase. However this component comes with a hefty price tag of US$1,500.

Loaded with Legacy

If you are working on a project with a lot of legacy code getting down to zero violations in one go might be a herculean task (and is probably not even recommended). In this case you can configure the plugin to only fail the compilation in case of new violations. Then at least we know that we aren’t making things worse.

Structure101 comes with a web application for showing information about your project. By publishing snapshots of your project continuously you can maintain a good graphical overview of how the project is evolving. Here’s a screenshot of some of the graphs from the web app in regard to architectural complexity:

Here we can see that the excess complexity (XS) has gone down to zero. We also see a breakdown of the different types of contributors to the excess complexity. The reason why the “size” is reduced in the final version is because I excluded some 3rd party code (which contributed to the Fat/method complexity).

Summary

Structure101 is a great tool, although in my opinion slightly overpriced. The support that is included in the price is excellent. I do think code all around the world could smell a little bit less if more people knew about this product, so please spread the word. There’s a 30-day trial that you can sign up for to try it out today!

—
Viktor is a freelance software engineer who occasionally finds coding serenity at the Agile Digital office.

Node.js Showdown

In this technical article Stefan Schneider-Kennedy takes a closer look at Node.js.

I’ve been interested in looking at Node.js for some time. I was intrigued by the concept of the “evented” web server and I also quite like the JavaScript language. As best I can tell, these tools are what make Node.js “cool” and they’ve inspired me to take a closer look at this framework today.

First up: the “event driven” concept. Traditional web servers spawn a new thread for each client request, which is a fairly heavyweighted way of handling most requests. For example, in a database-driven web app, you spend most of the time waiting for the database to get back to you. Keeping a thread around while you wait is a waste of memory. Node.js takes a different approach. Everything runs in a single thread, which is woken up whenever an interesting event happens (such as data coming back from the database). This approach has the advantage of using a lot less memory in many situations. The disadvantage is that if you lock that single thread, no one gets data. Because of this, you have to try to keep the main thread as light as possible by delegating work to other processes or Web Workers (still in the planning stages).

To show the event-driven, single thread model at its best, I put it up against apache running PHP. Both node.js and PHP slept for 1s and then exited (perhaps simulating waiting for a complicated DB query to return). I then pointed apache benchmark against the two servers, 10000 requests at a concurrency of 1000 (ab -n 10000 -c 10000 http://localhost…). Node.js handled the load well, taking 13s (out of a possible 10) to finish the requests. On the other hand, apache ran out of threads around the 2000 mark, slowed down my machine, and then stopped answering requests.

The other nice thing about Node.js is that it’s JavaScript. Whatever you think of the language itself, the code-reuse benefits of running the same code on the server and client side is definitely interesting. For the purposes of my demonstration, I made use of a JavaScript implementation of the “Markdown” markup language called Showdown. I threw together a quick “wiki”-like service, which utilized Showdown to provide page rendering and a quick preview of the page. Making use of this library on the server using a small wrapper module was pretty easy. What really interested me about running JS server-side was that someone made an implementation of DOM1, allowing stuff like jQuery to be used on the server. I had the idea that you could use this to easily make an interactive, “ajaxy” app, which would still work if JS is disabled on the client side. Essentially, the client updates its DOM and any functions it runs to do so also run on the server to keep its copy of the DOM in sync. This would take a bit more memory than generating HTML on the fly, but might be cool for apps that display a common interface to all of its users (e.g., an administration panel). The stumbling block is that the node DOM1 implementation doesn’t have a serialization function (DOM -> a HTML string), and I haven’t got around to making one myself.

One of my favourite things about JavaScript is its event-driven nature, and doing this kind of thing on the server side appeals to me. Node.js isn’t the only framework that allows this (e.g., Python’s eventlet), and there are existing web servers (like Nginx) that use an event-driven design to deliver much higher concurrency than Apache. However Node.js forces this pattern more than most. Synchronous aspects are the exception in Node.js rather than the norm. This didn’t seem to get in the way much and this generally forces you to use a design pattern that scales well.

Sharing code between the server and client is interesting, but probably not generally too useful. Sharing rendering code, e.g., preview/render (like in my “wiki” example), or boring stuff like data validation, are probably the main applications. Combined with a DOM serialization function, you could have a client side app that gracefully degrades without as much effort as it normally takes. Generally though, client-side JavaScript has a rather different role than what runs on the server and, hence, there isn’t too much overlap.

Node.js is certainly interesting, and I’m looking forward to playing around with it a bit more. The event-driven thing is cool, but I’d have to see how it stacks up against other event-driven frameworks before being able to make an informed judgement about whether it offers anything extra.

—
Stefan is a software engineer who is currently melding his mind with the allhomes.com.au platform.