Saturday, December 31, 2011

Digital bookshelf - no really

As we prepare for our relocation, we've been digging through our substantial book collection attempting to figure out what to keep and what to shed.  Each time I have to remind myself that at this point, most of my book reading is done electronically.  The thought occurred to me that this will lead to the decline of the bookshelf, and therefore the hallowed tradition of inspecting the bookshelves of a host as a means of either distraction or a vague attempt at insight.

I wondered then if it would be possible to create a virtual bookshelf.  Not a kindle type deal with a little set of icons on a tablet, but an actual, virtual bookshelf.  A projection of some kind of a bookshelf, 3' x 6' on a wall that you could see and perhaps even interact with.  The obvious rendition is some kind of LCD screen that is the same size.  TVs are getting so big these days, I'm thinking it wouldn't be much of a stretch to use one as a virtual bookshelf.  I think this is a very narrow vision though.  I think it would be better to do something new or at least different.  I could envision libraries revolutionized with monolith type structures that are virtual bookshelves.  You could browse the shelves of the mystery novels just as you do now, pausing to flick through the pages of a random book whose title looks intriguing.  Just touch a button on the virtual book to add it to your rental queue, and on with the browsing.

Wednesday, December 28, 2011


If I can ever crack SBT and IntelliJ, I'm gonna post about it, a lot. I just can't seem to get to a point where I can get things working properly with a Scala config. Every time I try, I just get frustrated and give up. The biggest barrier into Scala for me right now is documentation. Critical features like SBT just don't have good docs. I can't seem to run Scalatest Spec tests in IntelliJ, I just end up with weird compiler errors. I can't seem to get the SBT console working in IntelliJ either, and I can't seem to generate an IntelliJ project with either IDEA plugin for SBT, I just end up with weird dependency errors or something else going on. The local Scala users group seems to be barely active. They have one event listed, in June 2012, and are asking people if they are going. It's December 2011, that's six months away, I don't even know if I'm going to be in this state in six months, let alone able to attend some event. They don't seem to have regular meet-ups from what I can gather from their page, which is not a whole lot. I've resorted now to posting questions on Stack overflow, which seems to be turning out pretty well to be honest. People answer quickly and often include lots of helpful information on questions that once I've read the answer seem like they probably made me look pretty dumb. Well, I guess if you don't ask, you'll never know, so I'll take dumb yesterday, smart today over dumb forever. I might separate actual Scala posts with code from the main blog at some point too. Seems like they are confusing each other.

Sunday, December 18, 2011

Scala, encore un fois!

How often do you have complex functionality in a declarative language work the first time out of the gate? Almost never right? Not so much with functional. The number of times a function I've written has worked out of the gate, or virtually of the gate is astounding me. Talk about getting this done faster. The only problem I'm having right now is that I'm still not that familiar with the syntax nuances, so I'm tripping up over that a bit. Really gotta sit down with the user guid and get up to speed on all the little bits and bobs that I'm missing right now. I read through the List class spec, and that was pretty helpful, but there's so much more!

Friday, December 16, 2011

Amazon EC2 - Failing the promise... again

I have a couple of instances on EC2 that I use to host my website, and a bunch of files. They were scheduled for a mandatory reboot for an upgrade. Guess what, they didn't come back up cleanly. Imagine my surprise, except, I wasn't surprised, I was anticipating this. I've worked with EC2 enough to dread a host reboot, mandatory or otherwise. It's a pain in the ass, and a crapshoot if the system will come back up or not. To add insult to injury, a reboot assigns a new IP address. If other systems in your configuration are using that system for something, which has an internal IP address, so unless you have your own BIND set up to deal with that, you're using host entries, the host, if it comes back up, is not at the same address it was before. It's great. Not. You can build a VPC of course, which I have thankfully, so the damage wasn't too bad. I figured out that the "upgrade" had caused my /etc/fstab to be "upgraded" and Linux was attempting to mount my EBS as ext3, but it was actually ext2, so it wasn't mounting. Other providers are providing increasingly compelling options for cloud hosting. The lure of EBS is starting to diminish as other providers are offering bigger disk allocations than they used to, sometimes dynamically. Not only that, but the performance of EBS has been pretty dismal, along with the instances themselves, so I'm not really at big EC2 fan at this point. Current contenders: Rackpace Linode Linode has a good system, but it's instances are bit pricey. has a nice system that more customizable, and the prices are pretty good, but I can't get a good bead on the exact terms of the contract, wether the billing is by hour, or by month, or how it exactly works. Rackspace's cloud offering is really great, BUT, lacks any kind of dynamic disk assignment last I checked. They have a good range of instances to suit all needs, far better than Amazon for small use systems. Just that darn disk allocation issue. I have loads of data acquired over the years, but my website doesn't get much traffic at all, so I don't need a big honking machine. claims to have the underlying disks on RAID 10, but if they are, then they must either be overloading nodes pretty hard, or they just have crappy hardware. I ran a bonnie++ benchmark, and it was downright awful. Yet again, caught between a rock and a hard place on hosting, SO annoying.

Monday, December 12, 2011

Coding, how to put things where they go.

My first rule of programming is "Put Shit Where it Goes".

For many years, I've struggled to effectively quantify this. It has just seemed "obvious" to me very often where shit goes.

I think talking with a remote team over the past couple of weeks has helped my try to think through this a little more.

Give what's to be received, take only what is given

Here's a part I was thinking of today. It's the concept of things that are provided or given, and things that are required or received. You don't get things from an object that requires them, and you don't give things to an object that provides them.

Here's an example:

MyBusinessObject o = new MyBusinessObject();

MyOutputFormatter f = new MyOutputFormatter():
When I see this, I want to reach through my screen and slap someone. I liken it to using an apple to make an Apple pie, then trying to use the apple from the the pie in a glass of cider! You can tell me that this analogy is wrong because it's an object, a thing in memory that multiple places can use, like an apple that exists in parallel universes where it is a pie in one and cider in another. I believe this counter-argument is mistaken. When your apple exists in two different contexts, the results can be unpredictable, even catastrophic when one context changes the apple, or the apple's previous existence is now removed.

If we remove MyBusinessobject from the code above, the assignment to MyOutputFormatter will then fail. My IDE will catch this, as will compilation, but it's a correction that should never have needed to be corrected, and you spend time correcting that syntax error that didn't need to be spent.

The reverse situation is even clearer. You don't give things to a provider. It's like taking an orange out of a bag and putting it on an orange tree.

FruitProvider orangeTree = new FruitProvider();
Fruitrovider appleTree = new FruitProvider();

FruitConsumer bakery = new Bakery();
FruitConsumer caterer = new Caterer();
FruitConsumer footballer = new Footballer();


Then to the astonished onlookers:


It may seem laughable when written like this, but I see code like this from time to time. It's also an example of why strong typing can save your ass (more on that later I think). If you were writing 'good' code, this would never compile. A "Tree" should simply not have a setter for its fruit. How many of us fire and forget in the IDE though? Just hit "generate code" and not think twice about it? How often does a framework force us to create setters we don't want to? Just like this fruit tree, if we are retrieving an object from a database, the ORM may use a setter to build our object from the database. The setter ends up being public because of it, and now any fool can put data in our object where it shouldn't be permitted.

There are some solutions to this that I've seen, though few good ones. I've seen the setter marked as @Deprecated so that at least the IDE and Compiler give you a big nasty warning if you try and use it, though it's not forcibly prohibited. It does seem like it should be a pretty fundamental feature to have. And in some ways, it is. This is what constructors are often used for. But, in the case of an ORM, a very common need is to initialize data lazily. We can't half create an object all that easily, so, the object gets created, then properties get set later. Almost like an orange tree. We can't have oranges until the tree is grown and the season has come. The problem is that the ORM is acting kind of like god in this scenario, who ends up being personally responsible for putting oranges on the tree. In fact, I think I like this analogy, I'm going to call this kind of code God-Code.

The problem with God-Code, is that it means the worshipers end up rather lacking in willpower and loose the sense to control their own destiny, so any old god can come along and mess with them.

Don't do for yourself what somebody else can do much better
or: don't Let the Plumber fix your Car

This is another instance of the "Put Shit where it Goes" rule. Let's say you want to get a car, make it a blue car, with tinted windows and a really great sound system. The dealership has the car for sale, but it doesn't come off the lot with tinted windows and an upgraded sound system. You don't buy the car from the dealer, then take it to your local plumber to get the windows tinted, or have the librarian put a premium sound system in. You have it done either by the deanship, or by a general mechanic, or by a specialist in that kind of installation.

The same principle applies to helper methods. If you ever intend to put tinted windows in more than one car, don't have your plumber tint the windows. If you have a Service Bean that sends XML, but gets given an object, you don't build XML Serialization into your Service Bean. You're gonna want to serialize to XML more than this one service call, so you make another bean that is responsible for serializing XML, or better yet, use someone else's bean/library.

This is a good example of a separate concern. XML Serialization is largely a concern that is independent from serving up data over HTTP. When you want to test your HTTP server, you don't want to end up having to test XML Serialization as part of that test. Vice-versa, you don't want to test HTTP functionality from your XML Serializer. You might ask, why not? It's simple math. If you have an object t that does two things, f and g which may be combined, potentially tightly coupled, you have to write test for the following for a given test fixture x:


Don't forget to write those test cases that demonstrate the functionality fails when it's supposed to!

If we have separated concerns that are decoupled in two objects a and b, we end up writing only:

We've now tested our system with the same efficacy with four tests instead of eight. Now work out the situation when you have an object that does three things, or four things. It's exponentially more work to test as you get more tightly coupled concerns within an object.

In additional thinking, you can sometimes fix the car yourself.  Save all that money on labour, spent it on something fun right?  If you're not a mechanism, think again.  The mechanic has the right tools to do the job.  He has contacts for getting the materials, and this is his job, he's good at it and can get it done more quickly than you (if he's any good).  You might be able to buy an off-label turbo cheaper, which is great, until you break off some fitting when you try to install it and have to buy another.  Or until you realize you need some special set of wrenches that costs twice as much as the turbo itself.  Then you end up spending a month getting it installed instead of getting a mechanic who could have done it in an afternoon.  You end up spending four times as much doing it yourself than it would have cost to buy brand-name and have the mechanic install it.

Don't reinvent the wheel is the usual metaphor.  If somebody else has built an XML serialization library, check it out first.  Be sure it can't meet your needs before you dive into writing your own.

Save yourself a permanent headache, and put shit where it goes!

Of course, if your goal is purely fun, or you need XML serialized in a special magical kind of way, you roll your own.  If you're nice, you make it free software so we can all have a magical XML serializer.  If everyone writes their own XML serializers, the world only ends up with a whole bunch of esoteric serializers that aren't any good to anyone but the people who made them, and they are probably going to end up wishing they'd spent their time building code that did something truly unique and useful.

I'm gonna cover tangling and scattering at some point I guess too.

Tuesday, December 6, 2011

Data Payload Facet Pattern

I have no idea if this already exists, or if there's a better way, but here's a little thing I came up with yesterday.  I'm calling it the data payload facet pattern. I guess it's pretty similar to the Façade pattern, but a bit more specific.

What is this pattern?  It's a way of creating what is essentially an Object Oriented version of a SQL view.

If you have a generic payload object that has a Map<String,Object> concept, then anything you set or get from this Map is, to a point unsafe.  You can use the class argument improvement at least on getters, but it's still pretty weak.

Enter the Payload Facet Object.
public class DataPayload {
  properties = new HashMap<String,Object>();
  private Object payload;

  public <T> T getProperty(String name, Class<? extends T> c) {
    if (!c.containsKey(name) || c.isInstance(properties.get(name))) {
      return properties.get(name);
    throw new IllegalArgumentException("The property "+name+" has type "+properties.get(name).getClass().getName()+" but you requested an object of type "+c.getName());

  public DataPayload setProperty(String name, Object value) {
    properties.put(name, value); // not worrying about replacing here
    return this;

So we have the pretty awful property object that is horribly error prone, particularly with String names.

We layer a Facet Object on top of this:

public class UserEntryPayloadFacet {
  public static final String USERNAME="Username";
  public static final String PASSWORD="Password";
  public static final String EMAIL="Email";

  private DataPayload proxied;

  public String getUsername() {
    return super.getProperty(UserEntryPayloadFacet.Username, String.class);

  public UserEntryPayloadFacet setUsername(String username) {
    super.setProperty(UserEntryPayloadFacet.Username, username);
    return this;

  public String getPassword() {
    return super.getProperty(UserEntryPayloadFacet.Password, String.class);

  public UserEntryPayloadFacet setPassword(String password) {
    super.setProperty(UserEntryPayloadFacet.Password, password); // of course - passwords are never in plain text, right?
    return this;
  // If you feel like it...
  public DataPayload unwrap() {
    return proxied;

You get the general idea.  In my implementation I've defined an interface to include an unwrap() method call to get at the payload, and in my case, I'm also exposing the underlying data object properties via proxy method calls, so the facet would have a getProperty(...) call and a setProperty(...) call.  Your mileage may vary.

Sunday, December 4, 2011

Scala vs Groovy

I've been working recently on a project based in Scala and just started to use SBT for the build tool. A significant project I have, my accounting system (G'Kané), is written in Grails.

As I'm flipping back and forth right now, a few things are sticking out.  I'll probably update this as I go along, but so far:

I really like the ? operator in Groovy, getThing()?.doSomething(), only runs doSomething() if getThing() returns a non-null value.  The only way I know how to do this right now in Scala is via a match statement, possibly using the Option[] construct, which is really clunky in comparison:

getThing() match {
  case Some(p) => p.doSomething()
  case _ => null

On the plus side for Scala is case classes.  The same thing that causes the above travesty is also something to be rejoiced in.  I know I can pass a map to a Groovy constructor:

new Thing([name : "myThing", value: 4])

but it's completely type unsafe.  In comparison, a Scala case class has a default constructor with the properties:

new Thing("myThing", 4)

Granted, it does mean the order has to be correct, which is a bit of a pain, but you do get type sanity.

In the black-eye department for Groovy is the bewildering mix of Groovy method declarations and Java style one, and in-between.

def thing = { "something here" }

but if I want it typed, suddenly it's back to almost Java:

def String thing() { "something here" }

Having untyped methods in Scala seems pretty near impossible though, so I suppose it's not quite fair, but:

def thing : String = { "something here" }

in and of itself seems closer to the syntax that would seem like it should be idiomatic for Groovy, though it's actually Scala. This debacle leaves us wondering if we should be calling Bar.thing, or Bar.thing() in Groovy, and it might change, even though the function of the method didn't.

The Grails compilation on the fly feature, whilst nice, at least in theory, is a source of major pain for an old hacker like me who has the Save key-press burned into the brain as a function to be performed after every single change made in a file.  The poor on-the-fly compilation just gets really angry because I frequently save a file multiple times per minute, and also when it's far from working.  SBT has a similar feature, but it's not coupled to my app in progress (at least not yet).

On traits and interfaces: I'm really loving traits in Scala.  They are fun and a neat way to implement cross-cutting concerns in a way that's not quite as all-powerful as Aspects, but yet still code-visible to a mere mortal.  Interfaces in Groovy, I honestly still don't understand how to work that.  I've looked it up a few times, but it either won't stick, or just hasn't made sense to me yet.  For instance, I have a set of services which all do similar things that I would normally group in an Interface in Java.  They are responsible for creating a Mapping description object.  They have a single method columnMap() which returns a ColumnMap object.  I'd create an interface called ColumnMapper or something in Java to describe this contract.  In Groovy, I can create an abstract class, but where does it live?  In my Grails view in IntelliJ, it seems a bit unclear.  Do I make it a service that's just declared abstract?  What happens if I try to bind that abstract class?  If it were Java I'd get a compiler error, I'm guessing I'd get a runtime error in Groovy, and I'm not sure if my IDE is smart enough to figure it out and show it as an error.

Speaking of IDEs, the Scala plugin for IntelliJ 11 is improved, but still has a ways to go.  It gets confused sometimes about a project, and will only let me create scripts, which then turn into the appropriate icon once I create the code.  Bit weird.  Still, given the very weak type binding in Groovy, which leaves the poor IDE guessing more often than not, I'm currently leaning to Scala on the matter of code inspection, at least in IntelliJ.

Saturday, December 3, 2011

OMG <3 Scala

Today was a good day. I wrote a somewhat complex piece of logic for a jump planner for EVE online, and, because of the awesomeness of Scala, it worked as conceived, first time.

This is one of the many reasons I love functional programming. As you take your problem and reduce it down to it's most basic components, strip out logic, simplify to a mathematical expression, simplify that expression, suddenly errors become clear in the very writing of it.

Scala feels so much closer to that expression than Java. So much verbosity is inherently prone to error, writing error free code in Scala seems dramatically more possible than in Java.

SBT and Gradle

Liistening to a not quite cent episode of Java Posse podcast, I heard them talk about two new build tools that are in the running to succeed Maven and Apache Ivy. They are SBT and Gradle respectively.

As with the rest of the Java community who are moving on, one is Scala based and the other is Groovy based.

I'm not going to get into a Scala vs Groovy thing here, as i can't say I know either of them to a high enough level to really make a fair comparison.

I've had a poke at both of them, and so far, I'm really diggin the simplicity of SBT and it's interactive features for working with Scala, which are super nice.

I've only got a small example going with my EVE project, and it so small it's barely worth posting. Having said that, it did clue me in to working with scalatest, which is much more elegant than JUnit. I'm curious to use it also with JUnit and java code, I've heard that it can work well with both Scala sources and Java. The interactive test suite features seem like a really awesome feature, continuous integratinon built right in.