Groovy, the Python of Java

I was a bona fide Java programmer for 5 years before I started working on Aleph Point and Parse.ly. I truly believe that Python and JavaScript are fundamentally better languages than Java for a variety of reasons born out of experience with each of them. (Note: Before this gets marked as flamebait, please notice that not only was I Java programmer for more than 5 years, but I was also a Java open source contributor!) I have enormous respect for the Java open source community, which has produced some of the highest quality modules available anywhere.

Now, don’t get me wrong — Python also has batteries included, and usually, when I think that I’m missing a great module I used to use in Java, it already exists in a much more powerful form in Python’s Standard Library or the wealth of modules on PyPI, GitHub, and Bitbucket. However, I believe in not reinventing the wheel, and so if a great open source tool exists in Java, I will want to interact with it.

One of these modules which we use extensively at Parse.ly is Apache Solr, and its surrounding Lucene project modules. Lucene is an extremely mature framework for document indexing, and Solr is a powerful server-ization of that technology that fits well into complex, mixed language distributed systems. I know there are efforts — like Whoosh — to build fast search engines atop the Python language. And I applaud these efforts — more projects means more competition, and more competition means better products. However, I still believe that you go with the best of breed tools available for production software, and you try not to let religious arguments about programming language get in the way.

Lately, I have come across more and more Java open source projects that have no equivalent in Python, and which I would like to access. Knowing that I wanted to feel comfortable incorporating Java open source projects — beyond Solr, which was already nicely wrapped as a web service — I, at first, thought that I’d be forced to still live among the weeds of complex class and interface definitions, cumbersome Java IDEs, XML configuration files, and (IMO) time-wasting rabbit holes like dependency injection, configuration management, and classpath hell. And then I found Groovy.

The most under-rated language ever

Groovy was released as one of the earliest dynamic languages written atop the JVM, back in 2003. I remember hearing about it back then and not “taking it seriously”, as we programmers often do with important, new technologies. I had the same ill feelings toward JavaScript until as late as 2007, but now consider myself more a JavaScript programmer than a Java one!

In truth, had I read James Strachan’s blog post discussing why he created Groovy, I would have probably paid more attention:

So I’ve been musing a little while if its time the Java platform had its own dynamic language designed from the ground up to work real nice with existing code; creating/extending objects normal Java can use and vice versa. Python/Jython’s a pretty good base – add the nice stuff from Ruby and maybe sprinkle on some AOP features and we could have a really Groovy new language for scripting Java objects, writing test cases and who knows, even doing real development in it.

I had thoughts like these back in my Java programming days, too. I would see features that were standard in other languages — concise list/map syntax, “bare” functions (that could live outside of class definitions), optional/dynamic typing, metaprogramming, a proper REPL, first-class functions — and think to myself, “Aren’t we missing a whole lot by lacking these features in Java?” The more I started to code Python and Java side-by-side, the more I started to realize that many “core” Java technologies were basically created to cope with inadequacies of the language itself.

For example, unit testing “frameworks” and formal IDE/debuggers are more popular in Java than other languages because of the lack of the REPL — dynamic language programmers tend to test their code from an interactive shell as a regular part of programming. Utility libraries like Apache Common Utils were created to cope with syntactic deficiencies around core types like lists and maps. And on and on.

Groovy is a “post-modern language” dynamic language. By that, I mean it is clearly inspired by and learns from the lessons of both Python and Ruby. It is also the world’s first programming language that was written to target an existing community of programmers. Let me explain. Python was not written “to target” C programmers or Visual Basic programmers. It was written as a new language altogether, not targeted at anyone in particular. However, Groovy was written “to target” Java programmers — it is the dynamic language Sun should have built for the JVM as dynamic languages gained popularity and the Java language stagnated.

Groovy is respectful of and cooperative with Java itself. One of its primary design goals is to live alongside existing Java code, even while Groovy’s syntax far surpasses that of Java. In this respect, Groovy plays a very similar role in the Java ecosystem that Python plays in the C ecosystem. However, unlike Python and C’s relationship, Groovy’s relationship with Java is bidirectional. What that means is that one can easily write Java libraries that execute Groovy code (provided the Groovy Language is on the classpath as a JAR), and Groovy libraries can easily execute Java code. In fact, you can even have complex relationships among these two languages, such as Groovy classes serving as base classes for Java classes and vice versa.

However, what’s perhaps most astonishing about Groovy is that it truly got a whole lot of things right. That is, the language simply is extremely well designed, and specifically greased for programmer productivity. It learned from the mistakes of existing dynamic languages, and made a dynamic language that is truly the “best of both worlds”. When I introduced the language to an engineer on my team at Parse.ly, he uttered some words that really stuck with me: “This language was clearly designed by very, very lazy programmers.”

So true. Typing is a chore — let’s do less of it to code without sacrificing readability. That’s the Groovy way.

Over the last several years, it has matured to the point where I feel extremely comfortable writing production code in Groovy, especially thanks to the surrounding ecosystem of the Grails project.

Python is still my muse, but Groovy is my Winston Wolf

A famous character in the movie Pulp Fiction is Harvey Keitel’s “Winston Wolf”. He’s the underworld problem solver — a guy who gets things done with a no-nonsense attitude.

Winston Wolf: If I’m curt with you, it’s because time is a factor. I think fast, I talk fast, and I need you two guys to act fast if you want to get out of this.

Python is a beautiful language and will continue to be my language of choice for production code. However, I simply can’t ignore the enormous wealth of production-quality Java code that has been written for which there is no good Python equivalent. I also can’t ignore the unbelievable engineering effort that went into making the JVM a powerful, scalable, and stable platform for production software. I thought I’d have to throw this community away (for myself and my companies) due to the deficiencies of the Java language. However, I have come to realize that Groovy has proven itself the worthy successor to Java. (Yes, successor — as in, you would be a fool to write your code directly in Java these days.) And personally, I will never work on a Java project again without Groovy. Java is a mess, but Groovy is my clean-up man.

I have also observed that the “ported” JVM languages have failed. This was a surprise to me as it was happening, but in retrospect, is not as surprising. Jython and JRuby are simply not viable long-term open source projects. The target community is only those programmers who know both Python and Java or both Ruby and Java. And there was no great impetus to truly “learn” those environments, because Python programmers felt that they already “knew” Jython, since the language was the same. Is the benefit of having access to Java libraries worth the downside of using a non-standard VM for my primary programming language? Usually not.

Now, while I wasn’t even paying attention — slowly learning Groovy in my spare time and using it for small projects dependent on Java — it seems that the world of independent dynamic JVM languages has taken off. Groovy has not had nearly as much popularity as languages like Scala (used at Twitter and Foursquare) or Clojure. And Groovy lacks “sex appeal”, possibly because no significant large-scale projects have been documented atop Groovy and because the language is an incremental improvement on successful mixed-paradigm languages.

However, unlike Clojure and Scala, the Groovy language can be learned in a weekend by an existing Java programmer with dynamic language experience, and can be learned in a couple weeks by anyone else. And unlike those other languages, it does not try to be a fundamentally new programming language with new programming models. Instead, it simply dramatically improves the Java language itself using a syntax that becomes quickly familiar to dynamic programmers.

When you consider the cost/benefit analysis of learning Groovy — for a couple weeks of my time, I gain access to being able to easily script — and, with Grails, wrap as web services — an almost limitless supply of Java open source code, you realize you’d be a fool not to learn the language.

I like new stuff as much as the next guy, but Java is a language that is still used by millions of professional programmers across the world, and has proven its ability to be the basis for large projects. And Groovy is a 2X or 3X productivity improvement over Java, with full bi-directional compatibility with existing Java code? I’ll take that over a new-fangled “revolutionary” language any day!

So, what’s so groovy about Groovy?

Oh, nothing… just things like this:

["Rob", "Christopher", "Joe", "John"].findAll{ 
  it.size() < = 4 
}.each { 
  println it 
}
// ==> prints "Rob", "Joe" and "John"

Yes, we’re really not in Java anymore. Let’s walk through the pain that is Java step by step to arrive at our Groovy salvation.

First, Java loves boilerplate. It really is an affliction of the language and the community. A great example of this is Java’s take on module imports.

Even though Java has a great standard library that is extremely well-designed and complete (for the most part), it fails to make this standard library easily accessible to you. That’s because all of Java’s best libraries are hidden in obtuse, deeply-nested library locations, and one can never remember where they are. IDEs solve this problem by providing an “Organize Imports” facility, which expert Java programmers use instead of managing imports themselves. This problem, alone, puts Java IDEs on significantly more productive footing than simple text editors like vim.

Groovy says, “this is nonsense — stop the madness”. In Groovy, a slew of commonly-used Java modules are simply imported automatically. If you run into a namespace conflict, that’s OK — we’re all adults here.

// these are all automatic in every Groovy module you write
import java.lang.*;
import java.util.*;
import java.net.*;
import java.io.*;
import java.math.BigInteger;
import java.math.BigDecimal;
import groovy.lang.*;
import groovy.util.*;

Note that groovy.lang and groovy.util add significant utilities above and beyond Java.

Next up are the major data structures used by any modern programmer — lists and maps (aka dictionaries). If you haven’t been doing Java in awhile, you won’t miss this kind of code.

List<Integer> someItems = Arrays.asList(new Integer[] {1, 2, 3, 4});
for (Integer item : someItems) {
    System.out.println(item);
}
Map<String, String> someMapping = new HashMap<String , String>() {{
    put("ST", "started");
    put("IP", "in progress");
    put("DN", "done");
}}
for (Map.Entry<String, String> entry : someMapping.entrySet()) {
    String key = entry.getKey();
    String value = entry.getValue();
    System.out.println(key + " => " + value);
}

Note that the above is about the most concise possible code you can write for these operations using standard Java facilities. In fact, I have even tapped into some obscure features, using Arrays.asList to initialize the ArrayList (leveraging the fact that arrays, but not lists, have a concise initialization syntax) and leveraging double curly brace initialization for the Map, which is an extremely non-standard and perhaps even overly inefficient technique. (The alternative, though, is to write out someMapping.put(key, val) for each entry in the Map — ick).

Let’s look at this same syntax, but in Groovy.

someItems = [1, 2, 3, 4]
someItems.each {
  println it
}
someMapping = ["ST": "started", "IP": "in progress", "DN": "done"]
someMapping.each { key, val ->
  println "${key} => ${val}"
}

Not only is this code more concise, it is also much more readable. The signal-to-noise ratio is much better.

Groovy has great support for Functional Programming and Domain-Specific Languages.

// first-class functions/methods and DSL syntax
clean = { text ->
  text.replaceAll("[^A-Za-z]*", "")
}
clean "my-text-*foo"
remove = someItems.&remove
remove 3
someItems

It retains support for loved Java features like switch statements, but these become much more powerful and concise, e.g. in this factory function that actually returns cleaner functions depending on the passed-in type.

// proper closure, switches and variable interpolation
cleanerFactory = { type -> 
  switch (type) {
    case "text":
        return clean
    case "numbers":
        return { text ->
            text.replaceAll("[^0-9]*", "")
        }
    default:
        throw new IllegalArgumentException("invalid type ${type}")
  }
}

Using the cleanerFactory is a breeze, simply call it and you get back a function which you may then call. Yes, this is basically impossible in Java. To get something close, you’d have to make the factory return “Runnable” instances and then call .run() on the results.

// try/catch and bean syntax shorthand
test = "my-text-*foo-1234"
println cleanerFactory("text")(test)
println cleanerFactory("numbers")(test)
try {
  println cleanerFactory("foo")(test)
} catch (IllegalArgumentException ex) {
  println "cleanerFactory returned exception: ${ex.class.simpleName}, ${ex.message}"
}

Just like Java, Groovy supports classes. But it improves them by adding a concise syntax for declaring proper JavaBeans. The following class has getters, setters, and a keyword constructor autogenerated for you.

// classes and types, if you need them
class Article {
  String title
  String summary
  String content
  Date date
}
article = new Article(
    title: "Parse.ly uses Groovy, oh noes!", 
    summary: "Didier resigns in protest", 
    content: "Keith goes back to .Net, Toms returns to Ireland.  Ahhhh!",
    date: new Date(2011, 3, 7)
)
article.properties.each{println it}

Also notice the special “.properties” property, which allows you to iterate over a JavaBean’s property without using Java’s obtuse reflection API.

// inheritance if you need it
class WSJArticle extends Article {
    List topics
}
wsj = new WSJArticle(topics: ["US", "Finance", "World News"])
wsj.topics.each{ println it }

Inheritance works exactly as expected, and follows Java rules.

// and sure, even call and inherit from Java code
class BetterDate extends Date {
    def getTimeString = {
        return "{this.time}"
    }
}
println new BetterDate().timeString

The interop works beautifully, too. Let’s say you want to now use BetterDate from your Java code. You have several options available to you. The most convenient is Groovy’s “joint compiler”, groovyc, which will compile your Java code and Groovy code alongside one another, making sure all dependencies are resolved.

// and sure, use Groovy scripting constructs with existing Java APIs!
lotsaItems = [1, 1, 2, 2, 2, 3, 3, 3, 3, 3]
println Collections.frequency(lotsaItems, 3)
Collections.shuffle(lotsaItems)
println lotsaItems

The above is an example of using existing Java APIs. Collections is surely a nice static class. Groovy lets you use it to your heart’s content.

The API for the Collections.binarySearch method is written as:

static <T> int binarySearch(List<? extends T> list, T key, Comparator<? super T> c)

Yes, that is “documentation” 🙂 So basically, I need to create a class that implements the Comparator interface, and then I need to pass that comparator to this method along with my list. That will require a lot of angle brackets to get the generics right. Hmm, who cares, let’s just do it the easy way: in Groovy.

// even existing, complex APIs work fine in Groovy, and with less code!
comparator = [
  compare: { a, b -> a.equals(b) ? 0 : Math.abs(a) < Math.abs(b) ? -1 : 1 }
] as Comparator
println Collections.binarySearch([0, 2, 5, 7, 8], 5, comparator)

This shows off the ability of Groovy to easily coerce values using some very intelligent rules. Comparator is an interface that expects a “compare” method — therefore, we can coerce a map with a “compare” key and a function value into an implementation of that interface.

// convenience features abound, like heredocs and safe pointer access
myDocument = """
  This is a long document...
"""
foo = [myDoc: null]
println foo.myDoc?.size() // returns null rather than an exception

If I had a dime for every time I noticed Java programmers using StringBuilder to work around its lack of multi-line strings, or adding extra methods just to do null checks… well, I’d be very rich indeed 🙂 Groovy says — no need to complicate your life with that noise.

// and type coercion that actually works!
set = [1, 2, 3, 4, 5] as Set
5 in set
// ==> true!

Yes, the in keyword. Yes, lists can be converted to sets. Because that makes sense!

// this is a feature I wish Python had -- JavaScript-like map syntax
map = [:]
map.item1 = "foo"
map.item2 = "bar"
map.each { println it }
// ==> "item1"="foo", "item2"="bar"
println "done"

Groovy also learns a little from JavaScript — maps are accessible using either key index or dot syntax. I personally like this, I know some Python people won’t.

// nice range and slice syntax
items = [1, 2, 3, 4, 5]
println items[0..2]
for (i in 0..5) { println i }

Similar to Python’s slice syntax, Groovy supports a “Range” type using two dots. It works where you expect it to: array indexing, for loops.

Using Grails, the standard web framework for Groovy, you can write simple RESTful, JSON web services in just tens of lines of code. e.g., here’s a Grails controller that renders the result of a function as JSON.

package service
import com.thirdparty.library.ContentExtractor
import grails.converters.JSON
 
class ParseController {
  def contentExtractor = new ContentExtractor()
  def index = {
    def article = contentExtractor.extractContent(params.url)
    render article as JSON
  }
}

This will respond to a service like http://service:8080/parse?url=http://google.com with a JSON document containing the Java properties from the article object.

Seriously, how much easier can it get to wrap up a Java library as a JSON web service? Not much.

// in short, a pretty language worth looking at
println "DONE."

Download Groovy and Grails to start playing around!

—

Hi, Hacker News! (11/18/2013 – 10pm ET) There’s one discussion thread about this post and another thread about the recent Groovy 2.2 release, both sending traffic from the frontpage. Feel free to also check out some other posts I’ve written that are relevant to startups:

• What One Does
• The Startup Diet
• Build a web app fast
• Why Startups Die

… or flame me on Twitter at @amontalenti!

Also, do you want to use the “Python of Python” in your everyday work? (That is: Python.) Well, Parse.ly is hiring a Senior Software Engineer, ideally in Eastern or Central Time Zone. This is a remote position for our fully distributed team.