Offbeat: Scala by the end of 2011 – No Drama but Frustration is Growing
gridgaintech.wordpress.comMy general feeling is that there is a business-model problem in the tech industry. The most important technolgies are underfunded. They are created by smart people in their free time, or by companies who do not have enough money. For example I've looked at Scala's Eclipse plugin: it was a horror. It is just question of money (hiring a bunch of good developers) to create IDE support which is not totally buggy and not horribly slow.
If 1/100th of the VC money going into social startups would go into actual technology companies, if it were an accpeted business model to sell (maybe read-only open source) technology for money, then the tech landscape would be totally different.
TL;DR: to replace Java, a lot of directed professional development effort would be needed: money would help a lot.
If you compare to where the previous generation of solid PLs and infrastructure research/development came from, I think what's missing currently is well-funded industrial research labs where such developers used to be able to park themselves while still releasing their languages/compilers for free, sidestepping the business-model question.
Not all languages came that way, but some of the major ones did: C was Bell Labs, C++ was also Bell Labs, Smalltalk was Xerox PARC. The main contemporary example in that vein is probably Go coming out of Google, though it remains to be seen how mature and widely used it'll get. For compiler projects, V8 coming out of Google and LLVM now being funded by Apple are two examples. Java is an example developed by a company with an eye towards product/monetization rather than as a research project, but is probably these days seen as a cautionary tale of why not to approach PLs as a business opportunity, because Sun pretty convincingly failed to monetize it.
Microsoft works that way with C#, VB and F#. And they even fund Haskell research. Judging by the quality of their output, it seems to be working very well.
True, those are also good examples, though with the non-Haskell examples it's a bit closer to the Eiffel model, where a company develops a high-quality proprietary compiler/environment to sell.
The market has in big parts shown that the kind of development we have know wins out. Sure there are some people left who can make money developing languages but most that did this died.
From my understanding, Sun's Java division was actually profitable. However, their Java division was a very small part of the overall company, and it was their hardware divisions that tanked.
One problem with infrastructure technology (programming languages, databases, IDEs, etc) is that all of the above is expected to be available for free by developers. So there is little incentive to develop a business around infrastructure technology. Sure people still do it, but not like the 80s when people still paid for PLs, IDEs, databases,etc. These markets have been drastically devalued by open source.
The problem isn't that developers "expect" this stuff to be free. It's that the quality of a particular development platform is related to how many people are using it - how many tutorials are there out there, how much sample code, how many libraries, etc.
There may be some social bias against paying for stuff, I think partially because people got burned with vendor lock-in. Technology changes so quickly, it's hard to know what a tool is worth before the fact.
But there's also a very real economic reality at play here. The most valuable platforms will be the ones with the most people collaborating on them, and that's always going to be the free or open platforms.
As you move up the infrastructure chain, I think people are more willing to pay for useful libraries or services, especially for ancillary stuff. But what rational person would build their business on something they didn't own?
I agree about the former state of the Eclipse plugin; that's why Typesafe has spent considerable resources in changing that. I encourage you to give a try to the current release candidate of Scala IDE 2.0:
http://www.scala-ide.orgThanks, I will have look at it.
I pretty much like the Eclipse plugin. The IntelliJ one is still better, but the Eclipse one definitely gets its stuff done.
Sounds a bit like you used an old version ...
For me, the eclipse plugin is crashing (or the whole IDE is freezing for a few seconds) too often when I try to get code completion. On the other hand, in IntelliJ the Scala plugin is pretty good! (Use the nightly builds!)
Again, Scala is just another tool it's evolving and it's a very nice tool to know...
Of course Java will always be around because it's easy to learn(I earn my money with it BTW) and it protects you from all kinds of stupid errors that newcomers could do in C , C++ or Javascript (memory management, Static Typed, etc...) but Scala provides a very flexible way to code a way that most Java devs are not used to, and therefor it takes a while to learn, but when you do learn it's a Joy I am a proof of that I really like coding in Scala while java is like my work horse, scala is what I use for my fun coding.
BUT... as an young language it have some major drawbacks, like lack of a proper IDE, slow compiling time, lack of Frameworks, lack of documentation and so on... but I am really happy to see Typesafe writing the problems down and fixing them when possible, it will take a while but when it's done it's going to be a very good language to know, and they are slowly I agree but fixing it.
i'm in the same boat with you. anyway have you tried IntelliJ? its Scala support rocks, and combined with JRebel (free for Scala), it can help reduce development time.
anyway i'm very sad to hear about current situation. i've been secretly wishing scala will become big.
Thanks I will try it. About Scala maybe it will, just give the Typesafe guys some time( or help them) and hope for the best :D. I myself will not quit studying it because of this, I do it because I like it.
I downvoted you because your 2nd sentence consists of 107 words.
I was missing a goal to pratice my Scala so I wrote a script to find if you follow your rule, I stopped after getting something bigger than 107 words in order to not hurt HN.
After several 90+ words paragraphs I found this 138 words beauty :D.
" Maro 321 days ago | link | parent | on: "Nothing like this will be built again": a tour of...
When I was getting my degree is Physics we had a lab in the "student reactor", which was still in a large building of its own. As part of the lab we got to sit down to the control console of the reactor and control it, on our own. I remember I was turning one knob increasing the power output when suddenly there was a loud noise, everything shook, and the power output went to zero. Turned out it was the failsafe (I think cadmium rods dropped into the core) as I was too agressive with the controls --- it was a normal occurence for students. (All modern reactors are designed in a way that you can't overdrive them, they'll automatically shut down.) It was great fun and a story I'll be telling for the rest of my life. "
P.S: I didn't count the --- as a word !
Maro complained about a single sentence, not a whole paragraph.
Point taken, maybe I could rewrite the script :D. Just kidding. I just wanted to show that a constructive argument like the one you made is appreciated and if you down vote me with that I will be glad, but the point that he gave was not useful to make my sentence skills better :D
This is the best reply I have ever seen to a grammar nazi comment. Kudos.
Thanks but it's not about "grammar nazi", English is not my first lang and I appreciate when people help. I wanted to point that it was a silly rule.
Ironically, that paragraph is the best argument against Scala. Scala doesn't have have failsafes for IT.
Thanks for the explanation, but where is the rule in English that states that a sentence cannot have 107 words ? I see posts with far more than that everyday here.
You should probably have some kind of punctuation after the word "code", and start a new sentence after the word "Joy". But it's not worth downvoting you.
He practiced his Scala to get back at me, which is pretty cool =)
Totally worth the downvotes my comment receives.
+1 for that and not to get back, not mad or something at all, did it just to ask for a better explanation next time :D
Would it be wrong to think of Scala and Clojure as the F# of the JVM?
I had been thinking of Scala as though it was Java++ (in analogy to C vs C++): a language that adds a few features on top of Java for better abstractions, but regular Java programmers can basically program Java in it (and read it as though it was Java). However, it seems like truly idiomatic Scala code involves heavy use of advanced functional and type system concepts that cannot simply be glossed over.
Will Scala and/or Clojure end up as languages in the JVM used by a small number of senior developers to be more efficient on particular problems (like F# on the CLR) rather than the next big language used by everyone (like C++ superseding C for most problems)? And, is that an accurate portrayal of how F# is currently being used?
I would say yes. From C to C++ you can use the same tools and everything. Java to Scala is still a new language.
more likely scala than clojure - i think a lot of the clojure community are there for the language, and see the jvm as incidental (or even an annoyance). newbie questions on the clojure mailing list are divided between "new to fp" and "new to the jvm ecosystem", and the clojure-in-clojure (with an eye on eventually targeting multiple platforms with a core layer) project has a lot of excitement around it.
Interesting that they feel this way. As a Groovy enthusiast, it feels to me like Scala's gotten all the press/mindshare over the last year, while Groovy is potentially doomed to sideline status. Lift gets some praise, and Play! and Roo are the new hotness, while Groovy/Grails doesn't get the love I'd like it to.
Probably all "alt.java" tech will remain somewhat fringe, simply due to inertia. And although I wasn't in the Groovy community at the beginning, I do think that Groovy, in some small part, helped further along the "non Java on the JVM" story, giving people solid examples of using alt.java languages without having to leave the JVM altogether.
It's Groovy still, like, slower than Ruby? That was always the big attraction with Scala - all the fun stuff, but less than a 5% performance penalty over vanilla java.
It's faster than it was a couple years ago, but doesn't win all the benchmarks out there, no.
That said, I've got Grails apps running in production just fine, and to whatever degree noticeable speed is an issue, it's almost always down to database indexing.
As Hamlet said, I don't think execution speed is the primary reason people aren't using Groovy. Personally, I think all the "alt.java" languages are fighting for attention from the few people left in the Java/JVM arena who are truly interested in exploring new ideas/techniques. Many Java devs who were interested in getting better years ago left for other platforms, (and many of those people will likely jump to new platforms every few years out of curiosity more than anything else).
You're assuming that the only people interested in these languages are only people locked in to Java-the-platform. That defiantly isn't the case, especially for Scala and Clojure, as they offer combinations of features and language traits that aren't really available elsewhere.
Is speed /really/ the reason people don't use Groovy?
If that were true, then I think JRuby be more popular. It is faster yet remains dynamic. There are various efforts to make Groovy fast, but the ones I have seen require you to turn off some of the metaprogramming capabilities.
True here. Groovy has truly pathological performance in some areas.
I wrote a web service that read and processed XML documents. They were large, but not enormous... Couple hundred kilobytes, usually. My first cut, using Groovy and its built in XML DSL, took up to 15 seconds to process a single document. Rewriting in Java/StAX, it was a couple hundred milliseconds.
From then on, my MO was to write my code in Groovy and then rewrite anything that got called more than a few times per request/run in Java. Groovy's fine for glue code, but its performance was a real barrier to regular use.
As for JRuby, I suspect it's because no matter how good JRuby is, it's a second-class citizen to "real" ruby in the Ruby ecosystem, and to built-for-the-JVM languages on the JVM side. It's a great product, but it does have a few impedance mismatches coming from either side.
Groovy is still pretty slow, but last I checked the Groovy++ compiler can take unchanged Groovy code and compile it so that it is often as fast as Java.
unchanged? Are they doing massiv typeanalisis? What are the tradeoffs? This seams to "to good to be true". Everybody else that tries to staticlly compling dynamic languages has more or less failed.
Groovy still seems slow. I used it as the template language for webapps and it was slow. I switched to Japid and rendering was lightning fast.
Just being curious, how's it compared to JSP or StringTemplate?
Might be too late for reply.
It's better than JSP, clearer and simpler syntax, better support for componenization with shared tags, layouts, and includes.
I have no experience with StringTemplate.
It seems like I get calls from recruiters every other week looking for people with Grails experience.
I used Grails for a project four years ago - I think it made a lot of sense then. We wanted a simple web UI for something that needed to deeply integrate with some java libraries. We had just done a relatively simple UI with Spring and it was pretty amazing how much better it was to work in Groovy.
However, these days I don't see why anyone would pick Grails over Rails. You can deploy rails on the jvm just fine, and it's much more mature. There are tons of tutorials and example code out there.
It's not a performance thing - it's just that ruby/rails is pretty mature now and runs on the JVM very well.
I don't really know much about Scala or its ecosystem but I'm curious, what do they mean by:
"Ridicule about parallel collections in 2.9 (one of its major new features)"To sum it up (feel free to add corrections) one guy put a blog post saying that Parallel collections had many issues, etc etc... and the reality was that he misunderstood them somehow.
As one guy put it, the collections do nice work but you have to know what you are doing (not all work can be broken into fragments that can be computed in parallel). He was kind of expecting otherwise.
Not saying they are perfect, just what happened (more or less)
I used them a bit with some image processing. Was awesome to see all 8 cores pegged w/ one line of code!
That does not say much, without actually specifying how much data is processed. It's great if you can spread calculations among 8 cores, but if a single-threaded C process provides better performance (let alone a parallel C implementation), what does it say?
Could you provide more data points?
(By the way, in C such loops can typically be parallelized by adding one OpenMP pragma.)
> if a single-threaded C process provides better performance (let alone a parallel C implementation), what does it say?
It doesn't say much until you define what's most important to you. Is it more important that you can use Scala and get your code working more quickly or do you have more time to write and debug a C program and do you really need to do that for performance reasons? Does the code need to function as part of a larger Java/Scala or C application, etc. In other words, and as always, you need to pick to right tool for the job.
The JVM is very fast now days, in my work the issue of performance vs. C never comes up. Obviously there are still valid uses for C these days, but you can get very decent performance from the JVM and that often means you can finish your work faster.
Java or Scala?
list.par().map(f);I've found that if the list is very large, using list.par.map(...) hits the memory limit & then simply hangs.
I process upwards of a million facilities ( a facility is the exposure on a commercial loan a bank makes to a client ), trying to forecast their expected loss over 12 future quarters.
I rewrote my code like so :
val facs:List[Facility] = ...populated via jdbc query ///
val N = 1000
facs.grouped(N).foreach( group => {
})group.par.map( fac => expectedloss(fac)Now the outer loop is sequential but the inner loop is parallel. It chugs along, and processes the entire million plus facilities! I get some control via the N. Currently my N is 1000 and all 8 cores running at 100% capacity, but I can slow down the workload with smaller N.
yep, that's what I did!
Given the amount of comments on Scala (positive or negative, that's irrelevant) I'm not so sure it will be a fringe language for long. It must be somehow relevant for so many people to spend so much time talking about it...
I disagree that the positive/negative is irrelevant. Much of the discussion around Scala has been around how complicated it is, and ensuing discussions about how the person who thinks it's confusing is not smart enough, and how Scala is the "right way" to do things, because it's elegant, etc.
Having a lot of those sorts of comments I don't think will guarantee future success.
I am a half step above a blub, but I've been doing some work in Scala and generally enjoy it. I do, however, think the Scala community needs to have a more convincing reply to the complexity argument than "You aren't smart enough to see its brilliance".
My fear is that if things don't change, Scala will fall into a perl-like situation where it won't be able to shake its reputation and people will reject the language out of hand before taking a look at what it has to offer.
It's too late for that. Scala's community has been branded as a bunch of condescending intellectuals. Just like Ruby's community was branded as a bunch of pierced, macbook users in 2005. Neither are really true -- but first impressions mean everything.
Comments like that always remind me of this image...
http://www.picshag.com/show.php?f=pics/112010/java-c-php-rub...
To be fair, most of the "not smart enough for Scala" comments come from people who don't like Scala and complains that the "average Java programmer" (whatever that means) won't get it ( which I find very insulting )
Exactly. Those “type systems experts” and “academics” who usually get bashed for mocking “Java Joe”s are normally those who say “have a look at it, it isn't hard”.
I meant irrelevant to the fact that they are talking about it. Obviously, if they mention negative points on the language (and those points are true) it's bad for the language. If it's pure FUD it's also bad, yes.
But even if it's only to spew hate, the fact that so many people talk about it, must mean something. Too much attention for a fringe language...
What makes it even more disturbing is that those who say “it is too complicated” are the arm-chair experts who never used the language (just have a look at the recent flame baits). :-/
A lot of people spent a lot of time talking about Lisp machines, 4GL, web portals, push technologies... the list goes on.
4GL like SQL? Push technologies like SMS, IM, and websockets? You could use better examples :)
I think I got the number wrong; 5GL was the "generation" that really fizzled. As for "push technologies", that is of course a much too broad term. I was thinking specifically of PointCast, which I believe brought the technical term into the business-hype limelight.
I like Scala, sad to see that it is not gaining the required momentum. There are many people complaining about Scala and we don't see many people who are using it successfully talk about its abilities and elegance, I would love to see the scala experts/Typesafe respond with something that demonstrates the usefulness/applicability of Scala to the masses, and may be Typesafe/Partners can offer an easy to follow video tutorial?. I am also a little bothered when the Scala community answers these criticism with - "You are wrong.You haven't learned it properly" instead of trying to help.
I also would really like to read more about successful use of Scala in various companies. Twitter uses it, but is it happy with Scala?
that is the point, we should also think on how much of their software stack is built on Scala.
> "You are wrong.You haven't learned it properly"
Do you have a link? Maybe this unhelpful behavior can be figured out in person.
I always experienced the Scala community as a very friendly and helpful one, focused on getting stuff done.
Imho the talk about “bad behavior” outnumbers the actual occurrences of bad behavior by a few magnitudes.
> Do you have a link?
http://blog.joda.org/2011/11/scala-ejb-2-feedback.html?showC...
I don't question the intent of the Scala community. There's no mystic group of evildoers there trying to degrade newcomers. But the mailing list is very intimidating. It doesn't feel like a place where newbies can come in and ask "How do I do a 'foreach' loop?".
While it takes a while to understand how he approaches problems and solves them, his offer is honest and his judgment is correct.
He basically says that if the blog author actually had a look at the language before bashing it, he would have found valid issues.
I agree with that. The author's complaints are unsubstantiated and don't reflect the issues people encounter in reality.
It's exactly what you asked for, an example of "You are wrong. You haven't learned it properly." You are also saying this kind of response (You haven't learned it properly) is appropriate. This attitude is what turns people away.
You can be honest, correct and still turn people off.
The problem with Scala is that you end up having to use lost of Java libraries.
And those were not written to be used with Scala.