Settings

Theme

Show HN: God mode in production code – a new way to debug in Scala

takipi.com

146 points by chookrl 13 years ago · 83 comments

Reader

mpeg 13 years ago

I'd give my firstborn for a debugger/profiler like this in .NET

saryant 13 years ago

I've been tracking down a NullPointerError in Play's new JSON inception handling. Eclipse's debugger has gotten me nowhere, I look forward to seeing how this one works!

spuz 13 years ago

This looks simliar to Chronon which is a similar debugging tool that works on Java (though not other JVM languages) applications.

  • LinaLauneBaer 13 years ago

    I have used Chronon for several weeks during a very strange project. The project was about integrating a highly complex financial system in our own infrastructure. The challenging part about this financial system was that it was written in a multitude of languages (Java, Perl, C, C++, ...) and that it had to communicate with external services and it had hard timeouts built in which were enforced externally. So if something went wrong and I wanted to find out I had only 20 seconds in the debugger to find out because then the timeouts would kick in and there was nothing I could do about it. With Chronon I was able to step through the code after it had already run with no time constraints. Saved me weeks...

    Chronon is also useful to find all thrown exceptions - even the ones who are catched properly.

    I am not an employee of Chronon Systems and I only used their tool once but it helped a lot.

  • nivstein 13 years ago

    Unlike Chronon, Takipi is built for monitoring your JVM in production, imposing minimal and predictable overhead.

elvinj 13 years ago

I know that at least from my experience debugging our apps in production did become harder as we scaled out in terms of servers, users and code. I do wonder if its something that should be handled at the code level (by being more defensive, taking a more immutable approach) or by having better debugging tools. Anyone knows if this supports Ruby?

  • wereHamster 13 years ago

    Or by using a better type system. Or by using the type system that's available in the language. Because it's all too easy to ignore the types and for example call `get` on a Scala Option value, or `fromJust` on a Haskell Maybe. Both will throw an exception at runtime if the Option/Maybe is empty.

    • elvinj 13 years ago

      Type safety does help a lot. The real hard ones I found to track down are the ones where you have to chase for a particular state to re-occur and you can't get a repro from ops or qa. At least for me these are the main bummers...

    • taeric 13 years ago

      Of course, the retort there is as much "write simpler programs" as it is "use the type system." I've seen and been a part of many a tangled knot in a type system that, while ostensibly safe, was so much more involved to do the simple things that I think it caused more bugs.

      • wereHamster 13 years ago

        I'm not saying strongly typed languages are bugfree. But they eliminate whole classes of bugs. The OP was asking about programming techniques to improve code quality or tools to help with debugging. Well, a type system is such a technique/tool that checks for bugs at compile time.

        • taeric 13 years ago

          Indeed. Apologies for what was too trite of a response. I can't explain why, but I have been getting heavily lured by the dynamic typing of lisp, lately. I blame that I am finally watching the SICP lectures online. :)

  • timdorr 13 years ago

    Passenger Enterprise supports a live irb and debugger: https://www.phusionpassenger.com/enterprise#debugger_support

gurdo 13 years ago

There's a very strong founding team around this company. I don't do Java too much these days, but if I were, I'd definitely use Takipi.

shpxnvz 13 years ago

Signup is broken right now, so maybe someone else can answer. From the website it appears that the debugger is a SaaS app that connects to the agent on your production box from their servers... is this the case?

  • nivstein 13 years ago

    The product consists of a slim JVM agent which connects and offloads the heavy crunching to Takipi's backend.

    • shpxnvz 13 years ago

      Given the nature of JVM agents, this means that the Tikipi backend, should it choose, may access all the data in the production application. Correct?

      ETA - Thanks, missed that page on the site.

      • nivstein 13 years ago

        The moment you sign-up for Takipi a secret AES-256 key is generated for you (and never stored by Takipi in any way or form, and you manually enter it on your machine when installing the agent). This key is used to encrypt every piece of application data which leaves the machine. The source code is also encrypted similarly. The source code and the data are only decrypted in your browser when viewing the details of the event.

        You can read more about security at Takipi here: http://www.takipi.com/features.html?nav=security

        • sillysaurus 13 years ago

          This makes no sense whatsoever. Their application is doing the encrypting. That means their application can do whatever it wants, including making it trivially easy to reverse the encryption.

          I get that the intentions are good, but https would provide exactly the same trust guarantees, and ultimately it bothers me that they would try to mislead people into a false sense of security like this. If their application is encrypting data which is sent to their servers, then that means their severs have complete access to your data. (Whether they choose to make use of this power is up to them. They certainly have it, though.)

          Encrypting the data prevents it from being leaked if their servers are hacked and their database stolen, which is good. But acting like they have no power to access your data is absurd.

          • jcheng 13 years ago

            I think the distinction is meaningful. A rogue employee would not have the power to access your data (as we've seen at Google for example[1]) unless they were able to sneak some code into the application code (as, sadly, we've also seen at Google[2]). That seems like a significantly higher bar and also would require a great deal more premeditation. Not to mention the possibility you mentioned of them getting hacked (which would be a FAR bigger concern to me).

            Now that I think about it, I never did move my stuff from Dropbox to SpiderOak, did I... hmmm, maybe today.

            [1] http://www.theatlanticwire.com/technology/2010/09/rogue-goog... [2] http://tech.fortune.cnn.com/2010/06/06/google-blames-wi-spy-...

          • baboo 13 years ago

            They are probably either lying or incompetent.

            If they really had no access, then their server cannot do any "analysis" beyond storing it, and the app could as well just store it locally, or with Amazon S3 or whatever storage system the website already uses.

            Obviously not everything is encrypted, or otherwise they don't realize that there is no reason for them to provide a simple storage service, considering most websites already have some way to store data.

            Or maybe they want to hold the data hostage.

            • sillysaurus 13 years ago

              No, there's no reason to go looking for conspiracies. Nor do I think their intent was malicious. I think they genuinely believed that this was beneficial. We should support them if they change their presentation from "we can't access your data" to "we're not evil, so if you trust that, then use us." We should let their actions speak for themselves.

            • pron 13 years ago

              Your tone in all of your comments on this post is inappropriate for HN. You have a new account so if you are indeed new here, I suggest you read other comments on the site to get a feel for the community.

            • jcheng 13 years ago

              See "Graph Analysis" section on the linked page, they make it pretty clear that there is info that is sent unencrypted but those parts aren't useful for reverse engineering. The data and source code itself sounds like it is encrypted though.

      • mseebach 13 years ago

        For tons of applications this falls somewhere in the range "impossible" -> "totally unacceptable" -> "very worrisome, requires extensive debate before signoff"

trailfox 13 years ago

Wow. Wish I could deploy this sort of tooling behind a firewall.

  • nivstein 13 years ago

    The agent only requires outbound HTTPS over port 443, and fully supports proxies.

  • wheaties 13 years ago

    Ditto, just sent an email asking about this. There's a number of industries which frown upon allowing any sort of code or production data to be sent to a 3rd party host.

koker 13 years ago

Going to give it a go on our servers. We have a relentless Heisenbug we've been chasing down lately, which only happens in production.

protomyth 13 years ago

Many of the Smalltalks allow hooking to a running image to do live debugging.

  • seanmcdirmid 13 years ago

    No they don't. They just allow for fix and continue; they give you no insight into why a problem occurred because they provide no history.

    • protomyth 13 years ago

      uhm... that's not universally true - I worked with some folks that had the history when they did it. Not sure if it was vendor provided or home grown, but it was the call history and call stack.

      • seanmcdirmid 13 years ago

        Call stack is provided in most debuggers these days. Are you honestly saying you saw something comparable to the linked article in some Smalltalk system? If so, I would be very interested in some kind of reference.

        • protomyth 13 years ago

          It was a VMWare debug image and it had all that information although it was not as fancy. It did have a good amount of custom code.

zaptheimpaler 13 years ago

This looks awesome! Is there anything like this for local code though? The IntelliJ/Eclipse debuggers don't seem as well designed as this one.

  • nivstein 13 years ago

    Takipi may be installed locally; the same installation process will work. However, it does not provide live debugging like in your IDE, but rather a post-event drill-down (a "time travelling debugger"), as it is designed for testing/staging/deployment environments.

orenbarzilai 13 years ago

What about python?

  • spatz 13 years ago

    From the FAQ:

      Which languages do you support?
    
      Takipi works at the JVM level. This means we support all languages running on the Java Virtual Machine. These include - Java, Scala, Groovy, Clojure, etc.. Click here to learn more about viewing Scala code in Takipi.
      We plan to support more software VMs such as CPython and Ruby MRI in the future.
  • thejash 13 years ago

    Just FYI, most of this is a lot easier in python. You can get the full stack trace and all of the values of all of the variables when there is an exception, something that is orders of magnitude more difficult to do with Java/Scala.

    You'd have to be a bit creative to get the "full variable value history" stuff working, but it would probably be possible with some hacking on the setattr methods for your objects (assuming you have full control over the system)

    I had a bunch of code like that for my previous (python) projects. I've really missed it now that I've moved to scala/java, so I'm pretty excited for this project.

eli_gottlieb 13 years ago

You've released? MAZAL TOV! I've been stalking you guys for a while now.

  • irisshoor 13 years ago

    Thanks, you're welcome to take Takipi for a test drive and let us know what you think :)

drorweiss 13 years ago

Looks great!

szeevim 13 years ago

likes the monsters

stasix 13 years ago

looks promising.

vovafeldman 13 years ago

Looks cool!

bizkeep2 13 years ago

US-based?

baboo 13 years ago

"to log and send data to Takipi's analysis servers" ???!!!

Is this for real?

I'm surely going to send all my private code, data and who knows what else to your servers! I have been waiting for this since forever, and I'm glad to have this opportunity now!

And... it slows everything by 5% constantly?!? To basically take a core dump for later analysis? (in case it's not obvious, such a thing should have no overhead whatsoever when fatal exceptions are not thrown)

Lol, is it 1st April again?

  • hmottestad 13 years ago

    You know, preferably it should actually make your servers run 5% faster :)

    Seriously though. Did you really expect a tool that continuously monitors your app and can produce stunning and relevant information about states leading up to a crash to have no effect on performance?

zer0gravity 13 years ago

Tesla said that he could simulate the behaviour of his contraptions to the smalles detail in his head and without exception, they ran that way once he built them for real.

I think that a good programmer should be able to do this with his programms, and without flattering myself I can say that in 99% of the cases when an error happened, the answer occured to me almost instantly or after a few moments of thinking and trying to simulate what happened in my head. I have to confess that these were instances when the whole code was mine. Working on code you don't really understand is a major mistake. But this is another issue..

Sure it remains the 1%.. but my point is that developing the thinking abbilities instead of debugging tools is a far better alternative.

  • millerm 13 years ago

    This is just an observation and not meant to be an insult. I am just using your comment as an example of why I believe you are incorrect in your final statement.

    In your post you have several misspelled words (smalles, programms,abbilities). In only a small number of lines of text you have many errors. A spellchecker is a wonderful tool too and you would have benefited from it. Imagine this now being hundreds, thousands, even millions of lines of code. A simple misspelling in code may lead to a compile error. It may actually be a call to a different function or perhaps just the wrong seed value was set. Saying that only thinking about something rather than using a tool that can help you is just a strange and dangerous advice. Not to mention quite expensive. I don't believe the point of this tool is to "debug your way into a new application".

    Again, this I just my opinion and I used your comment as my example of how tooling is important. Your misspellings could have caused confusion or a complete misunderstanding of what you were trying to convey. The same problem exists in the programming world. If you say the wrong thing it's very difficult to find out what happened without digging in. Any tool that can help you achieve the answer faster than yourself is a good tool.

    As for the actual Scala tool, I like the monster graphics. I think I need to ask them for a shirt.

  • hmottestad 13 years ago

    So you don't need any tools. Good for you.

    For the rest of us, having a tool to step through complex things like threaded recursion is really convenient (not that I would ever consider threaded recursion in any situation).

  • lmm 13 years ago

    Tesla was a genius; he often bemoaned the way Edison would perform a hundred experiments to discover something which was (to him) obvious if he just thought about it for a bit. But you know what? Edison succeeded where Tesla failed, and a good part of that can be attributed to reducing the art of inventing to an almost mechanical process, leaving his intellect free to concentrate on more important matters.

    It's the same with debugging. I've worked with many programmers who took your attitude; some of them were more talented than I. But I fixed bugs faster than them, and got more done, because I let the tools take care of figuring out what was happening in a program to cause a bug, leaving me more time to think about other things.

    • zer0gravity 13 years ago

      I'm not shunning tools. I'm just stating that taking time to really understand the overall architecture, and allowing yourself some time to think at the problem before jumping into debugging can take you very far.

      Now, depends what you understand by success, but for me success != productivity, and I would say it has more to do with the real long term value you deliver...

    • ZenoArrow 13 years ago

      "Edison succeeded where Tesla failed" Edison was probably a better businessman, but a better inventor? Are you sure?

  • campnic 13 years ago

    Of course, everyone would prefer to know the answer to their issues without any tools. As your code base becomes contributed to by more people, grows in size and leverages libraries you don't write, it becomes less feasible to always divine the state of the entire program. As component of a programmers skill set as a whole, better tools produce better results.

  • the1 13 years ago

    you haven't seen aged code base with 3000+ JSP with many ways of accessing different data sources and no coherent paradigm in general.

    • zer0gravity 13 years ago

      I must confess, I'm a coward. If I would ever see something like this I would run like hell. Programmers all over the world are taking too much bulshit.. seriously..

    • willismichael 13 years ago

      I was feeling all smug thinking that I could one-up you, but in the end you win:

      find -name "*.jsp" | wc -l

      1344

  • zer0gravity 13 years ago

    Wow, I didn't know about the down votes. Nice lesson.

    I guess I could have formulated this differently. Attempt 2:

    "Sure more powerful debugging tools is a great thing but the best ones are those you never use".

  • octo_t 13 years ago

    I can see you've never had to debug multi-threaded code or use 3rd party software ever or a project too large for one person to understand or a misconfigured webserver on the other end.

  • seanmcdirmid 13 years ago

    Why require me to think like a computer when the computer is right there and can think for me? Just because I can do math in my head doesn't mean I should.

  • jacobparker 13 years ago

    Tesla was also nuts. So, there is that.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection