Log driven programming (2013)
antirez.comhttps://orgmode.org/manual/Capture-_002d-Refile-_002d-Archiv...
> An important part of any organization system is the ability to quickly capture new ideas and tasks, and to associate reference material with them. Org does this using a process called capture.
This, and it's something baked into Spacemacs (easy to replicate with a vanilla Emacs, thought): working within a projectile-initialized folder, hitting <leader>po will open a TODO.org at the root folder of your project, where you can write anything you want (including inserting an Org-mode link to the file/line you were editing), and save/switch back to your file in a handful of keystrokes.
Downside: you end up with too much notes.
I use postits for this.
I used to use TODOs, but then if I wasn't looking at the code I forgot about it, and a lot of the stuff didn't have a spot it needed to go. It was just stuff I thought about while I was doing something else.
Daily (or twice daily) I reorder and prioritize. Every week or two there's a postit cleanup party where old postits get trashed or logged. A lot of the stuff that I trash isn't bad....it's just not in scope for the stuff I'm currently doing.
ADD: One of the beauties of postits on my monitors is that when my mind wanders -- as it tends to do -- I'm naturally looking at other stuff I should be thinking about. So when I postit stuff and put it somewhere in my field of vision, I'm telling my subconscious to start working on what I want to do with this stuff.
A couple of years ago I started using TODO:HIGH, TODO:MED and TODO:LOW.
TODO:HIGH is for things that need to be completed for the current feature or bug fix.
TODO:MED is for things that should be completed before the next public build.
TODO:LOW is for stuff that would be good to get done eventually. (A function is complete but doesn't read well so I'll add a note to refactor it at later date.)
When writing new code I try to get something up and running ASAP by filling in functionality with broad strokes. I'll go back and fill in the details once the larger picture is in place. With less code written, it's easier to refactor (or restart), if the design is not quite right.
Writing lots of TODOs helps ensure nothing gets forgotten about.
Picking an easy TODO from the backlog can be good way to warm up when starting the day too.
I'm used to working solo however. Not sure how this approach would go down on a team.
Exactly. I don't see why you would do this in another application. Doing it in another application means you also have to take note of _where_ the piece of your TODO is. That means filename and line number?
While I don't use priorities, I do liberally use "// TODO: <problem statement>" if it's something I need to do that takes me out of the current problem. Then when I'm finishing up a feature or bug fix, I just walk past the list of TODO's.
A recent example would be: On this new project I was on, while implementing a feature, I didn't know how resource files were handled yet -- so I hardcoded some strings with a "// TODO: Put these strings in resource files". Researching the way resource files are handled will pull you out of the current problem, while it's something that can easily be solved later.
That's what I do as well. Also, in OO languages like Java or C#, you can throw "NotImplemented" style exceptions. So that if you mistakenly wrote "TO DO" instead of "TODO" and can't find it statically, the runtime will let you know.
I create stories in JIRA. It's a lot more effort but the defects are at least visible to management and actually stand a chance of getting scheduled. You can also include a lot more context than you can in a comment.
This is my workflow and has been for the past 10 years or so.
Whenever you think of something else that needs doing, write it down so you stop feeling the need to keep it in mind.
Excellent; but does it ever get done?
Yes. In particular, if you use the same place to note down things you want to do in the first place. Then it boils down to continuously reviewing the list and picking the next thing to work on.
This is a reason why I generally avoid using TODO/FIXME type comments in my code. I'm much more likely to forget about them, whereas I always need to look at my org-mode project file to see what I need to do. So that's where I put them.
Ostensibly, this is what TODO comments have always been for.
Realistically, I guess it's probably better that these types of comments remain out of the code itself unless you and your team can practice proper rigor around them, e.g. using prehooks to enforce that no TODOs are left around before committing or merging.
>I guess it's probably better that these types of comments remain out of the code itself unless you and your team can practice proper rigor around them, e.g. using prehooks to enforce that no TODOs are left around before committing or merging.
Why? If they're "out of the code" they still can be ignored before committing or merging just as well...
That's a silly statement. Yes, they can be ignored, but perhaps for many people they're less likely to be. That's my experience, anyways.
I could train myself to regularly grep for TODO comments, and perhaps I will, but so far adding them to my actual 'todo app' is much more effective.
I'm more likely to write a stub implementation around more complex behaviour that still functions in a minor way first, especially if I want to observe that some component functions even in absence of a complete implementation of another component.
My personal layers of organization:
Stubs with TODOs for staking out an interface to be implemented later and the like. Almost always doing "git add -p" is a good way to jog your memory about these. They can be committed but probably shouldn't survive until a merge into master.
A todo file entry for items which are heftier and/or not likely to be returned to today. My todo files are date-based so I try not to add todos for items with a time horizon beyond about three days in the future. I also try to keep these files small enough to actually get through in a day.
A single wish list/ideas file. Keep it in approximately priority order and pull items from the top when you finish your daily todos. This is also often feature-based so a single line might generate a bunch of tasks. Biased towards a personal backlog instead of "just add to JIRA/Trello/whatever" to avoid eternal agile deprioritization hell.
Works fine for me, though of course I'm not out here writing Redis either.
personally I just write the function call, wait for my IDE to complain that the function doesnt exist, press command enter to have it generated with a throw new NotImplementedException().
works well for me, gives me the illusion of a working api, and I get autocompletion.
This requires you to give your functions well thought out names though
Yep, a notepad with a direct step by step 1 liners is very effective to get a ton done in one day.
Have it between you and the keyboard so it can't be ignored, presumably not ruining your typing experience.
Straight up notepad or sublime works too.
"Straight up notepad" means notepad.exe nowadays? That is really interesting
Haha, thank you for making me feel young for a minute :-)
I find it best to put a TODO file in the project that I'm working on. Sometimes I commit it, sometimes I hide it from git. I think it's a big advantage to have it all in one place, not very far from the code, and it won't get lost.
TODOs in comments end up getting forgotten about. Bug ticketing systems are too much work for the kind of notes I write to myself. I've tried using paper, but as I turn pages, notes on previous pages get lost forever. Staying in the text editor helps me not lose my flow.
> For my log I use Evernote because the log needs to have one characteristic: No save, No filenames, Nothing more than typing something.
I got emacs configured with a keybinding to open org files by date name (e.g. 2018-10-08.org), with the current date selected by default, creating them if they don't exist. I use `(org-read-date)` so I can, for example, get yesterday's notes with syntax like "-1d" or select it from a 3 month calendar. It frees me from having to think of a name for the notes I want to take and simply make a log of notes. It also autosaves.
(defun find-diary-file ()
(interactive)
(find-file (concat "~/org/diary/" (org-read-date) ".org")))
(global-set-key (kbd "C-c d") 'find-diary-file)For those unaware, the author of this article (Salvatore Sanfilippo) is a prolific programmer and leader in open source. He's been very productive in this role, authoring systems that many use today, such as Redis.
Obviously, Salvatore didn't invent a new creative workflow that he's branded "log driven programming". His intent with this article more likely was to share with others how he works productively. I can imagine that people asked him how he gets so much work done, considering how complex programming is, especially low level systems programming in C.
While the article was written in 2013, it shares timeless advice.
Had the same idea as I noticed that working in a nested way slowed me down a lot. Been trying to do this but I often lack the discipline to actually work this way and not dive into the local improvement I just thought of.
I use TODO and FIXME for this. This way, it's directly anchored to the relevant code. Both the IDE and the build server produce a list when asked.
Every project has a readme attached for bigger notes
This is essentially the process that Things 3 (I’m not affiliated in any way) for MacOS/iOS uses. It separates adding new tasks (for your inbox) from the step of organizing and scheduling them. So if you think of something you need to do, you can use a keyboard shortcut to pull up a prompt quick add it (from whatever app you’re using) and carry on with whatever else you’re working on.
I wrote a simple app for myself for this: it's basically a text editor with included checklist UI, optimized for a keyboard based workflow. I find it very useful to have a "conversation" with my notes while I'm developing to avoid going down too many rabbit-holes, and to keep focus on the primary task.
I usually just make the additional changes but I'm pretty good at keeping pretty massive mental models in place while I do so with out interrupting my work flow. If it's something truly massive i add a @TODO ... @PRI-N line in the comments.
How is something as frivolous as taking notes while programming, which many people do, worth a blog post and a mention here on HN? Let alone turning it into some dogmatic principle to work by, by calling it 'log driven' programming. What is next? Keyboard driven programming? 3-monitor-driven programming?
I do this:
While programming it pays off to stop and think about the code. I call this, thought driven programming.
This comment breaks a number of the site guidelines: the one that asks you to respond to the strongest plausible interpretation of a thing, the one that asks you not to post shallow dismissals of others' work, and the implicit one about not being a jerk on HN. If you'd please review https://news.ycombinator.com/newsguidelines.html and follow them in the future, we'd appreciate it.
Over the past 20something years of being a developer I've found there are a lot of things that seem obvious to me now that junior developers don't do by default. Looking back I wonder when I started doing all these obvious things. Presumably, if I didn't discover them myself, it was when I was told about them or I read about them. If no one is writing these "frivolous" things down then how will juniors learn?
If this helps a few new devs who don't have access to a good mentor (because, say, their mentor makes assumptions about what's obvious..) to tell them about this idea then it's automatically a useful post.
The best habit that any developer can have is reading code. It’s shocking how few devs spend time reading code; whether the code is from library dependencies or entirely unrelated third party projects, I always learn something new just by reading it.
Not every developer is blogging their best practices. If you want to see them and learn from them, the best place to look is in the code.
Honestly I think we should be reading at least 10x the amount of code we write. For every novel an author writes, how many novels do you think they read? Certainly more than ten. Why shouldn’t we do the same? In a world awash with open source code, there is really no excuse not to be reading as much of it as possible.
There are two interesting parts to reading code.
The former is usually not too bad. Occasionally there will be a comment to help with the latter (although rarely ime). The latter is basically impossible. I can ask 3 other devs, they'll all disagree. Some like the style, some hate it. Each will complain about different aspects, and none of them would write it quite the same for ~reasons~- Working out what it actually does, and why - Working out why somebody *wrote it in that way*Maybe it's just me vOv
Not sure if this is "obvious" or not, but a very simple thing I've been doing for the past 15 years is to write a short log entry every time I encounter an especially tricky bug.
I've found this to be a really good way to learn from bugs. There is something in the act of writing it down that makes the lessons of it stick better in my mind.
More details here: https://henrikwarne.com/2016/06/16/18-lessons-from-13-years-...
You’re oversimplifying. The key point isn’t the note-taking, it’s about putting newly discovered sub-tasks on a backlog rather than tackling them immediately. It’s good advice, and not something that comes naturally to everyone.
“Thought-driven programming” is a good idea, of course, but are you thinking in an effective and productive way? Are there approaches you can use to be more effective? I hope you agree that there might be some value in identifying and teaching such approaches.
He's also overreacting.
"over-reaction driven commenting"
Its not just taking notes. Its a common pattern while fixing bug, you encounter a code path that with a 3 minute time you can make it so much better, but should you continue on your original plan or take 3 minutes to make the code better is the question. The blog says don't spend time on diverging the original plan instead mark it for later. Another good name would be "breath first traversal development"
Sure, or, the name can be left out and the title could have been 'How I use notes while programming'. It doesn't need to be turned into a named concept, because it isn't a concept, it is just some advice that is so shallow that I question its relevance to HN.
It is the equivalent of articles and videos on bullet journaling that you find everywhere, portraying it as the silver bullet to becoming organised. "12 ways to organise your life". It might work for some, it might not for others.
In my experience, not a lot of developers manage to organize their thoughts well enough to write them down.
Maybe you manage to work with only people who can write down their thoughts coherently. But that's the rare exception and not the norm.
Consider what programming is, then consider the implications of the suggestion that few developers manage to organize their thoughts well enough to write them down.
That's exactly why there's so much bad code out there. I don't agree with GP that this applies to most developers, but certainly to a depressingly-high number.
"Maybe you manage to work with only people who can write down their thoughts coherently."
This is exactly what I mean with something becoming a dogma when you call it 'xyz driven' programming. You are painting a picture where taking notes while programming is a requirement of being a good programming. I am saying that it doesn't have to be.
Peter Buwalda is a Dutch writer who takes multiple showers a day. He says it helps him overcome his writers block. There is also some science behind this; it turns out your senses will be so busy with the steady flow of sound, the feeling of the water and the fact that there is no visual distraction that you can focus on your thoughts better. He isn't the first to realise this, hence the term 'shower-thoughts'.
So, one could argue, 'shower driven programming' is something we should all do. Installing showers in offices for this purpose or expecting people who work at home to take multiple showers a day in order to focus would be nonsense, but then I could also counter by saying "Well maybe you manage to work with people who can focus without taking constant showers, but that is not the norm."
Ridiculous right? So why isn't it ridiculous to demand that people take notes when programming?
Some of the replies on my comment have been along the lines of 'will someone please think of the children!' with reference to junior programmers. What junior programmers don't need is yet another dogma by which they should 'drive' their development.
It would have been completely different if the 'post' would have been 'Taking notes while programming can be helpful'. That strikes a completely different chord, namely, a helpful one. The word 'driven' does what you demonstrate: it makes people feel inferior when they don't do whatever the given driver is.
demand
Who, exactly, is demanding anything? You're way overblowing what is just a useful suggestion from someone who probably realized that would have been useful earlier in their career, and hence is trying to help others.
It's "log driven" because the logs, when read back, direct what you work on. Making that title to be some sort of mandate is inferring something that, frankly, simply isn't there.
It's surprising how much objectionable you find in what to me is simply an experienced programmer sharing a detail of his personal workflow. Naming doesn't turn something into a "dogmatic principle".
Before I begin again, I grant you that the length of my replies seem to imply that I care a lot. I don't, I just like to talk about semantics and when people raise new arguments, I like to explore the subject some more with a reply. For all I care you will write an article on 'boxer short driven programming' on how you sit around in your underwear programming and think everyone should do it.
"It's surprising how much objectionable you find in..."
It isn't much, it is just the one word that I find wrong. The comment you are responding to is an analysis of the effect of the chosen wording as demonstrated by @thachmai.
The semantics of the word 'driven' here ARE turning the given workflow into something that should be adhered to. If this weren't the case the word 'driven' would have no meaning and could thus be left out. Which would have been the better choice.
Furthermore, if you look at the article itself you can see that the whole tone is in line with my feeling about the title:
"The worst thing you can do is to interrupt what you are currently doing in order to fix the new problem. Instead just write freeMyObject() and don't care, but at the same time, open a different editor, and write:"
This doesn't come across as "Do you ever get distracted while programming by small pieces of irrelevant code? Try keeping a todo list to keep track of little cleanup so that you can come back to them later."
The author seems to think that you are a bad programmer if you don't take notes while programming and cleanup things you find along the way. @thachmai picked up on this and repeated this frame in his reply to my first comment. I got excited by this as it is exactly what I meant.
Words matter.
Workflows being a dime a dozen in both the software industry and all industries more generally, saying that any word makes a workflow “something that should be adhered to” seems like an overstatement. It's absolutely clear that the author thinks it is worth adhering to, however.
“-driven” doesn't imply demand, it implies the center of a given workflow. antirez is stating that this approach (which is more than just taking notes, since taking notes just means “writing stuff down” and says nothing about interruption) boosts productivity, and that it is centered around the process of stashing thoughts and observations without interrupting the present task. He also outlines why he thinks this is the case.
You are mischaracterizing the blog post by underspecifying what it says, then attacking the mischaracterization… I'm not sure that approach tracks with your claim to care about semantics. It's not just words that matter, it's also how they're collected into titles, sentences, paragraphs, documents, and how they exist in their broader context (software industry parlance, in this case).
You are, in turn, oversimplifying what I am saying. 'Language matters' might be closer to what I should have said and in that sense I oversimplified my own words.
This 'article' explains the trivial task of keeping track of todos and separating main and sub tasks. Something that I most people know how to do. 'Centralising' what we do around the written word is what separates us from animals and you find it everywhere.
I find coming up with a name with the word 'driven' in it for such a common thing somewhat grandiose. That is my critique.
Language matters indeed. I guess that if you'd said "I find coming up with a name with the word 'driven' in it for such a common thing somewhat grandiose." you'd not have provoked such a strong reaction, but you didn't. You said note-taking is "frivolous" (which to me suggested you think it's a bad idea!). You don't acknowledge the difference to "just taking notes", which might make it a not-so-common thing.
I for one find myself very curious about this shower-driven programming.
It sounds similar to Rich Hickey's famous concept of hammock-driven development.
Hammock-driven development requires less water, which is good.
"Common sense is not so common." - Voltaire
Yep.
One of the (painful) things I learned and continue to learn once I got into tech coaching is how much stuff we do without thinking about it. It gets into muscle memory and it's gone, as it should be. This is one of the reasons mobbing is so successful -- and why it doesn't make a lick of sense to anybody who codes for a living.
So no, there's no level of detail here that I think would be too much. (It may or may not belong on HN, and there's a question around how to organize this detail so that it's useful for folks, but those are different questions from whether or not it has value)
Yep. I like books like this https://www.amazon.com/Effective-Engineer-Engineering-Dispro... for this reason.
1-star commenters will say they learned nothing new, while 5-star commenters will be happy they found knowledge from experience enginners in (kinda compact) book form...
(I swear I'm not trying to plug anything)
It's weird. I did a blog series "Program F# like a stupid person". My point was 1) everybody makes mistakes, and 2) with the right attitude, you can plow through and make really cool complex solutions for folks. You just have to stick with it.
I don't think it was very popular. For the internet crowd, who only only have ten seconds to skim the material, "Program F# like a stupid person" became "Stupid person programming F#". It was just far, far too easy not to grok it. There are those 1-star folks.
A few did. Maybe 10%? Out of maybe 300 readers I got 6 or 7 people who absolutely loved it. 50 or 60 got it but didn't need it, and the vast majority didn't like it. They thought it was a Bad. Thing. They wanted tech porn. "New cloud-based logging app processes 14 Trillion records per nanosecond! Now with graphs!". That's their kind of story, whether or not they're processing one record per hour or not. It just feels cool. I get it.
I would love to find a way to cut through that noise and help people. I've thought about twitch, but that almost seems too unstructured. There's a way of organizing and presenting this material that I haven't mastered yet. It'll be fun to figure out.
I type while programming, I call it typing driven programming, it's part of the Voltaire family of programming methods.
Have you considered publishing a manifesto about this.. "thought driven programming"? :)
antirez shared a successful methodology for programming. He gave that methodology a name. It is helpful to name something when communicating the idea to others.
Sure, but names matter, don't they?
Naming is really hard work. English isn't even Salvatore's primary language. He's done better than I could in Italian.
I've personally found great benefit reading this and I am sure a lot of people will. As obvious as it may sound, it's always nice to be reminded every once in a while.
I think a lot of writers like the idea of coining a term that's in mass usage. It's like a personal brand marketing thing maybe. It's often injected into all of their writing in out of place ways and comes across as forceful and unnatural.
Examples include Cory Doctorow and Ted Nelson.
Don't give them ideas. Thought driven programming is now a thing.
Sliderule