Confessions of an Intermediate Programmer
michaelbromley.co.ukI've found the following really sets apart my programming now vs. earlier in my career:
1. I write and rewrite more now. I view the first time the code works as just the first draft (like I would with a paper). Earlier in my career I thought that making it work meant I was done.
2. I spend a lot more time thinking about what's happening in the code and why. I have found some of my best work to happen after days of thinking about small bits of functionality.
3. I get the domain (ie. the big picture) much better. This is insanely valuable and helps me step back.
4. I listen more to opposing views. That's not to say I don't argue my point, but afterwards I spend time to rehash what the other person said when I have no pressure to respond. This guides my thinking a lot as it give me an opportunity to change views/positions.
5. I'm increasingly tolerant of other technologies. I use x, but y must be good too b/c smart people use it.
5.a While I'm more tolerant, I'm less distracted. When I was younger, I spent more time switching technologies to learn "new things". Now I spend more time going deeper on my current stack to learn new things.
6. I now think its ok to write code in a "non optimal" way if it makes it easier for the entire team to work with. I don't believe in writing to the LCD, however I do believe programming just a little above the team's moving average capability is probably better than writing above everyone's head (whether by me or someone better than me).
I hate to sound like an old fogie here (I am almost 40) but one thing that occurs to me is that as we age our thinking changes and all the things you mention often develop over time.
When you are 20 you are pretty much at the height of your mathematical abilities. This is fine for certain kinds of programming, but it is not really ideal for others. It ensures that algorithms primary and the domains in a real sense become secondary. As you get older, these tend to flip. You understand the domains better and may not be quite as good at generating optimal algorithms, but the tradeoff is generally positive.
Additionally I think there is a tradeoff that happens between dreaming of building great things and developing a craftsmenship approach to the small things. Engineer the small pieces well, and the big pieces will take care of themselves.
In most areas, I think one is better with a very mixed-age and mixed-experience team. Older, more experienced programmers bring something very special to the table, but the same is true for younger programmers as well.
This being said, I have a few very minor disagreements with you ;-). These may be split hairs though.
First, while I rewrite more in some ways, I actually find on the whole I rewrite less because I am not afraid to procrastinate while trying to understand the domain. People wonder why my code seems to come out fully formed and it is because I am not afraid to spend my time thinking about the problem instead of writing a rough draft.
What has changed is the sort of rewrites I do. I will typically re-engineer a small piece to remove cruft based on lessons learned from the last two years, rather than rewriting a big piece to meet a new requirement (because things tend to be designed more extensably now). So I am less afraid to rewrite and more willing to do so, but at the same time, I tend to rewrite less.
Finally there is the question of optimal way to program. I am of the opinion (probably in part because of how much work I do with Perl) that there is no singular optimal way. Optimal has to be globally defined and one of the jobs of the engineer is to ensure that the framework is optimal so that the team can program optimally. If the team is not part of the definition of what is optimal, then I do agree it is a mess.
But I don't think we are dealing with a single dimension here. Understanding the problems the team is struggling with is a good way to ensure that framework can be built in a way that drives that average capability up.
> When you are 20 you are pretty much at the height of your mathematical abilities.
Is that really true?
There are many examples of people who came to mathematics late in life and did good things:
http://mathoverflow.net/questions/3591/mathematicians-who-we...
I think you only get better with time if you keep yourself sharp. What has changed for me is that I can't sit still and pound back Redbulls for six-hour coding sessions and then hit the gym. My process is also more methodical.
But mathematics is certainly one area I've only improved in with time.
And also see this: http://www.massey.ac.nz/~rmclachl/overthehill.html
However, I wonder if there is something else going on here. What I noticed in the list (and in the article I am linking to) is that the early peak seems less obvious from a historiography perspective than it does from a contemporary perspective. So I can't help but wonder if there is a qualitative difference in the sorts of contributions that younger and older mathematicians make.
Thinking more in terms of fluid vs crystalized intelligence here, but wondering if there is something else. I know my fluid intelligence is not where it was ten years ago, but part of that is a greater understanding of the problem domain. I can see more possibilities, and I see different ones than I would have before.
So I don't think it is as simple as we might want to think but I do think that thinking patterns (and abilities) change qualitatively as we age in ways which are favorable for important contributions when viewed through the light of history. Whether a thesis review board would agree is a different matter.
>Is that really true?
It's almost certainly not true, at least among those that would be called mathematicians. This follows if for no other reason than the fact that it takes years to get up to speed with the forefront of any particular branch of mathematics, which is the prerequisite for testing out one's mathematical abilities (in the context of research). While you occasionally have a few teenage prodigies, most mathematicians won't get started on their career until much later.
I can see how it would be true of normal programmers though -
Computer science courses usually teach something very different from every day programming. You learn linear algebra, AI, dynamic programming.
Then you go to work and spend 10 years mostly not doing those things. They are generic, and there are libraries for generic things - you just need to understand how they behave.
A decade later, most people will only retain the specific techniques they use in their domain. Maybe you know more than you ever did about indexing algorithms, but the math behind the deep learning algorithm you wrote at college is gone.
I feel like a big force that changes programming style is the gradual transition from guess-and-check to gradual bricolage, exploration, creation. I feel in my perspective like that explains at least a bit of the difference between "young" and "old" programmers here.
Guess and check programming tends to optimize around fast reset cycles so that many, many solutions can each be tried with little investment. This is the programmer who loves to refresh their window every second to see how their changes impacted the product. It's high energy and exciting. It also lends itself to prepackaged solutions—be they existing frameworks of specific transliterations of known algorithms. I also find that it tends to generate spaghetti code more quickly (all styles do, but GnC does so more quickly) and to fall flat on tasks which have either slow reset cycles, long dependency chains, or are difficult to visualize.
"Slow" programming, for the lack of a better word, orients itself around a deep understanding of both the domain and the natural "seams" in the problem space. It involves a lot of quick drafts that are perhaps never even observed—the mere act of trying and failing to express the problem elegantly is highly informative. It tends to find less "obviously optimal" solutions because it's in search of novel optimal solutions. It also tends to be more easily guided by high-level goals such as business value or elegance. Finally, programming slowly requires great reasoning skills and thus is more amenable to tools and processes which enhance reasoning. It can take stabs at deep, difficult domains because it happily uses great domain modeling or mathematical skills to break those challenging domains into clever, interlocking, and simple pieces. For all this though it suffers from having a design aesthetic and becomes less willing to "just dive in there and try all 500 solutions" in case one of them works.
I think this cuts broadly across several stylistic differences in coders. I think that age and experience pushes one toward the latter style. I also think that they both have strengths and weaknesses—so I strongly agree that mixed style teams are valuable.
> I'm increasingly tolerant of other technologies.
I would like to think I am, but recently I have noticed that everyone seems to want to jump on the NoSQL bandwagon. A lot of the jobs I see I would be a good fit want experience using MongoDB (or something similar).
I have been looking to learn them, but having read up on the technologies, I can't find a reasonable use case for any of them in my own work. One that would not be equally or better suited to Postgres or MySQL (which I know well).
I have come to the conclusion that 90% of people using NoSQL don't actually need it, and are throwing away a lot of useful features just so they can play with the latest fad.
NoSQL DBs definitely have their place. For instance my current project uses one to handle and do limited querying on data from other systems without needing to have a full schema for that data. You could do it in SQL by breaking out the columns we need and storing the rest in an XML blob, but it would be annoying. One rule of thumb might be that anytime you want to stick an XML blob into SQL, consider NoSQL instead.
That said, I think you're right that 90% of the time SQL is a better solution. You don't fully realize all the great stuff a real SQL engine does for you until you have to go without it.
Perhaps with age comes increasing tolerance and increasing ability in discerning hypes from useful technologies?
I'm a pretty 'young' person, and a much younger programmer, so I can't quite say I got the 'oo shiny lemme use that' thing out of my system, but I'm okay with that (for now it doesn't negatively impact my work, and I learn tons of stuff). But in other areas of life, I've noticed that I generally become less dichotomous in my approach to new and other things, but more discerning too.
NoSQL DBs have real advantages - e.g. you have a huge production relational DB, you want to make schema changes - you have to take your app offline for hours.
No. You can migrate big production relational DBs with much less than hours of downtime; sometimes there's no downtime at all. Please, either qualify your statement, or stop speaking in such an absolute sense.
Would you mind expanding on how to minimize downtime when applying migrations to large DBs? It's clear the grandparent isn't familiar enough with the relevant techniques to realize their statement needed qualification, or could count as "nonsense", so they could probably benefit from the explanation.
Sure, it can be accomplished in a variety of ways: swapping a migrated replica; breaking big schema changes into multiple incremental migrations that minimize the amount of time the database is locked and back-filling data; or spinning up an entirely new cluster, migrating everything, and then switching traffic over from the old cluster to the new cluster at the load balancer. Any number of these techniques can be mixed and matched to accomplish minimal and/or zero downtime depending on your constraints. [1] The most important thing is making sure your application code is able to operate with multiple schema versions.
I used to have a presentation from Facebook where they spoke about their zero-downtime migration process, but I can't find it, so this SO post [2] will do. Of course, there's a bunch more information out there to read up on.
[1] https://www.honeybadger.io/blog/2013/08/06/zero-downtime-mig...
[2] http://stackoverflow.com/questions/6740856/how-do-big-compan...
If you're taking down a SQL DB to make schema changes then you're doing it wrong. Even resource-intensive changes can be done online by applying the changes in stages.
This article did an excellent job verbalizing the incredible feeling of mental potency one gets when learning to program. When you see that first "Hello, World!", you are hooked. I still get this feeling whenever I learn a new platform. When I learned Android I was amazed that I had made a real-life app run on my phone. When I learned Ruby on Rails I proudly stuck my flag in the surface of the internet and declared myself ruler of my domain (pun intentded). When I learned to program an Arduino I felt my code-laden tendrils inching out into the real world: that light is blinking because I demanded it do so!
This is why i'd like to teach programming to kids. I really don't care if they continue with it or ever learn how to do anything useful, I just want to expose everyone to that feeling of power you can get when you're behind a keyboard and in front of the right tools. It makes you think you can do something amazing, and everyone should have a chance to feel that.
The desire to empower kids is basically one of the best motivations available to everyone. I'd recommend poking around for after school programs where you can volunteer to do it. I spent a month or two teaching HTML to a small group of middle schoolers and it was easily one of my favorite experiences in life.
It's such a great feeling. My favorite part is how, no matter how long you've been doing it for, you still get that feeling time and time again.
That bit is good. Fixing a bug that has haunted you is even better. Everything I've ever made is pathetically simple, so I can't imaging how good it must feel to fix something that is actually complicated.
For me the thing I had to learn was commenting more than anything else; working alone I never saw as much use for it. Then I worked with a guy that commented his code very well and suddenly I saw massive value in it.
I'm definitely still an intermediate programmer, but I think I was always fine admitting that to myself. It was getting over the fear of others reading and modifying my code that took me a while.
The big thing I learned (at a company where all changes are code reviewed) is to rewrite the code until it seems obviously correct to someone who doesn't know what I know. I rename variables, methods and extract code into small methods until someone can read the code out loud and it sounds like a comment. for example: if (!thereIsEnoughSpace(user.getDataRequestSize()) { if (canAllocateAdditionalStorage()) { reserveStorageForUser(user, user.getDataRequestSize()); ...
I've found many bugs after I renamed variables like 'i', 'j', 'k', etc to something more meaningful. That made it easier to realized I was using them incorrectly.
This is basically why I think I'm an ok developer...
I pretend to be the smartest developer in my group (let's call him Bob). I look at a line of code and ask myself, why will Bob ask me the change this line of code? What will he prevent me from checking in? How would he solve this problem?
I do this even if I have a really difficult problem and Bob is the "go to" person. I pretend to be Bob and ask myself, What question will Bob ask me when I tell him about the problem? He will ask me if I've searched google, ok I will search first. He will ask me if I've checked for an upgrade to a library, I will do that. He will ask if I've set a break point here and checked for a memory corruption there. etc. 80% of the time, I find the problem myself and the rest of the time, Bob doesn't know the answer either. But at the end of the day co-workers are fooled into thinking that I'm as good as he is. But I have no doubt that Bob is better because it is important to him to learn everything he can about some key technology. I only approximate Bob when I need to interact with others. If you ask Bob a question, you get a quick and correct response. If you ask me, I will tell you I'm in the middle of something else and I will get back to you in a few minutes but really I have to figure out the answer first hahaha. I try my best to avoid getting help from Bob because, honestly, I don't think Bob respects people who aren't as passionate as he is (most people confuse passion with intelligence). The downside, is that it takes me a day to solve a problem that Bob can solve in a few minutes. Managers hate that but it is just too boring to learn technology to the degree that Bob knows it in my free time.
Both good comments :)
Another couple of ways to improve your coding...
Start with comments outlining what you're going to do. If you are pair programming, this shows the other person what your overall plan is. It also acts as a mini todo list. They should be "What comments". That is, what does this piece of code do, not how it should be done. Comments are better than just coding because you can do that quickly as a sketch to show your partner. Then they don't just sit there waiting for you to type out a lot of the algorithm.
You can also start doing this with tests first, which help you outline the interface for your object/method/function/module. This also helps you think about how easy your piece of code is to use by users of your code. ALotOf. ProgrammersDoNotThinkHowEasyYourFunctionIsToCall(x,a,3). Is it obvious what it does, and what the required arguments are for the caller? Is error handling sane for the caller?
Coding considering Revision control is very important, and these days acts like great documentation. With tools like git blame, you can see comments for pieces of code inside the revision control system. If you limit commits to nice small chunks on one topic at a time this is very helpful for people. Don't include white space fixes with 5 days of changes to 30 files at once, with a comment like "update". If they are nicely explained by the revision control comments, and perhaps even reference your issue tracker, then other people reviewing, reading, or bug fixing your code later will have a much easier time.
Finally, know when you are in prototype mode, pitch/demo mode, quick script mode, or Serious engineering mode. Each call for different approaches, and have different requirements. Don't let the Serious engineer get you to do a 10 day development cycle when you have a time limit of 1 day to deliver. That's a fail. Also, it's a fail if you hack something up without following all the development guidelines for some embedded life critical piece of software.
Personally I only recommend the code with comments thing to novice programmers doing play projects.
Generally I think comments are bad and it's too easy to leave nonsense in the code with that technique.
The naming thing is huge. Give things good names. One can also factor out some specialized code into its own method / class - but because you want to re-use it necessarily but because you want to describe what it is doing. Divide et impera.
Having to explain to another what you're doing - I found that to be great as well. It removes bugs and produces cleaner code. You don't even need another - at check-in time, for every change, you can ask yourself "what is this doing" and "why". And if the answer isn't convincing, change what you're doing.
The value of peer-review was rarely on the peer's side - the other person doesn't really know what you're doing and is unlikely to catch a bug. But the peer would ask some question so you would start thinking about it and discover a problem.
The best thing about asking somebody for advice is often to come up with a good question. If you ask the right question, it contains the answer.
I totally agree. When I first started programming my comments largely echo'd the code, so I saw them as less valuable. Now I'm able to stand back from the code and write comments that reflect the "why", and let the code do the talking for the "what / how".
I see this a lot where the more experienced someone is the more they comment, but also where the comments are higher quality.
It's good you recognized that. In many cases, an intermediate programmer with expert-level group skills (including things like commenting) is worth much more than an advanced programmer with poor group skills. But often people neglect their group skills; they aren't "sexy".
I had the opposite experience; I found it easy to scan through comments explaining what the code does, and hard to read the code itself. So I would comment every single thing to explain it in English so that I didn't have to resort to code. Of course I could drop into code mode and rely on logical axioms and learned patterns, but it wasn't as intellectually satisfying as "getting it" with pure intuition.
Something similar still serves me well as a general mental framework for approaching programming and debugging at a very local (procedure) level:
1. Make sure that the English language description of the algorithm is logically correct.
2. Make sure the code actually corresponds to the English description.
You can take this a little further by separating your comments into two groups:
1. An english language description of what the function (or code block) does as a black box (forget the internals as they are subject to change). What you put in and what you get out, or section headings to structure how you read the code.
2. Comments in the code block to tell you why a given decision was made.
What this gives you is the framework you are describing but it further cements it into three layers:
1. Function and block documentation describe contract and what the code is expected to do from outside. If the code does not implement the contract the code is wrong.
2. When reading through the code as contract-implementation you can understand why certain decisions were made.
As bonus points, you get good API documentation.
Yeah well...
Great code needs no comments.
Bad code needs lots of comments.
I've seen examples of both recently.
I randomly met a guy who told me of his idea to implement a UX framework with windows, buttons, views, etc in python.
A week later he sends me the finished framework. And it's perfect. It's easy to understand, all the names of things make sense, it's orthogonal. Not a single comment anywhere, but perfectly clear. Great code basically.
And there's a project I am maintaining where some parts of the code are pretty much impossible to understand, full of duplicated code - somebody loved copy&paste - duplicated method and class names, and threads going off in all directions. There is a comment for every line of code but despite the fact that I consider myself pretty good at reading other people's code, I am not touching that thing with a 10 foot pole.
My goal is to write code that doesn't require a lot of comments.
Great code needs no comments.
Sure it does.
« The simple and obvious way to do this misses important edge cases X, Y and Z. »
« This is to work around a bug in the framework we're using; see http://... . »
« This is deliberately wrong, for compatibility with ABC system. »
> Great code needs no comments.
Great code implementing a particular algorithm for a particular reason might want a comment (1) identifying the algorithm and providing a reference to a published description, and (2) identifying the reason the algorithm was chosen.
Great code that is clear and readable and well-fit to purpose can still, very often, benefit from comments. Not everything that might want to be made clear to readers of code can be expressed in the code.
The most valuable comments that code can't obviate are comments that explain why the code was written this way, the intent, and a summary of how the code fits into the overall picture -- maybe at least a file-level comment.> Great code needs no comments.I think declaring intent is the most valuable comment. What was the author going for with this function? Why is there a nil-guard here when the database column is `NOT NULL`? Is there an actual edge-case or is our data bad or was the author just being arbitrarily defensive against a boogeyman nil?
When reading other people's code, I've found that most of my time is spent figuring out not what the code does, but rather the author's goals and their higher level game plan.
While I do agree to some extent that great code doesn't need any comments, it raises some questions. How can one be sure at the time of writing that the code is easy enough to understand for others? I'm sure that it's understandable for the author, who's been staring at it for a few hours, but will the same hold for a future maintainer?
I agree with other commentators that with comments one can at least express the intent and reasons for particular decisions.
And on a side note, it's wonderful to strive to write great code, but I believe it's easy to misjudge and think "Oh, I clearly write understandable code, thus I'll skip comments" even when it's not that clear. That certainly has happened to me, very likely to others at some point as well.
Sorry, I downvoted this. Great code doesn't need no comments. Lots of comments is definitely a code smell, but anything of reasonable cognitive complexity will almost certainly require some comments.
I can't be certain about your first example, but writing a UX framework is a pretty straightforward thing in terms of what it needs to do, so there probably wouldn't be much need to comment about what it's doing. Why, though - that's a whole 'nother question. Surely it's missing some of those whys.
It depends on the people working with you, sometimes I feel commenting should be left for edge cases only. More often than not it seemed that people read the comments and stop focusing when reading the code; they just check if it vaguely matches the comments. When foregoing comments entirely there was much better feedbacks and questions on important points, the understandig of the code base seemed much better.
The latter also forces you to keep a very clean code, and tricky parts (things you couldn't simplify nor make self explanatory) surface better.
From both my own experience and witnessing others, one major factor that sets apart from beginner and expert is working with legacy code. And by legacy I mean legacy from previous developer, previous team, whatever, not necessarily a decade old code base.
Beginners are keen to get rid of all the legacy code. It's tightly coupled, it's hard to read, hard to maintain, yada yada yada. And then they tend to make a way too optimistic estimate on how long it will take. While it might look like nothing fancy from outside, under the hood it often handles loads of edge cases etc. Been there, almost done that (didn't fit in the budget which was way too optimistic already). Eh.
Experienced programmers understand that it's not perfect, but they just live with that. Surely there are some cases when maintaining the codebase would be waste of money because the previous coder was a disaster, and sooner or later it will have to be scrapped away. But it's more tempting to do that than actually needed.
I had a nightmare of VB6 code to work on. Something like 3/4 of a million lines, and lots of deadlines to meet. No consideration given to quality - just have it working by friday. It was in the financial industry.
On more than one occasion you would write a subroutine to do something, then a month or two later, you would see someone had already written that same functionality.
Also I'd argue one shouldn't be considered an expert programmer until they've been forced to maintain lots of legacy code. It's an incredible lesson in what not to do, knowing when to comment and when not, proper naming, which things to optimize "prematurely" based on your experience-based predictions of how your application will scale, how to organize code in support of a growing codebase, etc.
Experienced programmers refuse to take the job.
We once tried to contract this really fancy team to take over some code. They asked if we had unit tests. "No". "Then we can't do it, sorry, bye."
And I agree with them.
Really fancy, hmm... They strike me as being not very useful if they aren't willing to touch the code unless it meets some quality standard. I can understand turning down a contract if the code is total mess and the client wants major work done in an unrealistic amount of time or for too little money. In this case, why not reply more like "We're going to have to do some refactoring before we do X. If your budget allows for that, then we'll take the job."
On the other side if all possible edge cases break it may be time to rewrite your code.
AFAIK this is actually normal -- you learn the basics from tutorials, then the things that you need to make more complicated projects easier seem unnecessary, then you attempt a large project without them, and then you see why they're necessary.
It's normal but can be really, really scary if you don't have a mentor or someone to help you out or point out some of the pitfalls on the way.
Even something as fundamental as version control is, I think, passed up by a lot of beginning/intermediate programmers because "it's complicated" and "I don't need it now because I'm the only one working on this." But a mentor can help push you into that earlier and walk you through some of the more common stuff in a more accessible, interactive way than an article or a forum, which goes a long way.
What's worked for you all for making the leap from lone programmer to working on a team with a mentor/people who are smarter than you?
I ask this as someone somewhere between "beginner" and "intermediate". I've used git on a bunch of projects, but normally get hung up with branches or with trying to undo changes. Even harder than version control, I think, is getting a lone beginner programmer to write tests. When I'm writing something, it's always tricky figuring out exactly what it is I should be testing. When time is a limiting factor, it's hard to get from "I know I should be doing this" to actually doing this.
When you are good enough to make your own projects work the way you want them to work, but not good enough to contribute to the open source projects you actually use, how do you break out of your bad habits that you know you will need to break when part of a team/large project?
I've been there. As far as testing goes, my vote is that, when you're at the beginner level for a framework/language/technology, put off testing until later. Get it to work normally first, and then write your automated tests later.
You already have a hard problem on your hands, figuring out how to get something to work normally in this new environment. Don't compound that right off the bat by trying to learn a new testing system too. In addition, reality is the ultimate test. By getting it working normally first, you know that it actually works, so if you write a test against it, and the test fails, then the test is probably wrong.
Once you get good with it, you should be able to reverse it and go to a more TDD-like workflow - write tests and code first, then manually test, and have confidence that if the tests work, then the app will work for the users too.
> When time is a limiting factor, it's hard to get from "I know I should be doing this" to actually doing this.
If you don't have time to write tests then you've overscoped the features for the available time. Once you start treating testing as essential and budgeting a realistic amount of time to write test code (probably on the order of 1:1 feature code time:test code time) then you'll be able to get it done. Your total number of features will go down but quality will go up.
I'm sure I'm not the only one who got down the road to overscoped features by saying "gee, wouldn't it be cool if my users could access this info that's living in a mysql table" and threw something together with a little php mysql_query(SELECT...), and it worked! except for some edge cases with non-ascii characters and the fact that you knew you were opening yourself up to SQL injection attacks, but you weren't really concerned because you only have 100 users and they're already authenticated by a more robust system. Anyway, you decide to do it the "right way" and rewrite it using SQLAlchemy. In the mean time, your coworkers wonder why you've burned half a day rewriting a feature that worked just fine after you had only been spending 30 minutes on it.
Sorry, that was a long way of pointing out that us novice programmers often don't have good methods of determining how long something should take because we've never done it before.
Sorry, that was a long way of pointing out that us novice programmers often don't have good methods of determining how long something should take because we've never done it before.
That is basically the core difficulty in software estimation: none of us have good methods for determining how long something should take because most of the time we haven't done it before. If it had been done before in a way we can reuse, we'd reuse it. The meat of most projects is always something that's in some way new, even when the new bit is integrating a bunch of pre-existing parts in a new way.
Another part of it is simply being good at automated testing. For me this was the biggest point of TDD—to force myself to learn to test anything and everything. At the end of the day all code must be tested, it's just a question of whether you are doing it manually or not. For one-off scripts and such manual testing is the sensible way to go, but for long-lived codes tests are an investment. Of course the value and necessity of tests varies quite a bit by language as well.
I agree with your point that it might be worth paring back features to improve code quality.
But there is no one true code to test ratio. If you're trying to become a better programmer be wary of dogma.
As posted here the other day, airbnb didn't have much testing until a year ago: http://nerds.airbnb.com/testing-at-airbnb/
I'm not saying either way is right or wrong, which is kind of my point.
Version control is one of those areas that is very difficult for a lone programmer to understand the need for (I was one). After all, I have backups, right? So why would I need it? Then I hit a snag, where something that worked before somehow didn't work now. And it was a lot harder tracking that down by going through backups, vs doing a git bisect and git revert. Eventually I ended up restoring every backup, and checking them into git.
Same here regarding version control.. until one day I after graduating from college I nostalgically searched for my old college files... and... oh the horror... my files where somewhere between "project1.final_backup_almost_final.java" and "project15.final_not_working.java.backup3" going through more strange things which I can't even think of why I named files like that, along the lines of "epsiloneridani43.final_working_working.tosubmit.java" and of course the project file I ended up sending to the teacher was something like "project16.working_again3.java" ...sigh
You see a huge amount of this sort of poor-man's version control in other industries - everyone I have worked with in other industries has worked out some personal scheme of versioning using initials, dates, numbering in order to sequence the various states of their work as it passes through multiple people and multiple versions.
A lot of room for disruption there I think if someone works out how to make version control fit better with ordinary people's view of the world (i.e. not git, svn etc). Versions in word are the closest I've seen to it but it really shouldn't be in one specific program but in the OS or a helper program.
File History in Windows 8 is nice, sort of. It takes snapshots of changed files every hour. Sadly you have to enable it first and it's sort-of treated as a backup (so you have to store the copies on a different volume, etc.). So, not exactly what people have in their home unless they know about such things.
I think the missing solution here though is collaboration and versioning between people, rather than individual backups.
Things like File History and Time Machine are nice, but don't really address this need.
To be fair, though, once you get the hang of it, you have to take care not to apply it to everything. Back when I grokked Mercurial (first VCS for me), I used it for packaging scripts, dotfiles, /etc files, todo lists, shopping lists for the grocery store, financial accounting, and my own poor man's dropbox clone (please don't ask).
"When you have a ham^H^H^HVCS, everything looks like a programming project".
O to be young and innocent of version control.
I've actually been down this route when I was a kid experimenting with making programs. I did not understand what was the point of functions - why not just copy/paste blocks of logic?
It wasn't something important to me for a long time since I did other stuff instead, but coming back to programming in an effort to make a career out of it, a lot of that stuff became instantly obvious & I felt foolish for not recognizing it in the first place.
> It wasn't something important to me for a long time since I did other stuff instead, but coming back to programming in an effort to make a career out of it, a lot of that stuff became instantly obvious & I felt foolish for not recognizing it in the first place.
I think one of the key things experienced programmers finally get is that we never stop understanding that there are things which we are doing sub-optimally and can continue to be reflective and self-critical of that. How can I structure my code better? How can I solve this software development problem that seems to come up frequently?
And then as we solve them, the solutions seem so obvious we wonder why we didn't realize it sooner.
It's funny, as a kid I can remember thinking: "Right, everything has to go in the main function unless absolutely necessary, it will just be easier to read that way."
Nowadays, it's the exact opposite!
As a kid I wrote a few games using QBasic. I apparently didn't have a firm grasp on control flow because every call to a sub routine ended with a call to another subroutine; sometimes to a sub routine that was already called earlier in the stack. You win the game by beating it before crashing with a stack overflow.
Nah you just invented continuation passing style. You just needed tail recursion too, and you would be good to go ;)
My code changed in a similar way, however I recently devolved.
I've mostly worked with higher level languages, and recently delved back into C. I had forgotten what a complete pain in the ass it is to pass complex data structures around between functions.
I'm a bit ashamed to admit my functions got pretty damn beefy real quick.
Don't worry, you can do this in C# or Python as well. Just structure your small, gradually growing proof-of-concept and soon-to-be-finished-product program correctly. No classes! Or at the very least a few monolithic ones, which cannot be instantiated. Use static methods whenever possible. I have never ever ever done this.
(I am getting ready to write an essay titled "Confessions of a Shitty Programmer").
Try C++, join the dark side! In addition to chocolate chip cookies, we promise C++11 move semantics for returned objects/values. The cookies/cake might be a lie. :-)
I had to create a small ADO class for advanced C++ last year.
It was... terrifying.
&structure ... ?
> &structure
AFAIK, that's only a legal function signature in C++.
To the point of the question : when I wrote that, I was specifically thinking of 2-dimensional arrays, not structs per se.
I meant passing a pointer rather than a by-ref call (which is indeed C++ only).
i.e. void myfunc(thing* stuff) { return; }
int main() { thing stuff; myfunc(&stuff); }
Oh, totally agreed. What I had difficulty with, specifically, was passing around a struct that contained a multidimensional array of a size determined at runtime.
I'm sure the problems I had could be solved by either a full re-design (not really an option), or accomplished by someone with more C experience.
It shouldn't matter what the struct is holding, you are passing a pointer, it's always the same size.
This sounds like the mammoth version of "sleeping on it", which has worked remarkably well for me.
There are some absolute, rather than relative measures that you can use to gauge your skill. For engineering fields (not just software) the 'toolmaker' analogy is a good one i find. Until you understand what involved in making the specialist tools you need for your field, and can actually make those tools yourself then you are far from mastery.
This is not entirely straightforward though - a bit of a rabbit hole you could say. e.g. should a programmer stop at the compiler? The CPU? The circuitry? Electronic components? The laws of nature? Tools are built with other tools after all and some tools are provided by nature...
On the other hand, if you haven't made even the first step towards understanding your tools then thats not really a problem to worry about.
I found this so striking in part because of recent events: Mark Karpeles's mother defending him as a genius, Karpeles not delegating coding to experienced developers and focusing on being a CEO, the leaked PHP MtGox code...
The bit of leaked PHP code I saw seemed to be reasonably structured?
Now, I wouldn't necessarily use PHP for this sort of work, but...
I hope you're not referring to http://pastebin.com/W8B3CGiN
Those 100 line functions wouldn't make it through any half decent code review. Zero modularization, dal is inextricably linked to almost every piece of logic in the thing. Its an object called "Bitcoin" and I have a feeling it is 95% of mt gox' business logic if not all of it, it even writes raw xml to a temp pointer before dumping it into a cache and then making calls to header(). Separation of concerns? What are concerns?
Sure its tabbed neatly, and its got camelCasing, its using some form of orm/query thingy going on in there, but its still one hell of an unmaintainable, untestable mess.
I'd rather a thousand lines of spaghetti than ever having to work on something like that.
I once was hired to a gig because of the following words that came out of my face hole: "Some people see a giant hairball of code, and think to themselves 'I don't want to deal with that'. I see the same code and think 'I'm going to get a new boat'."
Bad, evil, horrible code written by misguided children makes the world go around.
Pretty much my job description.
I hope you get paid well.
> Those 100 line functions wouldn't make it through any half decent code review.
True but I have worked on worse. The codebase we inherited from SQL-Ledger is quite a bit worse and was even more poorly commented when we forked (the only comments, aside from two lines describing the module, and copyright notices, were magic comments which if you delete things, stuff breaks).
And some of the modules were 4-5k lines with 1000-line functions. In a codebase of over 100k lines :-P. As we continue to pull out this logic, our codebase increases in functionality and decreases rather significantly in size.
Oh, and it was Perl which relied on sloppy global scoping and so you couldn't test it.
But hey, we are replacing the code at a pretty good clip.
What is really frustrating with the SQL-Ledger codebase (at least as of 2.6 and 2.8, I haven't looked at 3.0) though and is probably the case in the code you are linking to is that the developer had over 10 years of progress in this and still never seemed to manage to improve:
Really? That's like pass by reference to dereferenced reference, right?IS->post_invoice(\%myconfig, \%$form);It's one thing to write crappy software. It's quite another to write it for over a decade and never improve.
I hate to be all, my pile of spaghetti is bigger than your pile of spaghetti, yet I just can't help myself
- 2 decades in development
- just under 3k "scripts", mixture of php html "fusion", php classes, and last decade or so templates have been added
- last wc -l check, was well into 1m lines in house code
- database (we have a few), >300 tables, it used to be around 4 but we culled some tables / systems
This is our oldest system, still in use, still extremely business critical, we're doing a similar thing to you moving things out, cleaning house so to speak. But its going to be ongoing for a long time to come thats for sure.
Some of the people in this thread should get together and start a new dailywtf clone. Pretty sure we could preload a years worth of content with nothing but grep and git-log.
> - just under 3k "scripts", mixture of php html "fusion", php classes, and last decade or so templates have been added
Unfortunately the fact is once you get to a certain point you pretty much have to accept that you are going to have some inconsistency and rewriting is going to happen gradually. With the LedgerSMB codebase, we have a mixture of code before the fork, which is a mixture of Perl spitting out HTML as strings in pieces and Perl assembling db queries in little pieces, so your comment made me somewhat uncomfortable because it reminded me of what I am going through in the rewrite process.
> - last wc -l check, was well into 1m lines in house code
wc -l was in my case closer to 200k, but once you exclude blanks readmes, documentation and comments we added, it got down to about 100-130k. Not really sure.
However, the code was messed up in ways that are only possible by taking powerful features of a language and systematically abusing them. In the db, needless polymorphic associations which served no purpose.
Almost every module had a name of exactly 2 letters, and the scope of some of the modules was just breathtaking. PE.pm for example handled logical groupings of parts for point of sale, customer discount groups, and project accounting.
Now if there were modules that did too much, there were also modules which were divided for no reason at all. You had, for example, ar/ap workflow and user interface split between aa.pl, arap.pl, and arapprn.pl (the latter being printing support functions).
This is code that every time I work with it, I find unpleasant surprises lurking beneath the surface. It is code that when we forked failed basic tests for things like rounding numbers (financial app). We fixed that right away.
If we are lucky we will be done with the rewrite in another five years.
> Some of the people in this thread should get together and start a new dailywtf clone. Pretty sure we could preload a years worth of content with nothing but grep and git-log.
Definitely. I am up for it. Should I start a blog and post the url here? I would be happy to add you.
I personally like the hard coded values and the extremely long line lengths.
That was my first thought too.
That and "this looks kind of structured from the top but the more I look at it, the less structured it becomes."
I find his experience is very fulfilling. In the beginning he had fun times doing stuff which actually delivered some results. Probably gradually became aware of wrong chooses he made and discovered architect side of coding... I kinda feel this is a way to learn it. I think you need this excitement of fast moving to keep engaged - it is hacker part of coding. When you are young it is okey to try something, may be break something and move on.
... Or may be I think like that only because I can see myself almost in every single line author posted :)
I agree with the other commenters here. This is part of your development. Yes, it could've possibly be avoided if you had a CS background or worked alongside other developers. But it's normal.
When I was still in high school I "invented" inheritance in PHP. I put one function per file, and used the include search path. When I discovered OO later I realized what I had done.
It happens :)
What are the best ways to get better?
I think the following is a reasonable path:
1. Read more books. Turn off Twitter, FB, whatever, and open your Kindle app. But be picky. Don't read crap.
2. Read every day. 30 minutes minimum at the same time every day.
3. Write code that implements what you learn.
4. Write code that is more consistent. Learn to write in patterns. Use patterns that you both learn from a.) books, b.) reading other people's code, and c.) that you come up with on your own.
5. Program by using coding standards. Either your own, or if you're in a company then follow the team's standards.
6. Review what you've written. Identify alternatives to the patterns you've used. How else can feature x be implemented? What do you like about each approach? What do the experts say about each approach. Then read everything you can about various approaches to a given problem and implement as many of them as you have time for.
7. Read some code written by someone way beyond your skill level in a language you know. Take in how well they write. Pick one thing they did well, read about it, understand it, then integrate it into your coding.
8. Rewrite something you wrote at least a year ago. Can you easily find a better way to implement this code? If yes, then rewrite. If not, then read more.
9. Learn one new tool that helps improve your workflow and get to a point of daily use.
10. Speed up. Use automation to eliminate the basic coding that you already know well and that is now wasting your time. Use whatever is easy for you whether its templates, macros, boilerplate, generators or automation. Just pick one and automate away the stuff you've already learned.
Repeat.
In my experience it's impossible to know what "pattern" to use where in real life code into you've made and realized the mistake that the pattern solves.
sitting down and learning a bunch of patterns and then using them without a deep understanding of why they are useful is misguided, and you'll end up with dogmatic code. not that it won't help, it's just not the most important step. the most important step is to write at least medium sized code bases without putting your ego into them so that you can be savvy to your mistakes.
My recommendation is to tackle ambitious projects as an open source programmer. Do things well beyond your comfort zone and expect to fail.
The fact is that you won't get better unless you actually fail and are introspective enough both to realize you did (there are programmers out there who never realize this and I have tried, unsuccessfully, to work with them). So pick something you aren't sure you can succeed at.
The second thing is to be evaluating your development challenges as you go, be introspective, and learn from them. Every development challenge is an opportunity to get better.
The third thing to do is to start reading about cracking software and start working on cracking your own software. If you can get others to crack your software, better yet. You will learn a lot about best practices from this.
The fourth thing to do is to try to work with teams of open source developers and learn from others. I have never learned anything from someone who always agreed with me, and a friend of mine is fond of saying, if a village has two wise men who always agree on everything, the village has one more wise man than it needs.
Read more code. Edit code others have written. Maintain a wide variety of projects. Try to start your own.
Failure can be tough and sneaky: I didn't want to fail, so I Googled to see how others did it until I was sure... Then found I couldn't reach a consensus, and ended up "researching" for months. Sometimes doing it wrong isn't failing, after all, you've just found a way that didn't work-- that's progress.
Oh and reading or knowing how to do it isn't doing it. There can be quite a bit of work to doing it yourself, don't skip the exercises at the end of the metaphorical chapter ;-)
I have been working on my own code base for the last 3 years. It is constantly evolving, and requirements constantly change. I often have to revisit a piece of code I worked on 6 months ago. That is the best time to tell. If your own code doesn't make sense to you fairly quickly, it is worth refactoring. Look it and ask yourself why it is difficult to understand and what you could do to make it better.
Good question...
I've worked on enterprise software and people in the company thought of me as one of the better programmers there, but when I do open-source stuff I feel like an imbecile.
I quit my job and wanted to learn and get better, but I don't know how to start.
Defense, defense, defense. Your code should be peppered with checks that should never go wrong. You should feel stupid writing them.
They will save you a lot of headache one day.
Really the only way to get better is to write stuff and ship it to someone who uses it, and you have to experience the pain of supporting it. That's when all the important lessons are learned.
write more code. regret it.
The scariest place to be as a programmer is when you look back at what you did 6 months ago and still think to yourself that its pretty good. A little regret is a good, good thing.
Five years ago, I would add a new feature and 50% of the time it would be a fair bit harder than I expected, then maybe 30% of the time I would find it easier than I expected, as I had designed the code well, and it was set up for extending / reuse.
Nowadays, I am getting better. If I don't understand a bit of code I wrote a few months back, then I use it as an opportunity to look at it freshly. Why is this code crap? Why is it making what I want to do difficult? How could it be better? That's a good place to start refactoring from.
It still amazes me that I'm still getting better as a programmer every day, over ten years later.
Unfortunately, I repeated the same year of experience in the middle a few times...
In addition to this, reading and being able to understand other people's code will provide great returns. You learn about styling, coding standards, architecture, patterns, etc that you might have never stumbled upon if you attempted to code everything yourself.
That must depend o having decent code to work on in the first place. Seeing some of the comments in the code bases I have worked on, I am not sure I was going to learn anything there.
To rephrase: write more code. months later, look back at your early code, see if it still makes any sense to you today ;-) rinse, repeat in different languages, situations or version numbers.
Programming is 80% debugging, 20% writing bugs, and 1% off-by-one errors.
It amazes me that despite all the efforts made to improve programming languages, the debugging tools we're using are so clunky. I really wish more effort was made to improve them.
Nice article but i was wondering why didn't you go to university/IT School to learn computer science?
--
"To me, the object-oriented approach was just a bunch of unnecessary overhead and boilerplate"
If you had made real C++, you would'nt have think of object oriented programming as an overhead. I suggest you to buy a few O'Reilly books (Probably the best code book )
Not the author, but personally, I chose Maths as I felt it gave me a broader basis to approach my career. I am now happily employed as a Software Engineer. I do find I've missed out on some things (TDD indoctrination, for example), but I've gained in others.
There are only very few universities that will actually give you a proper indoctrination in most modern software engineering practices. Usually you get a bit of horrible object orientation, a few stern words about coding standards, and the lovely advice to "always test your code". If you take a Compilers course chances are the words "static analysis" will be used a lot.
I'm sure there are exceptions to this, but they are hard to find and universities aren't entirely to blame for it. Some parts of software engineering move fast and there is a justified fear to teach students things that will be useless by the time they graduate. So the focus is more on laying a solid ground-work.
you can access the article from this link in case if isn't opening
http://webcache.googleusercontent.com/search?q=cache%3Awww.m...
Although I generally do not post something that is not constructive to an article, I have to say, I bought the same magazine when I was younger with Borland on and that is what made me want to become a programmer!
I was amazed when I read that point. The only difference is that I couldn't get anything to build in Borland. I started out in BASIC making my motherboard beep to tunes in a BAT file.
Text-only cache: http://webcache.googleusercontent.com/search?q=cache:k4GBqYS...
This more or less sums up what I think about "duct tape programmers" (as Joel Spolsky calls it).
Code Complete is really great. I also recommend Clean Code. I started reading it before going to sleep, and it got me so excited I couldn't sleep. It was 4 or 5am when I finished it. All I could think about was refactoring all the code I had ever written.
You have to stop thinking about what's good about your work and focus instead on what sucks about it. Sounds depressing maybe, but that's how you get better. You can feel good about yourself, or you can get better.
The best part of that post was right under the title..
"File under ego"
A smashing read. Entertaining and honest, thank you.
I liked the "Fuck Generator". I've often said that many a programming career began with this:
10 PRINT "COCKS"
20 GOTO 10