Raising money for StackOverflow
joelonsoftware.comI've got mixed opinions of StackOverflow.
On the one hand, it's definitely better than ExpertsExchange, and does an adequate job of providing answers to many developer questions.
On the other hand, it's far (far!) from living up to it's potential. The basic premise of the site is that the wisdom of the crowd will separate the gold from the dross; however, the incentives lead to a plethora of mediocre answers which do not converge (Wikipedia-style) towards some platonic ideal, despite Joel's stated wishes to that effect. What's more, there's a well-documented problem ("the fastest gun in the West") whereby the first semi-correct answer will quickly rise to the top, drowning out later (but more correct, more thorough) answers.
In short: they haven't (yet) solved the basic problem of "changing the way people get answers".
What's more, gaining the critical mass of early users was leveraged largely through Joel and Jeff's celebrity. Which is not transferrable to other domains. Which means that the site can't easily scale out.
Based on the preceding, I'd say that VC money is a good way to go-- it should give them the resources to a) solve the basic problem, and b) build subscriber bases in other domains.
It is far from perfect, yes, but it's also much better than anything that's come along before.
More and more when I search for solutions to problems I'm having I've been coming up with SO links. And when I do and the SO question is related to my issue I usually get the solution, too.
The "fastest gun in the west" bug has been fixed for awhile; answers at equivalent ratings are ordered randomly. In my experience, that's eliminated the problem 90% of the time and ameliorated it the rest.
I agree, this does help fix the "fastest gun in the west" issue, but I don't think it eliminates the problem.
A mediocre answer that was posted one minute after the question was submitted often has more votes than a well thought answer posted several hours later.
A better solution might be to randomly order answers, regardless of votes, for the first couple of hours (maybe even day). This will further help remove any ordering bias.
I also feel that authors of answers should be hidden for the first couple of hours (or day). Just because John Resig answered a question, doesn't necessarily mean it's the best response.
reddit also tried to solve this same problem with their "best" ordering: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s...
If I understand it correctly, they introduced this algorithm to solve the exact same problem. Hence, it should work beautifully for solving the "Fastest gun in the West" problem also.
StackOverflow should seriously consider using this algorithm, at least for a day since the question was asked, as one previous commenter said.
Why not hide all submitted answers until 24 hours after the question is posted? Once this interval expires, there would presumably be enough of them to be shuffled and presented without any sort of fastest-gun bias.
Optionally, question submitters could pay for "Urgent" status, where the submitted answers are posted as they come in. No ratings or votes would be accepted for the same cooling-off period, however.
I don't think you should hide the answers, but you should definitely hide the ranking (votes) for about a day (or half a day) or so. During this stage the answers would be randomly sorted. After a day you get to see the votes for each answer...
i posted a question because im stuck and i want my answer nownownow
The "fastest gun in the west" problem is really only an issue if you want an answer within one day of asking the question. Long term I haven't noticed it being a big deal. Over time, if a question is relevant, good lower ranked answers will rise to the top even if faster, lower quality answers were voted up initially.
The worst case scenario is a question which is perfectly written such that it garners attention from experienced folks who know what they're talking about for only a tiny amount of time, and ever after only garners attention (via google searches, for example) from clueless noobs who can't judge the quality of the answers (and don't have the rep. on the site to vote answers up or down anyway). However, this is a pretty contrived scenario which doesn't play out often in real life. Important questions will garner attention by people with a range of experience and qualifications, which will have the side effect of improving the quality of answers over time through future voting etc.
My problem is the mandate of the cc-wiki by attribution license, when cc licenses are not intended for code. A pessimistic reading would appear to restrict even reimplementaitons of answers.
Of course, the main problem is that individuals should be able to license their contributions as they choose.
You can relicense your own contributions as you choose, as long as it's at least as permissive as cc-by-sa or you dual license your code under cc-by-sa. Just mention your license in your user info. I license all my code samples under the WTFPLv2.
Also, it doesn't matter for most code sample on there. Most are not anything you would want to copy (too specific, or too general, to apply directly to your problem), or short enough that copying them would be considered fair use.
What's more, there's a well-documented problem ("the fastest gun in the West") whereby the first semi-correct answer will quickly rise to the top, drowning out later (but more correct, more thorough) answers.
I haven't noticed this. I know I have personally answered questions months after an answer was accepted, and have gotten plenty of points. In some instances, the "accepted answer" was even changed to mine.
Stack Overflow's problem is that 80% of the answers are from talkative people with no clue, and 100% of the questions are from people who have never heard of Google.
I think your analysis is correct, but all of of your criticisms are most likely fixable. They may need lots of tweaking, but overall the concept is solid!
I agree the problems are fixable.
The more germane question is, are the problems best fixed via an infusion of VC cash?
In this case, I think the answer is "yes".
The fastest-gun-in-the-west problem can be a feature depending on your perspective.
If I come to Stack Overflow with a question, many times a quick, off the cuff but not perfect answer will solve my problem. Sure, it might not be the best possible answer, but if it solves my problem, why would I want to look for something else? The primary person getting value out of an answer should be the person who asked the question, or else no one will bother coming to ask their questions. If you put anything in to slow down the submission of answers, you reduce the value of the site to people asking questions.
And there's an easy solution if the top rated and accepted answer is actually incorrect or misleadingly incomplete; just edit it. Sure, it takes a bit more rep to be able to edit people's answers, but it doesn't take all that long to get that rep.
Why do Wikipedia articles converge to such high-quality content but that doesn't happen with other collaboratively-edited sites like SO? Maybe it's because there's such a well-defined culture to guide high-quality editing on Wikipedia (i.e. projects, bots, etc.) that doesn't exist on other sites?
What resources are you implying will help solve the basic problem?
Yes, there is plenty of room for improvement. At least make it possible to browse the site by more than one tag and fix the search engine.
depends on your perspective. as a googler, i skim all answers and get the info i need. as an asker, i want the fastest close-enough answer possible so i can figure it out and move on.
incenting 'cleanup' (the original intent of community wiki) would be great, but i doubt it would increase their traffic/revenue very much.
One problem that seems to have come up a few times is their inability to make serious revenue. This is a perennial issue for sites aimed at developers that don't sell products - coders + advertising don't mix (well).
I get the impression Joel's looking at expanding the SO model into other non-developer/non-geek niches where advertising could work, but I'm not convinced he's proven the revenue model enough to make it a tempting investment just yet. Anyone know any different?
Note: On another post, someone reminded me of the Careers move, so that's one easier-to-monetize stream at least.
If they allowed me to embed their system into my site to provide "smarter" message boards, I would gladly pay them a set-fee/month.
They don't need to care how I monetize my sites.
[Sadly now they do not allow this in any way]
You can't embed, that's true. You can pay a set fee and run your own however: http://stackexchange.com/
Not as part of another site. ie site that has more than just stack-exchange. (And they don't allow iframes)
Now we know what you've been up to.
"their inability to make serious revenue"
To somebody like me who doesn't really understand advertising, it seems like the value of an impression ought to be really high for StackOverflow. Based on the tags associated with my userId and the questions I've read, they pretty much know which technologies I use everyday. How much would it be worth for EnterpriseDB (the PostgreSQL folks)to advertise to someone who is wondering how to covert his Oracle database to MySQL? Perhaps, even with 6M users, there are so few people in any coveted niche that the friction overshadows the value?
I'm not sure I've seen anybody report hard numbers on this, but anecdotally what I've heard is this: geeks don't click on ads. Any site that targets geeks has a harder time making revenue from advertising than one targeting non-geeks, by a significant margin.
You're right, but since you follow me on Twitter, you must have missed my numbers on this.. ;-)
On Ruby Inside, Rails Inside, and Ruby Flow, I run a cpl million ad impressions each month and make an OK amount of money from it despite the CPM being deathly low (say $2-$3 on a per ad basis). The CTR, however? It's south of 0.5%.
I also ran an Adsense for Feeds experiment for a few days recently. Don't have the numbers to hand but it was 1 click in approximately 16,000 impressions.. and these are real, unblocked impressions. I made 40 cents. Developers don't click on ads. "Geeks" do, though, IMHO.. otherwise sites like Slashdot wouldn't bother.
Compare this to Adsense ads on my old, archived personal blog.. and they run at over 20%.
I click ads. They interest me often, but I cannot honestly think of the last time (if ever) that I bought something from one.
The less web savvy the audience the higher the click through, best CTRs I ever experienced were on myspace related sites, my target audience was 15 year old kids judging from profile links I saw.
Yeah, I had seen your numbers, but I was referring to some sort of larger study on the matter.
As for Slashdot, I always sort of wondered whether or not the appeal to those advertisers was less clickthrough and more name recognition. That is something that I think is a bit under-rated. I think that matters when evaluating products - I know that I tend to favor brands that I've heard of in some way vs something completely unfamiliar.
As for Slashdot, I always sort of wondered whether or not the appeal to those advertisers was less clickthrough and more name recognition.
That has been cited by some advertisers, for sure - especially those who are recently funded or have larger branding budgets.
Take New Relic, for example. They're sponsoring The Ruby Show for I'd guess (and I don't know for sure, but basing on rates of similar podcasts) at least $300-$500 per show? There's no way they're getting that much business from people listening to the show checking them out at that very moment. In terms of brand awareness, though, $5-8k a year is nothing if almost every Ruby developer has at least heard of you..
> ...geeks don't click on ads.
I think it's rather "geeks don't see ads".
It's worth noting that the sorts of ads most non-enterprise programming sites run aren't the sort of ads that the adblockers strip out by default (major networks, Flash, etc). You could manually add them, but I don't know if people go to the bother in large numbers if they're not horrid, intrusive ads.
I use EasyList, and all these ads are blocked.
Exactly. Many (most?) geeks use an adblocker.
And don't they like to mention so all the time. It's the computer equivalent of "I don't own a TV".
Probably more like "I have have HD reception" - opting out of ads improves your experience a great deal.
I do use an adblocker, but even before I did, I think I'd just used the web so much that it was almost like my brain had gotten used to the general pattern/shape of ads on web pages to the point where I just unconscionably ignored them.
That's called "banner blindness". It's important to keep in mind that banner blindness applies to things that look like ads, not just ads. http://www.useit.com/alertbox/banner-blindness.html
Good point. But I'd also be interested to know the experience level of users involved in Nielsen's study. My own anecdotal experience suggests that people less accustomed to the web are far slower at discerning what is an ad and what is content.
On a related topic from that link, I don't think I've ever come across a banner ad made to look like a dialog box that has been good enough to fool me. But again my experience seems to suggest that those with less computer experience are far more easily fooled.
How many programmers don't have some form of adblock in their browsers?
Judging by many of the questions on SO, I'm surprised the people asking them can even manage to use a web browser with the default settings.
If someone ran a malware-ad campaign on SO, I'm sure a significant number of people would be infected.
Considering how often I rebuild my dev machines, I couldn't imagine tracking down all these obscure plugins every time I built a box. Firefox gets installed, along with Firebug and the AWS plugins, then sits idle in the taskbar until I need to use it for testing something.
I'm actually surprised by how many programmers DO have adblock.
Some of us automate our installs.
Check out the FEBE and OPIE plugins for firefox. Host the archives someplace (dropbox works well) and write a little install script and you're done.
Installing stuff by hand is so last century...
I'm actually surprised by how many programmers still install stuff. :)
Advertising on SO. Sure, you can get highly targeted advertising that could help solve peoples problems. Converting Oracle to MySQL could have Oracle promoting tools to do just that on that very page. However, all those impressions are essentially people coming to the site, looking for an answer, and leaving. If the answers aren't there, they go back to Google and look for another answer.
Advertising on something like SO is almost like advertising your latest application on the Pirate Bay. I just don't see the value.
I'd be quite keen to advertise my to-be-launched-soon coder community site (that's not competing with SO) on SO, but they have no self serve system and I'm not paying the $20+ CPM they'd probably quote if I contacted them direct.. :-)
Do that and let me know what kind of return you make from advertising on SO. Until then, everyone is "keen" until they have to open up their wallet.
Appreciate the cynicism, but at the right price I (and doubtless many others) would do it for sure. An advertiser's idea of the "right price", though, is bound to differ from theirs ;-) Given that developers have a low CTR.. the CPM would have to be "very low."
I enjoy using StackOverflow.
Jeff Atwood seems to get a lot of flack, but in creating such a nice site and community, he's walked the walk as far as I'm concerned.
One thing that wasn't clear from the article, what does he want to spend the money on?
If it's marketing, I don't see why they can't do that with existing revenue. It's not as expensive as say a retail location for a starbucks and they must be doing some revenue with ads and job board already.
If it's hiring more developers that might make sense, although I can't think of any huge features they don't have yet.
Overall, not sure I'm convinced this is a good play for VC. They are growing nicely without it.
Obviously, they would need to target a larger audience if they are going after VC. I can see them rolling out the site to 500 different niches. Image answer sites for home improvement, gardening, small engine repair, microsoft office, botany, project management, cooking...
Reading the article, it would seem that the VC money will go towards launching StackOverflow clones in as many different verticals as possible. Thus the comparison with Starbucks, which had proven success in several markets before going big.
StackOverflow has launched what, 5 QA sites in total now? All of which seem to be thriving pretty well. It seems they're expecting similar success with QA sites for Sailing, Pregnancy Advice, Cat Owners, Subaru Mechanics, etc.
Jeff and I started out with a goal for StackOverflow of changing the way programmers and system administrators get answers to their questions on the Internet, which was deeply broken. In 18 months we’ve accomplish that
Have they? Whenever I have a question the first place I turn is Google, and I usually find my answer on a site that's not stack overflow.
"changed the way programmers... get their answers" != "changed the way I get my answers"
Clearly, based on their traffic, they have changed the way many, many programmers get their answers.
A lot of those Google results are now coming from StackOverflow. Arguably, they've done a good job of centralizing programming Q&A and making it manageable. Certainly light years ahead of Experts Exchange.
I use Google to start the process of finding answers (often with results pointing to SO). But when it comes time to ask a question, I go to SO immediately and almost always get a reasonable answer within a few hours. The more this keeps happening, the more the Googling step will become irrelevant.
This is the thing about SO that really works. You almost always get a good answer within hours of asking any programming question (that's been my experience anyway).
It's getting better, but StackOverflow is still pretty dominated by C# and .Net questions. If you're using something else there is a significantly smaller chance that they'll turn up in your search results.
Check out the "Tags" page to see the popularity of various technologies on StackOverflow. Although C# and .NET are pretty popular on StackOverflow, I don't think they are disproportionate to their overall popularity in the developer community.
Look at how StackOverflow has displaced Experts Exchange, though, and it tells a clearer story:
http://www.alexa.com/siteinfo/stackoverflow.com+experts-exch...
I don't think there is ANY other general-audience programming site with more traffic.
I've been watching that over time, as a potential data source for http://LangPop.com , and StackOverflow has definitely "improved", but it has historically had a .net/c# bias compared to what turns up elsewhere, just like github has had a Ruby bias, which is slowly fading as github gains in popularity with the developer population at large.
Joel and Jeff don't create the content for the site, the communities do. The fact that other communities haven't moved into their sections en masse is not their fault. It's YOUR (ie. everyone else) FAULT.
For example, I tried to get the Grails community to use StackOverFlow when the site first started and their attitude was that they would rather just use the mailing list and sift through Nabble.
http://archive.codehaus.org/lists/org.codehaus.grails.user/m...
The problem isn't with StackOverFlow, it's with the other communities refusing to adopt something new. "After all, IRC is where all the best people hang out...who needs this .Net thing..."
I'm currently learning Lisp. You wouldn't be able to get this kind of an answer on a mailing list or on IRC.
http://stackoverflow.com/questions/2264267/generating-a-quiz...
This .Net/C# site just made it easier to learn Haskell, Lisp, Erlang, Scala... a little irony not to be overlooked.
"The fact that other communities haven't moved into their sections en masse is not their fault. It's YOUR (ie. everyone else) FAULT."
The first sentence is fine. The second not so much. "Fault" assumes that moving to SO/contributing etc is unambigously a good thing. I would not be suprised if at least some developers thought their existing communication channels are just fine.
"For example, I tried to get the Grails community to use StackOverFlow when the site first started and their attitude was that they would rather just use the mailing list and sift through Nabble.
http://archive.codehaus.org/lists/org.codehaus.grails.user/m.... "
This sounds like a valid choice to me.
"The problem isn't with StackOverFlow, it's with the other communities refusing to adopt something new."
They may have good reasons to do so. Just because you are enthusiastic about SO doesn't mean everyone has to be.
(Due Disclosure : I don't use SO at all. I once saw an algorithm question on SO I could answer and when I tried, I found out I had to use OpenID to log in. Couldn't be bothered.)
StackOverFlow provides structure and organization to the information. I've dug through enough email threads that ended at a dead-end to be able to say that the structure is welcome.
People tag their questions, while others vote on the questions and answers. Question titles are matched against the database when people enter new questions. Admins "close" duplicates, and stop trolling. The reputation "economy" provides a way to get an answer.
StackOverflow isn't perfect but it is a big step forward in organizing the information so that that the data can be searched more effectively.
http://stackoverflow.com/search
There are 500,000+ questions tagged. Some are great, some are not. Joel and Jeff don't provide either of these. We do!
Don't try this on a mailing list.
http://stackoverflow.com/questions/101268/hidden-features-of...
The word "fault" implies that there is a problem that needs fixing.
The Perl community has a decade of history and over 800,000 posts of history invested in http://www.perlmonks.org/. Granted the site is creaking and could use a better design, but someone with a Perl question is likely still better off going there than to StackOverFlow.
The glass is half empty? I think you're completely missing the point. It's really up to each community (Haskell, Lisp, Scala, Clojure, etc.) to develop their appropriate sections of StackOverFlow. I've been on an Lisp bent lately, mainly Emacs Lisp. Think of a good question and ask it. You'll be throwing down breadcrumbs for the next guy who wants to learn an esoteric language. I've got 86 questions, and the last dozen have been mainly about Lisp, of some flavor.
http://stackoverflow.com/users/5020/melling
I'm going to turn Emacs Lisp into a mainstream developer tool. :-)
StackOverflow is a help when you don't find the answer with Google. It's not really new: it's more a successor to Usenet which is getting quite dated -- especially since most of the old folks have abandoned it.
That's a bit silly, it's structurally unlike usenet. It's advantage is that due to its structure it has the S/N that usenet once did, which is much to its credit since usenet collapsed under the explosive growth of the internet yet stackoverflow appears to just get stronger with growth.
It seems to vary a lot by area. In my usual day-to-day Ruby mode, I rarely come across Stack Overflow. For a couple of weeks, though, I've been building an iPhone game and SO seems to come up all the time for Objective C and iPhone framework stuff.
I often check Google first, not find anything helpful in the first few pages, then remember to try StackOverflow, and have a much higher success rate.
Actually, the best technique I've found is Google with "site:stackoverflow.com", as Google does a much better job searching StackOverflow than the built in search engine.
I wish they would change it back to the way it was when usenet was popular. Years ago, when I was looking for an answer, the first place I would go is dejanews and google destroyed that pretty much... so sad.
One of the things that comes to my mind is this:
Craigslist is widely described as leaving 90% of the money on the table. Most people don't think this is why they are this popular. They could have monetised more categories or cities and still kept their position. Maybe.
But maybe they did exactly what was necessary to become the number 1 classified site. What if the right amount of revenue for some top 200 site (or network of sites) is really a couple of million?
Can such a site survive VC or does the option of staying a certain size disappear??
Starbucks was a perfect example. They new exactly where the business was going if constraints got lifted. More stores. Potentially tens of thousands. If they were less capitalised they would have to go slower but they knew where they were going.
I wonder how this modifies their StackExchange offering. Does this mean that if you are doing anything beyond a internal knowledge base Stack Overflow is now a competitor?
The twist here is that StackOverflow is its own company (of which Joel is a founder) and StackExchange is a product of Fog Creek, currently available to everyone. If their pitch to VCs is "we can add sites for XYZ topics" what's preventing a competitor in that vertical starting up a StackExchange site on the same topic?
Nothing. It's not so hard to code, either, if you have time. That's why it's a land rush.
In this land rush you offer would-be competitors a turnkey solution to compete with you. So hypothetically let's say you were going to get into Automotive or Cooking Q&A and Car & Driver (or Cook's Illustrated) sees you get a little bit of traction and they sign up for StackExchange and heavily promote it faster and probably more effectively than you can since they have a dedicated core audience. This doesn't worry you? Or are you going to avoid verticals you can't quickly get a critical mass in?
Seems like a nice position to be in, actually.
Either you get a successful site in that vertical, or somebody else pays you $50k/year for the right to have it instead.
If this is intentional, that's an interesting idea. Once the concept is out there, it is a lot easier for someone to role their own competition. If there is going to be competition anyway, have some control and/or a stake in success.
If a site gets to the point where it is a substantial business, they may well want to partner with Stackoverflow more closely anyway to get features, advice, access to whatever revenue generating plan they come up with.
Lately, Stack Overflow has been a mixed blessing.
How it goes: I post a question(questions are rather specific, but technology stack is very common on SO)
I usually get one-two answers very quickly and maybe one more slightly delayed answer, all helpful but also rather generic. You definitely get the impression people are just point whoring.
After a day or two I end up accepting the closest answer, even if it is not that close.
Seems real experts have moved on or perhaps I am missing some trick in asking questions.
If this problem persists on their scaled out sites, I am not sure how much VC money will help them.
Someone has been envious on the Vark acquisition :-)
Of course VC makes sense in their case. There is no barrier to entry in this field, the model is clear:
* users tend to type questions in search engines
* search engines tend to rank up exact matches (especially with some trivial SEO like put the question in the title and url)
* given the proper environment, users will help each other answering those questions for free, generating lots of content - exposing ever bigger surface to the search engines
Why do they feel the need to publicize their desire to raise money? Surely they can get their foot in the door of potential VC's based on Joel's reputation with Fog Creek alone, so what do they gain by announcing that they are looking for investment and giving a bunch of reasons why it's the right thing for StackOverflow? Why not just quietly raise the money and then, if/when they get funded, announce it along with the reasons?
As a user of some of the more recent Q/A sites (StackOverflow and Fluther) I find one of the most valuable pieces to be the community. I don't think that necessarily scales to a venture-worthy size or that venture capital leads to long term care takers of that community. However, clearly the founders seem to think otherwise. Joel's looking for money and Fluther took seed funding.
"... There are a few indicators for the type of company that I believe can benefit from, and should take, VC. ..."
The question I'd be interested in knowing is, "if SO was in accepted into a YC intake, what advice would SO be given to evolve given the app is built, idea is validated?".
Joel's going to the Valley to pitch SO while Jeff's off in New Zealand at Webstock/on vacation. Do VCs care if only half of the founding team is there? What if one of them is 1/3 of the development team and the primary technical driving force?
No VC would invest based on one meeting. Nor would they expect the guy who is building the site to schlep along on every pitch meeting -- Jeff has better things to do!.
Presumably if we found some interested parties, they would want to meet the key team members, in a second or third meeting.
No, in fact I think it's expected that only one of founders be present for initial meetings. If I were a VC I'd wonder why they were all wasting their time when it could be handled by one person.
OT but the bronze statue of the two "guys" with the coin... anyone know what that is? There is a similar style statute outside of the ICC in Indianapolis.
Those statues are in the New York Subway at a couple of stations. I believe that particular one is at 14th St. and 8th Ave., on the A line, but I'm not 100% sure.
Tom Otterness
Jeff has to go in the new company. He is a liability for the VC.
Care to explain this a little more?
Jeff talks publicly all the time about how much of a fuckup he is, how proud he is that he sort-of figured out something you thought was obvious, how he just can't do anything right. It's his public persona, and the subject matter for his formulaic blog, where posts usually end with an explicit confirmation that it was a waste of your time: http://blog.wekeroad.com/blog/nothing-to-say/
He doesn't talk so much about what a successful badass he is (see Joel), despite obviously having accomplished some pretty major shit.
And all of the above presumably helped him to co-found a successful (in a make-something-people-want sense) startup. Not too bad after all, eh?
I disagree with the grandparent poster because it doesn't matter if you come off as a bumbler or a chancer in this environment - actually, Atwood's gungho informality is probably a plus. But I suspect he's getting at Atwood being routinely wrong or misinformed about all manner of developer related topics - just listen to the Stack Overflow podcast for a few weeks to get a feel for it. Joel's routinely correcting him on stuff that's not even in Joel's area of expertise. (But that, versus someone who's scared of ever looking wrong? I think I'd invest in Atwood.)