HN: The Good Parts (2016)
danluu.comI think besides dang and other moderators who keep the site somewhat cleaned up through brute force, the fact that HN discourages flashy features and headache-inducing levels of information density and imagery helps to keep a core audience present, unlike most other platforms that start out simple then quickly pile on feature after feature that make the original audience depart.
I almost agree, but in the HN tradition, I have to pick a nit:
> headache-inducing levels of information density
HN is one of the more information-dense sites of its kind - because the space that isn't wasted on "flashy features" and advertising is filled with content.
Information density is good and desirable. Bloated UIs full of ads and irrelevant distractions are dense with noise.
It definitely does and I wonder if there is a larger lesson for other forum / software. We can't seem to be able help ourself and not change things in the name of "improvements" and appart from OSS I can't think of many other example then HN that just held to it's roots.
Sites like HN tend to plateau unless there are added features to get new sign ups. Full-length profiles, social features like chat, reply notifications and following/subscribing to users are engagement tactics to keep the community engaged with each other. These are site improvements if the goal is to grow beyond the core audience.
HN has been growing at the same rate since shortly after it began over a dozen years ago: basically linear, with a lot of swings (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...).
We want the community to grow—it would be concerning if it didn't. But the engagement tactics you mention aren't necessary for the kind of growth we want. In fact we consciously avoid them. Certain kinds of engagement—probably just the sort that engagement tactics would juice—would harm what we're trying to optimize HN for (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...).
The interesting aspect of this, I think, is that HN doesn't need to pursue growth at all costs, the way that a startup would. It's not a startup, nor a business per se, but it's not noncommercial either—it's funded by a business that understands that it's more valuable with HN than it would be without it, and is smart enough not to try to squeeze profit out of it beyond that. YC's business interests are in having HN be as good as possible, not as big as possible. That's odd, and oddly satisfying. It seems to be a historical accident that HN ended up in that sweet spot. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
Late reply, but I was speaking in generalities. HN doesn't quite fit the bill for a general forum that would improve from those tactics. There is a somewhat focused expectation of submissions, and the comments typically need a certain caliber of quality. Forums like this that deviate from those core tenets converge on being a Reddit clone with fewer users and features.
It's natural for users to engage less and less over time. However, holding the community to a certain standard keeps users from outgrowing the submissions and discussions. No one is too old, too mature, or too "smart" for earnest discussion.
Is it okay to plateau? If HN keeps going, gaining and losing people over time and not growing above X% of the population, and eventually declining for whatever reason, that seems the way of things. Change happens, and the best we can do is guide it and prepare. It’s okay to live and then die.
It's cosmically ok for things to plateau, but from a local perspective I'd be concerned if HN were plateauing. It would mean that something else was wrong. Probably a lot else.
The mandate of the site is to be gratify intellectual curiosity (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...). Any website achieving such a thing will naturally grow.
Thank you for the reminder about the intentional limits of this forum:
> One important thing to realize is that we're trying to optimize HN for just one thing, which is curiosity
I’ve found it easier to practice creativity, for example, if I impose some boundaries.
For better or worse, given the contrarian nature of this community, many here would probably view a decline in membership as a good thing.
I think we've seen this before.
Comments, everywhere are terrible. Heck, this one probably is.
But I like HN more than anywhere else (so far). Maybe rot will set in, and the sun will set on our relationship, but not yet.
I find many comments to be quite correct. Not all, of course, but many.
The ones that I like, are the ones that make me think "orthogonally."
Yeah I come here to read opinions of intelligent / informed people who disagree with me. I even subtly and respectfully try to rile them up sometimes to see what they have to offer.
I have found that the best way to get correct information is to confidently state incorrect information.
That's Cunningham's Law. But please don't do it on purpose. That would make for worse discussion overall.
:)
Not to worry. I do it fine, by accident...
One thing I like about HN, is I can see many comments that I think are totally outrageously wrong and still get into a civil discussion with someone on why they think that way.
This would never go down on Twitter, Reddit etc. I think by virtue of being here, there's at least a semblance of something that unites us.
That being said, I've definitely seen a few situations where "logical fallacies" have been wheeled out to try and autowin an argument. However, this is usually the exception and when things get really bad the moderation is usually on point.
I don’t think the informed technical opinions are nearly as rare as danluu characterizes them to be, to wit: “it’s rare that a thread will have even a single comment that's well-informed”.
I do agree that the farther you get from computer tech, the more frequently you’ll see patently wrong comments (aviation is my eye-roll inducing topic here), but on computer tech topics, I’d say more than 50% of substantive comments have positive correlation with correctness and more than 30% are actively good.
I do wonder if the definition of well-informed includes agrees with my viewpoint somewhere in it.
Look at the examples he picks - I take "well informed" to mean not only knowledgable about the subject matter but also providing an insight beyond a platitude.
I think the following does not particularly offer any deep insight:
"Curious why he would need to move to a more prestigious position? Most people realize by their 30s that prestige is a sucker's game; it's a way of inducing people to do things that aren't much fun and they wouldn't really want to do on their own, by lauding them with accolades from people they don't really care about."
HN comments, by and large, are extremely polite, and people are open to discussion, criticism, etc without resorting to insults, in my experience. I don't really see the sort of complaints brought up in the post here, well, ever, outside of the rare ghostly, semi-transparent comments that have already been flagged or filtered.
You don't have to spend long on Reddit, Youtube or Facebook to really appreciate just how nice and non-combative it is here. Maybe it is just the relative difference from those sites' comment sections that blind me to how many assholes are regularly posting on HN, but I really don't see them.
Admittedly, as another comment pointed out, the further it gets away from tech, the less informed the comments tend to be (the recent thread about mystical experiences and psychedelics had the level of rhetoric and nuance I'd expect from a teenage Reddit capital-A Atheist on the subject, for example). But that's fine, it's not where I come here for.
> You don't have to spend long on Reddit, Youtube or Facebook to really appreciate just how nice and non-combative it is here
Let me offer a counterpoint (I'm glad you didn't mention Twitter, because I'd have to really agree that Twitter is a Hate Machine. And I'll grant you YouTube as well, to a lesser degree): I use facebook mostly for its closed, special interest groups. For each of these groups, which are relatively civil -- until there's the occasional namecalling flamewar and some people get expelled -- the most common assertion is by far something of this kind: "why, this is the best group about $SUBJECT, people here are so nice and civil and I haven't found a group like this one anywhere on the net!". If you belong to a couple of groups on facebook, you'll see this repeated in every one of them, each of these groups oblivious to the existence of other, similar or possibly nicer groups, each selectively forgetting the most recent fight.
I think some degree of this phenomenon is at play here on HN, too.
I agree, the only thing I use Facebook for these days is 2 private groups on specific subjects. While they're incredibly good for the kind of support group style 'ask a specific question, get a quick answer' type of thing, they definitely suffer from the problems in the article - mostly people getting very defensive and combative when their comments are questioned or criticised. That may be a characteristic of the type of group I'm in though.
He missed a couple of the things that bother me about HN:
* People fishing for upvotes by complaining about how X news site uses Javascript or has a paywall. We get it. I know. I agree. Stop posting it.
* Top comments being tangential responses to the title, not discussions of the linked content. I usually enjoy the tangential responses, but I want to see the more-relevant discussion voted higher. I can only imagine that these posts are being upvoted by people who also didn't read the article.
> On any topic I’m informed about, the vast majority of comments are pretty clearly wrong.
I'd be curious to see some examples, because this isn't really my experience. Once in a while, yes. But not that often and not the vast majority, and it mostly pops up in political-flavored threads. But I should defer to Luu, who is a lot more informed than I am on a lot of topics.
Ranking 'quality' is tough because it has multiple dimensions and aggregating multiple kinds of utility is problematic, see
https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theore...
developing a good ranking function for bag-of-words full text search is about carefully balancing the attraction of larger vs smaller documents in the collection. If you try to aggregate several bad search engines you get a bad search engine, if you try to add more factors you get a bad search engine, instead you have to walk the path of BM25 or one of the more modern 'information theoretic' ranking functions. (e.g. precious knowledge I got from years of reading conference proceedings and still having no idea how to make a good ranking function then finally reading the right review paper that pointed out the two discoveries the conference made in the first 10 years!)
Are you saying you do know the way to create a good ranking function? Does Arrows theroum stop that?
I know how to make some ranking functions that are good in some ways. They are not good in other ways so I don't break Arrow.
The shadow of Arrow however is that it is a lot of work to get a real improvement over a tfidf with random characteristics and you are limited by an asymptote which is less far away than you'd like: the rel teams at Google and Bing cannot beat the p=0.7@1 barrier for text search because 0.3 of the time they will get your intent wrong.
There are ranking functions in Lucene that can be tuned up as I described by making a test data set and using the methods developed at the TREC conference. That kind of tuning really works. At that conference you might see people fight over a point or two of AUC and I don't know if users can feel that but I am sure users can feel the 15 point and more difference we were seeing from tuning.
The odd thing is that hardly anybody does it and the knowledge seems pretty obscure. (e.g. I have talked to people at Lucene, OpenText and other full text search vendors and they are much more impressed with having 10,000 connectors than with the search results being good.)
OT: I hope Dan Luu write his blog more often. There is no post this year and he seems to be consumed by Twitter. His tweets are witty and I like it, but I miss his long form, super deep dive kind of writings. These are marvels. I also like the witty ones like this, and I would love to have it as blog rather than as tweets.
Totally off-topic: One of my favorite from his writing is "Sampling v. tracing" [1].
Like anything, great things die with popularity, or at least become just good.
When I started reading HN ten years ago it was a vastly different culture among users. Or maybe I just didn't recognize the underhanded asshole tones.
Either way, HN is still good enough compared to any alternatives I know about. I guess that's just life though.
> but maybe consider, at the margin, blogging more and commenting on HN less?
Have considered this seriously (and still am), but a blog post needs more meta context and exposition than an HN comment. The person browsing a blog post vs. playing the HN slot machine (time bandit?) is in a more passive mode than the give and take of HN comments.
The difference to me is that a blog post is a topic of conversation, where HN is a conversation. It's discourse, facilitated by candid pseudonymity, and not just a foil for someone to pose with. (whereas twitter is just the instagram of ideas.)
I have written at the pro level where it really is like a sport, and a lot of my job involves articulating complex things with clarity and pith, and to me, as a form, the blog post is too passive. However, if I get good enough at that form my opinion may change.
I'd like to see the worst-rated comments by anyone with -- say -- over 10k karma.
I have almost 30K points and one of my most recent comments was -1'd. Take a look. It happens a surprising amount, and is IMHO rarely legit (per the HN "rules").
It is certainly legit if someone buries a sarcastic or trivial comment of mine (which happens, though I don't do it often), but I don't bother to delete them -- downvoting them does that effectively anyway.
I'm not sure if there's any actual value to karma and I don't really know how it's assigned and wouldn't really notice if it weren't on the top bar of the page. I haven't tried to do any experiments though.
Some previous discussion: https://news.ycombinator.com/item?id=20092118
Also discussed at the time:
HN comments are underrated - https://news.ycombinator.com/item?id=12772925 - Oct 2016 (359 comments)
I, for one, am really impressed by the comments. The quantity is amazing for the quality, and the quality is amazing for the quantity. The top 1 comment isn't always great, but one of the top 3 usually is.
Or so I thought... till I read this Good Parts post and now I am left with this vague feeling I've been wrong all along. I wish he had said why some comments are so bad.
I now don't want to end up as the top comment because its always bad?
I sometimes wonder how HN’s audience is shaped by the site’s deliberate inaccessibility to visually impaired people. I suspect it is one part of the site’s tilt toward the younger crowd
> the site’s deliberate inaccessibility to visually impaired people
Can you expand? I naively thought that a site with markup as simple as HN's (zooming is flawless and it even works without JavaScript) would be very accessible to visually-impaired people.
I should have worded this a little differently. I don’t know how accessible it is to someone who is actually blind or uses a screen reader. For all I know it’s great for them.
I do know the tiny text and low contrast make it hard to use for someone like me with older eyes. This is especially true on mobile, where the text refuses to resize in accordance to my phone’s large font settings.
I'm not the parent poster, but I hear where their coming from.
I'm not visually impaired, just in need of bifocals; I zoom HN to 150% and it works fine.
But I am kinda surprised when I turn the zoom off. Even 25 years ago that text would have looked small to me.
Color ratio of a typical comment on this web site:
https://contrast-ratio.com/#%23dddddd-on-%23f6f6ef
1.25, which is well below the minimum of 4.5
I would not characterize the [dead] comments (which are #ddd) as "a typical comment". They're designed to be hard to read. If I turn off showdead (which I believe is an opt-in setting), it looks like the lightest comments are #9c9c9c, which is a 2.5 ratio (and of course are also comments which are intended to be less prominent).
They ought to have a button to put on a high-contrast stylesheet, in fact I bet some hacker has already done it.
I'd assumed anyone with visual impairment—but not outright blindness requiring a screen-reader—was using custom stylesheet stuff pretty much constantly all over the web. Controlling the appearance of sites client-site used to be a basic browser feature. I know that's receded into the background, but is something similar (either some accessibility feature, or a plugin) not the norm for those who need them? If not, that sucks, because control used to be so firmly on the client-side for those things and that was a good thing.
Maybe this is my inexperience with front-end design keeping a11y in mind: what about HN is inaccessible for visually impaired people? It is mostly plain-text so it is easy to resize. There is some low-contrast text but that can be adjusted in CSS with user styles. I haven't tried a screen reader but I imagine that it would work reasonably well. All not to mention that there are HN RSS feeds, so the content can be consumed in any way that a person might want.
I'm no expert, but the comment threading and indentation using table cells and spacer gifs doesn't seem like it would be straightforward for a screen reader.
Agreed, making the divs article tags or adding an article role would help, but personally, as a blind user, i don't care as much, the content is interesting enough that it's not a big deal. You can deduce the tree structure anyway, at least most of the time.
Right, any web page with sidebars, popups, cookie warnings, and other distraction is "not accessible" by default.
Interesting thought.
I'm wondering how effective non-text would be? Personally, text is my preferred medium for any non-simple communication.
I know teens who do it all on Tik-Tok.
Marshall McLuhan was right to point out that "television" would erode the need for people to read and write.
That reminds me of people who "learn it all on YouTube."
Maybe it's the best option for some people. Not for me, but I do love an audible book.
In conclusion, I think these other mediums simply lack the iterations that we've put into text over the centuries, leaving text as the clear winner in my book.
YouTube is great for things where the directions for the specific activity you want might not include "basic" things someone who knows it well wouldn't think to list in written instructions, or some little technique or movement they do automatically and don't even think about. I love it for house and car DIY stuff, for that reason and also because it makes it clear which parts will be a pain-in-the-ass, and how much, to help me gauge whether I want to do it, or hire it done.
It's also great for technique with tools in the kitchen—not learning recipes so much, but learning the right way to process a certain vegetable really fast, or exactly what a proper whipping motion for eggs ought to look like, or how a pro moves the pan on and off the flame while cooking this dish, exactly what it looks like when they deem a certain cooking technique "done", all of that.
As someone who isn't visually impaired I have little to go by or background to judge, but HN strikes me as not being particularly inaccessible, given its minimal UI and very little reliance on non-textual elements, in particular compared to most social media sites.
What in particular would you wish to have improved?
I mean, the site scales really well with browser size, default font size etc. I'd argue the real inaccessibility is lack of images, and general "boring" vibe that puts off entire demographics who post the type of content that might not be welcome here.
I think a large number of people are unaware that you can scale web pages up with Ctrl -, Ctrl +, particularly older people who complain that "text is too small".
I'm not even old (mid-20s), but I scale the site to 150%. HN is honestly one of the worst sites I have used with regards to default scaling across resolutions.
OTOH the simplicity of the site allows for very easy browser-based scaling, which means I only really think about how bad the text size is once per machine.
> The ranking scheme seems to penalize posts that have a lot of comments on the theory that flamebait topics will draw a lot of comments. That sometimes prematurely buries stories with good discussion, but much more often, it buries stories that draw pointless flamewars.
If you like stories that generate robust discussion, use: https://news.ycombinator.com/active
This 'hidden' view of HN tends to allow high-discussion topics to stay at the top, and not get insta-buried because they're controversial. I use it as my default now. In my opinion, it's much better than the main sorting algorithm.
Conversely, if you like stories that could generate good discussion but never got traction, the second chance pool is now open to the public. [0,1]
That is a bad sign. Someone who's only interested in the most overheated discussions and doesn't want to look at the front page is not using HN in the intended spirit of curiosity.
The intended use of HN is to run across things that one isn't already familiar with, not to argue about the same small number of hot topics over and over. People who want to do that should find a site that wants them to do that.
Different strokes for different folks. I'm glad HN offers both views! I personally think the flamewar-detector, while useful for keeping a good signal-to-noise ratio, is too sensitive and browsing /active lets you find things you would have only seen if you are one of those readers who refresh / every 5 minutes. Plus, there are only so many "Obscure Tool Rewritten In Rust" articles I can take :)
> there are only so many "Obscure Tool Rewritten In Rust" articles I can take
Yup, that's an issue for sure.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
Thanks for this! Awesome tip!
I like the idea of preserving the best comments as some form of blog. TheBestOfHN.com might be worth starting up by just compiling all comments with > 200 karma?
Those scores are not so interesting by themselves in that they were calculated based on votes at the time and aren't meant for comparing comments on different articles. (The top comment on a high-scoring article will automatically beat the top comment on a low-scoring article)
I would rather like to see what Dan did scaled up 1000x or so: that is, have experts pick out a few of the best comments over the very long term with a number of quality criteria in mind.
Like it or not individual people have characteristics that are mostly stable over time: they are more or less smart, mean, wordy, honest, loyal, interested in different things, make different mistakes, etc. Profiling the author is worthwhile.
The article though is a problem not a solution in terms of sampling: consider that people talked about topic A in 200 articles that hit the front page in the last 5 years; if you put the comments from those 200 articles in pool B and rank the articles in pool B and pick up the top ten you would get the "pure gold".
is this actually feasible - if we push it beyond HN then it becomes trying to crowd source wisdom or deep expertise across human experience. The Anti-Social-Media-Algorithm?
This sounds ... tempting...
The case of Wikipedia shows that efforts like that can be scaled but it's hard to find more examples.
A counter example is the old DMOZ directory which was a continuation of the tree-style Yahoo web directory.
That was subdivided into sections that were managed by "experts" (admins appointed to the sections) but the admins were lazy, corrupt or both. Even if they had been doing their jobs they would have been overwhelmed by hardly relevant "content marketing" submissions for web pages that are "200 OK", hot keywords, and a scam.
You have to draw from a spam-free pool.
For instance there is a data dump for stackoverflow
https://www.brentozar.com/archive/2015/10/how-to-download-th...
personally i think stackoverflow is a junk web site full of more wrong answers than right answers and wouldn't it be nice to pick just the best answer to the question and not see the code sample at the top that didn't work that the original poster was asking about?
If you are a real programmer, I mean you are shipping, you write a unit test, if it doesn't work they are gonna send it back to you, you may learn the hard way that answer #1 is not right (the compiler said so!) and scroll through a lot of half-baked discussion before you find that answer #7 passed the unit test, the acid test, all the other tests -- you are an expert and your opinion is worth much more than the "crowd".
If "Road Scholars" like us were getting our experience fed back into Stack Overflow how different would it look?
> (I remember posting a huge thing on slashdot when this article was originally posted)
Appropriate that _this_ comment is buried who-knows-where in slashdot's archives.
Whenever I find a good comment, I clip it.
On a lighter note and for humor read webshit weekly at http://n-gate.com/
Don't violate the prime directive.
I expect this post will get few comments out of self-consciousness.
On any forum, meta-posts are sometimes a lightning rod for stupid comments and so you often see meta-posts banned to prevent it!
"HN comments are horrible."
Proceeds to list pages of awesome comments.
Claims topics he's informed about have mostly wrong comments.
Doesn't give any examples.
HN comments are a conversation/discussion not a contest of who is right/wrong, and it's great to have differing viewpoints as long as those views don't degrade into logical fallacies like ad hominem, ,anecdotal, or others.
There are way less logical fallacies on HN than other sites which is what I love about it.
I think some people just want an echo chamber.
I think there's a blend of good and bad.
If you ever see a HN front page story where you have inside information about some event, you can rest assured there will be maddening amounts of completely wrong speculation written as if it were gospel by people with no connection to that event. And then people attack the strawmen with abandon.
And for professional reasons, you grit your teeth, keep your mouth shut, and let it all slide. I'm not in PR.
"Someone is wrong on the Internet": https://xkcd.com/386/
I agree that HN is better than a _lot_ of places. But it's not all wine and roses.
I guess I've never had inside information to see this firsthand.
Or maybe my internet spidey sense is so attuned that I've learned to subconsciously tune out B.S.
So I've never really seen that type of thing happening.
Does anyone really believe unverified information people post on the internet ?
I don't think that is a fair reading of danluu's post. It is titled "the good parts", after all. And did you read the closing paragraphs?
> "[...] Having flamebait drop off the front page quickly is significant, but it doesn’t seem sufficient to explain why there are so many more well-informed comments on HN than on other forums with roughly similar traffic."
> "Maybe the answer is that people come to HN for the same reason people come to Silicon Valley -- despite all the downsides, there’s a relatively large concentration of experts there across a wide variety of CS-related disciplines. If that’s true, and it’s a combination of path dependence on network effects, that’s pretty depressing since that’s not replicable."
So he does consider HN successful, he is merely pointing out that the gems are infrequent (though more frequent than on, say, Reddit), and that most comments are uninformed or wrong.
If you read anything from danluu's site, you'll see he is not really an "echo chamber" kind of guy.
PS: I've seen plenty of illogical, flat-out wrong and anecdotical comments on HN to know they are the norm rather than the exception. This doesn't preclude the existence of real gems, like danluu points out.
It's a sensational opening that he spends the entire rest of the article contradicting.
HN comments are horrible...except here's a thousand line blog about why they're not horrible.
Judging by the content of the entire rest of the post...a better opener should have been "Hacker news comments are mostly awesome and here's a thousand line blog post about it.... but they're not all perfect".
What a bizzare article.
It’s a sign of elitism to have this much contempt for internet comments. The internet is not academia or your fancy hierarchical tech job, it’s a place where anybody of any ilk can have a say. Yes there will be wrong comments or mean comments but that’s just because you’re amongst the people and the people are imperfect. Yet it’s the masses of people who form the audience of this site that make this site so valuable.
If one cannot get over their contempt of bad internet comments then it might be better for their stress levels to stick to reading academic journals or other elite publications. Much higher ratios of nice/correct opinions there.