When does it make sense to downvote an answer? Do downvotes help improve the overall quality of answers on the site?
I wrote this Q&A mainly because I wanted to share the results of that very fascinating paper I linked, and think of how to apply them to the JLSE context. I am quite open to others thoughts on this question as well, however :).– Darius JahandarieSep 15, 2014 at 21:29
Why do people downvote?
I speculate there are 3 reasons someone might downvote a post:
- Because they think the post is inaccurate and want to signal that to the community.
- Because they think the post is poorly formulated and want to signal that to the community.
- Because they want the author to learn to not post low-quality answers.
I'm going to discuss whether downvoting is the best way to achieve any of those three goals.
There was recently a very large study done on how community feedback shapes user behavior: http://cs.stanford.edu/people/jure/pubs/disqus-icwsm14.pdf
From the abstract:
"[...] By studying four large comment-based news communities, we find that negative feedback leads to significant behavioral changes that are detrimental to the community. Not only do authors of negatively-evaluated content contribute more, but also their future posts are of lower quality, and are perceived by the community as such. Moreover, these authors are more likely to subsequently evaluate their fellow users negatively, percolating these effects through the community. [...]"
I'm going to pull a lot of my factual grounding from this paper.
Goal 1: Signaling post inaccuracy
Often there are answers which simply have inaccurate information, e.g.,:
The most polite way to say "Could I please have some water?" is "今すぐ水をくれ。"
Such posts are often downvoted.
Alternative: Leave a comment. If you directly point out the inaccuracy, it will
- allow the author to fix it,
- signal the post inaccuracy far better than a downvote (which has an ambiguous meaning),
- will not have any of the downsides of downvotes which I will mention soon.
Goal 2: Signaling poorly formulated posts
Sometimes, while an answer may have accurate information, it formats it very poorly or misleadingly.
In this case, a comment sometimes may work ("Have you considered introducing concept X before concept Y?" "It seems a little misleading when you introduce 熱いです along with 医者です。"), but often feels like you are being too picky about how someone else should write their answer.
Alternative: Write your own answer (but make sure to credit any ideas you got from the initial answer). This way:
- there are two answers, one of which viewers may understand better, and
- hopefully, the initial answerer does not feel unwanted (due to a downvote) or stolen from (due to a very similar answer which does not give credit).
Alternative: Talk to the author on chat. While comments are not a great place for opinions, opening a dialog in chat seems to be a nice way to discuss something more opinionated or complicated. (Unfortunately, this sometimes fails because some users seem to not use the chat system.)
Goal 3: Make the author not post low-quality answers
Sometimes, you may find yourself thinking that an author's posts are consistently of low quality and want them to stop posting such answers.
However, according to the paper, downvotes are not just ineffective, they are counterproductive:
Downvoting authors does not cause them to write higher-quality posts in the future, it causes them to write lower-quality posts:
"Rather than evaluations increasing the user's post quality and steering the community towards higher quality discussions, we find that negative evaluations actually decrease post quality, with no clear trend for positive evaluations having an effect either way."
Downvoted authors does not cause them to post less, it causes them to post more:
"The fact that both types of evaluations encourage users to post more frequently suggests that providing negative feedback to 'bad' users might not be a good strategy for combating undesired behavior in a community. Given that users who receive no feedback post less frequently, a potentially effective strategy could be to ignore undesired behavior and provide no feedback at all."
Alternative: Like the paper suggests, do not personally vote on the answer. Also, I believe that having another answer available for a question will make people less likely to interact negatively with the low-quality answer. Finally, if there is a better answer, the author of the low-quality answer may realize that their Japanese was not as good as they thought it was without the negative feelings of someone calling them out on it.
4I think that the findings of this paper contradict the reality of what we have here. Occasionally users will join, post a few answers that, while written with good intentions, get downvoted because of their quality or accuracy. It has seemed to me that rather than ramping up their bad answers, they tend to get frustrated and leave instead of trying to conform to the strict standards here. I can see why it would be the opposite on news sites, which attract extremely passionate and opinionated people who might view negative feedback as a vindication, but I think people here don't fit that mold. Sep 16, 2014 at 6:34
1Good point. I admit, especially with new users, that I've noticed the same (downvotes seem to make them go away), though I haven't actually run any numbers to confirm that. I think their point about downvotes not changing post quality probably holds quite well here however. Sep 16, 2014 at 6:37
Also, I think with longer-standing members who could have things like pride in their answers and an image to keep up, I could easily see a downvote making them feel like they have to defend their image and make them post more to achieve that. But this is speculation, I should probably run some numbers if I actually want to drive a point home here. Sep 16, 2014 at 6:39
4I agree that downvotes probably won't help anyone improve, especially anonymous downvotes with no constructive criticism, and I don't necessarily disagree with the conclusion that we should avoid downvoting. It's just the applicability of this particular study to our community. There are plenty of people who have posted here trying their best to be helpful. Maybe some have continued the streak for a few answers, but let's face it: most people don't stick around here, and even longtime users get frustrated with the culture and quit. You don't need to run any data to know that's the case. Sep 16, 2014 at 6:44
2I should say though that I wholly endorse the first two goals and alternatives. Sep 16, 2014 at 6:50
1I agree that most people don't stick around here and longtime users get frustrated with the culture and quit. However, the culture includes a lot more than downvoting, and there are other reasons people may not stick around other than the culture. It's the correlation and/or causation with downvotes that I think really needs number crunching (if I want to make any serious claims about it). I am basically agreeing that there are reasons to believe the study doesn't directly apply and I should replicate bits of it using similar methodology. Sep 16, 2014 at 6:51
1As someone who has written answers that were downvoted, I can say that when downvoted, the key is whether the downvoting is explained. Conversely, over on philosophy, [I am an academic philosopher] I often comment before downvoting and get the weirdest resistance to changes that would polish answers.– virmaiorSep 20, 2014 at 10:34
@virmaior Maybe one difference is that there are more clearly "right" and "wrong" answers here than there are at Philosophy.SE? (Of course it's not perfectly clear in natural language, but certainly more so than philosophy I'd think). Pointing out factual inaccuracies seems like a much more accepted use of comments here than comments which kind of just complain about something which is not as quantifiable or verifiable. Sep 20, 2014 at 23:05
@DariusJahandarie the comment you make highlights part of the confusion philosophy.se faces. Just as Japanese attracts a fair share of "please translate this for me," philosophy attracts both questions that don't relate at all to academic philosophy (i.e. rants with a question mark) and answers that aren't at all grounded in philosophy... It's precisely the sort of factual errors that people tend to try to argue purposelessly.– virmaiorSep 21, 2014 at 1:04
3I found this post interesting and worth reading. I agree with ssb that downvoting helps with dealing with new users who produce content the community would rather not see here. It's a bit different with established users - they won't give up because of a downvote or two, their reason for leaving is probably more complex. I agree with virmaior that receiving downvotes with no comments makes little sense as as a poster I cannot improve if I don't know what's wrong. All in all, I find downvotes a useful tool for community but one that needs to be applied together with other means of moderating.– SzymonSep 22, 2014 at 10:46
In an ideal world, the way #1 is supposed to work on SE is a downvote signals wrong information in the answer (possibly accompanied by a comment), and if the information is fixed, the downvoter later notices and removes their vote, potentially even changing it to an upvote. In other words, if you care for your content, you can turn even a bad start into a reward. That doesn't always play out in reality (answerer unclear what is wrong, voter doesn't revisit after editing, etc). I can see downvoting for #2 on questions, but answers? Editing usually works better. I don't think #3 works.– TroyenOct 6, 2014 at 5:14
However, I think not down-voting on #1 is harmful to visitors: imagine two 0 score answers, one is meh, one is wrong. If you know one is wrong, do you upvote the "meh" or leave both at 0 for other readers to get the wrong impression? Comments aren't always visible to viewers and can be buried depending on activity levels.– TroyenOct 6, 2014 at 5:16
@Troyen I don't agree with your last point. A comment signals the inaccuracy to visitors far better. A comment only starts getting hidden when there's too many other comments and it doesn't have enough votes -- I don't think I've ever seen that happen for a comment which is correctly pointing out an inaccuracy. Your proposed scenario (one wrong answer, one meh answer, both with the same number of votes, and so many comments on the wrong answer + lack of votes that the comment pointing out the inaccuracy is behind an extra click) seems like such an edge case that it shouldn't define behavior. Oct 6, 2014 at 6:24
As someone who is something of a more established user who decided to leave over this, I'll pitch my two cents in.
The short of it is that I got sick of being publicly shamed because my understanding of concepts wasn't as complete as I thought it to be when posting some answers—especially when this seemed to become a trend where a certain user or two would make a point of doing so quite publicly and abrasively. I have no problem with people pointing out faults in my answers (after all, I'm here to learn, too!), but the environment got to the point where it felt like unless you were a native or a professor who could be reasonably assured of posting a flawless answer you shouldn't even be trying to answer. I finally got fed up with it when someone went so far as to call my answers "dangerous".
I occasionally poke my head in because I do learn things from the conversations from time to time, but I feel absolutely no compulsion to continue being an active participant in the community.
Thanks for popping in! Would you say the public shaming was downvoting, comments, both, and/or something else? Sep 26, 2014 at 3:50
It varied between being just an uncommented downvote and a downvote combined with a comment. After a string of heavily downvoted answers I deleted the ones that started hitting -3 (since that appears to be the expected behavior, per the badge system) because usually by that time there was already an answer that was receiving acceptance from the community and I wasn't getting help improving my own. My attempt at being a good member of the community by doing so ended up being cited during the incident when I was called out for being dangerous.– KajiSep 26, 2014 at 3:54
3I took a look at all 91 of your answers, including deleted answers and deleted comments, which spans a period of more than four months. In that time, you received one negative comment which was not very nice ("-1 Not even close, seriously. This is a learning site, not a guessing game. You do not have to answer if you do not know the answer."), which I removed as a moderator shortly after it was posted. I'm not sure that constitutes a trend.– user1478Sep 27, 2014 at 2:15
1I was unable to find the "dangerous" comment. Could you supply a link?– user1478Sep 27, 2014 at 2:27
I can't remember where it was specifically at this point, just that it was in a standoff with Nagoya. May have occurred in chat.– KajiSep 27, 2014 at 16:03
2Ah, thank you, I found it. I see that an unfortunate exchange ensued when you called him out on chat . . . We should try to avoid picking fights in chat or on the site itself, if possible.– user1478Sep 27, 2014 at 16:48