9

(This is tangentially related to the question about identifying native speakers.)

At present, answers to questions appear to derive from the following sources (in my subjective order of decreasing reliability):

  • Grammar texts (the good ones)
  • Personal experience (as a native speaker or fluent learner)
  • Statements from a Japanese teacher
  • Japanese Q&A sites and other internet resources
  • Old classroom memories and hearsay

Seeing as we can contribute to the proliferation of bad information just as easily as we can good information, if we wish to make this into a valuable site for students of Japanese, it would behoove us to have a high standard for accepting answers. While I don't propose we start tagging unreferenced statements with [citation needed] akin to Wikipedia, I do think we should attempt to ensure credibility in answers. A mere upvote can mean many things. We all hope it means, "Yes, I have checked this answer and can confirm it to be true," but it could easily mean, "I don't know if this is correct or not, but it makes sense to me."

Off the top of my head, here are some possible solutions:

  • Ask answerers to voluntarily provide a "References" section below their answer (面倒くさい for the answerer).
  • Ask high-reputation users to comment their concurrence when they upvote (面倒くさい for the upvoter).
  • Implement a standard of "minimum reference credibility" and draw the line somewhere inside the above list.

I'm not sure if I'm bringing up an issue worth discussing at this point, but I look forward to seeing everyone's comments nonetheless.

10

Like my response to the other question, enforcing such guidelines will be difficult and will only serve to tick people off and discourage people from answering questions.

I think we should just maintain the status quo as-is (where users answer questions and additionally provides references if they so desire). If someone has a doubt and wishes to challenge their statements, they can leave a comment asking for more details and/or a reference.

This is pretty much how editors and readers on Wikipedia operate. If you find a statement questionable, you use the {{fact}} tag to mark it as needing a citation. This warns the reader that the statement has been challenged by someone else. Of course, on Wikipedia someone could just revert that edit or it could be a random person up to some mischief but I don't see that manifesting into a problem here on the Japanese SE (at least not enough times to make it more than a nuisance).

Edit: I think that instead of asking the people answering questions to provide references, we should instead encourage users to be bold (sorry, another Wikipedia reference :)) and question the answers that are being posted. As a teacher of mine once said, if someone has a question to ask, there's probably another person out there with the same question. If somebody is curious or puzzled about (something in) the answer, they should not be afraid of making a follow-up comment asking for further clarifications.

As Troyen pointed out in the comment below, this sort of a feedback loop ultimately ends up being beneficial to all parties involved.

  • 3
    I also think it's working fine like this already. Sometimes someone will challenge an answer that was based off of memory and the answerer will go back and dig up a reference and both people end up learning from it as a result. – Troyen Jun 8 '11 at 21:35
  • This is SE format. This is how it works. I have 2 correct answers with a -1 score, I've seen grammar aberrations upvoted to the extreme, wrong/incomplete/inaccurate answers accepted. Even native speakers wouldn't know the finest grammar point or etymology of a character. How do you provide sources to something that comes with experience? Asking users to be bold doesn't work. The ones who challenge an answer from a high-rep is downvoted. People prefer to upvote and get rep. I've been worried at the quality of the answers since the first day of beta. This is SE and this is how it works. – repecmps Jun 9 '11 at 3:18
  • So I suppose you are suggesting that the SE system is broken for language questions since people may not be able to provide sources for topics that comes with experience and those users that are unsure about the credibility of an answer who challenges an answer with comments end up getting their comments flagged as being unconstructive and confrontational? When information is crowdsourced with an open-ended user base, there are going to be issues and this is unavoidable in my opinion. I do appreciate the fact that you are still contributing despite having been discouraged by -1s, repecmps. – rcjsuen Jun 9 '11 at 10:59
  • Perhaps what I worry about the most is the weight of the check mark and what effect that has on an answer's future. Suppose an incorrect (or even just misleading) answer gets past the checks in place and earns a check mark without opposition. How likely is it that several weeks or months down the road, someone will bother attempting to correct the answer? And even should this happen, would a new answer to an old "answered" question have a realistic chance of stealing the check mark? A stranger arriving from Google would probably just go with the check mark answer without reading further. – Derek Schaab Jun 9 '11 at 12:29
  • Derek, this is going to be a problem for any open-ended Q&A site that lets random people (on the Internet no less) make posts. I suggest you bring this up with Meta SE. I believe the reason users are noticing this as being a problem on this site is because of our low number of users. If there were more people visiting the site then the additional pairs of eyes monitoring the answers and comments would hugely improve their credibility. – rcjsuen Jun 9 '11 at 13:03
  • Actually, I don't think this is a problem at the moment precisely because we have enough users to effectively police answers. But if the site grows to the point of having too many answers to monitor, the chance a poor answer will get accepted will increase. And once an answer is accepted, it's practically set in stone, is it not? The activity level drops off so much that you'll never see it again. We could hypothetically mitigate this by asking answerers to police themselves; i.e. don't just type whatever comes to mind, but make an effort to preemptively fact-check and back up your statements. – Derek Schaab Jun 9 '11 at 18:00
  • If the site grows disproportionally where there are more people unfamiliar with the language than there are people familiar with the language then the problem you raised could happen. I certainly agree that if someone were to just accept an answer within sixty minutes then there could be a problem if users are just answering questions "randomly". But no, the acceptance of an answer is a toggle state so you can uncheck it if you wanted to. This also works the same way for voting, you can remove your vote if desired. – rcjsuen Jun 9 '11 at 18:30
  • 3
    This discussion has shown up on meta a few times as well. See here. – Troyen Jun 9 '11 at 19:06
  • 1
    @Troyen: Thank you for the link; it led me to some interesting reading. (Although it's admittedly depressing to see that SO, where you would expect answers to be empirically verifiable, often has the same issues we're discussing here.) – Derek Schaab Jun 9 '11 at 21:15
3

Thank you for bringing up this topic. I have been worried about the possibility that the site may become yet another place to reproduce popular false claims, and it is important to consider this problem, even if we do not have a silver bullet for it.

A mere upvote can mean many things. We all hope it means, "Yes, I have checked this answer and can confirm it to be true," […]

I am not sure about this. My feeling is that in the ideal world an up-vote should mean that the voter verified the answer, but I suspect that for most people, there is nothing they can do to verify the answer. As someone who is asking other users to vote often, I rather argue that users should be aware that high score does not always mean that the answer is correct.

Honestly, I find this problem daunting and I am not sure what we can do.

I do not think that it is right to restrict answers to verifiable ones (even if it is possible at all). If we do so, I suspect that few people will be able to post any answers. For example, some native speakers have trouble backing up their claims :).

Personally, if I cannot back up the claims in my answers, I try to state so (at least implicitly by using phrases such as “I feel …” and “my impression is …”). But I have probably already failed to do so in some of the answers I have posted so far, and I cannot require everyone to do the same.

  • 3
    I think you guys have been pretty on top of spotting wrong answers so far, and even if other people can't know for sure whether an answer is right, they should see the argument raging in the comments and know the answer might be sketchy. – Troyen Jun 8 '11 at 21:41
  • "...users should be aware that high score does not always mean that the answer is correct." This could be communicated to users who are familiar with our system, but what of those coming in from the outside? I worry not so much about propagating bad information within this community as I do about spreading it (however unintentionally) to the outside. – Derek Schaab Jun 9 '11 at 12:32
  • @Derek: I agree, it is a bigger problem to spread wrong information to the outside without bad intent. As I wrote, I do not know what we can do, but I do not think that it is right to restrict answers or votes to the verifiable cases, even if it is possible at all. – Tsuyoshi Ito Jun 9 '11 at 12:46
  • SO provides an imperfect process that hopefully will get more reliable from greater participation, just like Wikipedia. It is an interrogative process that is fundamentally incremental in reaching a state of accuracy. I agree with @Ito it would be unnecessarily limiting to, especially for nonnative English speakers, if we have strict standards of proof. As long as we have motivated, regular users (hey, like us) JLU should grow to be a source of quality information. – crunchyt Jun 22 '11 at 6:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .