10

Shortly after posting Is it possible to reduce the number of close votes needed for certain types of close reasons?, I discovered that Stack Overflow and other SE sites are testing moving to a 3-vote threshold for closing and reopening instead of 5-vote.

Personally, this seems like generally the right thing to do at JLSE too (to reduce noise, improve site quality, and reduce maintenance stress). One of the SE community managers suggested we had a discussion about it, so here's a meta thread! Please leave comments, answers, and votes if you have any opinions whether we should lower our threshold to 3.

5
  • 1
    I feel like it's not that hard and doesn't take that long to reach the 5 vote consensus as there seem to be a fairly high # of people who vote. But I also think changing it is fine.
    – Ringil
    Dec 22, 2019 at 15:43
  • @Ringil Hmm, I regularly see things which should clearly be closed stay open for half a day or longer pretty often. Dec 22, 2019 at 16:23
  • 2
    Given the overall level of activity, is half a day or even a day that long?
    – Ringil
    Dec 22, 2019 at 17:04
  • @Ringil Depends what the outcome metrics are. UX-wise, as a frequent visitor personally, seeing the same bad question over and over every time I visit without being able to do anything about it is fairly psychologically unpleasant. For new visitors, I think seeing low-quality questions leaves a bad impression (how many page views does the JLSE homepage get a day? I could imagine thousands easily...). I think aiming for sub-1hr cleanup times would be a decent goal. Dec 22, 2019 at 18:31
  • 2
    Since we’ve been hoping for this for a while now, and pinging CMs in chat to implement it doesn’t seem to be working, I’m going to try this new status-review thingy to draw staff attention to the post. Hope it works out :-)
    – user1478
    May 17, 2020 at 9:52

3 Answers 3

3

A bit over a year since y'all escalated this to the Community Team — apologies for the delay there! But the good news is you now officially only need 3 close votes to close a question on Japanese Language SE!

After looking at the data, I think it's clear that there's no reason for y'all to have this set at 5. Thank you so much for bearing with us while we got to this request.

We'll check back in with you all in a few months to see how things are going — if things are going great, we may not post an update but please feel free to leave a comment if you want us to post something.

Here's some basic stats of how things look:

Over the last two years, you've had decent numbers when looking into what percentage of questions that received at least one flag or vote to close actually ended up getting closed. While there are some dips here and there (the one in early 2022 is particularly noticeable), things have been generally in the 60-90% range and, the numbers of posts this represents are relatively small.

Graph showing the percentage of handled tasks, as described in the paragraph above

Additionally, the graph below shows that the community is doing the bulk of the closing — which is a good thing!

Graph showing all closed posts, posts closed by the community, and posts closed by the moderation team. The proportion of the latter two is around 2:1

When working on this project, Catija had found two primary use cases for this change:

  1. When too many items are going unhandled (not unclosed, just unhandled)
  2. When the moderators are doing the bulk of the closing/reopening.

The second point there is moot, given the fact that the community appears to be doing about twice as much closing as the mods. But the first graph still shows that there's around ~20% of tasks aging out from the review queue on any given month. The hope is that changing this to three votes to close/reopen will make it so that first graph grows closer to 100%!

1
  • Great! By the way, I think one of the real benefits of this change is that it's much more pleasant for power users to not be stressed out by the front page being filled with low quality / off-topic questions all the time. We have a rather small number of people who actually do moderation tasks like this, so even though eventually the posts do get closed, the time-from-post-to-close is pretty high, and as a result, anyone who goes through the new posts often has to sift through a bunch of junk. With this change I think the site will be more pleasant for people who are heavy answerers. May 25 at 18:43
9

I think we should do it. Here's my suggestion:

  1. Ask SE to turn it on.
  2. See how it is.
  3. In three months, we should start another meta discussion and decide if it's been a success or a failure. If we don't like it, we can ask SE to change it back at that point.

In chat, Flaw brought up one possible problem:

We want a faster close>edit>reopen cycle and a lower vote threshold seems to help this. But we also want a robust close>stay-closed process and that needs a higher vote threshold.

I'm not convinced this really needs a higher vote threshold, though. I think that most of the time a post reaches three close or reopen votes, it ends up being closed or reopened. It just takes longer in the current system. Changing to three votes would streamline the process and help the site run a bit smoother.

It's possible that under the new system posts will end up getting closed or reopened when they shouldn't really, but we can use the existing tools we have to solve those problems as they come up, if they come up. My feeling is that once we’ve tried it out, though, we’ll likely want to keep it that way. And trying it is the best way to test whether those problems are actual or purely theoretical.

In the worst case, if we feel it's a major problem, we can revisit it before the three months are up. The reasons I want to set a three month schedule are these:

  • To give us enough time to see if any problems do come up.
  • To make sure we don't forget to talk about it on meta eventually and decide whether it's been successful or not.

I'd really like to hear more feedback from our community before we decide to ask SE to try it out, though. I hope everyone will leave comments, answers, or votes as Darius asked in his post, so we can get more of a community consensus before moving forward.

2

I'm not opposed to the idea of lowering the vote threshold, but I do want to see what the community thinks of the idea.

I'll play the devil's advocate so we have some other points of view to consider. These are not my personal stances, just issues that I think may arise. So just to throw some thoughts out there:

  • I think we have enough active users to complete the 5 vote process. It may take slightly longer than 3 votes but I think this gives us some breadth in what kinds of questions are acceptable. Some questions appear too low quality to some users but may appear perfectly fine and innocent to other users and lowering the vote threshold reduces this breadth that we have.

  • I think it reduces the level of detail of the information from close votes. If we get 5/5 votes we know that the question most probably deserved to be closed; and if we get 3/5 votes we know some users think it may have some problems, but not enough users feel the same way. Seeing the question closed at 3/3 removes this sense of information that we may be currently used to.

  • A larger threshold represents the opinions of a larger group. Having a smaller threshold concentrates the power of defining what is off topic to a smaller group of users. This could be beneficial or not.


That being said, I'm not opposed to lowering the vote threshold mainly because all these changes are reversible. Even if the vote threshold was lowered we still can vote to reopen questions so it's not a big problem even if questions get closed slightly over-zealously as long as we have requests to reopen them.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .