Allow me to preface this with a bit of a disclaimer. I do not want to sound ungrateful or upset with the efforts of the SE team to support and promote this site. I think it's fantastic that SE is willing to create little contests and promotions and such for its sites, and I'm genuinely appreciative of the support they provide to all of the sites in the network. I very much enjoy Android Enthusiasts, and I don't see that changing any time soon.
Now, having said that, I feel the need to voice a bit of concern over the specifics of some of our site promotions. This was motivated initially by what I felt was an almost immediate flood of low-quality questions in audio for the current contest, so I decided to go back and do a little bit of datamining on the previous ones. What concerns me is that although we may be getting attention from the contests, we are:
- Generally not retaining the users we attract
- Causing a larger-than-usual influx of poor questions
- (In my view) Run the risk of possibly disillusioning some of the more regular and loyal members
Now that we've had a few goes at some contests, I think we can start to analyze the results a bit. I'm going to go through the past contests in what should essentially be chronological order with a bit of data I collected by simply scanning the relevant tags and user profiles:
This contest in some ways worked and in some ways felt like it kind of didn't. On the one hand, it brought attention to a lot of older unanswered questions, which is generally great. On the other hand, one user was so annoyed by what he perceived to be a flood of non-answers grabbing for loot, that he actually removed the tag from his own question so that it would not be part of the contest. There were certainly lots of cases of non-answers or half-answers attempting to get in on the contest, which provided a lot of clutter that people ended up needing to downvote since the rules dictated that anything with a score of zero was an eligible entry. In my opinion, this doesn't really encourage quality participation, but that's probably neither here nor there.
The winners of the contest also show some concerning stats. Of the 5 winners:
- 2 have not visited the site since the end of the contest
- 1 has visited but has made no other contributions to the site
- Additionally, this user has only one post total (mind you, this is the post which won the contest for them) and it currently sits at a net score of -1 (!!)
ICS week was the first tag-themed week. It was presented to the community on Meta before starting and received generally positive responses. Overall the week played out relatively well, but still seemed to encourage one-off posts to win sweet loot:
- 18 questions asked
- 1 of the 18 was closed
- 5 of the users (I count 14 total unique users, so ~35%) have no other activity on their profiles whatsoever
- The winning question ultimately had the 4.0-ice-cream-sandwich removed (!!) - It was quite arguably never about ICS to begin with.
Tablet week started immediately after ICS week, and was not presented to the userbase as a whole beforehand. It was also not one of the tags suggested in the ICS week's answers. I found this to be a bit unusual as tablet is a pretty broad tag, but it didn't seem like an unreasonable choice. This perhaps lent to the fact that there were very few entries (it's a bit of a low-traffic tag).
- 6 questions were asked (none closed)
- 1 user has no other posts on the site
I have to be blunt here: this was a terrible tag choice, in my opinion. applications is absolutely notorious for being a dumping ground for questions where the author can't come up with any other relevant tags. It's vague, it's broad, and it attracts junk and (I think) should be actively discouraged whenever possible. I firmly believe that if this tag had been pitched to the community via Meta it would have been met with substantial resistance. The stats for this week were:
- 18 questions asked
- 3 closed
- 7 users (out of 17) have no other activity outside of the contest or the contest timeframe
Similar to tablet week, this seems like an unusual tag choice to me. It is very broad, and it is also fairly low traffic (111 posts as of this writing, which includes the sudden influx of contest questions). Additionally, this week is based purely on views, which seems to further fly in the face of promoting high-quality contributions. Realistically, a winning entry needs only to be acceptable with effective link distribution. Audio week has also not fared so well, in my opinion, as it currently has:
- 13 questions asked
- 5 of the 13 (this is ~38%) have been closed, one has 3 close votes
- 7 of the users have (so far) no other activity on the site
I don't claim to have any solutions, but I do think that perhaps we should at least discuss the contests that have been run so far. As a community, I'm curious to know:
- How do we feel about these contests and this data? Are we concerned? Do we consider these contests successful?
- What other contest styles could we look at? Are there other options?
- Should we be concerned about the fact that established, loyal users have no benefit over one-off throwaway accounts in these contests (which, in many cases appear to never return)?
- Should the eligible posts be moderated more stringently or held to a higher standard than simply "Not a negative score and not closed"? Are view-based contests a good idea?