davidw wrote:... so many Quest reviewers rate all games highly, making it difficult to tell which is a good game and which is a bad one.
privateer wrote:
Since all games are checked by Mods, perhaps it would not be a significant increase in workload for them to categorise non-Sandpit games into 2 tiers, with Tier 1 games representing those which have (subjective issues of taste, etc not withstanding) obviously had lots of time and care invested in them, as opposed to those that have obviously not (but which were still not nonsensical enough to be condemned to the Sandpit.)
Pertex wrote:"privateer"
Since all games are checked by Mods, perhaps it would not be a significant increase in workload for them to categorise non-Sandpit games into 2 tiers, with Tier 1 games representing those which have (subjective issues of taste, etc not withstanding) obviously had lots of time and care invested in them, as opposed to those that have obviously not (but which were still not nonsensical enough to be condemned to the Sandpit.)
Alex is doing this most of the time but I am sure that he is not playing the games. I have checked some games , too, but I don't have time to check the complete game, just checking the first rooms/pages. It WOULD be a significant increase in workload to test the games more detailed
george wrote:I'm not certain but I think iFDB has a feature where if enough people rate a review as unhelpful the review disappears (or goes to the bottom or something).
jaynabonne wrote:Not to be a fly in the soup, and perhaps it's my programmer sort of mind that likes well-defined things, but trying to crack down on "reviews we don't like" seems like a task that is either going to be tough to do or, worse, potentially stifling.
As an example, let's say we have a system where we can flag reviews that are "not helpful" (or perhaps more in Facebook fashion, we can upvote reviews - a sort of meta-review). Now let's say there is someone like the one who has been mentioned. I don't know what her reviews are like - can you tell that her reviews are not sincere by reading them? Short of some sort of "smoking gun" message like what you have received, how much is it possible to differentiate inaccurate reviews of games vs those which simply differ from our opinion? I have played (or tried to play) some games by a highly-ranked author on this site, and I eventually gave up. And looking through the walkthroughs, I could see why, as the commands needed to be input were often nonsense. (Don't get me started!) And yet there were clearly people who played and succeeded at the games. So I just took that as a hint that we were on different wavelengths and left it at that.
How do we draw the line? Let's say this claire person posts a review which, superficially, appears to be legitimate. Would it make sense to downvote such a person based on what you have seen so far instead of on the actual review itself? What is the criteria used to downvote someone?
I think something more along the lines of a reputation system would work better, where people earn rep through good reviews, perhaps voted by others with good rep. A bit of a bootstrap, but it would get rolling eventually.
I don't know if the above makes sense. I just would hate to turn things into a witch hunt, where we're looking to excise "bad" reviews (as defined by someone) from the site. Given the variability of human opinion - and the ability, it seems, for people to play and enjoy just about anything, or at least to want to offer something positive, I think it's a slippery slope to try to establish norms where none may reasonably exist. Not without removing the ratings from "the common folk" and having some sort of judging panel.
(A note: a lot of my experience with reviewing things comes from my time spent on writing review sites. It seems to be a pervasive problem that you end up with glowing, positive reviews for mediocre product. Whether that is people wanting to not hurt people's feelings, the desire to focus on the positive, etc. I'm not sure. But it's not necessarily people being deceitful or trying to curry favor. In the end, you end up making up your own mind anyway.)
jaynabonne wrote:A couple more suggestions:
1) We can't necessarily keep people from posting glowing reviews for what's (arguably) barely passible games. The counter for that is to post a review of your own, perhaps with more critical detail. Unfortunately, it could get swamped by a torrent of positive reviews. One solution to that would be an Amazon-type rating display - how many 5 stars, how many 4 star, etc. with links to the those. I know I tend to skip the 5-star reviews and go down to the 1-3's to see where the problems are with a product. And then judge from there based on how reliable those seem.
2) This is even more work, but have people not only rank a game but actually rank various technical elements of a game: writing (including grammar), ease of command input, etc. That doesn't eliminate the subjective element, but that would provide more axes, and those specific elements could then be averaged out to provide a more broad summary.
None of the above eliminate "bad" reviews or unbalanced ratings. I don't think there's a way to do that, and tryiing to attack thing from that point of view is doomed to failure (IMO). But making the process more visible by allowing people to see more than just a single star rating might allow people to more easily weed through the fluff to get a more truthful impression.
davidw wrote:I'm not sure the reputation system would work very well for pretty much the same problem as we have with the current system: namely, you'd just have people voting for each other to better their own reputation. ("Hey, mate! Vote me up and tell all your friends to vote me up and I'll vote you up in return.")
For myself, I tend to ignore the ratings system because everyone rates games differently. I've known people who will rate a game 5 out of 5 even if it was so bad they quit a minute into it. I've also known other people who will really enjoy a game, but give it a poor rating because there was the occasional typo. I usually look at the reviews themselves to see if a game is worth playing, and by reviews I mean actual proper in-depth reviews written by people who know what they're talking about. "Dis is great!" isn't a review, nor is "This sucks!" If the review isn't at least a couple of paragraphs in length, I skip over it.
jaynabonne wrote:I see what you're talking about, but I don't see a good solution - at least not one that doesn't involve a lot of work for a number of people.
Looking at the reviews, it seems typical of what I see on other sites where "common man" reviews are given. I don't know what the motivation for these reviewers is. A cynical view could be that they are deliberately conspiring to intentionally give inflated reviews with the purpose of artificially pushing their friends' games up in the rankings. Another view (which I have seen more often) is that they are a circle of friends, and they all like each other, and they don't necessarily care about the larger scope or anyone outside their circle, and they give each other max reviews because they would never actually be harsh to friends (at least not in public), and they don't really play IF or have a gauge, and they're off in their own little world anyway.
In the end, I suppose it doesn't matter, as the result is the same - ratings are subjective and all over the map. It's frustrating, but short of eliminating people being able to review or having a "reviewer of reviews" with the power to delete (like Amazon, the bastards), there is not much you can do. To police it is to enforce a standard, and that has to come from somewhere. Which means someone has to decide.
One alternative is to have a panel of judges who alone review and rate. That keeps the reviews more consistent, but I think you'd need a broad selection of judges to keep it from being too one-sided - like you see on Rotten Tomatoes.
Another alternative is to have both, with people being able to review but not rate, and with Quest admins alone rating. Similar to the first option, but allows some "common" voice.
A third approach is to diminish the importance of ratings. Let people use them for their own purposes, but they don't factor into visibility or prominence on the site.
And the last I can think of at the moment is to get Quest games out there where the more valued reviewers can get hold of them. That's what was happening when Alex had Quest games automatically publish to IMDB. I think getting quality Quest games to IMDB is a good thing - but not automatically (just my personal preference as an author who wishes control over the crap he releases). Making it easier for authors to do that (some sort of link or button in the tool?) would help those who feel rightly proud of their games getting them out in the world. Would someone like the author of what you posted do that? I don't know. It depends on what their motivation is. If they're just posting games for their own circle of friends, then they won't. If there is text along with the button/link that basically says, "By doing this, you will be exposing your game to some of the toughest IF critics in the world who will rip your game to shreds. It's not pretty! Don't do it!" then one would hope they wouldn't.
In the meantime, the basic question is: can a high rating on the Quest actually mean anything? It seems not, just as it doesn't on any other site I have been on where all manners of people are able to freely express their point of view in an entirely subjective way, where you don't have large groups of reviews to statistically balance it out. Of course, if I went to that game and saw five-star reviews with comments like that, then I'd discount the reviews as well as the rating. It comes back to what you were saying about visibility, and perhaps that's the way to attack the problem (to the extent there is one? I'm still not sure, from a big picture point of view. If I saw that game, or others like it, on the textadventures home page to the detriment of quality games, then I'd worry.)
Sorry for making this so long.
jaynabonne wrote:Ok, one more thought. (Sorry. )
The reason there are so many "crap" Quest games is Quest's strength: it is accessible and usable by anyone. You don't have to be a writer-type (e.g. who can work out the English-like Inform script) or a programmer. It can be used by kids and adults alike, by amateurs and pros. I think that's a great thing, but then you end up with school kids alongside seasoned IF authors. There is no way to compare, and we really shouldn't. (One tack we took on SM was to split pieces up into Amateur and Advanced.)
What we really need is not so much to dis the honest efforts of people with less skill (to be honest, despite the poor spelling and grammar and over-usage of "colloquial" language, that game you highlighted had more logic in place than some Gamebook games I've seen on Quest. It looked like there was at least a decent amount of effort put in) as we need to promote what is good, to let people see what is possible with Quest in the right hands.
davidw wrote:Liam315:
I disagree. An overhaul of the caterigorization system might help, but it won't have much effect in the long run if people are still writing amazing reviews and giving great ratings to terrible games.
m4u wrote:And, whats is wrong with a rate based on plays and downloads? It seems like a good metric to me.
m4u wrote:And, whats is wrong with a rate based on plays and downloads? It seems like a good metric to me. Or at least create a section with the most playable and downloaded. I have seen this system somewhere in pages about games.
HegemonKhan wrote:
gamefaqs.com (just one site of many that) has a good system of reviews and user comments, for a *quick* example (using parasite eve game):
1. http://www.gamefaqs.com/ps/198265-parasite-eve/reviews
2. http://www.gamefaqs.com/boards/198265-p ... e/59649104
3. http://www.gamefaqs.com/boards/198265-p ... e/60548636
I'd suggest that would be a job for the moderators.
*Also for both #1 and #2 below, anyone (any user and~or mod) can submit reviews too, if they're good reviews, then they'll be allowed~included with the other (good~professional~quality) reviews.
1. the site's mods will have to check the reviews to see if they're good (actual reviews: a detailed and accurate summary of the game in relation to the various categories for their rating system of that game; ie they need to show that they actually played the game enough to have the knowledge to truly review and rate it), and deleting~removing the bad~fake "reviews".
m4u wrote:It seems a 10 points systems is better because I feel the 5 too close to the 1 or 3
Pertex wrote:So is there a solution? Sorry, but if I want to rate a game, I don't want to write a novel. Of how many characters are you thinking? 100,200, 500? Have a look at the top game The Mansion II ( http://textadventures.co.uk/games/view/ ... mansion-ii ). Which of the reviews there would be "valid"? 10%? In my opinion only 1!
Pertex wrote:So is there a solution? Sorry, but if I want to rate a game, I don't want to write a novel. Of how many characters are you thinking? 100,200, 500? Have a look at the top game The Mansion II ( http://textadventures.co.uk/games/view/ ... mansion-ii ). Which of the reviews there would be "valid"? 10%? In my opinion only 1!
jaynabonne wrote:Regarding "thoughtlessness", I think we had had so many thoughts beforehand that there wasn't much left to add.
jaynabonne wrote:I agree the limit would have to be set with some thought. That paragraph above is 87 words - hardly a novel. Perhaps 50 would be a better amount, as was proposed. It does require some thought and effort to come up with words, but I personally don't have a problem with people "earning" their rating by putting a few minutes effort in.
jaynabonne wrote:
I agree the limit would have to be set with some thought. That paragraph above is 87 words - hardly a novel. Perhaps 50 would be a better amount, as was proposed. It does require some thought and effort to come up with words, but I personally don't have a problem with people "earning" their rating by putting a few minutes effort in.
HegemonKhan wrote:I've posted this somewhere before, but here it is again, my '2 cents' comments:
gamefaqs.com has done this very well, it's a good system
best would be a U.S.~Britain legislation system:
U.S. <--> Britain
Actual REVIEWS:
Professional Quality Panel of Reviewers (moderators and~or certain users) and their Ratings: Senate (100: 50 states x 2 per state) <--> House of Lords
and
Public Panel of Reviewers (the 'mob~people', aka the rest of the users): Congress (~ around 420) <--> House of Commons
---------
AND
a like~dislike rating system (it will be abused, but maybe we can hope the sheer quantity~statistics will balance out the abuse to it... lol)
AND
(as has already been expressed by us and being worked on by Alex and co., a better categorization system for finding specific types of games)
what MOST+FIRST gets my attention is simply the NAME of the game, and its genre~category and~or description:
is this even a type of game that interests me?
then, I try to figure out of it's a good quality game or not... lol
------------------------
though, this is a lot more work, as it's a more comprehensive system, which Alex may not have the manpower~mods or time for implementing and~or managing such a big system on his website.
jaynabonne wrote:Here's a discussion about a weighting system based on both score and quantity. I have no idea how it works or if it's good, but it looks interesting.
http://math.stackexchange.com/questions ... view-score
davidw wrote:The problem is people writing dishonest reviews. If a game is great, say it's great. If it's bad, say it's bad. But don't give 5/5 reviews for barely playable messes.
Most of the reviews and ratings on this site are meaningless because there's no correlation between a 5/5 review and a great game.
Silver wrote:The problem is that I might feel a game is good and worthy of four stars. But if I award it four stars I'm saying it's not as good as games that have been five starred to death but are inferior. So I'm forced to give five. The other option is to go to all the highly rated games I feel have been unfairly placed into the league of excellence and down vote them. But then this puts noses out of joint and upsets people. It's a complete mess and life's too short.
HegemonKhan wrote:some people write what looks like really detailed and well-done reviews... but they've never played the game or just played its very beginning... and until you play the game yourself... you would never realize how totally BOGUS that review that you've read, was... the review could have absolutely nothing to do with the game at all, and you'd never know it, unless you actually decided to try the game out yourself (which you wouldn't do if that BOGUS review was a negative review...)
davidw wrote:I never really understand why people have to written a great game themselves in order to ‘qualify’ to review other games. Most film critics have never directed or acted in a film, yet you don’t get people saying they're not qualified to review a film due to their lack of directing / acting experience.
The simple fact is that sometimes games just suck. You don’t need to be a writer yourself to realise that.
Silver wrote:They'd probably sit there and do the maths though.
Why has my game got seven five star reviews and still isn't on the front page...
Cryophile wrote:I've also seen a few decent reviews end in "but you gave my game one star, so I am giving your game one star" so there's that
Review by JCB_diggerbrown
07 Jan 2015
it is impossible to progress in this game in a group activity of 15 mins (year 7 class of 30)
Silver wrote:Sounds like it came from a teacher and not a child. Or someone on a wind-up.
The Pixie wrote:If I had responsibility for placing games in catagories, I would definitely do it with a different account.
XanMag wrote:I went through and placed a few games fairly in proper categories. My comments are almost entirely positive or at least positive criticism (feel free to view them - tell me if they are not fair).
OurJud wrote:"The Pixie"
If I had responsibility for placing games in catagories, I would definitely do it with a different account.
I'm a bit confused by this. Are you both referring to the moderator rights that Alex is currently granting, for anyone willing to help categorise the backlog of uncategorised games?
So the author can see who decided what category their games go in?XanMag wrote:I went through and placed a few games fairly in proper categories. My comments are almost entirely positive or at least positive criticism (feel free to view them - tell me if they are not fair).
And Alex didn't say I had to add reviews for the games, too.
XanMag wrote:So much for trying to be helpful/thorough.