Blog Post

6 years ago

Rating System Improvements (Again)

◊ Posted by A β Pseudolonewolf
Categories: Features The SiteUFELTA
A while ago, I talked about making changes to the UFELTA system... but I haven't done yet because I've been busy with other things.
It's something that I want to have done before I make this version of the site properly public though.

I'm not satisfied with the system as it is. I mean, I like how it works, but the letters themselves are a problem. I usually don't know what to rate posts because none of the letters seem appropriate.

So I want to keep things generally the same, but to change only the letters and their meanings.

I'll repeat, since people in the chatroom and previous threads and such about this seemed to misinterpret for some reason, thinking that I wanted to add letters to the current ones, or use a different system entirely:

I only want to change the letters. So rather than being 'UFELTA', it could be 'ABOTS', or anything... With proper meanings for each letter, of course.

I don't yet know what the letters could be. I want them to be things that could apply easily to many posts, rather than being narrow and easily misused. Ideally, I'd want to find the criteria that people assess posts on in their minds anyway, so then they can use the rating system to show what their mind is doing without struggling to fit round pegs into square holes, or something like that.

I have some ideas at the moment, but all they are are ideas. I can't stress that enough. What I present here isn't a 'right, these are the new ones' sort of thing. Instead, it's 'here are some I came up with, but do you have any ideas? Then we can use find the most popular criteria through brainstorming'.

I repeat, since again, this seemed to be misunderstood before: I'm not presenting a set of decisions; only a set of vague, preliminary possibilities.

If people don't like them, then they can change. That's why I'm writing this rather than just implementing them.

Anyway, here are some letter ideas, not to add, but to make a whole new set, starting with a blank slate:

E - 'Emotionally stirring'.

I mentioned this one in the chatroom, and it seemed to be less clear to people than I though it would be.
Basically, we are emotional creatures, not robots, and we rate posts based on how we feel about them; we are not 'grading essays' as teachers or anything.
This rating would be a way of telling the poster 'your post made me experience positive emotions' - like posts that make you smile, which are 'touching' in some way, which amuse you, make you laugh; any positive emotional response - or 'your post made me experience negative emotions' - things like anger, frustration, fear, feeling upset, etc.

This really shouldn't take a whole lot of thought, but the exchange in the chatroom showed me how it can be overcomplicated.
Ratings should be a way of essentially telling the poster something without having to use words,
Keeping that in mind, this letter would be a way of saying things like 'your post made me smile/laugh/happy inside' or 'your post really frustrated me or made me mad or upset'.
Some posts may make you annoyed or happy, but you want to keep that to yourself. In that case, don't give a rating. It really is that simple.

This is obviously a very subjective thing, but a very valid one. You may write a very good post, which nevertheless angers people due to the views expressed in it. They'd be angered whether or not there was a rating for it; the rating would just allow us to see how people responded rather than not knowing. Or something.

I - 'Interesting'

This could be used instead of the 'thought-provoking' thing currently, with elements of 'useful' or 'engaging' too.
It's easier to use in more situations, and it's a word that people would naturally use to describe posts anyway.
Posts would be marked as 'interesting' if they were interesting to you. That's really all there is to it. It's just a way of compressing the message 'I found your post interesting' into a single click of a button.

The negative version could be used for posts that might now be 'Thoughtless' or 'Useless'; basically spam that doesn't contribute in any way.

A or T or G? 'Appreciate' or 'Thanks' or 'Gratitude'

This would be sort of like the basic Like/Dislike thing common to other sites and things. I don't know what letter to use though because I really want it to mean 'appreciate' rather than 'like', but A might be used for Agree/Disagree, if that continues to exist.

I might 'appreciate' bug reports even if I don't exactly 'like' them, because they make me aware of things, but also give me more work. I may not be 'emotionally stirred' by them, so that wouldn't fit; if anything I might be mildly annoyed, but marking them E- would be horrible and stupid.

Similarly, you might 'appreciate' receiving advice or criticism even if you don't 'like' it. It may even 'emotionally stir' you in a negative way, meaning you could give a E- and G+ (or whatever letter this may use).

It could also be used for what 'U' was meant for in the current system, to say 'thanks for solving my problem'.

Importantly, it would mean 'I personally appreciate this' rather than 'in an objective sense, this is a good post, and it may help someone'.

A negative for this could be given for things like unwanted advice, etc, but that might just be 'negatively emotionally stirring', so I don't know... Maybe the negative version of this letter could have a totally unrelated meaning, rather than being the 'opposite' of the positive.

L or F or P - Language/Length/Form/Presentation

These would all apply to the way the post is written rather than its content as such. Mainly they'd be used negatively, for posts with bad language, or which were tediously long, too brief, hard to understand, full of unnecessary fluff, etc. (A post like THIS one in the forum would certainly deserve a negative in this category!)

Again, though, the positive is difficult; it's harder to rate a post for 'good form' than it is for 'bad form'... So what I could do instead is see the different positives and negatives for each letter as unrelated 'slots'... So that, say, L+ could mean 'appreciate' while L- could mean 'bad form'...

But that's probably a bad idea.

A for Agree/Disagree

I may or may not keep this... It gives a way to show support, but the other ratings might be good enough for that anyway, so it may be unnecessary. I don't know. Not including it would mean I could use the A for something else.

Those are some of my ideas, and I want to stress that they are just ideas that you could pick apart, expand upon, dismiss, improve upon, or you could just come up with entirely new suggestions of your own. They are certainly not intentions or decisions.

So, that's that. Congratulations if you read this far. It's irritating writing about this at such length, since I've been getting rather annoyed at long posts lately...

Something that I really want to stress is that this rating system is less about 'this post is...' and more about 'I feel'...
So rather than things like 'I will rate this post up because it is a good post that will be useful to people', instead ratings should be more personal, like 'I will rate this up because I personally appreciate it; it helped /me/'.
You're not grading essays robotically. You're expressing your feelings towards the post and the person that posted it.

(I'm also tempted to add a 'Pretentious' rating which would be negative, and given to all the young people who put on an air of Being Intelligent And Deep And Philosophical without any actual depth to what they say... o_O)

on 4 Roots



Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Maybe the appreciate thing doesn't have to have an opposite at all. It could be another kind of button, when you click it it means "I appreciate this" (or whatever it will be called) and not clicking it would mean that you feel neutral about it. Similar to facebook's "Like", I guess.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Abbx 22 Pakistan SanguineMelancholic ENFJ 3w4 573C
I'm not going to blab on, but would just state my opinions of what Pseudo should do in place of the UFELTA system.

I (Interesting)
P (Presentation)
A (Attitude { Friendly/ Hostile/ Fun/ Boring})
G (Gratitude)
E (Emotionally-Stirring)

One thing I rather dislike is the Agree/ Disagree option. People should be open to others' opinions and should accept them as having the right to voice their own, rather than putting him/her down by disagreeing with him, which seems harsh.

Also, I also dislike the misuse of the UFELTA, as people rate disagreeable or hostile posts with a negative in all the categories, e.g. in 'L' or 'E.'

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Wait a second. You suggested a rating system that lets me potentially rate a post as uninteresting, poorly presented, hostile/boring, I didn't like it (-gratitude), and made me angry (-emotionally-stirring), but you don't like the idea of agree/disagree because it seems harsh?!

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Abbx 22 Pakistan SanguineMelancholic ENFJ 3w4 573C
I just feel that people should accept that the other person has a right to voice his/her own opinion, be it as ridiculous as possible, without a long list of disagreements to make it seem as if that person is wrong.
Kind of like what happened with Onisuzume.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
So people are allowed to express their opinion about anything, but others aren't allowed to express their opinion that the person is wrong?

Agree/Disagree ratings have many uses. One of them that would be useful to me would be to see which site or game suggestions have the most support, so I'll know what to add to please the most people.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Abbx 22 Pakistan SanguineMelancholic ENFJ 3w4 573C
Yes, that makes sense.

Still, what would better describe this, is that some people sometimes disagree, and then offer no explanation. It seems upsetting to have something rejected after putting a lot of thought into writing it.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Onisuzume just made some objectively bad posts. It happens. Any rating system would have savaged those comments, agree/disagree option or not.

Regardless of this though, Onisuzume still has way more Agree ratings than Disagree ratings, and has pretty much the same Agree/Disagree ratio that I do. And I'm not really worried about my own ratio, because agree/disagree isn't a measure of post quality, and it definitely doesn't mean someone is right or wrong. I would definitely prefer to be marked 'disagree' than to be marked 'useless' 'unfunny' 'boring' 'poor grammar' or 'thoughtless'.

I believe that people benefit from exposure to different opinions and ideas, even if they disagree with them. To disagree with my opinion, you still had to read it. You were still exposed to it. That's fine with me, I only ask for you to hear me out. Opinions change. It might be that in a few months time, you could reread that same opinion, and not really disagree with it anymore. I could even reread my own post in a few months time and not be in favor of the opinions expressed.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Abbx 22 Pakistan SanguineMelancholic ENFJ 3w4 573C
Yes, that makes sense.

Still, what would better describe this, is that some people sometimes disagree, and then offer no explanation. It seems upsetting to have something rejected after putting a lot of thought into writing it.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
Orcris 20 United States MelancholicCholeric INTP 5w6 71C
Maybe you don't like people disagreeing with you; I don't know, but I think that most people seem to accept that people will disagree with them. Like A β Pseudolonewolf said, people shouldn't be forced to not say that they disagree with them.

Most people don't put disagrees down because they think that someone shouldn't voice their opinion. Disagrees just show that they disagree with that opinion, not that it shouldn't be posted. I'm sorry, but just because I don't like someone's opinion doesn't mean that it shouldn't be voiced.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
(Hmm. I clicked your blog post while in the middle of writing my post to copy some text, but discovered that it linked me to this page I was already in instead, making me lose my progress. Unpleasant.)

Those ideas seem to be quite good, rating things based on subjective feelings rather than objective quality is indeed easier. I think.

Emotionally Stirring and Interesting seem like valid criteria. However, I have some doubts about Appreciate/Thanks/Gratitude (which I will refer to as "A"). Firstly, I think it might be a bit too subjective. The posts one appreciates the most are posts written directly to them, as an answer to a question, for example. However, such posts would likely end up getting only one positive A, because other's can't really appreciate a post written for someone else. Of course, posts written to everyone in general could still get high A ratings, but I feel that such posts might not deserve to get a lot higher A ratings than direct answers. I know that while I might appreciate a general post enought to give it a +A, I would appreciate a personal post a lot more, and would like to see a post like that getting more than just one positive A rating.

Another thing is that while E and A are meant to be completely separate, people would still probably rate them very similarly most of the time. After all, people usually appreciate things that make them feel happy, and unappreciate things that make them feel angry.

So, I think the A rating should perhaps be changed to something you can give to posts written for someone else, and something a bit more clearly separate from E. Hmm... "Helpful" doesn't quite capture the intended purpose, since you can appreciate something which is not helpful and unappreciate something which is helpful. Alas, I am out of ideas.

The Language/Length/Form/Presentation rating... A while ago, M β Sunflower suggested in the chat that the current system's Language and Entertaining ratings could be merged. I think that might work here, having a rating like "Readability", which would say how much you enjoyed reading the comment, not the contents itself. A post with clever wordplay and no tedious filler could recieve a +R, because it was pleasant and engaging to read. A post written in bad grammar or a very boring post would recieve a -R.

About the Pretentiousness rating, perhaps there could be a "Humility" criteria? Though that would probably encourage fake self-deprecation as an attempt to fish for positive Humility.

Additionally, I think it might be a good idea not to wait too long before implementing a new system, since we should probably have some time to test it properly before releasing this new site to the public, but pushing back the grand opening of this site in order to test a new system might not be a good thing. After all, you probably want to make this new site open to everyone as soon as possible, no?

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11
At the moment, I'm not really fond of seeing posts with loads of +U ratings from people, because it seems like piling them on like that makes them sort of meaningless, like a green U on a post is expected rather than rare. So I actually like the idea of more subjective ratings.
Though even if something is aimed at a specific person, it may benefit others. If someone posts a solution to a programming problem, for example, then the person with the problem might Appreciate it, but so might others who had the same problem that learned from that post. I find this to be far more meaningful than loads of people coming along and basically saying 'this post seems objectively useful', since rather than assessing posts, 'grading essays', I want the ratings to reflect the post's 'affect on the community' in an emotional, subjective sense.

I'd see the Appreciate thing is the basic 'thanks' or 'I like this' which would be given out regularly, probably, for all manner of posts, but often it'd be given out in an emotionally neutral way. I mean, I would like to be able to Appreciate many posts that I get for the sake of saying 'thanks for taking the time to write that, your feedback has not been ignored', but they don't really make me positively emotional...
I'd mean for the emotional thing to be used for times of noticeable emotion; like a genuine feeling of being emotionally touched rather than a faint smile or something, and certainly not for a robotic 'this post seems to be meant to be amusing, so even though I didn't laugh, I'll mark it as positively emotionally stirring'.

It would also be interesting to me to see which posts might be Appreciated but given negative Emotional ratings by the same person.

'Readability' is sort of what I had in mind with the Language/Form thing; I should add the R as an alternate possibility for the letter. I don't know how many posts would even have clever wordplay though, so it seems like the positive might be infrequently used... I need to think more about that.
Readability seems like a better word in some ways, like being able to more easily encompass things like 'too long so it's hard to get through', but would a too-short post be 'unreadable'?

With my ideas though, 'Entertaining' - which encompasses 'Engaging', 'Amusing', 'Boring' and such, would be spread out over 'emotionally stirring' (funny things), 'language/form' (engaging/boring), or just 'appreciate'...

I want to add this new system 'within a few days', ideally. Though obviously I'll need to have my ideas worked out first.

Notice: Undefined index: FID in /home4/yalort/public_html/charcoal/code/common.php on line 11

I have a feeling that Emotionally Stirring will have to be reworded to something more intuitive, because if even the people who you were explaining it to in chat were getting confused, then it's going to be horribly misused.

But you're asking for our suggestions for the rating systems we would use, so I'll give that a shot. The ideal rating system would be one where all ratings are used roughly equally - if a rating isn't getting used much, it's either too specific or obsolete, and if it's getting used all the time, it's probably too general to be used as a guideline.



(T) THANKS // NO THANKS - Self explainatory. If I liked the post, if I think it made the site a better place, or made my life a bit better for reading it, I say Thanks. If I think the post was useless, or stupid, or spam, or wasted precious moments of my life to read, I say No Thanks. You're not expected to jump through hoops to get these ratings, or have to be stingy in giving them out - for example, if people respond to my comments, or post in my thread, I can Thank them just for that.

(H) HUMOROUS // HOSTILE - Was the post witty? Did you smile while reading it, or even actually laugh out loud? Rate it Humorous. But if the post is unfairly critical towards other people, maybe the joke falls flat or is offensive, or even if the tone just makes you feel uncomfortable, rate it Hostile. (These ratings might not seem like exact opposites, but I didn't think there was much worth in a +Friendly rating, and didn't see -Entertaining ratings given out much, so I merged the two to create something that's meaningful both ways).

(B) BALANCED // BIASED - Hey, we're in an argument! But there's a way to show respect for your opponents' position here. If they consider the other side of the argument, if they admit shortcomings in their postition, if they substantiate their claims, or if they respond properly to the arguments you've raised, you rate them Balanced. But if I'm just ranting, or not responding to contrary evidence, or have selective hearing, I'm biased. For suggestions, if my suggestion considers its shortcomings or alternatives, I'm Balanced, if I didn't really think things through ("Let me use images and .gifs in my sig!"), I'm biased. Criticism with alternatives is balanced ("The new site is bad because of X and X, but could be improved with Y!"), otherwise it is biased ("The new site sucks!").

(A) AGREE // DISAGREE - As before.

(M) MERIT // MODERATION (*Special rules apply to this one) - Some posts here seriously go beyond the call of duty in the amount of effort people put into them, and in the worth they bring to the site. Those posts that are long, well reasoned and written, and would have gotten cyans or greens on the previous site are awarded with Merit. On the other hand... If a post breaks the rules in such a way that it needs to be flagged for moderator attention, you mark it for Moderation.

I'm a big fan of the "Revert, Block, and Ignore" that was used and promoted by some users on the old site. Giving Red post and personal ratings on the old site didn't seem to do much when trolls were liking the attention they got from it, with members piling on 20+ red ratings on someone writing a troll post, while a someone writing a cyan-level post might only get 3-5 ratings. (M) is a HIDDEN rating. You don't see if anyone's given you Merit, but you don't see if anyone's marked you for Moderation either. And if you mark someone for Moderation, you can't give them any other ratings - that defeats the purpose of this. We don't want to give troll posts visibility through negative ratings. If I rate a post -No Thanks and then mark it -Moderation, my -No Thanks rating disappears and I can't rate this post -No Thanks anymore (unless I remove my Moderation rating). Of course, all posts with a Moderation flag are added to a Mods-only shortlist for ease of moderation.