Amanda, has just been employed as a “process executive” through a third- party firm for a social media platform, has been one of many content moderators given the heavy task of flagging posts containing hate speech, violence, misinformation. Amanda’s job is by no means luxurious, it has taken a heavy toll on her mental health since she started. She finds a few racist jokes, some neo-Nazi posts, she even came across a video of a man being beaten to death. Amanda spends less than 30 seconds on each item she finds and will do this up to 400 times a day.

One day during her shift, Amanda finds a post in her queue of someone saying that “autistic people should be sterilized”. Amanda decides to flag the post because she has deemed it to be offensive and to be a form of hate speech. 

 Unfortunately, her company’s policy is such that autism is not considered to be a protected characteristic in the same way that gender and race are so technically the post does not violate policy. Amanda is reprimanded for her mistake and this is being counted against her “accuracy score”. Amanda is new and has made previous errors during her shift. 

Amanda decides to appeal the decision she made on the autism post, but she knows that it will most likely be overturned. Amanda is one of the many who’s job it is to purge the site of its most heinous content. Her job puts her in the face of traumatizing material on a daily basis and is expected to conduct herself with as much accuracy as possible. 

Evidently, this is not possible, and she will most likely make similar mistakes again. Her job as a fact-checker on a platform with endless forms of violence will continue to take a toll on her. Perhaps this is where AI can stand to relinquish the burden from human moderators.