TRK

Our calculator, a mathematical provocation, helps show how underpriced data labeling or data training tasks are. This calculator focuses on a worker's time and payment towards entire workdays and a living wage, as opposed to pricing a bunch of tasks in aggregate. We aim for a living wage, and our living wage focuses on Washington State, where Amazon is headquartered.

TRK

Wage Calculator

    If you priced...
    10000 images labeled
    minimum
    maximum
    At $10000 per task
    minimum
    maximum
    At around 10000 per task
    minimum
    maximum
happy emojiunhappy emoji

TRK

Methodology

This project focuses on wage inequality, and we are aiming for a living wage. For this project, we read and reference the following listed materials, as well as interviewed fifteen different gig economy style workers who work across CrowdFlower, Mechanical Turk and Fiverr. For structuring the wage calculator, we looked at the largest demographics of Mechanical Turkers are workers, these workers were based in India and the United States. Because this project focuses on a living wage for all Turkers, we decided to focus on the US which has a higher minimum wage. Amazon is headquartered in Washington State, which has the highest minimum wage in the United States at $11USD per hour. The majority of Mechanical Turkers areestimated to be paid around $2 USD. As critical designers, this information adds a poetic provocation—shouldn't freelancers/gig workers of Amazon be paid fairly?

In a time where even Amazon US employees are underpaid, wage inequality has never been more important. Amazon, CrowdFlower, any company using freelancers or gig economy employees should pay their workers not just a minimum wage, but a living wage. Pay equity is important for a better future.

For this research, we looked at:

Wage Calculator
TRK

Process

We spoke to creators, Turkers, Fiverr workers, research labs that use Mechanical Turk, and AI artists to analyze and understand the product design of Mechanical Turk, and the labor injustice implications of the tool as well as worker and client interactions within the tool.

From this, we created the Wage Calculator. An intervention interface designed to price tasks more fairly. But how does a “fairer” task system operate? What are some constraints that are likely to be constant across location, needs, and workflows? How is work audited and paid for, and who ensures that workers are paid on time? How are disputes mediated? We outlined the following parameters:

  • time per task
  • max hours per workers
  • location
  • processing fees
  • project budget

We then selected about 0.04 for a task that is 4 seconds long to test the outcomes.

$0.04 at 4 seconds per task is 15 tasks per minute. 15 tasks per minute is 900 tasks per hour. 900 tasks per hour is a total, when tasks are priced at .04, is $36 per hour.

limitations

This equation doesn't take into account human variants like starting a new task, or the task even changing slightly. Training and labeling can be hard because nothing is ever completely uniform and the same, a take away both artists and Turkers revealed in our user interviews. Additionally, it's unsustainable to assume a human can work with the same rigor and quickness 1000 times in a row.

Adjusted calculation:

$0.04 task at 20 seconds which is 3 tasks per minute. 3 tasks per minute is 180 tasks per hour. 180 tasks per hour which is $7.2 per hour. However this assumes people pay per task, for example labeling one image versus labeling like 10 images. We noticed that in some tasks listed on Mechanical Turk, a single task could be priced at a few cents but include a bundle of multiple tasks, like labeling multiple images. The payment then didn’t reflect the longer time in regards to that bundled task.

How can we push people to price better? Does this tool do it?

For a living wage in Washington State, our target due rate is: $127.40 USD/day working at 3 tasks per minute.

To allow for client mistakes, worker breaks such as 5 mins at the top of every hour and a 45 minute lunch break, and human variants: 6.5 hours = 390 minutes = 1170 tasks

The calculation breakdown of $127.40 USD / 1170 tasks = $0.1089 USD / task. At a minimum, workers should be at around 11cents per task to just clear a living wage before taxes.

Wage Calculator