"Amazon Mechanical Turk (MTurk) is a marketplace for work that requires human intelligence. The MTurk service gives businesses access to a diverse, on-demand, scalable workforce and gives Workers a selection of thousands of tasks to complete whenever it's convenient.
MTurk is based on the idea that there are still many things that human beings can do much more effectively than computers, such as identifying objects in a photo or video performing data de-duplication, transcribing audio records, or researching data details. Traditionally, tasks like this have been accomplished by hiring a large temporary workforce (which is time consuming, expensive, and difficult to scale) or have gone undone."
Researchers are attracted to the service for several reasons... compared to traditional samples recruited through universities or panels, the Mturk service is timely, inexpensive, and targeted. Academics can either choose to dissect their studies and create smaller Human Intelligence Tasks (HITs) or they can post full surveys for Workers to complete in an expedited fashion. Although MTurk has proven to be a efficient and cheap alternative to expensive panel sampling, and just as qualified as university samples, there are some things to keep in mind about the service:
(1) Money does not work as a big moderator for data quality--setting qualifying criteria does.
(2) To avoid the haunting 40 percent Mturk fee, and as best practice, consider running at least 1 soft launch (i.e. 9 or under assignments at a time) anything 10 or over incurs the 40 percent fee rather than the standard 20 percent fee.
(3) Roughly a 50 percent retention rate for longitudinal studies
(4) For cross-sectional and longitudinal studies pay the same each round, no need to increase.
(5) Workers are hyper-sensitive to attention checks. Set timers and force responses whenever it makes sense to do so.
(6) Setting qualifications are important, but you are going to get people outside the USA whether you like it or not due to falsified geo-locations. As a good rule of thumb, you should set your qualifications to 95 percent approved hits, over 1000 approved hits, and location is United States.
(7) It is thought that 97% of workers are overworked.
Some other things to keep in mind when considering MTurk as a sampling alternative is that it is good practice to balance with student sample to put down nay-sayers that will likely surface in the journal review stages. However, some reviewers frown upon split samples--so weigh your options and do what's best for your study. Keep an eye on MTurk Forums--people will likely talk about your study. Check up on them and consider legitimate concerns. If using Qualtrics as your landing site, it works well with the Mturk but you must include a unique ID code for payment. For researchers interested in viewing blogs dedicated to Mturk, visit A Computer Scientist in a Business School and Expirimental Turk.
For those of you wondering about a monetary average, researchers commented in an open SPSP forum and decided on minimum, mid-range and maximum payment averages. The average pay was $0.1379 per minute (SD = 0.059). The max pay mentioned was $0.25 per minute ($15 per hour) and the minimum was $0.05 per minute ($.50 for 10 minutes). If you'd like to read more on this forum topic, as a member of SPSP, you can read the thread here.