Trustworthiness in crowdsourcing

Abstract

Crowdsourcing is an exciting new trend where a group of geographically distributed individuals contribute willingly, sometimes for free, towards a common goal. Previous research and studies have shown how Crowdsourcing can be effectively used for Software Development. In this work, we investigate one of the key concerns in Crowdsourcing of Software Development Trustworthiness of the Crowd. Through a comprehensive review of related work, we present a taxonomy of the various concerns around the Trustworthiness of the Crowd. We study the current state of the art techniques to address these concerns both from academic literature and the Crowdsourcing vendors (platforms). From the study, we nd that there are algorithmic techniques to tackle a few of the issues. However, other issues would still need to have a process. Thus to tackle Trustworthiness of the Crowd in a comprehensive way, a combination of algorithmic techniques and a process is required. We also investigate the performance of the algorithmic techniques in the context of Software development through a simulation of crowd behavior.