Signals matter: understanding popularity and impact of users on stack overflow

Abstract

Stack Overflow, a Q&A site on programming, awards reputation points and badges (game elements) to users on performing various actions. Situating our work in Digital Signaling Theory, we investigate the role of these game elements in characterizing social qualities (specifically, popularity and impact) of its users. We operationalize these attributes using common metrics and apply statistical modeling to quantify and validate the strength of these signals empirically. Our results are based on a rich dataset of 3,831,147 users and their activities spanning nearly a decade since the site's inception in 2008. We present evidence that specific non-trivial badges, reputation scores, and age of the user on the site positively correlate with popularity and impact. Further, we _nd that the presence of costly to earn and hard to observe signals qualitatively differentiates highly impactful users from highly popular users. Extending the work, we take a deeper look at the impact of users and how it changes across different communities. To operationalise this study, we define communities based on the user-specified tags on the questions. We aim to understand the dynamics of communities and learn about the impact of generalists vs. specialists users.