Skip to content


  • Poster presentation
  • Open Access

Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk

  • 1Email author,
  • 1,
  • 2,
  • 1 and
  • 1
BMC Neuroscience201516 (Suppl 1) :P278

  • Published:


  • Spatial Frequency
  • Discrimination Task
  • Human Visual System
  • Natural Image
  • Comparable Quality

The human visual system must employ mechanisms to minimize informational redundancy whilst maintaining that which is behaviorally relevant [1, 2]. Previous research has concentrated on two-point correlations via spatial frequency and orientation tuning. Higher-order correlations are less studied, but they may inform us about cortical functioning [3]. Isotrigon textures can be used to probe the sensitivity of the human visual system as their structure is exclusively due to 4th and higher-order spatial correlations [4]. Although artificially generated, the same features that give isotrigons salience also create salience in natural images [2]. We implemented an isotrigon discrimination task using the crowdsourcing platform Amazon Mechanical Turk (mTurk) [5]. An important secondary aim was to evaluate the suitability of mTurk for visual psychometric studies as very few exist [6].

960 HITs were uploaded to mTurk and 121 naïve subjects participated. Based on data quality, 91% of HITs were retained at a cost of $0.132 AUD per HIT. The mTurk data was compared to two supervised lab datasets. Lab and mTurk performance functions were very similar (Figure 1A) and highly correlated (Figure 1B). Bland-Altman plots were examined and the mean lab/mTurk coefficient of repeatability was 15.5%. Factor analysis was performed on the combined data and 2 principal factors were identified. Previous studies support that the number of mechanisms is less than 10 [7] and more likely 2-4 [8, 9]. The congruence between the lab and mTurk data is striking considering the unsupervised mode of delivery. In conclusion, mTurk is an underutilized platform for visual psychometric research which can produce data of comparable quality to lab samples at reduced cost and increased scale.
Figure 1
Figure 1

1A: Color map of median texture discrimination performance versus isotrigon texture type. Lab datasets DC (84 HITs) and M1-6 (270 HITs). mTurk datasets Live1 (480 HITs) and Live2 (480 HITs). 1B: Pearson's correlation coefficients between datasets (abbreviations as above).

Authors’ Affiliations

Eccles Institute for Neuroscience, John Curtin School of Medical Research, ANU, Canberra, ACT 0200, Australia
Department of Neurology & Neuroscience, Weill Cornell Medical College, 1300 York Ave, New York, 10021, USA


  1. H Barlow: Redundancy reduction revisited. Network. 2001, 12 (3): 241-253.View ArticleGoogle Scholar
  2. G Tkacik, JS Prentice, JD Victor, V Balasubramanian: Local statistics in natural scenes predict the saliency of synthetic textures. Proc Natl Acad Sci U S A. 2010, 107 (42): 18149-18154.View ArticleGoogle Scholar
  3. JD Victor: Isodipole Textures: A Window on Cortical Mechanisms of Form Processing. Early Vision and Beyond. Edited by: Papathomas TVC, C.; Gorea, A; Kowler, E. 1995, MIT Press, 99-107.Google Scholar
  4. T Maddess, Y Nagai, AC James, A Ankiewcz: Binary and ternary textures containing higher-order spatial correlations. Vision Res. 2004, 44 (11): 1093-1113.View ArticleGoogle Scholar
  5. W Mason, S Suri: Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods. 2012, 44 (1): 1-23.View ArticleGoogle Scholar
  6. J Freeman, CM Ziemba, DJ Heeger, EP Simoncelli, JA Movshon: A functional and perceptual signature of the second visual area in primates. Nature neuroscience. 2013, 16 (7): 974-981.View ArticleGoogle Scholar
  7. RR Taylor, T Maddess, Y Nagai: Spatial biases and computational constraints on the encoding of complex local image structure. J Vis. 2008, 8 (7): 19 11-13.Google Scholar
  8. T Maddess, Y Nagai: Discriminating isotrigon textures [corrected]. Vision Res. 2001, 41 (28): 3837-3860.View ArticleGoogle Scholar
  9. T Maddess, Y Nagai, JD Victor, RR Taylor: Multilevel isotrigon textures. J Opt Soc Am A Opt Image Sci Vis. 2007, 24 (2): 278-293.View ArticleGoogle Scholar


© Seamons et al. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.