Skip to main content

Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk

The human visual system must employ mechanisms to minimize informational redundancy whilst maintaining that which is behaviorally relevant [1, 2]. Previous research has concentrated on two-point correlations via spatial frequency and orientation tuning. Higher-order correlations are less studied, but they may inform us about cortical functioning [3]. Isotrigon textures can be used to probe the sensitivity of the human visual system as their structure is exclusively due to 4th and higher-order spatial correlations [4]. Although artificially generated, the same features that give isotrigons salience also create salience in natural images [2]. We implemented an isotrigon discrimination task using the crowdsourcing platform Amazon Mechanical Turk (mTurk) [5]. An important secondary aim was to evaluate the suitability of mTurk for visual psychometric studies as very few exist [6].

960 HITs were uploaded to mTurk and 121 naïve subjects participated. Based on data quality, 91% of HITs were retained at a cost of $0.132 AUD per HIT. The mTurk data was compared to two supervised lab datasets. Lab and mTurk performance functions were very similar (Figure 1A) and highly correlated (Figure 1B). Bland-Altman plots were examined and the mean lab/mTurk coefficient of repeatability was 15.5%. Factor analysis was performed on the combined data and 2 principal factors were identified. Previous studies support that the number of mechanisms is less than 10 [7] and more likely 2-4 [8, 9]. The congruence between the lab and mTurk data is striking considering the unsupervised mode of delivery. In conclusion, mTurk is an underutilized platform for visual psychometric research which can produce data of comparable quality to lab samples at reduced cost and increased scale.

Figure 1
figure1

1A: Color map of median texture discrimination performance versus isotrigon texture type. Lab datasets DC (84 HITs) and M1-6 (270 HITs). mTurk datasets Live1 (480 HITs) and Live2 (480 HITs). 1B: Pearson's correlation coefficients between datasets (abbreviations as above).

References

  1. 1.

    H Barlow: Redundancy reduction revisited. Network. 2001, 12 (3): 241-253.

    Article  Google Scholar 

  2. 2.

    G Tkacik, JS Prentice, JD Victor, V Balasubramanian: Local statistics in natural scenes predict the saliency of synthetic textures. Proc Natl Acad Sci U S A. 2010, 107 (42): 18149-18154.

    Article  Google Scholar 

  3. 3.

    JD Victor: Isodipole Textures: A Window on Cortical Mechanisms of Form Processing. Early Vision and Beyond. Edited by: Papathomas TVC, C.; Gorea, A; Kowler, E. 1995, MIT Press, 99-107.

    Google Scholar 

  4. 4.

    T Maddess, Y Nagai, AC James, A Ankiewcz: Binary and ternary textures containing higher-order spatial correlations. Vision Res. 2004, 44 (11): 1093-1113.

    Article  Google Scholar 

  5. 5.

    W Mason, S Suri: Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods. 2012, 44 (1): 1-23.

    Article  Google Scholar 

  6. 6.

    J Freeman, CM Ziemba, DJ Heeger, EP Simoncelli, JA Movshon: A functional and perceptual signature of the second visual area in primates. Nature neuroscience. 2013, 16 (7): 974-981.

    Article  Google Scholar 

  7. 7.

    RR Taylor, T Maddess, Y Nagai: Spatial biases and computational constraints on the encoding of complex local image structure. J Vis. 2008, 8 (7): 19 11-13.

    Google Scholar 

  8. 8.

    T Maddess, Y Nagai: Discriminating isotrigon textures [corrected]. Vision Res. 2001, 41 (28): 3837-3860.

    Article  Google Scholar 

  9. 9.

    T Maddess, Y Nagai, JD Victor, RR Taylor: Multilevel isotrigon textures. J Opt Soc Am A Opt Image Sci Vis. 2007, 24 (2): 278-293.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to John WG Seamons.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Seamons, J.W., Barbosa, M.S., Victor, J.D. et al. Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk. BMC Neurosci 16, P278 (2015). https://doi.org/10.1186/1471-2202-16-S1-P278

Download citation

Keywords

  • Spatial Frequency
  • Discrimination Task
  • Human Visual System
  • Natural Image
  • Comparable Quality