- Poster presentation
- Open Access
Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk
BMC Neuroscience volume 16, Article number: P278 (2015)
The human visual system must employ mechanisms to minimize informational redundancy whilst maintaining that which is behaviorally relevant [1, 2]. Previous research has concentrated on two-point correlations via spatial frequency and orientation tuning. Higher-order correlations are less studied, but they may inform us about cortical functioning . Isotrigon textures can be used to probe the sensitivity of the human visual system as their structure is exclusively due to 4th and higher-order spatial correlations . Although artificially generated, the same features that give isotrigons salience also create salience in natural images . We implemented an isotrigon discrimination task using the crowdsourcing platform Amazon Mechanical Turk (mTurk) . An important secondary aim was to evaluate the suitability of mTurk for visual psychometric studies as very few exist .
960 HITs were uploaded to mTurk and 121 naïve subjects participated. Based on data quality, 91% of HITs were retained at a cost of $0.132 AUD per HIT. The mTurk data was compared to two supervised lab datasets. Lab and mTurk performance functions were very similar (Figure 1A) and highly correlated (Figure 1B). Bland-Altman plots were examined and the mean lab/mTurk coefficient of repeatability was 15.5%. Factor analysis was performed on the combined data and 2 principal factors were identified. Previous studies support that the number of mechanisms is less than 10  and more likely 2-4 [8, 9]. The congruence between the lab and mTurk data is striking considering the unsupervised mode of delivery. In conclusion, mTurk is an underutilized platform for visual psychometric research which can produce data of comparable quality to lab samples at reduced cost and increased scale.
H Barlow: Redundancy reduction revisited. Network. 2001, 12 (3): 241-253.
G Tkacik, JS Prentice, JD Victor, V Balasubramanian: Local statistics in natural scenes predict the saliency of synthetic textures. Proc Natl Acad Sci U S A. 2010, 107 (42): 18149-18154.
JD Victor: Isodipole Textures: A Window on Cortical Mechanisms of Form Processing. Early Vision and Beyond. Edited by: Papathomas TVC, C.; Gorea, A; Kowler, E. 1995, MIT Press, 99-107.
T Maddess, Y Nagai, AC James, A Ankiewcz: Binary and ternary textures containing higher-order spatial correlations. Vision Res. 2004, 44 (11): 1093-1113.
W Mason, S Suri: Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods. 2012, 44 (1): 1-23.
J Freeman, CM Ziemba, DJ Heeger, EP Simoncelli, JA Movshon: A functional and perceptual signature of the second visual area in primates. Nature neuroscience. 2013, 16 (7): 974-981.
RR Taylor, T Maddess, Y Nagai: Spatial biases and computational constraints on the encoding of complex local image structure. J Vis. 2008, 8 (7): 19 11-13.
T Maddess, Y Nagai: Discriminating isotrigon textures [corrected]. Vision Res. 2001, 41 (28): 3837-3860.
T Maddess, Y Nagai, JD Victor, RR Taylor: Multilevel isotrigon textures. J Opt Soc Am A Opt Image Sci Vis. 2007, 24 (2): 278-293.
About this article
Cite this article
Seamons, J.W., Barbosa, M.S., Victor, J.D. et al. Developing and validating an isotrigon texture discrimination task using Amazon Mechanical Turk. BMC Neurosci 16, P278 (2015). https://doi.org/10.1186/1471-2202-16-S1-P278
- Spatial Frequency
- Discrimination Task
- Human Visual System
- Natural Image
- Comparable Quality