Micro perceptual human computation for visual tasks

ACM Transactions on Graphics

Volumn 31, Issue 5, 2012, Pages

  Gingold, Yotam ^a,b   Shamir, Ariel ^b   Cohen Or, Daniel ^a  

^a TEL AVIV UNIVERSITY   (Israel)

^b INTERDISCIPLINARY CENTER   (Israel)

Author keywords

Crowdsourcing; Depth; Graphics; Human computation; Images; Normals; Perception; Symmetry; Vision

Indexed keywords

CROWDSOURCING; DEPTH; GRAPHICS; HUMAN COMPUTATION; IMAGES; NORMALS;

ALGORITHMS; CRYSTAL SYMMETRY; PHOTOGRAPHY; SENSORY PERCEPTION; VISION;

PROBLEM SOLVING;

EID: 84867316251     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2231816.2231817     Document Type: Article

References (58)

1. ACHANTA, R., SHAJI, A., SMITH, K., LUCCHI, A., FUA, P., AND SUSSTRUNK, S. 2010. Superpixels. Tech. rep., EPFL.
2. ADAR, E. 2011. Why I hate Mechanical Turk research. In Proceedings of the CHI' Workshop on Crowdsourcing and Human Computation.
3. ADOMAVICIUS, G. AND TUZHILIN, A. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. Trans. Knowl. Data Engin. 17, 734-749.
4. AHN, L. V., BLUM, M., HOPPER, N. J., AND LANGFORD, J. 2003. CAPTCHA: Using hard AI problems for security. In Proceedings of the Conference on Advances in Cryptology (Eurocrypt). 294-311.
5. AMAZON. 2005. Mechanical turk. http://www.mturk.com/.
6. AMER, M., RAICH, R., AND TODOROVIC, S. 2010. Monocular extraction of 2.1D sketch. In Proceedings of the International Conference on Image Processing (ICIP). 3437-3440.
7. ARMBRUST, M., FOX, A., GRIFFITH, R., JOSEPH, A. D., KATZ, R., KONWINSKI, A., LEE, G., PATTERSON, D., RABKIN, A., STOICA, I., AND ZAHARIA, M. 2010. A view of cloud computing. Comm. ACM 53, 50-58.
8. ASSA, J. AND WOLF, I. 2007. Diorama construction from a single image. In Proceedings of the Eurographics Conference. Eurographics Association.
9. BELHUMEUR, P. N., KRIEGMAN, D. J., AND YUILLE, A. L. 1997. The bas-relief ambiguity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1060-1066.
10. BERNSTEIN, M. S., BRANDT, J., MILLER, R. C., AND KARGER, D. R. 2011. Crowds in two seconds: Enabling real-time crowd-powered interfaces. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). 32-42.
11. BERNSTEIN, M. S., LITTLE, G., MILLER, R. C., HARTMANN, B., ACKERMAN, M. S., KARGER, D. R., CROWELL, D., AND PANOVICH, K. 2010. Soylent: A word processor with a crowd inside. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). 313-322.
12. BHAT, P., ZITNICK, C. L., COHEN, M., AND CURLESS, B. 2010. Gradient Shop: A gradient-domain optimization framework for image and video filtering. ACM Trans. Graph. 29, 10:1-10:14.
13. BIGHAM, J. P., JAYANT, C., JI, H., LITTLE, G., MILLER, A., MILLER, R. C., MILLER, R., TATAROWICZ, A., WHITE, B., WHITE, S., AND YEH, T. 2010. Viz Wiz: Nearly real-time answers to visual questions. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). 333-342.
14. BRANSON, S., WAH, C., BABENKO, B., SCHROFF, F., WELINDER, P., PERONA, P., AND BELONGIE, S. 2010. Visual recognition with humans in the loop. In Proceedings of the European Conference on Computer Vision (ECCV).
15. CHEN, P.-C., HAYS, J. H., LEE, S., PARK, M., AND LIU, Y. 2007. A quantitative evaluation of symmetry detection algorithms. Tech. rep. CMU-RI-TR-07-36, Robotics Institute, Pittsburgh, PA.
16. CHEN, X., GOLOVINSKIY, A., AND FUNKHOUSER, T. 2009. A benchmark for 3D mesh segmentation. ACM Trans. Graph. 28, 3.
17. CHILTON, L. B., HORTON, J. J., MILLER, R. C., AND AZENKOT, S. 2010. Task search in a human computation market. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP). 1-9.
18. COLE, F., SANIK, K., DE CARLO, D., FINKELSTEIN, A., FUNKHOUSER, T., RUSINKIEWICZ, S., AND SINGH, M. 2009. How well do line drawings depict shape? ACM Trans. Graph. 28, 3.
19. COMANICIU, D. AND MEER, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603-619.
20. CORNELIUS, H., PERD'OCH, M., MATAS, J., AND LOY, G. 2007. Efficient symmetry detection using local affine frames. In Proceedings of the Scandinavian Conference on Image Analysis (SCIA). 152-161.
21. CROWDFLOWER. 2007. Crowdflower. http://crowdflower.com/.
22. DUROU, J.-D., FALCONE, M., AND SAGONA, M. 2008. Numerical methods for shape-from-shading: A new survey with benchmarks. Comput. Vis. Image Understand. 109, 22-43.
23. FARIDANI, S., HARTMANN, B., AND IPEIROTIS, P. 2011. What's the right price? Pricing tasks for finishing on time. In Proceedings of the AAAI Workshop on Human Computation (HCOMP).
24. GOLDBERG, D., NICHOLS, D., OKI, B. M., AND TERRY, D. 1992. Using collaborative filtering to weave an information tapestry. Comm. ACM 35, 61-70.
25. GRIER, D. A. 2005. When Computers Were Human. Princeton University Press.
26. HAYES, B. 2008. Cloud computing. Comm. ACM 51, 7, 9-11.
27. HEALY, A. F., PROCTOR, R. W., AND WEINER, I. B., EDS. 2003. Experimental Psychology. Handbook of Psychology. Vol. 4. Wiley.
28. HEER, J. AND BOSTOCK, M. 2010. Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI). 203-212.
29. HOIEM, D., EFROS, A. A., AND HEBERT, M. 2005. Automatic photo pop-up. http://www.cs.uiuc.edu/homes/dhoiem/projects/popup/.
30. HUANG, E., ZHANG, H., PARKES, D. C., GAJOS, K. Z., AND CHEN, Y. 2010. Toward automatic task design: A progress report. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP).
31. IPEIROTIS, P. G. 2010. Analyzing the amazon mechanical turk marketplace. ACM Crossroads 17, 16-21.
32. IPEIROTIS, P. G., PROVOST, F., AND WANG, J. 2010. Quality management on amazon mechanical turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP).
33. KALOGERAKIS, E., HERTZMANN, A., AND SINGH, K. 2010. Learning 3D mesh segmentation and labeling. ACM Trans. Graph. 29, 3.
34. KOENDERINK, J. J., VAN DOORN, A. J., AND KAPPERS, A. M. L. 1992. Surface perception in pictures. Percept. Psycophys. 52, 5, 487-496.
35. KOENDERINK, J. J., VAN DOORN, A. J., KAPPERS, A. M. L., AND TODD, J. T. 2001. Ambiguity and the 'mental eye' in pictorial relief. Percept. 30, 431-448.
36. LEVINSHTEIN, A., STERE, A., KUTULAKOS, K. N., FLEET, D. J., DICKINSON, S. J., AND SIDDIQI, K. 2009. TurboPixels: Fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31, 2290-2297.
37. LITTLE, G., CHILTON, L. B., GOLDMAN, M., AND MILLER, R. C. 2010. TurKit: Human computation algorithms on Mechanical Turk. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST).
38. LIU, Y., HEL-OR, H., KAPLAN, C. S., AND GOOL, L. V. 2010. Computational symmetryin computer vision and computer graphics. Found. Trends Comput. Graph. Vis. 5, 1-195.
39. MASON, W. AND SURI, S. 2011. Conducting behavioral research on amazon's mechanical turk. Behav. Res. Methods 44, 1.
40. MASON, W. AND WATTS, D. J. 2010. Financial incentives and the "performance of crowds". SIGKDD Explor. Newslett. 11, 100-108.
41. OH, B. M., CHEN, M., DORSEY, J., AND DURAND, F. 2001. Image-Based modeling and photo editing. In Proceedings of the ACM SIGGRAPH Conference. 433-442.
42. QUINN, A. J. AND BEDERSON, B. B. 2011. Human computation: A survey and taxonomy of a growing field. In Proceedings of the ACM SIGCHI Conference. 1403-1412.
43. RUSSEL, B. C., TORRALBA, A., MURPHY, K. P., AND FREEMAN, W. T. 2008. LabelMe: A database and Web-based tool for image annotation. Int. J. Comput. Vis. 77, 1-3, 157-173.
44. SAMASOURCE. 2008. Samasource. http://www.samasource.org/.
45. SAXENA, A., SUN, M., AND NG, A. Y. 2009. Make3D: Learning 3D scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31, 824-840.
46. SCHMIDT, R., KHAN, A., KURTENBACH, G., AND SINGH, K. 2009. On expert performance in 3D curve-drawing tasks. In Proceedings of the Eurographics Workshop on Sketch-Based Interfaces and Modeling (SBIM). 133-140.
47. SHAHAF, D. AND HORVITZ, E. 2010. Generalized task markets for human and machine computation. In Proceedings of the National Conference on Artificial Intelligence.
48. SOROKIN, A., BERENSON, D., SRINIVASA, S., AND HEBERT, M. 2010. People helping robots helping people: Crowdsourcing for grasping novel objects. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
49. SPIRO, I., TAYLOR, G., WILLIAMS, G., AND BREGLER, C. 2010. Hands by hand: Crowd-Sourced motion tracking for gesture annotation. In Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPRW). 17-24.
50. SYKORA, D., SEDLACEK, D., JINCHAO, S., DINGLIANA, J., AND COLLINS, S. 2010. Adding depth to cartoons using sparse depth (in) equalities. Comput. Graph. Forum 29, 2.
51. TALTON, J. O., GIBSON, D., YANG, L., HANRAHAN, P., AND KOLTUN, V. 2009. Exploratory modeling with collaborative design spaces. ACM Trans. Graph. 28, 167:1-167:10.
52. TXTEAGLE. 2009. Txteagle. http://txteagle.com/.
53. VENTURA, J., DI VERDI, S., AND HOLLERER, T. 2009. A sketch-based interface for photo pop-up. In Proceedings of the Eurographics Workshop on Sketch-Based Interfaces and Modeling (SBIM).
54. VON AHN, L. 2005. Human computation. Ph. D. thesis, Carnegie Mellon University, Pittsburgh, PA.
55. VON AHN, L. AND DABBISH, L. 2004. Labeling images with a computer game. In Proceedings of the ACM SIGCHI Conference. 319-326.
56. VON AHN, L. AND DABBISH, L. 2008. General techniques for designing games with a purpose. Comm. ACM 51, 8, 58-67.
57. WU, T.-P., SUN, J., TANG, C.-K., AND SHUM, H.-Y. 2008. Interactive normal reconstruction from a single image. ACM Trans. Graph. 27, 119:1-119:9.
58. th International Conference on Computer Vision (ICCV). 1451-1458.