Online Experiments involving Surveys
I’ve spent the last couple of months working on an experiment design framework in follow-up to an initial study concerning pragmatic understanding of UI designs.
Though I discovered a few frameworks out there for constructing online experiments. I’m working on several experiments involving hundreds of submissions and requiring:
- Subject anonymity - In at least one experiment, questions are sensitive and anonymity will be essential
- Relatively, browser neutral - I don’t want to exclude people on the basis that they don’t have the most current browser technology
- Extensible - I want to be able to easily re-use code across experiments
- Deployable to Amazon Mechanical Turk - I should be able to deploy an online experiment across multiple venues, but especially via AMT.
- Secure, reliable, … - It’s a lot of work to ensure all layers of your software stack is resilient on the open Internet.
I started down the path of writing my own JS front-end built on top of node.js server and mongoDB. I learned early on that I could avoid back-end development by using services from firebase.com. But I spent quite a time spinning up on the angular.js front-end framework because it was something I started to use for other purposes. Both Firebase and angular.js are spectacular new technologies and I wish I had more time to go down this route!
However – time is flying and progress has still been slow. Following my schedule, I set aside front-end development and moved on to learning about how to integrate with mechanical turk. And… I quickly discovered an alternative development option.
Hopefully, I will have a new blog post in a few weeks detailing useful information on how to extend Qualtrics for design of behavioral UI experiments. Thus far, everything seems doable. Qualtrics uses Prototype.js and has made it incredibly easy to interact with their Question API.