Online Experiments involving Surveys

The challenge of designing online experiments using survey software.


I’ve spent the last couple of months working on an experiment design framework in follow-up to an initial study concerning pragmatic understanding of UI designs.

Though I discovered a few frameworks out there for constructing online experiments. I’m working on several experiments involving hundreds of submissions and requiring:

I started down the path of writing my own JS front-end built on top of node.js server and mongoDB. I learned early on that I could avoid back-end development by using services from firebase.com. But I spent quite a time spinning up on the angular.js front-end framework because it was something I started to use for other purposes. Both Firebase and angular.js are spectacular new technologies and I wish I had more time to go down this route!

However – time is flying and progress has still been slow. Following my schedule, I set aside front-end development and moved on to learning about how to integrate with mechanical turk. And… I quickly discovered an alternative development option.

I discovered Qualtrics research suite software which allows researchers to create surveys with branching display logic and with a JavaScript API that makes it easy to embed custom JavaScript and interact with AMT. Even more lucky - my University has an account and, that by logging in with my.edu credentials, I have full access to this capability.

qualtrics survey flow

Hopefully, I will have a new blog post in a few weeks detailing useful information on how to extend Qualtrics for design of behavioral UI experiments. Thus far, everything seems doable. Qualtrics uses Prototype.js and has made it incredibly easy to interact with their Question API.