From Living Lab

Mobile Living Labs 09: Methods and Tools for Evaluation in the Wild

Workshop at MobileHCI09, September 15-18th, 2009, Bonn, Germany.
http://mll09.novay.nl

New: MLL09#Workshop_Results are available

Contents 1 Important Dates 2 Submission 2.1 Workshop Topics 2.2 Review of Papers 3 Workshop Agenda 4 Workshop Results 5 Committee 6 Acknowledgements

In a Mobile Living Lab, mobile devices are used to evaluate concepts and prototypes in real-life settings. In other words: the lab is brought to the people. This one-day workshop provides a forum for designers, researchers and practitioners who are interested in sharing experiences and issues with methods and tools for Mobile Living Labs. In particular, we seek to attract both people who have applied Mobile Living Lab methods as well as those who build tools for Mobile Living Labs.

We invite those designers, researchers and practitioners who want to contribute to a vivid discussion leading to improved methods and tools to study user experience in the real context of use to submit a position paper, on any relevant topic for the workshop, including challenges and solutions for Mobile Living Labs, innovative methods and tools for data collection and analysis.

For more background information, see MLL09/Background

Important Dates

Submission deadline of workshop paper (passed): Friday, May 15th, 2009
Notification of acceptance: Monday, June 8th, 2009
Early registration deadline: Monday, June 29th, 2009
Late registration deadline: Monday, August 10th, 2009
Workshop: Tuesday, September 15th, 2009

Submission

Due to the interactive form of the workshop, we encourage you to submit position papers that take a critical view on existing practices.

Submissions should be 2-4 pages in the MobileHCI09 other tracks paper template format, on any relevant topic for the workshop, including challenges and solutions for Mobile Living Labs, innovative methods and tools for data collection and analysis. A list of further topics is given below.

Submissions should be e-mailed as a PDF attachment to mll09(at)novay.nl by Monday, May 4th. Participants will be selected to represent diverse perspectives and the organizers are looking for positions that can stimulate discussion.

Workshop Topics

Topics and questions to be addressed in the workshop include:

• When to choose taking the lab to the people instead of taking people to the lab?
  
• What are the merits and limitations of Mobile Living Lab methods in general?
  
• For which kind of applications are Mobile Living Labs beneficial? (e.g., only applications that involve social networks and context-aware applications, or are there other categories as well?)
  
• What are the merits and limitations of Mobile Living Lab evaluation methods in general? What are the merits and limitation using it for formative evaluation (i.e., to inform the design of new applications) and for summative evaluation (i.e., to assess the user interface of an application)?
  
• What are the relative merits and limitations of self-report methods, measurement methods and observation methods for studying various aspects of mobile user experience?
• How to deal with long-term and large-scale Mobile Living Labs?
  
• Which tool support is needed for configuring, deploying, data collection and analysis of studies in Mobile Living Labs?
  
• What are the merits and limitations of using people's own mobile devices versus handing out new devices for a study?
  
• To what extent can we study mobile user experience with tools embedded in stationary infrastructure?
  
• What is known about the reliability and validity of these methods and tools? What are the open issues?
  
• How to deal with informed consent, privacy, data/device theft, loss and corruption?
  
• What have we learned so far? What have been the big outcomes from previous Mobile Living Lab workshops?
  

Review of Papers

Position papers will be reviewed by the workshop’s programme committee, which includes the workshop organizers. Participants will be selected to represent diverse perspectives and the organizers are searching for positions that can stimulate discussion. Notification of acceptance will be sent on or before Monday, June 8th, 2009. One author of each accepted paper needs to register for the workshop and for at least one day of the conference.

We will circulate accepted position papers among all workshop participants well before the workshop, thus enabling people to read each other’s positions.

Workshop Agenda

See MLL09/Agenda

Workshop Results

• Accepted position papers are available for download (click on the previous link to download them in a single file). For the list of accepted position papers, see MLL09/Papers
• Results of the pre-workshop survey are available
• Photos of the posters with overviews of methods with their pros and cons created by the subgroups on early stage methods, self-report mthods and the observation-based methods are also available
• Transcription of the posters

Committee

Organizers

Henri ter Hofte, Novay^*, The Netherlands
Kasper Løvborg Jensen, Aalborg University, Denmark
Petteri Nurmi, HIIT, Finland
Jon Froehlich, University of Washington, USA

Program Committee

Tom Broens, Novay^*, The Netherlands
Sunny Consolvo, Intel Research, USA
Alexandre Fleury, Aalborg University, Denmark
Anne Marie Kanstrup, Aalborg University, Denmark
Joke Kort, TNO Information & Communication Technology, The Netherlands
James Landay, University of Washington, USA
Yelena Nakhimovsky, Google, USA
Ingrid Mulder, Delft University of Technology & Rotterdam University, The Netherlands

^*In April 2009, Telematica Instituut changed its name to Novay.

Acknowledgements

The organisation of this workshop is supported in part by the Amsterdam Living Lab project.