In recent years, real time analysis of user state based on signals from the brain and peripheral physiology has made progress, typically within separate branches of research.  However, the ability to predict user intention from these inferred states is still a grand challenge in real-world applications.

Solution

As a novel solution, MindSee proposes to fuse  EEG - as the main sensor - with peripheral physiological sensors (EDR, fEMG, eye gaze and pupillometry) and contextual information for unobtrusive acquisition of implicit measures of perception, cognition and emotion. 

Real-world application 

Real-time estimates of these implicit, or hidden, user states will be used to complement keyboard and gestural input in a real-world application of scientific literature search where the information exploration of the user is guided by co-adaptation with the computer. The proposed Symbiotic Information Seeking System will provide a wide range of visualization resources that adapt the information retrieved according to its relevance, cognitive ergonomic complexity, and aesthetic properties. For the target application of scientific literature search, MindSee builds upon a cutting-edge retrieval system that has access to 50 million documents from the main scientific databases.

MindSee will be developed using an iterative approach with three full cycles of implementation and evaluation of increasingly complex symbiotic interactions in information seeking. The iterative evaluation of MindSee technology and validation of underlying methods will be conducted in realistic experiments with user groups that vary in skill and content area.

The MindSee project will develop a new symbiotic information retrieval system capable of more than doubling the performance of information seeking in realistic tasks, compared to mainstream tools. MindSee symbiotic interaction will deliver solutions to increase productivity and creative potential. Several MindSee results are exploitable and applicable to other information seeking contexts beyond scientific literature search!

Project Details

  • Project Name: Symbiotic Mind Computer Interaction for Information Seeking
  • Project Acronym: MindSee
  • Project number: 611570
  • Call (part) identifier: FP7-ICT-2013-10
  • Project Start Date: 1st October 2013
  • Project Duration: 36 months
  • Project Coordinator: University of Helsinki, FI, Prof Giulio Jacucci