DriveBy: Mapping the interactive experience while driving

Interaction Research is heavily contextual. Assessing behaviour is not enough unless viewed in light of the subject's environment, the present situation, the demands exerted on him, and the objects and activities that surround him. This work showcases how AI and technology can push the boundaries of sense-making of all possible unstructured data in a complex environment.

Informing the design of an interactive system

The job was to understand driver behaviour with respect to automotive interaction systems, to assess friction and failure points, and design for a successful product launch. 

Why subject video processing is never enough

• What interaction actually impliesInteraction is about engagement which is dynamic and involves other subjects, entities and activities. Its as much about the surrounding environment and current situation, as it is about the subject and object in question. And for all these reasons studying interaction is more than just mapping the user behaviour.  

• Sentiment analysis is good to have, behavioural data is a must
  Unfortunately, too much has been said about assessing the subjects feelings. And studies also expect drivers to speak up because their AI replies heavily on transcripts. But to be true to the study, its important to capture not just what the subject is saying and feeling, but also doing. Behaviour is a dynamic descriptor and so we need video processing that actually tells us what's actually going on it the video. Especially when using Qual@Scale which could either be multiple drivers across different countries in the same time, or longitudinal data of the driver over time, large datasets need to be processed to explain driver behaviour, and this is the foremost requirement of any video analytics system.   

• Analysing the environment and contextAnother key requirement for an interaction study is about decoding what's happening in the background. Is the driver driving under heavy traffic conditions? Is he in the city or the countryside? What's happening outside the automobile? What conditions are the roads in? All of these elements constitute the environmental context and need decoding because they dictate the driver behaviour.
  Context is also about who's in the car. Interaction systems are often shared by co-passengers, especially for entertainment. And sometimes even administered by the pilot driver. The system must factor in such a context as well when decoding for driver behaviour, expectations and friction level.   

• Dashboards, journeys, moments and more
  Delivery outputs are usually in custom dashboard formats as custom processing requires custom presentation. Hence such projects take a few weeks to setup, but once the dashboards are up and running, they update results within minutes of the experience.
  For this study, the system  mapped moments of the journey, tagged driver events, and summarised environmental context. Based on these, it was possible to create journey timelines, and review actual video snippets/ clips for friction points experienced during the driving journey.        

Forward Paths

The study captured in live video format, all subject interactions within the automobile through 2 cameras positioned for sufficient coverage. Humanify's proprietary, patent-pending system  analysed the video feed into moments, events, and friction points. Analysing the context, environment and outlining where the interactive system failed and more importantly, why it did so. Discovering insights that were meaningful to creating amazing user experiences.