In-car user testing. How to test a mobile app with drivers?

In-car user testing. How to test a mobile app with drivers?

Natalia Pięta
January 03, 2018  | 7 min read

Parkanizer is an awarded parking application for which we conducted very interesting in-car user tests. In this article I would like to share my knowledge with you for the sake of next user-friendly mobile applications for drivers.

If you have any questions along the way, don't be afraid to ask in the comments!

In-car user testing. Preparation phase

Do you know the Pareto rule? 80% of preparation and 20% of work give great results. When conducting the in-car tests with real users preparations are the essentials. One needs to use the imagination to ask the right questions in the right moment.

So before research, you need to have an overall overview on what you want to study. In your imagination, try to recreate the natural circumstances and route for your scenario as much as possible.

Try to imagine how would the test looks like in details

Here are some questions I was asking myself before conducting in-car tests for Parkenizer application.

  • Will I conduct the full test in the car or you can I start it in the lab, office, café or at user’s home before we hit the road? Maybe I should invite the user for a cup of hot coffee after the drive to sum up the test?
  • What route would I like the user to take? Do I want her/him to drive the usual way somewhere or go to some specific location? Will the user drive me back? If not, how will I get back home/ office from the test?
  • Will the whole test be recorded? What will I record? Will it take place at day or after dusk? What is the weather, so also the light, going to be like on the day of the test?
  • What recording tools will I need? Where will I place them? How will I make sure they will not run out of power or storage space during the test? How will I carry them before and after the test?
  • Will I need internet/GPS access during the whole test or only at some stages? How will I make sure it is available in different places? What will I do when the access is lost?
  • Will I use my phone or the user’s smartphone? Will it be paired with the in-car screen? Do we need any backup? What happens if the smartphone battery goes low?
  • Where will be the smartphone located? Do I want to record the way the user uses it on everyday basis (so sometimes against the law) or would I rather concentrate on the safety requirements?
  • Where will I sit in the car? When to ask which questions? When it will be best not to ask questions?

As I mentioned, these are just some questions ;)

In-car user testing. Where to start and finish the tests

There are different logistic, but also technological factors that will determine the best place to start and finish the usability test with a driver. I recommend considering at least 4 of them:

  • In what circumstances would the user usually use the app?
  • How would the user normally start using the app?
  • What is the stage of development of the app for drivers you want to test?
  • How much time and effort will it take for you to adjust to the user’s natural use scenario?

The design of mobile application should be tested in various circumstances due to gaining real-life knowledge in different situations. The key is to recreate the natural circumstances and route for your scenario as much as possible.

Example: If the user should use the app at the way to work, try to accompany him/her some day in the morning, when he/she actually drives to work. Then it would be natural to start the test at the user’s home and have a cup of coffee before hitting the road. Insights about the context of the service you are providing may turn out invaluable.

Pay attention to details!

Again, your application must work smoothly in different driving situations. Observe what activities a driver does when driving. He or she can e.g.:

  • turn the engine on/off,
  • find the nearest gas station,
  • stop a car at the lights.

Of course, there is much more. If the app is usually searched for or turned on in the car – start by sitting in and let the driver show you how she/he does it her/his way.

But if you want to test something that would naturally be done out of the car, you can do it in the lab or any calm place where you can sit down with the driver.

The place your test ends also influences what information you get.

Driver's focus and satisfaction

Another important factor is driver's focus. In-car user testing may include situations whit talking, not talking, searching for a specific place in a hurry, using the app in the traffic jam and so on.

Collect some opinions about the satisfaction with using the app. Talk with the driver for several minutes in some quiet place and sum it up. If he/she is in hurry, ask the user if you can call him in the afternoon or tomorrow to ask what he thinks after several hours.

What do you test? Functionality vs mockup

There is a difference whether you test the app or mockup. 
  • If you have the functional app, adjust 100% to user’s habits when possible.
  • If you are checking some functionalities with existing users – ask them where and how they use the app when you recruit them and try to shadow them.
  • If your application is by now only a bunch of sketches or mockups, start in calm environment to teach the user a little bit about these artifacts and instruct him how to think about and interact with them. Move to the car when the user is ready.
 in studio user testing app

How much will it take to adjust to the user’s natural use scenario?

You are always trying to balance pros and cons of some research decisions. Decide on the taken roads (known / unknown) but just make sure you pick the right combination of factors that make it possible to validate the product and get a valuable feedback from the driver.

For me – it usually means getting as close to the natural use of the app as possible, on the routes that the drivers would pick themselves.

Tools for in-car user testing

A video recorder, a voice recorded, a smartphone... What to pick? Consider lenght and purpose of your test.

If you check only one or two functionalities, it may be sufficient to record the interview with the voice recorder on your smartphone and use a smartphone as a video recording device in the crucial moments.

in car usability testing with phone
But be aware that with a smartphone you set some kind of a device-barrier between you and the user. What is more, due to concentrating on recording with your phone you may be less concentrated and less attentive.

From my experience it is better to record the user tests with a small portable camera used together with several types of grips.

user testing go pro on tripodFor Parkanizer we have used a Go Pro HERO 4 Silver for the last 18 months – it is just sufficient for our needs. It is light and can be used even outside the lab.

Extra tip: Pick the lowest quality possible which looks OK for you for the recordings – you will save storage space and energy of the camera. If you need, you can use the mode which includes both recording video and taking pictures every several seconds.

in car usability app Grips can really help when installing a camera.
 

A preparation checklist

At least one day before you take off for the test with a driver:

  • check what kind of frames you get when placing the camera in different places in your own car. Experiment also with the placement and position of the smartphone. That will help you make a decision and set up the camera fast when you enter any car;
  • try recording in-car in the different times of the day and different lighting conditions. The settings you need for the night or cloudy January afternoon will be different than for a sunny July noon. In case of a lot of sunlight, you may need some sun shield for a smartphone’s screen. Otherwise nothing will be visible on the video;
  • check how much time will the battery in your device last and how much video recordings will your micro-SD card (or any other storage device) hold;
  • fully charge your smartphone before you start the user testing – you may need it for communication, but also as a hotspot if there is a problem with the internet connection on the user’s smartphone.;

  • at the end of the day check if the camera is charged with the storage card inside ;-).

As a practitioner who always tries to counteract the worst scenarios before they become reality, I would also recommend that you take your laptop to the field with you.

Thanks to that you will be able to download the files immediately after recording and recharge the camera a bit before another test.

Learn as much as possible about the driver in advance

As a good researcher you want to know exactly who will use your app. I mean not a demographic profile, but drivers habits.

In Parkanizer we asked eg. about: how the user drive, where he puts a smartphone in the car, does he pair it with any devices, how much does he use it during the drive and during the day. We asked all the questions that supported the development of a useful app.

Phone holder. Important item

One of the most important things you have to learn is whether the driver uses any kind of car phone holder. It may occur that the user has it, but does not use it or does not have one at all.

This is the place where the research gets really tricky: you have to ask yourself if you want to test the application with the minimal intervention and maximum realism or do you put safety first.

in car usability testing detail distraction I very rarely make the decision that the need for full realism of the research context beats the need for a responsible behaviour behind the wheel. Safety first.

Distracted drivers kill 9 people per day

The U.S. Center For Disease Control and Prevention estimates that 9 people are killed every day in the U.S. as a result of crashes involving a distracted driver.

It is though very important to reduce the distractions to minimum – design the application so it requires minimum attention and interaction, but also make sure the cell phone is located in the spot where having a look at it does not require the driver to take her eyes off the road.

If the driver you are meeting with is still unaware of the possible effects of being distracted by her phone, it is your role to educate him.

Avoid distracting the driver

But not only the electronic devices may distract the driver during the test. It can be also you and your questions that cause tension, pull the driver's attention off the road and make it difficult to stay concentrated on driving.

That is why you should:

  • ask most of your questions before you start a ride or after you finish it;
  • ask the driver to talk-aloud what is he doing when driving, but try not to ask further questions or make him concentrate on the app. If you see it requires driver’s effort, tell him to stop. You will see how much attention does the app draw anyways by observing what is happening;

  • ask some questions carefully when you stand in the traffic jam or stop at a red traffic light, but make sure answering them does not require more than 1-2 clicks in the app or much thinking.

A sign for redesign

Remember – a small talk between a passenger and a driver is not a crime, but you must be sensitive to the capabilities of the person you are interviewing in this specific circumstances.

If you can see that some functionality of the app causes distraction and may be a source of a dangerous behaviour on the road, make sure your team redesigns or removes it from the app.

You are a guest

People treat their cars as extensions of themselves or their homes. They decorate it, they keep their things inside, they let it be maximally messy or keep it as tidy as they like. Each car is different.

As Michael Margolis remarks in one of his latest posts on Medium - when you are conducting a test with a user, you should behave like a good host - do your best to make her comfortable and build a rapport by being open, attentive and encouraging, but not overwhelming.

In Parkanizer we did nothing more and nothing less than be a gracious guest.

That is why our application got awarded!

Keep reading

tech development

Building a custom Prometheus exporter

In this post I'd like to guide you through the process of ...

Marcin Budny

Dec 02, 2019

6 min read

business

4 Business Model Canvas Tips

In May, 2019, I had the pleasure of attending the ...

Jarosław Pilarczyk

Sep 11, 2019

4 min read

tech development

Bluetooth with Raspberry Pi and bleno - part 1: iBeacon

Check how easy it is to create a Bluetooth peripheral ...

Marcin Budny

Dec 14, 2017

2 min read

tech ux design mobile

How to make progress animation with Lottie?

Animations created with Lottie save a lot of effort and ...

Kamil Chlebuś

Mar 12, 2019

3 min read

Show More Posts