Sabine Engelhardt and Alexander Mankowsky from our Future Technologies department explain in an interview what we can learn from three ponies and four GoPro cameras about the role of perception and empathy when working with automated vehicles.
Ms Engelhardt, how is your little herd of three ponies helping with the development of automated vehicles?
Sabine Engelhardt: What our four-legged team has taught us, most of all, on our various wanderings, is how important the visualization of awareness is in enabling us to understand others around us. One example of this is the sensor systems: a pony will usually be aware of a cyclist long before we have even noticed them. What we do notice, however, is that the animal's ears twitch suddenly - as if homing in on a specific point … until we, too, become aware of the cyclist.
What does this mean in terms of the interaction between human beings and machines?
Sabine Engelhardt: The animal's sensor is made visible for us human beings via its ears, which makes it so much easier for us to handle it. And that's why this sensory awareness also needs to be made visible in automated vehicles. You see, for people to start putting their trust in the machine, they need to be able to recognize immediately and intuitively what the autonomous vehicle is going to do. Mercedes-Benz is researching this concept of "informed trust" with the help of the "cooperative vehicle". The cooperative vehicle, built on the basis of an S-Class, features a 360-degree beacon and lights on the roof that provide information about the automated driving mode and the proposed movements of the vehicle.
When we think about the mobility of the future, automated driving is one of the key topics that immediately come to mind. The challenge is to create acceptance for automated vehicles. Where is the crux of this issue?
Sabine Engelhardt: The crux of the matter is to overcome our own hesitation. I first became really conscious of this at the Ars Electronica 2015 in Linz, Austria. Alexander was there presenting the F015, the visionary research vehicle from Mercedes-Benz, and gave a talk about it. During the panel discussion that followed, a somewhat anxious-seeming older gentleman raised a question: "Will we have to learn a new language?" I was just so taken aback and my immediate thought was: No way! The inclusion of automated driving in our lives must be intuitive – the way we understand animals, and so many other things that we come up against on a day-to-day basis, without ever having studied them and without the need for an instruction manual.
What role does empathy play in all this?
Alexander Mankowsky: Empathy is a term we use to describe precisely this sensitivity to other people, other situations or animals. One current theory suggests that we can sense a person's intentions by replicating their situation within ourselves, analyzing our own feelings and then mentally transposing these back – assuming that we have at some point experienced those feelings ourselves. As just one example of this, we are constantly trying to read the facial expression of the person we are talking to. By doing so we are trying to work out whether what we are doing or saying is right. And we do so in order to predict the immediate future.
So are we unconsciously also trying to "read" vehicles that we meet on the road?
Alexander Mankowsky: Yes. This is why empathy is so important for the acceptance of automated vehicles.
But the fact of the matter is that automated vehicles, in other words machines, are incapable of developing empathy…
Sabine Engelhardt: But we humans do it nonetheless. We try to read minds. How often do we say that someone is driving aggressively or with a lack of confidence on the freeway? How do we know this? We can't see the people at all. Just from the way the vehicle is moving and our observation of the style of driving, we determine how a person is apparently behaving. Incidentally, we also do it with other machines, for example our computers: "Why are you playing up again?!" And since neither the vehicle nor the computer responds to our efforts: our empathy is completely wasted.
What are the implications of this for automated vehicles?
Alexander Mankowsky: Ultimately, automated vehicles are automatons, which means that we have to make their behavior understandable to our perceptual facilities.
Sabine Engelhardt: What we're trying to do with the cooperative vehicle is to apply this knowledge very consciously and to offer a system that really allows us to understand automated vehicles. The aim is to design automated vehicles in such a way that their movements can be intuitively predicted and anticipated in much the same way as with manually controlled vehicles or pedestrians. And it's precisely this that we're looking to achieve.
For example with your project "See like a pony – SLAP"?
Sabine Engelhardt: That's right. The situation at the Ars Electronica was what gave us the initial impulse to explore our idea further: how can we make that apparently instinctive interaction, triggered by our perceptions, visible? We had previously already spent quite a lot of time considering how my three ponies might make suitable protagonists for this project. I've spent a lot of time roaming with my three four-legged friends over the years, so that I'm now an established fourth member of this little herd. During our wanderings, I've always been amazed to see how attentively they use their ears to keep track of and monitor me and their surroundings. These days the herd is equipped with cameras, allowing us to draw inferences about the communication between people and automatically driving vehicles.
Alexander Mankowsky: SLAP is a way of studying how the indicators of body language can be applied to a vehicle, so that our sensory system perceives the intention and does not miss, or misinterpret, any signals.
How do you go about your observational study?
Sabine Engelhardt: The GoPros are attached to the horses' halters in such a way that their ears are always in shot. I myself wear a helmet, on which my camera is mounted. Then off we go.
Alexander Mankowsky: In order to help us understand how the members of the group adapt to one another and who reacts to whom, and how, the four different views taken on a walk are evaluated in parallel and in sync with one another. We quickly noticed that technical aids support the process of developing empathy. After all: we learn very quickly. This is very instructive as far as the application to automated vehicles is concerned, in that these need to convey information about what they are doing.
Sabine Engelhardt: There's a lot of time and effort involved in this project, as everything needs to sit properly and firmly, but it's worth it! When we go for a walk, I never know what I'm going to learn this time around. The "WOW effect" can come quite unexpectedly when we evaluate the videos – it's very exciting.
How should automated vehicles communicate?
Alexander Mankowsky: The "I can see you" signal, in particular, should be visible but not cause us alarm. Thanks to our ability to empathize, we human beings are very adaptable. Technical devices can be very useful in this regard, as the footage we have shot has shown very clearly. There's no need for any sort of shrill alarm, since we are perfectly capable of understanding the messages without anything like that.
Are there any other findings that you can describe in more detail?
Alexander Makowsky: Observing the way the body moves has been very useful in developing the concept of the cooperative vehicle. The use of movement is key to making automatic driving as intuitively understandable to people as possible. After all: our perceptions are very sensitive to the slightest movement of the body.
How did you use that?
Alexander Mankowsky: For example in the way the cooperative vehicle "wakes up": when the vehicle wants to drive out of a parking space, it "gets up" rather than making loud bleeping noises. The car rises, the mirrors unfold, the lights go on like eyes opening– it stretches itself and in one harmonious movement is suddenly there. Completely without any fuss and bother, we become aware that the car is now active.
Sabine Engelhardt: Today I would be able to tell the gentleman at the Ars Electronica with all certainty: you don't need to learn a new language, spoken or written. It all works perfectly well, even without words!
Thinking about where you go from here with the project, what other areas might be of interest?
Alexander Mankowsky: The next question is: how do we persuade people to help the vehicle or to do what the vehicle wants? Consider what might happen when the vehicle wants to leave a parking space. How can any bystanders blocking the vehicle's route be encouraged to let the vehicle out? We need to find the means of expression to deal with "social negotiating situations" like this where we need to ask human beings for cooperation. There are various ways of approaching this.
One more question to both of you: what elements of SLAP make you particularly proud?
Sabine Engelhardt: What I'm proud of (laughs)? Hard to say, since I've been so much a part of the whole thing myself. The success of the whole learning process has been exciting - the things I have learned myself from the project and the things we have managed to make visible. I would never have thought that what is an inherent part of my everyday life could bring about such findings. I'm really proud of my little herd as it wanders the world, enjoying its adventures and delivering new insights into what, certainly at first glance, is a completely different subject.
What do you think, Mr Mankowsky?
Alexander Mankowsky: The introduction of automated vehicles into society is a groundbreaking innovation. We are proud of our part in making automated driving compatible with the way people live. First and foremost, it's about people. And that's why the vehicles have to be designed in such a way as to make us humans feel comfortable.
The more successful we are at adapting automated vehicles to the perceptive capabilities of human beings, the better the chances for their acceptance in society?
Alexander Mankowsky: Yes, that is extremely important. The very one-sided view that the car needs to recognize people suggests that people are passive beings, which they are not. Human beings need to understand the automated vehicle – and we achieve this by designing it in line with their needs and expectations. Turning the idea on its head like this is essential if we are to deliver a future that offers quality of life.
That sounds promising!