Le design de la Privacy
Technology critic and privacy designer Tijmen Schep wrote the book ‘Design My Privacy’ in 2016, and this is used by design schools across the Netherlands to teach the principles of privacy design. Now, two years later, he offers a first look at one of his latest projects: a privacy friendly smart home. He will show and dive into some of the prototypes, which will be the first time these are publicly shown, followed by a discussion of the ethical and design principles that emerged from this hands-on exploration.
Transcription de la conférence réalisée par une bénévole anonyme (un immense merci à elle :))
So hi everybody. I’m glad that you’re here. I hope that we’re going to have a lot of fun together.
I’m very excited about today. My name is Tijmen Schep. I’m a privacy designer and a technology critic. If you are curious what that means – technology critic basically means that I am a researcher.
I explore new technologies and try to understand what they do with society and how I can help a wider audience understand these issues.
For example something that is really at the top of my radar is the social credit system in China, and the rise of reputation systems all over the world really. The rise of the reputation economy. We are going from an information society to a reputation society.
So I’ve done a lot of work of around that for example I’ve launched the word ‘Social Cooling’. Here in France you might know it as ‘Refroidissement Social’. I’m not going to get into that because I could give a whole talk about that, but a part of this is this interest in psychological security which I’ll come back to later. It’s the idea that you can feel free to be crazy to be yourself to feel secure to say what you want.
Another important topic or way of thinking about what I do is through the lenses of surveillance. You have a number of types of surveillance. The classical surveillance itself – you know top down. Then you have sousveillance which is bottom up for example when you film a police officer with your phone that’s sousveillance. And you have one which is really exploding all over the place right now which is ‘coveillance’ where we film each other, where we ‘surveil’ each other, either aware of it or often unaware of it.
Actually Facebook is a good example of, or Google Glass, is a great example of a coveillance device.
So I’m also a privacy designer, so for example I helped start a study in Amsterdam at the Design Acadamy, at the design school called UbiComp design. That was years ago when we didn’t call it ‘Internet of Things’ yet we called it UbiComp.
And I wrote a book about it called ¨Design My Privacy » which is used by a lot of design schools in the Netherlands to teach about privacy design.
I’m also since recently a member of the SHERPA project which is an EU project that explores technological issues in the near future. And you could say I’m a bit of an artist or an activist. I’ve co-founded an organisation called SETUP in Utrecht in the Netherlands and I’m going to show you two small art works, art pieces from an exhibit that we organised a couple of years ago. This was at Dutch Design Week. They are both « design fictions ». So the first one is a coffee machine that basically gives you good or bad coffee based on your area code.
So the idea here was that people could feel that behind their backs a lot of data is being gathered about them and this is used to influence their lives in a lot of ways. Their data is coming to bite them in the arse you could say. It’s coming back to them. We wanted to let people feel that a little bit.
So you get these funny events where you give a cup of coffee to people. Some people didn’t like it at all. Bad coffee …
How many stars do these people have? What’s your perception? Do they live in a good neighbourhood or a bad neighbourhood? Anyone?
Here’s the answer – actually really good.
The second work I’d like to show is the Home Control Kit by Jasper van Loenen. He’s another artist we work with. I like working with him a lot.
And he thought – you know we all want to have great children when we grow up. You know 5-star children. And we also live increasingly in Smart Homes – connected homes. So what if you combine the two. He thought you might be able to automate parenting. So for example in his Smart Home, if the children don’t do the dishes, their score goes down and then the PlayStation won’t turn on. So he’s experimented with all these types of things.
So he had some prototypes like a screen that showed how it all worked and allowed parents to control it and some little toys. Now this is the screen. So here you can see that the parents can decide what they find important in the children’s lives. You know like social interaction is more important or not, taking care of pets is that important or not, and basically what he is doing here is creating a miniature version of what is going on in China. Taking it down to the family level to make it very tangible and understandable what’s going on here.
For example these systems are not neutral. Your parents are deciding what’s good and bad, just like in China where of course the Chinese government is deciding what is good and bad.
This was a small display that the kids could wear that would track all their activities, and the one on the right is for the very smallest babies. It’s a little toy and the head would glow orange or red or green depending on the score of the baby. So yeah – it’s supposed to be funny.
So these works are criticisms of this whole Smart Home idea. Is it perhaps a little bit too smart? This is the question that we asked.
Because this stuff is very powerful, smart homes, and these smart systems, smart cities. They are growing and they have a lot of power over us and at the same time they are very ‘intransparent’. Right? We’ve been talking about this all day, we have no idea how the algorithms work, what’s going on, there is a power imbalance.
You could say that the user is increasingly visible to the outside world, to the institutions, while the system itself is very hidden. Personally I blame Mark Weiser. I wrote my thesis on this man, and – he meant well. He was a researcher in Xerox PARC in the 80s.
And he wrote a paper in which he said: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they become indistinguishable from it”. This is the father of UbiComp. This is the father of the Internet of Things. And he’s telling us to make these things invisible. Well I think he’s very wrong and that was my criticism of his work.
I think making things invisible is a big issue when this technology spreads around because it creates learned helplessness. You create a world where at the one end you are putting technology everywhere and at the same time your story is ‘people should know how technology works’. That creates a big rift – a big power imbalance.
So what I think we are building right now with the internet of things and how it is currently designed is a big trust time-bomb. We’ve already seen some small explosions going off. But I think when this gap widens people will really start to doubt buying smart homes and actually that’s what research is pointing out. More and more people are hesitant to buy smart home stuff. They’re not as into it any more, because of all these tech issues – the ‘techlash’ as it’s called.
So if you say right now that the user is visible and the system is hidden I think we have to swap that around basically, and say well we should make the user very hidden and the system very visible.
So I’ve tried to create a prototype that I’m going to show you that does this to an extreme.
And this is because, after I wrote the book, I wrote about a lot of principles, and I’ll go through them a little bit. And you’ll recognise them you know from the Canadian principles.
But I got a question afterwards from students … and I’ll get into that in a sec. So let’s take a quick look at the book and what’s wrong with the book. Okay.
So it says things like ‘Privacy from the start’. It should be obvious that any product should start in the beginning with privacy. I think we all understand that one by now. ‘Think like a hacker’. For me there is very much an imbalance between the optimistic stories, what technology can bring, and very much a lack of stories about what the risks are. Long term risks, short term risks.
I think we are often talking about the extremes and there is very little good talk and not enough talk about the darker sides.
‘Minimal data gathering’ – that was already the law anyway but you know – who does that? But I put it in the book because it has to be there.
‘Protect your data’ of course. This is another practical one. If you gather data, protect it, encrypt it, etc.
Understand Identity – this one is really important. Because for example so few designers are able to do this. This is very much about for example the case where Google and Facebook tried to implement real name policies. So people had to use their real name on social media, and that created all kinds of problems with people who have very good reasons to not to use their real name, like people in oppressed minorities, gay people, also writers, activists.
So I think this is one example of how a lot of these technology thinkers in silicone valley and other designers have a hard time understanding the complexities of identity.
‘Open the black box’ was another one. I think we’ve heard this one today. The idea that it shouldn’t be a black box. You should be able to look into it on multiple levels. You should be able to perhaps look at how the algorithm works, have some design that explains that or could influence it. Maybe you could even look at the source code.
‘Make the user a designer’ was another one. So this one is very much about the idea that if you design for a screen your user is very simple. From the computer’s perspective there is basically a mouse and a keyboard. When you design for things that go into the real world, smart cars, etc, then you have to anticipate all the ways that people use normal things. All different cultures and all their behaviours. That is incredibly difficult.
A designer can no longer say he can understand or foresee all the uses of technology, how different systems will be used. It’s incredibly difficult.
Lucy Suchman talked a lot about this. She was a thinker who worked with Mark Weiser at the time, and already told him back then ‘Mark this is not a good idea’.
And finally ‘Technology is not neutral’. This is more of a general lesson I think that we have to learn as a society. Oftentimes we fall victim to math-washing or other narratives where it’s all about well the technology is neutral so don’t look too closely at it, let’s not have a debate, it’s neutral. I think we often confuse math with … you know math is neutral, science is neutral, and this uses data so it must be science. It’s not.
But the question that I got from students after I presented the book was this: ‘Great – all these principles – but do you have some examples?’.
I think already today we also talked about principles. How does it work in practice? So I started on a search for two years, how do you design this, how does it work?
So I’m going to give you a big warning – what you are going to see today is a work in progress. It’s not done yet. I’m going to give you a sneak preview. Actually I’m going to give you the first sneak preview ever to an audience of my project. Be gentle!
So it’s called Candle. The reason is very simple. I’m going to talk about the things and the thoughts and I’m going to mix it a little bit.
Start with the things. Why it’s called candle is that the controller is basically hidden inside a candle. You can hide it in whatever you want but I tend to hide it inside LED candles.
Right now I’ve totally forgot the candle, so this is the controller. It’s a Raspberry Pi. It’s not very exciting but it fits inside a candle.
So what is it? This is actually a little wifi hotspot. IF you feel like it you could actually try to connect to it. And everything is stored on this little device. This is somewhere inside your home and this is the centre hub of the smart home.
This device is secure and all that but it also has secure communication with all the nodes – all the different devices in your home. It uses not wifi for that, it uses different technology. That’s on purpose. Putting wifi enabled devices on your home network is a really bad idea, as we’ve seen countless times. You know if you put a wifi-enabled security camera on your home network, it’ll get hacked and that will give people access to your entire network.
So we have to separate these networks. That’s what candle very much does by being a hotspot totally on its own. This is a smart island. It’s not connected to the internet at all. It’s not even connected to your network. It is connected to all the devices though. But how do you control something that is not connected to the internet. Well you can connect to it with your tablet or smartphone … or by using your voice. Because nowadays voice can actually be done locally, quite well, so you don’t need to connect to the cloud for that.
And of course like I said you can connect it to a tablet, so my suggestion is always to use a dedicated tablet that you put on your wall, and that gives you access to the interface and the data on the candle.
So what is inside the candle? So there’s the little Raspberry Pi and for me it is very important that this project can be educational, and easy to build for everybody.
So it’s a little Raspberry Pi, and you connect a little board onto that, that you buy, and you connect a little radio onto that, and that’s the same radio that all the parts use.
That’s it, so no soldering. You put in an SD card and away you go. It becomes a hotspot, you connect to it, you set everything up.
So I tried to make it very easy. It is easy, relatively, for a ‘maker project’. Of course it is open source. All of the things I build are open source. You can see how it works, you can play with it.
And it’s meant to be educational as I said. Making it yourself you have a different kind of experience that when you buy something complete.
It uses a lot of open source projects – for those interested these are the names of those projects.
So let’s look at the things. So I’m going to give you another warning, you are not going to get any blue and white, white plastic with blue LED orbs from the future. Because that’s usually the narrative that is hidden inside these devices.
They’re gorgeous, they’re perfect, they’re clean, as if someone transported it from the future into your home. Blue LEDs, it’s the future, science fiction.
Because this is an anti-project it has specifically no cleanliness at all. So a short look at the website and you can see what type of devices you can build. You can build a controller, you can build a door lock, you can build a barometer, a dust sensor, a smart thermostat, a plant health sensor. And the idea is that you shop for all this yourself. You buy the parts. You click an icon. You get everything in your Aliexpress basket. You buy it. You put it together like LEGO. You’re done.
So this is what it looks like. You get all the parts. It’s actually cheaper than a smart home. It doesn’t cost as much money because you buy the parts. It’s kind of like the IKEA of smart homes. This is the weather sensor for example. You connect all the parts together – you plug it in – kind of like LEGO.
And there’s an explanation on the website how that works, how you build the sensor, what the parts are etc. It tries to make it as easy as possible.
So then you have your device and it looks really ugly because it’s the technology. Then comes the fun part. So where usually you would find these perfectly greatly designed devices, here you go to your local cheap store and look at what normal people buy. What normal people have in their homes. It is not glowing orbs. Nobody wants to have that in their living room. They want to have something like a picture frame for example. So I tried to play with that design narrative.
So this is the first example. This is an air quality sensor, a dust sensor and it is hidden inside a hedgehog. On the back you can see the sensor itself and the Arduino and all those parts I mentioned earlier. They are not hidden, you could hide them but I purposefully didn’t, because I kind of like this idea of the hedgehog being behind there.
So I also try to envision which kind of people would do this or buy this and have this in their home. For me this would be an elderly woman who’s worried about the dust in her house. Perhaps there’s a project nearby that had created a lot of dust.
Another example is this. This is a plant health sensor. So this is a very tiny, again small device. It has six sensors that come out of it. In this case I’ve hidden it inside a small watering device.
And again, I try to imagine what kind of person would love something as cheesy as this in their home.
Actually in my home I use this. I hide all the sensors and devices in books. And I try to make it into jokes. I used to work a lot with the library so I got access to free old books.
So this is a carbon monoxide sensor inside a book called Poisonous which is a thriller. So I have a number of these devices. This is a book about picnicking in nature that has a CO2 sensor in it.
It’s an example of how you can have a lot of fun with it. My electricity sensor at home is in a book about geopolitics called the End of Power.
This is another example, this is a bit of a more recent example. It is a smart alarm clock because I thought maybe that is easy to make – who knows. I try to explore popular devices that you see in Kickstarter, and other project sites. You know, smart thermostat of course, smart doorlock. This is kind of a new one.
So you can actually look inside. You can take the lid off, look inside and see the parts that you put together yourself. The recent edition is the BBC microbit. This is a really interesting device.
It is donated to children all over the world as an educational tool. It’s got a built-in motion sensor. So basically you put the software in it and you’re done, you’ve got a really nice motion sensor to measure the motion of cookie-jars, doors, whatever you want.
You can hide it inside for example a little painting that you put on your door, put a battery on it, and it lasts for a very long time.
A final example that I haven’t finished yet, but that is very high on my list is Anemone. And this is an example where I don’t just try to emulate existing products, I try to create something very new which I think is very much in line with the philosophy of the project.
So Anemone is a small box that goes in between your wall where you plug in your router, and your router. What it does is it basically allows you to disconnect your entire home from the internet.
Why would you want that? Well there are a lot of reasons actually. For example when you disconnect at night, when you sleep, or when you go on holiday. It disconnects you automatically when nobody’s home. This will create a much lower attack factor. Your security is basically infinite because you cannot hack what is not connected to the internet. That’s the same reason behind all air gapping of this system. If it is not connected to the internet you cannot really hack it because it is not connected to the internet. Very secure.
And this allows you to extend your entire network. Of course I didn’t mention some of these like electricity use, smart lock etc but that’s for another time.
So that in a nutshell is candle. It’s the hardware. I’m very much trying to create narrative around that of how it’s also a good idea to build candle – you know the privacy by design aspect, being respectful to people in your home, very secure because it is not connected to the internet, and tries to be more empowering by giving you insight and creating an educational experience.
So those are the things let’s look a bit more at the thoughts. So you’ve seen my eight principles but I’ve also tried to create eight very practical rules – or at least I tried to create more rules – it doesn’t have to be eight but it is so far.
Privacy from the start I’ll have a look at this list and try and come with some of the more practical ones and see how they relate.
So for example go cloudless – or at least allow cloudless use of your smart device. This is a really basic one but really important. If it were up to me, if I could write the law, I would make it mandatory that every smart device has a little slider on it – hardware – and you can set it to only local or network – or not connected to the network or internet. That way you always have control about whether the device is allowed to go online or not – very clear.
In my case it’s a bit … I want to put it in as a demo but whole device is not connected to the internet at all so that makes it less necessary to even have that switch.
And of course devices should work fine without an internet connection. And that is very important because right now we have a lot of reports of companies that go bankrupt and then people’s smart home doesn’t work anymore. You know – two hundred dollar paper weights that are created by this.
So I think it is very important that these things also work without the internet, so that’s why since a few months all these devices now have screens. They didn’t have that before. So that they can totally work standalone. Even without the network capability you still have an air quality sensor just on its own. You can still see what’s going on.
Another rule is minimal viable hardware. This is where I try and take a limitation and turn it into a positive thing. So these devices use Arduino which is a very simple type of technology. It is just enough to have encryption. It is just enough for basic functionality. But don’t expect to run Linux on it. But why would you want to run Linux on it? You don’t! As soon as you can run Linux on a device, then it becomes interesting to hackers. Your smart camera becomes interesting to hackers because it is so powerful.
These devices are not powerful and they are very difficult to hack and if you would do it, it would become immediately obvious because the functionality would not, you know there would be no room for that.
So because they are so simple, so minimal, you can’t really hack them – it would be immediately obvious.
So that’s a few of those. Let’s look at transparency. I think we should allow access to the judging part of algorithms. This project is open source so you can see all the algorithms and there is not much judging going on. But if your project did do that, for example a Nest thermostat, you should be able to see how it rates people’s behaviour.
This project is very educational. It tries to help people learn about the black box. It is designed to be easy, to be accessible, and to be honest.
That’s something that I’ve seen in a lot of smart projects where people call something an ‘optical sensor’ – it has an ‘optical sensor’. No it has a camera – please call it a camera. It has a ‘noise sensor’ – no it has a microphone – call it a microphone.
So I think that this being very critical of language and being aware of that is important. I think we have to call out other designers who play around with that.
A lot of these things we have mentioned so far have ticked a few of the boxes. So privacy from the start is ticked. Think like a hacker is ticked. I mean hackers are totally on my mind. Open the black box very much so. Make the user a designer. There’s an interface in here to make your own connections and you are very welcome to change any of the code. And understand that technology is not neutral.
Here we go … be Scandinavian.
I saw this great start-up on Kickstarter who created a sensor for people who do Airbnb basically. It was an Airbnb sensor. So it had a motion sensor to make sure people didn’t use it, it had a smoke sensor, it had a loud noises sensor, but in all of these choices they chose to have the simplest sensor they could use. They could have put a camera in it for example and just watched the people in their home but you don’t want to do that – that would be rude. So they tried to use the minimal sensors required to detect a certain type of behaviour. So I think that is really a great, logical type of rule.
Okay, so we’ve had a couple of these now. If I had mentioned Scandinavian method why am I not ticking the box Minimal data, Protect data, and Understanding identity.
That’s because what I think I very much learned by building this project is how limited the current smart home technology really is, how it is stuck in a very old fashioned way of thinking.
What’s the problem? Well I’ve talked to you about this model, of surveillance, sousveillance and coveillance. And what you’ll notice from how I’ve been talking is that I’ve been very much trying to protect the user from surveillance. If you’re not online you can’t be surveiled by the evil corporate or whatever. But as I’ve been going to conferences about this subject and talking to people and talking to researchers.
What I’ve increasingly learned is that a really big issue with smart homes is that people use it against each other. Like family members use it against each other because they know exactly what happened – no you were at home and you said you weren’t but you were etc.
So there was a really great researcher I talked to and he was just totally – he didn’t expect that at all. It sounds obvious now when I say it but when he started doing smart home research he was totally taken aback by the way the people abused the system and used the system in a social way.
So the issue here is coveillance. And this is something that my system doesn’t really take into account well enough. The system has basic things like user accounts and stuff like that but dad still has the admin account, and he has control of everything.
So we’ve solved the problem of Google having access to everything but now it is still your Dad who has access to everything and solving that is a lot more difficult actually. So when talking about psychological security I think you want to have a smart home that allows for that, that allows for children to feel secure, as well. People who are not the boss at home to feel secure.
And even more than that, we’ve talked about children but what about visitors. If you buy an Alexa device in your smart home, that basically means that if I visit your smart home you’re saying that I should subject to having my voice pattern and my voice fingerprint, you know, detected by Amazon, and stored. I don’t think that’s a decision, you know, I find it very rude when people make that decision for me, so that is why I don’t like those devices.
But this is very difficult to deal with, so in one of the projects I’ve done I’ve realised we almost need something like photoshop for data. We need the ability to remove certain data points that are unflattering for example, make them look better. And that might sound totally weird, because weren’t all these systems about finding the truth, and finding real good data, and now you are proposing that we make systems that allow people to doctor the data, to make themselves look better.
Well I see a couple of projects that allow for this for example car insurance systems where you can say well this drive wasn’t me it was a friend. So there’s a way to make it socially acceptable to say you hide some data points or remove them. But I think this is really important, I think, talking about identity, this is what made the smartphone a success. The smartphone wasn’t a success because you could make calls with it – it’s not because of the functional device – who even still makes calls – nobody does. What makes smart phones such a success is because it allowed for complex forms of digital identity formation. It allows us to create amazing selfies, allows us to show others how great we are and that is a fundamental human need. To be control of your own identity, to create your own identity, to sculpt your identity.
And when I’m looking at all these smart home systems that I’ve done, I’ve very much realised that none of these systems think in that way yet. They’re all still basically routers. You know the interface is very much like routers. There’s a list of your devices and your sensors – it’s not at all like a smart phone which has appstores and thanks that allow you to have fun with this and to use it as a status symbol for example.
You know I can imagine that an app that you can download to play tricks on your Dad. You know a kid would download the app to their smart home and use that for fun, but we are nowhere near that level of thinking, that Appstore vibe.
So I think that … this is one of the things where I’m totally stumped because I can make the technology well and I can hide it from the internet, but I totally cannot fix right now the problem with identity formation and doing that in a responsible way. So that’s a big problem.
Okay so the conclusion. I’ve shown you a sneak preview of Candle, a weird project that I’ve been working on with a lot of fun and learning a lot from. I’ve tried to put all these ethics into it. Privacy by design, … you know … if I wanted to do it really well how would I do it. What would be the ideal smart home for me. Totally safe and secure and privacy friendly. But, like I said, I couldn’t really make it perfect.
So I guess this is a work in progress, I’ll continue this research by doing part of my work. So what I’ll say is that for now this project is very much a statement. I don’t think this will be on Kickstarter any time soon.
What I hope is to create such an extreme version of the smart home that other companies will go ‘hey there are some interesting ideas in there, maybe we could create a light version of that or take some of those ideas, we could create some more privacy -firendly products.
For example I don’t think many companies will create a system that is totally not connected to the cloud. I hope they will but it will take a couple more years and more scandals before that happens.
But I hope that this project can disable the smart time bomb to be able to make a smart home that people totally can trust, and work from that, instead of trying to create a cloud system that will be secure – I don’t think that is the right approach.
I hope we can rebuild trust together with this research, and if there is one thing I’ve learned it is that people really do care about privacy. People say people don’t care about it – that’s nonsense. There’s a situation where people want to buy privacy products, they’re just not available.
So I hope this project also helps to push us in a direction where we create some middle ground, where we create some new type of products that is more privacy friendly. I think there is a huge opportunity as well. This is not just something I do because I’m an idealist – this is a market opportunity!
There will be more scandals I promise you. Cambridge Analytica was the beginning. People will learn all about how the data economy works. They will want these type of devices.
I compare it very much to global warming – that’s the social cooling part I talked about. I think that comparison is very useful. Because of the global warming we start to learn more about the environment and start to care about, you know, ecological food. And ecological food in ten years has exploded from being a niche market to, you know, you can buy it at the supermarket. Right? That’s incredible. You can even buy vegan pizzas at Dominos now.
And I think the same thing will happen here. I think we’ll see the smart home, the pricing with the technology market, is kind of small right now. It will explode in the next 10 yrs. That’s my prediction. I hope it comes true. I think that EU will help with that. I think we’ve already seen the GDPR has been incredibly helpful.
One of the projects I’m working on right now is a privacy label. This is awful – it won’t look like this at all. This is for a washing machine. And I think in ten years we might get this, if we’re honest.
Because it is a reality, you know, Cambridge Analyticas would love to know your smart thermostat settings, and they probably do that’s the reality of the world we live in right now.
So I think it is the time right now for alternatives and what I take away from the Privacy by Design narrative is that it goes beyond compliance. When I talk to people in the law sector they are all talking about compliance still and compliance is about doing the minimum necessary to not be sued.
Well what I find so amazing about GDPR is that basically, politicians have said, dear designers, do better. You know, be proactive. I think that is really rare that it happens and I think it’s great you know it gives people like me a lot of work (not really 🙂
So I would conclude that privacy is something that we should see as an opportunity. We need like Elon Musk for privacy design who also said ecological design is an opportunity, we can make it really sexy, like a car, and we can make solar panels awesome.
I think we need to have the same thing with privacy by design. I don’t think these devices will be it, but I hope that some of you can bring it in that direction.
That was my story thank you very much.