This transcript was generated automatically and lightly processed; it may contain small errors. Please check against the audio before quoting.
Meredith Whittaker There's a woman right now in the U.S. living in prison, and she's living in prison because her private messages that her and her daughter sent each other in the state of Nebraska documented their accessing and then dealing with the aftermath of abortion care in a state that had criminalized that. Facebook handed over the data, and that was the evidence used to convict her. Now, that's not the only data that Facebook has. That's not the only case of this type. One of the issues is that we have corporate secrecy on one side, which means we don't know all the data Facebook is handing over. (.....)
Sanam That's Meredith Whitaker, my guest this week on If You Were In Charge, with me, Sanam Naraghi Anderlini.
Welcome, everyone. The show is brought to you by the Robert Bosch Academy and my organization, ICANN. Meredith is the president of the nonprofit Signal Technology Foundation that's responsible for the Signal Communications app. Now, if you know about Signal and WhatsApp and so forth, we often think that they're all the same and their encryption issues and privacy issues are the same. But actually, Signal has a very specific focus on valuing privacy. They say, quote, we can't read your messages or listen to your calls, and no one else can either. Privacy isn't an optional mode. It's just the way that Signal works. Every message, every call, every time. They also say that they don't sell, rent, or monetize any personal data or content, and that's why they are a nonprofit foundation. Of course, they comply with governmental laws, regulations, and legal processes that are enforceable, but they basically don't monetize and collect our data. They say that the message history is stored on our own devices. I'm a Luddite when it comes to technology issues. Of course, I use whatever the apps are, but I don't really understand, necessarily, the ins and outs of all these things. But I wanted to talk to somebody who really does, and Meredith is quite extraordinary. She, apart from being currently the president of Signal, she was the founder of the AI Institute, and she continues to be a chief advisor to them now. She's also advised the White House, the U.S. Federal Communications Commission, and the Trade Commission, the EU, the City of New York, and works on these questions of artificial intelligence, internet policy, privacy, and security. As you'll hear, we talked a lot about what privacy actually means and what is going on, what the sort of infrastructure of the systems are going back to the 1990s.
We talked about the question of surveillance and AI being used in defense and weapons systems right now, which I found quite insidious.
We will come back at the end with Sole to hear her comments and the perspectives of her generation on these issues. And I hope you find the conversation interesting. See you later. Bye. (.....) Meredith, hi. How are you? (.) Hi, I'm doing well. Nice to see you, Salam. Nice to see you. Thank you for taking the time. I wanted to talk to you about so many different issues that you've worked on and you work on right now, both in terms of what you do at Signal and what Signal represents in terms of security and confidentiality issues, but also your work on AI and some of the concerns that you've raised in the past around tech and defense and tech and surveillance and so forth. So there's a lot of issues. But I have to tell you, I start with a confession, which is that I'm very conscious of the fact that there's a lot of stuff that I don't know. And I also assume that I don't know what I don't know. So bear with me as we have this conversation.
Meredith Whittaker Well, that makes the two of us. There's nothing like not knowing to catalyze learning. This is true. And it's what I tell myself.
Sanam And it's also, I was, as I was learning more about you, I was thinking, how did you go from being an English lit and rhetoric major into the world of tech? And no wonder I, we got along when I first met you, because I'm a big fan of everybody having to study literature as part of their kind of evolution, if you want. But what was your first step from Berkeley into the world of technology and how was it, what was it like to be, to be working with tech folks? (.)
Meredith Whittaker Yeah. I mean, I wish the story were more, almost more intentional, but it certainly was not at the beginning. At least Google was the first company to hire me out of Berkeley is the story I put my resume on monster, which was old school LinkedIn for, for the youngs here. (..) And they reached out and I was looking for a job because I didn't have any money, you know, the way it goes in this old world. And, and, you know, it, it was interesting, they were paying the most. And, you know, I realized after I took the job and was, you know, initiated into the world I now occupy for nearly 20 years, that it was a very interesting and, and kind of weird place. And I just got really curious and really, I just kept having a list of questions I had to answer. And that's, that's basically the formula to how I became who I am from an English and rhetoric degree to, you know, a person who's been across the tech industry and academia and thinks and writes and, and researches and, and builds technology. But I do think, you know, literature and close reading skills are perhaps the fundamental skill in my mind. If you can read carefully and closely, you can kind of learn most things. I think that really helped me as I was entering this new world with this new language and this drastically new culture to be able to, okay, well, if everything is strange, if the runic symbols are completely foreign to me, at least I can sit down with a, a one-on-one textbook and start to get some of the fundamentals.
Sanam And I, when I, when I, when I entered my field, it was, for a while I was very intimidated and then I was like, no, actually, if you just read things carefully, one thing you realize is sometimes it's just badly written actually. And then, yeah, then you analyze and, and, and, and you can learn. Um, so just, again, from a sort of a broad, you know, 30,000 foot view, when we think about the universe of tech right now, which, which is kind of integrated in all of our lives. And certainly, you know, for me, the simplest things, you know, the emails and your signal app and, you know, our communications tools are the things that make our work possible and feasible. So there's, there's, there's a lot of good that's been done, but in your, in your perspective and from where, from the work that you've done, if I wanted to say, what's the good, what's the bad and what's the ugly, what would, what are the things that, that come to mind for you?
Meredith Whittaker Look, we have to look at tech as part of a broad set of political economic drivers, or at least as, as shaped by those. And a lot of the work that I've done in my scholarship, which is, it's kind of what I do on the side, but it, it deeply informs how I think about signals place in the world and how I think about situating and strategizing to grow and sustain signals. So they're, they're, they're deeply interrelated in my practice, and I've spent a fair amount of time in the early internet history, so to speak, looking not only at the, the history of military investment in networked computation, which happens, it was kind of post-Cold War, and, you know, very much tied to these grand visions of US empire and hegemony, which were, you know, kind of the, the narrative that was really lofting those, those, those cold world war ambitions and that, that justified extraordinary expenditures in computational research and the build out of these computational networks that then in the 1990s and 1980s returned, or in the 1980s, 90s, starting in the 1980s, sorry, this is, dear reader, we are recording in the morning and I am drinking my coffee now, so pardon the inaccuracy. But anyway, we all know that in the 90s, the internet economy was sort of created the commercial internet and this was, this was really an attempt in that neoliberal era, driven by the Clinton administration, but with broad consensus, to find a way to get the US economy out of stagflation, without resorting to redistributive policies. So, you know, you can look at Matthew Crane's book, who did a really good history of this period, called Profit Over Privacy, and he quotes an official at that time, you know, calling the potential of the commercial internet, the new deal without the socialism. So there was a lot of hope riding on the idea that we could turn over the keys to these networks, to this research that had been done, and we could create an economic miracle. Now, some of that arguably did happen, right, but a miracle for whom, and how was, how was the benefits and harms distributed, and what we see from that time is that, you know, there were some, some real fundamental errors that we're still paying for, and that I believe we still need to go back and, and fundamentally correct. And two of those that really matter in my analysis were the failure to put any restrictions on commercial surveillance, so the private sector could create and collect as much data as they wanted, could surveil as much as they wanted, much less restricted than government. And, and importantly, the Clinton administration endorsed advertising as the economic engine of this commercial internet, and that matters in the context of this surveillance business model that was created out of these policies, because, of course, advertising is a, an imperative to know your customer, right, and to know your customer, you want to know more and more and more about them, to be the best advertising firm, to be the platform that targets your ad desk, to, you know, know, know that Sanam really wants to see an ad for athletic greens and is likely to click on it, or whatever it is, right, the justification there is we need more and more and more information, so we can target your ad the best, so you will, you advertisers will become our customer, and this creates the, everything is free, but you're the product paradigm that we're all familiar with when we open our phones, and, you know, the, the early versions were the assumption that if you're, I don't know, 45, then you are premenopausal, and, and you're gonna, you know, you want the facelift or something, and, and
Sanam And, and, and since then, it's become so much more targeted, I, I'm like, my phone listens to me all the time, I, I think it seems to know exactly what conversations I've had, so that on my Instagram feed, I get certain things that come up based on where I've been, and, you know, what conversation I had last night, it's very, very sort of insidious, it seems.
Meredith Whittaker You know, sadly, Instagram's pretty good. Like, I have to resist, I'll be like, damn it, that is very cute. Yeah. I do not want to click on it. I don't want to acknowledge that there's that sort of, you know, whatever, that algorithmic pressure. But, yeah, they, you know, and maybe unsurprisingly, given that you can buy my credit card data from a data broker, you can see my location data, you can see who I hang out with, you could see, you know, did I start doing Duolingo, because the app is probably scanning my phone to see what I accessed or not. Am I, did I stop doing that? Okay, well, let's pile on a bunch of language learning alternatives. There's a lot of data sources out there.
Sanam And it's, it's a generational thing, because in a way, a lot of people just think, well, so what, it doesn't matter, right? Like, that's, that's part of what we're hearing. So what if I'm getting surveyed, and I'm getting these targeted ads? In fact, it's, it's great, because it's stuff that I might like, right? So what's your answer to that kind of side of the argument?
Meredith Whittaker Well, I don't, in a sense, I, I get it, right? It's not a targeted ad that's hurting us, right? It's not me seeing a cute pair of boots on Instagram, that is causing me any harm, right? And so focusing on that and saying, like, this is bad, doesn't actually make a lot of sense in the context to the average privileged iPhone users day to day. But it's when we kind of flip that and say, like, actually, that's one side of a crystalline structure. And if we turn it to another facet, we see another side, which is that, you know, there's a woman right now, in the US living in prison, and she's living in prison, because her private messages that her and her daughter sent each other in the state of Nebraska, after the Supreme Court's Dobbs decision, which, you know, tossed abortion care back to the states, documented their accessing and then dealing with the aftermath of abortion care in a state that had criminalized that. And that was the evidence used to convict her.
Sanam That data was picked off from, what, their WhatsApp messages? Where was it? Where was it? And who? Yeah, who got it?
Meredith Whittaker Facebook messages. And it was turned over to law enforcement in the case. So Facebook handed over the data? Yeah, Facebook handed over the data. Now, that's not the only data that Facebook has. That's not the only case of this type. But, you know, one of the issues is that we have corporate secrecy on one side, which means we don't know all the data Facebook is handing over. Snowden gave us a peek inside of that ecosystem over a decade ago. But it's changed radically. And the quantities of data that you can create and collect and the presence of powerful machine learning and AI models that, you know, create more data and use more data more elegantly. It's a different world now, but we should assume those relationships are intact and we should assume that a lot of them are sealed. We should assume that a lot of these cases don't make it to a journalist in a feeble news ecosystem who has the time and the energy to dig into it and to report on it.
Sanam It's so hard to do that. And actually, I mean, when I think about surveillance because of the work that I do in war zones and with peace builders and activists, obviously a lot of what we're worried about is the rise of authoritarianism and the hyper surveillance of, you know, face recognition in protests and things like that that are being used. So we've had, you know, Palestinian colleagues talk about that in Iran, they talk about that. And then, of course, what's happening in Gaza with the targeting, you know, with the quadcopters. Is that part of the kind of universe that you're also looking at and thinking about in terms of both the surveillance side of it, but also now with AI or machine learning or whatever, whatever it is that this term that we have to talk about as well?
Meredith Whittaker Absolutely. And I, you know, these aren't identically the same things, but we're looking at continua and we're looking at a universe that is ultimately largely controlled by a handful of large tech companies due to economies of scale, due to network effects and the characteristic of particular business models in the context of tendencies toward monopolization. And I won't get into the economic structure there, but, you know, the net is there are a handful of these companies, they are largely based in the US or China, and they are the ones that really have the have the market, they have the technical capacity, and they got there first. And so we rely on them for a lot of these infrastructures and services. So whether it's facial recognition technologies, which, you know, may be sold by a defense contractor or a company to militaries, or it's large language models, which are being trained for commercial use, and then sold now, really rushing to be sold to militaries, but, you know, also used for a number of different commercial applications. And otherwise, these are all reliance on this surveillance business model, on this collection of data, so facial recognition would generally use publicly posted data, so you can think about Flickr, which was an early photo sharing site, I don't know if everyone is familiar with it, but it was, you know, it was online in the early internet, and it was a way for photographers and others to organize your digital photos. And this was well before the machine learning and AI boom of today, so there was no thought you could have had that, like, these photos will become, will effectively impart my private biometric information to third parties who will use it to create a database of photos that is then linked to national ID photos that they're also using, that is then sold to militaries that will monitor me and my family at checkpoints, right? Or direct a drone, or direct a drone at me, or something, as we've seen in Gaza.
Sanam Yeah, exactly, exactly.
Meredith Whittaker The Gaza situation is really alarming to be diplomatic about it, because what we're seeing is an example of just how entangled these commercial surveillance capabilities are with the asymmetric surveillance of various oppressed populations. And so Gaza is, and so Gaza is, of course, one of the, you know, it's a place where, it's one of the most surveilled places in Earth, the Palestinian people are some of the most surveilled people on Earth, and the machine learning and AI capabilities, whether it's lavender, or the gospel, or where's daddy, these technologies that have been reported on as targeting and identification technologies that have very much exacerbated the civilian death toll. (.) Those rely on that kind of surveillance data.
Sanam Can you just, what is, what is, where's daddy, or what, what is lavender? Like, what, what do they do? How does it work? What's the effect of it?
Meredith Whittaker These are machine learning and AI systems reported by Yuval Abraham in 972 Magazine. I want to make sure I'm being precise, so I actually would direct the listeners, and maybe in the show notes, we can include links here to the original reporting, because it's, it's pretty in-depth and long form. But these are systems, I've called them ad targeting for death. In the past, they have a similar logic. So for listeners who are involved in humanitarian issues, I'm, I assume you're familiar with a signature strike, introduced as part of the drone war during the Obama era, a method of targeting people for death, really, based on their data profile. So instead of knowing Sanam, who you are and what you've done and saying, you know, oh, Sanam, you know, Sanam, very bad. We don't want Sanam around. We're going to target Sanam because we know what she's done, our intelligence and shows us this. (.) We say, hey, this data profile of this person who has a podcast, who has worked in peace and conflict, who has this or that background, who is in these different group chats, who clicked on this link, who went through these checkpoints, who associates with this person who also has a data profile like that, that data profile, according to our model, is the data profile of someone with a high probability of being a terrorist. And so regardless, so based on that probability, we're going to kill that person. We don't know who they are. I don't know your name is Sanam. I don't know who your family is. I don't know, you know, you love cats, not dogs. I don't know anything about you. I know that this data profile, based on our assumptions meets a threshold, and then I kill you. And of course, to build that data profile, you need data. And that comes from the surveillance apparatus. It comes from location data, checkpoint data, WhatsApp group data, all of this, right? It becomes a sort of guilt without evidence based on mass surveillance that, importantly, only actors with surveillance capacities can actually deploy. Ukraine, for instance, couldn't deploy these systems in Russia because they don't have mass surveillance data of all of Russia's populations in order to build these systems, irrespective of whether these systems should exist or not, which under the international order, it seems pretty clear they certainly should not, but simply the capacity is asymmetric. And it is a capacity of those who are surveilling that can be deployed on surveilled populations. It is not a universal technology that exists for everyone to use.
Sanam And actually, this brings me to you and Signal. So what is it that Signal does which is different? And, you know, how do you maintain it, actually? Because it's a nonprofit foundation, right? So you're not working for profit, but how does it work? And how do you maintain the security that you guys have?
Meredith Whittaker I mean, Signal is like this beacon of light that shows tech can be done otherwise. That's my short description. But, you know, what we do is we are building a mass tech communications platform that effectively competes at the level of the big guys, the WhatsApp, the iMessage, etc., but does so in a really different way in that we collect as close to no data about you as possible. (.) And our commitment is to actually provide all of the features, all of the functionalities, but to do so in a way that is not contributing to this mass surveillance paradigm. And I think it's important to name that, you know, we're a nonprofit not because that's a nice charitable touch on top, right? We're a nonprofit because in a tech business model, sort of going back to the 90s and this sort of incentive structure that got put in place for surveillance, this tech business model is effectively built around collecting and monetizing data. Either you collect it and you use it to target ads or you use it to train AI models or you're providing goods and services to someone who does. So the assumption is that you're, you know, surveillance. And because there's no real business model in tech for privacy, it would be very dangerous to a project focused on privacy such as Signal to be a for profit with a board with a fiduciary duty who would have to push on that privacy guarantee and say, hey, you may need to weaken this because we have to be bringing in the revenue and our duty is to make sure that, you know, you can think a little bit about the board at Twitter when they made the sale to Elon, right? There are a number of people who went out publicly after us and like, I didn't want to, but ultimately, like as a director with a fiduciary duty, I didn't have a choice structurally. And that's what we need to avoid for Signal. And I think that's kind of the acid test that shows just how broken the fundamentals of the tech industry's economic structure are.
Sanam Kind of probably obvious to a lot of people, but I've been in the nonprofit sector forever. And the question of what is the board's responsibility? It's around governance and morality and integrity and keeping to the mission. And of course, fiduciary responsibility, but not profit related because we're a nonprofit. So when I constantly hear, well, you know, the bottom line is the most important thing. You're like, no, but there are other things. And what you're saying is, no, at the end of the day, the boards, you know, their primary goal or responsibility is to supposedly the shareholders. In a for-profit structure and in an industry where profit is made via surveillance. Exactly. What would you say if there are groups or people that are using your platform for, you know, I don't know, human trafficking or other kinds of criminal activity? How do you respond to that? Like, is there any way of kicking them out or keeping them out? Is that just one of the risks of having a communications app that anybody can use?
Meredith Whittaker Well, look, I think that question, I hear it a lot. And it's, it's kind of similar to, you know, if you ask the guys who maintain the autobahn, what do you think about bad trucks on your highways, right? Of course, we don't like it. I have strong views on the type of behavior and treatment of each other that is appropriate and not appropriate. But when we think about questions like that, I think we think a lot about how we can design more safely without ever being in a position to monitor or scan people's private communications, the technology that we use to ensure privacy, ensure that we are collecting as close to no data as possible, is encryption. And I won't go into this, but basically it uses complex math to guarantee that only the people in the conversation. So, you know, when we're messaging Sanam, it's only me and you, not signal, not law enforcement, not a hostile government, not anyone else can see those messages. So, that technology, it's a binary. It's actually black and white. It's the type of thing that policymakers hate because there's no, like, middle ground warm compromise. It's either hot or it is cold. It either works for everyone or it is broken for everyone. And we have ultimately undermined the sanctity of what we do. The way that we think about those types of harms, what about bad guys using this platform, is there's a lot of features we don't build so that it's harder to do certain things on the platform. So, we don't have channels. We don't have mass distribution of content. So, you know, kind of the dissemination of harmful content or what have you is not really possible on Signal. You can't go viral on Signal. We don't have a directory. So, you can't open up Signal and say, like, I want to search all people on Signal with the name Sanam and reach out with a DM and say hi. So, very different from Twitter or Telegram or what have you. And there are a lot of other ways we really maintain our platform as a pure interpersonal communications app. You can use it to talk, but it doesn't have social features and it doesn't have a lot of the features that enable self-contained harmful activity. You know, I think there are also questions about how much of the effective intervention for gnarly social problems like trafficking that often have to do with deep economic disenfranchisement, post-colonial legacies, a lack of work or ability to thrive in different contexts. There's a nexus of gnarly social issues there. I also want to be careful that we're not abstracting issues that we may not want to look at directly in the face online and thus assuming there is a technical answer for what are ultimately questions that we need to solve in the political and social realm.
Sanam And it's interesting because it's sometimes I feel as if when we're hearing things from the tech world, it's as if they think that through tech, we're going to solve all sorts of problems, whereas the reality is, as you say, it's much more complicated. There's social dynamics, cultural dynamics, etc. going on. And for me, one of the issues when I look at social media is that, you know, these algorithms that are in play, whether it's because they have been targeted to each of us individually, as in TikTok, the effect of it is polarizing, right? What I see about the U.S. elections or Trump or what's happening in Gaza can be radically different to even some of my closest friends, right? And so it's polarizing and atomizing us. Do you think that there is a way that that actually these platforms could be a force of good and helping with social cohesion and trying to help with kind of help, you know, being providing us with objective facts collectively as opposed to sort of stratifying and atomizing us? Do you think that there is a possibility that that could be done?
Meredith Whittaker It is entirely possible, right? My question is, how do you realign the economic incentives and how do you manufacture the political will when, you know, we're in a pretty tricky position now? There are about five major platforms in the Western world, largely, but well beyond that. And you have a handful from China and only one is not jurisdictions outside of the U.S. And these are kind of the gatekeepers to the information environment for billions and billions and billions of people. That's a very enticing target for anyone in power who might want to take a hand at information controls or shaping public opinion or what have you. You know, when I think about what would be better instead, I think there are a lot of alternatives. And anyone who's a kind of early Internet person will remember a teeming ecosystem of blogs and kind of special interest groups. And, you know, that was actually very, very vibrant. You can get on somebody's blog where they were meticulously documenting, say, something, you know, the movement of some particular European law. And then you'd see in the comments, there are all these people who are very interested in that, you know, it was the opposite of siloed. You were able to sort of dip into different realms of expertise that were heterogeneous and informed and in some sense, I think, kept in check by a sort of expert community that was arguing and adding and what have you. And this was kind of a small Internet and then you would use an RSS feeder and you would, you know, have have all of those organized. And it was it was a very vibrant ecosystem that got, I don't know, I think of like pouring bleach in a coral reef, right, is the sort of violent metaphor that comes to mind with the platforms sort of scooping up the heterogeneity and diversity. And I think, you know, healthy version of various internet news and information properties into these homogenous, you know, massive platforms that ultimately have a single point of control, a single point of failure, a decision around what is and is not, you know, criminalized speech, which is really dangerous at a time when things like reproductive health care and access to trans literature and what have you are becoming criminalized in places like the US.
Sanam So this brings me to the sort of the question that, you know, I ask everyone on these podcasts is, so if you were in charge, what would you do differently, you know, and I think about it in terms of policy legislation, but right now looking at the future, what would you do?
Meredith Whittaker Part of what I do is just assemble a team of real experts to help make that map, right? I think on the broad sweeping approach, I think there needs to be, you know, really strong federal privacy laws that would, you know, begin to curb some of the issues we're seeing now. And yes, I think it would require vastly scaling back a lot of these platform business models, a lot of these paradigms, but that that would be something that is, I think, necessary. I would look at tools like structural separation to put it colloquially decouple the kind of infrastructure from the application layer across a number of these firms. You know, right now we have just vertically integrated monopolies controlling global tech ecosystems at the infrastructure information and, you know, surveillance and data levels. And I just don't think that's safe or advisable for all sorts of reasons for the, for the, for the non techie, what does that, you know, what, what would that mean? And for somebody like me, I would mean breaking up these companies and what breaking up actually means is a, is a delicate issue in it. You know, you have a handful of companies, but they're largely based in the U S and they have global reach that is very, very difficult to replicate. So if I want to deploy a service, you know, like signal where you can make a phone call from Kingston, Jamaica to Phuket, Thailand, I need to have server capacity across those geographies, right? (.) I can't just, you know, have one server placed in the U S or one server in Kingston. That's not going to actually facilitate that real time communications, video streaming, et cetera. The type of infrastructures we're talking about are global in reach, incredibly expensive to build and maintain. And, you know, it's difficult to think of like a single jurisdiction, simply snapping their hands and then building and maintaining an alternative. So there are real questions, I think, kind of get us back to the literature of public utilities and managed telecom monopolies. When we're talking about network effects and economies of scale like this, when we're talking about services like communications or even interoperable networks, where we all at some level need to be speaking the same language, using the same protocols, linking to each other on the same rules, right? We don't want to have to buy like eight different telephones because each telephone uses a different protocol and a different network, right? But if there's only one telephone, then we need to be managing and overseeing that network with a lot of care so that it's not weaponized, so that the centralized control of whoever is managing that isn't turned against us. And that's, you know, through the history of telecom infrastructure, starting with the telegraph, there's been real questions in government and statecraft around, you know, how do you manage something like this? How do we make sure that it isn't simply in the hands of its owner doing their bidding like Western Union in the late 19th century that was refusing to send telegraphs if they were being sent by a political campaign they didn't endorse, right? Like, that's the issue we're seeing. However, we can't have 80 different telegraph networks that don't interoperate because that's actually not useful. So there are real questions there that people with more expertise, I think, would be tasked to answer. But that's a key problem, but that problem at a global scale. And then we have these actors that have massive amounts of data, and I think data that can easily be weaponized, that is not always even accurate, that is not always even fungible, but that nonetheless is giving them the authority and power to define our worlds in ways that are very dangerous. I almost would want to bring in people who have deep expertise and disarmament processes, right? Like, how do you draw down from something? How do you stop doing a thing? How do you, you know, melt the databases back into plowshares or what have you so that we're building a world that has less of a vulnerability in the form of the kind of surveillance that can be weaponized by those in power?
Sanam It's going back to some of the foundations of the entire system and structures from the question of laws around selling our privacy data to all the structures. What is the advice that you would give to people around what they should or shouldn't be doing?
Meredith Whittaker Well, I want to, you know, before I sort of give individualized advice, I do want to say there's kind of an optimistic side to this as well, because I think in the, you know, in the shadow of this massive, tall monolith that is the tech industry and the handful of giants, there's a lot of actually interesting and innovative things that just haven't been given the attention or the capital. And, you know, in part, that's because people just don't know how much tech costs, right? Signal is about $50 million a year to run. And that's an incredibly lean tech organization because we have to pay for servers. We have to pay for bandwidth. We have to pay for very good engineers. You know, all of that is a cost center. And so I think there is an opening in this moment as people begin to realize just the stakes here, which I think is truly happening in a way that I haven't seen in the past to actually marshal the capital and political will to do tech that is much more interesting. And, you know, again, kind of enables real creativity and innovation of the kind I saw some of in my early career, you know, when I was entering tech in 2006. But that seems to have been ultimately just overshadowed. The analogy I think of is just like a giant skyscraper that won't let any light into the garden. And we really need to sort of clear the airway so that we get light back in and can begin to build an ecosystem. And I think young people can think and participate in that. There's actually exciting work to do in tech. It's not just a money job or there could be if we can really push on this. And there are things like Signal that I think just prove that, right? We are one of the most innovative organizations in tech, full stop. We rewrite the stack. We rewrite core functions in order to ensure that they can be done actually privately. We are proof that it doesn't have to be this way and that actually doing it a different way is exciting and gives you really interesting problems to work on. And there should be many, many, many different tech organizations like Signal. And there really could be. But, you know, we have to tackle both of these problems at once. We have to clear the air so we can get some sun in. And then we have to build.
Sanam If you were a billionaire, like if you had the resources that Elon Musk has or others, what would you be doing now? How would you deploy that right now?
Meredith Whittaker If I had those resources, I had not prepped for this question. And now it's going to haunt me that I didn't think of, like, the 8 million different answers I could have given. But I think there is, you know, we should be, you know, one thing I would do is I would just quickly, you know, set up an endowment for Signal so that we could have a perpetual motion machine of sustainability. And we're not, we're not doing sort of yearly fundraising to support the nervous system of private communications around the world, right? Like core infrastructure should be sustainably funded, full stop. This is a, you know, again, it's a weird product of this broken business model that this exists this way. I would be looking at research into, you know, safe structural separation. I would be, you know, there's so much work that I would be funding. I would be setting up, honestly, alternative infrastructures for news and academic work, kind of outside of some of those institutions that I think are not doing very well, often as a consequence of some of these factors.
Sanam There's, you know, that's a really interesting point because one of the things that I see and I worry about is that actually in my universe where we're talking about, you know, women's experiences or women's work in peace and security, which is so badly funded and so invisible already with some of the AI, you know, like chat, GPT and so forth. It's just simple things when I ask, you know, just like saying, what's a peace builder, what is, you know, where is the, how does it answer these questions, we're becoming even more invisible and it's like, you know, it's kind of erasing our existence. So you, you kind of have an existential moment when, when you, when you realize that you just don't exist in this universe that's being created. So, yeah, so having...
Meredith Whittaker And that universe that's being created is created out of the data that's been deposited on the accessible internet and in the private archives of these companies over the last 20, 30 years. These are ultimately like aggregates and if, if you weren't part central to that discussion, then, you know, these prediction machines are not going to predict that peace builder would be, will not know who you are. If you're not in the archive, you will not be known and the archive is this sort of random cache of surveillance data that was largely created in English, largely created by men, et cetera, et cetera. Over the past, you know, 20, 30 years, it is not the sum total of human knowledge, right? It's whatever got deposited on the internet by somebody, you know, who used to be able to write HTML and then like had a blog, which meant they had access to, you know, power and internet connectivity and technical training in some cases and a laptop and, you know, a bedroom door that had a lock on it. So their mom couldn't come in so they could, you know, program, you know, whatever it is, right? They, it's, it's very highly contingent and again, these AI systems are not magic, right? They're taking all of this data and then they're predicting the next word in a sentence and if peace builder isn't in many sentences that they've seen, they're not going to predict it.
Sanam And as you say, I mean, all, all the, all the conditions you said, it's 90% of the time, it's not going to be women because they're not necessarily doing the writing. They're not necessarily having the access to the technical, you know, knowledge or the laptop or even the electricity, certainly in the places that, that we work where it's war zones and so forth. So, so it's sort of invisible from the beginning, but they, they exist in the real world, but they're, they're completely erased out of, or they never made it into this universe, right?
Meredith Whittaker That's, that's the, and then what we're doing is naturalizing what is a historically contingent erasure as proper as the way things are, right? The smart computer says, this is the answer. This must be the answer, let alone all the indigenous knowledge and so forth that also never probably made it here. And that raises the question, do we want that knowledge there, right? Because ultimately, if we're talking about GPT, that's a Microsoft product that is pulling that data back into a centralized, you know, model or, or archive that is in the hands of one of these handful of companies that is based in the U.S.
Sanam U.S. election results, Elon Musk making it into the government, Elon Musk putting so much money into, into the political space. In fact, this year we saw the highest percentage of billionaires kind of affecting the politics, some obviously on the democratic sides, but some, a lot of it on, on the Republican side and the tech universe coming in on the side of Trump. Does it keep you sleep, sleepless at night? Like what do you see coming into the future in the next four years? (..)
Meredith Whittaker I'm going to give a flip answer to this. I, for whatever reason, I'm the best sleeper I know. And, you know, I just continue to get my rest because ultimately I have to get up every day and I have to make sure that Signal thrives and survives and grows over the next 5, 10, 20, 40 years. That's, that is my role here. And so, you know, I do think, you know, we should have repealed Citizens United. And there's a lot of, there's a lot of prior art here that needs to be examined as the cracks in some of these electoral systems and democratic processes are laid bare. But, you know, what this has done is just really focus my attention and galvanize my commitment, you know, ensuring that truly private global communications remain accessible and available to everyone in the world.
Sanam And that's, that's, that's a huge contribution that, um, that you're making. Thank you. Cause we definitely use Signal. Yeah. And then.
Meredith Whittaker Well, I'm, I'm glad to hear that.
Sanam Thank you. And, and then finally, um, what gives you hope? (...)
Meredith Whittaker Oh, well, you know, so much actually does give me hope. I think working on Signal, I have the privilege of meeting a lot of people who rely on Signal and just seeing, seeing the intelligence and dedication that are happening across, across sectors from, you know, journalists who are using this to, you know, report on, really important issues to, you know, the human rights and peace building work that, that you do, Sanam. I think, you know, hope for me is a sort of the stippled picture that comes together in aggregate when you realize that once you get away from the feeds and the screens and, you know, whatever, and actually are in community with people, talking to people, most people are really good. And most people really want good things and, you know, whatever systemic issues we are, are struggling under, whatever, you know, handful of pathologies seem to have infected different sectors or, you know, different governments. There is still a mass majority of people who are good human beings who want a livable future and a better life for everyone. And I think, you know, all right, so how do we empower them? How do we make sure that they're leading? And that's, that's kind of a thought that I, I keep with me through it all.
Sanam Thank you. That's, that's, it resonates enormously with me as well because I, I sometimes, you know, I look at my, the network that I work with and sometimes we are, we are the light in the tunnel and, and, you know, we're giving them hope when they're having a dark moment. And then other times I look and I'm like, you know, if they can do the work that they're doing in Nigeria or Cameroon or Yemen under the circumstances of, you know, violence and poverty and, and war, but still focus on peace, then, then that inspires me to, to do, to do the bit that I can as well. And I, and I think you're right. It's kind of, you know, holding each other's hands or seeing each other's light in the, in the noise and the fury and all the, you know, the rubble that's out there. But yeah, it's, it's, it's, it's the people.
Meredith Whittaker And, and thank you for yours.
Sanam Good morning, sunshine. How are you doing this morning? (.)
Soleh I'm good. You know, it's early. Did you have a busy weekend? All the cousins came up for formal, which was fun, you know, in the kind of old hall. Okay.
Sanam And, and do, do I dare ask you about the bacteria this week? Anything, anything exciting happening? (..)
Soleh No.
Sanam Do they look different?
Soleh No. Why would they look different?
Sanam I don't know. Sometimes they turn pink or green or something. No, no, no. Nothing. All right. Okay. So our conversation today was about AI and issues of surveillance and then issues of, you know, how technology is being used in terms of ongoing violence in Gaza. What was your reaction? Were there sort of questions you wish I'd asked? Did I sound like a complete Luddite as I was talking to her?
Soleh I think one thing that was particularly interesting or that, like, you guys talked about quite a bit was the, like, profit structures, which I think is something that people don't always think about. And I didn't know that Signal was non-profit, maybe. I mean, I don't know that much about Signal generally, but, like, it makes sense, I guess, that in order to be private, like, they have to be non-profit because they're not obliged to sell data and all that to keep the business going because it's not a business in the same way.
Sanam We've all sort of taken it for granted that these services are free, but they're not really free. And she's explaining why they're not really free. It's because we're, in a way, the commodity or our information is the commodity that gets sold one way or another, I guess. When you think about, I mean, you know, again, generationally, you know, we so often hear young people saying, well, what does it matter? You know, why do I need privacy? Like, you know, everything is out there. People are posting their pictures and so forth.
Soleh I don't think people think, like, why do I need privacy? I think the flippant, like, kind of comments come from the fact that we all know that there's not really any way to actually be private.
Sanam I see. So you feel as if you're under surveillance anyway.
Soleh There's not much that you can do as an individual to, like, actually be private, right? Like, in the example that she gives with the mother and daughter who were communicating about, you know, getting an abortion, which is just a horrible example, but there might be, like, incidences where you think about, like, messages needing to be private, but, like, the idea that I could actually make my entire, like, that I could take myself off the internet internet is completely absurd, right? You know, at this point, what's the difference? Like, I had friends in undergrad or, you know, I knew people in undergrad who, you know, would have, you had to, like, plug two things into the laptop in order to then get access to the laptop and then it could only, like, when the second thing was plugged in would it even connect to the internet type of thing. Like, you can, sure, you can do these things to stay private, but who's doing that? I don't even know what the things they were plugging in did. But I think that's where the flippant thing comes from. Like, it's not, or the kind of disregard that it sounds like our generation has. It's not because we're not concerned about privacy. It's because there's nothing to do.
Sanam What else did you find interesting or different or new in terms of what she was saying?
Soleh For sure, what she was talking about, these kind of profiles that they make and then you don't need to know anything about them. Like, actually, they just need to fit a profile. Like, their data needs to fit a profile and that's sufficient. And also her point about how that's only something that it's always, it's only the bigger power who's able to do it. So, like, Ukraine probably isn't able to use those kinds of things, whereas, like,
Sanam Russia might be able to, you know, that technology is so scary because it might be in the hands of the U.S. and the Israelis now, but it's probably going to make its way out. But from a surveillance standpoint, as she was speaking, I didn't get a chance really to sort of get into it with her. But I was thinking about, you know, how the Chinese have all these pervasive ways doing facial recognition detection. In Iran, they, you know, people talk about how you don't get stopped for, you know, let's say if you're driving and you don't have your scarf on or if you go to a protest, you don't necessarily get stopped there and then. It's just that there is surveillance and facial recognition stuff going on and three days later they can just come knock on your door or they fine you through your, you know, through your electronic banking system and your, I don't know, electronic ID systems for not wearing your scarf properly and, you know, in a car. So it's kind of 24-7 surveillance in such an insidious way. I imagine, as I say, once it gets out, it's going to be in the hands of as many, you know, both state and non-state actors going forward, which is just, you know, it's pretty dystopic.
Soleh I mean, yeah, I mean, in that sense, what I was saying before about not having to be that worried in a way about our privacy is then, like, also a privilege of the fact that we don't, I mean, we do live in a surveillance state in a way, but not in, like, a police state in these.
Sanam Okay, so on the one hand, you guys have a sense that there's, you know, the surveillance is there and there's very little that you can do. But are there ways in which you have conversations or with your friends that people even attempt to try and circumvent or try and sort of maintain some degree of privacy? Do you think, like, people are more conscious of what to put on, you know, even if it's photographs and things like that, is there an awareness?
Soleh I think people are conscious of the fact that, like, whatever you put online is there forever and is probably accessible to someone else. Back when there wasn't so much media literacy, I don't know what photographs people are putting, like, search history, that kind of stuff. So now people are very conscious of the fact that, like, yeah, you would never assume that anything that goes online is private. Yeah, even, like, incognito mode or the idea of disappearing messages or something like Snapchat. It's kind of, it's like, okay, it disappeared for you, but we all know that it's there somewhere, you know. (..) What did you find most interesting about your conversation? Or what did you, like, wish you talked about more?
Sanam I think the question of understanding the sort of fundamental commercial structure or the underlying sort of legal commercial structure was, was really interesting. That the explanation of how the surveillance is on that profiling question. So it's not that you, it's not that someone is necessarily proven to have done something wrong. It's just the assumption that, oh, because I'm, because they're talking to X and Y and Z and they happen to be in spot A and spot B, we are inferring that they are guilty and therefore we can take them out or we can go after them. That to me is so disturbing because it profoundly undermines the essence of you are innocent until proven guilty in the legal structures that we know. And it, and it, and it puts things on its head because in authoritarian states, um, the assumption is you are guilty, um, because if you were innocent, you wouldn't be in spot A or B or you wouldn't be talking to, you know, X, Y, and Z. So it, this, this idea that this has become normalized that, oh, because, because you happen to be somewhere and you happen to be talking to somebody, either you're guilty by association or you are associated with that person. And therefore you should, it's fine to take you out. Right. That, that I think is, is very, very dangerous. And of course we're seeing it in the context of both Gaza and, and, uh, Lebanon right now.
Soleh You know, at the end of the day, a program or some kind of data collection, whatever is still made by a person. So it's like to have such complete faith in like how we're sorting this data, you know, tracking someone that we say, because this program says that you are likely guilty. Like we trust it so completely that we can say like, for certainty you are. It's also scary because people like to think that these things are objective, but they're not. They're, they're coded by a person. Like it's, it's still made by, you know, with biases and all that kind of, I mean, obviously a surveillance program is made with biases, but like, you know what I mean?
Sanam And it's, and it's interesting you say that because I was in India a couple of years ago and, um, talking about AI and the question of coding and, and, you know, what they look for. And, and it's something like 70% of the people doing this are, are men and a lot of information. So on the one hand, you could argue, well, in a way it's good because it, you know, we could say we're, we're sort of still protected or some, you know, not as surveyed, or you could argue as I see in my work that it's even further erasing women's presence and work and real life experiences, um, and existence in the world. And, and, and her point again about, you know, AI or machine learning as, as I think she mentioned, um, is basically taking what's there, but if you, if you're coding and not being conscious of what's there, then you're again, further erasing histories. So, so that, yeah, so those parts of it, I thought were both interesting and just somewhat just overwhelming. I was like, how, you know, it feels as if we're constantly paying, playing catch up or if it's not on the internet, does it mean it doesn't exist? Clearly not, right? There's lots of things that don't exist on the internet, but, um. Pics or it didn't happen, mommy. Huh? Pics or it didn't happen. Pics or it didn't happen. Is that, what does that mean? What, you have to have a photograph? I was like, yeah, exactly. So, I mean, last night I was thinking about you because we were watching a, a documentary about Cadbury's chocolate and, and they were showing how in the old days it would just be like, you know, many of the boxes would just have plain chocolate and now everything's wrapped up in plastic. And so, like, what happens to all of that bloody plastic? So, Junjun and I were watching it together, thinking about you.
Soleh I went to see Wicked last night. That was fun, actually. You should go see it.
Sanam Okay. Yeah, yeah, Lyd said she wanted to see it. I, I was, I was aiming for Paddington in Peru, but I think I might watch that on a plane soon. So, all right. Okay, we've talked to you soon. Have a good week. //S00: Bye.// Bye. Bye. Miss you. Bye. (10 seconds pause)
This is an Africa digital audio production. My executive producer is Pierce Lynch and it contains original music by Little Monster Media. Remember, you can catch a new episode of If You Were In Charge every Tuesday. A huge thanks from me to the Bosch Academy for making these conversations possible and to my own organization, ICANN, for allowing me the time to explore these big issues with my guests and spend some time with my daughters. If you want to get in touch, you can find all our social media links on our website and the link is in the description of the episode. So, that's it for If You Were In Charge. I'm Sanam Naraghi Anderlini. See you next time. Bye.