Interview with Christopher Soghoian about Analyst’s Notebook
For our story about the Denver Police Department’s use of the surveillance analytics software program Analyst’s Notebook, we were happy to be able to speak by telephone with Christopher Soghoian, Principal Technologist with the American Civil Liberties Union. He answered some questions for us about the specific functions of the software, and helped contextualize its use by intelligence agencies, the military, and law enforcement.
Here is a transcript of the phone interview:
Unicorn Riot: What is Analyst’s Notebook? How is it used? When someone uses it, what capabilities does it give them?
Christopher Soghoian: IBM’s marketing materials probably do an even better job [answering this than I can.] From what we gather, Analyst’s Notebook is an essential tool used by many, many people in the intelligence community, in the military, and to a lesser extent, people in law enforcement, to make sense of large data sets. Whether it is credit card information, travel records, cell phone location records, calling records, emails, you know, if you have a pile of data, and you’re looking for the needle in the haystack, Analyst’s Notebook is going to be one of the tools that you use to try and understand relationships between people, figure out who they’re talking to, who they’re meeting with, who they’re collaborating with, and who they’re communicating with.
Unicorn Riot: NSA uses Analyst’s Notebook, and Analyst’s Notebook is specifically mentioned in the ACLU’s lawsuit against the NSA, can you say something about that?
Christopher Soghoian: Yeah, so about the expert brief in our lawsuit against the US government… you know the government’s position essentially is, all they’re collecting is metadata, all they’re collecting are records of who calls who, which isn’t sensitive stuff, so we wanted to help the court understand why metadata is sensitive. And part of that is to let the court understand what tools the government has at its disposal to make sense of metadata. One of the big problems with traditional wiretapping, whether listening to telephone calls or intercepting emails, is that it’s tough to make sense of them. You have to pay someone to listen to calls, even to transcribe them, you have to pay someone to read emails, and when your target says, “the package is being delivered today at midnight”, does that mean that FedEx is coming, or does that mean that there’s an attack? It’s very difficult to make sense of many intercepted communications. The thing for law enforcement and for the government more broadly, with metadata, is that metadata is very easy to analyze, and to do so with automated tools. Regardless of whether your target speaks English or Spanish or Pashto or Farsi or Mandarin, on every email there is a ‘From’ and a ‘To.’ On every telephone call, there is a ‘From’ and a ‘To’ and if it’s a cell phone, there are location records. And metadata can be analyzed in automated fashion in a way that regular wiretaps can’t. And more importantly, you can understand a hell of a lot of information about someone’s life with metadata alone. If someone receives a call from their OBGYN, then an hour later calls their mother, and then two hours after that calls Planned Parenthood, you have a pretty good idea of what’s going on, even without knowing the contents of what they say on the phone. If someone makes a three hour telephone call to a suicide hotline, does it really matter what they’re saying?
Unicorn Riot: Based on the research for the ACLU’s lawsuit against the NSA, we have some idea of how the NSA as a federal agency is using Analyst’s Notebook. What do you know about local police departments using this software, are they using it in the same way?
Christopher Soghoian: What we know really comes from things like LinkedIn. From what we do know about Analyst’s Notebook, it’s pretty expensive, and so its probably gonna be too expensive for many state and local law enforcement agencies. Its the kind of thing that you would expect a state fusion center to have, but not a local police department to have. As for what are they doing with it, certainly I would imagine the intelligence community can do more, simply because it”s easier for them to get records. The intelligence community can get records of every person who flies anywhere. They can get international telephone records. They can get international email records. They can get financial records. They can get all of these records and then put them together. Whereas law enforcement is probably going to have to seek individual records about individual people. If you’re looking for the needle in the haystack, it helps to have the whole haystack, and the intelligence community, more than law enforcement, has easy access to that whole haystack.
Unicorn Riot: How does this overlap with what’s called “open source intelligence” Like social media, Facebook, Twitter, etc… we know that the police in Denver will monitor social media to figure out who is planning certain protests which then informs how they plan and prepare…
Christopher Soghoian: Certainly these kinds of tools can pull in open source data sets, including social media and all those things.
Unicorn Riot: You mentioned that the software was really expensive, and it was something more likely for a fusion center to have than a local law enforcement agency to have. Actually, the resume that we found that led us to look into Analyst’s Notebook, a Denver Police crime analyst named Nathaniel Niles, who left the FBI to come work for Denver Police doing work using this software. Also some of the Analyst’s Notebook documents that we have mention “CopLink”, a connection to this IBM CopLink database, which is information sharing between statewide law enforcement. And CopLink is connected to the Colorado Information Sharing Consortium, which isn’t actually the Colorado fusion center which is a different entity, but seems like it serves a similar function.
Reveal the truth about surveillance tech: Use your cursor or touchscreen to scratch off the censored software package on the link below!
Christopher Soghoian: Here’s the thing with all of these tools, right – if you have a tool that helps you make sense of large amounts of data, you’re going to then want large amounts of data. These tools are not very useful if you only have one person’s telephone records. So, they’re going to be the most useful to those agencies that either have access to large amounts of data by themselves, or can get it through sharing data with other agencies.
Unicorn Riot: Some of the product literature for Analyst’s Notebook specifically mentions the connection to Cellebrite (Cellebrite has been in the news lately).
Christopher Soghoian: It makes sense. So let’s say you’re an army intelligence officer in Iraq, and you have at your disposal records from every telephone call made by anyone in the country. Your special forces team go out, they do a midnight raid, they go to someone’s house, they shake them down and they find a cellphone in their pocket and they hook up a Cellebrite device to it, and they dump the records of who that person has called. With Analyst’s Notebook you can then look up the call records of all the people that person has called, and where they’ve been. It makes sense that you’re going to want to have the ability to import data from varying sources. One of the sort of pieces of software like this is just special software to import things in different formats. You would imagine that Visa and Mastercard are gonna give you data in a different format than American Airlines. You’re going to need custom software that can import the data regardless of which format it’s coming in, and where it’s coming from.
Unicorn Riot: Yeah, some of the other Denver Police emails we have of them discussing the purchase of the newest version of Analyst’s Notebook, they specifically mention Analyst’s Notebook SDK, which is “Software Developer Kit”, to customize their own additions to it. Why I asked about Cellebrite specifically is that we also know that the Denver Police Department uses Cellebrite, we have their policy on it, and it was recently updated after the Supreme Court ruling that a warrant was required to search mobile devices. They claim that they adhere to that decision in their new policy, it says a warrant is preferred but not required. I’m imagining hypothetically – since this investigation is tied to a bigger picture attempt to learn how the DPD relates to protests, how they surveil and monitor different groups of protesters – so I’m imagining, before that Supreme Court case, people are arrested at a demonstration, charged with obstructing the street, in a mass arrest. Hypothetically, would information such as metadata from their cell phones be extracted via Cellebrite, and fed into Analyst’s Notebook for further analysis?
Christopher Soghoian: You know, I would imagine that while the rules for when they have searched have changed, their policies for “do we hang onto the data for as long as we can and then search it in other cases?” have not changed. I would imagine that many agencies feel that once they have data, regardless of how they got it, they can use it and search it as many times as they want n the future.
Unicorn Riot: Yeah, a lot of us were arrested during the Occupy demonstrations, in a fairly classic mass arrest scenario, I didn’t encrypt my phone back then, I wasn’t at the time educated about privacy techniques, most people who I know who were arrested during that time period weren’t either – it’s only since Snowden that we’ve really given a fuck. I don’t know if we can say for sure that this is happening in terms of protest arrestees phone data being put into Analyst’s Notebook, but it seems entirely possible given what we know so far.
Christopher Soghoian: There was an article in one of the Virginia newspapers last year I think talking about how the authorities in, I want to say, Hampton Roads, Virginia had signed a Memorandum of Understanding where in every investigation where data was taken from a phone, they would dump that into a central database that all the police departments would share and have access to. They were dumping this into a central database and then other agencies could search it. And again if you’re using a tool like this, like PenLink or like Analyst’s Notebook, the more data you have, the happier you’re gonna be. These kinds of tools in many ways incentivize law enforcement agencies to want to hang onto stuff longer than they need, to search it in other cases, and then to share it with other agencies. And the more data you have the more likely that you are to stumble across something later.
Unicorn Riot: And that connects to the whole fusion center model of like, “if we get all the hay, we’ll find more needles”, or something
Christopher Soghoian: “If we get more hay, we’ll find more needles.”
Unicorn Riot: Are you aware of any specific examples of law enforcement using Analyst’s Notebook? We have an email from the IBM sales rep who says what the Denver Police Department mainly uses it for, but that part of the email is redacted.
Christopher Soghoian: I don’t know, again, I’ve seen Analyst’s Notebook mentioned on some LinkedIn profiles before, but really that’s all I’ve seen. In many ways this is a great find on your part, and hopefully this will inspire future open records requests from people around the country. So good work.
Unicorn Riot: Thanks! The way we see it, MC McGrath & Transparency Toolkit deserve all the credit since we found this scoop via ICWatch
Christopher Soghoian is the Principal Technologist with the American Civil Liberties Union. You can see his work on his website or follow him on twitter at @csoghoian.
Unicorn Riot reporter Dan Feidt recently caught up with MC McGrath at the #LibrePlanet conference and interviewed him about the ICWatch project. Watch it here.
Also, watch MC’s #LibrePlanet talk about ICWatch – “Taking back our freedom: free software for sousveillance”
To help our volunteer-operated, horizontally-organized, non-profit media collective please consider a tax-deductible donation:
