Ian Todd: Hello and welcome to the third episode of the new realities of cyber security podcast series. My name is Ian Todd, I’m a Data Privacy and Cyber Security Consultant here at PwC. Today I’m joined by Kris McConkey, Cyber Security Partner, to discuss Digital Crime Scene Forensics. When you think of Cyber Security, you’ll think of the fascinating work Kris and his team do on a daily basis. We will talk about chasing down the bad guys, setting up online traps and carrying out digital crime scene investigations. Kris thank you so much for joining us in the podcast. I think this is a really exciting area and when I think people think about cyber security, what you do is what people imagine, what people see in the movies and it’s the kind of fun part of cyber so if we can get a little bit of introduction from yourself and a little bit about your area of expertise.
Kris McConkey: Sure thing, probably better talking about the expertise as a team. You wouldn’t spend too long if you were just focused on mine. I guess probably at very high level the team is a pretty kind of hardcore squad of geeks and like you said this is often viewed as the sexy side of cyber security and I think to a large degree the team often think of themselves as almost doing kind of digital hand to hand combat with the bad guys in some shape or form. So the three main things that we are focused on is what we call threat intelligence. It’s basically who are the bad guys, what are they doing, who are they targeting, what tools are they developing to do that, what data are they trying to get, what are they doing with it, all that sort of stuff, and that’s generally so that organisations can understand why they are being targeted or what to watch out for, what’s making them a target but also how to defend themselves against that type of activity. The second area is really putting a lot of that intelligence into practice and helping clients find evidence of some sort of malicious activity on their networks. It might be an attacker using the same credentials as the IT admins do to move around the network, it might be malware, it might be an insider taking loads of data out and the third bit, which is probably where a lot of our heritage is in the UK because we’ve been doing it since about 1998, is in the incident response and digital forensics side of things so for the most part we act as the emergency panic button for clients whenever they get hacked we’re the team that helps them figure out what happened and what they need to do to stop it and resolve the issue.
Ian: Cool, I think we talked briefly on one of the other podcasts actually with Charlie McMurdie and she said that was part of breach aid and your name was brought up there. Like you say when the emergency button is hit, you guys come in, in parachutes and try and help the organisations out.
Kris: Yes so the whole breach aid concept is really bringing together the full spectrum of stuff that you need in the midst of a crisis because the technical side of it is just one bit and it might often be the first bit through the door because you obviously need to know what’s going on in order to figure out how to deal with it but helping a client, for example, navigate the decision making process in the midst of a crisis, so do we take systems offline, impact customer or service availability, or do we keep stuff online even though it might be compromised, who do we tell, what do we tell them, when do we tell them, all of those types of decisions, it’s really important to have that kind of broader crisis management expertise and/or the legal and regulatory side of it as well, which as you will know from all the data protection, EU and GDPR stuff is hugely important.
Ian: Yes, it’s such a complex beast to try and react when something goes wrong. So I think one of the key things I had in my mind is around forensics and when I think of forensics I instantly think of CSI and fingerprints and this is the kind of thing I think about. Am I a million miles away thinking that from a digital perspective, when you go into a crime scene or into a breach, are you looking for the same things that we would look for in the physical world in a digital environment?
Kris: Yes it’s really about following the trail of digital breadcrumbs in some shape or form. So like you described a sort of digital crime scene I guess in most cases you kind of have a thread to pull on so in a physical crime scene you would start with a dead body, you know that’s there and you have to work backwards as to what happened, how did it happen, who did it, etc. The two common scenarios that we get called into in the digital world, one is you have something that you can work from so either a system that you know is compromised, or you know a specific piece of data has been lost and found itself on Paste Bin and that gives you a starting point so it’s all about piecing together the little breadcrumbs of evidence across an organisation to put together the full fact pattern of what’s happened. The other side of things is where, probably the last 5/6 years, a lot of the Government agencies have been much more proactive in notifying companies when they’ve had a problem, but the amount of information they get to act on is actually pretty limited so what you’ll usually get is some sort of notification that says “at some point you’ve had a system somewhere on your network that’s communicated with this domain, it’s bad, you should look at it”. Now that actually doesn’t give clients an awful lot to go because they don’t have a huge amount of historical log data to go and analyse, so we end up knowing there’s a crime scene somewhere in the company’s network but that network might span 50 countries and have 200,000 systems on it and so we had to build a lot of different technology approaches and analytical approaches to basically sweep networks to find out, ok where is the crime scene and then start working from there.
Ian: Incredible. And how do you start? Like you say, a crime scene in the physical sense is there’s a dead body there, let’s cordon this area off and we’ll start, but when you start doing that over the entire globe, where do you realistically start? How do you start that?
Kris: It varies a little bit by each client, so basically if you’re looking for evidence you’ve mainly got 3 sources of it, you’ve got either something on the end points so like processors that are running like malware that are on the actual servers and user systems round the enterprise, you’ve got evidence of something in network traffic so really data in motion and that might be data being taken out of the enterprise, it might be malware commanding control traffic in the networks and then you’ve got historical evidence of stuff that you get in firewall logs and DNS logs, proxy logs, all of those sorts of things. And so quite often our focus is on getting as much visibility across those 3 sources as we can and then applying all of our intelligence about what we know is bad and a lot of statistical anomaly detection methods and things to figure out where do we focus down on? And as you described in the physical crime scene you actually want to cordon stuff off, it’s actually reasonably similar to that. If you’ve got a malicious hacker inside a network that has control of the network, part of your containment strategy in terms of kicking them out of the network is really how do you cordon off areas of it and contain them into little areas so that you can then remediate what is happening in that area and limit their ability to get into other bits again.
Ian: Fascinating. A question I think about when I’m listening to you talk there is how many organisations don’t know that these crime scenes exist? I imagine that must be a big issue as well, they just don’t know this murder has happened somewhere if that makes sense?
Kris: Absolutely, so some of the stamps from the last few years, I think this was probably a few years old but the one that has always stuck in my mind was something in the region of 94% of organisations were notified about security incidents by third parties, they hadn’t actually spotted it themselves, so I think what’s probably changed over the last few years if you look at some of the metrics that have come out in various annual reports, is that organisations are generally getting better at finding incidents themselves but the containment time, so the actual time to resolve the unauthorised access to the network or the IT asset, is actually going up so the bad guys are actually getting better at withstanding attempts by the company to get rid of them.
Ian: Right. So I guess that leads us quite nicely to the next part. So I guess the fundamental theme of this podcast is about the baddies and the good guys. So what are we seeing from the bad guys right now, what are they using, what are they trying to do, what is their motivation right now?
Kris: So I guess, well let’s start with the motivation piece because that’s probably easiest to break down into a few categories. We generally break it into 4/5, depending on which way we’re thinking about it. The way I like doing is probably focus on 4 with the 5th potentially being part of all of them. So you’ve got the very standard organised crime, anybody that’s focused on getting something they can monetise quickly, that’s everything from your personal online banking credentials, the latest dry decks and spam runs try to pick up through to what happened at the bank in Bangladesh where they’ve got 10’s of millions of unauthorised wire transfers being conducted by somebody that’s actually inside the network that shouldn’t be there. That predominantly affects financial services probably more than other sectors, it does affect everybody but financial services obviously get hit most heavily because the bad guys always follow where the money is, so that’s one category. Another is around sabotage and that is, in some cases, can a nation state capability if you look at the very old but very relevant Stuxnet example first kind of disruptive thing that was really observed. I think that’s probably the case now a little bit with some other nation states and their ability to sabotage things if they so needed but the instances of that are reasonably few around the world. You’ve got hacktivist side of things which is really doing anything that’s disruptive and embarrassing and that’s usually because a company is acting in, well they have a different kind of stance compared to some group’s ideological view on something.
Ian: We’re going to see more of this aren’t we, I guess? It seems in the media we are around, anonymous I guess would be potentially in that category and we saw the big Ashley Madison hack or insider threat or whatever that might have been, but there was a lot of talk around that as well and I guess like you say, these are growing in the media it seems, their presence seems to be growing.
Kris: Absolutely and they’re probably the ones that get the most media attention because that’s the rule for all goal, they want to draw attention to stuff so anything they can do to take over an organisations Twitter account, deface their website, leak emails, all of that sort of stuff is just designed to get publicity. It’s usually fairly effective. And then the fourth category is espionage. That might be nation state espionage focused on intellectual property or merger and acquisition deal information, those types of things, it could also be competitors seeking an unfair advantage in something. But across all of those you’ve already alluded to the term insider threat, there is the potential for insiders to be involved as well.
Ian: So what are we seeing as the attack vessels if that’s the right term. What are the bad guys using to get into the castle? What are they doing right now?
Kris: So a few fairly standard things. Nothing has really changed in this space for the last while and the bad guys usually come up with little tweaks on some things but the general principles are generally the same. You’ve either got individuals just being targeted through spear fishing and that’s either to collect credentials that can then be used for something else, or to drop some kind of malware that gives attackers a foothold on a network so they can move around and find whatever data they’re interested in and get it out.
Ian: And that’s an email that’s focused on a particular person, so a spear phishing attack may be something to a Partner at PwC and they’ll look directly and aim at that person.
Kris: Absolutely and that’s a great example of that from recent history that’s been really prevalent over the last 12 months is what’s called BEC fraud, Business Email Compromise, and anybody in a big organisation has probably seen emails appearing to come from their CFO to probably some overworked person in the finance function saying “hey we’re just about to close an urgent deal, we need you to make this wire transfer payment today” and people, because they’re getting an email from the CFO circumvent the normal approval processes, they’re already overworked and are like “I have to get this done as quickly as possible” and will suddenly wire a couple of hundred thousand pounds out to somebody, and the FBI did a pretty good report on this a few months ago which estimated that several billion has been lost to those types of …
Ian: A very, very basic
Kris: Basic confident scams in the last 12 months. So that’s people focused on getting into specific users’ systems. On the other side of things you’ve got attackers that are trying to come in via the perimeter so they’ll obviously look at vulnerable web servers, email servers, those types of things, get into them and then either get data straight off their systems or somehow burrow their way into the network a bit further from there.
Ian: Incredible. So I think we’ve got a bit of a flavour for the bad guys. What are the good guys doing? I think the perception people have is that the bad guys are always one step ahead and we’re always chasing. So how do we keep up with them if they’re sprinting ahead and what are the good guys doing?
Kris: Great question. If you start at a very simple level there’s stuff that we could bang on about which everybody already knows, and which is just generally quite difficult to do in very large organisations and that is stuff that actually mitigates a huge amount of just general attacks but also quite targeted ones. And it’s basic stuff. It’s keeping applications and operating systems up to date and patched, it’s limiting administrative privileges so that not every user has the ability to do whatever they want in the network.
Ian: That helps the insider threat. That minimises that a bit?
Kris: Absolutely, it just exercises a bit more control over what people can access and the fourth is around application whitelisting. So rather than trying to block bad stuff, only let stuff that you know is good and is trusted, that you want to be in your environment actually run. And so there’s a really good paper released a few years ago by the Australian Signals Directorate which is Australia’s version of GCHQ, and they studied about 1,000 security incidents that they had investigated and worked out what controls would have mitigated those, and they came up with a list of 35 that would have mitigated pretty much everything but the top 4 of those which are the ones I’ve just mentioned I think they calculated between 80% and 90% of stuff those would have deflected. And so that’s the kind of hygiene side of things, if you can’t do that stuff really well - there will always be little gaps somewhere. In most organisations there are very good reasons why they can’t do all of those very well in their environment. You’ve then got the concept of threat intelligence which is kind of working out who is likely to target you and why, and want bits of your business and the things you are doing put you at risk, and then how do you protect bits of data systems that the bad guys are going to be interested in and then the other, well there’s lots of strategies, but the other thing that has re-emerged recently is the concept of deception and so a lot of people are talking about deception honeypots and things inside organisations which is really part of a broader active defence strategy, making your own network a hostile place for an attacker to operate in and I described it recently as almost like a house of mirrors so whenever you come through the front door, you want an attacker to really not know where they are or whether a system is real that they’re looking at or whether it’s a fake one, and so there are more and more things popping up in that space as well.
Ian: Fascinating. So I think of honeypots as it attracts people into that area. Is that what you’re doing? You’re intentionally diverting their attention to a place where you’ve got more control and you can monitor over, is that what you’re doing?
Kris: Systems that look like ones that attackers are interested in but are actually fake so as soon as an attacker touches them it will raise an alarm somewhere so nobody should in a legitimate business be touching those systems. If somebody is trying to access them it’s probably a bad guy on the inside.
Ian: Interesting there are little trip wires there.
Kris: That’s exactly the concept.
Ian: Great stuff. So one of the things I’ve heard you talk about is sinkholes and I guess this kind of goes in with the idea of honeypot so can you give me a little bit of an idea what a sinkhole is and what you are using them for?
Kris: Sure it’s one of many research tools in our arsenal. We obviously have a full time research team that tries to keep track of bad guys, they track the existing ones that we know about and find new ways of identifying new bad guys, but whenever we’re researching a threat actor we’re quite often interested in their targeting, are they targeting pharmaceutical companies from Western Europe, are they targeting oil and gas companies in Northern Africa, or financial services companies in the US? It’s quite useful just to know what they’re actually up to at the minute. One of the ways that we often do that is, for most malware families once they’re inside or onto somebody’s computer, they have to basically phone home to say “hey I’m here, what instructions do you have for me, what do you want me to do?” and that’s known as commander control traffic so they basically have to phone home, wait for a command and then execute it. And that might be they’re phoning home every 2 minutes to a www.baddomain.com for example, but what we’ll often do is work with the domain registrars those domains were registered through to take control of them so we now own those domains, not the bad guys, and all the infected systems are basically phoning home to a server under our control not the bad guys’. We never send anything back, we don’t communicate, but we can actually log where those are coming from. So we can see if all the connections are coming from the US, Northern Africa, Western Europe, Japan or kind of spread all over the world. And we can quite often then trace back which companies or individuals are actually, not right down to names of individuals, but companies for example we can somehow work back so we can actually tell if it’s a pharmaceutical company or a gas company or so on. And that gives us a really good idea for that specific campaign, who were the attackers targeting. And some attackers only target very niche sectors, some have actually a very broad remit in terms of targeting so we see a real mix of companies involved in it. What we use that for aside from just the research is actually to notify victims where we can. In some cases we actually work with law enforcement and cert agencies to try and make victim notifications so they can actually resolve the issue. The other thing it’s quite useful for is part of an incident response containment strategy so whenever we’re kicking an attacker out of our network we’ll usually at the same time try to take control of all of their commander control infrastructure that they’re using on that specific attack and it’s only a very small part of the containment strategy but it’s also very useful to figure out, once you think you’ve cleaned up the intrusion, do you still see any systems from that client beaconing back to the commander control server. And it’s useful to be able to correlate containments executed, don’t see any traffic back to the server.
Ian: Incredible. It gives you a good picture of everything that’s going on.
Kris: Yes, it’s very useful research tool.
Ian: And do the bad guys, well will they know – it might seem like a very simple question – are they aware that you’ve kind of held hostage that domain? Do they know, and at what point do they know?
Kris: I’m pretty sure that they know reasonably quickly when their stuff stops working. In reality the two ways you can get in control of those things, you can either work with the domain registrars to take them over when they’re still in use by the bad guys, but in many cases the bad guys have been using them for a year, they don’t really care about the victims anymore, they’ve got everything they wanted out of them and then they let the domains expire so the other way of picking them up is waiting until they expire and then grab them, and you quite often still see some victims.
Ian: Interesting, incredible. So I think this is quite relevant to what’s happening right now in the political sphere in the UK is the introduction of this Snoopers Charter. I think it’s going to change the way people use the internet. From what I’ve read people are going to start using VPN’s and different proxies and tor servers and obviously on the other side of that we’re going to see a huge amount of databases that store huge amounts of personal information. How do you see this changing the landscape in the future or will it change it? What do you see from there?
Kris: It’s going to be interesting to see how this one pans out actually. I think if you look at it cold there are probably going to be both positives and negatives associated with it. In any sort of surveillance capacity if you look at a lot of the other international regimes where the internet is very heavily policed, there is a very prevalent switch to exactly, as you said VPN’s and other methods of circumventing some of the surveillance on it, not necessarily because anybody is up to anything bad, but just because they like their privacy.
Ian: I think people forget that, sometimes people just want to hold their privacy. It’s not like they’re doing anything malicious at all but they like to keep that privacy to themselves.
Kris: Absolutely. The other side of it is if you look at what the National Cyber Security Centre says they want to do to make the UK a much tougher placer for bad guys to target, they almost need that visibility into traffic to be able to stop it. Over the last few years we’ve seen a huge switch from, I guess in general terms, malware – which is communicating in plain text all switching over the SSL traffic and unless, actually Mozilla for the very first time stated that over 50% of all web traffic is now encrypted. So unless you’ve got some sort of inspection capability either at an ISP level or corporate level, you have no idea whether that’s malware traffic or legitimate stuff. So the ability to inspect that actually makes it much easier to catch malicious activity but it also, at a National level, would mean that the UK can apply greater filtering to try and prevent external state sponsored hackers in different bits of the World, from actually getting into corporates over here.
Ian: That’s really interesting because I haven’t seen that talked about as much, obviously people work very passionately around the privacy side of things but I think that’s a really important part and maybe has been looked over slightly as well.
Kris: And then, as you said, the other side is this huge additional set of data that the ISP’s are going to have to collect. I used to run an ISP, I know the volumes of data that even small ones can push in and out of their networks, that’s going to be a huge technological burden for a lot of the major ISP’s, but it’s also going to be a data set that will inevitably get targeted at some point because it does have really rich telemetry on all of the subscribers.
Ian: I think that’s the big fear a lot of people have. It’s that huge data storage and how secure is it and what can people extract from there. It’s a fair enough fear I think for people to have. So I think the question that I’d like to ask is what is PwC doing to help the good guys with the bad guys? What are we doing?
Kris: Quite a lot. I used to say that our time was really split 50/50 between helping clients prepare pre-incident so that if one happened they’re able to respond to it effectively and to reduce the likelihood of one actually happening, and 50% of our time actually helping clients to respond to incidents. I think more recently it’s probably been 40/60 so 40% on the prep side, 60% on the response side, just because there are some fairly major incidents happening in the UK at the minute and over the last few months but I guess across the spectrum of what we do, I guess the bits that I’m responsible for primarily helping organisations understand about the bad guys and that’s everything from reporting through to technical data that they can use to protect their networks, and understand why they’ve been targeted. Helping them actually develop monitoring and detection strategies so they get better at finding the malicious activity themselves or in some cases doing that for them either as part of an assessment or as part of a managed service, and then a lot around the response side of things so everything from forensic readiness plans to insert response playbooks through to being there as we described earlier as a panic button, so that we can actually sweep in whenever organisations need help.
Ian: And once a panic button is depressed you can be there relatively quickly can’t you?
Kris: Absolutely, we’ve 150 incident response and digital forensic people scattered around the UK so they’re usually close to where our clients are.
Ian: Perfect, well thank you so much. I think the content was fascinating so I do appreciate your time. I’m sure there will be a bunch of questions and hopefully if listeners do have questions we’ll feed them back to you and get some answers.
Kris: Cool, thanks for having me.
Ian: Thank you for joining us today. Next week I’ll be joined by Richard Mardling, discussing the importance of identity and access management, an increasingly important area for organisations wishing to improve their security from external and internal threats. In the meantime, if you have any questions, comments or suggestions, please feel free to contact me directly on Twitter at @iantodd86 or email me at email@example.com, Please remember to subscribe for future episodes.
Cyber Threat Operations - Manager, PwC United Kingdom
Tel: +44 (0)7841 803680