AI transcript
0:00:05 Hi, and welcome to the A16Z Podcast Goes Remote. I’m Doss, and in this podcast,
0:00:11 I call A16Z security expert Joel Delegarza to chat about what the rapid, widespread
0:00:16 move to remote work means for security. With so many people going remote the same way that we are,
0:00:22 what’s top of mind for you as a security expert? There is a concept in information security, which
0:00:28 is the belief in defense in depth. And that means that you don’t rely on any one thing to protect
0:00:33 you. You have a series of things that you use, and you stack them on top of each other, and you
0:00:38 use those series of things to offer multiple layers of protection. You don’t just put a moat
0:00:44 around the castle. You also put walls, and you have archers, and you have hot oil ready to pour on
0:00:49 people that try to storm it. And so in security, we have those same sorts of controls. The challenge
0:00:55 for security teams is that a lot of those controls for a lot of companies only live in their office
0:00:59 and only live in their corporate network. And so when users take their machines home with them,
0:01:04 or they’re remotely accessing, they don’t necessarily have the same controls in the office
0:01:09 as they do at home. And if you look at some of the large breaches over the last, let’s say, five
0:01:15 years, you’d see that there are a number of instances where remote employees using a home
0:01:20 computer that’s perhaps shared with someone in the house that doesn’t have protections on it,
0:01:24 is used to access internal corporate information by an attacker that’s hacked it.
0:01:29 Are the things that we’re dealing with new things or just things that are underway happening a lot
0:01:34 faster? We’ve had multiple scenarios in the corporate and enterprise world where we’ve had to
0:01:40 make employees work from home and work remotely. The first real encounter in at least my adult life
0:01:45 with this sort of a scenario was obviously 9/11, when we had fundamentally a city that became
0:01:50 unavailable in the workforce there being mostly unavailable or having to move to disaster recovery
0:01:57 sites. And I think 9/11 really taught a lot of large corporations about the importance of building
0:02:03 really resilient business continuity programs. The actual new thing about this is just the scale,
0:02:07 is just the entirety of a workforce for a company being forced to work remote,
0:02:13 as well as their suppliers, as well as their customers. We had the advent of things like
0:02:19 SaaS and Salesforce and Box and all these tools that were basically derived so that people could
0:02:25 access their work materials anywhere. And that it sort of became expected that some percentage,
0:02:29 usually salespeople because they’re in the field, but some percentage of your workforce would be
0:02:35 remote. And so we’ve been building infrastructure to support that workforce for some time now.
0:02:39 This is less of like, oh, it’s a new way to work and we have to change everything. This is more
0:02:45 like, we have to re-engineer everything to handle the capacity and just the sheer number.
0:02:49 How are the best security teams you know, properly preparing their organizations
0:02:52 with this really rapid shift to remote work?
0:02:58 I think the right way to think about it is to just build a matrix in your mind that sort of
0:03:03 enumerates all the different security controls you have available to you in the workplace,
0:03:08 in the office, and have some understanding of how they translate to the different scenarios
0:03:13 all of your employees will find themselves in now. And so I think there’s two things that the
0:03:17 really good security teams are fundamentally doing. The first is getting their people stood up
0:03:23 online outside of the office because security teams don’t necessarily always have great disaster
0:03:28 recovery and business continuity plans. And then second, making sure that what they’re doing is
0:03:31 actually safe and secure. If you were in an organization right now and say you were going
0:03:39 from 20% to now 90% of your workforce is remote, break down for me very specifically how you would
0:03:44 do a risk assessment. Over the last couple years, most things have left the building. And so most
0:03:50 services are provided by third parties. Most of the infrastructure that you run isn’t running
0:03:55 on your premise. And so for the last three or four years, most CISOs or Chief Information
0:04:00 Security Officer have spent a tremendous amount of time thinking about their third party risk.
0:04:05 Who are their vendors? Who are their counterparties? Who are the people that they transact with?
0:04:10 And you have to think about them not just from a security perspective because that’s
0:04:14 a little bit narrow in terms of impact of the business, but you need to be more comprehensive
0:04:21 and think of terms of like confidentiality. So is shifting all of your voice traffic to this
0:04:25 third party, does that provide you with the confidentiality you need to run your business?
0:04:30 Well, it may be okay to have a sales call with a customer where you don’t discuss anything
0:04:35 confidential over a video conferencing system. Now you’re having your board meetings over this
0:04:39 video conferencing system. Does it meet the requirements that you have? And then you have
0:04:44 to think in terms of integrity. Do the systems that you’re relying on, now that you’ve moved
0:04:50 everybody onto them, have the controls in place to ensure the integrity of the operations of your
0:04:54 business? Are they going to lose your data? Is there going to be some sort of a disruption
0:04:59 to the quality of the output? Are the systems of record truly capable of being systems of record?
0:05:04 And then finally, you have to think in terms of availability. Not just you as a company are
0:05:09 moving your entire workforce to this service provider. The entire planet is, will the service
0:05:15 provider be up and running in the face of this kind of demand? Or will they just fall over because of
0:05:20 the excess utilization? I like the way that you broke that down. So it sounded like the first
0:05:26 bucket there was really around confidentiality and what transactions were happening in person,
0:05:31 providing a measure of security now happening virtually. So let’s focus in on that for a second.
0:05:37 How would you go about assessing that? It really depends on the vertical. And it depends on the
0:05:43 industry. There’s a very, very rich tapestry of requirements and regulations that you have to
0:05:47 really understand. And it’s very specific to the business that you’re in specifically if you’re
0:05:53 regulated. And you have to make sure that the tools that you’re using can support those industry
0:05:59 specific regulations. If you are, for example, in the healthcare industry, and let’s say you’re a
0:06:06 hospital network, and hospitals right now are rushing to provide telemedicine and to remotely
0:06:12 treat potentially sick people, the issue with that is that there are these regulations called HIPAA
0:06:17 and high tech that mean that you actually have to work to maintain the confidentiality of your
0:06:22 patient’s information. So then I guess looking at the second bucket that you talked about,
0:06:26 which was really selecting these new tools and introducing these new third party vendors that
0:06:32 you maybe weren’t using before. So for instance, you and I are using a totally new tool for A16Z
0:06:36 that we rolled out as soon as we went remote so that we could keep running our podcast.
0:06:41 How are you or security professionals thinking about these third party tools and how do you go
0:06:46 about assessing them? Well, it’s always about the data. For example, we’re recording a podcast,
0:06:50 this is public information, eventually it’s going to be released. And so the sensitivity of our
0:06:55 discussion that we’re recording right now is low. It’s fundamentally public data.
0:07:00 Whereas if we were talking about a portfolio company, this might not be an appropriate tool
0:07:06 because it might not adequately protect those discussions. And so we really have to understand
0:07:11 first the sensitivity of the data and then match that data sensitivity to the security features
0:07:17 and capabilities of the tool. Generally, marketing teams are kind of free to experiment with tools
0:07:21 that are maybe not industrial grade security. But the moment that you start talking about
0:07:26 transferring customer records or transferring personal information of your customers or
0:07:31 any intellectual property, then you really need to understand the tools and a very quick adoption
0:07:35 and migration path could potentially get you into a not so great place.
0:07:39 It’s interesting you mentioned quick adoption because that is absolutely what we’re seeing
0:07:44 right now. When you suddenly have, in our case, all of A16Z going remote, we suddenly needed all
0:07:49 these new communication tools that we didn’t use before. So we are rolling them out relatively
0:07:54 quickly. How are IT and security teams keeping up with the fact that people are rapidly adapting to
0:07:59 this, things are changing daily? How do they balance that with security?
0:08:06 At A16Z, we’ve been fortunate in that we’ve probably spent the last two years really focusing
0:08:12 on eliminating any kind of custom solutions, not having servers under people’s desks,
0:08:17 not having servers at all, focusing on using cloud infrastructure and SaaS.
0:08:21 And so when this event happened and we had to pivot credit to our IT team, they did some
0:08:26 wonderful work, but we were really well positioned. There wasn’t a whole lot of stuff other than
0:08:31 adding a few new services that were disruptive. I think the way that the modern enterprise has
0:08:36 built their data stores is somewhat similar so that a lot of the data that a company has that
0:08:41 could very easily flow out of the organization are generally pretty well controlled.
0:08:45 Often we’re used to these large enterprise rollouts of new tools. They take a long time,
0:08:50 but now you have a workforce going remote and you maybe need to roll tools out faster.
0:08:56 What steps are you seeing people cut or needing to add to get the tools out and into the hands of
0:09:03 workers in order to do virtual work? Usually one of the longest pulls on any of these kinds of tool
0:09:09 deployments is the legal and contract negotiations. It’s the kind of thing where the length of your
0:09:12 proof of concept is probably half the length of the debate you’re going to have with the
0:09:17 vendor about limits of liability. Like people complain about IT, but if you really want to
0:09:23 prolong something, bring a couple lawyers. And especially when you have to have IT people,
0:09:28 technical people work with lawyers, it compounds it. So I think where I’ve seen things getting
0:09:33 quicker is just on the procurement side, on the contracting side. We’ve gone through a three-year
0:09:40 process of large enterprises telling employees, don’t use your credit card to buy a SaaS service.
0:09:46 That window seems to have opened up a little bit. And so you’re seeing people paying for things with
0:09:51 personal or corporate cards to get services deployed and unrolled. And I think IT and legal,
0:09:54 they’re going to be flexible. They’re going to keep the business moving. There’s probably going to be a
0:09:59 lot of contract review and a lot of teeth gnashing over the next couple months as they figure out
0:10:05 what they’ve allowed into the enterprise. What in your mind works and doesn’t work to be communicating
0:10:10 out to the organization at this time? And what would you be reiterating to individual workers?
0:10:15 The user tends to be the weakest link in any security system. And so there is this desire to
0:10:20 blame. And then the products that grow out of the desire to blame users tend to be of the variety
0:10:27 that look to shame users into behaving better. So building tests that try to get users to fail
0:10:32 and then highlighting their failure. And we’ve seen more tools that take that approach. They’re
0:10:37 really good at getting the level of compliance up, but only to a certain point. I think the real
0:10:43 key is going to be figuring out how to decadetize employees and users, how to make them feel part
0:10:49 of this, instead of smacking them on the hand for making mistakes. And then that’s really hard for
0:10:53 security people because we do tend to be a bit pessimistic. But building systems that reward
0:10:59 good behavior, I think will go a lot further than the desire to name and shame. From a most
0:11:04 important tips perspective, I think for me, it’s always two-factor authentication. At its most
0:11:12 simplest level, two-factor authentication is the way you log into a system using two factors or two
0:11:18 things. And from a security perspective, you want one of those things to be something you know,
0:11:22 like a password. I’ve got a password and that’s the thing I know. And then the other one of those
0:11:28 things to be something you have, like a hardware security key. And so it becomes very difficult
0:11:33 for an attacker to get access to your system because not only do they have to have your
0:11:38 password, they also have to have access to your key. And so it really frustrates what is ultimately
0:11:44 the single largest source of hacking in the world, which is stealing someone’s username and password.
0:11:49 In general, while using a phone is better than just using a username and password, from our
0:11:54 perspective, it’s not as strong as using a dedicated piece of hardware to protect your login accounts.
0:11:59 So that text message that gives you the code on your phone, probably not as good as some sort of
0:12:03 hardware key you’re plugging into your computer when it comes to two-factor.
0:12:07 Correct. And for systems that you care about, I mean, you should really use a hardware
0:12:13 security key. And if you’re at home and you’re not using strong two-factor on your corporate
0:12:18 resources or even on your personal laptop, then certainly make sure that you enable that. I also
0:12:25 think at home, if you’re not using a corporate-issued laptop or workstation and you’re using your own
0:12:30 equipment to access your workplace, double-click on the security of your own machines, you make
0:12:35 sure that they’ve got usernames and passwords, that you’re running some kind of antivirus software,
0:12:39 that you’re patching your systems. Ideally, you’re not sharing computers.
0:12:44 So you’ve mentioned business continuity planning a couple of times. Explain to me kind of what
0:12:49 that concept means to a security professional. It’s kind of the job of a security professional
0:12:53 and more broadly risk professionals in an organization to sit around thinking about what’s
0:12:58 the worst possible thing that could happen to the business. And so you come out with this
0:13:02 list of things that could potentially disrupt your business. Now, they may be hurricanes,
0:13:07 they could be earthquakes, it could be a hacker attack, it could be a breach, it could be ransomware,
0:13:12 it could be a nation state attack, it could be war, whatever the case may be. You estimate their
0:13:16 risk to the business, like if this happened, how big of an impact would it have? What’s the
0:13:20 probability of a global pandemic happening, for example, or an event that forces all of your
0:13:25 employees to work from home. And a business continuity plan is developed to help manage
0:13:32 those risks so that you can continue to run your business through any sort of adverse changes.
0:13:36 It’s not dissimilar from what a CFO or a financial planner would do where they
0:13:40 try to figure out their risks from a credit perspective, like our credit market’s going to
0:13:46 shut. Do we have market risk? Is our stock price going to fall? Which industries and orgs are having
0:13:51 a harder time with that business continuity and maintaining going remote? And why? What are the
0:13:56 unique challenges if you start to break it down by industry? I think if you break it down by industry,
0:14:01 you’d see that the businesses that are having the biggest challenges are the ones that have never
0:14:07 had a significant disruption. Whereas if you look at banks who were primarily the ones impacted by
0:14:12 9/11, they’ve been able to fairly seamlessly transition to remote work. They’ve been able
0:14:17 to take up different locations and implement their pandemic response plan. There haven’t been
0:14:23 any disruptions to the financial system. We’ve seen people doing panic ATM withdrawals and the ATM
0:14:28 and banking infrastructure doing just fine with it. And if you look at Deutsche Bank on 9/11,
0:14:32 Deutsche Bank invested a bunch of money in business continuity. They could seat their
0:14:37 employees on the other side of the river. They had backups. They were running off-site. In response
0:14:41 to that catastrophe happening, they were able to quickly resume business, settle their trades,
0:14:47 not suffer material financial impacts. I’m sure in every meeting leading up to the event,
0:14:52 there was probably someone saying, “We should cut that budget.” But lo and behold, you hold fast and
0:14:56 it turns out to be an investment that’s worthwhile. I also feel like there’s certain industries where
0:15:01 either regulation or the nature of the critical infrastructure, say a power utility, they have
0:15:06 some unique challenges. I’m curious if that’s something that you’re seeing or hearing about.
0:15:10 I think the power utilities and a lot of these critical infrastructure components,
0:15:15 they sort of have their zombie apocalypse plan. They plan for that and I generally have faith
0:15:19 that they’re doing it well. I think the one industry, the one segment that’s going to be
0:15:24 really impacted and we’re seeing that is actually pharma and healthcare. I think that there are
0:15:30 just major capacity constraints in a lot of countries that just won’t be able to handle a
0:15:36 major flood of inbound requests for care. Ultimately, the reason why we are all
0:15:41 working from home is to protect our healthcare system. Whether we’re conscious of it or not,
0:15:48 we are all engaged in a business continuity plan for the public health system right now.
0:15:53 I mean, that is what working from home is doing so that we can keep capacity available to treat
0:15:58 and care for people. I want to shift now and talk a little bit, not just about the security
0:16:05 practices, but what this means for the architectures that organizations have, because as the workforce
0:16:10 goes more distributed, it does seem like there might be a need to rearchitect the way that we do
0:16:15 things. What are your thoughts on how this might impact organizational architectures?
0:16:20 I think the cleanest example of where there needs to be a massive rearchitecture is when it comes
0:16:28 to traditional VPN or virtual private networking technology. VPNs are mostly based on IPsec,
0:16:34 which is an internet security protocol that was developed many years ago. These protocols and
0:16:38 these infrastructures were designed to be point-to-point. You would have many, many points
0:16:44 around a central hub that would aggregate all of that information and then send it to other
0:16:49 central hubs. In that architecture, if one point on the hub wants to talk to another point,
0:16:55 it has to go through a central point. When you move your entire workforce onto that kind of hub
0:17:01 and spoke point-to-point infrastructure, you get traffic jams. Security systems tend to fail closed.
0:17:09 If a VPN or a firewall starts falling over, they tend to shut down and stop all traffic. It’s
0:17:14 really clear that we have to get away from the centralized, the ring of trust model. We’ve got
0:17:19 to go more towards a web of trust. You’re seeing this with a lot of the new security technology
0:17:23 that’s coming out where they’re creating these more distributed trust environments,
0:17:27 cryptocurrencies, and blockchain are very much about that distributed trust model.
0:17:33 Is it too broad of a generalization to say that the ability for us to scale and to
0:17:39 not be real-time stress testing our systems is really directly related to how fast we can
0:17:46 re-architect to distributed trust? The point-to-point architecture scales fairly linearly,
0:17:51 and so for every increase in capacity or increase in utilization, you have to add
0:17:58 a fixed amount of capacity, and it’s just not a great way to scale from an infrastructure
0:18:03 perspective. And so we have to get to a way where we can use capacity that’s more at the edge
0:18:08 and get away from the centralized infrastructure. You talked about this process of re-architecting,
0:18:13 and I’ve also heard about this concept of shifting to zero trust. Is that the same shift or are those
0:18:19 things different? They kind of cohabitate the same space, and I’d say there’s a lot of overlap,
0:18:25 but zero trust is, it’s an idea that was kicked off, I think, by a forester researcher in the late
0:18:32 ’90s, and the idea was that you have to eliminate transitive trust. Transitive trust is basically
0:18:38 the principle that if I trust you and you trust Bob, then I trust Bob. And as you could imagine,
0:18:42 that is what attackers would use to exfiltrate data, to get access to intellectual property,
0:18:48 to do generally bad things. Transitive trust is a very dangerous thing, and I guess the layman’s
0:18:54 way to say that is, in the old world, if you went to the office and you plugged into the corporate
0:18:58 network, on your corporate network, you had access to a bunch of systems, and a lot of that data
0:19:04 didn’t have passwords or logins or encryption, because it was on the corporate network, and the
0:19:09 corporate network was considered safe. The moment that you got access to the corporate network,
0:19:15 if you were an attacker, you had access to all the data. And so zero trust is about creating
0:19:19 a distributed trust environment. We’re taking away the castle and moat, and every person’s
0:19:25 home is becoming a castle, to reuse that phrase. With the changes that you see underway, with the
0:19:31 shift away from a hub and spoke, how would you advise startups to start thinking about
0:19:35 security and their products? I think that you’re going to see a lot of companies that historically
0:19:41 wouldn’t use bleeding edge technology, actually moving towards adopting a lot of bleeding edge
0:19:47 technology just because of the disruption. And I think it’s a really wonderful opportunity for
0:19:51 entrepreneurs that are making enterprise tech right now. I think this is their time to really
0:19:57 get significant adoption from customers that in the old days would have wanted to see something on
0:20:02 prem, but now you can’t get access to your premises. So you’ve got to try something new.
0:20:07 Generally, we tell our startups, obviously, security is important, but as you get bigger and
0:20:11 larger and later in your fundraising, it becomes more and more important. And then finally,
0:20:17 when you IPO, there are specific public company security requirements that you have to meet before
0:20:22 you even get to go public. So it is a blocker at that level. I think the focus on security is
0:20:28 kind of shifting. I think it’s going to come a lot earlier now. Typically, you’d see series B
0:20:32 companies, sometimes series A companies focusing on security. I think it’s going to be like a seed
0:20:38 stage thing. So as we wrap up, what here is a passing challenge security teams have to meet
0:20:45 and what is just a longer term shift in how we think about security? What’s our new world order?
0:20:50 I think the growing pains are a passing challenge. I think a lot of the large cloud providers and
0:20:54 service providers are going to add capacity. And to be quite honest, a lot of the services I’m
0:21:00 using right now are working fine. So I’m not super concerned about the capacity. I think the
0:21:06 longer term change is just going to be more about keeping the security mentality. I think a lot of
0:21:13 this ultimately comes down to users. And in a workplace where we see each other every day,
0:21:18 you still had people falling for scams where a co-worker sends you a request for money from a
0:21:23 sketchy Gmail account and you send the money. So I think that when you put more of a social
0:21:30 isolation in there, I think the risk of targeting users going for social engineering to defraud
0:21:35 people will potentially become more successful. And so I think the real focus for these organizations
0:21:40 is finding ways to keep employees who are at home in their pajamas, still thinking like foot
0:21:44 soldiers in the battle to protect their company and their data. That’s going to be a real challenge.
0:21:49 And I think training is always proven to be one of the best returns on investment.
0:21:54 That is a terrific note to end on. Joel, thank you so much for joining.
0:21:54 Thank you. My pleasure.
0:00:11 I call A16Z security expert Joel Delegarza to chat about what the rapid, widespread
0:00:16 move to remote work means for security. With so many people going remote the same way that we are,
0:00:22 what’s top of mind for you as a security expert? There is a concept in information security, which
0:00:28 is the belief in defense in depth. And that means that you don’t rely on any one thing to protect
0:00:33 you. You have a series of things that you use, and you stack them on top of each other, and you
0:00:38 use those series of things to offer multiple layers of protection. You don’t just put a moat
0:00:44 around the castle. You also put walls, and you have archers, and you have hot oil ready to pour on
0:00:49 people that try to storm it. And so in security, we have those same sorts of controls. The challenge
0:00:55 for security teams is that a lot of those controls for a lot of companies only live in their office
0:00:59 and only live in their corporate network. And so when users take their machines home with them,
0:01:04 or they’re remotely accessing, they don’t necessarily have the same controls in the office
0:01:09 as they do at home. And if you look at some of the large breaches over the last, let’s say, five
0:01:15 years, you’d see that there are a number of instances where remote employees using a home
0:01:20 computer that’s perhaps shared with someone in the house that doesn’t have protections on it,
0:01:24 is used to access internal corporate information by an attacker that’s hacked it.
0:01:29 Are the things that we’re dealing with new things or just things that are underway happening a lot
0:01:34 faster? We’ve had multiple scenarios in the corporate and enterprise world where we’ve had to
0:01:40 make employees work from home and work remotely. The first real encounter in at least my adult life
0:01:45 with this sort of a scenario was obviously 9/11, when we had fundamentally a city that became
0:01:50 unavailable in the workforce there being mostly unavailable or having to move to disaster recovery
0:01:57 sites. And I think 9/11 really taught a lot of large corporations about the importance of building
0:02:03 really resilient business continuity programs. The actual new thing about this is just the scale,
0:02:07 is just the entirety of a workforce for a company being forced to work remote,
0:02:13 as well as their suppliers, as well as their customers. We had the advent of things like
0:02:19 SaaS and Salesforce and Box and all these tools that were basically derived so that people could
0:02:25 access their work materials anywhere. And that it sort of became expected that some percentage,
0:02:29 usually salespeople because they’re in the field, but some percentage of your workforce would be
0:02:35 remote. And so we’ve been building infrastructure to support that workforce for some time now.
0:02:39 This is less of like, oh, it’s a new way to work and we have to change everything. This is more
0:02:45 like, we have to re-engineer everything to handle the capacity and just the sheer number.
0:02:49 How are the best security teams you know, properly preparing their organizations
0:02:52 with this really rapid shift to remote work?
0:02:58 I think the right way to think about it is to just build a matrix in your mind that sort of
0:03:03 enumerates all the different security controls you have available to you in the workplace,
0:03:08 in the office, and have some understanding of how they translate to the different scenarios
0:03:13 all of your employees will find themselves in now. And so I think there’s two things that the
0:03:17 really good security teams are fundamentally doing. The first is getting their people stood up
0:03:23 online outside of the office because security teams don’t necessarily always have great disaster
0:03:28 recovery and business continuity plans. And then second, making sure that what they’re doing is
0:03:31 actually safe and secure. If you were in an organization right now and say you were going
0:03:39 from 20% to now 90% of your workforce is remote, break down for me very specifically how you would
0:03:44 do a risk assessment. Over the last couple years, most things have left the building. And so most
0:03:50 services are provided by third parties. Most of the infrastructure that you run isn’t running
0:03:55 on your premise. And so for the last three or four years, most CISOs or Chief Information
0:04:00 Security Officer have spent a tremendous amount of time thinking about their third party risk.
0:04:05 Who are their vendors? Who are their counterparties? Who are the people that they transact with?
0:04:10 And you have to think about them not just from a security perspective because that’s
0:04:14 a little bit narrow in terms of impact of the business, but you need to be more comprehensive
0:04:21 and think of terms of like confidentiality. So is shifting all of your voice traffic to this
0:04:25 third party, does that provide you with the confidentiality you need to run your business?
0:04:30 Well, it may be okay to have a sales call with a customer where you don’t discuss anything
0:04:35 confidential over a video conferencing system. Now you’re having your board meetings over this
0:04:39 video conferencing system. Does it meet the requirements that you have? And then you have
0:04:44 to think in terms of integrity. Do the systems that you’re relying on, now that you’ve moved
0:04:50 everybody onto them, have the controls in place to ensure the integrity of the operations of your
0:04:54 business? Are they going to lose your data? Is there going to be some sort of a disruption
0:04:59 to the quality of the output? Are the systems of record truly capable of being systems of record?
0:05:04 And then finally, you have to think in terms of availability. Not just you as a company are
0:05:09 moving your entire workforce to this service provider. The entire planet is, will the service
0:05:15 provider be up and running in the face of this kind of demand? Or will they just fall over because of
0:05:20 the excess utilization? I like the way that you broke that down. So it sounded like the first
0:05:26 bucket there was really around confidentiality and what transactions were happening in person,
0:05:31 providing a measure of security now happening virtually. So let’s focus in on that for a second.
0:05:37 How would you go about assessing that? It really depends on the vertical. And it depends on the
0:05:43 industry. There’s a very, very rich tapestry of requirements and regulations that you have to
0:05:47 really understand. And it’s very specific to the business that you’re in specifically if you’re
0:05:53 regulated. And you have to make sure that the tools that you’re using can support those industry
0:05:59 specific regulations. If you are, for example, in the healthcare industry, and let’s say you’re a
0:06:06 hospital network, and hospitals right now are rushing to provide telemedicine and to remotely
0:06:12 treat potentially sick people, the issue with that is that there are these regulations called HIPAA
0:06:17 and high tech that mean that you actually have to work to maintain the confidentiality of your
0:06:22 patient’s information. So then I guess looking at the second bucket that you talked about,
0:06:26 which was really selecting these new tools and introducing these new third party vendors that
0:06:32 you maybe weren’t using before. So for instance, you and I are using a totally new tool for A16Z
0:06:36 that we rolled out as soon as we went remote so that we could keep running our podcast.
0:06:41 How are you or security professionals thinking about these third party tools and how do you go
0:06:46 about assessing them? Well, it’s always about the data. For example, we’re recording a podcast,
0:06:50 this is public information, eventually it’s going to be released. And so the sensitivity of our
0:06:55 discussion that we’re recording right now is low. It’s fundamentally public data.
0:07:00 Whereas if we were talking about a portfolio company, this might not be an appropriate tool
0:07:06 because it might not adequately protect those discussions. And so we really have to understand
0:07:11 first the sensitivity of the data and then match that data sensitivity to the security features
0:07:17 and capabilities of the tool. Generally, marketing teams are kind of free to experiment with tools
0:07:21 that are maybe not industrial grade security. But the moment that you start talking about
0:07:26 transferring customer records or transferring personal information of your customers or
0:07:31 any intellectual property, then you really need to understand the tools and a very quick adoption
0:07:35 and migration path could potentially get you into a not so great place.
0:07:39 It’s interesting you mentioned quick adoption because that is absolutely what we’re seeing
0:07:44 right now. When you suddenly have, in our case, all of A16Z going remote, we suddenly needed all
0:07:49 these new communication tools that we didn’t use before. So we are rolling them out relatively
0:07:54 quickly. How are IT and security teams keeping up with the fact that people are rapidly adapting to
0:07:59 this, things are changing daily? How do they balance that with security?
0:08:06 At A16Z, we’ve been fortunate in that we’ve probably spent the last two years really focusing
0:08:12 on eliminating any kind of custom solutions, not having servers under people’s desks,
0:08:17 not having servers at all, focusing on using cloud infrastructure and SaaS.
0:08:21 And so when this event happened and we had to pivot credit to our IT team, they did some
0:08:26 wonderful work, but we were really well positioned. There wasn’t a whole lot of stuff other than
0:08:31 adding a few new services that were disruptive. I think the way that the modern enterprise has
0:08:36 built their data stores is somewhat similar so that a lot of the data that a company has that
0:08:41 could very easily flow out of the organization are generally pretty well controlled.
0:08:45 Often we’re used to these large enterprise rollouts of new tools. They take a long time,
0:08:50 but now you have a workforce going remote and you maybe need to roll tools out faster.
0:08:56 What steps are you seeing people cut or needing to add to get the tools out and into the hands of
0:09:03 workers in order to do virtual work? Usually one of the longest pulls on any of these kinds of tool
0:09:09 deployments is the legal and contract negotiations. It’s the kind of thing where the length of your
0:09:12 proof of concept is probably half the length of the debate you’re going to have with the
0:09:17 vendor about limits of liability. Like people complain about IT, but if you really want to
0:09:23 prolong something, bring a couple lawyers. And especially when you have to have IT people,
0:09:28 technical people work with lawyers, it compounds it. So I think where I’ve seen things getting
0:09:33 quicker is just on the procurement side, on the contracting side. We’ve gone through a three-year
0:09:40 process of large enterprises telling employees, don’t use your credit card to buy a SaaS service.
0:09:46 That window seems to have opened up a little bit. And so you’re seeing people paying for things with
0:09:51 personal or corporate cards to get services deployed and unrolled. And I think IT and legal,
0:09:54 they’re going to be flexible. They’re going to keep the business moving. There’s probably going to be a
0:09:59 lot of contract review and a lot of teeth gnashing over the next couple months as they figure out
0:10:05 what they’ve allowed into the enterprise. What in your mind works and doesn’t work to be communicating
0:10:10 out to the organization at this time? And what would you be reiterating to individual workers?
0:10:15 The user tends to be the weakest link in any security system. And so there is this desire to
0:10:20 blame. And then the products that grow out of the desire to blame users tend to be of the variety
0:10:27 that look to shame users into behaving better. So building tests that try to get users to fail
0:10:32 and then highlighting their failure. And we’ve seen more tools that take that approach. They’re
0:10:37 really good at getting the level of compliance up, but only to a certain point. I think the real
0:10:43 key is going to be figuring out how to decadetize employees and users, how to make them feel part
0:10:49 of this, instead of smacking them on the hand for making mistakes. And then that’s really hard for
0:10:53 security people because we do tend to be a bit pessimistic. But building systems that reward
0:10:59 good behavior, I think will go a lot further than the desire to name and shame. From a most
0:11:04 important tips perspective, I think for me, it’s always two-factor authentication. At its most
0:11:12 simplest level, two-factor authentication is the way you log into a system using two factors or two
0:11:18 things. And from a security perspective, you want one of those things to be something you know,
0:11:22 like a password. I’ve got a password and that’s the thing I know. And then the other one of those
0:11:28 things to be something you have, like a hardware security key. And so it becomes very difficult
0:11:33 for an attacker to get access to your system because not only do they have to have your
0:11:38 password, they also have to have access to your key. And so it really frustrates what is ultimately
0:11:44 the single largest source of hacking in the world, which is stealing someone’s username and password.
0:11:49 In general, while using a phone is better than just using a username and password, from our
0:11:54 perspective, it’s not as strong as using a dedicated piece of hardware to protect your login accounts.
0:11:59 So that text message that gives you the code on your phone, probably not as good as some sort of
0:12:03 hardware key you’re plugging into your computer when it comes to two-factor.
0:12:07 Correct. And for systems that you care about, I mean, you should really use a hardware
0:12:13 security key. And if you’re at home and you’re not using strong two-factor on your corporate
0:12:18 resources or even on your personal laptop, then certainly make sure that you enable that. I also
0:12:25 think at home, if you’re not using a corporate-issued laptop or workstation and you’re using your own
0:12:30 equipment to access your workplace, double-click on the security of your own machines, you make
0:12:35 sure that they’ve got usernames and passwords, that you’re running some kind of antivirus software,
0:12:39 that you’re patching your systems. Ideally, you’re not sharing computers.
0:12:44 So you’ve mentioned business continuity planning a couple of times. Explain to me kind of what
0:12:49 that concept means to a security professional. It’s kind of the job of a security professional
0:12:53 and more broadly risk professionals in an organization to sit around thinking about what’s
0:12:58 the worst possible thing that could happen to the business. And so you come out with this
0:13:02 list of things that could potentially disrupt your business. Now, they may be hurricanes,
0:13:07 they could be earthquakes, it could be a hacker attack, it could be a breach, it could be ransomware,
0:13:12 it could be a nation state attack, it could be war, whatever the case may be. You estimate their
0:13:16 risk to the business, like if this happened, how big of an impact would it have? What’s the
0:13:20 probability of a global pandemic happening, for example, or an event that forces all of your
0:13:25 employees to work from home. And a business continuity plan is developed to help manage
0:13:32 those risks so that you can continue to run your business through any sort of adverse changes.
0:13:36 It’s not dissimilar from what a CFO or a financial planner would do where they
0:13:40 try to figure out their risks from a credit perspective, like our credit market’s going to
0:13:46 shut. Do we have market risk? Is our stock price going to fall? Which industries and orgs are having
0:13:51 a harder time with that business continuity and maintaining going remote? And why? What are the
0:13:56 unique challenges if you start to break it down by industry? I think if you break it down by industry,
0:14:01 you’d see that the businesses that are having the biggest challenges are the ones that have never
0:14:07 had a significant disruption. Whereas if you look at banks who were primarily the ones impacted by
0:14:12 9/11, they’ve been able to fairly seamlessly transition to remote work. They’ve been able
0:14:17 to take up different locations and implement their pandemic response plan. There haven’t been
0:14:23 any disruptions to the financial system. We’ve seen people doing panic ATM withdrawals and the ATM
0:14:28 and banking infrastructure doing just fine with it. And if you look at Deutsche Bank on 9/11,
0:14:32 Deutsche Bank invested a bunch of money in business continuity. They could seat their
0:14:37 employees on the other side of the river. They had backups. They were running off-site. In response
0:14:41 to that catastrophe happening, they were able to quickly resume business, settle their trades,
0:14:47 not suffer material financial impacts. I’m sure in every meeting leading up to the event,
0:14:52 there was probably someone saying, “We should cut that budget.” But lo and behold, you hold fast and
0:14:56 it turns out to be an investment that’s worthwhile. I also feel like there’s certain industries where
0:15:01 either regulation or the nature of the critical infrastructure, say a power utility, they have
0:15:06 some unique challenges. I’m curious if that’s something that you’re seeing or hearing about.
0:15:10 I think the power utilities and a lot of these critical infrastructure components,
0:15:15 they sort of have their zombie apocalypse plan. They plan for that and I generally have faith
0:15:19 that they’re doing it well. I think the one industry, the one segment that’s going to be
0:15:24 really impacted and we’re seeing that is actually pharma and healthcare. I think that there are
0:15:30 just major capacity constraints in a lot of countries that just won’t be able to handle a
0:15:36 major flood of inbound requests for care. Ultimately, the reason why we are all
0:15:41 working from home is to protect our healthcare system. Whether we’re conscious of it or not,
0:15:48 we are all engaged in a business continuity plan for the public health system right now.
0:15:53 I mean, that is what working from home is doing so that we can keep capacity available to treat
0:15:58 and care for people. I want to shift now and talk a little bit, not just about the security
0:16:05 practices, but what this means for the architectures that organizations have, because as the workforce
0:16:10 goes more distributed, it does seem like there might be a need to rearchitect the way that we do
0:16:15 things. What are your thoughts on how this might impact organizational architectures?
0:16:20 I think the cleanest example of where there needs to be a massive rearchitecture is when it comes
0:16:28 to traditional VPN or virtual private networking technology. VPNs are mostly based on IPsec,
0:16:34 which is an internet security protocol that was developed many years ago. These protocols and
0:16:38 these infrastructures were designed to be point-to-point. You would have many, many points
0:16:44 around a central hub that would aggregate all of that information and then send it to other
0:16:49 central hubs. In that architecture, if one point on the hub wants to talk to another point,
0:16:55 it has to go through a central point. When you move your entire workforce onto that kind of hub
0:17:01 and spoke point-to-point infrastructure, you get traffic jams. Security systems tend to fail closed.
0:17:09 If a VPN or a firewall starts falling over, they tend to shut down and stop all traffic. It’s
0:17:14 really clear that we have to get away from the centralized, the ring of trust model. We’ve got
0:17:19 to go more towards a web of trust. You’re seeing this with a lot of the new security technology
0:17:23 that’s coming out where they’re creating these more distributed trust environments,
0:17:27 cryptocurrencies, and blockchain are very much about that distributed trust model.
0:17:33 Is it too broad of a generalization to say that the ability for us to scale and to
0:17:39 not be real-time stress testing our systems is really directly related to how fast we can
0:17:46 re-architect to distributed trust? The point-to-point architecture scales fairly linearly,
0:17:51 and so for every increase in capacity or increase in utilization, you have to add
0:17:58 a fixed amount of capacity, and it’s just not a great way to scale from an infrastructure
0:18:03 perspective. And so we have to get to a way where we can use capacity that’s more at the edge
0:18:08 and get away from the centralized infrastructure. You talked about this process of re-architecting,
0:18:13 and I’ve also heard about this concept of shifting to zero trust. Is that the same shift or are those
0:18:19 things different? They kind of cohabitate the same space, and I’d say there’s a lot of overlap,
0:18:25 but zero trust is, it’s an idea that was kicked off, I think, by a forester researcher in the late
0:18:32 ’90s, and the idea was that you have to eliminate transitive trust. Transitive trust is basically
0:18:38 the principle that if I trust you and you trust Bob, then I trust Bob. And as you could imagine,
0:18:42 that is what attackers would use to exfiltrate data, to get access to intellectual property,
0:18:48 to do generally bad things. Transitive trust is a very dangerous thing, and I guess the layman’s
0:18:54 way to say that is, in the old world, if you went to the office and you plugged into the corporate
0:18:58 network, on your corporate network, you had access to a bunch of systems, and a lot of that data
0:19:04 didn’t have passwords or logins or encryption, because it was on the corporate network, and the
0:19:09 corporate network was considered safe. The moment that you got access to the corporate network,
0:19:15 if you were an attacker, you had access to all the data. And so zero trust is about creating
0:19:19 a distributed trust environment. We’re taking away the castle and moat, and every person’s
0:19:25 home is becoming a castle, to reuse that phrase. With the changes that you see underway, with the
0:19:31 shift away from a hub and spoke, how would you advise startups to start thinking about
0:19:35 security and their products? I think that you’re going to see a lot of companies that historically
0:19:41 wouldn’t use bleeding edge technology, actually moving towards adopting a lot of bleeding edge
0:19:47 technology just because of the disruption. And I think it’s a really wonderful opportunity for
0:19:51 entrepreneurs that are making enterprise tech right now. I think this is their time to really
0:19:57 get significant adoption from customers that in the old days would have wanted to see something on
0:20:02 prem, but now you can’t get access to your premises. So you’ve got to try something new.
0:20:07 Generally, we tell our startups, obviously, security is important, but as you get bigger and
0:20:11 larger and later in your fundraising, it becomes more and more important. And then finally,
0:20:17 when you IPO, there are specific public company security requirements that you have to meet before
0:20:22 you even get to go public. So it is a blocker at that level. I think the focus on security is
0:20:28 kind of shifting. I think it’s going to come a lot earlier now. Typically, you’d see series B
0:20:32 companies, sometimes series A companies focusing on security. I think it’s going to be like a seed
0:20:38 stage thing. So as we wrap up, what here is a passing challenge security teams have to meet
0:20:45 and what is just a longer term shift in how we think about security? What’s our new world order?
0:20:50 I think the growing pains are a passing challenge. I think a lot of the large cloud providers and
0:20:54 service providers are going to add capacity. And to be quite honest, a lot of the services I’m
0:21:00 using right now are working fine. So I’m not super concerned about the capacity. I think the
0:21:06 longer term change is just going to be more about keeping the security mentality. I think a lot of
0:21:13 this ultimately comes down to users. And in a workplace where we see each other every day,
0:21:18 you still had people falling for scams where a co-worker sends you a request for money from a
0:21:23 sketchy Gmail account and you send the money. So I think that when you put more of a social
0:21:30 isolation in there, I think the risk of targeting users going for social engineering to defraud
0:21:35 people will potentially become more successful. And so I think the real focus for these organizations
0:21:40 is finding ways to keep employees who are at home in their pajamas, still thinking like foot
0:21:44 soldiers in the battle to protect their company and their data. That’s going to be a real challenge.
0:21:49 And I think training is always proven to be one of the best returns on investment.
0:21:54 That is a terrific note to end on. Joel, thank you so much for joining.
0:21:54 Thank you. My pleasure.
We are in the midst of a rapid and unprecedented shift to remote work. What does it mean for security when the airgap between work and life is gone? How prepared are organizations? And what should security professionals as well as individual workers be doing to protect themselves and their companies?
In this podcast, a16z security expert Joel de la Garza breaks down the current risks and how to defend against them. But beyond just immediate security needs, he explains what bigger transformations may be happening, most notably a shift from the traditional hub-and-spoke, point to point, security architectures to a more distributed approach to workloads as well as trust.