A Lacework autopsy

Failure Mode. Lessons from Lacework’s defeat snatched from the jaws of victory.

https://charity.wtf/2024/07/24/pragmatism-neutrality-and-leadership/

Fisher’s fundamental law of natural selection

munsoned (v.) - to be up a creek without a paddle; to have the whole world in the palm of your hand and blow it. a figure of speech.

from the great movie "kingpin".

Lacework: Snatching defeat from the jaws of victory

Google’s $32 billion acquisition of Wiz is noteworthy from many angles. I’m happy for my friends, former colleagues and their bank accounts. It’s also emotional in other complex ways.

In short: That was supposed to be Lacework.

For the uninitiated, Lacework was incubated in 2014 down the hall from Snowflake at Sutter Hill Ventures. The founders and Sutter Hill were early to recognize that security for software businesses built on clouds like Amazon Web Services fundamentally differed from security in traditional static data centers. By late 2019, enough product-market fit had been found for Sutter Hill and the leadership to invest in scaling the company. By February 2020, a sizable go-to-market team had been assembled. In the following pandemic months, revenues grew 3.5x, while headcount grew 3x.

On the competition front, we were shooting fish in the barrel. First-generation products couldn’t deal with dynamic cloud environments, while vendors steeped in security didn’t speak, much less serve, cloud and DevOps professionals. Palo Alto had acquired a group of point products. slapped them together in iframes, and called their “platform” Prisma. Threat Stack’s rules couldn’t scale to accommodate constant cloud changes, AWS Guard Duty was a good start, but got expensive and difficult to manage at scale, Crowdstrike’s product didn’t work yet, we rarely ran into Trend Micro, Orca secured packaged apps for CISOs at companies overpaying to run those apps in the cloud, and Wix had just launched a website.

In our adjacent Sutter Hill universe, Snowflake’s September IPO blew the doors off the market, delivering one of the largest venture capital returns ever to Sutter Hill. Roughly $12B from a $200 million investment. Lacework quickly became Sutter Hill’s answer to “what’s next?”

On January 5th and 6th, 2021 yes, that Jan 6, we briefed 15+ press and analysts for a January 7th announcement of $525m, the largest funding round in security with a unicorn valuation to match. In the following 11 months, the company would replace the CEO twice-ish, hire ~1,000 people, and raise another $1.5 billion in November at an eye-popping $8.3B valuation.

Fast forward to June 204, as the rumors of the first Google-Wiz deal at $23 billion were being leaked, Lacework shareholders were asked to approve a $200m firesale to Fortinet.

How the heck did that happen?

I’d like to start with the elevator pitch version of the story/first-call deck I built for Lacework in November of 2020.

1bcfdee242e380baa216c9051e23d7dd1bcfdee242e380688898ea414afad48c


In the cloud, everything is constantly changing.

1bcfdee242e3806ead40fbf426dd5cc71bcfdee242e3809bb8cbfc9b484c74c0


If you can't differentiate between good and bad changes, you can’t run a secure business in the cloud.

1bcfdee242e3805aae81c826d17e5dcb1bcfdee242e38038bdefcbacbf0f3ffd


Lacework was built to understand cloud changes so devops and security teams can work together to build a secure business.

I share this story not just because it’s good. It also aligns with the key lessons of this now legendary train wreck.  It’s easy to point fingers and blame people for key mistakes [2], but in the spirit of blameless company-failure-post-mortems. For the most part, I’m going to stay away from blaming indiviuals, especially since key decisions and cultural norms are made and set by groups of people. A little warning: I’m going to dive into more of the background and cloud security definitions than you bargined for when clicking here. I deprobably necessary before I get to the lessons. Here we go.

• ****

This post is sponsored by The Right Stack. If you are a technical leader of a software team building revenue-generating products in the cloud with fewer than 150 developers and engineers, please take this short survey.

• ****

First, the facts:

Lacework background

The cloud security opportunity

Cloud security definitions.

Lacework in 2020 and 2021

The opportunity

Yes, those ZIRPy billions are massive but not completely nuts. The opportunity is enormous, especially by the standards of the security market. At the time, we forecasted the opportunity by looking at cloud infrastructure spending, which was growing at 30% a year to reach $80+ billion in 2020. Businesses built on the cloud typically spend between 1% and 8% of their cloud spend on security, with the median at ~2% and rising with the adoption of dynamic architectures. Finally, cloud infrastructure spending was forecasted to crack $100B by 2022, pegging the near-term opportunity at $3B+ in 2022 and growing fast. In hindsight, this forecast was largely correct, even with the post-ZIRP-optimization correction.

WTF is “cloud security”?

There is a ton of disingenuous positioning in this market, preying on the overworked security teams and CTOs forced to buy this stuff. There are three runtime layers and two parallel pillars when securing and monitoring applications running on cloud services.

Maslow’s cloud security hierarchy is roughly:

#1 Are my cloud services appropriately configured? Ala “Are my S3 buckets open to the public or private?” The Capital One breach and countless open S3 buckets stem from misconfigured services. Gartner calls this Cloud Security Posture Management or CSPM.

#2 Are my containers and instances appropriately configured? Why did a new set of instances appear in my AWS account? Which version am I running of what? What vulnerabilities in my operating systems and containers should I care about? This is “Container and Instance Security” with a dash of operating system vulnerability management.

Let’s pause here for a second.

#1 and #2 can be accomplished by pulling data and logs from a cloud service provider’s APIs and/or by running a container with a bit of software inside the customer’s cloud account. This means a customer’s security team can secure a cloud account with minimal interaction with the customer’s DevOps and product team. If your go-to-market team speaks CISO, this can be handy to infiltrate customers looking to get going and check a box with minimal fuss. This has led to certain vendors braying about being agentless… until they invariably add an agent to gain visibility *inside* an instance, container, or container cluster, which brings us to #3.

#3 What’s the behavior of the apps running inside my instances and containers? What are these apps doing, and what are they connecting to? (Parents of teenagers can relate) Since every product with visibility into application behaviors also operates in the first two layers and “application security” already meant something else (#5), Gartner calls 1-3 Cloud Native Workload Protection Platforms or CNAPP.

Here is the kicker:

Building #3 is significantly harder than #1 and #2 from a company and software development perspective.

Getting detailed visibility into the web of application behaviors and relationships requires deploying an agent embedded in an application. That agent becomes part of the software application and release process, which means it can’t and won’t get deployed without active participation from the devops and product teams. Wham, your go-to-market just changed: now your company is no longer a security company selling to security people.

Buyers - don't be shy about testing agents before you sign any contracts. Agents mature through battle scars. There are no shortcuts.

Cloud security companies have acquired and consolidated outside the three runtime layers.

Finally, to help customers “shift left,” and consolidate cloud security functions, cloud security companies have acquired companies in the two parallel pillars.

One pillar is managing and monitoring the permissions and authorizations across the actions humans, automation tools, and cloud services are allowed to perform. It’s hideously complicated and causes much gnashing of teeth. Gartner calls this Cloud Infrastructure Entitlement Management, or CIEM.

The other pillar includes several layers of checking and scanning application and infrastructure code for vulnerabilities and misconfigurations in the development process. This has been the “application security” market long before the cloud appeared. The infrastructure as code bits are, of course, newer.

I’m ignoring incident, compliance management, policy-as-code, and a bunch of other stuff. This has already been WAY more context than anyone wanted and/or it’s own post. Don’t @ me about eBPF either.

Phew, sorry about that

THE LESSONS

Building a company to help software teams build and run a business?

#1 - In the cloud, everything is changing constantly.

Your customer’s software products, architectures, cloud provider services, cloud service APIs, cloud provider security products, security threats, regulations, the countries customers want to operate in, competitor’s products, skills on your team, vulnerabilities, etc. It is all changing all the time. What does this mean for companies building products serving builders in the cloud?

In the cloud, the search for product market fit never ends.

The only anecdote is relentless customer obsession, transparent loss analysis, and a healthy dose of paranoia. Bezos calls this the beginner mindset. At it’s core, a beginner mindset requires a level of humility at odds with a hard charging sales culture. And yet, today’s best sales teams are inquisitive and helpful. Lacework had some early sales hires who were absolute masters at listening to customers, aligning the value of the product to the ways the customer’s business was changing, and working with product teams to craft a roadmap.

1. WWJD? CSPM as AWS’s problem

What would Jassy do?

1. We did this before, it’s the same thing, but SaaS now.
2. Never (publically) count your chickens before they hatch
1. Two of my top ten nauseating moments at Lacework involved internal meetings where a form of “We’re all going to get rich.” was declared. This actually got worse after I left with one leader declaring the company was going to create “generational wealth.” Look, I get it’s all about the money, but never count your cloud chickens until they hatch. Software architectures and key technologies can change inside of three years - which sounds like a long time, but innovator

1c2fdee242e38061a49dc4cf204db9a0

If you can’t differentiate good and bad changes, you can’t run a secure business on the cloud.

When something does change, can the company effectively deal with that change? Let’s take a look at some examples of changes Lacework faced.

1. Assumptions:
1. The first question investors ask is: why won’t the cloud providers invest in solving this problem?
1. When building functionality adjacent to a cloud provider, questions regarding the cloud provider’s
2. Roadmap priorities and commitments
1. When a customer asks for a change to your roadmap, or more likely a sales exec asks for a change to land a prospect, is that a good change or not? The result of collaboration between functions should result in refined focus - qualifying a customer out or shifting the ICP. Here is a good riff on considerations for https://www.saastr.com/where-do-you-draw-the-line-when-companies-ask-for-customization-for-a-saas-product/?
3. Premature scaling and understanding changes:
1. Readiness to scale includes every aspect of the company. A desire to scale based on funding round timing is different than a company’s readiness to scale. You can even say things like “the timing is ideal, so we’re going to take money before we need it.” And then the money hits the bank account and all of a sudden it’s giddy up time. Note: just because your board thinks growth is entirely dependent on how much money is poured into the business, you need to have a frank conversation with the top 20% of the company to discuss what accelerating scale takes. If you hired “the best” they will tell you which corners can be cut and which ones cannot.
2. Corollary: People who are skeptical or don’t react to your Slootman “always ask to go faster” with enthusiasm might see something you don’t. Intel used to allow space for dissent before committing. Netflix, famously “farms dissent.”
4. Board desires
1. I’ve now been part of two companies where the board had more control than the founders. It must work enough to perpetuate the model. However -
1. When a board member suggests a company is ready for X or should do Y, the incentive for managers to say yes is high. After all, the board members are often wealthy guys who have seen many companies in your position and you serve at their will. The check here, is to remember the management team has a different view of the reality of the company’s capabilities than the board.
2. From my perspective, investor and manager motivations and desires are mismatched enough to generate a view of the company that can become divorced from the reality of the changes the company must navigate.
3. OR - Managers should have a better handle on reality
4. This mismatch can also lead to times where “raise when you can” which can separate the company’s ability to change (scale) from reality to what the newly “rich” founders and the investors believe.
1. Too much money is a real thing and the only thing missing from this HBO Silicon Valley clip is the fact that a high valuation can make hiring harder now that the company has to “grow into” and then 10x their new orbit.

https://www.saastr.com/why-is-it-dangerous-to-accept-too-high-of-a-valuation-from-an-investor-for-a-startup-company/

Lacework was built to understand cloud changes so devops and security teams can work together to build a secure business.

The nexus of building blocks power innovation and growth

A common innovation misconception is the notion some smarty pants says “Eureka” and something completely new appears out of thin air. The reality is someone figures out how to take existing components and combine them into something new. At that point, they say “Eureka?” and  extends far beyond the products into business models and more. Innovation compounds over time. Silicon Valley is particularly good at this partly because you can go find people who built or know about a component you are combining into something new and put them in a room with experts in the other components.

Lacework sat at the intersection of public cloud, security, devops, monitoring, compliance, and SaaS. To avoid writing that list out five times in the next two paragraphs, I’m going to call that nexus list the CloudSec Vigor. It’s kind of like hybrid vigor, but with a specific set of traits.

One other key element to the vigor, is learning speed.

When I ran an analyst team, the hiring rule of thumb was you could teach someone how to be an analyst or teach an analyst a new market, but not both. Asking what you or a candidate will have to learn to be successful at a new job is

At Lacework, the most successful sales engineers didn’t have a background in security, they came from monitoring companies like AppDynamics, DataDog, and New Relic. They spoke public cloud, devops, monitoring, and SaaS. We could teach them security. Sales engineers from the security world could be successful, but that profile was tougher than the former.

Ideally, the nexus vigor needs to be reflected across every function in the org, especially on the leadership team. I don’t care how brilliantly someone has run another business, if the bulk of the executive team isn’t well-versed in speaking to your ideal customer profile, you’re going to get trounced by a company with a superior ability to connect with customers. Lesson #2 in action.

If you look at Wiz as the counter-example, the founding team built a security product Adaonme (sp) was acquired by a public cloud Microsoft Azure. The founding team could all connect with and had personal networks of people within multiple elements of the CloudSec Vigor.

I didn’t use the term hybrid vigor. Yes, it’s helpful to have perspectives from outside the innovation nexus, but early on, if you hire too far from your innovation nexus, you need to be willing to accept the drag that hire creates. For a leadership hire, it’s a risk to your mission and doesn’t set up the leader to succeed.

At this point, I’m getting dangerously close to pointing fingers and naming names, so I’ll say this: At Lacework, too many leaders came from businesses outside the CloudSec Nexus. Some of those leaders hired more people outside the CloudSec Nexus, and those from inside the Nexus spent too much time teaching these new arrivals to the CloudSec Nexus before giving up in some fashion.

Ok, I am going to mention one name, partly because I’ve never met him, but mostly because I believe selecting him to be a co-CEO was the pinnacle of hiring outside the CloudSec Nexus rather than the root cause of the problem. Jay Parak was hired initially to compliment another CEO’s lack of CloudSec Nexus experience, sorry “technical depth.” Mr. Parak had spent a decade managing the thousands of engineers and billions of dollars that build and run Facebook’s bespoke private infrastructure. Based on his bio, he had no background in public cloud services, a bit of B2B SaaS via Akamai 20 years ago, and I’m guessing his security knowledge, while potentially deep in technology weeds, he probably wasn’t hanging out with enterprise CISOs and DevOps leads a whole lot at Facebook.

Corollary: if you are not well versed in elements of vigor your company has pulled together, study up (remember that humble thing?)

Why did Lacework hire leaders from outside the cloud nexus?

Now I’m playing with blameless fire, but I think there are two reasons. Firstly, hubris.  A board coming off of the biggest exit in Silicon Valley must be heady stuff. I can only speculate what that feels like, but following the Snowflake exit with closing the largest security funding round transferred a belief and a desire the company was more mature than reality. The transition from “take the funding when you can” to “the only thing between Lacework and an IPO is the speed we can scale the sales team” .

Parting thoughts:

My good friend Chris Christiansen called M&A (Mergers & Acquisitions), Murders & Acqesence.

At RSA, a few Lacework alumni gathered in a bar. I was struck by how much of the conversion centered around various lapses in integrity.

When all you have is your integrity, people may not remember how much  Wiz is hopping to integrate not one, but two companies at the same time. That’s aggressive.

Lacework shopping the company was leaked, widely. Deals are only ever leaked in order to shift the value of the deal. In this case the same journalist who leaked the potential Wiz/Sential deal, leaked the Lacework/Wiz deal. If you leaked, your bank account might end up in a dandy spot, but that integrity lapse means no one trusts you.

We’re still in the early innings. Wiz might think they now have this whole sebang wrapped up, but go back to principle #1, this game isn’t over yet and the opportunity is massive.

Footnotes:

[1] Exhausted by an onslaught of “large funding round stories,” several press members passed on the story. That and the attack on the capital just didn’t seem like a time to be writing about a big round. Shout out to Jen Lankford (PR), Dan (CEO),  for all the work that went into that announcement.

[2] One individual in the Lacework alumni slack channel has suggested shorting companies where particular executives go next. Someday, digi-archeologists will find equally spicey takes in text message inboxes and email draft folders.

A few other Lacework post-mortems of note:

https://www.theinformation.com/articles/laceworks-fire-sale-tarnishes-sutter-hills-incubation-strategy

https://www.linkedin.com/posts/sakalra_wiz-lacework-ugcPost-7218620261443461120-tSdB

From Charity WTF

If a bunch of your employees are waving a flag and urgently saying “we have a problem”, they are very likely doing you a favor. Either way, they deserve to be heard.

You don’t have to do what they want. But you ought to listen to them, and reserve judgment. Open your eyes. Look around. Do some reading. Talk to people. Consider whether you might be missing something. Then make a decision and give an honest answer. They may or may not agree, and they may or may not choose to stay, but that’s what treating them with respect looks like, just like you ask them to treat you, and each other.

To instead say “Sorry, your feedback is a distraction from the mission and will no longer be tolerated” is so unbelievably disrespectful, and wrapping your decision in the noble flag of the mission is dishonest. It’s hard to tell sometime whether people are deluding themselves or only trying to delude other people, but holy shit, what a doozy.

Ok, I think the let’s see if I have this straight:

Jay joins Lacework in ~July 21.  The board gives him a loan to buy $26m in equity at the $1.3B valuation. They choose Jay because Hat needs someone who understands some combination of cloud, devops, security, and a SaaS productt and who could be better than a guy with $xx million in Facebook stock who built a highly bespoke infrastructure for a massive ad-driven consumer company??

You can’t just give someone $26m in equity and call that an incentive, you also have to give them options, cash, and other prizes. I’m going to go out on a limb here and guess that Jay’s total package was $57m and a lawyer put a change of control clause in his contract that stated he got that number if triggered.

The board