Your friendly neighborhood AppSec advisor and honeypot enthusiast. Formerly @ Goldman Sachs and Ernst & Young. Find his thoughts in code form committed to Github.
After spending 15 plus years of my career in large enterprise environments, and most of that time focused on information security and risk management, I decided to take the plunge and join a start up! For anyone that has enjoyed the stability of large organizations, a change like this can justify feeling a bit nervous. (I am also a new parent as well, so extra nervousness all around.) So why would I take such a chance? Why did I decide to take that chance with Signal Sciences? And now that I’m almost two months into it how do I feel about it? In this post I’ll answer those questions and perhaps this can help you if you’re thinking about a change of direction in your career.
Did somebody say risk? Being in the field of information security for so long I tend to think about everything in life in terms of risk and risk management. To better understand the risks involved i needed more information about what I was getting myself (and my family) into. Fortunately I have some very valuable friends, colleagues, and family members that have experience in this space. Their combined experience covered the full spectrum of founding a startup, working for several startups, and funding startups. In speaking with them at length I was able to understand what to expect, what questions to ask, and ultimately that I needed to reset my own expectations about the risk/reward nature of startups.
So there is risk, but how can I mitigate the risk? In the end, the lead mitigating factor was actually the current state of employment opportunities the information security field. For those of us in the field, we are very fortunate to be in high demand while the supply of experienced information security professionals is scarce. I knew that if things did not workout in this startup adventure I would very likely be able bounce back into the enterprise. Considering all the factors carefully, this was a risk I could manage and was willing to take.
But Not Just Any Startup Would Do
In the last eight years my focus has been in application security. In that time there have been many tools, both open source and commercial, to help application security teams find bugs in applications before they are released to fend for themselves in production. You hope your security reviews and testing were enough, and if you wanted any additional layer of protection in production the primary option available was a Web Application Firewall (WAF).
As it turns out, and those with experience know, WAFs are not the most effective or scalable tool to deploy. However, some organizations have been willing to endure the limitations of WAFs to either meet a compliance requirement or attempt to gain some sense of additional protection. But there are other organizations that are not able to tolerate WAF shortcomings. Enter Signal Sciences, in which its very origin stems from the pain and shallow effectiveness of WAFs in an environment requiring scalable infrastructure and fast paced application development.
WAF Solutions so far have been Ineffective
While the concept of a WAF is good, block all requests containing malicious payloads, it is far from being a silver bullet. A WAF is one of those tools where its effectiveness is only as good as its implementation, and its implementation is only as good as much as you can invest in it. And this is not a one time investment, it is an ongoing investment of resources for oversight and maintenance. To be fair, if you do have the time, money, and resources to make this investment then you are more likely to get value out of your WAF. However, you could still be spending more time tuning and maintaining the tool, rather than consuming what the tool should be producing.
In the organizations I’ve been in, the consensus was the return on investment was not worth the resourcing and effort. Instead, we relied upon enforcing secure coding practices and security reviews. However, there were a few exceptions where a WAF made some sense in our environment. These exception use cases were:
Vendor applications where the vendor provided no assurance on their application security controls or practices. In these scenarios we would not employ the standard WAF rules, but only rules that had been finely tuned to the vendor’s application.
Hot patching. Deploy a WAF in a passive log only mode only, and if there were a critical vulnerability announced or discovered we could create a custom rule to protect the application until the vulnerability was repaired at the application layer.
Logging mode for visibility. Collecting WAF logs to help better understand the types of attacks were being thrown at our applications.
In reality only the first option was ever implemented for a few select vendor applications. For hot patching and visibility, the resourcing challenges to deploy sunk in and prevented progress.
Signal Sciences is the next generation of WAFs
To expand more on its origins, Signal Sciences’ founders come from the security team at Etsy. There, they were faced the challenge of getting a handle on the security of a large scale e-commerce web presence, compounded with a high volume of code releases on a daily basis. In an effort to leverage WAFs as a layer of protection they found that they couldn’t scale with their infrastructure, and the constant flow of false positives broke applications hindering development’s ability to move fast. As a result, the team realized there had to be a better way and began developing a solution to solve the problem of obtaining a high degree of application security assurance in such a rapidly changing environment. In achieving this goal they realized what was needed from a solution:
Visibility — Transparency into the what, where, and how your production applications are being attacked is paramount to prioritizing the resources of your application security team. Transparency into attack payloads and anomalous behavior in your application is absolutely necessary for being able to quickly respond if an attacker has discovered a vulnerability in your application. In addition, from a operational perspective, knowing what the normal behavior of your application looks like is instrumental to identifying buggy code releases.
Do away with static rules — The static rules approach (typically a bunch of regular expressions in a file) are highly prone to false positives, which unfortunately can break application functionality and upset a lot of people. The constant battle of tuning rules to fix false postivies and prevent application breakage ultimately results in tuning the rules down to a point where you are getting very little value from them.
Smart Blocking — Blocking an IP address can be very problematic, especially when that IP address represents a corporate proxy or mobile network. While it may block the bad actor attacking your site, it may also block all the legitimate users attempting to perform business transactions.
Scalable architecture — Network appliance based solutions don’t scale (unless you have unlimited time, money, and resources). As organizations move to more dynamic and cost effective infrastructure a security solution shouldn’t constrain your architectural decisions.
In the end what they had created from many lessons learned was not a tool, but a solution. A solution that gave application security, devops, and development teams the ability to understand and see what is happening within their applications in production. This visibility informed decisions on prioritizing resources and enabled teams to react quickly to real attacks as they occurred. As I followed what they had done at Etsy, I quickly recognized the value of what they had built. To me it was the missing component of a complete application security program.
This missing component was a solution that allows you to focus on its output of attack and anomaly data, rather than tuning and maintaining rules. So when Signal Sciences was formed I was very eager to get a first hand experience with the solution. Once I did, it did not disappoint. From my perspective it took our production application layer security capabilities from 0 to 100 mph right out of the box. This was very exciting and with the potential the solution has to transform an organization’s ability to obtain a better handle on application security made me want to be a part of it.
This is why I joined Signal Sciences.
Since I’ve joined Signal Sciences I’ve had the privilege of participating in numerous calls with future customers. It has been amazing to hear from one organization to the next how they all have experienced similar pains and limitations from their existing legacy WAF deployments. And as we take them through the Signal Sciences solution and the problems it is solving it just completely resonates with them. To put it simply, it has been a very cool experience so far. And I am thrilled that I made the decision to join the Signal Sciences family.
Thanks for reading this article. If you enjoyed it please let us know.
At Signal Sciences we are building the industry’s first Next Generation Web Application Firewall (NGWAF). Our NGWAF was built in response to our own frustrations of trying to use legacy WAFs while enabling business initiatives like DevOps, cloud adoption and continuous delivery. The Signal Sciences NGWAF works seamlessly across cloud, physical, and containerized infrastructure, providing security without breaking production traffic.