System administrator setting up server network in data center showing vulnerability management

Running in Place: Staying Afloat With Language-Level Vulnerability Management

The patch management process can be painful, tedious, and time and labor intensive. Often, all this effort is for no other purpose than to maintain the operational status quo. And for devs or sysadmins, patch management has to happen on top of handling every-day activities as well as any other additional challenges that occur during service interruptions or system reboots.

When it comes to language-level vulnerabilities, patching challenges today present a proverbial “one-step-forward-two-steps-back” environment for developers. You know what we’re talking about…the hop-on/hop-off/hop-on again merry-go-round of patch management just to ensure a reasonable level of operations, security and compliance. And despite best efforts, there’s always another vulnerability (or two or three or TEN!) right around the corner.

When it comes to vulnerabilities, every security professional worth their salt knows that there is no single security answer. Yes, you can implement advanced threat protection, zero trust, and endpoint security. But those solutions aren’t going to get you to the 99.999% solution. An ongoing vulnerability patch management process must be a key component of the overall security solution.

However, even with the best vulnerability management processes in place, including sufficient time and resources to patch immediately upon notification of a new CVE, the patching process can still be disruptive and inefficient. Patch installation is sometimes postponed due to required system and machine downtime from reboots. Unfortunately, delays in patching critical vulnerabilities increase opportunities for threat actors to infiltrate systems and exfiltrate data.

Added challenges may also include lack of agreement between security and operations on how to install a patch and finding time to take critical resources offline.

For businesses working with operating systems like Linux, additional issues can arise. Historically, Linux has been considered one of the more secure operating systems. However, Linux attacks are becoming more attractive to threat actors because those systems tend to have a higher payoff value. Businesses operating with Linux are faced with increasingly dangerous Linux-based vulnerabilities, like those related to remote control execution (RCE) and local privilege escalation (LPE). RCE is considered the holy grail for attackers, given the level of control it offers over machines and systems. And if RCE isn’t possible, attackers can always rely on an LPE vulnerability.

With Linux, traditional patching tools like apt, dnf, or yum (which are used for updating on-disk versions of vulnerable software), are problematic, since they don’t immediately apply to already running code, like the Linux kernel that is always running, or shared libraries already residing in memory as a dependency of multiple services. Changes occurring on disk will only translate to revisions in the running code when those services are restarted, or the whole system is restarted – something obviously disruptive to any workload. This means that some customer or user somewhere is going to experience a business disruption.

There are other situations beyond core Linux system components that are notoriously difficult to patch. Security vulnerabilities in languages like PHP, Python, and Java can create a scenario where managing the vulnerability involves updating the language. The problem is that when a language level update is released, it traditionally does not simply address security issues – it introduces other, unrelated, language changes which may break existing code. This, in turn, requires rewriting code to compensate for any language-level changes, which further delays the patching process. In addition, retesting and recertification may also be required.

When it comes to upgrading to new language versions, one option that businesses frequently employ is to delay the process, waiting for a time when resources are optimal and business disruptions can be lessened. But there are risks with this, not the least of which are regulatory and compliance concerns, which often require that vulnerabilities be patched within a certain time frame. Continuing to develop and release systems on language versions with known vulnerabilities clearly creates supply chain security risks.

What all this means for businesses is that the “hop-on/hop-off/hop-on again merry-go-round” patching process can be a ‘no win’ situation. You can patch quickly to ensure security, but risk system availability and downtime, or delay the patching process and risk threat actor intrusion.

Research by TuxCare into vulnerability management suggests that organizations are well aware of the feeling that they can never get ahead when it comes to patch management. Almost half (45%) of the respondents to a recent survey reported that they had no tolerance for patching downtime. Moreover, even with critical or high-priority vulnerabilities, more than half (56%) took over a month to patch their system.

All of this shows how complicated vulnerability management is and how quickly the process can break down. To avoid the patching “hop-on/hop-off again merry-go-round,” many businesses adopt an automated ‘frictionless’ patching process, involving tools that enable ‘live’ patching – removing notable burdens for already busy sysadmins.

Requiring ops and security to otherwise manually manage the patching process takes a toll on teams. With threats growing and budgets as well as skilled staff shrinking, exploring alternatives to get your staff off the vulnerability management merry-go-round can improve productivity, enhance security, and support a better overall business operations environment.