‘Superflaws’ in real-time operating systems

Why have hackers conventionally targeted Microsoft Windows operating systems? One reason is that this family of operating systems traditionally have had a large number of vulnerabilities that have taken a long time to fix (and that remedial work is still continuing). Another reason is that Windows is usually the primary desktop operating system that is utilised by millions of users around the world. This combination of factors makes Windows a big target with a worrying return on hacking investment.

Recently, we have witnessed an unprecedented growth in the number of gadgets and devices that go to make up what we are now calling the Internet of Things (IoT). Indeed, it is estimated that by 2025, there will be over 75 billion connected IoT devices. This proliferation of IoT devices is now presenting us with a security problem similar to that of Windows - another widely used operating system with security vulnerabilities.

The reality is that each IoT device has its own software stack, often sourced from a third party, which may have vulnerable components. Real-Time Operating Systems, also known as RTOS, are among the most commonly outsourced parts of an IoT device. RTOS is an operating system that interfaces with IoT hardware to provide real time application data processing. The defining aspect of such operating systems is that all data processing is conducted near enough immediately, with little or no buffering. If processing delays do occur, jitter will be introduced into the operating system leading to possible device failure.  

99 (Or in this case, 200 million) problems

In 2019 it was disclosed by Armis Labs that one of the most popular RTOS, Wind River’s VxWorks, possessed eleven critical vulnerabilities. The discovery was named Urgent/11. Out of the eleven vulnerabilities, six of them exposed devices such as network firewalls, printer devices and magnetic resonance imaging (MRI), to Remote Code Execution (RCE) attacks. In other words, permitting bad actors to access and alter data on these devices, despite being physically located elsewhere.
Indeed, the vulnerabilities allow for an attack to occur either by commandeering the perimeter firewalls of an organisation and then compromising the rest of the network, or by undertaking an attack from an external network to directly access a device.

From there, it is easy enough for cybercriminals to jump laterally from one device to the next if they are all connected through a shared network. This could result, in extreme cases, in the complete cessation of an organisation’s operations. This could lead to physically detrimental, or harmful consequences, as well as substantial financial costs; particularly when ransom money is demanded.

Whilst these vulnerabilities originally impacted two million devices, the actual spread of Urgent/11 goes far beyond this figure as the set of vulnerabilities are present in six other RTOS suppliers that utilise the same IPnet TCP/IP stack. For example, OSE by ENEA, INTEGRITY by Green Hills, Nucleus RTOS by Mentor, TRON by TRON Forum and ZebOS by IP Infusion. As such, as many as 200 million devices have been potentially compromised. This includes millions of additional medical, industrial and enterprise devices from some of the world’s largest corporations: Siemens, ABB, Emerson Electric, Rockwell Automation, Mitsubishi Electronic, Samsung, Ricoh, Xerox, NEC and Arris, among others.

The alarming scale of these vulnerabilities have proven to be a headache for organisations. The sheer number of impacted devices provides an exceptional challenge to those trying to fix the problem; that is, if they realise there is a problem to fix in the first place. Similar to a needle in a haystack, often times, many organisations remain oblivious to the existence of Urgent/11.  

Unfortunately, the only means of dealing with Urgent/11 is through the use of patches. Yet, this assumes such patches can be made available, and then applied to the huge number of IoT devices that may be vulnerable.

Patching’s long tail problem

While individuals have the luxury to leave their personal devices updating over several hours or overnight, large corporations running complex industrial control systems simply do not have that option available to them. Every minute that their operation has an unplanned shut down could result in the loss of millions in revenue.

In certain circumstances, critical industrial and IoT control systems may require 100% uptime. For that reason, companies tend to enforce strict patching and maintenance schedules to protect their operations from this potential problem. Every aspect is meticulously planned and prudently rehearsed to avoid unexpected delays, typically years in advance. Despite such planning, cybersecurity teams are generally overwhelmed with their day to day workload and device owners do not always have the necessary skills to apply some of the low-level RTOS patches. This leads to an inevitable pile-up of vulnerabilities that go unaddressed, many with vulnerabilities that extend back decades.

Entering the post-patching era?

As of yet, no one has been able to offer a comprehensive solution to this problem. Nevertheless, the development of new security systems should be considered a priority to resolve the most critical and urgent elements of this issue. That said, it might also be necessary for us to adopt a shift in our approach. We might need to recognise that we are entering a post-patching era whereby many devices with a vulnerable RTOS cannot be patched due to practical reasons. Rather, it may be necessary to consider new forms of mitigation. An example of this could be a move towards the supervision of devices and preventing out of band or abnormal behaviour.

As traditional patching methods progressively fail because of legitimate business challenges that delay timely updates, a new method of monitoring and acting on security threats needs to be thought through for high availability or critical industrial processes.