Canape – Dicing ESXi into Byte Size Portions

Canape – Dicing ESXi into Byte Size Portions

On the 21st October Context consultant Alex Chapman will be presenting at the Ruxcon conference in Melbourne. Alex will be describing how to inspect, manipulate and exploit the main remote administration protocol used by both VMware ESXi and other VMware products such as Workstation, using Context’s network protocol analysis tool, Canape.

Using Canape, Context found numerous security vulnerabilities which could lead to remote code execution and denial of service within the ESXi protocol which the vendor, VMware, are currently working on resolutions for. The full details of these issues are not included in this blog due to Context’s responsible disclosure policy, although further details may be released in subsequent posts once the vendor has mitigated the vulnerabilities. This post will however cover some of the SSL weaknesses that Context discovered within the protocol , and will describe how some of the new features of the latest Canape release can be used to analyse SSL streams that are embedded within binary network protocols streams.

Canape is Context’s free network protocol analysis and manipulation tool for Windows, which aims to reduce the amount of work required during a security review to assess an arbitrary protocol. It is designed to act in a similar fashion to pre-existing web application testing tools such as CAT, providing an interface to capture, manipulate and then replay network traffic in any protocol, not just HTTP. More information about Canape, as well as downloads and worked examples, can be found on Context’s Canape website located at

In order to coincide with the Ruxcon talk a new version of Canape (v1.1) is being released, containing some important new features that will be discussed later in this post. This blog provides an overviejw of one of the new advanced features of Canape, Layer Sections, and how they can be used to strip away SSL encryption from the middle of complex network protocols such as the VMware ESXi management protocol.

This blog post will begin with some background on the ESXi network protocol.

ESXi network protocols

ESXi is VMware’s enterprise grade hypervisor which is a bare-metal hypervisor, meaning that it does not require a separate host operating system to be present. It is designed for use in larger infrastructure deployments, but can be installed and managed as a single isolated system. As ESXi is an enterprise product, it assumed that physical access will not be the main way of interacting and managing the server. It therefore runs a number of network exposed services to provide remote management, services that are utilised by the vSphere Client shown below.  

The vSphere management protocol actually uses a combination of HTTP, plain text and binary protocols, each utilising SSL encryption running over TCP ports 443 and 902. Of most interest to a security researcher are the set of protocols running over TCP port 902, as that is likely to be where the interesting findings will be, rather than within standard HTTPS traffic.

Through inspection, it is clear TCP port 902 is used to provide a number of different services; including VNC access and file transfer, using a number of different protocols . It contains three separate states:

    1. Initial State

    2. Authentication and Service Selection State

    3. Main Protocol State

The initial state solely consists of the server sending a particular banner to the client application as shown below:

The banner contains information on what protocols the server supports, such as support for VNC or SSL. This is important as the next packet sent by the client looks like this:

This it turns out is a TLSv1 Client Hello packet: the protocol “upgrades” to include SSL encryption to perform authentication. The authentication details which need to be provided depends on the intended subsequent protocol: it could be a username and password combination, an authentication ticket or a session token. Once the authentication process has been completed, the final step to issue a command to switch into the intended protocol to use. For example the following Canape screenshot shows the vSphere client connecting up to the VNC service:

Whereas this Canape screenshot shows the client connecting to the file transfer service:

The main protocol now starts; in the case of the VNC protocol this establishes a brand new SSL connection. However, in the case of the file transfer protocol it turns out then the protocol downgrades back to plain text before commencing, so it is worth knowing that potentially file transfers are at risk of being extracted by anyone capable of monitoring the network traffic even though almost everything else is protected by SSL. This is shown below:

New updates in Canape version 1.1 allow us to decrypt these upgraded SSL protocols, even if they are situated right in the middle of the protocol stream.

The following example Netgraph shows what a Layer Section looks like when added to a graph, the dashed blue line indicates the pairing of the nodes and from a user interface perspective they are treated as a single unit. Deleting one of them deletes the other and they share a single configuration, removing the complexity of ensuring the correct settings are applied to both nodes.

Canape updates

As mentioned in the introduction, to coincide with the Ruxcon presentation a new version of Canape, v1.1, is being released. This release contains a number of improvements and fixes to make it easier to use and test with.

The main focus of Canape improvements has been to reduce the amount of coding required to work with complex network protocol s. We don’t want to force users to become expert developers just to perform basic man-in-the-middle protocol analysis.

Layer sections

One of the more difficult to manage scenarios in Canape without implementing custom code is removing transport layers. This can be things like fragmentation of data, multiplexing of independent streams (for example SPDY) or negotiated encryption wrappers.

The ESXi protocol fits into this latter category, it doesn’t necessarily use SSL unless the ESXi server requests it in the connection banner and even then it only does so for certain types of services. This technique is also common in other protocols too, for example SMTP supports the STARTTLS command which will allow a connection to upgrade from plain-text to SSL encryption. Even with custom code this required a reasonable amount of effort to support in Canape v1.0, as the existing SSL man-in-the-middle functionality was "all or nothing".

To better handle encryption upgrades of this sort, the latest Canape version includes a new feature called Layer Sections. Layer Sections are similar in concept to the existing Netgraph Containers which allow Netgraphs to contain sub-graphs, however Layer Sections act in a different way. Containers effectively are a convenience mechanism to reduce the complexity of a graph, while a Layer Sections offer the functionality of Netgraph Containers, they also provide extra processing upon active connections. This extra processing allows Layer Sections to do clever tricks such as negotiating SSL individually with each end of a proxied connection, which allows us to man-in-the-middle upgraded encryption of the type implemented within the ESXi protocol.

While the infrastructure is now in place to do some interesting stuff with Layer Sections, v1.1 currently only exposes a canned SSL implementation from the interface. This is good enough for a number of use cases out of the box, but will be augmented in future versions to support custom implementations in script as well as many more built-in functions including simple demultiplexers and defragmenters.

The State Graph has also been extended to allow the use of Layer Sections; this is a relatively unknown feature of Canape which allows you to quickly build up a state model graph without manually drawing out individual decision elements. If you are lucky enough to see the presentation at Ruxcon there will be a demonstration to show how you can use this to quickly put together an ESXi parser with very little development effort.

To conclude, features included in the new version of Canape, such as Layer Sections, greatly increase the power of the Canape tool to capture, manipulate and replay data from complex network protocol with minimal writing of source code. The new version of Canape can be downloaded from the Context website at

Subscribe for more Research like this

CHECK IT Health Check Service
Cyber Essentials
CESG Certified Service
First - Improving Security Together
BSI ISO 9001 FS 581360
BSI ISO 27001 IS 553326
PCI - Approved Scanning Vendor