TAP as a Service (TAPaaS)¶

Problem Description¶

Performing network protocol review, checking firewall rules and / or performing network security audit is part of the daily job of the system administrator. To be able to provide this functionality in the cloud, an idea to create a tap-as-a-service was born. For example when enabled and configured it will allow the system administrator to run a Snort IDS on his tenant network.

In short, we want to create a mechanism to forward traffic for example from one VM to another. When packet will be forwarded, the original value of source and target ip/ports information will not be altered and the system administrator will be able to run, for example tcpdump, on the target VM to trace these packets. The administrator, will have to enable a promiscuous mode on the target VM network card to be able to see these packets. [1]

Currently this document concentrates on tapping traffic coming in and out of VMs.

Additional kinds of ports will not be supported in the first version of the spec.

1. Tapping DHCP port
2. Tapping router port
3. Tapping floating ip traffic

Terminology and API¶

The API [2] and SPEC [3] define two objects:

1. TapService
2. TapFlow

TapService represents the port to which the mirrored traffic is delivered. For example this port can be attached to VM where a Snort IDS running.

'tap_services': {
    'id': {'allow_post': False, 'allow_put': False,
           'validate': {'type:uuid': None}, 'is_visible': True,
           'primary_key': True},
    'tenant_id': {'allow_post': True, 'allow_put': False,
                  'validate': {'type:string': None},
                  'required_by_policy': True, 'is_visible': True},
    'name': {'allow_post': True, 'allow_put': True,
             'validate': {'type:string': None},
             'is_visible': True, 'default': ''},
    'description': {'allow_post': True, 'allow_put': True,
                    'validate': {'type:string': None},
                    'is_visible': True, 'default': ''},
    'port_id': {'allow_post': True, 'allow_put': False,
                'validate': {'type:uuid': None},
                'is_visible': True},
}

TapFlow represents the port from which the traffic needs to be mirrored. It can be a port associated with VM on another CN.

'tap_flows': {
    'id': {'allow_post': False, 'allow_put': False,
           'validate': {'type:uuid': None}, 'is_visible': True,
           'primary_key': True},
    'tenant_id': {'allow_post': True, 'allow_put': False,
                  'validate': {'type:string': None},
                  'required_by_policy': True, 'is_visible': True},
    'name': {'allow_post': True, 'allow_put': True,
             'validate': {'type:string': None},
             'is_visible': True, 'default': ''},
    'description': {'allow_post': True, 'allow_put': True,
                    'validate': {'type:string': None},
                    'is_visible': True, 'default': ''},
    'tap_service_id': {'allow_post': True, 'allow_put': False,
                       'validate': {'type:uuid': None},
                       'required_by_policy': True, 'is_visible': True},
    'source_port': {'allow_post': True, 'allow_put': False,
                    'validate': {'type:uuid': None},
                    'required_by_policy': True, 'is_visible': True},
    'position': {'allow_post': True, 'allow_put': False,
                  'validate': {'type:values': position_enum},
                  'is_visible': True}
    'direction': {'allow_post': True, 'allow_put': False,
                  'validate': {'type:values': direction_enum},
                  'is_visible': True}
}

The DirectionEnum states whether the TAP is attached to the ingress, egress, or both directions of the VM port.

direction_enum = ['IN', 'OUT', 'BOTH']

The PositionEnum states where to place the tapping point. It is defined as follows:

position_enum = ['VNIC', 'PORT']

VNIC specifies that the packet should be intercepted close to VM, while PORT specifies that packet should be intercepted on the network level.

In case we are talking about tapping outgoing packets:

1. VNIC is before security group firewall.
2. PORT is after security group firewall.

In case we are talking about tapping incoming packet:

1. VNIC is after the security group firewall.
2. PORT is before security group firewall.

Multiple TapFlow instances can be associated with a single TapService instance. I.e. one Snort VM can check traffic of multiple virtual servers.

Tap packet forwarding table¶

After the packet is mirrored, it will be forwarded to the new packet forwarding table (TAP_FORWARD_TABLE).

Packets can come from local CN and external CN. To minimize number of changes, we propose to create a new table to handle mirrored packets.

This table may not be highly optimized, but improves the modular design.

Each mirrored packet, coming either from the same CN, or from external CN, will have a marked tunnel id.

In case the packet coming from the local or external CN needs to be forwarded locally (to the same CN), the following rules will be applied:

filter: tun_id=DEST_TUN_ID action:output:DEST_LOCAL_PORT

In case the packet is coming from the local CN and should be forwarded to external CN:

filter: tun_id=DEST_TUN_ID action:output:OVERLAY_NET_PORT

Field Name	Description
DEST_TUN_ID	a tunnel number will specify a destination VM
DEST_LOCAL_PORT	destination OVS port number (in case it is on same CN)
OVERLAY_NET_PORT	packet will be forwarded to other CN

Packet mirroring¶

In order to support Tap as a Service, a TapFlow packet mirroring rule can be installed in multiple locations, relative to the port:

1. Tap rule on output
2. Tap rule on input
3. Both

In addition, tapping flows can be installed before and after SG firewall rules.

Filter:reg7=0x8 Actions:resubmit(,79),
  $DEST_TUN_ID1->tun_id,goto_table:TAP_FORWARDING

Filter:in_port:6 Actions:resubmit(,79),
  $DEST_TUN_ID1->tun_id,goto_table:TAP_FORWARDING,
  $DEST_TUN_ID2->tun_id,goto_table:TAP_FORWARDING

Receiving mirrored packets from other CNs¶

To be able to forward packets received from other CNs on each CN that has a TapService we will add relevant rules to forward rules to a TAP_FORWARDING table.

We will add new rule in table=0:

Filter:tun_id=$DEST_TUN_ID1 Actions:goto_table:TAP_FORWARDING
Filter:tun_id=$DEST_TUN_ID2 Actions:goto_table:TAP_FORWARDING

Database Changes¶

In order to support TapServices the following table will be added to distributed nosql database.

Attribute Name	Description
key	record identify
topic	tenant ID
port_id	port id of the destination VM
segmentation_id	overlay network distinguishing tunnel id

The following fields were omitted here:

• name
• description

In order to support TapFlows the following table will be added to distributed nosql database.

Attribute Name	Description
key	record identify
topic	tenant ID
tap_service_id	id of the destination TapService
source_port_id	port id of the tapped machine
position	enum [‘VNIC’, ‘PORT’]
direction_enum	enum [‘IN’, ‘OUT’, ‘BOTH’]

The following fields were omitted here:

• name
• description

List of relevant OpenFlow tables¶

INGRESS_CLASSIFICATION_DISPATCH_TABLE = 0
EGRESS_PORT_SECURITY_TABLE = 1
SERVICES_CLASSIFICATION_TABLE = 9
INGRESS_SECURITY_GROUP_TABLE = 77
INGRESS_DISPATCH_TABLE = 78

References¶